2024-04-05T22:06:56.019 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-05T22:06:56.020 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --archive /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672 -- /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/orig.config.yaml 2024-04-05T22:06:56.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-05T22:06:56.158 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672 branch: wip-yuri4-testing-2024-04-04-1128-reef description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: yweinste@redhat.com first_in_suite: false job_id: '7641672' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2024-04-04-1128-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2024-04-04-1128-reef sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5779 sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2024-04-04-1128-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8919605b0a5dc1f1650c8aec3051c77c0e3e240e/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbjYUP5vK2Qbb96znHaYroZFrZBg/XI0PMZan49bBI9xSKPeaNJDnXaZtsKsnRWmuhzAoICbPCRqEGiWnhE+VM= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCnXSpAkWuSJg4Pt6OmsfWAKlOC9R3c3W4g9ge5OjmfvIsslsovZ1PLYWw8SmZK5Y0xBJs7GyUrzdfPi/UgmuQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-05_18:13:21 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-05T22:06:56.158 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8919605b0a5dc1f1650c8aec3051c77c0e3e240e/qa; will attempt to use it 2024-04-05T22:06:56.159 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8919605b0a5dc1f1650c8aec3051c77c0e3e240e/qa/tasks 2024-04-05T22:06:56.159 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-05T22:06:56.160 INFO:teuthology.task.internal:Checking packages... 2024-04-05T22:06:56.188 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8919605b0a5dc1f1650c8aec3051c77c0e3e240e' 2024-04-05T22:06:56.189 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-05T22:06:56.189 INFO:teuthology.packaging:ref: None 2024-04-05T22:06:56.189 INFO:teuthology.packaging:tag: None 2024-04-05T22:06:56.189 INFO:teuthology.packaging:branch: wip-yuri4-testing-2024-04-04-1128-reef 2024-04-05T22:06:56.189 INFO:teuthology.packaging:sha1: 8919605b0a5dc1f1650c8aec3051c77c0e3e240e 2024-04-05T22:06:56.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2024-04-04-1128-reef 2024-04-05T22:06:56.466 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1173.g8919605b 2024-04-05T22:06:56.468 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-05T22:06:56.476 INFO:teuthology.task.internal:no buildpackages task found 2024-04-05T22:06:56.476 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-05T22:06:56.511 INFO:teuthology.task.internal:Saving configuration 2024-04-05T22:06:56.529 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-05T22:06:56.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-05T22:06:56.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-05 22:01:05.931000', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbjYUP5vK2Qbb96znHaYroZFrZBg/XI0PMZan49bBI9xSKPeaNJDnXaZtsKsnRWmuhzAoICbPCRqEGiWnhE+VM='} 2024-04-05T22:06:56.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-05 22:01:05.932694', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCnXSpAkWuSJg4Pt6OmsfWAKlOC9R3c3W4g9ge5OjmfvIsslsovZ1PLYWw8SmZK5Y0xBJs7GyUrzdfPi/UgmuQ='} 2024-04-05T22:06:56.580 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-05T22:06:56.586 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-05T22:06:56.587 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-05T22:06:56.587 INFO:teuthology.run_tasks:Running task console_log... 2024-04-05T22:06:56.657 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d40e39160>, signals=[15]) 2024-04-05T22:06:56.658 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-05T22:06:56.665 INFO:teuthology.task.internal:Opening connections... 2024-04-05T22:06:56.665 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:06:56.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:06:56.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:06:56.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:06:56.863 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-05T22:06:56.870 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-04-05T22:06:56.888 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-04-05T22:06:56.888 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-05T22:06:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-04-05T22:06:56.946 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-05T22:06:56.946 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-05T22:06:56.946 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-05T22:06:56.946 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-04-05T22:06:56.972 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-05T22:06:56.990 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-05T22:06:56.990 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-05T22:06:57.045 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-05T22:06:57.046 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-05T22:06:57.046 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-05T22:06:57.046 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-05T22:06:57.046 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-05T22:06:57.046 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-05T22:06:57.046 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-05T22:06:57.063 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-05T22:06:57.073 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-05T22:06:57.080 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-05T22:06:57.080 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-04-05T22:06:57.083 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-05T22:06:57.099 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-05T22:06:57.107 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-05T22:06:57.107 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-04-05T22:06:57.138 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-05T22:06:57.248 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-05T22:06:57.303 INFO:teuthology.run_tasks:Running task kernel... 2024-04-05T22:06:57.320 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-05T22:06:57.320 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-05T22:06:57.320 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-05T22:06:57.321 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-05T22:06:57.321 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-05T22:06:57.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:06:57.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-05T22:06:57.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:06:57.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-05T22:06:57.593 DEBUG:teuthology.task.kernel:sha1 for client.1 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:06:57.593 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-05T22:06:57.593 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-05T22:06:57.596 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:06:57.597 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-05T22:06:57.597 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-04-05T22:06:57.610 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-361.el9.x86_64 2024-04-05T22:06:57.610 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:06:57.611 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-05T22:06:57.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum upgrade -y linux-firmware 2024-04-05T22:06:57.613 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-361.el9.x86_64 2024-04-05T22:06:57.614 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:06:57.614 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-05T22:06:57.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum upgrade -y linux-firmware 2024-04-05T22:07:00.010 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-04-05T22:07:00.536 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS-9 - AppStream 6.4 MB/s | 8.1 MB 00:01 2024-04-05T22:07:02.310 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.5 MB 00:01 2024-04-05T22:07:02.536 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.5 MB 00:00 2024-04-05T22:07:04.054 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.0 MB 00:01 2024-04-05T22:07:06.228 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-05T22:07:10.406 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.1 MB 00:00 2024-04-05T22:07:12.039 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-05T22:07:12.141 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 877 kB/s | 8.0 MB 00:09 2024-04-05T22:07:13.847 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-05T22:07:14.567 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-05T22:07:18.818 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.1 MB 00:00 2024-04-05T22:07:18.981 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-05T22:07:20.443 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-05T22:07:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:Modular dependency problems: 2024-04-05T22:07:21.101 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 1: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 2: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 3: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 4: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 5: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 6: conflicting requests 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.102 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 7: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 8: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 9: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 10: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 11: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 12: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 13: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 14: conflicting requests 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.103 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 15: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 16: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 17: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 18: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 19: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 20: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 21: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 22: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 23: conflicting requests 2024-04-05T22:07:21.104 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 24: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 25: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 26: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 27: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 28: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 29: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 30: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 31: conflicting requests 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.105 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 32: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 33: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 34: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 35: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 36: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 37: conflicting requests 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 38: cannot install the best candidate for the job 2024-04-05T22:07:21.106 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 39: conflicting requests 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 40: cannot install the best candidate for the job 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.107 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 41: cannot install the best candidate for the job 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 42: cannot install the best candidate for the job 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 43: cannot install the best candidate for the job 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 44: cannot install the best candidate for the job 2024-04-05T22:07:21.108 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 45: cannot install the best candidate for the job 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.109 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-05T22:07:21.662 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 1 Package 2024-04-05T22:07:21.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:07:21.677 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 376 M 2024-04-05T22:07:21.677 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-04-05T22:07:22.755 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-04-05T22:07:27.632 INFO:teuthology.orchestra.run.smithi073.stdout:linux-firmware-20240219-143.el9.noarch.rpm 66 MB/s | 376 MB 00:05 2024-04-05T22:07:27.632 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-04-05T22:07:27.632 INFO:teuthology.orchestra.run.smithi073.stdout:Total 63 MB/s | 376 MB 00:05 2024-04-05T22:07:27.719 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-05T22:07:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:Modular dependency problems: 2024-04-05T22:07:29.891 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 1: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 2: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 3: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 4: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 5: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 6: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 7: conflicting requests 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.892 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 8: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 9: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 10: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 11: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 12: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 13: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 14: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 15: conflicting requests 2024-04-05T22:07:29.893 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 16: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 17: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 18: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 19: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 20: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 21: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 22: conflicting requests 2024-04-05T22:07:29.894 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 23: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 24: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 25: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 26: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 27: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 28: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 29: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 30: conflicting requests 2024-04-05T22:07:29.895 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 31: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 32: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 33: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 34: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 35: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 36: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 37: conflicting requests 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.896 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 38: cannot install the best candidate for the job 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 39: conflicting requests 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 40: cannot install the best candidate for the job 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.897 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 41: cannot install the best candidate for the job 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 42: cannot install the best candidate for the job 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 43: cannot install the best candidate for the job 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.898 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 44: cannot install the best candidate for the job 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 45: cannot install the best candidate for the job 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-05T22:07:29.899 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-05T22:07:30.404 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-05T22:07:30.418 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 1 Package 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 376 M 2024-04-05T22:07:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-05T22:07:30.656 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-04-05T22:07:30.824 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-04-05T22:07:30.825 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-04-05T22:07:34.046 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-04-05T22:07:34.046 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-04-05T22:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:linux-firmware-20240219-143.el9.noarch.rpm 70 MB/s | 376 MB 00:05 2024-04-05T22:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-05T22:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:Total 69 MB/s | 376 MB 00:05 2024-04-05T22:07:38.870 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-05T22:07:39.021 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-05T22:07:39.021 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-05T22:07:41.081 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-04-05T22:07:41.484 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-05T22:07:41.568 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:42.250 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-05T22:07:42.250 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-05T22:07:42.919 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:42.919 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-05T22:07:43.368 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:43.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:07:43.368 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-04-05T22:07:43.369 INFO:teuthology.orchestra.run.smithi073.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-05T22:07:43.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:07:43.369 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-04-05T22:07:43.685 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-05T22:07:43.685 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:07:43.947 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-05T22:07:43.947 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-05T22:07:43.950 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi073.front.sepia.ceph.com to complete... 2024-04-05T22:07:49.545 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-05T22:07:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-05T22:07:49.989 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:51.244 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:51.244 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-05T22:07:51.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-05T22:07:51.640 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:07:51.640 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-05T22:07:51.640 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-05T22:07:51.640 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:07:51.641 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-05T22:07:51.948 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-04-05T22:07:51.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-05T22:07:52.177 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-05T22:07:52.178 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-05T22:07:52.180 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-04-05T22:08:31.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05 22:08:31 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-05T22:08:31.505 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2024-04-05 22:08:31-- 2024-04-05T22:08:31.505 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 39s 2024-04-05T22:08:31.505 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 321M in 39s (8.21 MB/s) 2024-04-05T22:08:31.508 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-04-05T22:08:31.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-05T22:08:31.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05 22:08:31 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-05T22:08:31.813 INFO:teuthology.orchestra.run.smithi073.stderr:FINISHED --2024-04-05 22:08:31-- 2024-04-05T22:08:31.813 INFO:teuthology.orchestra.run.smithi073.stderr:Total wall clock time: 48s 2024-04-05T22:08:31.813 INFO:teuthology.orchestra.run.smithi073.stderr:Downloaded: 1 files, 321M in 48s (6.75 MB/s) 2024-04-05T22:08:31.816 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-05T22:08:31.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-05T22:08:33.445 INFO:teuthology.orchestra.run.smithi157.stdout:Verifying... ######################################## 2024-04-05T22:08:33.498 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing... ######################################## 2024-04-05T22:08:33.577 INFO:teuthology.orchestra.run.smithi157.stdout:Updating / installing... 2024-04-05T22:08:33.749 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying... ######################################## 2024-04-05T22:08:33.802 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing... ######################################## 2024-04-05T22:08:33.881 INFO:teuthology.orchestra.run.smithi073.stdout:Updating / installing... 2024-04-05T22:08:41.157 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-05T22:08:41.380 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-05T22:08:55.168 INFO:teuthology.orchestra.run.smithi157.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-05T22:08:56.155 INFO:teuthology.orchestra.run.smithi073.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-05T22:09:17.466 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-05T22:09:17.466 DEBUG:teuthology.orchestra.run.smithi157:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-05T22:09:17.535 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-05T22:09:17.536 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-05T22:09:17.537 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-05T22:09:17.538 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-05T22:09:17.539 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-05T22:09:17.540 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-05T22:09:17.541 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-05T22:09:17.542 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-05T22:09:17.543 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-05T22:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-05T22:09:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-05T22:09:17.546 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-05T22:09:17.547 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-05T22:09:17.548 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-05T22:09:17.549 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-05T22:09:17.550 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-05T22:09:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-05T22:09:17.552 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-05T22:09:17.553 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-05T22:09:17.554 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-05T22:09:17.555 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-05T22:09:17.556 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-05T22:09:17.557 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-05T22:09:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-05T22:09:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-05T22:09:17.560 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-05T22:09:17.561 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-05T22:09:17.562 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-05T22:09:17.563 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-05T22:09:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-05T22:09:17.565 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-05T22:09:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-05T22:09:17.567 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-05T22:09:17.568 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-05T22:09:17.569 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-05T22:09:17.570 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-05T22:09:17.571 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-05T22:09:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-05T22:09:17.573 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-05T22:09:17.574 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-05T22:09:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-05T22:09:17.576 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-05T22:09:17.577 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-05T22:09:17.578 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-05T22:09:17.579 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-05T22:09:17.580 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-05T22:09:17.581 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-05T22:09:17.582 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-05T22:09:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-05T22:09:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-05T22:09:17.585 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-05T22:09:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-05T22:09:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-05T22:09:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-05T22:09:17.589 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-05T22:09:17.590 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-05T22:09:17.591 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-05T22:09:17.592 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-05T22:09:17.593 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-05T22:09:17.594 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-05T22:09:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-05T22:09:17.596 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-05T22:09:17.597 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-05T22:09:17.598 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-05T22:09:17.599 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-05T22:09:17.600 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-05T22:09:17.601 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-05T22:09:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-05T22:09:17.603 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-05T22:09:17.604 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-05T22:09:17.605 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-05T22:09:17.606 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-05T22:09:17.607 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-05T22:09:17.608 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-05T22:09:17.609 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-05T22:09:17.610 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-05T22:09:17.611 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-05T22:09:17.612 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-05T22:09:17.613 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-05T22:09:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-05T22:09:17.615 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-05T22:09:17.616 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-05T22:09:17.617 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-05T22:09:17.618 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-05T22:09:17.619 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-05T22:09:17.620 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-05T22:09:17.621 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-05T22:09:17.622 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-05T22:09:17.623 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-05T22:09:17.624 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-05T22:09:17.625 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-05T22:09:17.626 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-05T22:09:17.627 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-05T22:09:17.628 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-05T22:09:17.629 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-05T22:09:17.630 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-05T22:09:17.631 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-05T22:09:17.632 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-05T22:09:17.633 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-05T22:09:17.634 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-05T22:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-05T22:09:17.636 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-05T22:09:17.637 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-05T22:09:17.638 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-05T22:09:17.639 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-05T22:09:17.640 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-05T22:09:17.641 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-05T22:09:17.642 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-05T22:09:17.643 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-05T22:09:17.644 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-05T22:09:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-05T22:09:17.646 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-05T22:09:17.647 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-05T22:09:17.648 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-05T22:09:17.649 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-05T22:09:17.650 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-05T22:09:17.651 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-05T22:09:17.652 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-05T22:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-05T22:09:17.654 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-05T22:09:17.655 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-05T22:09:17.656 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-05T22:09:17.657 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-05T22:09:17.658 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-05T22:09:17.659 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-05T22:09:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-05T22:09:17.661 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-05T22:09:17.662 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-05T22:09:17.663 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-05T22:09:17.664 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-05T22:09:17.665 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-05T22:09:17.666 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-05T22:09:17.670 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-05T22:09:17.671 DEBUG:teuthology.orchestra.run.smithi157:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout:postinstall scriptlet (using /bin/sh): 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout:for file in vmlinuz System.map config; do 2024-04-05T22:09:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: fi 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:done 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-05T22:09:17.695 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-05T22:09:17.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-05T22:09:17.772 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-05T22:09:17.773 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-05T22:09:17.774 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-04-05T22:09:17.774 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:09:17.774 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-04-05T22:09:17.774 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-05T22:09:17.775 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 6.8.0-g1df0c7e8a235 2024-04-05T22:09:17.775 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-05T22:09:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-04-05T22:09:18.646 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-05T22:09:18.647 DEBUG:teuthology.orchestra.run.smithi073:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-05T22:09:18.719 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-05T22:09:18.720 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-05T22:09:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-05T22:09:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-05T22:09:18.723 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-05T22:09:18.724 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-05T22:09:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-05T22:09:18.726 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-05T22:09:18.727 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-05T22:09:18.728 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-05T22:09:18.729 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-05T22:09:18.730 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-05T22:09:18.731 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-05T22:09:18.732 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-05T22:09:18.733 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-05T22:09:18.734 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-05T22:09:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-05T22:09:18.736 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-05T22:09:18.737 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-05T22:09:18.738 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-05T22:09:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-05T22:09:18.740 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-05T22:09:18.741 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-05T22:09:18.742 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-05T22:09:18.743 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-05T22:09:18.744 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-05T22:09:18.745 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-05T22:09:18.746 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-05T22:09:18.747 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-05T22:09:18.748 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-05T22:09:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-05T22:09:18.750 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-05T22:09:18.751 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-05T22:09:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-05T22:09:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-05T22:09:18.754 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-05T22:09:18.755 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-05T22:09:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-05T22:09:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-05T22:09:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-05T22:09:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-05T22:09:18.760 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-05T22:09:18.761 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-05T22:09:18.762 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-05T22:09:18.763 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-05T22:09:18.764 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-05T22:09:18.765 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-05T22:09:18.766 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-05T22:09:18.767 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-05T22:09:18.768 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-05T22:09:18.769 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-05T22:09:18.770 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-05T22:09:18.771 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-05T22:09:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-05T22:09:18.773 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-05T22:09:18.774 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-05T22:09:18.775 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-05T22:09:18.776 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-05T22:09:18.777 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-05T22:09:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-05T22:09:18.779 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-05T22:09:18.780 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-05T22:09:18.781 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-05T22:09:18.782 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-05T22:09:18.783 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-05T22:09:18.784 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-05T22:09:18.785 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-05T22:09:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-05T22:09:18.787 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-05T22:09:18.788 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-05T22:09:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-05T22:09:18.790 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-05T22:09:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-05T22:09:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-05T22:09:18.793 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-05T22:09:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-05T22:09:18.795 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-05T22:09:18.796 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-05T22:09:18.797 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-05T22:09:18.798 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-05T22:09:18.799 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-05T22:09:18.800 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-05T22:09:18.801 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-05T22:09:18.802 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-05T22:09:18.803 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-05T22:09:18.804 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-05T22:09:18.805 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-05T22:09:18.806 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-05T22:09:18.807 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-05T22:09:18.808 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-05T22:09:18.809 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-05T22:09:18.810 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-05T22:09:18.811 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-05T22:09:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-05T22:09:18.813 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-05T22:09:18.814 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-05T22:09:18.815 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-05T22:09:18.816 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-05T22:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-05T22:09:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-05T22:09:18.819 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-05T22:09:18.820 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-05T22:09:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-05T22:09:18.822 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-05T22:09:18.823 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-05T22:09:18.824 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-05T22:09:18.825 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-05T22:09:18.826 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-05T22:09:18.827 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-05T22:09:18.828 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-05T22:09:18.829 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-05T22:09:18.830 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-05T22:09:18.831 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-05T22:09:18.832 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-05T22:09:18.833 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-05T22:09:18.834 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-05T22:09:18.835 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-05T22:09:18.836 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-05T22:09:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-05T22:09:18.838 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-05T22:09:18.839 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-05T22:09:18.840 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-05T22:09:18.841 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-05T22:09:18.842 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-05T22:09:18.843 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-05T22:09:18.844 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-05T22:09:18.845 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-05T22:09:18.846 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-05T22:09:18.847 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-05T22:09:18.848 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-05T22:09:18.849 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-05T22:09:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-05T22:09:18.851 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-05T22:09:18.852 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-05T22:09:18.853 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-05T22:09:18.853 INFO:teuthology.orchestra.run.smithi073.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-05T22:09:18.856 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-05T22:09:18.856 DEBUG:teuthology.orchestra.run.smithi073:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-05T22:09:18.878 INFO:teuthology.orchestra.run.smithi073.stdout:postinstall scriptlet (using /bin/sh): 2024-04-05T22:09:18.878 INFO:teuthology.orchestra.run.smithi073.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-05T22:09:18.878 INFO:teuthology.orchestra.run.smithi073.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-05T22:09:18.878 INFO:teuthology.orchestra.run.smithi073.stdout:fi 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:for file in vmlinuz System.map config; do 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout: fi 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:done 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:fi 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-05T22:09:18.879 INFO:teuthology.orchestra.run.smithi073.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-05T22:09:18.880 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-05T22:09:18.880 INFO:teuthology.orchestra.run.smithi073.stdout:fi 2024-04-05T22:09:18.880 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -qi grub2-tools 2024-04-05T22:09:18.954 INFO:teuthology.orchestra.run.smithi073.stdout:Name : grub2-tools 2024-04-05T22:09:18.954 INFO:teuthology.orchestra.run.smithi073.stdout:Epoch : 1 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Version : 2.06 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Release : 68.el9 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: x86_64 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Group : Unspecified 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Size : 8264254 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:License : GPLv3+ 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Packager : builder@centos.org 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:Vendor : CentOS 2024-04-05T22:09:18.955 INFO:teuthology.orchestra.run.smithi073.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:Summary : Support tools for GRUB. 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:Description : 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:hardware devices. 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:09:18.956 INFO:teuthology.orchestra.run.smithi073.stdout:This subpackage provides tools for support of all platforms. 2024-04-05T22:09:18.957 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-05T22:09:18.957 INFO:teuthology.task.kernel:Updating grub on smithi073 to boot 6.8.0-g1df0c7e8a235 2024-04-05T22:09:18.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-05T22:09:19.595 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-04-05T22:09:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-05T22:09:19.835 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-04-05T22:09:19.838 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-05T22:09:19.853 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.z8akAWbjMh 2024-04-05T22:09:19.853 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z8akAWbjMh 2024-04-05T22:09:19.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.z8akAWbjMh 2024-04-05T22:09:20.064 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.z8akAWbjMh is 6KB 2024-04-05T22:09:20.113 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.z8akAWbjMh 2024-04-05T22:09:20.127 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-04-05T22:09:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-05T22:09:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-05T22:09:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-05T22:09:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-05T22:09:20.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-05T22:09:20.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-04-05T22:09:21.038 INFO:teuthology.orchestra.run.smithi073.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-05T22:09:21.062 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-04-05T22:09:21.065 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-04-05T22:09:21.079 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.Iih2vbwnT8 2024-04-05T22:09:21.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Iih2vbwnT8 2024-04-05T22:09:21.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.Iih2vbwnT8 2024-04-05T22:09:21.267 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.Iih2vbwnT8 is 6KB 2024-04-05T22:09:21.286 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.Iih2vbwnT8 2024-04-05T22:09:21.300 DEBUG:teuthology.orchestra.run.smithi073:> sudo /bin/ls /boot/loader/entries || true 2024-04-05T22:09:21.370 INFO:teuthology.orchestra.run.smithi073.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-05T22:09:21.370 INFO:teuthology.orchestra.run.smithi073.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-05T22:09:21.370 INFO:teuthology.orchestra.run.smithi073.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-05T22:09:21.370 INFO:teuthology.orchestra.run.smithi073.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-05T22:09:21.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-05T22:09:21.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-04-05T22:09:50.386 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-05T22:09:50.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:09:50.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:09:51.539 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-05T22:09:51.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:09:51.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:10:08.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-05T22:10:10.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-04-05T22:10:17.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:10:17.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:10:19.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:10:19.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:10:20.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-05T22:10:22.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-04-05T22:10:32.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:10:32.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:10:34.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:10:34.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:10:35.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-05T22:10:50.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:10:50.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:11:06.004 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-04-05T22:11:06.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:11:06.617 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-04-05T22:11:06.617 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-04-05T22:11:06.657 INFO:teuthology.orchestra.run.smithi073.stdout:6.8.0-g1df0c7e8a235 2024-04-05T22:11:06.657 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-05T22:11:06.657 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-05T22:11:06.657 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-05T22:11:07.659 DEBUG:teuthology.parallel:result is None 2024-04-05T22:11:09.473 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-05T22:11:27.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:11:27.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:11:27.881 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-05T22:11:28.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:11:28.556 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-g1df0c7e8a235"... 2024-04-05T22:11:28.557 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-05T22:11:28.594 INFO:teuthology.orchestra.run.smithi157.stdout:6.8.0-g1df0c7e8a235 2024-04-05T22:11:28.594 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-05T22:11:28.594 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-05T22:11:28.594 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-05T22:11:29.595 DEBUG:teuthology.parallel:result is None 2024-04-05T22:11:29.595 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-05T22:11:29.607 INFO:teuthology.task.internal:Creating test directory... 2024-04-05T22:11:29.607 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-05T22:11:29.616 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-05T22:11:29.667 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-05T22:11:29.673 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-05T22:11:29.696 INFO:teuthology.task.internal:Creating archive directory... 2024-04-05T22:11:29.696 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-05T22:11:29.703 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-05T22:11:29.766 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-05T22:11:29.773 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-05T22:11:29.773 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-05T22:11:29.778 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-05T22:11:29.922 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-05T22:11:29.961 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-05T22:11:30.024 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-05T22:11:30.032 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-05T22:11:30.037 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-05T22:11:30.044 INFO:teuthology.task.internal:Configuring sudo... 2024-04-05T22:11:30.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-05T22:11:30.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-05T22:11:30.141 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-05T22:11:30.150 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-05T22:11:30.153 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-05T22:11:30.160 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-05T22:11:30.208 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-05T22:11:30.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-05T22:11:30.389 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-05T22:11:30.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-05T22:11:30.569 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:11:30.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-05T22:11:30.664 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-05T22:11:30.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-05T22:11:30.819 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-05T22:11:30.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-05T22:11:30.975 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:11:30.975 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-05T22:11:31.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-04-05T22:11:31.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-05T22:11:31.182 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-05T22:11:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-05T22:11:31.535 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-05T22:11:31.543 INFO:teuthology.task.internal:Starting timer... 2024-04-05T22:11:31.543 INFO:teuthology.run_tasks:Running task pcp... 2024-04-05T22:11:31.553 INFO:teuthology.run_tasks:Running task selinux... 2024-04-05T22:11:31.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-04-05T22:11:31.675 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-04-05T22:11:31.680 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-05T22:11:31.794 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-05T22:11:31.797 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-05T22:11:31.797 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-04-05T22:11:31.855 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-04-05T22:11:31.855 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-05T22:11:31.903 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-05T22:11:31.903 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-04-05T22:11:31.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-05T22:11:31.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:11:31.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-05T22:11:32.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:11:32.073 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-05T22:11:32.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-04-05T22:11:32.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-05T22:11:32.233 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-05T22:11:32.243 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-05T22:11:32.260 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-05T22:11:32.261 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-05T22:17:47.014 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-05T22:17:47.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:17:47.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:17:47.158 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-04-05T22:17:47.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-05T22:17:47.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:17:47.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-05T22:17:47.435 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-05T22:17:47.574 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-05T22:17:47.574 INFO:teuthology.run_tasks:Running task clock... 2024-04-05T22:17:47.586 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-05T22:17:47.586 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-05T22:17:47.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-05T22:17:47.594 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-05T22:17:47.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-05T22:17:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-05T22:17:47.703 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-05T22:17:47.741 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-05T22:17:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-05T22:17:47.835 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-04-05T22:17:47.836 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-05T22:17:47.876 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-05T22:17:47.878 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-04-05T22:17:47.924 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-05T22:17:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-05T22:17:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-05T22:17:47.983 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-05T22:17:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-05T22:17:48.088 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-05T22:17:48.088 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-05T22:17:48.088 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.089 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.089 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.089 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:17:48.123 INFO:teuthology.run_tasks:Running task install... 2024-04-05T22:17:48.169 DEBUG:teuthology.task.install:project ceph 2024-04-05T22:17:48.170 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'}} 2024-04-05T22:17:48.170 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'} 2024-04-05T22:17:48.170 INFO:teuthology.task.install:Using flavor: default 2024-04-05T22:17:48.176 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-05T22:17:48.176 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-04-05T22:17:48.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e', 'tag': None, 'wait_for_package': False} 2024-04-05T22:17:48.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8919605b0a5dc1f1650c8aec3051c77c0e3e240e 2024-04-05T22:17:48.179 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e', 'tag': None, 'wait_for_package': False} 2024-04-05T22:17:48.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8919605b0a5dc1f1650c8aec3051c77c0e3e240e 2024-04-05T22:17:48.341 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/ 2024-04-05T22:17:48.341 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1173.g8919605b 2024-04-05T22:17:48.345 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/ 2024-04-05T22:17:48.345 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1173.g8919605b 2024-04-05T22:17:48.460 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-05T22:17:48.460 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:17:48.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-05T22:17:48.465 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-04-04-1128-reef/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-05T22:17:48.465 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:17:48.465 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-05T22:17:48.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-05T22:17:48.562 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-05T22:17:48.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-05T22:17:48.564 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8919605b0a5dc1f1650c8aec3051c77c0e3e240e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-05T22:17:48.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-05T22:17:48.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-05T22:17:48.833 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-05T22:17:48.836 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-05T22:17:48.922 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-04-05T22:17:48.925 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-05T22:17:48.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-04-05T22:17:48.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-05T22:17:49.272 INFO:teuthology.orchestra.run.smithi073.stdout:74 files removed 2024-04-05T22:17:49.292 INFO:teuthology.orchestra.run.smithi157.stdout:74 files removed 2024-04-05T22:17:49.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-05T22:17:49.331 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-05T22:17:50.304 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 158 kB/s | 77 kB 00:00 2024-04-05T22:17:50.356 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 150 kB/s | 77 kB 00:00 2024-04-05T22:17:50.805 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-05T22:17:50.847 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-05T22:17:51.599 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-05T22:17:52.234 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-05T22:17:52.789 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 1.0 kB/s | 1.9 kB 00:01 2024-04-05T22:17:53.425 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-05T22:17:54.110 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.5 MB 00:00 2024-04-05T22:17:55.157 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-05T22:17:55.293 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.5 MB 00:00 2024-04-05T22:18:04.754 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 919 kB/s | 8.0 MB 00:08 2024-04-05T22:18:07.244 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-05T22:18:11.905 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.1 MB 00:00 2024-04-05T22:18:13.508 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-05T22:18:17.588 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 913 kB/s | 19 MB 00:21 2024-04-05T22:18:23.156 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.1 MB 00:01 2024-04-05T22:18:25.085 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-05T22:18:26.209 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-05T22:18:28.633 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 21 MB 00:14 2024-04-05T22:18:32.050 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-05T22:18:34.500 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-05T22:18:34.550 INFO:teuthology.orchestra.run.smithi157.stderr:Modular dependency problems: 2024-04-05T22:18:34.550 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 1: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 2: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 3: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 4: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 5: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 6: conflicting requests 2024-04-05T22:18:34.551 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 7: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 8: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 9: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 10: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 11: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 12: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 13: conflicting requests 2024-04-05T22:18:34.552 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 14: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 15: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 16: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 17: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 18: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 19: conflicting requests 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.553 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 20: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 21: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 22: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 23: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 24: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 25: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 26: conflicting requests 2024-04-05T22:18:34.554 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 27: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 28: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 29: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 30: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 31: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 32: conflicting requests 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.555 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 33: conflicting requests 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 34: conflicting requests 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 35: conflicting requests 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 36: conflicting requests 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 37: conflicting requests 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.556 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 38: cannot install the best candidate for the job 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 39: conflicting requests 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.557 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 40: cannot install the best candidate for the job 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 41: cannot install the best candidate for the job 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 42: cannot install the best candidate for the job 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.558 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 43: cannot install the best candidate for the job 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 44: cannot install the best candidate for the job 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.559 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: Problem 45: cannot install the best candidate for the job 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-05T22:18:34.560 INFO:teuthology.orchestra.run.smithi157.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-05T22:18:35.149 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-05T22:18:35.150 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-05T22:18:35.170 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-05T22:18:35.171 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-05T22:18:35.172 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-05T22:18:35.174 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-05T22:18:35.176 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-05T22:18:35.180 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-05T22:18:35.181 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-05T22:18:35.280 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.2.2-1173.g8919605b.el9 ceph 6.5 k 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.2.2-1173.g8919605b.el9 ceph 5.1 M 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 848 k 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1173.g8919605b.el9 ceph 143 k 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.2.2-1173.g8919605b.el9 ceph 1.5 M 2024-04-05T22:18:35.301 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 138 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 3.5 M 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 7.4 M 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 49 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.2.2-1173.g8919605b.el9 ceph 7.7 M 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.2.2-1173.g8919605b.el9 ceph 40 M 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 264 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 224 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.2.2-1173.g8919605b.el9 ceph 222 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 18 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-05T22:18:35.302 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.2.2-1173.g8919605b.el9 ceph 31 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 708 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.2.2-1173.g8919605b.el9 ceph 127 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.2.2-1173.g8919605b.el9 ceph 162 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.2.2-1173.g8919605b.el9 ceph 321 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 298 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.2.2-1173.g8919605b.el9 ceph 99 k 2024-04-05T22:18:35.303 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 87 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.0 M 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 172 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-05T22:18:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.3 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.0 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-05T22:18:35.305 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.2.2-1173.g8919605b.el9 ceph 18 M 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 24 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.2.2-1173.g8919605b.el9 ceph 2.1 M 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 246 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.2.2-1173.g8919605b.el9 ceph 4.7 M 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 17 M 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 15 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.2.2-1173.g8919605b.el9 ceph 25 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-05T22:18:35.306 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.2.2-1173.g8919605b.el9 ceph 166 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 474 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 4.5 M 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-05T22:18:35.307 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 45 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.2.2-1173.g8919605b.el9 ceph 129 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-05T22:18:35.308 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-05T22:18:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-05T22:18:35.310 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-05T22:18:35.311 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-04-05T22:18:35.312 INFO:teuthology.orchestra.run.smithi157.stdout:Install 132 Packages 2024-04-05T22:18:35.313 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 24 Packages 2024-04-05T22:18:35.313 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:18:35.313 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 207 M 2024-04-05T22:18:35.313 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-05T22:18:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:(1/156): ceph-18.2.2-1173.g8919605b.el9.x86_64. 33 kB/s | 6.5 kB 00:00 2024-04-05T22:18:36.027 INFO:teuthology.orchestra.run.smithi157.stdout:(2/156): ceph-fuse-18.2.2-1173.g8919605b.el9.x8 4.0 MB/s | 848 kB 00:00 2024-04-05T22:18:36.093 INFO:teuthology.orchestra.run.smithi157.stdout:(3/156): ceph-immutable-object-cache-18.2.2-117 2.1 MB/s | 143 kB 00:00 2024-04-05T22:18:36.312 INFO:teuthology.orchestra.run.smithi157.stdout:(4/156): ceph-mds-18.2.2-1173.g8919605b.el9.x86 9.6 MB/s | 2.1 MB 00:00 2024-04-05T22:18:36.469 INFO:teuthology.orchestra.run.smithi157.stdout:(5/156): ceph-mgr-18.2.2-1173.g8919605b.el9.x86 9.3 MB/s | 1.5 MB 00:00 2024-04-05T22:18:36.886 INFO:teuthology.orchestra.run.smithi157.stdout:(6/156): ceph-mon-18.2.2-1173.g8919605b.el9.x86 11 MB/s | 4.7 MB 00:00 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr:Modular dependency problems: 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 1: conflicting requests 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 2: conflicting requests 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 3: conflicting requests 2024-04-05T22:18:36.993 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 4: conflicting requests 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 5: conflicting requests 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 6: conflicting requests 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 7: conflicting requests 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 8: conflicting requests 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.994 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 9: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 10: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 11: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 12: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 13: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 14: conflicting requests 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.995 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 15: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 16: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 17: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 18: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 19: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 20: conflicting requests 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.996 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 21: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 22: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 23: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 24: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 25: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 26: conflicting requests 2024-04-05T22:18:36.997 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 27: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 28: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 29: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 30: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 31: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 32: conflicting requests 2024-04-05T22:18:36.998 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:36.999 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 33: conflicting requests 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 34: conflicting requests 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 35: conflicting requests 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 36: conflicting requests 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.002 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 37: conflicting requests 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 38: cannot install the best candidate for the job 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 39: conflicting requests 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.003 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 40: cannot install the best candidate for the job 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 41: cannot install the best candidate for the job 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 42: cannot install the best candidate for the job 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.004 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 43: cannot install the best candidate for the job 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 44: cannot install the best candidate for the job 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: Problem 45: cannot install the best candidate for the job 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.005 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.006 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.006 INFO:teuthology.orchestra.run.smithi073.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-05T22:18:37.561 INFO:teuthology.orchestra.run.smithi157.stdout:(7/156): ceph-common-18.2.2-1173.g8919605b.el9. 9.5 MB/s | 18 MB 00:01 2024-04-05T22:18:37.620 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-05T22:18:37.622 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-05T22:18:37.642 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-05T22:18:37.643 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-05T22:18:37.644 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-05T22:18:37.646 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-05T22:18:37.648 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-05T22:18:37.651 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-05T22:18:37.653 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-05T22:18:37.748 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-05T22:18:37.768 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================= 2024-04-05T22:18:37.768 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-04-05T22:18:37.768 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================= 2024-04-05T22:18:37.768 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-04-05T22:18:37.768 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:18.2.2-1173.g8919605b.el9 ceph 6.5 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:18.2.2-1173.g8919605b.el9 ceph 5.1 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 848 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1173.g8919605b.el9 ceph 143 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:18.2.2-1173.g8919605b.el9 ceph 1.5 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 138 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 3.5 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 7.4 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 49 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:18.2.2-1173.g8919605b.el9 ceph 7.7 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:18.2.2-1173.g8919605b.el9 ceph 40 M 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 264 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 224 k 2024-04-05T22:18:37.769 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:18.2.2-1173.g8919605b.el9 ceph 222 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 18 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:18.2.2-1173.g8919605b.el9 ceph 31 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 708 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:18.2.2-1173.g8919605b.el9 ceph 127 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-05T22:18:37.770 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:18.2.2-1173.g8919605b.el9 ceph 162 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:18.2.2-1173.g8919605b.el9 ceph 321 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 298 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:18.2.2-1173.g8919605b.el9 ceph 99 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 87 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.0 M 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 172 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-04-05T22:18:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.3 M 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 3.0 M 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-05T22:18:37.772 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:18.2.2-1173.g8919605b.el9 ceph 18 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 24 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:18.2.2-1173.g8919605b.el9 ceph 2.1 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 246 k 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:18.2.2-1173.g8919605b.el9 ceph 4.7 M 2024-04-05T22:18:37.773 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:18.2.2-1173.g8919605b.el9 ceph 17 M 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1173.g8919605b.el9 ceph-noarch 15 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:18.2.2-1173.g8919605b.el9 ceph 25 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:18.2.2-1173.g8919605b.el9 ceph 166 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-05T22:18:37.774 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 474 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:18.2.2-1173.g8919605b.el9 ceph 4.5 M 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-05T22:18:37.775 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:18.2.2-1173.g8919605b.el9 ceph 45 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:18.2.2-1173.g8919605b.el9 ceph 129 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-05T22:18:37.776 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-05T22:18:37.777 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-05T22:18:37.778 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-05T22:18:37.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-05T22:18:37.780 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================= 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout:Install 132 Packages 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 24 Packages 2024-04-05T22:18:37.781 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:18:37.782 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 207 M 2024-04-05T22:18:37.782 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-04-05T22:18:38.145 INFO:teuthology.orchestra.run.smithi157.stdout:(8/156): ceph-osd-18.2.2-1173.g8919605b.el9.x86 13 MB/s | 17 MB 00:01 2024-04-05T22:18:38.187 INFO:teuthology.orchestra.run.smithi157.stdout:(9/156): ceph-selinux-18.2.2-1173.g8919605b.el9 620 kB/s | 25 kB 00:00 2024-04-05T22:18:38.343 INFO:teuthology.orchestra.run.smithi073.stdout:(1/156): ceph-18.2.2-1173.g8919605b.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2024-04-05T22:18:38.353 INFO:teuthology.orchestra.run.smithi157.stdout:(10/156): ceph-radosgw-18.2.2-1173.g8919605b.el 9.8 MB/s | 7.7 MB 00:00 2024-04-05T22:18:38.438 INFO:teuthology.orchestra.run.smithi157.stdout:(11/156): cephfs-mirror-18.2.2-1173.g8919605b.e 2.6 MB/s | 222 kB 00:00 2024-04-05T22:18:38.513 INFO:teuthology.orchestra.run.smithi157.stdout:(12/156): libcephfs-devel-18.2.2-1173.g8919605b 418 kB/s | 31 kB 00:00 2024-04-05T22:18:38.638 INFO:teuthology.orchestra.run.smithi157.stdout:(13/156): libcephfs2-18.2.2-1173.g8919605b.el9. 5.6 MB/s | 708 kB 00:00 2024-04-05T22:18:38.730 INFO:teuthology.orchestra.run.smithi157.stdout:(14/156): libcephsqlite-18.2.2-1173.g8919605b.e 1.8 MB/s | 166 kB 00:00 2024-04-05T22:18:38.830 INFO:teuthology.orchestra.run.smithi157.stdout:(15/156): librados-devel-18.2.2-1173.g8919605b. 1.2 MB/s | 127 kB 00:00 2024-04-05T22:18:38.939 INFO:teuthology.orchestra.run.smithi157.stdout:(16/156): libradosstriper1-18.2.2-1173.g8919605 4.3 MB/s | 474 kB 00:00 2024-04-05T22:18:39.168 INFO:teuthology.orchestra.run.smithi073.stdout:(2/156): ceph-fuse-18.2.2-1173.g8919605b.el9.x8 1.0 MB/s | 848 kB 00:00 2024-04-05T22:18:39.419 INFO:teuthology.orchestra.run.smithi073.stdout:(3/156): ceph-immutable-object-cache-18.2.2-117 788 kB/s | 143 kB 00:00 2024-04-05T22:18:39.472 INFO:teuthology.orchestra.run.smithi157.stdout:(17/156): ceph-base-18.2.2-1173.g8919605b.el9.x 1.3 MB/s | 5.1 MB 00:03 2024-04-05T22:18:39.573 INFO:teuthology.orchestra.run.smithi157.stdout:(18/156): librgw2-18.2.2-1173.g8919605b.el9.x86 7.1 MB/s | 4.5 MB 00:00 2024-04-05T22:18:39.598 INFO:teuthology.orchestra.run.smithi157.stdout:(19/156): python3-ceph-argparse-18.2.2-1173.g89 362 kB/s | 45 kB 00:00 2024-04-05T22:18:39.632 INFO:teuthology.orchestra.run.smithi157.stdout:(20/156): python3-ceph-common-18.2.2-1173.g8919 2.2 MB/s | 129 kB 00:00 2024-04-05T22:18:39.666 INFO:teuthology.orchestra.run.smithi157.stdout:(21/156): python3-cephfs-18.2.2-1173.g8919605b. 2.4 MB/s | 162 kB 00:00 2024-04-05T22:18:39.724 INFO:teuthology.orchestra.run.smithi157.stdout:(22/156): python3-rados-18.2.2-1173.g8919605b.e 3.4 MB/s | 321 kB 00:00 2024-04-05T22:18:39.775 INFO:teuthology.orchestra.run.smithi157.stdout:(23/156): python3-rbd-18.2.2-1173.g8919605b.el9 2.7 MB/s | 298 kB 00:00 2024-04-05T22:18:39.825 INFO:teuthology.orchestra.run.smithi157.stdout:(24/156): python3-rgw-18.2.2-1173.g8919605b.el9 1.0 MB/s | 99 kB 00:00 2024-04-05T22:18:39.850 INFO:teuthology.orchestra.run.smithi157.stdout:(25/156): rbd-fuse-18.2.2-1173.g8919605b.el9.x8 1.1 MB/s | 87 kB 00:00 2024-04-05T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout:(26/156): rbd-nbd-18.2.2-1173.g8919605b.el9.x86 1.6 MB/s | 172 kB 00:00 2024-04-05T22:18:40.076 INFO:teuthology.orchestra.run.smithi157.stdout:(27/156): ceph-grafana-dashboards-18.2.2-1173.g 205 kB/s | 24 kB 00:00 2024-04-05T22:18:40.176 INFO:teuthology.orchestra.run.smithi157.stdout:(28/156): ceph-mgr-cephadm-18.2.2-1173.g8919605 1.4 MB/s | 138 kB 00:00 2024-04-05T22:18:40.236 INFO:teuthology.orchestra.run.smithi073.stdout:(4/156): ceph-mds-18.2.2-1173.g8919605b.el9.x86 2.6 MB/s | 2.1 MB 00:00 2024-04-05T22:18:40.251 INFO:teuthology.orchestra.run.smithi157.stdout:(29/156): rbd-mirror-18.2.2-1173.g8919605b.el9. 7.0 MB/s | 3.0 MB 00:00 2024-04-05T22:18:40.853 INFO:teuthology.orchestra.run.smithi073.stdout:(5/156): ceph-mgr-18.2.2-1173.g8919605b.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-04-05T22:18:41.036 INFO:teuthology.orchestra.run.smithi073.stdout:(6/156): ceph-base-18.2.2-1173.g8919605b.el9.x8 1.7 MB/s | 5.1 MB 00:02 2024-04-05T22:18:41.327 INFO:teuthology.orchestra.run.smithi157.stdout:(30/156): ceph-test-18.2.2-1173.g8919605b.el9.x 13 MB/s | 40 MB 00:03 2024-04-05T22:18:41.410 INFO:teuthology.orchestra.run.smithi157.stdout:(31/156): ceph-mgr-dashboard-18.2.2-1173.g89196 2.9 MB/s | 3.5 MB 00:01 2024-04-05T22:18:41.444 INFO:teuthology.orchestra.run.smithi157.stdout:(32/156): ceph-mgr-modules-core-18.2.2-1173.g89 2.1 MB/s | 246 kB 00:00 2024-04-05T22:18:41.502 INFO:teuthology.orchestra.run.smithi157.stdout:(33/156): ceph-mgr-rook-18.2.2-1173.g8919605b.e 542 kB/s | 49 kB 00:00 2024-04-05T22:18:41.550 INFO:teuthology.orchestra.run.smithi157.stdout:(34/156): ceph-prometheus-alerts-18.2.2-1173.g8 142 kB/s | 15 kB 00:00 2024-04-05T22:18:41.586 INFO:teuthology.orchestra.run.smithi157.stdout:(35/156): ceph-volume-18.2.2-1173.g8919605b.el9 3.1 MB/s | 264 kB 00:00 2024-04-05T22:18:41.728 INFO:teuthology.orchestra.run.smithi157.stdout:(36/156): ceph-mgr-diskprediction-local-18.2.2- 5.0 MB/s | 7.4 MB 00:01 2024-04-05T22:18:41.761 INFO:teuthology.orchestra.run.smithi157.stdout:(37/156): cephadm-18.2.2-1173.g8919605b.el9.noa 1.0 MB/s | 224 kB 00:00 2024-04-05T22:18:41.895 INFO:teuthology.orchestra.run.smithi157.stdout:(38/156): libaio-devel-0.3.112-1.el8.x86_64.rpm 106 kB/s | 18 kB 00:00 2024-04-05T22:18:41.920 INFO:teuthology.orchestra.run.smithi157.stdout:(39/156): cephfs-top-18.2.2-1173.g8919605b.el9. 55 kB/s | 18 kB 00:00 2024-04-05T22:18:42.145 INFO:teuthology.orchestra.run.smithi073.stdout:(7/156): ceph-mon-18.2.2-1173.g8919605b.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-04-05T22:18:42.213 INFO:teuthology.orchestra.run.smithi157.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 178 kB/s | 80 kB 00:00 2024-04-05T22:18:42.395 INFO:teuthology.orchestra.run.smithi073.stdout:(8/156): ceph-osd-18.2.2-1173.g8919605b.el9.x86 12 MB/s | 17 MB 00:01 2024-04-05T22:18:42.446 INFO:teuthology.orchestra.run.smithi073.stdout:(9/156): ceph-selinux-18.2.2-1173.g8919605b.el9 514 kB/s | 25 kB 00:00 2024-04-05T22:18:42.454 INFO:teuthology.orchestra.run.smithi157.stdout:(41/156): jq-1.6-16.el9.x86_64.rpm 335 kB/s | 187 kB 00:00 2024-04-05T22:18:42.521 INFO:teuthology.orchestra.run.smithi157.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 634 kB/s | 195 kB 00:00 2024-04-05T22:18:42.688 INFO:teuthology.orchestra.run.smithi157.stdout:(43/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 936 kB/s | 218 kB 00:00 2024-04-05T22:18:42.696 INFO:teuthology.orchestra.run.smithi073.stdout:(10/156): ceph-common-18.2.2-1173.g8919605b.el9 3.9 MB/s | 18 MB 00:04 2024-04-05T22:18:42.730 INFO:teuthology.orchestra.run.smithi157.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-05T22:18:42.763 INFO:teuthology.orchestra.run.smithi073.stdout:(11/156): cephfs-mirror-18.2.2-1173.g8919605b.e 3.3 MB/s | 222 kB 00:00 2024-04-05T22:18:42.822 INFO:teuthology.orchestra.run.smithi073.stdout:(12/156): libcephfs-devel-18.2.2-1173.g8919605b 541 kB/s | 31 kB 00:00 2024-04-05T22:18:42.830 INFO:teuthology.orchestra.run.smithi157.stdout:(45/156): libgfortran-11.4.1-3.el9.x86_64.rpm 883 kB/s | 803 kB 00:00 2024-04-05T22:18:42.856 INFO:teuthology.orchestra.run.smithi157.stdout:(46/156): python3-ply-3.11-14.el9.noarch.rpm 854 kB/s | 106 kB 00:00 2024-04-05T22:18:42.939 INFO:teuthology.orchestra.run.smithi157.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-05T22:18:42.973 INFO:teuthology.orchestra.run.smithi157.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-05T22:18:43.055 INFO:teuthology.orchestra.run.smithi073.stdout:(13/156): libcephfs2-18.2.2-1173.g8919605b.el9. 3.0 MB/s | 708 kB 00:00 2024-04-05T22:18:43.056 INFO:teuthology.orchestra.run.smithi157.stdout:(49/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-05T22:18:43.107 INFO:teuthology.orchestra.run.smithi157.stdout:(50/156): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-04-05T22:18:43.164 INFO:teuthology.orchestra.run.smithi073.stdout:(14/156): libcephsqlite-18.2.2-1173.g8919605b.e 1.5 MB/s | 166 kB 00:00 2024-04-05T22:18:43.322 INFO:teuthology.orchestra.run.smithi073.stdout:(15/156): ceph-radosgw-18.2.2-1173.g8919605b.el 6.6 MB/s | 7.7 MB 00:01 2024-04-05T22:18:43.381 INFO:teuthology.orchestra.run.smithi073.stdout:(16/156): librados-devel-18.2.2-1173.g8919605b. 587 kB/s | 127 kB 00:00 2024-04-05T22:18:43.422 INFO:teuthology.orchestra.run.smithi073.stdout:(17/156): libradosstriper1-18.2.2-1173.g8919605 4.7 MB/s | 474 kB 00:00 2024-04-05T22:18:43.424 INFO:teuthology.orchestra.run.smithi157.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 431 kB/s | 107 kB 00:00 2024-04-05T22:18:43.483 INFO:teuthology.orchestra.run.smithi157.stdout:(52/156): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 689 kB 00:00 2024-04-05T22:18:43.490 INFO:teuthology.orchestra.run.smithi073.stdout:(18/156): python3-ceph-argparse-18.2.2-1173.g89 680 kB/s | 45 kB 00:00 2024-04-05T22:18:43.508 INFO:teuthology.orchestra.run.smithi157.stdout:(53/156): elfutils-libelf-devel-0.190-2.el9.x86 259 kB/s | 21 kB 00:00 2024-04-05T22:18:43.567 INFO:teuthology.orchestra.run.smithi157.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-05T22:18:43.598 INFO:teuthology.orchestra.run.smithi073.stdout:(19/156): python3-ceph-common-18.2.2-1173.g8919 1.2 MB/s | 129 kB 00:00 2024-04-05T22:18:43.600 INFO:teuthology.orchestra.run.smithi157.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-05T22:18:43.625 INFO:teuthology.orchestra.run.smithi157.stdout:(56/156): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-04-05T22:18:43.684 INFO:teuthology.orchestra.run.smithi157.stdout:(57/156): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-05T22:18:43.724 INFO:teuthology.orchestra.run.smithi073.stdout:(20/156): python3-cephfs-18.2.2-1173.g8919605b. 1.3 MB/s | 162 kB 00:00 2024-04-05T22:18:43.734 INFO:teuthology.orchestra.run.smithi157.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-05T22:18:43.751 INFO:teuthology.orchestra.run.smithi157.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 685 kB/s | 45 kB 00:00 2024-04-05T22:18:43.843 INFO:teuthology.orchestra.run.smithi157.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-05T22:18:43.894 INFO:teuthology.orchestra.run.smithi157.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-05T22:18:43.944 INFO:teuthology.orchestra.run.smithi157.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-05T22:18:43.969 INFO:teuthology.orchestra.run.smithi157.stdout:(63/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-04-05T22:18:43.994 INFO:teuthology.orchestra.run.smithi157.stdout:(64/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 476 kB/s | 47 kB 00:00 2024-04-05T22:18:44.028 INFO:teuthology.orchestra.run.smithi157.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-05T22:18:44.053 INFO:teuthology.orchestra.run.smithi157.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 428 kB/s | 35 kB 00:00 2024-04-05T22:18:44.245 INFO:teuthology.orchestra.run.smithi157.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-05T22:18:44.374 INFO:teuthology.orchestra.run.smithi073.stdout:(21/156): librgw2-18.2.2-1173.g8919605b.el9.x86 4.5 MB/s | 4.5 MB 00:00 2024-04-05T22:18:44.396 INFO:teuthology.orchestra.run.smithi157.stdout:(68/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2024-04-05T22:18:44.449 INFO:teuthology.orchestra.run.smithi073.stdout:(22/156): python3-rbd-18.2.2-1173.g8919605b.el9 4.0 MB/s | 298 kB 00:00 2024-04-05T22:18:44.479 INFO:teuthology.orchestra.run.smithi157.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-05T22:18:44.483 INFO:teuthology.orchestra.run.smithi073.stdout:(23/156): python3-rados-18.2.2-1173.g8919605b.e 424 kB/s | 321 kB 00:00 2024-04-05T22:18:44.517 INFO:teuthology.orchestra.run.smithi073.stdout:(24/156): python3-rgw-18.2.2-1173.g8919605b.el9 1.5 MB/s | 99 kB 00:00 2024-04-05T22:18:44.550 INFO:teuthology.orchestra.run.smithi073.stdout:(25/156): rbd-fuse-18.2.2-1173.g8919605b.el9.x8 1.3 MB/s | 87 kB 00:00 2024-04-05T22:18:44.554 INFO:teuthology.orchestra.run.smithi157.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-04-05T22:18:44.613 INFO:teuthology.orchestra.run.smithi157.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 615 kB/s | 133 kB 00:00 2024-04-05T22:18:44.759 INFO:teuthology.orchestra.run.smithi157.stdout:(72/156): python3-devel-3.9.17-2.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-04-05T22:18:44.872 INFO:teuthology.orchestra.run.smithi157.stdout:(73/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-05T22:18:45.006 INFO:teuthology.orchestra.run.smithi157.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-05T22:18:45.031 INFO:teuthology.orchestra.run.smithi157.stdout:(75/156): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 26 kB 00:00 2024-04-05T22:18:45.081 INFO:teuthology.orchestra.run.smithi157.stdout:(76/156): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-04-05T22:18:45.206 INFO:teuthology.orchestra.run.smithi157.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-04-05T22:18:45.209 INFO:teuthology.orchestra.run.smithi073.stdout:(26/156): ceph-test-18.2.2-1173.g8919605b.el9.x 14 MB/s | 40 MB 00:02 2024-04-05T22:18:45.240 INFO:teuthology.orchestra.run.smithi157.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 828 kB/s | 172 kB 00:00 2024-04-05T22:18:45.242 INFO:teuthology.orchestra.run.smithi073.stdout:(27/156): rbd-nbd-18.2.2-1173.g8919605b.el9.x86 249 kB/s | 172 kB 00:00 2024-04-05T22:18:45.265 INFO:teuthology.orchestra.run.smithi157.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 190 kB/s | 35 kB 00:00 2024-04-05T22:18:45.268 INFO:teuthology.orchestra.run.smithi073.stdout:(28/156): ceph-grafana-dashboards-18.2.2-1173.g 415 kB/s | 24 kB 00:00 2024-04-05T22:18:45.310 INFO:teuthology.orchestra.run.smithi073.stdout:(29/156): ceph-mgr-cephadm-18.2.2-1173.g8919605 2.0 MB/s | 138 kB 00:00 2024-04-05T22:18:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:(80/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-05T22:18:45.383 INFO:teuthology.orchestra.run.smithi157.stdout:(81/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-05T22:18:45.410 INFO:teuthology.orchestra.run.smithi073.stdout:(30/156): rbd-mirror-18.2.2-1173.g8919605b.el9. 3.3 MB/s | 3.0 MB 00:00 2024-04-05T22:18:45.425 INFO:teuthology.orchestra.run.smithi157.stdout:(82/156): python3-packaging-20.9-5.el9.noarch.r 846 kB/s | 77 kB 00:00 2024-04-05T22:18:45.450 INFO:teuthology.orchestra.run.smithi157.stdout:(83/156): python3-prettytable-0.7.2-27.el9.noar 640 kB/s | 42 kB 00:00 2024-04-05T22:18:45.494 INFO:teuthology.orchestra.run.smithi073.stdout:(31/156): ceph-mgr-modules-core-18.2.2-1173.g89 2.9 MB/s | 246 kB 00:00 2024-04-05T22:18:45.500 INFO:teuthology.orchestra.run.smithi157.stdout:(84/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-05T22:18:45.550 INFO:teuthology.orchestra.run.smithi157.stdout:(85/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-05T22:18:45.576 INFO:teuthology.orchestra.run.smithi157.stdout:(86/156): python3-pytz-2021.1-5.el9.noarch.rpm 685 kB/s | 51 kB 00:00 2024-04-05T22:18:45.577 INFO:teuthology.orchestra.run.smithi073.stdout:(32/156): ceph-mgr-dashboard-18.2.2-1173.g89196 11 MB/s | 3.5 MB 00:00 2024-04-05T22:18:45.603 INFO:teuthology.orchestra.run.smithi073.stdout:(33/156): ceph-mgr-rook-18.2.2-1173.g8919605b.e 457 kB/s | 49 kB 00:00 2024-04-05T22:18:45.628 INFO:teuthology.orchestra.run.smithi073.stdout:(34/156): ceph-prometheus-alerts-18.2.2-1173.g8 296 kB/s | 15 kB 00:00 2024-04-05T22:18:45.662 INFO:teuthology.orchestra.run.smithi073.stdout:(35/156): ceph-volume-18.2.2-1173.g8919605b.el9 4.4 MB/s | 264 kB 00:00 2024-04-05T22:18:45.692 INFO:teuthology.orchestra.run.smithi157.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-05T22:18:45.695 INFO:teuthology.orchestra.run.smithi073.stdout:(36/156): cephadm-18.2.2-1173.g8919605b.el9.noa 3.3 MB/s | 224 kB 00:00 2024-04-05T22:18:45.718 INFO:teuthology.orchestra.run.smithi157.stdout:(88/156): python3-requests-oauthlib-1.3.0-12.el 322 kB/s | 54 kB 00:00 2024-04-05T22:18:45.721 INFO:teuthology.orchestra.run.smithi073.stdout:(37/156): cephfs-top-18.2.2-1173.g8919605b.el9. 315 kB/s | 18 kB 00:00 2024-04-05T22:18:45.768 INFO:teuthology.orchestra.run.smithi157.stdout:(89/156): python3-toml-0.10.2-6.el9.noarch.rpm 556 kB/s | 42 kB 00:00 2024-04-05T22:18:45.835 INFO:teuthology.orchestra.run.smithi157.stdout:(90/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-05T22:18:45.846 INFO:teuthology.orchestra.run.smithi073.stdout:(38/156): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 18 kB 00:00 2024-04-05T22:18:45.886 INFO:teuthology.orchestra.run.smithi157.stdout:(91/156): uuid-1.6.2-55.el9.x86_64.rpm 498 kB/s | 58 kB 00:00 2024-04-05T22:18:45.905 INFO:teuthology.orchestra.run.smithi073.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 439 kB/s | 80 kB 00:00 2024-04-05T22:18:45.919 INFO:teuthology.orchestra.run.smithi157.stdout:(92/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-05T22:18:45.970 INFO:teuthology.orchestra.run.smithi157.stdout:(93/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 768 kB/s | 64 kB 00:00 2024-04-05T22:18:46.003 INFO:teuthology.orchestra.run.smithi157.stdout:(94/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 545 kB/s | 45 kB 00:00 2024-04-05T22:18:46.030 INFO:teuthology.orchestra.run.smithi073.stdout:(40/156): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-04-05T22:18:46.072 INFO:teuthology.orchestra.run.smithi073.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-04-05T22:18:46.086 INFO:teuthology.orchestra.run.smithi157.stdout:(95/156): fmt-8.1.1-5.el9.x86_64.rpm 956 kB/s | 111 kB 00:00 2024-04-05T22:18:46.105 INFO:teuthology.orchestra.run.smithi073.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-05T22:18:46.120 INFO:teuthology.orchestra.run.smithi157.stdout:(96/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.6 MB/s | 309 kB 00:00 2024-04-05T22:18:46.139 INFO:teuthology.orchestra.run.smithi073.stdout:(43/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 218 kB 00:00 2024-04-05T22:18:46.162 INFO:teuthology.orchestra.run.smithi157.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 636 kB/s | 26 kB 00:00 2024-04-05T22:18:46.173 INFO:teuthology.orchestra.run.smithi073.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-05T22:18:46.204 INFO:teuthology.orchestra.run.smithi157.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-04-05T22:18:46.240 INFO:teuthology.orchestra.run.smithi073.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-05T22:18:46.246 INFO:teuthology.orchestra.run.smithi157.stdout:(99/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-05T22:18:46.290 INFO:teuthology.orchestra.run.smithi073.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-04-05T22:18:46.321 INFO:teuthology.orchestra.run.smithi157.stdout:(100/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-05T22:18:46.340 INFO:teuthology.orchestra.run.smithi073.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-05T22:18:46.382 INFO:teuthology.orchestra.run.smithi073.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-05T22:18:46.416 INFO:teuthology.orchestra.run.smithi073.stdout:(49/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-05T22:18:46.422 INFO:teuthology.orchestra.run.smithi157.stdout:(101/156): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-05T22:18:46.462 INFO:teuthology.orchestra.run.smithi157.stdout:(102/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-05T22:18:46.487 INFO:teuthology.orchestra.run.smithi157.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 674 kB/s | 43 kB 00:00 2024-04-05T22:18:46.513 INFO:teuthology.orchestra.run.smithi157.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 650 kB/s | 32 kB 00:00 2024-04-05T22:18:46.538 INFO:teuthology.orchestra.run.smithi157.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-04-05T22:18:46.580 INFO:teuthology.orchestra.run.smithi157.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-05T22:18:46.614 INFO:teuthology.orchestra.run.smithi157.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-05T22:18:46.647 INFO:teuthology.orchestra.run.smithi157.stdout:(108/156): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-05T22:18:46.667 INFO:teuthology.orchestra.run.smithi073.stdout:(50/156): ceph-mgr-diskprediction-local-18.2.2- 5.5 MB/s | 7.4 MB 00:01 2024-04-05T22:18:46.673 INFO:teuthology.orchestra.run.smithi157.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-04-05T22:18:46.698 INFO:teuthology.orchestra.run.smithi157.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 361 kB/s | 18 kB 00:00 2024-04-05T22:18:46.724 INFO:teuthology.orchestra.run.smithi157.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 467 kB/s | 23 kB 00:00 2024-04-05T22:18:46.749 INFO:teuthology.orchestra.run.smithi157.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-04-05T22:18:46.765 INFO:teuthology.orchestra.run.smithi157.stdout:(113/156): python3-jaraco-text-3.2.0-6.el9.noar 479 kB/s | 20 kB 00:00 2024-04-05T22:18:46.799 INFO:teuthology.orchestra.run.smithi157.stdout:(114/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-04-05T22:18:46.841 INFO:teuthology.orchestra.run.smithi157.stdout:(115/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-05T22:18:46.858 INFO:teuthology.orchestra.run.smithi157.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-05T22:18:46.867 INFO:teuthology.orchestra.run.smithi073.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 537 kB/s | 107 kB 00:00 2024-04-05T22:18:46.875 INFO:teuthology.orchestra.run.smithi157.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-04-05T22:18:46.917 INFO:teuthology.orchestra.run.smithi073.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-05T22:18:46.934 INFO:teuthology.orchestra.run.smithi073.stdout:(53/156): elfutils-libelf-devel-0.190-2.el9.x86 326 kB/s | 21 kB 00:00 2024-04-05T22:18:46.942 INFO:teuthology.orchestra.run.smithi157.stdout:(118/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-05T22:18:46.959 INFO:teuthology.orchestra.run.smithi157.stdout:(119/156): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-04-05T22:18:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:(54/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-05T22:18:46.993 INFO:teuthology.orchestra.run.smithi157.stdout:(120/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-05T22:18:47.010 INFO:teuthology.orchestra.run.smithi157.stdout:(121/156): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2024-04-05T22:18:47.010 INFO:teuthology.orchestra.run.smithi073.stdout:(55/156): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-04-05T22:18:47.043 INFO:teuthology.orchestra.run.smithi157.stdout:(122/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-05T22:18:47.066 INFO:teuthology.orchestra.run.smithi157.stdout:(123/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-05T22:18:47.068 INFO:teuthology.orchestra.run.smithi073.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-05T22:18:47.094 INFO:teuthology.orchestra.run.smithi157.stdout:(124/156): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-04-05T22:18:47.102 INFO:teuthology.orchestra.run.smithi073.stdout:(57/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-05T22:18:47.111 INFO:teuthology.orchestra.run.smithi157.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 1.1 MB/s | 51 kB 00:00 2024-04-05T22:18:47.136 INFO:teuthology.orchestra.run.smithi073.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-05T22:18:47.145 INFO:teuthology.orchestra.run.smithi157.stdout:(126/156): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-05T22:18:47.162 INFO:teuthology.orchestra.run.smithi157.stdout:(127/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-05T22:18:47.169 INFO:teuthology.orchestra.run.smithi073.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 682 kB/s | 45 kB 00:00 2024-04-05T22:18:47.204 INFO:teuthology.orchestra.run.smithi157.stdout:(128/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-05T22:18:47.229 INFO:teuthology.orchestra.run.smithi157.stdout:(129/156): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2024-04-05T22:18:47.254 INFO:teuthology.orchestra.run.smithi157.stdout:(130/156): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-04-05T22:18:47.321 INFO:teuthology.orchestra.run.smithi073.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-05T22:18:47.338 INFO:teuthology.orchestra.run.smithi157.stdout:(131/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-05T22:18:47.387 INFO:teuthology.orchestra.run.smithi073.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 393 kB/s | 25 kB 00:00 2024-04-05T22:18:47.429 INFO:teuthology.orchestra.run.smithi073.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 2.2 MB/s | 582 kB 00:00 2024-04-05T22:18:47.454 INFO:teuthology.orchestra.run.smithi073.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 717 kB/s | 47 kB 00:00 2024-04-05T22:18:47.513 INFO:teuthology.orchestra.run.smithi073.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 613 kB/s | 35 kB 00:00 2024-04-05T22:18:47.580 INFO:teuthology.orchestra.run.smithi073.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-05T22:18:47.655 INFO:teuthology.orchestra.run.smithi157.stdout:(132/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.8 MB/s | 16 MB 00:02 2024-04-05T22:18:47.788 INFO:teuthology.orchestra.run.smithi073.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-04-05T22:18:48.055 INFO:teuthology.orchestra.run.smithi073.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 6.0 MB/s | 1.6 MB 00:00 2024-04-05T22:18:48.139 INFO:teuthology.orchestra.run.smithi157.stdout:(133/156): librados2-18.2.2-1173.g8919605b.el9. 3.7 MB/s | 3.3 MB 00:00 2024-04-05T22:18:48.155 INFO:teuthology.orchestra.run.smithi073.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 7.4 MB/s | 4.7 MB 00:00 2024-04-05T22:18:48.247 INFO:teuthology.orchestra.run.smithi073.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-05T22:18:48.298 INFO:teuthology.orchestra.run.smithi073.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-04-05T22:18:48.498 INFO:teuthology.orchestra.run.smithi073.stdout:(71/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 4.1 MB 00:00 2024-04-05T22:18:48.505 INFO:teuthology.orchestra.run.smithi157.stdout:(134/156): NetworkManager-1.47.2-1.el9.x86_64.r 2.7 MB/s | 2.3 MB 00:00 2024-04-05T22:18:48.582 INFO:teuthology.orchestra.run.smithi073.stdout:(72/156): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-04-05T22:18:48.631 INFO:teuthology.orchestra.run.smithi157.stdout:(135/156): NetworkManager-team-1.47.2-1.el9.x86 339 kB/s | 42 kB 00:00 2024-04-05T22:18:48.657 INFO:teuthology.orchestra.run.smithi073.stdout:(73/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-04-05T22:18:48.708 INFO:teuthology.orchestra.run.smithi073.stdout:(74/156): python3-jsonpatch-1.21-16.el9.noarch. 522 kB/s | 26 kB 00:00 2024-04-05T22:18:48.739 INFO:teuthology.orchestra.run.smithi157.stdout:(136/156): NetworkManager-tui-1.47.2-1.el9.x86_ 2.3 MB/s | 248 kB 00:00 2024-04-05T22:18:48.825 INFO:teuthology.orchestra.run.smithi073.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-05T22:18:48.839 INFO:teuthology.orchestra.run.smithi157.stdout:(137/156): acl-2.3.1-4.el9.x86_64.rpm 717 kB/s | 71 kB 00:00 2024-04-05T22:18:48.841 INFO:teuthology.orchestra.run.smithi073.stdout:(76/156): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-05T22:18:48.948 INFO:teuthology.orchestra.run.smithi157.stdout:(138/156): elfutils-debuginfod-client-0.190-2.e 338 kB/s | 36 kB 00:00 2024-04-05T22:18:48.958 INFO:teuthology.orchestra.run.smithi073.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.0 MB/s | 6.0 MB 00:00 2024-04-05T22:18:48.975 INFO:teuthology.orchestra.run.smithi073.stdout:(78/156): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-04-05T22:18:49.001 INFO:teuthology.orchestra.run.smithi073.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 984 kB/s | 172 kB 00:00 2024-04-05T22:18:49.043 INFO:teuthology.orchestra.run.smithi073.stdout:(80/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-05T22:18:49.057 INFO:teuthology.orchestra.run.smithi157.stdout:(139/156): NetworkManager-libnm-1.47.2-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-04-05T22:18:49.076 INFO:teuthology.orchestra.run.smithi073.stdout:(81/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-05T22:18:49.090 INFO:teuthology.orchestra.run.smithi157.stdout:(140/156): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-04-05T22:18:49.109 INFO:teuthology.orchestra.run.smithi073.stdout:(82/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-05T22:18:49.135 INFO:teuthology.orchestra.run.smithi073.stdout:(83/156): python3-prettytable-0.7.2-27.el9.noar 728 kB/s | 42 kB 00:00 2024-04-05T22:18:49.174 INFO:teuthology.orchestra.run.smithi157.stdout:(141/156): elfutils-libs-0.190-2.el9.x86_64.rpm 2.2 MB/s | 256 kB 00:00 2024-04-05T22:18:49.177 INFO:teuthology.orchestra.run.smithi073.stdout:(84/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-05T22:18:49.208 INFO:teuthology.orchestra.run.smithi157.stdout:(142/156): libacl-2.3.1-4.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-04-05T22:18:49.244 INFO:teuthology.orchestra.run.smithi073.stdout:(85/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-05T22:18:49.261 INFO:teuthology.orchestra.run.smithi073.stdout:(86/156): python3-pytz-2021.1-5.el9.noarch.rpm 615 kB/s | 51 kB 00:00 2024-04-05T22:18:49.283 INFO:teuthology.orchestra.run.smithi157.stdout:(143/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 108 kB 00:00 2024-04-05T22:18:49.295 INFO:teuthology.orchestra.run.smithi073.stdout:(87/156): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-05T22:18:49.316 INFO:teuthology.orchestra.run.smithi157.stdout:(144/156): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-04-05T22:18:49.392 INFO:teuthology.orchestra.run.smithi157.stdout:(145/156): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-04-05T22:18:49.403 INFO:teuthology.orchestra.run.smithi073.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-05T22:18:49.420 INFO:teuthology.orchestra.run.smithi073.stdout:(89/156): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-05T22:18:49.425 INFO:teuthology.orchestra.run.smithi157.stdout:(146/156): libsmartcols-2.37.4-18.el9.x86_64.rp 591 kB/s | 64 kB 00:00 2024-04-05T22:18:49.471 INFO:teuthology.orchestra.run.smithi073.stdout:(90/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-05T22:18:49.488 INFO:teuthology.orchestra.run.smithi073.stdout:(91/156): uuid-1.6.2-55.el9.x86_64.rpm 871 kB/s | 58 kB 00:00 2024-04-05T22:18:49.500 INFO:teuthology.orchestra.run.smithi157.stdout:(147/156): libuuid-2.37.4-18.el9.x86_64.rpm 266 kB/s | 29 kB 00:00 2024-04-05T22:18:49.546 INFO:teuthology.orchestra.run.smithi073.stdout:(92/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-05T22:18:49.563 INFO:teuthology.orchestra.run.smithi073.stdout:(93/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 854 kB/s | 64 kB 00:00 2024-04-05T22:18:49.597 INFO:teuthology.orchestra.run.smithi073.stdout:(94/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 913 kB/s | 45 kB 00:00 2024-04-05T22:18:49.660 INFO:teuthology.orchestra.run.smithi157.stdout:(148/156): openssl-3.0.7-27.el9.x86_64.rpm 5.2 MB/s | 1.2 MB 00:00 2024-04-05T22:18:49.818 INFO:teuthology.orchestra.run.smithi157.stdout:(149/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.8 MB/s | 2.2 MB 00:00 2024-04-05T22:18:49.910 INFO:teuthology.orchestra.run.smithi157.stdout:(150/156): util-linux-2.37.4-18.el9.x86_64.rpm 9.2 MB/s | 2.3 MB 00:00 2024-04-05T22:18:49.954 INFO:teuthology.orchestra.run.smithi157.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-04-05T22:18:49.997 INFO:teuthology.orchestra.run.smithi073.stdout:(95/156): fmt-8.1.1-5.el9.x86_64.rpm 256 kB/s | 111 kB 00:00 2024-04-05T22:18:50.029 INFO:teuthology.orchestra.run.smithi157.stdout:(152/156): librbd1-18.2.2-1173.g8919605b.el9.x8 1.1 MB/s | 3.0 MB 00:02 2024-04-05T22:18:50.113 INFO:teuthology.orchestra.run.smithi157.stdout:(153/156): util-linux-user-2.37.4-18.el9.x86_64 154 kB/s | 31 kB 00:00 2024-04-05T22:18:50.181 INFO:teuthology.orchestra.run.smithi073.stdout:(96/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 530 kB/s | 309 kB 00:00 2024-04-05T22:18:50.197 INFO:teuthology.orchestra.run.smithi157.stdout:(154/156): xfsdump-3.1.12-4.el9.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-04-05T22:18:50.222 INFO:teuthology.orchestra.run.smithi157.stdout:(155/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 690 kB/s | 74 kB 00:00 2024-04-05T22:18:50.298 INFO:teuthology.orchestra.run.smithi073.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-05T22:18:50.515 INFO:teuthology.orchestra.run.smithi073.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-05T22:18:50.540 INFO:teuthology.orchestra.run.smithi073.stdout:(99/156): liboath-2.6.7-2.el9.x86_64.rpm 201 kB/s | 49 kB 00:00 2024-04-05T22:18:50.799 INFO:teuthology.orchestra.run.smithi073.stdout:(100/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-05T22:18:50.899 INFO:teuthology.orchestra.run.smithi073.stdout:(101/156): libarrow-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-04-05T22:18:50.932 INFO:teuthology.orchestra.run.smithi073.stdout:(102/156): libunwind-1.6.2-1.el9.x86_64.rpm 162 kB/s | 67 kB 00:00 2024-04-05T22:18:50.972 INFO:teuthology.orchestra.run.smithi157.stdout:(156/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 1.1 MB 00:00 2024-04-05T22:18:50.974 INFO:teuthology.orchestra.run.smithi073.stdout:(103/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-05T22:18:50.982 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-05T22:18:50.983 INFO:teuthology.orchestra.run.smithi157.stdout:Total 13 MB/s | 207 MB 00:15 2024-04-05T22:18:51.016 INFO:teuthology.orchestra.run.smithi073.stdout:(104/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 373 kB/s | 43 kB 00:00 2024-04-05T22:18:51.083 INFO:teuthology.orchestra.run.smithi073.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-04-05T22:18:51.159 INFO:teuthology.orchestra.run.smithi073.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-05T22:18:51.209 INFO:teuthology.orchestra.run.smithi073.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-05T22:18:51.268 INFO:teuthology.orchestra.run.smithi073.stdout:(108/156): python3-cachetools-4.2.4-1.el9.noarc 96 kB/s | 32 kB 00:00 2024-04-05T22:18:51.293 INFO:teuthology.orchestra.run.smithi073.stdout:(109/156): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-04-05T22:18:51.310 INFO:teuthology.orchestra.run.smithi073.stdout:(110/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-05T22:18:51.377 INFO:teuthology.orchestra.run.smithi073.stdout:(111/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-05T22:18:51.402 INFO:teuthology.orchestra.run.smithi073.stdout:(112/156): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-05T22:18:51.427 INFO:teuthology.orchestra.run.smithi073.stdout:(113/156): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-05T22:18:51.486 INFO:teuthology.orchestra.run.smithi073.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-05T22:18:51.528 INFO:teuthology.orchestra.run.smithi073.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-05T22:18:51.578 INFO:teuthology.orchestra.run.smithi073.stdout:(116/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-05T22:18:51.629 INFO:teuthology.orchestra.run.smithi073.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-04-05T22:18:51.654 INFO:teuthology.orchestra.run.smithi073.stdout:(118/156): python3-more-itertools-8.12.0-2.el9. 472 kB/s | 79 kB 00:00 2024-04-05T22:18:51.704 INFO:teuthology.orchestra.run.smithi073.stdout:(119/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-05T22:18:51.738 INFO:teuthology.orchestra.run.smithi073.stdout:(120/156): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-04-05T22:18:51.814 INFO:teuthology.orchestra.run.smithi073.stdout:(121/156): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-05T22:18:51.839 INFO:teuthology.orchestra.run.smithi073.stdout:(122/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 490 kB/s | 90 kB 00:00 2024-04-05T22:18:51.872 INFO:teuthology.orchestra.run.smithi073.stdout:(123/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-05T22:18:51.923 INFO:teuthology.orchestra.run.smithi073.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 548 kB/s | 59 kB 00:00 2024-04-05T22:18:51.956 INFO:teuthology.orchestra.run.smithi073.stdout:(125/156): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-04-05T22:18:51.981 INFO:teuthology.orchestra.run.smithi073.stdout:(126/156): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-04-05T22:18:52.048 INFO:teuthology.orchestra.run.smithi073.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-05T22:18:52.124 INFO:teuthology.orchestra.run.smithi073.stdout:(128/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-05T22:18:52.149 INFO:teuthology.orchestra.run.smithi073.stdout:(129/156): python3-websocket-client-1.2.3-2.el9 466 kB/s | 90 kB 00:00 2024-04-05T22:18:52.174 INFO:teuthology.orchestra.run.smithi073.stdout:(130/156): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-04-05T22:18:52.236 INFO:teuthology.orchestra.run.smithi073.stdout:(131/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-05T22:18:52.370 INFO:teuthology.orchestra.run.smithi073.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 7.2 MB/s | 1.6 MB 00:00 2024-04-05T22:18:52.637 INFO:teuthology.orchestra.run.smithi073.stdout:(133/156): librbd1-18.2.2-1173.g8919605b.el9.x8 7.5 MB/s | 3.0 MB 00:00 2024-04-05T22:18:52.721 INFO:teuthology.orchestra.run.smithi073.stdout:(134/156): NetworkManager-1.47.2-1.el9.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2024-04-05T22:18:52.763 INFO:teuthology.orchestra.run.smithi073.stdout:(135/156): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-04-05T22:18:52.846 INFO:teuthology.orchestra.run.smithi073.stdout:(136/156): librados2-18.2.2-1173.g8919605b.el9. 4.9 MB/s | 3.3 MB 00:00 2024-04-05T22:18:52.880 INFO:teuthology.orchestra.run.smithi073.stdout:(137/156): NetworkManager-tui-1.47.2-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-04-05T22:18:52.922 INFO:teuthology.orchestra.run.smithi073.stdout:(138/156): elfutils-debuginfod-client-0.190-2.e 907 kB/s | 36 kB 00:00 2024-04-05T22:18:52.981 INFO:teuthology.orchestra.run.smithi073.stdout:(139/156): NetworkManager-libnm-1.47.2-1.el9.x8 5.4 MB/s | 1.8 MB 00:00 2024-04-05T22:18:53.006 INFO:teuthology.orchestra.run.smithi073.stdout:(140/156): acl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 71 kB 00:00 2024-04-05T22:18:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-04-05T22:18:53.082 INFO:teuthology.orchestra.run.smithi073.stdout:(142/156): libacl-2.3.1-4.el9.x86_64.rpm 310 kB/s | 23 kB 00:00 2024-04-05T22:18:53.115 INFO:teuthology.orchestra.run.smithi073.stdout:(143/156): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-04-05T22:18:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:(144/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-05T22:18:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:(145/156): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-04-05T22:18:53.191 INFO:teuthology.orchestra.run.smithi073.stdout:(146/156): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-04-05T22:18:53.217 INFO:teuthology.orchestra.run.smithi073.stdout:(147/156): libsmartcols-2.37.4-18.el9.x86_64.rp 852 kB/s | 64 kB 00:00 2024-04-05T22:18:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:(148/156): libuuid-2.37.4-18.el9.x86_64.rpm 383 kB/s | 29 kB 00:00 2024-04-05T22:18:53.317 INFO:teuthology.orchestra.run.smithi073.stdout:(149/156): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-04-05T22:18:53.409 INFO:teuthology.orchestra.run.smithi073.stdout:(150/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-05T22:18:53.451 INFO:teuthology.orchestra.run.smithi073.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-04-05T22:18:53.476 INFO:teuthology.orchestra.run.smithi073.stdout:(152/156): util-linux-user-2.37.4-18.el9.x86_64 471 kB/s | 31 kB 00:00 2024-04-05T22:18:53.527 INFO:teuthology.orchestra.run.smithi073.stdout:(153/156): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2024-04-05T22:18:53.594 INFO:teuthology.orchestra.run.smithi073.stdout:(154/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.3 MB/s | 1.1 MB 00:00 2024-04-05T22:18:53.661 INFO:teuthology.orchestra.run.smithi073.stdout:(155/156): util-linux-2.37.4-18.el9.x86_64.rpm 5.5 MB/s | 2.3 MB 00:00 2024-04-05T22:18:53.711 INFO:teuthology.orchestra.run.smithi073.stdout:(156/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 406 kB/s | 74 kB 00:00 2024-04-05T22:18:53.717 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-04-05T22:18:53.718 INFO:teuthology.orchestra.run.smithi073.stdout:Total 13 MB/s | 207 MB 00:15 2024-04-05T22:18:54.343 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-05T22:18:54.726 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-05T22:18:54.726 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-05T22:18:57.056 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-05T22:18:57.057 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-05T22:18:57.099 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-04-05T22:18:57.479 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-04-05T22:18:57.479 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-04-05T22:18:59.757 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-04-05T22:18:59.757 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-04-05T22:19:03.733 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-05T22:19:03.890 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-04-05T22:19:03.936 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-04-05T22:19:03.974 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-04-05T22:19:04.020 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-04-05T22:19:04.105 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-04-05T22:19:04.208 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-04-05T22:19:04.265 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-04-05T22:19:04.300 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-04-05T22:19:04.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-04-05T22:19:04.796 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-04-05T22:19:04.840 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:19:05.003 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:19:05.032 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:19:05.138 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:19:05.172 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 12/180 2024-04-05T22:19:05.239 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/180 2024-04-05T22:19:05.520 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-04-05T22:19:05.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/180 2024-04-05T22:19:05.613 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:19:05.793 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:19:05.830 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 17/180 2024-04-05T22:19:05.921 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 17/180 2024-04-05T22:19:06.001 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-04-05T22:19:06.037 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-04-05T22:19:06.349 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-04-05T22:19:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-04-05T22:19:06.519 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-04-05T22:19:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-04-05T22:19:06.591 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-04-05T22:19:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-04-05T22:19:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-04-05T22:19:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-04-05T22:19:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-04-05T22:19:06.790 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:06.847 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-04-05T22:19:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-04-05T22:19:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-04-05T22:19:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-04-05T22:19:07.350 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:07.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-04-05T22:19:07.444 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-04-05T22:19:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-04-05T22:19:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:19:07.552 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/180 2024-04-05T22:19:07.599 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/180 2024-04-05T22:19:07.632 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/180 2024-04-05T22:19:07.703 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:19:07.731 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:19:07.744 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/180 2024-04-05T22:19:07.821 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/180 2024-04-05T22:19:07.853 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:19:07.859 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/180 2024-04-05T22:19:07.881 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 12/180 2024-04-05T22:19:07.917 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/180 2024-04-05T22:19:07.937 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/180 2024-04-05T22:19:08.024 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/180 2024-04-05T22:19:08.063 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.2.2-1173.g8919605b.el9. 33/180 2024-04-05T22:19:08.113 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.2.2-1173.g8919605b.el 34/180 2024-04-05T22:19:08.218 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_6 35/180 2024-04-05T22:19:08.234 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-04-05T22:19:08.304 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/180 2024-04-05T22:19:08.336 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:19:08.399 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-04-05T22:19:08.473 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-04-05T22:19:08.513 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:19:08.546 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 17/180 2024-04-05T22:19:08.629 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 17/180 2024-04-05T22:19:08.668 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-04-05T22:19:08.720 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-04-05T22:19:09.013 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-04-05T22:19:09.174 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-04-05T22:19:09.186 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-04-05T22:19:09.272 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-04-05T22:19:09.274 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-04-05T22:19:09.371 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-04-05T22:19:09.422 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-04-05T22:19:09.454 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-04-05T22:19:09.486 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:09.503 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-04-05T22:19:09.548 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.2.2-1173.g8919605b.el9.x86_6 43/180 2024-04-05T22:19:09.584 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-04-05T22:19:09.597 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:09.959 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-04-05T22:19:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-04-05T22:19:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-04-05T22:19:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-04-05T22:19:10.156 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-04-05T22:19:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/180 2024-04-05T22:19:10.243 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-04-05T22:19:10.256 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/180 2024-04-05T22:19:10.290 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/180 2024-04-05T22:19:10.340 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-04-05T22:19:10.427 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/180 2024-04-05T22:19:10.500 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/180 2024-04-05T22:19:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/180 2024-04-05T22:19:10.548 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-04-05T22:19:10.592 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/180 2024-04-05T22:19:10.594 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-04-05T22:19:10.700 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/180 2024-04-05T22:19:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:18.2.2-1173.g8919605b.el9. 33/180 2024-04-05T22:19:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-04-05T22:19:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:18.2.2-1173.g8919605b.el 34/180 2024-04-05T22:19:10.789 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-04-05T22:19:10.853 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-04-05T22:19:10.862 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_6 35/180 2024-04-05T22:19:10.942 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-04-05T22:19:10.981 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-04-05T22:19:11.020 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-04-05T22:19:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-04-05T22:19:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-04-05T22:19:11.100 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-04-05T22:19:11.154 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-04-05T22:19:11.214 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-04-05T22:19:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-04-05T22:19:11.454 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-04-05T22:19:11.500 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-04-05T22:19:11.626 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-04-05T22:19:11.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-04-05T22:19:11.705 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-04-05T22:19:11.744 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-04-05T22:19:11.782 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-04-05T22:19:11.784 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-04-05T22:19:11.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-04-05T22:19:11.862 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 69/180 2024-04-05T22:19:11.888 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-04-05T22:19:11.914 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/180 2024-04-05T22:19:11.974 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/180 2024-04-05T22:19:11.996 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-04-05T22:19:12.019 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 72/180 2024-04-05T22:19:12.058 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-04-05T22:19:12.073 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 73/180 2024-04-05T22:19:12.095 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-04-05T22:19:12.123 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/180 2024-04-05T22:19:12.158 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-04-05T22:19:12.198 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/180 2024-04-05T22:19:12.212 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:18.2.2-1173.g8919605b.el9.x86_6 43/180 2024-04-05T22:19:12.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/180 2024-04-05T22:19:12.271 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-04-05T22:19:12.292 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/180 2024-04-05T22:19:12.401 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/180 2024-04-05T22:19:12.566 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 79/180 2024-04-05T22:19:12.637 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/180 2024-04-05T22:19:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-04-05T22:19:12.879 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-04-05T22:19:12.968 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-04-05T22:19:13.025 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-04-05T22:19:13.220 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-04-05T22:19:13.279 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-04-05T22:19:13.433 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-04-05T22:19:13.474 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-04-05T22:19:13.539 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-04-05T22:19:13.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/180 2024-04-05T22:19:13.624 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-04-05T22:19:13.668 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-04-05T22:19:13.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/180 2024-04-05T22:19:13.706 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-04-05T22:19:13.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/180 2024-04-05T22:19:13.766 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-04-05T22:19:13.821 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-04-05T22:19:13.835 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/180 2024-04-05T22:19:13.872 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-04-05T22:19:13.878 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/180 2024-04-05T22:19:14.054 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-04-05T22:19:14.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/180 2024-04-05T22:19:14.108 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-04-05T22:19:14.150 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/180 2024-04-05T22:19:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-04-05T22:19:14.242 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/180 2024-04-05T22:19:14.288 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-04-05T22:19:14.324 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-04-05T22:19:14.368 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-04-05T22:19:14.398 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-04-05T22:19:14.436 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-04-05T22:19:14.470 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-04-05T22:19:14.504 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 69/180 2024-04-05T22:19:14.550 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/180 2024-04-05T22:19:14.616 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/180 2024-04-05T22:19:15.043 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/180 2024-04-05T22:19:15.098 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 72/180 2024-04-05T22:19:15.177 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/180 2024-04-05T22:19:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 73/180 2024-04-05T22:19:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/180 2024-04-05T22:19:15.310 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/180 2024-04-05T22:19:15.356 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/180 2024-04-05T22:19:15.403 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/180 2024-04-05T22:19:15.494 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/180 2024-04-05T22:19:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 79/180 2024-04-05T22:19:15.716 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/180 2024-04-05T22:19:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/180 2024-04-05T22:19:16.683 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/180 2024-04-05T22:19:16.723 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/180 2024-04-05T22:19:16.756 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/180 2024-04-05T22:19:16.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/180 2024-04-05T22:19:16.824 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/180 2024-04-05T22:19:16.878 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/180 2024-04-05T22:19:16.915 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/180 2024-04-05T22:19:17.138 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/180 2024-04-05T22:19:17.165 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 94/180 2024-04-05T22:19:17.179 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/180 2024-04-05T22:19:17.201 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 95/180 2024-04-05T22:19:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/180 2024-04-05T22:19:17.355 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 95/180 2024-04-05T22:19:17.391 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 96/180 2024-04-05T22:19:17.449 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 97/180 2024-04-05T22:19:17.905 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 98/180 2024-04-05T22:19:17.937 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 99/180 2024-04-05T22:19:18.029 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 99/180 2024-04-05T22:19:18.056 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 100/180 2024-04-05T22:19:18.071 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/180 2024-04-05T22:19:18.198 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/180 2024-04-05T22:19:19.726 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/180 2024-04-05T22:19:19.812 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/180 2024-04-05T22:19:19.846 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/180 2024-04-05T22:19:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 94/180 2024-04-05T22:19:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 95/180 2024-04-05T22:19:20.327 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 95/180 2024-04-05T22:19:20.363 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 96/180 2024-04-05T22:19:20.430 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 97/180 2024-04-05T22:19:20.578 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:20.621 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:20.885 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 98/180 2024-04-05T22:19:20.918 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 99/180 2024-04-05T22:19:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 99/180 2024-04-05T22:19:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 100/180 2024-04-05T22:19:21.234 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:21.274 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 102/180 2024-04-05T22:19:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 102/180 2024-04-05T22:19:21.441 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-05T22:19:21.441 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-05T22:19:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 103/180 2024-04-05T22:19:23.435 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:23.490 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 101/180 2024-04-05T22:19:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 102/180 2024-04-05T22:19:24.323 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 102/180 2024-04-05T22:19:24.323 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-05T22:19:24.324 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-05T22:19:24.324 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 103/180 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 103/180 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-05T22:19:33.755 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-05T22:19:33.756 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-05T22:19:33.756 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:33.984 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 104/180 2024-04-05T22:19:34.026 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 104/180 2024-04-05T22:19:34.026 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:34.026 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-05T22:19:34.026 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:35.604 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 105/180 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 105/180 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-05T22:19:35.682 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:35.912 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 106/180 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 106/180 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-05T22:19:35.979 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:36.463 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 107/180 2024-04-05T22:19:36.532 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 107/180 2024-04-05T22:19:36.532 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:36.532 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-05T22:19:36.533 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-05T22:19:36.533 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-05T22:19:36.533 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 103/180 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-04-05T22:19:36.578 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-04-05T22:19:36.579 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-04-05T22:19:36.579 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-04-05T22:19:36.579 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 108/180 2024-04-05T22:19:36.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 109/180 2024-04-05T22:19:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 104/180 2024-04-05T22:19:36.829 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 110/180 2024-04-05T22:19:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 104/180 2024-04-05T22:19:36.835 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:36.835 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-05T22:19:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:37.067 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 111/180 2024-04-05T22:19:37.117 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 112/180 2024-04-05T22:19:37.201 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 113/180 2024-04-05T22:19:37.301 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 114/180 2024-04-05T22:19:37.468 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 115/180 2024-04-05T22:19:37.515 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 116/180 2024-04-05T22:19:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 117/180 2024-04-05T22:19:37.589 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 118/180 2024-04-05T22:19:37.627 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-16.el9.x86_64 119/180 2024-04-05T22:19:37.658 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/180 2024-04-05T22:19:38.296 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.2.2-1173.g8919605b.el9.noarch 121/180 2024-04-05T22:19:38.343 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.2.2-1173.g8919605b.el9.noarch 121/180 2024-04-05T22:19:38.396 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.e 122/180 2024-04-05T22:19:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 105/180 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 105/180 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-05T22:19:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:38.508 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1173.g8919605b. 123/180 2024-04-05T22:19:38.546 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 124/180 2024-04-05T22:19:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 106/180 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 106/180 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-05T22:19:38.806 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:39.287 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 107/180 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 107/180 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-05T22:19:39.380 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:39.512 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 108/180 2024-04-05T22:19:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 109/180 2024-04-05T22:19:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 110/180 2024-04-05T22:19:39.851 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 124/180 2024-04-05T22:19:39.904 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 111/180 2024-04-05T22:19:39.932 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 125/180 2024-04-05T22:19:39.967 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 112/180 2024-04-05T22:19:40.057 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 113/180 2024-04-05T22:19:40.168 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 114/180 2024-04-05T22:19:40.342 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 115/180 2024-04-05T22:19:40.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 116/180 2024-04-05T22:19:40.449 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 117/180 2024-04-05T22:19:40.479 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 118/180 2024-04-05T22:19:40.520 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : jq-1.6-16.el9.x86_64 119/180 2024-04-05T22:19:40.549 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/180 2024-04-05T22:19:41.030 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 125/180 2024-04-05T22:19:41.063 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:18.2.2-1173.g8919605b.el9.noarch 121/180 2024-04-05T22:19:41.064 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 126/180 2024-04-05T22:19:41.105 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:18.2.2-1173.g8919605b.el9.noarch 121/180 2024-04-05T22:19:41.154 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.e 122/180 2024-04-05T22:19:41.206 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 126/180 2024-04-05T22:19:41.266 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1173.g8919605b. 123/180 2024-04-05T22:19:41.304 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 124/180 2024-04-05T22:19:41.380 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el 127/180 2024-04-05T22:19:41.408 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 128/180 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 128/180 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-05T22:19:41.477 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:41.534 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 129/180 2024-04-05T22:19:41.570 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 129/180 2024-04-05T22:19:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.2.2-1173.g8919605b.el9.x86_64 130/180 2024-04-05T22:19:42.596 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 124/180 2024-04-05T22:19:42.631 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 125/180 2024-04-05T22:19:43.711 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 125/180 2024-04-05T22:19:43.748 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 126/180 2024-04-05T22:19:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 126/180 2024-04-05T22:19:44.062 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el 127/180 2024-04-05T22:19:44.117 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 128/180 2024-04-05T22:19:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 128/180 2024-04-05T22:19:44.184 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:44.184 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-05T22:19:44.185 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-05T22:19:44.185 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-05T22:19:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:44.242 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 129/180 2024-04-05T22:19:44.282 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 129/180 2024-04-05T22:19:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:18.2.2-1173.g8919605b.el9.x86_64 130/180 2024-04-05T22:19:45.591 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 131/180 2024-04-05T22:19:45.667 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 132/180 2024-04-05T22:19:46.429 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 133/180 2024-04-05T22:19:46.473 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 134/180 2024-04-05T22:19:46.549 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 134/180 2024-04-05T22:19:46.549 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:46.549 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-05T22:19:46.549 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-05T22:19:46.549 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-05T22:19:46.550 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:46.599 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1173.g89196 135/180 2024-04-05T22:19:46.691 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1173.g89196 135/180 2024-04-05T22:19:46.691 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:46.691 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-05T22:19:46.691 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:46.766 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 136/180 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 136/180 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-05T22:19:46.862 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:47.234 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 137/180 2024-04-05T22:19:47.314 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 137/180 2024-04-05T22:19:47.314 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:47.314 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-05T22:19:47.314 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-05T22:19:47.315 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-05T22:19:47.315 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:47.374 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 138/180 2024-04-05T22:19:47.409 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 139/180 2024-04-05T22:19:47.706 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/180 2024-04-05T22:19:47.778 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 141/180 2024-04-05T22:19:47.839 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-04-05T22:19:47.902 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-04-05T22:19:47.954 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-04-05T22:19:47.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_ 145/180 2024-04-05T22:19:48.043 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-04-05T22:19:48.094 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-04-05T22:19:48.181 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-04-05T22:19:48.314 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-04-05T22:19:48.351 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-04-05T22:19:48.390 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 131/180 2024-04-05T22:19:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 132/180 2024-04-05T22:19:49.226 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:49.234 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-45.el9.x86_64 133/180 2024-04-05T22:19:49.273 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 134/180 2024-04-05T22:19:49.275 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:49.341 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 134/180 2024-04-05T22:19:49.341 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:49.341 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-05T22:19:49.342 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-05T22:19:49.342 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-05T22:19:49.342 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:49.342 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:49.390 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1173.g89196 135/180 2024-04-05T22:19:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1173.g89196 135/180 2024-04-05T22:19:49.477 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:49.477 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-05T22:19:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 136/180 2024-04-05T22:19:49.637 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 136/180 2024-04-05T22:19:49.637 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:49.637 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-05T22:19:49.638 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-05T22:19:49.638 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-05T22:19:49.638 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:50.008 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 137/180 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 137/180 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-05T22:19:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 138/180 2024-04-05T22:19:50.192 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 139/180 2024-04-05T22:19:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/180 2024-04-05T22:19:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 141/180 2024-04-05T22:19:50.591 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-04-05T22:19:50.651 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-04-05T22:19:50.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 152/180 2024-04-05T22:19:50.698 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-04-05T22:19:50.741 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_ 145/180 2024-04-05T22:19:50.776 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-04-05T22:19:50.784 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-04-05T22:19:50.870 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-04-05T22:19:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/180 2024-04-05T22:19:50.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/180 2024-04-05T22:19:50.949 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 156/180 2024-04-05T22:19:50.953 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-04-05T22:19:50.982 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-04-05T22:19:51.017 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-04-05T22:19:51.060 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-04-05T22:19:51.060 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-04-05T22:19:51.071 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-04-05T22:19:51.118 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-04-05T22:19:51.205 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-04-05T22:19:51.246 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-04-05T22:19:51.247 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-04-05T22:19:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-04-05T22:19:51.449 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-04-05T22:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-04-05T22:19:51.532 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-04-05T22:19:51.565 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-04-05T22:19:51.644 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-04-05T22:19:51.721 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-04-05T22:19:51.779 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-04-05T22:19:51.811 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-04-05T22:19:51.845 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-04-05T22:19:51.879 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-04-05T22:19:51.912 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-04-05T22:19:51.949 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:51.950 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:52.027 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/180 2024-04-05T22:19:52.279 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:52.279 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-04-05T22:19:52.426 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-04-05T22:19:52.457 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-04-05T22:19:52.490 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-04-05T22:19:52.560 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-04-05T22:19:52.600 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-04-05T22:19:52.720 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-04-05T22:19:52.975 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-04-05T22:19:52.976 INFO:teuthology.orchestra.run.smithi157.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-05T22:19:52.976 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:19:53.450 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 152/180 2024-04-05T22:19:53.576 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-04-05T22:19:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/180 2024-04-05T22:19:53.739 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/180 2024-04-05T22:19:53.765 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 156/180 2024-04-05T22:19:53.799 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-04-05T22:19:53.842 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-04-05T22:19:53.893 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-04-05T22:19:53.894 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-04-05T22:19:54.044 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-04-05T22:19:54.095 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-04-05T22:19:54.095 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-04-05T22:19:54.225 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-04-05T22:19:54.275 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-04-05T22:19:54.331 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-04-05T22:19:54.374 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-04-05T22:19:54.406 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-04-05T22:19:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-04-05T22:19:54.554 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-04-05T22:19:54.603 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-04-05T22:19:54.644 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-04-05T22:19:54.702 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-04-05T22:19:54.736 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-04-05T22:19:54.769 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-04-05T22:19:54.806 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:54.807 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:55.127 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-04-05T22:19:55.128 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-04-05T22:19:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-04-05T22:19:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-04-05T22:19:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-04-05T22:19:55.359 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-04-05T22:19:55.407 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-04-05T22:19:55.527 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-04-05T22:19:55.799 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-04-05T22:19:55.799 INFO:teuthology.orchestra.run.smithi073.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-05T22:19:55.800 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.2.2-1173.g8919605b.el9.x86_64 1/180 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 2/180 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 3/180 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 4/180 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1173.g89196 5/180 2024-04-05T22:19:58.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 6/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 7/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 8/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 9/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 12/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 13/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_ 14/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 15/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.2.2-1173.g8919605b.el9.x86_6 17/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 18/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 19/180 2024-04-05T22:19:58.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1173.g8919605b.el 20/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.2.2-1173.g8919605b.el9. 21/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_6 22/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 23/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 24/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 25/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 26/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 27/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 28/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1173.g8919605b. 29/180 2024-04-05T22:19:58.280 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 30/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 31/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 32/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el 33/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 34/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.e 35/180 2024-04-05T22:19:58.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 36/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.2.2-1173.g8919605b.el9.noarch 37/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 38/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 39/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-16.el9.x86_64 41/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 42/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 43/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 44/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 45/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 46/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 47/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 48/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 49/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 50/180 2024-04-05T22:19:58.295 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 51/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 52/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 53/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 54/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 55/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 56/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 57/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 58/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 62/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 63/180 2024-04-05T22:19:58.296 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 73/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 74/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 75/180 2024-04-05T22:19:58.297 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 76/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 77/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 78/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 79/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 80/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 81/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 82/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 83/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 84/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 85/180 2024-04-05T22:19:58.298 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 86/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 87/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 89/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 90/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 91/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 92/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 93/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 94/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 95/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-04-05T22:19:58.299 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-04-05T22:19:58.300 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-04-05T22:19:58.301 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-04-05T22:19:58.302 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.2.2-1173.g8919605b.el9.x86_64 133/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 135/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 137/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 138/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 139/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 140/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 141/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 142/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 143/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-4.el9.x86_64 145/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-3.el9.x86_64 146/180 2024-04-05T22:19:58.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 147/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 148/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 149/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 150/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 151/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 152/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 153/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 154/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 155/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 156/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 157/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 158/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 159/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 160/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 161/180 2024-04-05T22:19:58.304 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 162/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 163/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 164/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 165/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 166/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 167/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 168/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 169/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 170/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 171/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 172/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 173/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 174/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 175/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 176/180 2024-04-05T22:19:58.305 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-04-05T22:19:58.306 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-04-05T22:19:58.306 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: acl-2.3.1-4.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-05T22:20:00.657 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.658 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.659 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.660 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-16.el9.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-05T22:20:00.661 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-05T22:20:00.662 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-05T22:20:00.663 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-05T22:20:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-05T22:20:00.665 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-05T22:20:00.666 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-05T22:20:00.667 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-05T22:20:00.668 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-05T22:20:00.669 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-05T22:20:00.669 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:00.669 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-05T22:20:01.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:18.2.2-1173.g8919605b.el9.x86_64 1/180 2024-04-05T22:20:01.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 2/180 2024-04-05T22:20:01.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 3/180 2024-04-05T22:20:01.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 4/180 2024-04-05T22:20:01.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1173.g89196 5/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 6/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 7/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 8/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 9/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 10/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 11/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 12/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 13/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_ 14/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 15/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 16/180 2024-04-05T22:20:01.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:18.2.2-1173.g8919605b.el9.x86_6 17/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86 18/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 19/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1173.g8919605b.el 20/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:18.2.2-1173.g8919605b.el9. 21/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_6 22/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 23/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 24/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 25/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 26/180 2024-04-05T22:20:01.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 27/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 28/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1173.g8919605b. 29/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noa 30/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.n 31/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1173.g891 32/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el 33/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 34/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.e 35/180 2024-04-05T22:20:01.079 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 36/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:18.2.2-1173.g8919605b.el9.noarch 37/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 38/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 39/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : jq-1.6-16.el9.x86_64 41/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 42/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 43/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 44/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 45/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 46/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 47/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 48/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 49/180 2024-04-05T22:20:01.080 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 50/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 51/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 52/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 53/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 54/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 55/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 56/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 57/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 58/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 62/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 63/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/180 2024-04-05T22:20:01.081 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 73/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 74/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 75/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 76/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 77/180 2024-04-05T22:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 78/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 79/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 80/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 81/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 82/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 83/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 84/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 85/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 86/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 87/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 89/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 90/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 91/180 2024-04-05T22:20:01.083 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 92/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 93/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 94/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 95/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-04-05T22:20:01.084 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-04-05T22:20:01.085 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-04-05T22:20:01.086 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:18.2.2-1173.g8919605b.el9.x86_64 133/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 135/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 137/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 138/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 139/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 140/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 141/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 142/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 143/180 2024-04-05T22:20:01.087 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : acl-2.3.1-4.el9.x86_64 145/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : acl-2.3.1-3.el9.x86_64 146/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 147/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 148/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 149/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 150/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 151/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 152/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 153/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 154/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 155/180 2024-04-05T22:20:01.088 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 156/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 157/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 158/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 159/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 160/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 161/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 162/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 163/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 164/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 165/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 166/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 167/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 168/180 2024-04-05T22:20:01.089 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 169/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 170/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 171/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 172/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 173/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 174/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 175/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 176/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-04-05T22:20:01.090 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-04-05T22:20:01.355 DEBUG:teuthology.parallel:result is None 2024-04-05T22:20:03.513 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-04-05T22:20:03.513 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:03.513 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-04-05T22:20:03.513 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:03.513 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: acl-2.3.1-4.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.514 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-05T22:20:03.515 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.516 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:18.2.2-1173.g8919605b.el9.noarch 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-05T22:20:03.517 INFO:teuthology.orchestra.run.smithi073.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: jq-1.6-16.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.518 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-05T22:20:03.519 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.14-1.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-05T22:20:03.520 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-05T22:20:03.521 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-05T22:20:03.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-05T22:20:03.523 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:18.2.2-1173.g8919605b.el9.x86_64 2024-04-05T22:20:03.524 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-3.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:03.525 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-04-05T22:20:04.363 DEBUG:teuthology.parallel:result is None 2024-04-05T22:20:04.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8919605b0a5dc1f1650c8aec3051c77c0e3e240e 2024-04-05T22:20:04.509 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-05T22:20:04.568 INFO:teuthology.orchestra.run.smithi073.stdout:18.2.2-1173.g8919605b.el9 2024-04-05T22:20:04.569 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1173.g8919605b.el9 2024-04-05T22:20:04.569 INFO:teuthology.task.install:The correct ceph version 18.2.2-1173.g8919605b is installed. 2024-04-05T22:20:04.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8919605b0a5dc1f1650c8aec3051c77c0e3e240e 2024-04-05T22:20:04.709 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-05T22:20:04.764 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.2-1173.g8919605b.el9 2024-04-05T22:20:04.764 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1173.g8919605b.el9 2024-04-05T22:20:04.764 INFO:teuthology.task.install:The correct ceph version 18.2.2-1173.g8919605b is installed. 2024-04-05T22:20:04.766 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-05T22:20:04.766 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:04.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-05T22:20:04.847 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:04.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-05T22:20:04.927 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-05T22:20:04.928 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:04.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-04-05T22:20:05.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-05T22:20:05.113 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:05.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-05T22:20:05.189 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-05T22:20:05.303 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-05T22:20:05.303 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:05.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-05T22:20:05.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-05T22:20:05.499 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:05.499 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-05T22:20:05.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-05T22:20:05.693 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-05T22:20:05.694 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:05.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-04-05T22:20:05.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-05T22:20:05.877 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:05.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-05T22:20:05.949 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-05T22:20:06.065 INFO:teuthology.run_tasks:Running task ceph... 2024-04-05T22:20:06.148 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-05T22:20:06.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-04-05T22:20:06.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-04-05T22:20:06.244 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-05T22:20:06.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-05T22:20:06.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-05T22:20:06.338 INFO:tasks.ceph:Creating extra log directories... 2024-04-05T22:20:06.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-05T22:20:06.342 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-05T22:20:06.442 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-05T22:20:06.442 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-05T22:20:06.442 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-05T22:20:06.442 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:06.442 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-05T22:20:06.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:06.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:06.758 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:06.758 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-05T22:20:06.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:07.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:07.163 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-05T22:20:07.163 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-05T22:20:07.164 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672', 'branch': 'wip-yuri4-testing-2024-04-04-1128-reef', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7641672', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2024-04-04-1128-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2024-04-04-1128-reef', 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5779, 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2024-04-04-1128-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8919605b0a5dc1f1650c8aec3051c77c0e3e240e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbjYUP5vK2Qbb96znHaYroZFrZBg/XI0PMZan49bBI9xSKPeaNJDnXaZtsKsnRWmuhzAoICbPCRqEGiWnhE+VM=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCnXSpAkWuSJg4Pt6OmsfWAKlOC9R3c3W4g9ge5OjmfvIsslsovZ1PLYWw8SmZK5Y0xBJs7GyUrzdfPi/UgmuQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8919605b0a5dc1f1650c8aec3051c77c0e3e240e', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-05_18:13:21', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-05T22:20:07.164 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-05T22:20:07.169 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-05T22:20:07.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-04-05T22:20:07.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-04-05T22:20:07.343 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:07.352 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-04-05T22:20:07.391 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-05T22:20:07.391 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-04-05T22:20:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-05T22:20:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:07.472 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-04-05T22:20:07.472 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:07.473 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:07.473 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-05 22:19:56.217328129 +0000 2024-04-05T22:20:07.473 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-05 22:17:14.921300759 +0000 2024-04-05T22:20:07.473 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-05 22:17:14.922300734 +0000 2024-04-05T22:20:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-05 22:17:14.921300759 +0000 2024-04-05T22:20:07.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-05T22:20:07.580 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:20:07.581 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:20:07.581 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000370634 s, 1.4 MB/s 2024-04-05T22:20:07.583 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-05T22:20:07.625 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:07.719 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-05 22:19:56.218328104 +0000 2024-04-05T22:20:07.720 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-05 22:17:15.484286954 +0000 2024-04-05T22:20:07.720 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-05 22:17:15.484286954 +0000 2024-04-05T22:20:07.720 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-05 22:17:15.484286954 +0000 2024-04-05T22:20:07.720 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-05T22:20:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:20:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:20:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000337018 s, 1.5 MB/s 2024-04-05T22:20:07.811 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-05T22:20:07.864 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-04-05T22:20:07.910 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-05T22:20:07.910 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:07.910 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-04-05T22:20:07.910 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:07.911 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:07.911 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-05 22:19:56.219328078 +0000 2024-04-05T22:20:07.911 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-05 22:17:16.069272610 +0000 2024-04-05T22:20:07.911 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-05 22:17:16.069272610 +0000 2024-04-05T22:20:07.911 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-05 22:17:16.069272610 +0000 2024-04-05T22:20:07.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-05T22:20:07.995 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:20:07.996 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:20:07.996 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000324007 s, 1.6 MB/s 2024-04-05T22:20:08.000 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-05T22:20:08.051 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-04-05T22:20:08.096 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-05 22:19:56.219328078 +0000 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-05 22:17:16.644258511 +0000 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-05 22:17:16.644258511 +0000 2024-04-05T22:20:08.097 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-05 22:17:16.644258511 +0000 2024-04-05T22:20:08.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-05T22:20:08.226 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:20:08.226 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:20:08.226 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000391189 s, 1.3 MB/s 2024-04-05T22:20:08.231 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-05T22:20:08.284 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:08.285 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:08.285 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-05T22:20:08.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-05T22:20:08.343 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-05 22:19:53.433991954 +0000 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-05 22:17:05.168638433 +0000 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-05 22:17:05.168638433 +0000 2024-04-05T22:20:08.395 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-05 22:17:05.168638433 +0000 2024-04-05T22:20:08.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-05T22:20:08.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-05T22:20:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-05T22:20:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00022611 s, 2.3 MB/s 2024-04-05T22:20:08.489 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-05T22:20:08.544 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-05 22:19:53.434991936 +0000 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-05 22:17:05.780627311 +0000 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-05 22:17:05.780627311 +0000 2024-04-05T22:20:08.590 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-05 22:17:05.780627311 +0000 2024-04-05T22:20:08.591 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-05T22:20:08.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-05T22:20:08.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-05T22:20:08.724 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000225098 s, 2.3 MB/s 2024-04-05T22:20:08.728 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-05T22:20:08.785 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-05 22:19:53.435991918 +0000 2024-04-05T22:20:08.838 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-05 22:17:06.348616987 +0000 2024-04-05T22:20:08.838 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-05 22:17:06.348616987 +0000 2024-04-05T22:20:08.838 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-05 22:17:06.348616987 +0000 2024-04-05T22:20:08.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-05T22:20:08.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-05T22:20:08.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-05T22:20:08.928 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00024979 s, 2.0 MB/s 2024-04-05T22:20:08.932 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-05T22:20:08.987 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-05 22:19:53.437991882 +0000 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-05 22:17:06.904606883 +0000 2024-04-05T22:20:09.035 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-05 22:17:06.904606883 +0000 2024-04-05T22:20:09.036 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-05 22:17:06.904606883 +0000 2024-04-05T22:20:09.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-05T22:20:09.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-05T22:20:09.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-05T22:20:09.110 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000246871 s, 2.1 MB/s 2024-04-05T22:20:09.113 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-05T22:20:09.168 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:09.168 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-05T22:20:09.168 INFO:tasks.ceph:Generating config... 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] debug client = 20 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] debug ms = 1 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] pid file = 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-05T22:20:09.169 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-05T22:20:09.169 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-05T22:20:09.169 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-05T22:20:09.169 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-05T22:20:09.169 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-05T22:20:09.169 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-05T22:20:09.169 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-05T22:20:09.170 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-05T22:20:09.170 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-05T22:20:09.170 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-05T22:20:09.171 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-05T22:20:09.171 INFO:tasks.ceph:Setting up mon.a... 2024-04-05T22:20:09.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-05T22:20:09.307 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-04-05T22:20:09.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-05T22:20:09.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-05T22:20:09.541 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-04-05T22:20:09.541 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-05T22:20:09.542 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:09.542 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-05T22:20:09.592 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid 38861c87-67cf-4180-a71f-97401becb77d 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = pacific 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:fsid 38861c87-67cf-4180-a71f-97401becb77d 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-04-05T22:20:09.743014+0000 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-04-05T22:20:09.743014+0000 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 16 (pacific) 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 1 2024-04-05T22:20:09.742 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-04-05T22:20:09.743 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-04-05T22:20:09.743 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-04-05T22:20:09.743 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-05T22:20:09.745 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-05T22:20:09.794 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 38861c87-67cf-4180-a71f-97401becb77d... 2024-04-05T22:20:09.795 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-05T22:20:09.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-05T22:20:09.965 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-04-05T22:20:09.965 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-04-05T22:20:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-04-05T22:20:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-04-05T22:20:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on skipped message = False 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = 38861c87-67cf-4180-a71f-97401becb77d 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-04-05T22:20:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-04-05T22:20:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-05T22:20:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-05T22:20:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug finisher = 20 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-05T22:20:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-04-05T22:20:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-04-05T22:20:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-04-05T22:20:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-04-05T22:20:09.977 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.978 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-05T22:20:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.73,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 38861c87-67cf-4180-a71f-97401becb77d 2024-04-05T22:20:09.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-04-05T22:20:09.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.982 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-04-05T22:20:09.983 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: debug finisher = 20 2024-04-05T22:20:09.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-05T22:20:09.985 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-05T22:20:09.986 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-04-05T22:20:09.987 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-04-05T22:20:09.988 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-05T22:20:09.989 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-05T22:20:09.989 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-05T22:20:10.015 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-05T22:20:10.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-05T22:20:10.159 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-05T22:20:10.159 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:10.159 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-05T22:20:10.208 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:10.208 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-05T22:20:10.256 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:20:10.256 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:10.256 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-05T22:20:10.256 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-05T22:20:10.452 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:10.452 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:10.548 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:20:10.548 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:10.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-05T22:20:10.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-05T22:20:10.700 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:10.700 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:10.796 INFO:tasks.ceph:Setting up mon nodes... 2024-04-05T22:20:10.796 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-05T22:20:10.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-05T22:20:10.982 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-05T22:20:10.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-05T22:20:11.180 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-05T22:20:11.192 INFO:tasks.ceph:Setting up mds nodes... 2024-04-05T22:20:11.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-05T22:20:11.375 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-05T22:20:11.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-05T22:20:11.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-05T22:20:11.639 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-05T22:20:11.649 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-05T22:20:11.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-05T22:20:11.926 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-05T22:20:11.936 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-05T22:20:12.031 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-05T22:20:12.206 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-05T22:20:12.215 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-05T22:20:12.304 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-05T22:20:12.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-05T22:20:12.435 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-05T22:20:12.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-05T22:20:12.608 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-05T22:20:12.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-05T22:20:12.801 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-05T22:20:12.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-05T22:20:12.994 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-05T22:20:13.043 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-05T22:20:13.043 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-05T22:20:13.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-05T22:20:13.141 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:13.141 INFO:tasks.ceph:role: osd.0 2024-04-05T22:20:13.141 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:20:13.141 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:13.238 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:13.239 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:13.760 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-05T22:20:13.767 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-05T22:20:13.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-05T22:20:13.952 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-05T22:20:14.057 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-05T22:20:14.135 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:14.135 INFO:tasks.ceph:role: osd.1 2024-04-05T22:20:14.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:20:14.135 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:14.219 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:14.220 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:14.671 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-05T22:20:14.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-05T22:20:14.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-05T22:20:14.795 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-05T22:20:14.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-05T22:20:14.962 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:14.962 INFO:tasks.ceph:role: osd.2 2024-04-05T22:20:14.962 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:20:14.962 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:15.052 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:15.579 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-05T22:20:15.586 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-05T22:20:15.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-05T22:20:15.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-05T22:20:15.861 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-05T22:20:15.944 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:15.944 INFO:tasks.ceph:role: osd.3 2024-04-05T22:20:15.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:20:15.944 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-05T22:20:16.037 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:16.038 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:16.502 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-05T22:20:16.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-05T22:20:16.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-05T22:20:16.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-05T22:20:16.704 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:16.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.848+0000 7fd97aa1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-05T22:20:16.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.849+0000 7fd97aa1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-05T22:20:16.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.849+0000 7fd97aa1d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-05T22:20:16.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.849+0000 7fd97aa1d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-05T22:20:16.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.849+0000 7fd97aa1d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-05T22:20:16.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:16.852+0000 7fd97aa1d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-05T22:20:20.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-05T22:20:20.132 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:20.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.271+0000 7fb38ad06640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-05T22:20:20.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.272+0000 7fb38ad06640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-05T22:20:20.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.272+0000 7fb38ad06640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-05T22:20:20.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.272+0000 7fb38ad06640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-05T22:20:20.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.272+0000 7fb38ad06640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-05T22:20:20.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:20.275+0000 7fb38ad06640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-05T22:20:23.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-05T22:20:23.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:23.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.663+0000 7f210c4a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-05T22:20:23.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.664+0000 7f210c4a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-05T22:20:23.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.664+0000 7f210c4a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-05T22:20:23.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.664+0000 7f210c4a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-05T22:20:23.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.664+0000 7f210c4a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-05T22:20:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:23.666+0000 7f210c4a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-05T22:20:26.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-05T22:20:26.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:27.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.068+0000 7f1327c37640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-05T22:20:27.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.068+0000 7f1327c37640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-05T22:20:27.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.068+0000 7f1327c37640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-05T22:20:27.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.068+0000 7f1327c37640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-05T22:20:27.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.068+0000 7f1327c37640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-05T22:20:27.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:27.071+0000 7f1327c37640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-05T22:20:29.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-05T22:20:30.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-05T22:20:30.191 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:30.191 INFO:tasks.ceph:role: osd.4 2024-04-05T22:20:30.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:20:30.191 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-05T22:20:30.283 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:30.284 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:30.284 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:30.284 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:30.284 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:30.284 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:30.285 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:30.285 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:30.285 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:30.285 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:30.317 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-05T22:20:30.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-05T22:20:30.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-05T22:20:30.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-05T22:20:30.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-05T22:20:30.588 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:30.589 INFO:tasks.ceph:role: osd.5 2024-04-05T22:20:30.589 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:20:30.589 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-05T22:20:30.667 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:30.667 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:30.667 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:30.667 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:30.668 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-05T22:20:30.705 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-05T22:20:30.706 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-05T22:20:30.803 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-05T22:20:30.875 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-05T22:20:30.943 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:30.944 INFO:tasks.ceph:role: osd.6 2024-04-05T22:20:30.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:20:30.944 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:31.018 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:31.019 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:31.052 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-05T22:20:31.058 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-05T22:20:31.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-05T22:20:31.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-05T22:20:31.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-05T22:20:31.298 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-05T22:20:31.298 INFO:tasks.ceph:role: osd.7 2024-04-05T22:20:31.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:20:31.298 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-05T22:20:31.374 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-05T22:20:31.375 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-05T22:20:31.407 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-05T22:20:31.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-05T22:20:31.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-05T22:20:31.507 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-05T22:20:31.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:31.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.722+0000 7f27c3213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-05T22:20:31.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.722+0000 7f27c3213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-05T22:20:31.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.722+0000 7f27c3213640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-05T22:20:31.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.722+0000 7f27c3213640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-05T22:20:31.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.722+0000 7f27c3213640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-05T22:20:31.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:31.725+0000 7f27c3213640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-05T22:20:34.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-05T22:20:34.752 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.890+0000 7f1ed8cb3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-05T22:20:34.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.891+0000 7f1ed8cb3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-05T22:20:34.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.891+0000 7f1ed8cb3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-05T22:20:34.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.891+0000 7f1ed8cb3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-05T22:20:34.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.891+0000 7f1ed8cb3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-05T22:20:34.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:34.893+0000 7f1ed8cb3640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-05T22:20:37.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:37.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:20:38.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-05T22:20:38.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.313+0000 7fb00a211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-05T22:20:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.313+0000 7fb00a211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-05T22:20:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.314+0000 7fb00a211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-05T22:20:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.314+0000 7fb00a211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-05T22:20:38.313 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.314+0000 7fb00a211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-05T22:20:38.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:38.316+0000 7fb00a211640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-05T22:20:41.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-05T22:20:41.590 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:41.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.721+0000 7f5690173640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-05T22:20:41.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.722+0000 7f5690173640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-05T22:20:41.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.722+0000 7f5690173640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-05T22:20:41.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.722+0000 7f5690173640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-05T22:20:41.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.722+0000 7f5690173640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-05T22:20:41.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:41.724+0000 7f5690173640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-05T22:20:44.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-05T22:20:44.748 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-05T22:20:44.748 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:44.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-05T22:20:44.844 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:44.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-05T22:20:44.937 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:44.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-05T22:20:45.029 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:45.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-05T22:20:45.119 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:45.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-05T22:20:45.207 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:45.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-05T22:20:45.345 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:45.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-05T22:20:45.439 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-05T22:20:45.534 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.535 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-05T22:20:45.625 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-05T22:20:45.717 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-05T22:20:45.805 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.805 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-05T22:20:45.896 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-05T22:20:45.991 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:45.991 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-05T22:20:46.079 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:46.079 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-05T22:20:46.134 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:46.134 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-05T22:20:46.184 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:46.184 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-05T22:20:46.241 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:46.241 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-05T22:20:46.291 INFO:tasks.ceph:Adding keys to all mons... 2024-04-05T22:20:46.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-05T22:20:46.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g== 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw== 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA== 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-04-05T22:20:46.381 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg== 2024-04-05T22:20:46.382 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw== 2024-04-05T22:20:46.383 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-04-05T22:20:46.384 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA== 2024-04-05T22:20:46.384 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-04-05T22:20:46.384 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g== 2024-04-05T22:20:46.384 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-04-05T22:20:46.384 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA== 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-04-05T22:20:46.385 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[client.2] 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g== 2024-04-05T22:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:[client.1] 2024-04-05T22:20:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw== 2024-04-05T22:20:46.387 INFO:teuthology.orchestra.run.smithi073.stdout:[client.3] 2024-04-05T22:20:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA== 2024-04-05T22:20:46.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-05T22:20:46.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-05T22:20:46.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:46.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:46.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:46.729 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:46.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:46.879 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-05T22:20:47.456 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-05T22:20:47.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:47.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:47.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:47.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-05T22:20:47.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:47.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.030 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-05T22:20:48.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.757 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:48.917 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-05T22:20:49.057 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-05T22:20:49.057 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-05T22:20:49.152 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-05T22:20:49.296 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-05T22:20:49.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-05T22:20:49.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-05T22:20:49.639 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-05T22:20:49.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-05T22:20:49.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-05T22:20:49.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-05T22:20:50.050 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:50.058 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-05T22:20:50.107 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-05T22:20:50.107 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-05T22:20:50.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-05T22:20:50.114 INFO:tasks.ceph.mon.a:Started 2024-04-05T22:20:50.114 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-05T22:20:50.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-05T22:20:50.151 INFO:tasks.ceph.mon.b:Started 2024-04-05T22:20:50.152 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-05T22:20:50.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-05T22:20:50.159 INFO:tasks.ceph.mon.c:Started 2024-04-05T22:20:50.159 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-05T22:20:50.159 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-05T22:20:50.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-05T22:20:50.164 INFO:tasks.ceph.mgr.y:Started 2024-04-05T22:20:50.164 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-05T22:20:50.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-05T22:20:50.169 INFO:tasks.ceph.mgr.x:Started 2024-04-05T22:20:50.169 INFO:tasks.ceph:Setting crush tunables to default 2024-04-05T22:20:50.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-04-05T22:20:50.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.345+0000 7f7916800640 1 Processor -- start 2024-04-05T22:20:50.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.345+0000 7f7916800640 1 -- start start 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f7916800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f7916800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f7916800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7910070060 con 0x7f791015ab60 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f7916800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79100adb80 con 0x7f79100ccbc0 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f7916800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7910071220 con 0x7f79100ad720 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33792/0 (socket says 172.21.15.73:33792) 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 learned_addr learned my addr 172.21.15.73:0/2561624295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.346+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.347+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-05T22:20:50.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.548+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.548+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.548+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-05T22:20:50.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.548+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.548+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.549+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-05T22:20:50.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.549+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-05T22:20:50.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.549+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.549+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.549+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.950+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.950+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.950+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.950+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.950+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.951+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.951+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.951+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.951+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:50.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:50.951+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-05T22:20:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.752+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.752+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-05T22:20:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-05T22:20:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-05T22:20:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:51.753+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 msgr2=0x7f791006bb60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.157:3300/0 conn(0x7f79100ad720 0x7f791006bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 -- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 msgr2=0x7f79100ccff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 --2- 172.21.15.73:0/2561624295 >> v2:172.21.15.73:3300/0 conn(0x7f79100ccbc0 0x7f79100ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 -- 172.21.15.73:0/2561624295 >> v1:172.21.15.73:6789/0 conn(0x7f791015ab60 legacy=0x7f791006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:53.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:53.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.346+0000 7f790cc00640 1 -- 172.21.15.73:0/2561624295 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7910071500 con 0x7f78f0003690 2024-04-05T22:20:53.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.347+0000 7f790cc00640 1 -- 172.21.15.73:0/2561624295 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7910071220 con 0x7f78f00072c0 2024-04-05T22:20:53.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.347+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:53.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.348+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-05T22:20:53.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.348+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:53.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.348+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:53.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.348+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-05T22:20:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.550+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.550+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-05T22:20:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.550+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.550+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.550+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-05T22:20:53.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.951+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:53.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.952+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-05T22:20:53.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.952+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:53.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.952+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:53.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:53.952+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-05T22:20:54.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:54.754+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:54.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:54.754+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-05T22:20:54.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:54.754+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-05T22:20:54.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:54.754+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-05T22:20:54.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:54.754+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-05T22:20:55.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.579+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355379297 0 0) 0x7f7910071500 con 0x7f78f0003690 2024-04-05T22:20:55.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.579+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f4003520 con 0x7f78f0003690 2024-04-05T22:20:55.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160061492 0 0) 0x7f78f4003520 con 0x7f78f0003690 2024-04-05T22:20:55.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f0009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-05T22:20:55.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f0009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f000afa0 con 0x7f78f0003690 2024-04-05T22:20:55.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f79000016f0 con 0x7f78f0003690 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.580+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 >> v1:172.21.15.157:6789/0 conn(0x7f78f0003690 legacy=0x7f78f0005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790fe00640 1 -- 172.21.15.73:0/2561624295 reap_dead start 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 0x7f78f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4004d00 0x7f78f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7910071220 con 0x7f78f4003f80 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f400bc60 con 0x7f78f4004d00 2024-04-05T22:20:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f400be20 con 0x7f78f00072c0 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.581+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 0x7f78f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.582+0000 7f790f400640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.582+0000 7f790fe00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4004d00 0x7f78f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.583+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 msgr2=0x7f78f4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.583+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.583+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4004d00 msgr2=0x7f78f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.583+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4004d00 0x7f78f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.583+0000 7f790ea00640 1 -- 172.21.15.73:0/2561624295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4003520 con 0x7f78f4003f80 2024-04-05T22:20:55.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.585+0000 7f790ea00640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 0x7f78f40047c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78fc019d00 tx=0x7f78f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fbb9236c81144c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:55.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.586+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004030 con 0x7f78f4003f80 2024-04-05T22:20:55.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.586+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc0041b0 con 0x7f78f4003f80 2024-04-05T22:20:55.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.586+0000 7f790e000640 1 -- 172.21.15.73:0/2561624295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc003610 con 0x7f78f4003f80 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.587+0000 7f7916800640 1 -- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 msgr2=0x7f78f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.587+0000 7f7916800640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 0x7f78f40047c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78fc019d00 tx=0x7f78f40057f0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 -- 172.21.15.73:0/2561624295 shutdown_connections 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4004d00 0x7f78f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4003f80 0x7f78f40047c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 --2- 172.21.15.73:0/2561624295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f00072c0 0x7f78f4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 -- 172.21.15.73:0/2561624295 >> 172.21.15.73:0/2561624295 conn(0x7f7910069e90 msgr2=0x7f791006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:55.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 -- 172.21.15.73:0/2561624295 shutdown_connections 2024-04-05T22:20:55.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.588+0000 7f7916800640 1 -- 172.21.15.73:0/2561624295 wait complete. 2024-04-05T22:20:55.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.592+0000 7f7916800640 1 Processor -- start 2024-04-05T22:20:55.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.592+0000 7f7916800640 1 -- start start 2024-04-05T22:20:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 0x7f79100ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f4004d00 0x7f79100aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79100b0bd0 con 0x7f78f00072c0 2024-04-05T22:20:55.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7910153210 con 0x7f78f4003f80 2024-04-05T22:20:55.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f7916800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79101534f0 con 0x7f78f4004d00 2024-04-05T22:20:55.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f790f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f790f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33904/0 (socket says 172.21.15.73:33904) 2024-04-05T22:20:55.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.594+0000 7f790f400640 1 -- 172.21.15.73:0/1452589492 learned_addr learned my addr 172.21.15.73:0/1452589492 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790fe00640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f4004d00 0x7f79100aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 -- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f4004d00 msgr2=0x7f79100aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f4004d00 0x7f79100aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 -- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 msgr2=0x7f79100ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790ea00640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 0x7f79100ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:55.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 0x7f79100ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 -- 172.21.15.73:0/1452589492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79101537d0 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790ea00640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 0x7f79100ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.595+0000 7f790f400640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f790000f770 tx=0x7f790000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000fe80 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7910153ab0 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79000052e0 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000d970 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79100b0f60 con 0x7f78f00072c0 2024-04-05T22:20:55.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.596+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f79000035f0 con 0x7f78f00072c0 2024-04-05T22:20:55.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.598+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79100b10f0 con 0x7f78f00072c0 2024-04-05T22:20:55.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.599+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f79000038a0 con 0x7f78f00072c0 2024-04-05T22:20:55.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.602+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f7900003b50 con 0x7f78f00072c0 2024-04-05T22:20:55.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.677+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7910067690 con 0x7f78f00072c0 2024-04-05T22:20:55.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.685+0000 7f7907400640 1 -- 172.21.15.73:0/1452589492 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7900003cf0 con 0x7f78f00072c0 2024-04-05T22:20:55.686 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-04-05T22:20:55.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 msgr2=0x7f7910148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:55.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f790000f770 tx=0x7f790000fc30 comp rx=0 tx=0).stop 2024-04-05T22:20:55.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 shutdown_connections 2024-04-05T22:20:55.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78f4004d00 0x7f79100aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78f4003f80 0x7f79100ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 --2- 172.21.15.73:0/1452589492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f00072c0 0x7f7910148710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 >> 172.21.15.73:0/1452589492 conn(0x7f7910069e90 msgr2=0x7f791014d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 shutdown_connections 2024-04-05T22:20:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:55.687+0000 7f7916800640 1 -- 172.21.15.73:0/1452589492 wait complete. 2024-04-05T22:20:55.705 INFO:tasks.ceph:check_enable_crimson: False 2024-04-05T22:20:55.705 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-05T22:20:55.706 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:55.706 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-05T22:20:55.782 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:55.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-05T22:20:55.906 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:55.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-05T22:20:56.032 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:20:56.032 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-05T22:20:56.160 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:56.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-05T22:20:56.219 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:56.218+0000 7f1e0fa32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-05T22:20:56.220 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:56.220+0000 7f5588e37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-05T22:20:56.243 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:56.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-05T22:20:56.283 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:56.283+0000 7f5588e37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-05T22:20:56.291 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:56.288+0000 7f1e0fa32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-05T22:20:56.371 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:56.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-05T22:20:56.497 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:20:56.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-05T22:20:56.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 0c57082f-7b11-49ad-9aa2-3d7c81942802 0 2024-04-05T22:20:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.849+0000 7fe8b6600640 1 Processor -- start 2024-04-05T22:20:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.849+0000 7fe8b6600640 1 -- start start 2024-04-05T22:20:56.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8b6600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8b015ad30 0x7fe8b00b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:56.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:56.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8b6600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00cf170 con 0x7fe8b00b4440 2024-04-05T22:20:56.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8b6600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00ce290 con 0x7fe8b015ad30 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8b6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00ce560 con 0x7fe8b00cde30 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49276/0 (socket says 172.21.15.157:49276) 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8a7e00640 1 -- 172.21.15.157:0/555454062 learned_addr learned my addr 172.21.15.157:0/555454062 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.851+0000 7fe8af400640 1 --2- 172.21.15.157:0/555454062 >> v2:172.21.15.157:3300/0 conn(0x7fe8b015ad30 0x7fe8b00b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.852+0000 7fe8a7e00640 1 -- 172.21.15.157:0/555454062 >> v2:172.21.15.157:3300/0 conn(0x7fe8b015ad30 msgr2=0x7fe8b00b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.852+0000 7fe8a7e00640 1 --2- 172.21.15.157:0/555454062 >> v2:172.21.15.157:3300/0 conn(0x7fe8b015ad30 0x7fe8b00b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.852+0000 7fe8a7e00640 1 -- 172.21.15.157:0/555454062 >> v1:172.21.15.73:6789/0 conn(0x7fe8b00b4440 legacy=0x7fe8b00b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.852+0000 7fe8a7e00640 1 -- 172.21.15.157:0/555454062 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00ce840 con 0x7fe8b00cde30 2024-04-05T22:20:56.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.853+0000 7fe8a7e00640 1 --2- 172.21.15.157:0/555454062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc362e87617c3c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:56.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.853+0000 7fe8aea00640 1 -- 172.21.15.157:0/555454062 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898031640 con 0x7fe8b00cde30 2024-04-05T22:20:56.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.853+0000 7fe8aea00640 1 -- 172.21.15.157:0/555454062 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8980317a0 con 0x7fe8b00cde30 2024-04-05T22:20:56.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.853+0000 7fe8aea00640 1 -- 172.21.15.157:0/555454062 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898031b30 con 0x7fe8b00cde30 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 -- 172.21.15.157:0/555454062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 msgr2=0x7fe8b015b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 --2- 172.21.15.157:0/555454062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 -- 172.21.15.157:0/555454062 shutdown_connections 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 --2- 172.21.15.157:0/555454062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00cde30 0x7fe8b015b110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 --2- 172.21.15.157:0/555454062 >> v2:172.21.15.157:3300/0 conn(0x7fe8b015ad30 0x7fe8b00b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 -- 172.21.15.157:0/555454062 >> 172.21.15.157:0/555454062 conn(0x7fe8b0099a60 msgr2=0x7fe8b0099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 -- 172.21.15.157:0/555454062 shutdown_connections 2024-04-05T22:20:56.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.854+0000 7fe8b6600640 1 -- 172.21.15.157:0/555454062 wait complete. 2024-04-05T22:20:56.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.856+0000 7fe8b6600640 1 Processor -- start 2024-04-05T22:20:56.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.856+0000 7fe8b6600640 1 -- start start 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.856+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b00cde30 0x7fe8b01e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b015ad30 0x7fe8b006c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8b6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b01e7c70 con 0x7fe8b00cde30 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8b6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b01e77e0 con 0x7fe8b015ad30 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8b6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b01e8bc0 con 0x7fe8b00b4440 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49284/0 (socket says 172.21.15.157:49284) 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 -- 172.21.15.157:0/4232345213 learned_addr learned my addr 172.21.15.157:0/4232345213 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8afe00640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b015ad30 0x7fe8b006c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8a7e00640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b00cde30 0x7fe8b01e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:56.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 -- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b015ad30 msgr2=0x7fe8b006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b015ad30 0x7fe8b006c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 -- 172.21.15.157:0/4232345213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b00cde30 msgr2=0x7fe8b01e8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b00cde30 0x7fe8b01e8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.857+0000 7fe8af400640 1 -- 172.21.15.157:0/4232345213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980094d0 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8af400640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e8a0 tx=0x7fe8a000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00094f0 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a000c940 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000ccf0 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b01e6bd0 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01e7190 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe8a0005c90 con 0x7fe8b00b4440 2024-04-05T22:20:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.858+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe8a00137b0 con 0x7fe8b00b4440 2024-04-05T22:20:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.859+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0068160 con 0x7fe8b00b4440 2024-04-05T22:20:56.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.862+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7fe8a0021060 con 0x7fe8b00b4440 2024-04-05T22:20:56.885 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:56.885+0000 7f5588e37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-05T22:20:56.928 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:56.928+0000 7f1e0fa32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-05T22:20:56.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.939+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0c57082f-7b11-49ad-9aa2-3d7c81942802", "id": 0} v 0) v1 -- 0x7fe8b006cdb0 con 0x7fe8b00b4440 2024-04-05T22:20:56.946 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:56.946+0000 7f5588e37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-05T22:20:56.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.948+0000 7fe8a7400640 1 -- 172.21.15.157:0/4232345213 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c57082f-7b11-49ad-9aa2-3d7c81942802", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8a0017070 con 0x7fe8b00b4440 2024-04-05T22:20:56.949 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 msgr2=0x7fe8b01e6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e8a0 tx=0x7fe8a000ed60 comp rx=0 tx=0).stop 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 shutdown_connections 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b015ad30 0x7fe8b006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b00cde30 0x7fe8b01e8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 --2- 172.21.15.157:0/4232345213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8b00b4440 0x7fe8b01e6540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 >> 172.21.15.157:0/4232345213 conn(0x7fe8b0099a60 msgr2=0x7fe8b006f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 shutdown_connections 2024-04-05T22:20:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:56.950+0000 7fe8b6600640 1 -- 172.21.15.157:0/4232345213 wait complete. 2024-04-05T22:20:56.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 620a6081-b2dd-4f3d-90de-3cb57d1f6788 1 2024-04-05T22:20:56.994 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:56.993+0000 7f1e0fa32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-05T22:20:56.997 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:56.997+0000 7f5588e37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-05T22:20:57.048 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.048+0000 7f1e0fa32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-05T22:20:57.058 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.058+0000 7f5588e37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-05T22:20:57.106 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.106+0000 7f5588e37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-05T22:20:57.115 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.114+0000 7f1e0fa32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-05T22:20:57.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.137+0000 7f2d25400640 1 Processor -- start 2024-04-05T22:20:57.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.138+0000 7f2d25400640 1 -- start start 2024-04-05T22:20:57.140 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.139+0000 7f2d25400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2d200b1bf0 0x7f2d200b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.139+0000 7f2d25400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d200b1750 0x7f2d20166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.139+0000 7f2d25400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2006adc0 con 0x7f2d200b0720 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.139+0000 7f2d25400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200b2460 con 0x7f2d200b1bf0 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.139+0000 7f2d25400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2015d4f0 con 0x7f2d200b1750 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d200b1750 0x7f2d20166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d200b1750 0x7f2d20166c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36456/0 (socket says 172.21.15.157:36456) 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 -- 172.21.15.157:0/4250635871 learned_addr learned my addr 172.21.15.157:0/4250635871 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d17e00640 1 --2- 172.21.15.157:0/4250635871 >> v2:172.21.15.73:3300/0 conn(0x7f2d200b1bf0 0x7f2d200b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 -- 172.21.15.157:0/4250635871 >> v2:172.21.15.73:3300/0 conn(0x7f2d200b1bf0 msgr2=0x7f2d200b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 --2- 172.21.15.157:0/4250635871 >> v2:172.21.15.73:3300/0 conn(0x7f2d200b1bf0 0x7f2d200b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 -- 172.21.15.157:0/4250635871 >> v1:172.21.15.73:6789/0 conn(0x7f2d200b0720 legacy=0x7f2d200b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:57.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.140+0000 7f2d1f200640 1 -- 172.21.15.157:0/4250635871 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200b51c0 con 0x7f2d200b1750 2024-04-05T22:20:57.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.141+0000 7f2d1f200640 1 --2- 172.21.15.157:0/4250635871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1750 0x7f2d20166c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d200afcc0 tx=0x7f2d1802be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=325861c9479717b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.142+0000 7f2d1de00640 1 -- 172.21.15.157:0/4250635871 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1802c9f0 con 0x7f2d200b1750 2024-04-05T22:20:57.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.142+0000 7f2d1de00640 1 -- 172.21.15.157:0/4250635871 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1802cb50 con 0x7f2d200b1750 2024-04-05T22:20:57.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.142+0000 7f2d1de00640 1 -- 172.21.15.157:0/4250635871 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180343f0 con 0x7f2d200b1750 2024-04-05T22:20:57.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 -- 172.21.15.157:0/4250635871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1750 msgr2=0x7f2d20166c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 --2- 172.21.15.157:0/4250635871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1750 0x7f2d20166c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d200afcc0 tx=0x7f2d1802be50 comp rx=0 tx=0).stop 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 -- 172.21.15.157:0/4250635871 shutdown_connections 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 --2- 172.21.15.157:0/4250635871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1750 0x7f2d20166c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 --2- 172.21.15.157:0/4250635871 >> v2:172.21.15.73:3300/0 conn(0x7f2d200b1bf0 0x7f2d200b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 -- 172.21.15.157:0/4250635871 >> 172.21.15.157:0/4250635871 conn(0x7f2d20099a60 msgr2=0x7f2d20099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 -- 172.21.15.157:0/4250635871 shutdown_connections 2024-04-05T22:20:57.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.143+0000 7f2d25400640 1 -- 172.21.15.157:0/4250635871 wait complete. 2024-04-05T22:20:57.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.144+0000 7f2d25400640 1 Processor -- start 2024-04-05T22:20:57.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.145+0000 7f2d25400640 1 -- start start 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.145+0000 7f2d25400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.145+0000 7f2d25400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d200b1750 0x7f2d2015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d25400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 0x7f2d20152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d25400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d201571e0 con 0x7f2d200b1750 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d25400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015db70 con 0x7f2d200b1bf0 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d25400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20159910 con 0x7f2d200b0720 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 0x7f2d20152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49298/0 (socket says 172.21.15.157:49298) 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 0x7f2d20152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36470/0 (socket says 172.21.15.157:36470) 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 -- 172.21.15.157:0/3235486614 learned_addr learned my addr 172.21.15.157:0/3235486614 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:57.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d17e00640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d200b1750 0x7f2d2015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 -- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 msgr2=0x7f2d20152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 0x7f2d20152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 -- 172.21.15.157:0/3235486614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d200b1750 msgr2=0x7f2d2015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d200b1750 0x7f2d2015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.146+0000 7f2d1e800640 1 -- 172.21.15.157:0/3235486614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009b30 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d1e800640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b3f0 tx=0x7f2d1000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10011830 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10011990 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10011d40 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d20156c20 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2015d800 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2d10004320 con 0x7f2d200b0720 2024-04-05T22:20:57.149 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.147+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2d1001a400 con 0x7f2d200b0720 2024-04-05T22:20:57.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.148+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2015d660 con 0x7f2d200b0720 2024-04-05T22:20:57.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.150+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f2d10038c20 con 0x7f2d200b0720 2024-04-05T22:20:57.168 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.168+0000 7f1e0fa32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-05T22:20:57.230 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.229+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "620a6081-b2dd-4f3d-90de-3cb57d1f6788", "id": 1} v 0) v1 -- 0x7f2d20068160 con 0x7f2d200b0720 2024-04-05T22:20:57.236 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.235+0000 7f5588e37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-05T22:20:57.239 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.238+0000 7f2d16a00640 1 -- 172.21.15.157:0/3235486614 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "620a6081-b2dd-4f3d-90de-3cb57d1f6788", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d10027070 con 0x7f2d200b0720 2024-04-05T22:20:57.239 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:20:57.240 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.239+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 msgr2=0x7f2d201565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.240 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.239+0000 7f2d25400640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b3f0 tx=0x7f2d1000b8b0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 shutdown_connections 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d200b1bf0 0x7f2d20152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d200b1750 0x7f2d2015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 --2- 172.21.15.157:0/3235486614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d200b0720 0x7f2d201565b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 >> 172.21.15.157:0/3235486614 conn(0x7f2d20099a60 msgr2=0x7f2d20159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 shutdown_connections 2024-04-05T22:20:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.240+0000 7f2d25400640 1 -- 172.21.15.157:0/3235486614 wait complete. 2024-04-05T22:20:57.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 236682ef-6293-4b5c-837f-c010a131e2c5 2 2024-04-05T22:20:57.306 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.306+0000 7f1e0fa32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-05T22:20:57.315 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.314+0000 7f5588e37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-05T22:20:57.367 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.367+0000 7f5588e37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-05T22:20:57.392 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.391+0000 7f1e0fa32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-05T22:20:57.426 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.426+0000 7f5588e37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-05T22:20:57.449 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.448+0000 7f1e0fa32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-05T22:20:57.480 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.479+0000 7f5588e37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-05T22:20:57.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.489+0000 7f6bbbe00640 1 Processor -- start 2024-04-05T22:20:57.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.490+0000 7f6bbbe00640 1 -- start start 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bbbe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bbc0d14a0 0x7f6bbc0af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bbbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bbc0b3b80 0x7f6bbc0afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bbbe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0b26e0 con 0x7f6bbc15d0a0 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bbbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b1050 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bbbe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b1330 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bb3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bbc0d14a0 0x7f6bbc0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bb3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bbc0d14a0 0x7f6bbc0af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36478/0 (socket says 172.21.15.157:36478) 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.491+0000 7f6bb3e00640 1 -- 172.21.15.157:0/387231114 learned_addr learned my addr 172.21.15.157:0/387231114 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.492+0000 7f6bbb400640 1 --2- 172.21.15.157:0/387231114 >> v2:172.21.15.73:3300/0 conn(0x7f6bbc0b3b80 0x7f6bbc0afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.492+0000 7f6bb3e00640 1 -- 172.21.15.157:0/387231114 >> v2:172.21.15.73:3300/0 conn(0x7f6bbc0b3b80 msgr2=0x7f6bbc0afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.492+0000 7f6bb3e00640 1 --2- 172.21.15.157:0/387231114 >> v2:172.21.15.73:3300/0 conn(0x7f6bbc0b3b80 0x7f6bbc0afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.492+0000 7f6bb3e00640 1 -- 172.21.15.157:0/387231114 >> v1:172.21.15.73:6789/0 conn(0x7f6bbc15d0a0 legacy=0x7f6bbc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.492+0000 7f6bb3e00640 1 -- 172.21.15.157:0/387231114 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0b2370 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.493+0000 7f6bb3e00640 1 --2- 172.21.15.157:0/387231114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0d14a0 0x7f6bbc0af610 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0b8b10 tx=0x7f6ba402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f17ca8849851890 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.493+0000 7f6bba000640 1 -- 172.21.15.157:0/387231114 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4031870 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.493+0000 7f6bba000640 1 -- 172.21.15.157:0/387231114 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40319d0 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.493+0000 7f6bba000640 1 -- 172.21.15.157:0/387231114 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4031d60 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 -- 172.21.15.157:0/387231114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0d14a0 msgr2=0x7f6bbc0af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/387231114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0d14a0 0x7f6bbc0af610 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0b8b10 tx=0x7f6ba402b910 comp rx=0 tx=0).stop 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 -- 172.21.15.157:0/387231114 shutdown_connections 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/387231114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0d14a0 0x7f6bbc0af610 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/387231114 >> v2:172.21.15.73:3300/0 conn(0x7f6bbc0b3b80 0x7f6bbc0afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 -- 172.21.15.157:0/387231114 >> 172.21.15.157:0/387231114 conn(0x7f6bbc0d2a60 msgr2=0x7f6bbc0d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 -- 172.21.15.157:0/387231114 shutdown_connections 2024-04-05T22:20:57.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.494+0000 7f6bbbe00640 1 -- 172.21.15.157:0/387231114 wait complete. 2024-04-05T22:20:57.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.496+0000 7f6bbbe00640 1 Processor -- start 2024-04-05T22:20:57.496 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.496+0000 7f6bbbe00640 1 -- start start 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 0x7f6bbc0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bbc15d0a0 0x7f6bbc0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b5680 con 0x7f6bbc15d0a0 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b58d0 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0b5bb0 con 0x7f6bbc0d14a0 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bb3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 0x7f6bbc0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36492/0 (socket says 172.21.15.157:36492) 2024-04-05T22:20:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 -- 172.21.15.157:0/3988873023 learned_addr learned my addr 172.21.15.157:0/3988873023 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbb400640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bbc15d0a0 0x7f6bbc0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 -- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 msgr2=0x7f6bbc0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 0x7f6bbc0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 -- 172.21.15.157:0/3988873023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bbc15d0a0 msgr2=0x7f6bbc0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bbc15d0a0 0x7f6bbc0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bbaa00640 1 -- 172.21.15.157:0/3988873023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40094d0 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.497+0000 7f6bb3e00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 0x7f6bbc0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bbaa00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b3f0 tx=0x7f6bac00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac011830 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac011990 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0b5e90 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac011d40 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc15bc10 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.498+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6bac004320 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.499+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f6bac003a10 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.501 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.499+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc0b63b0 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.502+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f6bac038c20 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.513 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.513+0000 7f1e0fa32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-05T22:20:57.571 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:57.570+0000 7f1e0fa32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-05T22:20:57.583 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.582+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "236682ef-6293-4b5c-837f-c010a131e2c5", "id": 2} v 0) v1 -- 0x7f6bbc067250 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.592 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.592+0000 7f6bb2a00640 1 -- 172.21.15.157:0/3988873023 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "236682ef-6293-4b5c-837f-c010a131e2c5", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6bac027070 con 0x7f6bbc0b3b80 2024-04-05T22:20:57.592 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 msgr2=0x7f6bbc15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b3f0 tx=0x7f6bac00b8b0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 shutdown_connections 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bbc15d0a0 0x7f6bbc0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bbc0d14a0 0x7f6bbc0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 --2- 172.21.15.157:0/3988873023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bbc0b3b80 0x7f6bbc15b6d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 >> 172.21.15.157:0/3988873023 conn(0x7f6bbc0d2a60 msgr2=0x7f6bbc0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 shutdown_connections 2024-04-05T22:20:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.593+0000 7f6bbbe00640 1 -- 172.21.15.157:0/3988873023 wait complete. 2024-04-05T22:20:57.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 48454405-0107-42fb-a71e-ef13d6c8198c 3 2024-04-05T22:20:57.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.848+0000 7fb50ea00640 1 Processor -- start 2024-04-05T22:20:57.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.848+0000 7fb50ea00640 1 -- start start 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 0x7fb5080cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5080cdde0 0x7fb5080ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5080b44e0 con 0x7fb508160720 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080d11a0 con 0x7fb5080cdde0 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080d1c40 con 0x7fb5080cfce0 2024-04-05T22:20:57.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb507400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 0x7fb5080cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb507e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb508160720 0x7fb5080cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57458/0 (socket says 172.21.15.157:57458) 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb507e00640 1 -- 172.21.15.157:0/3383635363 learned_addr learned my addr 172.21.15.157:0/3383635363 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb50ca00640 1 --2- 172.21.15.157:0/3383635363 >> v2:172.21.15.157:3300/0 conn(0x7fb5080cdde0 0x7fb5080ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb507400640 1 -- 172.21.15.157:0/3383635363 >> v2:172.21.15.157:3300/0 conn(0x7fb5080cdde0 msgr2=0x7fb5080ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.851+0000 7fb507400640 1 --2- 172.21.15.157:0/3383635363 >> v2:172.21.15.157:3300/0 conn(0x7fb5080cdde0 0x7fb5080ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.851+0000 7fb507400640 1 -- 172.21.15.157:0/3383635363 >> v1:172.21.15.157:6789/0 conn(0x7fb508160720 legacy=0x7fb5080cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.850+0000 7fb506a00640 1 -- 172.21.15.157:0/3383635363 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478460413 0 0) 0x7fb5080b44e0 con 0x7fb508160720 2024-04-05T22:20:57.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.851+0000 7fb507400640 1 -- 172.21.15.157:0/3383635363 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080b3f20 con 0x7fb5080cfce0 2024-04-05T22:20:57.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.852+0000 7fb507400640 1 --2- 172.21.15.157:0/3383635363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 0x7fb5080cd8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40097a0 tx=0x7fb4f402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634edad28ec6a675 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.852+0000 7fb506a00640 1 -- 172.21.15.157:0/3383635363 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4031780 con 0x7fb5080cfce0 2024-04-05T22:20:57.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.852+0000 7fb506a00640 1 -- 172.21.15.157:0/3383635363 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f40318e0 con 0x7fb5080cfce0 2024-04-05T22:20:57.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.852+0000 7fb506a00640 1 -- 172.21.15.157:0/3383635363 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4031c70 con 0x7fb5080cfce0 2024-04-05T22:20:57.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 -- 172.21.15.157:0/3383635363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 msgr2=0x7fb5080cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 --2- 172.21.15.157:0/3383635363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 0x7fb5080cd8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40097a0 tx=0x7fb4f402b820 comp rx=0 tx=0).stop 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 -- 172.21.15.157:0/3383635363 shutdown_connections 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 --2- 172.21.15.157:0/3383635363 >> v2:172.21.15.157:3300/0 conn(0x7fb5080cdde0 0x7fb5080ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 --2- 172.21.15.157:0/3383635363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5080cfce0 0x7fb5080cd8a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 -- 172.21.15.157:0/3383635363 >> 172.21.15.157:0/3383635363 conn(0x7fb50809c000 msgr2=0x7fb5080d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 -- 172.21.15.157:0/3383635363 shutdown_connections 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.853+0000 7fb50ea00640 1 -- 172.21.15.157:0/3383635363 wait complete. 2024-04-05T22:20:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.855+0000 7fb50ea00640 1 Processor -- start 2024-04-05T22:20:57.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.855+0000 7fb50ea00640 1 -- start start 2024-04-05T22:20:57.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5080cdde0 0x7fb5080c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080cb040 con 0x7fb5080cdde0 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50815b850 con 0x7fb5080cfce0 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c9b20 con 0x7fb508160720 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb507400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb507400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36496/0 (socket says 172.21.15.157:36496) 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb50ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:49326/0 (socket says 172.21.15.157:49326) 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.856+0000 7fb507400640 1 -- 172.21.15.157:0/2644318140 learned_addr learned my addr 172.21.15.157:0/2644318140 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507e00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5080cdde0 0x7fb5080c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 -- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 msgr2=0x7fb5080c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 -- 172.21.15.157:0/2644318140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5080cdde0 msgr2=0x7fb5080c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5080cdde0 0x7fb5080c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 -- 172.21.15.157:0/2644318140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009400 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb50ca00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb507400640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4034040 tx=0x7fb4f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.857+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4032820 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4032980 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4032d30 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080ca430 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5080ca9f0 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb4f404e440 con 0x7fb5080cfce0 2024-04-05T22:20:57.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.858+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb4f404eb80 con 0x7fb5080cfce0 2024-04-05T22:20:57.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.859+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080d1400 con 0x7fb5080cfce0 2024-04-05T22:20:57.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.861+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7fb4f404e6f0 con 0x7fb5080cfce0 2024-04-05T22:20:57.927 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:57.926+0000 7f5588e37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-05T22:20:57.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.939+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "48454405-0107-42fb-a71e-ef13d6c8198c", "id": 3} v 0) v1 -- 0x7fb5080cb1b0 con 0x7fb5080cfce0 2024-04-05T22:20:57.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.949+0000 7fb4ffe00640 1 -- 172.21.15.157:0/2644318140 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48454405-0107-42fb-a71e-ef13d6c8198c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb4f404e910 con 0x7fb5080cfce0 2024-04-05T22:20:57.950 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 msgr2=0x7fb50815b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4034040 tx=0x7fb4f40046c0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 shutdown_connections 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb508160720 0x7fb5080c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5080cfce0 0x7fb50815b310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 --2- 172.21.15.157:0/2644318140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5080cdde0 0x7fb5080c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 >> 172.21.15.157:0/2644318140 conn(0x7fb50809c000 msgr2=0x7fb5080b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 shutdown_connections 2024-04-05T22:20:57.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:57.950+0000 7fb50ea00640 1 -- 172.21.15.157:0/2644318140 wait complete. 2024-04-05T22:20:57.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 3be8f606-e96f-484d-9db0-a5b60f39ba79 4 2024-04-05T22:20:58.050 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.049+0000 7f1e0fa32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-05T22:20:58.086 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.086+0000 7f5588e37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-05T22:20:58.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.159+0000 7f7253e00640 1 Processor -- start 2024-04-05T22:20:58.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.160+0000 7f7253e00640 1 -- start start 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72540d14a0 0x7f72540af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72540b3b80 0x7f72540afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540b26e0 con 0x7f725415d0a0 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540b1050 con 0x7f72540d14a0 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540b1330 con 0x7f72540b3b80 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.161+0000 7f7253400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72540b3b80 0x7f72540afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72540b3b80 0x7f72540afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38422/0 (socket says 172.21.15.157:38422) 2024-04-05T22:20:58.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 -- 172.21.15.157:0/1851559626 learned_addr learned my addr 172.21.15.157:0/1851559626 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:58.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7252000640 1 --2- 172.21.15.157:0/1851559626 >> v2:172.21.15.73:3300/0 conn(0x7f72540d14a0 0x7f72540af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 -- 172.21.15.157:0/1851559626 >> v2:172.21.15.73:3300/0 conn(0x7f72540d14a0 msgr2=0x7f72540af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:58.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 --2- 172.21.15.157:0/1851559626 >> v2:172.21.15.73:3300/0 conn(0x7f72540d14a0 0x7f72540af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 -- 172.21.15.157:0/1851559626 >> v1:172.21.15.73:6789/0 conn(0x7f725415d0a0 legacy=0x7f72540d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.162+0000 7f7253400640 1 -- 172.21.15.157:0/1851559626 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540b2370 con 0x7f72540b3b80 2024-04-05T22:20:58.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.163+0000 7f7253400640 1 --2- 172.21.15.157:0/1851559626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540afb50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f725408bde0 tx=0x7f724c02bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a917487dcc8708 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.163+0000 7f7251600640 1 -- 172.21.15.157:0/1851559626 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c02cb20 con 0x7f72540b3b80 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.163+0000 7f7251600640 1 -- 172.21.15.157:0/1851559626 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c02cc80 con 0x7f72540b3b80 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.163+0000 7f7251600640 1 -- 172.21.15.157:0/1851559626 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0343f0 con 0x7f72540b3b80 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 -- 172.21.15.157:0/1851559626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 msgr2=0x7f72540afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 --2- 172.21.15.157:0/1851559626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540afb50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f725408bde0 tx=0x7f724c02bf80 comp rx=0 tx=0).stop 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 -- 172.21.15.157:0/1851559626 shutdown_connections 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 --2- 172.21.15.157:0/1851559626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540afb50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 --2- 172.21.15.157:0/1851559626 >> v2:172.21.15.73:3300/0 conn(0x7f72540d14a0 0x7f72540af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 -- 172.21.15.157:0/1851559626 >> 172.21.15.157:0/1851559626 conn(0x7f72540d2a60 msgr2=0x7f72540d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 -- 172.21.15.157:0/1851559626 shutdown_connections 2024-04-05T22:20:58.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.164+0000 7f7253e00640 1 -- 172.21.15.157:0/1851559626 wait complete. 2024-04-05T22:20:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.166+0000 7f7253e00640 1 Processor -- start 2024-04-05T22:20:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.166+0000 7f7253e00640 1 -- start start 2024-04-05T22:20:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72540d14a0 0x7f725415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725415d0a0 0x7f72540b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725415b8c0 con 0x7f72540d14a0 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540b8270 con 0x7f72540b3b80 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7253e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cad70 con 0x7f725415d0a0 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7252a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7252a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38430/0 (socket says 172.21.15.157:38430) 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.167+0000 7f7252a00640 1 -- 172.21.15.157:0/2542820504 learned_addr learned my addr 172.21.15.157:0/2542820504 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7253400640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725415d0a0 0x7f72540b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252000640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72540d14a0 0x7f725415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 -- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725415d0a0 msgr2=0x7f72540b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725415d0a0 0x7f72540b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 -- 172.21.15.157:0/2542820504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72540d14a0 msgr2=0x7f725415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72540d14a0 0x7f725415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 -- 172.21.15.157:0/2542820504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009b30 con 0x7f72540b3b80 2024-04-05T22:20:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f7252a00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f724400c400 tx=0x7f724400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.168+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440052d0 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244005430 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72540cb050 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244003830 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72540cb610 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7244003c90 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.169+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7244011aa0 con 0x7f72540b3b80 2024-04-05T22:20:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.170+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7254067250 con 0x7f72540b3b80 2024-04-05T22:20:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.172+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f7244017070 con 0x7f72540b3b80 2024-04-05T22:20:58.216 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.215+0000 7f1e0fa32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-05T22:20:58.226 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.226+0000 7f5588e37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-05T22:20:58.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.250+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3be8f606-e96f-484d-9db0-a5b60f39ba79", "id": 4} v 0) v1 -- 0x7f725404e160 con 0x7f72540b3b80 2024-04-05T22:20:58.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.260+0000 7f724aa00640 1 -- 172.21.15.157:0/2542820504 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3be8f606-e96f-484d-9db0-a5b60f39ba79", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7244011590 con 0x7f72540b3b80 2024-04-05T22:20:58.260 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 msgr2=0x7f72540ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f724400c400 tx=0x7f724400c8c0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 shutdown_connections 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725415d0a0 0x7f72540b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72540d14a0 0x7f725415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 --2- 172.21.15.157:0/2542820504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72540b3b80 0x7f72540ca770 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 >> 172.21.15.157:0/2542820504 conn(0x7f72540d2a60 msgr2=0x7f72540cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 shutdown_connections 2024-04-05T22:20:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.261+0000 7f7253e00640 1 -- 172.21.15.157:0/2542820504 wait complete. 2024-04-05T22:20:58.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 5dcdb399-2827-4da5-911d-dc7ced11744c 5 2024-04-05T22:20:58.279 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.279+0000 7f5588e37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-05T22:20:58.361 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.361+0000 7f1e0fa32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-05T22:20:58.402 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.402+0000 7f5588e37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:20:58.416 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.415+0000 7f1e0fa32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-05T22:20:58.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.461+0000 7f0786000640 1 Processor -- start 2024-04-05T22:20:58.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.462+0000 7f0786000640 1 -- start start 2024-04-05T22:20:58.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.463+0000 7f0786000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0780160490 0x7f07800cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.463+0000 7f0786000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07800ce460 con 0x7f07800b3c70 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.463+0000 7f0786000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07800ce740 con 0x7f07800cfd70 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.463+0000 7f0786000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07800cea20 con 0x7f0780160490 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f0777e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f07800b3c70 0x7f07800cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.157:45782/0 (socket says 172.21.15.157:45782) 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f0777e00640 1 -- 172.21.15.157:0/3123732385 learned_addr learned my addr 172.21.15.157:0/3123732385 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-05T22:20:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077f400640 1 --2- 172.21.15.157:0/3123732385 >> v2:172.21.15.73:3300/0 conn(0x7f0780160490 0x7f07800cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970222151 0 0) 0x7f07800ce740 con 0x7f07800cfd70 2024-04-05T22:20:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0764003520 con 0x7f07800cfd70 2024-04-05T22:20:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794777467 0 0) 0x7f07800ce460 con 0x7f07800b3c70 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07800ce740 con 0x7f07800b3c70 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882795959 0 0) 0x7f0764003520 con 0x7f07800cfd70 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077f400640 1 -- 172.21.15.157:0/3123732385 >> v1:172.21.15.157:6789/0 conn(0x7f07800cfd70 legacy=0x7f07800d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077f400640 1 -- 172.21.15.157:0/3123732385 >> v1:172.21.15.73:6789/0 conn(0x7f07800b3c70 legacy=0x7f07800cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.464+0000 7f077f400640 1 -- 172.21.15.157:0/3123732385 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07800ced00 con 0x7f0780160490 2024-04-05T22:20:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.466+0000 7f077f400640 1 --2- 172.21.15.157:0/3123732385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780160490 0x7f07800cd9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0770009720 tx=0x7f077002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8934152b5012583 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.466+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780019f0 con 0x7f0780160490 2024-04-05T22:20:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.466+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770031520 con 0x7f0780160490 2024-04-05T22:20:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.466+0000 7f077ea00640 1 -- 172.21.15.157:0/3123732385 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700316b0 con 0x7f0780160490 2024-04-05T22:20:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 -- 172.21.15.157:0/3123732385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780160490 msgr2=0x7f07800cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 --2- 172.21.15.157:0/3123732385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780160490 0x7f07800cd9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0770009720 tx=0x7f077002b7c0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 -- 172.21.15.157:0/3123732385 shutdown_connections 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 --2- 172.21.15.157:0/3123732385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780160490 0x7f07800cd9a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 -- 172.21.15.157:0/3123732385 >> 172.21.15.157:0/3123732385 conn(0x7f078009c020 msgr2=0x7f07800d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 -- 172.21.15.157:0/3123732385 shutdown_connections 2024-04-05T22:20:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.467+0000 7f0786000640 1 -- 172.21.15.157:0/3123732385 wait complete. 2024-04-05T22:20:58.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.468+0000 7f0786000640 1 Processor -- start 2024-04-05T22:20:58.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.469+0000 7f0786000640 1 -- start start 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.469+0000 7f0786000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.469+0000 7f0786000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07800cfd70 0x7f07800d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f0786000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 0x7f07800d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f0786000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c7270 con 0x7f07800cfd70 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f0786000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800caeb0 con 0x7f0780160490 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f0786000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c6940 con 0x7f07800b3c70 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 0x7f07800d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:57610/0 (socket says 172.21.15.157:57610) 2024-04-05T22:20:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077f400640 1 -- 172.21.15.157:0/1375484904 learned_addr learned my addr 172.21.15.157:0/1375484904 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f0777e00640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07800cfd70 0x7f07800d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 -- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 msgr2=0x7f078014a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 -- 172.21.15.157:0/1375484904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07800cfd70 msgr2=0x7f07800d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07800cfd70 0x7f07800d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077fe00640 1 -- 172.21.15.157:0/1375484904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009400 con 0x7f0780160490 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.470+0000 7f077f400640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:20:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f077fe00640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 0x7f07800d32d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0778001460 tx=0x7f077800cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778003d80 con 0x7f0780160490 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778005070 con 0x7f0780160490 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778005420 con 0x7f0780160490 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078014a900 con 0x7f0780160490 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07800c6e90 con 0x7f0780160490 2024-04-05T22:20:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f077800a610 con 0x7f0780160490 2024-04-05T22:20:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.471+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f0778012400 con 0x7f0780160490 2024-04-05T22:20:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.472+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07800c73b0 con 0x7f0780160490 2024-04-05T22:20:58.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.476+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f07800c73b0 con 0x7f0780160490 2024-04-05T22:20:58.512 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.512+0000 7f5588e37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-05T22:20:58.549 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.549+0000 7f1e0fa32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:20:58.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.554+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5dcdb399-2827-4da5-911d-dc7ced11744c", "id": 5} v 0) v1 -- 0x7f0780068160 con 0x7f0780160490 2024-04-05T22:20:58.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.564+0000 7f0777400640 1 -- 172.21.15.157:0/1375484904 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5dcdb399-2827-4da5-911d-dc7ced11744c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f077800a990 con 0x7f0780160490 2024-04-05T22:20:58.565 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 msgr2=0x7f07800d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 0x7f07800d32d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0778001460 tx=0x7f077800cd00 comp rx=0 tx=0).stop 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 shutdown_connections 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780160490 0x7f07800d32d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07800cfd70 0x7f07800d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 --2- 172.21.15.157:0/1375484904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07800b3c70 0x7f078014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 >> 172.21.15.157:0/1375484904 conn(0x7f078009c020 msgr2=0x7f07800cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 shutdown_connections 2024-04-05T22:20:58.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.566+0000 7f0786000640 1 -- 172.21.15.157:0/1375484904 wait complete. 2024-04-05T22:20:58.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 8047a6f8-c2c1-462e-876e-7700088d2ca7 6 2024-04-05T22:20:58.670 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.669+0000 7f1e0fa32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-05T22:20:58.689 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.688+0000 7f5588e37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:20:58.737 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.737+0000 7f5588e37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-05T22:20:58.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.769+0000 7f1761800640 1 Processor -- start 2024-04-05T22:20:58.770 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.769+0000 7f1761800640 1 -- start start 2024-04-05T22:20:58.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f1761800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f1761800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c0b1b30 con 0x7f175c0cbe00 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f1761800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c0b1e10 con 0x7f175c0b32b0 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f1761800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0ce7b0 con 0x7f175c06c630 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f175aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f175b400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f175c0b32b0 0x7f175c1647c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55514/0 (socket says 172.21.15.157:55514) 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.771+0000 7f175b400640 1 -- 172.21.15.157:0/2447496661 learned_addr learned my addr 172.21.15.157:0/2447496661 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.772+0000 7f175aa00640 1 -- 172.21.15.157:0/2447496661 >> v1:172.21.15.157:6789/0 conn(0x7f175c0b32b0 legacy=0x7f175c1647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.772+0000 7f175a000640 1 -- 172.21.15.157:0/2447496661 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689712915 0 0) 0x7f175c0b1e10 con 0x7f175c0b32b0 2024-04-05T22:20:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.772+0000 7f175aa00640 1 -- 172.21.15.157:0/2447496661 >> v1:172.21.15.73:6789/0 conn(0x7f175c0cbe00 legacy=0x7f175c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.772+0000 7f175aa00640 1 -- 172.21.15.157:0/2447496661 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c06cf50 con 0x7f175c06c630 2024-04-05T22:20:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.773+0000 7f175aa00640 1 --2- 172.21.15.157:0/2447496661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c06ca10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f174c009dc0 tx=0x7f174c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30fbae18f2d7d3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.773+0000 7f175a000640 1 -- 172.21.15.157:0/2447496661 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c02cc40 con 0x7f175c06c630 2024-04-05T22:20:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.773+0000 7f175a000640 1 -- 172.21.15.157:0/2447496661 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c02cda0 con 0x7f175c06c630 2024-04-05T22:20:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.773+0000 7f175a000640 1 -- 172.21.15.157:0/2447496661 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c034660 con 0x7f175c06c630 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 -- 172.21.15.157:0/2447496661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 msgr2=0x7f175c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 --2- 172.21.15.157:0/2447496661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c06ca10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f174c009dc0 tx=0x7f174c02be90 comp rx=0 tx=0).stop 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 -- 172.21.15.157:0/2447496661 shutdown_connections 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 --2- 172.21.15.157:0/2447496661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c06ca10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 -- 172.21.15.157:0/2447496661 >> 172.21.15.157:0/2447496661 conn(0x7f175c0b0830 msgr2=0x7f175c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 -- 172.21.15.157:0/2447496661 shutdown_connections 2024-04-05T22:20:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.774+0000 7f1761800640 1 -- 172.21.15.157:0/2447496661 wait complete. 2024-04-05T22:20:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.776+0000 7f1761800640 1 Processor -- start 2024-04-05T22:20:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.777+0000 7f1761800640 1 -- start start 2024-04-05T22:20:58.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f175c0b32b0 0x7f175c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175c0cbe00 0x7f175c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c143370 con 0x7f175c0b32b0 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1f2220 con 0x7f175c0cbe00 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f1761800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c070930 con 0x7f175c06c630 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f175aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f175aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:57634/0 (socket says 172.21.15.157:57634) 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.778+0000 7f175aa00640 1 -- 172.21.15.157:0/424637247 learned_addr learned my addr 172.21.15.157:0/424637247 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175b400640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175c0cbe00 0x7f175c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f1752000640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f175c0b32b0 0x7f175c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 -- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175c0cbe00 msgr2=0x7f175c06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175c0cbe00 0x7f175c06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 -- 172.21.15.157:0/424637247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f175c0b32b0 msgr2=0x7f175c06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f175c0b32b0 0x7f175c06f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 -- 172.21.15.157:0/424637247 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c009a70 con 0x7f175c06c630 2024-04-05T22:20:58.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f175aa00640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f174c000f80 tx=0x7f174c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.779+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c040650 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0407b0 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c070c10 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c040b60 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c143000 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f174c003990 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.780+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f174c004660 con 0x7f175c06c630 2024-04-05T22:20:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.781+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c068160 con 0x7f175c06c630 2024-04-05T22:20:58.783 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.783+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f174c044020 con 0x7f175c06c630 2024-04-05T22:20:58.858 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.857+0000 7f1e0fa32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:20:58.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.860+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8047a6f8-c2c1-462e-876e-7700088d2ca7", "id": 6} v 0) v1 -- 0x7f175c0ae520 con 0x7f175c06c630 2024-04-05T22:20:58.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.870+0000 7f1753400640 1 -- 172.21.15.157:0/424637247 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8047a6f8-c2c1-462e-876e-7700088d2ca7", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f174c039070 con 0x7f175c06c630 2024-04-05T22:20:58.871 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 msgr2=0x7f175c072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f174c000f80 tx=0x7f174c004a40 comp rx=0 tx=0).stop 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 shutdown_connections 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175c0cbe00 0x7f175c06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f175c0b32b0 0x7f175c06f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 --2- 172.21.15.157:0/424637247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f175c06c630 0x7f175c072f60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 >> 172.21.15.157:0/424637247 conn(0x7f175c0b0830 msgr2=0x7f175c15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 shutdown_connections 2024-04-05T22:20:58.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:58.872+0000 7f1761800640 1 -- 172.21.15.157:0/424637247 wait complete. 2024-04-05T22:20:58.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 09e1e050-1cef-45dc-9aa2-2fa40597a903 7 2024-04-05T22:20:58.909 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:58.908+0000 7f5588e37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-05T22:20:58.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:58.910+0000 7f1e0fa32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-05T22:20:59.073 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.073+0000 7fa8f3200640 1 Processor -- start 2024-04-05T22:20:59.074 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.073+0000 7fa8f3200640 1 -- start start 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ec0cf710 con 0x7fa8ec06a3c0 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f3200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ec0cd760 con 0x7fa8ec0af210 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0cd340 con 0x7fa8ec15aad0 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa8ec0af210 0x7fa8ec069cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55526/0 (socket says 172.21.15.157:55526) 2024-04-05T22:20:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1400640 1 -- 172.21.15.157:0/3708448360 learned_addr learned my addr 172.21.15.157:0/3708448360 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-05T22:20:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f0a00640 1 -- 172.21.15.157:0/3708448360 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990028010 0 0) 0x7fa8ec0cd760 con 0x7fa8ec0af210 2024-04-05T22:20:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1e00640 1 -- 172.21.15.157:0/3708448360 >> v1:172.21.15.157:6789/0 conn(0x7fa8ec0af210 legacy=0x7fa8ec069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1e00640 1 -- 172.21.15.157:0/3708448360 >> v1:172.21.15.73:6789/0 conn(0x7fa8ec06a3c0 legacy=0x7fa8ec15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.075+0000 7fa8f1e00640 1 -- 172.21.15.157:0/3708448360 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec0cf150 con 0x7fa8ec15aad0 2024-04-05T22:20:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f1e00640 1 --2- 172.21.15.157:0/3708448360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec06e890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00097d0 tx=0x7fa8e002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5300a97ad4ec4a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f0a00640 1 -- 172.21.15.157:0/3708448360 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e002e070 con 0x7fa8ec15aad0 2024-04-05T22:20:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f0a00640 1 -- 172.21.15.157:0/3708448360 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e00315a0 con 0x7fa8ec15aad0 2024-04-05T22:20:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f0a00640 1 -- 172.21.15.157:0/3708448360 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0031930 con 0x7fa8ec15aad0 2024-04-05T22:20:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f3200640 1 -- 172.21.15.157:0/3708448360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 msgr2=0x7fa8ec06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.077+0000 7fa8f3200640 1 --2- 172.21.15.157:0/3708448360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec06e890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00097d0 tx=0x7fa8e002b840 comp rx=0 tx=0).stop 2024-04-05T22:20:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.078+0000 7fa8f3200640 1 -- 172.21.15.157:0/3708448360 shutdown_connections 2024-04-05T22:20:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.078+0000 7fa8f3200640 1 --2- 172.21.15.157:0/3708448360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec06e890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.078+0000 7fa8f3200640 1 -- 172.21.15.157:0/3708448360 >> 172.21.15.157:0/3708448360 conn(0x7fa8ec0706c0 msgr2=0x7fa8ec070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.078+0000 7fa8f3200640 1 -- 172.21.15.157:0/3708448360 shutdown_connections 2024-04-05T22:20:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.078+0000 7fa8f3200640 1 -- 172.21.15.157:0/3708448360 wait complete. 2024-04-05T22:20:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.079+0000 7fa8f3200640 1 Processor -- start 2024-04-05T22:20:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 -- start start 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ec06a3c0 0x7fa8ec06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec06e250 con 0x7fa8ec06a3c0 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0c4a70 con 0x7fa8ec0af210 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.080+0000 7fa8f3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec06cd60 con 0x7fa8ec15aad0 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38472/0 (socket says 172.21.15.157:38472) 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 -- 172.21.15.157:0/1266551556 learned_addr learned my addr 172.21.15.157:0/1266551556 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-05T22:20:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1e00640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ec06a3c0 0x7fa8ec06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 -- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 msgr2=0x7fa8ec0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 -- 172.21.15.157:0/1266551556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ec06a3c0 msgr2=0x7fa8ec06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ec06a3c0 0x7fa8ec06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 -- 172.21.15.157:0/1266551556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009400 con 0x7fa8ec0af210 2024-04-05T22:20:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.081+0000 7fa8f1400640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c440 tx=0x7fa8dc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0074e0 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc007660 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc004a20 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec06d040 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec06d600 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa8dc01d3f0 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.082+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fa8dc01dd00 con 0x7fa8ec0af210 2024-04-05T22:20:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.083+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec04e160 con 0x7fa8ec0af210 2024-04-05T22:20:59.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.085+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7fa8dc01d6a0 con 0x7fa8ec0af210 2024-04-05T22:20:59.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:20:59.094+0000 7f1e0fa32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-05T22:20:59.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.163+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "09e1e050-1cef-45dc-9aa2-2fa40597a903", "id": 7} v 0) v1 -- 0x7fa8ec0aec80 con 0x7fa8ec0af210 2024-04-05T22:20:59.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.170+0000 7fa8e6000640 1 -- 172.21.15.157:0/1266551556 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09e1e050-1cef-45dc-9aa2-2fa40597a903", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa8dc01d910 con 0x7fa8ec0af210 2024-04-05T22:20:59.171 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 msgr2=0x7fa8ec06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c440 tx=0x7fa8dc001e70 comp rx=0 tx=0).stop 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 shutdown_connections 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ec15aad0 0x7fa8ec0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ec0af210 0x7fa8ec06dd10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 --2- 172.21.15.157:0/1266551556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ec06a3c0 0x7fa8ec06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.171+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 >> 172.21.15.157:0/1266551556 conn(0x7fa8ec0706c0 msgr2=0x7fa8ec0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:59.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.172+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 shutdown_connections 2024-04-05T22:20:59.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-05T22:20:59.172+0000 7fa8f3200640 1 -- 172.21.15.157:0/1266551556 wait complete. 2024-04-05T22:20:59.189 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-05T22:20:59.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-05T22:20:59.196 INFO:tasks.ceph.osd.0:Started 2024-04-05T22:20:59.196 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-05T22:20:59.196 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-05T22:20:59.207 INFO:tasks.ceph.osd.1:Started 2024-04-05T22:20:59.207 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-05T22:20:59.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-05T22:20:59.211 INFO:tasks.ceph.osd.2:Started 2024-04-05T22:20:59.212 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-05T22:20:59.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-05T22:20:59.217 INFO:tasks.ceph.osd.3:Started 2024-04-05T22:20:59.217 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-05T22:20:59.217 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-05T22:20:59.236 INFO:tasks.ceph.osd.4:Started 2024-04-05T22:20:59.236 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-05T22:20:59.236 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-05T22:20:59.243 INFO:tasks.ceph.osd.5:Started 2024-04-05T22:20:59.243 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-05T22:20:59.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-05T22:20:59.247 INFO:tasks.ceph.osd.6:Started 2024-04-05T22:20:59.248 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-05T22:20:59.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-05T22:20:59.253 INFO:tasks.ceph.osd.7:Started 2024-04-05T22:20:59.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:20:59.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.477+0000 7f22b6a00640 1 Processor -- start 2024-04-05T22:20:59.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.477+0000 7f22b6a00640 1 -- start start 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b6a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22b006cb80 0x7f22b006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22b015e850 0x7f22b015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b6a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b006f5b0 con 0x7f22b01587d0 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b6a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b006d500 con 0x7f22b015e850 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b6a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b015f220 con 0x7f22b006cb80 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22b015e850 0x7f22b015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22b015e850 0x7f22b015ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54396/0 (socket says 172.21.15.73:54396) 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22b4a00640 1 -- 172.21.15.73:0/767629514 learned_addr learned my addr 172.21.15.73:0/767629514 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:20:59.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.478+0000 7f22a7400640 1 --2- 172.21.15.73:0/767629514 >> v2:172.21.15.157:3300/0 conn(0x7f22b006cb80 0x7f22b006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.479+0000 7f22b4a00640 1 -- 172.21.15.73:0/767629514 >> v2:172.21.15.157:3300/0 conn(0x7f22b006cb80 msgr2=0x7f22b006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:59.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.479+0000 7f22b4a00640 1 --2- 172.21.15.73:0/767629514 >> v2:172.21.15.157:3300/0 conn(0x7f22b006cb80 0x7f22b006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.479+0000 7f22b4a00640 1 -- 172.21.15.73:0/767629514 >> v1:172.21.15.157:6789/0 conn(0x7f22b01587d0 legacy=0x7f22b00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:20:59.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.479+0000 7f22b4a00640 1 -- 172.21.15.73:0/767629514 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0070410 con 0x7f22b015e850 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22b4a00640 1 --2- 172.21.15.73:0/767629514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b015e850 0x7f22b015ece0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f22b006b560 tx=0x7f22a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b952cb42f0162c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22af400640 1 -- 172.21.15.73:0/767629514 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a802ca80 con 0x7f22b015e850 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22af400640 1 -- 172.21.15.73:0/767629514 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a802cbe0 con 0x7f22b015e850 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22af400640 1 -- 172.21.15.73:0/767629514 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80343f0 con 0x7f22b015e850 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22b6a00640 1 -- 172.21.15.73:0/767629514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b015e850 msgr2=0x7f22b015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.480+0000 7f22b6a00640 1 --2- 172.21.15.73:0/767629514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b015e850 0x7f22b015ece0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f22b006b560 tx=0x7f22a802bee0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 -- 172.21.15.73:0/767629514 shutdown_connections 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 --2- 172.21.15.73:0/767629514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b015e850 0x7f22b015ece0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 --2- 172.21.15.73:0/767629514 >> v2:172.21.15.157:3300/0 conn(0x7f22b006cb80 0x7f22b006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 -- 172.21.15.73:0/767629514 >> 172.21.15.73:0/767629514 conn(0x7f22b00ce8e0 msgr2=0x7f22b00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 -- 172.21.15.73:0/767629514 shutdown_connections 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.481+0000 7f22b6a00640 1 -- 172.21.15.73:0/767629514 wait complete. 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.482+0000 7f22b6a00640 1 Processor -- start 2024-04-05T22:20:59.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.482+0000 7f22b6a00640 1 -- start start 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b01587d0 0x7f22b01efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b015e850 0x7f22b00aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00a9e60 con 0x7f22b006cb80 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00a8dc0 con 0x7f22b01587d0 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22b6a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00a90a0 con 0x7f22b015e850 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.483+0000 7f22afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54410/0 (socket says 172.21.15.73:54410) 2024-04-05T22:20:59.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 -- 172.21.15.73:0/1066830041 learned_addr learned my addr 172.21.15.73:0/1066830041 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22a7400640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b01587d0 0x7f22b01efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22b4a00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b015e850 0x7f22b00aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 -- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b015e850 msgr2=0x7f22b00aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b015e850 0x7f22b00aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 -- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b01587d0 msgr2=0x7f22b01efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b01587d0 0x7f22b01efc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 -- 172.21.15.73:0/1066830041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8009b30 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22afe00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c430 tx=0x7f22a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011790 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a00118f0 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b00a9380 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.484+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011ce0 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.485+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b00a9940 con 0x7f22b006cb80 2024-04-05T22:20:59.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.485+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f22a001c760 con 0x7f22b006cb80 2024-04-05T22:20:59.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.485+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f22a0011e40 con 0x7f22b006cb80 2024-04-05T22:20:59.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.486+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b0066af0 con 0x7f22b006cb80 2024-04-05T22:20:59.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.488+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f22a0012740 con 0x7f22b006cb80 2024-04-05T22:20:59.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.552+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22b0068160 con 0x7f22b006cb80 2024-04-05T22:20:59.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.553+0000 7f22acc00640 1 -- 172.21.15.73:0/1066830041 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f22a00128e0 con 0x7f22b006cb80 2024-04-05T22:20:59.553 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:20:59.553 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":10,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:20:59.166718+0000","last_up_change":"0.000000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 msgr2=0x7f22b00a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c430 tx=0x7f22a000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 shutdown_connections 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b015e850 0x7f22b00aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b01587d0 0x7f22b01efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 --2- 172.21.15.73:0/1066830041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22b006cb80 0x7f22b00a84c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 >> 172.21.15.73:0/1066830041 conn(0x7f22b00ce8e0 msgr2=0x7f22b00708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:20:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 shutdown_connections 2024-04-05T22:20:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:20:59.554+0000 7f22b6a00640 1 -- 172.21.15.73:0/1066830041 wait complete. 2024-04-05T22:20:59.570 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-05T22:20:59.570 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-05T22:20:59.884 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-05T22:20:59.884+0000 7f1f8d215640 -1 Falling back to public interface 2024-04-05T22:20:59.920 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-05T22:20:59.920+0000 7f621741d640 -1 Falling back to public interface 2024-04-05T22:20:59.925 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-05T22:20:59.925+0000 7f841981c640 -1 Falling back to public interface 2024-04-05T22:20:59.928 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-05T22:20:59.928+0000 7f532498f640 -1 Falling back to public interface 2024-04-05T22:20:59.930 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-05T22:20:59.930+0000 7fb9bf593640 -1 Falling back to public interface 2024-04-05T22:20:59.954 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-05T22:20:59.954+0000 7f74af421640 -1 Falling back to public interface 2024-04-05T22:20:59.962 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-05T22:20:59.962+0000 7f8d28a11640 -1 Falling back to public interface 2024-04-05T22:20:59.966 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:20:59.966+0000 7f5588e37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-05T22:20:59.968 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-05T22:20:59.968+0000 7f385fdaa640 -1 Falling back to public interface 2024-04-05T22:21:00.215 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:21:00.215+0000 7f1e0fa32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-05T22:21:00.275 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:21:00.275+0000 7f5588e37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-05T22:21:00.333 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:21:00.333+0000 7f5588e37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-05T22:21:00.376 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:00.466 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:21:00.466+0000 7f5588e37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-05T22:21:00.529 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:21:00.529+0000 7f1e0fa32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-05T22:21:00.538 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.538+0000 7f7401200640 1 Processor -- start 2024-04-05T22:21:00.538 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.538+0000 7f7401200640 1 -- start start 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f7401200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 0x7f73fc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f7401200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc15aa60 con 0x7f73fc1541a0 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f7401200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc0b3b60 con 0x7f73fc15aef0 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f7401200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0701c0 con 0x7f73fc06d710 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f73f9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f73fc1541a0 0x7f73fc0b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35926/0 (socket says 172.21.15.73:35926) 2024-04-05T22:21:00.539 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f73f9e00640 1 -- 172.21.15.73:0/243724176 learned_addr learned my addr 172.21.15.73:0/243724176 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:00.540 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.539+0000 7f73f9400640 1 --2- 172.21.15.73:0/243724176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 0x7f73fc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:00.540 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.540+0000 7f73f8a00640 1 -- 172.21.15.73:0/243724176 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568542053 0 0) 0x7f73fc15aa60 con 0x7f73fc1541a0 2024-04-05T22:21:00.540 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.540+0000 7f73f8a00640 1 -- 172.21.15.73:0/243724176 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e0003520 con 0x7f73fc1541a0 2024-04-05T22:21:00.541 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.540+0000 7f73f9400640 1 -- 172.21.15.73:0/243724176 >> v1:172.21.15.157:6789/0 conn(0x7f73fc15aef0 legacy=0x7f73fc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.541 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.540+0000 7f73f9400640 1 -- 172.21.15.73:0/243724176 >> v1:172.21.15.73:6789/0 conn(0x7f73fc1541a0 legacy=0x7f73fc0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.541 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.540+0000 7f73f9400640 1 -- 172.21.15.73:0/243724176 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0b3880 con 0x7f73fc06d710 2024-04-05T22:21:00.542 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.542+0000 7f73f9400640 1 --2- 172.21.15.73:0/243724176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 0x7f73fc0cd8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f73e80097d0 tx=0x7f73e802db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1193e5a8c3463189 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.542+0000 7f73f8a00640 1 -- 172.21.15.73:0/243724176 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e802f900 con 0x7f73fc06d710 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.542+0000 7f73f8a00640 1 -- 172.21.15.73:0/243724176 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e802fa60 con 0x7f73fc06d710 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.542+0000 7f73f8a00640 1 -- 172.21.15.73:0/243724176 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e802fdf0 con 0x7f73fc06d710 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 -- 172.21.15.73:0/243724176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 msgr2=0x7f73fc0cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 --2- 172.21.15.73:0/243724176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 0x7f73fc0cd8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f73e80097d0 tx=0x7f73e802db40 comp rx=0 tx=0).stop 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 -- 172.21.15.73:0/243724176 shutdown_connections 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 --2- 172.21.15.73:0/243724176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc06d710 0x7f73fc0cd8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.543 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 -- 172.21.15.73:0/243724176 >> 172.21.15.73:0/243724176 conn(0x7f73fc0ce8e0 msgr2=0x7f73fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:00.544 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 -- 172.21.15.73:0/243724176 shutdown_connections 2024-04-05T22:21:00.544 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.543+0000 7f7401200640 1 -- 172.21.15.73:0/243724176 wait complete. 2024-04-05T22:21:00.545 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.544+0000 7f7401200640 1 Processor -- start 2024-04-05T22:21:00.545 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 -- start start 2024-04-05T22:21:00.547 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73fc06d710 0x7f73fc1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:00.547 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc1541a0 0x7f73fc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:00.547 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:00.547 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc156480 con 0x7f73fc15aef0 2024-04-05T22:21:00.547 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc155580 con 0x7f73fc06d710 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.545+0000 7f7401200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc155860 con 0x7f73fc1541a0 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54494/0 (socket says 172.21.15.73:54494) 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 -- 172.21.15.73:0/1525421623 learned_addr learned my addr 172.21.15.73:0/1525421623 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73f9400640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc1541a0 0x7f73fc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73f9e00640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73fc06d710 0x7f73fc1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 -- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc1541a0 msgr2=0x7f73fc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc1541a0 0x7f73fc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 -- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73fc06d710 msgr2=0x7f73fc1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.548 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73fc06d710 0x7f73fc1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.546+0000 7f73fa800640 1 -- 172.21.15.73:0/1525421623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8009400 con 0x7f73fc15aef0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f73fa800640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c6b0 tx=0x7f73f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4003b70 con 0x7f73fc15aef0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4003cf0 con 0x7f73fc15aef0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc155dc0 con 0x7f73fc15aef0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4003ea0 con 0x7f73fc15aef0 2024-04-05T22:21:00.549 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.547+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc156de0 con 0x7f73fc15aef0 2024-04-05T22:21:00.550 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.548+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc066af0 con 0x7f73fc15aef0 2024-04-05T22:21:00.550 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.548+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 2) v1 ==== 44528+0+0 (secure 0 0 0) 0x7f73f4005310 con 0x7f73fc15aef0 2024-04-05T22:21:00.550 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.548+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f73f40491f0 con 0x7f73fc15aef0 2024-04-05T22:21:00.550 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.550+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72706 (secure 0 0 0) 0x7f73f40106f0 con 0x7f73fc15aef0 2024-04-05T22:21:00.594 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:21:00.593+0000 7f1e0fa32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-05T22:21:00.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.617+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73fc1565c0 con 0x7f73fc15aef0 2024-04-05T22:21:00.618 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.618+0000 7f73f2000640 1 -- 172.21.15.73:0/1525421623 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f73fc1565c0 con 0x7f73fc15aef0 2024-04-05T22:21:00.618 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-05T22:21:00.618 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":10,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:20:59.166718+0000","last_up_change":"0.000000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:00.619 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 msgr2=0x7f73fc0ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c6b0 tx=0x7f73f400cb70 comp rx=0 tx=0).stop 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 shutdown_connections 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73fc15aef0 0x7f73fc0ccd20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73fc1541a0 0x7f73fc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 --2- 172.21.15.73:0/1525421623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73fc06d710 0x7f73fc1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 >> 172.21.15.73:0/1525421623 conn(0x7f73fc0ce8e0 msgr2=0x7f73fc0b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 shutdown_connections 2024-04-05T22:21:00.620 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:00.619+0000 7f7401200640 1 -- 172.21.15.73:0/1525421623 wait complete. 2024-04-05T22:21:00.632 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-05T22:21:00.723 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:21:00.723+0000 7f1e0fa32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-05T22:21:01.510 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:21:01.510+0000 7f5529600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-05T22:21:01.510 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:21:01.510+0000 7f5529600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-05T22:21:03.406 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-05T22:21:03.406+0000 7f1f8d215640 -1 osd.0 0 log_to_monitors true 2024-04-05T22:21:03.434 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-05T22:21:03.434+0000 7f621741d640 -1 osd.4 0 log_to_monitors true 2024-04-05T22:21:03.454 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-05T22:21:03.454+0000 7fb9bf593640 -1 osd.2 0 log_to_monitors true 2024-04-05T22:21:03.463 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-05T22:21:03.463+0000 7f841981c640 -1 osd.1 0 log_to_monitors true 2024-04-05T22:21:03.488 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-05T22:21:03.488+0000 7f74af421640 -1 osd.3 0 log_to_monitors true 2024-04-05T22:21:03.532 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-05T22:21:03.533+0000 7f8d28a11640 -1 osd.7 0 log_to_monitors true 2024-04-05T22:21:03.548 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-05T22:21:03.548+0000 7f385fdaa640 -1 osd.5 0 log_to_monitors true 2024-04-05T22:21:03.765 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-05T22:21:03.765+0000 7f532498f640 -1 osd.6 0 log_to_monitors true 2024-04-05T22:21:06.082 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-05T22:21:06.082+0000 7f3859e00640 -1 osd.5 0 waiting for initial osdmap 2024-04-05T22:21:06.088 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-05T22:21:06.089+0000 7f8d24400640 -1 osd.7 0 waiting for initial osdmap 2024-04-05T22:21:06.093 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-05T22:21:06.093+0000 7f8d19000640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:06.095 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-05T22:21:06.096+0000 7f3850800640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:06.107 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-05T22:21:06.108+0000 7f6211000640 -1 osd.4 0 waiting for initial osdmap 2024-04-05T22:21:06.117 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-05T22:21:06.118+0000 7f6207a00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:06.868 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-05T22:21:06.868+0000 7f531ea00640 -1 osd.6 0 waiting for initial osdmap 2024-04-05T22:21:06.883 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-05T22:21:06.884+0000 7f5315400640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:06.965 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-05T22:21:06.965+0000 7f1f88c00640 -1 osd.0 0 waiting for initial osdmap 2024-04-05T22:21:06.973 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-05T22:21:06.973+0000 7f1f7d800640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:06.993 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-05T22:21:06.993+0000 7f74aae00640 -1 osd.3 0 waiting for initial osdmap 2024-04-05T22:21:06.998 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-05T22:21:06.999+0000 7fb9b9600640 -1 osd.2 0 waiting for initial osdmap 2024-04-05T22:21:06.999 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-05T22:21:06.999+0000 7f749fa00640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:07.009 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-05T22:21:07.009+0000 7fb9b0000640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:07.020 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-05T22:21:07.020+0000 7f8413400640 -1 osd.1 0 waiting for initial osdmap 2024-04-05T22:21:07.026 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-05T22:21:07.026+0000 7f8409e00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-05T22:21:07.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:21:07.342 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:21:07.440 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:07.611 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.611+0000 7fab0de00640 1 Processor -- start 2024-04-05T22:21:07.611 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.611+0000 7fab0de00640 1 -- start start 2024-04-05T22:21:07.612 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.612+0000 7fab0de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 0x7fab08070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.612+0000 7fab0de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.612+0000 7fab0de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0815a830 con 0x7fab08159440 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.612+0000 7fab0de00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab080b2280 con 0x7fab080b3440 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.612+0000 7fab0de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab080b3a20 con 0x7fab08071340 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.613+0000 7fab07400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:07.613 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.613+0000 7fab07400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:49408/0 (socket says 172.21.15.73:49408) 2024-04-05T22:21:07.614 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.613+0000 7fab07400640 1 -- 172.21.15.73:0/2171472326 learned_addr learned my addr 172.21.15.73:0/2171472326 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:07.614 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.613+0000 7faaffe00640 1 --2- 172.21.15.73:0/2171472326 >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 0x7fab08070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:07.614 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab06000640 1 -- 172.21.15.73:0/2171472326 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14418787 0 0) 0x7faaf8001110 con 0x7fab08159440 2024-04-05T22:21:07.614 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab06000640 1 -- 172.21.15.73:0/2171472326 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaec003520 con 0x7fab08159440 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab07400640 1 -- 172.21.15.73:0/2171472326 >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 msgr2=0x7fab08070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab07400640 1 --2- 172.21.15.73:0/2171472326 >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 0x7fab08070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab07400640 1 -- 172.21.15.73:0/2171472326 >> v1:172.21.15.157:6789/0 conn(0x7fab08159440 legacy=0x7fab0806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7fab07400640 1 -- 172.21.15.73:0/2171472326 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080b2520 con 0x7fab08071340 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.614+0000 7faaffe00640 1 --2- 172.21.15.73:0/2171472326 >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 0x7fab08070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:07.615 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.615+0000 7fab07400640 1 --2- 172.21.15.73:0/2171472326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab00009e50 tx=0x7fab0002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3650a0c8f82cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:07.616 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab06000640 1 -- 172.21.15.73:0/2171472326 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0002caf0 con 0x7fab08071340 2024-04-05T22:21:07.616 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab06000640 1 -- 172.21.15.73:0/2171472326 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0002cc50 con 0x7fab08071340 2024-04-05T22:21:07.616 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab06000640 1 -- 172.21.15.73:0/2171472326 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000343f0 con 0x7fab08071340 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 -- 172.21.15.73:0/2171472326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 msgr2=0x7fab080ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 --2- 172.21.15.73:0/2171472326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab00009e50 tx=0x7fab0002bf50 comp rx=0 tx=0).stop 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 -- 172.21.15.73:0/2171472326 shutdown_connections 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 --2- 172.21.15.73:0/2171472326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab080ad510 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 --2- 172.21.15.73:0/2171472326 >> v2:172.21.15.157:3300/0 conn(0x7fab080b3440 0x7fab08070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.616+0000 7fab0de00640 1 -- 172.21.15.73:0/2171472326 >> 172.21.15.73:0/2171472326 conn(0x7fab080ce8e0 msgr2=0x7fab080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.617+0000 7fab0de00640 1 -- 172.21.15.73:0/2171472326 shutdown_connections 2024-04-05T22:21:07.617 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.617+0000 7fab0de00640 1 -- 172.21.15.73:0/2171472326 wait complete. 2024-04-05T22:21:07.618 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.618+0000 7fab0de00640 1 Processor -- start 2024-04-05T22:21:07.618 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 -- start start 2024-04-05T22:21:07.621 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab0814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.621 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab080b3440 0x7fab081e2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.621 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.621 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081f2e50 con 0x7fab08159440 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081f3130 con 0x7fab080b3440 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.619+0000 7fab0de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab081e1c20 con 0x7fab08071340 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50758/0 (socket says 172.21.15.73:50758) 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 -- 172.21.15.73:0/2701560765 learned_addr learned my addr 172.21.15.73:0/2701560765 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 -- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 msgr2=0x7fab0814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab0814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 -- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab080b3440 msgr2=0x7fab081e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7faaffe00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab080b3440 0x7fab081e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab080b3440 0x7fab081e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 -- 172.21.15.73:0/2701560765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab00009b30 con 0x7fab08159440 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.620+0000 7fab07400640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fab00036040 tx=0x7fab00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:07.622 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.621+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00009510 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.621+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00004250 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.621+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab081e1f00 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.621+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000045c0 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.621+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab081e2960 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.622+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab08066af0 con 0x7fab08159440 2024-04-05T22:21:07.623 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.623+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fab00004720 con 0x7fab08159440 2024-04-05T22:21:07.625 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.624+0000 7faaff400640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 0x7faae406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:07.625 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.625+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fab000aba40 con 0x7fab08159440 2024-04-05T22:21:07.625 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.625+0000 7fab06a00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 0x7faae406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:07.625 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.625+0000 7fab06a00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 0x7faae406d030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001270 tx=0x7faaf80077e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:07.631 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.631+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fab000772e0 con 0x7fab08159440 2024-04-05T22:21:07.813 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.813+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab080cf780 con 0x7fab08159440 2024-04-05T22:21:07.815 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.814+0000 7faaff400640 1 -- 172.21.15.73:0/2701560765 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10440 (secure 0 0 0) 0x7fab0007a900 con 0x7fab08159440 2024-04-05T22:21:07.815 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-05T22:21:07.815 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":14,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:07.094714+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 msgr2=0x7faae406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 0x7faae406d030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001270 tx=0x7faaf80077e0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 msgr2=0x7fab081f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fab00036040 tx=0x7fab00004a40 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 shutdown_connections 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faae406ab70 0x7faae406d030 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab08159440 0x7fab081f2330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab080b3440 0x7fab081e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 --2- 172.21.15.73:0/2701560765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab08071340 0x7fab0814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 >> 172.21.15.73:0/2701560765 conn(0x7fab080ce8e0 msgr2=0x7fab0815c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 shutdown_connections 2024-04-05T22:21:07.818 INFO:teuthology.misc.health.smithi073.stderr:2024-04-05T22:21:07.817+0000 7fab0de00640 1 -- 172.21.15.73:0/2701560765 wait complete. 2024-04-05T22:21:07.830 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-05T22:21:07.831 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-05T22:21:07.831 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:21:07.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:21:07.836 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:21:07.836 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:21:07.836 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:21:07.841 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:21:07.841 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:21:07.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:21:07.849 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:21:07.849 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:21:07.849 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:21:07.855 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:21:07.856 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-05T22:21:07.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:07.982 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:21:07.986 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:21:08.016 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:21:08.017 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:21:08.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.035+0000 7f09c3600640 1 Processor -- start 2024-04-05T22:21:08.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.035+0000 7f09c3600640 1 -- start start 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c3600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f09bc1541a0 0x7f09bc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c3600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc15aa60 con 0x7f09bc15aef0 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c3600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc0b3b60 con 0x7f09bc1541a0 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0701c0 con 0x7f09bc06d710 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f09bc1541a0 0x7f09bc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f09bc1541a0 0x7f09bc0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50792/0 (socket says 172.21.15.73:50792) 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09c0c00640 1 -- 172.21.15.73:0/3300209021 learned_addr learned my addr 172.21.15.73:0/3300209021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.036+0000 7f09bbe00640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.037+0000 7f09c0c00640 1 -- 172.21.15.73:0/3300209021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 msgr2=0x7f09bc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.037+0000 7f09c0c00640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.037+0000 7f09c0c00640 1 -- 172.21.15.73:0/3300209021 >> v1:172.21.15.157:6789/0 conn(0x7f09bc15aef0 legacy=0x7f09bc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.037+0000 7f09c0c00640 1 -- 172.21.15.73:0/3300209021 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0b3880 con 0x7f09bc1541a0 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c0c00640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc1541a0 0x7f09bc0b3050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b002db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74705ae2a6fdeca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09bb400640 1 -- 172.21.15.73:0/3300209021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b002e940 con 0x7f09bc1541a0 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09bb400640 1 -- 172.21.15.73:0/3300209021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b002eaa0 con 0x7f09bc1541a0 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09bb400640 1 -- 172.21.15.73:0/3300209021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b002ee30 con 0x7f09bc1541a0 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 -- 172.21.15.73:0/3300209021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc1541a0 msgr2=0x7f09bc0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc1541a0 0x7f09bc0b3050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b002db90 comp rx=0 tx=0).stop 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 -- 172.21.15.73:0/3300209021 shutdown_connections 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 --2- 172.21.15.73:0/3300209021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc1541a0 0x7f09bc0b3050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 -- 172.21.15.73:0/3300209021 >> 172.21.15.73:0/3300209021 conn(0x7f09bc0ce8e0 msgr2=0x7f09bc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 -- 172.21.15.73:0/3300209021 shutdown_connections 2024-04-05T22:21:08.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.038+0000 7f09c3600640 1 -- 172.21.15.73:0/3300209021 wait complete. 2024-04-05T22:21:08.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.040+0000 7f09c3600640 1 Processor -- start 2024-04-05T22:21:08.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.040+0000 7f09c3600640 1 -- start start 2024-04-05T22:21:08.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc1541a0 0x7f09bc0ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc144f00 con 0x7f09bc15aef0 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc06bde0 con 0x7f09bc1541a0 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc06c050 con 0x7f09bc06d710 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09bbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc1541a0 0x7f09bc0ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50802/0 (socket says 172.21.15.73:50802) 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c1600640 1 -- 172.21.15.73:0/1243001332 learned_addr learned my addr 172.21.15.73:0/1243001332 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.041+0000 7f09c0c00640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 -- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 msgr2=0x7f09bc0acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 -- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc1541a0 msgr2=0x7f09bc0ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc1541a0 0x7f09bc0ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 -- 172.21.15.73:0/1243001332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00094d0 con 0x7f09bc15aef0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c0c00640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c1600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c6e0 tx=0x7f09b400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4003b70 con 0x7f09bc15aef0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4003cf0 con 0x7f09bc15aef0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005310 con 0x7f09bc15aef0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc06c330 con 0x7f09bc15aef0 2024-04-05T22:21:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.042+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc0a8aa0 con 0x7f09bc15aef0 2024-04-05T22:21:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.043+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc066af0 con 0x7f09bc15aef0 2024-04-05T22:21:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.044+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f09b4005470 con 0x7f09bc15aef0 2024-04-05T22:21:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.045+0000 7f09b8c00640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 0x7f099c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.045+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f09b4088ad0 con 0x7f09bc15aef0 2024-04-05T22:21:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.045+0000 7f09c0c00640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 0x7f099c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.046+0000 7f09c0c00640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 0x7f099c06d030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09b0038000 tx=0x7f09b0032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.050+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f09b40541c0 con 0x7f09bc15aef0 2024-04-05T22:21:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.232+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09bc068160 con 0x7f09bc15aef0 2024-04-05T22:21:08.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.234+0000 7f09b8c00640 1 -- 172.21.15.73:0/1243001332 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10624 (secure 0 0 0) 0x7f09b4003e60 con 0x7f09bc15aef0 2024-04-05T22:21:08.234 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:08.234 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:08.099123+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 msgr2=0x7f099c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 0x7f099c06d030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09b0038000 tx=0x7f09b0032000 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 msgr2=0x7f09bc1ed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c6e0 tx=0x7f09b400cba0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 shutdown_connections 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f099c06ab70 0x7f099c06d030 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09bc15aef0 0x7f09bc1ed650 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc1541a0 0x7f09bc0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 --2- 172.21.15.73:0/1243001332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc06d710 0x7f09bc0acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 >> 172.21.15.73:0/1243001332 conn(0x7f09bc0ce8e0 msgr2=0x7f09bc152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 shutdown_connections 2024-04-05T22:21:08.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.236+0000 7f09c3600640 1 -- 172.21.15.73:0/1243001332 wait complete. 2024-04-05T22:21:08.254 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-05T22:21:08.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:08.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.448+0000 7fdf8fc00640 1 Processor -- start 2024-04-05T22:21:08.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.448+0000 7fdf8fc00640 1 -- start start 2024-04-05T22:21:08.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.450+0000 7fdf8fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 0x7fdf8806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.450+0000 7fdf8fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf880d09b0 0x7fdf880d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.450+0000 7fdf8fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf880cb950 con 0x7fdf880cd090 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.450+0000 7fdf8fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880ce990 con 0x7fdf880d09b0 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.450+0000 7fdf8fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880d1350 con 0x7fdf8815cf50 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf880d09b0 0x7fdf880d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf8dc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdf880cd090 0x7fdf880cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51822/0 (socket says 172.21.15.73:51822) 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf8dc00640 1 -- 172.21.15.73:0/1058296229 learned_addr learned my addr 172.21.15.73:0/1058296229 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf8d200640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 0x7fdf8806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87400640 1 -- 172.21.15.73:0/1058296229 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688387115 0 0) 0x7fdf880cb950 con 0x7fdf880cd090 2024-04-05T22:21:08.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87e00640 1 -- 172.21.15.73:0/1058296229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 msgr2=0x7fdf8806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:08.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87e00640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 0x7fdf8806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87e00640 1 -- 172.21.15.73:0/1058296229 >> v1:172.21.15.73:6789/0 conn(0x7fdf880cd090 legacy=0x7fdf880cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf87e00640 1 -- 172.21.15.73:0/1058296229 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880cdce0 con 0x7fdf880d09b0 2024-04-05T22:21:08.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.451+0000 7fdf8d200640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 0x7fdf8806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.452+0000 7fdf87e00640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf880d09b0 0x7fdf880d0e10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097a0 tx=0x7fdf7802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac71d8c21aed3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.452+0000 7fdf87400640 1 -- 172.21.15.73:0/1058296229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7802d040 con 0x7fdf880d09b0 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf87400640 1 -- 172.21.15.73:0/1058296229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78031810 con 0x7fdf880d09b0 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf87400640 1 -- 172.21.15.73:0/1058296229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78031ba0 con 0x7fdf880d09b0 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 -- 172.21.15.73:0/1058296229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf880d09b0 msgr2=0x7fdf880d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf880d09b0 0x7fdf880d0e10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097a0 tx=0x7fdf7802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 -- 172.21.15.73:0/1058296229 shutdown_connections 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf880d09b0 0x7fdf880d0e10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/1058296229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf8815cf50 0x7fdf8806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 -- 172.21.15.73:0/1058296229 >> 172.21.15.73:0/1058296229 conn(0x7fdf88099a60 msgr2=0x7fdf88099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 -- 172.21.15.73:0/1058296229 shutdown_connections 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.453+0000 7fdf8fc00640 1 -- 172.21.15.73:0/1058296229 wait complete. 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.455+0000 7fdf8fc00640 1 Processor -- start 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.455+0000 7fdf8fc00640 1 -- start start 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 0x7fdf881516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880d09b0 0x7fdf880b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b3520 con 0x7fdf8815cf50 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b2c80 con 0x7fdf880d09b0 2024-04-05T22:21:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b2f60 con 0x7fdf880cd090 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50826/0 (socket says 172.21.15.73:50826) 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8dc00640 1 -- 172.21.15.73:0/576523091 learned_addr learned my addr 172.21.15.73:0/576523091 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.456+0000 7fdf8d200640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 0x7fdf881516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf87e00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880d09b0 0x7fdf880b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 -- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 msgr2=0x7fdf881516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 0x7fdf881516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 -- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880d09b0 msgr2=0x7fdf880b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880d09b0 0x7fdf880b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 -- 172.21.15.73:0/576523091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009400 con 0x7fdf8815cf50 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8d200640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 0x7fdf881516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8dc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000c7e0 tx=0x7fdf8000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000a640 con 0x7fdf8815cf50 2024-04-05T22:21:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8000a7c0 con 0x7fdf8815cf50 2024-04-05T22:21:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000ab70 con 0x7fdf8815cf50 2024-04-05T22:21:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf880b3240 con 0x7fdf8815cf50 2024-04-05T22:21:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.457+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf881f2ec0 con 0x7fdf8815cf50 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.458+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8806d550 con 0x7fdf8815cf50 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.459+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fdf8000acd0 con 0x7fdf8815cf50 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.460+0000 7fdf84c00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 0x7fdf6406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.460+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fdf800881d0 con 0x7fdf8815cf50 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.461+0000 7fdf8d200640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 0x7fdf6406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.461+0000 7fdf8d200640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 0x7fdf6406d030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf70006fd0 tx=0x7fdf70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.465+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdf80053950 con 0x7fdf8815cf50 2024-04-05T22:21:08.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.685+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf881f2ec0 con 0x7fdf8815cf50 2024-04-05T22:21:08.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.686+0000 7fdf84c00640 1 -- 172.21.15.73:0/576523091 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7fdf881f2ec0 con 0x7fdf8815cf50 2024-04-05T22:21:08.686 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:08.686 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-04-05T22:21:08.687 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 2 2024-04-05T22:21:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.688+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 msgr2=0x7fdf6406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.688+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 0x7fdf6406d030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf70006fd0 tx=0x7fdf70008040 comp rx=0 tx=0).stop 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.688+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 msgr2=0x7fdf881f27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.688+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000c7e0 tx=0x7fdf8000cca0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 shutdown_connections 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf6406ab70 0x7fdf6406d030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8815cf50 0x7fdf881f27a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880d09b0 0x7fdf880b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 --2- 172.21.15.73:0/576523091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf880cd090 0x7fdf881516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 >> 172.21.15.73:0/576523091 conn(0x7fdf88099a60 msgr2=0x7fdf880cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 shutdown_connections 2024-04-05T22:21:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:08.689+0000 7fdf8fc00640 1 -- 172.21.15.73:0/576523091 wait complete. 2024-04-05T22:21:10.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:10.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.893+0000 7f87af000640 1 Processor -- start 2024-04-05T22:21:10.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.893+0000 7f87af000640 1 -- start start 2024-04-05T22:21:10.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87af000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87a80d1930 0x7f87a80d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87af000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 0x7f87a80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87af000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80ceee0 con 0x7f87a815cf70 2024-04-05T22:21:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87af000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80ce5b0 con 0x7f87a80d1930 2024-04-05T22:21:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87af000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80ce890 con 0x7f87a80d0bd0 2024-04-05T22:21:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87a7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87a80d1930 0x7f87a80d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87a7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87a80d1930 0x7f87a80d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50864/0 (socket says 172.21.15.73:50864) 2024-04-05T22:21:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.895+0000 7f87a7400640 1 -- 172.21.15.73:0/2075893427 learned_addr learned my addr 172.21.15.73:0/2075893427 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87ad000640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 0x7f87a80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87a7400640 1 -- 172.21.15.73:0/2075893427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 msgr2=0x7f87a80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87a7400640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 0x7f87a80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87a7400640 1 -- 172.21.15.73:0/2075893427 >> v1:172.21.15.157:6789/0 conn(0x7f87a815cf70 legacy=0x7f87a80d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87a7400640 1 -- 172.21.15.73:0/2075893427 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80ceb70 con 0x7f87a80d1930 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.896+0000 7f87ad000640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 0x7f87a80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.897+0000 7f87a7400640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d1930 0x7f87a80d3910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8794009870 tx=0x7f879402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b6775f51d61f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.897+0000 7f87a6a00640 1 -- 172.21.15.73:0/2075893427 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794031640 con 0x7f87a80d1930 2024-04-05T22:21:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.897+0000 7f87a6a00640 1 -- 172.21.15.73:0/2075893427 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87940317a0 con 0x7f87a80d1930 2024-04-05T22:21:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.897+0000 7f87a6a00640 1 -- 172.21.15.73:0/2075893427 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794031b30 con 0x7f87a80d1930 2024-04-05T22:21:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 -- 172.21.15.73:0/2075893427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d1930 msgr2=0x7f87a80d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d1930 0x7f87a80d3910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8794009870 tx=0x7f879402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 -- 172.21.15.73:0/2075893427 shutdown_connections 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a80d0bd0 0x7f87a80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 --2- 172.21.15.73:0/2075893427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d1930 0x7f87a80d3910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 -- 172.21.15.73:0/2075893427 >> 172.21.15.73:0/2075893427 conn(0x7f87a80d2df0 msgr2=0x7f87a80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 -- 172.21.15.73:0/2075893427 shutdown_connections 2024-04-05T22:21:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.898+0000 7f87af000640 1 -- 172.21.15.73:0/2075893427 wait complete. 2024-04-05T22:21:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.900+0000 7f87af000640 1 Processor -- start 2024-04-05T22:21:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.900+0000 7f87af000640 1 -- start start 2024-04-05T22:21:10.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a80d1930 0x7f87a80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a815cf70 0x7f87a81f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a814cb70 con 0x7f87a80d0bd0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80b5a10 con 0x7f87a80d1930 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87af000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80b8090 con 0x7f87a815cf70 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50870/0 (socket says 172.21.15.73:50870) 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 -- 172.21.15.73:0/3878633047 learned_addr learned my addr 172.21.15.73:0/3878633047 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 -- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a815cf70 msgr2=0x7f87a81f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87ad000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a815cf70 0x7f87a81f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a815cf70 0x7f87a81f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7e00640 1 -- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a80d1930 msgr2=0x7f87a80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.901+0000 7f87a7400640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a80d1930 0x7f87a80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f87a7e00640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a80d1930 0x7f87a80cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f87a7e00640 1 -- 172.21.15.73:0/3878633047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940094d0 con 0x7f87a80d0bd0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f87a7e00640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f879800c430 tx=0x7f879800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980052d0 con 0x7f87a80d0bd0 2024-04-05T22:21:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798005430 con 0x7f87a80d0bd0 2024-04-05T22:21:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798003830 con 0x7f87a80d0bd0 2024-04-05T22:21:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a80b5f40 con 0x7f87a80d0bd0 2024-04-05T22:21:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.902+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80b8610 con 0x7f87a80d0bd0 2024-04-05T22:21:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.903+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a8068160 con 0x7f87a80d0bd0 2024-04-05T22:21:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.904+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f8798003990 con 0x7f87a80d0bd0 2024-04-05T22:21:10.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.905+0000 7f879fe00640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 0x7f878806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:10.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.905+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8798088540 con 0x7f87a80d0bd0 2024-04-05T22:21:10.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.905+0000 7f87a7400640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 0x7f878806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:10.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.906+0000 7f87a7400640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 0x7f878806d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8794036000 tx=0x7f879403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:10.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:10.910+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8798053ab0 con 0x7f87a80d0bd0 2024-04-05T22:21:11.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.152+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87a80cff60 con 0x7f87a80d0bd0 2024-04-05T22:21:11.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.153+0000 7f879fe00640 1 -- 172.21.15.73:0/3878633047 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f8798017090 con 0x7f87a80d0bd0 2024-04-05T22:21:11.154 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:11.154 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-04-05T22:21:11.154 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 3 2024-04-05T22:21:11.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 msgr2=0x7f878806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 0x7f878806d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8794036000 tx=0x7f879403c040 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 msgr2=0x7f87a80c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f879800c430 tx=0x7f879800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 shutdown_connections 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f878806ab70 0x7f878806d030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a815cf70 0x7f87a81f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a80d1930 0x7f87a80cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 --2- 172.21.15.73:0/3878633047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80d0bd0 0x7f87a80c9200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 >> 172.21.15.73:0/3878633047 conn(0x7f87a80d2df0 msgr2=0x7f87a815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 shutdown_connections 2024-04-05T22:21:11.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.156+0000 7f87af000640 1 -- 172.21.15.73:0/3878633047 wait complete. 2024-04-05T22:21:11.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:11.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.350+0000 7f5397e00640 1 Processor -- start 2024-04-05T22:21:11.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.350+0000 7f5397e00640 1 -- start start 2024-04-05T22:21:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f539806d460 0x7f53981607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f539806b100 0x7f539815d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539815dea0 0x7f5398069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539815d230 con 0x7f539806b100 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5398160cf0 con 0x7f539806d460 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.352+0000 7f5397e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5398160f90 con 0x7f539815dea0 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f539806b100 0x7f539815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f539806b100 0x7f539815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50880/0 (socket says 172.21.15.73:50880) 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 -- 172.21.15.73:0/1591480233 learned_addr learned my addr 172.21.15.73:0/1591480233 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5397400640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539815dea0 0x7f5398069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 -- 172.21.15.73:0/1591480233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539815dea0 msgr2=0x7f5398069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539815dea0 0x7f5398069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 -- 172.21.15.73:0/1591480233 >> v2:172.21.15.157:3300/0 conn(0x7f539806d460 msgr2=0x7f53981607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396a00640 1 --2- 172.21.15.73:0/1591480233 >> v2:172.21.15.157:3300/0 conn(0x7f539806d460 0x7f53981607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 --2- 172.21.15.73:0/1591480233 >> v2:172.21.15.157:3300/0 conn(0x7f539806d460 0x7f53981607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.353+0000 7f5396000640 1 -- 172.21.15.73:0/1591480233 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398161270 con 0x7f539806b100 2024-04-05T22:21:11.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.354+0000 7f5396000640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806b100 0x7f539815d960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f53980d0090 tx=0x7f538402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3081c20debcff71a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5395600640 1 -- 172.21.15.73:0/1591480233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384031870 con 0x7f539806b100 2024-04-05T22:21:11.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5395600640 1 -- 172.21.15.73:0/1591480233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53840319d0 con 0x7f539806b100 2024-04-05T22:21:11.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5395600640 1 -- 172.21.15.73:0/1591480233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384031d60 con 0x7f539806b100 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 -- 172.21.15.73:0/1591480233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806b100 msgr2=0x7f539815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806b100 0x7f539815d960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f53980d0090 tx=0x7f538402b910 comp rx=0 tx=0).stop 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 -- 172.21.15.73:0/1591480233 shutdown_connections 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806b100 0x7f539815d960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 --2- 172.21.15.73:0/1591480233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539815dea0 0x7f5398069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 --2- 172.21.15.73:0/1591480233 >> v2:172.21.15.157:3300/0 conn(0x7f539806d460 0x7f53981607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.355+0000 7f5397e00640 1 -- 172.21.15.73:0/1591480233 >> 172.21.15.73:0/1591480233 conn(0x7f53980b0c00 msgr2=0x7f53980b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:11.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.356+0000 7f5397e00640 1 -- 172.21.15.73:0/1591480233 shutdown_connections 2024-04-05T22:21:11.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.356+0000 7f5397e00640 1 -- 172.21.15.73:0/1591480233 wait complete. 2024-04-05T22:21:11.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.357+0000 7f5397e00640 1 Processor -- start 2024-04-05T22:21:11.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.357+0000 7f5397e00640 1 -- start start 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539806b100 0x7f539815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539815dea0 0x7f539815c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539809cb40 con 0x7f539806d460 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53981599d0 con 0x7f539815dea0 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5397e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53981f2a30 con 0x7f539806b100 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5396000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5396000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50892/0 (socket says 172.21.15.73:50892) 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.358+0000 7f5396000640 1 -- 172.21.15.73:0/1528537874 learned_addr learned my addr 172.21.15.73:0/1528537874 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 -- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539806b100 msgr2=0x7f539815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539806b100 0x7f539815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 -- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539815dea0 msgr2=0x7f539815c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539815dea0 0x7f539815c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 -- 172.21.15.73:0/1528537874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840094d0 con 0x7f539806d460 2024-04-05T22:21:11.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5396000640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f53840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384031860 con 0x7f539806d460 2024-04-05T22:21:11.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53840443f0 con 0x7f539806d460 2024-04-05T22:21:11.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840447a0 con 0x7f539806d460 2024-04-05T22:21:11.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53981f2c90 con 0x7f539806d460 2024-04-05T22:21:11.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.359+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53981f3250 con 0x7f539806d460 2024-04-05T22:21:11.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.360+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539804e160 con 0x7f539806d460 2024-04-05T22:21:11.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.362+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f5384044900 con 0x7f539806d460 2024-04-05T22:21:11.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.363+0000 7f538ea00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 0x7f537806d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.363+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f538402d040 con 0x7f539806d460 2024-04-05T22:21:11.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.363+0000 7f5396a00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 0x7f537806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.364+0000 7f5396a00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 0x7f537806d060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5388006fd0 tx=0x7f5388008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.368+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5384076cf0 con 0x7f539806d460 2024-04-05T22:21:11.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.564+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f539806aee0 con 0x7f539806d460 2024-04-05T22:21:11.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.565+0000 7f538ea00640 1 -- 172.21.15.73:0/1528537874 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f538407a310 con 0x7f539806d460 2024-04-05T22:21:11.566 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:11.566 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":18,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:11.116689+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 msgr2=0x7f537806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 0x7f537806d060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5388006fd0 tx=0x7f5388008040 comp rx=0 tx=0).stop 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 msgr2=0x7f53981f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f53840046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 shutdown_connections 2024-04-05T22:21:11.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f537806aba0 0x7f537806d060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539815dea0 0x7f539815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f539806d460 0x7f53981f24f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 --2- 172.21.15.73:0/1528537874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f539806b100 0x7f539815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 >> 172.21.15.73:0/1528537874 conn(0x7f53980b0c00 msgr2=0x7f5398155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 shutdown_connections 2024-04-05T22:21:11.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.568+0000 7f5397e00640 1 -- 172.21.15.73:0/1528537874 wait complete. 2024-04-05T22:21:11.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:11.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:11.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.768+0000 7f4b6c800640 1 Processor -- start 2024-04-05T22:21:11.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.768+0000 7f4b6c800640 1 -- start start 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6c800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 0x7f4b6406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6c800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b6406ba70 0x7f4b64164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6c800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6406bea0 con 0x7f4b6415abe0 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6c800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6415afc0 con 0x7f4b6406ba70 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6c800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6406f5b0 con 0x7f4b6406c9a0 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b6406ba70 0x7f4b64164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b6406ba70 0x7f4b64164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50904/0 (socket says 172.21.15.73:50904) 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 -- 172.21.15.73:0/1941831311 learned_addr learned my addr 172.21.15.73:0/1941831311 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b69400640 1 --2- 172.21.15.73:0/1941831311 >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 0x7f4b6406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 -- 172.21.15.73:0/1941831311 >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 msgr2=0x7f4b6406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:11.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 --2- 172.21.15.73:0/1941831311 >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 0x7f4b6406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 -- 172.21.15.73:0/1941831311 >> v1:172.21.15.73:6789/0 conn(0x7f4b6415abe0 legacy=0x7f4b640712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b6a800640 1 -- 172.21.15.73:0/1941831311 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6406f890 con 0x7f4b6406ba70 2024-04-05T22:21:11.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.770+0000 7f4b69400640 1 --2- 172.21.15.73:0/1941831311 >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 0x7f4b6406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.771+0000 7f4b6a800640 1 --2- 172.21.15.73:0/1941831311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b64164740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b6406ce30 tx=0x7f4b6002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd4ceae7b48d23b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b68a00640 1 -- 172.21.15.73:0/1941831311 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6002c9f0 con 0x7f4b6406ba70 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b68a00640 1 -- 172.21.15.73:0/1941831311 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6002cb50 con 0x7f4b6406ba70 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b68a00640 1 -- 172.21.15.73:0/1941831311 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600343f0 con 0x7f4b6406ba70 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 -- 172.21.15.73:0/1941831311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 msgr2=0x7f4b64164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1941831311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b64164740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b6406ce30 tx=0x7f4b6002be50 comp rx=0 tx=0).stop 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 -- 172.21.15.73:0/1941831311 shutdown_connections 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1941831311 >> v2:172.21.15.157:3300/0 conn(0x7f4b6406c9a0 0x7f4b6406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1941831311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b64164740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 -- 172.21.15.73:0/1941831311 >> 172.21.15.73:0/1941831311 conn(0x7f4b64069cb0 msgr2=0x7f4b6406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:11.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 -- 172.21.15.73:0/1941831311 shutdown_connections 2024-04-05T22:21:11.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.772+0000 7f4b6c800640 1 -- 172.21.15.73:0/1941831311 wait complete. 2024-04-05T22:21:11.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.774+0000 7f4b6c800640 1 Processor -- start 2024-04-05T22:21:11.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.774+0000 7f4b6c800640 1 -- start start 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 0x7f4b6415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415abe0 0x7f4b6415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641f2900 con 0x7f4b6406ba70 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415dff0 con 0x7f4b6415abe0 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b6c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640b1af0 con 0x7f4b6406c9a0 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b69e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b69e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50906/0 (socket says 172.21.15.73:50906) 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.775+0000 7f4b69e00640 1 -- 172.21.15.73:0/1901270635 learned_addr learned my addr 172.21.15.73:0/1901270635 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69400640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 0x7f4b6415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b6a800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415abe0 0x7f4b6415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 -- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 msgr2=0x7f4b6415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 0x7f4b6415d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 -- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415abe0 msgr2=0x7f4b6415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415abe0 0x7f4b6415dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 -- 172.21.15.73:0/1901270635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b60009b30 con 0x7f4b6406ba70 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69400640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 0x7f4b6415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b69e00640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c4d0 tx=0x7f4b5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b580052d0 con 0x7f4b6406ba70 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58005430 con 0x7f4b6406ba70 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58012850 con 0x7f4b6406ba70 2024-04-05T22:21:11.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b640b1d90 con 0x7f4b6406ba70 2024-04-05T22:21:11.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.776+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b640b2350 con 0x7f4b6406ba70 2024-04-05T22:21:11.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.777+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b64068160 con 0x7f4b6406ba70 2024-04-05T22:21:11.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.778+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f4b580129b0 con 0x7f4b6406ba70 2024-04-05T22:21:11.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.779+0000 7f4b5e000640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 0x7f4b4806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:11.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.779+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b58088ac0 con 0x7f4b6406ba70 2024-04-05T22:21:11.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.779+0000 7f4b69400640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 0x7f4b4806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:11.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.780+0000 7f4b69400640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 0x7f4b4806d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b54006fd0 tx=0x7f4b54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:11.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.783+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4b58054030 con 0x7f4b6406ba70 2024-04-05T22:21:11.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.962+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b641584e0 con 0x7f4b6406ba70 2024-04-05T22:21:11.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.962+0000 7f4b5e000640 1 -- 172.21.15.73:0/1901270635 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b58057650 con 0x7f4b6406ba70 2024-04-05T22:21:11.963 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:11.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 msgr2=0x7f4b4806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 0x7f4b4806d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b54006fd0 tx=0x7f4b54008040 comp rx=0 tx=0).stop 2024-04-05T22:21:11.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 msgr2=0x7f4b641401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c4d0 tx=0x7f4b5800c990 comp rx=0 tx=0).stop 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 shutdown_connections 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4b4806ab70 0x7f4b4806d030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415abe0 0x7f4b6415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6406c9a0 0x7f4b6415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 --2- 172.21.15.73:0/1901270635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b6406ba70 0x7f4b641401c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 >> 172.21.15.73:0/1901270635 conn(0x7f4b64069cb0 msgr2=0x7f4b640af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 shutdown_connections 2024-04-05T22:21:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:11.965+0000 7f4b6c800640 1 -- 172.21.15.73:0/1901270635 wait complete. 2024-04-05T22:21:11.983 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:21:12.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.188+0000 7fefd2800640 1 Processor -- start 2024-04-05T22:21:12.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.188+0000 7fefd2800640 1 -- start start 2024-04-05T22:21:12.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefd2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefcc1587d0 0x7fefcc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefd2800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefcc0ccd20 0x7fefcc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefd2800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc06f5b0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefd2800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc070410 con 0x7fefcc1587d0 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefd2800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc158bb0 con 0x7fefcc0ccd20 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefcb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefcc1587d0 0x7fefcc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefcaa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefcc0ccd20 0x7fefcc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefcaa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefcc0ccd20 0x7fefcc06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:35230/0 (socket says 172.21.15.73:35230) 2024-04-05T22:21:12.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.190+0000 7fefcaa00640 1 -- 172.21.15.73:0/783933367 learned_addr learned my addr 172.21.15.73:0/783933367 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:12.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.191+0000 7fefcaa00640 1 -- 172.21.15.73:0/783933367 >> v2:172.21.15.73:3300/0 conn(0x7fefcc1587d0 msgr2=0x7fefcc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:12.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.191+0000 7fefcaa00640 1 --2- 172.21.15.73:0/783933367 >> v2:172.21.15.73:3300/0 conn(0x7fefcc1587d0 0x7fefcc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.191+0000 7fefcaa00640 1 -- 172.21.15.73:0/783933367 >> v1:172.21.15.157:6789/0 conn(0x7fefcc06d0c0 legacy=0x7fefcc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.191+0000 7fefcaa00640 1 -- 172.21.15.73:0/783933367 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc06f800 con 0x7fefcc0ccd20 2024-04-05T22:21:12.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.192+0000 7fefcaa00640 1 --2- 172.21.15.73:0/783933367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0ccd20 0x7fefcc06cb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009870 tx=0x7fefb402b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac7e07b25a8d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.193+0000 7fefca000640 1 -- 172.21.15.73:0/783933367 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4030850 con 0x7fefcc0ccd20 2024-04-05T22:21:12.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.193+0000 7fefca000640 1 -- 172.21.15.73:0/783933367 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb40309b0 con 0x7fefcc0ccd20 2024-04-05T22:21:12.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.193+0000 7fefca000640 1 -- 172.21.15.73:0/783933367 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4030d40 con 0x7fefcc0ccd20 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.193+0000 7fefd2800640 1 -- 172.21.15.73:0/783933367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0ccd20 msgr2=0x7fefcc06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.193+0000 7fefd2800640 1 --2- 172.21.15.73:0/783933367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0ccd20 0x7fefcc06cb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009870 tx=0x7fefb402b990 comp rx=0 tx=0).stop 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 -- 172.21.15.73:0/783933367 shutdown_connections 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 --2- 172.21.15.73:0/783933367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0ccd20 0x7fefcc06cb80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 --2- 172.21.15.73:0/783933367 >> v2:172.21.15.73:3300/0 conn(0x7fefcc1587d0 0x7fefcc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 -- 172.21.15.73:0/783933367 >> 172.21.15.73:0/783933367 conn(0x7fefcc0ce8e0 msgr2=0x7fefcc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 -- 172.21.15.73:0/783933367 shutdown_connections 2024-04-05T22:21:12.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.194+0000 7fefd2800640 1 -- 172.21.15.73:0/783933367 wait complete. 2024-04-05T22:21:12.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.195+0000 7fefd2800640 1 Processor -- start 2024-04-05T22:21:12.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.195+0000 7fefd2800640 1 -- start start 2024-04-05T22:21:12.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc1587d0 0x7fefcc157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefcc0ad080 0x7fefcc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc1523d0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0ada50 con 0x7fefcc1587d0 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.196+0000 7fefd2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc151750 con 0x7fefcc0ad080 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50912/0 (socket says 172.21.15.73:50912) 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 -- 172.21.15.73:0/1659999960 learned_addr learned my addr 172.21.15.73:0/1659999960 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcaa00640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc1587d0 0x7fefcc157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcbe00640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefcc0ad080 0x7fefcc0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 -- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefcc0ad080 msgr2=0x7fefcc0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefcc0ad080 0x7fefcc0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 -- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc1587d0 msgr2=0x7fefcc157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc1587d0 0x7fefcc157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 -- 172.21.15.73:0/1659999960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40094d0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.197+0000 7fefcb400640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e870 tx=0x7fefbc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.198+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0094f0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.198+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc00c940 con 0x7fefcc06d0c0 2024-04-05T22:21:12.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.198+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00ccf0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.198+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc151a30 con 0x7fefcc06d0c0 2024-04-05T22:21:12.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.198+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc151ff0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.199+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc066af0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.200+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fefbc013470 con 0x7fefcc06d0c0 2024-04-05T22:21:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.201+0000 7fefc3400640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 0x7fefa806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.201+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fefbc089310 con 0x7fefcc06d0c0 2024-04-05T22:21:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.201+0000 7fefcaa00640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 0x7fefa806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.202+0000 7fefcaa00640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 0x7fefa806d030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fefb4036000 tx=0x7fefb4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.206+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fefbc0549b0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.386+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fefcc152640 con 0x7fefcc06d0c0 2024-04-05T22:21:12.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.387+0000 7fefc3400640 1 -- 172.21.15.73:0/1659999960 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fefbc057fd0 con 0x7fefcc06d0c0 2024-04-05T22:21:12.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:12.387 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:21:12.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:21:12.387 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:21:12.388 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:21:12.388 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:21:12.388 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:21:12.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 msgr2=0x7fefa806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 0x7fefa806d030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fefb4036000 tx=0x7fefb4008030 comp rx=0 tx=0).stop 2024-04-05T22:21:12.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 msgr2=0x7fefcc13c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e870 tx=0x7fefbc00ed30 comp rx=0 tx=0).stop 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 shutdown_connections 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefa806ab70 0x7fefa806d030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefcc0ad080 0x7fefcc0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc1587d0 0x7fefcc157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 --2- 172.21.15.73:0/1659999960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefcc06d0c0 0x7fefcc13c7b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 >> 172.21.15.73:0/1659999960 conn(0x7fefcc0ce8e0 msgr2=0x7fefcc15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 shutdown_connections 2024-04-05T22:21:12.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.390+0000 7fefd2800640 1 -- 172.21.15.73:0/1659999960 wait complete. 2024-04-05T22:21:12.407 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:21:12.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:21:12.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.619+0000 7ff49a000640 1 Processor -- start 2024-04-05T22:21:12.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.619+0000 7ff49a000640 1 -- start start 2024-04-05T22:21:12.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff49a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff49415ae50 0x7ff4940b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff49a000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49415b230 con 0x7ff4940cf3e0 2024-04-05T22:21:12.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff49a000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49415f4a0 con 0x7ff4940cce90 2024-04-05T22:21:12.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff49a000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4940cd320 con 0x7ff49415ae50 2024-04-05T22:21:12.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff493400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff49415ae50 0x7ff4940b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff493e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff4940cf3e0 0x7ff4940b1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51840/0 (socket says 172.21.15.73:51840) 2024-04-05T22:21:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.621+0000 7ff493e00640 1 -- 172.21.15.73:0/3099921483 learned_addr learned my addr 172.21.15.73:0/3099921483 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.622+0000 7ff493400640 1 -- 172.21.15.73:0/3099921483 >> v1:172.21.15.157:6789/0 conn(0x7ff4940cce90 legacy=0x7ff4940cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.622+0000 7ff493400640 1 -- 172.21.15.73:0/3099921483 >> v1:172.21.15.73:6789/0 conn(0x7ff4940cf3e0 legacy=0x7ff4940b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.622+0000 7ff493400640 1 -- 172.21.15.73:0/3099921483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49415f780 con 0x7ff49415ae50 2024-04-05T22:21:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.622+0000 7ff492000640 1 -- 172.21.15.73:0/3099921483 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823808717 0 0) 0x7ff49415b230 con 0x7ff4940cf3e0 2024-04-05T22:21:12.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.623+0000 7ff493400640 1 --2- 172.21.15.73:0/3099921483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff49415ae50 0x7ff4940b3aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff484009d90 tx=0x7ff48402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70928bbd87e018fe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.623+0000 7ff492000640 1 -- 172.21.15.73:0/3099921483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48402cc60 con 0x7ff49415ae50 2024-04-05T22:21:12.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.623+0000 7ff492000640 1 -- 172.21.15.73:0/3099921483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48402cdc0 con 0x7ff49415ae50 2024-04-05T22:21:12.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.623+0000 7ff492000640 1 -- 172.21.15.73:0/3099921483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484034660 con 0x7ff49415ae50 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 -- 172.21.15.73:0/3099921483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff49415ae50 msgr2=0x7ff4940b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 --2- 172.21.15.73:0/3099921483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff49415ae50 0x7ff4940b3aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff484009d90 tx=0x7ff48402beb0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 -- 172.21.15.73:0/3099921483 shutdown_connections 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 --2- 172.21.15.73:0/3099921483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff49415ae50 0x7ff4940b3aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 -- 172.21.15.73:0/3099921483 >> 172.21.15.73:0/3099921483 conn(0x7ff49406a0d0 msgr2=0x7ff49406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 -- 172.21.15.73:0/3099921483 shutdown_connections 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.624+0000 7ff49a000640 1 -- 172.21.15.73:0/3099921483 wait complete. 2024-04-05T22:21:12.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.626+0000 7ff49a000640 1 Processor -- start 2024-04-05T22:21:12.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.626+0000 7ff49a000640 1 -- start start 2024-04-05T22:21:12.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940cf3e0 0x7ff494155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 0x7ff4941f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff494157b30 con 0x7ff4940cce90 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff494157800 con 0x7ff4940cf3e0 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff49a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941f2b10 con 0x7ff49415ae50 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50936/0 (socket says 172.21.15.73:50936) 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 -- 172.21.15.73:0/4121233577 learned_addr learned my addr 172.21.15.73:0/4121233577 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 -- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 msgr2=0x7ff4941f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493e00640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 0x7ff4941f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff492a00640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940cf3e0 0x7ff494155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 0x7ff4941f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 -- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940cf3e0 msgr2=0x7ff494155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940cf3e0 0x7ff494155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:12.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493400640 1 -- 172.21.15.73:0/4121233577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484009a70 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.628+0000 7ff493e00640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 0x7ff4941f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff493400640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff484037040 tx=0x7ff484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484040650 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4840407b0 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484040b60 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4941f2df0 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.629+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4941f2760 con 0x7ff4940cce90 2024-04-05T22:21:12.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.630+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494068160 con 0x7ff4940cce90 2024-04-05T22:21:12.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.631+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7ff484040cc0 con 0x7ff4940cce90 2024-04-05T22:21:12.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.632+0000 7ff48b400640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 0x7ff47406d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:12.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.632+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4840abc40 con 0x7ff4940cce90 2024-04-05T22:21:12.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.632+0000 7ff492a00640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 0x7ff47406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:12.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.633+0000 7ff492a00640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 0x7ff47406d060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cf200 tx=0x7ff480009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:12.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.637+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff4840772d0 con 0x7ff4940cce90 2024-04-05T22:21:12.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:12.830+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff494073770 con 0x7ff4940cce90 2024-04-05T22:21:13.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.403+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4840776a0 con 0x7ff4940cce90 2024-04-05T22:21:13.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.405+0000 7ff48b400640 1 -- 172.21.15.73:0/4121233577 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff484090440 con 0x7ff4940cce90 2024-04-05T22:21:13.405 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.409+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 msgr2=0x7ff47406d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 0x7ff47406d060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cf200 tx=0x7ff480009040 comp rx=0 tx=0).stop 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 msgr2=0x7ff49406baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff484037040 tx=0x7ff484004a40 comp rx=0 tx=0).stop 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 shutdown_connections 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff47406aba0 0x7ff47406d060 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cf200 tx=0x7ff480009040 comp rx=0 tx=0).stop 2024-04-05T22:21:13.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff49415ae50 0x7ff4941f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940cf3e0 0x7ff494155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 --2- 172.21.15.73:0/4121233577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4940cce90 0x7ff49406baf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.410+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 >> 172.21.15.73:0/4121233577 conn(0x7ff49406a0d0 msgr2=0x7ff4940cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:13.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.411+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 shutdown_connections 2024-04-05T22:21:13.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.411+0000 7ff49a000640 1 -- 172.21.15.73:0/4121233577 wait complete. 2024-04-05T22:21:13.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:21:13.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.634+0000 7f703aa00640 1 Processor -- start 2024-04-05T22:21:13.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.634+0000 7f703aa00640 1 -- start start 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.635+0000 7f703aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f7034165750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.635+0000 7f703aa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703415afc0 con 0x7f703415abe0 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.635+0000 7f703aa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703406f5b0 con 0x7f703406c9a0 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.635+0000 7f703aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703406f890 con 0x7f703406bc40 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.636+0000 7f7033e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f703415abe0 0x7f70340712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51848/0 (socket says 172.21.15.73:51848) 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.636+0000 7f7033e00640 1 -- 172.21.15.73:0/2940003414 learned_addr learned my addr 172.21.15.73:0/2940003414 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:13.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.636+0000 7f7038a00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f7034165750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.636+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605937770 0 0) 0x7f703415afc0 con 0x7f703415abe0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.636+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7018003520 con 0x7f703415abe0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255098475 0 0) 0x7f7018003520 con 0x7f703415abe0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 msgr2=0x7f7034165750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f7034165750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 >> v1:172.21.15.157:6789/0 conn(0x7f703406c9a0 legacy=0x7f703406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703406fb70 con 0x7f703415abe0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7024001670 con 0x7f703415abe0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 >> v1:172.21.15.73:6789/0 conn(0x7f703415abe0 legacy=0x7f70340712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f70180039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f70180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004d00 0x7f7018005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703406f890 con 0x7f7018003ee0 2024-04-05T22:21:13.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701800bb40 con 0x7f7018004d00 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701800bd00 con 0x7f703406bc40 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f702be00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f70180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.637+0000 7f7033e00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f70180039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f7038a00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004d00 0x7f7018005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f702be00640 1 -- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 msgr2=0x7f70180039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f702be00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f70180039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f702be00640 1 -- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004d00 msgr2=0x7f7018005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f702be00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004d00 0x7f7018005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f702be00640 1 -- 172.21.15.73:0/2940003414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018003520 con 0x7f7018003ee0 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f7033e00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703406bc40 0x7f70180039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:13.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.638+0000 7f7038a00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7018004d00 0x7f7018005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:13.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.639+0000 7f702be00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f70180047c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f701c001910 tx=0x7f70180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d056de54acfab44 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.639+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c032620 con 0x7f7018003ee0 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.639+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c032780 con 0x7f7018003ee0 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.639+0000 7f7033400640 1 -- 172.21.15.73:0/2940003414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c032b50 con 0x7f7018003ee0 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 -- 172.21.15.73:0/2940003414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 msgr2=0x7f70180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 --2- 172.21.15.73:0/2940003414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f70180047c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f701c001910 tx=0x7f70180057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f7038a00640 1 -- 172.21.15.73:0/2940003414 reap_dead start 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 -- 172.21.15.73:0/2940003414 shutdown_connections 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 -- 172.21.15.73:0/2940003414 >> 172.21.15.73:0/2940003414 conn(0x7f7034069cb0 msgr2=0x7f703406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 -- 172.21.15.73:0/2940003414 shutdown_connections 2024-04-05T22:21:13.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.640+0000 7f703aa00640 1 -- 172.21.15.73:0/2940003414 wait complete. 2024-04-05T22:21:13.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.642+0000 7f703aa00640 1 Processor -- start 2024-04-05T22:21:13.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.642+0000 7f703aa00640 1 -- start start 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.643+0000 7f703aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.643+0000 7f703aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7034149310 0x7f70340b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f703aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70340b3460 0x7f703415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f703aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703415c380 con 0x7f7018003ee0 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f703aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70340b38f0 con 0x7f7034149310 2024-04-05T22:21:13.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f703aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703413ed60 con 0x7f70340b3460 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50966/0 (socket says 172.21.15.73:50966) 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 -- 172.21.15.73:0/2864559453 learned_addr learned my addr 172.21.15.73:0/2864559453 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7038a00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70340b3460 0x7f703415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f702be00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7034149310 0x7f70340b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 -- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70340b3460 msgr2=0x7f703415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70340b3460 0x7f703415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 -- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7034149310 msgr2=0x7f70340b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7034149310 0x7f70340b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.644+0000 7f7033e00640 1 -- 172.21.15.73:0/2864559453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034145b50 con 0x7f7018003ee0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f7033e00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f702400e780 tx=0x7f702400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400ee90 con 0x7f7018003ee0 2024-04-05T22:21:13.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702400c870 con 0x7f7018003ee0 2024-04-05T22:21:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7034145e30 con 0x7f7018003ee0 2024-04-05T22:21:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400cbe0 con 0x7f7018003ee0 2024-04-05T22:21:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.645+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70341463f0 con 0x7f7018003ee0 2024-04-05T22:21:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.646+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70341465e0 con 0x7f7018003ee0 2024-04-05T22:21:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.647+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f702400cd40 con 0x7f7018003ee0 2024-04-05T22:21:13.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.648+0000 7f7030c00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 0x7f701006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:13.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.648+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f70240891f0 con 0x7f7018003ee0 2024-04-05T22:21:13.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.648+0000 7f702be00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 0x7f701006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:13.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.649+0000 7f702be00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 0x7f701006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f701c036000 tx=0x7f701c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:13.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.654+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7024054510 con 0x7f7018003ee0 2024-04-05T22:21:13.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:13.846+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7034068160 con 0x7f7018003ee0 2024-04-05T22:21:14.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.407+0000 7f7030c00640 1 -- 172.21.15.73:0/2864559453 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7024057b30 con 0x7f7018003ee0 2024-04-05T22:21:14.408 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:21:14.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.413+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 msgr2=0x7f701006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.413+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 0x7f701006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f701c036000 tx=0x7f701c009030 comp rx=0 tx=0).stop 2024-04-05T22:21:14.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.413+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 msgr2=0x7f703413e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.413+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f702400e780 tx=0x7f702400ec40 comp rx=0 tx=0).stop 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 shutdown_connections 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f701006ac00 0x7f701006d0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70340b3460 0x7f703415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7034149310 0x7f70340b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 --2- 172.21.15.73:0/2864559453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7018003ee0 0x7f703413e820 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f702400e780 tx=0x7f702400ec40 comp rx=0 tx=0).stop 2024-04-05T22:21:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.414+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 >> 172.21.15.73:0/2864559453 conn(0x7f7034069cb0 msgr2=0x7f70340ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.417+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 shutdown_connections 2024-04-05T22:21:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.417+0000 7f703aa00640 1 -- 172.21.15.73:0/2864559453 wait complete. 2024-04-05T22:21:14.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:21:14.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.633+0000 7fb77fe00640 1 Processor -- start 2024-04-05T22:21:14.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.633+0000 7fb77fe00640 1 -- start start 2024-04-05T22:21:14.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb78015ac30 0x7fb78006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78006bc50 con 0x7fb78015a850 2024-04-05T22:21:14.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78015f530 con 0x7fb78015ac30 2024-04-05T22:21:14.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800708e0 con 0x7fb780070470 2024-04-05T22:21:14.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb78015ac30 0x7fb78006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb78015a850 0x7fb78015e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51850/0 (socket says 172.21.15.73:51850) 2024-04-05T22:21:14.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb78015ac30 0x7fb78006ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50978/0 (socket says 172.21.15.73:50978) 2024-04-05T22:21:14.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77ea00640 1 -- 172.21.15.73:0/629485998 learned_addr learned my addr 172.21.15.73:0/629485998 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77f400640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.635+0000 7fb77e000640 1 -- 172.21.15.73:0/629485998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 msgr2=0x7fb78006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.636+0000 7fb77e000640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78006c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.636+0000 7fb77e000640 1 -- 172.21.15.73:0/629485998 >> v1:172.21.15.73:6789/0 conn(0x7fb78015a850 legacy=0x7fb78015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.636+0000 7fb77e000640 1 -- 172.21.15.73:0/629485998 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78015ef70 con 0x7fb78015ac30 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.636+0000 7fb77d600640 1 -- 172.21.15.73:0/629485998 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853035704 0 0) 0x7fb78006bc50 con 0x7fb78015a850 2024-04-05T22:21:14.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.637+0000 7fb77e000640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb78006ff30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097d0 tx=0x7fb76c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7c990d0a59a76b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:14.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.637+0000 7fb77d600640 1 -- 172.21.15.73:0/629485998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c02d040 con 0x7fb78015ac30 2024-04-05T22:21:14.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.637+0000 7fb77d600640 1 -- 172.21.15.73:0/629485998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c031810 con 0x7fb78015ac30 2024-04-05T22:21:14.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.637+0000 7fb77d600640 1 -- 172.21.15.73:0/629485998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c031ba0 con 0x7fb78015ac30 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 -- 172.21.15.73:0/629485998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 msgr2=0x7fb78006ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb78006ff30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097d0 tx=0x7fb76c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 -- 172.21.15.73:0/629485998 shutdown_connections 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78006c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 --2- 172.21.15.73:0/629485998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb78006ff30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 -- 172.21.15.73:0/629485998 >> 172.21.15.73:0/629485998 conn(0x7fb780069cb0 msgr2=0x7fb78006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 -- 172.21.15.73:0/629485998 shutdown_connections 2024-04-05T22:21:14.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.638+0000 7fb77fe00640 1 -- 172.21.15.73:0/629485998 wait complete. 2024-04-05T22:21:14.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.639+0000 7fb77fe00640 1 Processor -- start 2024-04-05T22:21:14.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.640+0000 7fb77fe00640 1 -- start start 2024-04-05T22:21:14.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb78015a850 0x7fb78014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801549f0 con 0x7fb78015ac30 2024-04-05T22:21:14.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78014f3f0 con 0x7fb78015a850 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78014ee30 con 0x7fb780070470 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50988/0 (socket says 172.21.15.73:50988) 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 -- 172.21.15.73:0/1626885365 learned_addr learned my addr 172.21.15.73:0/1626885365 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77ea00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 -- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 msgr2=0x7fb78014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77e000640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb78015a850 0x7fb78014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78014e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 -- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb78015a850 msgr2=0x7fb78014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb78015a850 0x7fb78014c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.641+0000 7fb77f400640 1 -- 172.21.15.73:0/1626885365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c009400 con 0x7fb78015ac30 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb77f400640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb77800cd70 tx=0x7fb77800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778003b90 con 0x7fb78015ac30 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778003cf0 con 0x7fb78015ac30 2024-04-05T22:21:14.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78014f110 con 0x7fb78015ac30 2024-04-05T22:21:14.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778004240 con 0x7fb78015ac30 2024-04-05T22:21:14.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.642+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78015ba00 con 0x7fb78015ac30 2024-04-05T22:21:14.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.643+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78015bbf0 con 0x7fb78015ac30 2024-04-05T22:21:14.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.644+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb7780043a0 con 0x7fb78015ac30 2024-04-05T22:21:14.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.645+0000 7fb777400640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 0x7fb76006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:14.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.645+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb778012070 con 0x7fb78015ac30 2024-04-05T22:21:14.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.645+0000 7fb77ea00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 0x7fb76006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:14.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.646+0000 7fb77ea00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 0x7fb76006d0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb770000f60 tx=0x7fb770009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:14.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.650+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb778054f50 con 0x7fb78015ac30 2024-04-05T22:21:14.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.881+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb78004e160 con 0x7fb78015ac30 2024-04-05T22:21:14.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.894+0000 7fb777400640 1 -- 172.21.15.73:0/1626885365 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb778058570 con 0x7fb78015ac30 2024-04-05T22:21:14.900 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 2 and data pool 3 2024-04-05T22:21:14.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.901+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 msgr2=0x7fb76006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.901+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 0x7fb76006d0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb770000f60 tx=0x7fb770009040 comp rx=0 tx=0).stop 2024-04-05T22:21:14.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.901+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 msgr2=0x7fb7801544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:14.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.901+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb77800cd70 tx=0x7fb77800a660 comp rx=0 tx=0).stop 2024-04-05T22:21:14.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 shutdown_connections 2024-04-05T22:21:14.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb76006ac30 0x7fb76006d0f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb78015ac30 0x7fb7801544b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb78015a850 0x7fb78014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 --2- 172.21.15.73:0/1626885365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb780070470 0x7fb78014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 >> 172.21.15.73:0/1626885365 conn(0x7fb780069cb0 msgr2=0x7fb780157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 shutdown_connections 2024-04-05T22:21:14.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:14.902+0000 7fb77fe00640 1 -- 172.21.15.73:0/1626885365 wait complete. 2024-04-05T22:21:14.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.098+0000 7f1a0f200640 1 Processor -- start 2024-04-05T22:21:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.098+0000 7f1a0f200640 1 -- start start 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 0x7f1a1015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a100d1600 0x7f1a100d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a100b2420 con 0x7f1a100d35e0 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a100b2770 con 0x7f1a100d1600 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a100ce380 con 0x7f1a1015ccd0 2024-04-05T22:21:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a100d1600 0x7f1a100d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a100d1600 0x7f1a100d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50996/0 (socket says 172.21.15.73:50996) 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.100+0000 7f1a0e800640 1 -- 172.21.15.73:0/3091079540 learned_addr learned my addr 172.21.15.73:0/3091079540 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0e800640 1 -- 172.21.15.73:0/3091079540 >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 msgr2=0x7f1a1015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0de00640 1 --2- 172.21.15.73:0/3091079540 >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 0x7f1a1015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0e800640 1 --2- 172.21.15.73:0/3091079540 >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 0x7f1a1015d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0e800640 1 -- 172.21.15.73:0/3091079540 >> v1:172.21.15.157:6789/0 conn(0x7f1a100d35e0 legacy=0x7f1a100d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0e800640 1 -- 172.21.15.73:0/3091079540 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a100ce660 con 0x7f1a100d1600 2024-04-05T22:21:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.101+0000 7f1a0de00640 1 --2- 172.21.15.73:0/3091079540 >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 0x7f1a1015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0e800640 1 --2- 172.21.15.73:0/3091079540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100d08a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a1008b200 tx=0x7f1a0802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cae08c3b8cceedb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0ca00640 1 -- 172.21.15.73:0/3091079540 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08031750 con 0x7f1a100d1600 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0ca00640 1 -- 172.21.15.73:0/3091079540 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a080318b0 con 0x7f1a100d1600 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0ca00640 1 -- 172.21.15.73:0/3091079540 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08031c40 con 0x7f1a100d1600 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 -- 172.21.15.73:0/3091079540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 msgr2=0x7f1a100d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 --2- 172.21.15.73:0/3091079540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100d08a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a1008b200 tx=0x7f1a0802b7f0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 -- 172.21.15.73:0/3091079540 shutdown_connections 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 --2- 172.21.15.73:0/3091079540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100d08a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 --2- 172.21.15.73:0/3091079540 >> v2:172.21.15.157:3300/0 conn(0x7f1a1015ccd0 0x7f1a1015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.102+0000 7f1a0f200640 1 -- 172.21.15.73:0/3091079540 >> 172.21.15.73:0/3091079540 conn(0x7f1a100d2ac0 msgr2=0x7f1a100d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.103+0000 7f1a0f200640 1 -- 172.21.15.73:0/3091079540 shutdown_connections 2024-04-05T22:21:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.103+0000 7f1a0f200640 1 -- 172.21.15.73:0/3091079540 wait complete. 2024-04-05T22:21:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.104+0000 7f1a0f200640 1 Processor -- start 2024-04-05T22:21:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.104+0000 7f1a0f200640 1 -- start start 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a100d35e0 0x7f1a1006b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a1015ccd0 0x7f1a101f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100b7880 con 0x7f1a100d1600 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100b7b60 con 0x7f1a1015ccd0 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100b7e40 con 0x7f1a100d35e0 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51004/0 (socket says 172.21.15.73:51004) 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.105+0000 7f1a0de00640 1 -- 172.21.15.73:0/4188614041 learned_addr learned my addr 172.21.15.73:0/4188614041 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 -- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a100d35e0 msgr2=0x7f1a1006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0d400640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a100d35e0 0x7f1a1006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0e800640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a1015ccd0 0x7f1a101f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a100d35e0 0x7f1a1006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 -- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a1015ccd0 msgr2=0x7f1a101f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a1015ccd0 0x7f1a101f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 -- 172.21.15.73:0/4188614041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080094d0 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0de00640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c7f0 tx=0x7f1a0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00007820 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00007980 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a100b8120 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.106+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00007cf0 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.107+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a100b74c0 con 0x7f1a100d1600 2024-04-05T22:21:15.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.108+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a10067250 con 0x7f1a100d1600 2024-04-05T22:21:15.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.108+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1a000113f0 con 0x7f1a100d1600 2024-04-05T22:21:15.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.109+0000 7f1a06000640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 0x7f19ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.109+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a000896c0 con 0x7f1a100d1600 2024-04-05T22:21:15.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.109+0000 7f1a0d400640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 0x7f19ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.110+0000 7f1a0d400640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 0x7f19ec06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007fd0 tx=0x7f19f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.114+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1a000548a0 con 0x7f1a100d1600 2024-04-05T22:21:15.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.293+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a100d30d0 con 0x7f1a100d1600 2024-04-05T22:21:15.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.295+0000 7f1a06000640 1 -- 172.21.15.73:0/4188614041 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f1a00057ec0 con 0x7f1a100d1600 2024-04-05T22:21:15.296 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:15.296 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":21,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:14.884045+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 msgr2=0x7f19ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 0x7f19ec06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007fd0 tx=0x7f19f8009040 comp rx=0 tx=0).stop 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 msgr2=0x7f1a100b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c7f0 tx=0x7f1a0000ccb0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 shutdown_connections 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19ec06ac00 0x7f19ec06d0c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a1015ccd0 0x7f1a101f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a100d35e0 0x7f1a1006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 --2- 172.21.15.73:0/4188614041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a100d1600 0x7f1a100b6f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 >> 172.21.15.73:0/4188614041 conn(0x7f1a100d2ac0 msgr2=0x7f1a100b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 shutdown_connections 2024-04-05T22:21:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.298+0000 7f1a0f200640 1 -- 172.21.15.73:0/4188614041 wait complete. 2024-04-05T22:21:15.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:15.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.521+0000 7f4705a00640 1 Processor -- start 2024-04-05T22:21:15.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.521+0000 7f4705a00640 1 -- start start 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.522+0000 7f4705a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47000b2b90 0x7f4700070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.522+0000 7f4705a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47000b1b50 0x7f470006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.522+0000 7f4705a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470006d3e0 con 0x7f47000b20a0 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.522+0000 7f4705a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470006d610 con 0x7f47000b2b90 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.522+0000 7f4705a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4700070230 con 0x7f47000b1b50 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47000b2b90 0x7f4700070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ffe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f47000b20a0 0x7f47000b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51864/0 (socket says 172.21.15.73:51864) 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47000b2b90 0x7f4700070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51018/0 (socket says 172.21.15.73:51018) 2024-04-05T22:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ffe00640 1 -- 172.21.15.73:0/770542753 learned_addr learned my addr 172.21.15.73:0/770542753 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f4705000640 1 --2- 172.21.15.73:0/770542753 >> v2:172.21.15.157:3300/0 conn(0x7f47000b1b50 0x7f470006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 -- 172.21.15.73:0/770542753 >> v2:172.21.15.157:3300/0 conn(0x7f47000b1b50 msgr2=0x7f470006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 --2- 172.21.15.73:0/770542753 >> v2:172.21.15.157:3300/0 conn(0x7f47000b1b50 0x7f470006cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 -- 172.21.15.73:0/770542753 >> v1:172.21.15.73:6789/0 conn(0x7f47000b20a0 legacy=0x7f47000b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46ff400640 1 -- 172.21.15.73:0/770542753 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700070510 con 0x7f47000b2b90 2024-04-05T22:21:15.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.523+0000 7f46fea00640 1 -- 172.21.15.73:0/770542753 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969786931 0 0) 0x7f470006d3e0 con 0x7f47000b20a0 2024-04-05T22:21:15.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.525+0000 7f46ff400640 1 --2- 172.21.15.73:0/770542753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b2b90 0x7f4700070ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47000b4560 tx=0x7f46ec02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4705dd6b88015306 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.525+0000 7f46fea00640 1 -- 172.21.15.73:0/770542753 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec031710 con 0x7f47000b2b90 2024-04-05T22:21:15.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.525+0000 7f46fea00640 1 -- 172.21.15.73:0/770542753 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec02f050 con 0x7f47000b2b90 2024-04-05T22:21:15.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.525+0000 7f46fea00640 1 -- 172.21.15.73:0/770542753 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec031a90 con 0x7f47000b2b90 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 -- 172.21.15.73:0/770542753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b2b90 msgr2=0x7f4700070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 --2- 172.21.15.73:0/770542753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b2b90 0x7f4700070ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47000b4560 tx=0x7f46ec02b7b0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 -- 172.21.15.73:0/770542753 shutdown_connections 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 --2- 172.21.15.73:0/770542753 >> v2:172.21.15.157:3300/0 conn(0x7f47000b1b50 0x7f470006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 --2- 172.21.15.73:0/770542753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b2b90 0x7f4700070ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 -- 172.21.15.73:0/770542753 >> 172.21.15.73:0/770542753 conn(0x7f47000b06c0 msgr2=0x7f47000b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:15.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 -- 172.21.15.73:0/770542753 shutdown_connections 2024-04-05T22:21:15.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.526+0000 7f4705a00640 1 -- 172.21.15.73:0/770542753 wait complete. 2024-04-05T22:21:15.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.527+0000 7f4705a00640 1 Processor -- start 2024-04-05T22:21:15.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.528+0000 7f4705a00640 1 -- start start 2024-04-05T22:21:15.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.528+0000 7f4705a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f4705a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47000b20a0 0x7f4700150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f4705a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47000b2b90 0x7f47000ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f4705a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470014fd30 con 0x7f47000b1b50 2024-04-05T22:21:15.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f4705a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001f5ad0 con 0x7f47000b2b90 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f4705a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47001f5db0 con 0x7f47000b20a0 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51022/0 (socket says 172.21.15.73:51022) 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 -- 172.21.15.73:0/1314219144 learned_addr learned my addr 172.21.15.73:0/1314219144 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 -- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47000b20a0 msgr2=0x7f4700150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47000b20a0 0x7f4700150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 -- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47000b2b90 msgr2=0x7f47000ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47000b2b90 0x7f47000ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 -- 172.21.15.73:0/1314219144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009400 con 0x7f47000b1b50 2024-04-05T22:21:15.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.529+0000 7f46ffe00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c4d0 tx=0x7f46f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.530+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00074e0 con 0x7f47000b1b50 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.530+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47001f6090 con 0x7f47000b1b50 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.530+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0007660 con 0x7f47000b1b50 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.530+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004a20 con 0x7f47000b1b50 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.530+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001f6650 con 0x7f47000b1b50 2024-04-05T22:21:15.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.531+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470004e160 con 0x7f47000b1b50 2024-04-05T22:21:15.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.532+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f46f0004b80 con 0x7f47000b1b50 2024-04-05T22:21:15.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.533+0000 7f46f7e00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 0x7f46e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.533+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46f0088620 con 0x7f47000b1b50 2024-04-05T22:21:15.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.533+0000 7f46ff400640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 0x7f46e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.534+0000 7f46ff400640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 0x7f46e006d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46ec035000 tx=0x7f46ec030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.538+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f46f00536e0 con 0x7f47000b1b50 2024-04-05T22:21:15.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.733+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47000b2920 con 0x7f47000b1b50 2024-04-05T22:21:15.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.735+0000 7f46f7e00640 1 -- 172.21.15.73:0/1314219144 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f46f0056d00 con 0x7f47000b1b50 2024-04-05T22:21:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:15.736 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":21,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:14.884045+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:15.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 msgr2=0x7f46e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 0x7f46e006d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46ec035000 tx=0x7f46ec030000 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 msgr2=0x7f470014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c4d0 tx=0x7f46f000c990 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 shutdown_connections 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f46e006ac00 0x7f46e006d0c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47000b2b90 0x7f47000ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47000b20a0 0x7f4700150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 --2- 172.21.15.73:0/1314219144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47000b1b50 0x7f470014f7f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 >> 172.21.15.73:0/1314219144 conn(0x7f47000b06c0 msgr2=0x7f4700070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 shutdown_connections 2024-04-05T22:21:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.738+0000 7f4705a00640 1 -- 172.21.15.73:0/1314219144 wait complete. 2024-04-05T22:21:15.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:21:15.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.967+0000 7fc9dd400640 1 Processor -- start 2024-04-05T22:21:15.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.967+0000 7fc9dd400640 1 -- start start 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9dd400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9d815aa70 0x7fc9d8070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9dd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d80714a0 0x7fc9d815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9dd400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d815ae50 con 0x7fc9d815dbc0 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9dd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d8071930 con 0x7fc9d80714a0 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9dd400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d815e3b0 con 0x7fc9d815aa70 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d80714a0 0x7fc9d815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d80714a0 0x7fc9d815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50754/0 (socket says 172.21.15.73:50754) 2024-04-05T22:21:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 -- 172.21.15.73:0/3031039758 learned_addr learned my addr 172.21.15.73:0/3031039758 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 -- 172.21.15.73:0/3031039758 >> v2:172.21.15.157:3300/0 conn(0x7fc9d815aa70 msgr2=0x7fc9d8070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 --2- 172.21.15.73:0/3031039758 >> v2:172.21.15.157:3300/0 conn(0x7fc9d815aa70 0x7fc9d8070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 -- 172.21.15.73:0/3031039758 >> v1:172.21.15.157:6789/0 conn(0x7fc9d815dbc0 legacy=0x7fc9d81609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:15.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.969+0000 7fc9d5e00640 1 -- 172.21.15.73:0/3031039758 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d806ca50 con 0x7fc9d80714a0 2024-04-05T22:21:15.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.971+0000 7fc9d5e00640 1 --2- 172.21.15.73:0/3031039758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d815d680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80b16a0 tx=0x7fc9c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99000cb840887f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.971+0000 7fc9d5400640 1 -- 172.21.15.73:0/3031039758 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031870 con 0x7fc9d80714a0 2024-04-05T22:21:15.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.971+0000 7fc9d5400640 1 -- 172.21.15.73:0/3031039758 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c40319d0 con 0x7fc9d80714a0 2024-04-05T22:21:15.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.971+0000 7fc9d5400640 1 -- 172.21.15.73:0/3031039758 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031d60 con 0x7fc9d80714a0 2024-04-05T22:21:15.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 -- 172.21.15.73:0/3031039758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 msgr2=0x7fc9d815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3031039758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d815d680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80b16a0 tx=0x7fc9c402b910 comp rx=0 tx=0).stop 2024-04-05T22:21:15.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 -- 172.21.15.73:0/3031039758 shutdown_connections 2024-04-05T22:21:15.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3031039758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d815d680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3031039758 >> v2:172.21.15.157:3300/0 conn(0x7fc9d815aa70 0x7fc9d8070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 -- 172.21.15.73:0/3031039758 >> 172.21.15.73:0/3031039758 conn(0x7fc9d8069cb0 msgr2=0x7fc9d806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 -- 172.21.15.73:0/3031039758 shutdown_connections 2024-04-05T22:21:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.972+0000 7fc9dd400640 1 -- 172.21.15.73:0/3031039758 wait complete. 2024-04-05T22:21:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.974+0000 7fc9dd400640 1 Processor -- start 2024-04-05T22:21:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.975+0000 7fc9dd400640 1 -- start start 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d815aa70 0x7fc9d80afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d815dbc0 0x7fc9d80cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d8152cc0 con 0x7fc9d80714a0 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80aed20 con 0x7fc9d815dbc0 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9dd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80af000 con 0x7fc9d815aa70 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50770/0 (socket says 172.21.15.73:50770) 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.976+0000 7fc9d6800640 1 -- 172.21.15.73:0/3045851001 learned_addr learned my addr 172.21.15.73:0/3045851001 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 -- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d815aa70 msgr2=0x7fc9d80afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d815aa70 0x7fc9d80afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 -- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d815dbc0 msgr2=0x7fc9d80cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d815dbc0 0x7fc9d80cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:15.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 -- 172.21.15.73:0/3045851001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40094d0 con 0x7fc9d80714a0 2024-04-05T22:21:15.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9d6800640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b4a0 tx=0x7fc9c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8010610 con 0x7fc9d80714a0 2024-04-05T22:21:15.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8010770 con 0x7fc9d80714a0 2024-04-05T22:21:15.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8010b20 con 0x7fc9d80714a0 2024-04-05T22:21:15.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d80af2e0 con 0x7fc9d80714a0 2024-04-05T22:21:15.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.977+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d8153e30 con 0x7fc9d80714a0 2024-04-05T22:21:15.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.978+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d80af520 con 0x7fc9d80714a0 2024-04-05T22:21:15.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.979+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc9c8010c80 con 0x7fc9d80714a0 2024-04-05T22:21:15.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.980+0000 7fc9cea00640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 0x7fc9b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:15.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.981+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c80890f0 con 0x7fc9d80714a0 2024-04-05T22:21:15.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.981+0000 7fc9d5e00640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 0x7fc9b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:15.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.981+0000 7fc9d5e00640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 0x7fc9b806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036000 tx=0x7fc9c4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:15.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:15.985+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc9d80af520 con 0x7fc9d80714a0 2024-04-05T22:21:16.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.219+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc9d81539b0 con 0x7fc9d80714a0 2024-04-05T22:21:16.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.898+0000 7fc9cea00640 1 -- 172.21.15.73:0/3045851001 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc9d81539b0 con 0x7fc9d80714a0 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 msgr2=0x7fc9b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 0x7fc9b806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036000 tx=0x7fc9c4030000 comp rx=0 tx=0).stop 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 msgr2=0x7fc9d8151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b4a0 tx=0x7fc9c800b960 comp rx=0 tx=0).stop 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 shutdown_connections 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9b806ac00 0x7fc9b806d0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d815dbc0 0x7fc9d80cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:16.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d815aa70 0x7fc9d80afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 --2- 172.21.15.73:0/3045851001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d80714a0 0x7fc9d8151c00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 >> 172.21.15.73:0/3045851001 conn(0x7fc9d8069cb0 msgr2=0x7fc9d806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.901+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 shutdown_connections 2024-04-05T22:21:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:16.902+0000 7fc9dd400640 1 -- 172.21.15.73:0/3045851001 wait complete. 2024-04-05T22:21:16.924 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:21:16.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:21:17.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.134+0000 7f996aa00640 1 Processor -- start 2024-04-05T22:21:17.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.134+0000 7f996aa00640 1 -- start start 2024-04-05T22:21:17.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f996aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 0x7f99640cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f996aa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996415ddd0 con 0x7f996415d1f0 2024-04-05T22:21:17.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f996aa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996415e0b0 con 0x7f99640b10a0 2024-04-05T22:21:17.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f996aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99640cd460 con 0x7f996415ce10 2024-04-05T22:21:17.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f995b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f996415d1f0 0x7f99640b0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48718/0 (socket says 172.21.15.73:48718) 2024-04-05T22:21:17.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.136+0000 7f995b400640 1 -- 172.21.15.73:0/1235524720 learned_addr learned my addr 172.21.15.73:0/1235524720 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:17.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963e00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 0x7f99640cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930550909 0 0) 0x7f996415ddd0 con 0x7f996415d1f0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9948003520 con 0x7f996415d1f0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928506779 0 0) 0x7f9948003520 con 0x7f996415d1f0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 msgr2=0x7f99640cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 0x7f99640cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 >> v1:172.21.15.157:6789/0 conn(0x7f99640b10a0 legacy=0x7f99640cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99640cd740 con 0x7f996415d1f0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f994c001670 con 0x7f996415d1f0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 >> v1:172.21.15.73:6789/0 conn(0x7f996415d1f0 legacy=0x7f99640b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f9948003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99480047c0 0x7f9948004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 0x7f9948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640cd460 con 0x7f99480039a0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994800ba30 con 0x7f99480047c0 2024-04-05T22:21:17.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994800bbf0 con 0x7f996415ce10 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.137+0000 7f9963e00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f9948003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f995b400640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99480047c0 0x7f9948004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f9963e00640 1 -- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 msgr2=0x7f9948005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f9963e00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996415ce10 0x7f9948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f9963e00640 1 -- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99480047c0 msgr2=0x7f9948004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f9963e00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99480047c0 0x7f9948004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.138+0000 7f9963e00640 1 -- 172.21.15.73:0/1235524720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948003520 con 0x7f99480039a0 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.139+0000 7f9963e00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f9948003dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9954009a90 tx=0x7f99480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644d1bea7394c06a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.139+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995402c6b0 con 0x7f99480039a0 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.139+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995402c810 con 0x7f99480039a0 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.139+0000 7f9963400640 1 -- 172.21.15.73:0/1235524720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995402cba0 con 0x7f99480039a0 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 -- 172.21.15.73:0/1235524720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 msgr2=0x7f9948003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 --2- 172.21.15.73:0/1235524720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f9948003dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9954009a90 tx=0x7f99480057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f9968a00640 1 -- 172.21.15.73:0/1235524720 reap_dead start 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 -- 172.21.15.73:0/1235524720 shutdown_connections 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 -- 172.21.15.73:0/1235524720 >> 172.21.15.73:0/1235524720 conn(0x7f9964099a60 msgr2=0x7f9964099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 -- 172.21.15.73:0/1235524720 shutdown_connections 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.140+0000 7f996aa00640 1 -- 172.21.15.73:0/1235524720 wait complete. 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.142+0000 7f996aa00640 1 Processor -- start 2024-04-05T22:21:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.142+0000 7f996aa00640 1 -- start start 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f996414be00 0x7f996415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996414c240 0x7f996415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996415c030 con 0x7f99480039a0 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996415bd50 con 0x7f996414be00 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f996aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9964159fc0 con 0x7f996414c240 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50790/0 (socket says 172.21.15.73:50790) 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 -- 172.21.15.73:0/1340711781 learned_addr learned my addr 172.21.15.73:0/1340711781 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f995b400640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f996414be00 0x7f996415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 -- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996414c240 msgr2=0x7f996415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996414c240 0x7f996415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 -- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f996414be00 msgr2=0x7f996415b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f996414be00 0x7f996415b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.143+0000 7f9963e00640 1 -- 172.21.15.73:0/1340711781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996415a2a0 con 0x7f99480039a0 2024-04-05T22:21:17.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f9963e00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f995402c250 tx=0x7f9954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:17.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954003810 con 0x7f99480039a0 2024-04-05T22:21:17.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954003970 con 0x7f99480039a0 2024-04-05T22:21:17.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954003ce0 con 0x7f99480039a0 2024-04-05T22:21:17.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996415a580 con 0x7f99480039a0 2024-04-05T22:21:17.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.144+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996415b860 con 0x7f99480039a0 2024-04-05T22:21:17.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.145+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964067480 con 0x7f99480039a0 2024-04-05T22:21:17.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.146+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9954040450 con 0x7f99480039a0 2024-04-05T22:21:17.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.147+0000 7f9960c00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 0x7f994006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:17.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.147+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99540ab270 con 0x7f99480039a0 2024-04-05T22:21:17.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.147+0000 7f995b400640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 0x7f994006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:17.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.148+0000 7f995b400640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 0x7f994006d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f994c001200 tx=0x7f994c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:17.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.151+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9954076320 con 0x7f99480039a0 2024-04-05T22:21:17.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.382+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9964068160 con 0x7f99480039a0 2024-04-05T22:21:17.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.901+0000 7f9960c00640 1 -- 172.21.15.73:0/1340711781 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99540044e0 con 0x7f99480039a0 2024-04-05T22:21:17.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.904+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 msgr2=0x7f994006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 0x7f994006d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f994c001200 tx=0x7f994c009040 comp rx=0 tx=0).stop 2024-04-05T22:21:17.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 msgr2=0x7f996415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:17.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f995402c250 tx=0x7f9954004730 comp rx=0 tx=0).stop 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 shutdown_connections 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f994006ac00 0x7f994006d0c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f996414c240 0x7f996415efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f996414be00 0x7f996415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 --2- 172.21.15.73:0/1340711781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99480039a0 0x7f996415aaa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 >> 172.21.15.73:0/1340711781 conn(0x7f9964099a60 msgr2=0x7f996406f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 shutdown_connections 2024-04-05T22:21:17.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:17.905+0000 7f996aa00640 1 -- 172.21.15.73:0/1340711781 wait complete. 2024-04-05T22:21:17.925 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:21:17.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:18.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.134+0000 7f80cb400640 1 Processor -- start 2024-04-05T22:21:18.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.134+0000 7f80cb400640 1 -- start start 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.135+0000 7f80cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 0x7f80c40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80cb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c406f5b0 con 0x7f80c40ccd20 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80cb400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c4070410 con 0x7f80c415e850 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80cb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4158bb0 con 0x7f80c41587d0 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80c3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f80c40ccd20 0x7f80c406cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48722/0 (socket says 172.21.15.73:48722) 2024-04-05T22:21:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80c3e00640 1 -- 172.21.15.73:0/216429715 learned_addr learned my addr 172.21.15.73:0/216429715 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80c8a00640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 0x7f80c40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665166877 0 0) 0x7f80c406f5b0 con 0x7f80c40ccd20 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.136+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a8007930 con 0x7f80c40ccd20 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098926126 0 0) 0x7f80a8007930 con 0x7f80c40ccd20 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 msgr2=0x7f80c40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 0x7f80c40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 >> v1:172.21.15.157:6789/0 conn(0x7f80c415e850 legacy=0x7f80c40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c406f800 con 0x7f80c40ccd20 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f80b0001670 con 0x7f80c40ccd20 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 >> v1:172.21.15.73:6789/0 conn(0x7f80c40ccd20 legacy=0x7f80c406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 0x7f80a8007db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a8008330 0x7f80a8008bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80a8009590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4158bb0 con 0x7f80a8009110 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a800fe70 con 0x7f80a8008330 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8010030 con 0x7f80c41587d0 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80a8009590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 -- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 msgr2=0x7f80a8007db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c41587d0 0x7f80a8007db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c3e00640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a8008330 0x7f80a8008bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 -- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a8008330 msgr2=0x7f80a8008bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a8008330 0x7f80a8008bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.137+0000 7f80c9400640 1 -- 172.21.15.73:0/216429715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a8007930 con 0x7f80a8009110 2024-04-05T22:21:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.138+0000 7f80c9400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80a8009590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001e50 tx=0x7f80a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa6d1f011a6dc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc003b70 con 0x7f80a8009110 2024-04-05T22:21:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc003cd0 con 0x7f80a8009110 2024-04-05T22:21:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80c3400640 1 -- 172.21.15.73:0/216429715 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc005310 con 0x7f80a8009110 2024-04-05T22:21:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 -- 172.21.15.73:0/216429715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 msgr2=0x7f80a8009590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 --2- 172.21.15.73:0/216429715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80a8009590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001e50 tx=0x7f80a8009c00 comp rx=0 tx=0).stop 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80c9400640 1 -- 172.21.15.73:0/216429715 reap_dead start 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 -- 172.21.15.73:0/216429715 shutdown_connections 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 -- 172.21.15.73:0/216429715 >> 172.21.15.73:0/216429715 conn(0x7f80c40ce8e0 msgr2=0x7f80c40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 -- 172.21.15.73:0/216429715 shutdown_connections 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.139+0000 7f80cb400640 1 -- 172.21.15.73:0/216429715 wait complete. 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.141+0000 7f80cb400640 1 Processor -- start 2024-04-05T22:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.141+0000 7f80cb400640 1 -- start start 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c4158cf0 0x7f80c4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c40cdcb0 0x7f80c40ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40ce670 con 0x7f80a8009110 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41507c0 con 0x7f80c40cdcb0 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80cb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4159de0 con 0x7f80c4158cf0 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50808/0 (socket says 172.21.15.73:50808) 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.142+0000 7f80c8a00640 1 -- 172.21.15.73:0/1577769810 learned_addr learned my addr 172.21.15.73:0/1577769810 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c3e00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c4158cf0 0x7f80c4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 -- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c4158cf0 msgr2=0x7f80c4153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c4158cf0 0x7f80c4153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 -- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c40cdcb0 msgr2=0x7f80c40ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c40cdcb0 0x7f80c40ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 -- 172.21.15.73:0/1577769810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c4159260 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c8a00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e820 tx=0x7f80b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400eed0 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b400c8c0 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c4159540 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.143+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400cc50 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.144+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c4159b00 con 0x7f80a8009110 2024-04-05T22:21:18.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.145+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c4066af0 con 0x7f80a8009110 2024-04-05T22:21:18.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.146+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f80b4013470 con 0x7f80a8009110 2024-04-05T22:21:18.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.146+0000 7f80c0c00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 0x7f80a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.147+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80b4089340 con 0x7f80a8009110 2024-04-05T22:21:18.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.147+0000 7f80c3e00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 0x7f80a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.147+0000 7f80c3e00640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 0x7f80a406d0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80b0001200 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.151+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f80b4054520 con 0x7f80a8009110 2024-04-05T22:21:18.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.380+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80c414fd70 con 0x7f80a8009110 2024-04-05T22:21:18.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.382+0000 7f80c0c00640 1 -- 172.21.15.73:0/1577769810 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f80b4057b40 con 0x7f80a8009110 2024-04-05T22:21:18.382 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:18.382 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:21:18.383 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 msgr2=0x7f80a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 0x7f80a406d0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80b0001200 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 msgr2=0x7f80c414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e820 tx=0x7f80b400ece0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 shutdown_connections 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80a406ac30 0x7f80a406d0f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c40cdcb0 0x7f80c40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c4158cf0 0x7f80c4153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 --2- 172.21.15.73:0/1577769810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80a8009110 0x7f80c414f830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.384+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 >> 172.21.15.73:0/1577769810 conn(0x7f80c40ce8e0 msgr2=0x7f80c415ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.385+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 shutdown_connections 2024-04-05T22:21:18.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.385+0000 7f80cb400640 1 -- 172.21.15.73:0/1577769810 wait complete. 2024-04-05T22:21:18.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.609+0000 7f6b9fe00640 1 Processor -- start 2024-04-05T22:21:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.609+0000 7f6b9fe00640 1 -- start start 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b9fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ba00b20a0 0x7f6ba0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b9fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ba00b1b50 0x7f6ba006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b9fe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba006d3e0 con 0x7f6ba00b2480 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b9fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba006d610 con 0x7f6ba00b1b50 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b9fe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba0070230 con 0x7f6ba00b20a0 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ba00b1b50 0x7f6ba006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ba00b1b50 0x7f6ba006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50824/0 (socket says 172.21.15.73:50824) 2024-04-05T22:21:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 -- 172.21.15.73:0/144804483 learned_addr learned my addr 172.21.15.73:0/144804483 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:18.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 -- 172.21.15.73:0/144804483 >> v2:172.21.15.157:3300/0 conn(0x7f6ba00b20a0 msgr2=0x7f6ba0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:18.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 --2- 172.21.15.73:0/144804483 >> v2:172.21.15.157:3300/0 conn(0x7f6ba00b20a0 0x7f6ba0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 -- 172.21.15.73:0/144804483 >> v1:172.21.15.157:6789/0 conn(0x7f6ba00b2480 legacy=0x7f6ba00b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:18.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.611+0000 7f6b96000640 1 -- 172.21.15.73:0/144804483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0070510 con 0x7f6ba00b1b50 2024-04-05T22:21:18.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.612+0000 7f6b96000640 1 --2- 172.21.15.73:0/144804483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b1b50 0x7f6ba006cea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00b40b0 tx=0x7f6b8802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8743a5b2375a51be server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.612+0000 7f6b9e000640 1 -- 172.21.15.73:0/144804483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88031750 con 0x7f6ba00b1b50 2024-04-05T22:21:18.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.612+0000 7f6b9e000640 1 -- 172.21.15.73:0/144804483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b880318b0 con 0x7f6ba00b1b50 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.612+0000 7f6b9e000640 1 -- 172.21.15.73:0/144804483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88031c40 con 0x7f6ba00b1b50 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 -- 172.21.15.73:0/144804483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b1b50 msgr2=0x7f6ba006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/144804483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b1b50 0x7f6ba006cea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00b40b0 tx=0x7f6b8802b7f0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 -- 172.21.15.73:0/144804483 shutdown_connections 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/144804483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b1b50 0x7f6ba006cea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/144804483 >> v2:172.21.15.157:3300/0 conn(0x7f6ba00b20a0 0x7f6ba0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 -- 172.21.15.73:0/144804483 >> 172.21.15.73:0/144804483 conn(0x7f6ba00b06c0 msgr2=0x7f6ba00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 -- 172.21.15.73:0/144804483 shutdown_connections 2024-04-05T22:21:18.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.613+0000 7f6b9fe00640 1 -- 172.21.15.73:0/144804483 wait complete. 2024-04-05T22:21:18.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.615+0000 7f6b9fe00640 1 Processor -- start 2024-04-05T22:21:18.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.615+0000 7f6b9fe00640 1 -- start start 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba00b1b50 0x7f6ba0144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba00b2480 0x7f6ba0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba0159f90 con 0x7f6ba00b20a0 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba0147f20 con 0x7f6ba00b2480 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba015a200 con 0x7f6ba00b1b50 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b96000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b96000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50832/0 (socket says 172.21.15.73:50832) 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b96000640 1 -- 172.21.15.73:0/1119460827 learned_addr learned my addr 172.21.15.73:0/1119460827 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9ea00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba00b1b50 0x7f6ba0144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.616+0000 7f6b9f400640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba00b2480 0x7f6ba0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 -- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba00b1b50 msgr2=0x7f6ba0144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba00b1b50 0x7f6ba0144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 -- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba00b2480 msgr2=0x7f6ba0154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba00b2480 0x7f6ba0154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 -- 172.21.15.73:0/1119460827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880094d0 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b96000640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88033c60 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b88033de0 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8803d6b0 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.617+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba015a4e0 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.618+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba0145520 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.619+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba015aa00 con 0x7f6ba00b20a0 2024-04-05T22:21:18.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.619+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b8803a040 con 0x7f6ba00b20a0 2024-04-05T22:21:18.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.620+0000 7f6b97400640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 0x7f6b7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:18.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.620+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b880ab3e0 con 0x7f6ba00b20a0 2024-04-05T22:21:18.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.621+0000 7f6b9ea00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 0x7f6b7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:18.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.621+0000 7f6b9ea00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 0x7f6b7c06d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009890 tx=0x7f6b90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:18.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.625+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6ba015aa00 con 0x7f6ba00b20a0 2024-04-05T22:21:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.818+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba015aa00 con 0x7f6ba00b20a0 2024-04-05T22:21:18.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.820+0000 7f6b97400640 1 -- 172.21.15.73:0/1119460827 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f6ba015aa00 con 0x7f6ba00b20a0 2024-04-05T22:21:18.821 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:18.821 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 msgr2=0x7f6b7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 0x7f6b7c06d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009890 tx=0x7f6b90008040 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 msgr2=0x7f6ba01546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b880046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 shutdown_connections 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7c06ac00 0x7f6b7c06d0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba00b2480 0x7f6ba0154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ba00b20a0 0x7f6ba01546e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 --2- 172.21.15.73:0/1119460827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba00b1b50 0x7f6ba0144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 >> 172.21.15.73:0/1119460827 conn(0x7f6ba00b06c0 msgr2=0x7f6ba00af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 shutdown_connections 2024-04-05T22:21:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:18.823+0000 7f6b9fe00640 1 -- 172.21.15.73:0/1119460827 wait complete. 2024-04-05T22:21:18.842 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:21:18.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:21:19.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.045+0000 7faeea400640 1 Processor -- start 2024-04-05T22:21:19.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.045+0000 7faeea400640 1 -- start start 2024-04-05T22:21:19.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faeea400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faee413fa60 0x7faee4142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faeea400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 0x7faee41429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faeea400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee4143330 con 0x7faee406b500 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faeea400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee406d3a0 con 0x7faee413fa60 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faeea400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee4143050 con 0x7faee413fe40 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faee9a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faee406b500 0x7faee406f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48732/0 (socket says 172.21.15.73:48732) 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.047+0000 7faee9a00640 1 -- 172.21.15.73:0/3262774199 learned_addr learned my addr 172.21.15.73:0/3262774199 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faedbe00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 0x7faee41429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee9000640 1 --2- 172.21.15.73:0/3262774199 >> v2:172.21.15.157:3300/0 conn(0x7faee413fa60 0x7faee4142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014464013 0 0) 0x7faee4143330 con 0x7faee406b500 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec8003520 con 0x7faee406b500 2024-04-05T22:21:19.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 136116148 0 0) 0x7faec8003520 con 0x7faee406b500 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 msgr2=0x7faee41429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 0x7faee41429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 >> v2:172.21.15.157:3300/0 conn(0x7faee413fa60 msgr2=0x7faee4142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 --2- 172.21.15.73:0/3262774199 >> v2:172.21.15.157:3300/0 conn(0x7faee413fa60 0x7faee4142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40ac970 con 0x7faee406b500 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7faedc0019f0 con 0x7faee406b500 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.048+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 >> v1:172.21.15.73:6789/0 conn(0x7faee406b500 legacy=0x7faee406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8003970 0x7faec8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec80047c0 0x7faec8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 0x7faec8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee406d3a0 con 0x7faec8003970 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4143050 con 0x7faec80047c0 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec800b980 con 0x7faee413fe40 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8003970 0x7faec8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faedbe00640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec80047c0 0x7faec8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 -- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 msgr2=0x7faec8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee413fe40 0x7faec8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 -- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec80047c0 msgr2=0x7faec8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec80047c0 0x7faec8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.049+0000 7faee9000640 1 -- 172.21.15.73:0/3262774199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8003520 con 0x7faec8003970 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.050+0000 7faee9000640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8003970 0x7faec8003d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faed4009ac0 tx=0x7faec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6285365ca591c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed402c710 con 0x7faec8003970 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed402c870 con 0x7faec8003970 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faee3e00640 1 -- 172.21.15.73:0/3262774199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed402cc00 con 0x7faec8003970 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 -- 172.21.15.73:0/3262774199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8003970 msgr2=0x7faec8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 --2- 172.21.15.73:0/3262774199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8003970 0x7faec8003d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faed4009ac0 tx=0x7faec80057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faee9a00640 1 -- 172.21.15.73:0/3262774199 reap_dead start 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 -- 172.21.15.73:0/3262774199 shutdown_connections 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 -- 172.21.15.73:0/3262774199 >> 172.21.15.73:0/3262774199 conn(0x7faee40b06c0 msgr2=0x7faee40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 -- 172.21.15.73:0/3262774199 shutdown_connections 2024-04-05T22:21:19.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.051+0000 7faeea400640 1 -- 172.21.15.73:0/3262774199 wait complete. 2024-04-05T22:21:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.053+0000 7faeea400640 1 Processor -- start 2024-04-05T22:21:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.053+0000 7faeea400640 1 -- start start 2024-04-05T22:21:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec8003970 0x7faee41f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e160 0x7faee40cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41f2440 con 0x7faee40b3250 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40b3c20 con 0x7faec8003970 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faeea400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee415d030 con 0x7faee415e160 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.054+0000 7faee9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50850/0 (socket says 172.21.15.73:50850) 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 -- 172.21.15.73:0/3855487575 learned_addr learned my addr 172.21.15.73:0/3855487575 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9000640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec8003970 0x7faee41f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faedbe00640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e160 0x7faee40cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 -- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e160 msgr2=0x7faee40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e160 0x7faee40cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 -- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec8003970 msgr2=0x7faee41f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec8003970 0x7faee41f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 -- 172.21.15.73:0/3855487575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40cd5c0 con 0x7faee40b3250 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee9a00640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faedc0013d0 tx=0x7faedc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00a6d0 con 0x7faee40b3250 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc00a850 con 0x7faee40b3250 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee415d240 con 0x7faee40b3250 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.055+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00ac00 con 0x7faee40b3250 2024-04-05T22:21:19.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.056+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee415d800 con 0x7faee40b3250 2024-04-05T22:21:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.057+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee41f2580 con 0x7faee40b3250 2024-04-05T22:21:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.058+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faedc00ad60 con 0x7faee40b3250 2024-04-05T22:21:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.058+0000 7faee1600640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 0x7faec006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.059+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faedc0884a0 con 0x7faee40b3250 2024-04-05T22:21:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.059+0000 7faee9000640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 0x7faec006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.059+0000 7faee9000640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 0x7faec006d0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faed4036000 tx=0x7faed4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.063+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7faedc053680 con 0x7faee40b3250 2024-04-05T22:21:19.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.247+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faee40acf30 con 0x7faec006ac30 2024-04-05T22:21:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.266+0000 7faee1600640 1 -- 172.21.15.73:0/3855487575 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264497 (secure 0 0 0) 0x7faed4004040 con 0x7faec006ac30 2024-04-05T22:21:19.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 msgr2=0x7faec006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 0x7faec006d0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faed4036000 tx=0x7faed4030000 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 msgr2=0x7faee40b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faedc0013d0 tx=0x7faedc00cf00 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 shutdown_connections 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faec006ac30 0x7faec006d0f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faee40b3250 0x7faee40b36e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e160 0x7faee40cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 --2- 172.21.15.73:0/3855487575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faec8003970 0x7faee41f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 >> 172.21.15.73:0/3855487575 conn(0x7faee40b06c0 msgr2=0x7faee40ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 shutdown_connections 2024-04-05T22:21:19.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.271+0000 7faeea400640 1 -- 172.21.15.73:0/3855487575 wait complete. 2024-04-05T22:21:19.289 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-05T22:21:18.488848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":3055616,"data_stored":1816372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103198"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982064+0000","last_change":"2024-04-05T22:21:14.923383+0000","last_active":"2024-04-05T22:21:15.982064+0000","last_peered":"2024-04-05T22:21:15.982064+0000","last_clean":"2024-04-05T22:21:15.982064+0000","last_became_active":"2024-04-05T22:21:14.922950+0000","last_became_peered":"2024-04-05T22:21:14.922950+0000","last_unstale":"2024-04-05T22:21:15.982064+0000","last_undegraded":"2024-04-05T22:21:15.982064+0000","last_fullsized":"2024-04-05T22:21:15.982064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900565+0000","last_change":"2024-04-05T22:21:14.912375+0000","last_active":"2024-04-05T22:21:15.900565+0000","last_peered":"2024-04-05T22:21:15.900565+0000","last_clean":"2024-04-05T22:21:15.900565+0000","last_became_active":"2024-04-05T22:21:14.912009+0000","last_became_peered":"2024-04-05T22:21:14.912009+0000","last_unstale":"2024-04-05T22:21:15.900565+0000","last_undegraded":"2024-04-05T22:21:15.900565+0000","last_fullsized":"2024-04-05T22:21:15.900565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976623+0000","last_change":"2024-04-05T22:21:14.914402+0000","last_active":"2024-04-05T22:21:15.976623+0000","last_peered":"2024-04-05T22:21:15.976623+0000","last_clean":"2024-04-05T22:21:15.976623+0000","last_became_active":"2024-04-05T22:21:14.914120+0000","last_became_peered":"2024-04-05T22:21:14.914120+0000","last_unstale":"2024-04-05T22:21:15.976623+0000","last_undegraded":"2024-04-05T22:21:15.976623+0000","last_fullsized":"2024-04-05T22:21:15.976623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899962+0000","last_change":"2024-04-05T22:21:14.921662+0000","last_active":"2024-04-05T22:21:15.899962+0000","last_peered":"2024-04-05T22:21:15.899962+0000","last_clean":"2024-04-05T22:21:15.899962+0000","last_became_active":"2024-04-05T22:21:14.921218+0000","last_became_peered":"2024-04-05T22:21:14.921218+0000","last_unstale":"2024-04-05T22:21:15.899962+0000","last_undegraded":"2024-04-05T22:21:15.899962+0000","last_fullsized":"2024-04-05T22:21:15.899962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979973+0000","last_change":"2024-04-05T22:21:14.935767+0000","last_active":"2024-04-05T22:21:15.979973+0000","last_peered":"2024-04-05T22:21:15.979973+0000","last_clean":"2024-04-05T22:21:15.979973+0000","last_became_active":"2024-04-05T22:21:14.935145+0000","last_became_peered":"2024-04-05T22:21:14.935145+0000","last_unstale":"2024-04-05T22:21:15.979973+0000","last_undegraded":"2024-04-05T22:21:15.979973+0000","last_fullsized":"2024-04-05T22:21:15.979973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898816+0000","last_change":"2024-04-05T22:21:14.923031+0000","last_active":"2024-04-05T22:21:15.898816+0000","last_peered":"2024-04-05T22:21:15.898816+0000","last_clean":"2024-04-05T22:21:15.898816+0000","last_became_active":"2024-04-05T22:21:14.922585+0000","last_became_peered":"2024-04-05T22:21:14.922585+0000","last_unstale":"2024-04-05T22:21:15.898816+0000","last_undegraded":"2024-04-05T22:21:15.898816+0000","last_fullsized":"2024-04-05T22:21:15.898816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979244+0000","last_change":"2024-04-05T22:21:14.925099+0000","last_active":"2024-04-05T22:21:15.979244+0000","last_peered":"2024-04-05T22:21:15.979244+0000","last_clean":"2024-04-05T22:21:15.979244+0000","last_became_active":"2024-04-05T22:21:14.921678+0000","last_became_peered":"2024-04-05T22:21:14.921678+0000","last_unstale":"2024-04-05T22:21:15.979244+0000","last_undegraded":"2024-04-05T22:21:15.979244+0000","last_fullsized":"2024-04-05T22:21:15.979244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975561+0000","last_change":"2024-04-05T22:21:14.916195+0000","last_active":"2024-04-05T22:21:15.975561+0000","last_peered":"2024-04-05T22:21:15.975561+0000","last_clean":"2024-04-05T22:21:15.975561+0000","last_became_active":"2024-04-05T22:21:14.915947+0000","last_became_peered":"2024-04-05T22:21:14.915947+0000","last_unstale":"2024-04-05T22:21:15.975561+0000","last_undegraded":"2024-04-05T22:21:15.975561+0000","last_fullsized":"2024-04-05T22:21:15.975561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982147+0000","last_change":"2024-04-05T22:21:14.931207+0000","last_active":"2024-04-05T22:21:15.982147+0000","last_peered":"2024-04-05T22:21:15.982147+0000","last_clean":"2024-04-05T22:21:15.982147+0000","last_became_active":"2024-04-05T22:21:14.930509+0000","last_became_peered":"2024-04-05T22:21:14.930509+0000","last_unstale":"2024-04-05T22:21:15.982147+0000","last_undegraded":"2024-04-05T22:21:15.982147+0000","last_fullsized":"2024-04-05T22:21:15.982147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981506+0000","last_change":"2024-04-05T22:21:14.916586+0000","last_active":"2024-04-05T22:21:15.981506+0000","last_peered":"2024-04-05T22:21:15.981506+0000","last_clean":"2024-04-05T22:21:15.981506+0000","last_became_active":"2024-04-05T22:21:14.916004+0000","last_became_peered":"2024-04-05T22:21:14.916004+0000","last_unstale":"2024-04-05T22:21:15.981506+0000","last_undegraded":"2024-04-05T22:21:15.981506+0000","last_fullsized":"2024-04-05T22:21:15.981506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:02.978295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981439+0000","last_change":"2024-04-05T22:21:14.913056+0000","last_active":"2024-04-05T22:21:15.981439+0000","last_peered":"2024-04-05T22:21:15.981439+0000","last_clean":"2024-04-05T22:21:15.981439+0000","last_became_active":"2024-04-05T22:21:14.912748+0000","last_became_peered":"2024-04-05T22:21:14.912748+0000","last_unstale":"2024-04-05T22:21:15.981439+0000","last_undegraded":"2024-04-05T22:21:15.981439+0000","last_fullsized":"2024-04-05T22:21:15.981439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:11:59.394291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983331+0000","last_change":"2024-04-05T22:21:14.926927+0000","last_active":"2024-04-05T22:21:15.983331+0000","last_peered":"2024-04-05T22:21:15.983331+0000","last_clean":"2024-04-05T22:21:15.983331+0000","last_became_active":"2024-04-05T22:21:14.913316+0000","last_became_peered":"2024-04-05T22:21:14.913316+0000","last_unstale":"2024-04-05T22:21:15.983331+0000","last_undegraded":"2024-04-05T22:21:15.983331+0000","last_fullsized":"2024-04-05T22:21:15.983331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:14.865832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976116+0000","last_change":"2024-04-05T22:21:14.919462+0000","last_active":"2024-04-05T22:21:15.976116+0000","last_peered":"2024-04-05T22:21:15.976116+0000","last_clean":"2024-04-05T22:21:15.976116+0000","last_became_active":"2024-04-05T22:21:14.919180+0000","last_became_peered":"2024-04-05T22:21:14.919180+0000","last_unstale":"2024-04-05T22:21:15.976116+0000","last_undegraded":"2024-04-05T22:21:15.976116+0000","last_fullsized":"2024-04-05T22:21:15.976116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975694+0000","last_change":"2024-04-05T22:21:14.912391+0000","last_active":"2024-04-05T22:21:15.975694+0000","last_peered":"2024-04-05T22:21:15.975694+0000","last_clean":"2024-04-05T22:21:15.975694+0000","last_became_active":"2024-04-05T22:21:14.912013+0000","last_became_peered":"2024-04-05T22:21:14.912013+0000","last_unstale":"2024-04-05T22:21:15.975694+0000","last_undegraded":"2024-04-05T22:21:15.975694+0000","last_fullsized":"2024-04-05T22:21:15.975694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899930+0000","last_change":"2024-04-05T22:21:14.900459+0000","last_active":"2024-04-05T22:21:15.899930+0000","last_peered":"2024-04-05T22:21:15.899930+0000","last_clean":"2024-04-05T22:21:15.899930+0000","last_became_active":"2024-04-05T22:21:14.900144+0000","last_became_peered":"2024-04-05T22:21:14.900144+0000","last_unstale":"2024-04-05T22:21:15.899930+0000","last_undegraded":"2024-04-05T22:21:15.899930+0000","last_fullsized":"2024-04-05T22:21:15.899930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898646+0000","last_change":"2024-04-05T22:21:14.913235+0000","last_active":"2024-04-05T22:21:15.898646+0000","last_peered":"2024-04-05T22:21:15.898646+0000","last_clean":"2024-04-05T22:21:15.898646+0000","last_became_active":"2024-04-05T22:21:14.912913+0000","last_became_peered":"2024-04-05T22:21:14.912913+0000","last_unstale":"2024-04-05T22:21:15.898646+0000","last_undegraded":"2024-04-05T22:21:15.898646+0000","last_fullsized":"2024-04-05T22:21:15.898646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900293+0000","last_change":"2024-04-05T22:21:14.931728+0000","last_active":"2024-04-05T22:21:15.900293+0000","last_peered":"2024-04-05T22:21:15.900293+0000","last_clean":"2024-04-05T22:21:15.900293+0000","last_became_active":"2024-04-05T22:21:14.931500+0000","last_became_peered":"2024-04-05T22:21:14.931500+0000","last_unstale":"2024-04-05T22:21:15.900293+0000","last_undegraded":"2024-04-05T22:21:15.900293+0000","last_fullsized":"2024-04-05T22:21:15.900293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897627+0000","last_change":"2024-04-05T22:21:14.921738+0000","last_active":"2024-04-05T22:21:15.897627+0000","last_peered":"2024-04-05T22:21:15.897627+0000","last_clean":"2024-04-05T22:21:15.897627+0000","last_became_active":"2024-04-05T22:21:14.919781+0000","last_became_peered":"2024-04-05T22:21:14.919781+0000","last_unstale":"2024-04-05T22:21:15.897627+0000","last_undegraded":"2024-04-05T22:21:15.897627+0000","last_fullsized":"2024-04-05T22:21:15.897627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981513+0000","last_change":"2024-04-05T22:21:14.920265+0000","last_active":"2024-04-05T22:21:15.981513+0000","last_peered":"2024-04-05T22:21:15.981513+0000","last_clean":"2024-04-05T22:21:15.981513+0000","last_became_active":"2024-04-05T22:21:14.919985+0000","last_became_peered":"2024-04-05T22:21:14.919985+0000","last_unstale":"2024-04-05T22:21:15.981513+0000","last_undegraded":"2024-04-05T22:21:15.981513+0000","last_fullsized":"2024-04-05T22:21:15.981513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981857+0000","last_change":"2024-04-05T22:21:14.929527+0000","last_active":"2024-04-05T22:21:15.981857+0000","last_peered":"2024-04-05T22:21:15.981857+0000","last_clean":"2024-04-05T22:21:15.981857+0000","last_became_active":"2024-04-05T22:21:14.929222+0000","last_became_peered":"2024-04-05T22:21:14.929222+0000","last_unstale":"2024-04-05T22:21:15.981857+0000","last_undegraded":"2024-04-05T22:21:15.981857+0000","last_fullsized":"2024-04-05T22:21:15.981857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:15:04.491398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975985+0000","last_change":"2024-04-05T22:21:14.933523+0000","last_active":"2024-04-05T22:21:15.975985+0000","last_peered":"2024-04-05T22:21:15.975985+0000","last_clean":"2024-04-05T22:21:15.975985+0000","last_became_active":"2024-04-05T22:21:14.933340+0000","last_became_peered":"2024-04-05T22:21:14.933340+0000","last_unstale":"2024-04-05T22:21:15.975985+0000","last_undegraded":"2024-04-05T22:21:15.975985+0000","last_fullsized":"2024-04-05T22:21:15.975985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898875+0000","last_change":"2024-04-05T22:21:14.933905+0000","last_active":"2024-04-05T22:21:15.898875+0000","last_peered":"2024-04-05T22:21:15.898875+0000","last_clean":"2024-04-05T22:21:15.898875+0000","last_became_active":"2024-04-05T22:21:14.933507+0000","last_became_peered":"2024-04-05T22:21:14.933507+0000","last_unstale":"2024-04-05T22:21:15.898875+0000","last_undegraded":"2024-04-05T22:21:15.898875+0000","last_fullsized":"2024-04-05T22:21:15.898875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975831+0000","last_change":"2024-04-05T22:21:14.915842+0000","last_active":"2024-04-05T22:21:15.975831+0000","last_peered":"2024-04-05T22:21:15.975831+0000","last_clean":"2024-04-05T22:21:15.975831+0000","last_became_active":"2024-04-05T22:21:14.915391+0000","last_became_peered":"2024-04-05T22:21:14.915391+0000","last_unstale":"2024-04-05T22:21:15.975831+0000","last_undegraded":"2024-04-05T22:21:15.975831+0000","last_fullsized":"2024-04-05T22:21:15.975831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:21.773200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898993+0000","last_change":"2024-04-05T22:21:14.924789+0000","last_active":"2024-04-05T22:21:15.898993+0000","last_peered":"2024-04-05T22:21:15.898993+0000","last_clean":"2024-04-05T22:21:15.898993+0000","last_became_active":"2024-04-05T22:21:14.924488+0000","last_became_peered":"2024-04-05T22:21:14.924488+0000","last_unstale":"2024-04-05T22:21:15.898993+0000","last_undegraded":"2024-04-05T22:21:15.898993+0000","last_fullsized":"2024-04-05T22:21:15.898993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976451+0000","last_change":"2024-04-05T22:21:14.926693+0000","last_active":"2024-04-05T22:21:15.976451+0000","last_peered":"2024-04-05T22:21:15.976451+0000","last_clean":"2024-04-05T22:21:15.976451+0000","last_became_active":"2024-04-05T22:21:14.926415+0000","last_became_peered":"2024-04-05T22:21:14.926415+0000","last_unstale":"2024-04-05T22:21:15.976451+0000","last_undegraded":"2024-04-05T22:21:15.976451+0000","last_fullsized":"2024-04-05T22:21:15.976451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981744+0000","last_change":"2024-04-05T22:21:14.940178+0000","last_active":"2024-04-05T22:21:15.981744+0000","last_peered":"2024-04-05T22:21:15.981744+0000","last_clean":"2024-04-05T22:21:15.981744+0000","last_became_active":"2024-04-05T22:21:14.938042+0000","last_became_peered":"2024-04-05T22:21:14.938042+0000","last_unstale":"2024-04-05T22:21:15.981744+0000","last_undegraded":"2024-04-05T22:21:15.981744+0000","last_fullsized":"2024-04-05T22:21:15.981744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:42.750931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900215+0000","last_change":"2024-04-05T22:21:14.916877+0000","last_active":"2024-04-05T22:21:15.900215+0000","last_peered":"2024-04-05T22:21:15.900215+0000","last_clean":"2024-04-05T22:21:15.900215+0000","last_became_active":"2024-04-05T22:21:14.916585+0000","last_became_peered":"2024-04-05T22:21:14.916585+0000","last_unstale":"2024-04-05T22:21:15.900215+0000","last_undegraded":"2024-04-05T22:21:15.900215+0000","last_fullsized":"2024-04-05T22:21:15.900215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976229+0000","last_change":"2024-04-05T22:21:14.918894+0000","last_active":"2024-04-05T22:21:15.976229+0000","last_peered":"2024-04-05T22:21:15.976229+0000","last_clean":"2024-04-05T22:21:15.976229+0000","last_became_active":"2024-04-05T22:21:14.918074+0000","last_became_peered":"2024-04-05T22:21:14.918074+0000","last_unstale":"2024-04-05T22:21:15.976229+0000","last_undegraded":"2024-04-05T22:21:15.976229+0000","last_fullsized":"2024-04-05T22:21:15.976229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976071+0000","last_change":"2024-04-05T22:21:14.922850+0000","last_active":"2024-04-05T22:21:15.976071+0000","last_peered":"2024-04-05T22:21:15.976071+0000","last_clean":"2024-04-05T22:21:15.976071+0000","last_became_active":"2024-04-05T22:21:14.920770+0000","last_became_peered":"2024-04-05T22:21:14.920770+0000","last_unstale":"2024-04-05T22:21:15.976071+0000","last_undegraded":"2024-04-05T22:21:15.976071+0000","last_fullsized":"2024-04-05T22:21:15.976071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899082+0000","last_change":"2024-04-05T22:21:14.917603+0000","last_active":"2024-04-05T22:21:15.899082+0000","last_peered":"2024-04-05T22:21:15.899082+0000","last_clean":"2024-04-05T22:21:15.899082+0000","last_became_active":"2024-04-05T22:21:14.917325+0000","last_became_peered":"2024-04-05T22:21:14.917325+0000","last_unstale":"2024-04-05T22:21:15.899082+0000","last_undegraded":"2024-04-05T22:21:15.899082+0000","last_fullsized":"2024-04-05T22:21:15.899082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:46.139845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.896840+0000","last_change":"2024-04-05T22:21:14.911372+0000","last_active":"2024-04-05T22:21:15.896840+0000","last_peered":"2024-04-05T22:21:15.896840+0000","last_clean":"2024-04-05T22:21:15.896840+0000","last_became_active":"2024-04-05T22:21:14.911083+0000","last_became_peered":"2024-04-05T22:21:14.911083+0000","last_unstale":"2024-04-05T22:21:15.896840+0000","last_undegraded":"2024-04-05T22:21:15.896840+0000","last_fullsized":"2024-04-05T22:21:15.896840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979172+0000","last_change":"2024-04-05T22:21:14.915897+0000","last_active":"2024-04-05T22:21:15.979172+0000","last_peered":"2024-04-05T22:21:15.979172+0000","last_clean":"2024-04-05T22:21:15.979172+0000","last_became_active":"2024-04-05T22:21:14.915223+0000","last_became_peered":"2024-04-05T22:21:14.915223+0000","last_unstale":"2024-04-05T22:21:15.979172+0000","last_undegraded":"2024-04-05T22:21:15.979172+0000","last_fullsized":"2024-04-05T22:21:15.979172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:19:20.576451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982664+0000","last_change":"2024-04-05T22:21:14.901329+0000","last_active":"2024-04-05T22:21:15.982664+0000","last_peered":"2024-04-05T22:21:15.982664+0000","last_clean":"2024-04-05T22:21:15.982664+0000","last_became_active":"2024-04-05T22:21:14.900870+0000","last_became_peered":"2024-04-05T22:21:14.900870+0000","last_unstale":"2024-04-05T22:21:15.982664+0000","last_undegraded":"2024-04-05T22:21:15.982664+0000","last_fullsized":"2024-04-05T22:21:15.982664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:12.554588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976297+0000","last_change":"2024-04-05T22:21:14.437934+0000","last_active":"2024-04-05T22:21:15.976297+0000","last_peered":"2024-04-05T22:21:15.976297+0000","last_clean":"2024-04-05T22:21:15.976297+0000","last_became_active":"2024-04-05T22:21:14.437696+0000","last_became_peered":"2024-04-05T22:21:14.437696+0000","last_unstale":"2024-04-05T22:21:15.976297+0000","last_undegraded":"2024-04-05T22:21:15.976297+0000","last_fullsized":"2024-04-05T22:21:15.976297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976230+0000","last_change":"2024-04-05T22:21:14.920473+0000","last_active":"2024-04-05T22:21:15.976230+0000","last_peered":"2024-04-05T22:21:15.976230+0000","last_clean":"2024-04-05T22:21:15.976230+0000","last_became_active":"2024-04-05T22:21:14.917453+0000","last_became_peered":"2024-04-05T22:21:14.917453+0000","last_unstale":"2024-04-05T22:21:15.976230+0000","last_undegraded":"2024-04-05T22:21:15.976230+0000","last_fullsized":"2024-04-05T22:21:15.976230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980837+0000","last_change":"2024-04-05T22:21:14.449143+0000","last_active":"2024-04-05T22:21:15.980837+0000","last_peered":"2024-04-05T22:21:15.980837+0000","last_clean":"2024-04-05T22:21:15.980837+0000","last_became_active":"2024-04-05T22:21:14.448341+0000","last_became_peered":"2024-04-05T22:21:14.448341+0000","last_unstale":"2024-04-05T22:21:15.980837+0000","last_undegraded":"2024-04-05T22:21:15.980837+0000","last_fullsized":"2024-04-05T22:21:15.980837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:14.544598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975917+0000","last_change":"2024-04-05T22:21:14.441239+0000","last_active":"2024-04-05T22:21:15.975917+0000","last_peered":"2024-04-05T22:21:15.975917+0000","last_clean":"2024-04-05T22:21:15.975917+0000","last_became_active":"2024-04-05T22:21:14.440898+0000","last_became_peered":"2024-04-05T22:21:14.440898+0000","last_unstale":"2024-04-05T22:21:15.975917+0000","last_undegraded":"2024-04-05T22:21:15.975917+0000","last_fullsized":"2024-04-05T22:21:15.975917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975636+0000","last_change":"2024-04-05T22:21:14.922371+0000","last_active":"2024-04-05T22:21:15.975636+0000","last_peered":"2024-04-05T22:21:15.975636+0000","last_clean":"2024-04-05T22:21:15.975636+0000","last_became_active":"2024-04-05T22:21:14.922074+0000","last_became_peered":"2024-04-05T22:21:14.922074+0000","last_unstale":"2024-04-05T22:21:15.975636+0000","last_undegraded":"2024-04-05T22:21:15.975636+0000","last_fullsized":"2024-04-05T22:21:15.975636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898990+0000","last_change":"2024-04-05T22:21:14.446427+0000","last_active":"2024-04-05T22:21:15.898990+0000","last_peered":"2024-04-05T22:21:15.898990+0000","last_clean":"2024-04-05T22:21:15.898990+0000","last_became_active":"2024-04-05T22:21:14.445487+0000","last_became_peered":"2024-04-05T22:21:14.445487+0000","last_unstale":"2024-04-05T22:21:15.898990+0000","last_undegraded":"2024-04-05T22:21:15.898990+0000","last_fullsized":"2024-04-05T22:21:15.898990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899850+0000","last_change":"2024-04-05T22:21:14.917346+0000","last_active":"2024-04-05T22:21:15.899850+0000","last_peered":"2024-04-05T22:21:15.899850+0000","last_clean":"2024-04-05T22:21:15.899850+0000","last_became_active":"2024-04-05T22:21:14.917085+0000","last_became_peered":"2024-04-05T22:21:14.917085+0000","last_unstale":"2024-04-05T22:21:15.899850+0000","last_undegraded":"2024-04-05T22:21:15.899850+0000","last_fullsized":"2024-04-05T22:21:15.899850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981190+0000","last_change":"2024-04-05T22:21:14.441415+0000","last_active":"2024-04-05T22:21:15.981190+0000","last_peered":"2024-04-05T22:21:15.981190+0000","last_clean":"2024-04-05T22:21:15.981190+0000","last_became_active":"2024-04-05T22:21:14.440493+0000","last_became_peered":"2024-04-05T22:21:14.440493+0000","last_unstale":"2024-04-05T22:21:15.981190+0000","last_undegraded":"2024-04-05T22:21:15.981190+0000","last_fullsized":"2024-04-05T22:21:15.981190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:25.845173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899687+0000","last_change":"2024-04-05T22:21:14.932959+0000","last_active":"2024-04-05T22:21:15.899687+0000","last_peered":"2024-04-05T22:21:15.899687+0000","last_clean":"2024-04-05T22:21:15.899687+0000","last_became_active":"2024-04-05T22:21:14.932656+0000","last_became_peered":"2024-04-05T22:21:14.932656+0000","last_unstale":"2024-04-05T22:21:15.899687+0000","last_undegraded":"2024-04-05T22:21:15.899687+0000","last_fullsized":"2024-04-05T22:21:15.899687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899902+0000","last_change":"2024-04-05T22:21:14.444991+0000","last_active":"2024-04-05T22:21:15.899902+0000","last_peered":"2024-04-05T22:21:15.899902+0000","last_clean":"2024-04-05T22:21:15.899902+0000","last_became_active":"2024-04-05T22:21:14.444667+0000","last_became_peered":"2024-04-05T22:21:14.444667+0000","last_unstale":"2024-04-05T22:21:15.899902+0000","last_undegraded":"2024-04-05T22:21:15.899902+0000","last_fullsized":"2024-04-05T22:21:15.899902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.915276+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.914411+0000","last_became_peered":"2024-04-05T22:21:14.914411+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900261+0000","last_change":"2024-04-05T22:21:14.438342+0000","last_active":"2024-04-05T22:21:15.900261+0000","last_peered":"2024-04-05T22:21:15.900261+0000","last_clean":"2024-04-05T22:21:15.900261+0000","last_became_active":"2024-04-05T22:21:14.438070+0000","last_became_peered":"2024-04-05T22:21:14.438070+0000","last_unstale":"2024-04-05T22:21:15.900261+0000","last_undegraded":"2024-04-05T22:21:15.900261+0000","last_fullsized":"2024-04-05T22:21:15.900261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:13.607794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983525+0000","last_change":"2024-04-05T22:21:14.928911+0000","last_active":"2024-04-05T22:21:15.983525+0000","last_peered":"2024-04-05T22:21:15.983525+0000","last_clean":"2024-04-05T22:21:15.983525+0000","last_became_active":"2024-04-05T22:21:14.928657+0000","last_became_peered":"2024-04-05T22:21:14.928657+0000","last_unstale":"2024-04-05T22:21:15.983525+0000","last_undegraded":"2024-04-05T22:21:15.983525+0000","last_fullsized":"2024-04-05T22:21:15.983525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982110+0000","last_change":"2024-04-05T22:21:14.442377+0000","last_active":"2024-04-05T22:21:15.982110+0000","last_peered":"2024-04-05T22:21:15.982110+0000","last_clean":"2024-04-05T22:21:15.982110+0000","last_became_active":"2024-04-05T22:21:14.441914+0000","last_became_peered":"2024-04-05T22:21:14.441914+0000","last_unstale":"2024-04-05T22:21:15.982110+0000","last_undegraded":"2024-04-05T22:21:15.982110+0000","last_fullsized":"2024-04-05T22:21:15.982110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:53.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980822+0000","last_change":"2024-04-05T22:21:14.919936+0000","last_active":"2024-04-05T22:21:15.980822+0000","last_peered":"2024-04-05T22:21:15.980822+0000","last_clean":"2024-04-05T22:21:15.980822+0000","last_became_active":"2024-04-05T22:21:14.919602+0000","last_became_peered":"2024-04-05T22:21:14.919602+0000","last_unstale":"2024-04-05T22:21:15.980822+0000","last_undegraded":"2024-04-05T22:21:15.980822+0000","last_fullsized":"2024-04-05T22:21:15.980822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976328+0000","last_change":"2024-04-05T22:21:14.437206+0000","last_active":"2024-04-05T22:21:15.976328+0000","last_peered":"2024-04-05T22:21:15.976328+0000","last_clean":"2024-04-05T22:21:15.976328+0000","last_became_active":"2024-04-05T22:21:14.436852+0000","last_became_peered":"2024-04-05T22:21:14.436852+0000","last_unstale":"2024-04-05T22:21:15.976328+0000","last_undegraded":"2024-04-05T22:21:15.976328+0000","last_fullsized":"2024-04-05T22:21:15.976328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975906+0000","last_change":"2024-04-05T22:21:14.933572+0000","last_active":"2024-04-05T22:21:15.975906+0000","last_peered":"2024-04-05T22:21:15.975906+0000","last_clean":"2024-04-05T22:21:15.975906+0000","last_became_active":"2024-04-05T22:21:14.933310+0000","last_became_peered":"2024-04-05T22:21:14.933310+0000","last_unstale":"2024-04-05T22:21:15.975906+0000","last_undegraded":"2024-04-05T22:21:15.975906+0000","last_fullsized":"2024-04-05T22:21:15.975906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:10:27.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899122+0000","last_change":"2024-04-05T22:21:14.435387+0000","last_active":"2024-04-05T22:21:15.899122+0000","last_peered":"2024-04-05T22:21:15.899122+0000","last_clean":"2024-04-05T22:21:15.899122+0000","last_became_active":"2024-04-05T22:21:14.435061+0000","last_became_peered":"2024-04-05T22:21:14.435061+0000","last_unstale":"2024-04-05T22:21:15.899122+0000","last_undegraded":"2024-04-05T22:21:15.899122+0000","last_fullsized":"2024-04-05T22:21:15.899122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980213+0000","last_change":"2024-04-05T22:21:14.935722+0000","last_active":"2024-04-05T22:21:15.980213+0000","last_peered":"2024-04-05T22:21:15.980213+0000","last_clean":"2024-04-05T22:21:15.980213+0000","last_became_active":"2024-04-05T22:21:14.935463+0000","last_became_peered":"2024-04-05T22:21:14.935463+0000","last_unstale":"2024-04-05T22:21:15.980213+0000","last_undegraded":"2024-04-05T22:21:15.980213+0000","last_fullsized":"2024-04-05T22:21:15.980213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980372+0000","last_change":"2024-04-05T22:21:14.449636+0000","last_active":"2024-04-05T22:21:15.980372+0000","last_peered":"2024-04-05T22:21:15.980372+0000","last_clean":"2024-04-05T22:21:15.980372+0000","last_became_active":"2024-04-05T22:21:14.449257+0000","last_became_peered":"2024-04-05T22:21:14.449257+0000","last_unstale":"2024-04-05T22:21:15.980372+0000","last_undegraded":"2024-04-05T22:21:15.980372+0000","last_fullsized":"2024-04-05T22:21:15.980372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:56:39.430553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980094+0000","last_change":"2024-04-05T22:21:14.916105+0000","last_active":"2024-04-05T22:21:15.980094+0000","last_peered":"2024-04-05T22:21:15.980094+0000","last_clean":"2024-04-05T22:21:15.980094+0000","last_became_active":"2024-04-05T22:21:14.915590+0000","last_became_peered":"2024-04-05T22:21:14.915590+0000","last_unstale":"2024-04-05T22:21:15.980094+0000","last_undegraded":"2024-04-05T22:21:15.980094+0000","last_fullsized":"2024-04-05T22:21:15.980094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:21:38.340157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897721+0000","last_change":"2024-04-05T22:21:14.429654+0000","last_active":"2024-04-05T22:21:15.897721+0000","last_peered":"2024-04-05T22:21:15.897721+0000","last_clean":"2024-04-05T22:21:15.897721+0000","last_became_active":"2024-04-05T22:21:14.428871+0000","last_became_peered":"2024-04-05T22:21:14.428871+0000","last_unstale":"2024-04-05T22:21:15.897721+0000","last_undegraded":"2024-04-05T22:21:15.897721+0000","last_fullsized":"2024-04-05T22:21:15.897721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979204+0000","last_change":"2024-04-05T22:21:14.921522+0000","last_active":"2024-04-05T22:21:15.979204+0000","last_peered":"2024-04-05T22:21:15.979204+0000","last_clean":"2024-04-05T22:21:15.979204+0000","last_became_active":"2024-04-05T22:21:14.921331+0000","last_became_peered":"2024-04-05T22:21:14.921331+0000","last_unstale":"2024-04-05T22:21:15.979204+0000","last_undegraded":"2024-04-05T22:21:15.979204+0000","last_fullsized":"2024-04-05T22:21:15.979204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983583+0000","last_change":"2024-04-05T22:21:14.437535+0000","last_active":"2024-04-05T22:21:15.983583+0000","last_peered":"2024-04-05T22:21:15.983583+0000","last_clean":"2024-04-05T22:21:15.983583+0000","last_became_active":"2024-04-05T22:21:14.437228+0000","last_became_peered":"2024-04-05T22:21:14.437228+0000","last_unstale":"2024-04-05T22:21:15.983583+0000","last_undegraded":"2024-04-05T22:21:15.983583+0000","last_fullsized":"2024-04-05T22:21:15.983583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981661+0000","last_change":"2024-04-05T22:21:14.935527+0000","last_active":"2024-04-05T22:21:15.981661+0000","last_peered":"2024-04-05T22:21:15.981661+0000","last_clean":"2024-04-05T22:21:15.981661+0000","last_became_active":"2024-04-05T22:21:14.935121+0000","last_became_peered":"2024-04-05T22:21:14.935121+0000","last_unstale":"2024-04-05T22:21:15.981661+0000","last_undegraded":"2024-04-05T22:21:15.981661+0000","last_fullsized":"2024-04-05T22:21:15.981661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897618+0000","last_change":"2024-04-05T22:21:14.434360+0000","last_active":"2024-04-05T22:21:15.897618+0000","last_peered":"2024-04-05T22:21:15.897618+0000","last_clean":"2024-04-05T22:21:15.897618+0000","last_became_active":"2024-04-05T22:21:14.434064+0000","last_became_peered":"2024-04-05T22:21:14.434064+0000","last_unstale":"2024-04-05T22:21:15.897618+0000","last_undegraded":"2024-04-05T22:21:15.897618+0000","last_fullsized":"2024-04-05T22:21:15.897618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982201+0000","last_change":"2024-04-05T22:21:14.931694+0000","last_active":"2024-04-05T22:21:15.982201+0000","last_peered":"2024-04-05T22:21:15.982201+0000","last_clean":"2024-04-05T22:21:15.982201+0000","last_became_active":"2024-04-05T22:21:14.931452+0000","last_became_peered":"2024-04-05T22:21:14.931452+0000","last_unstale":"2024-04-05T22:21:15.982201+0000","last_undegraded":"2024-04-05T22:21:15.982201+0000","last_fullsized":"2024-04-05T22:21:15.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900462+0000","last_change":"2024-04-05T22:21:14.431053+0000","last_active":"2024-04-05T22:21:15.900462+0000","last_peered":"2024-04-05T22:21:15.900462+0000","last_clean":"2024-04-05T22:21:15.900462+0000","last_became_active":"2024-04-05T22:21:14.430719+0000","last_became_peered":"2024-04-05T22:21:14.430719+0000","last_unstale":"2024-04-05T22:21:15.900462+0000","last_undegraded":"2024-04-05T22:21:15.900462+0000","last_fullsized":"2024-04-05T22:21:15.900462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:02.241437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899853+0000","last_change":"2024-04-05T22:21:14.921494+0000","last_active":"2024-04-05T22:21:15.899853+0000","last_peered":"2024-04-05T22:21:15.899853+0000","last_clean":"2024-04-05T22:21:15.899853+0000","last_became_active":"2024-04-05T22:21:14.920872+0000","last_became_peered":"2024-04-05T22:21:14.920872+0000","last_unstale":"2024-04-05T22:21:15.899853+0000","last_undegraded":"2024-04-05T22:21:15.899853+0000","last_fullsized":"2024-04-05T22:21:15.899853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897333+0000","last_change":"2024-04-05T22:21:14.439971+0000","last_active":"2024-04-05T22:21:15.897333+0000","last_peered":"2024-04-05T22:21:15.897333+0000","last_clean":"2024-04-05T22:21:15.897333+0000","last_became_active":"2024-04-05T22:21:14.439620+0000","last_became_peered":"2024-04-05T22:21:14.439620+0000","last_unstale":"2024-04-05T22:21:15.897333+0000","last_undegraded":"2024-04-05T22:21:15.897333+0000","last_fullsized":"2024-04-05T22:21:15.897333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981058+0000","last_change":"2024-04-05T22:21:14.930698+0000","last_active":"2024-04-05T22:21:15.981058+0000","last_peered":"2024-04-05T22:21:15.981058+0000","last_clean":"2024-04-05T22:21:15.981058+0000","last_became_active":"2024-04-05T22:21:14.930442+0000","last_became_peered":"2024-04-05T22:21:14.930442+0000","last_unstale":"2024-04-05T22:21:15.981058+0000","last_undegraded":"2024-04-05T22:21:15.981058+0000","last_fullsized":"2024-04-05T22:21:15.981058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-05T22:21:14.902603+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:21:14.902603+0000","last_peered":"2024-04-05T22:21:14.902603+0000","last_clean":"2024-04-05T22:21:14.902603+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:21:14.902603+0000","last_undegraded":"2024-04-05T22:21:14.902603+0000","last_fullsized":"2024-04-05T22:21:14.902603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898869+0000","last_change":"2024-04-05T22:21:14.440806+0000","last_active":"2024-04-05T22:21:15.898869+0000","last_peered":"2024-04-05T22:21:15.898869+0000","last_clean":"2024-04-05T22:21:15.898869+0000","last_became_active":"2024-04-05T22:21:14.439437+0000","last_became_peered":"2024-04-05T22:21:14.439437+0000","last_unstale":"2024-04-05T22:21:15.898869+0000","last_undegraded":"2024-04-05T22:21:15.898869+0000","last_fullsized":"2024-04-05T22:21:15.898869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976007+0000","last_change":"2024-04-05T22:21:14.919101+0000","last_active":"2024-04-05T22:21:15.976007+0000","last_peered":"2024-04-05T22:21:15.976007+0000","last_clean":"2024-04-05T22:21:15.976007+0000","last_became_active":"2024-04-05T22:21:14.916214+0000","last_became_peered":"2024-04-05T22:21:14.916214+0000","last_unstale":"2024-04-05T22:21:15.976007+0000","last_undegraded":"2024-04-05T22:21:15.976007+0000","last_fullsized":"2024-04-05T22:21:15.976007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979251+0000","last_change":"2024-04-05T22:21:14.438539+0000","last_active":"2024-04-05T22:21:15.979251+0000","last_peered":"2024-04-05T22:21:15.979251+0000","last_clean":"2024-04-05T22:21:15.979251+0000","last_became_active":"2024-04-05T22:21:14.437939+0000","last_became_peered":"2024-04-05T22:21:14.437939+0000","last_unstale":"2024-04-05T22:21:15.979251+0000","last_undegraded":"2024-04-05T22:21:15.979251+0000","last_fullsized":"2024-04-05T22:21:15.979251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975766+0000","last_change":"2024-04-05T22:21:14.916601+0000","last_active":"2024-04-05T22:21:15.975766+0000","last_peered":"2024-04-05T22:21:15.975766+0000","last_clean":"2024-04-05T22:21:15.975766+0000","last_became_active":"2024-04-05T22:21:14.916330+0000","last_became_peered":"2024-04-05T22:21:14.916330+0000","last_unstale":"2024-04-05T22:21:15.975766+0000","last_undegraded":"2024-04-05T22:21:15.975766+0000","last_fullsized":"2024-04-05T22:21:15.975766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897037+0000","last_change":"2024-04-05T22:21:14.434305+0000","last_active":"2024-04-05T22:21:15.897037+0000","last_peered":"2024-04-05T22:21:15.897037+0000","last_clean":"2024-04-05T22:21:15.897037+0000","last_became_active":"2024-04-05T22:21:14.433877+0000","last_became_peered":"2024-04-05T22:21:14.433877+0000","last_unstale":"2024-04-05T22:21:15.897037+0000","last_undegraded":"2024-04-05T22:21:15.897037+0000","last_fullsized":"2024-04-05T22:21:15.897037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899989+0000","last_change":"2024-04-05T22:21:14.912112+0000","last_active":"2024-04-05T22:21:15.899989+0000","last_peered":"2024-04-05T22:21:15.899989+0000","last_clean":"2024-04-05T22:21:15.899989+0000","last_became_active":"2024-04-05T22:21:14.910925+0000","last_became_peered":"2024-04-05T22:21:14.910925+0000","last_unstale":"2024-04-05T22:21:15.899989+0000","last_undegraded":"2024-04-05T22:21:15.899989+0000","last_fullsized":"2024-04-05T22:21:15.899989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899675+0000","last_change":"2024-04-05T22:21:14.438374+0000","last_active":"2024-04-05T22:21:15.899675+0000","last_peered":"2024-04-05T22:21:15.899675+0000","last_clean":"2024-04-05T22:21:15.899675+0000","last_became_active":"2024-04-05T22:21:14.437921+0000","last_became_peered":"2024-04-05T22:21:14.437921+0000","last_unstale":"2024-04-05T22:21:15.899675+0000","last_undegraded":"2024-04-05T22:21:15.899675+0000","last_fullsized":"2024-04-05T22:21:15.899675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983069+0000","last_change":"2024-04-05T22:21:14.914627+0000","last_active":"2024-04-05T22:21:15.983069+0000","last_peered":"2024-04-05T22:21:15.983069+0000","last_clean":"2024-04-05T22:21:15.983069+0000","last_became_active":"2024-04-05T22:21:14.914234+0000","last_became_peered":"2024-04-05T22:21:14.914234+0000","last_unstale":"2024-04-05T22:21:15.983069+0000","last_undegraded":"2024-04-05T22:21:15.983069+0000","last_fullsized":"2024-04-05T22:21:15.983069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900119+0000","last_change":"2024-04-05T22:21:14.431183+0000","last_active":"2024-04-05T22:21:15.900119+0000","last_peered":"2024-04-05T22:21:15.900119+0000","last_clean":"2024-04-05T22:21:15.900119+0000","last_became_active":"2024-04-05T22:21:14.430912+0000","last_became_peered":"2024-04-05T22:21:14.430912+0000","last_unstale":"2024-04-05T22:21:15.900119+0000","last_undegraded":"2024-04-05T22:21:15.900119+0000","last_fullsized":"2024-04-05T22:21:15.900119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:23.153789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899958+0000","last_change":"2024-04-05T22:21:14.921009+0000","last_active":"2024-04-05T22:21:15.899958+0000","last_peered":"2024-04-05T22:21:15.899958+0000","last_clean":"2024-04-05T22:21:15.899958+0000","last_became_active":"2024-04-05T22:21:14.920657+0000","last_became_peered":"2024-04-05T22:21:14.920657+0000","last_unstale":"2024-04-05T22:21:15.899958+0000","last_undegraded":"2024-04-05T22:21:15.899958+0000","last_fullsized":"2024-04-05T22:21:15.899958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981703+0000","last_change":"2024-04-05T22:21:14.443668+0000","last_active":"2024-04-05T22:21:15.981703+0000","last_peered":"2024-04-05T22:21:15.981703+0000","last_clean":"2024-04-05T22:21:15.981703+0000","last_became_active":"2024-04-05T22:21:14.442178+0000","last_became_peered":"2024-04-05T22:21:14.442178+0000","last_unstale":"2024-04-05T22:21:15.981703+0000","last_undegraded":"2024-04-05T22:21:15.981703+0000","last_fullsized":"2024-04-05T22:21:15.981703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900022+0000","last_change":"2024-04-05T22:21:14.905357+0000","last_active":"2024-04-05T22:21:15.900022+0000","last_peered":"2024-04-05T22:21:15.900022+0000","last_clean":"2024-04-05T22:21:15.900022+0000","last_became_active":"2024-04-05T22:21:14.905068+0000","last_became_peered":"2024-04-05T22:21:14.905068+0000","last_unstale":"2024-04-05T22:21:15.900022+0000","last_undegraded":"2024-04-05T22:21:15.900022+0000","last_fullsized":"2024-04-05T22:21:15.900022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900480+0000","last_change":"2024-04-05T22:21:14.442815+0000","last_active":"2024-04-05T22:21:15.900480+0000","last_peered":"2024-04-05T22:21:15.900480+0000","last_clean":"2024-04-05T22:21:15.900480+0000","last_became_active":"2024-04-05T22:21:14.442217+0000","last_became_peered":"2024-04-05T22:21:14.442217+0000","last_unstale":"2024-04-05T22:21:15.900480+0000","last_undegraded":"2024-04-05T22:21:15.900480+0000","last_fullsized":"2024-04-05T22:21:15.900480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976452+0000","last_change":"2024-04-05T22:21:14.933779+0000","last_active":"2024-04-05T22:21:15.976452+0000","last_peered":"2024-04-05T22:21:15.976452+0000","last_clean":"2024-04-05T22:21:15.976452+0000","last_became_active":"2024-04-05T22:21:14.933248+0000","last_became_peered":"2024-04-05T22:21:14.933248+0000","last_unstale":"2024-04-05T22:21:15.976452+0000","last_undegraded":"2024-04-05T22:21:15.976452+0000","last_fullsized":"2024-04-05T22:21:15.976452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981318+0000","last_change":"2024-04-05T22:21:14.449034+0000","last_active":"2024-04-05T22:21:15.981318+0000","last_peered":"2024-04-05T22:21:15.981318+0000","last_clean":"2024-04-05T22:21:15.981318+0000","last_became_active":"2024-04-05T22:21:14.448183+0000","last_became_peered":"2024-04-05T22:21:14.448183+0000","last_unstale":"2024-04-05T22:21:15.981318+0000","last_undegraded":"2024-04-05T22:21:15.981318+0000","last_fullsized":"2024-04-05T22:21:15.981318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:29:59.320001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.977308+0000","last_change":"2024-04-05T22:21:14.921617+0000","last_active":"2024-04-05T22:21:15.977308+0000","last_peered":"2024-04-05T22:21:15.977308+0000","last_clean":"2024-04-05T22:21:15.977308+0000","last_became_active":"2024-04-05T22:21:14.921153+0000","last_became_peered":"2024-04-05T22:21:14.921153+0000","last_unstale":"2024-04-05T22:21:15.977308+0000","last_undegraded":"2024-04-05T22:21:15.977308+0000","last_fullsized":"2024-04-05T22:21:15.977308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979031+0000","last_change":"2024-04-05T22:21:14.442038+0000","last_active":"2024-04-05T22:21:15.979031+0000","last_peered":"2024-04-05T22:21:15.979031+0000","last_clean":"2024-04-05T22:21:15.979031+0000","last_became_active":"2024-04-05T22:21:14.441431+0000","last_became_peered":"2024-04-05T22:21:14.441431+0000","last_unstale":"2024-04-05T22:21:15.979031+0000","last_undegraded":"2024-04-05T22:21:15.979031+0000","last_fullsized":"2024-04-05T22:21:15.979031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982667+0000","last_change":"2024-04-05T22:21:14.930105+0000","last_active":"2024-04-05T22:21:15.982667+0000","last_peered":"2024-04-05T22:21:15.982667+0000","last_clean":"2024-04-05T22:21:15.982667+0000","last_became_active":"2024-04-05T22:21:14.927238+0000","last_became_peered":"2024-04-05T22:21:14.927238+0000","last_unstale":"2024-04-05T22:21:15.982667+0000","last_undegraded":"2024-04-05T22:21:15.982667+0000","last_fullsized":"2024-04-05T22:21:15.982667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900204+0000","last_change":"2024-04-05T22:21:14.437077+0000","last_active":"2024-04-05T22:21:15.900204+0000","last_peered":"2024-04-05T22:21:15.900204+0000","last_clean":"2024-04-05T22:21:15.900204+0000","last_became_active":"2024-04-05T22:21:14.436767+0000","last_became_peered":"2024-04-05T22:21:14.436767+0000","last_unstale":"2024-04-05T22:21:15.900204+0000","last_undegraded":"2024-04-05T22:21:15.900204+0000","last_fullsized":"2024-04-05T22:21:15.900204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975658+0000","last_change":"2024-04-05T22:21:14.920674+0000","last_active":"2024-04-05T22:21:15.975658+0000","last_peered":"2024-04-05T22:21:15.975658+0000","last_clean":"2024-04-05T22:21:15.975658+0000","last_became_active":"2024-04-05T22:21:14.919722+0000","last_became_peered":"2024-04-05T22:21:14.919722+0000","last_unstale":"2024-04-05T22:21:15.975658+0000","last_undegraded":"2024-04-05T22:21:15.975658+0000","last_fullsized":"2024-04-05T22:21:15.975658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900313+0000","last_change":"2024-04-05T22:21:14.432768+0000","last_active":"2024-04-05T22:21:15.900313+0000","last_peered":"2024-04-05T22:21:15.900313+0000","last_clean":"2024-04-05T22:21:15.900313+0000","last_became_active":"2024-04-05T22:21:14.432330+0000","last_became_peered":"2024-04-05T22:21:14.432330+0000","last_unstale":"2024-04-05T22:21:15.900313+0000","last_undegraded":"2024-04-05T22:21:15.900313+0000","last_fullsized":"2024-04-05T22:21:15.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976206+0000","last_change":"2024-04-05T22:21:14.919559+0000","last_active":"2024-04-05T22:21:15.976206+0000","last_peered":"2024-04-05T22:21:15.976206+0000","last_clean":"2024-04-05T22:21:15.976206+0000","last_became_active":"2024-04-05T22:21:14.916617+0000","last_became_peered":"2024-04-05T22:21:14.916617+0000","last_unstale":"2024-04-05T22:21:15.976206+0000","last_undegraded":"2024-04-05T22:21:15.976206+0000","last_fullsized":"2024-04-05T22:21:15.976206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980370+0000","last_change":"2024-04-05T22:21:14.445535+0000","last_active":"2024-04-05T22:21:15.980370+0000","last_peered":"2024-04-05T22:21:15.980370+0000","last_clean":"2024-04-05T22:21:15.980370+0000","last_became_active":"2024-04-05T22:21:14.438718+0000","last_became_peered":"2024-04-05T22:21:14.438718+0000","last_unstale":"2024-04-05T22:21:15.980370+0000","last_undegraded":"2024-04-05T22:21:15.980370+0000","last_fullsized":"2024-04-05T22:21:15.980370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980726+0000","last_change":"2024-04-05T22:21:14.928694+0000","last_active":"2024-04-05T22:21:15.980726+0000","last_peered":"2024-04-05T22:21:15.980726+0000","last_clean":"2024-04-05T22:21:15.980726+0000","last_became_active":"2024-04-05T22:21:14.928392+0000","last_became_peered":"2024-04-05T22:21:14.928392+0000","last_unstale":"2024-04-05T22:21:15.980726+0000","last_undegraded":"2024-04-05T22:21:15.980726+0000","last_fullsized":"2024-04-05T22:21:15.980726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:02:26.504384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900241+0000","last_change":"2024-04-05T22:21:14.919963+0000","last_active":"2024-04-05T22:21:15.900241+0000","last_peered":"2024-04-05T22:21:15.900241+0000","last_clean":"2024-04-05T22:21:15.900241+0000","last_became_active":"2024-04-05T22:21:14.917075+0000","last_became_peered":"2024-04-05T22:21:14.917075+0000","last_unstale":"2024-04-05T22:21:15.900241+0000","last_undegraded":"2024-04-05T22:21:15.900241+0000","last_fullsized":"2024-04-05T22:21:15.900241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980519+0000","last_change":"2024-04-05T22:21:14.440258+0000","last_active":"2024-04-05T22:21:15.980519+0000","last_peered":"2024-04-05T22:21:15.980519+0000","last_clean":"2024-04-05T22:21:15.980519+0000","last_became_active":"2024-04-05T22:21:14.439946+0000","last_became_peered":"2024-04-05T22:21:14.439946+0000","last_unstale":"2024-04-05T22:21:15.980519+0000","last_undegraded":"2024-04-05T22:21:15.980519+0000","last_fullsized":"2024-04-05T22:21:15.980519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899588+0000","last_change":"2024-04-05T22:21:14.914728+0000","last_active":"2024-04-05T22:21:15.899588+0000","last_peered":"2024-04-05T22:21:15.899588+0000","last_clean":"2024-04-05T22:21:15.899588+0000","last_became_active":"2024-04-05T22:21:14.914450+0000","last_became_peered":"2024-04-05T22:21:14.914450+0000","last_unstale":"2024-04-05T22:21:15.899588+0000","last_undegraded":"2024-04-05T22:21:15.899588+0000","last_fullsized":"2024-04-05T22:21:15.899588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981282+0000","last_change":"2024-04-05T22:21:14.451271+0000","last_active":"2024-04-05T22:21:15.981282+0000","last_peered":"2024-04-05T22:21:15.981282+0000","last_clean":"2024-04-05T22:21:15.981282+0000","last_became_active":"2024-04-05T22:21:14.450977+0000","last_became_peered":"2024-04-05T22:21:14.450977+0000","last_unstale":"2024-04-05T22:21:15.981282+0000","last_undegraded":"2024-04-05T22:21:15.981282+0000","last_fullsized":"2024-04-05T22:21:15.981282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.921420+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.919777+0000","last_became_peered":"2024-04-05T22:21:14.919777+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976482+0000","last_change":"2024-04-05T22:21:14.442123+0000","last_active":"2024-04-05T22:21:15.976482+0000","last_peered":"2024-04-05T22:21:15.976482+0000","last_clean":"2024-04-05T22:21:15.976482+0000","last_became_active":"2024-04-05T22:21:14.441885+0000","last_became_peered":"2024-04-05T22:21:14.441885+0000","last_unstale":"2024-04-05T22:21:15.976482+0000","last_undegraded":"2024-04-05T22:21:15.976482+0000","last_fullsized":"2024-04-05T22:21:15.976482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976454+0000","last_change":"2024-04-05T22:21:14.923884+0000","last_active":"2024-04-05T22:21:15.976454+0000","last_peered":"2024-04-05T22:21:15.976454+0000","last_clean":"2024-04-05T22:21:15.976454+0000","last_became_active":"2024-04-05T22:21:14.923545+0000","last_became_peered":"2024-04-05T22:21:14.923545+0000","last_unstale":"2024-04-05T22:21:15.976454+0000","last_undegraded":"2024-04-05T22:21:15.976454+0000","last_fullsized":"2024-04-05T22:21:15.976454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976809+0000","last_change":"2024-04-05T22:21:14.439558+0000","last_active":"2024-04-05T22:21:15.976809+0000","last_peered":"2024-04-05T22:21:15.976809+0000","last_clean":"2024-04-05T22:21:15.976809+0000","last_became_active":"2024-04-05T22:21:14.439082+0000","last_became_peered":"2024-04-05T22:21:14.439082+0000","last_unstale":"2024-04-05T22:21:15.976809+0000","last_undegraded":"2024-04-05T22:21:15.976809+0000","last_fullsized":"2024-04-05T22:21:15.976809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.946}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:21:19.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:21:19.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.541+0000 7f00ba200640 1 Processor -- start 2024-04-05T22:21:19.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.541+0000 7f00ba200640 1 -- start start 2024-04-05T22:21:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00ba200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f00b40b1870 0x7f00b40ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00ba200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b40ce5d0 con 0x7f00b40ad6b0 2024-04-05T22:21:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00ba200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b40ce010 con 0x7f00b4160620 2024-04-05T22:21:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00ba200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b40ce2f0 con 0x7f00b40b1870 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00b2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f00b40b1870 0x7f00b40ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00b3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f00b40ad6b0 0x7f00b40cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48740/0 (socket says 172.21.15.73:48740) 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00b3e00640 1 -- 172.21.15.73:0/4206549005 learned_addr learned my addr 172.21.15.73:0/4206549005 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.543+0000 7f00b2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f00b40b1870 0x7f00b40ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50856/0 (socket says 172.21.15.73:50856) 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.544+0000 7f00b2a00640 1 -- 172.21.15.73:0/4206549005 >> v1:172.21.15.157:6789/0 conn(0x7f00b4160620 legacy=0x7f00b40caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.544+0000 7f00b2a00640 1 -- 172.21.15.73:0/4206549005 >> v1:172.21.15.73:6789/0 conn(0x7f00b40ad6b0 legacy=0x7f00b40cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.544+0000 7f00b2a00640 1 -- 172.21.15.73:0/4206549005 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40ac970 con 0x7f00b40b1870 2024-04-05T22:21:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00b2a00640 1 --2- 172.21.15.73:0/4206549005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40ad170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00b40ce7c0 tx=0x7f00a002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420016591f79db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00b2000640 1 -- 172.21.15.73:0/4206549005 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0030800 con 0x7f00b40b1870 2024-04-05T22:21:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00b2000640 1 -- 172.21.15.73:0/4206549005 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0030960 con 0x7f00b40b1870 2024-04-05T22:21:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00b2000640 1 -- 172.21.15.73:0/4206549005 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0030cf0 con 0x7f00b40b1870 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00ba200640 1 -- 172.21.15.73:0/4206549005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 msgr2=0x7f00b40ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.545+0000 7f00ba200640 1 --2- 172.21.15.73:0/4206549005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40ad170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00b40ce7c0 tx=0x7f00a002b940 comp rx=0 tx=0).stop 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.546+0000 7f00ba200640 1 -- 172.21.15.73:0/4206549005 shutdown_connections 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.546+0000 7f00ba200640 1 --2- 172.21.15.73:0/4206549005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40ad170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.546+0000 7f00ba200640 1 -- 172.21.15.73:0/4206549005 >> 172.21.15.73:0/4206549005 conn(0x7f00b40ce8e0 msgr2=0x7f00b40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.546+0000 7f00ba200640 1 -- 172.21.15.73:0/4206549005 shutdown_connections 2024-04-05T22:21:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.546+0000 7f00ba200640 1 -- 172.21.15.73:0/4206549005 wait complete. 2024-04-05T22:21:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 Processor -- start 2024-04-05T22:21:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 -- start start 2024-04-05T22:21:19.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 0x7f00b406a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b4160620 0x7f00b4160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40c37e0 con 0x7f00b40b1870 2024-04-05T22:21:19.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40c7990 con 0x7f00b40ad6b0 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.547+0000 7f00ba200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4161870 con 0x7f00b4160620 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50872/0 (socket says 172.21.15.73:50872) 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 -- 172.21.15.73:0/493492064 learned_addr learned my addr 172.21.15.73:0/493492064 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b3400640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 0x7f00b406a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b3e00640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b4160620 0x7f00b4160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 -- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b4160620 msgr2=0x7f00b4160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b4160620 0x7f00b4160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 -- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 msgr2=0x7f00b406a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 0x7f00b406a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 -- 172.21.15.73:0/493492064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0009400 con 0x7f00b40b1870 2024-04-05T22:21:19.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b3400640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 0x7f00b406a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.549+0000 7f00b2a00640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009890 tx=0x7f00a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.550+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a003e040 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.550+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0039e40 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.550+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0044630 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.550+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b406a830 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.550+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b40c3d10 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.551+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b4066af0 con 0x7f00b40b1870 2024-04-05T22:21:19.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.552+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f00a0044790 con 0x7f00b40b1870 2024-04-05T22:21:19.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.552+0000 7f00ab400640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 0x7f009406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:19.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.553+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00a00aba80 con 0x7f00b40b1870 2024-04-05T22:21:19.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.553+0000 7f00b3400640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 0x7f009406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:19.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.553+0000 7f00b3400640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 0x7f009406d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f00a4000f60 tx=0x7f00a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:19.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.557+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00a0076ce0 con 0x7f00b40b1870 2024-04-05T22:21:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.749+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00b40cc7e0 con 0x7f009406ac00 2024-04-05T22:21:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.757+0000 7f00ab400640 1 -- 172.21.15.73:0/493492064 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264497 (secure 0 0 0) 0x7f00a4001c90 con 0x7f009406ac00 2024-04-05T22:21:19.758 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:19.765 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.762+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 msgr2=0x7f009406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.762+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 0x7f009406d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f00a4000f60 tx=0x7f00a4009040 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.762+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 msgr2=0x7f00b40c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.762+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009890 tx=0x7f00a0004730 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 shutdown_connections 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f009406ac00 0x7f009406d0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b4160620 0x7f00b4160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b40b1870 0x7f00b40c7450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 --2- 172.21.15.73:0/493492064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b40ad6b0 0x7f00b406a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:19.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 >> 172.21.15.73:0/493492064 conn(0x7f00b40ce8e0 msgr2=0x7f00b40b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:19.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 shutdown_connections 2024-04-05T22:21:19.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:19.763+0000 7f00ba200640 1 -- 172.21.15.73:0/493492064 wait complete. 2024-04-05T22:21:19.782 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-05T22:21:18.488848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":3055616,"data_stored":1816372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.103198"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982064+0000","last_change":"2024-04-05T22:21:14.923383+0000","last_active":"2024-04-05T22:21:15.982064+0000","last_peered":"2024-04-05T22:21:15.982064+0000","last_clean":"2024-04-05T22:21:15.982064+0000","last_became_active":"2024-04-05T22:21:14.922950+0000","last_became_peered":"2024-04-05T22:21:14.922950+0000","last_unstale":"2024-04-05T22:21:15.982064+0000","last_undegraded":"2024-04-05T22:21:15.982064+0000","last_fullsized":"2024-04-05T22:21:15.982064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900565+0000","last_change":"2024-04-05T22:21:14.912375+0000","last_active":"2024-04-05T22:21:15.900565+0000","last_peered":"2024-04-05T22:21:15.900565+0000","last_clean":"2024-04-05T22:21:15.900565+0000","last_became_active":"2024-04-05T22:21:14.912009+0000","last_became_peered":"2024-04-05T22:21:14.912009+0000","last_unstale":"2024-04-05T22:21:15.900565+0000","last_undegraded":"2024-04-05T22:21:15.900565+0000","last_fullsized":"2024-04-05T22:21:15.900565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976623+0000","last_change":"2024-04-05T22:21:14.914402+0000","last_active":"2024-04-05T22:21:15.976623+0000","last_peered":"2024-04-05T22:21:15.976623+0000","last_clean":"2024-04-05T22:21:15.976623+0000","last_became_active":"2024-04-05T22:21:14.914120+0000","last_became_peered":"2024-04-05T22:21:14.914120+0000","last_unstale":"2024-04-05T22:21:15.976623+0000","last_undegraded":"2024-04-05T22:21:15.976623+0000","last_fullsized":"2024-04-05T22:21:15.976623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899962+0000","last_change":"2024-04-05T22:21:14.921662+0000","last_active":"2024-04-05T22:21:15.899962+0000","last_peered":"2024-04-05T22:21:15.899962+0000","last_clean":"2024-04-05T22:21:15.899962+0000","last_became_active":"2024-04-05T22:21:14.921218+0000","last_became_peered":"2024-04-05T22:21:14.921218+0000","last_unstale":"2024-04-05T22:21:15.899962+0000","last_undegraded":"2024-04-05T22:21:15.899962+0000","last_fullsized":"2024-04-05T22:21:15.899962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979973+0000","last_change":"2024-04-05T22:21:14.935767+0000","last_active":"2024-04-05T22:21:15.979973+0000","last_peered":"2024-04-05T22:21:15.979973+0000","last_clean":"2024-04-05T22:21:15.979973+0000","last_became_active":"2024-04-05T22:21:14.935145+0000","last_became_peered":"2024-04-05T22:21:14.935145+0000","last_unstale":"2024-04-05T22:21:15.979973+0000","last_undegraded":"2024-04-05T22:21:15.979973+0000","last_fullsized":"2024-04-05T22:21:15.979973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898816+0000","last_change":"2024-04-05T22:21:14.923031+0000","last_active":"2024-04-05T22:21:15.898816+0000","last_peered":"2024-04-05T22:21:15.898816+0000","last_clean":"2024-04-05T22:21:15.898816+0000","last_became_active":"2024-04-05T22:21:14.922585+0000","last_became_peered":"2024-04-05T22:21:14.922585+0000","last_unstale":"2024-04-05T22:21:15.898816+0000","last_undegraded":"2024-04-05T22:21:15.898816+0000","last_fullsized":"2024-04-05T22:21:15.898816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979244+0000","last_change":"2024-04-05T22:21:14.925099+0000","last_active":"2024-04-05T22:21:15.979244+0000","last_peered":"2024-04-05T22:21:15.979244+0000","last_clean":"2024-04-05T22:21:15.979244+0000","last_became_active":"2024-04-05T22:21:14.921678+0000","last_became_peered":"2024-04-05T22:21:14.921678+0000","last_unstale":"2024-04-05T22:21:15.979244+0000","last_undegraded":"2024-04-05T22:21:15.979244+0000","last_fullsized":"2024-04-05T22:21:15.979244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975561+0000","last_change":"2024-04-05T22:21:14.916195+0000","last_active":"2024-04-05T22:21:15.975561+0000","last_peered":"2024-04-05T22:21:15.975561+0000","last_clean":"2024-04-05T22:21:15.975561+0000","last_became_active":"2024-04-05T22:21:14.915947+0000","last_became_peered":"2024-04-05T22:21:14.915947+0000","last_unstale":"2024-04-05T22:21:15.975561+0000","last_undegraded":"2024-04-05T22:21:15.975561+0000","last_fullsized":"2024-04-05T22:21:15.975561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982147+0000","last_change":"2024-04-05T22:21:14.931207+0000","last_active":"2024-04-05T22:21:15.982147+0000","last_peered":"2024-04-05T22:21:15.982147+0000","last_clean":"2024-04-05T22:21:15.982147+0000","last_became_active":"2024-04-05T22:21:14.930509+0000","last_became_peered":"2024-04-05T22:21:14.930509+0000","last_unstale":"2024-04-05T22:21:15.982147+0000","last_undegraded":"2024-04-05T22:21:15.982147+0000","last_fullsized":"2024-04-05T22:21:15.982147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981506+0000","last_change":"2024-04-05T22:21:14.916586+0000","last_active":"2024-04-05T22:21:15.981506+0000","last_peered":"2024-04-05T22:21:15.981506+0000","last_clean":"2024-04-05T22:21:15.981506+0000","last_became_active":"2024-04-05T22:21:14.916004+0000","last_became_peered":"2024-04-05T22:21:14.916004+0000","last_unstale":"2024-04-05T22:21:15.981506+0000","last_undegraded":"2024-04-05T22:21:15.981506+0000","last_fullsized":"2024-04-05T22:21:15.981506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:02.978295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981439+0000","last_change":"2024-04-05T22:21:14.913056+0000","last_active":"2024-04-05T22:21:15.981439+0000","last_peered":"2024-04-05T22:21:15.981439+0000","last_clean":"2024-04-05T22:21:15.981439+0000","last_became_active":"2024-04-05T22:21:14.912748+0000","last_became_peered":"2024-04-05T22:21:14.912748+0000","last_unstale":"2024-04-05T22:21:15.981439+0000","last_undegraded":"2024-04-05T22:21:15.981439+0000","last_fullsized":"2024-04-05T22:21:15.981439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:11:59.394291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983331+0000","last_change":"2024-04-05T22:21:14.926927+0000","last_active":"2024-04-05T22:21:15.983331+0000","last_peered":"2024-04-05T22:21:15.983331+0000","last_clean":"2024-04-05T22:21:15.983331+0000","last_became_active":"2024-04-05T22:21:14.913316+0000","last_became_peered":"2024-04-05T22:21:14.913316+0000","last_unstale":"2024-04-05T22:21:15.983331+0000","last_undegraded":"2024-04-05T22:21:15.983331+0000","last_fullsized":"2024-04-05T22:21:15.983331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:14.865832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976116+0000","last_change":"2024-04-05T22:21:14.919462+0000","last_active":"2024-04-05T22:21:15.976116+0000","last_peered":"2024-04-05T22:21:15.976116+0000","last_clean":"2024-04-05T22:21:15.976116+0000","last_became_active":"2024-04-05T22:21:14.919180+0000","last_became_peered":"2024-04-05T22:21:14.919180+0000","last_unstale":"2024-04-05T22:21:15.976116+0000","last_undegraded":"2024-04-05T22:21:15.976116+0000","last_fullsized":"2024-04-05T22:21:15.976116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975694+0000","last_change":"2024-04-05T22:21:14.912391+0000","last_active":"2024-04-05T22:21:15.975694+0000","last_peered":"2024-04-05T22:21:15.975694+0000","last_clean":"2024-04-05T22:21:15.975694+0000","last_became_active":"2024-04-05T22:21:14.912013+0000","last_became_peered":"2024-04-05T22:21:14.912013+0000","last_unstale":"2024-04-05T22:21:15.975694+0000","last_undegraded":"2024-04-05T22:21:15.975694+0000","last_fullsized":"2024-04-05T22:21:15.975694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899930+0000","last_change":"2024-04-05T22:21:14.900459+0000","last_active":"2024-04-05T22:21:15.899930+0000","last_peered":"2024-04-05T22:21:15.899930+0000","last_clean":"2024-04-05T22:21:15.899930+0000","last_became_active":"2024-04-05T22:21:14.900144+0000","last_became_peered":"2024-04-05T22:21:14.900144+0000","last_unstale":"2024-04-05T22:21:15.899930+0000","last_undegraded":"2024-04-05T22:21:15.899930+0000","last_fullsized":"2024-04-05T22:21:15.899930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898646+0000","last_change":"2024-04-05T22:21:14.913235+0000","last_active":"2024-04-05T22:21:15.898646+0000","last_peered":"2024-04-05T22:21:15.898646+0000","last_clean":"2024-04-05T22:21:15.898646+0000","last_became_active":"2024-04-05T22:21:14.912913+0000","last_became_peered":"2024-04-05T22:21:14.912913+0000","last_unstale":"2024-04-05T22:21:15.898646+0000","last_undegraded":"2024-04-05T22:21:15.898646+0000","last_fullsized":"2024-04-05T22:21:15.898646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900293+0000","last_change":"2024-04-05T22:21:14.931728+0000","last_active":"2024-04-05T22:21:15.900293+0000","last_peered":"2024-04-05T22:21:15.900293+0000","last_clean":"2024-04-05T22:21:15.900293+0000","last_became_active":"2024-04-05T22:21:14.931500+0000","last_became_peered":"2024-04-05T22:21:14.931500+0000","last_unstale":"2024-04-05T22:21:15.900293+0000","last_undegraded":"2024-04-05T22:21:15.900293+0000","last_fullsized":"2024-04-05T22:21:15.900293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897627+0000","last_change":"2024-04-05T22:21:14.921738+0000","last_active":"2024-04-05T22:21:15.897627+0000","last_peered":"2024-04-05T22:21:15.897627+0000","last_clean":"2024-04-05T22:21:15.897627+0000","last_became_active":"2024-04-05T22:21:14.919781+0000","last_became_peered":"2024-04-05T22:21:14.919781+0000","last_unstale":"2024-04-05T22:21:15.897627+0000","last_undegraded":"2024-04-05T22:21:15.897627+0000","last_fullsized":"2024-04-05T22:21:15.897627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981513+0000","last_change":"2024-04-05T22:21:14.920265+0000","last_active":"2024-04-05T22:21:15.981513+0000","last_peered":"2024-04-05T22:21:15.981513+0000","last_clean":"2024-04-05T22:21:15.981513+0000","last_became_active":"2024-04-05T22:21:14.919985+0000","last_became_peered":"2024-04-05T22:21:14.919985+0000","last_unstale":"2024-04-05T22:21:15.981513+0000","last_undegraded":"2024-04-05T22:21:15.981513+0000","last_fullsized":"2024-04-05T22:21:15.981513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981857+0000","last_change":"2024-04-05T22:21:14.929527+0000","last_active":"2024-04-05T22:21:15.981857+0000","last_peered":"2024-04-05T22:21:15.981857+0000","last_clean":"2024-04-05T22:21:15.981857+0000","last_became_active":"2024-04-05T22:21:14.929222+0000","last_became_peered":"2024-04-05T22:21:14.929222+0000","last_unstale":"2024-04-05T22:21:15.981857+0000","last_undegraded":"2024-04-05T22:21:15.981857+0000","last_fullsized":"2024-04-05T22:21:15.981857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:15:04.491398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975985+0000","last_change":"2024-04-05T22:21:14.933523+0000","last_active":"2024-04-05T22:21:15.975985+0000","last_peered":"2024-04-05T22:21:15.975985+0000","last_clean":"2024-04-05T22:21:15.975985+0000","last_became_active":"2024-04-05T22:21:14.933340+0000","last_became_peered":"2024-04-05T22:21:14.933340+0000","last_unstale":"2024-04-05T22:21:15.975985+0000","last_undegraded":"2024-04-05T22:21:15.975985+0000","last_fullsized":"2024-04-05T22:21:15.975985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898875+0000","last_change":"2024-04-05T22:21:14.933905+0000","last_active":"2024-04-05T22:21:15.898875+0000","last_peered":"2024-04-05T22:21:15.898875+0000","last_clean":"2024-04-05T22:21:15.898875+0000","last_became_active":"2024-04-05T22:21:14.933507+0000","last_became_peered":"2024-04-05T22:21:14.933507+0000","last_unstale":"2024-04-05T22:21:15.898875+0000","last_undegraded":"2024-04-05T22:21:15.898875+0000","last_fullsized":"2024-04-05T22:21:15.898875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975831+0000","last_change":"2024-04-05T22:21:14.915842+0000","last_active":"2024-04-05T22:21:15.975831+0000","last_peered":"2024-04-05T22:21:15.975831+0000","last_clean":"2024-04-05T22:21:15.975831+0000","last_became_active":"2024-04-05T22:21:14.915391+0000","last_became_peered":"2024-04-05T22:21:14.915391+0000","last_unstale":"2024-04-05T22:21:15.975831+0000","last_undegraded":"2024-04-05T22:21:15.975831+0000","last_fullsized":"2024-04-05T22:21:15.975831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:21.773200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898993+0000","last_change":"2024-04-05T22:21:14.924789+0000","last_active":"2024-04-05T22:21:15.898993+0000","last_peered":"2024-04-05T22:21:15.898993+0000","last_clean":"2024-04-05T22:21:15.898993+0000","last_became_active":"2024-04-05T22:21:14.924488+0000","last_became_peered":"2024-04-05T22:21:14.924488+0000","last_unstale":"2024-04-05T22:21:15.898993+0000","last_undegraded":"2024-04-05T22:21:15.898993+0000","last_fullsized":"2024-04-05T22:21:15.898993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976451+0000","last_change":"2024-04-05T22:21:14.926693+0000","last_active":"2024-04-05T22:21:15.976451+0000","last_peered":"2024-04-05T22:21:15.976451+0000","last_clean":"2024-04-05T22:21:15.976451+0000","last_became_active":"2024-04-05T22:21:14.926415+0000","last_became_peered":"2024-04-05T22:21:14.926415+0000","last_unstale":"2024-04-05T22:21:15.976451+0000","last_undegraded":"2024-04-05T22:21:15.976451+0000","last_fullsized":"2024-04-05T22:21:15.976451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981744+0000","last_change":"2024-04-05T22:21:14.940178+0000","last_active":"2024-04-05T22:21:15.981744+0000","last_peered":"2024-04-05T22:21:15.981744+0000","last_clean":"2024-04-05T22:21:15.981744+0000","last_became_active":"2024-04-05T22:21:14.938042+0000","last_became_peered":"2024-04-05T22:21:14.938042+0000","last_unstale":"2024-04-05T22:21:15.981744+0000","last_undegraded":"2024-04-05T22:21:15.981744+0000","last_fullsized":"2024-04-05T22:21:15.981744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:42.750931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900215+0000","last_change":"2024-04-05T22:21:14.916877+0000","last_active":"2024-04-05T22:21:15.900215+0000","last_peered":"2024-04-05T22:21:15.900215+0000","last_clean":"2024-04-05T22:21:15.900215+0000","last_became_active":"2024-04-05T22:21:14.916585+0000","last_became_peered":"2024-04-05T22:21:14.916585+0000","last_unstale":"2024-04-05T22:21:15.900215+0000","last_undegraded":"2024-04-05T22:21:15.900215+0000","last_fullsized":"2024-04-05T22:21:15.900215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976229+0000","last_change":"2024-04-05T22:21:14.918894+0000","last_active":"2024-04-05T22:21:15.976229+0000","last_peered":"2024-04-05T22:21:15.976229+0000","last_clean":"2024-04-05T22:21:15.976229+0000","last_became_active":"2024-04-05T22:21:14.918074+0000","last_became_peered":"2024-04-05T22:21:14.918074+0000","last_unstale":"2024-04-05T22:21:15.976229+0000","last_undegraded":"2024-04-05T22:21:15.976229+0000","last_fullsized":"2024-04-05T22:21:15.976229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976071+0000","last_change":"2024-04-05T22:21:14.922850+0000","last_active":"2024-04-05T22:21:15.976071+0000","last_peered":"2024-04-05T22:21:15.976071+0000","last_clean":"2024-04-05T22:21:15.976071+0000","last_became_active":"2024-04-05T22:21:14.920770+0000","last_became_peered":"2024-04-05T22:21:14.920770+0000","last_unstale":"2024-04-05T22:21:15.976071+0000","last_undegraded":"2024-04-05T22:21:15.976071+0000","last_fullsized":"2024-04-05T22:21:15.976071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899082+0000","last_change":"2024-04-05T22:21:14.917603+0000","last_active":"2024-04-05T22:21:15.899082+0000","last_peered":"2024-04-05T22:21:15.899082+0000","last_clean":"2024-04-05T22:21:15.899082+0000","last_became_active":"2024-04-05T22:21:14.917325+0000","last_became_peered":"2024-04-05T22:21:14.917325+0000","last_unstale":"2024-04-05T22:21:15.899082+0000","last_undegraded":"2024-04-05T22:21:15.899082+0000","last_fullsized":"2024-04-05T22:21:15.899082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:46.139845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.896840+0000","last_change":"2024-04-05T22:21:14.911372+0000","last_active":"2024-04-05T22:21:15.896840+0000","last_peered":"2024-04-05T22:21:15.896840+0000","last_clean":"2024-04-05T22:21:15.896840+0000","last_became_active":"2024-04-05T22:21:14.911083+0000","last_became_peered":"2024-04-05T22:21:14.911083+0000","last_unstale":"2024-04-05T22:21:15.896840+0000","last_undegraded":"2024-04-05T22:21:15.896840+0000","last_fullsized":"2024-04-05T22:21:15.896840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979172+0000","last_change":"2024-04-05T22:21:14.915897+0000","last_active":"2024-04-05T22:21:15.979172+0000","last_peered":"2024-04-05T22:21:15.979172+0000","last_clean":"2024-04-05T22:21:15.979172+0000","last_became_active":"2024-04-05T22:21:14.915223+0000","last_became_peered":"2024-04-05T22:21:14.915223+0000","last_unstale":"2024-04-05T22:21:15.979172+0000","last_undegraded":"2024-04-05T22:21:15.979172+0000","last_fullsized":"2024-04-05T22:21:15.979172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:19:20.576451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982664+0000","last_change":"2024-04-05T22:21:14.901329+0000","last_active":"2024-04-05T22:21:15.982664+0000","last_peered":"2024-04-05T22:21:15.982664+0000","last_clean":"2024-04-05T22:21:15.982664+0000","last_became_active":"2024-04-05T22:21:14.900870+0000","last_became_peered":"2024-04-05T22:21:14.900870+0000","last_unstale":"2024-04-05T22:21:15.982664+0000","last_undegraded":"2024-04-05T22:21:15.982664+0000","last_fullsized":"2024-04-05T22:21:15.982664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:12.554588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976297+0000","last_change":"2024-04-05T22:21:14.437934+0000","last_active":"2024-04-05T22:21:15.976297+0000","last_peered":"2024-04-05T22:21:15.976297+0000","last_clean":"2024-04-05T22:21:15.976297+0000","last_became_active":"2024-04-05T22:21:14.437696+0000","last_became_peered":"2024-04-05T22:21:14.437696+0000","last_unstale":"2024-04-05T22:21:15.976297+0000","last_undegraded":"2024-04-05T22:21:15.976297+0000","last_fullsized":"2024-04-05T22:21:15.976297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976230+0000","last_change":"2024-04-05T22:21:14.920473+0000","last_active":"2024-04-05T22:21:15.976230+0000","last_peered":"2024-04-05T22:21:15.976230+0000","last_clean":"2024-04-05T22:21:15.976230+0000","last_became_active":"2024-04-05T22:21:14.917453+0000","last_became_peered":"2024-04-05T22:21:14.917453+0000","last_unstale":"2024-04-05T22:21:15.976230+0000","last_undegraded":"2024-04-05T22:21:15.976230+0000","last_fullsized":"2024-04-05T22:21:15.976230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980837+0000","last_change":"2024-04-05T22:21:14.449143+0000","last_active":"2024-04-05T22:21:15.980837+0000","last_peered":"2024-04-05T22:21:15.980837+0000","last_clean":"2024-04-05T22:21:15.980837+0000","last_became_active":"2024-04-05T22:21:14.448341+0000","last_became_peered":"2024-04-05T22:21:14.448341+0000","last_unstale":"2024-04-05T22:21:15.980837+0000","last_undegraded":"2024-04-05T22:21:15.980837+0000","last_fullsized":"2024-04-05T22:21:15.980837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:14.544598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975917+0000","last_change":"2024-04-05T22:21:14.441239+0000","last_active":"2024-04-05T22:21:15.975917+0000","last_peered":"2024-04-05T22:21:15.975917+0000","last_clean":"2024-04-05T22:21:15.975917+0000","last_became_active":"2024-04-05T22:21:14.440898+0000","last_became_peered":"2024-04-05T22:21:14.440898+0000","last_unstale":"2024-04-05T22:21:15.975917+0000","last_undegraded":"2024-04-05T22:21:15.975917+0000","last_fullsized":"2024-04-05T22:21:15.975917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975636+0000","last_change":"2024-04-05T22:21:14.922371+0000","last_active":"2024-04-05T22:21:15.975636+0000","last_peered":"2024-04-05T22:21:15.975636+0000","last_clean":"2024-04-05T22:21:15.975636+0000","last_became_active":"2024-04-05T22:21:14.922074+0000","last_became_peered":"2024-04-05T22:21:14.922074+0000","last_unstale":"2024-04-05T22:21:15.975636+0000","last_undegraded":"2024-04-05T22:21:15.975636+0000","last_fullsized":"2024-04-05T22:21:15.975636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898990+0000","last_change":"2024-04-05T22:21:14.446427+0000","last_active":"2024-04-05T22:21:15.898990+0000","last_peered":"2024-04-05T22:21:15.898990+0000","last_clean":"2024-04-05T22:21:15.898990+0000","last_became_active":"2024-04-05T22:21:14.445487+0000","last_became_peered":"2024-04-05T22:21:14.445487+0000","last_unstale":"2024-04-05T22:21:15.898990+0000","last_undegraded":"2024-04-05T22:21:15.898990+0000","last_fullsized":"2024-04-05T22:21:15.898990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899850+0000","last_change":"2024-04-05T22:21:14.917346+0000","last_active":"2024-04-05T22:21:15.899850+0000","last_peered":"2024-04-05T22:21:15.899850+0000","last_clean":"2024-04-05T22:21:15.899850+0000","last_became_active":"2024-04-05T22:21:14.917085+0000","last_became_peered":"2024-04-05T22:21:14.917085+0000","last_unstale":"2024-04-05T22:21:15.899850+0000","last_undegraded":"2024-04-05T22:21:15.899850+0000","last_fullsized":"2024-04-05T22:21:15.899850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981190+0000","last_change":"2024-04-05T22:21:14.441415+0000","last_active":"2024-04-05T22:21:15.981190+0000","last_peered":"2024-04-05T22:21:15.981190+0000","last_clean":"2024-04-05T22:21:15.981190+0000","last_became_active":"2024-04-05T22:21:14.440493+0000","last_became_peered":"2024-04-05T22:21:14.440493+0000","last_unstale":"2024-04-05T22:21:15.981190+0000","last_undegraded":"2024-04-05T22:21:15.981190+0000","last_fullsized":"2024-04-05T22:21:15.981190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:25.845173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899687+0000","last_change":"2024-04-05T22:21:14.932959+0000","last_active":"2024-04-05T22:21:15.899687+0000","last_peered":"2024-04-05T22:21:15.899687+0000","last_clean":"2024-04-05T22:21:15.899687+0000","last_became_active":"2024-04-05T22:21:14.932656+0000","last_became_peered":"2024-04-05T22:21:14.932656+0000","last_unstale":"2024-04-05T22:21:15.899687+0000","last_undegraded":"2024-04-05T22:21:15.899687+0000","last_fullsized":"2024-04-05T22:21:15.899687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899902+0000","last_change":"2024-04-05T22:21:14.444991+0000","last_active":"2024-04-05T22:21:15.899902+0000","last_peered":"2024-04-05T22:21:15.899902+0000","last_clean":"2024-04-05T22:21:15.899902+0000","last_became_active":"2024-04-05T22:21:14.444667+0000","last_became_peered":"2024-04-05T22:21:14.444667+0000","last_unstale":"2024-04-05T22:21:15.899902+0000","last_undegraded":"2024-04-05T22:21:15.899902+0000","last_fullsized":"2024-04-05T22:21:15.899902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.915276+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.914411+0000","last_became_peered":"2024-04-05T22:21:14.914411+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900261+0000","last_change":"2024-04-05T22:21:14.438342+0000","last_active":"2024-04-05T22:21:15.900261+0000","last_peered":"2024-04-05T22:21:15.900261+0000","last_clean":"2024-04-05T22:21:15.900261+0000","last_became_active":"2024-04-05T22:21:14.438070+0000","last_became_peered":"2024-04-05T22:21:14.438070+0000","last_unstale":"2024-04-05T22:21:15.900261+0000","last_undegraded":"2024-04-05T22:21:15.900261+0000","last_fullsized":"2024-04-05T22:21:15.900261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:13.607794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983525+0000","last_change":"2024-04-05T22:21:14.928911+0000","last_active":"2024-04-05T22:21:15.983525+0000","last_peered":"2024-04-05T22:21:15.983525+0000","last_clean":"2024-04-05T22:21:15.983525+0000","last_became_active":"2024-04-05T22:21:14.928657+0000","last_became_peered":"2024-04-05T22:21:14.928657+0000","last_unstale":"2024-04-05T22:21:15.983525+0000","last_undegraded":"2024-04-05T22:21:15.983525+0000","last_fullsized":"2024-04-05T22:21:15.983525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982110+0000","last_change":"2024-04-05T22:21:14.442377+0000","last_active":"2024-04-05T22:21:15.982110+0000","last_peered":"2024-04-05T22:21:15.982110+0000","last_clean":"2024-04-05T22:21:15.982110+0000","last_became_active":"2024-04-05T22:21:14.441914+0000","last_became_peered":"2024-04-05T22:21:14.441914+0000","last_unstale":"2024-04-05T22:21:15.982110+0000","last_undegraded":"2024-04-05T22:21:15.982110+0000","last_fullsized":"2024-04-05T22:21:15.982110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:53.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980822+0000","last_change":"2024-04-05T22:21:14.919936+0000","last_active":"2024-04-05T22:21:15.980822+0000","last_peered":"2024-04-05T22:21:15.980822+0000","last_clean":"2024-04-05T22:21:15.980822+0000","last_became_active":"2024-04-05T22:21:14.919602+0000","last_became_peered":"2024-04-05T22:21:14.919602+0000","last_unstale":"2024-04-05T22:21:15.980822+0000","last_undegraded":"2024-04-05T22:21:15.980822+0000","last_fullsized":"2024-04-05T22:21:15.980822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976328+0000","last_change":"2024-04-05T22:21:14.437206+0000","last_active":"2024-04-05T22:21:15.976328+0000","last_peered":"2024-04-05T22:21:15.976328+0000","last_clean":"2024-04-05T22:21:15.976328+0000","last_became_active":"2024-04-05T22:21:14.436852+0000","last_became_peered":"2024-04-05T22:21:14.436852+0000","last_unstale":"2024-04-05T22:21:15.976328+0000","last_undegraded":"2024-04-05T22:21:15.976328+0000","last_fullsized":"2024-04-05T22:21:15.976328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975906+0000","last_change":"2024-04-05T22:21:14.933572+0000","last_active":"2024-04-05T22:21:15.975906+0000","last_peered":"2024-04-05T22:21:15.975906+0000","last_clean":"2024-04-05T22:21:15.975906+0000","last_became_active":"2024-04-05T22:21:14.933310+0000","last_became_peered":"2024-04-05T22:21:14.933310+0000","last_unstale":"2024-04-05T22:21:15.975906+0000","last_undegraded":"2024-04-05T22:21:15.975906+0000","last_fullsized":"2024-04-05T22:21:15.975906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:10:27.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899122+0000","last_change":"2024-04-05T22:21:14.435387+0000","last_active":"2024-04-05T22:21:15.899122+0000","last_peered":"2024-04-05T22:21:15.899122+0000","last_clean":"2024-04-05T22:21:15.899122+0000","last_became_active":"2024-04-05T22:21:14.435061+0000","last_became_peered":"2024-04-05T22:21:14.435061+0000","last_unstale":"2024-04-05T22:21:15.899122+0000","last_undegraded":"2024-04-05T22:21:15.899122+0000","last_fullsized":"2024-04-05T22:21:15.899122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980213+0000","last_change":"2024-04-05T22:21:14.935722+0000","last_active":"2024-04-05T22:21:15.980213+0000","last_peered":"2024-04-05T22:21:15.980213+0000","last_clean":"2024-04-05T22:21:15.980213+0000","last_became_active":"2024-04-05T22:21:14.935463+0000","last_became_peered":"2024-04-05T22:21:14.935463+0000","last_unstale":"2024-04-05T22:21:15.980213+0000","last_undegraded":"2024-04-05T22:21:15.980213+0000","last_fullsized":"2024-04-05T22:21:15.980213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980372+0000","last_change":"2024-04-05T22:21:14.449636+0000","last_active":"2024-04-05T22:21:15.980372+0000","last_peered":"2024-04-05T22:21:15.980372+0000","last_clean":"2024-04-05T22:21:15.980372+0000","last_became_active":"2024-04-05T22:21:14.449257+0000","last_became_peered":"2024-04-05T22:21:14.449257+0000","last_unstale":"2024-04-05T22:21:15.980372+0000","last_undegraded":"2024-04-05T22:21:15.980372+0000","last_fullsized":"2024-04-05T22:21:15.980372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:56:39.430553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980094+0000","last_change":"2024-04-05T22:21:14.916105+0000","last_active":"2024-04-05T22:21:15.980094+0000","last_peered":"2024-04-05T22:21:15.980094+0000","last_clean":"2024-04-05T22:21:15.980094+0000","last_became_active":"2024-04-05T22:21:14.915590+0000","last_became_peered":"2024-04-05T22:21:14.915590+0000","last_unstale":"2024-04-05T22:21:15.980094+0000","last_undegraded":"2024-04-05T22:21:15.980094+0000","last_fullsized":"2024-04-05T22:21:15.980094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:21:38.340157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897721+0000","last_change":"2024-04-05T22:21:14.429654+0000","last_active":"2024-04-05T22:21:15.897721+0000","last_peered":"2024-04-05T22:21:15.897721+0000","last_clean":"2024-04-05T22:21:15.897721+0000","last_became_active":"2024-04-05T22:21:14.428871+0000","last_became_peered":"2024-04-05T22:21:14.428871+0000","last_unstale":"2024-04-05T22:21:15.897721+0000","last_undegraded":"2024-04-05T22:21:15.897721+0000","last_fullsized":"2024-04-05T22:21:15.897721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979204+0000","last_change":"2024-04-05T22:21:14.921522+0000","last_active":"2024-04-05T22:21:15.979204+0000","last_peered":"2024-04-05T22:21:15.979204+0000","last_clean":"2024-04-05T22:21:15.979204+0000","last_became_active":"2024-04-05T22:21:14.921331+0000","last_became_peered":"2024-04-05T22:21:14.921331+0000","last_unstale":"2024-04-05T22:21:15.979204+0000","last_undegraded":"2024-04-05T22:21:15.979204+0000","last_fullsized":"2024-04-05T22:21:15.979204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983583+0000","last_change":"2024-04-05T22:21:14.437535+0000","last_active":"2024-04-05T22:21:15.983583+0000","last_peered":"2024-04-05T22:21:15.983583+0000","last_clean":"2024-04-05T22:21:15.983583+0000","last_became_active":"2024-04-05T22:21:14.437228+0000","last_became_peered":"2024-04-05T22:21:14.437228+0000","last_unstale":"2024-04-05T22:21:15.983583+0000","last_undegraded":"2024-04-05T22:21:15.983583+0000","last_fullsized":"2024-04-05T22:21:15.983583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981661+0000","last_change":"2024-04-05T22:21:14.935527+0000","last_active":"2024-04-05T22:21:15.981661+0000","last_peered":"2024-04-05T22:21:15.981661+0000","last_clean":"2024-04-05T22:21:15.981661+0000","last_became_active":"2024-04-05T22:21:14.935121+0000","last_became_peered":"2024-04-05T22:21:14.935121+0000","last_unstale":"2024-04-05T22:21:15.981661+0000","last_undegraded":"2024-04-05T22:21:15.981661+0000","last_fullsized":"2024-04-05T22:21:15.981661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897618+0000","last_change":"2024-04-05T22:21:14.434360+0000","last_active":"2024-04-05T22:21:15.897618+0000","last_peered":"2024-04-05T22:21:15.897618+0000","last_clean":"2024-04-05T22:21:15.897618+0000","last_became_active":"2024-04-05T22:21:14.434064+0000","last_became_peered":"2024-04-05T22:21:14.434064+0000","last_unstale":"2024-04-05T22:21:15.897618+0000","last_undegraded":"2024-04-05T22:21:15.897618+0000","last_fullsized":"2024-04-05T22:21:15.897618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982201+0000","last_change":"2024-04-05T22:21:14.931694+0000","last_active":"2024-04-05T22:21:15.982201+0000","last_peered":"2024-04-05T22:21:15.982201+0000","last_clean":"2024-04-05T22:21:15.982201+0000","last_became_active":"2024-04-05T22:21:14.931452+0000","last_became_peered":"2024-04-05T22:21:14.931452+0000","last_unstale":"2024-04-05T22:21:15.982201+0000","last_undegraded":"2024-04-05T22:21:15.982201+0000","last_fullsized":"2024-04-05T22:21:15.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900462+0000","last_change":"2024-04-05T22:21:14.431053+0000","last_active":"2024-04-05T22:21:15.900462+0000","last_peered":"2024-04-05T22:21:15.900462+0000","last_clean":"2024-04-05T22:21:15.900462+0000","last_became_active":"2024-04-05T22:21:14.430719+0000","last_became_peered":"2024-04-05T22:21:14.430719+0000","last_unstale":"2024-04-05T22:21:15.900462+0000","last_undegraded":"2024-04-05T22:21:15.900462+0000","last_fullsized":"2024-04-05T22:21:15.900462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:02.241437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899853+0000","last_change":"2024-04-05T22:21:14.921494+0000","last_active":"2024-04-05T22:21:15.899853+0000","last_peered":"2024-04-05T22:21:15.899853+0000","last_clean":"2024-04-05T22:21:15.899853+0000","last_became_active":"2024-04-05T22:21:14.920872+0000","last_became_peered":"2024-04-05T22:21:14.920872+0000","last_unstale":"2024-04-05T22:21:15.899853+0000","last_undegraded":"2024-04-05T22:21:15.899853+0000","last_fullsized":"2024-04-05T22:21:15.899853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897333+0000","last_change":"2024-04-05T22:21:14.439971+0000","last_active":"2024-04-05T22:21:15.897333+0000","last_peered":"2024-04-05T22:21:15.897333+0000","last_clean":"2024-04-05T22:21:15.897333+0000","last_became_active":"2024-04-05T22:21:14.439620+0000","last_became_peered":"2024-04-05T22:21:14.439620+0000","last_unstale":"2024-04-05T22:21:15.897333+0000","last_undegraded":"2024-04-05T22:21:15.897333+0000","last_fullsized":"2024-04-05T22:21:15.897333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981058+0000","last_change":"2024-04-05T22:21:14.930698+0000","last_active":"2024-04-05T22:21:15.981058+0000","last_peered":"2024-04-05T22:21:15.981058+0000","last_clean":"2024-04-05T22:21:15.981058+0000","last_became_active":"2024-04-05T22:21:14.930442+0000","last_became_peered":"2024-04-05T22:21:14.930442+0000","last_unstale":"2024-04-05T22:21:15.981058+0000","last_undegraded":"2024-04-05T22:21:15.981058+0000","last_fullsized":"2024-04-05T22:21:15.981058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-05T22:21:14.902603+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:21:14.902603+0000","last_peered":"2024-04-05T22:21:14.902603+0000","last_clean":"2024-04-05T22:21:14.902603+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:21:14.902603+0000","last_undegraded":"2024-04-05T22:21:14.902603+0000","last_fullsized":"2024-04-05T22:21:14.902603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898869+0000","last_change":"2024-04-05T22:21:14.440806+0000","last_active":"2024-04-05T22:21:15.898869+0000","last_peered":"2024-04-05T22:21:15.898869+0000","last_clean":"2024-04-05T22:21:15.898869+0000","last_became_active":"2024-04-05T22:21:14.439437+0000","last_became_peered":"2024-04-05T22:21:14.439437+0000","last_unstale":"2024-04-05T22:21:15.898869+0000","last_undegraded":"2024-04-05T22:21:15.898869+0000","last_fullsized":"2024-04-05T22:21:15.898869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976007+0000","last_change":"2024-04-05T22:21:14.919101+0000","last_active":"2024-04-05T22:21:15.976007+0000","last_peered":"2024-04-05T22:21:15.976007+0000","last_clean":"2024-04-05T22:21:15.976007+0000","last_became_active":"2024-04-05T22:21:14.916214+0000","last_became_peered":"2024-04-05T22:21:14.916214+0000","last_unstale":"2024-04-05T22:21:15.976007+0000","last_undegraded":"2024-04-05T22:21:15.976007+0000","last_fullsized":"2024-04-05T22:21:15.976007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979251+0000","last_change":"2024-04-05T22:21:14.438539+0000","last_active":"2024-04-05T22:21:15.979251+0000","last_peered":"2024-04-05T22:21:15.979251+0000","last_clean":"2024-04-05T22:21:15.979251+0000","last_became_active":"2024-04-05T22:21:14.437939+0000","last_became_peered":"2024-04-05T22:21:14.437939+0000","last_unstale":"2024-04-05T22:21:15.979251+0000","last_undegraded":"2024-04-05T22:21:15.979251+0000","last_fullsized":"2024-04-05T22:21:15.979251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975766+0000","last_change":"2024-04-05T22:21:14.916601+0000","last_active":"2024-04-05T22:21:15.975766+0000","last_peered":"2024-04-05T22:21:15.975766+0000","last_clean":"2024-04-05T22:21:15.975766+0000","last_became_active":"2024-04-05T22:21:14.916330+0000","last_became_peered":"2024-04-05T22:21:14.916330+0000","last_unstale":"2024-04-05T22:21:15.975766+0000","last_undegraded":"2024-04-05T22:21:15.975766+0000","last_fullsized":"2024-04-05T22:21:15.975766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897037+0000","last_change":"2024-04-05T22:21:14.434305+0000","last_active":"2024-04-05T22:21:15.897037+0000","last_peered":"2024-04-05T22:21:15.897037+0000","last_clean":"2024-04-05T22:21:15.897037+0000","last_became_active":"2024-04-05T22:21:14.433877+0000","last_became_peered":"2024-04-05T22:21:14.433877+0000","last_unstale":"2024-04-05T22:21:15.897037+0000","last_undegraded":"2024-04-05T22:21:15.897037+0000","last_fullsized":"2024-04-05T22:21:15.897037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899989+0000","last_change":"2024-04-05T22:21:14.912112+0000","last_active":"2024-04-05T22:21:15.899989+0000","last_peered":"2024-04-05T22:21:15.899989+0000","last_clean":"2024-04-05T22:21:15.899989+0000","last_became_active":"2024-04-05T22:21:14.910925+0000","last_became_peered":"2024-04-05T22:21:14.910925+0000","last_unstale":"2024-04-05T22:21:15.899989+0000","last_undegraded":"2024-04-05T22:21:15.899989+0000","last_fullsized":"2024-04-05T22:21:15.899989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899675+0000","last_change":"2024-04-05T22:21:14.438374+0000","last_active":"2024-04-05T22:21:15.899675+0000","last_peered":"2024-04-05T22:21:15.899675+0000","last_clean":"2024-04-05T22:21:15.899675+0000","last_became_active":"2024-04-05T22:21:14.437921+0000","last_became_peered":"2024-04-05T22:21:14.437921+0000","last_unstale":"2024-04-05T22:21:15.899675+0000","last_undegraded":"2024-04-05T22:21:15.899675+0000","last_fullsized":"2024-04-05T22:21:15.899675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983069+0000","last_change":"2024-04-05T22:21:14.914627+0000","last_active":"2024-04-05T22:21:15.983069+0000","last_peered":"2024-04-05T22:21:15.983069+0000","last_clean":"2024-04-05T22:21:15.983069+0000","last_became_active":"2024-04-05T22:21:14.914234+0000","last_became_peered":"2024-04-05T22:21:14.914234+0000","last_unstale":"2024-04-05T22:21:15.983069+0000","last_undegraded":"2024-04-05T22:21:15.983069+0000","last_fullsized":"2024-04-05T22:21:15.983069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900119+0000","last_change":"2024-04-05T22:21:14.431183+0000","last_active":"2024-04-05T22:21:15.900119+0000","last_peered":"2024-04-05T22:21:15.900119+0000","last_clean":"2024-04-05T22:21:15.900119+0000","last_became_active":"2024-04-05T22:21:14.430912+0000","last_became_peered":"2024-04-05T22:21:14.430912+0000","last_unstale":"2024-04-05T22:21:15.900119+0000","last_undegraded":"2024-04-05T22:21:15.900119+0000","last_fullsized":"2024-04-05T22:21:15.900119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:23.153789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899958+0000","last_change":"2024-04-05T22:21:14.921009+0000","last_active":"2024-04-05T22:21:15.899958+0000","last_peered":"2024-04-05T22:21:15.899958+0000","last_clean":"2024-04-05T22:21:15.899958+0000","last_became_active":"2024-04-05T22:21:14.920657+0000","last_became_peered":"2024-04-05T22:21:14.920657+0000","last_unstale":"2024-04-05T22:21:15.899958+0000","last_undegraded":"2024-04-05T22:21:15.899958+0000","last_fullsized":"2024-04-05T22:21:15.899958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981703+0000","last_change":"2024-04-05T22:21:14.443668+0000","last_active":"2024-04-05T22:21:15.981703+0000","last_peered":"2024-04-05T22:21:15.981703+0000","last_clean":"2024-04-05T22:21:15.981703+0000","last_became_active":"2024-04-05T22:21:14.442178+0000","last_became_peered":"2024-04-05T22:21:14.442178+0000","last_unstale":"2024-04-05T22:21:15.981703+0000","last_undegraded":"2024-04-05T22:21:15.981703+0000","last_fullsized":"2024-04-05T22:21:15.981703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900022+0000","last_change":"2024-04-05T22:21:14.905357+0000","last_active":"2024-04-05T22:21:15.900022+0000","last_peered":"2024-04-05T22:21:15.900022+0000","last_clean":"2024-04-05T22:21:15.900022+0000","last_became_active":"2024-04-05T22:21:14.905068+0000","last_became_peered":"2024-04-05T22:21:14.905068+0000","last_unstale":"2024-04-05T22:21:15.900022+0000","last_undegraded":"2024-04-05T22:21:15.900022+0000","last_fullsized":"2024-04-05T22:21:15.900022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900480+0000","last_change":"2024-04-05T22:21:14.442815+0000","last_active":"2024-04-05T22:21:15.900480+0000","last_peered":"2024-04-05T22:21:15.900480+0000","last_clean":"2024-04-05T22:21:15.900480+0000","last_became_active":"2024-04-05T22:21:14.442217+0000","last_became_peered":"2024-04-05T22:21:14.442217+0000","last_unstale":"2024-04-05T22:21:15.900480+0000","last_undegraded":"2024-04-05T22:21:15.900480+0000","last_fullsized":"2024-04-05T22:21:15.900480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976452+0000","last_change":"2024-04-05T22:21:14.933779+0000","last_active":"2024-04-05T22:21:15.976452+0000","last_peered":"2024-04-05T22:21:15.976452+0000","last_clean":"2024-04-05T22:21:15.976452+0000","last_became_active":"2024-04-05T22:21:14.933248+0000","last_became_peered":"2024-04-05T22:21:14.933248+0000","last_unstale":"2024-04-05T22:21:15.976452+0000","last_undegraded":"2024-04-05T22:21:15.976452+0000","last_fullsized":"2024-04-05T22:21:15.976452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981318+0000","last_change":"2024-04-05T22:21:14.449034+0000","last_active":"2024-04-05T22:21:15.981318+0000","last_peered":"2024-04-05T22:21:15.981318+0000","last_clean":"2024-04-05T22:21:15.981318+0000","last_became_active":"2024-04-05T22:21:14.448183+0000","last_became_peered":"2024-04-05T22:21:14.448183+0000","last_unstale":"2024-04-05T22:21:15.981318+0000","last_undegraded":"2024-04-05T22:21:15.981318+0000","last_fullsized":"2024-04-05T22:21:15.981318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:29:59.320001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.977308+0000","last_change":"2024-04-05T22:21:14.921617+0000","last_active":"2024-04-05T22:21:15.977308+0000","last_peered":"2024-04-05T22:21:15.977308+0000","last_clean":"2024-04-05T22:21:15.977308+0000","last_became_active":"2024-04-05T22:21:14.921153+0000","last_became_peered":"2024-04-05T22:21:14.921153+0000","last_unstale":"2024-04-05T22:21:15.977308+0000","last_undegraded":"2024-04-05T22:21:15.977308+0000","last_fullsized":"2024-04-05T22:21:15.977308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979031+0000","last_change":"2024-04-05T22:21:14.442038+0000","last_active":"2024-04-05T22:21:15.979031+0000","last_peered":"2024-04-05T22:21:15.979031+0000","last_clean":"2024-04-05T22:21:15.979031+0000","last_became_active":"2024-04-05T22:21:14.441431+0000","last_became_peered":"2024-04-05T22:21:14.441431+0000","last_unstale":"2024-04-05T22:21:15.979031+0000","last_undegraded":"2024-04-05T22:21:15.979031+0000","last_fullsized":"2024-04-05T22:21:15.979031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982667+0000","last_change":"2024-04-05T22:21:14.930105+0000","last_active":"2024-04-05T22:21:15.982667+0000","last_peered":"2024-04-05T22:21:15.982667+0000","last_clean":"2024-04-05T22:21:15.982667+0000","last_became_active":"2024-04-05T22:21:14.927238+0000","last_became_peered":"2024-04-05T22:21:14.927238+0000","last_unstale":"2024-04-05T22:21:15.982667+0000","last_undegraded":"2024-04-05T22:21:15.982667+0000","last_fullsized":"2024-04-05T22:21:15.982667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900204+0000","last_change":"2024-04-05T22:21:14.437077+0000","last_active":"2024-04-05T22:21:15.900204+0000","last_peered":"2024-04-05T22:21:15.900204+0000","last_clean":"2024-04-05T22:21:15.900204+0000","last_became_active":"2024-04-05T22:21:14.436767+0000","last_became_peered":"2024-04-05T22:21:14.436767+0000","last_unstale":"2024-04-05T22:21:15.900204+0000","last_undegraded":"2024-04-05T22:21:15.900204+0000","last_fullsized":"2024-04-05T22:21:15.900204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975658+0000","last_change":"2024-04-05T22:21:14.920674+0000","last_active":"2024-04-05T22:21:15.975658+0000","last_peered":"2024-04-05T22:21:15.975658+0000","last_clean":"2024-04-05T22:21:15.975658+0000","last_became_active":"2024-04-05T22:21:14.919722+0000","last_became_peered":"2024-04-05T22:21:14.919722+0000","last_unstale":"2024-04-05T22:21:15.975658+0000","last_undegraded":"2024-04-05T22:21:15.975658+0000","last_fullsized":"2024-04-05T22:21:15.975658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900313+0000","last_change":"2024-04-05T22:21:14.432768+0000","last_active":"2024-04-05T22:21:15.900313+0000","last_peered":"2024-04-05T22:21:15.900313+0000","last_clean":"2024-04-05T22:21:15.900313+0000","last_became_active":"2024-04-05T22:21:14.432330+0000","last_became_peered":"2024-04-05T22:21:14.432330+0000","last_unstale":"2024-04-05T22:21:15.900313+0000","last_undegraded":"2024-04-05T22:21:15.900313+0000","last_fullsized":"2024-04-05T22:21:15.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976206+0000","last_change":"2024-04-05T22:21:14.919559+0000","last_active":"2024-04-05T22:21:15.976206+0000","last_peered":"2024-04-05T22:21:15.976206+0000","last_clean":"2024-04-05T22:21:15.976206+0000","last_became_active":"2024-04-05T22:21:14.916617+0000","last_became_peered":"2024-04-05T22:21:14.916617+0000","last_unstale":"2024-04-05T22:21:15.976206+0000","last_undegraded":"2024-04-05T22:21:15.976206+0000","last_fullsized":"2024-04-05T22:21:15.976206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980370+0000","last_change":"2024-04-05T22:21:14.445535+0000","last_active":"2024-04-05T22:21:15.980370+0000","last_peered":"2024-04-05T22:21:15.980370+0000","last_clean":"2024-04-05T22:21:15.980370+0000","last_became_active":"2024-04-05T22:21:14.438718+0000","last_became_peered":"2024-04-05T22:21:14.438718+0000","last_unstale":"2024-04-05T22:21:15.980370+0000","last_undegraded":"2024-04-05T22:21:15.980370+0000","last_fullsized":"2024-04-05T22:21:15.980370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980726+0000","last_change":"2024-04-05T22:21:14.928694+0000","last_active":"2024-04-05T22:21:15.980726+0000","last_peered":"2024-04-05T22:21:15.980726+0000","last_clean":"2024-04-05T22:21:15.980726+0000","last_became_active":"2024-04-05T22:21:14.928392+0000","last_became_peered":"2024-04-05T22:21:14.928392+0000","last_unstale":"2024-04-05T22:21:15.980726+0000","last_undegraded":"2024-04-05T22:21:15.980726+0000","last_fullsized":"2024-04-05T22:21:15.980726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:02:26.504384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900241+0000","last_change":"2024-04-05T22:21:14.919963+0000","last_active":"2024-04-05T22:21:15.900241+0000","last_peered":"2024-04-05T22:21:15.900241+0000","last_clean":"2024-04-05T22:21:15.900241+0000","last_became_active":"2024-04-05T22:21:14.917075+0000","last_became_peered":"2024-04-05T22:21:14.917075+0000","last_unstale":"2024-04-05T22:21:15.900241+0000","last_undegraded":"2024-04-05T22:21:15.900241+0000","last_fullsized":"2024-04-05T22:21:15.900241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980519+0000","last_change":"2024-04-05T22:21:14.440258+0000","last_active":"2024-04-05T22:21:15.980519+0000","last_peered":"2024-04-05T22:21:15.980519+0000","last_clean":"2024-04-05T22:21:15.980519+0000","last_became_active":"2024-04-05T22:21:14.439946+0000","last_became_peered":"2024-04-05T22:21:14.439946+0000","last_unstale":"2024-04-05T22:21:15.980519+0000","last_undegraded":"2024-04-05T22:21:15.980519+0000","last_fullsized":"2024-04-05T22:21:15.980519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899588+0000","last_change":"2024-04-05T22:21:14.914728+0000","last_active":"2024-04-05T22:21:15.899588+0000","last_peered":"2024-04-05T22:21:15.899588+0000","last_clean":"2024-04-05T22:21:15.899588+0000","last_became_active":"2024-04-05T22:21:14.914450+0000","last_became_peered":"2024-04-05T22:21:14.914450+0000","last_unstale":"2024-04-05T22:21:15.899588+0000","last_undegraded":"2024-04-05T22:21:15.899588+0000","last_fullsized":"2024-04-05T22:21:15.899588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981282+0000","last_change":"2024-04-05T22:21:14.451271+0000","last_active":"2024-04-05T22:21:15.981282+0000","last_peered":"2024-04-05T22:21:15.981282+0000","last_clean":"2024-04-05T22:21:15.981282+0000","last_became_active":"2024-04-05T22:21:14.450977+0000","last_became_peered":"2024-04-05T22:21:14.450977+0000","last_unstale":"2024-04-05T22:21:15.981282+0000","last_undegraded":"2024-04-05T22:21:15.981282+0000","last_fullsized":"2024-04-05T22:21:15.981282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.921420+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.919777+0000","last_became_peered":"2024-04-05T22:21:14.919777+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976482+0000","last_change":"2024-04-05T22:21:14.442123+0000","last_active":"2024-04-05T22:21:15.976482+0000","last_peered":"2024-04-05T22:21:15.976482+0000","last_clean":"2024-04-05T22:21:15.976482+0000","last_became_active":"2024-04-05T22:21:14.441885+0000","last_became_peered":"2024-04-05T22:21:14.441885+0000","last_unstale":"2024-04-05T22:21:15.976482+0000","last_undegraded":"2024-04-05T22:21:15.976482+0000","last_fullsized":"2024-04-05T22:21:15.976482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976454+0000","last_change":"2024-04-05T22:21:14.923884+0000","last_active":"2024-04-05T22:21:15.976454+0000","last_peered":"2024-04-05T22:21:15.976454+0000","last_clean":"2024-04-05T22:21:15.976454+0000","last_became_active":"2024-04-05T22:21:14.923545+0000","last_became_peered":"2024-04-05T22:21:14.923545+0000","last_unstale":"2024-04-05T22:21:15.976454+0000","last_undegraded":"2024-04-05T22:21:15.976454+0000","last_fullsized":"2024-04-05T22:21:15.976454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976809+0000","last_change":"2024-04-05T22:21:14.439558+0000","last_active":"2024-04-05T22:21:15.976809+0000","last_peered":"2024-04-05T22:21:15.976809+0000","last_clean":"2024-04-05T22:21:15.976809+0000","last_became_active":"2024-04-05T22:21:14.439082+0000","last_became_peered":"2024-04-05T22:21:14.439082+0000","last_unstale":"2024-04-05T22:21:15.976809+0000","last_undegraded":"2024-04-05T22:21:15.976809+0000","last_fullsized":"2024-04-05T22:21:15.976809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.946}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:21:19.787 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:21:19.787 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-05T22:21:19.787 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-05T22:21:19.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-05T22:21:19.788 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-05T22:21:20.017 INFO:tasks.ceph.mds.b.smithi157.stderr:2024-04-05T22:21:20.017+0000 7fcfb2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:21:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.026+0000 7fc092a00640 1 Processor -- start 2024-04-05T22:21:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.026+0000 7fc092a00640 1 -- start start 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc092a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc08c06cb80 0x7fc08c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc092a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc08c15e850 0x7fc08c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc092a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c06f5b0 con 0x7fc08c1587d0 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc092a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c06d500 con 0x7fc08c15e850 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc092a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c15f220 con 0x7fc08c06cb80 2024-04-05T22:21:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc08be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc08c1587d0 0x7fc08c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48744/0 (socket says 172.21.15.73:48744) 2024-04-05T22:21:20.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc08be00640 1 -- 172.21.15.73:0/769261473 learned_addr learned my addr 172.21.15.73:0/769261473 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:20.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc090a00640 1 --2- 172.21.15.73:0/769261473 >> v2:172.21.15.73:3300/0 conn(0x7fc08c15e850 0x7fc08c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc083400640 1 --2- 172.21.15.73:0/769261473 >> v2:172.21.15.157:3300/0 conn(0x7fc08c06cb80 0x7fc08c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc090a00640 1 -- 172.21.15.73:0/769261473 >> v2:172.21.15.157:3300/0 conn(0x7fc08c06cb80 msgr2=0x7fc08c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc090a00640 1 --2- 172.21.15.73:0/769261473 >> v2:172.21.15.157:3300/0 conn(0x7fc08c06cb80 0x7fc08c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc090a00640 1 -- 172.21.15.73:0/769261473 >> v1:172.21.15.73:6789/0 conn(0x7fc08c1587d0 legacy=0x7fc08c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc090a00640 1 -- 172.21.15.73:0/769261473 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c070410 con 0x7fc08c15e850 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.028+0000 7fc08b400640 1 -- 172.21.15.73:0/769261473 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323118492 0 0) 0x7fc08c06f5b0 con 0x7fc08c1587d0 2024-04-05T22:21:20.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc090a00640 1 --2- 172.21.15.73:0/769261473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15ece0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc08c06b560 tx=0x7fc08402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36345404721f51b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc08b400640 1 -- 172.21.15.73:0/769261473 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08402cc80 con 0x7fc08c15e850 2024-04-05T22:21:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc08b400640 1 -- 172.21.15.73:0/769261473 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08402cde0 con 0x7fc08c15e850 2024-04-05T22:21:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc08b400640 1 -- 172.21.15.73:0/769261473 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084034660 con 0x7fc08c15e850 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 -- 172.21.15.73:0/769261473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 msgr2=0x7fc08c15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 --2- 172.21.15.73:0/769261473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15ece0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc08c06b560 tx=0x7fc08402bed0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 -- 172.21.15.73:0/769261473 shutdown_connections 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 --2- 172.21.15.73:0/769261473 >> v2:172.21.15.157:3300/0 conn(0x7fc08c06cb80 0x7fc08c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 --2- 172.21.15.73:0/769261473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15ece0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.030+0000 7fc092a00640 1 -- 172.21.15.73:0/769261473 >> 172.21.15.73:0/769261473 conn(0x7fc08c0ce8e0 msgr2=0x7fc08c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.031+0000 7fc092a00640 1 -- 172.21.15.73:0/769261473 shutdown_connections 2024-04-05T22:21:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.031+0000 7fc092a00640 1 -- 172.21.15.73:0/769261473 wait complete. 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.032+0000 7fc092a00640 1 Processor -- start 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.032+0000 7fc092a00640 1 -- start start 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 0x7fc08c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc08c1587d0 0x7fc08c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0c3e40 con 0x7fc08c15e850 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0affc0 con 0x7fc08c1587d0 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc092a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0c3880 con 0x7fc08c06cb80 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc090a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc090a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50900/0 (socket says 172.21.15.73:50900) 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.033+0000 7fc090a00640 1 -- 172.21.15.73:0/2964309640 learned_addr learned my addr 172.21.15.73:0/2964309640 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc083400640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc08c1587d0 0x7fc08c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc08be00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 0x7fc08c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 -- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 msgr2=0x7fc08c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 0x7fc08c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 -- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc08c1587d0 msgr2=0x7fc08c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc08c1587d0 0x7fc08c0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 -- 172.21.15.73:0/2964309640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009a70 con 0x7fc08c15e850 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc08be00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 0x7fc08c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc090a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc084009e80 tx=0x7fc084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084040650 con 0x7fc08c15e850 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0840407b0 con 0x7fc08c15e850 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084040b60 con 0x7fc08c15e850 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.034+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c0c3b60 con 0x7fc08c15e850 2024-04-05T22:21:20.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.035+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c15ce70 con 0x7fc08c15e850 2024-04-05T22:21:20.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.036+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c066af0 con 0x7fc08c15e850 2024-04-05T22:21:20.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.037+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc084040cc0 con 0x7fc08c15e850 2024-04-05T22:21:20.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.037+0000 7fc088c00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 0x7fc06806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.038+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0840ac1a0 con 0x7fc08c15e850 2024-04-05T22:21:20.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.038+0000 7fc08be00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 0x7fc06806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.038+0000 7fc08be00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 0x7fc06806d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000f60 tx=0x7fc07c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.042+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc0840772e0 con 0x7fc08c15e850 2024-04-05T22:21:20.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.284+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc08c0b0130 con 0x7fc08c15e850 2024-04-05T22:21:20.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.290+0000 7fc088c00640 1 -- 172.21.15.73:0/2964309640 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170935 (secure 0 0 0) 0x7fc084009450 con 0x7fc08c15e850 2024-04-05T22:21:20.292 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.294+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 msgr2=0x7fc06806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.294+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 0x7fc06806d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000f60 tx=0x7fc07c009040 comp rx=0 tx=0).stop 2024-04-05T22:21:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.294+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 msgr2=0x7fc08c15d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.294+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc084009e80 tx=0x7fc084004a40 comp rx=0 tx=0).stop 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 shutdown_connections 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc06806ac00 0x7fc06806d0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c15e850 0x7fc08c15d390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc08c1587d0 0x7fc08c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 --2- 172.21.15.73:0/2964309640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc08c06cb80 0x7fc08c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 >> 172.21.15.73:0/2964309640 conn(0x7fc08c0ce8e0 msgr2=0x7fc08c15a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 shutdown_connections 2024-04-05T22:21:20.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.295+0000 7fc092a00640 1 -- 172.21.15.73:0/2964309640 wait complete. 2024-04-05T22:21:20.311 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":37063},{"type":"v1","addr":"172.21.15.73:6833","nonce":37063}]},"active_addr":"172.21.15.73:6833/37063","active_change":"2024-04-05T22:21:00.469624+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3883631467}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2340596025}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":745661956}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":206467021}]}]} 2024-04-05T22:21:20.315 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-05T22:21:20.315 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-05T22:21:20.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:20.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.521+0000 7f0d50c00640 1 Processor -- start 2024-04-05T22:21:20.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.521+0000 7f0d50c00640 1 -- start start 2024-04-05T22:21:20.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d50c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d4c0b1870 0x7f0d4c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d50c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c0ce5d0 con 0x7f0d4c160620 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d50c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c0ce010 con 0x7f0d4c0ad6b0 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d50c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c0ce2f0 con 0x7f0d4c0b1870 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d49e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d4c160620 0x7f0d4c0caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48752/0 (socket says 172.21.15.73:48752) 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d49e00640 1 -- 172.21.15.73:0/1638801648 learned_addr learned my addr 172.21.15.73:0/1638801648 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030304888 0 0) 0x7f0d4c0ce5d0 con 0x7f0d4c160620 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.523+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d30003520 con 0x7f0d4c160620 2024-04-05T22:21:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d41400640 1 --2- 172.21.15.73:0/1638801648 >> v2:172.21.15.157:3300/0 conn(0x7f0d4c0b1870 0x7f0d4c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 405044369 0 0) 0x7f0d30003520 con 0x7f0d4c160620 2024-04-05T22:21:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 >> v2:172.21.15.157:3300/0 conn(0x7f0d4c0b1870 msgr2=0x7f0d4c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 --2- 172.21.15.73:0/1638801648 >> v2:172.21.15.157:3300/0 conn(0x7f0d4c0b1870 0x7f0d4c0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 >> v1:172.21.15.157:6789/0 conn(0x7f0d4c0ad6b0 legacy=0x7f0d4c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0ac970 con 0x7f0d4c160620 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0d3c001670 con 0x7f0d4c160620 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 >> v1:172.21.15.73:6789/0 conn(0x7f0d4c160620 legacy=0x7f0d4c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d30003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d300047c0 0x7f0d30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d300051d0 0x7f0d3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0ce2f0 con 0x7f0d30003970 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3000c100 con 0x7f0d300047c0 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3000c2c0 con 0x7f0d300051d0 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.524+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d30003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d49e00640 1 -- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d300051d0 msgr2=0x7f0d3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d4a800640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d300051d0 0x7f0d3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d41400640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d300047c0 0x7f0d30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d300051d0 0x7f0d3000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d49e00640 1 -- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d300047c0 msgr2=0x7f0d30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d300047c0 0x7f0d30004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d49e00640 1 -- 172.21.15.73:0/1638801648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30003520 con 0x7f0d30003970 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.525+0000 7f0d4a800640 1 -- 172.21.15.73:0/1638801648 reap_dead start 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.526+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d30003db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c240 tx=0x7f0d300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9561057dd542ae server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.526+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c005ad0 con 0x7f0d30003970 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.526+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c005c30 con 0x7f0d30003970 2024-04-05T22:21:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.526+0000 7f0d49400640 1 -- 172.21.15.73:0/1638801648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c02caf0 con 0x7f0d30003970 2024-04-05T22:21:20.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 -- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 msgr2=0x7f0d30003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d30003db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c240 tx=0x7f0d300057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 -- 172.21.15.73:0/1638801648 shutdown_connections 2024-04-05T22:21:20.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1638801648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d30003db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 -- 172.21.15.73:0/1638801648 >> 172.21.15.73:0/1638801648 conn(0x7f0d4c0ce8e0 msgr2=0x7f0d4c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 -- 172.21.15.73:0/1638801648 shutdown_connections 2024-04-05T22:21:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.527+0000 7f0d50c00640 1 -- 172.21.15.73:0/1638801648 wait complete. 2024-04-05T22:21:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.529+0000 7f0d50c00640 1 Processor -- start 2024-04-05T22:21:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.529+0000 7f0d50c00640 1 -- start start 2024-04-05T22:21:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c15a2e0 0x7f0d4c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c1580b0 0x7f0d4c0b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c15a810 con 0x7f0d30003970 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c15aaf0 con 0x7f0d4c1580b0 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d50c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c15add0 con 0x7f0d4c15a2e0 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50908/0 (socket says 172.21.15.73:50908) 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 -- 172.21.15.73:0/1019768586 learned_addr learned my addr 172.21.15.73:0/1019768586 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 -- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c15a2e0 msgr2=0x7f0d4c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c15a2e0 0x7f0d4c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 -- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c1580b0 msgr2=0x7f0d4c0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c1580b0 0x7f0d4c0b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.530+0000 7f0d49e00640 1 -- 172.21.15.73:0/1019768586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c15b0b0 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d49e00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c02c450 tx=0x7f0d3c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c03c040 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c02ccc0 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c15b390 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c007290 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.531+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c0b3740 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.532+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c066af0 con 0x7f0d30003970 2024-04-05T22:21:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.533+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d3c032070 con 0x7f0d30003970 2024-04-05T22:21:20.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.534+0000 7f0d42a00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 0x7f0d2806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.534+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d3c0aa9f0 con 0x7f0d30003970 2024-04-05T22:21:20.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.534+0000 7f0d41400640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 0x7f0d2806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.535+0000 7f0d41400640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 0x7f0d2806d110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006a90 tx=0x7f0d34006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.539+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0d3c075d80 con 0x7f0d30003970 2024-04-05T22:21:20.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.732+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d4c068160 con 0x7f0d30003970 2024-04-05T22:21:20.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.735+0000 7f0d42a00640 1 -- 172.21.15.73:0/1019768586 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0d3c046020 con 0x7f0d30003970 2024-04-05T22:21:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:20.734 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:20.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.737+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 msgr2=0x7f0d2806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.737+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 0x7f0d2806d110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006a90 tx=0x7f0d34006a20 comp rx=0 tx=0).stop 2024-04-05T22:21:20.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.737+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 msgr2=0x7f0d4c156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.737+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c02c450 tx=0x7f0d3c02e070 comp rx=0 tx=0).stop 2024-04-05T22:21:20.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 shutdown_connections 2024-04-05T22:21:20.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0d2806ac50 0x7f0d2806d110 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c1580b0 0x7f0d4c0b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c15a2e0 0x7f0d4c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 --2- 172.21.15.73:0/1019768586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30003970 0x7f0d4c156f30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 >> 172.21.15.73:0/1019768586 conn(0x7f0d4c0ce8e0 msgr2=0x7f0d4c0cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 shutdown_connections 2024-04-05T22:21:20.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.738+0000 7f0d50c00640 1 -- 172.21.15.73:0/1019768586 wait complete. 2024-04-05T22:21:20.755 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-05T22:21:20.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.959+0000 7fe71ce00640 1 Processor -- start 2024-04-05T22:21:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.959+0000 7fe71ce00640 1 -- start start 2024-04-05T22:21:20.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe71ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7181587d0 0x7fe7180cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71815e850 0x7fe71815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe71ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71806f5b0 con 0x7fe7180ccd20 2024-04-05T22:21:20.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe71ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71815f240 con 0x7fe7181587d0 2024-04-05T22:21:20.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe71ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe718070410 con 0x7fe71815e850 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe715e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7181587d0 0x7fe7180cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe715e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7181587d0 0x7fe7180cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50916/0 (socket says 172.21.15.73:50916) 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe715400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe7180ccd20 0x7fe71806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48760/0 (socket says 172.21.15.73:48760) 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe715e00640 1 -- 172.21.15.73:0/1104543042 learned_addr learned my addr 172.21.15.73:0/1104543042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.961+0000 7fe716800640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71815e850 0x7fe71815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.962+0000 7fe715e00640 1 -- 172.21.15.73:0/1104543042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71815e850 msgr2=0x7fe71815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.962+0000 7fe715e00640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71815e850 0x7fe71815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.962+0000 7fe715e00640 1 -- 172.21.15.73:0/1104543042 >> v1:172.21.15.73:6789/0 conn(0x7fe7180ccd20 legacy=0x7fe71806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.962+0000 7fe714a00640 1 -- 172.21.15.73:0/1104543042 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261677983 0 0) 0x7fe71806f5b0 con 0x7fe7180ccd20 2024-04-05T22:21:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.962+0000 7fe715e00640 1 -- 172.21.15.73:0/1104543042 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718158bb0 con 0x7fe7181587d0 2024-04-05T22:21:20.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.963+0000 7fe715e00640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7181587d0 0x7fe7180cc7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe708009870 tx=0x7fe70802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6513a31e44582b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.963+0000 7fe714a00640 1 -- 172.21.15.73:0/1104543042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708030840 con 0x7fe7181587d0 2024-04-05T22:21:20.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.963+0000 7fe714a00640 1 -- 172.21.15.73:0/1104543042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7080309a0 con 0x7fe7181587d0 2024-04-05T22:21:20.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.963+0000 7fe714a00640 1 -- 172.21.15.73:0/1104543042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708030d30 con 0x7fe7181587d0 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 -- 172.21.15.73:0/1104543042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7181587d0 msgr2=0x7fe7180cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7181587d0 0x7fe7180cc7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe708009870 tx=0x7fe70802b980 comp rx=0 tx=0).stop 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 -- 172.21.15.73:0/1104543042 shutdown_connections 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71815e850 0x7fe71815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1104543042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7181587d0 0x7fe7180cc7e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 -- 172.21.15.73:0/1104543042 >> 172.21.15.73:0/1104543042 conn(0x7fe7180ce8e0 msgr2=0x7fe7180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 -- 172.21.15.73:0/1104543042 shutdown_connections 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.964+0000 7fe71ce00640 1 -- 172.21.15.73:0/1104543042 wait complete. 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.966+0000 7fe71ce00640 1 Processor -- start 2024-04-05T22:21:20.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.966+0000 7fe71ce00640 1 -- start start 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 0x7fe7180ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 0x7fe718148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71814a110 con 0x7fe71815e850 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718145cb0 con 0x7fe718148000 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe71ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe718149500 con 0x7fe7181587d0 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe715400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe715400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50918/0 (socket says 172.21.15.73:50918) 2024-04-05T22:21:20.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.967+0000 7fe715400640 1 -- 172.21.15.73:0/1802715604 learned_addr learned my addr 172.21.15.73:0/1802715604 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715e00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 0x7fe7180ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe716800640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 0x7fe718148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 -- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 msgr2=0x7fe7180ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 0x7fe7180ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 -- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 msgr2=0x7fe718148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 0x7fe718148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 -- 172.21.15.73:0/1802715604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080094d0 con 0x7fe71815e850 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe716800640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 0x7fe718148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715e00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 0x7fe7180ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe715400640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c4d0 tx=0x7fe70400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040074e0 con 0x7fe71815e850 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704007660 con 0x7fe71815e850 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7181497e0 con 0x7fe71815e850 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704004a20 con 0x7fe71815e850 2024-04-05T22:21:20.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.968+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe718149da0 con 0x7fe71815e850 2024-04-05T22:21:20.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.969+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe718066af0 con 0x7fe71815e850 2024-04-05T22:21:20.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.970+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe704004b80 con 0x7fe71815e850 2024-04-05T22:21:20.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.971+0000 7fe70e000640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 0x7fe6f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:20.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.971+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7040889c0 con 0x7fe71815e850 2024-04-05T22:21:20.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.971+0000 7fe715e00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 0x7fe6f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:20.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.972+0000 7fe715e00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 0x7fe6f806d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe708036000 tx=0x7fe70803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:20.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:20.976+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe704053b00 con 0x7fe71815e850 2024-04-05T22:21:21.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.172+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe718068160 con 0x7fe71815e850 2024-04-05T22:21:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.175+0000 7fe70e000640 1 -- 172.21.15.73:0/1802715604 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fe704018020 con 0x7fe71815e850 2024-04-05T22:21:21.174 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:21.174 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 msgr2=0x7fe6f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 0x7fe6f806d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe708036000 tx=0x7fe70803b040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 msgr2=0x7fe7180accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c4d0 tx=0x7fe70400c990 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 shutdown_connections 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe6f806ac00 0x7fe6f806d0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe718148000 0x7fe718148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71815e850 0x7fe7180accb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 --2- 172.21.15.73:0/1802715604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7181587d0 0x7fe7180ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.177+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 >> 172.21.15.73:0/1802715604 conn(0x7fe7180ce8e0 msgr2=0x7fe718158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.178+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 shutdown_connections 2024-04-05T22:21:21.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.178+0000 7fe71ce00640 1 -- 172.21.15.73:0/1802715604 wait complete. 2024-04-05T22:21:21.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-05T22:21:21.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-05T22:21:21.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-05T22:21:21.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-05T22:21:21.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-05T22:21:21.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-05T22:21:21.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-05T22:21:21.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-05T22:21:21.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.493+0000 7f7392600640 1 Processor -- start 2024-04-05T22:21:21.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.493+0000 7f7392600640 1 -- start start 2024-04-05T22:21:21.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.493+0000 7f7bc5400640 1 Processor -- start 2024-04-05T22:21:21.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.493+0000 7f0dd3e00640 1 Processor -- start 2024-04-05T22:21:21.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.493+0000 7f7bc5400640 1 -- start start 2024-04-05T22:21:21.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.494+0000 7f0dd3e00640 1 -- start start 2024-04-05T22:21:21.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.495+0000 7f7392600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c15cf70 0x7f738c0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.495+0000 7f7392600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.495+0000 7f7bc5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc015ce40 0x7f7bc00d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f7bc5400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 0x7f7bc00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f7392600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0ceee0 con 0x7f738c15d350 2024-04-05T22:21:21.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f7392600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c0ce5b0 con 0x7f738c15cf70 2024-04-05T22:21:21.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f0dd3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ce70 0x7f0dd40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f7392600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0ce890 con 0x7f738c0d0bd0 2024-04-05T22:21:21.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f738b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c15cf70 0x7f738c0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f738b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c15cf70 0x7f738c0d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50932/0 (socket says 172.21.15.73:50932) 2024-04-05T22:21:21.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f738b400640 1 -- 172.21.15.73:0/3728843594 learned_addr learned my addr 172.21.15.73:0/3728843594 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f0dd3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0dd40d13e0 0x7f0dd40b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f738aa00640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f738b400640 1 -- 172.21.15.73:0/3728843594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 msgr2=0x7f738c0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f738b400640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f738b400640 1 -- 172.21.15.73:0/3728843594 >> v1:172.21.15.157:6789/0 conn(0x7f738c15d350 legacy=0x7f738c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f738b400640 1 -- 172.21.15.73:0/3728843594 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0ceb70 con 0x7f738c15cf70 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f0dd3e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40d03a0 con 0x7f0dd40d0840 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f0dd3e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd40ce400 con 0x7f0dd40d13e0 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.496+0000 7f0dd3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd40ce6e0 con 0x7f0dd415ce70 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd2a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ce70 0x7f0dd40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0dd40d13e0 0x7f0dd40b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bbf200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 0x7f7bc00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0dd40d13e0 0x7f0dd40b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50952/0 (socket says 172.21.15.73:50952) 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 -- 172.21.15.73:0/1870542584 learned_addr learned my addr 172.21.15.73:0/1870542584 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bbe800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc015ce40 0x7f7bc00d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bbe800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc015ce40 0x7f7bc00d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50946/0 (socket says 172.21.15.73:50946) 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bbe800640 1 -- 172.21.15.73:0/342582138 learned_addr learned my addr 172.21.15.73:0/342582138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc00ce9d0 con 0x7f7bc00c9210 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00c6de0 con 0x7f7bc015ce40 2024-04-05T22:21:21.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00d1b90 con 0x7f7bc00ce0d0 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 -- 172.21.15.73:0/1870542584 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ce70 msgr2=0x7f0dd40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 --2- 172.21.15.73:0/1870542584 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ce70 0x7f0dd40d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 -- 172.21.15.73:0/1870542584 >> v1:172.21.15.157:6789/0 conn(0x7f0dd40d0840 legacy=0x7f0dd40d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f0dd3400640 1 -- 172.21.15.73:0/1870542584 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40ce9c0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.497+0000 7f7bbf200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 0x7f7bc00d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57574/0 (socket says 172.21.15.73:57574) 2024-04-05T22:21:21.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f738b400640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c0d3910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2383450df0488620 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f7bbe800640 1 -- 172.21.15.73:0/342582138 >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 msgr2=0x7f7bc00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f7bbe800640 1 --2- 172.21.15.73:0/342582138 >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 0x7f7bc00d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f7bbe800640 1 -- 172.21.15.73:0/342582138 >> v1:172.21.15.73:6789/0 conn(0x7f7bc00c9210 legacy=0x7f7bc00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f7bbe800640 1 -- 172.21.15.73:0/342582138 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00ce6f0 con 0x7f7bc015ce40 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f738a000640 1 -- 172.21.15.73:0/3728843594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0317c0 con 0x7f738c15cf70 2024-04-05T22:21:21.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f738a000640 1 -- 172.21.15.73:0/3728843594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c031920 con 0x7f738c15cf70 2024-04-05T22:21:21.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f738a000640 1 -- 172.21.15.73:0/3728843594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c031cb0 con 0x7f738c15cf70 2024-04-05T22:21:21.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.500+0000 7f39e5e00640 1 Processor -- start 2024-04-05T22:21:21.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f7392600640 1 -- 172.21.15.73:0/3728843594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 msgr2=0x7f738c0d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f7392600640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c0d3910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c02b860 comp rx=0 tx=0).stop 2024-04-05T22:21:21.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.500+0000 7f39e5e00640 1 -- start start 2024-04-05T22:21:21.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f7bbe800640 1 --2- 172.21.15.73:0/342582138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00d1110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d1040 tx=0x7f7bb002b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bbeaa32566371 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f7bbd400640 1 -- 172.21.15.73:0/342582138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0030710 con 0x7f7bc015ce40 2024-04-05T22:21:21.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7bbd400640 1 -- 172.21.15.73:0/342582138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb002e050 con 0x7f7bc015ce40 2024-04-05T22:21:21.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7bbd400640 1 -- 172.21.15.73:0/342582138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0030a90 con 0x7f7bc015ce40 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 -- 172.21.15.73:0/3728843594 shutdown_connections 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 --2- 172.21.15.73:0/3728843594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c0d3910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 -- 172.21.15.73:0/3728843594 >> 172.21.15.73:0/3728843594 conn(0x7f738c0d2df0 msgr2=0x7f738c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 -- 172.21.15.73:0/3728843594 shutdown_connections 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.499+0000 7f0dd3400640 1 --2- 172.21.15.73:0/1870542584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40b26a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd408b470 tx=0x7f0dcc02bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43c5eb4dded79ec server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.500+0000 7f0dd1600640 1 -- 172.21.15.73:0/1870542584 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc02caa0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.500+0000 7f0dd1600640 1 -- 172.21.15.73:0/1870542584 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc02cc00 con 0x7f0dd40d13e0 2024-04-05T22:21:21.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.500+0000 7f0dd1600640 1 -- 172.21.15.73:0/1870542584 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0343f0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f0dd3e00640 1 -- 172.21.15.73:0/1870542584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd40d13e0 msgr2=0x7f0dd40b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.501+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/1870542584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40b26a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd408b470 tx=0x7f0dcc02bf00 comp rx=0 tx=0).stop 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 -- 172.21.15.73:0/1870542584 shutdown_connections 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/1870542584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40b26a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/1870542584 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ce70 0x7f0dd40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 -- 172.21.15.73:0/1870542584 >> 172.21.15.73:0/1870542584 conn(0x7f0dd40d2a60 msgr2=0x7f0dd40d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 -- 172.21.15.73:0/1870542584 shutdown_connections 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f0dd3e00640 1 -- 172.21.15.73:0/1870542584 wait complete. 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f39e5e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39e00b1870 0x7f39e00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f39e5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f39e00add80 0x7f39e00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f39e5e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00ae1d0 con 0x7f39e014ffd0 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f39e5e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e015a450 con 0x7f39e00add80 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f39e5e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00b1d00 con 0x7f39e00b1870 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39dea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f39e014ffd0 0x7f39e00b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48782/0 (socket says 172.21.15.73:48782) 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39dea00640 1 -- 172.21.15.73:0/635376389 learned_addr learned my addr 172.21.15.73:0/635376389 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39df400640 1 --2- 172.21.15.73:0/635376389 >> v2:172.21.15.73:3300/0 conn(0x7f39e00add80 0x7f39e00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.502+0000 7f7392600640 1 -- 172.21.15.73:0/3728843594 wait complete. 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39df400640 1 -- 172.21.15.73:0/635376389 >> v2:172.21.15.157:3300/0 conn(0x7f39e00b1870 msgr2=0x7f39e00ad840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39df400640 1 --2- 172.21.15.73:0/635376389 >> v2:172.21.15.157:3300/0 conn(0x7f39e00b1870 0x7f39e00ad840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39df400640 1 -- 172.21.15.73:0/635376389 >> v1:172.21.15.73:6789/0 conn(0x7f39e014ffd0 legacy=0x7f39e00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.503+0000 7f39df400640 1 -- 172.21.15.73:0/635376389 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e01503b0 con 0x7f39e00add80 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.504+0000 7f7392600640 1 Processor -- start 2024-04-05T22:21:21.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 msgr2=0x7f7bc00d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f0dd3e00640 1 Processor -- start 2024-04-05T22:21:21.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7392600640 1 -- start start 2024-04-05T22:21:21.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 --2- 172.21.15.73:0/342582138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00d1110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d1040 tx=0x7f7bb002b750 comp rx=0 tx=0).stop 2024-04-05T22:21:21.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 shutdown_connections 2024-04-05T22:21:21.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 --2- 172.21.15.73:0/342582138 >> v2:172.21.15.157:3300/0 conn(0x7f7bc00ce0d0 0x7f7bc00d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 --2- 172.21.15.73:0/342582138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00d1110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 >> 172.21.15.73:0/342582138 conn(0x7f7bc009be00 msgr2=0x7f7bc00d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f39df400640 1 --2- 172.21.15.73:0/635376389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00add80 0x7f39e00ccb10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39e00ae4b0 tx=0x7f39d802cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5522c8aae8bee05b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 shutdown_connections 2024-04-05T22:21:21.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7bc5400640 1 -- 172.21.15.73:0/342582138 wait complete. 2024-04-05T22:21:21.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f39dd600640 1 -- 172.21.15.73:0/635376389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d802d940 con 0x7f39e00add80 2024-04-05T22:21:21.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.505+0000 7f0dd3e00640 1 -- start start 2024-04-05T22:21:21.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7fe06f800640 1 Processor -- start 2024-04-05T22:21:21.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7fe06f800640 1 -- start start 2024-04-05T22:21:21.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f9e43000640 1 Processor -- start 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 0x7f0dd4158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd415ce70 0x7f0dd40cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40cb550 con 0x7f0dd415ce70 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40caf90 con 0x7f0dd40d13e0 2024-04-05T22:21:21.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40cb270 con 0x7f0dd40d0840 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c15d350 0x7f738c09d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c09cad0 con 0x7f738c15cf70 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15a8b0 con 0x7f738c15d350 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.506+0000 7f7392600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c09dd40 con 0x7f738c0d0bd0 2024-04-05T22:21:21.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f39dd600640 1 -- 172.21.15.73:0/635376389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d802daa0 con 0x7f39e00add80 2024-04-05T22:21:21.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 0x7f0dd4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 0x7f0dd4158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56126/0 (socket says 172.21.15.73:56126) 2024-04-05T22:21:21.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f0dd2a00640 1 -- 172.21.15.73:0/2329640509 learned_addr learned my addr 172.21.15.73:0/2329640509 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f0dd2000640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.507+0000 7f9e43000640 1 -- start start 2024-04-05T22:21:21.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f738be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c15d350 0x7f738c09d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f738b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd3400640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd415ce70 0x7f0dd40cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f738b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56140/0 (socket says 172.21.15.73:56140) 2024-04-05T22:21:21.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f738b400640 1 -- 172.21.15.73:0/1558265107 learned_addr learned my addr 172.21.15.73:0/1558265107 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f738aa00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd2000640 1 -- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 msgr2=0x7f0dd4158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f7bc5400640 1 Processor -- start 2024-04-05T22:21:21.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f7bc5400640 1 -- start start 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd2000640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 0x7f0dd4158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd2000640 1 -- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd415ce70 msgr2=0x7f0dd40cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd2000640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dd415ce70 0x7f0dd40cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f0dd2000640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009b30 con 0x7f0dd40d13e0 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f0dd2a00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40d0840 0x7f0dd4158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f39e5e00640 1 -- 172.21.15.73:0/635376389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00add80 msgr2=0x7f39e00ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.508+0000 7f39e5e00640 1 --2- 172.21.15.73:0/635376389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00add80 0x7f39e00ccb10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39e00ae4b0 tx=0x7f39d802cb90 comp rx=0 tx=0).stop 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f9e43000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 0x7f9e34054050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f9e43000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e340531f0 0x7f9e340073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f39e5e00640 1 -- 172.21.15.73:0/635376389 shutdown_connections 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f39e5e00640 1 --2- 172.21.15.73:0/635376389 >> v2:172.21.15.157:3300/0 conn(0x7f39e00b1870 0x7f39e00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f39e5e00640 1 --2- 172.21.15.73:0/635376389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00add80 0x7f39e00ccb10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f39e5e00640 1 -- 172.21.15.73:0/635376389 >> 172.21.15.73:0/635376389 conn(0x7f39e00ce8e0 msgr2=0x7f39e00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f738be00640 1 -- 172.21.15.73:0/1558265107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 msgr2=0x7f738c0a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f738be00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738c0d0bd0 0x7f738c0a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f738be00640 1 -- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 msgr2=0x7f738c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f738be00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c15cf70 0x7f738c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f738be00640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0094d0 con 0x7f738c15d350 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7f0dd2000640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40ca4f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b3e0 tx=0x7f0dc400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004020 con 0x7f0dd40d13e0 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7fe06f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 0x7fe0680b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7fe06f800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0680c3eb0 0x7fe068161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7fe06f800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680d1550 con 0x7fe0680c3a50 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7fe06f800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0680b13a0 con 0x7fe0680c3eb0 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.509+0000 7fe06f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680ceda0 con 0x7fe0680b0fc0 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e43000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e3404fca0 0x7f9e34050130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e41200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e340531f0 0x7f9e340073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e43000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340df7a0 con 0x7f9e340df3c0 2024-04-05T22:21:21.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e43000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e34054590 con 0x7f9e340531f0 2024-04-05T22:21:21.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4004180 con 0x7f0dd40d13e0 2024-04-05T22:21:21.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011630 con 0x7f0dd40d13e0 2024-04-05T22:21:21.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e41c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 0x7f9e34054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7fe06ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 0x7fe0680b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f39e5e00640 1 -- 172.21.15.73:0/635376389 shutdown_connections 2024-04-05T22:21:21.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e43000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34050670 con 0x7f9e3404fca0 2024-04-05T22:21:21.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7fe067e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe0680c3a50 0x7fe068160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60202/0 (socket says 172.21.15.73:60202) 2024-04-05T22:21:21.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7fe067e00640 1 -- 172.21.15.73:0/906332625 learned_addr learned my addr 172.21.15.73:0/906332625 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e41c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 0x7f9e34054050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50976/0 (socket says 172.21.15.73:50976) 2024-04-05T22:21:21.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e41200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e340531f0 0x7f9e340073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57616/0 (socket says 172.21.15.73:57616) 2024-04-05T22:21:21.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.510+0000 7f9e41c00640 1 -- 172.21.15.73:0/1349845484 learned_addr learned my addr 172.21.15.73:0/1349845484 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e44600640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e3404fca0 0x7f9e34050130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc00c9210 0x7f7bc01ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc00ce0d0 0x7f7bc015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc01ff610 con 0x7f7bc015ce40 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc015a700 con 0x7f7bc00c9210 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bc5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0154af0 con 0x7f7bc00ce0d0 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f738be00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c15d350 0x7f738c09d800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f738400be00 tx=0x7f738400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f7bbe800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc00c9210 0x7f7bc01ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd40b57b0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd4158a50 con 0x7f0dd40d13e0 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f39e5e00640 1 -- 172.21.15.73:0/635376389 wait complete. 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e41200640 1 -- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e3404fca0 msgr2=0x7f9e34050130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e41200640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e3404fca0 0x7f9e34050130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e41200640 1 -- 172.21.15.73:0/1349845484 >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 msgr2=0x7f9e34054050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e41200640 1 --2- 172.21.15.73:0/1349845484 >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 0x7f9e34054050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.511+0000 7f9e41200640 1 -- 172.21.15.73:0/1349845484 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34006030 con 0x7f9e340531f0 2024-04-05T22:21:21.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f7bbde00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc00ce0d0 0x7f7bc015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0dd40cef80 con 0x7f0dd40d13e0 2024-04-05T22:21:21.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f7bbde00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc00ce0d0 0x7f7bc015ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56166/0 (socket says 172.21.15.73:56166) 2024-04-05T22:21:21.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f7bbde00640 1 -- 172.21.15.73:0/441737638 learned_addr learned my addr 172.21.15.73:0/441737638 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f7bbf200640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7f0a0ea00640 1 Processor -- start 2024-04-05T22:21:21.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7fe067400640 1 -- 172.21.15.73:0/906332625 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552461799 0 0) 0x7fe0680d1550 con 0x7fe0680c3a50 2024-04-05T22:21:21.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.512+0000 7fe06d800640 1 --2- 172.21.15.73:0/906332625 >> v2:172.21.15.73:3300/0 conn(0x7fe0680c3eb0 0x7fe068161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400ce70 con 0x7f738c15d350 2024-04-05T22:21:21.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f0a0ea00640 1 -- start start 2024-04-05T22:21:21.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384003970 con 0x7f738c15d350 2024-04-05T22:21:21.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384003d20 con 0x7f738c15d350 2024-04-05T22:21:21.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7fe067400640 1 -- 172.21.15.73:0/906332625 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003520 con 0x7fe0680c3a50 2024-04-05T22:21:21.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 -- 172.21.15.73:0/441737638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc00ce0d0 msgr2=0x7f7bc015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc00ce0d0 0x7f7bc015ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 -- 172.21.15.73:0/441737638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc00c9210 msgr2=0x7f7bc01ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc00c9210 0x7f7bc01ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00094d0 con 0x7f7bc015ce40 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7bbf200640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00b48a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8007d90 tx=0x7f7bb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8018070 con 0x7f7bc015ce40 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800edd0 con 0x7f7bc015ce40 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8003bb0 con 0x7f7bc015ce40 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f7bc5400640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc015a0b0 con 0x7f7bc015ce40 2024-04-05T22:21:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c0c5fa0 con 0x7f738c15d350 2024-04-05T22:21:21.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f39e5e00640 1 Processor -- start 2024-04-05T22:21:21.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7fe06ce00640 1 -- 172.21.15.73:0/906332625 >> v2:172.21.15.73:3300/0 conn(0x7fe0680c3eb0 msgr2=0x7fe068161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7f9e41200640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e340531f0 0x7f9e340073b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400b590 tx=0x7f9e2802fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d9e43352c56819 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f9e3be00640 1 -- 172.21.15.73:0/1349845484 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280307f0 con 0x7f9e340531f0 2024-04-05T22:21:21.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f9e3be00640 1 -- 172.21.15.73:0/1349845484 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28030950 con 0x7f9e340531f0 2024-04-05T22:21:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f8ab9e00640 1 Processor -- start 2024-04-05T22:21:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f8ab9e00640 1 -- start start 2024-04-05T22:21:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f7bc5400640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0154780 con 0x7f7bc015ce40 2024-04-05T22:21:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f9e3be00640 1 -- 172.21.15.73:0/1349845484 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28030ce0 con 0x7f9e340531f0 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.514+0000 7f39e5e00640 1 -- start start 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39e00add80 0x7f39e006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e00b1870 0x7f39e01f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e014ffd0 0x7f39e0149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0145990 con 0x7f39e014ffd0 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00cd090 con 0x7f39e00b1870 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f39e5e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0145640 con 0x7f39e00add80 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f39de000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e00b1870 0x7f39e01f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f39dea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39e00add80 0x7f39e006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f39df400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e014ffd0 0x7f39e0149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f39dea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39e00add80 0x7f39e006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56180/0 (socket says 172.21.15.73:56180) 2024-04-05T22:21:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f39dea00640 1 -- 172.21.15.73:0/3418428544 learned_addr learned my addr 172.21.15.73:0/3418428544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c09d000 con 0x7f738c15d350 2024-04-05T22:21:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 0x7f8ab40d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab40d0bd0 0x7f8ab40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f9e43000640 1 -- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e340531f0 msgr2=0x7f9e340073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.515+0000 7f9e43000640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e340531f0 0x7f9e340073b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400b590 tx=0x7f9e2802fc50 comp rx=0 tx=0).stop 2024-04-05T22:21:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f0a0ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7fe06ce00640 1 --2- 172.21.15.73:0/906332625 >> v2:172.21.15.73:3300/0 conn(0x7fe0680c3eb0 0x7fe068161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7fe06ce00640 1 -- 172.21.15.73:0/906332625 >> v1:172.21.15.157:6789/0 conn(0x7fe0680c3a50 legacy=0x7fe068160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.513+0000 7fe06ce00640 1 -- 172.21.15.73:0/906332625 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0680ceac0 con 0x7fe0680b0fc0 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f0a0ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a080b1870 0x7f0a080b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 0x7f8ab40d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40ceee0 con 0x7f8ab415d350 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40ce5b0 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40ce890 con 0x7f8ab415cf70 2024-04-05T22:21:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7fe06ce00640 1 --2- 172.21.15.73:0/906332625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 0x7fe0680b1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0540097a0 tx=0x7fe05402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517fdcd37cf769d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 -- 172.21.15.73:0/1349845484 shutdown_connections 2024-04-05T22:21:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e3404fca0 0x7f9e34050130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 --2- 172.21.15.73:0/1349845484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e340531f0 0x7f9e340073b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39df400640 1 -- 172.21.15.73:0/3418428544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39e00add80 msgr2=0x7f39e006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39df400640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39e00add80 0x7f39e006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39df400640 1 -- 172.21.15.73:0/3418428544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e00b1870 msgr2=0x7f39e01f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39df400640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e00b1870 0x7f39e01f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39df400640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8009400 con 0x7f39e014ffd0 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.516+0000 7f0a0ea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a080ae300 con 0x7f0a081609e0 2024-04-05T22:21:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f0a0ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a080b21f0 con 0x7f0a080b1870 2024-04-05T22:21:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f0a0ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080ae020 con 0x7f0a08145c70 2024-04-05T22:21:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f39de000640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e00b1870 0x7f39e01f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 --2- 172.21.15.73:0/1349845484 >> v2:172.21.15.73:3300/0 conn(0x7f9e340df3c0 0x7f9e34054050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 -- 172.21.15.73:0/1349845484 >> 172.21.15.73:0/1349845484 conn(0x7f9e34054c60 msgr2=0x7f9e34055070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f9e43000640 1 -- 172.21.15.73:0/1349845484 shutdown_connections 2024-04-05T22:21:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f738c068160 con 0x7f738c15d350 2024-04-05T22:21:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7fe067400640 1 -- 172.21.15.73:0/906332625 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054031830 con 0x7fe0680b0fc0 2024-04-05T22:21:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7bb8014420 con 0x7f7bc015ce40 2024-04-05T22:21:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.517+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b90000fc0 con 0x7f7bc015ce40 2024-04-05T22:21:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39df400640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e014ffd0 0x7f39e0149c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7fe067400640 1 -- 172.21.15.73:0/906332625 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe054031990 con 0x7fe0680b0fc0 2024-04-05T22:21:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80408f0 con 0x7f39e014ffd0 2024-04-05T22:21:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8040a50 con 0x7f39e014ffd0 2024-04-05T22:21:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7fe067400640 1 -- 172.21.15.73:0/906332625 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054031d60 con 0x7fe0680b0fc0 2024-04-05T22:21:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f0a07e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8040e00 con 0x7f39e014ffd0 2024-04-05T22:21:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e006cd50 con 0x7f39e014ffd0 2024-04-05T22:21:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f7bb6a00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7b9c06acd0 0x7f7b9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bb80894b0 con 0x7f7bc015ce40 2024-04-05T22:21:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f0a07e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56198/0 (socket says 172.21.15.73:56198) 2024-04-05T22:21:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f0a07e00640 1 -- 172.21.15.73:0/1011325303 learned_addr learned my addr 172.21.15.73:0/1011325303 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f09ff400640 1 --2- 172.21.15.73:0/1011325303 >> v2:172.21.15.157:3300/0 conn(0x7f0a080b1870 0x7f0a080b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e006d310 con 0x7f39e014ffd0 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f0a07e00640 1 -- 172.21.15.73:0/1011325303 >> v2:172.21.15.157:3300/0 conn(0x7f0a080b1870 msgr2=0x7f0a080b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f0a07e00640 1 --2- 172.21.15.73:0/1011325303 >> v2:172.21.15.157:3300/0 conn(0x7f0a080b1870 0x7f0a080b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f0a07e00640 1 -- 172.21.15.73:0/1011325303 >> v1:172.21.15.157:6789/0 conn(0x7f0a081609e0 legacy=0x7f0a08160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f0a07e00640 1 -- 172.21.15.73:0/1011325303 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0806f6e0 con 0x7f0a08145c70 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f8ab2000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab40d0bd0 0x7f8ab40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f8ab2000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab40d0bd0 0x7f8ab40d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57660/0 (socket says 172.21.15.73:57660) 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f8ab2000640 1 -- 172.21.15.73:0/1294821660 learned_addr learned my addr 172.21.15.73:0/1294821660 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f8ab3400640 1 --2- 172.21.15.73:0/1294821660 >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 0x7f8ab40d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f7bb6a00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7f7b9c072ef0 0x7f7b9c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f8ab2000640 1 -- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 msgr2=0x7f8ab40d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.520+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0dc4011790 con 0x7f0dd40d13e0 2024-04-05T22:21:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.518+0000 7f9e43000640 1 -- 172.21.15.73:0/1349845484 wait complete. 2024-04-05T22:21:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b9c0759c0 con 0x7f7b9c072ef0 2024-04-05T22:21:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb808ba10 con 0x7f7bc015ce40 2024-04-05T22:21:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.519+0000 7f7bbe800640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7b9c06acd0 0x7f7b9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.520+0000 7f7bbde00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7f7b9c072ef0 0x7f7b9c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.520+0000 7f0a07e00640 1 --2- 172.21.15.73:0/1011325303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009870 tx=0x7f09f802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54e45e20e62f7637 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f7bbde00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7f7b9c072ef0 0x7f7b9c075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f0a07400640 1 -- 172.21.15.73:0/1011325303 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8031870 con 0x7f0a08145c70 2024-04-05T22:21:21.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f0a07400640 1 -- 172.21.15.73:0/1011325303 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80319d0 con 0x7f0a08145c70 2024-04-05T22:21:21.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f0a07400640 1 -- 172.21.15.73:0/1011325303 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8031da0 con 0x7f0a08145c70 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f7bbe800640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7b9c06acd0 0x7f7b9c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7fe06f800640 1 -- 172.21.15.73:0/906332625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 msgr2=0x7fe0680b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7fe06f800640 1 --2- 172.21.15.73:0/906332625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 0x7fe0680b1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0540097a0 tx=0x7fe05402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7384003e80 con 0x7f738c15d350 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f0dcaa00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0db406ac00 0x7f0db406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dc40898f0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f0dcaa00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f0db4072e20 0x7f0db4075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0db40758f0 con 0x7f0db4072e20 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dc4089ca0 con 0x7f0dd40d13e0 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.521+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39e0159b20 con 0x7f39e014ffd0 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2a00640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 0x7f8ab40d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2000640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 0x7f8ab40d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2000640 1 -- 172.21.15.73:0/1294821660 >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 msgr2=0x7f8ab40d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2000640 1 --2- 172.21.15.73:0/1294821660 >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 0x7f8ab40d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2000640 1 -- 172.21.15.73:0/1294821660 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40ceb70 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab3400640 1 --2- 172.21.15.73:0/1294821660 >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 0x7f8ab40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:21.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f8ab2a00640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 0x7f8ab40d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:21.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== osd.0 v2:172.21.15.73:6800/37310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7b9c0759c0 con 0x7f7b9c072ef0 2024-04-05T22:21:21.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f0dd3400640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f0db4072e20 0x7f0db4075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f0dd2a00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0db406ac00 0x7f0db406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7f9e43000640 1 Processor -- start 2024-04-05T22:21:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f9e43000640 1 -- start start 2024-04-05T22:21:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f0dd2a00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0db406ac00 0x7f0db406d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0006fd0 tx=0x7f0dc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f0dd3400640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f0db4072e20 0x7f0db4075240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f8ab2000640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab40d1220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ab408b890 tx=0x7f8aa002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116930d25c6427dc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f39d803c040 con 0x7f39e014ffd0 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f7383400640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f736c06acd0 0x7f736c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7384089180 con 0x7f738c15d350 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f7383400640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7f736c072ef0 0x7f736c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f736c0759c0 con 0x7f736c072ef0 2024-04-05T22:21:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f738408b7a0 con 0x7f738c15d350 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7fe06f800640 1 -- 172.21.15.73:0/906332625 shutdown_connections 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7fe06f800640 1 --2- 172.21.15.73:0/906332625 >> v2:172.21.15.73:3300/0 conn(0x7fe0680c3eb0 0x7fe068161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7fe06f800640 1 --2- 172.21.15.73:0/906332625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680b0fc0 0x7fe0680b1780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.522+0000 7fe06f800640 1 -- 172.21.15.73:0/906332625 >> 172.21.15.73:0/906332625 conn(0x7fe068099a60 msgr2=0x7fe068099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7fe06f800640 1 -- 172.21.15.73:0/906332625 shutdown_connections 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.524+0000 7fe06f800640 1 -- 172.21.15.73:0/906332625 wait complete. 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f0a0ea00640 1 -- 172.21.15.73:0/1011325303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 msgr2=0x7f0a080ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.523+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/1011325303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009870 tx=0x7f09f802b910 comp rx=0 tx=0).stop 2024-04-05T22:21:21.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f738b400640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f736c06acd0 0x7f736c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f0a0ea00640 1 -- 172.21.15.73:0/1011325303 shutdown_connections 2024-04-05T22:21:21.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/1011325303 >> v2:172.21.15.157:3300/0 conn(0x7f0a080b1870 0x7f0a080b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/1011325303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a08145c70 0x7f0a080ad2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f0a0ea00640 1 -- 172.21.15.73:0/1011325303 >> 172.21.15.73:0/1011325303 conn(0x7f0a080ce8e0 msgr2=0x7f0a080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f39d7400640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f39bc06ac00 0x7f39bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39d80aafc0 con 0x7f39e014ffd0 2024-04-05T22:21:21.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f738b400640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f736c06acd0 0x7f736c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f738aa00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7f736c072ef0 0x7f736c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== osd.2 v2:172.21.15.73:6813/37323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f0db40758f0 con 0x7f0db4072e20 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f39d7400640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7f39bc072da0 0x7f39bc0751c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39bc075870 con 0x7f39bc072da0 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d80ab340 con 0x7f39e014ffd0 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f9e43000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 0x7f9e340d7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f9e43000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e340531f0 0x7f9e340e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e43000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e340df3c0 0x7f9e340d4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e43000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340d8290 con 0x7f9e340531f0 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e43000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e341781e0 con 0x7f9e3404fca0 2024-04-05T22:21:21.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e43000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340d7fb0 con 0x7f9e340df3c0 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f0a0ea00640 1 -- 172.21.15.73:0/1011325303 shutdown_connections 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f0a0ea00640 1 -- 172.21.15.73:0/1011325303 wait complete. 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.525+0000 7f8ab1600640 1 -- 172.21.15.73:0/1294821660 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00317e0 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f8ab1600640 1 -- 172.21.15.73:0/1294821660 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0031940 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f8ab1600640 1 -- 172.21.15.73:0/1294821660 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0031cd0 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f738aa00640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7f736c072ef0 0x7f736c075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e41c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 0x7f9e340d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e41200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e340531f0 0x7f9e340e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f9e41c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 0x7f9e340d7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57676/0 (socket says 172.21.15.73:57676) 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f9e41c00640 1 -- 172.21.15.73:0/2733437573 learned_addr learned my addr 172.21.15.73:0/2733437573 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f9e44600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e340df3c0 0x7f9e340d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 -- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab40d0bd0 msgr2=0x7f8ab40d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab40d1220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ab408b890 tx=0x7f8aa002b880 comp rx=0 tx=0).stop 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7fe06f800640 1 Processor -- start 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 -- 172.21.15.73:0/1294821660 shutdown_connections 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/1294821660 >> v2:172.21.15.73:3300/0 conn(0x7f8ab415d350 0x7f8ab40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab40d1220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/1294821660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415cf70 0x7f8ab40d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f8ab9e00640 1 -- 172.21.15.73:0/1294821660 >> 172.21.15.73:0/1294821660 conn(0x7f8ab40d2df0 msgr2=0x7f8ab40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.526+0000 7f39dea00640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f39bc06ac00 0x7f39bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7f39de000640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7f39bc072da0 0x7f39bc0751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f39dea00640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f39bc06ac00 0x7f39bc06d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000f60 tx=0x7f39c8005f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.527+0000 7fe06f800640 1 -- start start 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 -- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e340df3c0 msgr2=0x7f9e340d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e340df3c0 0x7f9e340d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 -- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e340531f0 msgr2=0x7f9e340e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e340531f0 0x7f9e340e14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 -- 172.21.15.73:0/2733437573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2800d8a0 con 0x7f9e3404fca0 2024-04-05T22:21:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f0a0ea00640 1 Processor -- start 2024-04-05T22:21:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f8ab9e00640 1 -- 172.21.15.73:0/1294821660 shutdown_connections 2024-04-05T22:21:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f8ab9e00640 1 -- 172.21.15.73:0/1294821660 wait complete. 2024-04-05T22:21:21.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== osd.1 v2:172.21.15.73:6808/37311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f736c0759c0 con 0x7f736c072ef0 2024-04-05T22:21:21.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f0a0ea00640 1 -- start start 2024-04-05T22:21:21.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.528+0000 7f9e41c00640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 0x7f9e340d7980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000aa80 tx=0x7f9e3000af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30010070 con 0x7f9e3404fca0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000da20 con 0x7f9e3404fca0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000ddd0 con 0x7f9e3404fca0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e34177b90 con 0x7f9e3404fca0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e340d6380 con 0x7f9e3404fca0 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0680b0fc0 0x7fe0681f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 0x7f0a0815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08145c70 0x7f0a0815e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081609e0 0x7f0a0806cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815c850 con 0x7f0a080b1870 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815ed60 con 0x7f0a08145c70 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a0ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081f3640 con 0x7f0a081609e0 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f8ab9e00640 1 Processor -- start 2024-04-05T22:21:21.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a07e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 0x7f0a0815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a07e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 0x7f0a0815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51048/0 (socket says 172.21.15.73:51048) 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7f0a07e00640 1 -- 172.21.15.73:0/4045156271 learned_addr learned my addr 172.21.15.73:0/4045156271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7f8ab9e00640 1 -- start start 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab414e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 0x7f8ab40c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7f39de000640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7f39bc072da0 0x7f39bc0751c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415d350 0x7f8ab40c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40caaf0 con 0x7f8ab40d0bd0 2024-04-05T22:21:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab41e5850 con 0x7f8ab415cf70 2024-04-05T22:21:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab41e5b30 con 0x7f8ab415d350 2024-04-05T22:21:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 0x7fe06814bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680c3eb0 0x7fe06814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815e180 con 0x7fe0680c3a50 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681f6a90 con 0x7fe0680b0fc0 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.529+0000 7fe06f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680ca0a0 con 0x7fe0680c3eb0 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.530+0000 7fe067e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 0x7fe06814bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7fe067e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 0x7fe06814bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51042/0 (socket says 172.21.15.73:51042) 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7fe067e00640 1 -- 172.21.15.73:0/1851515877 learned_addr learned my addr 172.21.15.73:0/1851515877 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7fe06d800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680c3eb0 0x7fe06814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.531+0000 7f0a0ca00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081609e0 0x7f0a0806cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 -- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08145c70 msgr2=0x7f0a0815e820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08145c70 0x7f0a0815e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 -- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 msgr2=0x7f0a0815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 0x7f0a0815bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80094d0 con 0x7f0a081609e0 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a07e00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a080b1870 0x7f0a0815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7f0a0ca00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081609e0 0x7f0a0806cb00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c730 tx=0x7f0a0000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00003b70 con 0x7f0a081609e0 2024-04-05T22:21:21.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00003cd0 con 0x7f0a081609e0 2024-04-05T22:21:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000052f0 con 0x7f0a081609e0 2024-04-05T22:21:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a081f2d10 con 0x7f0a081609e0 2024-04-05T22:21:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7fe067e00640 1 -- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680c3eb0 msgr2=0x7fe06814c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7fe067e00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0680c3eb0 0x7fe06814c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7fe067e00640 1 -- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0680b0fc0 msgr2=0x7fe0681f6550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-05T22:21:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7fe067e00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0680b0fc0 0x7fe0681f6550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.532+0000 7fe067e00640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054009400 con 0x7fe0680c3a50 2024-04-05T22:21:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7fe067e00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 0x7fe06814bc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c500 tx=0x7fe05c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e3001e460 con 0x7f9e3404fca0 2024-04-05T22:21:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f8ab2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415d350 0x7f8ab40c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 0x7f8ab40c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.533+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a081f32d0 con 0x7f0a081609e0 2024-04-05T22:21:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005ce0 con 0x7fe0680c3a50 2024-04-05T22:21:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0680ca380 con 0x7fe0680c3a50 2024-04-05T22:21:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0680ca940 con 0x7fe0680c3a50 2024-04-05T22:21:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 0x7f8ab40c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57690/0 (socket says 172.21.15.73:57690) 2024-04-05T22:21:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a080ad820 con 0x7f0a081609e0 2024-04-05T22:21:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f8ab2000640 1 -- 172.21.15.73:0/2170962639 learned_addr learned my addr 172.21.15.73:0/2170962639 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f8ab2000640 1 -- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415d350 msgr2=0x7f8ab40c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f8ab2000640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415d350 0x7f8ab40c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f8ab2000640 1 -- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ab40d0bd0 msgr2=0x7f8ab414e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.534+0000 7f9e39600640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e1c06ac00 0x7f9e1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e300894f0 con 0x7f9e3404fca0 2024-04-05T22:21:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f9e43000640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f9e34178650 0x7f9e340d4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 --> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e340065f0 con 0x7f9e34178650 2024-04-05T22:21:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f9e41200640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e1c06ac00 0x7f9e1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f8ab2000640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab414e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f8ab2000640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac009a70 con 0x7f8ab415cf70 2024-04-05T22:21:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f8ab3400640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ab415d350 0x7f8ab40c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== osd.3 v2:172.21.15.73:6824/37326 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f39bc075870 con 0x7f39bc072da0 2024-04-05T22:21:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7f8ab2a00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ab40d0bd0 0x7f8ab414e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c005e40 con 0x7fe0680c3a50 2024-04-05T22:21:21.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0680c4340 con 0x7fe0680c3a50 2024-04-05T22:21:21.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.535+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005ce0 con 0x7fe0680c3a50 2024-04-05T22:21:21.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f8ab2000640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 0x7f8ab40c8230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00090b0 tx=0x7f8aa00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f9e41200640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e1c06ac00 0x7f9e1c06d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e28039000 tx=0x7f9e28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0044c60 con 0x7f8ab415cf70 2024-04-05T22:21:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0048070 con 0x7f8ab415cf70 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0032470 con 0x7f8ab415cf70 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.536+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa00094d0 con 0x7f8ab415cf70 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.537+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab41e60f0 con 0x7f8ab415cf70 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.538+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ab40d3750 con 0x7f8ab415cf70 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.539+0000 7f9e44600640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f9e34178650 0x7f9e340d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.540+0000 7f9e44600640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f9e34178650 0x7f9e340d4550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.539+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0a00005660 con 0x7f0a081609e0 2024-04-05T22:21:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.540+0000 7f0a04c00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f09e406ac00 0x7f09e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.540+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8aa00325d0 con 0x7f8ab415cf70 2024-04-05T22:21:21.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a00089c40 con 0x7f0a081609e0 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7f0a04c00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f09e4072e20 0x7f09e4075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09e40758f0 con 0x7f09e4072e20 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7f0a07e00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f09e406ac00 0x7f09e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a00089ff0 con 0x7f0a081609e0 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.541+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe05c0070f0 con 0x7fe0680c3a50 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f8aab400640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a9406ac50 0x7f8a9406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aa00b38f0 con 0x7f8ab415cf70 2024-04-05T22:21:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f09ff400640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f09e4072e20 0x7f09e4075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== osd.5 v2:172.21.15.157:6821/37594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9e3c08f040 con 0x7f9e34178650 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f8aab400640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f8a94072e70 0x7f8a94075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a94075940 con 0x7f8a94072e70 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.542+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa00b3d90 con 0x7f8ab415cf70 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.543+0000 7f8ab2a00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a9406ac50 0x7f8a9406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.543+0000 7f8ab3400640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f8a94072e70 0x7f8a94075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.544+0000 7f8ab2a00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a9406ac50 0x7f8a9406d110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4006b10 tx=0x7f8aa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.544+0000 7f8ab3400640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f8a94072e70 0x7f8a94075290 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.543+0000 7f09ff400640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f09e4072e20 0x7f09e4075240 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.544+0000 7f0a07e00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f09e406ac00 0x7f09e406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a080cc270 tx=0x7f09f803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.545+0000 7fe064c00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe04806acc0 0x7fe04806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.545+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe05c089070 con 0x7fe0680c3a50 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.545+0000 7fe06ce00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe04806acc0 0x7fe04806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.545+0000 7fe064c00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7fe048072e60 0x7fe048075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.545+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe048075930 con 0x7fe048072e60 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.546+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe05c005ce0 con 0x7fe0680c3a50 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.546+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== osd.6 v2:172.21.15.157:6808/37593 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f09e40758f0 con 0x7f09e4072e20 2024-04-05T22:21:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.546+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 --> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0dd40cb3e0 con 0x7f0db4072e20 2024-04-05T22:21:21.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.546+0000 7fe06d800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7fe048072e60 0x7fe048075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dcaa00640 1 -- 172.21.15.73:0/2329640509 <== osd.2 v2:172.21.15.73:6813/37323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0dd40cb3e0 con 0x7f0db4072e20 2024-04-05T22:21:21.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f0db4072e20 msgr2=0x7f0db4075240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f0db4072e20 0x7f0db4075240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.546+0000 7fe06ce00640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe04806acc0 0x7fe04806d180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe054036000 tx=0x7fe054008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0db406ac00 msgr2=0x7f0db406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0db406ac00 0x7f0db406d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0006fd0 tx=0x7f0dc0008040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd40d13e0 msgr2=0x7f0dd40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f0dd3e00640 1 --2- 172.21.15.73:0/2329640509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd40d13e0 0x7f0dd40ca4f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b3e0 tx=0x7f0dc400b8a0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.548+0000 7f0dd3400640 1 -- 172.21.15.73:0/2329640509 reap_dead start 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.548+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 shutdown_connections 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.548+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 >> 172.21.15.73:0/2329640509 conn(0x7f0dd40d2a60 msgr2=0x7f0dd40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.547+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== osd.7 v2:172.21.15.157:6816/37603 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8a94075940 con 0x7f8a94072e70 2024-04-05T22:21:21.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.548+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 shutdown_connections 2024-04-05T22:21:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.548+0000 7f0dd3e00640 1 -- 172.21.15.73:0/2329640509 wait complete. 2024-04-05T22:21:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.550+0000 7fe06d800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7fe048072e60 0x7fe048075280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.552+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== osd.4 v2:172.21.15.157:6800/37592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe048075930 con 0x7fe048072e60 2024-04-05T22:21:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.556+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 --> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f738c0c5d50 con 0x7f736c072ef0 2024-04-05T22:21:21.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.558+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 --> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b90002cb0 con 0x7f7b9c072ef0 2024-04-05T22:21:21.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.558+0000 7f7383400640 1 -- 172.21.15.73:0/1558265107 <== osd.1 v2:172.21.15.73:6808/37311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f738c0c5d50 con 0x7f736c072ef0 2024-04-05T22:21:21.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7f736c072ef0 msgr2=0x7f736c075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7f736c072ef0 0x7f736c075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f736c06acd0 msgr2=0x7f736c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f736c06acd0 0x7f736c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c03d040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c15d350 msgr2=0x7f738c09d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.559+0000 7f7392600640 1 --2- 172.21.15.73:0/1558265107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c15d350 0x7f738c09d800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f738400be00 tx=0x7f738400bef0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.560+0000 7f7bb6a00640 1 -- 172.21.15.73:0/441737638 <== osd.0 v2:172.21.15.73:6800/37310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7b90002cb0 con 0x7f7b9c072ef0 2024-04-05T22:21:21.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.560+0000 7f738be00640 1 -- 172.21.15.73:0/1558265107 reap_dead start 2024-04-05T22:21:21.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.560+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 shutdown_connections 2024-04-05T22:21:21.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.560+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 >> 172.21.15.73:0/1558265107 conn(0x7f738c0d2df0 msgr2=0x7f738c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 shutdown_connections 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7f7b9c072ef0 msgr2=0x7f7b9c075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7f7b9c072ef0 0x7f7b9c075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7b9c06acd0 msgr2=0x7f7b9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7b9c06acd0 0x7f7b9c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb0008030 comp rx=0 tx=0).stop 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 msgr2=0x7f7bc00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bafe00640 1 --2- 172.21.15.73:0/441737638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc015ce40 0x7f7bc00b48a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8007d90 tx=0x7f7bb800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7392600640 1 -- 172.21.15.73:0/1558265107 wait complete. 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.561+0000 7f7bbf200640 1 -- 172.21.15.73:0/441737638 reap_dead start 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.562+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 shutdown_connections 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.562+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 >> 172.21.15.73:0/441737638 conn(0x7f7bc009be00 msgr2=0x7f7bc00c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.562+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 shutdown_connections 2024-04-05T22:21:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.562+0000 7f7bafe00640 1 -- 172.21.15.73:0/441737638 wait complete. 2024-04-05T22:21:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.564+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 --> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39e0159840 con 0x7f39bc072da0 2024-04-05T22:21:21.595 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:21.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-05T22:21:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.567+0000 7f39d7400640 1 -- 172.21.15.73:0/3418428544 <== osd.3 v2:172.21.15.73:6824/37326 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39d0013040 con 0x7f39bc072da0 2024-04-05T22:21:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.568+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7f39bc072da0 msgr2=0x7f39bc0751c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.568+0000 7f39e5e00640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7f39bc072da0 0x7f39bc0751c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f39bc06ac00 msgr2=0x7f39bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f39bc06ac00 0x7f39bc06d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000f60 tx=0x7f39c8005f30 comp rx=0 tx=0).stop 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e014ffd0 msgr2=0x7f39e0149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 --2- 172.21.15.73:0/3418428544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e014ffd0 0x7f39e0149c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d80046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39df400640 1 -- 172.21.15.73:0/3418428544 reap_dead start 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 shutdown_connections 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.569+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 >> 172.21.15.73:0/3418428544 conn(0x7f39e00ce8e0 msgr2=0x7f39e015bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.570+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 shutdown_connections 2024-04-05T22:21:21.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.570+0000 7f39e5e00640 1 -- 172.21.15.73:0/3418428544 wait complete. 2024-04-05T22:21:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.580+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 --> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ab40cac30 con 0x7f8a94072e70 2024-04-05T22:21:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.580+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 --> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a0806fca0 con 0x7f09e4072e20 2024-04-05T22:21:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 --> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e34006bb0 con 0x7f9e34178650 2024-04-05T22:21:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a04c00640 1 -- 172.21.15.73:0/4045156271 <== osd.6 v2:172.21.15.157:6808/37593 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09f0009580 con 0x7f09e4072e20 2024-04-05T22:21:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f8aab400640 1 -- 172.21.15.73:0/2170962639 <== osd.7 v2:172.21.15.157:6816/37603 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ab40cac30 con 0x7f8a94072e70 2024-04-05T22:21:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f09e4072e20 msgr2=0x7f09e4075240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f09e4072e20 0x7f09e4075240 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f09e406ac00 msgr2=0x7f09e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f09e406ac00 0x7f09e406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a080cc270 tx=0x7f09f803c040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081609e0 msgr2=0x7f0a0806cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.581+0000 7f0a0ea00640 1 --2- 172.21.15.73:0/4045156271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081609e0 0x7f0a0806cb00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c730 tx=0x7f0a0000cbf0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f0a0ca00640 1 -- 172.21.15.73:0/4045156271 reap_dead start 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 shutdown_connections 2024-04-05T22:21:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 >> 172.21.15.73:0/4045156271 conn(0x7f0a080ce8e0 msgr2=0x7f0a080afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f9e39600640 1 -- 172.21.15.73:0/2733437573 <== osd.5 v2:172.21.15.157:6821/37594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e3c04e9b0 con 0x7f9e34178650 2024-04-05T22:21:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f8a94072e70 msgr2=0x7f8a94075290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f8a94072e70 0x7f8a94075290 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 shutdown_connections 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f0a0ea00640 1 -- 172.21.15.73:0/4045156271 wait complete. 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a9406ac50 msgr2=0x7f8a9406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a9406ac50 0x7f8a9406d110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4006b10 tx=0x7f8aa4008040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 msgr2=0x7f8ab40c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f8ab9e00640 1 --2- 172.21.15.73:0/2170962639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab415cf70 0x7f8ab40c8230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00090b0 tx=0x7f8aa00046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f9e34178650 msgr2=0x7f9e340d4550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.582+0000 7f9e43000640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f9e34178650 0x7f9e340d4550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e1c06ac00 msgr2=0x7f9e1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f8ab3400640 1 -- 172.21.15.73:0/2170962639 reap_dead start 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f9e43000640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e1c06ac00 0x7f9e1c06d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e28039000 tx=0x7f9e28008040 comp rx=0 tx=0).stop 2024-04-05T22:21:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 msgr2=0x7f9e340d7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f9e43000640 1 --2- 172.21.15.73:0/2733437573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e3404fca0 0x7f9e340d7980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000aa80 tx=0x7f9e3000af40 comp rx=0 tx=0).stop 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f9e44600640 1 -- 172.21.15.73:0/2733437573 reap_dead start 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 shutdown_connections 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 >> 172.21.15.73:0/2170962639 conn(0x7f8ab40d2df0 msgr2=0x7f8ab406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 shutdown_connections 2024-04-05T22:21:21.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.583+0000 7f8ab9e00640 1 -- 172.21.15.73:0/2170962639 wait complete. 2024-04-05T22:21:21.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.584+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 shutdown_connections 2024-04-05T22:21:21.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.584+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 >> 172.21.15.73:0/2733437573 conn(0x7f9e34054c60 msgr2=0x7f9e340db9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.584+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 shutdown_connections 2024-04-05T22:21:21.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.584+0000 7f9e43000640 1 -- 172.21.15.73:0/2733437573 wait complete. 2024-04-05T22:21:21.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.584+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 --> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0680d11e0 con 0x7fe048072e60 2024-04-05T22:21:21.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.586+0000 7fe064c00640 1 -- 172.21.15.73:0/1851515877 <== osd.4 v2:172.21.15.157:6800/37592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe06000f040 con 0x7fe048072e60 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7fe048072e60 msgr2=0x7fe048075280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7fe048072e60 0x7fe048075280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe04806acc0 msgr2=0x7fe04806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe04806acc0 0x7fe04806d180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe054036000 tx=0x7fe054008030 comp rx=0 tx=0).stop 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 msgr2=0x7fe06814bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.587+0000 7fe06f800640 1 --2- 172.21.15.73:0/1851515877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0680c3a50 0x7fe06814bc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c500 tx=0x7fe05c00c9c0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.608 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:21.609 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:21.609 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-05T22:21:21.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-05T22:21:21.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.588+0000 7fe06d800640 1 -- 172.21.15.73:0/1851515877 reap_dead start 2024-04-05T22:21:21.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.588+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 shutdown_connections 2024-04-05T22:21:21.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.588+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 >> 172.21.15.73:0/1851515877 conn(0x7fe068099a60 msgr2=0x7fe0680cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.588+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 shutdown_connections 2024-04-05T22:21:21.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.588+0000 7fe06f800640 1 -- 172.21.15.73:0/1851515877 wait complete. 2024-04-05T22:21:21.611 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:21.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-05T22:21:21.612 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:21.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-05T22:21:21.613 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:21.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-05T22:21:21.614 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:21.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-05T22:21:21.615 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:21.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.928+0000 7f183f000640 1 Processor -- start 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.928+0000 7f183f000640 1 -- start start 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.930+0000 7f183f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 0x7f1830006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.930+0000 7f183f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1830054300 con 0x7f18300eacd0 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.930+0000 7f183f000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18300536f0 con 0x7f1830005e60 2024-04-05T22:21:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.930+0000 7f183f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300539d0 con 0x7f18300069b0 2024-04-05T22:21:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.931+0000 7f183dc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f18300eacd0 0x7f183004fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48790/0 (socket says 172.21.15.73:48790) 2024-04-05T22:21:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.931+0000 7f183e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 0x7f1830006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.931+0000 7f183dc00640 1 -- 172.21.15.73:0/531307436 learned_addr learned my addr 172.21.15.73:0/531307436 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397464420 0 0) 0x7f18300536f0 con 0x7f1830005e60 2024-04-05T22:21:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1820003520 con 0x7f1830005e60 2024-04-05T22:21:21.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636015841 0 0) 0x7f1830054300 con 0x7f18300eacd0 2024-04-05T22:21:21.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18200048a0 con 0x7f18300eacd0 2024-04-05T22:21:21.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f183e600640 1 -- 172.21.15.73:0/531307436 >> v1:172.21.15.157:6789/0 conn(0x7f1830005e60 legacy=0x7f18300062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f183e600640 1 -- 172.21.15.73:0/531307436 >> v1:172.21.15.73:6789/0 conn(0x7f18300eacd0 legacy=0x7f183004fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.932+0000 7f183e600640 1 -- 172.21.15.73:0/531307436 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830053cb0 con 0x7f18300069b0 2024-04-05T22:21:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.934+0000 7f183e600640 1 --2- 172.21.15.73:0/531307436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 0x7f1830006e40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f183804fa60 tx=0x7f18380a5b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f98d6f1782d7106 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.934+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380a79b0 con 0x7f18300069b0 2024-04-05T22:21:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.935+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18380a7b10 con 0x7f18300069b0 2024-04-05T22:21:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.935+0000 7f1837e00640 1 -- 172.21.15.73:0/531307436 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380af420 con 0x7f18300069b0 2024-04-05T22:21:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7fb271400640 1 Processor -- start 2024-04-05T22:21:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 -- 172.21.15.73:0/531307436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 msgr2=0x7f1830006e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 --2- 172.21.15.73:0/531307436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 0x7f1830006e40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f183804fa60 tx=0x7f18380a5b90 comp rx=0 tx=0).stop 2024-04-05T22:21:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7fb271400640 1 -- start start 2024-04-05T22:21:21.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 -- 172.21.15.73:0/531307436 shutdown_connections 2024-04-05T22:21:21.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 --2- 172.21.15.73:0/531307436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300069b0 0x7f1830006e40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 -- 172.21.15.73:0/531307436 >> 172.21.15.73:0/531307436 conn(0x7f1830009f80 msgr2=0x7f183000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 -- 172.21.15.73:0/531307436 shutdown_connections 2024-04-05T22:21:21.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.936+0000 7f183f000640 1 -- 172.21.15.73:0/531307436 wait complete. 2024-04-05T22:21:21.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb271400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb26c0b2ba0 0x7fb26c0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb271400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c1627f0 con 0x7fb26c162370 2024-04-05T22:21:21.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb271400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0b3510 con 0x7fb26c161f90 2024-04-05T22:21:21.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb271400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c158790 con 0x7fb26c0b2ba0 2024-04-05T22:21:21.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb26a800640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb26c161f90 0x7fb26c164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60214/0 (socket says 172.21.15.73:60214) 2024-04-05T22:21:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.938+0000 7fb26a800640 1 -- 172.21.15.73:0/2346712442 learned_addr learned my addr 172.21.15.73:0/2346712442 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.939+0000 7f183f000640 1 Processor -- start 2024-04-05T22:21:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.939+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375862528 0 0) 0x7fb26c0b3510 con 0x7fb26c161f90 2024-04-05T22:21:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.939+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb250003520 con 0x7fb26c161f90 2024-04-05T22:21:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657090338 0 0) 0x7fb258001110 con 0x7fb26c162370 2024-04-05T22:21:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c0b3510 con 0x7fb26c162370 2024-04-05T22:21:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb26b200640 1 --2- 172.21.15.73:0/2346712442 >> v2:172.21.15.73:3300/0 conn(0x7fb26c0b2ba0 0x7fb26c0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7f183f000640 1 -- start start 2024-04-05T22:21:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7f63cee00640 1 Processor -- start 2024-04-05T22:21:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434621140 0 0) 0x7fb250003520 con 0x7fb26c161f90 2024-04-05T22:21:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 >> v2:172.21.15.73:3300/0 conn(0x7fb26c0b2ba0 msgr2=0x7fb26c0b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 --2- 172.21.15.73:0/2346712442 >> v2:172.21.15.73:3300/0 conn(0x7fb26c0b2ba0 0x7fb26c0b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 >> v1:172.21.15.73:6789/0 conn(0x7fb26c162370 legacy=0x7fb26c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c165cf0 con 0x7fb26c161f90 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb25c001670 con 0x7fb26c161f90 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 >> v1:172.21.15.157:6789/0 conn(0x7fb26c161f90 legacy=0x7fb26c164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb250003b00 0x7fb250003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7fb269400640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 0x7fb250004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.941+0000 7fb269400640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb250005180 0x7fb25000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.941+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580013c0 con 0x7fb250003b00 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.941+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c158790 con 0x7fb250005180 2024-04-05T22:21:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.941+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25000c0d0 con 0x7fb2500047c0 2024-04-05T22:21:21.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.940+0000 7f63cee00640 1 -- start start 2024-04-05T22:21:21.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.941+0000 7fb26a800640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb250003b00 0x7fb250003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18300069b0 0x7f18300366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300eacd0 0x7f18300495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830180750 con 0x7f18300069b0 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830037290 con 0x7f1830005e60 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f183f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830036fb0 con 0x7f18300eacd0 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7fb269e00640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 0x7fb250004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7fb26b200640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb250005180 0x7fb25000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f183dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f183dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57700/0 (socket says 172.21.15.73:57700) 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f183dc00640 1 -- 172.21.15.73:0/1830056268 learned_addr learned my addr 172.21.15.73:0/1830056268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7fb269e00640 1 -- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb250005180 msgr2=0x7fb25000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7fb269e00640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb250005180 0x7fb25000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7fb269e00640 1 -- 172.21.15.73:0/2346712442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb250003b00 msgr2=0x7fb250003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7fb269e00640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb250003b00 0x7fb250003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7fb269e00640 1 -- 172.21.15.73:0/2346712442 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c165cf0 con 0x7fb2500047c0 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f63cee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63c80d10f0 0x7f63c80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f63cee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c80b1f20 0x7f63c80adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f63cee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c80cf5a0 con 0x7f63c815e2e0 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f63cee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c80d0140 con 0x7f63c80d10f0 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.942+0000 7f63cee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80cf7d0 con 0x7f63c80b1f20 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f63c7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63c80d10f0 0x7f63c80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f183e600640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300eacd0 0x7f18300495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183d200640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18300069b0 0x7f18300366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f63c7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63c80d10f0 0x7f63c80cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51096/0 (socket says 172.21.15.73:51096) 2024-04-05T22:21:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f63c7400640 1 -- 172.21.15.73:0/2695516823 learned_addr learned my addr 172.21.15.73:0/2695516823 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.943+0000 7f63cce00640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c80b1f20 0x7f63c80adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183dc00640 1 -- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300eacd0 msgr2=0x7f18300495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7fb26b200640 1 -- 172.21.15.73:0/2346712442 reap_dead start 2024-04-05T22:21:21.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183dc00640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300eacd0 0x7f18300495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183dc00640 1 -- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18300069b0 msgr2=0x7f18300366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183dc00640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18300069b0 0x7f18300366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.944+0000 7f183dc00640 1 -- 172.21.15.73:0/1830056268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830180090 con 0x7f1830005e60 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7f63c7400640 1 -- 172.21.15.73:0/2695516823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c80b1f20 msgr2=0x7f63c80adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7f63c7400640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c80b1f20 0x7f63c80adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7f63c7400640 1 -- 172.21.15.73:0/2695516823 >> v1:172.21.15.73:6789/0 conn(0x7f63c815e2e0 legacy=0x7f63c80d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7f63c7400640 1 -- 172.21.15.73:0/2695516823 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80b1900 con 0x7f63c80d10f0 2024-04-05T22:21:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7f183dc00640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c450 tx=0x7f182c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.945+0000 7fb269e00640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 0x7fb250004c40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb258007f20 tx=0x7fb2500057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c521f7d3dda63855 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0074e0 con 0x7f1830005e60 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c007640 con 0x7f1830005e60 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0049e0 con 0x7f1830005e60 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830180370 con 0x7f1830005e60 2024-04-05T22:21:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830036c30 con 0x7f1830005e60 2024-04-05T22:21:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580052d0 con 0x7fb2500047c0 2024-04-05T22:21:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258005450 con 0x7fb2500047c0 2024-04-05T22:21:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.946+0000 7fb269400640 1 -- 172.21.15.73:0/2346712442 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258007290 con 0x7fb2500047c0 2024-04-05T22:21:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7f63c7400640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80cf060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009870 tx=0x7f63b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20613fc09fbc93a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7fb271400640 1 -- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 msgr2=0x7fb250004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7fb271400640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 0x7fb250004c40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb258007f20 tx=0x7fb2500057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7f63c6a00640 1 -- 172.21.15.73:0/2695516823 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8031640 con 0x7f63c80d10f0 2024-04-05T22:21:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7f63c6a00640 1 -- 172.21.15.73:0/2695516823 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b80317a0 con 0x7f63c80d10f0 2024-04-05T22:21:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7f63c6a00640 1 -- 172.21.15.73:0/2695516823 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8031b30 con 0x7f63c80d10f0 2024-04-05T22:21:21.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.947+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830005360 con 0x7f1830005e60 2024-04-05T22:21:21.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.948+0000 7fb271400640 1 -- 172.21.15.73:0/2346712442 shutdown_connections 2024-04-05T22:21:21.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.948+0000 7fb271400640 1 --2- 172.21.15.73:0/2346712442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2500047c0 0x7fb250004c40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.948+0000 7fb271400640 1 -- 172.21.15.73:0/2346712442 >> 172.21.15.73:0/2346712442 conn(0x7fb26c0ce8e0 msgr2=0x7fb26c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.948+0000 7fb271400640 1 -- 172.21.15.73:0/2346712442 shutdown_connections 2024-04-05T22:21:21.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.948+0000 7fb271400640 1 -- 172.21.15.73:0/2346712442 wait complete. 2024-04-05T22:21:21.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.949+0000 7f63cee00640 1 -- 172.21.15.73:0/2695516823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 msgr2=0x7f63c80cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.949+0000 7f63cee00640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80cf060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009870 tx=0x7f63b802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7fb271400640 1 Processor -- start 2024-04-05T22:21:21.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7f63cee00640 1 -- 172.21.15.73:0/2695516823 shutdown_connections 2024-04-05T22:21:21.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7f63cee00640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c80b1f20 0x7f63c80adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7f63cee00640 1 --2- 172.21.15.73:0/2695516823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80cf060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7f63cee00640 1 -- 172.21.15.73:0/2695516823 >> 172.21.15.73:0/2695516823 conn(0x7f63c8099a60 msgr2=0x7f63c8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.950+0000 7fb271400640 1 -- start start 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.951+0000 7f63cee00640 1 -- 172.21.15.73:0/2695516823 shutdown_connections 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.951+0000 7f63cee00640 1 -- 172.21.15.73:0/2695516823 wait complete. 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 0x7fb26c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c148210 0x7fb26c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c143400 con 0x7fb26c147cf0 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c142400 con 0x7fb2500047c0 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.952+0000 7fb271400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c145650 con 0x7fb26c148210 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7fb269e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7fb269e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51108/0 (socket says 172.21.15.73:51108) 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7fb269e00640 1 -- 172.21.15.73:0/3370877732 learned_addr learned my addr 172.21.15.73:0/3370877732 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7fb26a800640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 0x7fb26c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7fb26b200640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c148210 0x7fb26c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7f63cee00640 1 Processor -- start 2024-04-05T22:21:21.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.953+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f182c004b40 con 0x7f1830005e60 2024-04-05T22:21:21.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb26a800640 1 -- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c148210 msgr2=0x7fb26c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7f63cee00640 1 -- start start 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb26a800640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c148210 0x7fb26c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb26a800640 1 -- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 msgr2=0x7fb26c145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb26a800640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb26a800640 1 -- 172.21.15.73:0/3370877732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c1427f0 con 0x7fb2500047c0 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7fb269e00640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.954+0000 7f1835600640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 0x7f181806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.955+0000 7fc411c00640 1 Processor -- start 2024-04-05T22:21:21.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.955+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f182c088fd0 con 0x7f1830005e60 2024-04-05T22:21:21.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.955+0000 7f183d200640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 0x7f181806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.955+0000 7fc411c00640 1 -- start start 2024-04-05T22:21:21.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7fb26a800640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 0x7fb26c145bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00e750 tx=0x7fb25c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63c80b1f20 0x7f63c80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80c9f90 con 0x7f63c80d10f0 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80ca270 con 0x7f63c80b1f20 2024-04-05T22:21:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.956+0000 7f63cee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80ca550 con 0x7f63c815e2e0 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f183d200640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 0x7f181806d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f183004f710 tx=0x7f1824009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f63cce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f63c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f63cce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56300/0 (socket says 172.21.15.73:56300) 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f63cce00640 1 -- 172.21.15.73:0/2080899061 learned_addr learned my addr 172.21.15.73:0/2080899061 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f63c7e00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63c80b1f20 0x7f63c80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7fc411c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc40c14ffd0 0x7fc40c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7fc411c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0adf50 0x7fc40c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7fc411c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c1503b0 con 0x7fc40c0b3530 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7fc411c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c15a450 con 0x7fc40c14ffd0 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7fc411c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06f850 con 0x7fc40c0adf50 2024-04-05T22:21:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.957+0000 7f5b4f600640 1 Processor -- start 2024-04-05T22:21:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 -- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63c80b1f20 msgr2=0x7f63c80c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f5b4f600640 1 -- start start 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63c80b1f20 0x7f63c80c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 -- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 msgr2=0x7f63c80c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 -- 172.21.15.73:0/2080899061 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80094d0 con 0x7f63c815e2e0 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7fc40a000640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc40c0b3530 0x7fc40c0ad840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60216/0 (socket says 172.21.15.73:60216) 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7fc40a000640 1 -- 172.21.15.73:0/921623508 learned_addr learned my addr 172.21.15.73:0/921623508 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7fc40aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc40c14ffd0 0x7fc40c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7f63cce00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e7f0 tx=0x7f63b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.958+0000 7fc40b400640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0adf50 0x7fc40c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00094b0 con 0x7f63c815e2e0 2024-04-05T22:21:21.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b000c730 con 0x7f63c815e2e0 2024-04-05T22:21:21.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c80cbf60 con 0x7f63c815e2e0 2024-04-05T22:21:21.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fe98d200640 1 Processor -- start 2024-04-05T22:21:21.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00094b0 con 0x7f63c815e2e0 2024-04-05T22:21:21.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c806f040 con 0x7f63c815e2e0 2024-04-05T22:21:21.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fc40aa00640 1 -- 172.21.15.73:0/921623508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0adf50 msgr2=0x7fc40c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fc40aa00640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0adf50 0x7fc40c0ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fc40aa00640 1 -- 172.21.15.73:0/921623508 >> v1:172.21.15.157:6789/0 conn(0x7fc40c0b3530 legacy=0x7fc40c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fc40aa00640 1 -- 172.21.15.73:0/921623508 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c159840 con 0x7fc40c14ffd0 2024-04-05T22:21:21.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7fe98d200640 1 -- start start 2024-04-05T22:21:21.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f10fe800640 1 Processor -- start 2024-04-05T22:21:21.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.959+0000 7f5b4f600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b4815cf70 0x7f5b480d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b4f600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480ceee0 con 0x7f5b4815d350 2024-04-05T22:21:21.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b4f600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480ce5b0 con 0x7f5b4815cf70 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b4f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480ce890 con 0x7f5b480d0bd0 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56312/0 (socket says 172.21.15.73:56312) 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b3fe00640 1 -- 172.21.15.73:0/4221846070 learned_addr learned my addr 172.21.15.73:0/4221846070 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f5b4cc00640 1 --2- 172.21.15.73:0/4221846070 >> v2:172.21.15.157:3300/0 conn(0x7f5b4815cf70 0x7f5b480d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c80cbdc0 con 0x7f63c815e2e0 2024-04-05T22:21:21.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7f10fe800640 1 -- start start 2024-04-05T22:21:21.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.960+0000 7fc40aa00640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c14ffd0 0x7fc40c0b1870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00ba50 tx=0x7fc3fc02dfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615b28ba4a329fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7f5b3fe00640 1 -- 172.21.15.73:0/4221846070 >> v2:172.21.15.157:3300/0 conn(0x7f5b4815cf70 msgr2=0x7f5b480d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7f5b3fe00640 1 --2- 172.21.15.73:0/4221846070 >> v2:172.21.15.157:3300/0 conn(0x7f5b4815cf70 0x7f5b480d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fc409600640 1 -- 172.21.15.73:0/921623508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc02ed90 con 0x7fc40c14ffd0 2024-04-05T22:21:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7f5b3fe00640 1 -- 172.21.15.73:0/4221846070 >> v1:172.21.15.157:6789/0 conn(0x7f5b4815d350 legacy=0x7f5b480cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7f5b3fe00640 1 -- 172.21.15.73:0/4221846070 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480ceb70 con 0x7f5b480d0bd0 2024-04-05T22:21:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fc409600640 1 -- 172.21.15.73:0/921623508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc032070 con 0x7fc40c14ffd0 2024-04-05T22:21:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fc409600640 1 -- 172.21.15.73:0/921623508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc038620 con 0x7fc40c14ffd0 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fe98d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe988161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fe98d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 0x7fe9880cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fe98d200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe988161a80 con 0x7fe9881606a0 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fe98d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9880d3d50 con 0x7fe9880d34d0 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.961+0000 7fe98d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880ced40 con 0x7fe9881602c0 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fe97d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe9881606a0 0x7fe9880ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48810/0 (socket says 172.21.15.73:48810) 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fe97d400640 1 -- 172.21.15.73:0/1764532733 learned_addr learned my addr 172.21.15.73:0/1764532733 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fe985e00640 1 --2- 172.21.15.73:0/1764532733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe988161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 -- 172.21.15.73:0/921623508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c14ffd0 msgr2=0x7fc40c0b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c14ffd0 0x7fc40c0b1870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00ba50 tx=0x7fc3fc02dfe0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 -- 172.21.15.73:0/921623508 shutdown_connections 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0adf50 0x7fc40c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 --2- 172.21.15.73:0/921623508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c14ffd0 0x7fc40c0b1870 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00ba50 tx=0x7fc3fc02dfe0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.962+0000 7fc411c00640 1 -- 172.21.15.73:0/921623508 >> 172.21.15.73:0/921623508 conn(0x7fc40c0ce8e0 msgr2=0x7fc40c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f5b3fe00640 1 --2- 172.21.15.73:0/4221846070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009870 tx=0x7f5b3002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bff5a70c34d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7fe986800640 1 --2- 172.21.15.73:0/1764532733 >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 0x7fe9880cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10fe800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10f8162370 0x7f10f80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10fe800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f80b2ba0 0x7f10f80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10fe800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f81627f0 con 0x7f10f8161f90 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10fe800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f8158790 con 0x7f10f8162370 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10fe800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b3550 con 0x7f10f80b2ba0 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7fc411c00640 1 -- 172.21.15.73:0/921623508 shutdown_connections 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10f7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f10f8161f90 0x7f10f8164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48822/0 (socket says 172.21.15.73:48822) 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10f7400640 1 -- 172.21.15.73:0/3483964012 learned_addr learned my addr 172.21.15.73:0/3483964012 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:21.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7f10f6a00640 1 --2- 172.21.15.73:0/3483964012 >> v2:172.21.15.157:3300/0 conn(0x7f10f8162370 0x7f10f80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7fe985e00640 1 -- 172.21.15.73:0/1764532733 >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 msgr2=0x7fe9880cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7fe985e00640 1 --2- 172.21.15.73:0/1764532733 >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 0x7fe9880cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7fe985e00640 1 -- 172.21.15.73:0/1764532733 >> v1:172.21.15.73:6789/0 conn(0x7fe9881606a0 legacy=0x7fe9880ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7fe985e00640 1 -- 172.21.15.73:0/1764532733 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880ce1a0 con 0x7fe9881602c0 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.963+0000 7fc411c00640 1 -- 172.21.15.73:0/921623508 wait complete. 2024-04-05T22:21:21.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7fe986800640 1 --2- 172.21.15.73:0/1764532733 >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 0x7fe9880cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f7e00640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f80b2ba0 0x7f10f80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6000640 1 -- 172.21.15.73:0/3483964012 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248173332 0 0) 0x7f10e8001110 con 0x7f10f8161f90 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6000640 1 -- 172.21.15.73:0/3483964012 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10dc003520 con 0x7f10f8161f90 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6a00640 1 -- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f80b2ba0 msgr2=0x7f10f80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6a00640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f80b2ba0 0x7f10f80b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6a00640 1 -- 172.21.15.73:0/3483964012 >> v1:172.21.15.73:6789/0 conn(0x7f10f8161f90 legacy=0x7f10f8164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.964+0000 7f10f6a00640 1 -- 172.21.15.73:0/3483964012 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8165cf0 con 0x7f10f8162370 2024-04-05T22:21:21.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.965+0000 7fc411c00640 1 Processor -- start 2024-04-05T22:21:21.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.965+0000 7fe985e00640 1 --2- 172.21.15.73:0/1764532733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe988161540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9700097a0 tx=0x7fe97002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c86d1a3bc525b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.965+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f63b0005990 con 0x7f63c815e2e0 2024-04-05T22:21:21.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.965+0000 7fc411c00640 1 -- start start 2024-04-05T22:21:21.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.966+0000 7f10f6a00640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f8162370 0x7f10f80b2110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10e40097d0 tx=0x7f10e402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc4e93f14495735 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.966+0000 7fc411c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc411c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0b3530 0x7fc40c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc411c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c15a590 0x7fc40c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7f9909800640 1 Processor -- start 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc411c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c15af60 con 0x7fc40c0adf50 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc411c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c14ca40 con 0x7fc40c15a590 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc411c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06ba20 con 0x7fc40c0b3530 2024-04-05T22:21:21.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51146/0 (socket says 172.21.15.73:51146) 2024-04-05T22:21:21.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 -- 172.21.15.73:0/1894053947 learned_addr learned my addr 172.21.15.73:0/1894053947 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.966+0000 7f63bfe00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 0x7f63a406d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63b0088e30 con 0x7f63c815e2e0 2024-04-05T22:21:21.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7f9909800640 1 -- start start 2024-04-05T22:21:21.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40b400640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c15a590 0x7fc40c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7f63c7e00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 0x7f63a406d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 -- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0b3530 msgr2=0x7fc40c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0b3530 0x7fc40c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 -- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c15a590 msgr2=0x7fc40c15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c15a590 0x7fc40c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.967+0000 7fc40aa00640 1 -- 172.21.15.73:0/1894053947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc00b6b0 con 0x7fc40c0adf50 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc40aa00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00ba50 tx=0x7fc3fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7f63c7e00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 0x7f63a406d120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001f10 tx=0x7f63c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0423f0 con 0x7fc40c0adf50 2024-04-05T22:21:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc042550 con 0x7fc40c0adf50 2024-04-05T22:21:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0428c0 con 0x7fc40c0adf50 2024-04-05T22:21:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c15b450 con 0x7fc40c0adf50 2024-04-05T22:21:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c15b220 con 0x7fc40c0adf50 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7f9909800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99040b2050 0x7f99040d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7f9909800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99040b2430 0x7f99040d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.968+0000 7f9909800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99040cd8b0 con 0x7f99040cfe80 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9909800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040d3b30 con 0x7f99040b2430 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9909800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040d0320 con 0x7f99040b2050 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99040b2430 0x7f99040d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99040b2430 0x7f99040d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51160/0 (socket says 172.21.15.73:51160) 2024-04-05T22:21:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 -- 172.21.15.73:0/1258017598 learned_addr learned my addr 172.21.15.73:0/1258017598 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c00c6c0 con 0x7fb2500047c0 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c142ad0 con 0x7fb2500047c0 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c143090 con 0x7fb2500047c0 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 -- 172.21.15.73:0/1258017598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99040b2050 msgr2=0x7f99040d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 --2- 172.21.15.73:0/1258017598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99040b2050 0x7f99040d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 -- 172.21.15.73:0/1258017598 >> v1:172.21.15.157:6789/0 conn(0x7f99040cfe80 legacy=0x7f99041650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:21.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7f9902000640 1 -- 172.21.15.73:0/1258017598 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040d3570 con 0x7f99040b2430 2024-04-05T22:21:21.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c00c820 con 0x7fb2500047c0 2024-04-05T22:21:21.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.969+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c00cb90 con 0x7fb2500047c0 2024-04-05T22:21:21.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.970+0000 7f10f6000640 1 -- 172.21.15.73:0/3483964012 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4031780 con 0x7f10f8162370 2024-04-05T22:21:21.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.970+0000 7f10f6000640 1 -- 172.21.15.73:0/3483964012 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40318e0 con 0x7f10f8162370 2024-04-05T22:21:21.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.970+0000 7f10f6000640 1 -- 172.21.15.73:0/3483964012 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4031c70 con 0x7f10f8162370 2024-04-05T22:21:21.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.970+0000 7f9902000640 1 --2- 172.21.15.73:0/1258017598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2430 0x7f99040d13a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009870 tx=0x7f98f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb69116d9087a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb244005250 con 0x7fb2500047c0 2024-04-05T22:21:21.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7f9901600640 1 -- 172.21.15.73:0/1258017598 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0031870 con 0x7f99040b2430 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7f9901600640 1 -- 172.21.15.73:0/1258017598 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00319d0 con 0x7f99040b2430 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7f9901600640 1 -- 172.21.15.73:0/1258017598 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0031d40 con 0x7f99040b2430 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7f10fe800640 1 -- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f8162370 msgr2=0x7f10f80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7f10fe800640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f8162370 0x7f10f80b2110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10e40097d0 tx=0x7f10e402b820 comp rx=0 tx=0).stop 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.971+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc3fc041390 con 0x7fc40c0adf50 2024-04-05T22:21:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c066af0 con 0x7fc40c0adf50 2024-04-05T22:21:21.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f182c0542d0 con 0x7f1830005e60 2024-04-05T22:21:21.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 -- 172.21.15.73:0/1258017598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2430 msgr2=0x7f99040d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 --2- 172.21.15.73:0/1258017598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2430 0x7f99040d13a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009870 tx=0x7f98f002b910 comp rx=0 tx=0).stop 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 -- 172.21.15.73:0/1258017598 shutdown_connections 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 --2- 172.21.15.73:0/1258017598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2430 0x7f99040d13a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 --2- 172.21.15.73:0/1258017598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99040b2050 0x7f99040d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 -- 172.21.15.73:0/1258017598 >> 172.21.15.73:0/1258017598 conn(0x7f990409c050 msgr2=0x7f99040d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 -- 172.21.15.73:0/1258017598 shutdown_connections 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.972+0000 7f9909800640 1 -- 172.21.15.73:0/1258017598 wait complete. 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7fc402a00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 0x7fc3ec06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3fc0aadf0 con 0x7fc40c0adf50 2024-04-05T22:21:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7f10fe800640 1 -- 172.21.15.73:0/3483964012 shutdown_connections 2024-04-05T22:21:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7f10fe800640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f80b2ba0 0x7f10f80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb25c00ccf0 con 0x7fb2500047c0 2024-04-05T22:21:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7f10fe800640 1 --2- 172.21.15.73:0/3483964012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f8162370 0x7f10f80b2110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.973+0000 7f10fe800640 1 -- 172.21.15.73:0/3483964012 >> 172.21.15.73:0/3483964012 conn(0x7f10f80ce8e0 msgr2=0x7f10f80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:21.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.974+0000 7fc40a000640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 0x7fc3ec06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.974+0000 7f10fe800640 1 -- 172.21.15.73:0/3483964012 shutdown_connections 2024-04-05T22:21:21.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.974+0000 7f10fe800640 1 -- 172.21.15.73:0/3483964012 wait complete. 2024-04-05T22:21:21.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.975+0000 7f9909800640 1 Processor -- start 2024-04-05T22:21:21.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.974+0000 7fb262a00640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 0x7fb24c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.974+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb25c089a10 con 0x7fb2500047c0 2024-04-05T22:21:21.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.975+0000 7fb269e00640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 0x7fb24c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.975+0000 7fc40a000640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 0x7fc3ec06d0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b2100 tx=0x7fc3f8005c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7fb269e00640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 0x7fb24c06d0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb258000f60 tx=0x7fb25803a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:21.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.975+0000 7f9909800640 1 -- start start 2024-04-05T22:21:21.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99040cfe80 0x7f99040c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 0x7f99041582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:21.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99041587f0 con 0x7f99040b2050 2024-04-05T22:21:21.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990409d2f0 con 0x7f99040cfe80 2024-04-05T22:21:21.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.976+0000 7f9909800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990409d5d0 con 0x7f990409cb30 2024-04-05T22:21:21.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f10fe800640 1 Processor -- start 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99040cfe80 0x7f99040c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51166/0 (socket says 172.21.15.73:51166) 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 -- 172.21.15.73:0/1376413754 learned_addr learned my addr 172.21.15.73:0/1376413754 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9903400640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 0x7f99041582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f10fe800640 1 -- start start 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 -- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 msgr2=0x7f99041582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 0x7f99041582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.977+0000 7f9902a00640 1 -- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99040cfe80 msgr2=0x7f99040c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f9902a00640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99040cfe80 0x7f99040c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f9902a00640 1 -- 172.21.15.73:0/1376413754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00094d0 con 0x7f99040b2050 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f9903400640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 0x7f99041582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:21.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f9902a00640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98f400b4a0 tx=0x7f98f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f80b2ba0 0x7f10f8163d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f8162370 0x7f10f81f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80a9830 con 0x7f10f8161f90 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80af7f0 con 0x7f10f80b2ba0 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.978+0000 7f10fe800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80afad0 con 0x7f10f8162370 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51182/0 (socket says 172.21.15.73:51182) 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 -- 172.21.15.73:0/4134226033 learned_addr learned my addr 172.21.15.73:0/4134226033 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f7e00640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f8162370 0x7f10f81f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 -- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f8162370 msgr2=0x7f10f81f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f8162370 0x7f10f81f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 -- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f80b2ba0 msgr2=0x7f10f8163d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f7400640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f80b2ba0 0x7f10f8163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f80b2ba0 0x7f10f8163d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 -- 172.21.15.73:0/4134226033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009400 con 0x7f10f8161f90 2024-04-05T22:21:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.979+0000 7f10f6a00640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10e4034040 tx=0x7f10e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.980+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4010610 con 0x7f99040b2050 2024-04-05T22:21:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.980+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990409d8b0 con 0x7f99040b2050 2024-04-05T22:21:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.980+0000 7f5b47e00640 1 -- 172.21.15.73:0/4221846070 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30031640 con 0x7f5b480d0bd0 2024-04-05T22:21:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7f5b47e00640 1 -- 172.21.15.73:0/4221846070 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b300317a0 con 0x7f5b480d0bd0 2024-04-05T22:21:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f63b0054130 con 0x7f63c815e2e0 2024-04-05T22:21:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fe985400640 1 -- 172.21.15.73:0/1764532733 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97002f050 con 0x7fe9881602c0 2024-04-05T22:21:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fe985400640 1 -- 172.21.15.73:0/1764532733 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970031700 con 0x7fe9881602c0 2024-04-05T22:21:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7f5b47e00640 1 -- 172.21.15.73:0/4221846070 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30031640 con 0x7f5b480d0bd0 2024-04-05T22:21:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc3fc075f50 con 0x7fc40c0adf50 2024-04-05T22:21:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fe985400640 1 -- 172.21.15.73:0/1764532733 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970031a90 con 0x7fe9881602c0 2024-04-05T22:21:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.980+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990409de70 con 0x7f99040b2050 2024-04-05T22:21:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 -- 172.21.15.73:0/4221846070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 msgr2=0x7f5b480d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 --2- 172.21.15.73:0/4221846070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009870 tx=0x7f5b3002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99040cd230 con 0x7f99040b2050 2024-04-05T22:21:22.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fe98d200640 1 -- 172.21.15.73:0/1764532733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 msgr2=0x7fe988161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.981+0000 7fe98d200640 1 --2- 172.21.15.73:0/1764532733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe988161540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9700097a0 tx=0x7fe97002b7a0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 -- 172.21.15.73:0/1764532733 shutdown_connections 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 --2- 172.21.15.73:0/1764532733 >> v2:172.21.15.73:3300/0 conn(0x7fe9880d34d0 0x7fe9880cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 --2- 172.21.15.73:0/1764532733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe988161540 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 -- 172.21.15.73:0/1764532733 >> 172.21.15.73:0/1764532733 conn(0x7fe98809c050 msgr2=0x7fe9880d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 -- 172.21.15.73:0/1764532733 shutdown_connections 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7fe98d200640 1 -- 172.21.15.73:0/1764532733 wait complete. 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 -- 172.21.15.73:0/4221846070 shutdown_connections 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 --2- 172.21.15.73:0/4221846070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b480d1220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 --2- 172.21.15.73:0/4221846070 >> v2:172.21.15.157:3300/0 conn(0x7f5b4815cf70 0x7f5b480d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.982+0000 7f5b4f600640 1 -- 172.21.15.73:0/4221846070 >> 172.21.15.73:0/4221846070 conn(0x7f5b480d2df0 msgr2=0x7f5b480d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4032820 con 0x7f10f8161f90 2024-04-05T22:21:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40329c0 con 0x7f10f8161f90 2024-04-05T22:21:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4032d70 con 0x7f10f8161f90 2024-04-05T22:21:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4010770 con 0x7f99040b2050 2024-04-05T22:21:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4010ae0 con 0x7f99040b2050 2024-04-05T22:21:22.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80a9a10 con 0x7f10f8161f90 2024-04-05T22:21:22.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.985+0000 7fe98d200640 1 Processor -- start 2024-04-05T22:21:22.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.985+0000 7fe98d200640 1 -- start start 2024-04-05T22:21:22.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.985+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80ac4f0 con 0x7f10f8161f90 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.983+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f98f4010d00 con 0x7f99040b2050 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.984+0000 7f98faa00640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 0x7f98e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.985+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98f4088e60 con 0x7f99040b2050 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.985+0000 7f9902000640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 0x7f98e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9880d34d0 0x7fe988203000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe9880cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 0x7fe9880cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880cb270 con 0x7fe9880d34d0 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880b4270 con 0x7fe9881606a0 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7fe98d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880ca940 con 0x7fe9881602c0 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.986+0000 7f9902000640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 0x7f98e406d0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f0036000 tx=0x7f98f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7f5b4f600640 1 -- 172.21.15.73:0/4221846070 shutdown_connections 2024-04-05T22:21:22.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8066af0 con 0x7f10f8161f90 2024-04-05T22:21:22.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7fe97d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe9880cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7fe97d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe9880cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56376/0 (socket says 172.21.15.73:56376) 2024-04-05T22:21:22.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7fe97d400640 1 -- 172.21.15.73:0/1613699014 learned_addr learned my addr 172.21.15.73:0/1613699014 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:22.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7fe986800640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 0x7fe9880cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 -- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 msgr2=0x7fe9880cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe9880cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 -- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9880d34d0 msgr2=0x7fe988203000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9880d34d0 0x7fe988203000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 -- 172.21.15.73:0/1613699014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980009400 con 0x7fe9881606a0 2024-04-05T22:21:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb25c054d80 con 0x7fb2500047c0 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.988+0000 7fe986800640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 0x7fe9880cbd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e880 tx=0x7fe98000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.989+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980004020 con 0x7fe9881606a0 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.989+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980004180 con 0x7fe9881606a0 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.989+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980004020 con 0x7fe9881606a0 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.987+0000 7f5b4f600640 1 -- 172.21.15.73:0/4221846070 wait complete. 2024-04-05T22:21:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.990+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970009400 con 0x7fe9881606a0 2024-04-05T22:21:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.989+0000 7f5b4f600640 1 Processor -- start 2024-04-05T22:21:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.990+0000 7f5b4f600640 1 -- start start 2024-04-05T22:21:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.990+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9880caf00 con 0x7fe9881606a0 2024-04-05T22:21:22.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 0x7f5b48151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4815d350 0x7f5b481f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880caa80 con 0x7fe9881606a0 2024-04-05T22:21:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.992+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f98f4054220 con 0x7f99040b2050 2024-04-05T22:21:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10e404e450 con 0x7f10f8161f90 2024-04-05T22:21:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.995+0000 7f10ef400640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 0x7f10d806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.995+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e40ab420 con 0x7f10f8161f90 2024-04-05T22:21:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.993+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9800042f0 con 0x7fe9881606a0 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.995+0000 7fe97ea00640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 0x7fe96406d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 0x7f5b48151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.991+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 0x7f5b48151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57756/0 (socket says 172.21.15.73:57756) 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.992+0000 7f5b4cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.992+0000 7f5b4cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56390/0 (socket says 172.21.15.73:56390) 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.993+0000 7f5b4f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480ca620 con 0x7f5b4815d350 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.993+0000 7f5b4f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48151d20 con 0x7f5b4815cf70 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.993+0000 7f5b4f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480cce40 con 0x7f5b480d0bd0 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b3fe00640 1 -- 172.21.15.73:0/3534983509 learned_addr learned my addr 172.21.15.73:0/3534983509 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4d600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4815d350 0x7f5b481f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4cc00640 1 -- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 msgr2=0x7f5b48151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4cc00640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 0x7f5b48151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4cc00640 1 -- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4815d350 msgr2=0x7f5b481f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4cc00640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4815d350 0x7f5b481f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.994+0000 7f5b4cc00640 1 -- 172.21.15.73:0/3534983509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300094d0 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.995+0000 7f5b4cc00640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e850 tx=0x7f5b3800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.996+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800c940 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.996+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3800caa0 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.996+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800ce30 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.996+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b481e3b40 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.996+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b480c9f60 con 0x7f5b480d0bd0 2024-04-05T22:21:22.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.997+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b48068160 con 0x7f5b480d0bd0 2024-04-05T22:21:22.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.997+0000 7f10f7400640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 0x7f10d806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.995+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe980088b10 con 0x7fe9881606a0 2024-04-05T22:21:22.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.998+0000 7fe985e00640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 0x7fe96406d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.999+0000 7f10f7400640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 0x7f10d806d110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000f60 tx=0x7f10e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.999+0000 7fe985e00640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 0x7fe96406d120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe970035000 tx=0x7fe970030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:21.999+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5b38013470 con 0x7f5b480d0bd0 2024-04-05T22:21:22.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.000+0000 7f5b45600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 0x7f5b2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.000+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b38089800 con 0x7f5b480d0bd0 2024-04-05T22:21:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.001+0000 7f5b3fe00640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 0x7f5b2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.001+0000 7f5b3fe00640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 0x7f5b2406d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009870 tx=0x7f5b30008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.002+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f10e4076890 con 0x7f10f8161f90 2024-04-05T22:21:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.007+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe980053f10 con 0x7fe9881606a0 2024-04-05T22:21:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.012+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5b38054b70 con 0x7f5b480d0bd0 2024-04-05T22:21:22.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.390+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc40c068160 con 0x7fc40c0adf50 2024-04-05T22:21:22.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.390+0000 7fc402a00640 1 -- 172.21.15.73:0/1894053947 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3fc047070 con 0x7fc40c0adf50 2024-04-05T22:21:22.391 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:22.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.394+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 msgr2=0x7fc3ec06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.394+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 0x7fc3ec06d0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b2100 tx=0x7fc3f8005c00 comp rx=0 tx=0).stop 2024-04-05T22:21:22.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.394+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 msgr2=0x7fc40c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.394+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00ba50 tx=0x7fc3fc004730 comp rx=0 tx=0).stop 2024-04-05T22:21:22.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 shutdown_connections 2024-04-05T22:21:22.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3ec06ac30 0x7fc3ec06d0f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f183004e9d0 con 0x7f1830005e60 2024-04-05T22:21:22.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c15a590 0x7fc40c15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c0b3530 0x7fc40c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 --2- 172.21.15.73:0/1894053947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc40c0adf50 0x7fc40c15cc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.395+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 >> 172.21.15.73:0/1894053947 conn(0x7fc40c0ce8e0 msgr2=0x7fc40c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.396+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 shutdown_connections 2024-04-05T22:21:22.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.396+0000 7fc411c00640 1 -- 172.21.15.73:0/1894053947 wait complete. 2024-04-05T22:21:22.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.396+0000 7f1835600640 1 -- 172.21.15.73:0/1830056268 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f182c022090 con 0x7f1830005e60 2024-04-05T22:21:22.398 INFO:teuthology.orchestra.run.smithi073.stdout:60129542147 2024-04-05T22:21:22.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.400+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 msgr2=0x7f181806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.400+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 0x7f181806d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f183004f710 tx=0x7f1824009040 comp rx=0 tx=0).stop 2024-04-05T22:21:22.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.400+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 msgr2=0x7f18300e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.400+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c450 tx=0x7f182c00c910 comp rx=0 tx=0).stop 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 shutdown_connections 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181806ad00 0x7f181806d1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18300eacd0 0x7f18300495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18300069b0 0x7f18300366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 --2- 172.21.15.73:0/1830056268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1830005e60 0x7f18300e9170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.401+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 >> 172.21.15.73:0/1830056268 conn(0x7f1830009f80 msgr2=0x7f183004db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.402+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 shutdown_connections 2024-04-05T22:21:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.402+0000 7f183f000640 1 -- 172.21.15.73:0/1830056268 wait complete. 2024-04-05T22:21:22.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.407+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f63c80cc480 con 0x7f63c815e2e0 2024-04-05T22:21:22.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.408+0000 7f63bfe00640 1 -- 172.21.15.73:0/2080899061 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63c80cc480 con 0x7f63c815e2e0 2024-04-05T22:21:22.410 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:22.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.412+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f10f80a9f30 con 0x7f10f8161f90 2024-04-05T22:21:22.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.412+0000 7f10ef400640 1 -- 172.21.15.73:0/4134226033 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10f80a9f30 con 0x7f10f8161f90 2024-04-05T22:21:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.412+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 msgr2=0x7f63a406d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.412+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 0x7f63a406d120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001f10 tx=0x7f63c0009040 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 msgr2=0x7f63c80c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e7f0 tx=0x7f63b000ecb0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 shutdown_connections 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f63a406ac60 0x7f63a406d120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63c815e2e0 0x7f63c80c9890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63c80d10f0 0x7f63c80c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 --2- 172.21.15.73:0/2080899061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63c80b1f20 0x7f63c80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 >> 172.21.15.73:0/2080899061 conn(0x7f63c8099a60 msgr2=0x7f63c815c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 shutdown_connections 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.413+0000 7f63cee00640 1 -- 172.21.15.73:0/2080899061 wait complete. 2024-04-05T22:21:22.414 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.416+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 msgr2=0x7f10d806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.416+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 0x7f10d806d110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000f60 tx=0x7f10e8009040 comp rx=0 tx=0).stop 2024-04-05T22:21:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.417+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 msgr2=0x7f10f80aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.417+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb2440057d0 con 0x7fb2500047c0 2024-04-05T22:21:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.417+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10e4034040 tx=0x7f10e40046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 shutdown_connections 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10d806ac50 0x7f10d806d110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f8162370 0x7f10f81f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f8161f90 0x7f10f80aed20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 --2- 172.21.15.73:0/4134226033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f80b2ba0 0x7f10f8163d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 >> 172.21.15.73:0/4134226033 conn(0x7f10f80ce8e0 msgr2=0x7f10f80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9904068160 con 0x7f99040b2050 2024-04-05T22:21:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 shutdown_connections 2024-04-05T22:21:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7f10fe800640 1 -- 172.21.15.73:0/4134226033 wait complete. 2024-04-05T22:21:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.419+0000 7fb262a00640 1 -- 172.21.15.73:0/3370877732 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb25c0583a0 con 0x7fb2500047c0 2024-04-05T22:21:22.420 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:22.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.422+0000 7f98faa00640 1 -- 172.21.15.73:0/1376413754 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98f4014090 con 0x7f99040b2050 2024-04-05T22:21:22.421 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:22.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-04-05T22:21:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.423+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 msgr2=0x7fb24c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.423+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 0x7fb24c06d0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb258000f60 tx=0x7fb25803a040 comp rx=0 tx=0).stop 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.423+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 msgr2=0x7fb26c145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.423+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 0x7fb26c145bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00e750 tx=0x7fb25c00ec10 comp rx=0 tx=0).stop 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 shutdown_connections 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb24c06ac30 0x7fb24c06d0f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c148210 0x7fb26c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb26c147cf0 0x7fb26c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 --2- 172.21.15.73:0/3370877732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2500047c0 0x7fb26c145bb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.424+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 >> 172.21.15.73:0/3370877732 conn(0x7fb26c0ce8e0 msgr2=0x7fb26c164350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.426+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 shutdown_connections 2024-04-05T22:21:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.426+0000 7fb271400640 1 -- 172.21.15.73:0/3370877732 wait complete. 2024-04-05T22:21:22.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-04-05T22:21:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 msgr2=0x7f98e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 0x7f98e406d0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f0036000 tx=0x7f98f0008030 comp rx=0 tx=0).stop 2024-04-05T22:21:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 msgr2=0x7f990409e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98f400b4a0 tx=0x7f98f400b960 comp rx=0 tx=0).stop 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 shutdown_connections 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f98e406ac30 0x7f98e406d0f0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f0036000 tx=0x7f98f0008030 comp rx=0 tx=0).stop 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990409cb30 0x7f99041582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99040cfe80 0x7f99040c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 --2- 172.21.15.73:0/1376413754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99040b2050 0x7f990409e390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 >> 172.21.15.73:0/1376413754 conn(0x7f990409c050 msgr2=0x7f99040ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 shutdown_connections 2024-04-05T22:21:22.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.427+0000 7f9909800640 1 -- 172.21.15.73:0/1376413754 wait complete. 2024-04-05T22:21:22.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.429+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe9880caa80 con 0x7fe9881606a0 2024-04-05T22:21:22.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.430+0000 7fe97ea00640 1 -- 172.21.15.73:0/1613699014 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe980020020 con 0x7fe9881606a0 2024-04-05T22:21:22.432 INFO:teuthology.orchestra.run.smithi073.stdout:60129542147 2024-04-05T22:21:22.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.433+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5b480d3420 con 0x7f5b480d0bd0 2024-04-05T22:21:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 msgr2=0x7fe96406d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7f5b45600640 1 -- 172.21.15.73:0/3534983509 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b38058190 con 0x7f5b480d0bd0 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 0x7fe96406d120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe970035000 tx=0x7fe970030000 comp rx=0 tx=0).stop 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 msgr2=0x7fe9880cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 0x7fe9880cbd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e880 tx=0x7fe98000ed40 comp rx=0 tx=0).stop 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 shutdown_connections 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe96406ac60 0x7fe96406d120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9881606a0 0x7fe9880cbd30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.435 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:22.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9881602c0 0x7fe9880cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 --2- 172.21.15.73:0/1613699014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9880d34d0 0x7fe988203000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 >> 172.21.15.73:0/1613699014 conn(0x7fe98809c050 msgr2=0x7fe98806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.434+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 shutdown_connections 2024-04-05T22:21:22.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.435+0000 7fe98d200640 1 -- 172.21.15.73:0/1613699014 wait complete. 2024-04-05T22:21:22.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.438+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 msgr2=0x7f5b2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.438+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 0x7f5b2406d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009870 tx=0x7f5b30008030 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 msgr2=0x7f5b48150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e850 tx=0x7f5b3800ed10 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 shutdown_connections 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5b2406ac00 0x7f5b2406d0c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4815d350 0x7f5b481f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815cf70 0x7f5b48151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 --2- 172.21.15.73:0/3534983509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b480d0bd0 0x7f5b48150e40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 >> 172.21.15.73:0/3534983509 conn(0x7f5b480d2df0 msgr2=0x7f5b4815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 shutdown_connections 2024-04-05T22:21:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:22.439+0000 7f5b4f600640 1 -- 172.21.15.73:0/3534983509 wait complete. 2024-04-05T22:21:22.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-04-05T22:21:22.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-04-05T22:21:22.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.5 2024-04-05T22:21:22.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-04-05T22:21:22.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-04-05T22:21:23.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-05T22:21:23.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-05T22:21:23.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-05T22:21:23.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-05T22:21:23.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-05T22:21:23.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-05T22:21:23.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-05T22:21:23.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-05T22:21:23.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.721+0000 7f2182800640 1 Processor -- start 2024-04-05T22:21:23.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.721+0000 7f2182800640 1 -- start start 2024-04-05T22:21:23.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.723+0000 7f2182800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21740e32a0 0x7f2174052c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.723+0000 7f2182800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2174053160 0x7f217404f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.723+0000 7f2182800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740e3680 con 0x7f217404fd80 2024-04-05T22:21:23.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.724+0000 7f2182800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2174050210 con 0x7f21740e32a0 2024-04-05T22:21:23.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.724+0000 7f2182800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2174053670 con 0x7f2174053160 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21740e32a0 0x7f2174052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21740e32a0 0x7f2174052c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51202/0 (socket says 172.21.15.73:51202) 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 -- 172.21.15.73:0/819280466 learned_addr learned my addr 172.21.15.73:0/819280466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 -- 172.21.15.73:0/819280466 >> v2:172.21.15.157:3300/0 conn(0x7f2174053160 msgr2=0x7f217404f840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 --2- 172.21.15.73:0/819280466 >> v2:172.21.15.157:3300/0 conn(0x7f2174053160 0x7f217404f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.725+0000 7f2181400640 1 -- 172.21.15.73:0/819280466 >> v1:172.21.15.73:6789/0 conn(0x7f217404fd80 legacy=0x7f2174005e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.726+0000 7f2181400640 1 -- 172.21.15.73:0/819280466 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174053950 con 0x7f21740e32a0 2024-04-05T22:21:23.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.728+0000 7f2181400640 1 --2- 172.21.15.73:0/819280466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21740e32a0 0x7f2174052c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2174055a70 tx=0x7f2168034260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe279b0e3bcf1487 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.729+0000 7f217be00640 1 -- 172.21.15.73:0/819280466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216803c640 con 0x7f21740e32a0 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.729+0000 7f217be00640 1 -- 172.21.15.73:0/819280466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216803c7a0 con 0x7f21740e32a0 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.729+0000 7f217be00640 1 -- 172.21.15.73:0/819280466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216803c640 con 0x7f21740e32a0 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 -- 172.21.15.73:0/819280466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21740e32a0 msgr2=0x7f2174052c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 --2- 172.21.15.73:0/819280466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21740e32a0 0x7f2174052c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2174055a70 tx=0x7f2168034260 comp rx=0 tx=0).stop 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 -- 172.21.15.73:0/819280466 shutdown_connections 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 --2- 172.21.15.73:0/819280466 >> v2:172.21.15.157:3300/0 conn(0x7f2174053160 0x7f217404f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 --2- 172.21.15.73:0/819280466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21740e32a0 0x7f2174052c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.730+0000 7f2182800640 1 -- 172.21.15.73:0/819280466 >> 172.21.15.73:0/819280466 conn(0x7f2174009f80 msgr2=0x7f217400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.731+0000 7f2182800640 1 -- 172.21.15.73:0/819280466 shutdown_connections 2024-04-05T22:21:23.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.731+0000 7f2182800640 1 -- 172.21.15.73:0/819280466 wait complete. 2024-04-05T22:21:23.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.733+0000 7f2182800640 1 Processor -- start 2024-04-05T22:21:23.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.733+0000 7f2182800640 1 -- start start 2024-04-05T22:21:23.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.735+0000 7f2182800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2181400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2181400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51216/0 (socket says 172.21.15.73:51216) 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2181400640 1 -- 172.21.15.73:0/3558719229 learned_addr learned my addr 172.21.15.73:0/3558719229 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2182800640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 0x7f21740d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2182800640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 0x7f217404c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2182800640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740da7c0 con 0x7f217404fd80 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2182800640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740e5f50 con 0x7f21740e32a0 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.736+0000 7f2182800640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740e5c70 con 0x7f2174053160 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181400640 1 -- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 msgr2=0x7f21740d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181e00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 0x7f217404c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2180a00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 0x7f21740d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181400640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 0x7f21740d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181400640 1 -- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 msgr2=0x7f217404c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181400640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 0x7f217404c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.737+0000 7f2181400640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680094d0 con 0x7f217404fd80 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.738+0000 7f2181e00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 0x7f217404c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.738+0000 7f2180a00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 0x7f21740d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.738+0000 7f2181400640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f21680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.739+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168003990 con 0x7f217404fd80 2024-04-05T22:21:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.739+0000 7f2182800640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740da100 con 0x7f217404fd80 2024-04-05T22:21:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.739+0000 7f2182800640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21740d2080 con 0x7f217404fd80 2024-04-05T22:21:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.739+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168003af0 con 0x7f217404fd80 2024-04-05T22:21:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.739+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168003e80 con 0x7f217404fd80 2024-04-05T22:21:23.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.741+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2150005250 con 0x7f217404fd80 2024-04-05T22:21:23.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.744+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2168043040 con 0x7f217404fd80 2024-04-05T22:21:23.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.746+0000 7f2179600640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 0x7f215c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.750+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21680b44c0 con 0x7f217404fd80 2024-04-05T22:21:23.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.751+0000 7f2180a00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 0x7f215c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.756+0000 7f2180a00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 0x7f215c06d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f217404eb60 tx=0x7f2170008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.756+0000 7fe85d200640 1 Processor -- start 2024-04-05T22:21:23.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.756+0000 7fe85d200640 1 -- start start 2024-04-05T22:21:23.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.759+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f216807f720 con 0x7f217404fd80 2024-04-05T22:21:23.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.759+0000 7fe85d200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8580b20d0 0x7fe85813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.759+0000 7fe85d200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85813c270 con 0x7fe85815d2b0 2024-04-05T22:21:23.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.759+0000 7fe85d200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe858159810 con 0x7fe85813bc90 2024-04-05T22:21:23.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.759+0000 7fe85d200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85815d710 con 0x7fe8580b20d0 2024-04-05T22:21:23.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.761+0000 7fe856a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe85815d2b0 0x7fe8580b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48836/0 (socket says 172.21.15.73:48836) 2024-04-05T22:21:23.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.761+0000 7fe857400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe85813bc90 0x7fe858159100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60242/0 (socket says 172.21.15.73:60242) 2024-04-05T22:21:23.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.761+0000 7fe857400640 1 -- 172.21.15.73:0/2937098025 learned_addr learned my addr 172.21.15.73:0/2937098025 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:23.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.761+0000 7fe857e00640 1 --2- 172.21.15.73:0/2937098025 >> v2:172.21.15.157:3300/0 conn(0x7fe8580b20d0 0x7fe85813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.761+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169888480 0 0) 0x7fe858159810 con 0x7fe85813bc90 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.762+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe83c003520 con 0x7fe85813bc90 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.762+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023226829 0 0) 0x7fe85813c270 con 0x7fe85815d2b0 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.762+0000 7fe857e00640 1 -- 172.21.15.73:0/2937098025 >> v1:172.21.15.157:6789/0 conn(0x7fe85813bc90 legacy=0x7fe858159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.762+0000 7fe857e00640 1 -- 172.21.15.73:0/2937098025 >> v1:172.21.15.73:6789/0 conn(0x7fe85815d2b0 legacy=0x7fe8580b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.762+0000 7fe857e00640 1 -- 172.21.15.73:0/2937098025 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858159ad0 con 0x7fe8580b20d0 2024-04-05T22:21:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.764+0000 7fe857e00640 1 --2- 172.21.15.73:0/2937098025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8580b20d0 0x7fe85813e4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8480087a0 tx=0x7fe84802df60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7200be6bd5aa0c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.765+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84802ed10 con 0x7fe8580b20d0 2024-04-05T22:21:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.765+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848032070 con 0x7fe8580b20d0 2024-04-05T22:21:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.766+0000 7fe85d200640 1 -- 172.21.15.73:0/2937098025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8580b20d0 msgr2=0x7fe85813e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.766+0000 7fe85d200640 1 --2- 172.21.15.73:0/2937098025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8580b20d0 0x7fe85813e4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8480087a0 tx=0x7fe84802df60 comp rx=0 tx=0).stop 2024-04-05T22:21:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.765+0000 7fe856000640 1 -- 172.21.15.73:0/2937098025 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848038620 con 0x7fe8580b20d0 2024-04-05T22:21:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.767+0000 7fe85d200640 1 -- 172.21.15.73:0/2937098025 shutdown_connections 2024-04-05T22:21:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.767+0000 7fe85d200640 1 --2- 172.21.15.73:0/2937098025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8580b20d0 0x7fe85813e4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.767+0000 7fe85d200640 1 -- 172.21.15.73:0/2937098025 >> 172.21.15.73:0/2937098025 conn(0x7fe8580b06c0 msgr2=0x7fe8580b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.767+0000 7fe85d200640 1 -- 172.21.15.73:0/2937098025 shutdown_connections 2024-04-05T22:21:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.767+0000 7fe85d200640 1 -- 172.21.15.73:0/2937098025 wait complete. 2024-04-05T22:21:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.770+0000 7fe85d200640 1 Processor -- start 2024-04-05T22:21:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.770+0000 7fe85d200640 1 -- start start 2024-04-05T22:21:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.770+0000 7f0821000640 1 Processor -- start 2024-04-05T22:21:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.771+0000 7f0821000640 1 -- start start 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8580b20d0 0x7fe858155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 0x7fe8581eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581ef420 con 0x7fe85813bc90 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85813b7b0 con 0x7fe85815d2b0 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe85d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858155850 con 0x7fe8580b20d0 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.772+0000 7fe857e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.773+0000 7fe857e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57780/0 (socket says 172.21.15.73:57780) 2024-04-05T22:21:23.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.773+0000 7fe857e00640 1 -- 172.21.15.73:0/2554832561 learned_addr learned my addr 172.21.15.73:0/2554832561 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.773+0000 7fe2e0600640 1 Processor -- start 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.773+0000 7fe2e0600640 1 -- start start 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe856a00640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 0x7fe8581eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe857e00640 1 -- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8580b20d0 msgr2=0x7fe858155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe857e00640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8580b20d0 0x7fe858155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe857e00640 1 -- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 msgr2=0x7fe8581eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7f0821000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f081c06b0a0 0x7f081c164d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe857e00640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 0x7fe8581eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7fe857e00640 1 -- 172.21.15.73:0/2554832561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848008480 con 0x7fe85815d2b0 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7f0821000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c0cd7b0 con 0x7f081c065b10 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7f0821000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c06cc60 con 0x7f081c159f40 2024-04-05T22:21:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.774+0000 7f0821000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0ad740 con 0x7f081c06b0a0 2024-04-05T22:21:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.775+0000 7f0819e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f081c159f40 0x7f081c06be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60258/0 (socket says 172.21.15.73:60258) 2024-04-05T22:21:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.775+0000 7f0819e00640 1 -- 172.21.15.73:0/594427105 learned_addr learned my addr 172.21.15.73:0/594427105 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.775+0000 7fe856a00640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 0x7fe8581eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7fe857e00640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8480087a0 tx=0x7fe848004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f081a800640 1 --2- 172.21.15.73:0/594427105 >> v2:172.21.15.157:3300/0 conn(0x7f081c06b0a0 0x7f081c164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989690948 0 0) 0x7f081c0cd7b0 con 0x7f081c065b10 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0800003520 con 0x7f081c065b10 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429433353 0 0) 0x7f081c06cc60 con 0x7f081c159f40 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f081a800640 1 -- 172.21.15.73:0/594427105 >> v1:172.21.15.157:6789/0 conn(0x7f081c159f40 legacy=0x7f081c06be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f081a800640 1 -- 172.21.15.73:0/594427105 >> v1:172.21.15.73:6789/0 conn(0x7f081c065b10 legacy=0x7f081c06c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7f081a800640 1 -- 172.21.15.73:0/594427105 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0cd230 con 0x7f081c06b0a0 2024-04-05T22:21:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7fe2e0600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe2d8158940 0x7fe2d806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800b5f0 con 0x7fe85815d2b0 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7fe2e0600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d806f5b0 con 0x7fe2d806d0c0 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7fe2e0600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d8070410 con 0x7fe2d8158560 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.776+0000 7fe2e0600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d80b3980 con 0x7fe2d8158940 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe2ddc00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe2d8158560 0x7fe2d80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60270/0 (socket says 172.21.15.73:60270) 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe2ddc00640 1 -- 172.21.15.73:0/581225389 learned_addr learned my addr 172.21.15.73:0/581225389 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe2dd200640 1 --2- 172.21.15.73:0/581225389 >> v2:172.21.15.157:3300/0 conn(0x7fe2d8158940 0x7fe2d806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84800b750 con 0x7fe85815d2b0 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.777+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848038480 con 0x7fe85815d2b0 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe858155b30 con 0x7fe85815d2b0 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463527238 0 0) 0x7fe2d8070410 con 0x7fe2d8158560 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c0003520 con 0x7fe2d8158560 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622978323 0 0) 0x7fe2d806f5b0 con 0x7fe2d806d0c0 2024-04-05T22:21:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d8070410 con 0x7fe2d806d0c0 2024-04-05T22:21:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.779+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8581560f0 con 0x7fe85815d2b0 2024-04-05T22:21:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7f081a800640 1 --2- 172.21.15.73:0/594427105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c06b0a0 0x7f081c164d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85033de41b9de3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2dd200640 1 -- 172.21.15.73:0/581225389 >> v1:172.21.15.157:6789/0 conn(0x7fe2d8158560 legacy=0x7fe2d80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2dd200640 1 -- 172.21.15.73:0/581225389 >> v1:172.21.15.73:6789/0 conn(0x7fe2d806d0c0 legacy=0x7fe2d815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.778+0000 7fe2dd200640 1 -- 172.21.15.73:0/581225389 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80ccef0 con 0x7fe2d8158940 2024-04-05T22:21:23.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.779+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081402ccf0 con 0x7f081c06b0a0 2024-04-05T22:21:23.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.780+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081402ce50 con 0x7f081c06b0a0 2024-04-05T22:21:23.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.780+0000 7f0818a00640 1 -- 172.21.15.73:0/594427105 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814034660 con 0x7f081c06b0a0 2024-04-05T22:21:23.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7fea9aa00640 1 Processor -- start 2024-04-05T22:21:23.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7fe2dd200640 1 --2- 172.21.15.73:0/581225389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d8158940 0x7fe2d806cb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40097a0 tx=0x7fe2c402ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69b64cdad8303fd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85804e160 con 0x7fe85815d2b0 2024-04-05T22:21:23.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7f0821000640 1 -- 172.21.15.73:0/594427105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c06b0a0 msgr2=0x7f081c164d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7f0821000640 1 --2- 172.21.15.73:0/594427105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c06b0a0 0x7f081c164d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081402bf40 comp rx=0 tx=0).stop 2024-04-05T22:21:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.781+0000 7fea9aa00640 1 -- start start 2024-04-05T22:21:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c402f820 con 0x7fe2d8158940 2024-04-05T22:21:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7f0821000640 1 -- 172.21.15.73:0/594427105 shutdown_connections 2024-04-05T22:21:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7f0821000640 1 --2- 172.21.15.73:0/594427105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c06b0a0 0x7f081c164d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7f0821000640 1 -- 172.21.15.73:0/594427105 >> 172.21.15.73:0/594427105 conn(0x7f081c0cf190 msgr2=0x7f081c0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe8480385e0 con 0x7fe85815d2b0 2024-04-05T22:21:23.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7f0821000640 1 -- 172.21.15.73:0/594427105 shutdown_connections 2024-04-05T22:21:23.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7f0821000640 1 -- 172.21.15.73:0/594427105 wait complete. 2024-04-05T22:21:23.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c402f980 con 0x7fe2d8158940 2024-04-05T22:21:23.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.782+0000 7fe2d7e00640 1 -- 172.21.15.73:0/581225389 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c402fd50 con 0x7fe2d8158940 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fe2e0600640 1 -- 172.21.15.73:0/581225389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d8158940 msgr2=0x7fe2d806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fea9aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea940b1f20 0x7fea940cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fea9aa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940cf5a0 con 0x7fea9415e2e0 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fea9aa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940b1900 con 0x7fea940d10f0 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fea9aa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940d0140 con 0x7fea940b1f20 2024-04-05T22:21:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fe84f400640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 0x7fe83806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.783+0000 7fe2e0600640 1 --2- 172.21.15.73:0/581225389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d8158940 0x7fe2d806cb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40097a0 tx=0x7fe2c402ba00 comp rx=0 tx=0).stop 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fe2e0600640 1 -- 172.21.15.73:0/581225389 shutdown_connections 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fe2e0600640 1 --2- 172.21.15.73:0/581225389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d8158940 0x7fe2d806cb80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fe2e0600640 1 -- 172.21.15.73:0/581225389 >> 172.21.15.73:0/581225389 conn(0x7fe2d80ce8e0 msgr2=0x7fe2d80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe84807b020 con 0x7fe85815d2b0 2024-04-05T22:21:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fe857400640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 0x7fe83806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7fea98a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea940b1f20 0x7fea940cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.784+0000 7f649f000640 1 Processor -- start 2024-04-05T22:21:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fe2e0600640 1 -- 172.21.15.73:0/581225389 shutdown_connections 2024-04-05T22:21:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fea93400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fea940d10f0 0x7fea940adc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60272/0 (socket says 172.21.15.73:60272) 2024-04-05T22:21:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fe2e0600640 1 -- 172.21.15.73:0/581225389 wait complete. 2024-04-05T22:21:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7f649f000640 1 -- start start 2024-04-05T22:21:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fea93e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fea9415e2e0 0x7fea940d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48856/0 (socket says 172.21.15.73:48856) 2024-04-05T22:21:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fea93400640 1 -- 172.21.15.73:0/3824450228 learned_addr learned my addr 172.21.15.73:0/3824450228 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:23.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7fe857400640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 0x7fe83806d0f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8500012e0 tx=0x7fe850001650 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.785+0000 7f95da800640 1 Processor -- start 2024-04-05T22:21:23.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7f0821000640 1 Processor -- start 2024-04-05T22:21:23.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7fea98a00640 1 -- 172.21.15.73:0/3824450228 >> v1:172.21.15.157:6789/0 conn(0x7fea940d10f0 legacy=0x7fea940adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7f0821000640 1 -- start start 2024-04-05T22:21:23.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7fea92a00640 1 -- 172.21.15.73:0/3824450228 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280555549 0 0) 0x7fea940cf5a0 con 0x7fea9415e2e0 2024-04-05T22:21:23.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7fea98a00640 1 -- 172.21.15.73:0/3824450228 >> v1:172.21.15.73:6789/0 conn(0x7fea9415e2e0 legacy=0x7fea940d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7fea98a00640 1 -- 172.21.15.73:0/3824450228 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9406cde0 con 0x7fea940b1f20 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f649f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f649f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64980b1f20 0x7f64980d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f649f000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980cf5a0 con 0x7f64980d0f20 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f649f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980d0140 con 0x7f64980b1f20 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f649f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980cf7d0 con 0x7f649815e2e0 2024-04-05T22:21:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.786+0000 7f95da800640 1 -- start start 2024-04-05T22:21:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f6497400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64980b1f20 0x7f64980d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95da800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95d415cf70 0x7f95d40d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7fea98a00640 1 --2- 172.21.15.73:0/3824450228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940b1f20 0x7fea940cf060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea840097a0 tx=0x7fea8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20760080445e4998 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f6497400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64980b1f20 0x7f64980d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57814/0 (socket says 172.21.15.73:57814) 2024-04-05T22:21:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f6497400640 1 -- 172.21.15.73:0/3266315259 learned_addr learned my addr 172.21.15.73:0/3266315259 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.787+0000 7f6497e00640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95da800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95d40d1930 0x7f95d40b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95da800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d40ceee0 con 0x7f95d40d0bd0 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95da800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40ce5b0 con 0x7f95d40d1930 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95da800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40ce890 con 0x7f95d415cf70 2024-04-05T22:21:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c06b0a0 0x7f081c0c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081c159f40 0x7f081c2024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c1eff10 con 0x7f081c06b0a0 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c4350 con 0x7f081c065b10 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f0821000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c45d0 con 0x7f081c159f40 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95d2a00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f95d40d0bd0 0x7f95d40d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60290/0 (socket says 172.21.15.73:60290) 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f95d2a00640 1 -- 172.21.15.73:0/2852863804 learned_addr learned my addr 172.21.15.73:0/2852863804 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f6497400640 1 -- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 msgr2=0x7f64980cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.788+0000 7f6497400640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f6497400640 1 -- 172.21.15.73:0/3266315259 >> v1:172.21.15.157:6789/0 conn(0x7f64980d0f20 legacy=0x7f649815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f95d3400640 1 --2- 172.21.15.73:0/2852863804 >> v2:172.21.15.157:3300/0 conn(0x7f95d415cf70 0x7f95d40d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f95d3e00640 1 --2- 172.21.15.73:0/2852863804 >> v2:172.21.15.73:3300/0 conn(0x7f95d40d1930 0x7f95d40b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f0819400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c06b0a0 0x7f081c0c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f0819e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7fea92a00640 1 -- 172.21.15.73:0/3824450228 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84031640 con 0x7fea940b1f20 2024-04-05T22:21:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7fea92a00640 1 -- 172.21.15.73:0/3824450228 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840317a0 con 0x7fea940b1f20 2024-04-05T22:21:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7fea92a00640 1 -- 172.21.15.73:0/3824450228 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84031b30 con 0x7fea940b1f20 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f081a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081c159f40 0x7f081c2024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f0819e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57834/0 (socket says 172.21.15.73:57834) 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f0819e00640 1 -- 172.21.15.73:0/1945080294 learned_addr learned my addr 172.21.15.73:0/1945080294 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7fe2e0600640 1 Processor -- start 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7fe2e0600640 1 -- start start 2024-04-05T22:21:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f95d3400640 1 -- 172.21.15.73:0/2852863804 >> v2:172.21.15.73:3300/0 conn(0x7f95d40d1930 msgr2=0x7f95d40b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f95d3400640 1 --2- 172.21.15.73:0/2852863804 >> v2:172.21.15.73:3300/0 conn(0x7f95d40d1930 0x7f95d40b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f95d3400640 1 -- 172.21.15.73:0/2852863804 >> v1:172.21.15.157:6789/0 conn(0x7f95d40d0bd0 legacy=0x7f95d40d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f95d3400640 1 -- 172.21.15.73:0/2852863804 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40ceb70 con 0x7f95d415cf70 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f6b1be00640 1 Processor -- start 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f0819e00640 1 -- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081c159f40 msgr2=0x7f081c2024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f0819e00640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081c159f40 0x7f081c2024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f0819e00640 1 -- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c06b0a0 msgr2=0x7f081c0c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f0819e00640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c06b0a0 0x7f081c0c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f0819e00640 1 -- 172.21.15.73:0/1945080294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009a70 con 0x7f081c065b10 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.789+0000 7f6497400640 1 -- 172.21.15.73:0/3266315259 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980d13b0 con 0x7f64980b1f20 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.790+0000 7f6b1be00640 1 -- start start 2024-04-05T22:21:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7f0819e00640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f080800c3f0 tx=0x7f080800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 0x7fe2d814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 -- 172.21.15.73:0/3824450228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940b1f20 msgr2=0x7fea940cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3824450228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940b1f20 0x7fea940cf060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea840097a0 tx=0x7fea8402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d8158940 0x7fe2d814d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81384f0 con 0x7fe2d8158560 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81387d0 con 0x7fe2d806d0c0 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2e0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8138ab0 con 0x7fe2d8158940 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7f6497400640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980b1f20 0x7f64980d09e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6488009870 tx=0x7f648802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a3aba9642b2685 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 -- 172.21.15.73:0/3824450228 shutdown_connections 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3824450228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940b1f20 0x7fea940cf060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 -- 172.21.15.73:0/3824450228 >> 172.21.15.73:0/3824450228 conn(0x7fea94099a60 msgr2=0x7fea94099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2ddc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2ddc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57850/0 (socket says 172.21.15.73:57850) 2024-04-05T22:21:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2ddc00640 1 -- 172.21.15.73:0/3746938096 learned_addr learned my addr 172.21.15.73:0/3746938096 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fe2dd200640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 0x7fe2d814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7fea9aa00640 1 -- 172.21.15.73:0/3824450228 shutdown_connections 2024-04-05T22:21:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.791+0000 7f95d3400640 1 --2- 172.21.15.73:0/2852863804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d40d3910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95d40d08f0 tx=0x7f95c002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ebe353f3857796 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fea9aa00640 1 -- 172.21.15.73:0/3824450228 wait complete. 2024-04-05T22:21:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2de600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d8158940 0x7fe2d814d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b1be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 0x7f6b140d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b1be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b140d0bd0 0x7f6b140d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b1be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b140ceee0 con 0x7f6b1415d350 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b1be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140ce5b0 con 0x7f6b1415cf70 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b1be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140ce890 con 0x7f6b140d0bd0 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2dd200640 1 -- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d8158940 msgr2=0x7fe2d814d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2dd200640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d8158940 0x7fe2d814d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2dd200640 1 -- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 msgr2=0x7fe2d81f9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2dd200640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2dd200640 1 -- 172.21.15.73:0/3746938096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4009400 con 0x7fe2d8158560 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7fe2ddc00640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.792+0000 7f6b18a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b140d0bd0 0x7f6b140d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7fe2dd200640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 0x7fe2d814d460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4033000 tx=0x7fe2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b140d0bd0 0x7f6b140d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57856/0 (socket says 172.21.15.73:57856) 2024-04-05T22:21:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 -- 172.21.15.73:0/856335652 learned_addr learned my addr 172.21.15.73:0/856335652 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c403b040 con 0x7fe2d8158560 2024-04-05T22:21:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c4037c80 con 0x7fe2d8158560 2024-04-05T22:21:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b19400640 1 --2- 172.21.15.73:0/856335652 >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 0x7f6b140d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4042480 con 0x7fe2d8158560 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 -- 172.21.15.73:0/856335652 >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 msgr2=0x7f6b140d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 --2- 172.21.15.73:0/856335652 >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 0x7f6b140d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 -- 172.21.15.73:0/856335652 >> v1:172.21.15.157:6789/0 conn(0x7f6b1415d350 legacy=0x7f6b140cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b18a00640 1 -- 172.21.15.73:0/856335652 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140ceb70 con 0x7f6b140d0bd0 2024-04-05T22:21:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7f6b19400640 1 --2- 172.21.15.73:0/856335652 >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 0x7f6b140d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.793+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d8138d90 con 0x7fe2d8158560 2024-04-05T22:21:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.794+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d81fa450 con 0x7fe2d8158560 2024-04-05T22:21:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.794+0000 7fea9aa00640 1 Processor -- start 2024-04-05T22:21:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.794+0000 7fea9aa00640 1 -- start start 2024-04-05T22:21:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.795+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8066af0 con 0x7fe2d8158560 2024-04-05T22:21:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.795+0000 7f6b18a00640 1 --2- 172.21.15.73:0/856335652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b140d0bd0 0x7f6b140d1220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b14160ca0 tx=0x7f6b0002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67219834009b121 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.795+0000 7fea9aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b1f20 0x7fea9406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.795+0000 7fea9aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea9aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea9415e2e0 0x7fea9414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea9aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940c5d20 con 0x7fea940d10f0 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea9aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406f290 con 0x7fea940b1f20 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea9aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406efb0 con 0x7fea9415e2e0 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea93400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea93400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51280/0 (socket says 172.21.15.73:51280) 2024-04-05T22:21:23.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea93400640 1 -- 172.21.15.73:0/3703092894 learned_addr learned my addr 172.21.15.73:0/3703092894 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea93e00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b1f20 0x7fea9406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fea98a00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea9415e2e0 0x7fea9414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.796+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe2c402d040 con 0x7fe2d8158560 2024-04-05T22:21:23.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 -- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea9415e2e0 msgr2=0x7fea9414c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea9415e2e0 0x7fea9414c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 -- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b1f20 msgr2=0x7fea9406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b1f20 0x7fea9406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 -- 172.21.15.73:0/3703092894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84009400 con 0x7fea940d10f0 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.797+0000 7fea93400640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c500 tx=0x7fea8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.798+0000 7fe2d5600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 0x7fe2b806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.798+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c40ab660 con 0x7fe2d8158560 2024-04-05T22:21:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.798+0000 7fe2ddc00640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 0x7fe2b806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.799+0000 7fe2ddc00640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 0x7fe2b806d110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc000f60 tx=0x7fe2cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6b13e00640 1 -- 172.21.15.73:0/856335652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00031870 con 0x7f6b140d0bd0 2024-04-05T22:21:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f95d2000640 1 -- 172.21.15.73:0/2852863804 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c002e070 con 0x7f95d415cf70 2024-04-05T22:21:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08080074e0 con 0x7f081c065b10 2024-04-05T22:21:23.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6b13e00640 1 -- 172.21.15.73:0/856335652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000319d0 con 0x7f6b140d0bd0 2024-04-05T22:21:23.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6b13e00640 1 -- 172.21.15.73:0/856335652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00031d60 con 0x7f6b140d0bd0 2024-04-05T22:21:23.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0808007640 con 0x7f081c065b10 2024-04-05T22:21:23.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808004a00 con 0x7f081c065b10 2024-04-05T22:21:23.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6496a00640 1 -- 172.21.15.73:0/3266315259 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488031640 con 0x7f64980b1f20 2024-04-05T22:21:23.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80005ce0 con 0x7fea940d10f0 2024-04-05T22:21:23.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80005e40 con 0x7fea940d10f0 2024-04-05T22:21:23.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800072f0 con 0x7fea940d10f0 2024-04-05T22:21:23.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6496a00640 1 -- 172.21.15.73:0/3266315259 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64880317a0 con 0x7f64980b1f20 2024-04-05T22:21:23.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f6496a00640 1 -- 172.21.15.73:0/3266315259 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488031b30 con 0x7f64980b1f20 2024-04-05T22:21:23.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c0c48b0 con 0x7f081c065b10 2024-04-05T22:21:23.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.801+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe8480773f0 con 0x7fe85815d2b0 2024-04-05T22:21:23.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940c5660 con 0x7fea940d10f0 2024-04-05T22:21:23.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 -- 172.21.15.73:0/856335652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b140d0bd0 msgr2=0x7f6b140d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 --2- 172.21.15.73:0/856335652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b140d0bd0 0x7f6b140d1220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b14160ca0 tx=0x7f6b0002b910 comp rx=0 tx=0).stop 2024-04-05T22:21:23.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c0c4e70 con 0x7f081c065b10 2024-04-05T22:21:23.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f649f000640 1 -- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980b1f20 msgr2=0x7f64980d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f649f000640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980b1f20 0x7f64980d09e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6488009870 tx=0x7f648802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 -- 172.21.15.73:0/856335652 shutdown_connections 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 --2- 172.21.15.73:0/856335652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b140d0bd0 0x7f6b140d1220 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 --2- 172.21.15.73:0/856335652 >> v2:172.21.15.73:3300/0 conn(0x7f6b1415cf70 0x7f6b140d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f6b1be00640 1 -- 172.21.15.73:0/856335652 >> 172.21.15.73:0/856335652 conn(0x7f6b140d2df0 msgr2=0x7f6b140d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea941475b0 con 0x7fea940d10f0 2024-04-05T22:21:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.803+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe2c40769f0 con 0x7fe2d8158560 2024-04-05T22:21:23.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.803+0000 7f6b1be00640 1 -- 172.21.15.73:0/856335652 shutdown_connections 2024-04-05T22:21:23.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.803+0000 7f6b1be00640 1 -- 172.21.15.73:0/856335652 wait complete. 2024-04-05T22:21:23.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f95d2000640 1 -- 172.21.15.73:0/2852863804 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c00315a0 con 0x7f95d415cf70 2024-04-05T22:21:23.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.802+0000 7f95d2000640 1 -- 172.21.15.73:0/2852863804 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0031930 con 0x7f95d415cf70 2024-04-05T22:21:23.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.803+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c068160 con 0x7f081c065b10 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 -- 172.21.15.73:0/3266315259 shutdown_connections 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980b1f20 0x7f64980d09e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 --2- 172.21.15.73:0/3266315259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 -- 172.21.15.73:0/3266315259 >> 172.21.15.73:0/3266315259 conn(0x7f6498099a60 msgr2=0x7f6498099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 -- 172.21.15.73:0/3266315259 shutdown_connections 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f95da800640 1 -- 172.21.15.73:0/2852863804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 msgr2=0x7f95d40d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f95da800640 1 --2- 172.21.15.73:0/2852863804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d40d3910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95d40d08f0 tx=0x7f95c002b840 comp rx=0 tx=0).stop 2024-04-05T22:21:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f649f000640 1 -- 172.21.15.73:0/3266315259 wait complete. 2024-04-05T22:21:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f95da800640 1 -- 172.21.15.73:0/2852863804 shutdown_connections 2024-04-05T22:21:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7f95da800640 1 --2- 172.21.15.73:0/2852863804 >> v2:172.21.15.73:3300/0 conn(0x7f95d40d1930 0x7f95d40b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.805+0000 7f95da800640 1 --2- 172.21.15.73:0/2852863804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d40d3910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.805+0000 7f95da800640 1 -- 172.21.15.73:0/2852863804 >> 172.21.15.73:0/2852863804 conn(0x7f95d40d2df0 msgr2=0x7f95d40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.804+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea940cee60 con 0x7fea940d10f0 2024-04-05T22:21:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.805+0000 7f95da800640 1 -- 172.21.15.73:0/2852863804 shutdown_connections 2024-04-05T22:21:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.805+0000 7f95da800640 1 -- 172.21.15.73:0/2852863804 wait complete. 2024-04-05T22:21:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.806+0000 7f649f000640 1 Processor -- start 2024-04-05T22:21:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.806+0000 7f649f000640 1 -- start start 2024-04-05T22:21:23.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.806+0000 7f6b1be00640 1 Processor -- start 2024-04-05T22:21:23.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.807+0000 7f6b1be00640 1 -- start start 2024-04-05T22:21:23.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.807+0000 7f649f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64980b1f20 0x7f6498147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f649f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 0x7f64980c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f649f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f649f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980c4e10 con 0x7f64980b1f20 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f649f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980c5df0 con 0x7f64980d0f20 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f6497400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 0x7f64980c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f6497400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 0x7f64980c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57876/0 (socket says 172.21.15.73:57876) 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f6497400640 1 -- 172.21.15.73:0/3127471252 learned_addr learned my addr 172.21.15.73:0/3127471252 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f649d000640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649f000640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649806c500 con 0x7f649815e2e0 2024-04-05T22:21:23.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 0x7f6b140caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 0x7f6b14152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1415d350 0x7f6b140b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649d000640 1 -- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 msgr2=0x7f64980c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649d000640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 0x7f64980c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649d000640 1 -- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64980b1f20 msgr2=0x7f6498147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649d000640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64980b1f20 0x7f6498147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f649d000640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880094d0 con 0x7f649815e2e0 2024-04-05T22:21:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.809+0000 7f6b19400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 0x7f6b140caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b19400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 0x7f6b140caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56458/0 (socket says 172.21.15.73:56458) 2024-04-05T22:21:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b19400640 1 -- 172.21.15.73:0/2082496261 learned_addr learned my addr 172.21.15.73:0/2082496261 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b18a00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 0x7f6b14152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1406bd60 con 0x7f6b1415d350 2024-04-05T22:21:23.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14159010 con 0x7f6b1415cf70 2024-04-05T22:21:23.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1406b6a0 con 0x7f6b140d0bd0 2024-04-05T22:21:23.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f649d000640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980c58b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f649000c690 tx=0x7f649000cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.808+0000 7f95da800640 1 Processor -- start 2024-04-05T22:21:23.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.810+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0808004b60 con 0x7f081c065b10 2024-04-05T22:21:23.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490003b70 con 0x7f649815e2e0 2024-04-05T22:21:23.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f649f000640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64980c4b30 con 0x7f649815e2e0 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f649f000640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649806c950 con 0x7f649815e2e0 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f95da800640 1 -- start start 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f6b18a00640 1 -- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 msgr2=0x7f6b140caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f6b18a00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 0x7f6b140caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f6b18a00640 1 -- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1415d350 msgr2=0x7f6b140b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f6b18a00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1415d350 0x7f6b140b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fea80007510 con 0x7fea940d10f0 2024-04-05T22:21:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f0812000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 0x7f07fc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0808088f40 con 0x7f081c065b10 2024-04-05T22:21:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.811+0000 7f6b18a00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000094d0 con 0x7f6b1415cf70 2024-04-05T22:21:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490003cd0 con 0x7f649815e2e0 2024-04-05T22:21:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490005310 con 0x7f649815e2e0 2024-04-05T22:21:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d40d1930 0x7f95d40b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f0819400640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 0x7f07fc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415f840 con 0x7f95d40d0bd0 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41f6d60 con 0x7f95d415cf70 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.812+0000 7f95da800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40b69d0 con 0x7f95d40d1930 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f95d3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f95d3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51318/0 (socket says 172.21.15.73:51318) 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f95d3400640 1 -- 172.21.15.73:0/3197842117 learned_addr learned my addr 172.21.15.73:0/3197842117 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f95d3e00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f95d2a00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d40d1930 0x7f95d40b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7f6b18a00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 0x7f6b14152c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b00033000 tx=0x7f6b000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7fea8be00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 0x7fea7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.813+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea800887f0 con 0x7fea940d10f0 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3400640 1 -- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d40d1930 msgr2=0x7f95d40b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3400640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d40d1930 0x7f95d40b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3400640 1 -- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 msgr2=0x7f95d4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3400640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3400640 1 -- 172.21.15.73:0/3197842117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0009400 con 0x7f95d40d0bd0 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.814+0000 7f95d3e00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:21:23.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.815+0000 7f95d3400640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009890 tx=0x7f95c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.815+0000 7f0819400640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 0x7f07fc06d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f081c0b4060 tx=0x7f080c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.815+0000 7fea93e00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 0x7fea7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.816+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649814c8e0 con 0x7f649815e2e0 2024-04-05T22:21:23.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.816+0000 7fea93e00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 0x7fea7406d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0012e0 tx=0x7fea8c004d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.816+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6490005470 con 0x7f649815e2e0 2024-04-05T22:21:23.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.816+0000 7f648fe00640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 0x7f647406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.817+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6490058610 con 0x7f649815e2e0 2024-04-05T22:21:23.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.817+0000 7f6497e00640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 0x7f647406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.818+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0032840 con 0x7f95d40d0bd0 2024-04-05T22:21:23.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.818+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0031f00 con 0x7f95d40d0bd0 2024-04-05T22:21:23.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.818+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0032be0 con 0x7f95d40d0bd0 2024-04-05T22:21:23.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.818+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fea80053af0 con 0x7fea940d10f0 2024-04-05T22:21:23.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.819+0000 7f6497e00640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 0x7f647406d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6480006a90 tx=0x7f6480006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.819+0000 7f95da800640 1 -- 172.21.15.73:0/3197842117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d415ecc0 con 0x7f95d40d0bd0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.819+0000 7f95da800640 1 -- 172.21.15.73:0/3197842117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d415f280 con 0x7f95d40d0bd0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.821+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8005250 con 0x7f95d40d0bd0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.822+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f95c0032d40 con 0x7f95d40d0bd0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.823+0000 7f95cb400640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 0x7f95b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.823+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95c00ab180 con 0x7f95d40d0bd0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.823+0000 7f95d2a00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 0x7f95b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.825+0000 7f95d2a00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 0x7f95b406d110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f95d41595e0 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.828+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f95c00763f0 con 0x7f95d40d0bd0 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.836+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000443f0 con 0x7f6b1415cf70 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.836+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b00044570 con 0x7f6b1415cf70 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.836+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00044920 con 0x7f6b1415cf70 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.836+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1406b980 con 0x7f6b1415cf70 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.836+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b14158c50 con 0x7f6b1415cf70 2024-04-05T22:21:23.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.837+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f08080bc650 con 0x7f081c065b10 2024-04-05T22:21:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.838+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b14068160 con 0x7f6b1415cf70 2024-04-05T22:21:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.842+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b0004d450 con 0x7f6b1415cf70 2024-04-05T22:21:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.843+0000 7f6b11600640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 0x7f6af406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.843+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b0002d040 con 0x7f6b1415cf70 2024-04-05T22:21:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.845+0000 7f6b19400640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 0x7f6af406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:23.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.845+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f64900549e0 con 0x7f649815e2e0 2024-04-05T22:21:23.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.846+0000 7f6b19400640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 0x7f6af406d140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003b30 tx=0x7f6b080098b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:23.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:23.856+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6b00077090 con 0x7f6b1415cf70 2024-04-05T22:21:24.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.167+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2150005500 con 0x7f217404fd80 2024-04-05T22:21:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.168+0000 7f2179600640 1 -- 172.21.15.73:0/3558719229 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2168082d40 con 0x7f217404fd80 2024-04-05T22:21:24.168 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.171+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 msgr2=0x7f215c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.171+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 0x7f215c06d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f217404eb60 tx=0x7f2170008040 comp rx=0 tx=0).stop 2024-04-05T22:21:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.172+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 msgr2=0x7f21740dabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.172+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f21680046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 shutdown_connections 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f215c06ac00 0x7f215c06d0c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21740e32a0 0x7f217404c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2174053160 0x7f21740d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 --2- 172.21.15.73:0/3558719229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f217404fd80 0x7f21740dabe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 >> 172.21.15.73:0/3558719229 conn(0x7f2174009f80 msgr2=0x7f21740dbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 shutdown_connections 2024-04-05T22:21:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.173+0000 7f216ea00640 1 -- 172.21.15.73:0/3558719229 wait complete. 2024-04-05T22:21:24.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-04-05T22:21:24.198 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.200+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe85806d470 con 0x7fe85815d2b0 2024-04-05T22:21:24.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.201+0000 7fe84f400640 1 -- 172.21.15.73:0/2554832561 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe848077590 con 0x7fe85815d2b0 2024-04-05T22:21:24.200 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.204+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 msgr2=0x7fe83806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.204+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 0x7fe83806d0f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8500012e0 tx=0x7fe850001650 comp rx=0 tx=0).stop 2024-04-05T22:21:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.204+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 msgr2=0x7fe85813b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.204+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8480087a0 tx=0x7fe848004730 comp rx=0 tx=0).stop 2024-04-05T22:21:24.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 shutdown_connections 2024-04-05T22:21:24.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe83806ac30 0x7fe83806d0f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe85815d2b0 0x7fe85813b270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe85813bc90 0x7fe8581eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 --2- 172.21.15.73:0/2554832561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8580b20d0 0x7fe858155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 >> 172.21.15.73:0/2554832561 conn(0x7fe8580b06c0 msgr2=0x7fe85813ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.205+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 shutdown_connections 2024-04-05T22:21:24.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.206+0000 7fe85d200640 1 -- 172.21.15.73:0/2554832561 wait complete. 2024-04-05T22:21:24.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.217+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fea94068160 con 0x7fea940d10f0 2024-04-05T22:21:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.218+0000 7fea8be00640 1 -- 172.21.15.73:0/3703092894 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea80057110 con 0x7fea940d10f0 2024-04-05T22:21:24.219 INFO:teuthology.orchestra.run.smithi073.stdout:60129542148 2024-04-05T22:21:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 msgr2=0x7fea7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 0x7fea7406d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0012e0 tx=0x7fea8c004d40 comp rx=0 tx=0).stop 2024-04-05T22:21:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 msgr2=0x7fea940cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c500 tx=0x7fea8000c9c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 shutdown_connections 2024-04-05T22:21:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fea7406acd0 0x7fea7406d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea9415e2e0 0x7fea9414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea940d10f0 0x7fea940cdeb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 --2- 172.21.15.73:0/3703092894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b1f20 0x7fea9406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 >> 172.21.15.73:0/3703092894 conn(0x7fea94099a60 msgr2=0x7fea9414a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.223+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 shutdown_connections 2024-04-05T22:21:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.224+0000 7fea9aa00640 1 -- 172.21.15.73:0/3703092894 wait complete. 2024-04-05T22:21:24.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.2 2024-04-05T22:21:24.226 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.233+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe2d81392b0 con 0x7fe2d8158560 2024-04-05T22:21:24.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.234+0000 7fe2d5600640 1 -- 172.21.15.73:0/3746938096 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2d81392b0 con 0x7fe2d8158560 2024-04-05T22:21:24.233 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:24.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.237+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 msgr2=0x7fe2b806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.237+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 0x7fe2b806d110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc000f60 tx=0x7fe2cc009040 comp rx=0 tx=0).stop 2024-04-05T22:21:24.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 msgr2=0x7fe2d814d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 0x7fe2d814d460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4033000 tx=0x7fe2c40046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 shutdown_connections 2024-04-05T22:21:24.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe2b806ac50 0x7fe2b806d110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d8158940 0x7fe2d814d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2d8158560 0x7fe2d814d460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 --2- 172.21.15.73:0/3746938096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d806d0c0 0x7fe2d81f9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 >> 172.21.15.73:0/3746938096 conn(0x7fe2d80ce8e0 msgr2=0x7fe2d80ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 shutdown_connections 2024-04-05T22:21:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.238+0000 7fe2e0600640 1 -- 172.21.15.73:0/3746938096 wait complete. 2024-04-05T22:21:24.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.240+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f081c0b5110 con 0x7f081c065b10 2024-04-05T22:21:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.240+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f95a8005500 con 0x7f95d40d0bd0 2024-04-05T22:21:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.241+0000 7f95cb400640 1 -- 172.21.15.73:0/3197842117 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95c0079a10 con 0x7f95d40d0bd0 2024-04-05T22:21:24.240 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:24.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.241+0000 7f0812000640 1 -- 172.21.15.73:0/1945080294 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08080542b0 con 0x7f081c065b10 2024-04-05T22:21:24.241 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:24.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.242+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f649806c2d0 con 0x7f649815e2e0 2024-04-05T22:21:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.244+0000 7f648fe00640 1 -- 172.21.15.73:0/3127471252 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f649806c2d0 con 0x7f649815e2e0 2024-04-05T22:21:24.243 INFO:teuthology.orchestra.run.smithi073.stdout:60129542149 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 msgr2=0x7f95b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 0x7f95b406d110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f95d41595e0 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 msgr2=0x7f95d41f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009890 tx=0x7f95c00046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 msgr2=0x7f07fc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 0x7f07fc06d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f081c0b4060 tx=0x7f080c009040 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 msgr2=0x7f081c0b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f080800c3f0 tx=0x7f080800c8b0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 shutdown_connections 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95b406ac50 0x7f95b406d110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95d415cf70 0x7f95d4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95d40d1930 0x7f95d40b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 --2- 172.21.15.73:0/3197842117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95d40d0bd0 0x7f95d41f6820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 >> 172.21.15.73:0/3197842117 conn(0x7f95d40d2df0 msgr2=0x7f95d40c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 shutdown_connections 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f07fc06ac50 0x7f07fc06d110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081c159f40 0x7f081c2024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c06b0a0 0x7f081c0c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.245+0000 7f0821000640 1 --2- 172.21.15.73:0/1945080294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c065b10 0x7f081c0b3b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.246+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 >> 172.21.15.73:0/1945080294 conn(0x7f081c0cf190 msgr2=0x7f081c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.246+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 shutdown_connections 2024-04-05T22:21:24.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.246+0000 7f95c8c00640 1 -- 172.21.15.73:0/3197842117 wait complete. 2024-04-05T22:21:24.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.246+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 shutdown_connections 2024-04-05T22:21:24.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.246+0000 7f0821000640 1 -- 172.21.15.73:0/1945080294 wait complete. 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 msgr2=0x7f647406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 0x7f647406d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6480006a90 tx=0x7f6480006a20 comp rx=0 tx=0).stop 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 msgr2=0x7f64980c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980c58b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f649000c690 tx=0x7f649000cb50 comp rx=0 tx=0).stop 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 shutdown_connections 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f647406ac00 0x7f647406d0c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f649815e2e0 0x7f64980c58b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64980d0f20 0x7f64980c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 --2- 172.21.15.73:0/3127471252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64980b1f20 0x7f6498147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 >> 172.21.15.73:0/3127471252 conn(0x7f6498099a60 msgr2=0x7f649806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 shutdown_connections 2024-04-05T22:21:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.247+0000 7f648d600640 1 -- 172.21.15.73:0/3127471252 wait complete. 2024-04-05T22:21:24.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-04-05T22:21:24.250 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-04-05T22:21:24.261 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-04-05T22:21:24.266 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2024-04-05T22:21:24.266 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-04-05T22:21:24.271 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.274+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6b1406b7e0 con 0x7f6b1415cf70 2024-04-05T22:21:24.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.275+0000 7f6b11600640 1 -- 172.21.15.73:0/2082496261 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b1406b7e0 con 0x7f6b1415cf70 2024-04-05T22:21:24.274 INFO:teuthology.orchestra.run.smithi073.stdout:60129542150 2024-04-05T22:21:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.277+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 msgr2=0x7f6af406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.277+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 0x7f6af406d140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003b30 tx=0x7f6b080098b0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.277+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 msgr2=0x7f6b14152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.277+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 0x7f6b14152c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b00033000 tx=0x7f6b000046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 shutdown_connections 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6af406ac80 0x7f6af406d140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1415d350 0x7f6b140b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1415cf70 0x7f6b14152c10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2082496261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b140d0bd0 0x7f6b140caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 >> 172.21.15.73:0/2082496261 conn(0x7f6b140d2df0 msgr2=0x7f6b140b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 shutdown_connections 2024-04-05T22:21:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.278+0000 7f6b1be00640 1 -- 172.21.15.73:0/2082496261 wait complete. 2024-04-05T22:21:24.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-04-05T22:21:24.293 DEBUG:teuthology.parallel:result is None 2024-04-05T22:21:24.293 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-05T22:21:24.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:21:24.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.530+0000 7f6be0c00640 1 Processor -- start 2024-04-05T22:21:24.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.530+0000 7f6be0c00640 1 -- start start 2024-04-05T22:21:24.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.531+0000 7f6be0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bdc158860 0x7f6bdc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6be0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc0b1f00 0x7f6bdc15e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6be0c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bdc15eb60 0x7f6bdc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6be0c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc15eff0 con 0x7f6bdc158860 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6be0c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc0ccaf0 con 0x7f6bdc15eb60 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6be0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06b500 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bdc158860 0x7f6bdc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bdc158860 0x7f6bdc0b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51332/0 (socket says 172.21.15.73:51332) 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 -- 172.21.15.73:0/1515722510 learned_addr learned my addr 172.21.15.73:0/1515722510 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 -- 172.21.15.73:0/1515722510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc0b1f00 msgr2=0x7f6bdc15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 --2- 172.21.15.73:0/1515722510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc0b1f00 0x7f6bdc15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 -- 172.21.15.73:0/1515722510 >> v2:172.21.15.157:3300/0 conn(0x7f6bdc15eb60 msgr2=0x7f6bdc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:24.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 --2- 172.21.15.73:0/1515722510 >> v2:172.21.15.157:3300/0 conn(0x7f6bdc15eb60 0x7f6bdc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.532+0000 7f6bdb400640 1 -- 172.21.15.73:0/1515722510 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0b23a0 con 0x7f6bdc158860 2024-04-05T22:21:24.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.533+0000 7f6bdb400640 1 --2- 172.21.15.73:0/1515722510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc158860 0x7f6bdc0b19c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0cf9d0 tx=0x7f6bcc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93085311c2ec9b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:24.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.533+0000 7f6bdaa00640 1 -- 172.21.15.73:0/1515722510 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0317e0 con 0x7f6bdc158860 2024-04-05T22:21:24.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6bdaa00640 1 -- 172.21.15.73:0/1515722510 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc031940 con 0x7f6bdc158860 2024-04-05T22:21:24.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6bdaa00640 1 -- 172.21.15.73:0/1515722510 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc031cd0 con 0x7f6bdc158860 2024-04-05T22:21:24.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 -- 172.21.15.73:0/1515722510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc158860 msgr2=0x7f6bdc0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 --2- 172.21.15.73:0/1515722510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc158860 0x7f6bdc0b19c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0cf9d0 tx=0x7f6bcc02b880 comp rx=0 tx=0).stop 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 -- 172.21.15.73:0/1515722510 shutdown_connections 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 --2- 172.21.15.73:0/1515722510 >> v2:172.21.15.157:3300/0 conn(0x7f6bdc15eb60 0x7f6bdc15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 --2- 172.21.15.73:0/1515722510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc0b1f00 0x7f6bdc15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 --2- 172.21.15.73:0/1515722510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc158860 0x7f6bdc0b19c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 -- 172.21.15.73:0/1515722510 >> 172.21.15.73:0/1515722510 conn(0x7f6bdc0b06c0 msgr2=0x7f6bdc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 -- 172.21.15.73:0/1515722510 shutdown_connections 2024-04-05T22:21:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.534+0000 7f6be0c00640 1 -- 172.21.15.73:0/1515722510 wait complete. 2024-04-05T22:21:24.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.536+0000 7f6be0c00640 1 Processor -- start 2024-04-05T22:21:24.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.536+0000 7f6be0c00640 1 -- start start 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bdc15eb60 0x7f6bdc15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc155250 0x7f6bdc147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06f920 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc155a10 con 0x7f6bdc15eb60 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6be0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc155730 con 0x7f6bdc155250 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51344/0 (socket says 172.21.15.73:51344) 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 -- 172.21.15.73:0/413894497 learned_addr learned my addr 172.21.15.73:0/413894497 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 -- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc155250 msgr2=0x7f6bdc147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc155250 0x7f6bdc147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 -- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bdc15eb60 msgr2=0x7f6bdc15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bdc15eb60 0x7f6bdc15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.537+0000 7f6bdb400640 1 -- 172.21.15.73:0/413894497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0094d0 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6bdb400640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc02b8c0 tx=0x7f6bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc032940 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc032aa0 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc06fbc0 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc032e50 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.538+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc154e80 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.539+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc06f780 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.540+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6bcc04e450 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.541+0000 7f6bd3e00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 0x7f6bb806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.541+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bcc0ab670 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.541+0000 7f6bd2a00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 0x7f6bb806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:24.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.542+0000 7f6bd2a00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 0x7f6bb806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc1539b0 tx=0x7f6bc4006c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.545+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6bdc06f780 con 0x7f6bdc0b1f00 2024-04-05T22:21:24.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.721+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bdc06b750 con 0x7f6bb806ac00 2024-04-05T22:21:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.734+0000 7f6bd3e00640 1 -- 172.21.15.73:0/413894497 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264477 (secure 0 0 0) 0x7f6bc4013400 con 0x7f6bb806ac00 2024-04-05T22:21:24.737 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 msgr2=0x7f6bb806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 0x7f6bb806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc1539b0 tx=0x7f6bc4006c80 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 msgr2=0x7f6bdc154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc02b8c0 tx=0x7f6bcc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 shutdown_connections 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6bb806ac00 0x7f6bb806d0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bdc155250 0x7f6bdc147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bdc15eb60 0x7f6bdc15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 --2- 172.21.15.73:0/413894497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bdc0b1f00 0x7f6bdc154940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 >> 172.21.15.73:0/413894497 conn(0x7f6bdc0b06c0 msgr2=0x7f6bdc15e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 shutdown_connections 2024-04-05T22:21:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.740+0000 7f6be0c00640 1 -- 172.21.15.73:0/413894497 wait complete. 2024-04-05T22:21:24.756 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-05T22:21:24.491082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":3055616,"data_stored":1816372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.599141"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982064+0000","last_change":"2024-04-05T22:21:14.923383+0000","last_active":"2024-04-05T22:21:15.982064+0000","last_peered":"2024-04-05T22:21:15.982064+0000","last_clean":"2024-04-05T22:21:15.982064+0000","last_became_active":"2024-04-05T22:21:14.922950+0000","last_became_peered":"2024-04-05T22:21:14.922950+0000","last_unstale":"2024-04-05T22:21:15.982064+0000","last_undegraded":"2024-04-05T22:21:15.982064+0000","last_fullsized":"2024-04-05T22:21:15.982064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900565+0000","last_change":"2024-04-05T22:21:14.912375+0000","last_active":"2024-04-05T22:21:15.900565+0000","last_peered":"2024-04-05T22:21:15.900565+0000","last_clean":"2024-04-05T22:21:15.900565+0000","last_became_active":"2024-04-05T22:21:14.912009+0000","last_became_peered":"2024-04-05T22:21:14.912009+0000","last_unstale":"2024-04-05T22:21:15.900565+0000","last_undegraded":"2024-04-05T22:21:15.900565+0000","last_fullsized":"2024-04-05T22:21:15.900565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976623+0000","last_change":"2024-04-05T22:21:14.914402+0000","last_active":"2024-04-05T22:21:15.976623+0000","last_peered":"2024-04-05T22:21:15.976623+0000","last_clean":"2024-04-05T22:21:15.976623+0000","last_became_active":"2024-04-05T22:21:14.914120+0000","last_became_peered":"2024-04-05T22:21:14.914120+0000","last_unstale":"2024-04-05T22:21:15.976623+0000","last_undegraded":"2024-04-05T22:21:15.976623+0000","last_fullsized":"2024-04-05T22:21:15.976623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899962+0000","last_change":"2024-04-05T22:21:14.921662+0000","last_active":"2024-04-05T22:21:15.899962+0000","last_peered":"2024-04-05T22:21:15.899962+0000","last_clean":"2024-04-05T22:21:15.899962+0000","last_became_active":"2024-04-05T22:21:14.921218+0000","last_became_peered":"2024-04-05T22:21:14.921218+0000","last_unstale":"2024-04-05T22:21:15.899962+0000","last_undegraded":"2024-04-05T22:21:15.899962+0000","last_fullsized":"2024-04-05T22:21:15.899962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979973+0000","last_change":"2024-04-05T22:21:14.935767+0000","last_active":"2024-04-05T22:21:15.979973+0000","last_peered":"2024-04-05T22:21:15.979973+0000","last_clean":"2024-04-05T22:21:15.979973+0000","last_became_active":"2024-04-05T22:21:14.935145+0000","last_became_peered":"2024-04-05T22:21:14.935145+0000","last_unstale":"2024-04-05T22:21:15.979973+0000","last_undegraded":"2024-04-05T22:21:15.979973+0000","last_fullsized":"2024-04-05T22:21:15.979973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898816+0000","last_change":"2024-04-05T22:21:14.923031+0000","last_active":"2024-04-05T22:21:15.898816+0000","last_peered":"2024-04-05T22:21:15.898816+0000","last_clean":"2024-04-05T22:21:15.898816+0000","last_became_active":"2024-04-05T22:21:14.922585+0000","last_became_peered":"2024-04-05T22:21:14.922585+0000","last_unstale":"2024-04-05T22:21:15.898816+0000","last_undegraded":"2024-04-05T22:21:15.898816+0000","last_fullsized":"2024-04-05T22:21:15.898816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979244+0000","last_change":"2024-04-05T22:21:14.925099+0000","last_active":"2024-04-05T22:21:15.979244+0000","last_peered":"2024-04-05T22:21:15.979244+0000","last_clean":"2024-04-05T22:21:15.979244+0000","last_became_active":"2024-04-05T22:21:14.921678+0000","last_became_peered":"2024-04-05T22:21:14.921678+0000","last_unstale":"2024-04-05T22:21:15.979244+0000","last_undegraded":"2024-04-05T22:21:15.979244+0000","last_fullsized":"2024-04-05T22:21:15.979244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975561+0000","last_change":"2024-04-05T22:21:14.916195+0000","last_active":"2024-04-05T22:21:15.975561+0000","last_peered":"2024-04-05T22:21:15.975561+0000","last_clean":"2024-04-05T22:21:15.975561+0000","last_became_active":"2024-04-05T22:21:14.915947+0000","last_became_peered":"2024-04-05T22:21:14.915947+0000","last_unstale":"2024-04-05T22:21:15.975561+0000","last_undegraded":"2024-04-05T22:21:15.975561+0000","last_fullsized":"2024-04-05T22:21:15.975561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982147+0000","last_change":"2024-04-05T22:21:14.931207+0000","last_active":"2024-04-05T22:21:15.982147+0000","last_peered":"2024-04-05T22:21:15.982147+0000","last_clean":"2024-04-05T22:21:15.982147+0000","last_became_active":"2024-04-05T22:21:14.930509+0000","last_became_peered":"2024-04-05T22:21:14.930509+0000","last_unstale":"2024-04-05T22:21:15.982147+0000","last_undegraded":"2024-04-05T22:21:15.982147+0000","last_fullsized":"2024-04-05T22:21:15.982147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981506+0000","last_change":"2024-04-05T22:21:14.916586+0000","last_active":"2024-04-05T22:21:15.981506+0000","last_peered":"2024-04-05T22:21:15.981506+0000","last_clean":"2024-04-05T22:21:15.981506+0000","last_became_active":"2024-04-05T22:21:14.916004+0000","last_became_peered":"2024-04-05T22:21:14.916004+0000","last_unstale":"2024-04-05T22:21:15.981506+0000","last_undegraded":"2024-04-05T22:21:15.981506+0000","last_fullsized":"2024-04-05T22:21:15.981506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:02.978295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981439+0000","last_change":"2024-04-05T22:21:14.913056+0000","last_active":"2024-04-05T22:21:15.981439+0000","last_peered":"2024-04-05T22:21:15.981439+0000","last_clean":"2024-04-05T22:21:15.981439+0000","last_became_active":"2024-04-05T22:21:14.912748+0000","last_became_peered":"2024-04-05T22:21:14.912748+0000","last_unstale":"2024-04-05T22:21:15.981439+0000","last_undegraded":"2024-04-05T22:21:15.981439+0000","last_fullsized":"2024-04-05T22:21:15.981439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:11:59.394291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983331+0000","last_change":"2024-04-05T22:21:14.926927+0000","last_active":"2024-04-05T22:21:15.983331+0000","last_peered":"2024-04-05T22:21:15.983331+0000","last_clean":"2024-04-05T22:21:15.983331+0000","last_became_active":"2024-04-05T22:21:14.913316+0000","last_became_peered":"2024-04-05T22:21:14.913316+0000","last_unstale":"2024-04-05T22:21:15.983331+0000","last_undegraded":"2024-04-05T22:21:15.983331+0000","last_fullsized":"2024-04-05T22:21:15.983331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:14.865832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976116+0000","last_change":"2024-04-05T22:21:14.919462+0000","last_active":"2024-04-05T22:21:15.976116+0000","last_peered":"2024-04-05T22:21:15.976116+0000","last_clean":"2024-04-05T22:21:15.976116+0000","last_became_active":"2024-04-05T22:21:14.919180+0000","last_became_peered":"2024-04-05T22:21:14.919180+0000","last_unstale":"2024-04-05T22:21:15.976116+0000","last_undegraded":"2024-04-05T22:21:15.976116+0000","last_fullsized":"2024-04-05T22:21:15.976116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975694+0000","last_change":"2024-04-05T22:21:14.912391+0000","last_active":"2024-04-05T22:21:15.975694+0000","last_peered":"2024-04-05T22:21:15.975694+0000","last_clean":"2024-04-05T22:21:15.975694+0000","last_became_active":"2024-04-05T22:21:14.912013+0000","last_became_peered":"2024-04-05T22:21:14.912013+0000","last_unstale":"2024-04-05T22:21:15.975694+0000","last_undegraded":"2024-04-05T22:21:15.975694+0000","last_fullsized":"2024-04-05T22:21:15.975694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899930+0000","last_change":"2024-04-05T22:21:14.900459+0000","last_active":"2024-04-05T22:21:15.899930+0000","last_peered":"2024-04-05T22:21:15.899930+0000","last_clean":"2024-04-05T22:21:15.899930+0000","last_became_active":"2024-04-05T22:21:14.900144+0000","last_became_peered":"2024-04-05T22:21:14.900144+0000","last_unstale":"2024-04-05T22:21:15.899930+0000","last_undegraded":"2024-04-05T22:21:15.899930+0000","last_fullsized":"2024-04-05T22:21:15.899930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898646+0000","last_change":"2024-04-05T22:21:14.913235+0000","last_active":"2024-04-05T22:21:15.898646+0000","last_peered":"2024-04-05T22:21:15.898646+0000","last_clean":"2024-04-05T22:21:15.898646+0000","last_became_active":"2024-04-05T22:21:14.912913+0000","last_became_peered":"2024-04-05T22:21:14.912913+0000","last_unstale":"2024-04-05T22:21:15.898646+0000","last_undegraded":"2024-04-05T22:21:15.898646+0000","last_fullsized":"2024-04-05T22:21:15.898646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900293+0000","last_change":"2024-04-05T22:21:14.931728+0000","last_active":"2024-04-05T22:21:15.900293+0000","last_peered":"2024-04-05T22:21:15.900293+0000","last_clean":"2024-04-05T22:21:15.900293+0000","last_became_active":"2024-04-05T22:21:14.931500+0000","last_became_peered":"2024-04-05T22:21:14.931500+0000","last_unstale":"2024-04-05T22:21:15.900293+0000","last_undegraded":"2024-04-05T22:21:15.900293+0000","last_fullsized":"2024-04-05T22:21:15.900293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897627+0000","last_change":"2024-04-05T22:21:14.921738+0000","last_active":"2024-04-05T22:21:15.897627+0000","last_peered":"2024-04-05T22:21:15.897627+0000","last_clean":"2024-04-05T22:21:15.897627+0000","last_became_active":"2024-04-05T22:21:14.919781+0000","last_became_peered":"2024-04-05T22:21:14.919781+0000","last_unstale":"2024-04-05T22:21:15.897627+0000","last_undegraded":"2024-04-05T22:21:15.897627+0000","last_fullsized":"2024-04-05T22:21:15.897627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981513+0000","last_change":"2024-04-05T22:21:14.920265+0000","last_active":"2024-04-05T22:21:15.981513+0000","last_peered":"2024-04-05T22:21:15.981513+0000","last_clean":"2024-04-05T22:21:15.981513+0000","last_became_active":"2024-04-05T22:21:14.919985+0000","last_became_peered":"2024-04-05T22:21:14.919985+0000","last_unstale":"2024-04-05T22:21:15.981513+0000","last_undegraded":"2024-04-05T22:21:15.981513+0000","last_fullsized":"2024-04-05T22:21:15.981513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981857+0000","last_change":"2024-04-05T22:21:14.929527+0000","last_active":"2024-04-05T22:21:15.981857+0000","last_peered":"2024-04-05T22:21:15.981857+0000","last_clean":"2024-04-05T22:21:15.981857+0000","last_became_active":"2024-04-05T22:21:14.929222+0000","last_became_peered":"2024-04-05T22:21:14.929222+0000","last_unstale":"2024-04-05T22:21:15.981857+0000","last_undegraded":"2024-04-05T22:21:15.981857+0000","last_fullsized":"2024-04-05T22:21:15.981857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:15:04.491398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975985+0000","last_change":"2024-04-05T22:21:14.933523+0000","last_active":"2024-04-05T22:21:15.975985+0000","last_peered":"2024-04-05T22:21:15.975985+0000","last_clean":"2024-04-05T22:21:15.975985+0000","last_became_active":"2024-04-05T22:21:14.933340+0000","last_became_peered":"2024-04-05T22:21:14.933340+0000","last_unstale":"2024-04-05T22:21:15.975985+0000","last_undegraded":"2024-04-05T22:21:15.975985+0000","last_fullsized":"2024-04-05T22:21:15.975985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898875+0000","last_change":"2024-04-05T22:21:14.933905+0000","last_active":"2024-04-05T22:21:15.898875+0000","last_peered":"2024-04-05T22:21:15.898875+0000","last_clean":"2024-04-05T22:21:15.898875+0000","last_became_active":"2024-04-05T22:21:14.933507+0000","last_became_peered":"2024-04-05T22:21:14.933507+0000","last_unstale":"2024-04-05T22:21:15.898875+0000","last_undegraded":"2024-04-05T22:21:15.898875+0000","last_fullsized":"2024-04-05T22:21:15.898875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975831+0000","last_change":"2024-04-05T22:21:14.915842+0000","last_active":"2024-04-05T22:21:15.975831+0000","last_peered":"2024-04-05T22:21:15.975831+0000","last_clean":"2024-04-05T22:21:15.975831+0000","last_became_active":"2024-04-05T22:21:14.915391+0000","last_became_peered":"2024-04-05T22:21:14.915391+0000","last_unstale":"2024-04-05T22:21:15.975831+0000","last_undegraded":"2024-04-05T22:21:15.975831+0000","last_fullsized":"2024-04-05T22:21:15.975831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:21.773200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898993+0000","last_change":"2024-04-05T22:21:14.924789+0000","last_active":"2024-04-05T22:21:15.898993+0000","last_peered":"2024-04-05T22:21:15.898993+0000","last_clean":"2024-04-05T22:21:15.898993+0000","last_became_active":"2024-04-05T22:21:14.924488+0000","last_became_peered":"2024-04-05T22:21:14.924488+0000","last_unstale":"2024-04-05T22:21:15.898993+0000","last_undegraded":"2024-04-05T22:21:15.898993+0000","last_fullsized":"2024-04-05T22:21:15.898993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976451+0000","last_change":"2024-04-05T22:21:14.926693+0000","last_active":"2024-04-05T22:21:15.976451+0000","last_peered":"2024-04-05T22:21:15.976451+0000","last_clean":"2024-04-05T22:21:15.976451+0000","last_became_active":"2024-04-05T22:21:14.926415+0000","last_became_peered":"2024-04-05T22:21:14.926415+0000","last_unstale":"2024-04-05T22:21:15.976451+0000","last_undegraded":"2024-04-05T22:21:15.976451+0000","last_fullsized":"2024-04-05T22:21:15.976451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981744+0000","last_change":"2024-04-05T22:21:14.940178+0000","last_active":"2024-04-05T22:21:15.981744+0000","last_peered":"2024-04-05T22:21:15.981744+0000","last_clean":"2024-04-05T22:21:15.981744+0000","last_became_active":"2024-04-05T22:21:14.938042+0000","last_became_peered":"2024-04-05T22:21:14.938042+0000","last_unstale":"2024-04-05T22:21:15.981744+0000","last_undegraded":"2024-04-05T22:21:15.981744+0000","last_fullsized":"2024-04-05T22:21:15.981744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:42.750931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900215+0000","last_change":"2024-04-05T22:21:14.916877+0000","last_active":"2024-04-05T22:21:15.900215+0000","last_peered":"2024-04-05T22:21:15.900215+0000","last_clean":"2024-04-05T22:21:15.900215+0000","last_became_active":"2024-04-05T22:21:14.916585+0000","last_became_peered":"2024-04-05T22:21:14.916585+0000","last_unstale":"2024-04-05T22:21:15.900215+0000","last_undegraded":"2024-04-05T22:21:15.900215+0000","last_fullsized":"2024-04-05T22:21:15.900215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976229+0000","last_change":"2024-04-05T22:21:14.918894+0000","last_active":"2024-04-05T22:21:15.976229+0000","last_peered":"2024-04-05T22:21:15.976229+0000","last_clean":"2024-04-05T22:21:15.976229+0000","last_became_active":"2024-04-05T22:21:14.918074+0000","last_became_peered":"2024-04-05T22:21:14.918074+0000","last_unstale":"2024-04-05T22:21:15.976229+0000","last_undegraded":"2024-04-05T22:21:15.976229+0000","last_fullsized":"2024-04-05T22:21:15.976229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976071+0000","last_change":"2024-04-05T22:21:14.922850+0000","last_active":"2024-04-05T22:21:15.976071+0000","last_peered":"2024-04-05T22:21:15.976071+0000","last_clean":"2024-04-05T22:21:15.976071+0000","last_became_active":"2024-04-05T22:21:14.920770+0000","last_became_peered":"2024-04-05T22:21:14.920770+0000","last_unstale":"2024-04-05T22:21:15.976071+0000","last_undegraded":"2024-04-05T22:21:15.976071+0000","last_fullsized":"2024-04-05T22:21:15.976071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899082+0000","last_change":"2024-04-05T22:21:14.917603+0000","last_active":"2024-04-05T22:21:15.899082+0000","last_peered":"2024-04-05T22:21:15.899082+0000","last_clean":"2024-04-05T22:21:15.899082+0000","last_became_active":"2024-04-05T22:21:14.917325+0000","last_became_peered":"2024-04-05T22:21:14.917325+0000","last_unstale":"2024-04-05T22:21:15.899082+0000","last_undegraded":"2024-04-05T22:21:15.899082+0000","last_fullsized":"2024-04-05T22:21:15.899082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:46.139845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.896840+0000","last_change":"2024-04-05T22:21:14.911372+0000","last_active":"2024-04-05T22:21:15.896840+0000","last_peered":"2024-04-05T22:21:15.896840+0000","last_clean":"2024-04-05T22:21:15.896840+0000","last_became_active":"2024-04-05T22:21:14.911083+0000","last_became_peered":"2024-04-05T22:21:14.911083+0000","last_unstale":"2024-04-05T22:21:15.896840+0000","last_undegraded":"2024-04-05T22:21:15.896840+0000","last_fullsized":"2024-04-05T22:21:15.896840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979172+0000","last_change":"2024-04-05T22:21:14.915897+0000","last_active":"2024-04-05T22:21:15.979172+0000","last_peered":"2024-04-05T22:21:15.979172+0000","last_clean":"2024-04-05T22:21:15.979172+0000","last_became_active":"2024-04-05T22:21:14.915223+0000","last_became_peered":"2024-04-05T22:21:14.915223+0000","last_unstale":"2024-04-05T22:21:15.979172+0000","last_undegraded":"2024-04-05T22:21:15.979172+0000","last_fullsized":"2024-04-05T22:21:15.979172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:19:20.576451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982664+0000","last_change":"2024-04-05T22:21:14.901329+0000","last_active":"2024-04-05T22:21:15.982664+0000","last_peered":"2024-04-05T22:21:15.982664+0000","last_clean":"2024-04-05T22:21:15.982664+0000","last_became_active":"2024-04-05T22:21:14.900870+0000","last_became_peered":"2024-04-05T22:21:14.900870+0000","last_unstale":"2024-04-05T22:21:15.982664+0000","last_undegraded":"2024-04-05T22:21:15.982664+0000","last_fullsized":"2024-04-05T22:21:15.982664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:12.554588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976297+0000","last_change":"2024-04-05T22:21:14.437934+0000","last_active":"2024-04-05T22:21:15.976297+0000","last_peered":"2024-04-05T22:21:15.976297+0000","last_clean":"2024-04-05T22:21:15.976297+0000","last_became_active":"2024-04-05T22:21:14.437696+0000","last_became_peered":"2024-04-05T22:21:14.437696+0000","last_unstale":"2024-04-05T22:21:15.976297+0000","last_undegraded":"2024-04-05T22:21:15.976297+0000","last_fullsized":"2024-04-05T22:21:15.976297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976230+0000","last_change":"2024-04-05T22:21:14.920473+0000","last_active":"2024-04-05T22:21:15.976230+0000","last_peered":"2024-04-05T22:21:15.976230+0000","last_clean":"2024-04-05T22:21:15.976230+0000","last_became_active":"2024-04-05T22:21:14.917453+0000","last_became_peered":"2024-04-05T22:21:14.917453+0000","last_unstale":"2024-04-05T22:21:15.976230+0000","last_undegraded":"2024-04-05T22:21:15.976230+0000","last_fullsized":"2024-04-05T22:21:15.976230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980837+0000","last_change":"2024-04-05T22:21:14.449143+0000","last_active":"2024-04-05T22:21:15.980837+0000","last_peered":"2024-04-05T22:21:15.980837+0000","last_clean":"2024-04-05T22:21:15.980837+0000","last_became_active":"2024-04-05T22:21:14.448341+0000","last_became_peered":"2024-04-05T22:21:14.448341+0000","last_unstale":"2024-04-05T22:21:15.980837+0000","last_undegraded":"2024-04-05T22:21:15.980837+0000","last_fullsized":"2024-04-05T22:21:15.980837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:14.544598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975917+0000","last_change":"2024-04-05T22:21:14.441239+0000","last_active":"2024-04-05T22:21:15.975917+0000","last_peered":"2024-04-05T22:21:15.975917+0000","last_clean":"2024-04-05T22:21:15.975917+0000","last_became_active":"2024-04-05T22:21:14.440898+0000","last_became_peered":"2024-04-05T22:21:14.440898+0000","last_unstale":"2024-04-05T22:21:15.975917+0000","last_undegraded":"2024-04-05T22:21:15.975917+0000","last_fullsized":"2024-04-05T22:21:15.975917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975636+0000","last_change":"2024-04-05T22:21:14.922371+0000","last_active":"2024-04-05T22:21:15.975636+0000","last_peered":"2024-04-05T22:21:15.975636+0000","last_clean":"2024-04-05T22:21:15.975636+0000","last_became_active":"2024-04-05T22:21:14.922074+0000","last_became_peered":"2024-04-05T22:21:14.922074+0000","last_unstale":"2024-04-05T22:21:15.975636+0000","last_undegraded":"2024-04-05T22:21:15.975636+0000","last_fullsized":"2024-04-05T22:21:15.975636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898990+0000","last_change":"2024-04-05T22:21:14.446427+0000","last_active":"2024-04-05T22:21:15.898990+0000","last_peered":"2024-04-05T22:21:15.898990+0000","last_clean":"2024-04-05T22:21:15.898990+0000","last_became_active":"2024-04-05T22:21:14.445487+0000","last_became_peered":"2024-04-05T22:21:14.445487+0000","last_unstale":"2024-04-05T22:21:15.898990+0000","last_undegraded":"2024-04-05T22:21:15.898990+0000","last_fullsized":"2024-04-05T22:21:15.898990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899850+0000","last_change":"2024-04-05T22:21:14.917346+0000","last_active":"2024-04-05T22:21:15.899850+0000","last_peered":"2024-04-05T22:21:15.899850+0000","last_clean":"2024-04-05T22:21:15.899850+0000","last_became_active":"2024-04-05T22:21:14.917085+0000","last_became_peered":"2024-04-05T22:21:14.917085+0000","last_unstale":"2024-04-05T22:21:15.899850+0000","last_undegraded":"2024-04-05T22:21:15.899850+0000","last_fullsized":"2024-04-05T22:21:15.899850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981190+0000","last_change":"2024-04-05T22:21:14.441415+0000","last_active":"2024-04-05T22:21:15.981190+0000","last_peered":"2024-04-05T22:21:15.981190+0000","last_clean":"2024-04-05T22:21:15.981190+0000","last_became_active":"2024-04-05T22:21:14.440493+0000","last_became_peered":"2024-04-05T22:21:14.440493+0000","last_unstale":"2024-04-05T22:21:15.981190+0000","last_undegraded":"2024-04-05T22:21:15.981190+0000","last_fullsized":"2024-04-05T22:21:15.981190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:25.845173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899687+0000","last_change":"2024-04-05T22:21:14.932959+0000","last_active":"2024-04-05T22:21:15.899687+0000","last_peered":"2024-04-05T22:21:15.899687+0000","last_clean":"2024-04-05T22:21:15.899687+0000","last_became_active":"2024-04-05T22:21:14.932656+0000","last_became_peered":"2024-04-05T22:21:14.932656+0000","last_unstale":"2024-04-05T22:21:15.899687+0000","last_undegraded":"2024-04-05T22:21:15.899687+0000","last_fullsized":"2024-04-05T22:21:15.899687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899902+0000","last_change":"2024-04-05T22:21:14.444991+0000","last_active":"2024-04-05T22:21:15.899902+0000","last_peered":"2024-04-05T22:21:15.899902+0000","last_clean":"2024-04-05T22:21:15.899902+0000","last_became_active":"2024-04-05T22:21:14.444667+0000","last_became_peered":"2024-04-05T22:21:14.444667+0000","last_unstale":"2024-04-05T22:21:15.899902+0000","last_undegraded":"2024-04-05T22:21:15.899902+0000","last_fullsized":"2024-04-05T22:21:15.899902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.915276+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.914411+0000","last_became_peered":"2024-04-05T22:21:14.914411+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900261+0000","last_change":"2024-04-05T22:21:14.438342+0000","last_active":"2024-04-05T22:21:15.900261+0000","last_peered":"2024-04-05T22:21:15.900261+0000","last_clean":"2024-04-05T22:21:15.900261+0000","last_became_active":"2024-04-05T22:21:14.438070+0000","last_became_peered":"2024-04-05T22:21:14.438070+0000","last_unstale":"2024-04-05T22:21:15.900261+0000","last_undegraded":"2024-04-05T22:21:15.900261+0000","last_fullsized":"2024-04-05T22:21:15.900261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:13.607794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983525+0000","last_change":"2024-04-05T22:21:14.928911+0000","last_active":"2024-04-05T22:21:15.983525+0000","last_peered":"2024-04-05T22:21:15.983525+0000","last_clean":"2024-04-05T22:21:15.983525+0000","last_became_active":"2024-04-05T22:21:14.928657+0000","last_became_peered":"2024-04-05T22:21:14.928657+0000","last_unstale":"2024-04-05T22:21:15.983525+0000","last_undegraded":"2024-04-05T22:21:15.983525+0000","last_fullsized":"2024-04-05T22:21:15.983525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982110+0000","last_change":"2024-04-05T22:21:14.442377+0000","last_active":"2024-04-05T22:21:15.982110+0000","last_peered":"2024-04-05T22:21:15.982110+0000","last_clean":"2024-04-05T22:21:15.982110+0000","last_became_active":"2024-04-05T22:21:14.441914+0000","last_became_peered":"2024-04-05T22:21:14.441914+0000","last_unstale":"2024-04-05T22:21:15.982110+0000","last_undegraded":"2024-04-05T22:21:15.982110+0000","last_fullsized":"2024-04-05T22:21:15.982110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:53.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980822+0000","last_change":"2024-04-05T22:21:14.919936+0000","last_active":"2024-04-05T22:21:15.980822+0000","last_peered":"2024-04-05T22:21:15.980822+0000","last_clean":"2024-04-05T22:21:15.980822+0000","last_became_active":"2024-04-05T22:21:14.919602+0000","last_became_peered":"2024-04-05T22:21:14.919602+0000","last_unstale":"2024-04-05T22:21:15.980822+0000","last_undegraded":"2024-04-05T22:21:15.980822+0000","last_fullsized":"2024-04-05T22:21:15.980822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976328+0000","last_change":"2024-04-05T22:21:14.437206+0000","last_active":"2024-04-05T22:21:15.976328+0000","last_peered":"2024-04-05T22:21:15.976328+0000","last_clean":"2024-04-05T22:21:15.976328+0000","last_became_active":"2024-04-05T22:21:14.436852+0000","last_became_peered":"2024-04-05T22:21:14.436852+0000","last_unstale":"2024-04-05T22:21:15.976328+0000","last_undegraded":"2024-04-05T22:21:15.976328+0000","last_fullsized":"2024-04-05T22:21:15.976328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975906+0000","last_change":"2024-04-05T22:21:14.933572+0000","last_active":"2024-04-05T22:21:15.975906+0000","last_peered":"2024-04-05T22:21:15.975906+0000","last_clean":"2024-04-05T22:21:15.975906+0000","last_became_active":"2024-04-05T22:21:14.933310+0000","last_became_peered":"2024-04-05T22:21:14.933310+0000","last_unstale":"2024-04-05T22:21:15.975906+0000","last_undegraded":"2024-04-05T22:21:15.975906+0000","last_fullsized":"2024-04-05T22:21:15.975906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:10:27.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899122+0000","last_change":"2024-04-05T22:21:14.435387+0000","last_active":"2024-04-05T22:21:15.899122+0000","last_peered":"2024-04-05T22:21:15.899122+0000","last_clean":"2024-04-05T22:21:15.899122+0000","last_became_active":"2024-04-05T22:21:14.435061+0000","last_became_peered":"2024-04-05T22:21:14.435061+0000","last_unstale":"2024-04-05T22:21:15.899122+0000","last_undegraded":"2024-04-05T22:21:15.899122+0000","last_fullsized":"2024-04-05T22:21:15.899122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980213+0000","last_change":"2024-04-05T22:21:14.935722+0000","last_active":"2024-04-05T22:21:15.980213+0000","last_peered":"2024-04-05T22:21:15.980213+0000","last_clean":"2024-04-05T22:21:15.980213+0000","last_became_active":"2024-04-05T22:21:14.935463+0000","last_became_peered":"2024-04-05T22:21:14.935463+0000","last_unstale":"2024-04-05T22:21:15.980213+0000","last_undegraded":"2024-04-05T22:21:15.980213+0000","last_fullsized":"2024-04-05T22:21:15.980213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980372+0000","last_change":"2024-04-05T22:21:14.449636+0000","last_active":"2024-04-05T22:21:15.980372+0000","last_peered":"2024-04-05T22:21:15.980372+0000","last_clean":"2024-04-05T22:21:15.980372+0000","last_became_active":"2024-04-05T22:21:14.449257+0000","last_became_peered":"2024-04-05T22:21:14.449257+0000","last_unstale":"2024-04-05T22:21:15.980372+0000","last_undegraded":"2024-04-05T22:21:15.980372+0000","last_fullsized":"2024-04-05T22:21:15.980372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:56:39.430553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980094+0000","last_change":"2024-04-05T22:21:14.916105+0000","last_active":"2024-04-05T22:21:15.980094+0000","last_peered":"2024-04-05T22:21:15.980094+0000","last_clean":"2024-04-05T22:21:15.980094+0000","last_became_active":"2024-04-05T22:21:14.915590+0000","last_became_peered":"2024-04-05T22:21:14.915590+0000","last_unstale":"2024-04-05T22:21:15.980094+0000","last_undegraded":"2024-04-05T22:21:15.980094+0000","last_fullsized":"2024-04-05T22:21:15.980094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:21:38.340157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897721+0000","last_change":"2024-04-05T22:21:14.429654+0000","last_active":"2024-04-05T22:21:15.897721+0000","last_peered":"2024-04-05T22:21:15.897721+0000","last_clean":"2024-04-05T22:21:15.897721+0000","last_became_active":"2024-04-05T22:21:14.428871+0000","last_became_peered":"2024-04-05T22:21:14.428871+0000","last_unstale":"2024-04-05T22:21:15.897721+0000","last_undegraded":"2024-04-05T22:21:15.897721+0000","last_fullsized":"2024-04-05T22:21:15.897721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979204+0000","last_change":"2024-04-05T22:21:14.921522+0000","last_active":"2024-04-05T22:21:15.979204+0000","last_peered":"2024-04-05T22:21:15.979204+0000","last_clean":"2024-04-05T22:21:15.979204+0000","last_became_active":"2024-04-05T22:21:14.921331+0000","last_became_peered":"2024-04-05T22:21:14.921331+0000","last_unstale":"2024-04-05T22:21:15.979204+0000","last_undegraded":"2024-04-05T22:21:15.979204+0000","last_fullsized":"2024-04-05T22:21:15.979204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983583+0000","last_change":"2024-04-05T22:21:14.437535+0000","last_active":"2024-04-05T22:21:15.983583+0000","last_peered":"2024-04-05T22:21:15.983583+0000","last_clean":"2024-04-05T22:21:15.983583+0000","last_became_active":"2024-04-05T22:21:14.437228+0000","last_became_peered":"2024-04-05T22:21:14.437228+0000","last_unstale":"2024-04-05T22:21:15.983583+0000","last_undegraded":"2024-04-05T22:21:15.983583+0000","last_fullsized":"2024-04-05T22:21:15.983583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981661+0000","last_change":"2024-04-05T22:21:14.935527+0000","last_active":"2024-04-05T22:21:15.981661+0000","last_peered":"2024-04-05T22:21:15.981661+0000","last_clean":"2024-04-05T22:21:15.981661+0000","last_became_active":"2024-04-05T22:21:14.935121+0000","last_became_peered":"2024-04-05T22:21:14.935121+0000","last_unstale":"2024-04-05T22:21:15.981661+0000","last_undegraded":"2024-04-05T22:21:15.981661+0000","last_fullsized":"2024-04-05T22:21:15.981661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897618+0000","last_change":"2024-04-05T22:21:14.434360+0000","last_active":"2024-04-05T22:21:15.897618+0000","last_peered":"2024-04-05T22:21:15.897618+0000","last_clean":"2024-04-05T22:21:15.897618+0000","last_became_active":"2024-04-05T22:21:14.434064+0000","last_became_peered":"2024-04-05T22:21:14.434064+0000","last_unstale":"2024-04-05T22:21:15.897618+0000","last_undegraded":"2024-04-05T22:21:15.897618+0000","last_fullsized":"2024-04-05T22:21:15.897618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982201+0000","last_change":"2024-04-05T22:21:14.931694+0000","last_active":"2024-04-05T22:21:15.982201+0000","last_peered":"2024-04-05T22:21:15.982201+0000","last_clean":"2024-04-05T22:21:15.982201+0000","last_became_active":"2024-04-05T22:21:14.931452+0000","last_became_peered":"2024-04-05T22:21:14.931452+0000","last_unstale":"2024-04-05T22:21:15.982201+0000","last_undegraded":"2024-04-05T22:21:15.982201+0000","last_fullsized":"2024-04-05T22:21:15.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900462+0000","last_change":"2024-04-05T22:21:14.431053+0000","last_active":"2024-04-05T22:21:15.900462+0000","last_peered":"2024-04-05T22:21:15.900462+0000","last_clean":"2024-04-05T22:21:15.900462+0000","last_became_active":"2024-04-05T22:21:14.430719+0000","last_became_peered":"2024-04-05T22:21:14.430719+0000","last_unstale":"2024-04-05T22:21:15.900462+0000","last_undegraded":"2024-04-05T22:21:15.900462+0000","last_fullsized":"2024-04-05T22:21:15.900462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:02.241437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899853+0000","last_change":"2024-04-05T22:21:14.921494+0000","last_active":"2024-04-05T22:21:15.899853+0000","last_peered":"2024-04-05T22:21:15.899853+0000","last_clean":"2024-04-05T22:21:15.899853+0000","last_became_active":"2024-04-05T22:21:14.920872+0000","last_became_peered":"2024-04-05T22:21:14.920872+0000","last_unstale":"2024-04-05T22:21:15.899853+0000","last_undegraded":"2024-04-05T22:21:15.899853+0000","last_fullsized":"2024-04-05T22:21:15.899853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897333+0000","last_change":"2024-04-05T22:21:14.439971+0000","last_active":"2024-04-05T22:21:15.897333+0000","last_peered":"2024-04-05T22:21:15.897333+0000","last_clean":"2024-04-05T22:21:15.897333+0000","last_became_active":"2024-04-05T22:21:14.439620+0000","last_became_peered":"2024-04-05T22:21:14.439620+0000","last_unstale":"2024-04-05T22:21:15.897333+0000","last_undegraded":"2024-04-05T22:21:15.897333+0000","last_fullsized":"2024-04-05T22:21:15.897333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981058+0000","last_change":"2024-04-05T22:21:14.930698+0000","last_active":"2024-04-05T22:21:15.981058+0000","last_peered":"2024-04-05T22:21:15.981058+0000","last_clean":"2024-04-05T22:21:15.981058+0000","last_became_active":"2024-04-05T22:21:14.930442+0000","last_became_peered":"2024-04-05T22:21:14.930442+0000","last_unstale":"2024-04-05T22:21:15.981058+0000","last_undegraded":"2024-04-05T22:21:15.981058+0000","last_fullsized":"2024-04-05T22:21:15.981058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-05T22:21:14.902603+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:21:14.902603+0000","last_peered":"2024-04-05T22:21:14.902603+0000","last_clean":"2024-04-05T22:21:14.902603+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:21:14.902603+0000","last_undegraded":"2024-04-05T22:21:14.902603+0000","last_fullsized":"2024-04-05T22:21:14.902603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898869+0000","last_change":"2024-04-05T22:21:14.440806+0000","last_active":"2024-04-05T22:21:15.898869+0000","last_peered":"2024-04-05T22:21:15.898869+0000","last_clean":"2024-04-05T22:21:15.898869+0000","last_became_active":"2024-04-05T22:21:14.439437+0000","last_became_peered":"2024-04-05T22:21:14.439437+0000","last_unstale":"2024-04-05T22:21:15.898869+0000","last_undegraded":"2024-04-05T22:21:15.898869+0000","last_fullsized":"2024-04-05T22:21:15.898869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976007+0000","last_change":"2024-04-05T22:21:14.919101+0000","last_active":"2024-04-05T22:21:15.976007+0000","last_peered":"2024-04-05T22:21:15.976007+0000","last_clean":"2024-04-05T22:21:15.976007+0000","last_became_active":"2024-04-05T22:21:14.916214+0000","last_became_peered":"2024-04-05T22:21:14.916214+0000","last_unstale":"2024-04-05T22:21:15.976007+0000","last_undegraded":"2024-04-05T22:21:15.976007+0000","last_fullsized":"2024-04-05T22:21:15.976007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979251+0000","last_change":"2024-04-05T22:21:14.438539+0000","last_active":"2024-04-05T22:21:15.979251+0000","last_peered":"2024-04-05T22:21:15.979251+0000","last_clean":"2024-04-05T22:21:15.979251+0000","last_became_active":"2024-04-05T22:21:14.437939+0000","last_became_peered":"2024-04-05T22:21:14.437939+0000","last_unstale":"2024-04-05T22:21:15.979251+0000","last_undegraded":"2024-04-05T22:21:15.979251+0000","last_fullsized":"2024-04-05T22:21:15.979251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975766+0000","last_change":"2024-04-05T22:21:14.916601+0000","last_active":"2024-04-05T22:21:15.975766+0000","last_peered":"2024-04-05T22:21:15.975766+0000","last_clean":"2024-04-05T22:21:15.975766+0000","last_became_active":"2024-04-05T22:21:14.916330+0000","last_became_peered":"2024-04-05T22:21:14.916330+0000","last_unstale":"2024-04-05T22:21:15.975766+0000","last_undegraded":"2024-04-05T22:21:15.975766+0000","last_fullsized":"2024-04-05T22:21:15.975766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897037+0000","last_change":"2024-04-05T22:21:14.434305+0000","last_active":"2024-04-05T22:21:15.897037+0000","last_peered":"2024-04-05T22:21:15.897037+0000","last_clean":"2024-04-05T22:21:15.897037+0000","last_became_active":"2024-04-05T22:21:14.433877+0000","last_became_peered":"2024-04-05T22:21:14.433877+0000","last_unstale":"2024-04-05T22:21:15.897037+0000","last_undegraded":"2024-04-05T22:21:15.897037+0000","last_fullsized":"2024-04-05T22:21:15.897037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899989+0000","last_change":"2024-04-05T22:21:14.912112+0000","last_active":"2024-04-05T22:21:15.899989+0000","last_peered":"2024-04-05T22:21:15.899989+0000","last_clean":"2024-04-05T22:21:15.899989+0000","last_became_active":"2024-04-05T22:21:14.910925+0000","last_became_peered":"2024-04-05T22:21:14.910925+0000","last_unstale":"2024-04-05T22:21:15.899989+0000","last_undegraded":"2024-04-05T22:21:15.899989+0000","last_fullsized":"2024-04-05T22:21:15.899989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899675+0000","last_change":"2024-04-05T22:21:14.438374+0000","last_active":"2024-04-05T22:21:15.899675+0000","last_peered":"2024-04-05T22:21:15.899675+0000","last_clean":"2024-04-05T22:21:15.899675+0000","last_became_active":"2024-04-05T22:21:14.437921+0000","last_became_peered":"2024-04-05T22:21:14.437921+0000","last_unstale":"2024-04-05T22:21:15.899675+0000","last_undegraded":"2024-04-05T22:21:15.899675+0000","last_fullsized":"2024-04-05T22:21:15.899675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983069+0000","last_change":"2024-04-05T22:21:14.914627+0000","last_active":"2024-04-05T22:21:15.983069+0000","last_peered":"2024-04-05T22:21:15.983069+0000","last_clean":"2024-04-05T22:21:15.983069+0000","last_became_active":"2024-04-05T22:21:14.914234+0000","last_became_peered":"2024-04-05T22:21:14.914234+0000","last_unstale":"2024-04-05T22:21:15.983069+0000","last_undegraded":"2024-04-05T22:21:15.983069+0000","last_fullsized":"2024-04-05T22:21:15.983069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900119+0000","last_change":"2024-04-05T22:21:14.431183+0000","last_active":"2024-04-05T22:21:15.900119+0000","last_peered":"2024-04-05T22:21:15.900119+0000","last_clean":"2024-04-05T22:21:15.900119+0000","last_became_active":"2024-04-05T22:21:14.430912+0000","last_became_peered":"2024-04-05T22:21:14.430912+0000","last_unstale":"2024-04-05T22:21:15.900119+0000","last_undegraded":"2024-04-05T22:21:15.900119+0000","last_fullsized":"2024-04-05T22:21:15.900119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:23.153789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899958+0000","last_change":"2024-04-05T22:21:14.921009+0000","last_active":"2024-04-05T22:21:15.899958+0000","last_peered":"2024-04-05T22:21:15.899958+0000","last_clean":"2024-04-05T22:21:15.899958+0000","last_became_active":"2024-04-05T22:21:14.920657+0000","last_became_peered":"2024-04-05T22:21:14.920657+0000","last_unstale":"2024-04-05T22:21:15.899958+0000","last_undegraded":"2024-04-05T22:21:15.899958+0000","last_fullsized":"2024-04-05T22:21:15.899958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981703+0000","last_change":"2024-04-05T22:21:14.443668+0000","last_active":"2024-04-05T22:21:15.981703+0000","last_peered":"2024-04-05T22:21:15.981703+0000","last_clean":"2024-04-05T22:21:15.981703+0000","last_became_active":"2024-04-05T22:21:14.442178+0000","last_became_peered":"2024-04-05T22:21:14.442178+0000","last_unstale":"2024-04-05T22:21:15.981703+0000","last_undegraded":"2024-04-05T22:21:15.981703+0000","last_fullsized":"2024-04-05T22:21:15.981703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900022+0000","last_change":"2024-04-05T22:21:14.905357+0000","last_active":"2024-04-05T22:21:15.900022+0000","last_peered":"2024-04-05T22:21:15.900022+0000","last_clean":"2024-04-05T22:21:15.900022+0000","last_became_active":"2024-04-05T22:21:14.905068+0000","last_became_peered":"2024-04-05T22:21:14.905068+0000","last_unstale":"2024-04-05T22:21:15.900022+0000","last_undegraded":"2024-04-05T22:21:15.900022+0000","last_fullsized":"2024-04-05T22:21:15.900022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:17.628026+0000","last_change":"2024-04-05T22:21:14.442815+0000","last_active":"2024-04-05T22:21:17.628026+0000","last_peered":"2024-04-05T22:21:17.628026+0000","last_clean":"2024-04-05T22:21:17.628026+0000","last_became_active":"2024-04-05T22:21:14.442217+0000","last_became_peered":"2024-04-05T22:21:14.442217+0000","last_unstale":"2024-04-05T22:21:17.628026+0000","last_undegraded":"2024-04-05T22:21:17.628026+0000","last_fullsized":"2024-04-05T22:21:17.628026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976452+0000","last_change":"2024-04-05T22:21:14.933779+0000","last_active":"2024-04-05T22:21:15.976452+0000","last_peered":"2024-04-05T22:21:15.976452+0000","last_clean":"2024-04-05T22:21:15.976452+0000","last_became_active":"2024-04-05T22:21:14.933248+0000","last_became_peered":"2024-04-05T22:21:14.933248+0000","last_unstale":"2024-04-05T22:21:15.976452+0000","last_undegraded":"2024-04-05T22:21:15.976452+0000","last_fullsized":"2024-04-05T22:21:15.976452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981318+0000","last_change":"2024-04-05T22:21:14.449034+0000","last_active":"2024-04-05T22:21:15.981318+0000","last_peered":"2024-04-05T22:21:15.981318+0000","last_clean":"2024-04-05T22:21:15.981318+0000","last_became_active":"2024-04-05T22:21:14.448183+0000","last_became_peered":"2024-04-05T22:21:14.448183+0000","last_unstale":"2024-04-05T22:21:15.981318+0000","last_undegraded":"2024-04-05T22:21:15.981318+0000","last_fullsized":"2024-04-05T22:21:15.981318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:29:59.320001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.977308+0000","last_change":"2024-04-05T22:21:14.921617+0000","last_active":"2024-04-05T22:21:15.977308+0000","last_peered":"2024-04-05T22:21:15.977308+0000","last_clean":"2024-04-05T22:21:15.977308+0000","last_became_active":"2024-04-05T22:21:14.921153+0000","last_became_peered":"2024-04-05T22:21:14.921153+0000","last_unstale":"2024-04-05T22:21:15.977308+0000","last_undegraded":"2024-04-05T22:21:15.977308+0000","last_fullsized":"2024-04-05T22:21:15.977308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979031+0000","last_change":"2024-04-05T22:21:14.442038+0000","last_active":"2024-04-05T22:21:15.979031+0000","last_peered":"2024-04-05T22:21:15.979031+0000","last_clean":"2024-04-05T22:21:15.979031+0000","last_became_active":"2024-04-05T22:21:14.441431+0000","last_became_peered":"2024-04-05T22:21:14.441431+0000","last_unstale":"2024-04-05T22:21:15.979031+0000","last_undegraded":"2024-04-05T22:21:15.979031+0000","last_fullsized":"2024-04-05T22:21:15.979031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982667+0000","last_change":"2024-04-05T22:21:14.930105+0000","last_active":"2024-04-05T22:21:15.982667+0000","last_peered":"2024-04-05T22:21:15.982667+0000","last_clean":"2024-04-05T22:21:15.982667+0000","last_became_active":"2024-04-05T22:21:14.927238+0000","last_became_peered":"2024-04-05T22:21:14.927238+0000","last_unstale":"2024-04-05T22:21:15.982667+0000","last_undegraded":"2024-04-05T22:21:15.982667+0000","last_fullsized":"2024-04-05T22:21:15.982667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900204+0000","last_change":"2024-04-05T22:21:14.437077+0000","last_active":"2024-04-05T22:21:15.900204+0000","last_peered":"2024-04-05T22:21:15.900204+0000","last_clean":"2024-04-05T22:21:15.900204+0000","last_became_active":"2024-04-05T22:21:14.436767+0000","last_became_peered":"2024-04-05T22:21:14.436767+0000","last_unstale":"2024-04-05T22:21:15.900204+0000","last_undegraded":"2024-04-05T22:21:15.900204+0000","last_fullsized":"2024-04-05T22:21:15.900204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975658+0000","last_change":"2024-04-05T22:21:14.920674+0000","last_active":"2024-04-05T22:21:15.975658+0000","last_peered":"2024-04-05T22:21:15.975658+0000","last_clean":"2024-04-05T22:21:15.975658+0000","last_became_active":"2024-04-05T22:21:14.919722+0000","last_became_peered":"2024-04-05T22:21:14.919722+0000","last_unstale":"2024-04-05T22:21:15.975658+0000","last_undegraded":"2024-04-05T22:21:15.975658+0000","last_fullsized":"2024-04-05T22:21:15.975658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900313+0000","last_change":"2024-04-05T22:21:14.432768+0000","last_active":"2024-04-05T22:21:15.900313+0000","last_peered":"2024-04-05T22:21:15.900313+0000","last_clean":"2024-04-05T22:21:15.900313+0000","last_became_active":"2024-04-05T22:21:14.432330+0000","last_became_peered":"2024-04-05T22:21:14.432330+0000","last_unstale":"2024-04-05T22:21:15.900313+0000","last_undegraded":"2024-04-05T22:21:15.900313+0000","last_fullsized":"2024-04-05T22:21:15.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976206+0000","last_change":"2024-04-05T22:21:14.919559+0000","last_active":"2024-04-05T22:21:15.976206+0000","last_peered":"2024-04-05T22:21:15.976206+0000","last_clean":"2024-04-05T22:21:15.976206+0000","last_became_active":"2024-04-05T22:21:14.916617+0000","last_became_peered":"2024-04-05T22:21:14.916617+0000","last_unstale":"2024-04-05T22:21:15.976206+0000","last_undegraded":"2024-04-05T22:21:15.976206+0000","last_fullsized":"2024-04-05T22:21:15.976206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980370+0000","last_change":"2024-04-05T22:21:14.445535+0000","last_active":"2024-04-05T22:21:15.980370+0000","last_peered":"2024-04-05T22:21:15.980370+0000","last_clean":"2024-04-05T22:21:15.980370+0000","last_became_active":"2024-04-05T22:21:14.438718+0000","last_became_peered":"2024-04-05T22:21:14.438718+0000","last_unstale":"2024-04-05T22:21:15.980370+0000","last_undegraded":"2024-04-05T22:21:15.980370+0000","last_fullsized":"2024-04-05T22:21:15.980370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980726+0000","last_change":"2024-04-05T22:21:14.928694+0000","last_active":"2024-04-05T22:21:15.980726+0000","last_peered":"2024-04-05T22:21:15.980726+0000","last_clean":"2024-04-05T22:21:15.980726+0000","last_became_active":"2024-04-05T22:21:14.928392+0000","last_became_peered":"2024-04-05T22:21:14.928392+0000","last_unstale":"2024-04-05T22:21:15.980726+0000","last_undegraded":"2024-04-05T22:21:15.980726+0000","last_fullsized":"2024-04-05T22:21:15.980726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:02:26.504384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900241+0000","last_change":"2024-04-05T22:21:14.919963+0000","last_active":"2024-04-05T22:21:15.900241+0000","last_peered":"2024-04-05T22:21:15.900241+0000","last_clean":"2024-04-05T22:21:15.900241+0000","last_became_active":"2024-04-05T22:21:14.917075+0000","last_became_peered":"2024-04-05T22:21:14.917075+0000","last_unstale":"2024-04-05T22:21:15.900241+0000","last_undegraded":"2024-04-05T22:21:15.900241+0000","last_fullsized":"2024-04-05T22:21:15.900241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980519+0000","last_change":"2024-04-05T22:21:14.440258+0000","last_active":"2024-04-05T22:21:15.980519+0000","last_peered":"2024-04-05T22:21:15.980519+0000","last_clean":"2024-04-05T22:21:15.980519+0000","last_became_active":"2024-04-05T22:21:14.439946+0000","last_became_peered":"2024-04-05T22:21:14.439946+0000","last_unstale":"2024-04-05T22:21:15.980519+0000","last_undegraded":"2024-04-05T22:21:15.980519+0000","last_fullsized":"2024-04-05T22:21:15.980519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899588+0000","last_change":"2024-04-05T22:21:14.914728+0000","last_active":"2024-04-05T22:21:15.899588+0000","last_peered":"2024-04-05T22:21:15.899588+0000","last_clean":"2024-04-05T22:21:15.899588+0000","last_became_active":"2024-04-05T22:21:14.914450+0000","last_became_peered":"2024-04-05T22:21:14.914450+0000","last_unstale":"2024-04-05T22:21:15.899588+0000","last_undegraded":"2024-04-05T22:21:15.899588+0000","last_fullsized":"2024-04-05T22:21:15.899588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981282+0000","last_change":"2024-04-05T22:21:14.451271+0000","last_active":"2024-04-05T22:21:15.981282+0000","last_peered":"2024-04-05T22:21:15.981282+0000","last_clean":"2024-04-05T22:21:15.981282+0000","last_became_active":"2024-04-05T22:21:14.450977+0000","last_became_peered":"2024-04-05T22:21:14.450977+0000","last_unstale":"2024-04-05T22:21:15.981282+0000","last_undegraded":"2024-04-05T22:21:15.981282+0000","last_fullsized":"2024-04-05T22:21:15.981282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.921420+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.919777+0000","last_became_peered":"2024-04-05T22:21:14.919777+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976482+0000","last_change":"2024-04-05T22:21:14.442123+0000","last_active":"2024-04-05T22:21:15.976482+0000","last_peered":"2024-04-05T22:21:15.976482+0000","last_clean":"2024-04-05T22:21:15.976482+0000","last_became_active":"2024-04-05T22:21:14.441885+0000","last_became_peered":"2024-04-05T22:21:14.441885+0000","last_unstale":"2024-04-05T22:21:15.976482+0000","last_undegraded":"2024-04-05T22:21:15.976482+0000","last_fullsized":"2024-04-05T22:21:15.976482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976454+0000","last_change":"2024-04-05T22:21:14.923884+0000","last_active":"2024-04-05T22:21:15.976454+0000","last_peered":"2024-04-05T22:21:15.976454+0000","last_clean":"2024-04-05T22:21:15.976454+0000","last_became_active":"2024-04-05T22:21:14.923545+0000","last_became_peered":"2024-04-05T22:21:14.923545+0000","last_unstale":"2024-04-05T22:21:15.976454+0000","last_undegraded":"2024-04-05T22:21:15.976454+0000","last_fullsized":"2024-04-05T22:21:15.976454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976809+0000","last_change":"2024-04-05T22:21:14.439558+0000","last_active":"2024-04-05T22:21:15.976809+0000","last_peered":"2024-04-05T22:21:15.976809+0000","last_clean":"2024-04-05T22:21:15.976809+0000","last_became_active":"2024-04-05T22:21:14.439082+0000","last_became_peered":"2024-04-05T22:21:14.439082+0000","last_unstale":"2024-04-05T22:21:15.976809+0000","last_undegraded":"2024-04-05T22:21:15.976809+0000","last_fullsized":"2024-04-05T22:21:15.976809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.77}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7569999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:21:24.761 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:21:24.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.996+0000 7f674b000640 1 Processor -- start 2024-04-05T22:21:24.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.996+0000 7f674b000640 1 -- start start 2024-04-05T22:21:24.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.997+0000 7f674b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f674415acc0 0x7f67440ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.997+0000 7f674b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:24.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f674b000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674415f5f0 con 0x7f674415a8e0 2024-04-05T22:21:24.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f674b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440cb3a0 con 0x7f674415acc0 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f674b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674415f350 con 0x7f67440ca9f0 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f674415acc0 0x7f67440ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f674415acc0 0x7f67440ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51358/0 (socket says 172.21.15.73:51358) 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 -- 172.21.15.73:0/1930740361 learned_addr learned my addr 172.21.15.73:0/1930740361 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6749000640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 -- 172.21.15.73:0/1930740361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 msgr2=0x7f67440cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 -- 172.21.15.73:0/1930740361 >> v1:172.21.15.157:6789/0 conn(0x7f674415a8e0 legacy=0x7f67440c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:24.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.998+0000 7f6743400640 1 -- 172.21.15.73:0/1930740361 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440c3830 con 0x7f674415acc0 2024-04-05T22:21:24.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.999+0000 7f6743400640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ad720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6730009870 tx=0x7f673002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987f85efd668ef58 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:24.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.999+0000 7f6742a00640 1 -- 172.21.15.73:0/1930740361 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730031740 con 0x7f674415acc0 2024-04-05T22:21:24.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.999+0000 7f6742a00640 1 -- 172.21.15.73:0/1930740361 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67300318a0 con 0x7f674415acc0 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:24.999+0000 7f6742a00640 1 -- 172.21.15.73:0/1930740361 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730031c30 con 0x7f674415acc0 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 -- 172.21.15.73:0/1930740361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 msgr2=0x7f67440ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ad720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6730009870 tx=0x7f673002b7e0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 -- 172.21.15.73:0/1930740361 shutdown_connections 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 --2- 172.21.15.73:0/1930740361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ad720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 -- 172.21.15.73:0/1930740361 >> 172.21.15.73:0/1930740361 conn(0x7f67440b0490 msgr2=0x7f67440b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 -- 172.21.15.73:0/1930740361 shutdown_connections 2024-04-05T22:21:25.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.000+0000 7f674b000640 1 -- 172.21.15.73:0/1930740361 wait complete. 2024-04-05T22:21:25.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.002+0000 7f674b000640 1 Processor -- start 2024-04-05T22:21:25.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.002+0000 7f674b000640 1 -- start start 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f674415a8e0 0x7f67441f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cd850 con 0x7f674415acc0 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67441529c0 con 0x7f674415a8e0 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f674b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674415ca10 con 0x7f67440ca9f0 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51362/0 (socket says 172.21.15.73:51362) 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 -- 172.21.15.73:0/2227260784 learned_addr learned my addr 172.21.15.73:0/2227260784 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 -- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 msgr2=0x7f67440ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 -- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f674415a8e0 msgr2=0x7f67441f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f674415a8e0 0x7f67441f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 -- 172.21.15.73:0/2227260784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67300094d0 con 0x7f674415acc0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.003+0000 7f6749000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f673c00cc60 tx=0x7f673c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.004+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00ce70 con 0x7f674415acc0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.004+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c003990 con 0x7f674415acc0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.004+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c003d20 con 0x7f674415acc0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.004+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67440cd330 con 0x7f674415acc0 2024-04-05T22:21:25.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.004+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674415c3c0 con 0x7f674415acc0 2024-04-05T22:21:25.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.005+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6744068160 con 0x7f674415acc0 2024-04-05T22:21:25.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.006+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f673c003e80 con 0x7f674415acc0 2024-04-05T22:21:25.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.007+0000 7f673be00640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 0x7f672406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.007+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f673c089560 con 0x7f674415acc0 2024-04-05T22:21:25.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.007+0000 7f6743e00640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 0x7f672406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.008+0000 7f6743e00640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 0x7f672406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6734003cc0 tx=0x7f67340079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.011+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f673c054740 con 0x7f674415acc0 2024-04-05T22:21:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.188+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67440c4700 con 0x7f672406ac00 2024-04-05T22:21:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.195+0000 7f673be00640 1 -- 172.21.15.73:0/2227260784 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264477 (secure 0 0 0) 0x7f6734014400 con 0x7f672406ac00 2024-04-05T22:21:25.198 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:25.199 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 msgr2=0x7f672406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 0x7f672406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6734003cc0 tx=0x7f67340079b0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 msgr2=0x7f67440ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f673c00cc60 tx=0x7f673c00a5c0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 shutdown_connections 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f672406ac00 0x7f672406d0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f674415acc0 0x7f67440ac8d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f674415a8e0 0x7f67441f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 --2- 172.21.15.73:0/2227260784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67440ca9f0 0x7f67440ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.201+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 >> 172.21.15.73:0/2227260784 conn(0x7f67440b0490 msgr2=0x7f6744155780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.202+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 shutdown_connections 2024-04-05T22:21:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.202+0000 7f674b000640 1 -- 172.21.15.73:0/2227260784 wait complete. 2024-04-05T22:21:25.218 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-05T22:21:24.491082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":3055616,"data_stored":1816372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.599141"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982064+0000","last_change":"2024-04-05T22:21:14.923383+0000","last_active":"2024-04-05T22:21:15.982064+0000","last_peered":"2024-04-05T22:21:15.982064+0000","last_clean":"2024-04-05T22:21:15.982064+0000","last_became_active":"2024-04-05T22:21:14.922950+0000","last_became_peered":"2024-04-05T22:21:14.922950+0000","last_unstale":"2024-04-05T22:21:15.982064+0000","last_undegraded":"2024-04-05T22:21:15.982064+0000","last_fullsized":"2024-04-05T22:21:15.982064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900565+0000","last_change":"2024-04-05T22:21:14.912375+0000","last_active":"2024-04-05T22:21:15.900565+0000","last_peered":"2024-04-05T22:21:15.900565+0000","last_clean":"2024-04-05T22:21:15.900565+0000","last_became_active":"2024-04-05T22:21:14.912009+0000","last_became_peered":"2024-04-05T22:21:14.912009+0000","last_unstale":"2024-04-05T22:21:15.900565+0000","last_undegraded":"2024-04-05T22:21:15.900565+0000","last_fullsized":"2024-04-05T22:21:15.900565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976623+0000","last_change":"2024-04-05T22:21:14.914402+0000","last_active":"2024-04-05T22:21:15.976623+0000","last_peered":"2024-04-05T22:21:15.976623+0000","last_clean":"2024-04-05T22:21:15.976623+0000","last_became_active":"2024-04-05T22:21:14.914120+0000","last_became_peered":"2024-04-05T22:21:14.914120+0000","last_unstale":"2024-04-05T22:21:15.976623+0000","last_undegraded":"2024-04-05T22:21:15.976623+0000","last_fullsized":"2024-04-05T22:21:15.976623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899962+0000","last_change":"2024-04-05T22:21:14.921662+0000","last_active":"2024-04-05T22:21:15.899962+0000","last_peered":"2024-04-05T22:21:15.899962+0000","last_clean":"2024-04-05T22:21:15.899962+0000","last_became_active":"2024-04-05T22:21:14.921218+0000","last_became_peered":"2024-04-05T22:21:14.921218+0000","last_unstale":"2024-04-05T22:21:15.899962+0000","last_undegraded":"2024-04-05T22:21:15.899962+0000","last_fullsized":"2024-04-05T22:21:15.899962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979973+0000","last_change":"2024-04-05T22:21:14.935767+0000","last_active":"2024-04-05T22:21:15.979973+0000","last_peered":"2024-04-05T22:21:15.979973+0000","last_clean":"2024-04-05T22:21:15.979973+0000","last_became_active":"2024-04-05T22:21:14.935145+0000","last_became_peered":"2024-04-05T22:21:14.935145+0000","last_unstale":"2024-04-05T22:21:15.979973+0000","last_undegraded":"2024-04-05T22:21:15.979973+0000","last_fullsized":"2024-04-05T22:21:15.979973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898816+0000","last_change":"2024-04-05T22:21:14.923031+0000","last_active":"2024-04-05T22:21:15.898816+0000","last_peered":"2024-04-05T22:21:15.898816+0000","last_clean":"2024-04-05T22:21:15.898816+0000","last_became_active":"2024-04-05T22:21:14.922585+0000","last_became_peered":"2024-04-05T22:21:14.922585+0000","last_unstale":"2024-04-05T22:21:15.898816+0000","last_undegraded":"2024-04-05T22:21:15.898816+0000","last_fullsized":"2024-04-05T22:21:15.898816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:21.709331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979244+0000","last_change":"2024-04-05T22:21:14.925099+0000","last_active":"2024-04-05T22:21:15.979244+0000","last_peered":"2024-04-05T22:21:15.979244+0000","last_clean":"2024-04-05T22:21:15.979244+0000","last_became_active":"2024-04-05T22:21:14.921678+0000","last_became_peered":"2024-04-05T22:21:14.921678+0000","last_unstale":"2024-04-05T22:21:15.979244+0000","last_undegraded":"2024-04-05T22:21:15.979244+0000","last_fullsized":"2024-04-05T22:21:15.979244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975561+0000","last_change":"2024-04-05T22:21:14.916195+0000","last_active":"2024-04-05T22:21:15.975561+0000","last_peered":"2024-04-05T22:21:15.975561+0000","last_clean":"2024-04-05T22:21:15.975561+0000","last_became_active":"2024-04-05T22:21:14.915947+0000","last_became_peered":"2024-04-05T22:21:14.915947+0000","last_unstale":"2024-04-05T22:21:15.975561+0000","last_undegraded":"2024-04-05T22:21:15.975561+0000","last_fullsized":"2024-04-05T22:21:15.975561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982147+0000","last_change":"2024-04-05T22:21:14.931207+0000","last_active":"2024-04-05T22:21:15.982147+0000","last_peered":"2024-04-05T22:21:15.982147+0000","last_clean":"2024-04-05T22:21:15.982147+0000","last_became_active":"2024-04-05T22:21:14.930509+0000","last_became_peered":"2024-04-05T22:21:14.930509+0000","last_unstale":"2024-04-05T22:21:15.982147+0000","last_undegraded":"2024-04-05T22:21:15.982147+0000","last_fullsized":"2024-04-05T22:21:15.982147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981506+0000","last_change":"2024-04-05T22:21:14.916586+0000","last_active":"2024-04-05T22:21:15.981506+0000","last_peered":"2024-04-05T22:21:15.981506+0000","last_clean":"2024-04-05T22:21:15.981506+0000","last_became_active":"2024-04-05T22:21:14.916004+0000","last_became_peered":"2024-04-05T22:21:14.916004+0000","last_unstale":"2024-04-05T22:21:15.981506+0000","last_undegraded":"2024-04-05T22:21:15.981506+0000","last_fullsized":"2024-04-05T22:21:15.981506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:02.978295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981439+0000","last_change":"2024-04-05T22:21:14.913056+0000","last_active":"2024-04-05T22:21:15.981439+0000","last_peered":"2024-04-05T22:21:15.981439+0000","last_clean":"2024-04-05T22:21:15.981439+0000","last_became_active":"2024-04-05T22:21:14.912748+0000","last_became_peered":"2024-04-05T22:21:14.912748+0000","last_unstale":"2024-04-05T22:21:15.981439+0000","last_undegraded":"2024-04-05T22:21:15.981439+0000","last_fullsized":"2024-04-05T22:21:15.981439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:11:59.394291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983331+0000","last_change":"2024-04-05T22:21:14.926927+0000","last_active":"2024-04-05T22:21:15.983331+0000","last_peered":"2024-04-05T22:21:15.983331+0000","last_clean":"2024-04-05T22:21:15.983331+0000","last_became_active":"2024-04-05T22:21:14.913316+0000","last_became_peered":"2024-04-05T22:21:14.913316+0000","last_unstale":"2024-04-05T22:21:15.983331+0000","last_undegraded":"2024-04-05T22:21:15.983331+0000","last_fullsized":"2024-04-05T22:21:15.983331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:14.865832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976116+0000","last_change":"2024-04-05T22:21:14.919462+0000","last_active":"2024-04-05T22:21:15.976116+0000","last_peered":"2024-04-05T22:21:15.976116+0000","last_clean":"2024-04-05T22:21:15.976116+0000","last_became_active":"2024-04-05T22:21:14.919180+0000","last_became_peered":"2024-04-05T22:21:14.919180+0000","last_unstale":"2024-04-05T22:21:15.976116+0000","last_undegraded":"2024-04-05T22:21:15.976116+0000","last_fullsized":"2024-04-05T22:21:15.976116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975694+0000","last_change":"2024-04-05T22:21:14.912391+0000","last_active":"2024-04-05T22:21:15.975694+0000","last_peered":"2024-04-05T22:21:15.975694+0000","last_clean":"2024-04-05T22:21:15.975694+0000","last_became_active":"2024-04-05T22:21:14.912013+0000","last_became_peered":"2024-04-05T22:21:14.912013+0000","last_unstale":"2024-04-05T22:21:15.975694+0000","last_undegraded":"2024-04-05T22:21:15.975694+0000","last_fullsized":"2024-04-05T22:21:15.975694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899930+0000","last_change":"2024-04-05T22:21:14.900459+0000","last_active":"2024-04-05T22:21:15.899930+0000","last_peered":"2024-04-05T22:21:15.899930+0000","last_clean":"2024-04-05T22:21:15.899930+0000","last_became_active":"2024-04-05T22:21:14.900144+0000","last_became_peered":"2024-04-05T22:21:14.900144+0000","last_unstale":"2024-04-05T22:21:15.899930+0000","last_undegraded":"2024-04-05T22:21:15.899930+0000","last_fullsized":"2024-04-05T22:21:15.899930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898646+0000","last_change":"2024-04-05T22:21:14.913235+0000","last_active":"2024-04-05T22:21:15.898646+0000","last_peered":"2024-04-05T22:21:15.898646+0000","last_clean":"2024-04-05T22:21:15.898646+0000","last_became_active":"2024-04-05T22:21:14.912913+0000","last_became_peered":"2024-04-05T22:21:14.912913+0000","last_unstale":"2024-04-05T22:21:15.898646+0000","last_undegraded":"2024-04-05T22:21:15.898646+0000","last_fullsized":"2024-04-05T22:21:15.898646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:56.757361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900293+0000","last_change":"2024-04-05T22:21:14.931728+0000","last_active":"2024-04-05T22:21:15.900293+0000","last_peered":"2024-04-05T22:21:15.900293+0000","last_clean":"2024-04-05T22:21:15.900293+0000","last_became_active":"2024-04-05T22:21:14.931500+0000","last_became_peered":"2024-04-05T22:21:14.931500+0000","last_unstale":"2024-04-05T22:21:15.900293+0000","last_undegraded":"2024-04-05T22:21:15.900293+0000","last_fullsized":"2024-04-05T22:21:15.900293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897627+0000","last_change":"2024-04-05T22:21:14.921738+0000","last_active":"2024-04-05T22:21:15.897627+0000","last_peered":"2024-04-05T22:21:15.897627+0000","last_clean":"2024-04-05T22:21:15.897627+0000","last_became_active":"2024-04-05T22:21:14.919781+0000","last_became_peered":"2024-04-05T22:21:14.919781+0000","last_unstale":"2024-04-05T22:21:15.897627+0000","last_undegraded":"2024-04-05T22:21:15.897627+0000","last_fullsized":"2024-04-05T22:21:15.897627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981513+0000","last_change":"2024-04-05T22:21:14.920265+0000","last_active":"2024-04-05T22:21:15.981513+0000","last_peered":"2024-04-05T22:21:15.981513+0000","last_clean":"2024-04-05T22:21:15.981513+0000","last_became_active":"2024-04-05T22:21:14.919985+0000","last_became_peered":"2024-04-05T22:21:14.919985+0000","last_unstale":"2024-04-05T22:21:15.981513+0000","last_undegraded":"2024-04-05T22:21:15.981513+0000","last_fullsized":"2024-04-05T22:21:15.981513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981857+0000","last_change":"2024-04-05T22:21:14.929527+0000","last_active":"2024-04-05T22:21:15.981857+0000","last_peered":"2024-04-05T22:21:15.981857+0000","last_clean":"2024-04-05T22:21:15.981857+0000","last_became_active":"2024-04-05T22:21:14.929222+0000","last_became_peered":"2024-04-05T22:21:14.929222+0000","last_unstale":"2024-04-05T22:21:15.981857+0000","last_undegraded":"2024-04-05T22:21:15.981857+0000","last_fullsized":"2024-04-05T22:21:15.981857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:15:04.491398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975985+0000","last_change":"2024-04-05T22:21:14.933523+0000","last_active":"2024-04-05T22:21:15.975985+0000","last_peered":"2024-04-05T22:21:15.975985+0000","last_clean":"2024-04-05T22:21:15.975985+0000","last_became_active":"2024-04-05T22:21:14.933340+0000","last_became_peered":"2024-04-05T22:21:14.933340+0000","last_unstale":"2024-04-05T22:21:15.975985+0000","last_undegraded":"2024-04-05T22:21:15.975985+0000","last_fullsized":"2024-04-05T22:21:15.975985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898875+0000","last_change":"2024-04-05T22:21:14.933905+0000","last_active":"2024-04-05T22:21:15.898875+0000","last_peered":"2024-04-05T22:21:15.898875+0000","last_clean":"2024-04-05T22:21:15.898875+0000","last_became_active":"2024-04-05T22:21:14.933507+0000","last_became_peered":"2024-04-05T22:21:14.933507+0000","last_unstale":"2024-04-05T22:21:15.898875+0000","last_undegraded":"2024-04-05T22:21:15.898875+0000","last_fullsized":"2024-04-05T22:21:15.898875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975831+0000","last_change":"2024-04-05T22:21:14.915842+0000","last_active":"2024-04-05T22:21:15.975831+0000","last_peered":"2024-04-05T22:21:15.975831+0000","last_clean":"2024-04-05T22:21:15.975831+0000","last_became_active":"2024-04-05T22:21:14.915391+0000","last_became_peered":"2024-04-05T22:21:14.915391+0000","last_unstale":"2024-04-05T22:21:15.975831+0000","last_undegraded":"2024-04-05T22:21:15.975831+0000","last_fullsized":"2024-04-05T22:21:15.975831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:21.773200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898993+0000","last_change":"2024-04-05T22:21:14.924789+0000","last_active":"2024-04-05T22:21:15.898993+0000","last_peered":"2024-04-05T22:21:15.898993+0000","last_clean":"2024-04-05T22:21:15.898993+0000","last_became_active":"2024-04-05T22:21:14.924488+0000","last_became_peered":"2024-04-05T22:21:14.924488+0000","last_unstale":"2024-04-05T22:21:15.898993+0000","last_undegraded":"2024-04-05T22:21:15.898993+0000","last_fullsized":"2024-04-05T22:21:15.898993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976451+0000","last_change":"2024-04-05T22:21:14.926693+0000","last_active":"2024-04-05T22:21:15.976451+0000","last_peered":"2024-04-05T22:21:15.976451+0000","last_clean":"2024-04-05T22:21:15.976451+0000","last_became_active":"2024-04-05T22:21:14.926415+0000","last_became_peered":"2024-04-05T22:21:14.926415+0000","last_unstale":"2024-04-05T22:21:15.976451+0000","last_undegraded":"2024-04-05T22:21:15.976451+0000","last_fullsized":"2024-04-05T22:21:15.976451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981744+0000","last_change":"2024-04-05T22:21:14.940178+0000","last_active":"2024-04-05T22:21:15.981744+0000","last_peered":"2024-04-05T22:21:15.981744+0000","last_clean":"2024-04-05T22:21:15.981744+0000","last_became_active":"2024-04-05T22:21:14.938042+0000","last_became_peered":"2024-04-05T22:21:14.938042+0000","last_unstale":"2024-04-05T22:21:15.981744+0000","last_undegraded":"2024-04-05T22:21:15.981744+0000","last_fullsized":"2024-04-05T22:21:15.981744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:42.750931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900215+0000","last_change":"2024-04-05T22:21:14.916877+0000","last_active":"2024-04-05T22:21:15.900215+0000","last_peered":"2024-04-05T22:21:15.900215+0000","last_clean":"2024-04-05T22:21:15.900215+0000","last_became_active":"2024-04-05T22:21:14.916585+0000","last_became_peered":"2024-04-05T22:21:14.916585+0000","last_unstale":"2024-04-05T22:21:15.900215+0000","last_undegraded":"2024-04-05T22:21:15.900215+0000","last_fullsized":"2024-04-05T22:21:15.900215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976229+0000","last_change":"2024-04-05T22:21:14.918894+0000","last_active":"2024-04-05T22:21:15.976229+0000","last_peered":"2024-04-05T22:21:15.976229+0000","last_clean":"2024-04-05T22:21:15.976229+0000","last_became_active":"2024-04-05T22:21:14.918074+0000","last_became_peered":"2024-04-05T22:21:14.918074+0000","last_unstale":"2024-04-05T22:21:15.976229+0000","last_undegraded":"2024-04-05T22:21:15.976229+0000","last_fullsized":"2024-04-05T22:21:15.976229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:10.899784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976071+0000","last_change":"2024-04-05T22:21:14.922850+0000","last_active":"2024-04-05T22:21:15.976071+0000","last_peered":"2024-04-05T22:21:15.976071+0000","last_clean":"2024-04-05T22:21:15.976071+0000","last_became_active":"2024-04-05T22:21:14.920770+0000","last_became_peered":"2024-04-05T22:21:14.920770+0000","last_unstale":"2024-04-05T22:21:15.976071+0000","last_undegraded":"2024-04-05T22:21:15.976071+0000","last_fullsized":"2024-04-05T22:21:15.976071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899082+0000","last_change":"2024-04-05T22:21:14.917603+0000","last_active":"2024-04-05T22:21:15.899082+0000","last_peered":"2024-04-05T22:21:15.899082+0000","last_clean":"2024-04-05T22:21:15.899082+0000","last_became_active":"2024-04-05T22:21:14.917325+0000","last_became_peered":"2024-04-05T22:21:14.917325+0000","last_unstale":"2024-04-05T22:21:15.899082+0000","last_undegraded":"2024-04-05T22:21:15.899082+0000","last_fullsized":"2024-04-05T22:21:15.899082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:46.139845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.896840+0000","last_change":"2024-04-05T22:21:14.911372+0000","last_active":"2024-04-05T22:21:15.896840+0000","last_peered":"2024-04-05T22:21:15.896840+0000","last_clean":"2024-04-05T22:21:15.896840+0000","last_became_active":"2024-04-05T22:21:14.911083+0000","last_became_peered":"2024-04-05T22:21:14.911083+0000","last_unstale":"2024-04-05T22:21:15.896840+0000","last_undegraded":"2024-04-05T22:21:15.896840+0000","last_fullsized":"2024-04-05T22:21:15.896840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979172+0000","last_change":"2024-04-05T22:21:14.915897+0000","last_active":"2024-04-05T22:21:15.979172+0000","last_peered":"2024-04-05T22:21:15.979172+0000","last_clean":"2024-04-05T22:21:15.979172+0000","last_became_active":"2024-04-05T22:21:14.915223+0000","last_became_peered":"2024-04-05T22:21:14.915223+0000","last_unstale":"2024-04-05T22:21:15.979172+0000","last_undegraded":"2024-04-05T22:21:15.979172+0000","last_fullsized":"2024-04-05T22:21:15.979172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:19:20.576451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982664+0000","last_change":"2024-04-05T22:21:14.901329+0000","last_active":"2024-04-05T22:21:15.982664+0000","last_peered":"2024-04-05T22:21:15.982664+0000","last_clean":"2024-04-05T22:21:15.982664+0000","last_became_active":"2024-04-05T22:21:14.900870+0000","last_became_peered":"2024-04-05T22:21:14.900870+0000","last_unstale":"2024-04-05T22:21:15.982664+0000","last_undegraded":"2024-04-05T22:21:15.982664+0000","last_fullsized":"2024-04-05T22:21:15.982664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:12.554588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976297+0000","last_change":"2024-04-05T22:21:14.437934+0000","last_active":"2024-04-05T22:21:15.976297+0000","last_peered":"2024-04-05T22:21:15.976297+0000","last_clean":"2024-04-05T22:21:15.976297+0000","last_became_active":"2024-04-05T22:21:14.437696+0000","last_became_peered":"2024-04-05T22:21:14.437696+0000","last_unstale":"2024-04-05T22:21:15.976297+0000","last_undegraded":"2024-04-05T22:21:15.976297+0000","last_fullsized":"2024-04-05T22:21:15.976297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976230+0000","last_change":"2024-04-05T22:21:14.920473+0000","last_active":"2024-04-05T22:21:15.976230+0000","last_peered":"2024-04-05T22:21:15.976230+0000","last_clean":"2024-04-05T22:21:15.976230+0000","last_became_active":"2024-04-05T22:21:14.917453+0000","last_became_peered":"2024-04-05T22:21:14.917453+0000","last_unstale":"2024-04-05T22:21:15.976230+0000","last_undegraded":"2024-04-05T22:21:15.976230+0000","last_fullsized":"2024-04-05T22:21:15.976230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:29.600943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980837+0000","last_change":"2024-04-05T22:21:14.449143+0000","last_active":"2024-04-05T22:21:15.980837+0000","last_peered":"2024-04-05T22:21:15.980837+0000","last_clean":"2024-04-05T22:21:15.980837+0000","last_became_active":"2024-04-05T22:21:14.448341+0000","last_became_peered":"2024-04-05T22:21:14.448341+0000","last_unstale":"2024-04-05T22:21:15.980837+0000","last_undegraded":"2024-04-05T22:21:15.980837+0000","last_fullsized":"2024-04-05T22:21:15.980837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:14.544598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975917+0000","last_change":"2024-04-05T22:21:14.441239+0000","last_active":"2024-04-05T22:21:15.975917+0000","last_peered":"2024-04-05T22:21:15.975917+0000","last_clean":"2024-04-05T22:21:15.975917+0000","last_became_active":"2024-04-05T22:21:14.440898+0000","last_became_peered":"2024-04-05T22:21:14.440898+0000","last_unstale":"2024-04-05T22:21:15.975917+0000","last_undegraded":"2024-04-05T22:21:15.975917+0000","last_fullsized":"2024-04-05T22:21:15.975917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975636+0000","last_change":"2024-04-05T22:21:14.922371+0000","last_active":"2024-04-05T22:21:15.975636+0000","last_peered":"2024-04-05T22:21:15.975636+0000","last_clean":"2024-04-05T22:21:15.975636+0000","last_became_active":"2024-04-05T22:21:14.922074+0000","last_became_peered":"2024-04-05T22:21:14.922074+0000","last_unstale":"2024-04-05T22:21:15.975636+0000","last_undegraded":"2024-04-05T22:21:15.975636+0000","last_fullsized":"2024-04-05T22:21:15.975636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898990+0000","last_change":"2024-04-05T22:21:14.446427+0000","last_active":"2024-04-05T22:21:15.898990+0000","last_peered":"2024-04-05T22:21:15.898990+0000","last_clean":"2024-04-05T22:21:15.898990+0000","last_became_active":"2024-04-05T22:21:14.445487+0000","last_became_peered":"2024-04-05T22:21:14.445487+0000","last_unstale":"2024-04-05T22:21:15.898990+0000","last_undegraded":"2024-04-05T22:21:15.898990+0000","last_fullsized":"2024-04-05T22:21:15.898990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899850+0000","last_change":"2024-04-05T22:21:14.917346+0000","last_active":"2024-04-05T22:21:15.899850+0000","last_peered":"2024-04-05T22:21:15.899850+0000","last_clean":"2024-04-05T22:21:15.899850+0000","last_became_active":"2024-04-05T22:21:14.917085+0000","last_became_peered":"2024-04-05T22:21:14.917085+0000","last_unstale":"2024-04-05T22:21:15.899850+0000","last_undegraded":"2024-04-05T22:21:15.899850+0000","last_fullsized":"2024-04-05T22:21:15.899850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981190+0000","last_change":"2024-04-05T22:21:14.441415+0000","last_active":"2024-04-05T22:21:15.981190+0000","last_peered":"2024-04-05T22:21:15.981190+0000","last_clean":"2024-04-05T22:21:15.981190+0000","last_became_active":"2024-04-05T22:21:14.440493+0000","last_became_peered":"2024-04-05T22:21:14.440493+0000","last_unstale":"2024-04-05T22:21:15.981190+0000","last_undegraded":"2024-04-05T22:21:15.981190+0000","last_fullsized":"2024-04-05T22:21:15.981190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:25.845173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899687+0000","last_change":"2024-04-05T22:21:14.932959+0000","last_active":"2024-04-05T22:21:15.899687+0000","last_peered":"2024-04-05T22:21:15.899687+0000","last_clean":"2024-04-05T22:21:15.899687+0000","last_became_active":"2024-04-05T22:21:14.932656+0000","last_became_peered":"2024-04-05T22:21:14.932656+0000","last_unstale":"2024-04-05T22:21:15.899687+0000","last_undegraded":"2024-04-05T22:21:15.899687+0000","last_fullsized":"2024-04-05T22:21:15.899687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:22.318842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899902+0000","last_change":"2024-04-05T22:21:14.444991+0000","last_active":"2024-04-05T22:21:15.899902+0000","last_peered":"2024-04-05T22:21:15.899902+0000","last_clean":"2024-04-05T22:21:15.899902+0000","last_became_active":"2024-04-05T22:21:14.444667+0000","last_became_peered":"2024-04-05T22:21:14.444667+0000","last_unstale":"2024-04-05T22:21:15.899902+0000","last_undegraded":"2024-04-05T22:21:15.899902+0000","last_fullsized":"2024-04-05T22:21:15.899902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.915276+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.914411+0000","last_became_peered":"2024-04-05T22:21:14.914411+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900261+0000","last_change":"2024-04-05T22:21:14.438342+0000","last_active":"2024-04-05T22:21:15.900261+0000","last_peered":"2024-04-05T22:21:15.900261+0000","last_clean":"2024-04-05T22:21:15.900261+0000","last_became_active":"2024-04-05T22:21:14.438070+0000","last_became_peered":"2024-04-05T22:21:14.438070+0000","last_unstale":"2024-04-05T22:21:15.900261+0000","last_undegraded":"2024-04-05T22:21:15.900261+0000","last_fullsized":"2024-04-05T22:21:15.900261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:42:13.607794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983525+0000","last_change":"2024-04-05T22:21:14.928911+0000","last_active":"2024-04-05T22:21:15.983525+0000","last_peered":"2024-04-05T22:21:15.983525+0000","last_clean":"2024-04-05T22:21:15.983525+0000","last_became_active":"2024-04-05T22:21:14.928657+0000","last_became_peered":"2024-04-05T22:21:14.928657+0000","last_unstale":"2024-04-05T22:21:15.983525+0000","last_undegraded":"2024-04-05T22:21:15.983525+0000","last_fullsized":"2024-04-05T22:21:15.983525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:28.742147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982110+0000","last_change":"2024-04-05T22:21:14.442377+0000","last_active":"2024-04-05T22:21:15.982110+0000","last_peered":"2024-04-05T22:21:15.982110+0000","last_clean":"2024-04-05T22:21:15.982110+0000","last_became_active":"2024-04-05T22:21:14.441914+0000","last_became_peered":"2024-04-05T22:21:14.441914+0000","last_unstale":"2024-04-05T22:21:15.982110+0000","last_undegraded":"2024-04-05T22:21:15.982110+0000","last_fullsized":"2024-04-05T22:21:15.982110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:53.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980822+0000","last_change":"2024-04-05T22:21:14.919936+0000","last_active":"2024-04-05T22:21:15.980822+0000","last_peered":"2024-04-05T22:21:15.980822+0000","last_clean":"2024-04-05T22:21:15.980822+0000","last_became_active":"2024-04-05T22:21:14.919602+0000","last_became_peered":"2024-04-05T22:21:14.919602+0000","last_unstale":"2024-04-05T22:21:15.980822+0000","last_undegraded":"2024-04-05T22:21:15.980822+0000","last_fullsized":"2024-04-05T22:21:15.980822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:07:32.506402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976328+0000","last_change":"2024-04-05T22:21:14.437206+0000","last_active":"2024-04-05T22:21:15.976328+0000","last_peered":"2024-04-05T22:21:15.976328+0000","last_clean":"2024-04-05T22:21:15.976328+0000","last_became_active":"2024-04-05T22:21:14.436852+0000","last_became_peered":"2024-04-05T22:21:14.436852+0000","last_unstale":"2024-04-05T22:21:15.976328+0000","last_undegraded":"2024-04-05T22:21:15.976328+0000","last_fullsized":"2024-04-05T22:21:15.976328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975906+0000","last_change":"2024-04-05T22:21:14.933572+0000","last_active":"2024-04-05T22:21:15.975906+0000","last_peered":"2024-04-05T22:21:15.975906+0000","last_clean":"2024-04-05T22:21:15.975906+0000","last_became_active":"2024-04-05T22:21:14.933310+0000","last_became_peered":"2024-04-05T22:21:14.933310+0000","last_unstale":"2024-04-05T22:21:15.975906+0000","last_undegraded":"2024-04-05T22:21:15.975906+0000","last_fullsized":"2024-04-05T22:21:15.975906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:10:27.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899122+0000","last_change":"2024-04-05T22:21:14.435387+0000","last_active":"2024-04-05T22:21:15.899122+0000","last_peered":"2024-04-05T22:21:15.899122+0000","last_clean":"2024-04-05T22:21:15.899122+0000","last_became_active":"2024-04-05T22:21:14.435061+0000","last_became_peered":"2024-04-05T22:21:14.435061+0000","last_unstale":"2024-04-05T22:21:15.899122+0000","last_undegraded":"2024-04-05T22:21:15.899122+0000","last_fullsized":"2024-04-05T22:21:15.899122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980213+0000","last_change":"2024-04-05T22:21:14.935722+0000","last_active":"2024-04-05T22:21:15.980213+0000","last_peered":"2024-04-05T22:21:15.980213+0000","last_clean":"2024-04-05T22:21:15.980213+0000","last_became_active":"2024-04-05T22:21:14.935463+0000","last_became_peered":"2024-04-05T22:21:14.935463+0000","last_unstale":"2024-04-05T22:21:15.980213+0000","last_undegraded":"2024-04-05T22:21:15.980213+0000","last_fullsized":"2024-04-05T22:21:15.980213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:59:55.089408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980372+0000","last_change":"2024-04-05T22:21:14.449636+0000","last_active":"2024-04-05T22:21:15.980372+0000","last_peered":"2024-04-05T22:21:15.980372+0000","last_clean":"2024-04-05T22:21:15.980372+0000","last_became_active":"2024-04-05T22:21:14.449257+0000","last_became_peered":"2024-04-05T22:21:14.449257+0000","last_unstale":"2024-04-05T22:21:15.980372+0000","last_undegraded":"2024-04-05T22:21:15.980372+0000","last_fullsized":"2024-04-05T22:21:15.980372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:56:39.430553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980094+0000","last_change":"2024-04-05T22:21:14.916105+0000","last_active":"2024-04-05T22:21:15.980094+0000","last_peered":"2024-04-05T22:21:15.980094+0000","last_clean":"2024-04-05T22:21:15.980094+0000","last_became_active":"2024-04-05T22:21:14.915590+0000","last_became_peered":"2024-04-05T22:21:14.915590+0000","last_unstale":"2024-04-05T22:21:15.980094+0000","last_undegraded":"2024-04-05T22:21:15.980094+0000","last_fullsized":"2024-04-05T22:21:15.980094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:21:38.340157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897721+0000","last_change":"2024-04-05T22:21:14.429654+0000","last_active":"2024-04-05T22:21:15.897721+0000","last_peered":"2024-04-05T22:21:15.897721+0000","last_clean":"2024-04-05T22:21:15.897721+0000","last_became_active":"2024-04-05T22:21:14.428871+0000","last_became_peered":"2024-04-05T22:21:14.428871+0000","last_unstale":"2024-04-05T22:21:15.897721+0000","last_undegraded":"2024-04-05T22:21:15.897721+0000","last_fullsized":"2024-04-05T22:21:15.897721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979204+0000","last_change":"2024-04-05T22:21:14.921522+0000","last_active":"2024-04-05T22:21:15.979204+0000","last_peered":"2024-04-05T22:21:15.979204+0000","last_clean":"2024-04-05T22:21:15.979204+0000","last_became_active":"2024-04-05T22:21:14.921331+0000","last_became_peered":"2024-04-05T22:21:14.921331+0000","last_unstale":"2024-04-05T22:21:15.979204+0000","last_undegraded":"2024-04-05T22:21:15.979204+0000","last_fullsized":"2024-04-05T22:21:15.979204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983583+0000","last_change":"2024-04-05T22:21:14.437535+0000","last_active":"2024-04-05T22:21:15.983583+0000","last_peered":"2024-04-05T22:21:15.983583+0000","last_clean":"2024-04-05T22:21:15.983583+0000","last_became_active":"2024-04-05T22:21:14.437228+0000","last_became_peered":"2024-04-05T22:21:14.437228+0000","last_unstale":"2024-04-05T22:21:15.983583+0000","last_undegraded":"2024-04-05T22:21:15.983583+0000","last_fullsized":"2024-04-05T22:21:15.983583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:59:38.450307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981661+0000","last_change":"2024-04-05T22:21:14.935527+0000","last_active":"2024-04-05T22:21:15.981661+0000","last_peered":"2024-04-05T22:21:15.981661+0000","last_clean":"2024-04-05T22:21:15.981661+0000","last_became_active":"2024-04-05T22:21:14.935121+0000","last_became_peered":"2024-04-05T22:21:14.935121+0000","last_unstale":"2024-04-05T22:21:15.981661+0000","last_undegraded":"2024-04-05T22:21:15.981661+0000","last_fullsized":"2024-04-05T22:21:15.981661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897618+0000","last_change":"2024-04-05T22:21:14.434360+0000","last_active":"2024-04-05T22:21:15.897618+0000","last_peered":"2024-04-05T22:21:15.897618+0000","last_clean":"2024-04-05T22:21:15.897618+0000","last_became_active":"2024-04-05T22:21:14.434064+0000","last_became_peered":"2024-04-05T22:21:14.434064+0000","last_unstale":"2024-04-05T22:21:15.897618+0000","last_undegraded":"2024-04-05T22:21:15.897618+0000","last_fullsized":"2024-04-05T22:21:15.897618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982201+0000","last_change":"2024-04-05T22:21:14.931694+0000","last_active":"2024-04-05T22:21:15.982201+0000","last_peered":"2024-04-05T22:21:15.982201+0000","last_clean":"2024-04-05T22:21:15.982201+0000","last_became_active":"2024-04-05T22:21:14.931452+0000","last_became_peered":"2024-04-05T22:21:14.931452+0000","last_unstale":"2024-04-05T22:21:15.982201+0000","last_undegraded":"2024-04-05T22:21:15.982201+0000","last_fullsized":"2024-04-05T22:21:15.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:11.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900462+0000","last_change":"2024-04-05T22:21:14.431053+0000","last_active":"2024-04-05T22:21:15.900462+0000","last_peered":"2024-04-05T22:21:15.900462+0000","last_clean":"2024-04-05T22:21:15.900462+0000","last_became_active":"2024-04-05T22:21:14.430719+0000","last_became_peered":"2024-04-05T22:21:14.430719+0000","last_unstale":"2024-04-05T22:21:15.900462+0000","last_undegraded":"2024-04-05T22:21:15.900462+0000","last_fullsized":"2024-04-05T22:21:15.900462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:02.241437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899853+0000","last_change":"2024-04-05T22:21:14.921494+0000","last_active":"2024-04-05T22:21:15.899853+0000","last_peered":"2024-04-05T22:21:15.899853+0000","last_clean":"2024-04-05T22:21:15.899853+0000","last_became_active":"2024-04-05T22:21:14.920872+0000","last_became_peered":"2024-04-05T22:21:14.920872+0000","last_unstale":"2024-04-05T22:21:15.899853+0000","last_undegraded":"2024-04-05T22:21:15.899853+0000","last_fullsized":"2024-04-05T22:21:15.899853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897333+0000","last_change":"2024-04-05T22:21:14.439971+0000","last_active":"2024-04-05T22:21:15.897333+0000","last_peered":"2024-04-05T22:21:15.897333+0000","last_clean":"2024-04-05T22:21:15.897333+0000","last_became_active":"2024-04-05T22:21:14.439620+0000","last_became_peered":"2024-04-05T22:21:14.439620+0000","last_unstale":"2024-04-05T22:21:15.897333+0000","last_undegraded":"2024-04-05T22:21:15.897333+0000","last_fullsized":"2024-04-05T22:21:15.897333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:47.566095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981058+0000","last_change":"2024-04-05T22:21:14.930698+0000","last_active":"2024-04-05T22:21:15.981058+0000","last_peered":"2024-04-05T22:21:15.981058+0000","last_clean":"2024-04-05T22:21:15.981058+0000","last_became_active":"2024-04-05T22:21:14.930442+0000","last_became_peered":"2024-04-05T22:21:14.930442+0000","last_unstale":"2024-04-05T22:21:15.981058+0000","last_undegraded":"2024-04-05T22:21:15.981058+0000","last_fullsized":"2024-04-05T22:21:15.981058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-05T22:21:14.902603+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:21:14.902603+0000","last_peered":"2024-04-05T22:21:14.902603+0000","last_clean":"2024-04-05T22:21:14.902603+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:21:14.902603+0000","last_undegraded":"2024-04-05T22:21:14.902603+0000","last_fullsized":"2024-04-05T22:21:14.902603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.898869+0000","last_change":"2024-04-05T22:21:14.440806+0000","last_active":"2024-04-05T22:21:15.898869+0000","last_peered":"2024-04-05T22:21:15.898869+0000","last_clean":"2024-04-05T22:21:15.898869+0000","last_became_active":"2024-04-05T22:21:14.439437+0000","last_became_peered":"2024-04-05T22:21:14.439437+0000","last_unstale":"2024-04-05T22:21:15.898869+0000","last_undegraded":"2024-04-05T22:21:15.898869+0000","last_fullsized":"2024-04-05T22:21:15.898869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976007+0000","last_change":"2024-04-05T22:21:14.919101+0000","last_active":"2024-04-05T22:21:15.976007+0000","last_peered":"2024-04-05T22:21:15.976007+0000","last_clean":"2024-04-05T22:21:15.976007+0000","last_became_active":"2024-04-05T22:21:14.916214+0000","last_became_peered":"2024-04-05T22:21:14.916214+0000","last_unstale":"2024-04-05T22:21:15.976007+0000","last_undegraded":"2024-04-05T22:21:15.976007+0000","last_fullsized":"2024-04-05T22:21:15.976007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979251+0000","last_change":"2024-04-05T22:21:14.438539+0000","last_active":"2024-04-05T22:21:15.979251+0000","last_peered":"2024-04-05T22:21:15.979251+0000","last_clean":"2024-04-05T22:21:15.979251+0000","last_became_active":"2024-04-05T22:21:14.437939+0000","last_became_peered":"2024-04-05T22:21:14.437939+0000","last_unstale":"2024-04-05T22:21:15.979251+0000","last_undegraded":"2024-04-05T22:21:15.979251+0000","last_fullsized":"2024-04-05T22:21:15.979251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:27.484110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975766+0000","last_change":"2024-04-05T22:21:14.916601+0000","last_active":"2024-04-05T22:21:15.975766+0000","last_peered":"2024-04-05T22:21:15.975766+0000","last_clean":"2024-04-05T22:21:15.975766+0000","last_became_active":"2024-04-05T22:21:14.916330+0000","last_became_peered":"2024-04-05T22:21:14.916330+0000","last_unstale":"2024-04-05T22:21:15.975766+0000","last_undegraded":"2024-04-05T22:21:15.975766+0000","last_fullsized":"2024-04-05T22:21:15.975766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.897037+0000","last_change":"2024-04-05T22:21:14.434305+0000","last_active":"2024-04-05T22:21:15.897037+0000","last_peered":"2024-04-05T22:21:15.897037+0000","last_clean":"2024-04-05T22:21:15.897037+0000","last_became_active":"2024-04-05T22:21:14.433877+0000","last_became_peered":"2024-04-05T22:21:14.433877+0000","last_unstale":"2024-04-05T22:21:15.897037+0000","last_undegraded":"2024-04-05T22:21:15.897037+0000","last_fullsized":"2024-04-05T22:21:15.897037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:11:23.374330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899989+0000","last_change":"2024-04-05T22:21:14.912112+0000","last_active":"2024-04-05T22:21:15.899989+0000","last_peered":"2024-04-05T22:21:15.899989+0000","last_clean":"2024-04-05T22:21:15.899989+0000","last_became_active":"2024-04-05T22:21:14.910925+0000","last_became_peered":"2024-04-05T22:21:14.910925+0000","last_unstale":"2024-04-05T22:21:15.899989+0000","last_undegraded":"2024-04-05T22:21:15.899989+0000","last_fullsized":"2024-04-05T22:21:15.899989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899675+0000","last_change":"2024-04-05T22:21:14.438374+0000","last_active":"2024-04-05T22:21:15.899675+0000","last_peered":"2024-04-05T22:21:15.899675+0000","last_clean":"2024-04-05T22:21:15.899675+0000","last_became_active":"2024-04-05T22:21:14.437921+0000","last_became_peered":"2024-04-05T22:21:14.437921+0000","last_unstale":"2024-04-05T22:21:15.899675+0000","last_undegraded":"2024-04-05T22:21:15.899675+0000","last_fullsized":"2024-04-05T22:21:15.899675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.983069+0000","last_change":"2024-04-05T22:21:14.914627+0000","last_active":"2024-04-05T22:21:15.983069+0000","last_peered":"2024-04-05T22:21:15.983069+0000","last_clean":"2024-04-05T22:21:15.983069+0000","last_became_active":"2024-04-05T22:21:14.914234+0000","last_became_peered":"2024-04-05T22:21:14.914234+0000","last_unstale":"2024-04-05T22:21:15.983069+0000","last_undegraded":"2024-04-05T22:21:15.983069+0000","last_fullsized":"2024-04-05T22:21:15.983069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900119+0000","last_change":"2024-04-05T22:21:14.431183+0000","last_active":"2024-04-05T22:21:15.900119+0000","last_peered":"2024-04-05T22:21:15.900119+0000","last_clean":"2024-04-05T22:21:15.900119+0000","last_became_active":"2024-04-05T22:21:14.430912+0000","last_became_peered":"2024-04-05T22:21:14.430912+0000","last_unstale":"2024-04-05T22:21:15.900119+0000","last_undegraded":"2024-04-05T22:21:15.900119+0000","last_fullsized":"2024-04-05T22:21:15.900119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:23.153789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899958+0000","last_change":"2024-04-05T22:21:14.921009+0000","last_active":"2024-04-05T22:21:15.899958+0000","last_peered":"2024-04-05T22:21:15.899958+0000","last_clean":"2024-04-05T22:21:15.899958+0000","last_became_active":"2024-04-05T22:21:14.920657+0000","last_became_peered":"2024-04-05T22:21:14.920657+0000","last_unstale":"2024-04-05T22:21:15.899958+0000","last_undegraded":"2024-04-05T22:21:15.899958+0000","last_fullsized":"2024-04-05T22:21:15.899958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:34:49.579053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981703+0000","last_change":"2024-04-05T22:21:14.443668+0000","last_active":"2024-04-05T22:21:15.981703+0000","last_peered":"2024-04-05T22:21:15.981703+0000","last_clean":"2024-04-05T22:21:15.981703+0000","last_became_active":"2024-04-05T22:21:14.442178+0000","last_became_peered":"2024-04-05T22:21:14.442178+0000","last_unstale":"2024-04-05T22:21:15.981703+0000","last_undegraded":"2024-04-05T22:21:15.981703+0000","last_fullsized":"2024-04-05T22:21:15.981703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900022+0000","last_change":"2024-04-05T22:21:14.905357+0000","last_active":"2024-04-05T22:21:15.900022+0000","last_peered":"2024-04-05T22:21:15.900022+0000","last_clean":"2024-04-05T22:21:15.900022+0000","last_became_active":"2024-04-05T22:21:14.905068+0000","last_became_peered":"2024-04-05T22:21:14.905068+0000","last_unstale":"2024-04-05T22:21:15.900022+0000","last_undegraded":"2024-04-05T22:21:15.900022+0000","last_fullsized":"2024-04-05T22:21:15.900022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:02:02.157996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:17.628026+0000","last_change":"2024-04-05T22:21:14.442815+0000","last_active":"2024-04-05T22:21:17.628026+0000","last_peered":"2024-04-05T22:21:17.628026+0000","last_clean":"2024-04-05T22:21:17.628026+0000","last_became_active":"2024-04-05T22:21:14.442217+0000","last_became_peered":"2024-04-05T22:21:14.442217+0000","last_unstale":"2024-04-05T22:21:17.628026+0000","last_undegraded":"2024-04-05T22:21:17.628026+0000","last_fullsized":"2024-04-05T22:21:17.628026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976452+0000","last_change":"2024-04-05T22:21:14.933779+0000","last_active":"2024-04-05T22:21:15.976452+0000","last_peered":"2024-04-05T22:21:15.976452+0000","last_clean":"2024-04-05T22:21:15.976452+0000","last_became_active":"2024-04-05T22:21:14.933248+0000","last_became_peered":"2024-04-05T22:21:14.933248+0000","last_unstale":"2024-04-05T22:21:15.976452+0000","last_undegraded":"2024-04-05T22:21:15.976452+0000","last_fullsized":"2024-04-05T22:21:15.976452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:09:20.778324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981318+0000","last_change":"2024-04-05T22:21:14.449034+0000","last_active":"2024-04-05T22:21:15.981318+0000","last_peered":"2024-04-05T22:21:15.981318+0000","last_clean":"2024-04-05T22:21:15.981318+0000","last_became_active":"2024-04-05T22:21:14.448183+0000","last_became_peered":"2024-04-05T22:21:14.448183+0000","last_unstale":"2024-04-05T22:21:15.981318+0000","last_undegraded":"2024-04-05T22:21:15.981318+0000","last_fullsized":"2024-04-05T22:21:15.981318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:29:59.320001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.977308+0000","last_change":"2024-04-05T22:21:14.921617+0000","last_active":"2024-04-05T22:21:15.977308+0000","last_peered":"2024-04-05T22:21:15.977308+0000","last_clean":"2024-04-05T22:21:15.977308+0000","last_became_active":"2024-04-05T22:21:14.921153+0000","last_became_peered":"2024-04-05T22:21:14.921153+0000","last_unstale":"2024-04-05T22:21:15.977308+0000","last_undegraded":"2024-04-05T22:21:15.977308+0000","last_fullsized":"2024-04-05T22:21:15.977308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:03.499475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.979031+0000","last_change":"2024-04-05T22:21:14.442038+0000","last_active":"2024-04-05T22:21:15.979031+0000","last_peered":"2024-04-05T22:21:15.979031+0000","last_clean":"2024-04-05T22:21:15.979031+0000","last_became_active":"2024-04-05T22:21:14.441431+0000","last_became_peered":"2024-04-05T22:21:14.441431+0000","last_unstale":"2024-04-05T22:21:15.979031+0000","last_undegraded":"2024-04-05T22:21:15.979031+0000","last_fullsized":"2024-04-05T22:21:15.979031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.982667+0000","last_change":"2024-04-05T22:21:14.930105+0000","last_active":"2024-04-05T22:21:15.982667+0000","last_peered":"2024-04-05T22:21:15.982667+0000","last_clean":"2024-04-05T22:21:15.982667+0000","last_became_active":"2024-04-05T22:21:14.927238+0000","last_became_peered":"2024-04-05T22:21:14.927238+0000","last_unstale":"2024-04-05T22:21:15.982667+0000","last_undegraded":"2024-04-05T22:21:15.982667+0000","last_fullsized":"2024-04-05T22:21:15.982667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:08.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900204+0000","last_change":"2024-04-05T22:21:14.437077+0000","last_active":"2024-04-05T22:21:15.900204+0000","last_peered":"2024-04-05T22:21:15.900204+0000","last_clean":"2024-04-05T22:21:15.900204+0000","last_became_active":"2024-04-05T22:21:14.436767+0000","last_became_peered":"2024-04-05T22:21:14.436767+0000","last_unstale":"2024-04-05T22:21:15.900204+0000","last_undegraded":"2024-04-05T22:21:15.900204+0000","last_fullsized":"2024-04-05T22:21:15.900204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:11.296551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.975658+0000","last_change":"2024-04-05T22:21:14.920674+0000","last_active":"2024-04-05T22:21:15.975658+0000","last_peered":"2024-04-05T22:21:15.975658+0000","last_clean":"2024-04-05T22:21:15.975658+0000","last_became_active":"2024-04-05T22:21:14.919722+0000","last_became_peered":"2024-04-05T22:21:14.919722+0000","last_unstale":"2024-04-05T22:21:15.975658+0000","last_undegraded":"2024-04-05T22:21:15.975658+0000","last_fullsized":"2024-04-05T22:21:15.975658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:08.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900313+0000","last_change":"2024-04-05T22:21:14.432768+0000","last_active":"2024-04-05T22:21:15.900313+0000","last_peered":"2024-04-05T22:21:15.900313+0000","last_clean":"2024-04-05T22:21:15.900313+0000","last_became_active":"2024-04-05T22:21:14.432330+0000","last_became_peered":"2024-04-05T22:21:14.432330+0000","last_unstale":"2024-04-05T22:21:15.900313+0000","last_undegraded":"2024-04-05T22:21:15.900313+0000","last_fullsized":"2024-04-05T22:21:15.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:04:07.200821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976206+0000","last_change":"2024-04-05T22:21:14.919559+0000","last_active":"2024-04-05T22:21:15.976206+0000","last_peered":"2024-04-05T22:21:15.976206+0000","last_clean":"2024-04-05T22:21:15.976206+0000","last_became_active":"2024-04-05T22:21:14.916617+0000","last_became_peered":"2024-04-05T22:21:14.916617+0000","last_unstale":"2024-04-05T22:21:15.976206+0000","last_undegraded":"2024-04-05T22:21:15.976206+0000","last_fullsized":"2024-04-05T22:21:15.976206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.291668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980370+0000","last_change":"2024-04-05T22:21:14.445535+0000","last_active":"2024-04-05T22:21:15.980370+0000","last_peered":"2024-04-05T22:21:15.980370+0000","last_clean":"2024-04-05T22:21:15.980370+0000","last_became_active":"2024-04-05T22:21:14.438718+0000","last_became_peered":"2024-04-05T22:21:14.438718+0000","last_unstale":"2024-04-05T22:21:15.980370+0000","last_undegraded":"2024-04-05T22:21:15.980370+0000","last_fullsized":"2024-04-05T22:21:15.980370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:38:09.367546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980726+0000","last_change":"2024-04-05T22:21:14.928694+0000","last_active":"2024-04-05T22:21:15.980726+0000","last_peered":"2024-04-05T22:21:15.980726+0000","last_clean":"2024-04-05T22:21:15.980726+0000","last_became_active":"2024-04-05T22:21:14.928392+0000","last_became_peered":"2024-04-05T22:21:14.928392+0000","last_unstale":"2024-04-05T22:21:15.980726+0000","last_undegraded":"2024-04-05T22:21:15.980726+0000","last_fullsized":"2024-04-05T22:21:15.980726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:02:26.504384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.900241+0000","last_change":"2024-04-05T22:21:14.919963+0000","last_active":"2024-04-05T22:21:15.900241+0000","last_peered":"2024-04-05T22:21:15.900241+0000","last_clean":"2024-04-05T22:21:15.900241+0000","last_became_active":"2024-04-05T22:21:14.917075+0000","last_became_peered":"2024-04-05T22:21:14.917075+0000","last_unstale":"2024-04-05T22:21:15.900241+0000","last_undegraded":"2024-04-05T22:21:15.900241+0000","last_fullsized":"2024-04-05T22:21:15.900241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:17.284393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.980519+0000","last_change":"2024-04-05T22:21:14.440258+0000","last_active":"2024-04-05T22:21:15.980519+0000","last_peered":"2024-04-05T22:21:15.980519+0000","last_clean":"2024-04-05T22:21:15.980519+0000","last_became_active":"2024-04-05T22:21:14.439946+0000","last_became_peered":"2024-04-05T22:21:14.439946+0000","last_unstale":"2024-04-05T22:21:15.980519+0000","last_undegraded":"2024-04-05T22:21:15.980519+0000","last_fullsized":"2024-04-05T22:21:15.980519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:16:29.909780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.899588+0000","last_change":"2024-04-05T22:21:14.914728+0000","last_active":"2024-04-05T22:21:15.899588+0000","last_peered":"2024-04-05T22:21:15.899588+0000","last_clean":"2024-04-05T22:21:15.899588+0000","last_became_active":"2024-04-05T22:21:14.914450+0000","last_became_peered":"2024-04-05T22:21:14.914450+0000","last_unstale":"2024-04-05T22:21:15.899588+0000","last_undegraded":"2024-04-05T22:21:15.899588+0000","last_fullsized":"2024-04-05T22:21:15.899588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:44.545665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.981282+0000","last_change":"2024-04-05T22:21:14.451271+0000","last_active":"2024-04-05T22:21:15.981282+0000","last_peered":"2024-04-05T22:21:15.981282+0000","last_clean":"2024-04-05T22:21:15.981282+0000","last_became_active":"2024-04-05T22:21:14.450977+0000","last_became_peered":"2024-04-05T22:21:14.450977+0000","last_unstale":"2024-04-05T22:21:15.981282+0000","last_undegraded":"2024-04-05T22:21:15.981282+0000","last_fullsized":"2024-04-05T22:21:15.981282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:47.681714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976666+0000","last_change":"2024-04-05T22:21:14.921420+0000","last_active":"2024-04-05T22:21:15.976666+0000","last_peered":"2024-04-05T22:21:15.976666+0000","last_clean":"2024-04-05T22:21:15.976666+0000","last_became_active":"2024-04-05T22:21:14.919777+0000","last_became_peered":"2024-04-05T22:21:14.919777+0000","last_unstale":"2024-04-05T22:21:15.976666+0000","last_undegraded":"2024-04-05T22:21:15.976666+0000","last_fullsized":"2024-04-05T22:21:15.976666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:36.680220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976482+0000","last_change":"2024-04-05T22:21:14.442123+0000","last_active":"2024-04-05T22:21:15.976482+0000","last_peered":"2024-04-05T22:21:15.976482+0000","last_clean":"2024-04-05T22:21:15.976482+0000","last_became_active":"2024-04-05T22:21:14.441885+0000","last_became_peered":"2024-04-05T22:21:14.441885+0000","last_unstale":"2024-04-05T22:21:15.976482+0000","last_undegraded":"2024-04-05T22:21:15.976482+0000","last_fullsized":"2024-04-05T22:21:15.976482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976454+0000","last_change":"2024-04-05T22:21:14.923884+0000","last_active":"2024-04-05T22:21:15.976454+0000","last_peered":"2024-04-05T22:21:15.976454+0000","last_clean":"2024-04-05T22:21:15.976454+0000","last_became_active":"2024-04-05T22:21:14.923545+0000","last_became_peered":"2024-04-05T22:21:14.923545+0000","last_unstale":"2024-04-05T22:21:15.976454+0000","last_undegraded":"2024-04-05T22:21:15.976454+0000","last_fullsized":"2024-04-05T22:21:15.976454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:14.385613+0000","last_clean_scrub_stamp":"2024-04-05T22:21:14.385613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-05T22:21:15.976809+0000","last_change":"2024-04-05T22:21:14.439558+0000","last_active":"2024-04-05T22:21:15.976809+0000","last_peered":"2024-04-05T22:21:15.976809+0000","last_clean":"2024-04-05T22:21:15.976809+0000","last_became_active":"2024-04-05T22:21:14.439082+0000","last_became_peered":"2024-04-05T22:21:14.439082+0000","last_unstale":"2024-04-05T22:21:15.976809+0000","last_undegraded":"2024-04-05T22:21:15.976809+0000","last_fullsized":"2024-04-05T22:21:15.976809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:13.127576+0000","last_clean_scrub_stamp":"2024-04-05T22:21:13.127576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:40.832915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.77}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7569999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:21:25.222 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-05T22:21:25.222 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-05T22:21:25.222 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-05T22:21:25.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-05T22:21:25.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.456+0000 7f3002a00640 1 Processor -- start 2024-04-05T22:21:25.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.457+0000 7f3002a00640 1 -- start start 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3002a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2ffc0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3002a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc0ad880 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3002a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc0ce5d0 con 0x7f2ffc0b1870 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3002a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0ce010 con 0x7f2ffc160620 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3000a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ffc0cb2f0 0x7f2ffc0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48862/0 (socket says 172.21.15.73:48862) 2024-04-05T22:21:25.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f3000a00640 1 -- 172.21.15.73:0/1772336645 learned_addr learned my addr 172.21.15.73:0/1772336645 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:25.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2ffc0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857681160 0 0) 0x7f2ffc0ad880 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.458+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe0003520 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346545832 0 0) 0x7f2fe0003520 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 msgr2=0x7f2ffc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2ffc0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 >> v1:172.21.15.157:6789/0 conn(0x7f2ffc0b1870 legacy=0x7f2ffc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0ce2f0 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2ff40019f0 con 0x7f2ffc0cb2f0 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 >> v1:172.21.15.73:6789/0 conn(0x7f2ffc0cb2f0 legacy=0x7f2ffc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fe00039a0 0x7f2fe0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2fe00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe0004d00 0x7f2fe0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0ce010 con 0x7f2fe00039a0 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe000bb10 con 0x7f2fe0004d00 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe000bca0 con 0x7f2ffc160620 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fe00039a0 0x7f2fe0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f3000a00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe0004d00 0x7f2fe0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ffbe00640 1 -- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 msgr2=0x7f2fe00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.459+0000 7f2ff3400640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2fe00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.460+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc160620 0x7f2fe00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.460+0000 7f2ffbe00640 1 -- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe0004d00 msgr2=0x7f2fe0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.460+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe0004d00 0x7f2fe0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.460+0000 7f2ffbe00640 1 -- 172.21.15.73:0/1772336645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0003520 con 0x7f2fe00039a0 2024-04-05T22:21:25.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.460+0000 7f3000a00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe0004d00 0x7f2fe0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fe00039a0 0x7f2fe0003dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ba60 tx=0x7f2fe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423248e9ddd9da7e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec032620 con 0x7f2fe00039a0 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec032780 con 0x7f2fe00039a0 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f2ffb400640 1 -- 172.21.15.73:0/1772336645 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec032b10 con 0x7f2fe00039a0 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3002a00640 1 -- 172.21.15.73:0/1772336645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fe00039a0 msgr2=0x7f2fe0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3002a00640 1 --2- 172.21.15.73:0/1772336645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fe00039a0 0x7f2fe0003dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ba60 tx=0x7f2fe00057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3000a00640 1 -- 172.21.15.73:0/1772336645 reap_dead start 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3002a00640 1 -- 172.21.15.73:0/1772336645 shutdown_connections 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3002a00640 1 -- 172.21.15.73:0/1772336645 >> 172.21.15.73:0/1772336645 conn(0x7f2ffc0ce8e0 msgr2=0x7f2ffc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.461+0000 7f3002a00640 1 -- 172.21.15.73:0/1772336645 shutdown_connections 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.462+0000 7f3002a00640 1 -- 172.21.15.73:0/1772336645 wait complete. 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.463+0000 7f3002a00640 1 Processor -- start 2024-04-05T22:21:25.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.463+0000 7f3002a00640 1 -- start start 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe00039a0 0x7f2ffc14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc15a820 0x7f2ffc158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc159780 con 0x7f2ffc159290 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc159a60 con 0x7f2fe00039a0 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3002a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc159d40 con 0x7f2ffc15a820 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3000a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3000a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51390/0 (socket says 172.21.15.73:51390) 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.464+0000 7f3000a00640 1 -- 172.21.15.73:0/846220169 learned_addr learned my addr 172.21.15.73:0/846220169 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe00039a0 0x7f2ffc14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f2ff3400640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc15a820 0x7f2ffc158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 -- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc15a820 msgr2=0x7f2ffc158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc15a820 0x7f2ffc158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 -- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe00039a0 msgr2=0x7f2ffc14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe00039a0 0x7f2ffc14a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 -- 172.21.15.73:0/846220169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc15a020 con 0x7f2ffc159290 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3000a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0667b0 tx=0x7f2ff400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4004d60 con 0x7f2ffc159290 2024-04-05T22:21:25.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff400bb80 con 0x7f2ffc159290 2024-04-05T22:21:25.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4007110 con 0x7f2ffc159290 2024-04-05T22:21:25.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc15a300 con 0x7f2ffc159290 2024-04-05T22:21:25.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.465+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc0afaa0 con 0x7f2ffc159290 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.466+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc066af0 con 0x7f2ffc159290 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.467+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2ff4007270 con 0x7f2ffc159290 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.468+0000 7f2ff8c00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 0x7f2fd806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.468+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ff40895a0 con 0x7f2ffc159290 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.468+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 0x7f2fd806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.469+0000 7f2ffbe00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 0x7f2fd806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ba30 tx=0x7f2fec03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.473+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2ff4054770 con 0x7f2ffc159290 2024-04-05T22:21:25.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.691+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2ffc06d960 con 0x7f2ffc159290 2024-04-05T22:21:25.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.692+0000 7f2ff8c00640 1 -- 172.21.15.73:0/846220169 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2ff4057d90 con 0x7f2ffc159290 2024-04-05T22:21:25.691 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:25.691 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-05T22:21:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 msgr2=0x7f2fd806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 0x7f2fd806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ba30 tx=0x7f2fec03d040 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 msgr2=0x7f2ffc0af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0667b0 tx=0x7f2ff400b4a0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 shutdown_connections 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2fd806ac00 0x7f2fd806d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ffc159290 0x7f2ffc0af560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ffc15a820 0x7f2ffc158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 --2- 172.21.15.73:0/846220169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe00039a0 0x7f2ffc14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 >> 172.21.15.73:0/846220169 conn(0x7f2ffc0ce8e0 msgr2=0x7f2ffc0ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 shutdown_connections 2024-04-05T22:21:25.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.694+0000 7f3002a00640 1 -- 172.21.15.73:0/846220169 wait complete. 2024-04-05T22:21:25.709 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-05T22:21:25.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.907+0000 7fa48b800640 1 Processor -- start 2024-04-05T22:21:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.907+0000 7fa48b800640 1 -- start start 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa48b800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa48415cf70 0x7fa4840d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa48b800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840ceee0 con 0x7fa4840d0bd0 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa48b800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840ce5b0 con 0x7fa4840d1930 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa48b800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4840ce890 con 0x7fa48415cf70 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa483e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa4840d0bd0 0x7fa4840d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37678/0 (socket says 172.21.15.73:37678) 2024-04-05T22:21:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa483e00640 1 -- 172.21.15.73:0/185490275 learned_addr learned my addr 172.21.15.73:0/185490275 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.909+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011233862 0 0) 0x7fa4840ceee0 con 0x7fa4840d0bd0 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa468003520 con 0x7fa4840d0bd0 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa488e00640 1 --2- 172.21.15.73:0/185490275 >> v2:172.21.15.157:3300/0 conn(0x7fa48415cf70 0x7fa4840d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963144902 0 0) 0x7fa468003520 con 0x7fa4840d0bd0 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 >> v2:172.21.15.157:3300/0 conn(0x7fa48415cf70 msgr2=0x7fa4840d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 --2- 172.21.15.73:0/185490275 >> v2:172.21.15.157:3300/0 conn(0x7fa48415cf70 0x7fa4840d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 >> v1:172.21.15.157:6789/0 conn(0x7fa4840d1930 legacy=0x7fa4840b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840ceb70 con 0x7fa4840d0bd0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa46c001670 con 0x7fa4840d0bd0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 >> v1:172.21.15.73:6789/0 conn(0x7fa4840d0bd0 legacy=0x7fa4840d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa468003970 0x7fa468003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 0x7fa468004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4680052a0 0x7fa46800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840ce890 con 0x7fa4680047c0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46800c170 con 0x7fa468003970 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46800c330 con 0x7fa4680052a0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.910+0000 7fa483e00640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 0x7fa468004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa488e00640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa468003970 0x7fa468003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa483e00640 1 -- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4680052a0 msgr2=0x7fa46800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa489800640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4680052a0 0x7fa46800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa483e00640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4680052a0 0x7fa46800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa483e00640 1 -- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa468003970 msgr2=0x7fa468003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa483e00640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa468003970 0x7fa468003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa483e00640 1 -- 172.21.15.73:0/185490275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa468003520 con 0x7fa4680047c0 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.911+0000 7fa489800640 1 -- 172.21.15.73:0/185490275 reap_dead start 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa483e00640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 0x7fa468004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c210 tx=0x7fa4680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c89fb6b5e0435 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005ad0 con 0x7fa4680047c0 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c005c50 con 0x7fa4680047c0 2024-04-05T22:21:25.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa483400640 1 -- 172.21.15.73:0/185490275 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c02c870 con 0x7fa4680047c0 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa48b800640 1 -- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 msgr2=0x7fa468004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.912+0000 7fa48b800640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 0x7fa468004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c210 tx=0x7fa4680057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.913+0000 7fa48b800640 1 -- 172.21.15.73:0/185490275 shutdown_connections 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.913+0000 7fa48b800640 1 --2- 172.21.15.73:0/185490275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4680047c0 0x7fa468004c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.913+0000 7fa48b800640 1 -- 172.21.15.73:0/185490275 >> 172.21.15.73:0/185490275 conn(0x7fa4840d2df0 msgr2=0x7fa4840d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.913+0000 7fa48b800640 1 -- 172.21.15.73:0/185490275 shutdown_connections 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.913+0000 7fa48b800640 1 -- 172.21.15.73:0/185490275 wait complete. 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.914+0000 7fa48b800640 1 Processor -- start 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.914+0000 7fa48b800640 1 -- start start 2024-04-05T22:21:25.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4680047c0 0x7fa4840b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4841f1d00 0x7fa4841f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840b7f80 con 0x7fa4841f5f60 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4841f63e0 con 0x7fa4680047c0 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.915+0000 7fa48b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4841f6910 con 0x7fa4841f1d00 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38824/0 (socket says 172.21.15.73:38824) 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 -- 172.21.15.73:0/965008490 learned_addr learned my addr 172.21.15.73:0/965008490 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 -- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4841f1d00 msgr2=0x7fa4841f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4841f1d00 0x7fa4841f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 -- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4680047c0 msgr2=0x7fa4840b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4680047c0 0x7fa4840b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:25.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 -- 172.21.15.73:0/965008490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840ca5b0 con 0x7fa4841f5f60 2024-04-05T22:21:25.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa489800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4840b59e0 tx=0x7fa47c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003b70 con 0x7fa4841f5f60 2024-04-05T22:21:25.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c003cd0 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4840b8160 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.916+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c005270 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.917+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4840b8720 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.918+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4840b8940 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.918+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa47c0053d0 con 0x7fa4841f5f60 2024-04-05T22:21:25.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.919+0000 7fa480c00640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 0x7fa46006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:25.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.920+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa47c088c80 con 0x7fa4841f5f60 2024-04-05T22:21:25.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.920+0000 7fa488e00640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 0x7fa46006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:25.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.920+0000 7fa488e00640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 0x7fa46006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa474006e80 tx=0x7fa474006e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:25.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:25.924+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa47c053e70 con 0x7fa4841f5f60 2024-04-05T22:21:26.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.102+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa484068160 con 0x7fa4841f5f60 2024-04-05T22:21:26.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.104+0000 7fa480c00640 1 -- 172.21.15.73:0/965008490 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fa47c018070 con 0x7fa4841f5f60 2024-04-05T22:21:26.104 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:26.104 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 msgr2=0x7fa46006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 0x7fa46006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa474006e80 tx=0x7fa474006e10 comp rx=0 tx=0).stop 2024-04-05T22:21:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 msgr2=0x7fa48414ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4840b59e0 tx=0x7fa47c00dc00 comp rx=0 tx=0).stop 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 shutdown_connections 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa46006ac00 0x7fa46006d0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4841f5f60 0x7fa48414ef40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4841f1d00 0x7fa4841f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 --2- 172.21.15.73:0/965008490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4680047c0 0x7fa4840b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 >> 172.21.15.73:0/965008490 conn(0x7fa4840d2df0 msgr2=0x7fa4840cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 shutdown_connections 2024-04-05T22:21:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.107+0000 7fa48b800640 1 -- 172.21.15.73:0/965008490 wait complete. 2024-04-05T22:21:26.123 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:26.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.318+0000 7f0a3e400640 1 Processor -- start 2024-04-05T22:21:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.318+0000 7f0a3e400640 1 -- start start 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.319+0000 7f0a3e400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a3815abe0 0x7f0a380712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.319+0000 7f0a3e400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3815afc0 con 0x7f0a3806c9a0 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.319+0000 7f0a3e400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3806f5b0 con 0x7f0a3806bc40 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.319+0000 7f0a3e400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3806f890 con 0x7f0a3815abe0 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a3815abe0 0x7f0a380712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a3815abe0 0x7f0a380712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38840/0 (socket says 172.21.15.73:38840) 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 -- 172.21.15.73:0/2931891856 learned_addr learned my addr 172.21.15.73:0/2931891856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a2ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0a3806c9a0 0x7f0a3806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37684/0 (socket says 172.21.15.73:37684) 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 -- 172.21.15.73:0/2931891856 >> v1:172.21.15.157:6789/0 conn(0x7f0a3806bc40 legacy=0x7f0a38164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 -- 172.21.15.73:0/2931891856 >> v1:172.21.15.73:6789/0 conn(0x7f0a3806c9a0 legacy=0x7f0a3806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a37400640 1 -- 172.21.15.73:0/2931891856 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3806fb70 con 0x7f0a3815abe0 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.320+0000 7f0a36a00640 1 -- 172.21.15.73:0/2931891856 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900056918 0 0) 0x7f0a3815afc0 con 0x7f0a3806c9a0 2024-04-05T22:21:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.321+0000 7f0a37400640 1 --2- 172.21.15.73:0/2931891856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3815abe0 0x7f0a380712e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009870 tx=0x7f0a2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379089ca064aeeaf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.321+0000 7f0a36a00640 1 -- 172.21.15.73:0/2931891856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280317e0 con 0x7f0a3815abe0 2024-04-05T22:21:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a36a00640 1 -- 172.21.15.73:0/2931891856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28031940 con 0x7f0a3815abe0 2024-04-05T22:21:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a36a00640 1 -- 172.21.15.73:0/2931891856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28031cd0 con 0x7f0a3815abe0 2024-04-05T22:21:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 -- 172.21.15.73:0/2931891856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3815abe0 msgr2=0x7f0a380712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 --2- 172.21.15.73:0/2931891856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3815abe0 0x7f0a380712e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009870 tx=0x7f0a2802b880 comp rx=0 tx=0).stop 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 -- 172.21.15.73:0/2931891856 shutdown_connections 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 --2- 172.21.15.73:0/2931891856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3815abe0 0x7f0a380712e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 -- 172.21.15.73:0/2931891856 >> 172.21.15.73:0/2931891856 conn(0x7f0a38069cb0 msgr2=0x7f0a3806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 -- 172.21.15.73:0/2931891856 shutdown_connections 2024-04-05T22:21:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.322+0000 7f0a3e400640 1 -- 172.21.15.73:0/2931891856 wait complete. 2024-04-05T22:21:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.324+0000 7f0a3e400640 1 Processor -- start 2024-04-05T22:21:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.324+0000 7f0a3e400640 1 -- start start 2024-04-05T22:21:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3806c9a0 0x7f0a38147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815abe0 0x7f0a3815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38145a10 con 0x7f0a3806bc40 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381481b0 con 0x7f0a3806c9a0 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a3e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38148490 con 0x7f0a3815abe0 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38856/0 (socket says 172.21.15.73:38856) 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 -- 172.21.15.73:0/396608193 learned_addr learned my addr 172.21.15.73:0/396608193 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 -- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815abe0 msgr2=0x7f0a3815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815abe0 0x7f0a3815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 -- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3806c9a0 msgr2=0x7f0a38147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3806c9a0 0x7f0a38147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.325+0000 7f0a37400640 1 -- 172.21.15.73:0/396608193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280094d0 con 0x7f0a3806bc40 2024-04-05T22:21:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a37400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000c00 tx=0x7f0a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28032880 con 0x7f0a3806bc40 2024-04-05T22:21:26.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28032a00 con 0x7f0a3806bc40 2024-04-05T22:21:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a38148770 con 0x7f0a3806bc40 2024-04-05T22:21:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28032db0 con 0x7f0a3806bc40 2024-04-05T22:21:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.326+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a38149140 con 0x7f0a3806bc40 2024-04-05T22:21:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.327+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38067e90 con 0x7f0a3806bc40 2024-04-05T22:21:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.328+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0a2804e450 con 0x7f0a3806bc40 2024-04-05T22:21:26.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.329+0000 7f0a2fe00640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 0x7f0a1406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.329+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a280ab540 con 0x7f0a3806bc40 2024-04-05T22:21:26.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.329+0000 7f0a2ea00640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 0x7f0a1406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.330+0000 7f0a2ea00640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 0x7f0a1406d110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a20007fd0 tx=0x7f0a20007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.333+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0a28076830 con 0x7f0a3806bc40 2024-04-05T22:21:26.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.525+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a38068160 con 0x7f0a3806bc40 2024-04-05T22:21:26.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.526+0000 7f0a2fe00640 1 -- 172.21.15.73:0/396608193 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a28079e50 con 0x7f0a3806bc40 2024-04-05T22:21:26.525 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.528+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 msgr2=0x7f0a1406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.528+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 0x7f0a1406d110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a20007fd0 tx=0x7f0a20007920 comp rx=0 tx=0).stop 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.528+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 msgr2=0x7f0a381454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.528+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000c00 tx=0x7f0a280046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.528+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 shutdown_connections 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0a1406ac50 0x7f0a1406d110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815abe0 0x7f0a3815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3806c9a0 0x7f0a38147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 --2- 172.21.15.73:0/396608193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a3806bc40 0x7f0a381454d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 >> 172.21.15.73:0/396608193 conn(0x7f0a38069cb0 msgr2=0x7f0a380cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:26.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 shutdown_connections 2024-04-05T22:21:26.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.529+0000 7f0a3e400640 1 -- 172.21.15.73:0/396608193 wait complete. 2024-04-05T22:21:26.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:26.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.753+0000 7f4ac5a00640 1 Processor -- start 2024-04-05T22:21:26.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.753+0000 7f4ac5a00640 1 -- start start 2024-04-05T22:21:26.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.754+0000 7f4ac5a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4ac015a850 0x7f4ac015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.754+0000 7f4ac5a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 0x7f4ac006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.754+0000 7f4ac5a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac006bc50 con 0x7f4ac015e580 2024-04-05T22:21:26.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.754+0000 7f4ac5a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac015f530 con 0x7f4ac015a850 2024-04-05T22:21:26.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.754+0000 7f4ac5a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac015ef70 con 0x7f4ac0070640 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4ac015a850 0x7f4ac015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4ac015a850 0x7f4ac015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38860/0 (socket says 172.21.15.73:38860) 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 -- 172.21.15.73:0/435331106 learned_addr learned my addr 172.21.15.73:0/435331106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4ac5000640 1 --2- 172.21.15.73:0/435331106 >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 0x7f4ac006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 -- 172.21.15.73:0/435331106 >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 msgr2=0x7f4ac006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 --2- 172.21.15.73:0/435331106 >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 0x7f4ac006c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 -- 172.21.15.73:0/435331106 >> v1:172.21.15.157:6789/0 conn(0x7f4ac015e580 legacy=0x7f4ac006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4abfe00640 1 -- 172.21.15.73:0/435331106 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac015f250 con 0x7f4ac015a850 2024-04-05T22:21:26.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.755+0000 7f4ac5000640 1 --2- 172.21.15.73:0/435331106 >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 0x7f4ac006c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.756+0000 7f4abfe00640 1 --2- 172.21.15.73:0/435331106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac015ac30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00cbe50 tx=0x7f4ab002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c197889600b5946d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.756+0000 7f4abea00640 1 -- 172.21.15.73:0/435331106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00317e0 con 0x7f4ac015a850 2024-04-05T22:21:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.756+0000 7f4abea00640 1 -- 172.21.15.73:0/435331106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab0031940 con 0x7f4ac015a850 2024-04-05T22:21:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.756+0000 7f4abea00640 1 -- 172.21.15.73:0/435331106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0031cd0 con 0x7f4ac015a850 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 -- 172.21.15.73:0/435331106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 msgr2=0x7f4ac015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/435331106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac015ac30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00cbe50 tx=0x7f4ab002b880 comp rx=0 tx=0).stop 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 -- 172.21.15.73:0/435331106 shutdown_connections 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/435331106 >> v2:172.21.15.157:3300/0 conn(0x7f4ac0070640 0x7f4ac006c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/435331106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac015ac30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 -- 172.21.15.73:0/435331106 >> 172.21.15.73:0/435331106 conn(0x7f4ac0069cb0 msgr2=0x7f4ac006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 -- 172.21.15.73:0/435331106 shutdown_connections 2024-04-05T22:21:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.757+0000 7f4ac5a00640 1 -- 172.21.15.73:0/435331106 wait complete. 2024-04-05T22:21:26.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.759+0000 7f4ac5a00640 1 Processor -- start 2024-04-05T22:21:26.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.759+0000 7f4ac5a00640 1 -- start start 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0070640 0x7f4ac015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac015e580 0x7f4ac0143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00cd820 con 0x7f4ac015a850 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac015b6d0 con 0x7f4ac0070640 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4ac5a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00ccba0 con 0x7f4ac015e580 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4abf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4abf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38868/0 (socket says 172.21.15.73:38868) 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.760+0000 7f4abf400640 1 -- 172.21.15.73:0/2312879037 learned_addr learned my addr 172.21.15.73:0/2312879037 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 -- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac015e580 msgr2=0x7f4ac0143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac015e580 0x7f4ac0143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 -- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0070640 msgr2=0x7f4ac015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0070640 0x7f4ac015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 -- 172.21.15.73:0/2312879037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00094d0 con 0x7f4ac015a850 2024-04-05T22:21:26.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4abf400640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b420 tx=0x7f4aa800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8011830 con 0x7f4ac015a850 2024-04-05T22:21:26.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8011990 con 0x7f4ac015a850 2024-04-05T22:21:26.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8011d40 con 0x7f4ac015a850 2024-04-05T22:21:26.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac00cce80 con 0x7f4ac015a850 2024-04-05T22:21:26.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.761+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac00cd440 con 0x7f4ac015a850 2024-04-05T22:21:26.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.763+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac00cd960 con 0x7f4ac015a850 2024-04-05T22:21:26.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.763+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4aa801b460 con 0x7f4ac015a850 2024-04-05T22:21:26.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.764+0000 7f4ab7e00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 0x7f4a9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:26.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.764+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aa8088f20 con 0x7f4ac015a850 2024-04-05T22:21:26.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.764+0000 7f4abfe00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 0x7f4a9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:26.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.765+0000 7f4abfe00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 0x7f4a9c06d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0036000 tx=0x7f4ab0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:26.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.768+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4ac00cd960 con 0x7f4ac015a850 2024-04-05T22:21:26.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.961+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ac00cd960 con 0x7f4ac015a850 2024-04-05T22:21:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.962+0000 7f4ab7e00640 1 -- 172.21.15.73:0/2312879037 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ac00cd960 con 0x7f4ac015a850 2024-04-05T22:21:26.961 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 msgr2=0x7f4a9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 0x7f4a9c06d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0036000 tx=0x7f4ab0030000 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 msgr2=0x7f4ac00cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b420 tx=0x7f4aa800b8e0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 shutdown_connections 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a9c06ac00 0x7f4a9c06d0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac015e580 0x7f4ac0143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ac015a850 0x7f4ac00cdc40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 --2- 172.21.15.73:0/2312879037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0070640 0x7f4ac015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 >> 172.21.15.73:0/2312879037 conn(0x7f4ac0069cb0 msgr2=0x7f4ac006d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 shutdown_connections 2024-04-05T22:21:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:26.964+0000 7f4ac5a00640 1 -- 172.21.15.73:0/2312879037 wait complete. 2024-04-05T22:21:26.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:27.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.193+0000 7f2ca1400640 1 Processor -- start 2024-04-05T22:21:27.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.193+0000 7f2ca1400640 1 -- start start 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2ca1400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c9c15ad30 0x7f2c9c15f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2ca1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c9c06e230 0x7f2c9c06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2ca1400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0cdcf0 con 0x7f2c9c0ceed0 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2ca1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c15b110 con 0x7f2c9c06e230 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2ca1400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0cb300 con 0x7f2c9c15ad30 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.195+0000 7f2c93e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c9c06e230 0x7f2c9c06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c9c06e230 0x7f2c9c06a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38880/0 (socket says 172.21.15.73:38880) 2024-04-05T22:21:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 -- 172.21.15.73:0/1284844603 learned_addr learned my addr 172.21.15.73:0/1284844603 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c9be00640 1 --2- 172.21.15.73:0/1284844603 >> v2:172.21.15.157:3300/0 conn(0x7f2c9c15ad30 0x7f2c9c15f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 -- 172.21.15.73:0/1284844603 >> v2:172.21.15.157:3300/0 conn(0x7f2c9c15ad30 msgr2=0x7f2c9c15f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 --2- 172.21.15.73:0/1284844603 >> v2:172.21.15.157:3300/0 conn(0x7f2c9c15ad30 0x7f2c9c15f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 -- 172.21.15.73:0/1284844603 >> v1:172.21.15.157:6789/0 conn(0x7f2c9c0ceed0 legacy=0x7f2c9c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.196+0000 7f2c93e00640 1 -- 172.21.15.73:0/1284844603 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0cb5e0 con 0x7f2c9c06e230 2024-04-05T22:21:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.197+0000 7f2c93e00640 1 --2- 172.21.15.73:0/1284844603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c06e230 0x7f2c9c06a6c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0720d0 tx=0x7f2c8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a2b6184c389f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.197+0000 7f2c9b400640 1 -- 172.21.15.73:0/1284844603 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84031870 con 0x7f2c9c06e230 2024-04-05T22:21:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.197+0000 7f2c9b400640 1 -- 172.21.15.73:0/1284844603 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c840319d0 con 0x7f2c9c06e230 2024-04-05T22:21:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.197+0000 7f2c9b400640 1 -- 172.21.15.73:0/1284844603 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84031d60 con 0x7f2c9c06e230 2024-04-05T22:21:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 -- 172.21.15.73:0/1284844603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c06e230 msgr2=0x7f2c9c06a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1284844603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c06e230 0x7f2c9c06a6c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0720d0 tx=0x7f2c8402b910 comp rx=0 tx=0).stop 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 -- 172.21.15.73:0/1284844603 shutdown_connections 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1284844603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c06e230 0x7f2c9c06a6c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1284844603 >> v2:172.21.15.157:3300/0 conn(0x7f2c9c15ad30 0x7f2c9c15f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.198+0000 7f2ca1400640 1 -- 172.21.15.73:0/1284844603 >> 172.21.15.73:0/1284844603 conn(0x7f2c9c0ae4e0 msgr2=0x7f2c9c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.199+0000 7f2ca1400640 1 -- 172.21.15.73:0/1284844603 shutdown_connections 2024-04-05T22:21:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.199+0000 7f2ca1400640 1 -- 172.21.15.73:0/1284844603 wait complete. 2024-04-05T22:21:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.201+0000 7f2ca1400640 1 Processor -- start 2024-04-05T22:21:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.201+0000 7f2ca1400640 1 -- start start 2024-04-05T22:21:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9c06e230 0x7f2c9c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c9c0ceed0 0x7f2c9c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c15a180 con 0x7f2c9c15ad30 2024-04-05T22:21:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c155ce0 con 0x7f2c9c06e230 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c9ea0 con 0x7f2c9c0ceed0 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38882/0 (socket says 172.21.15.73:38882) 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 -- 172.21.15.73:0/1903281958 learned_addr learned my addr 172.21.15.73:0/1903281958 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2c93e00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c9c0ceed0 0x7f2c9c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2c9be00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9c06e230 0x7f2c9c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 -- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c9c0ceed0 msgr2=0x7f2c9c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c9c0ceed0 0x7f2c9c0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.202+0000 7f2ca0a00640 1 -- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9c06e230 msgr2=0x7f2c9c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2ca0a00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9c06e230 0x7f2c9c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2ca0a00640 1 -- 172.21.15.73:0/1903281958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c840094d0 con 0x7f2c9c15ad30 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2ca0a00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c780 tx=0x7f2c9400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003d80 con 0x7f2c9c15ad30 2024-04-05T22:21:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9400ce90 con 0x7f2c9c15ad30 2024-04-05T22:21:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940052b0 con 0x7f2c9c15ad30 2024-04-05T22:21:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c0ca3b0 con 0x7f2c9c15ad30 2024-04-05T22:21:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.203+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c155f10 con 0x7f2c9c15ad30 2024-04-05T22:21:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.204+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c04e160 con 0x7f2c9c15ad30 2024-04-05T22:21:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.205+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2c94005410 con 0x7f2c9c15ad30 2024-04-05T22:21:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.206+0000 7f2c98c00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 0x7f2c7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.206+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c94088740 con 0x7f2c9c15ad30 2024-04-05T22:21:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.207+0000 7f2c9be00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 0x7f2c7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.207+0000 7f2c9be00640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 0x7f2c7806d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c006fd0 tx=0x7f2c8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.210+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2c94053920 con 0x7f2c9c15ad30 2024-04-05T22:21:27.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.406+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c9c06d1f0 con 0x7f2c9c15ad30 2024-04-05T22:21:27.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.407+0000 7f2c98c00640 1 -- 172.21.15.73:0/1903281958 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c94056f40 con 0x7f2c9c15ad30 2024-04-05T22:21:27.406 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 msgr2=0x7f2c7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 0x7f2c7806d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c006fd0 tx=0x7f2c8c008040 comp rx=0 tx=0).stop 2024-04-05T22:21:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 msgr2=0x7f2c9c0cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c780 tx=0x7f2c9400cc40 comp rx=0 tx=0).stop 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 shutdown_connections 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2c7806ac00 0x7f2c7806d0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c9c15ad30 0x7f2c9c0cbe90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c9c0ceed0 0x7f2c9c0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 --2- 172.21.15.73:0/1903281958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9c06e230 0x7f2c9c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 >> 172.21.15.73:0/1903281958 conn(0x7f2c9c0ae4e0 msgr2=0x7f2c9c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 shutdown_connections 2024-04-05T22:21:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.410+0000 7f2ca1400640 1 -- 172.21.15.73:0/1903281958 wait complete. 2024-04-05T22:21:27.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:27.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.635+0000 7fefc5000640 1 Processor -- start 2024-04-05T22:21:27.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.635+0000 7fefc5000640 1 -- start start 2024-04-05T22:21:27.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefc00ccd20 0x7fefc006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 0x7fefc015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc006f5b0 con 0x7fefc00ccd20 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc0070410 con 0x7fefc006d0c0 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefc5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0158bb0 con 0x7fefc01587d0 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefbd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefc00ccd20 0x7fefc006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefbd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefc00ccd20 0x7fefc006cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38894/0 (socket says 172.21.15.73:38894) 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.637+0000 7fefbd400640 1 -- 172.21.15.73:0/4273465271 learned_addr learned my addr 172.21.15.73:0/4273465271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbde00640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbe800640 1 --2- 172.21.15.73:0/4273465271 >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 0x7fefc015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbd400640 1 -- 172.21.15.73:0/4273465271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 msgr2=0x7fefc00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbd400640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbd400640 1 -- 172.21.15.73:0/4273465271 >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 msgr2=0x7fefc015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbd400640 1 --2- 172.21.15.73:0/4273465271 >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 0x7fefc015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbd400640 1 -- 172.21.15.73:0/4273465271 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc006f800 con 0x7fefc00ccd20 2024-04-05T22:21:27.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbde00640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.638+0000 7fefbe800640 1 --2- 172.21.15.73:0/4273465271 >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 0x7fefc015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.639+0000 7fefbd400640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc00ccd20 0x7fefc006cb80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefc00ae4b0 tx=0x7fefa802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a0914a48dd1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.639+0000 7fefbca00640 1 -- 172.21.15.73:0/4273465271 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa802ca80 con 0x7fefc00ccd20 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.639+0000 7fefbca00640 1 -- 172.21.15.73:0/4273465271 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa802cbe0 con 0x7fefc00ccd20 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.639+0000 7fefbca00640 1 -- 172.21.15.73:0/4273465271 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80343f0 con 0x7fefc00ccd20 2024-04-05T22:21:27.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 -- 172.21.15.73:0/4273465271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc00ccd20 msgr2=0x7fefc006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc00ccd20 0x7fefc006cb80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefc00ae4b0 tx=0x7fefa802bee0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 -- 172.21.15.73:0/4273465271 shutdown_connections 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc00ccd20 0x7fefc006cb80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 --2- 172.21.15.73:0/4273465271 >> v2:172.21.15.157:3300/0 conn(0x7fefc006d0c0 0x7fefc015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 --2- 172.21.15.73:0/4273465271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 -- 172.21.15.73:0/4273465271 >> 172.21.15.73:0/4273465271 conn(0x7fefc00ce8e0 msgr2=0x7fefc00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:27.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 -- 172.21.15.73:0/4273465271 shutdown_connections 2024-04-05T22:21:27.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.640+0000 7fefc5000640 1 -- 172.21.15.73:0/4273465271 wait complete. 2024-04-05T22:21:27.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.642+0000 7fefc5000640 1 Processor -- start 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.642+0000 7fefc5000640 1 -- start start 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc0156930 0x7fefc00b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01595e0 con 0x7fefc006d0c0 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01562f0 con 0x7fefc0156930 2024-04-05T22:21:27.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefc5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00ce070 con 0x7fefc01587d0 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38898/0 (socket says 172.21.15.73:38898) 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 -- 172.21.15.73:0/2015193097 learned_addr learned my addr 172.21.15.73:0/2015193097 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbd400640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 -- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 msgr2=0x7fefc0155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc0155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 -- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc0156930 msgr2=0x7fefc00b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc0156930 0x7fefc00b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.643+0000 7fefbde00640 1 -- 172.21.15.73:0/2015193097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa8009b30 con 0x7fefc006d0c0 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefbde00640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d7d0 tx=0x7fefb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0004030 con 0x7fefc006d0c0 2024-04-05T22:21:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb0004190 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0004520 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc01564e0 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.644+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc0159200 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.645+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc0066af0 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.646+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fefb00096a0 con 0x7fefc006d0c0 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.647+0000 7fefb6000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 0x7fef9c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:27.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.647+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefb00894a0 con 0x7fefc006d0c0 2024-04-05T22:21:27.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.647+0000 7fefbd400640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 0x7fef9c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:27.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.648+0000 7fefbd400640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 0x7fef9c06d0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035000 tx=0x7fefa8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:27.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.651+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fefb0054670 con 0x7fefc006d0c0 2024-04-05T22:21:27.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.879+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefc0158d50 con 0x7fefc006d0c0 2024-04-05T22:21:27.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.880+0000 7fefb6000640 1 -- 172.21.15.73:0/2015193097 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7fefb0057c90 con 0x7fefc006d0c0 2024-04-05T22:21:27.880 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:27.880 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:21:27.880 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:21:27.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 msgr2=0x7fef9c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 0x7fef9c06d0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035000 tx=0x7fefa8030000 comp rx=0 tx=0).stop 2024-04-05T22:21:27.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 msgr2=0x7fefc0154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d7d0 tx=0x7fefb000dc90 comp rx=0 tx=0).stop 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 shutdown_connections 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9c06ac30 0x7fef9c06d0f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc0156930 0x7fefc00b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc01587d0 0x7fefc0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 --2- 172.21.15.73:0/2015193097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefc006d0c0 0x7fefc0154d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:27.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.882+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 >> 172.21.15.73:0/2015193097 conn(0x7fefc00ce8e0 msgr2=0x7fefc015b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:27.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.883+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 shutdown_connections 2024-04-05T22:21:27.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:27.883+0000 7fefc5000640 1 -- 172.21.15.73:0/2015193097 wait complete. 2024-04-05T22:21:27.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:28.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.108+0000 7f942be00640 1 Processor -- start 2024-04-05T22:21:28.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.108+0000 7f942be00640 1 -- start start 2024-04-05T22:21:28.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.109+0000 7f942be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f942c0b2b90 0x7f942c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.109+0000 7f942be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f942c0b1b50 0x7f942c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.109+0000 7f942be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c06d3e0 con 0x7f942c0b20a0 2024-04-05T22:21:28.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c06d610 con 0x7f942c0b1b50 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c070230 con 0x7f942c0b2b90 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f942c0b1b50 0x7f942c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f942c0b20a0 0x7f942c0b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37686/0 (socket says 172.21.15.73:37686) 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942aa00640 1 -- 172.21.15.73:0/745311851 learned_addr learned my addr 172.21.15.73:0/745311851 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942a000640 1 --2- 172.21.15.73:0/745311851 >> v2:172.21.15.157:3300/0 conn(0x7f942c0b2b90 0x7f942c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942b400640 1 -- 172.21.15.73:0/745311851 >> v2:172.21.15.157:3300/0 conn(0x7f942c0b2b90 msgr2=0x7f942c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:28.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942b400640 1 --2- 172.21.15.73:0/745311851 >> v2:172.21.15.157:3300/0 conn(0x7f942c0b2b90 0x7f942c070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942b400640 1 -- 172.21.15.73:0/745311851 >> v1:172.21.15.73:6789/0 conn(0x7f942c0b20a0 legacy=0x7f942c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f9429600640 1 -- 172.21.15.73:0/745311851 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854289997 0 0) 0x7f942c06d3e0 con 0x7f942c0b20a0 2024-04-05T22:21:28.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.110+0000 7f942b400640 1 -- 172.21.15.73:0/745311851 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c070510 con 0x7f942c0b1b50 2024-04-05T22:21:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.112+0000 7f942b400640 1 --2- 172.21.15.73:0/745311851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b1b50 0x7f942c06cea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f942c06f380 tx=0x7f942402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ea0e34aeee9b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.112+0000 7f9429600640 1 -- 172.21.15.73:0/745311851 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942402cb60 con 0x7f942c0b1b50 2024-04-05T22:21:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.112+0000 7f9429600640 1 -- 172.21.15.73:0/745311851 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942402ccc0 con 0x7f942c0b1b50 2024-04-05T22:21:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.112+0000 7f9429600640 1 -- 172.21.15.73:0/745311851 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424034660 con 0x7f942c0b1b50 2024-04-05T22:21:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 -- 172.21.15.73:0/745311851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b1b50 msgr2=0x7f942c06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 --2- 172.21.15.73:0/745311851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b1b50 0x7f942c06cea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f942c06f380 tx=0x7f942402bdb0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 -- 172.21.15.73:0/745311851 shutdown_connections 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 --2- 172.21.15.73:0/745311851 >> v2:172.21.15.157:3300/0 conn(0x7f942c0b2b90 0x7f942c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 --2- 172.21.15.73:0/745311851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b1b50 0x7f942c06cea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 -- 172.21.15.73:0/745311851 >> 172.21.15.73:0/745311851 conn(0x7f942c0b06c0 msgr2=0x7f942c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:28.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 -- 172.21.15.73:0/745311851 shutdown_connections 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.113+0000 7f942be00640 1 -- 172.21.15.73:0/745311851 wait complete. 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.114+0000 7f942be00640 1 Processor -- start 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.115+0000 7f942be00640 1 -- start start 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.115+0000 7f942be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c0b1b50 0x7f942c14ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f942c0b2b90 0x7f942c145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c153f80 con 0x7f942c0b20a0 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c154b00 con 0x7f942c0b1b50 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c14bee0 con 0x7f942c0b2b90 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38920/0 (socket says 172.21.15.73:38920) 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 -- 172.21.15.73:0/2097128950 learned_addr learned my addr 172.21.15.73:0/2097128950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 -- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f942c0b2b90 msgr2=0x7f942c145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f942c0b2b90 0x7f942c145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942aa00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c0b1b50 0x7f942c14ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 -- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c0b1b50 msgr2=0x7f942c14ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c0b1b50 0x7f942c14ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.116+0000 7f942a000640 1 -- 172.21.15.73:0/2097128950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009a70 con 0x7f942c0b20a0 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f942a000640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f941800c7f0 tx=0x7f941800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418004220 con 0x7f942c0b20a0 2024-04-05T22:21:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94180043a0 con 0x7f942c0b20a0 2024-04-05T22:21:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418007860 con 0x7f942c0b20a0 2024-04-05T22:21:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c1541d0 con 0x7f942c0b20a0 2024-04-05T22:21:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.117+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c154790 con 0x7f942c0b20a0 2024-04-05T22:21:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.118+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c154c70 con 0x7f942c0b20a0 2024-04-05T22:21:28.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.119+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9418007bd0 con 0x7f942c0b20a0 2024-04-05T22:21:28.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.120+0000 7f9422a00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 0x7f940c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.120+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94180892c0 con 0x7f942c0b20a0 2024-04-05T22:21:28.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.121+0000 7f942aa00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 0x7f940c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.121+0000 7f942aa00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 0x7f940c06d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f941c007fd0 tx=0x7f941c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.125+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9418054530 con 0x7f942c0b20a0 2024-04-05T22:21:28.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.317+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f942c04e160 con 0x7f942c0b20a0 2024-04-05T22:21:28.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.318+0000 7f9422a00640 1 -- 172.21.15.73:0/2097128950 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f9418057b50 con 0x7f942c0b20a0 2024-04-05T22:21:28.318 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:28.319 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:28.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.320+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 msgr2=0x7f940c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.320+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 0x7f940c06d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f941c007fd0 tx=0x7f941c009040 comp rx=0 tx=0).stop 2024-04-05T22:21:28.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.320+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 msgr2=0x7f942c14b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.320+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f941800c7f0 tx=0x7f941800ccb0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 shutdown_connections 2024-04-05T22:21:28.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f940c06ac00 0x7f940c06d0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f942c0b2b90 0x7f942c145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0b20a0 0x7f942c14b9a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 --2- 172.21.15.73:0/2097128950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c0b1b50 0x7f942c14ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 >> 172.21.15.73:0/2097128950 conn(0x7f942c0b06c0 msgr2=0x7f942c0ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:28.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 shutdown_connections 2024-04-05T22:21:28.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.321+0000 7f942be00640 1 -- 172.21.15.73:0/2097128950 wait complete. 2024-04-05T22:21:28.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:28.340 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.554+0000 7fefbb800640 1 Processor -- start 2024-04-05T22:21:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.554+0000 7fefbb800640 1 -- start start 2024-04-05T22:21:28.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefbb800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefb415aa70 0x7fefb4070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefbb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 0x7fefb406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefbb800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb415ae50 con 0x7fefb40714a0 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefbb800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb4071930 con 0x7fefb415aa70 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefbb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb415e3b0 con 0x7fefb415dd90 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefb415aa70 0x7fefb4070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefb415aa70 0x7fefb4070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38930/0 (socket says 172.21.15.73:38930) 2024-04-05T22:21:28.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 -- 172.21.15.73:0/3788378242 learned_addr learned my addr 172.21.15.73:0/3788378242 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb9800640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 0x7fefb406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 -- 172.21.15.73:0/3788378242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 msgr2=0x7fefb406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 0x7fefb406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.556+0000 7fefb8e00640 1 -- 172.21.15.73:0/3788378242 >> v1:172.21.15.73:6789/0 conn(0x7fefb40714a0 legacy=0x7fefb415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.557+0000 7fefb8e00640 1 -- 172.21.15.73:0/3788378242 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40ac9e0 con 0x7fefb415aa70 2024-04-05T22:21:28.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.557+0000 7fefb3400640 1 -- 172.21.15.73:0/3788378242 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612111236 0 0) 0x7fefb415ae50 con 0x7fefb40714a0 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.557+0000 7fefb9800640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 0x7fefb406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefb8e00640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415aa70 0x7fefb4070f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009870 tx=0x7fefa802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9422bb90983ad68 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefb3400640 1 -- 172.21.15.73:0/3788378242 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8030840 con 0x7fefb415aa70 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefb3400640 1 -- 172.21.15.73:0/3788378242 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa80309a0 con 0x7fefb415aa70 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefb3400640 1 -- 172.21.15.73:0/3788378242 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8030d30 con 0x7fefb415aa70 2024-04-05T22:21:28.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefbb800640 1 -- 172.21.15.73:0/3788378242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415aa70 msgr2=0x7fefb4070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.558+0000 7fefbb800640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415aa70 0x7fefb4070f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009870 tx=0x7fefa802b980 comp rx=0 tx=0).stop 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 -- 172.21.15.73:0/3788378242 shutdown_connections 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415dd90 0x7fefb406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 --2- 172.21.15.73:0/3788378242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415aa70 0x7fefb4070f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 -- 172.21.15.73:0/3788378242 >> 172.21.15.73:0/3788378242 conn(0x7fefb4069cb0 msgr2=0x7fefb406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:28.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 -- 172.21.15.73:0/3788378242 shutdown_connections 2024-04-05T22:21:28.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.559+0000 7fefbb800640 1 -- 172.21.15.73:0/3788378242 wait complete. 2024-04-05T22:21:28.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.561+0000 7fefbb800640 1 Processor -- start 2024-04-05T22:21:28.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.561+0000 7fefbb800640 1 -- start start 2024-04-05T22:21:28.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb40714a0 0x7fefb41411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415aa70 0x7fefb41f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4154fc0 con 0x7fefb415dd90 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40ae7e0 con 0x7fefb40714a0 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.562+0000 7fefbb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40aeac0 con 0x7fefb415aa70 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38934/0 (socket says 172.21.15.73:38934) 2024-04-05T22:21:28.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 -- 172.21.15.73:0/2310262253 learned_addr learned my addr 172.21.15.73:0/2310262253 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb3e00640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415aa70 0x7fefb41f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb8e00640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb40714a0 0x7fefb41411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 -- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415aa70 msgr2=0x7fefb41f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415aa70 0x7fefb41f1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 -- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb40714a0 msgr2=0x7fefb41411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb40714a0 0x7fefb41411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 -- 172.21.15.73:0/2310262253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80094d0 con 0x7fefb415dd90 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.563+0000 7fefb9800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefac00cd50 tx=0x7fefac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.564+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00f680 con 0x7fefb415dd90 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.564+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00f800 con 0x7fefb415dd90 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.564+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00fbb0 con 0x7fefb415dd90 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.564+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb40aeda0 con 0x7fefb415dd90 2024-04-05T22:21:28.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.564+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb4159ff0 con 0x7fefb415dd90 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.565+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb40af2c0 con 0x7fefb415dd90 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.566+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fefac00b040 con 0x7fefb415dd90 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.567+0000 7fefb0c00640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 0x7fef9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.567+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefac089e50 con 0x7fefb415dd90 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.567+0000 7fefb8e00640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 0x7fef9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.567+0000 7fefb8e00640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 0x7fef9406d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefa8036000 tx=0x7fefa803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.571+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fefac054f90 con 0x7fefb415dd90 2024-04-05T22:21:28.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.764+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefb4068160 con 0x7fefb415dd90 2024-04-05T22:21:28.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.764+0000 7fefb0c00640 1 -- 172.21.15.73:0/2310262253 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefac01d020 con 0x7fefb415dd90 2024-04-05T22:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 msgr2=0x7fef9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 0x7fef9406d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefa8036000 tx=0x7fefa803b040 comp rx=0 tx=0).stop 2024-04-05T22:21:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 msgr2=0x7fefb4154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefac00cd50 tx=0x7fefac00a650 comp rx=0 tx=0).stop 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 shutdown_connections 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fef9406ac00 0x7fef9406d0c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb415dd90 0x7fefb4154570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb415aa70 0x7fefb41f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 --2- 172.21.15.73:0/2310262253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb40714a0 0x7fefb41411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 >> 172.21.15.73:0/2310262253 conn(0x7fefb4069cb0 msgr2=0x7fefb415d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:28.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.767+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 shutdown_connections 2024-04-05T22:21:28.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.768+0000 7fefbb800640 1 -- 172.21.15.73:0/2310262253 wait complete. 2024-04-05T22:21:28.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:28.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.993+0000 7efe07200640 1 Processor -- start 2024-04-05T22:21:28.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.993+0000 7efe07200640 1 -- start start 2024-04-05T22:21:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.995+0000 7efe07200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000aee00 0x7efe0015de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.995+0000 7efe07200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe000b46d0 0x7efe00069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.995+0000 7efe07200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000cf850 con 0x7efe000af1e0 2024-04-05T22:21:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.995+0000 7efe07200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0015e430 con 0x7efe000b46d0 2024-04-05T22:21:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.995+0000 7efe07200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000717f0 con 0x7efe000aee00 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe000b46d0 0x7efe00069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe000b46d0 0x7efe00069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38944/0 (socket says 172.21.15.73:38944) 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 -- 172.21.15.73:0/1878423253 learned_addr learned my addr 172.21.15.73:0/1878423253 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efdffe00640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000aee00 0x7efe0015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 -- 172.21.15.73:0/1878423253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000aee00 msgr2=0x7efe0015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000aee00 0x7efe0015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 -- 172.21.15.73:0/1878423253 >> v1:172.21.15.157:6789/0 conn(0x7efe000af1e0 legacy=0x7efe0006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.996+0000 7efe05200640 1 -- 172.21.15.73:0/1878423253 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000b2b70 con 0x7efe000b46d0 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.997+0000 7efe05200640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000b46d0 0x7efe00069cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009870 tx=0x7efdf802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e4be70379512b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.997+0000 7efdfea00640 1 -- 172.21.15.73:0/1878423253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8030840 con 0x7efe000b46d0 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efdfea00640 1 -- 172.21.15.73:0/1878423253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf80309a0 con 0x7efe000b46d0 2024-04-05T22:21:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efdfea00640 1 -- 172.21.15.73:0/1878423253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8030d30 con 0x7efe000b46d0 2024-04-05T22:21:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 -- 172.21.15.73:0/1878423253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000b46d0 msgr2=0x7efe00069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000b46d0 0x7efe00069cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009870 tx=0x7efdf802b980 comp rx=0 tx=0).stop 2024-04-05T22:21:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 -- 172.21.15.73:0/1878423253 shutdown_connections 2024-04-05T22:21:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000b46d0 0x7efe00069cb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 --2- 172.21.15.73:0/1878423253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000aee00 0x7efe0015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 -- 172.21.15.73:0/1878423253 >> 172.21.15.73:0/1878423253 conn(0x7efe0006adc0 msgr2=0x7efe0006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 -- 172.21.15.73:0/1878423253 shutdown_connections 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:28.998+0000 7efe07200640 1 -- 172.21.15.73:0/1878423253 wait complete. 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.000+0000 7efe07200640 1 Processor -- start 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.000+0000 7efe07200640 1 -- start start 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 0x7efe0015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 0x7efe00160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00150100 con 0x7efe000aee00 2024-04-05T22:21:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0014c020 con 0x7efe000af1e0 2024-04-05T22:21:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efe07200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0014f7d0 con 0x7efe000b46d0 2024-04-05T22:21:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efdffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efdffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38950/0 (socket says 172.21.15.73:38950) 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.001+0000 7efdffe00640 1 -- 172.21.15.73:0/2222412465 learned_addr learned my addr 172.21.15.73:0/2222412465 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efe05200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 0x7efe00160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdff400640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 0x7efe0015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 -- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 msgr2=0x7efe00160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 0x7efe00160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 -- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 msgr2=0x7efe0015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 0x7efe0015fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 -- 172.21.15.73:0/2222412465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80094d0 con 0x7efe000aee00 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdff400640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 0x7efe0015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efe05200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 0x7efe00160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdffe00640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007ef0 tx=0x7efdf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0014620 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf0014780 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0014fa40 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.002+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0014b30 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.003+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe001f35b0 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.004+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00068160 con 0x7efe000aee00 2024-04-05T22:21:29.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.005+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efdf0014c90 con 0x7efe000aee00 2024-04-05T22:21:29.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.005+0000 7efdf7e00640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 0x7efddc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.006+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdf0089d20 con 0x7efe000aee00 2024-04-05T22:21:29.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.006+0000 7efdff400640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 0x7efddc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.006+0000 7efdff400640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 0x7efddc06d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe0015d140 tx=0x7efde8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.010+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7efdf0054f90 con 0x7efe000aee00 2024-04-05T22:21:29.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.202+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe000b1580 con 0x7efe000aee00 2024-04-05T22:21:29.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.203+0000 7efdf7e00640 1 -- 172.21.15.73:0/2222412465 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdf00585b0 con 0x7efe000aee00 2024-04-05T22:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 msgr2=0x7efddc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 0x7efddc06d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe0015d140 tx=0x7efde8009040 comp rx=0 tx=0).stop 2024-04-05T22:21:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 msgr2=0x7efe00150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007ef0 tx=0x7efdf000e4b0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 shutdown_connections 2024-04-05T22:21:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efddc06ac00 0x7efddc06d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe000b46d0 0x7efe00160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe000af1e0 0x7efe0015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 --2- 172.21.15.73:0/2222412465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe000aee00 0x7efe00150520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 >> 172.21.15.73:0/2222412465 conn(0x7efe0006adc0 msgr2=0x7efe000cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 shutdown_connections 2024-04-05T22:21:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.206+0000 7efe07200640 1 -- 172.21.15.73:0/2222412465 wait complete. 2024-04-05T22:21:29.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.429+0000 7f6b26800640 1 Processor -- start 2024-04-05T22:21:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.429+0000 7f6b26800640 1 -- start start 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b20157410 0x7f6b200b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b2015af80 0x7f6b200cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 0x7f6b200cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200b1e90 con 0x7f6b2015af80 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b201577f0 con 0x7f6b200cd570 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b26800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200cb630 con 0x7f6b20157410 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b24a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b2015af80 0x7f6b200cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b24a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b2015af80 0x7f6b200cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38958/0 (socket says 172.21.15.73:38958) 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b24a00640 1 -- 172.21.15.73:0/618914844 learned_addr learned my addr 172.21.15.73:0/618914844 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.431+0000 7f6b25400640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b20157410 0x7f6b200b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b25e00640 1 --2- 172.21.15.73:0/618914844 >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 0x7f6b200cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b24a00640 1 -- 172.21.15.73:0/618914844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b20157410 msgr2=0x7f6b200b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b24a00640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b20157410 0x7f6b200b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b24a00640 1 -- 172.21.15.73:0/618914844 >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 msgr2=0x7f6b200cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b24a00640 1 --2- 172.21.15.73:0/618914844 >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 0x7f6b200cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b24a00640 1 -- 172.21.15.73:0/618914844 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200b20a0 con 0x7f6b2015af80 2024-04-05T22:21:29.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.432+0000 7f6b25e00640 1 --2- 172.21.15.73:0/618914844 >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 0x7f6b200cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:29.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.433+0000 7f6b24a00640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200cd030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006d990 tx=0x7f6b0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe5e7047dd50c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.433+0000 7f6b1fe00640 1 -- 172.21.15.73:0/618914844 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c031870 con 0x7f6b2015af80 2024-04-05T22:21:29.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.433+0000 7f6b1fe00640 1 -- 172.21.15.73:0/618914844 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c0319d0 con 0x7f6b2015af80 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.433+0000 7f6b1fe00640 1 -- 172.21.15.73:0/618914844 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c031d60 con 0x7f6b2015af80 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 -- 172.21.15.73:0/618914844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 msgr2=0x7f6b200cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200cd030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006d990 tx=0x7f6b0c02b910 comp rx=0 tx=0).stop 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 -- 172.21.15.73:0/618914844 shutdown_connections 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200cd030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 --2- 172.21.15.73:0/618914844 >> v2:172.21.15.157:3300/0 conn(0x7f6b200cd570 0x7f6b200cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 --2- 172.21.15.73:0/618914844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b20157410 0x7f6b200b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 -- 172.21.15.73:0/618914844 >> 172.21.15.73:0/618914844 conn(0x7f6b200b0830 msgr2=0x7f6b200b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 -- 172.21.15.73:0/618914844 shutdown_connections 2024-04-05T22:21:29.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.434+0000 7f6b26800640 1 -- 172.21.15.73:0/618914844 wait complete. 2024-04-05T22:21:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.436+0000 7f6b26800640 1 Processor -- start 2024-04-05T22:21:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.436+0000 7f6b26800640 1 -- start start 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b200cd570 0x7f6b200a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b20157410 0x7f6b2015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201f2660 con 0x7f6b2015af80 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201f2940 con 0x7f6b20157410 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b26800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201f2c20 con 0x7f6b200cd570 2024-04-05T22:21:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38964/0 (socket says 172.21.15.73:38964) 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 -- 172.21.15.73:0/1936800504 learned_addr learned my addr 172.21.15.73:0/1936800504 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 -- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b200cd570 msgr2=0x7f6b200a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b200cd570 0x7f6b200a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 -- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b20157410 msgr2=0x7f6b2015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b20157410 0x7f6b2015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.437+0000 7f6b25e00640 1 -- 172.21.15.73:0/1936800504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0094d0 con 0x7f6b2015af80 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b25e00640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800cef0 tx=0x7f6b1800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800d840 con 0x7f6b2015af80 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1800d9c0 con 0x7f6b2015af80 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800dd70 con 0x7f6b2015af80 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b201f2f00 con 0x7f6b2015af80 2024-04-05T22:21:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.438+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2015ab70 con 0x7f6b2015af80 2024-04-05T22:21:29.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.439+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20158c70 con 0x7f6b2015af80 2024-04-05T22:21:29.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.440+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b1801e3b0 con 0x7f6b2015af80 2024-04-05T22:21:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.441+0000 7f6b1d600640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.441+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b18089ed0 con 0x7f6b2015af80 2024-04-05T22:21:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.441+0000 7f6b25400640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.442+0000 7f6b25400640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 0x7f6b0006d0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b200cacd0 tx=0x7f6b14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.445+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6b20158c70 con 0x7f6b2015af80 2024-04-05T22:21:29.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.643+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b20158c70 con 0x7f6b2015af80 2024-04-05T22:21:29.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.643+0000 7f6b1d600640 1 -- 172.21.15.73:0/1936800504 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b20158c70 con 0x7f6b2015af80 2024-04-05T22:21:29.643 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:29.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 msgr2=0x7f6b0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 0x7f6b0006d0c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b200cacd0 tx=0x7f6b14008040 comp rx=0 tx=0).stop 2024-04-05T22:21:29.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 msgr2=0x7f6b200b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800cef0 tx=0x7f6b1800a710 comp rx=0 tx=0).stop 2024-04-05T22:21:29.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 shutdown_connections 2024-04-05T22:21:29.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2015af80 0x7f6b200b3090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b20157410 0x7f6b2015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 --2- 172.21.15.73:0/1936800504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b200cd570 0x7f6b200a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.646+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 >> 172.21.15.73:0/1936800504 conn(0x7f6b200b0830 msgr2=0x7f6b2015b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.647+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 shutdown_connections 2024-04-05T22:21:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.647+0000 7f6b26800640 1 -- 172.21.15.73:0/1936800504 wait complete. 2024-04-05T22:21:29.664 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:29.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.872+0000 7ff9f2800640 1 Processor -- start 2024-04-05T22:21:29.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.872+0000 7ff9f2800640 1 -- start start 2024-04-05T22:21:29.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9f2800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9ec0d1930 0x7ff9ec0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 0x7ff9ec0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9f2800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0ceee0 con 0x7ff9ec15cf70 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9f2800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ce5b0 con 0x7ff9ec0d1930 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9f2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ce890 con 0x7ff9ec0d0bd0 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9eb400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff9ec15cf70 0x7ff9ec0d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37704/0 (socket says 172.21.15.73:37704) 2024-04-05T22:21:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9eb400640 1 -- 172.21.15.73:0/1930074440 learned_addr learned my addr 172.21.15.73:0/1930074440 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:29.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9ebe00640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 0x7ff9ec0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/1930074440 >> v2:172.21.15.157:3300/0 conn(0x7ff9ec0d1930 0x7ff9ec0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.874+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532572873 0 0) 0x7ff9ec0ceee0 con 0x7ff9ec15cf70 2024-04-05T22:21:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d0003520 con 0x7ff9ec15cf70 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2356346679 0 0) 0x7ff9d0003520 con 0x7ff9ec15cf70 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 msgr2=0x7ff9ec0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 0x7ff9ec0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 >> v2:172.21.15.157:3300/0 conn(0x7ff9ec0d1930 msgr2=0x7ff9ec0d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 --2- 172.21.15.73:0/1930074440 >> v2:172.21.15.157:3300/0 conn(0x7ff9ec0d1930 0x7ff9ec0d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0ceb70 con 0x7ff9ec15cf70 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff9dc0019f0 con 0x7ff9ec15cf70 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 >> v1:172.21.15.73:6789/0 conn(0x7ff9ec15cf70 legacy=0x7ff9ec0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d0003970 0x7ff9d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 0x7ff9d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ce5b0 con 0x7ff9d00047c0 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ce890 con 0x7ff9d0003970 2024-04-05T22:21:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d000b980 con 0x7ff9ec0d0bd0 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.875+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.876+0000 7ff9eaa00640 1 -- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 msgr2=0x7ff9d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.876+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0d0bd0 0x7ff9d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.876+0000 7ff9eaa00640 1 -- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d0003970 msgr2=0x7ff9d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.876+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d0003970 0x7ff9d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.876+0000 7ff9eaa00640 1 -- 172.21.15.73:0/1930074440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0003520 con 0x7ff9d00047c0 2024-04-05T22:21:29.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.877+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9d0004c40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009af0 tx=0x7ff9d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7759f2e4074ca493 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.877+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d802c710 con 0x7ff9d00047c0 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.877+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d802c870 con 0x7ff9d00047c0 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.877+0000 7ff9ea000640 1 -- 172.21.15.73:0/1930074440 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d802cc00 con 0x7ff9d00047c0 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 -- 172.21.15.73:0/1930074440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 msgr2=0x7ff9d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 --2- 172.21.15.73:0/1930074440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9d0004c40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009af0 tx=0x7ff9d00057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9ebe00640 1 -- 172.21.15.73:0/1930074440 reap_dead start 2024-04-05T22:21:29.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 -- 172.21.15.73:0/1930074440 shutdown_connections 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 -- 172.21.15.73:0/1930074440 >> 172.21.15.73:0/1930074440 conn(0x7ff9ec0d2df0 msgr2=0x7ff9ec0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 -- 172.21.15.73:0/1930074440 shutdown_connections 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.878+0000 7ff9f2800640 1 -- 172.21.15.73:0/1930074440 wait complete. 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.880+0000 7ff9f2800640 1 Processor -- start 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.880+0000 7ff9f2800640 1 -- start start 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.880+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ec0b5c70 0x7ff9ec0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0c93d0 0x7ff9ec0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9f2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0c9880 con 0x7ff9d00047c0 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9f2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0c9fd0 con 0x7ff9ec0b5c70 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9f2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ca2b0 con 0x7ff9ec0c93d0 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38968/0 (socket says 172.21.15.73:38968) 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 -- 172.21.15.73:0/3881278349 learned_addr learned my addr 172.21.15.73:0/3881278349 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 -- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0c93d0 msgr2=0x7ff9ec0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9ebe00640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0c93d0 0x7ff9ec0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0c93d0 0x7ff9ec0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ec0b5c70 0x7ff9ec0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 -- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ec0b5c70 msgr2=0x7ff9ec0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ec0b5c70 0x7ff9ec0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.881+0000 7ff9eb400640 1 -- 172.21.15.73:0/3881278349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0ca590 con 0x7ff9d00047c0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9eb400640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00cb10 tx=0x7ff9dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00a860 con 0x7ff9d00047c0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc00a9c0 con 0x7ff9d00047c0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00ad70 con 0x7ff9d00047c0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec0ca870 con 0x7ff9d00047c0 2024-04-05T22:21:29.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.882+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0c80f0 con 0x7ff9d00047c0 2024-04-05T22:21:29.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.883+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec0caab0 con 0x7ff9d00047c0 2024-04-05T22:21:29.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.884+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9dc0070d0 con 0x7ff9d00047c0 2024-04-05T22:21:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.885+0000 7ff9e3400640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 0x7ff9cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.885+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9dc088d50 con 0x7ff9d00047c0 2024-04-05T22:21:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.885+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 0x7ff9cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.886+0000 7ff9eaa00640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 0x7ff9cc06d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8036000 tx=0x7ff9d8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:29.889+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff9ec0caab0 con 0x7ff9d00047c0 2024-04-05T22:21:30.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.119+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9ec0caab0 con 0x7ff9d00047c0 2024-04-05T22:21:30.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.120+0000 7ff9e3400640 1 -- 172.21.15.73:0/3881278349 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7ff9ec0caab0 con 0x7ff9d00047c0 2024-04-05T22:21:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:30.120 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:21:30.120 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:21:30.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 msgr2=0x7ff9cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 0x7ff9cc06d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8036000 tx=0x7ff9d8030000 comp rx=0 tx=0).stop 2024-04-05T22:21:30.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 msgr2=0x7ff9ec15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00cb10 tx=0x7ff9dc00a060 comp rx=0 tx=0).stop 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 shutdown_connections 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9cc06ac00 0x7ff9cc06d0c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9ec0c93d0 0x7ff9ec0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9ec0b5c70 0x7ff9ec0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 --2- 172.21.15.73:0/3881278349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d00047c0 0x7ff9ec15b410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 >> 172.21.15.73:0/3881278349 conn(0x7ff9ec0d2df0 msgr2=0x7ff9ec0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 shutdown_connections 2024-04-05T22:21:30.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.123+0000 7ff9f2800640 1 -- 172.21.15.73:0/3881278349 wait complete. 2024-04-05T22:21:30.142 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:30.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.351+0000 7fbca2400640 1 Processor -- start 2024-04-05T22:21:30.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.351+0000 7fbca2400640 1 -- start start 2024-04-05T22:21:30.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbca2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc9c15cf50 0x7fbc9c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbca2400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c0cb950 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbca2400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c0cdce0 con 0x7fbc9c0cd7a0 2024-04-05T22:21:30.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbca2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c0ce990 con 0x7fbc9c15cf50 2024-04-05T22:21:30.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbc9aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbc9c0d09b0 0x7fbc9c0cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37710/0 (socket says 172.21.15.73:37710) 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbc9aa00640 1 -- 172.21.15.73:0/2498140050 learned_addr learned my addr 172.21.15.73:0/2498140050 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214288290 0 0) 0x7fbc9c0cb950 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc7c003520 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.353+0000 7fbc9b400640 1 --2- 172.21.15.73:0/2498140050 >> v2:172.21.15.157:3300/0 conn(0x7fbc9c15cf50 0x7fbc9c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2387477066 0 0) 0x7fbc7c003520 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 >> v2:172.21.15.157:3300/0 conn(0x7fbc9c15cf50 msgr2=0x7fbc9c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 --2- 172.21.15.73:0/2498140050 >> v2:172.21.15.157:3300/0 conn(0x7fbc9c15cf50 0x7fbc9c06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 >> v1:172.21.15.157:6789/0 conn(0x7fbc9c0cd7a0 legacy=0x7fbc9c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0cdfc0 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbc840019f0 con 0x7fbc9c0d09b0 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 >> v1:172.21.15.73:6789/0 conn(0x7fbc9c0d09b0 legacy=0x7fbc9c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c003970 0x7fbc7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc7c0047c0 0x7fbc7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 0x7fbc7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0ce990 con 0x7fbc7c005180 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c00c020 con 0x7fbc7c0047c0 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c00c1e0 con 0x7fbc7c003970 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.354+0000 7fbc9be00640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 0x7fbc7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9b400640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c003970 0x7fbc7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9aa00640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc7c0047c0 0x7fbc7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9be00640 1 -- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c003970 msgr2=0x7fbc7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9be00640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c003970 0x7fbc7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9be00640 1 -- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc7c0047c0 msgr2=0x7fbc7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9be00640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc7c0047c0 0x7fbc7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.355+0000 7fbc9be00640 1 -- 172.21.15.73:0/2498140050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c003520 con 0x7fbc7c005180 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.356+0000 7fbc9be00640 1 -- 172.21.15.73:0/2498140050 reap_dead start 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.356+0000 7fbc9be00640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 0x7fbc7c00b980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001720 tx=0x7fbc7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b9cd0b7da59077 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.356+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940052d0 con 0x7fbc7c005180 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.356+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9402e050 con 0x7fbc7c005180 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.356+0000 7fbc9a000640 1 -- 172.21.15.73:0/2498140050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940035f0 con 0x7fbc7c005180 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 -- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 msgr2=0x7fbc7c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 0x7fbc7c00b980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001720 tx=0x7fbc7c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 -- 172.21.15.73:0/2498140050 shutdown_connections 2024-04-05T22:21:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 --2- 172.21.15.73:0/2498140050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc7c005180 0x7fbc7c00b980 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 -- 172.21.15.73:0/2498140050 >> 172.21.15.73:0/2498140050 conn(0x7fbc9c099a60 msgr2=0x7fbc9c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:30.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 -- 172.21.15.73:0/2498140050 shutdown_connections 2024-04-05T22:21:30.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.357+0000 7fbca2400640 1 -- 172.21.15.73:0/2498140050 wait complete. 2024-04-05T22:21:30.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.358+0000 7fbca2400640 1 Processor -- start 2024-04-05T22:21:30.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 -- start start 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c005180 0x7fbc9c0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 0x7fbc9c0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c9300 con 0x7fbc9c157190 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0ca220 con 0x7fbc9c0ca6b0 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.359+0000 7fbca2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c98f0 con 0x7fbc7c005180 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38986/0 (socket says 172.21.15.73:38986) 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 -- 172.21.15.73:0/122208726 learned_addr learned my addr 172.21.15.73:0/122208726 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9aa00640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 0x7fbc9c0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 -- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c005180 msgr2=0x7fbc9c0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c005180 0x7fbc9c0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 -- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 msgr2=0x7fbc9c0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 0x7fbc9c0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9be00640 1 -- 172.21.15.73:0/122208726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0c9bd0 con 0x7fbc9c157190 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.360+0000 7fbc9aa00640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 0x7fbc9c0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbc9be00640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc9402bcb0 tx=0x7fbc94001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94007a30 con 0x7fbc9c157190 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9403d400 con 0x7fbc9c157190 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c0c9eb0 con 0x7fbc9c157190 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9403d790 con 0x7fbc9c157190 2024-04-05T22:21:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.361+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c06c440 con 0x7fbc9c157190 2024-04-05T22:21:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.362+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c0ca040 con 0x7fbc9c157190 2024-04-05T22:21:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.363+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbc94032400 con 0x7fbc9c157190 2024-04-05T22:21:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.364+0000 7fbc93400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 0x7fbc7806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.364+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc940a9f30 con 0x7fbc9c157190 2024-04-05T22:21:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.364+0000 7fbc9b400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 0x7fbc7806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.365+0000 7fbc9b400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 0x7fbc7806d0f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009890 tx=0x7fbc8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.369+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbc94075120 con 0x7fbc9c157190 2024-04-05T22:21:30.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.568+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc9c0c9640 con 0x7fbc9c157190 2024-04-05T22:21:30.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.570+0000 7fbc93400640 1 -- 172.21.15.73:0/122208726 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fbc94042070 con 0x7fbc9c157190 2024-04-05T22:21:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:30.570 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:30.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 msgr2=0x7fbc7806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 0x7fbc7806d0f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009890 tx=0x7fbc8c008040 comp rx=0 tx=0).stop 2024-04-05T22:21:30.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 msgr2=0x7fbc9c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc9402bcb0 tx=0x7fbc94001120 comp rx=0 tx=0).stop 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 shutdown_connections 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbc7806ac30 0x7fbc7806d0f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc9c157190 0x7fbc9c06bd20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc9c0ca6b0 0x7fbc9c0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 --2- 172.21.15.73:0/122208726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc7c005180 0x7fbc9c0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 >> 172.21.15.73:0/122208726 conn(0x7fbc9c099a60 msgr2=0x7fbc9c099e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 shutdown_connections 2024-04-05T22:21:30.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.573+0000 7fbca2400640 1 -- 172.21.15.73:0/122208726 wait complete. 2024-04-05T22:21:30.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:21:30.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.793+0000 7f061be00640 1 Processor -- start 2024-04-05T22:21:30.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.794+0000 7f061be00640 1 -- start start 2024-04-05T22:21:30.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f061be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f06140d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f061be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06140d0bd0 0x7f06140b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f061be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140ceee0 con 0x7f061415cf70 2024-04-05T22:21:30.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f061be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06140ce5b0 con 0x7f06140d0bd0 2024-04-05T22:21:30.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f061be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06140ce890 con 0x7f06140d1930 2024-04-05T22:21:30.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f0619400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f061415cf70 0x7f06140d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37726/0 (socket says 172.21.15.73:37726) 2024-04-05T22:21:30.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.795+0000 7f0619400640 1 -- 172.21.15.73:0/3803406037 learned_addr learned my addr 172.21.15.73:0/3803406037 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0618a00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f06140d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0619e00640 1 --2- 172.21.15.73:0/3803406037 >> v2:172.21.15.157:3300/0 conn(0x7f06140d0bd0 0x7f06140b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386863132 0 0) 0x7f06140ceee0 con 0x7f061415cf70 2024-04-05T22:21:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc003520 con 0x7f061415cf70 2024-04-05T22:21:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31219917 0 0) 0x7f05fc003520 con 0x7f061415cf70 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 msgr2=0x7f06140d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f06140d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 >> v2:172.21.15.157:3300/0 conn(0x7f06140d0bd0 msgr2=0x7f06140b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 --2- 172.21.15.73:0/3803406037 >> v2:172.21.15.157:3300/0 conn(0x7f06140d0bd0 0x7f06140b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140ceb70 con 0x7f061415cf70 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0608001670 con 0x7f061415cf70 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 >> v1:172.21.15.73:6789/0 conn(0x7f061415cf70 legacy=0x7f06140d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f05fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05fc0047c0 0x7f05fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f05fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140ce5b0 con 0x7f05fc003970 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140ce890 con 0x7f05fc0047c0 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc00b980 con 0x7f06140d1930 2024-04-05T22:21:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.796+0000 7f0619400640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f05fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0618a00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05fc0047c0 0x7f05fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619e00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f05fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619400640 1 -- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 msgr2=0x7f05fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619400640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06140d1930 0x7f05fc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619400640 1 -- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05fc0047c0 msgr2=0x7f05fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619400640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05fc0047c0 0x7f05fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.797+0000 7f0619400640 1 -- 172.21.15.73:0/3803406037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc003520 con 0x7f05fc003970 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.798+0000 7f0619400640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f05fc003d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f060800c270 tx=0x7f05fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dc1140ec73fa30 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.798+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060802cae0 con 0x7f05fc003970 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.798+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060802cc40 con 0x7f05fc003970 2024-04-05T22:21:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.798+0000 7f0613e00640 1 -- 172.21.15.73:0/3803406037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080058d0 con 0x7f05fc003970 2024-04-05T22:21:30.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 -- 172.21.15.73:0/3803406037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 msgr2=0x7f05fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 --2- 172.21.15.73:0/3803406037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f05fc003d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f060800c270 tx=0x7f05fc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f0619e00640 1 -- 172.21.15.73:0/3803406037 reap_dead start 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 -- 172.21.15.73:0/3803406037 shutdown_connections 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 -- 172.21.15.73:0/3803406037 >> 172.21.15.73:0/3803406037 conn(0x7f06140d2df0 msgr2=0x7f06140d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 -- 172.21.15.73:0/3803406037 shutdown_connections 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.799+0000 7f061be00640 1 -- 172.21.15.73:0/3803406037 wait complete. 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.801+0000 7f061be00640 1 Processor -- start 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.801+0000 7f061be00640 1 -- start start 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06141f5eb0 0x7f06141f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 0x7f06140cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f6e30 con 0x7f05fc003970 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f6870 con 0x7f06141f5eb0 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f061be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f6b50 con 0x7f06141f2560 2024-04-05T22:21:30.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f0619400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f0619400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38994/0 (socket says 172.21.15.73:38994) 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f0619400640 1 -- 172.21.15.73:0/1213287194 learned_addr learned my addr 172.21.15.73:0/1213287194 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.802+0000 7f0618a00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06141f5eb0 0x7f06141f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619e00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 0x7f06140cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 -- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 msgr2=0x7f06140cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 0x7f06140cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 -- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06141f5eb0 msgr2=0x7f06141f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06141f5eb0 0x7f06141f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 -- 172.21.15.73:0/1213287194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614159430 con 0x7f05fc003970 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619e00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 0x7f06140cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0619400640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f060800c210 tx=0x7f0608001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608039070 con 0x7f05fc003970 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0608004bf0 con 0x7f05fc003970 2024-04-05T22:21:30.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060803c040 con 0x7f05fc003970 2024-04-05T22:21:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0614159710 con 0x7f05fc003970 2024-04-05T22:21:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.803+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06140ca5e0 con 0x7f05fc003970 2024-04-05T22:21:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.805+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06141598a0 con 0x7f05fc003970 2024-04-05T22:21:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.805+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0608032070 con 0x7f05fc003970 2024-04-05T22:21:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.806+0000 7f0611600640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 0x7f05f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.806+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06080aa230 con 0x7f05fc003970 2024-04-05T22:21:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.806+0000 7f0618a00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 0x7f05f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.807+0000 7f0618a00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 0x7f05f806d0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06140b1e70 tx=0x7f0604008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:30.811+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0608075480 con 0x7f05fc003970 2024-04-05T22:21:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.050+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06140ca5e0 con 0x7f05fc003970 2024-04-05T22:21:31.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.051+0000 7f0611600640 1 -- 172.21.15.73:0/1213287194 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f06140ca5e0 con 0x7f05fc003970 2024-04-05T22:21:31.051 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:31.052 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:21:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 msgr2=0x7f05f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 0x7f05f806d0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06140b1e70 tx=0x7f0604008040 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 msgr2=0x7f06140c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f060800c210 tx=0x7f0608001a00 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 shutdown_connections 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f05f806ac00 0x7f05f806d0c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06141f2560 0x7f06140cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06141f5eb0 0x7f06141f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 --2- 172.21.15.73:0/1213287194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc003970 0x7f06140c9ec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.054+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 >> 172.21.15.73:0/1213287194 conn(0x7f06140d2df0 msgr2=0x7f061415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.055+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 shutdown_connections 2024-04-05T22:21:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.055+0000 7f061be00640 1 -- 172.21.15.73:0/1213287194 wait complete. 2024-04-05T22:21:31.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:21:14.885132+0000', 'modified': '2024-04-05T22:21:17.895240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/123005434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 123005434}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 123005434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:21:31.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:21:31.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-05T22:21:31.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.690+0000 7f691c200640 1 Processor -- start 2024-04-05T22:21:31.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.691+0000 7f691c200640 1 -- start start 2024-04-05T22:21:31.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691c200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69141541a0 0x7f69140b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691c200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f691406d710 0x7f69140cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691c200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691415aa60 con 0x7f691415aef0 2024-04-05T22:21:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691c200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140b3b60 con 0x7f691406d710 2024-04-05T22:21:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691c200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140701c0 con 0x7f69141541a0 2024-04-05T22:21:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f6918e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f691406d710 0x7f69140cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691a200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f691415aef0 0x7f691415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37732/0 (socket says 172.21.15.73:37732) 2024-04-05T22:21:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f691a200640 1 -- 172.21.15.73:0/2471507427 learned_addr learned my addr 172.21.15.73:0/2471507427 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.692+0000 7f6919800640 1 --2- 172.21.15.73:0/2471507427 >> v2:172.21.15.157:3300/0 conn(0x7f69141541a0 0x7f69140b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.693+0000 7f6913e00640 1 -- 172.21.15.73:0/2471507427 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238868523 0 0) 0x7f691415aa60 con 0x7f691415aef0 2024-04-05T22:21:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.693+0000 7f6918e00640 1 -- 172.21.15.73:0/2471507427 >> v2:172.21.15.157:3300/0 conn(0x7f69141541a0 msgr2=0x7f69140b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.693+0000 7f6918e00640 1 --2- 172.21.15.73:0/2471507427 >> v2:172.21.15.157:3300/0 conn(0x7f69141541a0 0x7f69140b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.693+0000 7f6918e00640 1 -- 172.21.15.73:0/2471507427 >> v1:172.21.15.73:6789/0 conn(0x7f691415aef0 legacy=0x7f691415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.693+0000 7f6918e00640 1 -- 172.21.15.73:0/2471507427 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140b3880 con 0x7f691406d710 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.694+0000 7f6918e00640 1 --2- 172.21.15.73:0/2471507427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691406d710 0x7f69140cd8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6914066850 tx=0x7f690002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93670c01d6f0e152 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.694+0000 7f6913e00640 1 -- 172.21.15.73:0/2471507427 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900031830 con 0x7f691406d710 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.694+0000 7f6913e00640 1 -- 172.21.15.73:0/2471507427 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900031990 con 0x7f691406d710 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.694+0000 7f6913e00640 1 -- 172.21.15.73:0/2471507427 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900031d20 con 0x7f691406d710 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 -- 172.21.15.73:0/2471507427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691406d710 msgr2=0x7f69140cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 --2- 172.21.15.73:0/2471507427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691406d710 0x7f69140cd8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6914066850 tx=0x7f690002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 -- 172.21.15.73:0/2471507427 shutdown_connections 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 --2- 172.21.15.73:0/2471507427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691406d710 0x7f69140cd8f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 --2- 172.21.15.73:0/2471507427 >> v2:172.21.15.157:3300/0 conn(0x7f69141541a0 0x7f69140b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 -- 172.21.15.73:0/2471507427 >> 172.21.15.73:0/2471507427 conn(0x7f69140ce8e0 msgr2=0x7f69140cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 -- 172.21.15.73:0/2471507427 shutdown_connections 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.695+0000 7f691c200640 1 -- 172.21.15.73:0/2471507427 wait complete. 2024-04-05T22:21:31.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.696+0000 7f691c200640 1 Processor -- start 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 -- start start 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691406d710 0x7f69140a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 0x7f69140af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691415e660 con 0x7f69141541a0 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140a8b30 con 0x7f691406d710 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.697+0000 7f691c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140a8e10 con 0x7f691415aef0 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39010/0 (socket says 172.21.15.73:39010) 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 -- 172.21.15.73:0/374825918 learned_addr learned my addr 172.21.15.73:0/374825918 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:31.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 -- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 msgr2=0x7f69140af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f691a200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 0x7f69140af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6919800640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691406d710 0x7f69140a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 0x7f69140af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 -- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691406d710 msgr2=0x7f69140a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691406d710 0x7f69140a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f6918e00640 1 -- 172.21.15.73:0/374825918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900009400 con 0x7f69141541a0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.698+0000 7f691a200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 0x7f69140af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f6918e00640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f69000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f6911600640 1 -- 172.21.15.73:0/374825918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900004230 con 0x7f69141541a0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f6911600640 1 -- 172.21.15.73:0/374825918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69000043b0 con 0x7f69141541a0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f6911600640 1 -- 172.21.15.73:0/374825918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900004520 con 0x7f69141541a0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140a90f0 con 0x7f69141541a0 2024-04-05T22:21:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.699+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140a96b0 con 0x7f69141541a0 2024-04-05T22:21:31.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.701+0000 7f6911600640 1 -- 172.21.15.73:0/374825918 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f690003a8a0 con 0x7f69141541a0 2024-04-05T22:21:31.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.702+0000 7f6911600640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 0x7f68f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.702+0000 7f6911600640 1 -- 172.21.15.73:0/374825918 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69000ab970 con 0x7f69141541a0 2024-04-05T22:21:31.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.702+0000 7f6919800640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 0x7f68f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.703+0000 7f691f1f6740 1 Processor -- start 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.703+0000 7f691f1f6740 1 -- start start 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.703+0000 7f6919800640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 0x7f68f406d0f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.703+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb5379c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.703+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb5379c6d0 0x55fb537cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f691f1f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5370c8c0 con 0x55fb537c95d0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5370a050 con 0x55fb537c91f0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb53705940 con 0x55fb5379c6d0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39012/0 (socket says 172.21.15.73:39012) 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 -- 172.21.15.73:0/3317261068 learned_addr learned my addr 172.21.15.73:0/3317261068 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 -- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb5379c6d0 msgr2=0x55fb537cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb5379c6d0 0x55fb537cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 -- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 msgr2=0x55fb5379c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb5379c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 -- 172.21.15.73:0/3317261068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb536d09e0 con 0x55fb537c95d0 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6918e00640 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f690007a790 tx=0x7f690007af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a586f00de5db0361 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.704+0000 7f6906000640 1 -- 172.21.15.73:0/3317261068 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690007e630 con 0x55fb537c95d0 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f6906000640 1 -- 172.21.15.73:0/3317261068 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900082bf0 con 0x55fb537c95d0 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f6906000640 1 -- 172.21.15.73:0/3317261068 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69000920d0 con 0x55fb537c95d0 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 -- 172.21.15.73:0/3317261068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 msgr2=0x55fb537cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f690007a790 tx=0x7f690007af00 comp rx=0 tx=0).stop 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 -- 172.21.15.73:0/3317261068 shutdown_connections 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb5379c6d0 0x55fb537cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537cbf80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3317261068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb5379c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 -- 172.21.15.73:0/3317261068 >> 172.21.15.73:0/3317261068 conn(0x55fb5379e400 msgr2=0x55fb537e6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 -- 172.21.15.73:0/3317261068 shutdown_connections 2024-04-05T22:21:31.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.705+0000 7f691f1f6740 1 -- 172.21.15.73:0/3317261068 wait complete. 2024-04-05T22:21:31.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.706+0000 7f691f1f6740 1 -- start start 2024-04-05T22:21:31.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.706+0000 7f691f1f6740 1 Processor -- start 2024-04-05T22:21:31.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c95d0 0x55fb537dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537ddaa0 0x55fb537f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb536a7070 con 0x55fb537c91f0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb536a6d10 con 0x55fb537c95d0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5346cf20 con 0x55fb537ddaa0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f691f1f6740 10 client.0 _ll_register_callbacks cb 0x55fb536fefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f6919800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f6919800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39024/0 (socket says 172.21.15.73:39024) 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.707+0000 7f6919800640 1 -- 172.21.15.73:0/3773343948 learned_addr learned my addr 172.21.15.73:0/3773343948 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 -- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537ddaa0 msgr2=0x55fb537f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537ddaa0 0x55fb537f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 -- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c95d0 msgr2=0x55fb537dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c95d0 0x55fb537dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 -- 172.21.15.73:0/3773343948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140a96b0 con 0x55fb537c91f0 2024-04-05T22:21:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f6919800640 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f690800ef80 tx=0x7f690800efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4adb79dbf53ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f68faa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908015e10 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6908020400 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.708+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb5372fae0 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69080205c0 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 10 client.4671 fetch_fsmap learned FSMap version 7 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55fb53703fc0 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5372fae0 con 0x55fb537c91f0 2024-04-05T22:21:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6908020720 con 0x55fb537c91f0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 10 client.4671 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 20 client.4671 populate_metadata read hostname 'smithi073' 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb53801fa0 0x55fb53804390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 4 client.4671 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.709+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 --> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb533f93c0 con 0x55fb53801fa0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.710+0000 7f6918e00640 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb53801fa0 0x55fb53804390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.710+0000 7f6918e00640 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb53801fa0 0x55fb53804390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22427b53ec540713 server_cookie=1d0e47ab34cdd988 in_seq=0 out_seq=0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.710+0000 7f68faa00640 10 client.4671 ms_handle_connect on v2:172.21.15.157:6832/123005434 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.712+0000 7f68faa00640 1 -- 172.21.15.73:0/3773343948 <== mds.0 v2:172.21.15.157:6832/123005434 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb533f93c0 con 0x55fb53801fa0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f68faa00640 10 client.4671 handle_command_reply: tid=0 2024-04-05T22:21:31.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 client.4671 shutdown 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 msgr2=0x55fb537c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f690800ef80 tx=0x7f690800efb0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 shutdown_connections 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537ddaa0 0x55fb537f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c95d0 0x55fb537dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb53801fa0 0x55fb53804390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3773343948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c91f0 0x55fb537c7b60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f68faa00640 0 client.4671 ms_handle_reset on v2:172.21.15.157:6832/123005434 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 >> 172.21.15.73:0/3773343948 conn(0x55fb5379c6d0 msgr2=0x55fb537a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 shutdown_connections 2024-04-05T22:21:31.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 1 -- 172.21.15.73:0/3773343948 wait complete. 2024-04-05T22:21:31.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.713+0000 7f691f1f6740 20 client.4671 trim_cache size 0 max 16384 2024-04-05T22:21:31.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.730+0000 7f691f1f6740 1 Processor -- start 2024-04-05T22:21:31.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.730+0000 7f691f1f6740 1 -- start start 2024-04-05T22:21:31.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb53800780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c95d0 0x55fb5379b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb53808550 con 0x55fb537ddaa0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb53797c30 con 0x55fb537c91f0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb537c7610 con 0x55fb537c95d0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39034/0 (socket says 172.21.15.73:39034) 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.731+0000 7f691a200640 1 -- 172.21.15.73:0/2153376767 learned_addr learned my addr 172.21.15.73:0/2153376767 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f6919800640 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb53800780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f6918e00640 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c95d0 0x55fb5379b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 -- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c95d0 msgr2=0x55fb5379b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c95d0 0x55fb5379b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 -- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 msgr2=0x55fb53800780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb53800780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 -- 172.21.15.73:0/2153376767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb53738f80 con 0x55fb537ddaa0 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f691a200640 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69140ad4c0 tx=0x7f690c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d3658da4156e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f6906a00640 1 -- 172.21.15.73:0/2153376767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00d740 con 0x55fb537ddaa0 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f6906a00640 1 -- 172.21.15.73:0/2153376767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00d8c0 con 0x55fb537ddaa0 2024-04-05T22:21:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.732+0000 7f6906a00640 1 -- 172.21.15.73:0/2153376767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00dc70 con 0x55fb537ddaa0 2024-04-05T22:21:31.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 -- 172.21.15.73:0/2153376767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 msgr2=0x55fb537c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69140ad4c0 tx=0x7f690c0020c0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 -- 172.21.15.73:0/2153376767 shutdown_connections 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537ddaa0 0x55fb537c70d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c95d0 0x55fb5379b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 --2- 172.21.15.73:0/2153376767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537c91f0 0x55fb53800780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 -- 172.21.15.73:0/2153376767 >> 172.21.15.73:0/2153376767 conn(0x55fb5379c6d0 msgr2=0x55fb537ea490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 -- 172.21.15.73:0/2153376767 shutdown_connections 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.733+0000 7f691f1f6740 1 -- 172.21.15.73:0/2153376767 wait complete. 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.734+0000 7f691f1f6740 1 -- start start 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.734+0000 7f691f1f6740 1 Processor -- start 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c91f0 0x55fb537a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537ddaa0 0x55fb5379fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb53845180 con 0x55fb537c95d0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb53844b60 con 0x55fb537ddaa0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5379bd80 con 0x55fb537c91f0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691f1f6740 10 client.0 _ll_register_callbacks cb 0x7f6908009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39044/0 (socket says 172.21.15.73:39044) 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 -- 172.21.15.73:0/3274215599 learned_addr learned my addr 172.21.15.73:0/3274215599 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6919800640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c91f0 0x55fb537a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 -- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c91f0 msgr2=0x55fb537a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c91f0 0x55fb537a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 -- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537ddaa0 msgr2=0x55fb5379fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f691a200640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537ddaa0 0x55fb5379fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.735+0000 7f6918e00640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537ddaa0 0x55fb5379fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f6918e00640 1 -- 172.21.15.73:0/3274215599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c001910 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f6918e00640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f690007a790 tx=0x7f6900009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e124ceca6a78dd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690007a5d0 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6900099260 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6900082bf0 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6900076f10 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f691f1f6740 10 client.4677 fetch_fsmap learned FSMap version 7 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55fb536a6280 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900082bf0 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.736+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f690004c060 con 0x55fb537c95d0 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 10 client.4677 fetch_fsmap finished waiting for FSMap version 7 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 10 client.4677 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 20 client.4677 populate_metadata read hostname 'smithi073' 2024-04-05T22:21:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb537a1910 0x55fb537a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:31.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 4 client.4677 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:21:31.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 --> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb5340a6c0 con 0x55fb537a1910 2024-04-05T22:21:31.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.737+0000 7f6919800640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb537a1910 0x55fb537a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.738+0000 7f6919800640 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb537a1910 0x55fb537a0230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a45471ecba2ca3 server_cookie=b2ab833969be67e in_seq=0 out_seq=0 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.738+0000 7f68faa00640 10 client.4677 ms_handle_connect on v2:172.21.15.157:6832/123005434 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.738+0000 7f68faa00640 1 -- 172.21.15.73:0/3274215599 <== mds.0 v2:172.21.15.157:6832/123005434 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55fb5340a6c0 con 0x55fb537a1910 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.738+0000 7f68faa00640 10 client.4677 handle_command_reply: tid=0 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.738+0000 7f691f1f6740 1 client.4677 shutdown 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 msgr2=0x55fb537a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f690007a790 tx=0x7f6900009190 comp rx=0 tx=0).stop 2024-04-05T22:21:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 shutdown_connections 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb537ddaa0 0x55fb5379fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:6832/123005434,v1:172.21.15.157:6835/123005434] conn(0x55fb537a1910 0x55fb537a0230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fb537c95d0 0x55fb537a13d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 --2- 172.21.15.73:0/3274215599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55fb537c91f0 0x55fb537a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f68faa00640 0 client.4677 ms_handle_reset on v2:172.21.15.157:6832/123005434 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 >> 172.21.15.73:0/3274215599 conn(0x55fb5379c6d0 msgr2=0x55fb538000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 shutdown_connections 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 1 -- 172.21.15.73:0/3274215599 wait complete. 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.739+0000 7f691f1f6740 20 client.4677 trim_cache size 0 max 16384 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:21:31.743 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4260, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 16.848646397, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 7, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 22, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 22, 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 23.716871104999999 2024-04-05T22:21:31.744 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 msgr2=0x7f68f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 0x7f68f406d0f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).stop 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 msgr2=0x7f691415e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f69000046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 shutdown_connections 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f68f406ac30 0x7f68f406d0f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f691415aef0 0x7f69140af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69141541a0 0x7f691415e120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 --2- 172.21.15.73:0/374825918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691406d710 0x7f69140a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:31.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 >> 172.21.15.73:0/374825918 conn(0x7f69140ce8e0 msgr2=0x7f6914153330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:31.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 shutdown_connections 2024-04-05T22:21:31.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:31.740+0000 7f691c200640 1 -- 172.21.15.73:0/374825918 wait complete. 2024-04-05T22:21:31.756 INFO:teuthology.run_tasks:Running task sequential... 2024-04-05T22:21:31.768 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-05T22:21:31.768 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-05T22:21:31.768 INFO:teuthology.task.print:Enabling mgr modules 2024-04-05T22:21:31.768 INFO:teuthology.run_tasks:Running task kclient... 2024-04-05T22:21:31.823 INFO:tasks.kclient:Mounting kernel clients... 2024-04-05T22:21:31.823 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-05T22:21:31.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:31.824 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:21:31.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:31.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-05T22:21:31.977 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-04-05T22:21:31.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:21:31.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:31.981 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:21:32.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-05T22:21:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-04-05T22:21:32.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:21:32.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.138 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:21:32.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.196 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-05T22:21:32.294 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-05T22:21:32.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:21:32.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.301 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:21:32.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-05T22:21:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-05T22:21:32.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:21:32.449 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-05T22:21:32.449 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-05T22:21:32.450 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:21:32.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:21:32.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.669+0000 7f239b000640 1 Processor -- start 2024-04-05T22:21:32.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.669+0000 7f239b000640 1 -- start start 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.670+0000 7f239b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f239415e620 0x7f239415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f239b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239406b500 con 0x7f239406bc70 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f239b000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940b20d0 con 0x7f2394158860 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f239b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239415eff0 con 0x7f239415e620 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f239415e620 0x7f239415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f239415e620 0x7f239415eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39060/0 (socket says 172.21.15.73:39060) 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f239a600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f239406bc70 0x7f239406c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37744/0 (socket says 172.21.15.73:37744) 2024-04-05T22:21:32.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 -- 172.21.15.73:0/1758823882 learned_addr learned my addr 172.21.15.73:0/1758823882 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:32.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 -- 172.21.15.73:0/1758823882 >> v1:172.21.15.157:6789/0 conn(0x7f2394158860 legacy=0x7f23940b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:32.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 -- 172.21.15.73:0/1758823882 >> v1:172.21.15.73:6789/0 conn(0x7f239406bc70 legacy=0x7f239406c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.671+0000 7f2399200640 1 -- 172.21.15.73:0/1758823882 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940b23a0 con 0x7f239415e620 2024-04-05T22:21:32.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.672+0000 7f2399200640 1 --2- 172.21.15.73:0/1758823882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239415e620 0x7f239415eab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2380009870 tx=0x7f238002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f958488a4468cc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:32.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.673+0000 7f2393e00640 1 -- 172.21.15.73:0/1758823882 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380031860 con 0x7f239415e620 2024-04-05T22:21:32.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.673+0000 7f2393e00640 1 -- 172.21.15.73:0/1758823882 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23800319c0 con 0x7f239415e620 2024-04-05T22:21:32.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.673+0000 7f2393e00640 1 -- 172.21.15.73:0/1758823882 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380031d50 con 0x7f239415e620 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.673+0000 7f239b000640 1 -- 172.21.15.73:0/1758823882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239415e620 msgr2=0x7f239415eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.673+0000 7f239b000640 1 --2- 172.21.15.73:0/1758823882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239415e620 0x7f239415eab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2380009870 tx=0x7f238002b900 comp rx=0 tx=0).stop 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.674+0000 7f239b000640 1 -- 172.21.15.73:0/1758823882 shutdown_connections 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.674+0000 7f239b000640 1 --2- 172.21.15.73:0/1758823882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239415e620 0x7f239415eab0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.674+0000 7f239b000640 1 -- 172.21.15.73:0/1758823882 >> 172.21.15.73:0/1758823882 conn(0x7f23940b06c0 msgr2=0x7f23940b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.674+0000 7f239b000640 1 -- 172.21.15.73:0/1758823882 shutdown_connections 2024-04-05T22:21:32.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.674+0000 7f239b000640 1 -- 172.21.15.73:0/1758823882 wait complete. 2024-04-05T22:21:32.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.676+0000 7f239b000640 1 Processor -- start 2024-04-05T22:21:32.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.676+0000 7f239b000640 1 -- start start 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239415e620 0x7f239415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23941e63d0 0x7f23941435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239415ac60 con 0x7f239406bc70 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394143ae0 con 0x7f239415e620 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f239b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239415d280 con 0x7f23941e63d0 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.677+0000 7f2399c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39072/0 (socket says 172.21.15.73:39072) 2024-04-05T22:21:32.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 -- 172.21.15.73:0/3669725868 learned_addr learned my addr 172.21.15.73:0/3669725868 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:32.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 -- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23941e63d0 msgr2=0x7f23941435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23941e63d0 0x7f23941435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 -- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239415e620 msgr2=0x7f239415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239415e620 0x7f239415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 -- 172.21.15.73:0/3669725868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23800094d0 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2399c00640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f238800c7c0 tx=0x7f238800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.678+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800aa50 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.679+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800abb0 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.679+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388012420 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.679+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23941e5610 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.679+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23941e5bd0 con 0x7f239406bc70 2024-04-05T22:21:32.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.680+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239404e160 con 0x7f239406bc70 2024-04-05T22:21:32.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.681+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2388012580 con 0x7f239406bc70 2024-04-05T22:21:32.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.682+0000 7f2391600640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 0x7f237406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:32.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.682+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23880892d0 con 0x7f239406bc70 2024-04-05T22:21:32.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.682+0000 7f2399200640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 0x7f237406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:32.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.682+0000 7f2399200640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 0x7f237406d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2380036000 tx=0x7f2380030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:32.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.686+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f23880544b0 con 0x7f239406bc70 2024-04-05T22:21:32.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.919+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23940b04a0 con 0x7f239406bc70 2024-04-05T22:21:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.920+0000 7f2391600640 1 -- 172.21.15.73:0/3669725868 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2388057ad0 con 0x7f239406bc70 2024-04-05T22:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:21:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.922+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 msgr2=0x7f237406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.922+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 0x7f237406d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2380036000 tx=0x7f2380030000 comp rx=0 tx=0).stop 2024-04-05T22:21:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.922+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 msgr2=0x7f239415cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.922+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f238800c7c0 tx=0x7f238800cc80 comp rx=0 tx=0).stop 2024-04-05T22:21:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 shutdown_connections 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f237406ac00 0x7f237406d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23941e63d0 0x7f23941435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239415e620 0x7f239415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 --2- 172.21.15.73:0/3669725868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239406bc70 0x7f239415cb70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 >> 172.21.15.73:0/3669725868 conn(0x7f23940b06c0 msgr2=0x7f2394072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 shutdown_connections 2024-04-05T22:21:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:32.923+0000 7f239b000640 1 -- 172.21.15.73:0/3669725868 wait complete. 2024-04-05T22:21:32.940 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-05T22:21:32.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:21:32.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:21:32.941 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:21:32.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.941 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:32.995 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-05T22:21:32.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-05T22:21:32.995 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:33.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:33.447 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:21:33.531 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:21:33.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:33.535 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:21:33.592 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:21:33.592 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:21:33.592 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:21:33.592 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:21:33.592 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:21:33.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:21:33.594 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:34.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:34.272 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:21:34.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:34.317 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-04-05T22:21:34.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-05T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:34.582 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-05T22:21:34.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-05T22:21:34.583 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:34.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-04-05T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:34.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:34.985 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:21:35.024 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:21:35.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:35.025 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:21:35.113 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:21:35.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:35.113 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:35.270 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:35.270 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:21:35.862 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:21:35.862 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:21:35.866 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:21:35.866 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:21:35.866 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:21:35.866 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:21:35.867 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:21:35.871 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:21:35.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:35.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:21:35.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:35.965 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:36.044 INFO:teuthology.orchestra.run.smithi073.stdout:client4486 2024-04-05T22:21:36.045 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:21:36.045 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:36.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:36.098 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:36.218 INFO:teuthology.orchestra.run.smithi073.stdout:client4486 2024-04-05T22:21:36.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:36.218 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:36.296 INFO:teuthology.orchestra.run.smithi073.stdout:client4486 2024-04-05T22:21:36.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:36.296 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38861c87-67cf-4180-a71f-97401becb77d.client4486/status') 2024-04-05T22:21:36.407 INFO:teuthology.orchestra.run.smithi073.stdout:instance: client.4486 (3)172.21.15.73:0/2322883712 2024-04-05T22:21:36.408 INFO:teuthology.orchestra.run.smithi073.stdout:blocklisted: false 2024-04-05T22:21:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records in 2024-04-05T22:21:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records out 2024-04-05T22:21:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:70 bytes copied, 8.9293e-05 s, 784 kB/s 2024-04-05T22:21:36.412 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-05T22:21:36.413 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-05T22:21:36.413 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-05T22:21:36.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:21:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.629+0000 7f9f77c00640 1 Processor -- start 2024-04-05T22:21:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.629+0000 7f9f77c00640 1 -- start start 2024-04-05T22:21:36.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.630+0000 7f9f77c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f7015ef60 0x7f9f7006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.630+0000 7f9f77c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7006f5b0 con 0x7f9f7006cb80 2024-04-05T22:21:36.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.630+0000 7f9f77c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f70070410 con 0x7f9f701587d0 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.630+0000 7f9f77c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7006d500 con 0x7f9f7015ef60 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f6fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9f7006cb80 0x7f9f7015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36406/0 (socket says 172.21.15.73:36406) 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f6fe00640 1 -- 172.21.15.73:0/1962906792 learned_addr learned my addr 172.21.15.73:0/1962906792 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f75c00640 1 --2- 172.21.15.73:0/1962906792 >> v2:172.21.15.157:3300/0 conn(0x7f9f7015ef60 0x7f9f7006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277281236 0 0) 0x7f9f7006f5b0 con 0x7f9f7006cb80 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f58003520 con 0x7f9f7006cb80 2024-04-05T22:21:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.631+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2009488724 0 0) 0x7f9f58003520 con 0x7f9f7006cb80 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 >> v2:172.21.15.157:3300/0 conn(0x7f9f7015ef60 msgr2=0x7f9f7006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 --2- 172.21.15.73:0/1962906792 >> v2:172.21.15.157:3300/0 conn(0x7f9f7015ef60 0x7f9f7006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 >> v1:172.21.15.157:6789/0 conn(0x7f9f701587d0 legacy=0x7f9f700cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700ccef0 con 0x7f9f7006cb80 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9f5c001670 con 0x7f9f7006cb80 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 >> v1:172.21.15.73:6789/0 conn(0x7f9f7006cb80 legacy=0x7f9f7015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f58003970 0x7f9f58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f58005180 0x7f9f5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7006d500 con 0x7f9f580047c0 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5800c020 con 0x7f9f58005180 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5800c1e0 con 0x7f9f58003970 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f75c00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f58005180 0x7f9f5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f75200640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f58003970 0x7f9f58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 -- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f58003970 msgr2=0x7f9f58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:36.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f58003970 0x7f9f58003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 -- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f58005180 msgr2=0x7f9f5800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f58005180 0x7f9f5800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f6fe00640 1 -- 172.21.15.73:0/1962906792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58003520 con 0x7f9f580047c0 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.632+0000 7f9f75c00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f58005180 0x7f9f5800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.633+0000 7f9f75c00640 1 -- 172.21.15.73:0/1962906792 reap_dead start 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.634+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f58004c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c001050 tx=0x7f9f580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50801fe4d5843e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.634+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c005ad0 con 0x7f9f580047c0 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.634+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c005c30 con 0x7f9f580047c0 2024-04-05T22:21:36.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.634+0000 7f9f6f400640 1 -- 172.21.15.73:0/1962906792 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c02ca50 con 0x7f9f580047c0 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 -- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 msgr2=0x7f9f58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f58004c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c001050 tx=0x7f9f580057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 -- 172.21.15.73:0/1962906792 shutdown_connections 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1962906792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f58004c40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 -- 172.21.15.73:0/1962906792 >> 172.21.15.73:0/1962906792 conn(0x7f9f700ce8e0 msgr2=0x7f9f700cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 -- 172.21.15.73:0/1962906792 shutdown_connections 2024-04-05T22:21:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.635+0000 7f9f77c00640 1 -- 172.21.15.73:0/1962906792 wait complete. 2024-04-05T22:21:36.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.637+0000 7f9f77c00640 1 Processor -- start 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.637+0000 7f9f77c00640 1 -- start start 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 0x7f9f7013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f7015b060 0x7f9f7015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701f6e70 con 0x7f9f580047c0 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7015ba00 con 0x7f9f700cad50 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f77c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701423a0 con 0x7f9f7015b060 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f75200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f75200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52690/0 (socket says 172.21.15.73:52690) 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f75200640 1 -- 172.21.15.73:0/1188233411 learned_addr learned my addr 172.21.15.73:0/1188233411 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.638+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 0x7f9f7013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f7015b060 0x7f9f7015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 -- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f7015b060 msgr2=0x7f9f7015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f7015b060 0x7f9f7015b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 -- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 msgr2=0x7f9f7013e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 0x7f9f7013e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 -- 172.21.15.73:0/1188233411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700ad460 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 0x7f9f7013e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f75200640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400d4d0 tx=0x7f9f6400d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64005480 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f64003610 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64003980 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f701f69e0 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.639+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f701f6390 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.641+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f70066af0 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.641+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9f64003ae0 con 0x7f9f580047c0 2024-04-05T22:21:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.642+0000 7f9f6cc00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 0x7f9f5006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.642+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f640890b0 con 0x7f9f580047c0 2024-04-05T22:21:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.642+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 0x7f9f5006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.643+0000 7f9f6fe00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 0x7f9f5006d0f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c034000 tx=0x7f9f5c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.647+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9f64054280 con 0x7f9f580047c0 2024-04-05T22:21:36.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.881+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f70068160 con 0x7f9f580047c0 2024-04-05T22:21:36.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.882+0000 7f9f6cc00640 1 -- 172.21.15.73:0/1188233411 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9f640578a0 con 0x7f9f580047c0 2024-04-05T22:21:36.882 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:21:36.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 msgr2=0x7f9f5006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 0x7f9f5006d0f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c034000 tx=0x7f9f5c007000 comp rx=0 tx=0).stop 2024-04-05T22:21:36.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 msgr2=0x7f9f700acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:36.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400d4d0 tx=0x7f9f6400d990 comp rx=0 tx=0).stop 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 shutdown_connections 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9f5006ac30 0x7f9f5006d0f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f7015b060 0x7f9f7015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f700cad50 0x7f9f7013e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 --2- 172.21.15.73:0/1188233411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580047c0 0x7f9f700acd80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 >> 172.21.15.73:0/1188233411 conn(0x7f9f700ce8e0 msgr2=0x7f9f7015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 shutdown_connections 2024-04-05T22:21:36.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:36.885+0000 7f9f77c00640 1 -- 172.21.15.73:0/1188233411 wait complete. 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-05T22:21:36.903 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-05T22:21:36.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-05T22:21:36.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:21:36.904 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-05T22:21:36.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:36.904 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:21:36.955 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:36.956 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42566sec preferred_lft 42566sec 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:36.957 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:21:36.958 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:36.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:36.959 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:21:37.046 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:21:37.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.050 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:21:37.103 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:21:37.103 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:21:37.104 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:21:37.104 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:21:37.104 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:21:37.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:21:37.105 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:37.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.286 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:21:37.335 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:21:37.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.336 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-04-05T22:21:37.386 INFO:teuthology.orchestra.run.smithi073.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-05T22:21:37.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.388 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:37.388 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:37.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:21:37.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-05T22:21:37.388 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:37.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:21:37.487 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:21:37.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-05T22:21:37.647 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 52:08:e6:01:b0:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::5008:e6ff:fe01:b085/64 scope link 2024-04-05T22:21:37.648 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.653 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-05T22:21:37.742 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:37.742 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:37.742 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:37.742 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 52:08:e6:01:b0:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::5008:e6ff:fe01:b085/64 scope link 2024-04-05T22:21:37.743 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:37.746 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-05T22:21:37.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-05T22:21:37.747 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:38.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.000 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:21:38.000 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:21:38.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.1 up 2024-04-05T22:21:38.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.1 master ceph-brx 2024-04-05T22:21:38.000 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:21:38.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:38.168 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-05T22:21:38.218 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-05T22:21:38.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:38.218 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-05T22:21:38.265 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-05T22:21:38.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.265 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:38.448 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:38.448 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:38.482 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:38.483 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:38.483 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:21:38.483 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 2@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:21:38.483 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:21:38.485 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-05T22:21:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:21:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:21:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:21:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:21:38.491 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:21:38.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-05T22:21:38.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.572 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:38.623 INFO:teuthology.orchestra.run.smithi073.stdout:client4379 2024-04-05T22:21:38.624 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:21:38.624 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:38.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.674 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:38.775 INFO:teuthology.orchestra.run.smithi073.stdout:client4379 2024-04-05T22:21:38.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.776 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:38.831 INFO:teuthology.orchestra.run.smithi073.stdout:client4379 2024-04-05T22:21:38.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:38.831 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38861c87-67cf-4180-a71f-97401becb77d.client4379/status') 2024-04-05T22:21:38.927 INFO:teuthology.orchestra.run.smithi073.stdout:instance: client.4379 (3)172.21.15.73:0/1360813457 2024-04-05T22:21:38.927 INFO:teuthology.orchestra.run.smithi073.stdout:blocklisted: false 2024-04-05T22:21:38.927 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records in 2024-04-05T22:21:38.927 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records out 2024-04-05T22:21:38.927 INFO:teuthology.orchestra.run.smithi073.stderr:70 bytes copied, 6.8863e-05 s, 1.0 MB/s 2024-04-05T22:21:38.931 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-05T22:21:38.931 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-05T22:21:38.931 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:21:38.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:21:39.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.133+0000 7f9efba00640 1 Processor -- start 2024-04-05T22:21:39.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.133+0000 7f9efba00640 1 -- start start 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.134+0000 7f9efba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ef4162370 0x7f9ef40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.134+0000 7f9efba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 0x7f9ef40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.134+0000 7f9efba00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef41627f0 con 0x7f9ef4161f90 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.134+0000 7f9efba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef4158790 con 0x7f9ef4162370 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.134+0000 7f9efba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b3550 con 0x7f9ef40b2ba0 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ef4162370 0x7f9ef40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ef4162370 0x7f9ef40b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52700/0 (socket says 172.21.15.73:52700) 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 -- 172.21.15.73:0/2379859150 learned_addr learned my addr 172.21.15.73:0/2379859150 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:39.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9ef9a00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 0x7f9ef40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:39.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 -- 172.21.15.73:0/2379859150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 msgr2=0x7f9ef40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:39.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 0x7f9ef40b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 -- 172.21.15.73:0/2379859150 >> v1:172.21.15.157:6789/0 conn(0x7f9ef4161f90 legacy=0x7f9ef4164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9eebe00640 1 -- 172.21.15.73:0/2379859150 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4165cf0 con 0x7f9ef4162370 2024-04-05T22:21:39.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.135+0000 7f9ef9a00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 0x7f9ef40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:39.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9eebe00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4162370 0x7f9ef40b2110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40098a0 tx=0x7f9ee402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4630c13b3d3348a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:39.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9ef3e00640 1 -- 172.21.15.73:0/2379859150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4030840 con 0x7f9ef4162370 2024-04-05T22:21:39.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9ef3e00640 1 -- 172.21.15.73:0/2379859150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40309a0 con 0x7f9ef4162370 2024-04-05T22:21:39.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9ef3e00640 1 -- 172.21.15.73:0/2379859150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4030d30 con 0x7f9ef4162370 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9efba00640 1 -- 172.21.15.73:0/2379859150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4162370 msgr2=0x7f9ef40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.137+0000 7f9efba00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4162370 0x7f9ef40b2110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40098a0 tx=0x7f9ee402b980 comp rx=0 tx=0).stop 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 -- 172.21.15.73:0/2379859150 shutdown_connections 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef40b2ba0 0x7f9ef40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 --2- 172.21.15.73:0/2379859150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4162370 0x7f9ef40b2110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 -- 172.21.15.73:0/2379859150 >> 172.21.15.73:0/2379859150 conn(0x7f9ef40ce8e0 msgr2=0x7f9ef40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 -- 172.21.15.73:0/2379859150 shutdown_connections 2024-04-05T22:21:39.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.138+0000 7f9efba00640 1 -- 172.21.15.73:0/2379859150 wait complete. 2024-04-05T22:21:39.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.139+0000 7f9efba00640 1 Processor -- start 2024-04-05T22:21:39.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.139+0000 7f9efba00640 1 -- start start 2024-04-05T22:21:39.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 0x7f9ef41f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef4162370 0x7f9ef40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41f9ed0 con 0x7f9ef4161f90 2024-04-05T22:21:39.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40ca7c0 con 0x7f9ef40b2ba0 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.140+0000 7f9efba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40caaa0 con 0x7f9ef4162370 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52704/0 (socket says 172.21.15.73:52704) 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 -- 172.21.15.73:0/563057207 learned_addr learned my addr 172.21.15.73:0/563057207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9ef9000640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 0x7f9ef41f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 -- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef4162370 msgr2=0x7f9ef40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef4162370 0x7f9ef40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 -- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 msgr2=0x7f9ef41f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 0x7f9ef41f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 -- 172.21.15.73:0/563057207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee40094d0 con 0x7f9ef4161f90 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9ef9000640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 0x7f9ef41f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:39.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.141+0000 7f9eebe00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee402bf10 tx=0x7f9ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.142+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4047070 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.142+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4039d90 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.142+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4043660 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.142+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40cad80 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.142+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40cb340 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.143+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4066af0 con 0x7f9ef4161f90 2024-04-05T22:21:39.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.144+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9ee404c410 con 0x7f9ef4161f90 2024-04-05T22:21:39.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.145+0000 7f9ef1600640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 0x7f9ed006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:39.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.145+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ee403e070 con 0x7f9ef4161f90 2024-04-05T22:21:39.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.145+0000 7f9ef9000640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 0x7f9ed006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:39.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.146+0000 7f9ef9000640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 0x7f9ed006d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9edc000f60 tx=0x7f9edc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:39.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.149+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9ee4076f20 con 0x7f9ef4161f90 2024-04-05T22:21:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.381+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ef40cabe0 con 0x7f9ef4161f90 2024-04-05T22:21:39.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.382+0000 7f9ef1600640 1 -- 172.21.15.73:0/563057207 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ef40cabe0 con 0x7f9ef4161f90 2024-04-05T22:21:39.382 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:21:39.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 msgr2=0x7f9ed006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 0x7f9ed006d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9edc000f60 tx=0x7f9edc009040 comp rx=0 tx=0).stop 2024-04-05T22:21:39.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 msgr2=0x7f9ef40c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee402bf10 tx=0x7f9ee4004730 comp rx=0 tx=0).stop 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 shutdown_connections 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ed006ac00 0x7f9ed006d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ef4162370 0x7f9ef40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ef4161f90 0x7f9ef40c9ec0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 --2- 172.21.15.73:0/563057207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ef40b2ba0 0x7f9ef41f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 >> 172.21.15.73:0/563057207 conn(0x7f9ef40ce8e0 msgr2=0x7f9ef4069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 shutdown_connections 2024-04-05T22:21:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:39.385+0000 7f9efba00640 1 -- 172.21.15.73:0/563057207 wait complete. 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-05T22:21:39.403 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:21:39.404 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:21:39.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:39.404 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:21:39.456 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:39.456 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:39.457 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:21:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:21:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-04-05T22:21:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:21:39.458 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:39.459 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-05T22:21:39.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-05T22:21:39.459 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:39.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:39.890 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:21:39.979 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:21:39.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:39.983 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:21:40.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:21:40.051 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:40.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:40.763 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:21:40.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:40.819 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-05T22:21:40.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:40.874 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:40.874 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:40.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:21:40.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-05T22:21:40.874 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:41.018 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-05T22:21:41.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-05T22:21:41.019 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:41.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:41.296 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:41.296 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:41.296 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:21:41.296 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-05T22:21:41.296 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:41.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:41.459 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:21:41.501 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:21:41.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:41.502 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:21:41.597 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:21:41.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:41.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:41.726 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:41.726 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:42.377 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:42.377 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:42.377 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:21:42.377 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:21:42.378 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:21:42.380 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:21:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:21:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:21:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:21:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:21:42.384 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:21:42.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:42.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:21:42.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:42.478 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:42.532 INFO:teuthology.orchestra.run.smithi157.stdout:client4701 2024-04-05T22:21:42.533 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:21:42.533 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:42.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:42.580 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:42.681 INFO:teuthology.orchestra.run.smithi157.stdout:client4701 2024-04-05T22:21:42.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:42.681 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:42.733 INFO:teuthology.orchestra.run.smithi157.stdout:client4701 2024-04-05T22:21:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:42.734 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38861c87-67cf-4180-a71f-97401becb77d.client4701/status') 2024-04-05T22:21:42.824 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4701 (3)172.21.15.157:0/604101846 2024-04-05T22:21:42.824 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-04-05T22:21:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-04-05T22:21:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-04-05T22:21:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:70 bytes copied, 6.6307e-05 s, 1.1 MB/s 2024-04-05T22:21:42.828 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-05T22:21:42.829 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-05T22:21:42.829 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-05T22:21:42.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:21:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.044+0000 7f81b9000640 1 Processor -- start 2024-04-05T22:21:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.044+0000 7f81b9000640 1 -- start start 2024-04-05T22:21:43.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81b9000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81b406cb80 0x7f81b406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81b9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81b9000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b406f5b0 con 0x7f81b41587d0 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81b9000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b406d500 con 0x7f81b406cb80 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81b9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b415f220 con 0x7f81b415e850 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81abe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81b406cb80 0x7f81b406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81abe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81b406cb80 0x7f81b406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52706/0 (socket says 172.21.15.73:52706) 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.046+0000 7f81abe00640 1 -- 172.21.15.73:0/980214330 learned_addr learned my addr 172.21.15.73:0/980214330 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:43.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.047+0000 7f81b2800640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.047+0000 7f81abe00640 1 -- 172.21.15.73:0/980214330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 msgr2=0x7f81b415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.047+0000 7f81abe00640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.047+0000 7f81abe00640 1 -- 172.21.15.73:0/980214330 >> v1:172.21.15.157:6789/0 conn(0x7f81b41587d0 legacy=0x7f81b40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.047+0000 7f81abe00640 1 -- 172.21.15.73:0/980214330 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4070410 con 0x7f81b406cb80 2024-04-05T22:21:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.048+0000 7f81abe00640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b406cfc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f819c0098a0 tx=0x7f819c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b619af8c48446c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.048+0000 7f81b1400640 1 -- 172.21.15.73:0/980214330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c030840 con 0x7f81b406cb80 2024-04-05T22:21:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.048+0000 7f81b1400640 1 -- 172.21.15.73:0/980214330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0309a0 con 0x7f81b406cb80 2024-04-05T22:21:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.048+0000 7f81b1400640 1 -- 172.21.15.73:0/980214330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c030d30 con 0x7f81b406cb80 2024-04-05T22:21:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 -- 172.21.15.73:0/980214330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 msgr2=0x7f81b406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b406cfc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f819c0098a0 tx=0x7f819c02b980 comp rx=0 tx=0).stop 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 -- 172.21.15.73:0/980214330 shutdown_connections 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 --2- 172.21.15.73:0/980214330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b406cfc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 -- 172.21.15.73:0/980214330 >> 172.21.15.73:0/980214330 conn(0x7f81b40ce8e0 msgr2=0x7f81b40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 -- 172.21.15.73:0/980214330 shutdown_connections 2024-04-05T22:21:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.049+0000 7f81b9000640 1 -- 172.21.15.73:0/980214330 wait complete. 2024-04-05T22:21:43.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.051+0000 7f81b9000640 1 Processor -- start 2024-04-05T22:21:43.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.051+0000 7f81b9000640 1 -- start start 2024-04-05T22:21:43.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 0x7f81b41525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b4157560 con 0x7f81b406cb80 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b41568e0 con 0x7f81b41587d0 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b4156bc0 con 0x7f81b415e850 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52712/0 (socket says 172.21.15.73:52712) 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81b1e00640 1 -- 172.21.15.73:0/131778431 learned_addr learned my addr 172.21.15.73:0/131778431 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.052+0000 7f81abe00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 0x7f81b41525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 -- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 msgr2=0x7f81b40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 -- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 msgr2=0x7f81b41525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 0x7f81b41525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 -- 172.21.15.73:0/131778431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0094d0 con 0x7f81b406cb80 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81abe00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 0x7f81b41525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b1e00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c450 tx=0x7f81a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:43.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40052d0 con 0x7f81b406cb80 2024-04-05T22:21:43.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4005430 con 0x7f81b406cb80 2024-04-05T22:21:43.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4012870 con 0x7f81b406cb80 2024-04-05T22:21:43.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b4156ea0 con 0x7f81b406cb80 2024-04-05T22:21:43.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.053+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b4152b10 con 0x7f81b406cb80 2024-04-05T22:21:43.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.054+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b4066af0 con 0x7f81b406cb80 2024-04-05T22:21:43.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.055+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f81a40129d0 con 0x7f81b406cb80 2024-04-05T22:21:43.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.057+0000 7f81aa000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 0x7f819006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:43.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.057+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81a4088e30 con 0x7f81b406cb80 2024-04-05T22:21:43.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.057+0000 7f81abe00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 0x7f819006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:43.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.057+0000 7f81abe00640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 0x7f819006d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f819c036000 tx=0x7f819c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:43.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.062+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f81a40540a0 con 0x7f81b406cb80 2024-04-05T22:21:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.286+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81b4068160 con 0x7f81b406cb80 2024-04-05T22:21:43.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.287+0000 7f81aa000640 1 -- 172.21.15.73:0/131778431 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f81a40576c0 con 0x7f81b406cb80 2024-04-05T22:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 msgr2=0x7f819006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 0x7f819006d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f819c036000 tx=0x7f819c03b040 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 msgr2=0x7f81b4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c450 tx=0x7f81a400c910 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 shutdown_connections 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f819006ac00 0x7f819006d0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b415e850 0x7f81b40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b41587d0 0x7f81b41525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 --2- 172.21.15.73:0/131778431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81b406cb80 0x7f81b4156300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:43.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.289+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 >> 172.21.15.73:0/131778431 conn(0x7f81b40ce8e0 msgr2=0x7f81b415dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:43.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.290+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 shutdown_connections 2024-04-05T22:21:43.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:43.290+0000 7f81b9000640 1 -- 172.21.15.73:0/131778431 wait complete. 2024-04-05T22:21:43.306 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-05T22:21:43.306 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:21:43.307 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-05T22:21:43.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.307 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:21:43.365 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42584sec preferred_lft 42584sec 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:21:43.366 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:21:43.367 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:43.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.368 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:21:43.460 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:21:43.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.465 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:21:43.515 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:21:43.515 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:21:43.515 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:21:43.515 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:21:43.515 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:21:43.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:21:43.516 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:43.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.694 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:21:43.743 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:21:43.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.744 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-05T22:21:43.791 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-05T22:21:43.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.793 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:43.793 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:43.793 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:21:43.793 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-05T22:21:43.793 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:43.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:20:12:72:f0:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::9420:12ff:fe72:f0ae/64 scope link 2024-04-05T22:21:44.019 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.022 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-05T22:21:44.109 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:21:44.109 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:21:44.109 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:21:44.109 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.109 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:20:12:72:f0:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::9420:12ff:fe72:f0ae/64 scope link 2024-04-05T22:21:44.110 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:21:44.113 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-05T22:21:44.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-05T22:21:44.114 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:44.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.389 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:21:44.389 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:21:44.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-04-05T22:21:44.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-04-05T22:21:44.389 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:21:44.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:44.561 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-05T22:21:44.609 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-05T22:21:44.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:21:44.610 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-05T22:21:44.658 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-05T22:21:44.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:44.841 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:44.841 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:44.873 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:21:44.890 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:21:44.890 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:21:44.891 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 3@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:21:44.891 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:21:44.892 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-05T22:21:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:21:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:21:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:21:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:21:44.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:21:44.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.893 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-05T22:21:44.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:44.983 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:45.039 INFO:teuthology.orchestra.run.smithi157.stdout:client4385 2024-04-05T22:21:45.039 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:21:45.039 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:45.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:45.091 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:client4385 2024-04-05T22:21:45.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:45.147 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:21:45.199 INFO:teuthology.orchestra.run.smithi157.stdout:client4385 2024-04-05T22:21:45.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:45.200 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38861c87-67cf-4180-a71f-97401becb77d.client4385/status') 2024-04-05T22:21:45.282 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4385 (3)192.168.0.2:0/1951877621 2024-04-05T22:21:45.282 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-04-05T22:21:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-04-05T22:21:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-04-05T22:21:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:69 bytes copied, 7.0909e-05 s, 973 kB/s 2024-04-05T22:21:45.286 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-05T22:21:45.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.514+0000 7f5c1be00640 1 Processor -- start 2024-04-05T22:21:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.515+0000 7f5c1be00640 1 -- start start 2024-04-05T22:21:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.516+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1406cb80 0x7f5c1406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.516+0000 7f5c1be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c1415e850 0x7f5c1415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.516+0000 7f5c1be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1406f5b0 con 0x7f5c141587d0 2024-04-05T22:21:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.516+0000 7f5c1be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1406d500 con 0x7f5c1415e850 2024-04-05T22:21:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.516+0000 7f5c1be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1415f220 con 0x7f5c1406cb80 2024-04-05T22:21:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c1415e850 0x7f5c1415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c1415e850 0x7f5c1415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:54500/0 (socket says 172.21.15.73:54500) 2024-04-05T22:21:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 -- 172.21.15.73:0/3586705674 learned_addr learned my addr 172.21.15.73:0/3586705674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c18a00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1406cb80 0x7f5c1406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 -- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1406cb80 msgr2=0x7f5c1406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1406cb80 0x7f5c1406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 -- 172.21.15.73:0/3586705674 >> v1:172.21.15.157:6789/0 conn(0x7f5c141587d0 legacy=0x7f5c140cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.517+0000 7f5c19e00640 1 -- 172.21.15.73:0/3586705674 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14070410 con 0x7f5c1415e850 2024-04-05T22:21:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.519+0000 7f5c19e00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1415e850 0x7f5c1415ece0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e80 tx=0x7f5c0c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cdd095ad3af9abf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.519+0000 7f5c13e00640 1 -- 172.21.15.73:0/3586705674 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c02caf0 con 0x7f5c1415e850 2024-04-05T22:21:45.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.519+0000 7f5c13e00640 1 -- 172.21.15.73:0/3586705674 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c02cc50 con 0x7f5c1415e850 2024-04-05T22:21:45.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.519+0000 7f5c13e00640 1 -- 172.21.15.73:0/3586705674 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0343f0 con 0x7f5c1415e850 2024-04-05T22:21:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 -- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1415e850 msgr2=0x7f5c1415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1415e850 0x7f5c1415ece0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e80 tx=0x7f5c0c02bf50 comp rx=0 tx=0).stop 2024-04-05T22:21:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 -- 172.21.15.73:0/3586705674 shutdown_connections 2024-04-05T22:21:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1415e850 0x7f5c1415ece0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 --2- 172.21.15.73:0/3586705674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1406cb80 0x7f5c1406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 -- 172.21.15.73:0/3586705674 >> 172.21.15.73:0/3586705674 conn(0x7f5c140ce8e0 msgr2=0x7f5c140cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:45.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 -- 172.21.15.73:0/3586705674 shutdown_connections 2024-04-05T22:21:45.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.520+0000 7f5c1be00640 1 -- 172.21.15.73:0/3586705674 wait complete. 2024-04-05T22:21:45.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.522+0000 7f5c1be00640 1 Processor -- start 2024-04-05T22:21:45.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.522+0000 7f5c1be00640 1 -- start start 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 0x7f5c1414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415e850 0x7f5c140c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140c4a40 con 0x7f5c141587d0 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1415b190 con 0x7f5c1406cb80 2024-04-05T22:21:45.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c1be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1415b470 con 0x7f5c1415e850 2024-04-05T22:21:45.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c18a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c18a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52720/0 (socket says 172.21.15.73:52720) 2024-04-05T22:21:45.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c18a00640 1 -- 172.21.15.73:0/2908251087 learned_addr learned my addr 172.21.15.73:0/2908251087 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:45.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c19400640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 0x7f5c1414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c19e00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415e850 0x7f5c140c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c18a00640 1 -- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415e850 msgr2=0x7f5c140c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.523+0000 7f5c18a00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415e850 0x7f5c140c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c18a00640 1 -- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 msgr2=0x7f5c1414e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c18a00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 0x7f5c1414e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c18a00640 1 -- 172.21.15.73:0/2908251087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c009b30 con 0x7f5c141587d0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c19400640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 0x7f5c1414e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c18a00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9b0 tx=0x7f5c0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00009670 con 0x7f5c141587d0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000c9a0 con 0x7f5c141587d0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1415b750 con 0x7f5c141587d0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000cd50 con 0x7f5c141587d0 2024-04-05T22:21:45.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.524+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c141f26c0 con 0x7f5c141587d0 2024-04-05T22:21:45.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.526+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c141f29a0 con 0x7f5c141587d0 2024-04-05T22:21:45.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.526+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c00013470 con 0x7f5c141587d0 2024-04-05T22:21:45.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.527+0000 7f5c11600640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 0x7f5bf406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.527+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c00089510 con 0x7f5c141587d0 2024-04-05T22:21:45.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.528+0000 7f5c19400640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 0x7f5bf406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.528+0000 7f5c19400640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 0x7f5bf406d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f60 tx=0x7f5c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.532+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5c00054780 con 0x7f5c141587d0 2024-04-05T22:21:45.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.727+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c14066af0 con 0x7f5c141587d0 2024-04-05T22:21:45.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.730+0000 7f5c11600640 1 -- 172.21.15.73:0/2908251087 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f5c00087e60 con 0x7f5c141587d0 2024-04-05T22:21:45.731 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:45.731 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 msgr2=0x7f5bf406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 0x7f5bf406d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f60 tx=0x7f5c08009040 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 msgr2=0x7f5c140c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9b0 tx=0x7f5c0000ee70 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 shutdown_connections 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf406ac00 0x7f5bf406d0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415e850 0x7f5c140c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141587d0 0x7f5c140c41f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 --2- 172.21.15.73:0/2908251087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406cb80 0x7f5c1414e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 >> 172.21.15.73:0/2908251087 conn(0x7f5c140ce8e0 msgr2=0x7f5c14158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.733+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 shutdown_connections 2024-04-05T22:21:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.734+0000 7f5c1be00640 1 -- 172.21.15.73:0/2908251087 wait complete. 2024-04-05T22:21:45.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:45.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:45.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.959+0000 7f1a12600640 1 Processor -- start 2024-04-05T22:21:45.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.959+0000 7f1a12600640 1 -- start start 2024-04-05T22:21:45.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a12600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a0c06cd50 0x7f1a0c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a12600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 0x7f1a0c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a12600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0b27f0 con 0x7f1a0c1599f0 2024-04-05T22:21:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a12600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c0b3890 con 0x7f1a0c06cd50 2024-04-05T22:21:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a12600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0b2ad0 con 0x7f1a0c0add00 2024-04-05T22:21:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a0b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1a0c1599f0 0x7f1a0c06c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33322/0 (socket says 172.21.15.73:33322) 2024-04-05T22:21:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a0b400640 1 -- 172.21.15.73:0/3651112712 learned_addr learned my addr 172.21.15.73:0/3651112712 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.961+0000 7f1a03e00640 1 --2- 172.21.15.73:0/3651112712 >> v2:172.21.15.73:3300/0 conn(0x7f1a0c06cd50 0x7f1a0c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a0be00640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 0x7f1a0c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a0aa00640 1 -- 172.21.15.73:0/3651112712 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12722168 0 0) 0x7f1a0c0b27f0 con 0x7f1a0c1599f0 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a0aa00640 1 -- 172.21.15.73:0/3651112712 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003520 con 0x7f1a0c1599f0 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a03e00640 1 -- 172.21.15.73:0/3651112712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 msgr2=0x7f1a0c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a03e00640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 0x7f1a0c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a03e00640 1 -- 172.21.15.73:0/3651112712 >> v1:172.21.15.73:6789/0 conn(0x7f1a0c1599f0 legacy=0x7f1a0c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a03e00640 1 -- 172.21.15.73:0/3651112712 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0b2db0 con 0x7f1a0c06cd50 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.962+0000 7f1a0be00640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 0x7f1a0c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:45.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.963+0000 7f1a03e00640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c070a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19f40097d0 tx=0x7f19f402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7fd385dc342860 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.963+0000 7f1a0aa00640 1 -- 172.21.15.73:0/3651112712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f402d040 con 0x7f1a0c06cd50 2024-04-05T22:21:45.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.963+0000 7f1a0aa00640 1 -- 172.21.15.73:0/3651112712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f4031810 con 0x7f1a0c06cd50 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.963+0000 7f1a0aa00640 1 -- 172.21.15.73:0/3651112712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4031b80 con 0x7f1a0c06cd50 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 -- 172.21.15.73:0/3651112712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c06cd50 msgr2=0x7f1a0c070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c070a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19f40097d0 tx=0x7f19f402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 -- 172.21.15.73:0/3651112712 shutdown_connections 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c0add00 0x7f1a0c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 --2- 172.21.15.73:0/3651112712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c070a40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 -- 172.21.15.73:0/3651112712 >> 172.21.15.73:0/3651112712 conn(0x7f1a0c069cb0 msgr2=0x7f1a0c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 -- 172.21.15.73:0/3651112712 shutdown_connections 2024-04-05T22:21:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.964+0000 7f1a12600640 1 -- 172.21.15.73:0/3651112712 wait complete. 2024-04-05T22:21:45.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.966+0000 7f1a12600640 1 Processor -- start 2024-04-05T22:21:45.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.966+0000 7f1a12600640 1 -- start start 2024-04-05T22:21:45.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c1599f0 0x7f1a0c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0cc070 con 0x7f1a0c0add00 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0cc350 con 0x7f1a0c06cd50 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a12600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0cc630 con 0x7f1a0c1599f0 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44494/0 (socket says 172.21.15.73:44494) 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 -- 172.21.15.73:0/1834650253 learned_addr learned my addr 172.21.15.73:0/1834650253 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a0b400640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 -- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c1599f0 msgr2=0x7f1a0c0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c1599f0 0x7f1a0c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 -- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 msgr2=0x7f1a0c0cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c0cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a03e00640 1 -- 172.21.15.73:0/1834650253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009400 con 0x7f1a0c0add00 2024-04-05T22:21:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.967+0000 7f1a0b400640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c0cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a03e00640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19f402b8f0 tx=0x7f19f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4006e10 con 0x7f1a0c0add00 2024-04-05T22:21:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f403ad60 con 0x7f1a0c0add00 2024-04-05T22:21:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c0cc910 con 0x7f1a0c0add00 2024-04-05T22:21:45.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4044650 con 0x7f1a0c0add00 2024-04-05T22:21:45.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.968+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c0cd650 con 0x7f1a0c0add00 2024-04-05T22:21:45.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.970+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c0cce30 con 0x7f1a0c0add00 2024-04-05T22:21:45.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.970+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f19f404d450 con 0x7f1a0c0add00 2024-04-05T22:21:45.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.971+0000 7f1a03400640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 0x7f19e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:45.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.971+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19f40ab560 con 0x7f1a0c0add00 2024-04-05T22:21:45.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.971+0000 7f1a0b400640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 0x7f19e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:45.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.972+0000 7f1a0b400640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 0x7f19e806d0c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000f60 tx=0x7f19fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:45.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:45.976+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1a0c0cce30 con 0x7f1a0c0add00 2024-04-05T22:21:46.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.167+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a0c0cce30 con 0x7f1a0c0add00 2024-04-05T22:21:46.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.168+0000 7f1a03400640 1 -- 172.21.15.73:0/1834650253 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a0c0cce30 con 0x7f1a0c0add00 2024-04-05T22:21:46.169 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 msgr2=0x7f19e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 0x7f19e806d0c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000f60 tx=0x7f19fc009040 comp rx=0 tx=0).stop 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 msgr2=0x7f1a0c0aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19f402b8f0 tx=0x7f19f40046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 shutdown_connections 2024-04-05T22:21:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f19e806ac00 0x7f19e806d0c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0c1599f0 0x7f1a0c0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0c0add00 0x7f1a0c0aba90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 --2- 172.21.15.73:0/1834650253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0c06cd50 0x7f1a0c0cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 >> 172.21.15.73:0/1834650253 conn(0x7f1a0c069cb0 msgr2=0x7f1a0c0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 shutdown_connections 2024-04-05T22:21:46.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.171+0000 7f1a12600640 1 -- 172.21.15.73:0/1834650253 wait complete. 2024-04-05T22:21:46.190 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:46.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.398+0000 7f2072400640 1 Processor -- start 2024-04-05T22:21:46.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.398+0000 7f2072400640 1 -- start start 2024-04-05T22:21:46.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f2072400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 0x7f206c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f2072400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f206c0add00 0x7f206c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f2072400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c0b27f0 con 0x7f206c1599f0 2024-04-05T22:21:46.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f2072400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c0b3890 con 0x7f206c0add00 2024-04-05T22:21:46.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f2072400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c0b2ad0 con 0x7f206c06cd50 2024-04-05T22:21:46.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f206b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f206c1599f0 0x7f206c06c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33328/0 (socket says 172.21.15.73:33328) 2024-04-05T22:21:46.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f206b400640 1 -- 172.21.15.73:0/1190723615 learned_addr learned my addr 172.21.15.73:0/1190723615 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:46.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.400+0000 7f206be00640 1 --2- 172.21.15.73:0/1190723615 >> v2:172.21.15.73:3300/0 conn(0x7f206c0add00 0x7f206c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206aa00640 1 --2- 172.21.15.73:0/1190723615 >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 0x7f206c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206a000640 1 -- 172.21.15.73:0/1190723615 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878954488 0 0) 0x7f206c0b27f0 con 0x7f206c1599f0 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206be00640 1 -- 172.21.15.73:0/1190723615 >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 msgr2=0x7f206c070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206be00640 1 --2- 172.21.15.73:0/1190723615 >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 0x7f206c070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206be00640 1 -- 172.21.15.73:0/1190723615 >> v1:172.21.15.73:6789/0 conn(0x7f206c1599f0 legacy=0x7f206c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206be00640 1 -- 172.21.15.73:0/1190723615 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0b2db0 con 0x7f206c0add00 2024-04-05T22:21:46.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.401+0000 7f206aa00640 1 --2- 172.21.15.73:0/1190723615 >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 0x7f206c070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.402+0000 7f206be00640 1 --2- 172.21.15.73:0/1190723615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c0add00 0x7f206c159dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f206c071ad0 tx=0x7f206402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a5e180a45a55b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.402+0000 7f206a000640 1 -- 172.21.15.73:0/1190723615 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206402cc80 con 0x7f206c0add00 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.402+0000 7f206a000640 1 -- 172.21.15.73:0/1190723615 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206402cde0 con 0x7f206c0add00 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.402+0000 7f206a000640 1 -- 172.21.15.73:0/1190723615 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064034660 con 0x7f206c0add00 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 -- 172.21.15.73:0/1190723615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c0add00 msgr2=0x7f206c159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 --2- 172.21.15.73:0/1190723615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c0add00 0x7f206c159dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f206c071ad0 tx=0x7f206402bed0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 -- 172.21.15.73:0/1190723615 shutdown_connections 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 --2- 172.21.15.73:0/1190723615 >> v2:172.21.15.157:3300/0 conn(0x7f206c06cd50 0x7f206c070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 --2- 172.21.15.73:0/1190723615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c0add00 0x7f206c159dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 -- 172.21.15.73:0/1190723615 >> 172.21.15.73:0/1190723615 conn(0x7f206c069cb0 msgr2=0x7f206c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 -- 172.21.15.73:0/1190723615 shutdown_connections 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.403+0000 7f2072400640 1 -- 172.21.15.73:0/1190723615 wait complete. 2024-04-05T22:21:46.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.405+0000 7f2072400640 1 Processor -- start 2024-04-05T22:21:46.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.405+0000 7f2072400640 1 -- start start 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c06cd50 0x7f206c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f206c0add00 0x7f206c1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c146a00 con 0x7f206c1599f0 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c146ce0 con 0x7f206c06cd50 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f2072400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c146fc0 con 0x7f206c0add00 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44500/0 (socket says 172.21.15.73:44500) 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 -- 172.21.15.73:0/3062827207 learned_addr learned my addr 172.21.15.73:0/3062827207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 -- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f206c0add00 msgr2=0x7f206c1f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f206c0add00 0x7f206c1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 -- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c06cd50 msgr2=0x7f206c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206b400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c06cd50 0x7f206c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c06cd50 0x7f206c149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.406+0000 7f206be00640 1 -- 172.21.15.73:0/3062827207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009a70 con 0x7f206c1599f0 2024-04-05T22:21:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f206be00640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2064009e80 tx=0x7f2064004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064040650 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640407b0 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064040b60 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c1472a0 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.407+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1f6270 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.408+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c1f6460 con 0x7f206c1599f0 2024-04-05T22:21:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.409+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2064040cc0 con 0x7f206c1599f0 2024-04-05T22:21:46.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.410+0000 7f2063400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 0x7f204806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.410+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20640ac300 con 0x7f206c1599f0 2024-04-05T22:21:46.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.410+0000 7f206b400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 0x7f204806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.411+0000 7f206b400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 0x7f204806d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f60 tx=0x7f205c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.414+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f20640776c0 con 0x7f206c1599f0 2024-04-05T22:21:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.610+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f206c1477c0 con 0x7f206c1599f0 2024-04-05T22:21:46.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.611+0000 7f2063400640 1 -- 172.21.15.73:0/3062827207 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f206c1477c0 con 0x7f206c1599f0 2024-04-05T22:21:46.611 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 msgr2=0x7f204806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 0x7f204806d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f60 tx=0x7f205c009040 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 msgr2=0x7f206c145db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2064009e80 tx=0x7f2064004a40 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 shutdown_connections 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f204806ac00 0x7f204806d0c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f206c1599f0 0x7f206c145db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f206c0add00 0x7f206c1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 --2- 172.21.15.73:0/3062827207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c06cd50 0x7f206c149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.613+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 >> 172.21.15.73:0/3062827207 conn(0x7f206c069cb0 msgr2=0x7f206c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:46.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.614+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 shutdown_connections 2024-04-05T22:21:46.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.614+0000 7f2072400640 1 -- 172.21.15.73:0/3062827207 wait complete. 2024-04-05T22:21:46.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:46.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.848+0000 7f0421800640 1 Processor -- start 2024-04-05T22:21:46.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.848+0000 7f0421800640 1 -- start start 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f041c06c630 0x7f041c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f041c0cbe00 0x7f041c0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0b1b30 con 0x7f041c06c630 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0b1e10 con 0x7f041c0cbe00 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f0421800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c06cf50 con 0x7f041c0b22a0 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f041aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f041c06c630 0x7f041c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f041aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f041c06c630 0x7f041c06ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44514/0 (socket says 172.21.15.73:44514) 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f041aa00640 1 -- 172.21.15.73:0/2980869138 learned_addr learned my addr 172.21.15.73:0/2980869138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f041a000640 1 --2- 172.21.15.73:0/2980869138 >> v2:172.21.15.157:3300/0 conn(0x7f041c0cbe00 0x7f041c0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.850+0000 7f041b400640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.851+0000 7f041aa00640 1 -- 172.21.15.73:0/2980869138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 msgr2=0x7f041c0b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:46.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.851+0000 7f041aa00640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c0b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.851+0000 7f041aa00640 1 -- 172.21.15.73:0/2980869138 >> v2:172.21.15.157:3300/0 conn(0x7f041c0cbe00 msgr2=0x7f041c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:46.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.851+0000 7f041aa00640 1 --2- 172.21.15.73:0/2980869138 >> v2:172.21.15.157:3300/0 conn(0x7f041c0cbe00 0x7f041c0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.851+0000 7f041aa00640 1 -- 172.21.15.73:0/2980869138 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0cc300 con 0x7f041c06c630 2024-04-05T22:21:46.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.852+0000 7f041aa00640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c06ca10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f041c04f0c0 tx=0x7f040c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8b49c93314ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0419600640 1 -- 172.21.15.73:0/2980869138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c031870 con 0x7f041c06c630 2024-04-05T22:21:46.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0419600640 1 -- 172.21.15.73:0/2980869138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c0319d0 con 0x7f041c06c630 2024-04-05T22:21:46.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0419600640 1 -- 172.21.15.73:0/2980869138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c031d60 con 0x7f041c06c630 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 -- 172.21.15.73:0/2980869138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 msgr2=0x7f041c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c06ca10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f041c04f0c0 tx=0x7f040c02b910 comp rx=0 tx=0).stop 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 -- 172.21.15.73:0/2980869138 shutdown_connections 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c0b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 --2- 172.21.15.73:0/2980869138 >> v2:172.21.15.157:3300/0 conn(0x7f041c0cbe00 0x7f041c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 --2- 172.21.15.73:0/2980869138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c06ca10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 -- 172.21.15.73:0/2980869138 >> 172.21.15.73:0/2980869138 conn(0x7f041c0b0830 msgr2=0x7f041c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 -- 172.21.15.73:0/2980869138 shutdown_connections 2024-04-05T22:21:46.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.853+0000 7f0421800640 1 -- 172.21.15.73:0/2980869138 wait complete. 2024-04-05T22:21:46.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.855+0000 7f0421800640 1 Processor -- start 2024-04-05T22:21:46.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.855+0000 7f0421800640 1 -- start start 2024-04-05T22:21:46.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0cbe00 0x7f041c1332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c133bd0 con 0x7f041c06c630 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c134680 con 0x7f041c0cbe00 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f0421800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c138380 con 0x7f041c0b22a0 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f041aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f041aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44516/0 (socket says 172.21.15.73:44516) 2024-04-05T22:21:46.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.856+0000 7f041aa00640 1 -- 172.21.15.73:0/799845536 learned_addr learned my addr 172.21.15.73:0/799845536 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:46.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041a000640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041b400640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0cbe00 0x7f041c1332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 -- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 msgr2=0x7f041c132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 -- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0cbe00 msgr2=0x7f041c1332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0cbe00 0x7f041c1332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 -- 172.21.15.73:0/799845536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0094d0 con 0x7f041c06c630 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041a000640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.857+0000 7f041aa00640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.858+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c03abe0 con 0x7f041c06c630 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.858+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c03ad40 con 0x7f041c06c630 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.858+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c044610 con 0x7f041c06c630 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.858+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c138c20 con 0x7f041c06c630 2024-04-05T22:21:46.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.858+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1385d0 con 0x7f041c06c630 2024-04-05T22:21:46.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.859+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c068160 con 0x7f041c06c630 2024-04-05T22:21:46.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.860+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f040c04d450 con 0x7f041c06c630 2024-04-05T22:21:46.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.861+0000 7f0412a00640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 0x7f03fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:46.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.861+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f040c02d040 con 0x7f041c06c630 2024-04-05T22:21:46.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.861+0000 7f041a000640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 0x7f03fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:46.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.862+0000 7f041a000640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 0x7f03fc06d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:46.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:46.865+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f040c076ba0 con 0x7f041c06c630 2024-04-05T22:21:47.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.063+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f041c0b10e0 con 0x7f041c06c630 2024-04-05T22:21:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.064+0000 7f0412a00640 1 -- 172.21.15.73:0/799845536 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f040c0350a0 con 0x7f041c06c630 2024-04-05T22:21:47.064 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:47.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 msgr2=0x7f03fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 0x7f03fc06d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 msgr2=0x7f041c156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 shutdown_connections 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f03fc06ac00 0x7f03fc06d0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0cbe00 0x7f041c1332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0b22a0 0x7f041c132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 --2- 172.21.15.73:0/799845536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f041c06c630 0x7f041c156670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 >> 172.21.15.73:0/799845536 conn(0x7f041c0b0830 msgr2=0x7f041c06f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 shutdown_connections 2024-04-05T22:21:47.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.067+0000 7f0421800640 1 -- 172.21.15.73:0/799845536 wait complete. 2024-04-05T22:21:47.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.290+0000 7fc98be00640 1 Processor -- start 2024-04-05T22:21:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.290+0000 7fc98be00640 1 -- start start 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.291+0000 7fc98be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc98c142ba0 0x7fc98c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.291+0000 7fc98be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c143330 con 0x7fc98c13fe40 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c143050 con 0x7fc98c13fa60 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98c06d3a0 con 0x7fc98c142ba0 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc98c13fe40 0x7fc98c15d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33332/0 (socket says 172.21.15.73:33332) 2024-04-05T22:21:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98b400640 1 -- 172.21.15.73:0/1114007728 learned_addr learned my addr 172.21.15.73:0/1114007728 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc982000640 1 --2- 172.21.15.73:0/1114007728 >> v2:172.21.15.157:3300/0 conn(0x7fc98c142ba0 0x7fc98c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032602192 0 0) 0x7fc98c143330 con 0x7fc98c13fe40 2024-04-05T22:21:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.292+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc970003520 con 0x7fc98c13fe40 2024-04-05T22:21:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196734798 0 0) 0x7fc970003520 con 0x7fc98c13fe40 2024-04-05T22:21:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 >> v2:172.21.15.157:3300/0 conn(0x7fc98c142ba0 msgr2=0x7fc98c06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 --2- 172.21.15.73:0/1114007728 >> v2:172.21.15.157:3300/0 conn(0x7fc98c142ba0 0x7fc98c06f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 >> v1:172.21.15.157:6789/0 conn(0x7fc98c13fa60 legacy=0x7fc98c142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0ac970 con 0x7fc98c13fe40 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc9840019f0 con 0x7fc98c13fe40 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 >> v1:172.21.15.73:6789/0 conn(0x7fc98c13fe40 legacy=0x7fc98c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 0x7fc970003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9700047c0 0x7fc970004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970005180 0x7fc97000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c06d3a0 con 0x7fc970003970 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97000c000 con 0x7fc970005180 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97000c1c0 con 0x7fc9700047c0 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98aa00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 0x7fc970003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98aa00640 1 -- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9700047c0 msgr2=0x7fc970004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98b400640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970005180 0x7fc97000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98aa00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9700047c0 0x7fc970004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.293+0000 7fc98aa00640 1 -- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970005180 msgr2=0x7fc97000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.294+0000 7fc98aa00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970005180 0x7fc97000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.294+0000 7fc98b400640 1 -- 172.21.15.73:0/1114007728 reap_dead start 2024-04-05T22:21:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.294+0000 7fc98aa00640 1 -- 172.21.15.73:0/1114007728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970003520 con 0x7fc970003970 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98aa00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 0x7fc970003d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc97c003db0 tx=0x7fc9700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68691107f251ede server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c032600 con 0x7fc970003970 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c02e050 con 0x7fc970003970 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98a000640 1 -- 172.21.15.73:0/1114007728 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0329e0 con 0x7fc970003970 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98be00640 1 -- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 msgr2=0x7fc970003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98be00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 0x7fc970003d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc97c003db0 tx=0x7fc9700057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98be00640 1 -- 172.21.15.73:0/1114007728 shutdown_connections 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98be00640 1 --2- 172.21.15.73:0/1114007728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc970003970 0x7fc970003d70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.295+0000 7fc98be00640 1 -- 172.21.15.73:0/1114007728 >> 172.21.15.73:0/1114007728 conn(0x7fc98c0b06c0 msgr2=0x7fc98c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:47.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.296+0000 7fc98be00640 1 -- 172.21.15.73:0/1114007728 shutdown_connections 2024-04-05T22:21:47.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.296+0000 7fc98be00640 1 -- 172.21.15.73:0/1114007728 wait complete. 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.297+0000 7fc98be00640 1 Processor -- start 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.297+0000 7fc98be00640 1 -- start start 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970003970 0x7fc98c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98c15f9f0 0x7fc98c1f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0cc770 con 0x7fc98c1f3130 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0cca50 con 0x7fc970003970 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.298+0000 7fc98be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0ccd30 con 0x7fc98c15f9f0 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44530/0 (socket says 172.21.15.73:44530) 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 -- 172.21.15.73:0/4196096470 learned_addr learned my addr 172.21.15.73:0/4196096470 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc982000640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98c15f9f0 0x7fc98c1f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 -- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98c15f9f0 msgr2=0x7fc98c1f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98c15f9f0 0x7fc98c1f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 -- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970003970 msgr2=0x7fc98c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970003970 0x7fc98c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 -- 172.21.15.73:0/4196096470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0cd010 con 0x7fc98c1f3130 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc98b400640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc98c157830 tx=0x7fc98400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.299+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98400de70 con 0x7fc98c1f3130 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.300+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98400bb80 con 0x7fc98c1f3130 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.300+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984007070 con 0x7fc98c1f3130 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.300+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c0cd2f0 con 0x7fc98c1f3130 2024-04-05T22:21:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.300+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c13d330 con 0x7fc98c1f3130 2024-04-05T22:21:47.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.301+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c143050 con 0x7fc98c1f3130 2024-04-05T22:21:47.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.302+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc9840071d0 con 0x7fc98c1f3130 2024-04-05T22:21:47.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.303+0000 7fc983400640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 0x7fc96806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.303+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9840897a0 con 0x7fc98c1f3130 2024-04-05T22:21:47.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.303+0000 7fc98aa00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 0x7fc96806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.304+0000 7fc98aa00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 0x7fc96806d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc97c034000 tx=0x7fc97c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.307+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc984054a40 con 0x7fc98c1f3130 2024-04-05T22:21:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.505+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98c04e160 con 0x7fc98c1f3130 2024-04-05T22:21:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.506+0000 7fc983400640 1 -- 172.21.15.73:0/4196096470 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fc984058060 con 0x7fc98c1f3130 2024-04-05T22:21:47.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:47.507 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 msgr2=0x7fc96806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 0x7fc96806d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc97c034000 tx=0x7fc97c02f000 comp rx=0 tx=0).stop 2024-04-05T22:21:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 msgr2=0x7fc98c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc98c157830 tx=0x7fc98400b4a0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 shutdown_connections 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc96806ac00 0x7fc96806d0c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc98c1f3130 0x7fc98c15da20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc98c15f9f0 0x7fc98c1f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 --2- 172.21.15.73:0/4196096470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc970003970 0x7fc98c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 >> 172.21.15.73:0/4196096470 conn(0x7fc98c0b06c0 msgr2=0x7fc98c06bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 shutdown_connections 2024-04-05T22:21:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.509+0000 7fc98be00640 1 -- 172.21.15.73:0/4196096470 wait complete. 2024-04-05T22:21:47.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:47.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:47.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.745+0000 7f0e2d000640 1 Processor -- start 2024-04-05T22:21:47.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.745+0000 7f0e2d000640 1 -- start start 2024-04-05T22:21:47.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e2d000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e280d0330 0x7f0e2806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e2d000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2815a530 con 0x7f0e280d0710 2024-04-05T22:21:47.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e2d000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2806cf90 con 0x7f0e28070dd0 2024-04-05T22:21:47.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e2d000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e280b3e10 con 0x7f0e280d0330 2024-04-05T22:21:47.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e25400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e280d0710 0x7f0e280706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33336/0 (socket says 172.21.15.73:33336) 2024-04-05T22:21:47.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e25400640 1 -- 172.21.15.73:0/3574406764 learned_addr learned my addr 172.21.15.73:0/3574406764 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e25e00640 1 --2- 172.21.15.73:0/3574406764 >> v2:172.21.15.157:3300/0 conn(0x7f0e280d0330 0x7f0e2806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.747+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298930214 0 0) 0x7f0e2815a530 con 0x7f0e280d0710 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c005730 con 0x7f0e280d0710 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3429025016 0 0) 0x7f0e0c005730 con 0x7f0e280d0710 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 >> v2:172.21.15.157:3300/0 conn(0x7f0e280d0330 msgr2=0x7f0e2806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 --2- 172.21.15.73:0/3574406764 >> v2:172.21.15.157:3300/0 conn(0x7f0e280d0330 0x7f0e2806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 >> v1:172.21.15.157:6789/0 conn(0x7f0e28070dd0 legacy=0x7f0e280b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280b40f0 con 0x7f0e280d0710 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0e10001670 con 0x7f0e280d0710 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 >> v1:172.21.15.73:6789/0 conn(0x7f0e280d0710 legacy=0x7f0e280706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c005b80 0x7f0e0c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 0x7f0e0c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c007390 0x7f0e0c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280b3e10 con 0x7f0e0c0069d0 2024-04-05T22:21:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00e230 con 0x7f0e0c005b80 2024-04-05T22:21:47.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00e3f0 con 0x7f0e0c007390 2024-04-05T22:21:47.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.748+0000 7f0e25400640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 0x7f0e0c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25e00640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c005b80 0x7f0e0c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e26800640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c007390 0x7f0e0c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25400640 1 -- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c007390 msgr2=0x7f0e0c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25400640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c007390 0x7f0e0c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25400640 1 -- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c005b80 msgr2=0x7f0e0c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:47.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25400640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c005b80 0x7f0e0c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e25400640 1 -- 172.21.15.73:0/3574406764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c005730 con 0x7f0e0c0069d0 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e26800640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c007390 0x7f0e0c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.749+0000 7f0e26800640 1 -- 172.21.15.73:0/3574406764 reap_dead start 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.750+0000 7f0e25400640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 0x7f0e0c006e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000c240 tx=0x7f0e0c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7003e55adf7f9873 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.750+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10005ad0 con 0x7f0e0c0069d0 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.750+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e10005c30 con 0x7f0e0c0069d0 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.750+0000 7f0e24a00640 1 -- 172.21.15.73:0/3574406764 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1002caf0 con 0x7f0e0c0069d0 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 -- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 msgr2=0x7f0e0c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 0x7f0e0c006e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000c240 tx=0x7f0e0c007a00 comp rx=0 tx=0).stop 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 -- 172.21.15.73:0/3574406764 shutdown_connections 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 --2- 172.21.15.73:0/3574406764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0c0069d0 0x7f0e0c006e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 -- 172.21.15.73:0/3574406764 >> 172.21.15.73:0/3574406764 conn(0x7f0e2806adc0 msgr2=0x7f0e2806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 -- 172.21.15.73:0/3574406764 shutdown_connections 2024-04-05T22:21:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.751+0000 7f0e2d000640 1 -- 172.21.15.73:0/3574406764 wait complete. 2024-04-05T22:21:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.753+0000 7f0e2d000640 1 Processor -- start 2024-04-05T22:21:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.753+0000 7f0e2d000640 1 -- start start 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 0x7f0e280aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e28143fc0 0x7f0e28144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f6580 con 0x7f0e2815db50 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f73e0 con 0x7f0e28143fc0 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e2d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2815e510 con 0x7f0e0c0069d0 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44548/0 (socket says 172.21.15.73:44548) 2024-04-05T22:21:47.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 -- 172.21.15.73:0/2698673017 learned_addr learned my addr 172.21.15.73:0/2698673017 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25e00640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 0x7f0e280aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 -- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 msgr2=0x7f0e280aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 0x7f0e280aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 -- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e28143fc0 msgr2=0x7f0e28144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e28143fc0 0x7f0e28144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25400640 1 -- 172.21.15.73:0/2698673017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e281f6760 con 0x7f0e2815db50 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.754+0000 7f0e25e00640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 0x7f0e280aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e25400640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001930 tx=0x7f0e1002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1003c040 con 0x7f0e2815db50 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1002cd80 con 0x7f0e2815db50 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10007290 con 0x7f0e2815db50 2024-04-05T22:21:47.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e281f6a40 con 0x7f0e2815db50 2024-04-05T22:21:47.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.755+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e281f7000 con 0x7f0e2815db50 2024-04-05T22:21:47.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.756+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e28068160 con 0x7f0e2815db50 2024-04-05T22:21:47.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.757+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e10032070 con 0x7f0e2815db50 2024-04-05T22:21:47.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.758+0000 7f0e1e000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 0x7f0e0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:47.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.758+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e100aa870 con 0x7f0e2815db50 2024-04-05T22:21:47.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.758+0000 7f0e25e00640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 0x7f0e0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:47.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.759+0000 7f0e25e00640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 0x7f0e0406d0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007440 tx=0x7f0e2000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:47.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.763+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0e10075a40 con 0x7f0e2815db50 2024-04-05T22:21:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.955+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e2806e560 con 0x7f0e2815db50 2024-04-05T22:21:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.955+0000 7f0e1e000640 1 -- 172.21.15.73:0/2698673017 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e10079060 con 0x7f0e2815db50 2024-04-05T22:21:47.956 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 msgr2=0x7f0e0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 0x7f0e0406d0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007440 tx=0x7f0e2000b040 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 msgr2=0x7f0e2815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001930 tx=0x7f0e1002e070 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 shutdown_connections 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e0406ac30 0x7f0e0406d0f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e28143fc0 0x7f0e28144440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2815db50 0x7f0e2815dfd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 --2- 172.21.15.73:0/2698673017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e0c0069d0 0x7f0e280aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 >> 172.21.15.73:0/2698673017 conn(0x7f0e2806adc0 msgr2=0x7f0e28144c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 shutdown_connections 2024-04-05T22:21:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:47.958+0000 7f0e2d000640 1 -- 172.21.15.73:0/2698673017 wait complete. 2024-04-05T22:21:47.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:48.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.185+0000 7f60cea00640 1 Processor -- start 2024-04-05T22:21:48.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.185+0000 7f60cea00640 1 -- start start 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 0x7f60c80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c806c9a0 0x7f60c806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60c806ba70 0x7f60c80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c806bec0 con 0x7f60c806ba70 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c815afc0 con 0x7f60c815abe0 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c806f5b0 con 0x7f60c806c9a0 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60c806ba70 0x7f60c80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60c806ba70 0x7f60c80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44564/0 (socket says 172.21.15.73:44564) 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.187+0000 7f60cca00640 1 -- 172.21.15.73:0/3058480596 learned_addr learned my addr 172.21.15.73:0/3058480596 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60c7e00640 1 --2- 172.21.15.73:0/3058480596 >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 0x7f60c80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60bf400640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c806c9a0 0x7f60c806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60cca00640 1 -- 172.21.15.73:0/3058480596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c806c9a0 msgr2=0x7f60c806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60cca00640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c806c9a0 0x7f60c806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60cca00640 1 -- 172.21.15.73:0/3058480596 >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 msgr2=0x7f60c80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60cca00640 1 --2- 172.21.15.73:0/3058480596 >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 0x7f60c80712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60cca00640 1 -- 172.21.15.73:0/3058480596 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c806f890 con 0x7f60c806ba70 2024-04-05T22:21:48.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.188+0000 7f60c7e00640 1 --2- 172.21.15.73:0/3058480596 >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 0x7f60c80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:21:48.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.189+0000 7f60cca00640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806ba70 0x7f60c80cccb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60c806d300 tx=0x7f60c002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc4cffeb154b16f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.189+0000 7f60c7400640 1 -- 172.21.15.73:0/3058480596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c002c960 con 0x7f60c806ba70 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.189+0000 7f60c7400640 1 -- 172.21.15.73:0/3058480596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c002cac0 con 0x7f60c806ba70 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.189+0000 7f60c7400640 1 -- 172.21.15.73:0/3058480596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c002ce50 con 0x7f60c806ba70 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 -- 172.21.15.73:0/3058480596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806ba70 msgr2=0x7f60c80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806ba70 0x7f60c80cccb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60c806d300 tx=0x7f60c002bdc0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 -- 172.21.15.73:0/3058480596 shutdown_connections 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806ba70 0x7f60c80cccb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 --2- 172.21.15.73:0/3058480596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c806c9a0 0x7f60c806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 --2- 172.21.15.73:0/3058480596 >> v2:172.21.15.157:3300/0 conn(0x7f60c815abe0 0x7f60c80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 -- 172.21.15.73:0/3058480596 >> 172.21.15.73:0/3058480596 conn(0x7f60c8069cb0 msgr2=0x7f60c806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 -- 172.21.15.73:0/3058480596 shutdown_connections 2024-04-05T22:21:48.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.190+0000 7f60cea00640 1 -- 172.21.15.73:0/3058480596 wait complete. 2024-04-05T22:21:48.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.192+0000 7f60cea00640 1 Processor -- start 2024-04-05T22:21:48.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.192+0000 7f60cea00640 1 -- start start 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c806ba70 0x7f60c8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c815abe0 0x7f60c81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8148cd0 con 0x7f60c806c9a0 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c814e510 con 0x7f60c806ba70 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60cea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c814e7f0 con 0x7f60c815abe0 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44576/0 (socket says 172.21.15.73:44576) 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60bf400640 1 -- 172.21.15.73:0/2411612707 learned_addr learned my addr 172.21.15.73:0/2411612707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:48.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.193+0000 7f60c7e00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c806ba70 0x7f60c8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60cca00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c815abe0 0x7f60c81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 -- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c815abe0 msgr2=0x7f60c81592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c815abe0 0x7f60c81592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 -- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c806ba70 msgr2=0x7f60c8153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c806ba70 0x7f60c8153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 -- 172.21.15.73:0/2411612707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0009b30 con 0x7f60c806c9a0 2024-04-05T22:21:48.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60bf400640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d800 tx=0x7f60b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000b6c0 con 0x7f60c806c9a0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b000b820 con 0x7f60c806c9a0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000bbb0 con 0x7f60c806c9a0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c814ead0 con 0x7f60c806c9a0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.194+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c814f090 con 0x7f60c806c9a0 2024-04-05T22:21:48.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.196+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c8068160 con 0x7f60c806c9a0 2024-04-05T22:21:48.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.197+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f60b000bd10 con 0x7f60c806c9a0 2024-04-05T22:21:48.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.197+0000 7f60c4c00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 0x7f60a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.198+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60b0089a30 con 0x7f60c806c9a0 2024-04-05T22:21:48.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.198+0000 7f60c7e00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 0x7f60a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.198+0000 7f60c7e00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 0x7f60a406d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006fd0 tx=0x7f60b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.202+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f60b0054ae0 con 0x7f60c806c9a0 2024-04-05T22:21:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.389+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60c8156de0 con 0x7f60c806c9a0 2024-04-05T22:21:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.389+0000 7f60c4c00640 1 -- 172.21.15.73:0/2411612707 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60b0058100 con 0x7f60c806c9a0 2024-04-05T22:21:48.390 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.391+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 msgr2=0x7f60a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.391+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 0x7f60a406d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006fd0 tx=0x7f60b8008040 comp rx=0 tx=0).stop 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.391+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 msgr2=0x7f60c81481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.391+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d800 tx=0x7f60b000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 shutdown_connections 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60a406ac00 0x7f60a406d0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60c815abe0 0x7f60c81592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60c806c9a0 0x7f60c81481d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 --2- 172.21.15.73:0/2411612707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c806ba70 0x7f60c8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 >> 172.21.15.73:0/2411612707 conn(0x7f60c8069cb0 msgr2=0x7f60c80af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:48.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 shutdown_connections 2024-04-05T22:21:48.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.392+0000 7f60cea00640 1 -- 172.21.15.73:0/2411612707 wait complete. 2024-04-05T22:21:48.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:48.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.612+0000 7f41e3200640 1 Processor -- start 2024-04-05T22:21:48.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.612+0000 7f41e3200640 1 -- start start 2024-04-05T22:21:48.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.613+0000 7f41e3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.613+0000 7f41e3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41e40cdbb0 0x7f41e40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.613+0000 7f41e3200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e406b7a0 con 0x7f41e4144260 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.613+0000 7f41e3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e406ba80 con 0x7f41e40cdbb0 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.613+0000 7f41e3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e406bd60 con 0x7f41e41435f0 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41e40cdbb0 0x7f41e40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41e40cdbb0 0x7f41e40cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44584/0 (socket says 172.21.15.73:44584) 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 -- 172.21.15.73:0/2718132766 learned_addr learned my addr 172.21.15.73:0/2718132766 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e1e00640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 -- 172.21.15.73:0/2718132766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 msgr2=0x7f41e406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e406c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 -- 172.21.15.73:0/2718132766 >> v1:172.21.15.157:6789/0 conn(0x7f41e4144260 legacy=0x7f41e41439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e2800640 1 -- 172.21.15.73:0/2718132766 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e406c040 con 0x7f41e40cdbb0 2024-04-05T22:21:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.614+0000 7f41e1e00640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:48.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.616+0000 7f41e2800640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e40cdfe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdaefcdbe05c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.616+0000 7f41e1400640 1 -- 172.21.15.73:0/2718132766 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc031640 con 0x7f41e40cdbb0 2024-04-05T22:21:48.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.616+0000 7f41e1400640 1 -- 172.21.15.73:0/2718132766 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0317a0 con 0x7f41e40cdbb0 2024-04-05T22:21:48.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.616+0000 7f41e1400640 1 -- 172.21.15.73:0/2718132766 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc031b30 con 0x7f41e40cdbb0 2024-04-05T22:21:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 -- 172.21.15.73:0/2718132766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 msgr2=0x7f41e40cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e40cdfe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 -- 172.21.15.73:0/2718132766 shutdown_connections 2024-04-05T22:21:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e40cdfe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 --2- 172.21.15.73:0/2718132766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 -- 172.21.15.73:0/2718132766 >> 172.21.15.73:0/2718132766 conn(0x7f41e40b0830 msgr2=0x7f41e40b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 -- 172.21.15.73:0/2718132766 shutdown_connections 2024-04-05T22:21:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.617+0000 7f41e3200640 1 -- 172.21.15.73:0/2718132766 wait complete. 2024-04-05T22:21:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.618+0000 7f41e3200640 1 Processor -- start 2024-04-05T22:21:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.619+0000 7f41e3200640 1 -- start start 2024-04-05T22:21:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.619+0000 7f41e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.619+0000 7f41e3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.619+0000 7f41e3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e4144260 0x7f41e41f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41599b0 con 0x7f41e40cdbb0 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e415bee0 con 0x7f41e4144260 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41e5fd0 con 0x7f41e41435f0 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44596/0 (socket says 172.21.15.73:44596) 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 -- 172.21.15.73:0/2433248456 learned_addr learned my addr 172.21.15.73:0/2433248456 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 -- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 msgr2=0x7f41e415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 -- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e4144260 msgr2=0x7f41e41f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e4144260 0x7f41e41f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 -- 172.21.15.73:0/2433248456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0094d0 con 0x7f41e40cdbb0 2024-04-05T22:21:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.620+0000 7f41e1e00640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41d400cf60 tx=0x7f41d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.621+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400d840 con 0x7f41e40cdbb0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.621+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400d9a0 con 0x7f41e40cdbb0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.621+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400dd30 con 0x7f41e40cdbb0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.621+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e41e62b0 con 0x7f41e40cdbb0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.621+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e415bb70 con 0x7f41e40cdbb0 2024-04-05T22:21:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.622+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e404e160 con 0x7f41e40cdbb0 2024-04-05T22:21:48.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.623+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f41d4015440 con 0x7f41e40cdbb0 2024-04-05T22:21:48.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.624+0000 7f41daa00640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 0x7f41c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:48.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.624+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41d4089d90 con 0x7f41e40cdbb0 2024-04-05T22:21:48.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.624+0000 7f41d9400640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 0x7f41c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:48.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.625+0000 7f41d9400640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 0x7f41c006d0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006fd0 tx=0x7f41cc006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:48.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.628+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f41d4055010 con 0x7f41e40cdbb0 2024-04-05T22:21:48.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.823+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41e406d290 con 0x7f41e40cdbb0 2024-04-05T22:21:48.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.823+0000 7f41daa00640 1 -- 172.21.15.73:0/2433248456 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41d401a020 con 0x7f41e40cdbb0 2024-04-05T22:21:48.824 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:48.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 msgr2=0x7f41c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 0x7f41c006d0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006fd0 tx=0x7f41cc006bc0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 msgr2=0x7f41e4159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41d400cf60 tx=0x7f41d400cf90 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 shutdown_connections 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41c006ac00 0x7f41c006d0c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e4144260 0x7f41e41f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e41435f0 0x7f41e415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 --2- 172.21.15.73:0/2433248456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41e40cdbb0 0x7f41e4159470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 >> 172.21.15.73:0/2433248456 conn(0x7f41e40b0830 msgr2=0x7f41e40cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 shutdown_connections 2024-04-05T22:21:48.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:48.826+0000 7f41e3200640 1 -- 172.21.15.73:0/2433248456 wait complete. 2024-04-05T22:21:48.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:49.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.055+0000 7f4f45600640 1 Processor -- start 2024-04-05T22:21:49.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.056+0000 7f4f45600640 1 -- start start 2024-04-05T22:21:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4f4013fa60 0x7f4f40142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fe40 0x7f4f401429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f4006b500 0x7f4f4006f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4006d3a0 con 0x7f4f4013fa60 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f40143330 con 0x7f4f4006b500 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f45600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f40143050 con 0x7f4f4013fe40 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.057+0000 7f4f3fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4f4013fa60 0x7f4f40142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4f4013fa60 0x7f4f40142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44612/0 (socket says 172.21.15.73:44612) 2024-04-05T22:21:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 -- 172.21.15.73:0/390657698 learned_addr learned my addr 172.21.15.73:0/390657698 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 -- 172.21.15.73:0/390657698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fe40 msgr2=0x7f4f401429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/390657698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fe40 0x7f4f401429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 -- 172.21.15.73:0/390657698 >> v2:172.21.15.157:3300/0 conn(0x7f4f4006b500 msgr2=0x7f4f4006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/390657698 >> v2:172.21.15.157:3300/0 conn(0x7f4f4006b500 0x7f4f4006f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.058+0000 7f4f3fe00640 1 -- 172.21.15.73:0/390657698 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400ac970 con 0x7f4f4013fa60 2024-04-05T22:21:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.059+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/390657698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4013fa60 0x7f4f40142490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f400b0650 tx=0x7f4f3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34f2dda195d3841 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f3f400640 1 -- 172.21.15.73:0/390657698 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30031870 con 0x7f4f4013fa60 2024-04-05T22:21:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f3f400640 1 -- 172.21.15.73:0/390657698 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f300319d0 con 0x7f4f4013fa60 2024-04-05T22:21:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f3f400640 1 -- 172.21.15.73:0/390657698 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30031d60 con 0x7f4f4013fa60 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 -- 172.21.15.73:0/390657698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4013fa60 msgr2=0x7f4f40142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 --2- 172.21.15.73:0/390657698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4013fa60 0x7f4f40142490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f400b0650 tx=0x7f4f3002b910 comp rx=0 tx=0).stop 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 -- 172.21.15.73:0/390657698 shutdown_connections 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 --2- 172.21.15.73:0/390657698 >> v2:172.21.15.157:3300/0 conn(0x7f4f4006b500 0x7f4f4006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 --2- 172.21.15.73:0/390657698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fe40 0x7f4f401429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 --2- 172.21.15.73:0/390657698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4013fa60 0x7f4f40142490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 -- 172.21.15.73:0/390657698 >> 172.21.15.73:0/390657698 conn(0x7f4f400b06c0 msgr2=0x7f4f400b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.060+0000 7f4f45600640 1 -- 172.21.15.73:0/390657698 shutdown_connections 2024-04-05T22:21:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.061+0000 7f4f45600640 1 -- 172.21.15.73:0/390657698 wait complete. 2024-04-05T22:21:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.062+0000 7f4f45600640 1 Processor -- start 2024-04-05T22:21:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.062+0000 7f4f45600640 1 -- start start 2024-04-05T22:21:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fa60 0x7f4f4013d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4013fe40 0x7f4f400cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4013d7d0 con 0x7f4f4006b500 2024-04-05T22:21:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400cbbd0 con 0x7f4f4013fe40 2024-04-05T22:21:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.063+0000 7f4f45600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400cbeb0 con 0x7f4f4013fa60 2024-04-05T22:21:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44614/0 (socket says 172.21.15.73:44614) 2024-04-05T22:21:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 -- 172.21.15.73:0/2023768837 learned_addr learned my addr 172.21.15.73:0/2023768837 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f44c00640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4013fe40 0x7f4f400cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f37400640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fa60 0x7f4f4013d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 -- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fa60 msgr2=0x7f4f4013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fa60 0x7f4f4013d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 -- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4013fe40 msgr2=0x7f4f400cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4013fe40 0x7f4f400cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.064+0000 7f4f3fe00640 1 -- 172.21.15.73:0/2023768837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300094d0 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f3fe00640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30031840 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f300443f0 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300447a0 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f400cc190 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.065+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f400ccda0 con 0x7f4f4006b500 2024-04-05T22:21:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.066+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400cd2e0 con 0x7f4f4006b500 2024-04-05T22:21:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.067+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f3004d450 con 0x7f4f4006b500 2024-04-05T22:21:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.068+0000 7f4f3cc00640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 0x7f4f1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.068+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f3002d040 con 0x7f4f4006b500 2024-04-05T22:21:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.068+0000 7f4f37400640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 0x7f4f1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.069+0000 7f4f37400640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 0x7f4f1c06d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f4013e7c0 tx=0x7f4f28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.072+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4f400cd2e0 con 0x7f4f4006b500 2024-04-05T22:21:49.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.266+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f400cd2e0 con 0x7f4f4006b500 2024-04-05T22:21:49.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.268+0000 7f4f3cc00640 1 -- 172.21.15.73:0/2023768837 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f4f30076d50 con 0x7f4f4006b500 2024-04-05T22:21:49.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:49.269 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 msgr2=0x7f4f1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 0x7f4f1c06d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f4013e7c0 tx=0x7f4f28008040 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 msgr2=0x7f4f4015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f300046c0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 shutdown_connections 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4f1c06ac00 0x7f4f1c06d0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4013fe40 0x7f4f400cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4013fa60 0x7f4f4013d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.270+0000 7f4f45600640 1 --2- 172.21.15.73:0/2023768837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f4006b500 0x7f4f4015d030 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.271+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 >> 172.21.15.73:0/2023768837 conn(0x7f4f400b06c0 msgr2=0x7f4f40142310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:49.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.271+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 shutdown_connections 2024-04-05T22:21:49.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.271+0000 7f4f45600640 1 -- 172.21.15.73:0/2023768837 wait complete. 2024-04-05T22:21:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:49.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:49.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.491+0000 7f270b000640 1 Processor -- start 2024-04-05T22:21:49.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.492+0000 7f270b000640 1 -- start start 2024-04-05T22:21:49.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f270b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 0x7f270415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f270b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270415ddd0 con 0x7f27040b0990 2024-04-05T22:21:49.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f270b000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270415e0b0 con 0x7f270415ce10 2024-04-05T22:21:49.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f270b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27040ce370 con 0x7f270415d1f0 2024-04-05T22:21:49.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f2709000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f27040b0990 0x7f27040b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33338/0 (socket says 172.21.15.73:33338) 2024-04-05T22:21:49.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.493+0000 7f2709000640 1 -- 172.21.15.73:0/987987841 learned_addr learned my addr 172.21.15.73:0/987987841 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2703400640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 0x7f270415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217221466 0 0) 0x7f270415ddd0 con 0x7f27040b0990 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8003520 con 0x7f27040b0990 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3369544211 0 0) 0x7f26e8003520 con 0x7f27040b0990 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 msgr2=0x7f270415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 0x7f270415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 >> v1:172.21.15.157:6789/0 conn(0x7f270415ce10 legacy=0x7f27040cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040cd460 con 0x7f27040b0990 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f26fc001670 con 0x7f27040b0990 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 >> v1:172.21.15.73:6789/0 conn(0x7f27040b0990 legacy=0x7f27040b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 0x7f26e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8004d00 0x7f26e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27040ce370 con 0x7f26e8003f20 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800ba60 con 0x7f26e8004d00 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800bc20 con 0x7f270415d1f0 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.494+0000 7f2703400640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.495+0000 7f2703400640 1 -- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 msgr2=0x7f26e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.495+0000 7f2703400640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f270415d1f0 0x7f26e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.495+0000 7f2703400640 1 -- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8004d00 msgr2=0x7f26e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.495+0000 7f2703400640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e8004d00 0x7f26e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.495+0000 7f2703400640 1 -- 172.21.15.73:0/987987841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8003520 con 0x7f26e8003f20 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.496+0000 7f2703400640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009fc0 tx=0x7f26e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d63202ed8200c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.496+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f402cc60 con 0x7f26e8003f20 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.496+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f400f040 con 0x7f26e8003f20 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.496+0000 7f2702a00640 1 -- 172.21.15.73:0/987987841 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4034440 con 0x7f26e8003f20 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 -- 172.21.15.73:0/987987841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 msgr2=0x7f26e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 --2- 172.21.15.73:0/987987841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009fc0 tx=0x7f26e80057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f2709000640 1 -- 172.21.15.73:0/987987841 reap_dead start 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 -- 172.21.15.73:0/987987841 shutdown_connections 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 -- 172.21.15.73:0/987987841 >> 172.21.15.73:0/987987841 conn(0x7f2704099a60 msgr2=0x7f2704099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 -- 172.21.15.73:0/987987841 shutdown_connections 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.497+0000 7f270b000640 1 -- 172.21.15.73:0/987987841 wait complete. 2024-04-05T22:21:49.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.499+0000 7f270b000640 1 Processor -- start 2024-04-05T22:21:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.499+0000 7f270b000640 1 -- start start 2024-04-05T22:21:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27040c51c0 0x7f27040ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 0x7f27040b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27040b36f0 con 0x7f26e8003f20 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041592f0 con 0x7f27040aed80 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f270b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27040af200 con 0x7f27040c51c0 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44626/0 (socket says 172.21.15.73:44626) 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 -- 172.21.15.73:0/264424734 learned_addr learned my addr 172.21.15.73:0/264424734 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2709000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 0x7f27040b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703400640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27040c51c0 0x7f27040ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 -- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27040c51c0 msgr2=0x7f27040ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27040c51c0 0x7f27040ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 -- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 msgr2=0x7f27040b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 0x7f27040b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2703e00640 1 -- 172.21.15.73:0/264424734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270415b550 con 0x7f26e8003f20 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.500+0000 7f2709000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 0x7f27040b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f2703e00640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c500 tx=0x7f26ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0052d0 con 0x7f26e8003f20 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec005430 con 0x7f26e8003f20 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec012870 con 0x7f26e8003f20 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270415b830 con 0x7f26e8003f20 2024-04-05T22:21:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.501+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270415bdf0 con 0x7f26e8003f20 2024-04-05T22:21:49.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.502+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2704067480 con 0x7f26e8003f20 2024-04-05T22:21:49.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.503+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f26ec0129d0 con 0x7f26e8003f20 2024-04-05T22:21:49.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.504+0000 7f26fbe00640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 0x7f26e006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.504+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26ec088ee0 con 0x7f26e8003f20 2024-04-05T22:21:49.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.504+0000 7f2703400640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 0x7f26e006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.505+0000 7f2703400640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 0x7f26e006d0f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26f4036000 tx=0x7f26f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.508+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f26ec054150 con 0x7f26e8003f20 2024-04-05T22:21:49.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.688+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2704068160 con 0x7f26e8003f20 2024-04-05T22:21:49.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.689+0000 7f26fbe00640 1 -- 172.21.15.73:0/264424734 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26ec057770 con 0x7f26e8003f20 2024-04-05T22:21:49.689 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 msgr2=0x7f26e006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 0x7f26e006d0f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26f4036000 tx=0x7f26f402e000 comp rx=0 tx=0).stop 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 msgr2=0x7f2704158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c500 tx=0x7f26ec00c9c0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 shutdown_connections 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26e006ac30 0x7f26e006d0f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27040aed80 0x7f27040b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27040c51c0 0x7f27040ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 --2- 172.21.15.73:0/264424734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e8003f20 0x7f2704158db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.691+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 >> 172.21.15.73:0/264424734 conn(0x7f2704099a60 msgr2=0x7f270406efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:49.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.692+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 shutdown_connections 2024-04-05T22:21:49.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.692+0000 7f270b000640 1 -- 172.21.15.73:0/264424734 wait complete. 2024-04-05T22:21:49.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:49.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.912+0000 7f21e2c00640 1 Processor -- start 2024-04-05T22:21:49.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.913+0000 7f21e2c00640 1 -- start start 2024-04-05T22:21:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15ce10 0x7f21dc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21dc15d1f0 0x7f21dc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f21dc0b0990 0x7f21dc0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc15dbb0 con 0x7f21dc15d1f0 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc15ddd0 con 0x7f21dc0b0990 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21e2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc15e0b0 con 0x7f21dc15ce10 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.914+0000 7f21db400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21dc15d1f0 0x7f21dc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21dc15d1f0 0x7f21dc15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44628/0 (socket says 172.21.15.73:44628) 2024-04-05T22:21:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 -- 172.21.15.73:0/1378912086 learned_addr learned my addr 172.21.15.73:0/1378912086 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21e0c00640 1 --2- 172.21.15.73:0/1378912086 >> v2:172.21.15.157:3300/0 conn(0x7f21dc0b0990 0x7f21dc0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21dbe00640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15ce10 0x7f21dc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 -- 172.21.15.73:0/1378912086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15ce10 msgr2=0x7f21dc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15ce10 0x7f21dc0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 -- 172.21.15.73:0/1378912086 >> v2:172.21.15.157:3300/0 conn(0x7f21dc0b0990 msgr2=0x7f21dc0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 --2- 172.21.15.73:0/1378912086 >> v2:172.21.15.157:3300/0 conn(0x7f21dc0b0990 0x7f21dc0b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.915+0000 7f21db400640 1 -- 172.21.15.73:0/1378912086 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0cd460 con 0x7f21dc15d1f0 2024-04-05T22:21:49.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.916+0000 7f21db400640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc15d1f0 0x7f21dc15d670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06a780 tx=0x7f21c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bafecdea7120a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.916+0000 7f21daa00640 1 -- 172.21.15.73:0/1378912086 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4031870 con 0x7f21dc15d1f0 2024-04-05T22:21:49.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.916+0000 7f21daa00640 1 -- 172.21.15.73:0/1378912086 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40319d0 con 0x7f21dc15d1f0 2024-04-05T22:21:49.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.916+0000 7f21daa00640 1 -- 172.21.15.73:0/1378912086 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4031d60 con 0x7f21dc15d1f0 2024-04-05T22:21:49.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 -- 172.21.15.73:0/1378912086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc15d1f0 msgr2=0x7f21dc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc15d1f0 0x7f21dc15d670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06a780 tx=0x7f21c402b910 comp rx=0 tx=0).stop 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 -- 172.21.15.73:0/1378912086 shutdown_connections 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc15d1f0 0x7f21dc15d670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1378912086 >> v2:172.21.15.157:3300/0 conn(0x7f21dc0b0990 0x7f21dc0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1378912086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15ce10 0x7f21dc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 -- 172.21.15.73:0/1378912086 >> 172.21.15.73:0/1378912086 conn(0x7f21dc099a60 msgr2=0x7f21dc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.917+0000 7f21e2c00640 1 -- 172.21.15.73:0/1378912086 shutdown_connections 2024-04-05T22:21:49.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.918+0000 7f21e2c00640 1 -- 172.21.15.73:0/1378912086 wait complete. 2024-04-05T22:21:49.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.919+0000 7f21e2c00640 1 Processor -- start 2024-04-05T22:21:49.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.919+0000 7f21e2c00640 1 -- start start 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15ce10 0x7f21dc1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d1f0 0x7f21dc0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc156830 con 0x7f21dc0b0990 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0ad990 con 0x7f21dc15ce10 2024-04-05T22:21:49.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.920+0000 7f21e2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0adc30 con 0x7f21dc15d1f0 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44634/0 (socket says 172.21.15.73:44634) 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 -- 172.21.15.73:0/1045874872 learned_addr learned my addr 172.21.15.73:0/1045874872 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 -- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d1f0 msgr2=0x7f21dc0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d1f0 0x7f21dc0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 -- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15ce10 msgr2=0x7f21dc1f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15ce10 0x7f21dc1f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 -- 172.21.15.73:0/1045874872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40094d0 con 0x7f21dc0b0990 2024-04-05T22:21:49.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.921+0000 7f21dbe00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00d800 tx=0x7f21cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.922+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00b6c0 con 0x7f21dc0b0990 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.922+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b820 con 0x7f21dc0b0990 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.922+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc0adf10 con 0x7f21dc0b0990 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.922+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bbb0 con 0x7f21dc0b0990 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.922+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc0ae4d0 con 0x7f21dc0b0990 2024-04-05T22:21:49.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.923+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc0ae660 con 0x7f21dc0b0990 2024-04-05T22:21:49.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.924+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f21cc00bd10 con 0x7f21dc0b0990 2024-04-05T22:21:49.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.925+0000 7f21d3e00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:49.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.925+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21cc089900 con 0x7f21dc0b0990 2024-04-05T22:21:49.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.925+0000 7f21db400640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:49.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.926+0000 7f21db400640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 0x7f21b806d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21c4036000 tx=0x7f21c4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:49.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:49.930+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f21cc054ae0 con 0x7f21dc0b0990 2024-04-05T22:21:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.122+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21dc067480 con 0x7f21dc0b0990 2024-04-05T22:21:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.123+0000 7f21d3e00640 1 -- 172.21.15.73:0/1045874872 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21cc058100 con 0x7f21dc0b0990 2024-04-05T22:21:50.123 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 msgr2=0x7f21b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 0x7f21b806d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21c4036000 tx=0x7f21c4030000 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 msgr2=0x7f21dc1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00d800 tx=0x7f21cc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 shutdown_connections 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f21b806ac00 0x7f21b806d0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d1f0 0x7f21dc0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15ce10 0x7f21dc1f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 --2- 172.21.15.73:0/1045874872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21dc0b0990 0x7f21dc1562f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 >> 172.21.15.73:0/1045874872 conn(0x7f21dc099a60 msgr2=0x7f21dc0b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 shutdown_connections 2024-04-05T22:21:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.126+0000 7f21e2c00640 1 -- 172.21.15.73:0/1045874872 wait complete. 2024-04-05T22:21:50.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.348+0000 7f08cfa00640 1 Processor -- start 2024-04-05T22:21:50.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.348+0000 7f08cfa00640 1 -- start start 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cfa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08c806c630 0x7f08c806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cfa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cfa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c80b1b30 con 0x7f08c80b22a0 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cfa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80b1e10 con 0x7f08c806c630 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cfa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c806cf50 con 0x7f08c80cbe00 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08c806c630 0x7f08c806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08c806c630 0x7f08c806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44640/0 (socket says 172.21.15.73:44640) 2024-04-05T22:21:50.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cda00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f08c80b22a0 0x7f08c80b2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33340/0 (socket says 172.21.15.73:33340) 2024-04-05T22:21:50.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cd000640 1 -- 172.21.15.73:0/1502524180 learned_addr learned my addr 172.21.15.73:0/1502524180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:50.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08cd000640 1 -- 172.21.15.73:0/1502524180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 msgr2=0x7f08c80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:50.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08c7e00640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.350+0000 7f08c7400640 1 -- 172.21.15.73:0/1502524180 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087041111 0 0) 0x7f08c80b1b30 con 0x7f08c80b22a0 2024-04-05T22:21:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.351+0000 7f08cd000640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.351+0000 7f08cd000640 1 -- 172.21.15.73:0/1502524180 >> v1:172.21.15.73:6789/0 conn(0x7f08c80b22a0 legacy=0x7f08c80b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.351+0000 7f08cd000640 1 -- 172.21.15.73:0/1502524180 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80cc300 con 0x7f08c806c630 2024-04-05T22:21:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.351+0000 7f08c7e00640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.352+0000 7f08cd000640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c806c630 0x7f08c806ca10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009870 tx=0x7f08bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73ffd9b9749a0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.352+0000 7f08c7400640 1 -- 172.21.15.73:0/1502524180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc030840 con 0x7f08c806c630 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.352+0000 7f08c7400640 1 -- 172.21.15.73:0/1502524180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc0309a0 con 0x7f08c806c630 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.352+0000 7f08c7400640 1 -- 172.21.15.73:0/1502524180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc030d30 con 0x7f08c806c630 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 -- 172.21.15.73:0/1502524180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c806c630 msgr2=0x7f08c806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c806c630 0x7f08c806ca10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009870 tx=0x7f08bc02b980 comp rx=0 tx=0).stop 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 -- 172.21.15.73:0/1502524180 shutdown_connections 2024-04-05T22:21:50.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1502524180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c806c630 0x7f08c806ca10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 -- 172.21.15.73:0/1502524180 >> 172.21.15.73:0/1502524180 conn(0x7f08c80b0830 msgr2=0x7f08c80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:50.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 -- 172.21.15.73:0/1502524180 shutdown_connections 2024-04-05T22:21:50.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.353+0000 7f08cfa00640 1 -- 172.21.15.73:0/1502524180 wait complete. 2024-04-05T22:21:50.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.354+0000 7f08cfa00640 1 Processor -- start 2024-04-05T22:21:50.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.355+0000 7f08cfa00640 1 -- start start 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.355+0000 7f08cfa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c806c630 0x7f08c8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.355+0000 7f08cfa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.355+0000 7f08cfa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08cfa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c815a540 con 0x7f08c80b22a0 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08cfa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c813f8e0 con 0x7f08c806c630 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08cfa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c813fbc0 con 0x7f08c80cbe00 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44648/0 (socket says 172.21.15.73:44648) 2024-04-05T22:21:50.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 -- 172.21.15.73:0/1194545496 learned_addr learned my addr 172.21.15.73:0/1194545496 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:50.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 -- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 msgr2=0x7f08c80ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 -- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c806c630 msgr2=0x7f08c8159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c806c630 0x7f08c8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 -- 172.21.15.73:0/1194545496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0094d0 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.356+0000 7f08c7e00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c7f0 tx=0x7f08b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.357+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4007820 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.357+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4007980 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.357+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c813fea0 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.357+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4007cf0 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.357+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c8140460 con 0x7f08c80b22a0 2024-04-05T22:21:50.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.358+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c8068160 con 0x7f08c80b22a0 2024-04-05T22:21:50.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.359+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08b40113f0 con 0x7f08c80b22a0 2024-04-05T22:21:50.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.360+0000 7f08c4c00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 0x7f08a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.360+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08b40896c0 con 0x7f08c80b22a0 2024-04-05T22:21:50.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.360+0000 7f08cd000640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 0x7f08a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.361+0000 7f08cd000640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 0x7f08a806d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08bc036000 tx=0x7f08bc03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.364+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f08b40548a0 con 0x7f08c80b22a0 2024-04-05T22:21:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.558+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08c80cb170 con 0x7f08c80b22a0 2024-04-05T22:21:50.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.559+0000 7f08c4c00640 1 -- 172.21.15.73:0/1194545496 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08b4057ec0 con 0x7f08c80b22a0 2024-04-05T22:21:50.559 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:50.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 msgr2=0x7f08a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 0x7f08a806d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08bc036000 tx=0x7f08bc03b040 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 msgr2=0x7f08c8140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c7f0 tx=0x7f08b400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 shutdown_connections 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f08a806ac00 0x7f08a806d0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cbe00 0x7f08c80ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08c80b22a0 0x7f08c8140980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 --2- 172.21.15.73:0/1194545496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c806c630 0x7f08c8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.561+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 >> 172.21.15.73:0/1194545496 conn(0x7f08c80b0830 msgr2=0x7f08c815b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.562+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 shutdown_connections 2024-04-05T22:21:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.562+0000 7f08cfa00640 1 -- 172.21.15.73:0/1194545496 wait complete. 2024-04-05T22:21:50.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:50.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.778+0000 7f960f000640 1 Processor -- start 2024-04-05T22:21:50.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.778+0000 7f960f000640 1 -- start start 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f960f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f960806cb80 0x7f960806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f960f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960815e850 0x7f960815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f960f000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960806f5b0 con 0x7f96081587d0 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f960f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960806d500 con 0x7f960806cb80 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f960f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960815f220 con 0x7f960815e850 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f95ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f960806cb80 0x7f960806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f95ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f960806cb80 0x7f960806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44656/0 (socket says 172.21.15.73:44656) 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.780+0000 7f95ff400640 1 -- 172.21.15.73:0/318172632 learned_addr learned my addr 172.21.15.73:0/318172632 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.781+0000 7f960d000640 1 --2- 172.21.15.73:0/318172632 >> v2:172.21.15.157:3300/0 conn(0x7f960815e850 0x7f960815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.781+0000 7f95ff400640 1 -- 172.21.15.73:0/318172632 >> v2:172.21.15.157:3300/0 conn(0x7f960815e850 msgr2=0x7f960815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.781+0000 7f95ff400640 1 --2- 172.21.15.73:0/318172632 >> v2:172.21.15.157:3300/0 conn(0x7f960815e850 0x7f960815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.781+0000 7f95ff400640 1 -- 172.21.15.73:0/318172632 >> v1:172.21.15.157:6789/0 conn(0x7f96081587d0 legacy=0x7f96080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:50.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.781+0000 7f95ff400640 1 -- 172.21.15.73:0/318172632 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608070410 con 0x7f960806cb80 2024-04-05T22:21:50.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.782+0000 7f95ff400640 1 --2- 172.21.15.73:0/318172632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960806cfc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f960815c270 tx=0x7f95f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cacaae9e5689f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.782+0000 7f9607400640 1 -- 172.21.15.73:0/318172632 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0031870 con 0x7f960806cb80 2024-04-05T22:21:50.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.782+0000 7f9607400640 1 -- 172.21.15.73:0/318172632 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00319d0 con 0x7f960806cb80 2024-04-05T22:21:50.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.782+0000 7f9607400640 1 -- 172.21.15.73:0/318172632 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0031d60 con 0x7f960806cb80 2024-04-05T22:21:50.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 -- 172.21.15.73:0/318172632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 msgr2=0x7f960806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 --2- 172.21.15.73:0/318172632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960806cfc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f960815c270 tx=0x7f95f002b910 comp rx=0 tx=0).stop 2024-04-05T22:21:50.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 -- 172.21.15.73:0/318172632 shutdown_connections 2024-04-05T22:21:50.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 --2- 172.21.15.73:0/318172632 >> v2:172.21.15.157:3300/0 conn(0x7f960815e850 0x7f960815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 --2- 172.21.15.73:0/318172632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960806cfc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 -- 172.21.15.73:0/318172632 >> 172.21.15.73:0/318172632 conn(0x7f96080ce8e0 msgr2=0x7f96080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:50.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 -- 172.21.15.73:0/318172632 shutdown_connections 2024-04-05T22:21:50.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.783+0000 7f960f000640 1 -- 172.21.15.73:0/318172632 wait complete. 2024-04-05T22:21:50.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.785+0000 7f960f000640 1 Processor -- start 2024-04-05T22:21:50.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.785+0000 7f960f000640 1 -- start start 2024-04-05T22:21:50.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96081587d0 0x7f96080b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960815e850 0x7f96080b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960814bf40 con 0x7f960806cb80 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960814ee50 con 0x7f960815e850 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f960f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080c37e0 con 0x7f96081587d0 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.786+0000 7f9607e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44658/0 (socket says 172.21.15.73:44658) 2024-04-05T22:21:50.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 -- 172.21.15.73:0/3446262648 learned_addr learned my addr 172.21.15.73:0/3446262648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f95ff400640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96081587d0 0x7f96080b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 -- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96081587d0 msgr2=0x7f96080b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96081587d0 0x7f96080b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 -- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960815e850 msgr2=0x7f96080b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960815e850 0x7f96080b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 -- 172.21.15.73:0/3446262648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00094d0 con 0x7f960806cb80 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.787+0000 7f9607e00640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c410 tx=0x7f95f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.788+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80035f0 con 0x7f960806cb80 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.788+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8003750 con 0x7f960806cb80 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.788+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8003b40 con 0x7f960806cb80 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.788+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96080b33f0 con 0x7f960806cb80 2024-04-05T22:21:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.788+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96080b2760 con 0x7f960806cb80 2024-04-05T22:21:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.789+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9608066af0 con 0x7f960806cb80 2024-04-05T22:21:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.790+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f95f8003ca0 con 0x7f960806cb80 2024-04-05T22:21:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.790+0000 7f9604c00640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 0x7f95e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.791+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95f8088610 con 0x7f960806cb80 2024-04-05T22:21:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.791+0000 7f95ff400640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 0x7f95e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.791+0000 7f95ff400640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 0x7f95e406d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f95f0036000 tx=0x7f95f003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.795+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f95f8053750 con 0x7f960806cb80 2024-04-05T22:21:50.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.991+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9608068160 con 0x7f960806cb80 2024-04-05T22:21:50.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.994+0000 7f9604c00640 1 -- 172.21.15.73:0/3446262648 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f95f8056d70 con 0x7f960806cb80 2024-04-05T22:21:50.994 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:50.995 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 msgr2=0x7f95e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 0x7f95e406d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f95f0036000 tx=0x7f95f003c040 comp rx=0 tx=0).stop 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 msgr2=0x7f960814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c410 tx=0x7f95f800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 shutdown_connections 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.996+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95e406ac00 0x7f95e406d0c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960815e850 0x7f96080b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96081587d0 0x7f96080b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 --2- 172.21.15.73:0/3446262648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f960806cb80 0x7f960814e910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 >> 172.21.15.73:0/3446262648 conn(0x7f96080ce8e0 msgr2=0x7f9608158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 shutdown_connections 2024-04-05T22:21:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:50.997+0000 7f960f000640 1 -- 172.21.15.73:0/3446262648 wait complete. 2024-04-05T22:21:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:51.017 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:51.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.228+0000 7f449fe00640 1 Processor -- start 2024-04-05T22:21:51.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.228+0000 7f449fe00640 1 -- start start 2024-04-05T22:21:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.229+0000 7f449fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44a006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a00715a0 con 0x7f44a015e0d0 2024-04-05T22:21:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449fe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a015d5f0 con 0x7f44a00cb7f0 2024-04-05T22:21:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0071880 con 0x7f44a0154090 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449e000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f44a015e0d0 0x7f44a00cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33342/0 (socket says 172.21.15.73:33342) 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449e000640 1 -- 172.21.15.73:0/2873238095 learned_addr learned my addr 172.21.15.73:0/2873238095 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449ea00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44a006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.230+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189972916 0 0) 0x7f44a00715a0 con 0x7f44a015e0d0 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4484005730 con 0x7f44a015e0d0 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623788279 0 0) 0x7f4484005730 con 0x7f44a015e0d0 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 msgr2=0x7f44a006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44a006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 >> v1:172.21.15.157:6789/0 conn(0x7f44a00cb7f0 legacy=0x7f44a015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a015d8d0 con 0x7f44a015e0d0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4488001690 con 0x7f44a015e0d0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 >> v1:172.21.15.73:6789/0 conn(0x7f44a015e0d0 legacy=0x7f44a00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4484005bb0 0x7f4484005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44840069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4484006f10 0x7f4484007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0071880 con 0x7f4484005bb0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448400dc40 con 0x7f4484006f10 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448400de00 con 0x7f44a0154090 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.231+0000 7f449ea00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4484005bb0 0x7f4484005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449e000640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44840069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449f400640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4484006f10 0x7f4484007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449ea00640 1 -- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 msgr2=0x7f44840069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449ea00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a0154090 0x7f44840069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449ea00640 1 -- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4484006f10 msgr2=0x7f4484007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449ea00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4484006f10 0x7f4484007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.232+0000 7f449ea00640 1 -- 172.21.15.73:0/2873238095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484005730 con 0x7f4484005bb0 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.233+0000 7f449ea00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4484005bb0 0x7f4484005fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4490009a90 tx=0x7f4484007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d7a1d79c2865ad server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449002c680 con 0x7f4484005bb0 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449002c7e0 con 0x7f4484005bb0 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449d600640 1 -- 172.21.15.73:0/2873238095 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449002cb70 con 0x7f4484005bb0 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 -- 172.21.15.73:0/2873238095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4484005bb0 msgr2=0x7f4484005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 --2- 172.21.15.73:0/2873238095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4484005bb0 0x7f4484005fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4490009a90 tx=0x7f4484007a00 comp rx=0 tx=0).stop 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449f400640 1 -- 172.21.15.73:0/2873238095 reap_dead start 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 -- 172.21.15.73:0/2873238095 shutdown_connections 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 -- 172.21.15.73:0/2873238095 >> 172.21.15.73:0/2873238095 conn(0x7f44a0069cb0 msgr2=0x7f44a006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 -- 172.21.15.73:0/2873238095 shutdown_connections 2024-04-05T22:21:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.234+0000 7f449fe00640 1 -- 172.21.15.73:0/2873238095 wait complete. 2024-04-05T22:21:51.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.236+0000 7f449fe00640 1 Processor -- start 2024-04-05T22:21:51.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.236+0000 7f449fe00640 1 -- start start 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a014d5e0 0x7f44a01f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a01522c0 0x7f44a0152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0153530 con 0x7f44a0155170 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0152c80 con 0x7f44a01522c0 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.237+0000 7f449fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a01555f0 con 0x7f44a014d5e0 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44686/0 (socket says 172.21.15.73:44686) 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 -- 172.21.15.73:0/2681568248 learned_addr learned my addr 172.21.15.73:0/2681568248 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449ea00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a014d5e0 0x7f44a01f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 -- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a014d5e0 msgr2=0x7f44a01f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a014d5e0 0x7f44a01f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449f400640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a01522c0 0x7f44a0152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 -- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a01522c0 msgr2=0x7f44a0152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a01522c0 0x7f44a0152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 -- 172.21.15.73:0/2681568248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a01f2ae0 con 0x7f44a0155170 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.238+0000 7f449e000640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f448800eb20 tx=0x7f44880016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.239+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800c870 con 0x7f44a0155170 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.239+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800c9d0 con 0x7f44a0155170 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.239+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a0155850 con 0x7f44a0155170 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.239+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800cd60 con 0x7f44a0155170 2024-04-05T22:21:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.239+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a01f2da0 con 0x7f44a0155170 2024-04-05T22:21:51.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.240+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a0155d70 con 0x7f44a0155170 2024-04-05T22:21:51.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.241+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4488013420 con 0x7f44a0155170 2024-04-05T22:21:51.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.242+0000 7f4496a00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 0x7f447c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.242+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4488088fe0 con 0x7f44a0155170 2024-04-05T22:21:51.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.242+0000 7f449ea00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 0x7f447c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.243+0000 7f449ea00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 0x7f447c06d0f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4490036000 tx=0x7f4490030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.246+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f44880541d0 con 0x7f44a0155170 2024-04-05T22:21:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.437+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44a004e160 con 0x7f44a0155170 2024-04-05T22:21:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.438+0000 7f4496a00640 1 -- 172.21.15.73:0/2681568248 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44880577f0 con 0x7f44a0155170 2024-04-05T22:21:51.438 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.440+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 msgr2=0x7f447c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.440+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 0x7f447c06d0f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4490036000 tx=0x7f4490030000 comp rx=0 tx=0).stop 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.440+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 msgr2=0x7f44a0152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.440+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f448800eb20 tx=0x7f44880016e0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 shutdown_connections 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f447c06ac30 0x7f447c06d0f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a01522c0 0x7f44a0152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0155170 0x7f44a0152ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 --2- 172.21.15.73:0/2681568248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a014d5e0 0x7f44a01f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 >> 172.21.15.73:0/2681568248 conn(0x7f44a0069cb0 msgr2=0x7f44a00ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:51.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 shutdown_connections 2024-04-05T22:21:51.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.441+0000 7f449fe00640 1 -- 172.21.15.73:0/2681568248 wait complete. 2024-04-05T22:21:51.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:51.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.666+0000 7f1bab200640 1 Processor -- start 2024-04-05T22:21:51.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.666+0000 7f1bab200640 1 -- start start 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1bab200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ba406c9a0 0x7f1ba406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1bab200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba406bea0 con 0x7f1ba415abe0 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1bab200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba415afc0 con 0x7f1ba406ba70 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1bab200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba406f5b0 con 0x7f1ba406c9a0 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ba406c9a0 0x7f1ba406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ba406c9a0 0x7f1ba406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44694/0 (socket says 172.21.15.73:44694) 2024-04-05T22:21:51.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 -- 172.21.15.73:0/3220597642 learned_addr learned my addr 172.21.15.73:0/3220597642 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:51.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 -- 172.21.15.73:0/3220597642 >> v1:172.21.15.157:6789/0 conn(0x7f1ba406ba70 legacy=0x7f1ba4164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:51.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 -- 172.21.15.73:0/3220597642 >> v1:172.21.15.73:6789/0 conn(0x7f1ba415abe0 legacy=0x7f1ba40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.668+0000 7f1b9b400640 1 -- 172.21.15.73:0/3220597642 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba406f890 con 0x7f1ba406c9a0 2024-04-05T22:21:51.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.669+0000 7f1b9b400640 1 --2- 172.21.15.73:0/3220597642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba406c9a0 0x7f1ba406b530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009870 tx=0x7f1b8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb02929edff258b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.670+0000 7f1ba3400640 1 -- 172.21.15.73:0/3220597642 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0317e0 con 0x7f1ba406c9a0 2024-04-05T22:21:51.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.670+0000 7f1ba3400640 1 -- 172.21.15.73:0/3220597642 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c031940 con 0x7f1ba406c9a0 2024-04-05T22:21:51.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.670+0000 7f1ba3400640 1 -- 172.21.15.73:0/3220597642 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c031cd0 con 0x7f1ba406c9a0 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.670+0000 7f1bab200640 1 -- 172.21.15.73:0/3220597642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba406c9a0 msgr2=0x7f1ba406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.670+0000 7f1bab200640 1 --2- 172.21.15.73:0/3220597642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba406c9a0 0x7f1ba406b530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009870 tx=0x7f1b8c02b880 comp rx=0 tx=0).stop 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.671+0000 7f1bab200640 1 -- 172.21.15.73:0/3220597642 shutdown_connections 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.671+0000 7f1bab200640 1 --2- 172.21.15.73:0/3220597642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba406c9a0 0x7f1ba406b530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.671+0000 7f1bab200640 1 -- 172.21.15.73:0/3220597642 >> 172.21.15.73:0/3220597642 conn(0x7f1ba4069cb0 msgr2=0x7f1ba406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.671+0000 7f1bab200640 1 -- 172.21.15.73:0/3220597642 shutdown_connections 2024-04-05T22:21:51.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.671+0000 7f1bab200640 1 -- 172.21.15.73:0/3220597642 wait complete. 2024-04-05T22:21:51.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.672+0000 7f1bab200640 1 Processor -- start 2024-04-05T22:21:51.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.673+0000 7f1bab200640 1 -- start start 2024-04-05T22:21:51.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.673+0000 7f1bab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ba406ba70 0x7f1ba4149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1bab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ba406c9a0 0x7f1ba415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1bab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1bab200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba415bf80 con 0x7f1ba415abe0 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1bab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba415c260 con 0x7f1ba406c9a0 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1bab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba415c540 con 0x7f1ba406ba70 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44708/0 (socket says 172.21.15.73:44708) 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 -- 172.21.15.73:0/1357887695 learned_addr learned my addr 172.21.15.73:0/1357887695 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1b9b400640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ba406c9a0 0x7f1ba415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba3e00640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ba406ba70 0x7f1ba4149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 -- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ba406ba70 msgr2=0x7f1ba4149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ba406ba70 0x7f1ba4149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 -- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ba406c9a0 msgr2=0x7f1ba415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ba406c9a0 0x7f1ba415b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.674+0000 7f1ba9200640 1 -- 172.21.15.73:0/1357887695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0094d0 con 0x7f1ba415abe0 2024-04-05T22:21:51.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1ba9200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c780 tx=0x7f1b9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003d80 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c00ce90 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba4150e30 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0052b0 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.675+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba4148f00 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.676+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba4068160 con 0x7f1ba415abe0 2024-04-05T22:21:51.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.677+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1b9c005410 con 0x7f1ba415abe0 2024-04-05T22:21:51.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.678+0000 7f1ba0c00640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 0x7f1b8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:51.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.679+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b9c088bb0 con 0x7f1ba415abe0 2024-04-05T22:21:51.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.679+0000 7f1ba3e00640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 0x7f1b8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:51.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.679+0000 7f1ba3e00640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 0x7f1b8006d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000f60 tx=0x7f1b94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:51.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.683+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1b9c053d80 con 0x7f1ba415abe0 2024-04-05T22:21:51.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.873+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ba415c0c0 con 0x7f1ba415abe0 2024-04-05T22:21:51.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.874+0000 7f1ba0c00640 1 -- 172.21.15.73:0/1357887695 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ba415c0c0 con 0x7f1ba415abe0 2024-04-05T22:21:51.875 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:51.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 msgr2=0x7f1b8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 0x7f1b8006d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000f60 tx=0x7f1b94009040 comp rx=0 tx=0).stop 2024-04-05T22:21:51.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 msgr2=0x7f1ba415b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:51.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c780 tx=0x7f1b9c00cc40 comp rx=0 tx=0).stop 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 shutdown_connections 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1b8006ac00 0x7f1b8006d0c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ba415abe0 0x7f1ba415b920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ba406c9a0 0x7f1ba415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 --2- 172.21.15.73:0/1357887695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ba406ba70 0x7f1ba4149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 >> 172.21.15.73:0/1357887695 conn(0x7f1ba4069cb0 msgr2=0x7f1ba40ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 shutdown_connections 2024-04-05T22:21:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:51.877+0000 7f1bab200640 1 -- 172.21.15.73:0/1357887695 wait complete. 2024-04-05T22:21:51.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:52.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.108+0000 7fd319000640 1 Processor -- start 2024-04-05T22:21:52.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.109+0000 7fd319000640 1 -- start start 2024-04-05T22:21:52.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd319000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd314157410 0x7fd3140b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd319000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd31415af80 0x7fd3140cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd319000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3141577f0 con 0x7fd3140cd570 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd319000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3140b1e90 con 0x7fd314157410 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd319000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3140cb630 con 0x7fd31415af80 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.110+0000 7fd313400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd314157410 0x7fd3140b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd314157410 0x7fd3140b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44716/0 (socket says 172.21.15.73:44716) 2024-04-05T22:21:52.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 -- 172.21.15.73:0/2526560129 learned_addr learned my addr 172.21.15.73:0/2526560129 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd30aa00640 1 --2- 172.21.15.73:0/2526560129 >> v2:172.21.15.157:3300/0 conn(0x7fd31415af80 0x7fd3140cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 -- 172.21.15.73:0/2526560129 >> v2:172.21.15.157:3300/0 conn(0x7fd31415af80 msgr2=0x7fd3140cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 --2- 172.21.15.73:0/2526560129 >> v2:172.21.15.157:3300/0 conn(0x7fd31415af80 0x7fd3140cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 -- 172.21.15.73:0/2526560129 >> v1:172.21.15.73:6789/0 conn(0x7fd3140cd570 legacy=0x7fd31415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd313400640 1 -- 172.21.15.73:0/2526560129 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140b20a0 con 0x7fd314157410 2024-04-05T22:21:52.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.111+0000 7fd312a00640 1 -- 172.21.15.73:0/2526560129 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101480058 0 0) 0x7fd3141577f0 con 0x7fd3140cd570 2024-04-05T22:21:52.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.112+0000 7fd313400640 1 --2- 172.21.15.73:0/2526560129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314157410 0x7fd3140b1950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd3140b1710 tx=0x7fd30402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457f4cf03b2c5b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.113+0000 7fd312a00640 1 -- 172.21.15.73:0/2526560129 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304030860 con 0x7fd314157410 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.113+0000 7fd312a00640 1 -- 172.21.15.73:0/2526560129 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3040309c0 con 0x7fd314157410 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.113+0000 7fd312a00640 1 -- 172.21.15.73:0/2526560129 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304030d50 con 0x7fd314157410 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.113+0000 7fd319000640 1 -- 172.21.15.73:0/2526560129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314157410 msgr2=0x7fd3140b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.113+0000 7fd319000640 1 --2- 172.21.15.73:0/2526560129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314157410 0x7fd3140b1950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd3140b1710 tx=0x7fd30402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 -- 172.21.15.73:0/2526560129 shutdown_connections 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 --2- 172.21.15.73:0/2526560129 >> v2:172.21.15.157:3300/0 conn(0x7fd31415af80 0x7fd3140cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 --2- 172.21.15.73:0/2526560129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314157410 0x7fd3140b1950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 -- 172.21.15.73:0/2526560129 >> 172.21.15.73:0/2526560129 conn(0x7fd3140b0830 msgr2=0x7fd3140b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 -- 172.21.15.73:0/2526560129 shutdown_connections 2024-04-05T22:21:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.114+0000 7fd319000640 1 -- 172.21.15.73:0/2526560129 wait complete. 2024-04-05T22:21:52.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.115+0000 7fd319000640 1 Processor -- start 2024-04-05T22:21:52.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.115+0000 7fd319000640 1 -- start start 2024-04-05T22:21:52.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314157410 0x7fd314154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31415af80 0x7fd31415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141545f0 con 0x7fd3140cd570 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd314149b70 con 0x7fd314157410 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.116+0000 7fd319000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd314154f50 con 0x7fd31415af80 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44720/0 (socket says 172.21.15.73:44720) 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 -- 172.21.15.73:0/4191877222 learned_addr learned my addr 172.21.15.73:0/4191877222 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd30aa00640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314157410 0x7fd314154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 -- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31415af80 msgr2=0x7fd31415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31415af80 0x7fd31415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 -- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314157410 msgr2=0x7fd314154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314157410 0x7fd314154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 -- 172.21.15.73:0/4191877222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040094d0 con 0x7fd3140cd570 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.117+0000 7fd313400640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3040090b0 tx=0x7fd304004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.118+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30403e040 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.118+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd304039e20 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.118+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304004270 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.118+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd314158ba0 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.118+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd314153f30 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.119+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31404e160 con 0x7fd3140cd570 2024-04-05T22:21:52.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.120+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd3040043d0 con 0x7fd3140cd570 2024-04-05T22:21:52.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.121+0000 7fd30be00640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 0x7fd2f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.121+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3040abd60 con 0x7fd3140cd570 2024-04-05T22:21:52.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.121+0000 7fd30aa00640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 0x7fd2f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.122+0000 7fd30aa00640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 0x7fd2f006d0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006fd0 tx=0x7fd2fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.125+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd304076f40 con 0x7fd3140cd570 2024-04-05T22:21:52.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.304+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3141e61f0 con 0x7fd3140cd570 2024-04-05T22:21:52.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.305+0000 7fd30be00640 1 -- 172.21.15.73:0/4191877222 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd304049020 con 0x7fd3140cd570 2024-04-05T22:21:52.306 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:52.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.307+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 msgr2=0x7fd2f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.307+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 0x7fd2f006d0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006fd0 tx=0x7fd2fc008040 comp rx=0 tx=0).stop 2024-04-05T22:21:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.307+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 msgr2=0x7fd3141531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.307+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3040090b0 tx=0x7fd304004730 comp rx=0 tx=0).stop 2024-04-05T22:21:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 shutdown_connections 2024-04-05T22:21:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd2f006ac00 0x7fd2f006d0c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31415af80 0x7fd31415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd314157410 0x7fd314154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 --2- 172.21.15.73:0/4191877222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3140cd570 0x7fd3141531c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 >> 172.21.15.73:0/4191877222 conn(0x7fd3140b0830 msgr2=0x7fd314155310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:52.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 shutdown_connections 2024-04-05T22:21:52.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.308+0000 7fd319000640 1 -- 172.21.15.73:0/4191877222 wait complete. 2024-04-05T22:21:52.424 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-04-05T22:21:52.424 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:21:52.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.633+0000 7f69efe00640 1 Processor -- start 2024-04-05T22:21:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.634+0000 7f69efe00640 1 -- start start 2024-04-05T22:21:52.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.635+0000 7f69efe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69f015ac50 0x7f69f006e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69efe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f006dfc0 con 0x7f69f006a3c0 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69efe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f015e9a0 con 0x7f69f006e990 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69efe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f015bf70 con 0x7f69f015ac50 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69ef400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f69f006a3c0 0x7f69f00cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33370/0 (socket says 172.21.15.73:33370) 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69ef400640 1 -- 172.21.15.73:0/658729546 learned_addr learned my addr 172.21.15.73:0/658729546 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:52.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69eea00640 1 --2- 172.21.15.73:0/658729546 >> v2:172.21.15.157:3300/0 conn(0x7f69f015ac50 0x7f69f006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429388224 0 0) 0x7f69f006dfc0 con 0x7f69f006a3c0 2024-04-05T22:21:52.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.636+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d4003520 con 0x7f69f006a3c0 2024-04-05T22:21:52.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520020134 0 0) 0x7f69d4003520 con 0x7f69f006a3c0 2024-04-05T22:21:52.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 >> v2:172.21.15.157:3300/0 conn(0x7f69f015ac50 msgr2=0x7f69f006e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:52.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 --2- 172.21.15.73:0/658729546 >> v2:172.21.15.157:3300/0 conn(0x7f69f015ac50 0x7f69f006e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 >> v1:172.21.15.157:6789/0 conn(0x7f69f006e990 legacy=0x7f69f0069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f015bc90 con 0x7f69f006a3c0 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f69e8001670 con 0x7f69f006a3c0 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 >> v1:172.21.15.73:6789/0 conn(0x7f69f006a3c0 legacy=0x7f69f00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69d40047c0 0x7f69d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69d4005180 0x7f69d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015bf70 con 0x7f69d4003970 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d400c020 con 0x7f69d4005180 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d400c1e0 con 0x7f69d40047c0 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69eea00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.637+0000 7f69ee000640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69d40047c0 0x7f69d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69eea00640 1 -- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69d40047c0 msgr2=0x7f69d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69eea00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69d40047c0 0x7f69d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69ef400640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69d4005180 0x7f69d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69eea00640 1 -- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69d4005180 msgr2=0x7f69d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69eea00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69d4005180 0x7f69d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69eea00640 1 -- 172.21.15.73:0/658729546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4003520 con 0x7f69d4003970 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69ef400640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69d4005180 0x7f69d400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.638+0000 7f69ef400640 1 -- 172.21.15.73:0/658729546 reap_dead start 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.639+0000 7f69eea00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fc0 tx=0x7f69d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa6a4442c3b4de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.639+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e002cc20 con 0x7f69d4003970 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.639+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e000f040 con 0x7f69d4003970 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.639+0000 7f69ed600640 1 -- 172.21.15.73:0/658729546 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0034440 con 0x7f69d4003970 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 -- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 msgr2=0x7f69d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fc0 tx=0x7f69d40057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 -- 172.21.15.73:0/658729546 shutdown_connections 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 --2- 172.21.15.73:0/658729546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 -- 172.21.15.73:0/658729546 >> 172.21.15.73:0/658729546 conn(0x7f69f00b1580 msgr2=0x7f69f00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 -- 172.21.15.73:0/658729546 shutdown_connections 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.640+0000 7f69efe00640 1 -- 172.21.15.73:0/658729546 wait complete. 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.642+0000 7f69efe00640 1 Processor -- start 2024-04-05T22:21:52.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.642+0000 7f69efe00640 1 -- start start 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f01528a0 0x7f69f0152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69f014fde0 0x7f69f0150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01507a0 con 0x7f69d4003970 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f013ed70 con 0x7f69f01528a0 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69efe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f013f050 con 0x7f69f014fde0 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44738/0 (socket says 172.21.15.73:44738) 2024-04-05T22:21:52.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.643+0000 7f69eea00640 1 -- 172.21.15.73:0/3498867859 learned_addr learned my addr 172.21.15.73:0/3498867859 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 -- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69f014fde0 msgr2=0x7f69f0150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69f014fde0 0x7f69f0150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 -- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f01528a0 msgr2=0x7f69f0152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f01528a0 0x7f69f0152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 -- 172.21.15.73:0/3498867859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f013f330 con 0x7f69d4003970 2024-04-05T22:21:52.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69eea00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e230 tx=0x7f69e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0034c50 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0034db0 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003bb0 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f013f610 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.644+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f013d240 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.646+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f004e160 con 0x7f69d4003970 2024-04-05T22:21:52.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.646+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f69e0040450 con 0x7f69d4003970 2024-04-05T22:21:52.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.648+0000 7f69e7400640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 0x7f69cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:52.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.648+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69e00aba60 con 0x7f69d4003970 2024-04-05T22:21:52.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.648+0000 7f69ee000640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 0x7f69cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:52.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.649+0000 7f69ee000640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 0x7f69cc06d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007fd0 tx=0x7f69d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:52.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.652+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f69e0076c30 con 0x7f69d4003970 2024-04-05T22:21:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.846+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69f0140d20 con 0x7f69d4003970 2024-04-05T22:21:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.848+0000 7f69e7400640 1 -- 172.21.15.73:0/3498867859 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f69e007a250 con 0x7f69d4003970 2024-04-05T22:21:52.849 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:52.849 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 msgr2=0x7f69cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 0x7f69cc06d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007fd0 tx=0x7f69d8009040 comp rx=0 tx=0).stop 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 msgr2=0x7f69f01394f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e230 tx=0x7f69e0009750 comp rx=0 tx=0).stop 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 shutdown_connections 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f69cc06ac00 0x7f69cc06d0c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.851+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69f014fde0 0x7f69f0150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.852+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f01528a0 0x7f69f0152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.852+0000 7f69efe00640 1 --2- 172.21.15.73:0/3498867859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69d4003970 0x7f69f01394f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.852+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 >> 172.21.15.73:0/3498867859 conn(0x7f69f00b1580 msgr2=0x7f69f00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.852+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 shutdown_connections 2024-04-05T22:21:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:52.852+0000 7f69efe00640 1 -- 172.21.15.73:0/3498867859 wait complete. 2024-04-05T22:21:52.871 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:52.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:53.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.074+0000 7f8ca7e00640 1 Processor -- start 2024-04-05T22:21:53.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.074+0000 7f8ca7e00640 1 -- start start 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8ca8160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca806d800 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80b3090 con 0x7f8ca8157030 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80b3370 con 0x7f8ca8157410 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8ca80cb0b0 0x7f8ca815d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33374/0 (socket says 172.21.15.73:33374) 2024-04-05T22:21:53.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca7400640 1 -- 172.21.15.73:0/3828781098 learned_addr learned my addr 172.21.15.73:0/3828781098 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:53.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8ca8160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.076+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516018789 0 0) 0x7f8ca806d800 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c88003520 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376693555 0 0) 0x7f8c88003520 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 msgr2=0x7f8ca8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8ca8160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 >> v1:172.21.15.157:6789/0 conn(0x7f8ca8157030 legacy=0x7f8ca80ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80b3650 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8ca00019f0 con 0x7f8ca80cb0b0 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 >> v1:172.21.15.73:6789/0 conn(0x7f8ca80cb0b0 legacy=0x7f8ca815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8c880039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c88003ee0 0x7f8c880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88004d00 0x7f8c88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80b3370 con 0x7f8c88003ee0 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8800b9e0 con 0x7f8c88004d00 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8800bba0 con 0x7f8ca8157410 2024-04-05T22:21:53.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.077+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c88003ee0 0x7f8c880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8ca6a00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8c880039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8ca7400640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88004d00 0x7f8c88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8c9fe00640 1 -- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 msgr2=0x7f8c880039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ca8157410 0x7f8c880039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8c9fe00640 1 -- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88004d00 msgr2=0x7f8c88005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88004d00 0x7f8c88005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8c9fe00640 1 -- 172.21.15.73:0/3828781098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88003520 con 0x7f8c88003ee0 2024-04-05T22:21:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.078+0000 7f8ca7400640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88004d00 0x7f8c88005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:53.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.079+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c88003ee0 0x7f8c880047c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ac0 tx=0x7f8c880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccae1e2243fa5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.079+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9002c600 con 0x7f8c88003ee0 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.079+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9002c760 con 0x7f8c88003ee0 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.079+0000 7f8ca6000640 1 -- 172.21.15.73:0/3828781098 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9002caf0 con 0x7f8c88003ee0 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 -- 172.21.15.73:0/3828781098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c88003ee0 msgr2=0x7f8c880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/3828781098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c88003ee0 0x7f8c880047c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ac0 tx=0x7f8c880057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7400640 1 -- 172.21.15.73:0/3828781098 reap_dead start 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 -- 172.21.15.73:0/3828781098 shutdown_connections 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 -- 172.21.15.73:0/3828781098 >> 172.21.15.73:0/3828781098 conn(0x7f8ca8069cb0 msgr2=0x7f8ca806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 -- 172.21.15.73:0/3828781098 shutdown_connections 2024-04-05T22:21:53.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.080+0000 7f8ca7e00640 1 -- 172.21.15.73:0/3828781098 wait complete. 2024-04-05T22:21:53.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.082+0000 7f8ca7e00640 1 Processor -- start 2024-04-05T22:21:53.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.082+0000 7f8ca7e00640 1 -- start start 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c88003ee0 0x7f8ca81531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca8154e50 0x7f8ca8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca806bf40 con 0x7f8ca80af1a0 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80afb60 con 0x7f8ca8154e50 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8155840 con 0x7f8c88003ee0 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44762/0 (socket says 172.21.15.73:44762) 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 -- 172.21.15.73:0/4035778557 learned_addr learned my addr 172.21.15.73:0/4035778557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8ca6a00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c88003ee0 0x7f8ca81531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 -- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c88003ee0 msgr2=0x7f8ca81531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c88003ee0 0x7f8ca81531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 -- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca8154e50 msgr2=0x7f8ca8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca8154e50 0x7f8ca8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.083+0000 7f8c9fe00640 1 -- 172.21.15.73:0/4035778557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8155a60 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8c9fe00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c90037040 tx=0x7f8c90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90003810 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c90003970 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca8155d40 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90003d00 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.084+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca81e67e0 con 0x7f8ca80af1a0 2024-04-05T22:21:53.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.085+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca804e160 con 0x7f8ca80af1a0 2024-04-05T22:21:53.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.086+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c90040450 con 0x7f8ca80af1a0 2024-04-05T22:21:53.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.088+0000 7f8c9ea00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 0x7f8c8406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.088+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c900ab2e0 con 0x7f8ca80af1a0 2024-04-05T22:21:53.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.088+0000 7f8ca6a00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 0x7f8c8406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.089+0000 7f8ca6a00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 0x7f8c8406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cb080 tx=0x7f8c98007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.092+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8c900764b0 con 0x7f8ca80af1a0 2024-04-05T22:21:53.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.286+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ca80cd860 con 0x7f8ca80af1a0 2024-04-05T22:21:53.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.287+0000 7f8c9ea00640 1 -- 172.21.15.73:0/4035778557 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c90003e60 con 0x7f8ca80af1a0 2024-04-05T22:21:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 msgr2=0x7f8c8406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 0x7f8c8406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cb080 tx=0x7f8c98007b70 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 msgr2=0x7f8ca80af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c90037040 tx=0x7f8c90004730 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 shutdown_connections 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c8406ac00 0x7f8c8406d0c0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cb080 tx=0x7f8c98007b70 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca8154e50 0x7f8ca8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca80af1a0 0x7f8ca80af620 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 --2- 172.21.15.73:0/4035778557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c88003ee0 0x7f8ca81531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 >> 172.21.15.73:0/4035778557 conn(0x7f8ca8069cb0 msgr2=0x7f8ca80cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 shutdown_connections 2024-04-05T22:21:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.290+0000 7f8ca7e00640 1 -- 172.21.15.73:0/4035778557 wait complete. 2024-04-05T22:21:53.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:53.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.523+0000 7fae75000640 1 Processor -- start 2024-04-05T22:21:53.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.523+0000 7fae75000640 1 -- start start 2024-04-05T22:21:53.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae75000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 0x7fae700712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae75000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae7006c9a0 0x7fae7006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae75000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7006bec0 con 0x7fae7006ba70 2024-04-05T22:21:53.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae75000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7015afc0 con 0x7fae7006c9a0 2024-04-05T22:21:53.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae75000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7006f5b0 con 0x7fae7015abe0 2024-04-05T22:21:53.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae6de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 0x7fae700712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.525+0000 7fae6d400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae7006c9a0 0x7fae7006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 0x7fae700712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:45642/0 (socket says 172.21.15.73:45642) 2024-04-05T22:21:53.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6de00640 1 -- 172.21.15.73:0/967476226 learned_addr learned my addr 172.21.15.73:0/967476226 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6d400640 1 -- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 msgr2=0x7fae700712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6d400640 1 --2- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 0x7fae700712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6d400640 1 -- 172.21.15.73:0/967476226 >> v1:172.21.15.157:6789/0 conn(0x7fae7006ba70 legacy=0x7fae700cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.526+0000 7fae6d400640 1 -- 172.21.15.73:0/967476226 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7006f890 con 0x7fae7006c9a0 2024-04-05T22:21:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.528+0000 7fae6d400640 1 --2- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006c9a0 0x7fae7006b530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae580097a0 tx=0x7fae5802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97095857711f89e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.528+0000 7fae6ca00640 1 -- 172.21.15.73:0/967476226 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5802f050 con 0x7fae7006c9a0 2024-04-05T22:21:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.528+0000 7fae6ca00640 1 -- 172.21.15.73:0/967476226 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58031700 con 0x7fae7006c9a0 2024-04-05T22:21:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.528+0000 7fae6ca00640 1 -- 172.21.15.73:0/967476226 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58031a90 con 0x7fae7006c9a0 2024-04-05T22:21:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 -- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006c9a0 msgr2=0x7fae7006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 --2- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006c9a0 0x7fae7006b530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae580097a0 tx=0x7fae5802b7a0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 -- 172.21.15.73:0/967476226 shutdown_connections 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 --2- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006c9a0 0x7fae7006b530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 --2- 172.21.15.73:0/967476226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7015abe0 0x7fae700712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 -- 172.21.15.73:0/967476226 >> 172.21.15.73:0/967476226 conn(0x7fae70069cb0 msgr2=0x7fae7006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 -- 172.21.15.73:0/967476226 shutdown_connections 2024-04-05T22:21:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.529+0000 7fae75000640 1 -- 172.21.15.73:0/967476226 wait complete. 2024-04-05T22:21:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.531+0000 7fae75000640 1 Processor -- start 2024-04-05T22:21:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.531+0000 7fae75000640 1 -- start start 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006ba70 0x7fae7015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7006c9a0 0x7fae701574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7015d170 con 0x7fae7015abe0 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7015c1e0 con 0x7fae7006ba70 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae75000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7015c7a0 con 0x7fae7006c9a0 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae6e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae6e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44770/0 (socket says 172.21.15.73:44770) 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae6e800640 1 -- 172.21.15.73:0/1374165333 learned_addr learned my addr 172.21.15.73:0/1374165333 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.532+0000 7fae6de00640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006ba70 0x7fae7015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6d400640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7006c9a0 0x7fae701574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 -- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7006c9a0 msgr2=0x7fae701574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7006c9a0 0x7fae701574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 -- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006ba70 msgr2=0x7fae7015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006ba70 0x7fae7015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 -- 172.21.15.73:0/1374165333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009400 con 0x7fae7015abe0 2024-04-05T22:21:53.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae6e800640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c3c0 tx=0x7fae6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680052d0 con 0x7fae7015abe0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68005430 con 0x7fae7015abe0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68003810 con 0x7fae7015abe0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.533+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70157a30 con 0x7fae7015abe0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.534+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae701f15c0 con 0x7fae7015abe0 2024-04-05T22:21:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.535+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70068160 con 0x7fae7015abe0 2024-04-05T22:21:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.536+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae68003970 con 0x7fae7015abe0 2024-04-05T22:21:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.536+0000 7fae66000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 0x7fae4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.536+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae680886e0 con 0x7fae7015abe0 2024-04-05T22:21:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.537+0000 7fae6de00640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 0x7fae4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.537+0000 7fae6de00640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 0x7fae4c06d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae70156370 tx=0x7fae6000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.541+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae680538b0 con 0x7fae7015abe0 2024-04-05T22:21:53.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.737+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae7006bec0 con 0x7fae7015abe0 2024-04-05T22:21:53.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.738+0000 7fae66000640 1 -- 172.21.15.73:0/1374165333 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae7006bec0 con 0x7fae7015abe0 2024-04-05T22:21:53.739 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 msgr2=0x7fae4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 0x7fae4c06d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae70156370 tx=0x7fae6000b040 comp rx=0 tx=0).stop 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 msgr2=0x7fae701f0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c3c0 tx=0x7fae6800c880 comp rx=0 tx=0).stop 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 shutdown_connections 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae4c06ac00 0x7fae4c06d0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7015abe0 0x7fae701f0770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7006c9a0 0x7fae701574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.741+0000 7fae75000640 1 --2- 172.21.15.73:0/1374165333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7006ba70 0x7fae7015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.742+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 >> 172.21.15.73:0/1374165333 conn(0x7fae70069cb0 msgr2=0x7fae700ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.742+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 shutdown_connections 2024-04-05T22:21:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.742+0000 7fae75000640 1 -- 172.21.15.73:0/1374165333 wait complete. 2024-04-05T22:21:53.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.971+0000 7fb905800640 1 Processor -- start 2024-04-05T22:21:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.971+0000 7fb905800640 1 -- start start 2024-04-05T22:21:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.972+0000 7fb905800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb90006c630 0x7fb9000ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.972+0000 7fb905800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb900070c50 0x7fb9000710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb905800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb900146ac0 con 0x7fb90015dd90 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb905800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb900070720 con 0x7fb900070c50 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb905800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9000acf60 con 0x7fb90006c630 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb900070c50 0x7fb9000710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb900070c50 0x7fb9000710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44774/0 (socket says 172.21.15.73:44774) 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 -- 172.21.15.73:0/934626674 learned_addr learned my addr 172.21.15.73:0/934626674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8fea00640 1 --2- 172.21.15.73:0/934626674 >> v2:172.21.15.157:3300/0 conn(0x7fb90006c630 0x7fb9000ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 -- 172.21.15.73:0/934626674 >> v2:172.21.15.157:3300/0 conn(0x7fb90006c630 msgr2=0x7fb9000ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 --2- 172.21.15.73:0/934626674 >> v2:172.21.15.157:3300/0 conn(0x7fb90006c630 0x7fb9000ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 -- 172.21.15.73:0/934626674 >> v1:172.21.15.73:6789/0 conn(0x7fb90015dd90 legacy=0x7fb90006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.973+0000 7fb8ff400640 1 -- 172.21.15.73:0/934626674 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000ad240 con 0x7fb900070c50 2024-04-05T22:21:53.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.975+0000 7fb8ff400640 1 --2- 172.21.15.73:0/934626674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb900070c50 0x7fb9000710e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb90006b1d0 tx=0x7fb8f802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9630dbb46cccac server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.975+0000 7fb8fe000640 1 -- 172.21.15.73:0/934626674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f802cb10 con 0x7fb900070c50 2024-04-05T22:21:53.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.975+0000 7fb8fe000640 1 -- 172.21.15.73:0/934626674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f802cc70 con 0x7fb900070c50 2024-04-05T22:21:53.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.975+0000 7fb8fe000640 1 -- 172.21.15.73:0/934626674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80343f0 con 0x7fb900070c50 2024-04-05T22:21:53.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 -- 172.21.15.73:0/934626674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb900070c50 msgr2=0x7fb9000710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 --2- 172.21.15.73:0/934626674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb900070c50 0x7fb9000710e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb90006b1d0 tx=0x7fb8f802bf70 comp rx=0 tx=0).stop 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 -- 172.21.15.73:0/934626674 shutdown_connections 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 --2- 172.21.15.73:0/934626674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb900070c50 0x7fb9000710e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 --2- 172.21.15.73:0/934626674 >> v2:172.21.15.157:3300/0 conn(0x7fb90006c630 0x7fb9000ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 -- 172.21.15.73:0/934626674 >> 172.21.15.73:0/934626674 conn(0x7fb9000b0830 msgr2=0x7fb9000b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 -- 172.21.15.73:0/934626674 shutdown_connections 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.976+0000 7fb905800640 1 -- 172.21.15.73:0/934626674 wait complete. 2024-04-05T22:21:53.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.978+0000 7fb905800640 1 Processor -- start 2024-04-05T22:21:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.978+0000 7fb905800640 1 -- start start 2024-04-05T22:21:53.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb900070c50 0x7fb9000cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90015dd90 0x7fb900145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900143220 con 0x7fb90006c630 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900143790 con 0x7fb90015dd90 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb905800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001458f0 con 0x7fb900070c50 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44788/0 (socket says 172.21.15.73:44788) 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 -- 172.21.15.73:0/3564742896 learned_addr learned my addr 172.21.15.73:0/3564742896 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 -- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb900070c50 msgr2=0x7fb9000cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb900070c50 0x7fb9000cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 -- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90015dd90 msgr2=0x7fb900145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90015dd90 0x7fb900145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.979+0000 7fb8fea00640 1 -- 172.21.15.73:0/3564742896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f8009b30 con 0x7fb90006c630 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb8fea00640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d5f0 tx=0x7fb8f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00125f0 con 0x7fb90006c630 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f0012750 con 0x7fb90006c630 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb900145020 con 0x7fb90006c630 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0012b00 con 0x7fb90006c630 2024-04-05T22:21:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.980+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000cbdb0 con 0x7fb90006c630 2024-04-05T22:21:53.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.981+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900146320 con 0x7fb90006c630 2024-04-05T22:21:53.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.982+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb8f0012c60 con 0x7fb90006c630 2024-04-05T22:21:53.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.983+0000 7fb8f7400640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 0x7fb8dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:53.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.984+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8f0089810 con 0x7fb90006c630 2024-04-05T22:21:53.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.984+0000 7fb8f6000640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 0x7fb8dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:53.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.984+0000 7fb8f6000640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 0x7fb8dc06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8000f60 tx=0x7fb8e80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:53.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:53.988+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb8f00549f0 con 0x7fb90006c630 2024-04-05T22:21:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.182+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb900068160 con 0x7fb90006c630 2024-04-05T22:21:54.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.183+0000 7fb8f7400640 1 -- 172.21.15.73:0/3564742896 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8f0058010 con 0x7fb90006c630 2024-04-05T22:21:54.183 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.185+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 msgr2=0x7fb8dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.185+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 0x7fb8dc06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8000f60 tx=0x7fb8e80079b0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 msgr2=0x7fb90015c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d5f0 tx=0x7fb8f000dab0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 shutdown_connections 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8dc06ac00 0x7fb8dc06d0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90015dd90 0x7fb900145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb900070c50 0x7fb9000cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 --2- 172.21.15.73:0/3564742896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb90006c630 0x7fb90015c790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 >> 172.21.15.73:0/3564742896 conn(0x7fb9000b0830 msgr2=0x7fb900160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 shutdown_connections 2024-04-05T22:21:54.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.186+0000 7fb905800640 1 -- 172.21.15.73:0/3564742896 wait complete. 2024-04-05T22:21:54.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:21:54.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.409+0000 7f13fca00640 1 Processor -- start 2024-04-05T22:21:54.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.410+0000 7f13fca00640 1 -- start start 2024-04-05T22:21:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13fca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13f81541a0 0x7f13f80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13fca00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13f806d710 0x7f13f80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13fca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f815aa60 con 0x7f13f815aef0 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13fca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f80b3b60 con 0x7f13f81541a0 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13fca00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f80701c0 con 0x7f13f806d710 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13f5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13f81541a0 0x7f13f80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13f6800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f13f815aef0 0x7f13f815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33404/0 (socket says 172.21.15.73:33404) 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13f5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13f81541a0 0x7f13f80b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44800/0 (socket says 172.21.15.73:44800) 2024-04-05T22:21:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.411+0000 7f13f6800640 1 -- 172.21.15.73:0/272720500 learned_addr learned my addr 172.21.15.73:0/272720500 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.412+0000 7f13efe00640 1 --2- 172.21.15.73:0/272720500 >> v2:172.21.15.157:3300/0 conn(0x7f13f806d710 0x7f13f80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.412+0000 7f13f5e00640 1 -- 172.21.15.73:0/272720500 >> v2:172.21.15.157:3300/0 conn(0x7f13f806d710 msgr2=0x7f13f80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.412+0000 7f13f5e00640 1 --2- 172.21.15.73:0/272720500 >> v2:172.21.15.157:3300/0 conn(0x7f13f806d710 0x7f13f80cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.412+0000 7f13f5e00640 1 -- 172.21.15.73:0/272720500 >> v1:172.21.15.73:6789/0 conn(0x7f13f815aef0 legacy=0x7f13f815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.412+0000 7f13f5e00640 1 -- 172.21.15.73:0/272720500 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80b3880 con 0x7f13f81541a0 2024-04-05T22:21:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.413+0000 7f13f5e00640 1 --2- 172.21.15.73:0/272720500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f80b3050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13f815e980 tx=0x7f13e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9d853fb4bc1bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.413+0000 7f13f5400640 1 -- 172.21.15.73:0/272720500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8031830 con 0x7f13f81541a0 2024-04-05T22:21:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.413+0000 7f13f5400640 1 -- 172.21.15.73:0/272720500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8031990 con 0x7f13f81541a0 2024-04-05T22:21:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.413+0000 7f13f5400640 1 -- 172.21.15.73:0/272720500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8031d20 con 0x7f13f81541a0 2024-04-05T22:21:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 -- 172.21.15.73:0/272720500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 msgr2=0x7f13f80b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 --2- 172.21.15.73:0/272720500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f80b3050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13f815e980 tx=0x7f13e802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 -- 172.21.15.73:0/272720500 shutdown_connections 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 --2- 172.21.15.73:0/272720500 >> v2:172.21.15.157:3300/0 conn(0x7f13f806d710 0x7f13f80cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 --2- 172.21.15.73:0/272720500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f80b3050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 -- 172.21.15.73:0/272720500 >> 172.21.15.73:0/272720500 conn(0x7f13f80ce8e0 msgr2=0x7f13f80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 -- 172.21.15.73:0/272720500 shutdown_connections 2024-04-05T22:21:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.414+0000 7f13fca00640 1 -- 172.21.15.73:0/272720500 wait complete. 2024-04-05T22:21:54.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.416+0000 7f13fca00640 1 Processor -- start 2024-04-05T22:21:54.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.416+0000 7f13fca00640 1 -- start start 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 0x7f13f815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13f815aef0 0x7f13f8149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8158060 con 0x7f13f81541a0 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8149990 con 0x7f13f815aef0 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13fca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f81573e0 con 0x7f13f806d710 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44802/0 (socket says 172.21.15.73:44802) 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.417+0000 7f13efe00640 1 -- 172.21.15.73:0/3732452481 learned_addr learned my addr 172.21.15.73:0/3732452481 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13f6800640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13f815aef0 0x7f13f8149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13f5e00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 0x7f13f815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 -- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 msgr2=0x7f13f815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 0x7f13f815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 -- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13f815aef0 msgr2=0x7f13f8149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13f815aef0 0x7f13f8149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 -- 172.21.15.73:0/3732452481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8009400 con 0x7f13f81541a0 2024-04-05T22:21:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13f5e00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 0x7f13f815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13efe00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d800 tx=0x7f13e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000b6c0 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e000b820 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000bbb0 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f81576c0 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.418+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f8157c80 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.420+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f8066af0 con 0x7f13f81541a0 2024-04-05T22:21:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.420+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f13e000bd10 con 0x7f13f81541a0 2024-04-05T22:21:54.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.421+0000 7f13ee000640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 0x7f13d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:54.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.421+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13e0089910 con 0x7f13f81541a0 2024-04-05T22:21:54.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.422+0000 7f13f5e00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 0x7f13d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:54.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.422+0000 7f13f5e00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 0x7f13d406d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13e8036000 tx=0x7f13e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:54.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.426+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f13e0054ae0 con 0x7f13f81541a0 2024-04-05T22:21:54.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:54.694+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f13f8068160 con 0x7f13f81541a0 2024-04-05T22:21:55.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.638+0000 7f13ee000640 1 -- 172.21.15.73:0/3732452481 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13e0058100 con 0x7f13f81541a0 2024-04-05T22:21:55.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 msgr2=0x7f13d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 0x7f13d406d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13e8036000 tx=0x7f13e8008030 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 msgr2=0x7f13f8148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d800 tx=0x7f13e000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 shutdown_connections 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f13d406ac00 0x7f13d406d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13f815aef0 0x7f13f8149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13f81541a0 0x7f13f8148f10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 --2- 172.21.15.73:0/3732452481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13f806d710 0x7f13f815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 >> 172.21.15.73:0/3732452481 conn(0x7f13f80ce8e0 msgr2=0x7f13f815b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 shutdown_connections 2024-04-05T22:21:55.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.642+0000 7f13fca00640 1 -- 172.21.15.73:0/3732452481 wait complete. 2024-04-05T22:21:55.664 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:21:55.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.875+0000 7fe915e00640 1 Processor -- start 2024-04-05T22:21:55.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.875+0000 7fe915e00640 1 -- start start 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.876+0000 7fe915e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe91006cd50 0x7fe910070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.876+0000 7fe915e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 0x7fe910159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.876+0000 7fe915e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9100b27f0 con 0x7fe9101599f0 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.876+0000 7fe915e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9100b3890 con 0x7fe91006cd50 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.876+0000 7fe915e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9100b2ad0 con 0x7fe9100add00 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe91006cd50 0x7fe910070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe91006cd50 0x7fe910070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44054/0 (socket says 172.21.15.73:44054) 2024-04-05T22:21:55.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 -- 172.21.15.73:0/2040432399 learned_addr learned my addr 172.21.15.73:0/2040432399 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe90f400640 1 --2- 172.21.15.73:0/2040432399 >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 0x7fe910159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 -- 172.21.15.73:0/2040432399 >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 msgr2=0x7fe910159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 --2- 172.21.15.73:0/2040432399 >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 0x7fe910159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 -- 172.21.15.73:0/2040432399 >> v1:172.21.15.73:6789/0 conn(0x7fe9101599f0 legacy=0x7fe91006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe906000640 1 -- 172.21.15.73:0/2040432399 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9100b2db0 con 0x7fe91006cd50 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe90f400640 1 --2- 172.21.15.73:0/2040432399 >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 0x7fe910159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.877+0000 7fe90e000640 1 -- 172.21.15.73:0/2040432399 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022678596 0 0) 0x7fe9100b27f0 con 0x7fe9101599f0 2024-04-05T22:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.878+0000 7fe906000640 1 --2- 172.21.15.73:0/2040432399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe91006cd50 0x7fe910070a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe91006d750 tx=0x7fe8f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1538f16e465a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:55.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe90e000640 1 -- 172.21.15.73:0/2040432399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8030860 con 0x7fe91006cd50 2024-04-05T22:21:55.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe90e000640 1 -- 172.21.15.73:0/2040432399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f80309c0 con 0x7fe91006cd50 2024-04-05T22:21:55.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe90e000640 1 -- 172.21.15.73:0/2040432399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8030d50 con 0x7fe91006cd50 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 -- 172.21.15.73:0/2040432399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe91006cd50 msgr2=0x7fe910070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 --2- 172.21.15.73:0/2040432399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe91006cd50 0x7fe910070a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe91006d750 tx=0x7fe8f802b9a0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 -- 172.21.15.73:0/2040432399 shutdown_connections 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 --2- 172.21.15.73:0/2040432399 >> v2:172.21.15.157:3300/0 conn(0x7fe9100add00 0x7fe910159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 --2- 172.21.15.73:0/2040432399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe91006cd50 0x7fe910070a40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 -- 172.21.15.73:0/2040432399 >> 172.21.15.73:0/2040432399 conn(0x7fe910069cb0 msgr2=0x7fe91006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 -- 172.21.15.73:0/2040432399 shutdown_connections 2024-04-05T22:21:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.879+0000 7fe915e00640 1 -- 172.21.15.73:0/2040432399 wait complete. 2024-04-05T22:21:55.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.881+0000 7fe915e00640 1 Processor -- start 2024-04-05T22:21:55.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.881+0000 7fe915e00640 1 -- start start 2024-04-05T22:21:55.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe91006cd50 0x7fe910149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 0x7fe91015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910149cb0 con 0x7fe9101599f0 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91015cd20 con 0x7fe9100add00 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.882+0000 7fe915e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91015d000 con 0x7fe91006cd50 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44062/0 (socket says 172.21.15.73:44062) 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 -- 172.21.15.73:0/4191718550 learned_addr learned my addr 172.21.15.73:0/4191718550 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90ea00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe91006cd50 0x7fe910149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe906000640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 0x7fe91015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 -- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe91006cd50 msgr2=0x7fe910149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe91006cd50 0x7fe910149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 -- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 msgr2=0x7fe91015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 0x7fe91015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:55.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 -- 172.21.15.73:0/4191718550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80094d0 con 0x7fe9101599f0 2024-04-05T22:21:55.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe906000640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 0x7fe91015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:55.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.883+0000 7fe90f400640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe90800cef0 tx=0x7fe90800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:55.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.884+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800d840 con 0x7fe9101599f0 2024-04-05T22:21:55.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.884+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90800d9c0 con 0x7fe9101599f0 2024-04-05T22:21:55.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.884+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800dd70 con 0x7fe9101599f0 2024-04-05T22:21:55.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.884+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91015d2e0 con 0x7fe9101599f0 2024-04-05T22:21:55.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.884+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91015d8a0 con 0x7fe9101599f0 2024-04-05T22:21:55.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.885+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91015da90 con 0x7fe9101599f0 2024-04-05T22:21:55.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.886+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe90801e3b0 con 0x7fe9101599f0 2024-04-05T22:21:55.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.887+0000 7fe907e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 0x7fe8ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:55.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.887+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe908089ed0 con 0x7fe9101599f0 2024-04-05T22:21:55.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.887+0000 7fe90ea00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 0x7fe8ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:55.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.888+0000 7fe90ea00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 0x7fe8ec06d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe900000f60 tx=0x7fe900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:55.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:55.891+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe9080550d0 con 0x7fe9101599f0 2024-04-05T22:21:56.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.086+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe910068160 con 0x7fe9101599f0 2024-04-05T22:21:56.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.092+0000 7fe907e00640 1 -- 172.21.15.73:0/4191718550 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe90801a020 con 0x7fe9101599f0 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 msgr2=0x7fe8ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 0x7fe8ec06d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe900000f60 tx=0x7fe900009040 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 msgr2=0x7fe910152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe90800cef0 tx=0x7fe90800a710 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 shutdown_connections 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8ec06ac00 0x7fe8ec06d0c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9101599f0 0x7fe910152f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9100add00 0x7fe91015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 --2- 172.21.15.73:0/4191718550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe91006cd50 0x7fe910149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 >> 172.21.15.73:0/4191718550 conn(0x7fe910069cb0 msgr2=0x7fe9100b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:56.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.095+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 shutdown_connections 2024-04-05T22:21:56.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.096+0000 7fe915e00640 1 -- 172.21.15.73:0/4191718550 wait complete. 2024-04-05T22:21:56.113 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:21:56.113 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:21:56.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:21:56.344 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:21:56.344 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:21:56.344 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:56.401 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:21:56.401 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:56.452 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:21:56.452 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:56.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:56.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.709+0000 7f3d04400640 1 Processor -- start 2024-04-05T22:21:56.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.710+0000 7f3d04400640 1 -- start start 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d04400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3cfc15cf40 0x7f3cfc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d04400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc06bd40 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d04400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc15d320 con 0x7f3cfc0d0850 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d04400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b2340 con 0x7f3cfc15cf40 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d02400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3cfc0d0cb0 0x7f3cfc0b1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34376/0 (socket says 172.21.15.73:34376) 2024-04-05T22:21:56.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.711+0000 7f3d02400640 1 -- 172.21.15.73:0/3911334762 learned_addr learned my addr 172.21.15.73:0/3911334762 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3d01a00640 1 --2- 172.21.15.73:0/3911334762 >> v2:172.21.15.157:3300/0 conn(0x7f3cfc15cf40 0x7f3cfc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080608162 0 0) 0x7f3cfc06bd40 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce0003520 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1016595484 0 0) 0x7f3ce0003520 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 >> v2:172.21.15.157:3300/0 conn(0x7f3cfc15cf40 msgr2=0x7f3cfc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 --2- 172.21.15.73:0/3911334762 >> v2:172.21.15.157:3300/0 conn(0x7f3cfc15cf40 0x7f3cfc0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 >> v1:172.21.15.157:6789/0 conn(0x7f3cfc0d0850 legacy=0x7f3cfc0b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc06b130 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3cf4001670 con 0x7f3cfc0d0cb0 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 >> v1:172.21.15.73:6789/0 conn(0x7f3cfc0d0cb0 legacy=0x7f3cfc0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0003970 0x7f3ce0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b2340 con 0x7f3ce0005180 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000c020 con 0x7f3ce0003970 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.712+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000c1e0 con 0x7f3ce00047c0 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d01a00640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0003970 0x7f3ce0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d01000640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 -- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce00047c0 msgr2=0x7f3ce0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 -- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0003970 msgr2=0x7f3ce0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0003970 0x7f3ce0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.713+0000 7f3d02400640 1 -- 172.21.15.73:0/3911334762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0003520 con 0x7f3ce0005180 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.714+0000 7f3d02400640 1 -- 172.21.15.73:0/3911334762 reap_dead start 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.714+0000 7f3d02400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c240 tx=0x7f3ce00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0a167b626e8359 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.714+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4005ad0 con 0x7f3ce0005180 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.714+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4005c70 con 0x7f3ce0005180 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.714+0000 7f3cfbe00640 1 -- 172.21.15.73:0/3911334762 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402c8b0 con 0x7f3ce0005180 2024-04-05T22:21:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 -- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 msgr2=0x7f3ce000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c240 tx=0x7f3ce00057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 -- 172.21.15.73:0/3911334762 shutdown_connections 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 --2- 172.21.15.73:0/3911334762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 -- 172.21.15.73:0/3911334762 >> 172.21.15.73:0/3911334762 conn(0x7f3cfc099a60 msgr2=0x7f3cfc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 -- 172.21.15.73:0/3911334762 shutdown_connections 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.715+0000 7f3d04400640 1 -- 172.21.15.73:0/3911334762 wait complete. 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.717+0000 7f3d04400640 1 Processor -- start 2024-04-05T22:21:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.717+0000 7f3d04400640 1 -- start start 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0005180 0x7f3cfc1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cfc0cfc70 0x7f3cfc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc1476e0 con 0x7f3cfc1560a0 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0d0610 con 0x7f3ce0005180 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d04400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc156a60 con 0x7f3cfc0cfc70 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d02400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d02400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44084/0 (socket says 172.21.15.73:44084) 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.718+0000 7f3d02400640 1 -- 172.21.15.73:0/1689699496 learned_addr learned my addr 172.21.15.73:0/1689699496 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d01000640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cfc0cfc70 0x7f3cfc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 -- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cfc0cfc70 msgr2=0x7f3cfc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d01a00640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0005180 0x7f3cfc1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cfc0cfc70 0x7f3cfc0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 -- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0005180 msgr2=0x7f3cfc1471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0005180 0x7f3cfc1471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 -- 172.21.15.73:0/1689699496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc156d20 con 0x7f3cfc1560a0 2024-04-05T22:21:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d02400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3cf402c520 tx=0x7f3cf4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40049e0 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4004b40 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402de80 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc157000 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.719+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc1575c0 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.721+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc068160 con 0x7f3cfc1560a0 2024-04-05T22:21:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.721+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3cf4032070 con 0x7f3cfc1560a0 2024-04-05T22:21:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.722+0000 7f3cf9600640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 0x7f3cdc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.722+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cf403c070 con 0x7f3cfc1560a0 2024-04-05T22:21:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.723+0000 7f3d01a00640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 0x7f3cdc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.723+0000 7f3d01a00640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 0x7f3cdc06d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0098d0 tx=0x7f3cec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.727+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3cf40757f0 con 0x7f3cfc1560a0 2024-04-05T22:21:56.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.919+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cfc15b630 con 0x7f3cfc1560a0 2024-04-05T22:21:56.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.921+0000 7f3cf9600640 1 -- 172.21.15.73:0/1689699496 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f3cf4045090 con 0x7f3cfc1560a0 2024-04-05T22:21:56.922 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:56.922 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 msgr2=0x7f3cdc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 0x7f3cdc06d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0098d0 tx=0x7f3cec008040 comp rx=0 tx=0).stop 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 msgr2=0x7f3cfc156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3cf402c520 tx=0x7f3cf4001a00 comp rx=0 tx=0).stop 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 shutdown_connections 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3cdc06ac00 0x7f3cdc06d0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cfc1560a0 0x7f3cfc156520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cfc0cfc70 0x7f3cfc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.924+0000 7f3d04400640 1 --2- 172.21.15.73:0/1689699496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce0005180 0x7f3cfc1471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:56.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.925+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 >> 172.21.15.73:0/1689699496 conn(0x7f3cfc099a60 msgr2=0x7f3cfc15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:56.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.925+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 shutdown_connections 2024-04-05T22:21:56.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:56.925+0000 7f3d04400640 1 -- 172.21.15.73:0/1689699496 wait complete. 2024-04-05T22:21:56.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:56.945 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:57.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.156+0000 7fe179a00640 1 Processor -- start 2024-04-05T22:21:57.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.156+0000 7fe179a00640 1 -- start start 2024-04-05T22:21:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe179a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe179a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 0x7fe17406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe179a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17406bc50 con 0x7fe17415e9a0 2024-04-05T22:21:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe179a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17415f530 con 0x7fe17415e580 2024-04-05T22:21:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe179a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17415ef70 con 0x7fe17415a850 2024-04-05T22:21:57.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe173400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 0x7fe17406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.158+0000 7fe173e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:34998/0 (socket says 172.21.15.73:34998) 2024-04-05T22:21:57.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 -- 172.21.15.73:0/3872796897 learned_addr learned my addr 172.21.15.73:0/3872796897 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:57.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 0x7fe17406ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:51444/0 (socket says 172.21.15.73:51444) 2024-04-05T22:21:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 -- 172.21.15.73:0/3872796897 >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 msgr2=0x7fe17406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 --2- 172.21.15.73:0/3872796897 >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 0x7fe17406ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 -- 172.21.15.73:0/3872796897 >> v1:172.21.15.157:6789/0 conn(0x7fe17415e9a0 legacy=0x7fe1740b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.159+0000 7fe173e00640 1 -- 172.21.15.73:0/3872796897 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17415f250 con 0x7fe17415a850 2024-04-05T22:21:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.160+0000 7fe173e00640 1 --2- 172.21.15.73:0/3872796897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe164009870 tx=0x7fe16402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9f39043436eb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.161+0000 7fe172a00640 1 -- 172.21.15.73:0/3872796897 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164030840 con 0x7fe17415a850 2024-04-05T22:21:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.161+0000 7fe172a00640 1 -- 172.21.15.73:0/3872796897 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1640309a0 con 0x7fe17415a850 2024-04-05T22:21:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.161+0000 7fe172a00640 1 -- 172.21.15.73:0/3872796897 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164030d30 con 0x7fe17415a850 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 -- 172.21.15.73:0/3872796897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 msgr2=0x7fe17415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 --2- 172.21.15.73:0/3872796897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe164009870 tx=0x7fe16402b980 comp rx=0 tx=0).stop 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 -- 172.21.15.73:0/3872796897 shutdown_connections 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 --2- 172.21.15.73:0/3872796897 >> v2:172.21.15.157:3300/0 conn(0x7fe17415e580 0x7fe17406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 --2- 172.21.15.73:0/3872796897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe17415ac30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 -- 172.21.15.73:0/3872796897 >> 172.21.15.73:0/3872796897 conn(0x7fe174069cb0 msgr2=0x7fe17406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 -- 172.21.15.73:0/3872796897 shutdown_connections 2024-04-05T22:21:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.162+0000 7fe179a00640 1 -- 172.21.15.73:0/3872796897 wait complete. 2024-04-05T22:21:57.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.163+0000 7fe179a00640 1 Processor -- start 2024-04-05T22:21:57.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.164+0000 7fe179a00640 1 -- start start 2024-04-05T22:21:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe1740cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe17415e9a0 0x7fe1741440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174150110 con 0x7fe17415e580 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740cc7c0 con 0x7fe17415e9a0 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740cca20 con 0x7fe17415a850 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44096/0 (socket says 172.21.15.73:44096) 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 -- 172.21.15.73:0/889852692 learned_addr learned my addr 172.21.15.73:0/889852692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173e00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe1740cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe179000640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe17415e9a0 0x7fe1741440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 -- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 msgr2=0x7fe1740cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe1740cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 -- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe17415e9a0 msgr2=0x7fe1741440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.165+0000 7fe173400640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe17415e9a0 0x7fe1741440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe173400640 1 -- 172.21.15.73:0/889852692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640094d0 con 0x7fe17415e580 2024-04-05T22:21:57.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe173e00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe1740cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe173400640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00efc0 tx=0x7fe15c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c014480 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00f040 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1740ccd00 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c014800 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.166+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740cd2c0 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.167+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17404e160 con 0x7fe17415e580 2024-04-05T22:21:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.168+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe15c014960 con 0x7fe17415e580 2024-04-05T22:21:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.169+0000 7fe16be00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 0x7fe15006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.169+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe15c089f90 con 0x7fe17415e580 2024-04-05T22:21:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.169+0000 7fe173e00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 0x7fe15006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.170+0000 7fe173e00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 0x7fe15006d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe164036000 tx=0x7fe16403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.174+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe15c0bd650 con 0x7fe17415e580 2024-04-05T22:21:57.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.365+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1740b1f80 con 0x7fe17415e580 2024-04-05T22:21:57.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.366+0000 7fe16be00640 1 -- 172.21.15.73:0/889852692 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe15c058c00 con 0x7fe17415e580 2024-04-05T22:21:57.366 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:57.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 msgr2=0x7fe15006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 0x7fe15006d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe164036000 tx=0x7fe16403b040 comp rx=0 tx=0).stop 2024-04-05T22:21:57.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 msgr2=0x7fe1740cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00efc0 tx=0x7fe15c00c490 comp rx=0 tx=0).stop 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 shutdown_connections 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe15006ac00 0x7fe15006d0c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe17415e9a0 0x7fe1741440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe17415e580 0x7fe1740cc280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 --2- 172.21.15.73:0/889852692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe17415a850 0x7fe1740cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 >> 172.21.15.73:0/889852692 conn(0x7fe174069cb0 msgr2=0x7fe1740af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 shutdown_connections 2024-04-05T22:21:57.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.369+0000 7fe179a00640 1 -- 172.21.15.73:0/889852692 wait complete. 2024-04-05T22:21:57.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:21:57.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.590+0000 7f829ec00640 1 Processor -- start 2024-04-05T22:21:57.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.591+0000 7f829ec00640 1 -- start start 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f829806c9a0 0x7f829806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f829806ba70 0x7f8298164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829806bea0 con 0x7f829815abe0 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829815afc0 con 0x7f829806ba70 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829806f5b0 con 0x7f829806c9a0 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f829cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f829806ba70 0x7f8298164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f8297e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f829815abe0 0x7f82980712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34386/0 (socket says 172.21.15.73:34386) 2024-04-05T22:21:57.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.592+0000 7f8297e00640 1 -- 172.21.15.73:0/2933239098 learned_addr learned my addr 172.21.15.73:0/2933239098 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f829cc00640 1 -- 172.21.15.73:0/2933239098 >> v2:172.21.15.157:3300/0 conn(0x7f829806c9a0 msgr2=0x7f829806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f828f400640 1 --2- 172.21.15.73:0/2933239098 >> v2:172.21.15.157:3300/0 conn(0x7f829806c9a0 0x7f829806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f829cc00640 1 --2- 172.21.15.73:0/2933239098 >> v2:172.21.15.157:3300/0 conn(0x7f829806c9a0 0x7f829806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f829cc00640 1 -- 172.21.15.73:0/2933239098 >> v1:172.21.15.73:6789/0 conn(0x7f829815abe0 legacy=0x7f82980712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f8297400640 1 -- 172.21.15.73:0/2933239098 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138498230 0 0) 0x7f829806bea0 con 0x7f829815abe0 2024-04-05T22:21:57.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.593+0000 7f829cc00640 1 -- 172.21.15.73:0/2933239098 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829806f890 con 0x7f829806ba70 2024-04-05T22:21:57.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.594+0000 7f829cc00640 1 --2- 172.21.15.73:0/2933239098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829806ba70 0x7f8298164740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829806ce30 tx=0x7f829002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6e40b74f01339 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.594+0000 7f8297400640 1 -- 172.21.15.73:0/2933239098 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829002cb60 con 0x7f829806ba70 2024-04-05T22:21:57.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.594+0000 7f8297400640 1 -- 172.21.15.73:0/2933239098 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829002ccc0 con 0x7f829806ba70 2024-04-05T22:21:57.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.594+0000 7f8297400640 1 -- 172.21.15.73:0/2933239098 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290034660 con 0x7f829806ba70 2024-04-05T22:21:57.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 -- 172.21.15.73:0/2933239098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829806ba70 msgr2=0x7f8298164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 --2- 172.21.15.73:0/2933239098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829806ba70 0x7f8298164740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829806ce30 tx=0x7f829002bdb0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 -- 172.21.15.73:0/2933239098 shutdown_connections 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 --2- 172.21.15.73:0/2933239098 >> v2:172.21.15.157:3300/0 conn(0x7f829806c9a0 0x7f829806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 --2- 172.21.15.73:0/2933239098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829806ba70 0x7f8298164740 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 -- 172.21.15.73:0/2933239098 >> 172.21.15.73:0/2933239098 conn(0x7f8298069cb0 msgr2=0x7f829806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 -- 172.21.15.73:0/2933239098 shutdown_connections 2024-04-05T22:21:57.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.595+0000 7f829ec00640 1 -- 172.21.15.73:0/2933239098 wait complete. 2024-04-05T22:21:57.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.597+0000 7f829ec00640 1 Processor -- start 2024-04-05T22:21:57.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.597+0000 7f829ec00640 1 -- start start 2024-04-05T22:21:57.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f829806ba70 0x7f8298143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f829806c9a0 0x7f82981520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980acc80 con 0x7f829815abe0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298143980 con 0x7f829806c9a0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980b1fd0 con 0x7f829806ba70 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44110/0 (socket says 172.21.15.73:44110) 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 -- 172.21.15.73:0/350695732 learned_addr learned my addr 172.21.15.73:0/350695732 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f828f400640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f829806c9a0 0x7f82981520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 -- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f829806ba70 msgr2=0x7f8298143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f829806ba70 0x7f8298143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 -- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f829806c9a0 msgr2=0x7f82981520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f829806c9a0 0x7f82981520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.598+0000 7f829cc00640 1 -- 172.21.15.73:0/350695732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009a70 con 0x7f829815abe0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f829cc00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8290037040 tx=0x7f8290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290040650 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82900407b0 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980b22b0 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290040b60 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.599+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82980b2870 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.600+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298068160 con 0x7f829815abe0 2024-04-05T22:21:57.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.601+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8290040cc0 con 0x7f829815abe0 2024-04-05T22:21:57.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.602+0000 7f8294c00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 0x7f827406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:57.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.602+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82900ac320 con 0x7f829815abe0 2024-04-05T22:21:57.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.602+0000 7f8297e00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 0x7f827406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:57.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.603+0000 7f8297e00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 0x7f827406d0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8288005880 tx=0x7f8288007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:57.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.606+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8290077610 con 0x7f829815abe0 2024-04-05T22:21:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.800+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8298157270 con 0x7f829815abe0 2024-04-05T22:21:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.800+0000 7f8294c00640 1 -- 172.21.15.73:0/350695732 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f829007ac30 con 0x7f829815abe0 2024-04-05T22:21:57.801 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.803+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 msgr2=0x7f827406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.803+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 0x7f827406d0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8288005880 tx=0x7f8288007a00 comp rx=0 tx=0).stop 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.803+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 msgr2=0x7f829814b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.803+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8290037040 tx=0x7f8290004a40 comp rx=0 tx=0).stop 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.803+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 shutdown_connections 2024-04-05T22:21:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f827406ac30 0x7f827406d0f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f829815abe0 0x7f829814b990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f829806c9a0 0x7f82981520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 --2- 172.21.15.73:0/350695732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f829806ba70 0x7f8298143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 >> 172.21.15.73:0/350695732 conn(0x7f8298069cb0 msgr2=0x7f8298157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 shutdown_connections 2024-04-05T22:21:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:57.804+0000 7f829ec00640 1 -- 172.21.15.73:0/350695732 wait complete. 2024-04-05T22:21:57.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:21:58.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.034+0000 7f5143e00640 1 Processor -- start 2024-04-05T22:21:58.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.034+0000 7f5143e00640 1 -- start start 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.035+0000 7f5143e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144160ff0 0x7f51440cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.035+0000 7f5143e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51441613d0 con 0x7f51440ae170 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5143e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514406b4f0 con 0x7f51440b2b90 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5143e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514406b760 con 0x7f5144160ff0 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144160ff0 0x7f51440cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144160ff0 0x7f51440cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44112/0 (socket says 172.21.15.73:44112) 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 -- 172.21.15.73:0/308102593 learned_addr learned my addr 172.21.15.73:0/308102593 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:58.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f513a000640 1 --1- 172.21.15.73:0/308102593 >> v1:172.21.15.73:6789/0 conn(0x7f51440ae170 0x7f51440b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34398/0 (socket says 172.21.15.73:34398) 2024-04-05T22:21:58.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 -- 172.21.15.73:0/308102593 >> v1:172.21.15.157:6789/0 conn(0x7f51440b2b90 legacy=0x7f514406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:58.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 -- 172.21.15.73:0/308102593 >> v1:172.21.15.73:6789/0 conn(0x7f51440ae170 legacy=0x7f51440b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142a00640 1 -- 172.21.15.73:0/308102593 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514406ba40 con 0x7f5144160ff0 2024-04-05T22:21:58.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.036+0000 7f5142000640 1 -- 172.21.15.73:0/308102593 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413040715 0 0) 0x7f51441613d0 con 0x7f51440ae170 2024-04-05T22:21:58.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.037+0000 7f5142a00640 1 --2- 172.21.15.73:0/308102593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f51440cb960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9719bd365945fbca server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5142000640 1 -- 172.21.15.73:0/308102593 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134030860 con 0x7f5144160ff0 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5142000640 1 -- 172.21.15.73:0/308102593 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51340309c0 con 0x7f5144160ff0 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5142000640 1 -- 172.21.15.73:0/308102593 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134030d50 con 0x7f5144160ff0 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 -- 172.21.15.73:0/308102593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 msgr2=0x7f51440cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 --2- 172.21.15.73:0/308102593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f51440cb960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 -- 172.21.15.73:0/308102593 shutdown_connections 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 --2- 172.21.15.73:0/308102593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f51440cb960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 -- 172.21.15.73:0/308102593 >> 172.21.15.73:0/308102593 conn(0x7f51440b0830 msgr2=0x7f51440b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 -- 172.21.15.73:0/308102593 shutdown_connections 2024-04-05T22:21:58.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.038+0000 7f5143e00640 1 -- 172.21.15.73:0/308102593 wait complete. 2024-04-05T22:21:58.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.040+0000 7f5143e00640 1 Processor -- start 2024-04-05T22:21:58.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.040+0000 7f5143e00640 1 -- start start 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51440ae170 0x7f5144164080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51440b2b90 0x7f51441625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144163630 con 0x7f5144160ff0 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514415d320 con 0x7f51440b2b90 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5144162b10 con 0x7f51440ae170 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.041+0000 7f5143400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44114/0 (socket says 172.21.15.73:44114) 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 -- 172.21.15.73:0/2976759937 learned_addr learned my addr 172.21.15.73:0/2976759937 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 -- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51440ae170 msgr2=0x7f5144164080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51440ae170 0x7f5144164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 -- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51440b2b90 msgr2=0x7f51441625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51440b2b90 0x7f51441625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 -- 172.21.15.73:0/2976759937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340094d0 con 0x7f5144160ff0 2024-04-05T22:21:58.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f5143400640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c7b0 tx=0x7f513c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c004b60 con 0x7f5144160ff0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c004cc0 con 0x7f5144160ff0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.042+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00aa90 con 0x7f5144160ff0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.043+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5144162d90 con 0x7f5144160ff0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.043+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5144163350 con 0x7f5144160ff0 2024-04-05T22:21:58.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.044+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404e160 con 0x7f5144160ff0 2024-04-05T22:21:58.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.045+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f513c00abf0 con 0x7f5144160ff0 2024-04-05T22:21:58.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.046+0000 7f513b400640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 0x7f512006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.046+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f513c089570 con 0x7f5144160ff0 2024-04-05T22:21:58.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.046+0000 7f5142a00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 0x7f512006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.047+0000 7f5142a00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 0x7f512006d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5134036000 tx=0x7f513403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.051+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f513c054740 con 0x7f5144160ff0 2024-04-05T22:21:58.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.242+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f514415d4d0 con 0x7f5144160ff0 2024-04-05T22:21:58.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.243+0000 7f513b400640 1 -- 172.21.15.73:0/2976759937 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f514415d4d0 con 0x7f5144160ff0 2024-04-05T22:21:58.244 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 msgr2=0x7f512006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 0x7f512006d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5134036000 tx=0x7f513403b040 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 msgr2=0x7f5144164a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c7b0 tx=0x7f513c00cc70 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 shutdown_connections 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f512006ac00 0x7f512006d0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144160ff0 0x7f5144164a90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51440b2b90 0x7f51441625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 --2- 172.21.15.73:0/2976759937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51440ae170 0x7f5144164080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 >> 172.21.15.73:0/2976759937 conn(0x7f51440b0830 msgr2=0x7f51440b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 shutdown_connections 2024-04-05T22:21:58.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.246+0000 7f5143e00640 1 -- 172.21.15.73:0/2976759937 wait complete. 2024-04-05T22:21:58.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:21:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.472+0000 7ff7ec000640 1 Processor -- start 2024-04-05T22:21:58.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.472+0000 7ff7ec000640 1 -- start start 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7ec000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e41587d0 0x7ff7e40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7ec000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 0x7ff7e406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7ec000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e406f5b0 con 0x7ff7e406d0c0 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7ec000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e4070410 con 0x7ff7e41587d0 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7ec000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4158bb0 con 0x7ff7e40ccd20 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7e9600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e41587d0 0x7ff7e40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7e9600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e41587d0 0x7ff7e40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44128/0 (socket says 172.21.15.73:44128) 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.474+0000 7ff7e9600640 1 -- 172.21.15.73:0/813522934 learned_addr learned my addr 172.21.15.73:0/813522934 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:58.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e8c00640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 0x7ff7e406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e9600640 1 -- 172.21.15.73:0/813522934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 msgr2=0x7ff7e406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e9600640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 0x7ff7e406cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e9600640 1 -- 172.21.15.73:0/813522934 >> v1:172.21.15.157:6789/0 conn(0x7ff7e406d0c0 legacy=0x7ff7e415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e9600640 1 -- 172.21.15.73:0/813522934 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e406f800 con 0x7ff7e41587d0 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.475+0000 7ff7e8c00640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 0x7ff7e406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:58.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.476+0000 7ff7e9600640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41587d0 0x7ff7e40cc7e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009870 tx=0x7ff7d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eea7ad76ca63758 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.476+0000 7ff7e3e00640 1 -- 172.21.15.73:0/813522934 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8031640 con 0x7ff7e41587d0 2024-04-05T22:21:58.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.476+0000 7ff7e3e00640 1 -- 172.21.15.73:0/813522934 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d80317a0 con 0x7ff7e41587d0 2024-04-05T22:21:58.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.476+0000 7ff7e3e00640 1 -- 172.21.15.73:0/813522934 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8031b30 con 0x7ff7e41587d0 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 -- 172.21.15.73:0/813522934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41587d0 msgr2=0x7ff7e40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41587d0 0x7ff7e40cc7e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009870 tx=0x7ff7d802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 -- 172.21.15.73:0/813522934 shutdown_connections 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e40ccd20 0x7ff7e406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 --2- 172.21.15.73:0/813522934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41587d0 0x7ff7e40cc7e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 -- 172.21.15.73:0/813522934 >> 172.21.15.73:0/813522934 conn(0x7ff7e40ce8e0 msgr2=0x7ff7e40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 -- 172.21.15.73:0/813522934 shutdown_connections 2024-04-05T22:21:58.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.477+0000 7ff7ec000640 1 -- 172.21.15.73:0/813522934 wait complete. 2024-04-05T22:21:58.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.479+0000 7ff7ec000640 1 Processor -- start 2024-04-05T22:21:58.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.479+0000 7ff7ec000640 1 -- start start 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e406d0c0 0x7ff7e415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e41587d0 0x7ff7e41f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41f65f0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4133040 con 0x7ff7e406d0c0 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ec000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41332e0 con 0x7ff7e41587d0 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ea000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ea000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44132/0 (socket says 172.21.15.73:44132) 2024-04-05T22:21:58.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ea000640 1 -- 172.21.15.73:0/1841834135 learned_addr learned my addr 172.21.15.73:0/1841834135 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7e8c00640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e41587d0 0x7ff7e41f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7e9600640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e406d0c0 0x7ff7e415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ea000640 1 -- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e41587d0 msgr2=0x7ff7e41f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.481+0000 7ff7ea000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e41587d0 0x7ff7e41f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ea000640 1 -- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e406d0c0 msgr2=0x7ff7e415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ea000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e406d0c0 0x7ff7e415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ea000640 1 -- 172.21.15.73:0/1841834135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d80094d0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ea000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c780 tx=0x7ff7dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc004b60 con 0x7ff7e41f5c20 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00ce90 con 0x7ff7e41f5c20 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00a850 con 0x7ff7e41f5c20 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e41335c0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.482+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e4133b80 con 0x7ff7e41f5c20 2024-04-05T22:21:58.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.483+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4066af0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.484+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff7dc00a9b0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.485+0000 7ff7e1600640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 0x7ff7c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:58.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.485+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7dc0895b0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.485+0000 7ff7e9600640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 0x7ff7c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:58.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.486+0000 7ff7e9600640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 0x7ff7c406d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8036000 tx=0x7ff7d803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:58.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.491+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff7dc054790 con 0x7ff7e41f5c20 2024-04-05T22:21:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.698+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7e4068160 con 0x7ff7e41f5c20 2024-04-05T22:21:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.699+0000 7ff7e1600640 1 -- 172.21.15.73:0/1841834135 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff7dc057db0 con 0x7ff7e41f5c20 2024-04-05T22:21:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:21:58.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 msgr2=0x7ff7c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 0x7ff7c406d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8036000 tx=0x7ff7d803c040 comp rx=0 tx=0).stop 2024-04-05T22:21:58.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 msgr2=0x7ff7e41f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:58.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c780 tx=0x7ff7dc00cc40 comp rx=0 tx=0).stop 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 shutdown_connections 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff7c406ac00 0x7ff7c406d0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e41f5c20 0x7ff7e41f60b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7e41587d0 0x7ff7e41f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 --2- 172.21.15.73:0/1841834135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e406d0c0 0x7ff7e415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:58.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.703+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 >> 172.21.15.73:0/1841834135 conn(0x7ff7e40ce8e0 msgr2=0x7ff7e40cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:58.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.704+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 shutdown_connections 2024-04-05T22:21:58.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:58.704+0000 7ff7ec000640 1 -- 172.21.15.73:0/1841834135 wait complete. 2024-04-05T22:21:58.723 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:21:58.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:58.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-05T22:21:58.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:21:58.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:21:58.824 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:21:58.881 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:21:58.882 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:58.940 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:21:58.941 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:58.990 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:21:58.991 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:21:59.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:21:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.255+0000 7f436dc00640 1 Processor -- start 2024-04-05T22:21:59.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.255+0000 7f436dc00640 1 -- start start 2024-04-05T22:21:59.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f436dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436815d120 0x7f4368071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f436dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f436806ede0 0x7f43680b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f436dc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436806bbc0 con 0x7f436806e920 2024-04-05T22:21:59.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f436dc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436815d500 con 0x7f436806ede0 2024-04-05T22:21:59.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f436dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436806b8e0 con 0x7f436815d120 2024-04-05T22:21:59.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f4366000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f436806ede0 0x7f43680b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f4366000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f436806ede0 0x7f43680b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44138/0 (socket says 172.21.15.73:44138) 2024-04-05T22:21:59.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f4366000640 1 -- 172.21.15.73:0/1169342498 learned_addr learned my addr 172.21.15.73:0/1169342498 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:59.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.257+0000 7f4366a00640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436815d120 0x7f4368071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.258+0000 7f4366000640 1 -- 172.21.15.73:0/1169342498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436815d120 msgr2=0x7f4368071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.258+0000 7f4366000640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436815d120 0x7f4368071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.258+0000 7f4366000640 1 -- 172.21.15.73:0/1169342498 >> v1:172.21.15.73:6789/0 conn(0x7f436806e920 legacy=0x7f43680cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.258+0000 7f4366000640 1 -- 172.21.15.73:0/1169342498 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368070760 con 0x7f436806ede0 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.258+0000 7f4365600640 1 -- 172.21.15.73:0/1169342498 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233809048 0 0) 0x7f436806bbc0 con 0x7f436806e920 2024-04-05T22:21:59.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f4366000640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f436806ede0 0x7f43680b4040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b0eecb65593b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f4365600640 1 -- 172.21.15.73:0/1169342498 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350030840 con 0x7f436806ede0 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f4365600640 1 -- 172.21.15.73:0/1169342498 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43500309a0 con 0x7f436806ede0 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f4365600640 1 -- 172.21.15.73:0/1169342498 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350030d30 con 0x7f436806ede0 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f436dc00640 1 -- 172.21.15.73:0/1169342498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f436806ede0 msgr2=0x7f43680b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.259+0000 7f436dc00640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f436806ede0 0x7f43680b4040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b980 comp rx=0 tx=0).stop 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 -- 172.21.15.73:0/1169342498 shutdown_connections 2024-04-05T22:21:59.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f436806ede0 0x7f43680b4040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 --2- 172.21.15.73:0/1169342498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436815d120 0x7f4368071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 -- 172.21.15.73:0/1169342498 >> 172.21.15.73:0/1169342498 conn(0x7f436806adc0 msgr2=0x7f436806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 -- 172.21.15.73:0/1169342498 shutdown_connections 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.260+0000 7f436dc00640 1 -- 172.21.15.73:0/1169342498 wait complete. 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.261+0000 7f436dc00640 1 Processor -- start 2024-04-05T22:21:59.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.261+0000 7f436dc00640 1 -- start start 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436806e920 0x7f436814d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436815d120 0x7f43681f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43681f36f0 con 0x7f43681f30f0 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43681f39d0 con 0x7f436815d120 2024-04-05T22:21:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.262+0000 7f436dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43681f40f0 con 0x7f436806e920 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44152/0 (socket says 172.21.15.73:44152) 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 -- 172.21.15.73:0/2581339074 learned_addr learned my addr 172.21.15.73:0/2581339074 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 -- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436806e920 msgr2=0x7f436814d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436806e920 0x7f436814d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 -- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436815d120 msgr2=0x7f43681f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436815d120 0x7f43681f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 -- 172.21.15.73:0/2581339074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500094d0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.263+0000 7f4367400640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4360001560 tx=0x7f436000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.264+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360003da0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.264+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360005070 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.264+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360005420 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.264+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43681f43d0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.264+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43681f3cb0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.265+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368066af0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.266+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4360005580 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.267+0000 7f435ea00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 0x7f434406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.267+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4360088cc0 con 0x7f43681f30f0 2024-04-05T22:21:59.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.267+0000 7f4366a00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 0x7f434406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.267+0000 7f4366a00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 0x7f434406d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4358006fd0 tx=0x7f4358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.271+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4360053e10 con 0x7f43681f30f0 2024-04-05T22:21:59.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.465+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43681f45c0 con 0x7f43681f30f0 2024-04-05T22:21:59.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.467+0000 7f435ea00640 1 -- 172.21.15.73:0/2581339074 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f4360017030 con 0x7f43681f30f0 2024-04-05T22:21:59.467 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:21:59.467 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:21:15.888823+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 msgr2=0x7f434406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 0x7f434406d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4358006fd0 tx=0x7f4358008040 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 msgr2=0x7f43681581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4360001560 tx=0x7f436000cc70 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 shutdown_connections 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f434406ac00 0x7f434406d0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43681f30f0 0x7f43681581e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436815d120 0x7f43681f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 --2- 172.21.15.73:0/2581339074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436806e920 0x7f436814d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 >> 172.21.15.73:0/2581339074 conn(0x7f436806adc0 msgr2=0x7f43680cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 shutdown_connections 2024-04-05T22:21:59.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.470+0000 7f436dc00640 1 -- 172.21.15.73:0/2581339074 wait complete. 2024-04-05T22:21:59.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:21:59.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:21:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.705+0000 7f8f47e00640 1 Processor -- start 2024-04-05T22:21:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.705+0000 7f8f47e00640 1 -- start start 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.706+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f40070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.706+0000 7f8f47e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f40071900 con 0x7f8f4015dd90 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.706+0000 7f8f47e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4015ae50 con 0x7f8f400714a0 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.706+0000 7f8f47e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015e3b0 con 0x7f8f4015aa70 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.707+0000 7f8f45e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f4015dd90 0x7f8f4006ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34424/0 (socket says 172.21.15.73:34424) 2024-04-05T22:21:59.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.707+0000 7f8f45e00640 1 -- 172.21.15.73:0/2189466142 learned_addr learned my addr 172.21.15.73:0/2189466142 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:21:59.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.707+0000 7f8f45400640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f40070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.707+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797670037 0 0) 0x7f8f40071900 con 0x7f8f4015dd90 2024-04-05T22:21:59.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.707+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f28003520 con 0x7f8f4015dd90 2024-04-05T22:21:59.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554820757 0 0) 0x7f8f28003520 con 0x7f8f4015dd90 2024-04-05T22:21:59.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 msgr2=0x7f8f40070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f40070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 >> v1:172.21.15.157:6789/0 conn(0x7f8f400714a0 legacy=0x7f8f4015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400ac9e0 con 0x7f8f4015dd90 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8f380019f0 con 0x7f8f4015dd90 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 >> v1:172.21.15.73:6789/0 conn(0x7f8f4015dd90 legacy=0x7f8f4006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f280039a0 0x7f8f28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f28004d00 0x7f8f28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015e3b0 con 0x7f8f28004d00 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2800bb10 con 0x7f8f280039a0 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2800bca0 con 0x7f8f4015aa70 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f45e00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f28004d00 0x7f8f28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f44a00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f45400640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f280039a0 0x7f8f28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.708+0000 7f8f45e00640 1 -- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 msgr2=0x7f8f280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.709+0000 7f8f45e00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.709+0000 7f8f45e00640 1 -- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f280039a0 msgr2=0x7f8f28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:21:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.709+0000 7f8f45e00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f280039a0 0x7f8f28003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.709+0000 7f8f45e00640 1 -- 172.21.15.73:0/2189466142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f28003520 con 0x7f8f28004d00 2024-04-05T22:21:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.709+0000 7f8f44a00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015aa70 0x7f8f280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f45e00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f28004d00 0x7f8f28005180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800a860 tx=0x7f8f280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6900dcbbceca7c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380072d0 con 0x7f8f28004d00 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3802f070 con 0x7f8f28004d00 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f3fe00640 1 -- 172.21.15.73:0/2189466142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380076b0 con 0x7f8f28004d00 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f47e00640 1 -- 172.21.15.73:0/2189466142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f28004d00 msgr2=0x7f8f28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2189466142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f28004d00 0x7f8f28005180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800a860 tx=0x7f8f280057f0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f45e00640 1 -- 172.21.15.73:0/2189466142 reap_dead start 2024-04-05T22:21:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f47e00640 1 -- 172.21.15.73:0/2189466142 shutdown_connections 2024-04-05T22:21:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.710+0000 7f8f47e00640 1 -- 172.21.15.73:0/2189466142 >> 172.21.15.73:0/2189466142 conn(0x7f8f40069cb0 msgr2=0x7f8f4006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.711+0000 7f8f47e00640 1 -- 172.21.15.73:0/2189466142 shutdown_connections 2024-04-05T22:21:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.711+0000 7f8f47e00640 1 -- 172.21.15.73:0/2189466142 wait complete. 2024-04-05T22:21:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.712+0000 7f8f47e00640 1 Processor -- start 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.712+0000 7f8f47e00640 1 -- start start 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28004d00 0x7f8f40144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015c7c0 0x7f8f401f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f401f6d60 con 0x7f8f40144b50 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015cc70 con 0x7f8f28004d00 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f47e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f40145950 con 0x7f8f4015c7c0 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.713+0000 7f8f44a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44174/0 (socket says 172.21.15.73:44174) 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 -- 172.21.15.73:0/2324762424 learned_addr learned my addr 172.21.15.73:0/2324762424 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:21:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f45400640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28004d00 0x7f8f40144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f45e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015c7c0 0x7f8f401f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 -- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015c7c0 msgr2=0x7f8f401f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015c7c0 0x7f8f401f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 -- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28004d00 msgr2=0x7f8f40144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28004d00 0x7f8f40144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 -- 172.21.15.73:0/2324762424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40145de0 con 0x7f8f40144b50 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.714+0000 7f8f44a00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d7a0 tx=0x7f8f2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.715+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c017070 con 0x7f8f40144b50 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.715+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c005070 con 0x7f8f40144b50 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.715+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40145020 con 0x7f8f40144b50 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.715+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c005420 con 0x7f8f40144b50 2024-04-05T22:21:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.715+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f401455e0 con 0x7f8f40144b50 2024-04-05T22:21:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.716+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f40068160 con 0x7f8f40144b50 2024-04-05T22:21:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.717+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8f2c007030 con 0x7f8f40144b50 2024-04-05T22:21:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.717+0000 7f8f3d600640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 0x7f8f2006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:21:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.718+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f2c088ce0 con 0x7f8f40144b50 2024-04-05T22:21:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.718+0000 7f8f45400640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 0x7f8f2006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:21:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.718+0000 7f8f45400640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 0x7f8f2006d0f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f40158dd0 tx=0x7f8f34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:21:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.722+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8f2c053e20 con 0x7f8f40144b50 2024-04-05T22:21:59.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.914+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f40145a90 con 0x7f8f40144b50 2024-04-05T22:21:59.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.915+0000 7f8f3d600640 1 -- 172.21.15.73:0/2324762424 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f2c057440 con 0x7f8f40144b50 2024-04-05T22:21:59.916 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 msgr2=0x7f8f2006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 0x7f8f2006d0f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f40158dd0 tx=0x7f8f34008040 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 msgr2=0x7f8f4015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d7a0 tx=0x7f8f2c00dc60 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 shutdown_connections 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8f2006ac30 0x7f8f2006d0f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f4015c7c0 0x7f8f401f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f40144b50 0x7f8f4015c280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 --2- 172.21.15.73:0/2324762424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28004d00 0x7f8f40144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 >> 172.21.15.73:0/2324762424 conn(0x7f8f40069cb0 msgr2=0x7f8f400b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 shutdown_connections 2024-04-05T22:21:59.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:21:59.918+0000 7f8f47e00640 1 -- 172.21.15.73:0/2324762424 wait complete. 2024-04-05T22:21:59.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:00.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.145+0000 7fba00c00640 1 Processor -- start 2024-04-05T22:22:00.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.145+0000 7fba00c00640 1 -- start start 2024-04-05T22:22:00.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fba00c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fba00c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0ceec0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fba00c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0ce590 con 0x7fb9fc0d1910 2024-04-05T22:22:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fba00c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc0ce870 con 0x7fb9fc15cf50 2024-04-05T22:22:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fb9f9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fb9f9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44184/0 (socket says 172.21.15.73:44184) 2024-04-05T22:22:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.146+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2162941625 learned_addr learned my addr 172.21.15.73:0/2162941625 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.147+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2162941625 >> v1:172.21.15.157:6789/0 conn(0x7fb9fc0d1910 legacy=0x7fb9fc0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.147+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2162941625 >> v1:172.21.15.73:6789/0 conn(0x7fb9fc0d0bb0 legacy=0x7fb9fc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.147+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2162941625 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0ceb50 con 0x7fb9fc15cf50 2024-04-05T22:22:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.147+0000 7fb9f9400640 1 -- 172.21.15.73:0/2162941625 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538942031 0 0) 0x7fb9fc0ceec0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.148+0000 7fb9f9e00640 1 --2- 172.21.15.73:0/2162941625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d13dc1373af1a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.148+0000 7fb9f9400640 1 -- 172.21.15.73:0/2162941625 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec031870 con 0x7fb9fc15cf50 2024-04-05T22:22:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.148+0000 7fb9f9400640 1 -- 172.21.15.73:0/2162941625 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0319d0 con 0x7fb9fc15cf50 2024-04-05T22:22:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.148+0000 7fb9f9400640 1 -- 172.21.15.73:0/2162941625 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec031d60 con 0x7fb9fc15cf50 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 -- 172.21.15.73:0/2162941625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc15cf50 msgr2=0x7fb9fc0d38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 --2- 172.21.15.73:0/2162941625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02b910 comp rx=0 tx=0).stop 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 -- 172.21.15.73:0/2162941625 shutdown_connections 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 --2- 172.21.15.73:0/2162941625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc15cf50 0x7fb9fc0d38f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 -- 172.21.15.73:0/2162941625 >> 172.21.15.73:0/2162941625 conn(0x7fb9fc0d2dd0 msgr2=0x7fb9fc0d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 -- 172.21.15.73:0/2162941625 shutdown_connections 2024-04-05T22:22:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.149+0000 7fba00c00640 1 -- 172.21.15.73:0/2162941625 wait complete. 2024-04-05T22:22:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.151+0000 7fba00c00640 1 Processor -- start 2024-04-05T22:22:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.151+0000 7fba00c00640 1 -- start start 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9fc0d1910 0x7fb9fc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9fc15cf50 0x7fb9fc0b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc14dd90 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc14d110 con 0x7fb9fc0d1910 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fba00c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc14d3f0 con 0x7fb9fc15cf50 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fb9f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fb9f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44196/0 (socket says 172.21.15.73:44196) 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.152+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2009965434 learned_addr learned my addr 172.21.15.73:0/2009965434 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:00.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9fc15cf50 msgr2=0x7fb9fc0b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9fc15cf50 0x7fb9fc0b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9fc0d1910 msgr2=0x7fb9fc06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9fc0d1910 0x7fb9fc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 -- 172.21.15.73:0/2009965434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0094d0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f9e00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec031820 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0443f0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0447a0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc14d6d0 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.153+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc06c280 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.155+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc14d530 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.156+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9ec04d450 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.156+0000 7fb9f2000640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 0x7fb9d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.156+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9ec02d040 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.157+0000 7fb9f3e00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 0x7fb9d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.157+0000 7fb9f3e00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 0x7fb9d806d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0ce4e0 tx=0x7fb9e4007910 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.161+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb9fc14d530 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.354+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9fc14d530 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.355+0000 7fb9f2000640 1 -- 172.21.15.73:0/2009965434 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9fc14d530 con 0x7fb9fc0d0bb0 2024-04-05T22:22:00.356 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 msgr2=0x7fb9d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 0x7fb9d806d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0ce4e0 tx=0x7fb9e4007910 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 msgr2=0x7fb9fc14e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec0046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 shutdown_connections 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb9d806ac00 0x7fb9d806d0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9fc15cf50 0x7fb9fc0b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9fc0d1910 0x7fb9fc06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 --2- 172.21.15.73:0/2009965434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9fc0d0bb0 0x7fb9fc14e1b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.357+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 >> 172.21.15.73:0/2009965434 conn(0x7fb9fc0d2dd0 msgr2=0x7fb9fc15c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.358+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 shutdown_connections 2024-04-05T22:22:00.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.358+0000 7fba00c00640 1 -- 172.21.15.73:0/2009965434 wait complete. 2024-04-05T22:22:00.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:00.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.602+0000 7fd0d7800640 1 Processor -- start 2024-04-05T22:22:00.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.602+0000 7fd0d7800640 1 -- start start 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.603+0000 7fd0d7800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0d006c630 0x7fd0d00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.603+0000 7fd0d7800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 0x7fd0d00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.603+0000 7fd0d7800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d0146ac0 con 0x7fd0d015dd90 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.603+0000 7fd0d7800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d0070720 con 0x7fd0d006c630 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.603+0000 7fd0d7800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00acf60 con 0x7fd0d0070c50 2024-04-05T22:22:00.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 0x7fd0d00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 0x7fd0d00710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:35120/0 (socket says 172.21.15.73:35120) 2024-04-05T22:22:00.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d5800640 1 -- 172.21.15.73:0/3465986612 learned_addr learned my addr 172.21.15.73:0/3465986612 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:00.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3465986612 >> v2:172.21.15.157:3300/0 conn(0x7fd0d006c630 0x7fd0d00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d4e00640 1 -- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 msgr2=0x7fd0d00710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:00.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 0x7fd0d00710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.604+0000 7fd0d4e00640 1 -- 172.21.15.73:0/3465986612 >> v1:172.21.15.157:6789/0 conn(0x7fd0d015dd90 legacy=0x7fd0d006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.605+0000 7fd0d4e00640 1 -- 172.21.15.73:0/3465986612 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00ad240 con 0x7fd0d006c630 2024-04-05T22:22:00.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.606+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00ac8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00097a0 tx=0x7fd0c002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9cea57114b3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.606+0000 7fd0cf400640 1 -- 172.21.15.73:0/3465986612 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c002d040 con 0x7fd0d006c630 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0cf400640 1 -- 172.21.15.73:0/3465986612 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c0031810 con 0x7fd0d006c630 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0cf400640 1 -- 172.21.15.73:0/3465986612 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0031ba0 con 0x7fd0d006c630 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 -- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 msgr2=0x7fd0d00ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00ac8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00097a0 tx=0x7fd0c002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 -- 172.21.15.73:0/3465986612 shutdown_connections 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d0070c50 0x7fd0d00710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3465986612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00ac8d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 -- 172.21.15.73:0/3465986612 >> 172.21.15.73:0/3465986612 conn(0x7fd0d00b0830 msgr2=0x7fd0d00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:00.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 -- 172.21.15.73:0/3465986612 shutdown_connections 2024-04-05T22:22:00.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.607+0000 7fd0d7800640 1 -- 172.21.15.73:0/3465986612 wait complete. 2024-04-05T22:22:00.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.609+0000 7fd0d7800640 1 Processor -- start 2024-04-05T22:22:00.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.609+0000 7fd0d7800640 1 -- start start 2024-04-05T22:22:00.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d015dd90 0x7fd0d00cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d013ffc0 con 0x7fd0d0070c50 2024-04-05T22:22:00.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01441d0 con 0x7fd0d006c630 2024-04-05T22:22:00.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0d7800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0144560 con 0x7fd0d015dd90 2024-04-05T22:22:00.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44206/0 (socket says 172.21.15.73:44206) 2024-04-05T22:22:00.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.610+0000 7fd0cfe00640 1 -- 172.21.15.73:0/3423618213 learned_addr learned my addr 172.21.15.73:0/3423618213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0d5800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d015dd90 0x7fd0d00cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 -- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d015dd90 msgr2=0x7fd0d00cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d015dd90 0x7fd0d00cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 -- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 msgr2=0x7fd0d00cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 -- 172.21.15.73:0/3423618213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0009400 con 0x7fd0d0070c50 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0cfe00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c400 tx=0x7fd0b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.611+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80052d0 con 0x7fd0d0070c50 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.612+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b8005430 con 0x7fd0d0070c50 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.612+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8003830 con 0x7fd0d0070c50 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.612+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0140210 con 0x7fd0d0070c50 2024-04-05T22:22:00.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.612+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01407d0 con 0x7fd0d0070c50 2024-04-05T22:22:00.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.613+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d0144730 con 0x7fd0d0070c50 2024-04-05T22:22:00.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.614+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd0b8003990 con 0x7fd0d0070c50 2024-04-05T22:22:00.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.615+0000 7fd0ccc00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 0x7fd0ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:00.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.615+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0b80888e0 con 0x7fd0d0070c50 2024-04-05T22:22:00.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.615+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 0x7fd0ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:00.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.616+0000 7fd0d4e00640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 0x7fd0ac06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0036000 tx=0x7fd0c003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:00.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.620+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd0d0144730 con 0x7fd0d0070c50 2024-04-05T22:22:00.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.824+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0d0144730 con 0x7fd0d0070c50 2024-04-05T22:22:00.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.824+0000 7fd0ccc00640 1 -- 172.21.15.73:0/3423618213 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0d0144730 con 0x7fd0d0070c50 2024-04-05T22:22:00.825 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:00.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 msgr2=0x7fd0ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 0x7fd0ac06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0036000 tx=0x7fd0c003c040 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 msgr2=0x7fd0d00cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c400 tx=0x7fd0b800c8c0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 shutdown_connections 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0ac06ac00 0x7fd0ac06d0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0d015dd90 0x7fd0d00cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d0070c50 0x7fd0d00cb840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 --2- 172.21.15.73:0/3423618213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d006c630 0x7fd0d00cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.827+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 >> 172.21.15.73:0/3423618213 conn(0x7fd0d00b0830 msgr2=0x7fd0d0141b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.828+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 shutdown_connections 2024-04-05T22:22:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:00.828+0000 7fd0d7800640 1 -- 172.21.15.73:0/3423618213 wait complete. 2024-04-05T22:22:00.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:22:01.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.054+0000 7fce20e00640 1 Processor -- start 2024-04-05T22:22:01.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.054+0000 7fce20e00640 1 -- start start 2024-04-05T22:22:01.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce20e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce1c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce20e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce1c06e920 0x7fce1c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce20e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c06bbc0 con 0x7fce1c06ede0 2024-04-05T22:22:01.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce20e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c15d500 con 0x7fce1c06e920 2024-04-05T22:22:01.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce20e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c06b8e0 con 0x7fce1c15d120 2024-04-05T22:22:01.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce19400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fce1c06ede0 0x7fce1c0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34450/0 (socket says 172.21.15.73:34450) 2024-04-05T22:22:01.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.056+0000 7fce19400640 1 -- 172.21.15.73:0/152358166 learned_addr learned my addr 172.21.15.73:0/152358166 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148382088 0 0) 0x7fce1c06bbc0 con 0x7fce1c06ede0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce19e00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce1c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce1a800640 1 --2- 172.21.15.73:0/152358166 >> v2:172.21.15.157:3300/0 conn(0x7fce1c06e920 0x7fce1c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce00003520 con 0x7fce1c06ede0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3310782831 0 0) 0x7fce00003520 con 0x7fce1c06ede0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 msgr2=0x7fce1c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce1c071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 >> v2:172.21.15.157:3300/0 conn(0x7fce1c06e920 msgr2=0x7fce1c0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 --2- 172.21.15.73:0/152358166 >> v2:172.21.15.157:3300/0 conn(0x7fce1c06e920 0x7fce1c0cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c070760 con 0x7fce1c06ede0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fce04001670 con 0x7fce1c06ede0 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 >> v1:172.21.15.73:6789/0 conn(0x7fce1c06ede0 legacy=0x7fce1c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce00003970 0x7fce00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce000047c0 0x7fce00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c15d500 con 0x7fce000047c0 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c06b8e0 con 0x7fce00003970 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.057+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0000ba30 con 0x7fce1c15d120 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce000047c0 0x7fce00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 -- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 msgr2=0x7fce00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce1a800640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce1c15d120 0x7fce00005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 -- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce00003970 msgr2=0x7fce00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce00003970 0x7fce00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.058+0000 7fce19400640 1 -- 172.21.15.73:0/152358166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00003520 con 0x7fce000047c0 2024-04-05T22:22:01.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.059+0000 7fce19400640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce000047c0 0x7fce00004c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c210 tx=0x7fce000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185eadbffe47d426 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.059+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04005ad0 con 0x7fce000047c0 2024-04-05T22:22:01.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.059+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04005c30 con 0x7fce000047c0 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.059+0000 7fce18a00640 1 -- 172.21.15.73:0/152358166 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0402c8a0 con 0x7fce000047c0 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 -- 172.21.15.73:0/152358166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce000047c0 msgr2=0x7fce00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 --2- 172.21.15.73:0/152358166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce000047c0 0x7fce00004c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c210 tx=0x7fce000057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce1a800640 1 -- 172.21.15.73:0/152358166 reap_dead start 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 -- 172.21.15.73:0/152358166 shutdown_connections 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 -- 172.21.15.73:0/152358166 >> 172.21.15.73:0/152358166 conn(0x7fce1c06adc0 msgr2=0x7fce1c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 -- 172.21.15.73:0/152358166 shutdown_connections 2024-04-05T22:22:01.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.060+0000 7fce20e00640 1 -- 172.21.15.73:0/152358166 wait complete. 2024-04-05T22:22:01.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.062+0000 7fce20e00640 1 Processor -- start 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.062+0000 7fce20e00640 1 -- start start 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce000047c0 0x7fce1c15fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce1c14a5d0 0x7fce1c1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c14aa50 con 0x7fce1c15d640 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c153fa0 con 0x7fce1c14a5d0 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce20e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c1f39e0 con 0x7fce000047c0 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44226/0 (socket says 172.21.15.73:44226) 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 -- 172.21.15.73:0/1284761545 learned_addr learned my addr 172.21.15.73:0/1284761545 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce19e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce000047c0 0x7fce1c15fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 -- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce000047c0 msgr2=0x7fce1c15fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce000047c0 0x7fce1c15fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 -- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce1c14a5d0 msgr2=0x7fce1c1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce1c14a5d0 0x7fce1c1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.063+0000 7fce1a800640 1 -- 172.21.15.73:0/1284761545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c160340 con 0x7fce1c15d640 2024-04-05T22:22:01.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce1a800640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0af0e0 tx=0x7fce1400e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400c070 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce14004020 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140043d0 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c1541f0 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.064+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c1547b0 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.065+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c066af0 con 0x7fce1c15d640 2024-04-05T22:22:01.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.066+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fce14003610 con 0x7fce1c15d640 2024-04-05T22:22:01.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.067+0000 7fce12000640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 0x7fcdf806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.067+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fce14088b50 con 0x7fce1c15d640 2024-04-05T22:22:01.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.067+0000 7fce19e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 0x7fcdf806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.068+0000 7fce19e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 0x7fcdf806d0f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce0c004af0 tx=0x7fce0c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.071+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fce14053c90 con 0x7fce1c15d640 2024-04-05T22:22:01.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.275+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce1c068160 con 0x7fce1c15d640 2024-04-05T22:22:01.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.276+0000 7fce12000640 1 -- 172.21.15.73:0/1284761545 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce14006d10 con 0x7fce1c15d640 2024-04-05T22:22:01.275 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:22:01.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 msgr2=0x7fcdf806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 0x7fcdf806d0f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce0c004af0 tx=0x7fce0c00b040 comp rx=0 tx=0).stop 2024-04-05T22:22:01.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 msgr2=0x7fce1c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0af0e0 tx=0x7fce1400e8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 shutdown_connections 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcdf806ac30 0x7fcdf806d0f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1c15d640 0x7fce1c15daf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce1c14a5d0 0x7fce1c1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 --2- 172.21.15.73:0/1284761545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce000047c0 0x7fce1c15fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 >> 172.21.15.73:0/1284761545 conn(0x7fce1c06adc0 msgr2=0x7fce1c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 shutdown_connections 2024-04-05T22:22:01.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.279+0000 7fce20e00640 1 -- 172.21.15.73:0/1284761545 wait complete. 2024-04-05T22:22:01.297 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:22:01.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:01.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-05T22:22:01.403 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:22:01.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:01.403 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:22:01.463 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:01.463 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:01.521 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:01.522 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:01.571 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:01.571 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:01.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:01.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.870+0000 7fbfffa00640 1 Processor -- start 2024-04-05T22:22:01.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.870+0000 7fbfffa00640 1 -- start start 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.871+0000 7fbfffa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbff815ef60 0x7fbff806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.871+0000 7fbfffa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff806f5b0 con 0x7fbff81587d0 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.871+0000 7fbfffa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff8070410 con 0x7fbff806cb80 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.871+0000 7fbfffa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff806d500 con 0x7fbff815ef60 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbff815ef60 0x7fbff806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbff815ef60 0x7fbff806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44228/0 (socket says 172.21.15.73:44228) 2024-04-05T22:22:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 -- 172.21.15.73:0/347097169 learned_addr learned my addr 172.21.15.73:0/347097169 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffd000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbff81587d0 0x7fbff80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34458/0 (socket says 172.21.15.73:34458) 2024-04-05T22:22:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 -- 172.21.15.73:0/347097169 >> v1:172.21.15.157:6789/0 conn(0x7fbff806cb80 legacy=0x7fbff815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 -- 172.21.15.73:0/347097169 >> v1:172.21.15.73:6789/0 conn(0x7fbff81587d0 legacy=0x7fbff80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.872+0000 7fbffda00640 1 -- 172.21.15.73:0/347097169 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80ccef0 con 0x7fbff815ef60 2024-04-05T22:22:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.873+0000 7fbffda00640 1 --2- 172.21.15.73:0/347097169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff815ef60 0x7fbff806cfc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e50 tx=0x7fbff002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c32aa4913e4631 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbff7400640 1 -- 172.21.15.73:0/347097169 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff002cb20 con 0x7fbff815ef60 2024-04-05T22:22:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbff7400640 1 -- 172.21.15.73:0/347097169 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff002cc80 con 0x7fbff815ef60 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbff7400640 1 -- 172.21.15.73:0/347097169 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00343f0 con 0x7fbff815ef60 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 -- 172.21.15.73:0/347097169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff815ef60 msgr2=0x7fbff806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 --2- 172.21.15.73:0/347097169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff815ef60 0x7fbff806cfc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e50 tx=0x7fbff002bf80 comp rx=0 tx=0).stop 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 -- 172.21.15.73:0/347097169 shutdown_connections 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 --2- 172.21.15.73:0/347097169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff815ef60 0x7fbff806cfc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 -- 172.21.15.73:0/347097169 >> 172.21.15.73:0/347097169 conn(0x7fbff80ce8e0 msgr2=0x7fbff80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 -- 172.21.15.73:0/347097169 shutdown_connections 2024-04-05T22:22:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.874+0000 7fbfffa00640 1 -- 172.21.15.73:0/347097169 wait complete. 2024-04-05T22:22:01.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.876+0000 7fbfffa00640 1 Processor -- start 2024-04-05T22:22:01.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.876+0000 7fbfffa00640 1 -- start start 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff806cb80 0x7fbff8145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff815ef60 0x7fbff806a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff814c810 con 0x7fbff81587d0 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8148130 con 0x7fbff806cb80 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.877+0000 7fbfffa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8147090 con 0x7fbff815ef60 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44244/0 (socket says 172.21.15.73:44244) 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 -- 172.21.15.73:0/3086368418 learned_addr learned my addr 172.21.15.73:0/3086368418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 -- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff815ef60 msgr2=0x7fbff806a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff815ef60 0x7fbff806a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 -- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff806cb80 msgr2=0x7fbff8145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff806cb80 0x7fbff8145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 -- 172.21.15.73:0/3086368418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009b30 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.878+0000 7fbff7e00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c430 tx=0x7fbfe800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.879+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8011790 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.879+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe80118f0 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.879+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8011ce0 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.879+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff8147370 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.879+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff8147930 con 0x7fbff81587d0 2024-04-05T22:22:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.880+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff8066af0 con 0x7fbff81587d0 2024-04-05T22:22:01.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.881+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbfe801c460 con 0x7fbff81587d0 2024-04-05T22:22:01.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.882+0000 7fbff4c00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 0x7fbfdc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:01.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.882+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbfe8089310 con 0x7fbff81587d0 2024-04-05T22:22:01.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.882+0000 7fbffd000640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 0x7fbfdc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:01.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.883+0000 7fbffd000640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 0x7fbfdc06d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0015e0 tx=0x7fbfec0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:01.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:01.886+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbfe80544f0 con 0x7fbff81587d0 2024-04-05T22:22:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.085+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbff8068160 con 0x7fbff81587d0 2024-04-05T22:22:02.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.087+0000 7fbff4c00640 1 -- 172.21.15.73:0/3086368418 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fbfe8057b10 con 0x7fbff81587d0 2024-04-05T22:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:01.664332+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 msgr2=0x7fbfdc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 0x7fbfdc06d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0015e0 tx=0x7fbfec0079c0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 msgr2=0x7fbff8069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c430 tx=0x7fbfe800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 shutdown_connections 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbfdc06ac00 0x7fbfdc06d0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.089+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff815ef60 0x7fbff806a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.090+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbff81587d0 0x7fbff8069cb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.090+0000 7fbfffa00640 1 --2- 172.21.15.73:0/3086368418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff806cb80 0x7fbff8145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.090+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 >> 172.21.15.73:0/3086368418 conn(0x7fbff80ce8e0 msgr2=0x7fbff81590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:02.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.090+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 shutdown_connections 2024-04-05T22:22:02.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.090+0000 7fbfffa00640 1 -- 172.21.15.73:0/3086368418 wait complete. 2024-04-05T22:22:02.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:02.108 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.319+0000 7fb830600640 1 Processor -- start 2024-04-05T22:22:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.319+0000 7fb830600640 1 -- start start 2024-04-05T22:22:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.320+0000 7fb830600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb82806c9a0 0x7fb82806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.320+0000 7fb830600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.320+0000 7fb830600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82806bea0 con 0x7fb82815abe0 2024-04-05T22:22:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb830600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82815afc0 con 0x7fb82806c9a0 2024-04-05T22:22:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb830600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82806f5b0 con 0x7fb82806ba70 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb82e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb82e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:35170/0 (socket says 172.21.15.73:35170) 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb82e600640 1 -- 172.21.15.73:0/285195825 learned_addr learned my addr 172.21.15.73:0/285195825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.321+0000 7fb82d200640 1 --2- 172.21.15.73:0/285195825 >> v2:172.21.15.157:3300/0 conn(0x7fb82806c9a0 0x7fb82806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.322+0000 7fb82e600640 1 -- 172.21.15.73:0/285195825 >> v2:172.21.15.157:3300/0 conn(0x7fb82806c9a0 msgr2=0x7fb82806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.322+0000 7fb82e600640 1 --2- 172.21.15.73:0/285195825 >> v2:172.21.15.157:3300/0 conn(0x7fb82806c9a0 0x7fb82806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.322+0000 7fb82e600640 1 -- 172.21.15.73:0/285195825 >> v1:172.21.15.157:6789/0 conn(0x7fb82815abe0 legacy=0x7fb8280712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.322+0000 7fb82e600640 1 -- 172.21.15.73:0/285195825 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82806f890 con 0x7fb82806ba70 2024-04-05T22:22:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.323+0000 7fb82e600640 1 --2- 172.21.15.73:0/285195825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb82002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0b3cbb6f53e12b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.324+0000 7fb827e00640 1 -- 172.21.15.73:0/285195825 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82002c9f0 con 0x7fb82806ba70 2024-04-05T22:22:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.324+0000 7fb827e00640 1 -- 172.21.15.73:0/285195825 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82002cb50 con 0x7fb82806ba70 2024-04-05T22:22:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.324+0000 7fb827e00640 1 -- 172.21.15.73:0/285195825 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200343f0 con 0x7fb82806ba70 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 -- 172.21.15.73:0/285195825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 msgr2=0x7fb828164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 --2- 172.21.15.73:0/285195825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb82002be50 comp rx=0 tx=0).stop 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 -- 172.21.15.73:0/285195825 shutdown_connections 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 --2- 172.21.15.73:0/285195825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806ba70 0x7fb828164740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 --2- 172.21.15.73:0/285195825 >> v2:172.21.15.157:3300/0 conn(0x7fb82806c9a0 0x7fb82806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 -- 172.21.15.73:0/285195825 >> 172.21.15.73:0/285195825 conn(0x7fb828069cb0 msgr2=0x7fb82806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:02.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 -- 172.21.15.73:0/285195825 shutdown_connections 2024-04-05T22:22:02.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.325+0000 7fb830600640 1 -- 172.21.15.73:0/285195825 wait complete. 2024-04-05T22:22:02.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.327+0000 7fb830600640 1 Processor -- start 2024-04-05T22:22:02.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.327+0000 7fb830600640 1 -- start start 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82806ba70 0x7fb828148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806c9a0 0x7fb82815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814f9b0 con 0x7fb82815abe0 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814fdd0 con 0x7fb82806ba70 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.328+0000 7fb830600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb828149a90 con 0x7fb82806c9a0 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44250/0 (socket says 172.21.15.73:44250) 2024-04-05T22:22:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 -- 172.21.15.73:0/1415897833 learned_addr learned my addr 172.21.15.73:0/1415897833 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82dc00640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82806ba70 0x7fb828148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82d200640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806c9a0 0x7fb82815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 -- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806c9a0 msgr2=0x7fb82815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806c9a0 0x7fb82815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 -- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82806ba70 msgr2=0x7fb828148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82806ba70 0x7fb828148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.329+0000 7fb82e600640 1 -- 172.21.15.73:0/1415897833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009b30 con 0x7fb82815abe0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb82e600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb820004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82003e450 con 0x7fb82815abe0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82003e5b0 con 0x7fb82815abe0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82814ed30 con 0x7fb82815abe0 2024-04-05T22:22:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82003e960 con 0x7fb82815abe0 2024-04-05T22:22:02.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.330+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82814f2f0 con 0x7fb82815abe0 2024-04-05T22:22:02.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.331+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82814faf0 con 0x7fb82815abe0 2024-04-05T22:22:02.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.332+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb82003eac0 con 0x7fb82815abe0 2024-04-05T22:22:02.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.333+0000 7fb825600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 0x7fb80806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.333+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb8200abfa0 con 0x7fb82815abe0 2024-04-05T22:22:02.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.333+0000 7fb82dc00640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 0x7fb80806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.334+0000 7fb82dc00640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 0x7fb80806d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8280b3180 tx=0x7fb81c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.337+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb8200df650 con 0x7fb82815abe0 2024-04-05T22:22:02.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.534+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8280cdc40 con 0x7fb82815abe0 2024-04-05T22:22:02.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.535+0000 7fb825600640 1 -- 172.21.15.73:0/1415897833 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb82814faf0 con 0x7fb82815abe0 2024-04-05T22:22:02.535 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:02.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 msgr2=0x7fb80806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 0x7fb80806d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8280b3180 tx=0x7fb81c009040 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 msgr2=0x7fb82815d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb820004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 shutdown_connections 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb80806ac00 0x7fb80806d0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb82815abe0 0x7fb82815d290 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb82806c9a0 0x7fb82815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 --2- 172.21.15.73:0/1415897833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82806ba70 0x7fb828148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 >> 172.21.15.73:0/1415897833 conn(0x7fb828069cb0 msgr2=0x7fb82806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 shutdown_connections 2024-04-05T22:22:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.538+0000 7fb830600640 1 -- 172.21.15.73:0/1415897833 wait complete. 2024-04-05T22:22:02.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:02.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.774+0000 7f2613200640 1 Processor -- start 2024-04-05T22:22:02.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.774+0000 7f2613200640 1 -- start start 2024-04-05T22:22:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.776+0000 7f2613200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f260c15abe0 0x7f260c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.776+0000 7f2613200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.776+0000 7f2613200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c15afc0 con 0x7f260c06c9a0 2024-04-05T22:22:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.776+0000 7f2613200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c06f5b0 con 0x7f260c15abe0 2024-04-05T22:22:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.776+0000 7f2613200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c06f890 con 0x7f260c06bc40 2024-04-05T22:22:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:35192/0 (socket says 172.21.15.73:35192) 2024-04-05T22:22:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 -- 172.21.15.73:0/3567718865 learned_addr learned my addr 172.21.15.73:0/3567718865 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f260be00640 1 --2- 172.21.15.73:0/3567718865 >> v2:172.21.15.157:3300/0 conn(0x7f260c15abe0 0x7f260c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 -- 172.21.15.73:0/3567718865 >> v2:172.21.15.157:3300/0 conn(0x7f260c15abe0 msgr2=0x7f260c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:02.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 --2- 172.21.15.73:0/3567718865 >> v2:172.21.15.157:3300/0 conn(0x7f260c15abe0 0x7f260c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.777+0000 7f2611200640 1 -- 172.21.15.73:0/3567718865 >> v1:172.21.15.157:6789/0 conn(0x7f260c06c9a0 legacy=0x7f260c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.778+0000 7f2611200640 1 -- 172.21.15.73:0/3567718865 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c06fb70 con 0x7f260c06bc40 2024-04-05T22:22:02.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.779+0000 7f2611200640 1 --2- 172.21.15.73:0/3567718865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2604009e50 tx=0x7f260402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2412f45e20c46dd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f260aa00640 1 -- 172.21.15.73:0/3567718865 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260402c9e0 con 0x7f260c06bc40 2024-04-05T22:22:02.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f260aa00640 1 -- 172.21.15.73:0/3567718865 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260402cb40 con 0x7f260c06bc40 2024-04-05T22:22:02.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f260aa00640 1 -- 172.21.15.73:0/3567718865 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26040343f0 con 0x7f260c06bc40 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 -- 172.21.15.73:0/3567718865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 msgr2=0x7f260c164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 --2- 172.21.15.73:0/3567718865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2604009e50 tx=0x7f260402be40 comp rx=0 tx=0).stop 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 -- 172.21.15.73:0/3567718865 shutdown_connections 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 --2- 172.21.15.73:0/3567718865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06bc40 0x7f260c164740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 --2- 172.21.15.73:0/3567718865 >> v2:172.21.15.157:3300/0 conn(0x7f260c15abe0 0x7f260c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 -- 172.21.15.73:0/3567718865 >> 172.21.15.73:0/3567718865 conn(0x7f260c069cb0 msgr2=0x7f260c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 -- 172.21.15.73:0/3567718865 shutdown_connections 2024-04-05T22:22:02.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.780+0000 7f2613200640 1 -- 172.21.15.73:0/3567718865 wait complete. 2024-04-05T22:22:02.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.782+0000 7f2613200640 1 Processor -- start 2024-04-05T22:22:02.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.782+0000 7f2613200640 1 -- start start 2024-04-05T22:22:02.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06c9a0 0x7f260c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f260c15abe0 0x7f260c14ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c14cdf0 con 0x7f260c06bc40 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c14e410 con 0x7f260c15abe0 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f2613200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c14d110 con 0x7f260c06c9a0 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f260be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.783+0000 7f260be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44260/0 (socket says 172.21.15.73:44260) 2024-04-05T22:22:02.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 -- 172.21.15.73:0/2577809312 learned_addr learned my addr 172.21.15.73:0/2577809312 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:02.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f2611200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f260c15abe0 0x7f260c14ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260b400640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06c9a0 0x7f260c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 -- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06c9a0 msgr2=0x7f260c0af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06c9a0 0x7f260c0af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 -- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f260c15abe0 msgr2=0x7f260c14ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f260c15abe0 0x7f260c14ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 -- 172.21.15.73:0/2577809312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604009b30 con 0x7f260c06bc40 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.784+0000 7f260be00640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9b0 tx=0x7f25fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.785+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009670 con 0x7f260c06bc40 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.785+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00c9a0 con 0x7f260c06bc40 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.785+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00cd50 con 0x7f260c06bc40 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.785+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c14d3f0 con 0x7f260c06bc40 2024-04-05T22:22:02.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.785+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c14d9b0 con 0x7f260c06bc40 2024-04-05T22:22:02.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.786+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c068160 con 0x7f260c06bc40 2024-04-05T22:22:02.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.787+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f25fc013470 con 0x7f260c06bc40 2024-04-05T22:22:02.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.787+0000 7f2603e00640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 0x7f25ec06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:02.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.788+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f25fc0895e0 con 0x7f260c06bc40 2024-04-05T22:22:02.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.788+0000 7f260b400640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 0x7f25ec06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:02.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.789+0000 7f260b400640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 0x7f25ec06d0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f260c1513a0 tx=0x7f25f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.792+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f25fc0547c0 con 0x7f260c06bc40 2024-04-05T22:22:02.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.990+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f260c0aea40 con 0x7f260c06bc40 2024-04-05T22:22:02.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.991+0000 7f2603e00640 1 -- 172.21.15.73:0/2577809312 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25fc057de0 con 0x7f260c06bc40 2024-04-05T22:22:02.990 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.993+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 msgr2=0x7f25ec06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.993+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 0x7f25ec06d0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f260c1513a0 tx=0x7f25f8009040 comp rx=0 tx=0).stop 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 msgr2=0x7f260c14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9b0 tx=0x7f25fc00ee70 comp rx=0 tx=0).stop 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 shutdown_connections 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25ec06ac30 0x7f25ec06d0f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f260c15abe0 0x7f260c14ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f260c06c9a0 0x7f260c0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 --2- 172.21.15.73:0/2577809312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260c06bc40 0x7f260c14ded0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:02.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 >> 172.21.15.73:0/2577809312 conn(0x7f260c069cb0 msgr2=0x7f260c0ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:02.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 shutdown_connections 2024-04-05T22:22:02.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:02.994+0000 7f2613200640 1 -- 172.21.15.73:0/2577809312 wait complete. 2024-04-05T22:22:03.012 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:03.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.221+0000 7f92aee00640 1 Processor -- start 2024-04-05T22:22:03.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.221+0000 7f92aee00640 1 -- start start 2024-04-05T22:22:03.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f92aee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92a80cbe00 0x7f92a80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f92aee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92a80b32b0 0x7f92a81647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f92aee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a80b1b30 con 0x7f92a806c630 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f92aee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a80b1e10 con 0x7f92a80cbe00 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f92aee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a80cc300 con 0x7f92a80b32b0 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92a80cbe00 0x7f92a80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92a80cbe00 0x7f92a80ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44268/0 (socket says 172.21.15.73:44268) 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 -- 172.21.15.73:0/3944546506 learned_addr learned my addr 172.21.15.73:0/3944546506 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 -- 172.21.15.73:0/3944546506 >> v2:172.21.15.157:3300/0 conn(0x7f92a80b32b0 msgr2=0x7f92a81647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 --2- 172.21.15.73:0/3944546506 >> v2:172.21.15.157:3300/0 conn(0x7f92a80b32b0 0x7f92a81647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 -- 172.21.15.73:0/3944546506 >> v1:172.21.15.157:6789/0 conn(0x7f92a806c630 legacy=0x7f92a806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:03.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.223+0000 7f929fe00640 1 -- 172.21.15.73:0/3944546506 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80cc5e0 con 0x7f92a80cbe00 2024-04-05T22:22:03.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.224+0000 7f929fe00640 1 --2- 172.21.15.73:0/3944546506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a80ce0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92a8144920 tx=0x7f929002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf611b6343506d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92a7400640 1 -- 172.21.15.73:0/3944546506 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290031870 con 0x7f92a80cbe00 2024-04-05T22:22:03.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92a7400640 1 -- 172.21.15.73:0/3944546506 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92900319d0 con 0x7f92a80cbe00 2024-04-05T22:22:03.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92a7400640 1 -- 172.21.15.73:0/3944546506 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290031d60 con 0x7f92a80cbe00 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 -- 172.21.15.73:0/3944546506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 msgr2=0x7f92a80ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 --2- 172.21.15.73:0/3944546506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a80ce0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92a8144920 tx=0x7f929002b910 comp rx=0 tx=0).stop 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 -- 172.21.15.73:0/3944546506 shutdown_connections 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 --2- 172.21.15.73:0/3944546506 >> v2:172.21.15.157:3300/0 conn(0x7f92a80b32b0 0x7f92a81647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 --2- 172.21.15.73:0/3944546506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a80ce0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 -- 172.21.15.73:0/3944546506 >> 172.21.15.73:0/3944546506 conn(0x7f92a80b0830 msgr2=0x7f92a80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 -- 172.21.15.73:0/3944546506 shutdown_connections 2024-04-05T22:22:03.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.225+0000 7f92aee00640 1 -- 172.21.15.73:0/3944546506 wait complete. 2024-04-05T22:22:03.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.227+0000 7f92aee00640 1 Processor -- start 2024-04-05T22:22:03.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.227+0000 7f92aee00640 1 -- start start 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92a806c630 0x7f92a813a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 0x7f92a813c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a813cad0 con 0x7f92a80cbe00 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a815ca70 con 0x7f92a80b32b0 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92aee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a815cd50 con 0x7f92a806c630 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92ace00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92ace00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44270/0 (socket says 172.21.15.73:44270) 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.228+0000 7f92ace00640 1 -- 172.21.15.73:0/3247336088 learned_addr learned my addr 172.21.15.73:0/3247336088 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f929fe00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 0x7f92a813c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92a7e00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92a806c630 0x7f92a813a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 -- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92a806c630 msgr2=0x7f92a813a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92a806c630 0x7f92a813a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 -- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 msgr2=0x7f92a813c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 0x7f92a813c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 -- 172.21.15.73:0/3247336088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92900094d0 con 0x7f92a80cbe00 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f929fe00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 0x7f92a813c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:03.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92ace00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b9e0 tx=0x7f92a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000cb20 con 0x7f92a80cbe00 2024-04-05T22:22:03.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a000cc80 con 0x7f92a80cbe00 2024-04-05T22:22:03.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0014680 con 0x7f92a80cbe00 2024-04-05T22:22:03.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.229+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a815d030 con 0x7f92a80cbe00 2024-04-05T22:22:03.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.230+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a813fcc0 con 0x7f92a80cbe00 2024-04-05T22:22:03.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.231+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a8068160 con 0x7f92a80cbe00 2024-04-05T22:22:03.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.231+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f92a001d3b0 con 0x7f92a80cbe00 2024-04-05T22:22:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.233+0000 7f92a4c00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 0x7f928406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.233+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f92a00893d0 con 0x7f92a80cbe00 2024-04-05T22:22:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.233+0000 7f92a7e00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 0x7f928406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.234+0000 7f92a7e00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 0x7f928406d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92980015d0 tx=0x7f9298009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.237+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f92a0054480 con 0x7f92a80cbe00 2024-04-05T22:22:03.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.431+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92a813feb0 con 0x7f92a80cbe00 2024-04-05T22:22:03.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.432+0000 7f92a4c00640 1 -- 172.21.15.73:0/3247336088 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92a0057aa0 con 0x7f92a80cbe00 2024-04-05T22:22:03.431 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 msgr2=0x7f928406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 0x7f928406d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92980015d0 tx=0x7f9298009040 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 msgr2=0x7f92a815c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b9e0 tx=0x7f92a000bea0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 shutdown_connections 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f928406ac00 0x7f928406d0c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a80cbe00 0x7f92a815c170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a80b32b0 0x7f92a813c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 --2- 172.21.15.73:0/3247336088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92a806c630 0x7f92a813a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.434+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 >> 172.21.15.73:0/3247336088 conn(0x7f92a80b0830 msgr2=0x7f92a815b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:03.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.435+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 shutdown_connections 2024-04-05T22:22:03.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.435+0000 7f92aee00640 1 -- 172.21.15.73:0/3247336088 wait complete. 2024-04-05T22:22:03.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:22:03.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.650+0000 7f5111400640 1 Processor -- start 2024-04-05T22:22:03.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.651+0000 7f5111400640 1 -- start start 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f5111400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 0x7f510c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f5111400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f510c15e850 0x7f510c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f5111400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c06f5b0 con 0x7f510c1587d0 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f5111400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c06d500 con 0x7f510c15e850 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f5111400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c15f220 con 0x7f510c06cb80 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.652+0000 7f510a800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f510c1587d0 0x7f510c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34472/0 (socket says 172.21.15.73:34472) 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510a800640 1 -- 172.21.15.73:0/780245654 learned_addr learned my addr 172.21.15.73:0/780245654 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510b200640 1 --2- 172.21.15.73:0/780245654 >> v2:172.21.15.73:3300/0 conn(0x7f510c15e850 0x7f510c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f5109e00640 1 --2- 172.21.15.73:0/780245654 >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 0x7f510c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f5109400640 1 -- 172.21.15.73:0/780245654 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169799042 0 0) 0x7f510c06f5b0 con 0x7f510c1587d0 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f5109400640 1 -- 172.21.15.73:0/780245654 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f0003520 con 0x7f510c1587d0 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510b200640 1 -- 172.21.15.73:0/780245654 >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 msgr2=0x7f510c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510b200640 1 --2- 172.21.15.73:0/780245654 >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 0x7f510c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510b200640 1 -- 172.21.15.73:0/780245654 >> v1:172.21.15.73:6789/0 conn(0x7f510c1587d0 legacy=0x7f510c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f510b200640 1 -- 172.21.15.73:0/780245654 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c070410 con 0x7f510c15e850 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.653+0000 7f5109e00640 1 --2- 172.21.15.73:0/780245654 >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 0x7f510c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.654+0000 7f510b200640 1 --2- 172.21.15.73:0/780245654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c15e850 0x7f510c15ece0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f510c06b560 tx=0x7f510402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ddcd1ec651386 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.654+0000 7f5109400640 1 -- 172.21.15.73:0/780245654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510402cc70 con 0x7f510c15e850 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.654+0000 7f5109400640 1 -- 172.21.15.73:0/780245654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510402cdd0 con 0x7f510c15e850 2024-04-05T22:22:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.654+0000 7f5109400640 1 -- 172.21.15.73:0/780245654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104034660 con 0x7f510c15e850 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 -- 172.21.15.73:0/780245654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c15e850 msgr2=0x7f510c15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 --2- 172.21.15.73:0/780245654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c15e850 0x7f510c15ece0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f510c06b560 tx=0x7f510402bec0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 -- 172.21.15.73:0/780245654 shutdown_connections 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 --2- 172.21.15.73:0/780245654 >> v2:172.21.15.157:3300/0 conn(0x7f510c06cb80 0x7f510c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 --2- 172.21.15.73:0/780245654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c15e850 0x7f510c15ece0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 -- 172.21.15.73:0/780245654 >> 172.21.15.73:0/780245654 conn(0x7f510c0ce8e0 msgr2=0x7f510c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 -- 172.21.15.73:0/780245654 shutdown_connections 2024-04-05T22:22:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.655+0000 7f5111400640 1 -- 172.21.15.73:0/780245654 wait complete. 2024-04-05T22:22:03.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.657+0000 7f5111400640 1 Processor -- start 2024-04-05T22:22:03.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.657+0000 7f5111400640 1 -- start start 2024-04-05T22:22:03.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510c1587d0 0x7f510c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f510c15e850 0x7f510c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b31b0 con 0x7f510c06cb80 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0c4290 con 0x7f510c1587d0 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f5111400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0c4570 con 0x7f510c15e850 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f510a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f510a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44286/0 (socket says 172.21.15.73:44286) 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.658+0000 7f510a800640 1 -- 172.21.15.73:0/8413432 learned_addr learned my addr 172.21.15.73:0/8413432 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5109e00640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510c1587d0 0x7f510c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 -- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f510c15e850 msgr2=0x7f510c069f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f510c15e850 0x7f510c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 -- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510c1587d0 msgr2=0x7f510c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510c1587d0 0x7f510c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 -- 172.21.15.73:0/8413432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104009a70 con 0x7f510c06cb80 2024-04-05T22:22:03.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f510a800640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c500 tx=0x7f50fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc005ce0 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc005e40 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c0c4850 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc007330 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.659+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c06a460 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.661+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c06a650 con 0x7f510c06cb80 2024-04-05T22:22:03.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.661+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f50fc007490 con 0x7f510c06cb80 2024-04-05T22:22:03.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.662+0000 7f5102a00640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 0x7f50ec06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:03.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.662+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f50fc088a80 con 0x7f510c06cb80 2024-04-05T22:22:03.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.662+0000 7f5109e00640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 0x7f50ec06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:03.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.663+0000 7f5109e00640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 0x7f50ec06d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f510c1f5540 tx=0x7f50f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.667+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f50fc053c80 con 0x7f510c06cb80 2024-04-05T22:22:03.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.873+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f510c066af0 con 0x7f510c06cb80 2024-04-05T22:22:03.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.874+0000 7f5102a00640 1 -- 172.21.15.73:0/8413432 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f50fc017020 con 0x7f510c06cb80 2024-04-05T22:22:03.873 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:22:03.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 msgr2=0x7f50ec06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 0x7f50ec06d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f510c1f5540 tx=0x7f50f8008040 comp rx=0 tx=0).stop 2024-04-05T22:22:03.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 msgr2=0x7f510c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c500 tx=0x7f50fc00c9c0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 shutdown_connections 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f50ec06ac30 0x7f50ec06d0f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f510c15e850 0x7f510c069f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510c1587d0 0x7f510c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 --2- 172.21.15.73:0/8413432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f510c06cb80 0x7f510c0b3640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 >> 172.21.15.73:0/8413432 conn(0x7f510c0ce8e0 msgr2=0x7f510c1574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 shutdown_connections 2024-04-05T22:22:03.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:03.877+0000 7f5111400640 1 -- 172.21.15.73:0/8413432 wait complete. 2024-04-05T22:22:03.894 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-05T22:22:03.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:03.895 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-05T22:22:03.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:22:03.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:03.993 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-05T22:22:04.044 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:04.044 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:04.101 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:04.101 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:04.153 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:04.153 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:04.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:04.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.420+0000 7f4c68800640 1 Processor -- start 2024-04-05T22:22:04.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.421+0000 7f4c68800640 1 -- start start 2024-04-05T22:22:04.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.422+0000 7f4c68800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 0x7f4c6006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.422+0000 7f4c68800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c600cd7a0 0x7f4c600d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.422+0000 7f4c68800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600cb950 con 0x7f4c600d09b0 2024-04-05T22:22:04.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.422+0000 7f4c68800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600ce990 con 0x7f4c600cd7a0 2024-04-05T22:22:04.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.422+0000 7f4c68800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600d1350 con 0x7f4c6015cf50 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c600cd7a0 0x7f4c600d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c600cd7a0 0x7f4c600d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44302/0 (socket says 172.21.15.73:44302) 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 -- 172.21.15.73:0/3750829155 learned_addr learned my addr 172.21.15.73:0/3750829155 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c65e00640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 0x7f4c6006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 -- 172.21.15.73:0/3750829155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 msgr2=0x7f4c6006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 0x7f4c6006ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 -- 172.21.15.73:0/3750829155 >> v1:172.21.15.157:6789/0 conn(0x7f4c600d09b0 legacy=0x7f4c600cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c66800640 1 -- 172.21.15.73:0/3750829155 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600cdce0 con 0x7f4c600cd7a0 2024-04-05T22:22:04.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.423+0000 7f4c65e00640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 0x7f4c6006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:04.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.424+0000 7f4c66800640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c600cd7a0 0x7f4c600d0e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009870 tx=0x7f4c5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242144baf7ca376a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.424+0000 7f4c64a00640 1 -- 172.21.15.73:0/3750829155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c031640 con 0x7f4c600cd7a0 2024-04-05T22:22:04.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c64a00640 1 -- 172.21.15.73:0/3750829155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0317a0 con 0x7f4c600cd7a0 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c64a00640 1 -- 172.21.15.73:0/3750829155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c031b30 con 0x7f4c600cd7a0 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 -- 172.21.15.73:0/3750829155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c600cd7a0 msgr2=0x7f4c600d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c600cd7a0 0x7f4c600d0e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009870 tx=0x7f4c5c02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 -- 172.21.15.73:0/3750829155 shutdown_connections 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c600cd7a0 0x7f4c600d0e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 --2- 172.21.15.73:0/3750829155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c6015cf50 0x7f4c6006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 -- 172.21.15.73:0/3750829155 >> 172.21.15.73:0/3750829155 conn(0x7f4c60099a60 msgr2=0x7f4c60099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 -- 172.21.15.73:0/3750829155 shutdown_connections 2024-04-05T22:22:04.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.425+0000 7f4c68800640 1 -- 172.21.15.73:0/3750829155 wait complete. 2024-04-05T22:22:04.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.427+0000 7f4c68800640 1 Processor -- start 2024-04-05T22:22:04.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.427+0000 7f4c68800640 1 -- start start 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c600cd7a0 0x7f4c601f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c600d09b0 0x7f4c600c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600c65f0 con 0x7f4c6015cf50 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b0390 con 0x7f4c600cd7a0 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c68800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b0600 con 0x7f4c600d09b0 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c66800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c66800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44318/0 (socket says 172.21.15.73:44318) 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.428+0000 7f4c66800640 1 -- 172.21.15.73:0/1259925852 learned_addr learned my addr 172.21.15.73:0/1259925852 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:04.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c65e00640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c600cd7a0 0x7f4c601f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 -- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c600d09b0 msgr2=0x7f4c600c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c600d09b0 0x7f4c600c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 -- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c600cd7a0 msgr2=0x7f4c601f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c600cd7a0 0x7f4c601f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 -- 172.21.15.73:0/1259925852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0094d0 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c66800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c02b930 tx=0x7f4c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c03ac60 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c03adc0 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c044630 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.429+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c600b08e0 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.430+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c600b0ea0 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.431+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c600b0740 con 0x7f4c6015cf50 2024-04-05T22:22:04.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.432+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4c5c02f050 con 0x7f4c6015cf50 2024-04-05T22:22:04.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.433+0000 7f4c56000640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 0x7f4c4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.433+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4c5c0ab410 con 0x7f4c6015cf50 2024-04-05T22:22:04.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.433+0000 7f4c65e00640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 0x7f4c4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.434+0000 7f4c65e00640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 0x7f4c4406d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c58007510 tx=0x7f4c580074a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.437+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4c600b0740 con 0x7f4c6015cf50 2024-04-05T22:22:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.631+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c600b0740 con 0x7f4c6015cf50 2024-04-05T22:22:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.633+0000 7f4c56000640 1 -- 172.21.15.73:0/1259925852 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16277 (secure 0 0 0) 0x7f4c600b0740 con 0x7f4c6015cf50 2024-04-05T22:22:04.632 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:04.633 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:01.664332+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 msgr2=0x7f4c4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 0x7f4c4406d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c58007510 tx=0x7f4c580074a0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 msgr2=0x7f4c600c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c02b930 tx=0x7f4c5c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 shutdown_connections 2024-04-05T22:22:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c4406ac00 0x7f4c4406d0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c6015cf50 0x7f4c600c60b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c600d09b0 0x7f4c600c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 --2- 172.21.15.73:0/1259925852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c600cd7a0 0x7f4c601f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 >> 172.21.15.73:0/1259925852 conn(0x7f4c60099a60 msgr2=0x7f4c600adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 shutdown_connections 2024-04-05T22:22:04.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.636+0000 7f4c68800640 1 -- 172.21.15.73:0/1259925852 wait complete. 2024-04-05T22:22:04.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:04.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:04.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.856+0000 7f2f99400640 1 Processor -- start 2024-04-05T22:22:04.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.856+0000 7f2f99400640 1 -- start start 2024-04-05T22:22:04.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f99400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f9415dd90 0x7f2f94167d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f99400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9415e170 con 0x7f2f94071670 2024-04-05T22:22:04.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f99400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9415ae50 con 0x7f2f9415aa70 2024-04-05T22:22:04.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f99400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9415e3b0 con 0x7f2f9415dd90 2024-04-05T22:22:04.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f91e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2f94071670 0x7f2f9415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34488/0 (socket says 172.21.15.73:34488) 2024-04-05T22:22:04.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.858+0000 7f2f91e00640 1 -- 172.21.15.73:0/4112575498 learned_addr learned my addr 172.21.15.73:0/4112575498 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:04.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57126843 0 0) 0x7f2f7c001490 con 0x7f2f94071670 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f78003520 con 0x7f2f94071670 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f93200640 1 --2- 172.21.15.73:0/4112575498 >> v2:172.21.15.157:3300/0 conn(0x7f2f9415dd90 0x7f2f94167d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164547994 0 0) 0x7f2f78003520 con 0x7f2f94071670 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 >> v2:172.21.15.157:3300/0 conn(0x7f2f9415dd90 msgr2=0x7f2f94167d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 --2- 172.21.15.73:0/4112575498 >> v2:172.21.15.157:3300/0 conn(0x7f2f9415dd90 0x7f2f94167d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 >> v1:172.21.15.157:6789/0 conn(0x7f2f9415aa70 legacy=0x7f2f94070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9406ca50 con 0x7f2f94071670 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2f7c001ba0 con 0x7f2f94071670 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 >> v1:172.21.15.73:6789/0 conn(0x7f2f94071670 legacy=0x7f2f9415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f78003970 0x7f2f78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f780047c0 0x7f2f78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9415e3b0 con 0x7f2f78005180 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7800c0d0 con 0x7f2f78003970 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7800c290 con 0x7f2f780047c0 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.859+0000 7f2f93200640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f91e00640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f780047c0 0x7f2f78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f93200640 1 -- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f780047c0 msgr2=0x7f2f78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f93200640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f780047c0 0x7f2f78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f92800640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f78003970 0x7f2f78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f93200640 1 -- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f78003970 msgr2=0x7f2f78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f93200640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f78003970 0x7f2f78003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f93200640 1 -- 172.21.15.73:0/4112575498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78003520 con 0x7f2f78005180 2024-04-05T22:22:04.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.860+0000 7f2f91e00640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f780047c0 0x7f2f78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.861+0000 7f2f93200640 1 -- 172.21.15.73:0/4112575498 reap_dead start 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.861+0000 7f2f93200640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f7800b980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400ba30 tx=0x7f2f780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3aeb13d955ff13 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.861+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84004020 con 0x7f2f78005180 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.861+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84004180 con 0x7f2f78005180 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.861+0000 7f2f91400640 1 -- 172.21.15.73:0/4112575498 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84003610 con 0x7f2f78005180 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 -- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 msgr2=0x7f2f7800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f7800b980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400ba30 tx=0x7f2f780057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 -- 172.21.15.73:0/4112575498 shutdown_connections 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 --2- 172.21.15.73:0/4112575498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f7800b980 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 -- 172.21.15.73:0/4112575498 >> 172.21.15.73:0/4112575498 conn(0x7f2f94069cb0 msgr2=0x7f2f9406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:04.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 -- 172.21.15.73:0/4112575498 shutdown_connections 2024-04-05T22:22:04.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.862+0000 7f2f99400640 1 -- 172.21.15.73:0/4112575498 wait complete. 2024-04-05T22:22:04.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.864+0000 7f2f99400640 1 Processor -- start 2024-04-05T22:22:04.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.864+0000 7f2f99400640 1 -- start start 2024-04-05T22:22:04.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f94158da0 0x7f2f9415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f9415c570 0x7f2f940ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9415ca20 con 0x7f2f78005180 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940cb510 con 0x7f2f94158da0 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f99400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f941591e0 con 0x7f2f9415c570 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44326/0 (socket says 172.21.15.73:44326) 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 -- 172.21.15.73:0/4189468441 learned_addr learned my addr 172.21.15.73:0/4189468441 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 -- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f9415c570 msgr2=0x7f2f940ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f9415c570 0x7f2f940ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 -- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f94158da0 msgr2=0x7f2f9415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f94158da0 0x7f2f9415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.865+0000 7f2f92800640 1 -- 172.21.15.73:0/4189468441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f94157fe0 con 0x7f2f78005180 2024-04-05T22:22:04.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f92800640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d570 tx=0x7f2f8c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c003610 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c003770 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f941582c0 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c003b20 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.866+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f94158880 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.867+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f94158a10 con 0x7f2f78005180 2024-04-05T22:22:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.868+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f8c003c80 con 0x7f2f78005180 2024-04-05T22:22:04.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.869+0000 7f2f8aa00640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 0x7f2f7006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:04.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.869+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2f8c089360 con 0x7f2f78005180 2024-04-05T22:22:04.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.870+0000 7f2f91e00640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 0x7f2f7006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:04.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.870+0000 7f2f91e00640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 0x7f2f7006d0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007c40 tx=0x7f2f7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:04.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:04.874+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2f8c054560 con 0x7f2f78005180 2024-04-05T22:22:05.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.052+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f94068160 con 0x7f2f78005180 2024-04-05T22:22:05.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.053+0000 7f2f8aa00640 1 -- 172.21.15.73:0/4189468441 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f8c057b80 con 0x7f2f78005180 2024-04-05T22:22:05.052 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 msgr2=0x7f2f7006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 0x7f2f7006d0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007c40 tx=0x7f2f7c009040 comp rx=0 tx=0).stop 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 msgr2=0x7f2f94152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d570 tx=0x7f2f8c00da30 comp rx=0 tx=0).stop 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 shutdown_connections 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2f7006ac30 0x7f2f7006d0f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f9415c570 0x7f2f940ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f94158da0 0x7f2f9415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.055+0000 7f2f99400640 1 --2- 172.21.15.73:0/4189468441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f78005180 0x7f2f94152050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.056+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 >> 172.21.15.73:0/4189468441 conn(0x7f2f94069cb0 msgr2=0x7f2f941551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:05.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.056+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 shutdown_connections 2024-04-05T22:22:05.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.056+0000 7f2f99400640 1 -- 172.21.15.73:0/4189468441 wait complete. 2024-04-05T22:22:05.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.280+0000 7f3be2000640 1 Processor -- start 2024-04-05T22:22:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.281+0000 7f3be2000640 1 -- start start 2024-04-05T22:22:05.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3be2000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bdc15ee80 0x7f3bdc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3be2000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bdc0b1950 0x7f3bdc164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3be2000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc15f260 con 0x7f3bdc165310 2024-04-05T22:22:05.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3be2000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc070d20 con 0x7f3bdc15ee80 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3be2000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc165780 con 0x7f3bdc0b1950 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.282+0000 7f3bdb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bdc15ee80 0x7f3bdc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdbe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3bdc165310 0x7f3bdc161850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34502/0 (socket says 172.21.15.73:34502) 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdbe00640 1 -- 172.21.15.73:0/2155935943 learned_addr learned my addr 172.21.15.73:0/2155935943 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bd3e00640 1 --2- 172.21.15.73:0/2155935943 >> v2:172.21.15.157:3300/0 conn(0x7f3bdc0b1950 0x7f3bdc164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdb400640 1 -- 172.21.15.73:0/2155935943 >> v2:172.21.15.157:3300/0 conn(0x7f3bdc0b1950 msgr2=0x7f3bdc164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdb400640 1 --2- 172.21.15.73:0/2155935943 >> v2:172.21.15.157:3300/0 conn(0x7f3bdc0b1950 0x7f3bdc164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdb400640 1 -- 172.21.15.73:0/2155935943 >> v1:172.21.15.73:6789/0 conn(0x7f3bdc165310 legacy=0x7f3bdc161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdb400640 1 -- 172.21.15.73:0/2155935943 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc06f5b0 con 0x7f3bdc15ee80 2024-04-05T22:22:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.283+0000 7f3bdaa00640 1 -- 172.21.15.73:0/2155935943 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687995123 0 0) 0x7f3bd4001490 con 0x7f3bdc165310 2024-04-05T22:22:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.284+0000 7f3bdb400640 1 --2- 172.21.15.73:0/2155935943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc0cecc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06d4a0 tx=0x7f3bcc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5b9f2092ed688d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.284+0000 7f3bdaa00640 1 -- 172.21.15.73:0/2155935943 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc031830 con 0x7f3bdc15ee80 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.284+0000 7f3bdaa00640 1 -- 172.21.15.73:0/2155935943 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc031990 con 0x7f3bdc15ee80 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.284+0000 7f3bdaa00640 1 -- 172.21.15.73:0/2155935943 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc031d20 con 0x7f3bdc15ee80 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 -- 172.21.15.73:0/2155935943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15ee80 msgr2=0x7f3bdc0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 --2- 172.21.15.73:0/2155935943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc0cecc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06d4a0 tx=0x7f3bcc02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 -- 172.21.15.73:0/2155935943 shutdown_connections 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 --2- 172.21.15.73:0/2155935943 >> v2:172.21.15.157:3300/0 conn(0x7f3bdc0b1950 0x7f3bdc164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 --2- 172.21.15.73:0/2155935943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc0cecc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 -- 172.21.15.73:0/2155935943 >> 172.21.15.73:0/2155935943 conn(0x7f3bdc069cb0 msgr2=0x7f3bdc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 -- 172.21.15.73:0/2155935943 shutdown_connections 2024-04-05T22:22:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.285+0000 7f3be2000640 1 -- 172.21.15.73:0/2155935943 wait complete. 2024-04-05T22:22:05.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.287+0000 7f3be2000640 1 Processor -- start 2024-04-05T22:22:05.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.287+0000 7f3be2000640 1 -- start start 2024-04-05T22:22:05.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc14e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc165310 0x7f3bdc0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c5900 con 0x7f3bdc0b1950 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc14e400 con 0x7f3bdc15ee80 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3be2000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c8410 con 0x7f3bdc165310 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44348/0 (socket says 172.21.15.73:44348) 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 -- 172.21.15.73:0/614855868 learned_addr learned my addr 172.21.15.73:0/614855868 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 -- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc165310 msgr2=0x7f3bdc0c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc165310 0x7f3bdc0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 -- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc15ee80 msgr2=0x7f3bdc14e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.288+0000 7f3bdb400640 1 -- 172.21.15.73:0/614855868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009400 con 0x7f3bdc0b1950 2024-04-05T22:22:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3bdb400640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0098c0 tx=0x7f3bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc03ba70 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc03bbf0 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0453f0 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc0c86f0 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.289+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc14dd70 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.290+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc068160 con 0x7f3bdc0b1950 2024-04-05T22:22:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.291+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3bcc04e450 con 0x7f3bdc0b1950 2024-04-05T22:22:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.293+0000 7f3bd3400640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 0x7f3bb806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.293+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3bcc0ab700 con 0x7f3bdc0b1950 2024-04-05T22:22:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.293+0000 7f3bd3e00640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 0x7f3bb806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.294+0000 7f3bd3e00640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 0x7f3bb806d0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.297+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3bcc076850 con 0x7f3bdc0b1950 2024-04-05T22:22:05.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.487+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bdc0c5aa0 con 0x7f3bdc0b1950 2024-04-05T22:22:05.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.488+0000 7f3bd3400640 1 -- 172.21.15.73:0/614855868 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bcc079e70 con 0x7f3bdc0b1950 2024-04-05T22:22:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 msgr2=0x7f3bb806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 0x7f3bb806d0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 msgr2=0x7f3bdc0c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0098c0 tx=0x7f3bcc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 shutdown_connections 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb806ac30 0x7f3bb806d0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc165310 0x7f3bdc0c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc15ee80 0x7f3bdc14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 --2- 172.21.15.73:0/614855868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc0b1950 0x7f3bdc0c7d50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 >> 172.21.15.73:0/614855868 conn(0x7f3bdc069cb0 msgr2=0x7f3bdc0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 shutdown_connections 2024-04-05T22:22:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.492+0000 7f3be2000640 1 -- 172.21.15.73:0/614855868 wait complete. 2024-04-05T22:22:05.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:05.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.721+0000 7f7731200640 1 Processor -- start 2024-04-05T22:22:05.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.722+0000 7f7731200640 1 -- start start 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 0x7f772c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772c144260 0x7f772c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c0cdbb0 0x7f772c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c06b7a0 con 0x7f772c144260 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c06ba80 con 0x7f772c1435f0 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.723+0000 7f7731200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c06bd60 con 0x7f772c0cdbb0 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772c144260 0x7f772c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772c144260 0x7f772c1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44354/0 (socket says 172.21.15.73:44354) 2024-04-05T22:22:05.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 -- 172.21.15.73:0/809082502 learned_addr learned my addr 172.21.15.73:0/809082502 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:05.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 -- 172.21.15.73:0/809082502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c0cdbb0 msgr2=0x7f772c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:05.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f772be00640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c0cdbb0 0x7f772c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f772b400640 1 --2- 172.21.15.73:0/809082502 >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 0x7f772c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c0cdbb0 0x7f772c0cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 -- 172.21.15.73:0/809082502 >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 msgr2=0x7f772c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 --2- 172.21.15.73:0/809082502 >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 0x7f772c06c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f7723e00640 1 -- 172.21.15.73:0/809082502 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c06c040 con 0x7f772c144260 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.724+0000 7f772b400640 1 --2- 172.21.15.73:0/809082502 >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 0x7f772c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.725+0000 7f7723e00640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c1439d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f772c06d4b0 tx=0x7f771402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7651228db824271 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.725+0000 7f772aa00640 1 -- 172.21.15.73:0/809082502 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714031870 con 0x7f772c144260 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f772aa00640 1 -- 172.21.15.73:0/809082502 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77140319d0 con 0x7f772c144260 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f772aa00640 1 -- 172.21.15.73:0/809082502 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714031d60 con 0x7f772c144260 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 -- 172.21.15.73:0/809082502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 msgr2=0x7f772c1439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c1439d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f772c06d4b0 tx=0x7f771402b910 comp rx=0 tx=0).stop 2024-04-05T22:22:05.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 -- 172.21.15.73:0/809082502 shutdown_connections 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c1439d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 --2- 172.21.15.73:0/809082502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c0cdbb0 0x7f772c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 --2- 172.21.15.73:0/809082502 >> v2:172.21.15.157:3300/0 conn(0x7f772c1435f0 0x7f772c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 -- 172.21.15.73:0/809082502 >> 172.21.15.73:0/809082502 conn(0x7f772c0b0830 msgr2=0x7f772c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 -- 172.21.15.73:0/809082502 shutdown_connections 2024-04-05T22:22:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.726+0000 7f7731200640 1 -- 172.21.15.73:0/809082502 wait complete. 2024-04-05T22:22:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.728+0000 7f7731200640 1 Processor -- start 2024-04-05T22:22:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.728+0000 7f7731200640 1 -- start start 2024-04-05T22:22:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 0x7f772c1f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c1435f0 0x7f772c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c1f1120 con 0x7f772c144260 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0ac960 con 0x7f772c0cdbb0 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f7731200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0a8910 con 0x7f772c1435f0 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f772be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f772be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44356/0 (socket says 172.21.15.73:44356) 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.729+0000 7f772be00640 1 -- 172.21.15.73:0/1826366762 learned_addr learned my addr 172.21.15.73:0/1826366762 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772b400640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 0x7f772c1f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7723e00640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c1435f0 0x7f772c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 -- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c1435f0 msgr2=0x7f772c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c1435f0 0x7f772c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 -- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 msgr2=0x7f772c1f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 0x7f772c1f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 -- 172.21.15.73:0/1826366762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140094d0 con 0x7f772c144260 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772b400640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 0x7f772c1f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f772be00640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f772400cd70 tx=0x7f772400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724003b90 con 0x7f772c144260 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7724003cf0 con 0x7f772c144260 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c0a8bf0 con 0x7f772c144260 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724004240 con 0x7f772c144260 2024-04-05T22:22:05.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.730+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0a91b0 con 0x7f772c144260 2024-04-05T22:22:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.732+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c0a94f0 con 0x7f772c144260 2024-04-05T22:22:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.732+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f77240043a0 con 0x7f772c144260 2024-04-05T22:22:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.733+0000 7f7723400640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 0x7f770806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.733+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7724012070 con 0x7f772c144260 2024-04-05T22:22:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.734+0000 7f772b400640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 0x7f770806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:05.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.734+0000 7f772b400640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 0x7f770806d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f771c006fd0 tx=0x7f771c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:05.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.738+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7724054fa0 con 0x7f772c144260 2024-04-05T22:22:05.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.930+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f772c04e160 con 0x7f772c144260 2024-04-05T22:22:05.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.931+0000 7f7723400640 1 -- 172.21.15.73:0/1826366762 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77240585c0 con 0x7f772c144260 2024-04-05T22:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 msgr2=0x7f770806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 0x7f770806d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f771c006fd0 tx=0x7f771c008040 comp rx=0 tx=0).stop 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 msgr2=0x7f772c0ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f772400cd70 tx=0x7f772400a660 comp rx=0 tx=0).stop 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 shutdown_connections 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770806ac00 0x7f770806d0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772c144260 0x7f772c0ca400 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f772c1435f0 0x7f772c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 --2- 172.21.15.73:0/1826366762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772c0cdbb0 0x7f772c1f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:05.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.934+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 >> 172.21.15.73:0/1826366762 conn(0x7f772c0b0830 msgr2=0x7f772c15e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:05.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.935+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 shutdown_connections 2024-04-05T22:22:05.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:05.935+0000 7f7731200640 1 -- 172.21.15.73:0/1826366762 wait complete. 2024-04-05T22:22:05.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:22:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.165+0000 7f3202000640 1 Processor -- start 2024-04-05T22:22:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.166+0000 7f3202000640 1 -- start start 2024-04-05T22:22:06.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.167+0000 7f3202000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31fc15ce50 0x7f31fc0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.167+0000 7f3202000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31fc0cf450 0x7f31fc0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.167+0000 7f3202000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0cef00 con 0x7f31fc0b2c30 2024-04-05T22:22:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.167+0000 7f3202000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0cfe20 con 0x7f31fc15ce50 2024-04-05T22:22:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.167+0000 7f3202000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0d0d60 con 0x7f31fc0cf450 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31fc15ce50 0x7f31fc0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31fc15ce50 0x7f31fc0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34116/0 (socket says 172.21.15.73:34116) 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 -- 172.21.15.73:0/605915447 learned_addr learned my addr 172.21.15.73:0/605915447 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f31fbe00640 1 --2- 172.21.15.73:0/605915447 >> v2:172.21.15.157:3300/0 conn(0x7f31fc0cf450 0x7f31fc0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 -- 172.21.15.73:0/605915447 >> v2:172.21.15.157:3300/0 conn(0x7f31fc0cf450 msgr2=0x7f31fc0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 --2- 172.21.15.73:0/605915447 >> v2:172.21.15.157:3300/0 conn(0x7f31fc0cf450 0x7f31fc0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 -- 172.21.15.73:0/605915447 >> v1:172.21.15.73:6789/0 conn(0x7f31fc0b2c30 legacy=0x7f31fc0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f31fb400640 1 -- 172.21.15.73:0/605915447 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939815298 0 0) 0x7f31fc0cef00 con 0x7f31fc0b2c30 2024-04-05T22:22:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.168+0000 7f3200c00640 1 -- 172.21.15.73:0/605915447 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0b2150 con 0x7f31fc15ce50 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.169+0000 7f3200c00640 1 --2- 172.21.15.73:0/605915447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc15ce50 0x7f31fc0d0820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31fc08b460 tx=0x7f31ec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745993317838a066 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f31fb400640 1 -- 172.21.15.73:0/605915447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec02cb20 con 0x7f31fc15ce50 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f31fb400640 1 -- 172.21.15.73:0/605915447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec02cc80 con 0x7f31fc15ce50 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f31fb400640 1 -- 172.21.15.73:0/605915447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0343f0 con 0x7f31fc15ce50 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 -- 172.21.15.73:0/605915447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc15ce50 msgr2=0x7f31fc0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 --2- 172.21.15.73:0/605915447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc15ce50 0x7f31fc0d0820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31fc08b460 tx=0x7f31ec02bf80 comp rx=0 tx=0).stop 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 -- 172.21.15.73:0/605915447 shutdown_connections 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 --2- 172.21.15.73:0/605915447 >> v2:172.21.15.157:3300/0 conn(0x7f31fc0cf450 0x7f31fc0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 --2- 172.21.15.73:0/605915447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc15ce50 0x7f31fc0d0820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 -- 172.21.15.73:0/605915447 >> 172.21.15.73:0/605915447 conn(0x7f31fc0d2a40 msgr2=0x7f31fc0d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.170+0000 7f3202000640 1 -- 172.21.15.73:0/605915447 shutdown_connections 2024-04-05T22:22:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.171+0000 7f3202000640 1 -- 172.21.15.73:0/605915447 wait complete. 2024-04-05T22:22:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.172+0000 7f3202000640 1 Processor -- start 2024-04-05T22:22:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.172+0000 7f3202000640 1 -- start start 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0cf450 0x7f31fc0b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc15ce50 0x7f31fc0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1f37c0 con 0x7f31fc0b2c30 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1f3490 con 0x7f31fc15ce50 2024-04-05T22:22:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3202000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0b5e60 con 0x7f31fc0cf450 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.173+0000 7f3200c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34126/0 (socket says 172.21.15.73:34126) 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 -- 172.21.15.73:0/42414198 learned_addr learned my addr 172.21.15.73:0/42414198 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 -- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0cf450 msgr2=0x7f31fc0b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0cf450 0x7f31fc0b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 -- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc15ce50 msgr2=0x7f31fc0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc15ce50 0x7f31fc0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 -- 172.21.15.73:0/42414198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec009b30 con 0x7f31fc0b2c30 2024-04-05T22:22:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f3200c00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000f80 tx=0x7f31ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec009510 con 0x7f31fc0b2c30 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec03e450 con 0x7f31fc0b2c30 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.174+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec03e800 con 0x7f31fc0b2c30 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.175+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc0b6140 con 0x7f31fc0b2c30 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.175+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0b6420 con 0x7f31fc0b2c30 2024-04-05T22:22:06.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.176+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc067250 con 0x7f31fc0b2c30 2024-04-05T22:22:06.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.177+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f31ec03e960 con 0x7f31fc0b2c30 2024-04-05T22:22:06.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.177+0000 7f31f8c00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 0x7f31d806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.177+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f31ec0abf90 con 0x7f31fc0b2c30 2024-04-05T22:22:06.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.178+0000 7f31fbe00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 0x7f31d806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.178+0000 7f31fbe00640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 0x7f31d806d0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b7720 tx=0x7f31e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.182+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f31ec077180 con 0x7f31fc0b2c30 2024-04-05T22:22:06.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.386+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f31fc04e160 con 0x7f31fc0b2c30 2024-04-05T22:22:06.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.387+0000 7f31f8c00640 1 -- 172.21.15.73:0/42414198 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f31ec043080 con 0x7f31fc0b2c30 2024-04-05T22:22:06.387 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.389+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 msgr2=0x7f31d806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.389+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 0x7f31d806d0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b7720 tx=0x7f31e4008040 comp rx=0 tx=0).stop 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.389+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 msgr2=0x7f31fc0c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.389+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000f80 tx=0x7f31ec004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 shutdown_connections 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31d806ac30 0x7f31d806d0f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc15ce50 0x7f31fc0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0cf450 0x7f31fc0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 --2- 172.21.15.73:0/42414198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31fc0b2c30 0x7f31fc0c8ef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 >> 172.21.15.73:0/42414198 conn(0x7f31fc0d2a40 msgr2=0x7f31fc15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 shutdown_connections 2024-04-05T22:22:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.390+0000 7f3202000640 1 -- 172.21.15.73:0/42414198 wait complete. 2024-04-05T22:22:06.405 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-05T22:22:06.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:06.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-05T22:22:06.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:22:06.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:06.511 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-05T22:22:06.561 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:22:06.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:06.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.767+0000 7f97ef800640 1 Processor -- start 2024-04-05T22:22:06.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.767+0000 7f97ef800640 1 -- start start 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.768+0000 7f97ef800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97e8161f90 0x7f97e8162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ef800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e8165cf0 con 0x7f97e8164e50 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ef800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e8165730 con 0x7f97e80b2ba0 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ef800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e8158790 con 0x7f97e8161f90 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97dfe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f97e8164e50 0x7f97e80b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53756/0 (socket says 172.21.15.73:53756) 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97dfe00640 1 -- 172.21.15.73:0/1147516981 learned_addr learned my addr 172.21.15.73:0/1147516981 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:06.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ece00640 1 --2- 172.21.15.73:0/1147516981 >> v2:172.21.15.73:3300/0 conn(0x7f97e8161f90 0x7f97e8162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ece00640 1 -- 172.21.15.73:0/1147516981 >> v1:172.21.15.157:6789/0 conn(0x7f97e80b2ba0 legacy=0x7f97e8070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:06.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ece00640 1 -- 172.21.15.73:0/1147516981 >> v1:172.21.15.73:6789/0 conn(0x7f97e8164e50 legacy=0x7f97e80b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97ece00640 1 -- 172.21.15.73:0/1147516981 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8165a10 con 0x7f97e8161f90 2024-04-05T22:22:06.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.769+0000 7f97e7e00640 1 -- 172.21.15.73:0/1147516981 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408316185 0 0) 0x7f97e8165cf0 con 0x7f97e8164e50 2024-04-05T22:22:06.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.770+0000 7f97ece00640 1 --2- 172.21.15.73:0/1147516981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e8161f90 0x7f97e8162370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097a0 tx=0x7f97d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea356bfd09db15b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.771+0000 7f97e7e00640 1 -- 172.21.15.73:0/1147516981 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031830 con 0x7f97e8161f90 2024-04-05T22:22:06.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.771+0000 7f97e7e00640 1 -- 172.21.15.73:0/1147516981 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8031990 con 0x7f97e8161f90 2024-04-05T22:22:06.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.771+0000 7f97e7e00640 1 -- 172.21.15.73:0/1147516981 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031d20 con 0x7f97e8161f90 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.771+0000 7f97ef800640 1 -- 172.21.15.73:0/1147516981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e8161f90 msgr2=0x7f97e8162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.771+0000 7f97ef800640 1 --2- 172.21.15.73:0/1147516981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e8161f90 0x7f97e8162370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097a0 tx=0x7f97d802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.772+0000 7f97ef800640 1 -- 172.21.15.73:0/1147516981 shutdown_connections 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.772+0000 7f97ef800640 1 --2- 172.21.15.73:0/1147516981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e8161f90 0x7f97e8162370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.772+0000 7f97ef800640 1 -- 172.21.15.73:0/1147516981 >> 172.21.15.73:0/1147516981 conn(0x7f97e80ce8e0 msgr2=0x7f97e80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.772+0000 7f97ef800640 1 -- 172.21.15.73:0/1147516981 shutdown_connections 2024-04-05T22:22:06.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.772+0000 7f97ef800640 1 -- 172.21.15.73:0/1147516981 wait complete. 2024-04-05T22:22:06.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.773+0000 7f97ef800640 1 Processor -- start 2024-04-05T22:22:06.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.773+0000 7f97ef800640 1 -- start start 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97e8161f90 0x7f97e814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97e8164e50 0x7f97e81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8159ee0 con 0x7f97e80b2ba0 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e815a180 con 0x7f97e8161f90 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ef800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e814d120 con 0x7f97e8164e50 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.774+0000 7f97ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34136/0 (socket says 172.21.15.73:34136) 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 -- 172.21.15.73:0/2855954901 learned_addr learned my addr 172.21.15.73:0/2855954901 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97dfe00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97e8161f90 0x7f97e814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 -- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97e8164e50 msgr2=0x7f97e81588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97e8164e50 0x7f97e81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 -- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97e8161f90 msgr2=0x7f97e814cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97e8161f90 0x7f97e814cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:06.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 -- 172.21.15.73:0/2855954901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8009400 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97ece00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97d80098c0 tx=0x7f97d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.775+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d803ca80 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.776+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d803cbe0 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.776+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81f6da0 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.776+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8004030 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.776+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e8159420 con 0x7f97e80b2ba0 2024-04-05T22:22:06.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.777+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8066af0 con 0x7f97e80b2ba0 2024-04-05T22:22:06.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.778+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f97d8004190 con 0x7f97e80b2ba0 2024-04-05T22:22:06.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.778+0000 7f97e5600640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 0x7f97c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:06.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.778+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f97d80abc10 con 0x7f97e80b2ba0 2024-04-05T22:22:06.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.779+0000 7f97dfe00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 0x7f97c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:06.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.779+0000 7f97dfe00640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 0x7f97c406d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97e8164850 tx=0x7f97d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:06.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:06.783+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f97d8076cc0 con 0x7f97e80b2ba0 2024-04-05T22:22:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.015+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e8066af0 con 0x7f97e80b2ba0 2024-04-05T22:22:07.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.016+0000 7f97e5600640 1 -- 172.21.15.73:0/2855954901 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f97d807a2e0 con 0x7f97e80b2ba0 2024-04-05T22:22:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:07.016 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:22:07.016 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:22:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 msgr2=0x7f97c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 0x7f97c406d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97e8164850 tx=0x7f97d0009040 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 msgr2=0x7f97e814eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97d80098c0 tx=0x7f97d80046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 shutdown_connections 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f97c406ac00 0x7f97c406d0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97e8164e50 0x7f97e81588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97e8161f90 0x7f97e814cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 --2- 172.21.15.73:0/2855954901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e80b2ba0 0x7f97e814eb50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 >> 172.21.15.73:0/2855954901 conn(0x7f97e80ce8e0 msgr2=0x7f97e806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 shutdown_connections 2024-04-05T22:22:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.019+0000 7f97ef800640 1 -- 172.21.15.73:0/2855954901 wait complete. 2024-04-05T22:22:07.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.234+0000 7f80d3e00640 1 Processor -- start 2024-04-05T22:22:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.235+0000 7f80d3e00640 1 -- start start 2024-04-05T22:22:07.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.236+0000 7f80d3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80cc0b1870 0x7f80cc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.236+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.236+0000 7f80d3e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0ce5d0 con 0x7f80cc160620 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.236+0000 7f80d3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0ce010 con 0x7f80cc0b1870 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.236+0000 7f80d3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0ce2f0 con 0x7f80cc0ad6b0 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.237+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.237+0000 7f80d1400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f80cc160620 0x7f80cc0caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:47778/0 (socket says 172.21.15.73:47778) 2024-04-05T22:22:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.237+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36132/0 (socket says 172.21.15.73:36132) 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.237+0000 7f80d1400640 1 -- 172.21.15.73:0/1521406378 learned_addr learned my addr 172.21.15.73:0/1521406378 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.237+0000 7f80d0a00640 1 --2- 172.21.15.73:0/1521406378 >> v2:172.21.15.157:3300/0 conn(0x7f80cc0b1870 0x7f80cc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.238+0000 7f80d0a00640 1 -- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 msgr2=0x7f80cc0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.238+0000 7f80d0a00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.238+0000 7f80d0a00640 1 -- 172.21.15.73:0/1521406378 >> v1:172.21.15.157:6789/0 conn(0x7f80cc160620 legacy=0x7f80cc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.238+0000 7f80d0a00640 1 -- 172.21.15.73:0/1521406378 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0ac970 con 0x7f80cc0b1870 2024-04-05T22:22:07.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.238+0000 7f80d1e00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:22:07.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.239+0000 7f80d0a00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc0b1870 0x7f80cc0ad170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80b80097a0 tx=0x7f80b802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef35f31bc11e4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.240+0000 7f80cbe00640 1 -- 172.21.15.73:0/1521406378 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b802d040 con 0x7f80cc0b1870 2024-04-05T22:22:07.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.240+0000 7f80cbe00640 1 -- 172.21.15.73:0/1521406378 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8031810 con 0x7f80cc0b1870 2024-04-05T22:22:07.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.240+0000 7f80cbe00640 1 -- 172.21.15.73:0/1521406378 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8031ba0 con 0x7f80cc0b1870 2024-04-05T22:22:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.240+0000 7f80d3e00640 1 -- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc0b1870 msgr2=0x7f80cc0ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.240+0000 7f80d3e00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc0b1870 0x7f80cc0ad170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80b80097a0 tx=0x7f80b802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 -- 172.21.15.73:0/1521406378 shutdown_connections 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 --2- 172.21.15.73:0/1521406378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc0b1870 0x7f80cc0ad170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 -- 172.21.15.73:0/1521406378 >> 172.21.15.73:0/1521406378 conn(0x7f80cc0ce8e0 msgr2=0x7f80cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 -- 172.21.15.73:0/1521406378 shutdown_connections 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.241+0000 7f80d3e00640 1 -- 172.21.15.73:0/1521406378 wait complete. 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.242+0000 7f80d3e00640 1 Processor -- start 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 -- start start 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 0x7f80cc0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c3f70 con 0x7f80cc0b1870 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0af840 con 0x7f80cc160620 2024-04-05T22:22:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.243+0000 7f80d3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0afb20 con 0x7f80cc0ad6b0 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34142/0 (socket says 172.21.15.73:34142) 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 -- 172.21.15.73:0/320042192 learned_addr learned my addr 172.21.15.73:0/320042192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d1400640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d1e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 0x7f80cc0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 -- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 msgr2=0x7f80cc1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc1f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 -- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 msgr2=0x7f80cc0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 0x7f80cc0c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 -- 172.21.15.73:0/320042192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c4009a70 con 0x7f80cc0b1870 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d1e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 0x7f80cc0c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.244+0000 7f80d0a00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098c0 tx=0x7f80b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.245+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8006e10 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.245+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b803ad60 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.245+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b8009400 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.245+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8044620 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.245+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1f2670 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.246+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc066af0 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.247+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f80b804d440 con 0x7f80cc0b1870 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.247+0000 7f80c9600640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 0x7f80ac06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.248+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f80b80ab640 con 0x7f80cc0b1870 2024-04-05T22:22:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.248+0000 7f80d1400640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 0x7f80ac06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.249+0000 7f80d1400640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 0x7f80ac06d0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000f60 tx=0x7f80c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.253+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f80b8076820 con 0x7f80cc0b1870 2024-04-05T22:22:07.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.450+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80cc068160 con 0x7f80cc0b1870 2024-04-05T22:22:07.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.453+0000 7f80c9600640 1 -- 172.21.15.73:0/320042192 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16277 (secure 0 0 0) 0x7f80b8079e40 con 0x7f80cc0b1870 2024-04-05T22:22:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:07.453 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:01.664332+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:07.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 msgr2=0x7f80ac06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 0x7f80ac06d0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000f60 tx=0x7f80c0009040 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 msgr2=0x7f80cc0aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098c0 tx=0x7f80b80046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 shutdown_connections 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f80ac06ac30 0x7f80ac06d0f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80cc160620 0x7f80cc0c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80cc0b1870 0x7f80cc0aedf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 --2- 172.21.15.73:0/320042192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80cc0ad6b0 0x7f80cc1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 >> 172.21.15.73:0/320042192 conn(0x7f80cc0ce8e0 msgr2=0x7f80cc15f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 shutdown_connections 2024-04-05T22:22:07.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.456+0000 7f80d3e00640 1 -- 172.21.15.73:0/320042192 wait complete. 2024-04-05T22:22:07.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:21:12.881456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:21:13.896817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:07.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:07.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:22:07.636 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:22:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.688+0000 7f8c8f200640 1 Processor -- start 2024-04-05T22:22:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.688+0000 7f8c8f200640 1 -- start start 2024-04-05T22:22:07.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.689+0000 7f8c8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 0x7f8c880712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.689+0000 7f8c8f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c8806bc40 0x7f8c88164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c8f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8815afc0 con 0x7f8c8806c9a0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c8f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8806f5b0 con 0x7f8c8806bc40 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8806f890 con 0x7f8c8815abe0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c87400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8c8806c9a0 0x7f8c8806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53768/0 (socket says 172.21.15.73:53768) 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c87400640 1 -- 172.21.15.73:0/2242777216 learned_addr learned my addr 172.21.15.73:0/2242777216 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c87e00640 1 --2- 172.21.15.73:0/2242777216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 0x7f8c880712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c8d200640 1 --2- 172.21.15.73:0/2242777216 >> v2:172.21.15.157:3300/0 conn(0x7f8c8806bc40 0x7f8c88164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c86a00640 1 -- 172.21.15.73:0/2242777216 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462450454 0 0) 0x7f8c8815afc0 con 0x7f8c8806c9a0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.690+0000 7f8c86a00640 1 -- 172.21.15.73:0/2242777216 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c003520 con 0x7f8c8806c9a0 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.691+0000 7f8c87e00640 1 -- 172.21.15.73:0/2242777216 >> v2:172.21.15.157:3300/0 conn(0x7f8c8806bc40 msgr2=0x7f8c88164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.691+0000 7f8c87e00640 1 --2- 172.21.15.73:0/2242777216 >> v2:172.21.15.157:3300/0 conn(0x7f8c8806bc40 0x7f8c88164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.691+0000 7f8c87e00640 1 -- 172.21.15.73:0/2242777216 >> v1:172.21.15.73:6789/0 conn(0x7f8c8806c9a0 legacy=0x7f8c8806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.691+0000 7f8c87e00640 1 -- 172.21.15.73:0/2242777216 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8806fb70 con 0x7f8c8815abe0 2024-04-05T22:22:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.692+0000 7f8c87e00640 1 --2- 172.21.15.73:0/2242777216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 0x7f8c880712e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097a0 tx=0x7f8c7802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8b9cc9f1987863 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.692+0000 7f8c86a00640 1 -- 172.21.15.73:0/2242777216 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7802f050 con 0x7f8c8815abe0 2024-04-05T22:22:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.692+0000 7f8c86a00640 1 -- 172.21.15.73:0/2242777216 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78031700 con 0x7f8c8815abe0 2024-04-05T22:22:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.692+0000 7f8c86a00640 1 -- 172.21.15.73:0/2242777216 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78031a90 con 0x7f8c8815abe0 2024-04-05T22:22:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 -- 172.21.15.73:0/2242777216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 msgr2=0x7f8c880712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 --2- 172.21.15.73:0/2242777216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 0x7f8c880712e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097a0 tx=0x7f8c7802b7a0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 -- 172.21.15.73:0/2242777216 shutdown_connections 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 --2- 172.21.15.73:0/2242777216 >> v2:172.21.15.157:3300/0 conn(0x7f8c8806bc40 0x7f8c88164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 --2- 172.21.15.73:0/2242777216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8815abe0 0x7f8c880712e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 -- 172.21.15.73:0/2242777216 >> 172.21.15.73:0/2242777216 conn(0x7f8c88069cb0 msgr2=0x7f8c8806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 -- 172.21.15.73:0/2242777216 shutdown_connections 2024-04-05T22:22:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.693+0000 7f8c8f200640 1 -- 172.21.15.73:0/2242777216 wait complete. 2024-04-05T22:22:07.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.695+0000 7f8c8f200640 1 Processor -- start 2024-04-05T22:22:07.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.695+0000 7f8c8f200640 1 -- start start 2024-04-05T22:22:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 0x7f8c881464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 0x7f8c88156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b35c0 con 0x7f8c8806c9a0 2024-04-05T22:22:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b32e0 con 0x7f8c8815abe0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88147b00 con 0x7f8c8806bc40 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34152/0 (socket says 172.21.15.73:34152) 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 -- 172.21.15.73:0/850723478 learned_addr learned my addr 172.21.15.73:0/850723478 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87e00640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 0x7f8c881464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c8d200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 0x7f8c88156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 -- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 msgr2=0x7f8c881464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 0x7f8c881464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.696+0000 7f8c87400640 1 -- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 msgr2=0x7f8c88156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c87400640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 0x7f8c88156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c87400640 1 -- 172.21.15.73:0/850723478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78009400 con 0x7f8c8806c9a0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c87e00640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 0x7f8c881464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c8d200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 0x7f8c88156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c87400640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c500 tx=0x7f8c7400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74005ce0 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74005e40 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74007330 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c88147de0 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.697+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c88146a20 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.698+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c88148300 con 0x7f8c8806c9a0 2024-04-05T22:22:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.699+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c74007550 con 0x7f8c8806c9a0 2024-04-05T22:22:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.700+0000 7f8c7fe00640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 0x7f8c6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.700+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8c74088a00 con 0x7f8c8806c9a0 2024-04-05T22:22:07.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.700+0000 7f8c87e00640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 0x7f8c6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:07.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.701+0000 7f8c87e00640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 0x7f8c6806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c78035000 tx=0x7f8c78008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.706+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8c88148300 con 0x7f8c8806c9a0 2024-04-05T22:22:07.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.896+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c88148300 con 0x7f8c8806c9a0 2024-04-05T22:22:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.897+0000 7f8c7fe00640 1 -- 172.21.15.73:0/850723478 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c88148300 con 0x7f8c8806c9a0 2024-04-05T22:22:07.896 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 msgr2=0x7f8c6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 0x7f8c6806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c78035000 tx=0x7f8c78008030 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 msgr2=0x7f8c88155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c500 tx=0x7f8c7400c9c0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 shutdown_connections 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8c6806ac00 0x7f8c6806d0c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8815abe0 0x7f8c88156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8806c9a0 0x7f8c88155ae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 --2- 172.21.15.73:0/850723478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8806bc40 0x7f8c881464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 >> 172.21.15.73:0/850723478 conn(0x7f8c88069cb0 msgr2=0x7f8c8815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.899+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 shutdown_connections 2024-04-05T22:22:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:07.900+0000 7f8c8f200640 1 -- 172.21.15.73:0/850723478 wait complete. 2024-04-05T22:22:07.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:08.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.125+0000 7f3f4dc00640 1 Processor -- start 2024-04-05T22:22:08.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.125+0000 7f3f4dc00640 1 -- start start 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.126+0000 7f3f4dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f4815abe0 0x7f3f480712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f4dc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4815afc0 con 0x7f3f4806c9a0 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f4dc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4806f5b0 con 0x7f3f4806bc40 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f4dc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4806f890 con 0x7f3f4815abe0 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f4815abe0 0x7f3f480712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f4815abe0 0x7f3f480712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34166/0 (socket says 172.21.15.73:34166) 2024-04-05T22:22:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 -- 172.21.15.73:0/42880578 learned_addr learned my addr 172.21.15.73:0/42880578 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:08.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46000640 1 --1- 172.21.15.73:0/42880578 >> v1:172.21.15.73:6789/0 conn(0x7f3f4806c9a0 0x7f3f4806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53782/0 (socket says 172.21.15.73:53782) 2024-04-05T22:22:08.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 -- 172.21.15.73:0/42880578 >> v1:172.21.15.157:6789/0 conn(0x7f3f4806bc40 legacy=0x7f3f48164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:08.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 -- 172.21.15.73:0/42880578 >> v1:172.21.15.73:6789/0 conn(0x7f3f4806c9a0 legacy=0x7f3f4806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f46a00640 1 -- 172.21.15.73:0/42880578 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4806fb70 con 0x7f3f4815abe0 2024-04-05T22:22:08.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.127+0000 7f3f45600640 1 -- 172.21.15.73:0/42880578 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779218527 0 0) 0x7f3f4815afc0 con 0x7f3f4806c9a0 2024-04-05T22:22:08.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.128+0000 7f3f46a00640 1 --2- 172.21.15.73:0/42880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f480712e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db14fdc311fffe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f45600640 1 -- 172.21.15.73:0/42880578 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f380317e0 con 0x7f3f4815abe0 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f45600640 1 -- 172.21.15.73:0/42880578 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38031940 con 0x7f3f4815abe0 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f45600640 1 -- 172.21.15.73:0/42880578 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38031cd0 con 0x7f3f4815abe0 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 -- 172.21.15.73:0/42880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 msgr2=0x7f3f480712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/42880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f480712e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f3802b880 comp rx=0 tx=0).stop 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 -- 172.21.15.73:0/42880578 shutdown_connections 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/42880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f480712e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 -- 172.21.15.73:0/42880578 >> 172.21.15.73:0/42880578 conn(0x7f3f48069cb0 msgr2=0x7f3f4806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 -- 172.21.15.73:0/42880578 shutdown_connections 2024-04-05T22:22:08.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.129+0000 7f3f4dc00640 1 -- 172.21.15.73:0/42880578 wait complete. 2024-04-05T22:22:08.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.131+0000 7f3f4dc00640 1 Processor -- start 2024-04-05T22:22:08.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.131+0000 7f3f4dc00640 1 -- start start 2024-04-05T22:22:08.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 0x7f3f481454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4806c9a0 0x7f3f48147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48145a10 con 0x7f3f4815abe0 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481481b0 con 0x7f3f4806bc40 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f4dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48148490 con 0x7f3f4806c9a0 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34168/0 (socket says 172.21.15.73:34168) 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.132+0000 7f3f47400640 1 -- 172.21.15.73:0/285035654 learned_addr learned my addr 172.21.15.73:0/285035654 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f46a00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 0x7f3f481454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f46000640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4806c9a0 0x7f3f48147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 -- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4806c9a0 msgr2=0x7f3f48147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4806c9a0 0x7f3f48147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 -- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 msgr2=0x7f3f481454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 0x7f3f481454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 -- 172.21.15.73:0/285035654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380094d0 con 0x7f3f4815abe0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f46a00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 0x7f3f481454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f47400640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c780 tx=0x7f3f4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40004b60 con 0x7f3f4815abe0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4000ce90 con 0x7f3f4815abe0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48148770 con 0x7f3f4815abe0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.133+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000a850 con 0x7f3f4815abe0 2024-04-05T22:22:08.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.134+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48149140 con 0x7f3f4815abe0 2024-04-05T22:22:08.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.135+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f48067e90 con 0x7f3f4815abe0 2024-04-05T22:22:08.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.135+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3f4000a9b0 con 0x7f3f4815abe0 2024-04-05T22:22:08.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.136+0000 7f3f3ea00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 0x7f3f2806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.136+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3f400895c0 con 0x7f3f4815abe0 2024-04-05T22:22:08.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.136+0000 7f3f46a00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 0x7f3f2806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.137+0000 7f3f46a00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 0x7f3f2806d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3f38036000 tx=0x7f3f3803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.140+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3f40054700 con 0x7f3f4815abe0 2024-04-05T22:22:08.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.332+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f48149330 con 0x7f3f4815abe0 2024-04-05T22:22:08.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.333+0000 7f3f3ea00640 1 -- 172.21.15.73:0/285035654 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f40057d20 con 0x7f3f4815abe0 2024-04-05T22:22:08.332 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 msgr2=0x7f3f2806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 0x7f3f2806d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3f38036000 tx=0x7f3f3803d040 comp rx=0 tx=0).stop 2024-04-05T22:22:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 msgr2=0x7f3f4815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c780 tx=0x7f3f4000cc40 comp rx=0 tx=0).stop 2024-04-05T22:22:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 shutdown_connections 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.335+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3f2806ac00 0x7f3f2806d0c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f4815abe0 0x7f3f4815d120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4806c9a0 0x7f3f48147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 --2- 172.21.15.73:0/285035654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4806bc40 0x7f3f481454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 >> 172.21.15.73:0/285035654 conn(0x7f3f48069cb0 msgr2=0x7f3f480cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 shutdown_connections 2024-04-05T22:22:08.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.336+0000 7f3f4dc00640 1 -- 172.21.15.73:0/285035654 wait complete. 2024-04-05T22:22:08.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:08.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.569+0000 7f4d05400640 1 Processor -- start 2024-04-05T22:22:08.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.569+0000 7f4d05400640 1 -- start start 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4d05400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d0006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4d05400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d00072040 0x7f4d0006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4d05400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0015d450 con 0x7f4d000713f0 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4d05400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0006a760 con 0x7f4d00072040 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4d05400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0006aa40 con 0x7f4d0015d070 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d00072040 0x7f4d0006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d00072040 0x7f4d0006ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34182/0 (socket says 172.21.15.73:34182) 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.571+0000 7f4cf7e00640 1 -- 172.21.15.73:0/1053507233 learned_addr learned my addr 172.21.15.73:0/1053507233 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.572+0000 7f4cfe800640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d0006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.572+0000 7f4cf7e00640 1 -- 172.21.15.73:0/1053507233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 msgr2=0x7f4d0006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:08.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.572+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d0006d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.572+0000 7f4cf7e00640 1 -- 172.21.15.73:0/1053507233 >> v1:172.21.15.157:6789/0 conn(0x7f4d000713f0 legacy=0x7f4d00160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:08.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.572+0000 7f4cf7e00640 1 -- 172.21.15.73:0/1053507233 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0006ad20 con 0x7f4d00072040 2024-04-05T22:22:08.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.573+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d00072040 0x7f4d0006ddc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb743794a82124c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.573+0000 7f4cfde00640 1 -- 172.21.15.73:0/1053507233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8031640 con 0x7f4d00072040 2024-04-05T22:22:08.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.573+0000 7f4cfde00640 1 -- 172.21.15.73:0/1053507233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80317a0 con 0x7f4d00072040 2024-04-05T22:22:08.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.573+0000 7f4cfde00640 1 -- 172.21.15.73:0/1053507233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8031b30 con 0x7f4d00072040 2024-04-05T22:22:08.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 -- 172.21.15.73:0/1053507233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d00072040 msgr2=0x7f4d0006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d00072040 0x7f4d0006ddc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 -- 172.21.15.73:0/1053507233 shutdown_connections 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d00072040 0x7f4d0006ddc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 --2- 172.21.15.73:0/1053507233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d0006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 -- 172.21.15.73:0/1053507233 >> 172.21.15.73:0/1053507233 conn(0x7f4d000b2c70 msgr2=0x7f4d000b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 -- 172.21.15.73:0/1053507233 shutdown_connections 2024-04-05T22:22:08.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.574+0000 7f4d05400640 1 -- 172.21.15.73:0/1053507233 wait complete. 2024-04-05T22:22:08.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.576+0000 7f4d05400640 1 Processor -- start 2024-04-05T22:22:08.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.576+0000 7f4d05400640 1 -- start start 2024-04-05T22:22:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.577+0000 7f4d05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4d05400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 0x7f4d00157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4d05400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d00144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4d05400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001576f0 con 0x7f4d000713f0 2024-04-05T22:22:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4d05400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00156ae0 con 0x7f4d00072040 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4d05400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00156dc0 con 0x7f4d0015d070 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34186/0 (socket says 172.21.15.73:34186) 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 -- 172.21.15.73:0/3537524978 learned_addr learned my addr 172.21.15.73:0/3537524978 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cff200640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d00144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 0x7f4d00157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 -- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 msgr2=0x7f4d00144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d00144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 -- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 msgr2=0x7f4d00157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 0x7f4d00157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cfe800640 1 -- 172.21.15.73:0/3537524978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80094d0 con 0x7f4d000713f0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.578+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 0x7f4d00157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4cfe800640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d5f0 tx=0x7f4cf000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00125f0 con 0x7f4d000713f0 2024-04-05T22:22:08.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0012750 con 0x7f4d000713f0 2024-04-05T22:22:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0012b00 con 0x7f4d000713f0 2024-04-05T22:22:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d001570a0 con 0x7f4d000713f0 2024-04-05T22:22:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.579+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d001449a0 con 0x7f4d000713f0 2024-04-05T22:22:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.580+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0006c3b0 con 0x7f4d000713f0 2024-04-05T22:22:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.581+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4cf0012c60 con 0x7f4d000713f0 2024-04-05T22:22:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.582+0000 7f4cf6a00640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 0x7f4cdc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.582+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4cf00898d0 con 0x7f4d000713f0 2024-04-05T22:22:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.582+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 0x7f4cdc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:08.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.583+0000 7f4cf7e00640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 0x7f4cdc06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8036000 tx=0x7f4ce803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.588+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4cf0054a20 con 0x7f4d000713f0 2024-04-05T22:22:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.783+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d00068160 con 0x7f4d000713f0 2024-04-05T22:22:08.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.784+0000 7f4cf6a00640 1 -- 172.21.15.73:0/3537524978 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cf0058040 con 0x7f4d000713f0 2024-04-05T22:22:08.783 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.786+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 msgr2=0x7f4cdc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.786+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 0x7f4cdc06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8036000 tx=0x7f4ce803c040 comp rx=0 tx=0).stop 2024-04-05T22:22:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.786+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 msgr2=0x7f4d00144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d5f0 tx=0x7f4cf000dab0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 shutdown_connections 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4cdc06ac00 0x7f4cdc06d0c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0015d070 0x7f4d00144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d00072040 0x7f4d00157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 --2- 172.21.15.73:0/3537524978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000713f0 0x7f4d00144460 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 >> 172.21.15.73:0/3537524978 conn(0x7f4d000b2c70 msgr2=0x7f4d0006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 shutdown_connections 2024-04-05T22:22:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:08.787+0000 7f4d05400640 1 -- 172.21.15.73:0/3537524978 wait complete. 2024-04-05T22:22:08.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:09.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.018+0000 7f8d7a800640 1 Processor -- start 2024-04-05T22:22:09.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.018+0000 7f8d7a800640 1 -- start start 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.019+0000 7f8d7a800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d7415e620 0x7f8d7415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.019+0000 7f8d7a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d7406bc70 0x7f8d7406c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.019+0000 7f8d7a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7406b500 con 0x7f8d74158860 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.019+0000 7f8d7a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7415eff0 con 0x7f8d7406bc70 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.019+0000 7f8d7a800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740ccaf0 con 0x7f8d7415e620 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8d74158860 0x7f8d740b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53790/0 (socket says 172.21.15.73:53790) 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79400640 1 -- 172.21.15.73:0/3739648331 learned_addr learned my addr 172.21.15.73:0/3739648331 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:09.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79e00640 1 --2- 172.21.15.73:0/3739648331 >> v2:172.21.15.73:3300/0 conn(0x7f8d7406bc70 0x7f8d7406c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79e00640 1 -- 172.21.15.73:0/3739648331 >> v2:172.21.15.157:3300/0 conn(0x7f8d7415e620 msgr2=0x7f8d7415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:09.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79e00640 1 --2- 172.21.15.73:0/3739648331 >> v2:172.21.15.157:3300/0 conn(0x7f8d7415e620 0x7f8d7415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79e00640 1 -- 172.21.15.73:0/3739648331 >> v1:172.21.15.73:6789/0 conn(0x7f8d74158860 legacy=0x7f8d740b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.020+0000 7f8d79e00640 1 -- 172.21.15.73:0/3739648331 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740b20d0 con 0x7f8d7406bc70 2024-04-05T22:22:09.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.021+0000 7f8d79e00640 1 --2- 172.21.15.73:0/3739648331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d7406c0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d74072570 tx=0x7f8d6c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7892c49675079aed server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d73e00640 1 -- 172.21.15.73:0/3739648331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c02cbe0 con 0x7f8d7406bc70 2024-04-05T22:22:09.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d73e00640 1 -- 172.21.15.73:0/3739648331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c02cd40 con 0x7f8d7406bc70 2024-04-05T22:22:09.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d73e00640 1 -- 172.21.15.73:0/3739648331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c034660 con 0x7f8d7406bc70 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 -- 172.21.15.73:0/3739648331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 msgr2=0x7f8d7406c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 --2- 172.21.15.73:0/3739648331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d7406c0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d74072570 tx=0x7f8d6c02be30 comp rx=0 tx=0).stop 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 -- 172.21.15.73:0/3739648331 shutdown_connections 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 --2- 172.21.15.73:0/3739648331 >> v2:172.21.15.157:3300/0 conn(0x7f8d7415e620 0x7f8d7415eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 --2- 172.21.15.73:0/3739648331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d7406c0c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.022+0000 7f8d7a800640 1 -- 172.21.15.73:0/3739648331 >> 172.21.15.73:0/3739648331 conn(0x7f8d740b06c0 msgr2=0x7f8d740b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.023+0000 7f8d7a800640 1 -- 172.21.15.73:0/3739648331 shutdown_connections 2024-04-05T22:22:09.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.023+0000 7f8d7a800640 1 -- 172.21.15.73:0/3739648331 wait complete. 2024-04-05T22:22:09.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.025+0000 7f8d7a800640 1 Processor -- start 2024-04-05T22:22:09.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.025+0000 7f8d7a800640 1 -- start start 2024-04-05T22:22:09.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7415e620 0x7f8d74144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741454d0 0x7f8d7415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74145960 con 0x7f8d7406bc70 2024-04-05T22:22:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74156920 con 0x7f8d741454d0 2024-04-05T22:22:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d7a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7415e0f0 con 0x7f8d7415e620 2024-04-05T22:22:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d79400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d79400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34216/0 (socket says 172.21.15.73:34216) 2024-04-05T22:22:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.026+0000 7f8d79400640 1 -- 172.21.15.73:0/1518901084 learned_addr learned my addr 172.21.15.73:0/1518901084 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 -- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7415e620 msgr2=0x7f8d74144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7415e620 0x7f8d74144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 -- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741454d0 msgr2=0x7f8d7415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741454d0 0x7f8d7415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 -- 172.21.15.73:0/1518901084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c009a70 con 0x7f8d7406bc70 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d79400640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c4a0 tx=0x7f8d6800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68005ce0 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68005e40 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007270 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7415d4e0 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.027+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7415daa0 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.028+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7404e160 con 0x7f8d7406bc70 2024-04-05T22:22:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.029+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d680073d0 con 0x7f8d7406bc70 2024-04-05T22:22:09.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.030+0000 7f8d71600640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 0x7f8d5406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.030+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8d680887c0 con 0x7f8d7406bc70 2024-04-05T22:22:09.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.030+0000 7f8d78a00640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 0x7f8d5406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.031+0000 7f8d78a00640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 0x7f8d5406d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d74156050 tx=0x7f8d60006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.034+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8d680539c0 con 0x7f8d7406bc70 2024-04-05T22:22:09.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.264+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d74154a90 con 0x7f8d7406bc70 2024-04-05T22:22:09.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.265+0000 7f8d71600640 1 -- 172.21.15.73:0/1518901084 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f8d68017090 con 0x7f8d7406bc70 2024-04-05T22:22:09.265 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:09.265 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:22:09.265 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:22:09.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 msgr2=0x7f8d5406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 0x7f8d5406d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d74156050 tx=0x7f8d60006bf0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 msgr2=0x7f8d740ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c4a0 tx=0x7f8d6800c960 comp rx=0 tx=0).stop 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 shutdown_connections 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d5406ac30 0x7f8d5406d0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d741454d0 0x7f8d7415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7415e620 0x7f8d74144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 --2- 172.21.15.73:0/1518901084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d7406bc70 0x7f8d740ced50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 >> 172.21.15.73:0/1518901084 conn(0x7f8d740b06c0 msgr2=0x7f8d740cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 shutdown_connections 2024-04-05T22:22:09.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.268+0000 7f8d7a800640 1 -- 172.21.15.73:0/1518901084 wait complete. 2024-04-05T22:22:09.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:09.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.498+0000 7f6d67000640 1 Processor -- start 2024-04-05T22:22:09.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.498+0000 7f6d67000640 1 -- start start 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.500+0000 7f6d67000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d60160620 0x7f6d600caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.500+0000 7f6d67000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d600cb2f0 0x7f6d600cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.500+0000 7f6d67000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d600ad880 con 0x7f6d600b1870 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.500+0000 7f6d67000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d600cbcc0 con 0x7f6d600cb2f0 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.500+0000 7f6d67000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600ce5d0 con 0x7f6d60160620 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d65000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d600cb2f0 0x7f6d600cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d57400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d600b1870 0x7f6d600ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53804/0 (socket says 172.21.15.73:53804) 2024-04-05T22:22:09.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d57400640 1 -- 172.21.15.73:0/787467633 learned_addr learned my addr 172.21.15.73:0/787467633 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d5fe00640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d60160620 0x7f6d600caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d65000640 1 -- 172.21.15.73:0/787467633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d60160620 msgr2=0x7f6d600caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d65000640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d60160620 0x7f6d600caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d65000640 1 -- 172.21.15.73:0/787467633 >> v1:172.21.15.73:6789/0 conn(0x7f6d600b1870 legacy=0x7f6d600ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d65000640 1 -- 172.21.15.73:0/787467633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600ce010 con 0x7f6d600cb2f0 2024-04-05T22:22:09.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.501+0000 7f6d5f400640 1 -- 172.21.15.73:0/787467633 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035918097 0 0) 0x7f6d600ad880 con 0x7f6d600b1870 2024-04-05T22:22:09.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d65000640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600cb780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d580097a0 tx=0x7f6d5802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5d4300e0413edd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d5f400640 1 -- 172.21.15.73:0/787467633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802d040 con 0x7f6d600cb2f0 2024-04-05T22:22:09.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d5f400640 1 -- 172.21.15.73:0/787467633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58031810 con 0x7f6d600cb2f0 2024-04-05T22:22:09.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d5f400640 1 -- 172.21.15.73:0/787467633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58031ba0 con 0x7f6d600cb2f0 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d67000640 1 -- 172.21.15.73:0/787467633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 msgr2=0x7f6d600cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d67000640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600cb780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d580097a0 tx=0x7f6d5802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d67000640 1 -- 172.21.15.73:0/787467633 shutdown_connections 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d67000640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600cb780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.503+0000 7f6d67000640 1 --2- 172.21.15.73:0/787467633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d60160620 0x7f6d600caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.504+0000 7f6d67000640 1 -- 172.21.15.73:0/787467633 >> 172.21.15.73:0/787467633 conn(0x7f6d600ce8e0 msgr2=0x7f6d600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.504+0000 7f6d67000640 1 -- 172.21.15.73:0/787467633 shutdown_connections 2024-04-05T22:22:09.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.504+0000 7f6d67000640 1 -- 172.21.15.73:0/787467633 wait complete. 2024-04-05T22:22:09.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.505+0000 7f6d67000640 1 Processor -- start 2024-04-05T22:22:09.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.505+0000 7f6d67000640 1 -- start start 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d600b1870 0x7f6d601f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d60160620 0x7f6d6006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60153d00 con 0x7f6d600cb2f0 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601f5de0 con 0x7f6d60160620 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.506+0000 7f6d67000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601f6680 con 0x7f6d600b1870 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34238/0 (socket says 172.21.15.73:34238) 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 -- 172.21.15.73:0/985557254 learned_addr learned my addr 172.21.15.73:0/985557254 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d5fe00640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d600b1870 0x7f6d601f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 -- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d600b1870 msgr2=0x7f6d601f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d600b1870 0x7f6d601f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 -- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d60160620 msgr2=0x7f6d6006f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d65000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d60160620 0x7f6d6006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d60160620 0x7f6d6006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 -- 172.21.15.73:0/985557254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d58009400 con 0x7f6d600cb2f0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.507+0000 7f6d57400640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c500 tx=0x7f6d4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.508+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007560 con 0x7f6d600cb2f0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.508+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d480047e0 con 0x7f6d600cb2f0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.508+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48004b90 con 0x7f6d600cb2f0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.508+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d600c5c20 con 0x7f6d600cb2f0 2024-04-05T22:22:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.508+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d601f6310 con 0x7f6d600cb2f0 2024-04-05T22:22:09.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.509+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60066af0 con 0x7f6d600cb2f0 2024-04-05T22:22:09.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.510+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d48004cf0 con 0x7f6d600cb2f0 2024-04-05T22:22:09.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.511+0000 7f6d5cc00640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 0x7f6d3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.511+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6d48088610 con 0x7f6d600cb2f0 2024-04-05T22:22:09.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.511+0000 7f6d5fe00640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 0x7f6d3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.512+0000 7f6d5fe00640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 0x7f6d3c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009e90 tx=0x7f6d5000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.515+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d48053750 con 0x7f6d600cb2f0 2024-04-05T22:22:09.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.709+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d60068160 con 0x7f6d600cb2f0 2024-04-05T22:22:09.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.711+0000 7f6d5cc00640 1 -- 172.21.15.73:0/985557254 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16277 (secure 0 0 0) 0x7f6d48056d70 con 0x7f6d600cb2f0 2024-04-05T22:22:09.710 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:09.711 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:01.664332+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:09.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 msgr2=0x7f6d3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 0x7f6d3c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009e90 tx=0x7f6d5000b040 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 msgr2=0x7f6d600c5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c500 tx=0x7f6d4800c9c0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 shutdown_connections 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d3c06ac00 0x7f6d3c06d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d60160620 0x7f6d6006f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d600cb2f0 0x7f6d600c5280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 --2- 172.21.15.73:0/985557254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d600b1870 0x7f6d601f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 >> 172.21.15.73:0/985557254 conn(0x7f6d600ce8e0 msgr2=0x7f6d600b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 shutdown_connections 2024-04-05T22:22:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.714+0000 7f6d67000640 1 -- 172.21.15.73:0/985557254 wait complete. 2024-04-05T22:22:09.734 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:22:09.734 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:09.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.936+0000 7f3089600640 1 Processor -- start 2024-04-05T22:22:09.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.937+0000 7f3089600640 1 -- start start 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.938+0000 7f3089600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f308415cf60 0x7f30840b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.938+0000 7f3089600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308415d340 con 0x7f308406bf80 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.938+0000 7f3089600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308415e3b0 con 0x7f30840b5940 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.938+0000 7f3089600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308415e690 con 0x7f308415cf60 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.938+0000 7f3083e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f308415cf60 0x7f30840b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.939+0000 7f3083e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f308415cf60 0x7f30840b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34250/0 (socket says 172.21.15.73:34250) 2024-04-05T22:22:09.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.939+0000 7f3083e00640 1 -- 172.21.15.73:0/751762169 learned_addr learned my addr 172.21.15.73:0/751762169 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:09.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.939+0000 7f3083e00640 1 -- 172.21.15.73:0/751762169 >> v1:172.21.15.157:6789/0 conn(0x7f30840b5940 legacy=0x7f308406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:09.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.939+0000 7f3083e00640 1 -- 172.21.15.73:0/751762169 >> v1:172.21.15.73:6789/0 conn(0x7f308406bf80 legacy=0x7f308415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.939+0000 7f3083e00640 1 -- 172.21.15.73:0/751762169 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308415e970 con 0x7f308415cf60 2024-04-05T22:22:09.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.940+0000 7f3083e00640 1 --2- 172.21.15.73:0/751762169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f308415cf60 0x7f30840b5400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3074009870 tx=0x7f307402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289608fe81f981ff server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.940+0000 7f3082a00640 1 -- 172.21.15.73:0/751762169 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074030860 con 0x7f308415cf60 2024-04-05T22:22:09.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.940+0000 7f3082a00640 1 -- 172.21.15.73:0/751762169 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30740309c0 con 0x7f308415cf60 2024-04-05T22:22:09.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.940+0000 7f3082a00640 1 -- 172.21.15.73:0/751762169 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074030d50 con 0x7f308415cf60 2024-04-05T22:22:09.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 -- 172.21.15.73:0/751762169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f308415cf60 msgr2=0x7f30840b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 --2- 172.21.15.73:0/751762169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f308415cf60 0x7f30840b5400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3074009870 tx=0x7f307402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 -- 172.21.15.73:0/751762169 shutdown_connections 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 --2- 172.21.15.73:0/751762169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f308415cf60 0x7f30840b5400 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 -- 172.21.15.73:0/751762169 >> 172.21.15.73:0/751762169 conn(0x7f30840b5d50 msgr2=0x7f30840b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 -- 172.21.15.73:0/751762169 shutdown_connections 2024-04-05T22:22:09.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.941+0000 7f3089600640 1 -- 172.21.15.73:0/751762169 wait complete. 2024-04-05T22:22:09.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.943+0000 7f3089600640 1 Processor -- start 2024-04-05T22:22:09.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.943+0000 7f3089600640 1 -- start start 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f308406bf80 0x7f30840c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f308415cf60 0x7f30841502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840ce610 con 0x7f30840ca5c0 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084150810 con 0x7f308415cf60 2024-04-05T22:22:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3089600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30840ca240 con 0x7f308406bf80 2024-04-05T22:22:09.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34262/0 (socket says 172.21.15.73:34262) 2024-04-05T22:22:09.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 -- 172.21.15.73:0/211339947 learned_addr learned my addr 172.21.15.73:0/211339947 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3083400640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f308415cf60 0x7f30841502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 -- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f308406bf80 msgr2=0x7f30840c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f308406bf80 0x7f30840c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 -- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f308415cf60 msgr2=0x7f30841502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.944+0000 7f3088c00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f308415cf60 0x7f30841502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f3088c00640 1 -- 172.21.15.73:0/211339947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740094d0 con 0x7f30840ca5c0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f3088c00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c6c0 tx=0x7f307c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c003b70 con 0x7f30840ca5c0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c003cd0 con 0x7f30840ca5c0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0052f0 con 0x7f30840ca5c0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30840cdc70 con 0x7f30840ca5c0 2024-04-05T22:22:09.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.945+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30840ce230 con 0x7f30840ca5c0 2024-04-05T22:22:09.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.946+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30840ce3f0 con 0x7f30840ca5c0 2024-04-05T22:22:09.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.947+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f307c005450 con 0x7f30840ca5c0 2024-04-05T22:22:09.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.948+0000 7f307be00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 0x7f306006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:09.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.948+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f307c089230 con 0x7f30840ca5c0 2024-04-05T22:22:09.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.948+0000 7f3083e00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 0x7f306006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:09.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.949+0000 7f3083e00640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 0x7f306006d0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3074036000 tx=0x7f307403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:09.953+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f307c0542e0 con 0x7f30840ca5c0 2024-04-05T22:22:10.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.171+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f308404e160 con 0x7f30840ca5c0 2024-04-05T22:22:10.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.172+0000 7f307be00640 1 -- 172.21.15.73:0/211339947 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f307c057900 con 0x7f30840ca5c0 2024-04-05T22:22:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:10.171 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:22:10.171 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:22:10.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.174+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 msgr2=0x7f306006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.174+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 0x7f306006d0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3074036000 tx=0x7f307403b040 comp rx=0 tx=0).stop 2024-04-05T22:22:10.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.174+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 msgr2=0x7f308406e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.174+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c6c0 tx=0x7f307c00cb80 comp rx=0 tx=0).stop 2024-04-05T22:22:10.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 shutdown_connections 2024-04-05T22:22:10.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f306006ac30 0x7f306006d0f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30840ca5c0 0x7f308406e820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f308415cf60 0x7f30841502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 --2- 172.21.15.73:0/211339947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f308406bf80 0x7f30840c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 >> 172.21.15.73:0/211339947 conn(0x7f30840b5d50 msgr2=0x7f30840b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:10.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 shutdown_connections 2024-04-05T22:22:10.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.175+0000 7f3089600640 1 -- 172.21.15.73:0/211339947 wait complete. 2024-04-05T22:22:10.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:10.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.376+0000 7f1929200640 1 Processor -- start 2024-04-05T22:22:10.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.376+0000 7f1929200640 1 -- start start 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19240ccd20 0x7f192406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 0x7f192415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406f5b0 con 0x7f19240ccd20 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1924070410 con 0x7f192406d0c0 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1929200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1924158bb0 con 0x7f19241587d0 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1921400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19240ccd20 0x7f192406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1921400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19240ccd20 0x7f192406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34266/0 (socket says 172.21.15.73:34266) 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.378+0000 7f1921400640 1 -- 172.21.15.73:0/1513357795 learned_addr learned my addr 172.21.15.73:0/1513357795 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921e00640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1922800640 1 --2- 172.21.15.73:0/1513357795 >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 0x7f192415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921400640 1 -- 172.21.15.73:0/1513357795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 msgr2=0x7f19240cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921400640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921400640 1 -- 172.21.15.73:0/1513357795 >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 msgr2=0x7f192415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921400640 1 --2- 172.21.15.73:0/1513357795 >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 0x7f192415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921400640 1 -- 172.21.15.73:0/1513357795 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192406f800 con 0x7f19240ccd20 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1922800640 1 --2- 172.21.15.73:0/1513357795 >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 0x7f192415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.379+0000 7f1921e00640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:10.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.380+0000 7f1921400640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240ccd20 0x7f192406cb80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19240ae4b0 tx=0x7f190c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e83c9db62c3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.380+0000 7f1920a00640 1 -- 172.21.15.73:0/1513357795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c031880 con 0x7f19240ccd20 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.380+0000 7f1920a00640 1 -- 172.21.15.73:0/1513357795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c0319e0 con 0x7f19240ccd20 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.380+0000 7f1920a00640 1 -- 172.21.15.73:0/1513357795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c031d70 con 0x7f19240ccd20 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 -- 172.21.15.73:0/1513357795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240ccd20 msgr2=0x7f192406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240ccd20 0x7f192406cb80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19240ae4b0 tx=0x7f190c02b920 comp rx=0 tx=0).stop 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 -- 172.21.15.73:0/1513357795 shutdown_connections 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240ccd20 0x7f192406cb80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 --2- 172.21.15.73:0/1513357795 >> v2:172.21.15.157:3300/0 conn(0x7f192406d0c0 0x7f192415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 --2- 172.21.15.73:0/1513357795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 -- 172.21.15.73:0/1513357795 >> 172.21.15.73:0/1513357795 conn(0x7f19240ce8e0 msgr2=0x7f19240cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 -- 172.21.15.73:0/1513357795 shutdown_connections 2024-04-05T22:22:10.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.381+0000 7f1929200640 1 -- 172.21.15.73:0/1513357795 wait complete. 2024-04-05T22:22:10.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.383+0000 7f1929200640 1 Processor -- start 2024-04-05T22:22:10.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.383+0000 7f1929200640 1 -- start start 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406d0c0 0x7f19240c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c4c40 con 0x7f19240af4b0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c4030 con 0x7f192406d0c0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1929200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c4310 con 0x7f19241587d0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1922800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1922800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34280/0 (socket says 172.21.15.73:34280) 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1922800640 1 -- 172.21.15.73:0/3669760797 learned_addr learned my addr 172.21.15.73:0/3669760797 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1921e00640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406d0c0 0x7f19240c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1921400640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.384+0000 7f1922800640 1 -- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 msgr2=0x7f19240b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1922800640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1922800640 1 -- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406d0c0 msgr2=0x7f19240c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1922800640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406d0c0 0x7f19240c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1922800640 1 -- 172.21.15.73:0/3669760797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0094d0 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1922800640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f191c00cef0 tx=0x7f191c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00d840 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00d9a0 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00dd50 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240c45f0 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.385+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240c3d20 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.386+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1924066af0 con 0x7f19240af4b0 2024-04-05T22:22:10.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.387+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f191c01e3b0 con 0x7f19240af4b0 2024-04-05T22:22:10.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.388+0000 7f191a000640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 0x7f190006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.388+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f191c089f00 con 0x7f19240af4b0 2024-04-05T22:22:10.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.388+0000 7f1921e00640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 0x7f190006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.389+0000 7f1921e00640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 0x7f190006d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f191400a950 tx=0x7f1914008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.393+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f191c054fb0 con 0x7f19240af4b0 2024-04-05T22:22:10.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.627+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1924068160 con 0x7f19240af4b0 2024-04-05T22:22:10.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.628+0000 7f191a000640 1 -- 172.21.15.73:0/3669760797 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f191c01a020 con 0x7f19240af4b0 2024-04-05T22:22:10.627 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:10.628 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2761997556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2761997556},{"type":"v1","addr":"172.21.15.157:6834","nonce":2761997556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3194761427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3194761427},{"type":"v1","addr":"172.21.15.73:6835","nonce":3194761427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3724183563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3724183563},{"type":"v1","addr":"172.21.15.73:6837","nonce":3724183563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:21:14.885132+0000","modified":"2024-04-05T22:21:17.895240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/123005434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":123005434},{"type":"v1","addr":"172.21.15.157:6835","nonce":123005434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-05T22:22:10.628 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 7 2024-04-05T22:22:10.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.630+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 msgr2=0x7f190006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.630+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 0x7f190006d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f191400a950 tx=0x7f1914008040 comp rx=0 tx=0).stop 2024-04-05T22:22:10.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 msgr2=0x7f19240c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f191c00cef0 tx=0x7f191c00a710 comp rx=0 tx=0).stop 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 shutdown_connections 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f190006ac00 0x7f190006d0c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19240af4b0 0x7f19240c55d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19241587d0 0x7f19240b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 --2- 172.21.15.73:0/3669760797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406d0c0 0x7f19240c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 >> 172.21.15.73:0/3669760797 conn(0x7f19240ce8e0 msgr2=0x7f19240cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 shutdown_connections 2024-04-05T22:22:10.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.631+0000 7f1929200640 1 -- 172.21.15.73:0/3669760797 wait complete. 2024-04-05T22:22:10.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:10.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.840+0000 7f5c19a00640 1 Processor -- start 2024-04-05T22:22:10.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.841+0000 7f5c19a00640 1 -- start start 2024-04-05T22:22:10.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.842+0000 7f5c19a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c1406d710 0x7f5c140cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.842+0000 7f5c19a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c1415aef0 0x7f5c140cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.842+0000 7f5c19a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1415aa60 con 0x7f5c141541a0 2024-04-05T22:22:10.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.842+0000 7f5c19a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c140b3b60 con 0x7f5c1415aef0 2024-04-05T22:22:10.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.842+0000 7f5c19a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c140701c0 con 0x7f5c1406d710 2024-04-05T22:22:10.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c1415aef0 0x7f5c140cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c1415aef0 0x7f5c140cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34296/0 (socket says 172.21.15.73:34296) 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 -- 172.21.15.73:0/2075373674 learned_addr learned my addr 172.21.15.73:0/2075373674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 -- 172.21.15.73:0/2075373674 >> v2:172.21.15.157:3300/0 conn(0x7f5c1406d710 msgr2=0x7f5c140cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 --2- 172.21.15.73:0/2075373674 >> v2:172.21.15.157:3300/0 conn(0x7f5c1406d710 0x7f5c140cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 -- 172.21.15.73:0/2075373674 >> v1:172.21.15.157:6789/0 conn(0x7f5c141541a0 legacy=0x7f5c140b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.843+0000 7f5c13400640 1 -- 172.21.15.73:0/2075373674 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140b3880 con 0x7f5c1415aef0 2024-04-05T22:22:10.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.844+0000 7f5c13400640 1 --2- 172.21.15.73:0/2075373674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1415aef0 0x7f5c140cde30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c1415e980 tx=0x7f5c0c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec3c7e893e46590 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.844+0000 7f5c12000640 1 -- 172.21.15.73:0/2075373674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c02ca80 con 0x7f5c1415aef0 2024-04-05T22:22:10.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.844+0000 7f5c12000640 1 -- 172.21.15.73:0/2075373674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c02cbe0 con 0x7f5c1415aef0 2024-04-05T22:22:10.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.844+0000 7f5c12000640 1 -- 172.21.15.73:0/2075373674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0343f0 con 0x7f5c1415aef0 2024-04-05T22:22:10.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 -- 172.21.15.73:0/2075373674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1415aef0 msgr2=0x7f5c140cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 --2- 172.21.15.73:0/2075373674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1415aef0 0x7f5c140cde30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c1415e980 tx=0x7f5c0c02bee0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 -- 172.21.15.73:0/2075373674 shutdown_connections 2024-04-05T22:22:10.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 --2- 172.21.15.73:0/2075373674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1415aef0 0x7f5c140cde30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 --2- 172.21.15.73:0/2075373674 >> v2:172.21.15.157:3300/0 conn(0x7f5c1406d710 0x7f5c140cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 -- 172.21.15.73:0/2075373674 >> 172.21.15.73:0/2075373674 conn(0x7f5c140ce8e0 msgr2=0x7f5c140cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:10.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 -- 172.21.15.73:0/2075373674 shutdown_connections 2024-04-05T22:22:10.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.845+0000 7f5c19a00640 1 -- 172.21.15.73:0/2075373674 wait complete. 2024-04-05T22:22:10.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.847+0000 7f5c19a00640 1 Processor -- start 2024-04-05T22:22:10.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.847+0000 7f5c19a00640 1 -- start start 2024-04-05T22:22:10.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406d710 0x7f5c14151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 0x7f5c1414a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406d330 con 0x7f5c141541a0 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406c4d0 con 0x7f5c1406d710 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c19a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406c7b0 con 0x7f5c1415aef0 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c0a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c0a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34300/0 (socket says 172.21.15.73:34300) 2024-04-05T22:22:10.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c0a000640 1 -- 172.21.15.73:0/1781100206 learned_addr learned my addr 172.21.15.73:0/1781100206 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.848+0000 7f5c13400640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 0x7f5c1414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 -- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 msgr2=0x7f5c1414a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c12a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406d710 0x7f5c14151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 0x7f5c1414a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 -- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406d710 msgr2=0x7f5c14151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:10.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406d710 0x7f5c14151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 -- 172.21.15.73:0/1781100206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c009b30 con 0x7f5c141541a0 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c13400640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 0x7f5c1414a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0a000640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b4a0 tx=0x7f5bfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc010610 con 0x7f5c141541a0 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc010770 con 0x7f5c141541a0 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1406ca90 con 0x7f5c141541a0 2024-04-05T22:22:10.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc010b20 con 0x7f5c141541a0 2024-04-05T22:22:10.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.849+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1406d050 con 0x7f5c141541a0 2024-04-05T22:22:10.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.850+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c14066af0 con 0x7f5c141541a0 2024-04-05T22:22:10.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.851+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5bfc010c80 con 0x7f5c141541a0 2024-04-05T22:22:10.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.852+0000 7f5c0b400640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 0x7f5bf006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:10.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.852+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5bfc0890f0 con 0x7f5c141541a0 2024-04-05T22:22:10.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.852+0000 7f5c12a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 0x7f5bf006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:10.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.853+0000 7f5c12a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 0x7f5bf006d0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c040015d0 tx=0x7f5c04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:10.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:10.856+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5bfc0541a0 con 0x7f5c141541a0 2024-04-05T22:22:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.039+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c14068160 con 0x7f5c141541a0 2024-04-05T22:22:11.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.042+0000 7f5c0b400640 1 -- 172.21.15.73:0/1781100206 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16277 (secure 0 0 0) 0x7f5bfc0577c0 con 0x7f5c141541a0 2024-04-05T22:22:11.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:11.042 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:01.664332+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:21:12.881456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-05T22:21:13.896817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:11.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 msgr2=0x7f5bf006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 0x7f5bf006d0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c040015d0 tx=0x7f5c04009040 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 msgr2=0x7f5c14152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b4a0 tx=0x7f5bfc00b960 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 shutdown_connections 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5bf006ac00 0x7f5bf006d0c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c1415aef0 0x7f5c1414a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c141541a0 0x7f5c14152e30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 --2- 172.21.15.73:0/1781100206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1406d710 0x7f5c14151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 >> 172.21.15.73:0/1781100206 conn(0x7f5c140ce8e0 msgr2=0x7f5c1414f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 shutdown_connections 2024-04-05T22:22:11.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.045+0000 7f5c19a00640 1 -- 172.21.15.73:0/1781100206 wait complete. 2024-04-05T22:22:11.062 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:22:11.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.276+0000 7f1880800640 1 Processor -- start 2024-04-05T22:22:11.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.276+0000 7f1880800640 1 -- start start 2024-04-05T22:22:11.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f1880800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1878160620 0x7f18780caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f1880800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780ad880 con 0x7f18780cb2f0 2024-04-05T22:22:11.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f1880800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780ce5d0 con 0x7f18780b1870 2024-04-05T22:22:11.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f1880800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18780ce010 con 0x7f1878160620 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f187e800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f18780cb2f0 0x7f18780cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53832/0 (socket says 172.21.15.73:53832) 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.278+0000 7f187e800640 1 -- 172.21.15.73:0/2807869907 learned_addr learned my addr 172.21.15.73:0/2807869907 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187de00640 1 --2- 172.21.15.73:0/2807869907 >> v2:172.21.15.157:3300/0 conn(0x7f1878160620 0x7f18780caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816214672 0 0) 0x7f18780ad880 con 0x7f18780cb2f0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1860003520 con 0x7f18780cb2f0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2122638519 0 0) 0x7f1860003520 con 0x7f18780cb2f0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 >> v2:172.21.15.157:3300/0 conn(0x7f1878160620 msgr2=0x7f18780caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 --2- 172.21.15.73:0/2807869907 >> v2:172.21.15.157:3300/0 conn(0x7f1878160620 0x7f18780caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 >> v1:172.21.15.157:6789/0 conn(0x7f18780b1870 legacy=0x7f18780ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780ce2f0 con 0x7f18780cb2f0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f18740019f0 con 0x7f18780cb2f0 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 >> v1:172.21.15.73:6789/0 conn(0x7f18780cb2f0 legacy=0x7f18780cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1860003970 0x7f1860003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18600047c0 0x7f1860004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 0x7f186000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780ce010 con 0x7f1860005180 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186000c020 con 0x7f18600047c0 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.279+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186000c1e0 con 0x7f1860003970 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 0x7f186000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 -- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1860003970 msgr2=0x7f1860003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187de00640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1860003970 0x7f1860003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187d400640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18600047c0 0x7f1860004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1860003970 0x7f1860003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 -- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18600047c0 msgr2=0x7f1860004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18600047c0 0x7f1860004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.280+0000 7f187e800640 1 -- 172.21.15.73:0/2807869907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860003520 con 0x7f1860005180 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.281+0000 7f187e800640 1 -- 172.21.15.73:0/2807869907 reap_dead start 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.281+0000 7f187e800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 0x7f186000b980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f187400a830 tx=0x7f18600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daf12698b09abdf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.281+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740072d0 con 0x7f1860005180 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.281+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187402f070 con 0x7f1860005180 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.281+0000 7f187ca00640 1 -- 172.21.15.73:0/2807869907 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740076b0 con 0x7f1860005180 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 -- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 msgr2=0x7f186000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 0x7f186000b980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f187400a830 tx=0x7f18600057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 -- 172.21.15.73:0/2807869907 shutdown_connections 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 --2- 172.21.15.73:0/2807869907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1860005180 0x7f186000b980 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 -- 172.21.15.73:0/2807869907 >> 172.21.15.73:0/2807869907 conn(0x7f18780ce8e0 msgr2=0x7f18780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 -- 172.21.15.73:0/2807869907 shutdown_connections 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.282+0000 7f1880800640 1 -- 172.21.15.73:0/2807869907 wait complete. 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.284+0000 7f1880800640 1 Processor -- start 2024-04-05T22:22:11.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.284+0000 7f1880800640 1 -- start start 2024-04-05T22:22:11.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1860005180 0x7f18781f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 0x7f18780af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781f6640 con 0x7f18781f6a60 2024-04-05T22:22:11.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781f2800 con 0x7f1860005180 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f1880800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1878148e80 con 0x7f187815d550 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f187d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f187d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34318/0 (socket says 172.21.15.73:34318) 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.285+0000 7f187d400640 1 -- 172.21.15.73:0/3732437608 learned_addr learned my addr 172.21.15.73:0/3732437608 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187e800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 0x7f18780af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187de00640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1860005180 0x7f18781f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 -- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 msgr2=0x7f18780af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 0x7f18780af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 -- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1860005180 msgr2=0x7f18781f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1860005180 0x7f18781f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 -- 172.21.15.73:0/3732437608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187815ddb0 con 0x7f18781f6a60 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187e800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 0x7f18780af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:11.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f187d400640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f186400d4f0 tx=0x7f186400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864005480 con 0x7f18781f6a60 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864003610 con 0x7f18781f6a60 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864003980 con 0x7f18781f6a60 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187815dad0 con 0x7f18781f6a60 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.286+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18781f6260 con 0x7f18781f6a60 2024-04-05T22:22:11.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.288+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1878066af0 con 0x7f18781f6a60 2024-04-05T22:22:11.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.288+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1864003ae0 con 0x7f18781f6a60 2024-04-05T22:22:11.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.289+0000 7f1872000640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 0x7f185806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.289+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f18640892f0 con 0x7f18781f6a60 2024-04-05T22:22:11.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.290+0000 7f187de00640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 0x7f185806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.290+0000 7f187de00640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 0x7f185806d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f186c009910 tx=0x7f186c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.294+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f18640543a0 con 0x7f18781f6a60 2024-04-05T22:22:11.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.507+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f18780cb160 con 0x7f18781f6a60 2024-04-05T22:22:11.507 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:22:11.508+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:22:11.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.520+0000 7f1872000640 1 -- 172.21.15.73:0/3732437608 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f18640579c0 con 0x7f18781f6a60 2024-04-05T22:22:11.520 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:22:11.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 msgr2=0x7f185806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 0x7f185806d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f186c009910 tx=0x7f186c008040 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 msgr2=0x7f187815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f186400d4f0 tx=0x7f186400d9b0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 shutdown_connections 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f185806ac00 0x7f185806d0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f187815d550 0x7f18780af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18781f6a60 0x7f187815d010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 --2- 172.21.15.73:0/3732437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1860005180 0x7f18781f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 >> 172.21.15.73:0/3732437608 conn(0x7f18780ce8e0 msgr2=0x7f18780ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 shutdown_connections 2024-04-05T22:22:11.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.523+0000 7f1880800640 1 -- 172.21.15.73:0/3732437608 wait complete. 2024-04-05T22:22:11.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:22:11.555 INFO:tasks.ceph.mds.b.smithi157.stderr: -7474> 2024-04-05T22:21:20.017+0000 7fcfb2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:22:11.648 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:22:11.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.735+0000 7f1acc200640 1 Processor -- start 2024-04-05T22:22:11.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.735+0000 7f1acc200640 1 -- start start 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1acc200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ac40b3240 0x7f1ac40cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1acc200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac40d0170 con 0x7f1ac40cd350 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1acc200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac40d0600 con 0x7f1ac40cdc70 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1acc200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac40cafa0 con 0x7f1ac40b3240 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1ac9800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ac40cd350 0x7f1ac40ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53838/0 (socket says 172.21.15.73:53838) 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1ac9800640 1 -- 172.21.15.73:0/4007849635 learned_addr learned my addr 172.21.15.73:0/4007849635 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:11.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1ac8e00640 1 --2- 172.21.15.73:0/4007849635 >> v2:172.21.15.73:3300/0 conn(0x7f1ac40b3240 0x7f1ac40cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.737+0000 7f1ac8e00640 1 -- 172.21.15.73:0/4007849635 >> v1:172.21.15.157:6789/0 conn(0x7f1ac40cdc70 legacy=0x7f1ac4164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:11.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.738+0000 7f1ac8e00640 1 -- 172.21.15.73:0/4007849635 >> v1:172.21.15.73:6789/0 conn(0x7f1ac40cd350 legacy=0x7f1ac40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.738+0000 7f1ac8e00640 1 -- 172.21.15.73:0/4007849635 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40cf560 con 0x7f1ac40b3240 2024-04-05T22:22:11.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.739+0000 7f1ac8e00640 1 --2- 172.21.15.73:0/4007849635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40b3240 0x7f1ac40cd730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b1c8ed3d391c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.739+0000 7f1ac3e00640 1 -- 172.21.15.73:0/4007849635 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4031830 con 0x7f1ac40b3240 2024-04-05T22:22:11.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.739+0000 7f1ac3e00640 1 -- 172.21.15.73:0/4007849635 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4031990 con 0x7f1ac40b3240 2024-04-05T22:22:11.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.739+0000 7f1ac3e00640 1 -- 172.21.15.73:0/4007849635 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4031d20 con 0x7f1ac40b3240 2024-04-05T22:22:11.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 -- 172.21.15.73:0/4007849635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40b3240 msgr2=0x7f1ac40cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 --2- 172.21.15.73:0/4007849635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40b3240 0x7f1ac40cd730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 -- 172.21.15.73:0/4007849635 shutdown_connections 2024-04-05T22:22:11.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 --2- 172.21.15.73:0/4007849635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40b3240 0x7f1ac40cd730 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 -- 172.21.15.73:0/4007849635 >> 172.21.15.73:0/4007849635 conn(0x7f1ac40b2720 msgr2=0x7f1ac40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:11.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 -- 172.21.15.73:0/4007849635 shutdown_connections 2024-04-05T22:22:11.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.740+0000 7f1acc200640 1 -- 172.21.15.73:0/4007849635 wait complete. 2024-04-05T22:22:11.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.742+0000 7f1acc200640 1 Processor -- start 2024-04-05T22:22:11.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.742+0000 7f1acc200640 1 -- start start 2024-04-05T22:22:11.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b3240 0x7f1ac4155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40cd350 0x7f1ac409aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac409d8b0 con 0x7f1ac40cdc70 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac409db90 con 0x7f1ac40b3240 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1acc200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac409de70 con 0x7f1ac40cd350 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34328/0 (socket says 172.21.15.73:34328) 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 -- 172.21.15.73:0/3594321047 learned_addr learned my addr 172.21.15.73:0/3594321047 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1ac8e00640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40cd350 0x7f1ac409aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1ac9800640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b3240 0x7f1ac4155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 -- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40cd350 msgr2=0x7f1ac409aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40cd350 0x7f1ac409aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 -- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b3240 msgr2=0x7f1ac4155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b3240 0x7f1ac4155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.743+0000 7f1aca200640 1 -- 172.21.15.73:0/3594321047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009400 con 0x7f1ac40cdc70 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1aca200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b970 tx=0x7f1abc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00cb40 con 0x7f1ac40cdc70 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc00cca0 con 0x7f1ac40cdc70 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc014680 con 0x7f1ac40cdc70 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac409e150 con 0x7f1ac40cdc70 2024-04-05T22:22:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.744+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac409e950 con 0x7f1ac40cdc70 2024-04-05T22:22:11.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.745+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac409e670 con 0x7f1ac40cdc70 2024-04-05T22:22:11.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.746+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1abc01d3b0 con 0x7f1ac40cdc70 2024-04-05T22:22:11.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.747+0000 7f1ac1600640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 0x7f1aa806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:11.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.747+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f1abc089270 con 0x7f1ac40cdc70 2024-04-05T22:22:11.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.747+0000 7f1ac9800640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 0x7f1aa806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:11.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.748+0000 7f1ac9800640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 0x7f1aa806d0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005880 tx=0x7f1ab8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:11.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.751+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1ac409e670 con 0x7f1ac40cdc70 2024-04-05T22:22:11.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:11.967+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ac409e670 con 0x7f1ac40cdc70 2024-04-05T22:22:12.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.526+0000 7f1ac1600640 1 -- 172.21.15.73:0/3594321047 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ac409e670 con 0x7f1ac40cdc70 2024-04-05T22:22:12.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 msgr2=0x7f1aa806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 0x7f1aa806d0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005880 tx=0x7f1ab8009040 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 msgr2=0x7f1ac409afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b970 tx=0x7f1abc00be30 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 shutdown_connections 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1aa806ac30 0x7f1aa806d0f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40cdc70 0x7f1ac409afc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40cd350 0x7f1ac409aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 --2- 172.21.15.73:0/3594321047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b3240 0x7f1ac4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 >> 172.21.15.73:0/3594321047 conn(0x7f1ac40b2720 msgr2=0x7f1ac40cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.529+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 shutdown_connections 2024-04-05T22:22:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.530+0000 7f1acc200640 1 -- 172.21.15.73:0/3594321047 wait complete. 2024-04-05T22:22:12.551 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:22:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.765+0000 7f8771000640 1 Processor -- start 2024-04-05T22:22:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.765+0000 7f8771000640 1 -- start start 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8771000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f876c0cda10 0x7f876c0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8771000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0ce360 con 0x7f876c0714b0 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8771000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0ccc50 con 0x7f876c15aa80 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8771000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0cd5f0 con 0x7f876c0cda10 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8762a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f876c0714b0 0x7f876c06b490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53852/0 (socket says 172.21.15.73:53852) 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.767+0000 7f8762a00640 1 -- 172.21.15.73:0/3688992909 learned_addr learned my addr 172.21.15.73:0/3688992909 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876be00640 1 --2- 172.21.15.73:0/3688992909 >> v2:172.21.15.157:3300/0 conn(0x7f876c0cda10 0x7f876c0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705432991 0 0) 0x7f876c0ce360 con 0x7f876c0714b0 2024-04-05T22:22:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8750003520 con 0x7f876c0714b0 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1839510181 0 0) 0x7f8750003520 con 0x7f876c0714b0 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 >> v2:172.21.15.157:3300/0 conn(0x7f876c0cda10 msgr2=0x7f876c0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 --2- 172.21.15.73:0/3688992909 >> v2:172.21.15.157:3300/0 conn(0x7f876c0cda10 0x7f876c0cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 >> v1:172.21.15.157:6789/0 conn(0x7f876c15aa80 legacy=0x7f876c06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0ccf30 con 0x7f876c0714b0 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8754001670 con 0x7f876c0714b0 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 >> v1:172.21.15.73:6789/0 conn(0x7f876c0714b0 legacy=0x7f876c06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750003970 0x7f8750003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500047c0 0x7f8750004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 0x7f875000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0cd5f0 con 0x7f87500052a0 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875000c170 con 0x7f8750003970 2024-04-05T22:22:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.768+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875000c330 con 0x7f87500047c0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 0x7f875000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876b400640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750003970 0x7f8750003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 -- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500047c0 msgr2=0x7f8750004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500047c0 0x7f8750004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 -- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750003970 msgr2=0x7f8750003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750003970 0x7f8750003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876be00640 1 -- 172.21.15.73:0/3688992909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8750003520 con 0x7f87500052a0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.769+0000 7f876b400640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750003970 0x7f8750003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.770+0000 7f876be00640 1 -- 172.21.15.73:0/3688992909 reap_dead start 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.770+0000 7f876be00640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 0x7f875000b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8764009f90 tx=0x7f87500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d27597c102d003c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.770+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876402cc20 con 0x7f87500052a0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.770+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400f040 con 0x7f87500052a0 2024-04-05T22:22:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.770+0000 7f876aa00640 1 -- 172.21.15.73:0/3688992909 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764003990 con 0x7f87500052a0 2024-04-05T22:22:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 -- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 msgr2=0x7f875000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 0x7f875000b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8764009f90 tx=0x7f87500057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 -- 172.21.15.73:0/3688992909 shutdown_connections 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 --2- 172.21.15.73:0/3688992909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87500052a0 0x7f875000b980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 -- 172.21.15.73:0/3688992909 >> 172.21.15.73:0/3688992909 conn(0x7f876c0b0360 msgr2=0x7f876c0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 -- 172.21.15.73:0/3688992909 shutdown_connections 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.771+0000 7f8771000640 1 -- 172.21.15.73:0/3688992909 wait complete. 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.773+0000 7f8771000640 1 Processor -- start 2024-04-05T22:22:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.773+0000 7f8771000640 1 -- start start 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87500052a0 0x7f876c0b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 0x7f876c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1f5df0 con 0x7f876c1f5430 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b30e0 con 0x7f87500052a0 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f8771000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c14e980 con 0x7f876c0cad40 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f876be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f876be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34338/0 (socket says 172.21.15.73:34338) 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.774+0000 7f876be00640 1 -- 172.21.15.73:0/1053239233 learned_addr learned my addr 172.21.15.73:0/1053239233 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876b400640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87500052a0 0x7f876c0b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f8762a00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 0x7f876c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 -- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 msgr2=0x7f876c1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 0x7f876c1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 -- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87500052a0 msgr2=0x7f876c0b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87500052a0 0x7f876c0b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 -- 172.21.15.73:0/1053239233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c151c30 con 0x7f876c1f5430 2024-04-05T22:22:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f8762a00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 0x7f876c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f876be00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f876402c690 tx=0x7f8764009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.775+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876402cea0 con 0x7f876c1f5430 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.776+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764004230 con 0x7f876c1f5430 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.776+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640045e0 con 0x7f876c1f5430 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.776+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c0b1b00 con 0x7f876c1f5430 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.776+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c0b20c0 con 0x7f876c1f5430 2024-04-05T22:22:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.777+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c04e160 con 0x7f876c1f5430 2024-04-05T22:22:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.778+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8764004740 con 0x7f876c1f5430 2024-04-05T22:22:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.779+0000 7f8763e00640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 0x7f874806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.779+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f87640ab9b0 con 0x7f876c1f5430 2024-04-05T22:22:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.779+0000 7f876b400640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 0x7f874806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.780+0000 7f876b400640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 0x7f874806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f875c007fd0 tx=0x7f875c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.784+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8764076bc0 con 0x7f876c1f5430 2024-04-05T22:22:12.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:12.977+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f876c0735e0 con 0x7f876c1f5430 2024-04-05T22:22:13.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.764+0000 7f8763e00640 1 -- 172.21.15.73:0/1053239233 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f876403d330 con 0x7f876c1f5430 2024-04-05T22:22:13.767 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:22:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.772+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 msgr2=0x7f874806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.772+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 0x7f874806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f875c007fd0 tx=0x7f875c009040 comp rx=0 tx=0).stop 2024-04-05T22:22:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.772+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 msgr2=0x7f876c1f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.772+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f876402c690 tx=0x7f8764009750 comp rx=0 tx=0).stop 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 shutdown_connections 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f874806ac00 0x7f874806d0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f876c1f5430 0x7f876c1f58b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f876c0cad40 0x7f876c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 --2- 172.21.15.73:0/1053239233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87500052a0 0x7f876c0b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.773+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 >> 172.21.15.73:0/1053239233 conn(0x7f876c0b0360 msgr2=0x7f876c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:13.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.775+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 shutdown_connections 2024-04-05T22:22:13.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.776+0000 7f8771000640 1 -- 172.21.15.73:0/1053239233 wait complete. 2024-04-05T22:22:13.795 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:22:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.988+0000 7fbce8c00640 1 Processor -- start 2024-04-05T22:22:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.988+0000 7fbce8c00640 1 -- start start 2024-04-05T22:22:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce40b36e0 0x7fbce40cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce40cf210 0x7fbce415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce8c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce415afe0 con 0x7fbce415ac00 2024-04-05T22:22:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce8c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce40cf6a0 con 0x7fbce40cf210 2024-04-05T22:22:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce4161760 con 0x7fbce40b36e0 2024-04-05T22:22:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.990+0000 7fbce2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce40cf210 0x7fbce415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce40cf210 0x7fbce415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34346/0 (socket says 172.21.15.73:34346) 2024-04-05T22:22:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 -- 172.21.15.73:0/3563133384 learned_addr learned my addr 172.21.15.73:0/3563133384 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce1400640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce40b36e0 0x7fbce40cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 -- 172.21.15.73:0/3563133384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce40b36e0 msgr2=0x7fbce40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce40b36e0 0x7fbce40cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 -- 172.21.15.73:0/3563133384 >> v1:172.21.15.157:6789/0 conn(0x7fbce415ac00 legacy=0x7fbce406cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.991+0000 7fbce2800640 1 -- 172.21.15.73:0/3563133384 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce4160b50 con 0x7fbce40cf210 2024-04-05T22:22:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.992+0000 7fbce2800640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce415d560 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009870 tx=0x7fbcdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece4902d70001e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:13.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.992+0000 7fbce0a00640 1 -- 172.21.15.73:0/3563133384 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc031640 con 0x7fbce40cf210 2024-04-05T22:22:13.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.992+0000 7fbce0a00640 1 -- 172.21.15.73:0/3563133384 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc0317a0 con 0x7fbce40cf210 2024-04-05T22:22:13.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.992+0000 7fbce0a00640 1 -- 172.21.15.73:0/3563133384 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc031b30 con 0x7fbce40cf210 2024-04-05T22:22:13.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 -- 172.21.15.73:0/3563133384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 msgr2=0x7fbce415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce415d560 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009870 tx=0x7fbcdc02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 -- 172.21.15.73:0/3563133384 shutdown_connections 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce415d560 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 --2- 172.21.15.73:0/3563133384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce40b36e0 0x7fbce40cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 -- 172.21.15.73:0/3563133384 >> 172.21.15.73:0/3563133384 conn(0x7fbce406a0d0 msgr2=0x7fbce406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 -- 172.21.15.73:0/3563133384 shutdown_connections 2024-04-05T22:22:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.993+0000 7fbce8c00640 1 -- 172.21.15.73:0/3563133384 wait complete. 2024-04-05T22:22:13.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.995+0000 7fbce8c00640 1 Processor -- start 2024-04-05T22:22:13.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.995+0000 7fbce8c00640 1 -- start start 2024-04-05T22:22:13.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce40b36e0 0x7fbce415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:13.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:13.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce415ac00 0x7fbce415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce4160040 con 0x7fbce40cf210 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce415f3c0 con 0x7fbce40b36e0 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce415f6a0 con 0x7fbce415ac00 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34358/0 (socket says 172.21.15.73:34358) 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce1400640 1 -- 172.21.15.73:0/2378482289 learned_addr learned my addr 172.21.15.73:0/2378482289 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce1e00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce40b36e0 0x7fbce415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.996+0000 7fbce2800640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce415ac00 0x7fbce415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 -- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce415ac00 msgr2=0x7fbce415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce415ac00 0x7fbce415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 -- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce40b36e0 msgr2=0x7fbce415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce40b36e0 0x7fbce415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 -- 172.21.15.73:0/2378482289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0094d0 con 0x7fbce40cf210 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce1400640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00d800 tx=0x7fbccc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00b6c0 con 0x7fbce40cf210 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00b820 con 0x7fbce40cf210 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00bbb0 con 0x7fbce40cf210 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce415f980 con 0x7fbce40cf210 2024-04-05T22:22:13.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.997+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce415f0b0 con 0x7fbce40cf210 2024-04-05T22:22:14.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.998+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce4068160 con 0x7fbce40cf210 2024-04-05T22:22:14.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:13.999+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbccc00bd10 con 0x7fbce40cf210 2024-04-05T22:22:14.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.000+0000 7fbcda000640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:14.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.000+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fbccc089810 con 0x7fbce40cf210 2024-04-05T22:22:14.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.001+0000 7fbce1e00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:14.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.001+0000 7fbce1e00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 0x7fbcc006d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbce4158d80 tx=0x7fbcd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:14.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.005+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbccc054ab0 con 0x7fbce40cf210 2024-04-05T22:22:14.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.196+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbce4070650 con 0x7fbce40cf210 2024-04-05T22:22:14.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.775+0000 7fbcda000640 1 -- 172.21.15.73:0/2378482289 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbccc0580d0 con 0x7fbce40cf210 2024-04-05T22:22:14.780 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:22:14.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.781+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 msgr2=0x7fbcc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:14.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.781+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 0x7fbcc006d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbce4158d80 tx=0x7fbcd4009040 comp rx=0 tx=0).stop 2024-04-05T22:22:14.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.781+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 msgr2=0x7fbce41547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:14.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.781+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00d800 tx=0x7fbccc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 shutdown_connections 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcc006ac00 0x7fbcc006d0c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbce415ac00 0x7fbce415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce40cf210 0x7fbce41547d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 --2- 172.21.15.73:0/2378482289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbce40b36e0 0x7fbce415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 >> 172.21.15.73:0/2378482289 conn(0x7fbce406a0d0 msgr2=0x7fbce406fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 shutdown_connections 2024-04-05T22:22:14.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:14.782+0000 7fbce8c00640 1 -- 172.21.15.73:0/2378482289 wait complete. 2024-04-05T22:22:14.807 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:22:14.807 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:22:14.807 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:22:14.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:22:14.808 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:22:14.808 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:22:14.808 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:22:14.808 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:22:14.808 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:22:14.808 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:22:14.808 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:22:14.808 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:22:14.809 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:22:14.809 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:22:14.809 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:22:14.809 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:22:14.827 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:22:14.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:22:14.829 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:22:14.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:22:14.830 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:22:14.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:22:14.831 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:22:14.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:22:14.876 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:22:14.876 DEBUG:teuthology.parallel:result is None 2024-04-05T22:22:14.878 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:22:14.878 DEBUG:teuthology.parallel:result is None 2024-04-05T22:22:14.882 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:22:14.882 DEBUG:teuthology.parallel:result is None 2024-04-05T22:22:14.884 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:22:14.884 DEBUG:teuthology.parallel:result is None 2024-04-05T22:22:14.884 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:22:15.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-05T22:22:15.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-05T22:22:15.040 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:22:15.041 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:22:15.043 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:22:15.043 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:22:15.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.082+0000 7f7840800640 1 Processor -- start 2024-04-05T22:22:15.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.083+0000 7f7840800640 1 -- start start 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838160620 0x7f78380caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 0x7f78380ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78380ad6b0 0x7f78380cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380ce5d0 con 0x7f7838160620 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380ce010 con 0x7f78380b1870 2024-04-05T22:22:15.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f7840800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380ce2f0 con 0x7f78380ad6b0 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.084+0000 7f783de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838160620 0x7f78380caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838160620 0x7f78380caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34392/0 (socket says 172.21.15.73:34392) 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 -- 172.21.15.73:0/4047582780 learned_addr learned my addr 172.21.15.73:0/4047582780 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783e800640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78380ad6b0 0x7f78380cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783d400640 1 --2- 172.21.15.73:0/4047582780 >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 0x7f78380ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 -- 172.21.15.73:0/4047582780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78380ad6b0 msgr2=0x7f78380cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78380ad6b0 0x7f78380cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 -- 172.21.15.73:0/4047582780 >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 msgr2=0x7f78380ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 --2- 172.21.15.73:0/4047582780 >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 0x7f78380ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783de00640 1 -- 172.21.15.73:0/4047582780 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380ac970 con 0x7f7838160620 2024-04-05T22:22:15.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.085+0000 7f783d400640 1 --2- 172.21.15.73:0/4047582780 >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 0x7f78380ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.086+0000 7f783de00640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838160620 0x7f78380caa60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f783806af00 tx=0x7f782c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494e4244bc898717 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f783ca00640 1 -- 172.21.15.73:0/4047582780 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c031870 con 0x7f7838160620 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f783ca00640 1 -- 172.21.15.73:0/4047582780 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c0319d0 con 0x7f7838160620 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f783ca00640 1 -- 172.21.15.73:0/4047582780 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c031d60 con 0x7f7838160620 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 -- 172.21.15.73:0/4047582780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838160620 msgr2=0x7f78380caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838160620 0x7f78380caa60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f783806af00 tx=0x7f782c02b910 comp rx=0 tx=0).stop 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 -- 172.21.15.73:0/4047582780 shutdown_connections 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78380ad6b0 0x7f78380cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 --2- 172.21.15.73:0/4047582780 >> v2:172.21.15.157:3300/0 conn(0x7f78380b1870 0x7f78380ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 --2- 172.21.15.73:0/4047582780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838160620 0x7f78380caa60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 -- 172.21.15.73:0/4047582780 >> 172.21.15.73:0/4047582780 conn(0x7f78380ce8e0 msgr2=0x7f78380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:15.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 -- 172.21.15.73:0/4047582780 shutdown_connections 2024-04-05T22:22:15.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.087+0000 7f7840800640 1 -- 172.21.15.73:0/4047582780 wait complete. 2024-04-05T22:22:15.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.089+0000 7f7840800640 1 Processor -- start 2024-04-05T22:22:15.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.089+0000 7f7840800640 1 -- start start 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78380ad6b0 0x7f783815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7838160620 0x7f78381594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838158280 con 0x7f78380b1870 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838159cc0 con 0x7f78380ad6b0 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f7840800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78381578e0 con 0x7f7838160620 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f783d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f783d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34404/0 (socket says 172.21.15.73:34404) 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.090+0000 7f783d400640 1 -- 172.21.15.73:0/2703442382 learned_addr learned my addr 172.21.15.73:0/2703442382 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:15.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783e800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7838160620 0x7f78381594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783de00640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78380ad6b0 0x7f783815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 -- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7838160620 msgr2=0x7f78381594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7838160620 0x7f78381594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 -- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78380ad6b0 msgr2=0x7f783815ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78380ad6b0 0x7f783815ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 -- 172.21.15.73:0/2703442382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0094d0 con 0x7f78380b1870 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f783d400640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f782800e7e0 tx=0x7f782800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.091+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828004030 con 0x7f78380b1870 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.092+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7828004190 con 0x7f78380b1870 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.092+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828004520 con 0x7f78380b1870 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.092+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7838157bc0 con 0x7f78380b1870 2024-04-05T22:22:15.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.092+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78381599e0 con 0x7f78380b1870 2024-04-05T22:22:15.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.093+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838066af0 con 0x7f78380b1870 2024-04-05T22:22:15.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.094+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f782800c6d0 con 0x7f78380b1870 2024-04-05T22:22:15.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.095+0000 7f7832000640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 0x7f781c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:15.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.095+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7828088cc0 con 0x7f78380b1870 2024-04-05T22:22:15.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.095+0000 7f783de00640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 0x7f781c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:15.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.096+0000 7f783de00640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 0x7f781c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f782c036000 tx=0x7f782c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:15.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.099+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f782808f080 con 0x7f78380b1870 2024-04-05T22:22:15.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:15.310+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f78380cdde0 con 0x7f78380b1870 2024-04-05T22:22:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.082+0000 7f7832000640 1 -- 172.21.15.73:0/2703442382 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7828020080 con 0x7f78380b1870 2024-04-05T22:22:16.083 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:22:16.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 msgr2=0x7f781c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 0x7f781c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f782c036000 tx=0x7f782c03c040 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 msgr2=0x7f78381586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f782800e7e0 tx=0x7f782800eca0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 shutdown_connections 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f781c06ac00 0x7f781c06d0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7838160620 0x7f78381594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78380b1870 0x7f78381586a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 --2- 172.21.15.73:0/2703442382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78380ad6b0 0x7f783815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 >> 172.21.15.73:0/2703442382 conn(0x7f78380ce8e0 msgr2=0x7f783815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.086+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 shutdown_connections 2024-04-05T22:22:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.087+0000 7f7840800640 1 -- 172.21.15.73:0/2703442382 wait complete. 2024-04-05T22:22:16.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:22:16.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.376+0000 7f1118400640 1 Processor -- start 2024-04-05T22:22:16.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.377+0000 7f1118400640 1 -- start start 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.378+0000 7f1118400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11101587d0 0x7f11100cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.378+0000 7f1118400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 0x7f111006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.378+0000 7f1118400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111006f5b0 con 0x7f111006d0c0 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.378+0000 7f1118400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1110070410 con 0x7f11101587d0 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.378+0000 7f1118400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1110158bb0 con 0x7f11100ccd20 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11101587d0 0x7f11100cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1116400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f111006d0c0 0x7f111015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49612/0 (socket says 172.21.15.73:49612) 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1116400640 1 -- 172.21.15.73:0/2939628303 learned_addr learned my addr 172.21.15.73:0/2939628303 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11101587d0 0x7f11100cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49282/0 (socket says 172.21.15.73:49282) 2024-04-05T22:22:16.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 -- 172.21.15.73:0/2939628303 >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 msgr2=0x7f111006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115000640 1 --2- 172.21.15.73:0/2939628303 >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 0x7f111006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f110fe00640 1 -- 172.21.15.73:0/2939628303 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920345520 0 0) 0x7f111006f5b0 con 0x7f111006d0c0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 --2- 172.21.15.73:0/2939628303 >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 0x7f111006cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 -- 172.21.15.73:0/2939628303 >> v1:172.21.15.73:6789/0 conn(0x7f111006d0c0 legacy=0x7f111015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115a00640 1 -- 172.21.15.73:0/2939628303 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111006f800 con 0x7f11101587d0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.379+0000 7f1115000640 1 --2- 172.21.15.73:0/2939628303 >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 0x7f111006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.380+0000 7f1115a00640 1 --2- 172.21.15.73:0/2939628303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f11100cc7e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f11000097a0 tx=0x7f110002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91843e933c41f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.380+0000 7f110fe00640 1 -- 172.21.15.73:0/2939628303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100031830 con 0x7f11101587d0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f110fe00640 1 -- 172.21.15.73:0/2939628303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100031990 con 0x7f11101587d0 2024-04-05T22:22:16.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f110fe00640 1 -- 172.21.15.73:0/2939628303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100031d20 con 0x7f11101587d0 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 -- 172.21.15.73:0/2939628303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 msgr2=0x7f11100cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 --2- 172.21.15.73:0/2939628303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f11100cc7e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f11000097a0 tx=0x7f110002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 -- 172.21.15.73:0/2939628303 shutdown_connections 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 --2- 172.21.15.73:0/2939628303 >> v2:172.21.15.157:3300/0 conn(0x7f11100ccd20 0x7f111006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 --2- 172.21.15.73:0/2939628303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f11100cc7e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 -- 172.21.15.73:0/2939628303 >> 172.21.15.73:0/2939628303 conn(0x7f11100ce8e0 msgr2=0x7f11100cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:16.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 -- 172.21.15.73:0/2939628303 shutdown_connections 2024-04-05T22:22:16.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.381+0000 7f1118400640 1 -- 172.21.15.73:0/2939628303 wait complete. 2024-04-05T22:22:16.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.383+0000 7f1118400640 1 Processor -- start 2024-04-05T22:22:16.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.383+0000 7f1118400640 1 -- start start 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111006d0c0 0x7f11100c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11100c4de0 0x7f11100b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c4220 con 0x7f11101587d0 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c4b80 con 0x7f11100c4de0 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1118400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c57f0 con 0x7f111006d0c0 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1115000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1115000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49286/0 (socket says 172.21.15.73:49286) 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.384+0000 7f1115000640 1 -- 172.21.15.73:0/1525347365 learned_addr learned my addr 172.21.15.73:0/1525347365 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:16.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 -- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111006d0c0 msgr2=0x7f11100c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111006d0c0 0x7f11100c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 -- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11100c4de0 msgr2=0x7f11100b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11100c4de0 0x7f11100b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 -- 172.21.15.73:0/1525347365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100009400 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1115000640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00cf90 tx=0x7f10fc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00d840 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00d9a0 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00dd30 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.385+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11100c3880 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.386+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11100c3e40 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.387+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1110066af0 con 0x7f11101587d0 2024-04-05T22:22:16.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.388+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10fc015440 con 0x7f11101587d0 2024-04-05T22:22:16.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.388+0000 7f110d600640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 0x7f10f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.389+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f10fc089a00 con 0x7f11101587d0 2024-04-05T22:22:16.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.389+0000 7f1115a00640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 0x7f10f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.390+0000 7f1115a00640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 0x7f10f006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1100036000 tx=0x7f1100030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.393+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f10fc054fd0 con 0x7f11101587d0 2024-04-05T22:22:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.663+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f11100c4360 con 0x7f11101587d0 2024-04-05T22:22:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.664+0000 7f110d600640 1 -- 172.21.15.73:0/1525347365 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f11100c4360 con 0x7f11101587d0 2024-04-05T22:22:16.664 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.665 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:22:16.671 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:22:16.673 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:22:16.674 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:22:16.675 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mds", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBmk73RIBAA7zar0Gzi9vlEPYGeNUbl3g==", 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mds" 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mgr", 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBmgN3RIBAA1U96+DDkQ4aLkF/tW/vXhA==", 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-osd", 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBm8vfRIBAAhj5Zr3T11sJwN6DxbzeLYA==", 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-osd" 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd", 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBmyBfSIBAAro1sZTSX2cTmzSpRfNi2VA==", 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBmATLSIBAAd1aJgDQOnBmuFIAYB8BeqA==", 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rgw", 2024-04-05T22:22:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBHeRBmtL3SIBAAcykDuSiUcN0Bg2vffu6Yfw==", 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:22:16.680 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:22:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 msgr2=0x7f10f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 0x7f10f006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1100036000 tx=0x7f1100030000 comp rx=0 tx=0).stop 2024-04-05T22:22:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 msgr2=0x7f1110156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00cf90 tx=0x7f10fc00a7d0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 shutdown_connections 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f10f006ac00 0x7f10f006d0c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11100c4de0 0x7f11100b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11101587d0 0x7f1110156710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 --2- 172.21.15.73:0/1525347365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111006d0c0 0x7f11100c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 >> 172.21.15.73:0/1525347365 conn(0x7f11100ce8e0 msgr2=0x7f111015a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 shutdown_connections 2024-04-05T22:22:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.667+0000 7f1118400640 1 -- 172.21.15.73:0/1525347365 wait complete. 2024-04-05T22:22:16.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-05T22:22:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.888+0000 7f6d03200640 1 Processor -- start 2024-04-05T22:22:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.888+0000 7f6d03200640 1 -- start start 2024-04-05T22:22:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d03200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d0415a8f0 0x7f6d040cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d03200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d0415acd0 0x7f6d040ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d03200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d04070f40 con 0x7f6d040cd990 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d03200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0415e7b0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d03200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04071190 con 0x7f6d0415acd0 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d0415a8f0 0x7f6d040cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d0415a8f0 0x7f6d040cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49288/0 (socket says 172.21.15.73:49288) 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 -- 172.21.15.73:0/3398169297 learned_addr learned my addr 172.21.15.73:0/3398169297 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6cf9400640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d0415acd0 0x7f6d040ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 -- 172.21.15.73:0/3398169297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d0415acd0 msgr2=0x7f6d040ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d0415acd0 0x7f6d040ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 -- 172.21.15.73:0/3398169297 >> v1:172.21.15.157:6789/0 conn(0x7f6d040cd990 legacy=0x7f6d04069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.890+0000 7f6d01e00640 1 -- 172.21.15.73:0/3398169297 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d04071470 con 0x7f6d0415a8f0 2024-04-05T22:22:16.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d01e00640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d040cbf50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12959be06e03451c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d01400640 1 -- 172.21.15.73:0/3398169297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4031640 con 0x7f6d0415a8f0 2024-04-05T22:22:16.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d01400640 1 -- 172.21.15.73:0/3398169297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40317a0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d01400640 1 -- 172.21.15.73:0/3398169297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4031b30 con 0x7f6d0415a8f0 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d03200640 1 -- 172.21.15.73:0/3398169297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 msgr2=0x7f6d040cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.892+0000 7f6d03200640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d040cbf50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 -- 172.21.15.73:0/3398169297 shutdown_connections 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d0415acd0 0x7f6d040ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 --2- 172.21.15.73:0/3398169297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d040cbf50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 -- 172.21.15.73:0/3398169297 >> 172.21.15.73:0/3398169297 conn(0x7f6d04066140 msgr2=0x7f6d0406adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:16.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 -- 172.21.15.73:0/3398169297 shutdown_connections 2024-04-05T22:22:16.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.893+0000 7f6d03200640 1 -- 172.21.15.73:0/3398169297 wait complete. 2024-04-05T22:22:16.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.894+0000 7f6d03200640 1 Processor -- start 2024-04-05T22:22:16.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.894+0000 7f6d03200640 1 -- start start 2024-04-05T22:22:16.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d040cd990 0x7f6d041f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d0415acd0 0x7f6d040b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d041522c0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0415c420 con 0x7f6d0415acd0 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.895+0000 7f6d03200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04151a20 con 0x7f6d040cd990 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49292/0 (socket says 172.21.15.73:49292) 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 -- 172.21.15.73:0/475229585 learned_addr learned my addr 172.21.15.73:0/475229585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 -- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d040cd990 msgr2=0x7f6d041f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d040cd990 0x7f6d041f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 -- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d0415acd0 msgr2=0x7f6d040b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d0415acd0 0x7f6d040b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 -- 172.21.15.73:0/475229585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40094d0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.896+0000 7f6cf9400640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c7c0 tx=0x7f6cec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.897+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec007820 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.897+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec007980 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.897+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d04151d00 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.897+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec007cf0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.897+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d04151690 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.898+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0415c280 con 0x7f6d0415a8f0 2024-04-05T22:22:16.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.899+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6cec0113f0 con 0x7f6d0415a8f0 2024-04-05T22:22:16.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.900+0000 7f6cfaa00640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 0x7f6ce006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:16.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.900+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6cec088d70 con 0x7f6d0415a8f0 2024-04-05T22:22:16.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.900+0000 7f6d01e00640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 0x7f6ce006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:16.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.901+0000 7f6d01e00640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 0x7f6ce006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4036000 tx=0x7f6cf4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:16.904+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d0415c280 con 0x7f6d0415a8f0 2024-04-05T22:22:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.164+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f6d0415c280 con 0x7f6d0415a8f0 2024-04-05T22:22:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.173+0000 7f6cfaa00640 1 -- 172.21.15.73:0/475229585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6d0415c280 con 0x7f6d0415a8f0 2024-04-05T22:22:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 msgr2=0x7f6ce006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 0x7f6ce006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4036000 tx=0x7f6cf4030000 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 msgr2=0x7f6d0414eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c7c0 tx=0x7f6cec00cc80 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 shutdown_connections 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6ce006ac00 0x7f6ce006d0c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d0415acd0 0x7f6d040b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0415a8f0 0x7f6d0414eee0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 --2- 172.21.15.73:0/475229585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d040cd990 0x7f6d041f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 >> 172.21.15.73:0/475229585 conn(0x7f6d04066140 msgr2=0x7f6d04069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 shutdown_connections 2024-04-05T22:22:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.176+0000 7f6d03200640 1 -- 172.21.15.73:0/475229585 wait complete. 2024-04-05T22:22:17.196 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-05T22:22:17.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.406+0000 7f2942e00640 1 Processor -- start 2024-04-05T22:22:17.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.407+0000 7f2942e00640 1 -- start start 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f2942e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f293c1587d0 0x7f293c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f2942e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f293c15e850 0x7f293c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f2942e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c06f5b0 con 0x7f293c0ccd20 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f2942e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c15f240 con 0x7f293c15e850 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f2942e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c070410 con 0x7f293c1587d0 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f293b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f293c0ccd20 0x7f293c06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49624/0 (socket says 172.21.15.73:49624) 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.408+0000 7f293b400640 1 -- 172.21.15.73:0/873964488 learned_addr learned my addr 172.21.15.73:0/873964488 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f2940e00640 1 --2- 172.21.15.73:0/873964488 >> v2:172.21.15.73:3300/0 conn(0x7f293c15e850 0x7f293c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f293be00640 1 --2- 172.21.15.73:0/873964488 >> v2:172.21.15.157:3300/0 conn(0x7f293c1587d0 0x7f293c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f293aa00640 1 -- 172.21.15.73:0/873964488 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245968359 0 0) 0x7f293c06f5b0 con 0x7f293c0ccd20 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f293aa00640 1 -- 172.21.15.73:0/873964488 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2920003520 con 0x7f293c0ccd20 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f2940e00640 1 -- 172.21.15.73:0/873964488 >> v2:172.21.15.157:3300/0 conn(0x7f293c1587d0 msgr2=0x7f293c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f2940e00640 1 --2- 172.21.15.73:0/873964488 >> v2:172.21.15.157:3300/0 conn(0x7f293c1587d0 0x7f293c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f2940e00640 1 -- 172.21.15.73:0/873964488 >> v1:172.21.15.73:6789/0 conn(0x7f293c0ccd20 legacy=0x7f293c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.409+0000 7f2940e00640 1 -- 172.21.15.73:0/873964488 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c158bb0 con 0x7f293c15e850 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.410+0000 7f2940e00640 1 --2- 172.21.15.73:0/873964488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c15ed00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f292c009d90 tx=0x7f292c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bcef2940ffb779 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f293aa00640 1 -- 172.21.15.73:0/873964488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c02cc60 con 0x7f293c15e850 2024-04-05T22:22:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f293aa00640 1 -- 172.21.15.73:0/873964488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c02cdc0 con 0x7f293c15e850 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f293aa00640 1 -- 172.21.15.73:0/873964488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c034660 con 0x7f293c15e850 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 -- 172.21.15.73:0/873964488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 msgr2=0x7f293c15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 --2- 172.21.15.73:0/873964488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c15ed00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f292c009d90 tx=0x7f292c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 -- 172.21.15.73:0/873964488 shutdown_connections 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 --2- 172.21.15.73:0/873964488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c15ed00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 --2- 172.21.15.73:0/873964488 >> v2:172.21.15.157:3300/0 conn(0x7f293c1587d0 0x7f293c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 -- 172.21.15.73:0/873964488 >> 172.21.15.73:0/873964488 conn(0x7f293c0ce8e0 msgr2=0x7f293c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 -- 172.21.15.73:0/873964488 shutdown_connections 2024-04-05T22:22:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.411+0000 7f2942e00640 1 -- 172.21.15.73:0/873964488 wait complete. 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.413+0000 7f2942e00640 1 Processor -- start 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.413+0000 7f2942e00640 1 -- start start 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 0x7f293c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f293c1574b0 0x7f293c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c13fe40 con 0x7f293c15e850 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c13ec50 con 0x7f293c1587d0 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f2942e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c13eee0 con 0x7f293c1574b0 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f293b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f293b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49312/0 (socket says 172.21.15.73:49312) 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.414+0000 7f293b400640 1 -- 172.21.15.73:0/4265340678 learned_addr learned my addr 172.21.15.73:0/4265340678 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293be00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 0x7f293c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 -- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f293c1574b0 msgr2=0x7f293c157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f293c1574b0 0x7f293c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 -- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 msgr2=0x7f293c1423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 0x7f293c1423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 -- 172.21.15.73:0/4265340678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c009a70 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293be00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 0x7f293c1423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f293b400640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f292400c500 tx=0x7f292400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924005ce0 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2924005e40 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924007330 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.415+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c13f1c0 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.416+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c13f780 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.417+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c066af0 con 0x7f293c15e850 2024-04-05T22:22:17.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.418+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2924007550 con 0x7f293c15e850 2024-04-05T22:22:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.418+0000 7f2933e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 0x7f291806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.419+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f29240885c0 con 0x7f293c15e850 2024-04-05T22:22:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.419+0000 7f293be00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 0x7f291806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.419+0000 7f293be00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 0x7f291806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2934006fd0 tx=0x7f2934008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.423+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2924053c40 con 0x7f293c15e850 2024-04-05T22:22:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.688+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f293c13ff80 con 0x7f293c15e850 2024-04-05T22:22:17.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.696+0000 7f2933e00640 1 -- 172.21.15.73:0/4265340678 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f293c13ff80 con 0x7f293c15e850 2024-04-05T22:22:17.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 msgr2=0x7f291806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 0x7f291806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2934006fd0 tx=0x7f2934008040 comp rx=0 tx=0).stop 2024-04-05T22:22:17.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 msgr2=0x7f293c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f292400c500 tx=0x7f292400c9c0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 shutdown_connections 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f291806ac00 0x7f291806d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f293c1574b0 0x7f293c157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f293c15e850 0x7f293c0b28c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 --2- 172.21.15.73:0/4265340678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293c1587d0 0x7f293c1423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 >> 172.21.15.73:0/4265340678 conn(0x7f293c0ce8e0 msgr2=0x7f293c15b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 shutdown_connections 2024-04-05T22:22:17.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.699+0000 7f2942e00640 1 -- 172.21.15.73:0/4265340678 wait complete. 2024-04-05T22:22:17.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-05T22:22:17.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.925+0000 7f40c8a00640 1 Processor -- start 2024-04-05T22:22:17.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.926+0000 7f40c8a00640 1 -- start start 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.927+0000 7f40c8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40c4142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.927+0000 7f40c8a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40c413fe40 0x7f40c41429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.927+0000 7f40c8a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c4143330 con 0x7f40c406b500 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.927+0000 7f40c8a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c406d3a0 con 0x7f40c413fe40 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.927+0000 7f40c8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4143050 con 0x7f40c413fa60 2024-04-05T22:22:17.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f40c406b500 0x7f40c406f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49628/0 (socket says 172.21.15.73:49628) 2024-04-05T22:22:17.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c3e00640 1 -- 172.21.15.73:0/2241901446 learned_addr learned my addr 172.21.15.73:0/2241901446 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:17.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2a00640 1 --2- 172.21.15.73:0/2241901446 >> v2:172.21.15.157:3300/0 conn(0x7f40c413fe40 0x7f40c41429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c3400640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40c4142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725143512 0 0) 0x7f40c4143330 con 0x7f40c406b500 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a8003520 con 0x7f40c406b500 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3116477200 0 0) 0x7f40a8003520 con 0x7f40c406b500 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 msgr2=0x7f40c4142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40c4142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 >> v2:172.21.15.157:3300/0 conn(0x7f40c413fe40 msgr2=0x7f40c41429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 --2- 172.21.15.73:0/2241901446 >> v2:172.21.15.157:3300/0 conn(0x7f40c413fe40 0x7f40c41429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.928+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40ac970 con 0x7f40c406b500 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f40bc0019f0 con 0x7f40c406b500 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 >> v1:172.21.15.73:6789/0 conn(0x7f40c406b500 legacy=0x7f40c406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a8003970 0x7f40a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8004d00 0x7f40a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c406d3a0 con 0x7f40a8003970 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4143050 con 0x7f40a8004d00 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800b980 con 0x7f40c413fa60 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a8003970 0x7f40a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 -- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 msgr2=0x7f40a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3e00640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8004d00 0x7f40a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2a00640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 -- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8004d00 msgr2=0x7f40a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8004d00 0x7f40a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3400640 1 -- 172.21.15.73:0/2241901446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8003520 con 0x7f40a8003970 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c3e00640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a8004d00 0x7f40a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.929+0000 7f40c2a00640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c413fa60 0x7f40a80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.931+0000 7f40c3400640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a8003970 0x7f40a8003d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ac0 tx=0x7f40a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b49e8909579e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.931+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b402c6b0 con 0x7f40a8003970 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.931+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b402c810 con 0x7f40a8003970 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.931+0000 7f40c2000640 1 -- 172.21.15.73:0/2241901446 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b402cba0 con 0x7f40a8003970 2024-04-05T22:22:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.931+0000 7f40c8a00640 1 -- 172.21.15.73:0/2241901446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a8003970 msgr2=0x7f40a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c8a00640 1 --2- 172.21.15.73:0/2241901446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a8003970 0x7f40a8003d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ac0 tx=0x7f40a80057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c3e00640 1 -- 172.21.15.73:0/2241901446 reap_dead start 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c8a00640 1 -- 172.21.15.73:0/2241901446 shutdown_connections 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c8a00640 1 -- 172.21.15.73:0/2241901446 >> 172.21.15.73:0/2241901446 conn(0x7f40c40b06c0 msgr2=0x7f40c40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c8a00640 1 -- 172.21.15.73:0/2241901446 shutdown_connections 2024-04-05T22:22:17.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.932+0000 7f40c8a00640 1 -- 172.21.15.73:0/2241901446 wait complete. 2024-04-05T22:22:17.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.933+0000 7f40c8a00640 1 Processor -- start 2024-04-05T22:22:17.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.934+0000 7f40c8a00640 1 -- start start 2024-04-05T22:22:17.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.934+0000 7f40c8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40a8003970 0x7f40c41f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.934+0000 7f40c8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c413dd00 0x7f40c413e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c413d870 con 0x7f40c415d0a0 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c413cc60 con 0x7f40c413dd00 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c413cf40 con 0x7f40a8003970 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49336/0 (socket says 172.21.15.73:49336) 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 -- 172.21.15.73:0/986010391 learned_addr learned my addr 172.21.15.73:0/986010391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 -- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40a8003970 msgr2=0x7f40c41f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40a8003970 0x7f40c41f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c3e00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c413dd00 0x7f40c413e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 -- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c413dd00 msgr2=0x7f40c413e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c413dd00 0x7f40c413e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.935+0000 7f40c2a00640 1 -- 172.21.15.73:0/986010391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c413d220 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40c2a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0109d0 tx=0x7f40ac010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac00ec20 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac00ed80 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c413d500 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01c620 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.936+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c415e460 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.937+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c404e160 con 0x7f40c415d0a0 2024-04-05T22:22:17.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.938+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f40ac01c780 con 0x7f40c415d0a0 2024-04-05T22:22:17.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.939+0000 7f40bb400640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 0x7f40a006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:17.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.939+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40ac0895f0 con 0x7f40c415d0a0 2024-04-05T22:22:17.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.939+0000 7f40c3400640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 0x7f40a006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:17.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.940+0000 7f40c3400640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 0x7f40a006d0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40b4036000 tx=0x7f40b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:17.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:17.943+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f40ac054b30 con 0x7f40c415d0a0 2024-04-05T22:22:18.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.213+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f40c413d9b0 con 0x7f40c415d0a0 2024-04-05T22:22:18.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.221+0000 7f40bb400640 1 -- 172.21.15.73:0/986010391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f40ac058150 con 0x7f40c415d0a0 2024-04-05T22:22:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 msgr2=0x7f40a006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 0x7f40a006d0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40b4036000 tx=0x7f40b402e000 comp rx=0 tx=0).stop 2024-04-05T22:22:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 msgr2=0x7f40c415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0109d0 tx=0x7f40ac010e90 comp rx=0 tx=0).stop 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 shutdown_connections 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f40a006ac30 0x7f40a006d0f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c413dd00 0x7f40c413e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c415d0a0 0x7f40c415df20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 --2- 172.21.15.73:0/986010391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40a8003970 0x7f40c41f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 >> 172.21.15.73:0/986010391 conn(0x7f40c40b06c0 msgr2=0x7f40c40ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 shutdown_connections 2024-04-05T22:22:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.224+0000 7f40c8a00640 1 -- 172.21.15.73:0/986010391 wait complete. 2024-04-05T22:22:18.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-05T22:22:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.437+0000 7fedd1a00640 1 Processor -- start 2024-04-05T22:22:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.438+0000 7fedd1a00640 1 -- start start 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.439+0000 7fedd1a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedcc15ccd0 0x7fedcc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.439+0000 7fedd1a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 0x7fedcc0d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.439+0000 7fedd1a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0b2420 con 0x7fedcc0d35e0 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.439+0000 7fedd1a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0b2770 con 0x7fedcc15ccd0 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.439+0000 7fedd1a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0ce380 con 0x7fedcc0d1600 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedcc15ccd0 0x7fedcc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedcc15ccd0 0x7fedcc15d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49350/0 (socket says 172.21.15.73:49350) 2024-04-05T22:22:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 -- 172.21.15.73:0/1967818556 learned_addr learned my addr 172.21.15.73:0/1967818556 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedd1000640 1 --2- 172.21.15.73:0/1967818556 >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 0x7fedcc0d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 -- 172.21.15.73:0/1967818556 >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 msgr2=0x7fedcc0d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 --2- 172.21.15.73:0/1967818556 >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 0x7fedcc0d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 -- 172.21.15.73:0/1967818556 >> v1:172.21.15.157:6789/0 conn(0x7fedcc0d35e0 legacy=0x7fedcc0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedcbe00640 1 -- 172.21.15.73:0/1967818556 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0ce660 con 0x7fedcc15ccd0 2024-04-05T22:22:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.440+0000 7fedd1000640 1 --2- 172.21.15.73:0/1967818556 >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 0x7fedcc0d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.441+0000 7fedcbe00640 1 --2- 172.21.15.73:0/1967818556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc15ccd0 0x7fedcc15d0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fedcc08cff0 tx=0x7fedbc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf95e9d2782bf02 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedcaa00640 1 -- 172.21.15.73:0/1967818556 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc031750 con 0x7fedcc15ccd0 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedcaa00640 1 -- 172.21.15.73:0/1967818556 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0318b0 con 0x7fedcc15ccd0 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedcaa00640 1 -- 172.21.15.73:0/1967818556 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc031c40 con 0x7fedcc15ccd0 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 -- 172.21.15.73:0/1967818556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc15ccd0 msgr2=0x7fedcc15d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 --2- 172.21.15.73:0/1967818556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc15ccd0 0x7fedcc15d0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fedcc08cff0 tx=0x7fedbc02b7f0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 -- 172.21.15.73:0/1967818556 shutdown_connections 2024-04-05T22:22:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 --2- 172.21.15.73:0/1967818556 >> v2:172.21.15.157:3300/0 conn(0x7fedcc0d1600 0x7fedcc0d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 --2- 172.21.15.73:0/1967818556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc15ccd0 0x7fedcc15d0b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 -- 172.21.15.73:0/1967818556 >> 172.21.15.73:0/1967818556 conn(0x7fedcc0d2ac0 msgr2=0x7fedcc0d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 -- 172.21.15.73:0/1967818556 shutdown_connections 2024-04-05T22:22:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.442+0000 7fedd1a00640 1 -- 172.21.15.73:0/1967818556 wait complete. 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.444+0000 7fedd1a00640 1 Processor -- start 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.444+0000 7fedd1a00640 1 -- start start 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0d1600 0x7fedcc09d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedcc15ccd0 0x7fedcc14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0a0890 con 0x7fedcc0d35e0 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc09fdc0 con 0x7fedcc0d1600 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedd1a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0cb030 con 0x7fedcc15ccd0 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedcb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedcb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49364/0 (socket says 172.21.15.73:49364) 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.445+0000 7fedcb400640 1 -- 172.21.15.73:0/964801120 learned_addr learned my addr 172.21.15.73:0/964801120 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcbe00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0d1600 0x7fedcc09d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 -- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedcc15ccd0 msgr2=0x7fedcc14ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedd1000640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedcc15ccd0 0x7fedcc14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedcc15ccd0 0x7fedcc14ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 -- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0d1600 msgr2=0x7fedcc09d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0d1600 0x7fedcc09d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 -- 172.21.15.73:0/964801120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0094d0 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedcb400640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c460 tx=0x7fedb800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8012830 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.446+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80129d0 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.447+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8012d80 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.447+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc14d3d0 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.447+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0a02d0 con 0x7fedcc0d35e0 2024-04-05T22:22:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.448+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc0cae00 con 0x7fedcc0d35e0 2024-04-05T22:22:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.449+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fedb801b400 con 0x7fedcc0d35e0 2024-04-05T22:22:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.449+0000 7fedc3e00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 0x7fedac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.449+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fedb8088ff0 con 0x7fedcc0d35e0 2024-04-05T22:22:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.450+0000 7fedcbe00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 0x7fedac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.450+0000 7fedcbe00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 0x7fedac06d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedbc035000 tx=0x7fedbc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.454+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fedb80bc650 con 0x7fedcc0d35e0 2024-04-05T22:22:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.719+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fedcc067250 con 0x7fedcc0d35e0 2024-04-05T22:22:18.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.726+0000 7fedc3e00640 1 -- 172.21.15.73:0/964801120 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fedcc0cae00 con 0x7fedcc0d35e0 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 msgr2=0x7fedac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 0x7fedac06d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedbc035000 tx=0x7fedbc008030 comp rx=0 tx=0).stop 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 msgr2=0x7fedcc09f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c460 tx=0x7fedb800c920 comp rx=0 tx=0).stop 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 shutdown_connections 2024-04-05T22:22:18.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fedac06ac00 0x7fedac06d0c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedcc15ccd0 0x7fedcc14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedcc0d35e0 0x7fedcc09f880 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 --2- 172.21.15.73:0/964801120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedcc0d1600 0x7fedcc09d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 >> 172.21.15.73:0/964801120 conn(0x7fedcc0d2ac0 msgr2=0x7fedcc0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 shutdown_connections 2024-04-05T22:22:18.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.729+0000 7fedd1a00640 1 -- 172.21.15.73:0/964801120 wait complete. 2024-04-05T22:22:18.746 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-05T22:22:18.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.946+0000 7fd2ad600640 1 Processor -- start 2024-04-05T22:22:18.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.946+0000 7fd2ad600640 1 -- start start 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.947+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd2a80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.947+0000 7fd2ad600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2a80ce000 0x7fd2a815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.947+0000 7fd2ad600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a815aa60 con 0x7fd2a806d710 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.947+0000 7fd2ad600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a80b3b60 con 0x7fd2a80ce000 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.947+0000 7fd2ad600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80701c0 con 0x7fd2a81541a0 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd29fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd2a806d710 0x7fd2a80cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49638/0 (socket says 172.21.15.73:49638) 2024-04-05T22:22:18.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd29fe00640 1 -- 172.21.15.73:0/2795080609 learned_addr learned my addr 172.21.15.73:0/2795080609 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:18.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279920588 0 0) 0x7fd2a815aa60 con 0x7fd2a806d710 2024-04-05T22:22:18.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c005730 con 0x7fd2a806d710 2024-04-05T22:22:18.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a7400640 1 --2- 172.21.15.73:0/2795080609 >> v2:172.21.15.157:3300/0 conn(0x7fd2a80ce000 0x7fd2a815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6a00640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd2a80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702512506 0 0) 0x7fd28c005730 con 0x7fd2a806d710 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 msgr2=0x7fd2a80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd2a80b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 >> v2:172.21.15.157:3300/0 conn(0x7fd2a80ce000 msgr2=0x7fd2a815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 --2- 172.21.15.73:0/2795080609 >> v2:172.21.15.157:3300/0 conn(0x7fd2a80ce000 0x7fd2a815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80b3880 con 0x7fd2a806d710 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fd290001670 con 0x7fd2a806d710 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 >> v1:172.21.15.73:6789/0 conn(0x7fd2a806d710 legacy=0x7fd2a80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.948+0000 7fd2a6000640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd28c005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6000640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c0061b0 0x7fd28c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6000640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c006f10 0x7fd28c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b3b60 con 0x7fd28c006f10 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80701c0 con 0x7fd28c0061b0 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00dc90 con 0x7fd2a81541a0 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c006f10 0x7fd28c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd29fe00640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c0061b0 0x7fd28c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a6a00640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd28c005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 -- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 msgr2=0x7fd28c005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a81541a0 0x7fd28c005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 -- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c0061b0 msgr2=0x7fd28c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c0061b0 0x7fd28c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd2a7400640 1 -- 172.21.15.73:0/2795080609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c005730 con 0x7fd28c006f10 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.949+0000 7fd29fe00640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c0061b0 0x7fd28c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:18.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.950+0000 7fd2a7400640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c006f10 0x7fd28c007390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007360 tx=0x7fd28c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860209764770731c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.950+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a002ccf0 con 0x7fd28c006f10 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a002ce50 con 0x7fd28c006f10 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2a6000640 1 -- 172.21.15.73:0/2795080609 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0034660 con 0x7fd28c006f10 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 -- 172.21.15.73:0/2795080609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c006f10 msgr2=0x7fd28c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 --2- 172.21.15.73:0/2795080609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c006f10 0x7fd28c007390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007360 tx=0x7fd28c007a00 comp rx=0 tx=0).stop 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2a7400640 1 -- 172.21.15.73:0/2795080609 reap_dead start 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 -- 172.21.15.73:0/2795080609 shutdown_connections 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 -- 172.21.15.73:0/2795080609 >> 172.21.15.73:0/2795080609 conn(0x7fd2a80ce8e0 msgr2=0x7fd2a80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 -- 172.21.15.73:0/2795080609 shutdown_connections 2024-04-05T22:22:18.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.951+0000 7fd2ad600640 1 -- 172.21.15.73:0/2795080609 wait complete. 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.953+0000 7fd2ad600640 1 Processor -- start 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.953+0000 7fd2ad600640 1 -- start start 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a8159b60 0x7fd2a80a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8151390 0x7fd2a80a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81411e0 con 0x7fd2a80a8f40 2024-04-05T22:22:18.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8141a80 con 0x7fd2a8151390 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2ad600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80aa5e0 con 0x7fd2a8159b60 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49382/0 (socket says 172.21.15.73:49382) 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.954+0000 7fd2a7400640 1 -- 172.21.15.73:0/1724325907 learned_addr learned my addr 172.21.15.73:0/1724325907 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a6a00640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a8159b60 0x7fd2a80a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 -- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a8159b60 msgr2=0x7fd2a80a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a8159b60 0x7fd2a80a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 -- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8151390 msgr2=0x7fd2a80a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8151390 0x7fd2a80a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 -- 172.21.15.73:0/1724325907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80ad140 con 0x7fd2a80a8f40 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2a7400640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007480 tx=0x7fd2a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003d600 con 0x7fd2a80a8f40 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a003d760 con 0x7fd2a80a8f40 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003db10 con 0x7fd2a80a8f40 2024-04-05T22:22:18.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8141430 con 0x7fd2a80a8f40 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.955+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a80acdc0 con 0x7fd2a80a8f40 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.956+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8066af0 con 0x7fd2a80a8f40 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.957+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd2a003b040 con 0x7fd2a80a8f40 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.959+0000 7fd29ea00640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 0x7fd28406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.959+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd2a00aa3a0 con 0x7fd2a80a8f40 2024-04-05T22:22:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.959+0000 7fd2a6a00640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 0x7fd28406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:18.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.959+0000 7fd2a6a00640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 0x7fd28406d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81f1e40 tx=0x7fd298008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:18.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:18.963+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd2a0075940 con 0x7fd2a80a8f40 2024-04-05T22:22:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.212+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd2a8068160 con 0x7fd2a80a8f40 2024-04-05T22:22:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.219+0000 7fd29ea00640 1 -- 172.21.15.73:0/1724325907 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd2a00390a0 con 0x7fd2a80a8f40 2024-04-05T22:22:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.222+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 msgr2=0x7fd28406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.222+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 0x7fd28406d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81f1e40 tx=0x7fd298008040 comp rx=0 tx=0).stop 2024-04-05T22:22:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.222+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 msgr2=0x7fd2a8143270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.222+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007480 tx=0x7fd2a0004ab0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 shutdown_connections 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd28406ac00 0x7fd28406d0c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2a80a8f40 0x7fd2a8143270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8151390 0x7fd2a80a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 --2- 172.21.15.73:0/1724325907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2a8159b60 0x7fd2a80a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 >> 172.21.15.73:0/1724325907 conn(0x7fd2a80ce8e0 msgr2=0x7fd2a806f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 shutdown_connections 2024-04-05T22:22:19.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.223+0000 7fd2ad600640 1 -- 172.21.15.73:0/1724325907 wait complete. 2024-04-05T22:22:19.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-05T22:22:19.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.436+0000 7fb7f4e00640 1 Processor -- start 2024-04-05T22:22:19.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.436+0000 7fb7f4e00640 1 -- start start 2024-04-05T22:22:19.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.437+0000 7fb7f4e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7f015e620 0x7fb7f015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.437+0000 7fb7f4e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7f006bc70 0x7fb7f006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.437+0000 7fb7f4e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f006b500 con 0x7fb7f0158860 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.437+0000 7fb7f4e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f015eff0 con 0x7fb7f015e620 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7f4e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00ccaf0 con 0x7fb7f006bc70 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7f015e620 0x7fb7f015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7f015e620 0x7fb7f015eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49388/0 (socket says 172.21.15.73:49388) 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 -- 172.21.15.73:0/4213182473 learned_addr learned my addr 172.21.15.73:0/4213182473 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7efe00640 1 --2- 172.21.15.73:0/4213182473 >> v2:172.21.15.157:3300/0 conn(0x7fb7f006bc70 0x7fb7f006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 -- 172.21.15.73:0/4213182473 >> v2:172.21.15.157:3300/0 conn(0x7fb7f006bc70 msgr2=0x7fb7f006c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 --2- 172.21.15.73:0/4213182473 >> v2:172.21.15.157:3300/0 conn(0x7fb7f006bc70 0x7fb7f006c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 -- 172.21.15.73:0/4213182473 >> v1:172.21.15.157:6789/0 conn(0x7fb7f0158860 legacy=0x7fb7f00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.438+0000 7fb7e7e00640 1 -- 172.21.15.73:0/4213182473 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00b20d0 con 0x7fb7f015e620 2024-04-05T22:22:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.440+0000 7fb7e7e00640 1 --2- 172.21.15.73:0/4213182473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f015e620 0x7fb7f015eab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00b16a0 tx=0x7fb7d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe515d15264e234 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.440+0000 7fb7eea00640 1 -- 172.21.15.73:0/4213182473 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80317e0 con 0x7fb7f015e620 2024-04-05T22:22:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.440+0000 7fb7eea00640 1 -- 172.21.15.73:0/4213182473 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8031940 con 0x7fb7f015e620 2024-04-05T22:22:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.440+0000 7fb7eea00640 1 -- 172.21.15.73:0/4213182473 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8031cd0 con 0x7fb7f015e620 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 -- 172.21.15.73:0/4213182473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f015e620 msgr2=0x7fb7f015eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/4213182473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f015e620 0x7fb7f015eab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00b16a0 tx=0x7fb7d802b880 comp rx=0 tx=0).stop 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 -- 172.21.15.73:0/4213182473 shutdown_connections 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/4213182473 >> v2:172.21.15.157:3300/0 conn(0x7fb7f006bc70 0x7fb7f006c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/4213182473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f015e620 0x7fb7f015eab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 -- 172.21.15.73:0/4213182473 >> 172.21.15.73:0/4213182473 conn(0x7fb7f00b06c0 msgr2=0x7fb7f00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 -- 172.21.15.73:0/4213182473 shutdown_connections 2024-04-05T22:22:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.441+0000 7fb7f4e00640 1 -- 172.21.15.73:0/4213182473 wait complete. 2024-04-05T22:22:19.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.442+0000 7fb7f4e00640 1 Processor -- start 2024-04-05T22:22:19.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 -- start start 2024-04-05T22:22:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7f015e620 0x7fb7f0142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7f00ca590 0x7fb7f0157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00c9080 con 0x7fb7f006bc70 2024-04-05T22:22:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0146800 con 0x7fb7f015e620 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.443+0000 7fb7f4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00c9290 con 0x7fb7f00ca590 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49394/0 (socket says 172.21.15.73:49394) 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 -- 172.21.15.73:0/3529501151 learned_addr learned my addr 172.21.15.73:0/3529501151 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 -- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7f00ca590 msgr2=0x7fb7f0157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7f00ca590 0x7fb7f0157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 -- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7f015e620 msgr2=0x7fb7f0142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7f015e620 0x7fb7f0142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 -- 172.21.15.73:0/3529501151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80094d0 con 0x7fb7f006bc70 2024-04-05T22:22:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.444+0000 7fb7ef400640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000c460 tx=0x7fb7e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.445+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0012830 con 0x7fb7f006bc70 2024-04-05T22:22:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.445+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e00129d0 con 0x7fb7f006bc70 2024-04-05T22:22:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.445+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0012d80 con 0x7fb7f006bc70 2024-04-05T22:22:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.445+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f00c9570 con 0x7fb7f006bc70 2024-04-05T22:22:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.445+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f00c9b30 con 0x7fb7f006bc70 2024-04-05T22:22:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.446+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f00c9d20 con 0x7fb7f006bc70 2024-04-05T22:22:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.447+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb7e001a3b0 con 0x7fb7f006bc70 2024-04-05T22:22:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.447+0000 7fb7e7400640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.447+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7e0088cc0 con 0x7fb7f006bc70 2024-04-05T22:22:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.448+0000 7fb7e7e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.448+0000 7fb7e7e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8036000 tx=0x7fb7d8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.453+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb7e00541f0 con 0x7fb7f006bc70 2024-04-05T22:22:19.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.694+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb7f004e160 con 0x7fb7f006bc70 2024-04-05T22:22:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.700+0000 7fb7e7400640 1 -- 172.21.15.73:0/3529501151 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb7e0057810 con 0x7fb7f006bc70 2024-04-05T22:22:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 msgr2=0x7fb7cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8036000 tx=0x7fb7d8030000 comp rx=0 tx=0).stop 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 msgr2=0x7fb7f00ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000c460 tx=0x7fb7e000c920 comp rx=0 tx=0).stop 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 shutdown_connections 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb7cc06ac00 0x7fb7cc06d0c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7f00ca590 0x7fb7f0157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7f015e620 0x7fb7f0142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 --2- 172.21.15.73:0/3529501151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7f006bc70 0x7fb7f00ca050 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 >> 172.21.15.73:0/3529501151 conn(0x7fb7f00b06c0 msgr2=0x7fb7f00b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 shutdown_connections 2024-04-05T22:22:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.703+0000 7fb7f4e00640 1 -- 172.21.15.73:0/3529501151 wait complete. 2024-04-05T22:22:19.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:19.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.916+0000 7f1136800640 1 Processor -- start 2024-04-05T22:22:19.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.916+0000 7f1136800640 1 -- start start 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1136800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f113015ccd0 0x7f113015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1136800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11300d1600 0x7f11300d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1136800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300b2420 con 0x7f11300d35e0 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1136800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11300b2770 con 0x7f11300d1600 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1136800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300ce380 con 0x7f113015ccd0 2024-04-05T22:22:19.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1135e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11300d1600 0x7f11300d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1134a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f11300d35e0 0x7f11300d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49642/0 (socket says 172.21.15.73:49642) 2024-04-05T22:22:19.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1134a00640 1 -- 172.21.15.73:0/4268240887 learned_addr learned my addr 172.21.15.73:0/4268240887 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:19.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1135400640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f113015ccd0 0x7f113015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.918+0000 7f1135e00640 1 -- 172.21.15.73:0/4268240887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f113015ccd0 msgr2=0x7f113015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:19.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.919+0000 7f1135e00640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f113015ccd0 0x7f113015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.919+0000 7f1135e00640 1 -- 172.21.15.73:0/4268240887 >> v1:172.21.15.73:6789/0 conn(0x7f11300d35e0 legacy=0x7f11300d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.919+0000 7f1135e00640 1 -- 172.21.15.73:0/4268240887 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11300ce660 con 0x7f11300d1600 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f1135e00640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300d08a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1128009d90 tx=0x7f112802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181668f56e3051d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f112fe00640 1 -- 172.21.15.73:0/4268240887 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112802cb40 con 0x7f11300d1600 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f112fe00640 1 -- 172.21.15.73:0/4268240887 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112802cca0 con 0x7f11300d1600 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f112fe00640 1 -- 172.21.15.73:0/4268240887 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128034660 con 0x7f11300d1600 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f1136800640 1 -- 172.21.15.73:0/4268240887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 msgr2=0x7f11300d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.920+0000 7f1136800640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300d08a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1128009d90 tx=0x7f112802bd90 comp rx=0 tx=0).stop 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 -- 172.21.15.73:0/4268240887 shutdown_connections 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300d08a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 --2- 172.21.15.73:0/4268240887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f113015ccd0 0x7f113015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 -- 172.21.15.73:0/4268240887 >> 172.21.15.73:0/4268240887 conn(0x7f11300d2ac0 msgr2=0x7f11300d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:19.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 -- 172.21.15.73:0/4268240887 shutdown_connections 2024-04-05T22:22:19.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.921+0000 7f1136800640 1 -- 172.21.15.73:0/4268240887 wait complete. 2024-04-05T22:22:19.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.922+0000 7f1136800640 1 Processor -- start 2024-04-05T22:22:19.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.922+0000 7f1136800640 1 -- start start 2024-04-05T22:22:19.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11300d35e0 0x7f11300b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f113015ccd0 0x7f113009f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300a0610 con 0x7f11300d1600 2024-04-05T22:22:19.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300a0050 con 0x7f113015ccd0 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.923+0000 7f1136800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300a0330 con 0x7f11300d35e0 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49406/0 (socket says 172.21.15.73:49406) 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 -- 172.21.15.73:0/630435873 learned_addr learned my addr 172.21.15.73:0/630435873 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 -- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11300d35e0 msgr2=0x7f11300b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11300d35e0 0x7f11300b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 -- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f113015ccd0 msgr2=0x7f113009f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f113015ccd0 0x7f113009f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 -- 172.21.15.73:0/630435873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128009a70 con 0x7f11300d1600 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.924+0000 7f1135400640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f112400c7f0 tx=0x7f112400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.925+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124004220 con 0x7f11300d1600 2024-04-05T22:22:19.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.925+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1124004380 con 0x7f11300d1600 2024-04-05T22:22:19.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.925+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124007840 con 0x7f11300d1600 2024-04-05T22:22:19.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.925+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11300cb170 con 0x7f11300d1600 2024-04-05T22:22:19.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.925+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113014d290 con 0x7f11300d1600 2024-04-05T22:22:19.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.926+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11300cb690 con 0x7f11300d1600 2024-04-05T22:22:19.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.927+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1124007bb0 con 0x7f11300d1600 2024-04-05T22:22:19.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.928+0000 7f112d600640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 0x7f111006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:19.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.928+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1124088e50 con 0x7f11300d1600 2024-04-05T22:22:19.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.928+0000 7f1134a00640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 0x7f111006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:19.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.929+0000 7f1134a00640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 0x7f111006d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f113009cb50 tx=0x7f111c007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:19.932+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f11300cb690 con 0x7f11300d1600 2024-04-05T22:22:20.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.126+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11300cb690 con 0x7f11300d1600 2024-04-05T22:22:20.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.128+0000 7f112d600640 1 -- 172.21.15.73:0/630435873 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f11300cb690 con 0x7f11300d1600 2024-04-05T22:22:20.128 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:20.128 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":32,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:16.075252+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.130+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 msgr2=0x7f111006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.130+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 0x7f111006d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f113009cb50 tx=0x7f111c007a40 comp rx=0 tx=0).stop 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.130+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 msgr2=0x7f11300b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.130+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f112400c7f0 tx=0x7f112400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 shutdown_connections 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f111006ac00 0x7f111006d0c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f113015ccd0 0x7f113009f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11300d35e0 0x7f11300b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 --2- 172.21.15.73:0/630435873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11300d1600 0x7f11300b3270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 >> 172.21.15.73:0/630435873 conn(0x7f11300d2ac0 msgr2=0x7f11300b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:20.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 shutdown_connections 2024-04-05T22:22:20.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.131+0000 7f1136800640 1 -- 172.21.15.73:0/630435873 wait complete. 2024-04-05T22:22:20.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:20.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:20.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.347+0000 7f5a19c00640 1 Processor -- start 2024-04-05T22:22:20.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.347+0000 7f5a19c00640 1 -- start start 2024-04-05T22:22:20.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.348+0000 7f5a19c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a14070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.348+0000 7f5a19c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140ae190 con 0x7f5a141599f0 2024-04-05T22:22:20.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.348+0000 7f5a19c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140b27f0 con 0x7f5a140add00 2024-04-05T22:22:20.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.348+0000 7f5a19c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b3890 con 0x7f5a1406cd50 2024-04-05T22:22:20.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.349+0000 7f5a12a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5a141599f0 0x7f5a1406c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49644/0 (socket says 172.21.15.73:49644) 2024-04-05T22:22:20.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.349+0000 7f5a12a00640 1 -- 172.21.15.73:0/163746252 learned_addr learned my addr 172.21.15.73:0/163746252 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.349+0000 7f5a0a000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a14070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.349+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040467780 0 0) 0x7f5a140ae190 con 0x7f5a141599f0 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.349+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f8003520 con 0x7f5a141599f0 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434423715 0 0) 0x7f59f8003520 con 0x7f5a141599f0 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 msgr2=0x7f5a14070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a14070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 >> v1:172.21.15.157:6789/0 conn(0x7f5a140add00 legacy=0x7f5a14159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140b2ad0 con 0x7f5a141599f0 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5a04001670 con 0x7f5a141599f0 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 >> v1:172.21.15.73:6789/0 conn(0x7f5a141599f0 legacy=0x7f5a1406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f80039a0 0x7f59f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f59f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8004d00 0x7f59f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b3890 con 0x7f59f8004d00 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f800ba60 con 0x7f59f80039a0 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f800de30 con 0x7f5a1406cd50 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a13400640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8004d00 0x7f59f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a0a000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f59f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.350+0000 7f5a13400640 1 -- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 msgr2=0x7f59f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:20.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.351+0000 7f5a13400640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f59f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.351+0000 7f5a13400640 1 -- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f80039a0 msgr2=0x7f59f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:20.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.351+0000 7f5a13400640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f80039a0 0x7f59f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.351+0000 7f5a13400640 1 -- 172.21.15.73:0/163746252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8003520 con 0x7f59f8004d00 2024-04-05T22:22:20.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.351+0000 7f5a0a000640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f59f80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:20.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a13400640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8004d00 0x7f59f8005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001e50 tx=0x7f59f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79267e6537d03e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c003b70 con 0x7f59f8004d00 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c003cd0 con 0x7f59f8004d00 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a12000640 1 -- 172.21.15.73:0/163746252 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c005310 con 0x7f59f8004d00 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a19c00640 1 -- 172.21.15.73:0/163746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8004d00 msgr2=0x7f59f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a19c00640 1 --2- 172.21.15.73:0/163746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8004d00 0x7f59f8005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001e50 tx=0x7f59f80057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a13400640 1 -- 172.21.15.73:0/163746252 reap_dead start 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a19c00640 1 -- 172.21.15.73:0/163746252 shutdown_connections 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.352+0000 7f5a19c00640 1 -- 172.21.15.73:0/163746252 >> 172.21.15.73:0/163746252 conn(0x7f5a14069cb0 msgr2=0x7f5a1406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.353+0000 7f5a19c00640 1 -- 172.21.15.73:0/163746252 shutdown_connections 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.353+0000 7f5a19c00640 1 -- 172.21.15.73:0/163746252 wait complete. 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.354+0000 7f5a19c00640 1 Processor -- start 2024-04-05T22:22:20.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.354+0000 7f5a19c00640 1 -- start start 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8004d00 0x7f5a14151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a141520f0 0x7f5a141544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415e010 con 0x7f5a141549f0 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14154e80 con 0x7f59f8004d00 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.355+0000 7f5a19c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415add0 con 0x7f5a141520f0 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49420/0 (socket says 172.21.15.73:49420) 2024-04-05T22:22:20.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 -- 172.21.15.73:0/1455607198 learned_addr learned my addr 172.21.15.73:0/1455607198 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 -- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a141520f0 msgr2=0x7f5a141544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a141520f0 0x7f5a141544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 -- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8004d00 msgr2=0x7f5a14151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8004d00 0x7f5a14151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 -- 172.21.15.73:0/1455607198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1415b0b0 con 0x7f5a141549f0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.356+0000 7f5a13400640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001f70 tx=0x7f5a0c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.357+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c006d50 con 0x7f5a141549f0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.357+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c003e40 con 0x7f5a141549f0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.357+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1415b390 con 0x7f5a141549f0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.357+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007270 con 0x7f5a141549f0 2024-04-05T22:22:20.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.357+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1415b950 con 0x7f5a141549f0 2024-04-05T22:22:20.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.358+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14068160 con 0x7f5a141549f0 2024-04-05T22:22:20.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.359+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a0c0073d0 con 0x7f5a141549f0 2024-04-05T22:22:20.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.360+0000 7f5a0b400640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.360+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5a0c0aa940 con 0x7f5a141549f0 2024-04-05T22:22:20.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.360+0000 7f5a12a00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.361+0000 7f5a12a00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 0x7f59f006d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a04000ff0 tx=0x7f5a040014d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.364+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5a0c075de0 con 0x7f5a141549f0 2024-04-05T22:22:20.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.558+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a140b2310 con 0x7f5a141549f0 2024-04-05T22:22:20.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.559+0000 7f5a0b400640 1 -- 172.21.15.73:0/1455607198 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a0c0470a0 con 0x7f5a141549f0 2024-04-05T22:22:20.559 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 msgr2=0x7f59f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 0x7f59f006d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a04000ff0 tx=0x7f5a040014d0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 msgr2=0x7f5a1415a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001f70 tx=0x7f5a0c0014a0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 shutdown_connections 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141549f0 0x7f5a1415a130 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a141520f0 0x7f5a141544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 --2- 172.21.15.73:0/1455607198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8004d00 0x7f5a14151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 >> 172.21.15.73:0/1455607198 conn(0x7f5a14069cb0 msgr2=0x7f5a140af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 shutdown_connections 2024-04-05T22:22:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.562+0000 7f5a19c00640 1 -- 172.21.15.73:0/1455607198 wait complete. 2024-04-05T22:22:20.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:22:20.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.787+0000 7f1165c00640 1 Processor -- start 2024-04-05T22:22:20.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.788+0000 7f1165c00640 1 -- start start 2024-04-05T22:22:20.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 0x7f11600b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f116006d710 0x7f11600cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116015aef0 0x7f11600cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116015aa60 con 0x7f116006d710 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600b3b60 con 0x7f11601541a0 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1165c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600701c0 con 0x7f116015aef0 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f116006d710 0x7f11600cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f116006d710 0x7f11600cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49436/0 (socket says 172.21.15.73:49436) 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 -- 172.21.15.73:0/26208049 learned_addr learned my addr 172.21.15.73:0/26208049 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f115ea00640 1 --2- 172.21.15.73:0/26208049 >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 0x7f11600b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f115f400640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116015aef0 0x7f11600cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 -- 172.21.15.73:0/26208049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116015aef0 msgr2=0x7f11600cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116015aef0 0x7f11600cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 -- 172.21.15.73:0/26208049 >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 msgr2=0x7f11600b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:20.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 --2- 172.21.15.73:0/26208049 >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 0x7f11600b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f1156000640 1 -- 172.21.15.73:0/26208049 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600b3880 con 0x7f116006d710 2024-04-05T22:22:20.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.790+0000 7f115ea00640 1 --2- 172.21.15.73:0/26208049 >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 0x7f11600b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:20.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.791+0000 7f1156000640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116006d710 0x7f11600cd8f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1160066850 tx=0x7f114802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf2afa4640a2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.792+0000 7f115e000640 1 -- 172.21.15.73:0/26208049 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114802ee90 con 0x7f116006d710 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.792+0000 7f115e000640 1 -- 172.21.15.73:0/26208049 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148036440 con 0x7f116006d710 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.792+0000 7f115e000640 1 -- 172.21.15.73:0/26208049 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11480367d0 con 0x7f116006d710 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.792+0000 7f1165c00640 1 -- 172.21.15.73:0/26208049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116006d710 msgr2=0x7f11600cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.792+0000 7f1165c00640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116006d710 0x7f11600cd8f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1160066850 tx=0x7f114802e0e0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 -- 172.21.15.73:0/26208049 shutdown_connections 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116006d710 0x7f11600cd8f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 --2- 172.21.15.73:0/26208049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116015aef0 0x7f11600cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 --2- 172.21.15.73:0/26208049 >> v2:172.21.15.157:3300/0 conn(0x7f11601541a0 0x7f11600b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 -- 172.21.15.73:0/26208049 >> 172.21.15.73:0/26208049 conn(0x7f11600ce8e0 msgr2=0x7f11600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 -- 172.21.15.73:0/26208049 shutdown_connections 2024-04-05T22:22:20.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.793+0000 7f1165c00640 1 -- 172.21.15.73:0/26208049 wait complete. 2024-04-05T22:22:20.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.794+0000 7f1165c00640 1 Processor -- start 2024-04-05T22:22:20.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 -- start start 2024-04-05T22:22:20.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 0x7f11600ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11601541a0 0x7f11600a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600aed50 con 0x7f116015aef0 2024-04-05T22:22:20.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600a8660 con 0x7f11601541a0 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.795+0000 7f1165c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600a8900 con 0x7f116006d710 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49440/0 (socket says 172.21.15.73:49440) 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 -- 172.21.15.73:0/3812033204 learned_addr learned my addr 172.21.15.73:0/3812033204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f1156000640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11601541a0 0x7f11600a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115ea00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 0x7f11600ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 -- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 msgr2=0x7f11600ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 0x7f11600ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 -- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11601541a0 msgr2=0x7f11600a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11601541a0 0x7f11600a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 -- 172.21.15.73:0/3812033204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114800b770 con 0x7f116015aef0 2024-04-05T22:22:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115ea00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 0x7f11600ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.796+0000 7f115f400640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1158007e40 tx=0x7f115800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.797+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158014620 con 0x7f116015aef0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.797+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115800ee90 con 0x7f116015aef0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.797+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580149e0 con 0x7f116015aef0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.797+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11600a8be0 con 0x7f116015aef0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.797+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11600aea40 con 0x7f116015aef0 2024-04-05T22:22:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.798+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160066af0 con 0x7f116015aef0 2024-04-05T22:22:20.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.799+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1158014b40 con 0x7f116015aef0 2024-04-05T22:22:20.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.800+0000 7f1157400640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 0x7f113c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:20.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.800+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1158089990 con 0x7f116015aef0 2024-04-05T22:22:20.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.800+0000 7f115ea00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 0x7f113c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:20.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.801+0000 7f115ea00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 0x7f113c06d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1150006fd0 tx=0x7f1150008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:20.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.804+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1158054eb0 con 0x7f116015aef0 2024-04-05T22:22:20.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:20.999+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f11600a9100 con 0x7f116015aef0 2024-04-05T22:22:21.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.000+0000 7f1157400640 1 -- 172.21.15.73:0/3812033204 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f11600a9100 con 0x7f116015aef0 2024-04-05T22:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:22:21.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.002+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 msgr2=0x7f113c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.002+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 0x7f113c06d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1150006fd0 tx=0x7f1150008040 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 msgr2=0x7f11600a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1158007e40 tx=0x7f115800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 shutdown_connections 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f113c06ac30 0x7f113c06d0f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f116015aef0 0x7f11600a9920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11601541a0 0x7f11600a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 --2- 172.21.15.73:0/3812033204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116006d710 0x7f11600ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 >> 172.21.15.73:0/3812033204 conn(0x7f11600ce8e0 msgr2=0x7f11600cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 shutdown_connections 2024-04-05T22:22:21.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.003+0000 7f1165c00640 1 -- 172.21.15.73:0/3812033204 wait complete. 2024-04-05T22:22:21.021 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:22:21.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:22:21.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.225+0000 7f5e1ee00640 1 Processor -- start 2024-04-05T22:22:21.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.225+0000 7f5e1ee00640 1 -- start start 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e1ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5e180cf3e0 0x7f5e180b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e1ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5e180b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e1ee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1815b230 con 0x7f5e1815ae50 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e1ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180cd320 con 0x7f5e180cf3e0 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e1ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180ce2f0 con 0x7f5e180cce90 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e17e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5e1815ae50 0x7f5e180cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49658/0 (socket says 172.21.15.73:49658) 2024-04-05T22:22:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.227+0000 7f5e17e00640 1 -- 172.21.15.73:0/446992550 learned_addr learned my addr 172.21.15.73:0/446992550 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279223952 0 0) 0x7f5e1815b230 con 0x7f5e1815ae50 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5df8003520 con 0x7f5e1815ae50 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e1ce00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5e180b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> v2:172.21.15.157:3300/0 conn(0x7f5e180cf3e0 0x7f5e180b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 563438255 0 0) 0x7f5df8003520 con 0x7f5e1815ae50 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 msgr2=0x7f5e180b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5e180b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 >> v2:172.21.15.157:3300/0 conn(0x7f5e180cf3e0 msgr2=0x7f5e180b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 --2- 172.21.15.73:0/446992550 >> v2:172.21.15.157:3300/0 conn(0x7f5e180cf3e0 0x7f5e180b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1815f4a0 con 0x7f5e1815ae50 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5e08001670 con 0x7f5e1815ae50 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e1ce00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5e180b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 >> v1:172.21.15.73:6789/0 conn(0x7f5e1815ae50 legacy=0x7f5e180cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8003970 0x7f5df8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df80047c0 0x7f5df8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.228+0000 7f5e16a00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5df800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180ce2f0 con 0x7f5df80047c0 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80054d0 con 0x7f5df8003970 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8005690 con 0x7f5e180cce90 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> v2:172.21.15.157:3300/0 conn(0x7f5e180cf3e0 0x7f5e180b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df80047c0 0x7f5df8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e1ce00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5df800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 -- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 msgr2=0x7f5df800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5df800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17e00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8003970 0x7f5df8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 -- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8003970 msgr2=0x7f5df8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8003970 0x7f5df8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17400640 1 -- 172.21.15.73:0/446992550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8003520 con 0x7f5df80047c0 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e1ce00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e180cce90 0x7f5df800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.229+0000 7f5e17e00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8003970 0x7f5df8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.230+0000 7f5e17400640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df80047c0 0x7f5df8004c40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5e180ae4b0 tx=0x7f5df80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552af9a01c2a6635 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00004220 con 0x7f5df80047c0 2024-04-05T22:22:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00004380 con 0x7f5df80047c0 2024-04-05T22:22:21.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e16a00640 1 -- 172.21.15.73:0/446992550 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00003830 con 0x7f5df80047c0 2024-04-05T22:22:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 -- 172.21.15.73:0/446992550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df80047c0 msgr2=0x7f5df8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/446992550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df80047c0 0x7f5df8004c40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5e180ae4b0 tx=0x7f5df80057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ce00640 1 -- 172.21.15.73:0/446992550 reap_dead start 2024-04-05T22:22:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 -- 172.21.15.73:0/446992550 shutdown_connections 2024-04-05T22:22:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 -- 172.21.15.73:0/446992550 >> 172.21.15.73:0/446992550 conn(0x7f5e1806a0d0 msgr2=0x7f5e1806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 -- 172.21.15.73:0/446992550 shutdown_connections 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.231+0000 7f5e1ee00640 1 -- 172.21.15.73:0/446992550 wait complete. 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.233+0000 7f5e1ee00640 1 Processor -- start 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.233+0000 7f5e1ee00640 1 -- start start 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df80047c0 0x7f5e1814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e180b0630 0x7f5e180b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1814cd80 con 0x7f5e1815e0f0 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b1000 con 0x7f5e180b0630 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e1ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180c8e00 con 0x7f5df80047c0 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49468/0 (socket says 172.21.15.73:49468) 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.234+0000 7f5e17400640 1 -- 172.21.15.73:0/2588740605 learned_addr learned my addr 172.21.15.73:0/2588740605 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17e00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df80047c0 0x7f5e1814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 -- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df80047c0 msgr2=0x7f5e1814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df80047c0 0x7f5e1814a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 -- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e180b0630 msgr2=0x7f5e180b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e180b0630 0x7f5e180b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 -- 172.21.15.73:0/2588740605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1814c200 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.235+0000 7f5e17400640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000a120 tx=0x7f5e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.236+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00037810 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.236+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0003d040 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.236+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00037b90 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.236+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1814c4e0 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.236+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1814caa0 con 0x7f5e1815e0f0 2024-04-05T22:22:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.237+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e180ad5a0 con 0x7f5e1815e0f0 2024-04-05T22:22:21.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.238+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5e00037cf0 con 0x7f5e1815e0f0 2024-04-05T22:22:21.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.239+0000 7f5e0fe00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 0x7f5df406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:21.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.239+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e000ab700 con 0x7f5e1815e0f0 2024-04-05T22:22:21.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.239+0000 7f5e17e00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 0x7f5df406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:21.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.239+0000 7f5e17e00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 0x7f5df406d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001eb0 tx=0x7f5e08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:21.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.243+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5e00076e00 con 0x7f5e1815e0f0 2024-04-05T22:22:21.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:21.437+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e180ad880 con 0x7f5e1815e0f0 2024-04-05T22:22:21.745 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-05T22:22:21.745 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-05T22:22:22.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.117+0000 7f5e0fe00640 1 -- 172.21.15.73:0/2588740605 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e0007a420 con 0x7f5e1815e0f0 2024-04-05T22:22:22.118 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:22:22.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 msgr2=0x7f5df406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:22.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 0x7f5df406d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001eb0 tx=0x7f5e08009040 comp rx=0 tx=0).stop 2024-04-05T22:22:22.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 msgr2=0x7f5e1814b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:22.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000a120 tx=0x7f5e00004730 comp rx=0 tx=0).stop 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 shutdown_connections 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5df406ac00 0x7f5df406d0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e180b0630 0x7f5e180b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e1815e0f0 0x7f5e1814b8b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 --2- 172.21.15.73:0/2588740605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df80047c0 0x7f5e1814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.125+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 >> 172.21.15.73:0/2588740605 conn(0x7f5e1806a0d0 msgr2=0x7f5e180af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.126+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 shutdown_connections 2024-04-05T22:22:22.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.126+0000 7f5e1ee00640 1 -- 172.21.15.73:0/2588740605 wait complete. 2024-04-05T22:22:22.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:22:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.365+0000 7fbe56c00640 1 Processor -- start 2024-04-05T22:22:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.365+0000 7fbe56c00640 1 -- start start 2024-04-05T22:22:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.367+0000 7fbe56c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe501599f0 0x7fbe50070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.367+0000 7fbe56c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbe500add00 0x7fbe5006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.367+0000 7fbe56c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe500b27f0 con 0x7fbe5006cb80 2024-04-05T22:22:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.367+0000 7fbe56c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5006d010 con 0x7fbe500add00 2024-04-05T22:22:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.367+0000 7fbe56c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500b3890 con 0x7fbe501599f0 2024-04-05T22:22:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbe500add00 0x7fbe5006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbe500add00 0x7fbe5006c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49472/0 (socket says 172.21.15.73:49472) 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 -- 172.21.15.73:0/3383923466 learned_addr learned my addr 172.21.15.73:0/3383923466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4fe00640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe501599f0 0x7fbe50070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 -- 172.21.15.73:0/3383923466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe501599f0 msgr2=0x7fbe50070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe501599f0 0x7fbe50070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 -- 172.21.15.73:0/3383923466 >> v1:172.21.15.157:6789/0 conn(0x7fbe5006cb80 legacy=0x7fbe5015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.368+0000 7fbe4f400640 1 -- 172.21.15.73:0/3383923466 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500b2ad0 con 0x7fbe500add00 2024-04-05T22:22:22.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.369+0000 7fbe4f400640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe500add00 0x7fbe5006c640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009e50 tx=0x7fbe3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3975c35cdf03ce1f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:22.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe4ea00640 1 -- 172.21.15.73:0/3383923466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3802ca70 con 0x7fbe500add00 2024-04-05T22:22:22.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe4ea00640 1 -- 172.21.15.73:0/3383923466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3802cbd0 con 0x7fbe500add00 2024-04-05T22:22:22.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe4ea00640 1 -- 172.21.15.73:0/3383923466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380343f0 con 0x7fbe500add00 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 -- 172.21.15.73:0/3383923466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe500add00 msgr2=0x7fbe5006c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe500add00 0x7fbe5006c640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009e50 tx=0x7fbe3802bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 -- 172.21.15.73:0/3383923466 shutdown_connections 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe500add00 0x7fbe5006c640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 --2- 172.21.15.73:0/3383923466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe501599f0 0x7fbe50070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.370+0000 7fbe56c00640 1 -- 172.21.15.73:0/3383923466 >> 172.21.15.73:0/3383923466 conn(0x7fbe50069cb0 msgr2=0x7fbe5006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.371+0000 7fbe56c00640 1 -- 172.21.15.73:0/3383923466 shutdown_connections 2024-04-05T22:22:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.371+0000 7fbe56c00640 1 -- 172.21.15.73:0/3383923466 wait complete. 2024-04-05T22:22:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.372+0000 7fbe56c00640 1 Processor -- start 2024-04-05T22:22:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.372+0000 7fbe56c00640 1 -- start start 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe5006cb80 0x7fbe50167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe500add00 0x7fbe50148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50167ad0 con 0x7fbe501599f0 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501487a0 con 0x7fbe500add00 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.373+0000 7fbe56c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500ceae0 con 0x7fbe5006cb80 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49478/0 (socket says 172.21.15.73:49478) 2024-04-05T22:22:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 -- 172.21.15.73:0/4287946802 learned_addr learned my addr 172.21.15.73:0/4287946802 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:22.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe4f400640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe500add00 0x7fbe50148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe4fe00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe5006cb80 0x7fbe50167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 -- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe5006cb80 msgr2=0x7fbe50167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe5006cb80 0x7fbe50167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 -- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe500add00 msgr2=0x7fbe50148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe500add00 0x7fbe50148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.374+0000 7fbe54c00640 1 -- 172.21.15.73:0/4287946802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe38009b30 con 0x7fbe501599f0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe54c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c410 tx=0x7fbe4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011790 con 0x7fbe501599f0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe480118f0 con 0x7fbe501599f0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011ce0 con 0x7fbe501599f0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe500ce520 con 0x7fbe501599f0 2024-04-05T22:22:22.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.375+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe500cbdc0 con 0x7fbe501599f0 2024-04-05T22:22:22.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.376+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe500cbc20 con 0x7fbe501599f0 2024-04-05T22:22:22.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.378+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe48007020 con 0x7fbe501599f0 2024-04-05T22:22:22.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.379+0000 7fbe47e00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 0x7fbe2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:22.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.379+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbe48089730 con 0x7fbe501599f0 2024-04-05T22:22:22.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.380+0000 7fbe4fe00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 0x7fbe2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:22.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.380+0000 7fbe4fe00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 0x7fbe2c06d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:22.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.383+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbe48054ad0 con 0x7fbe501599f0 2024-04-05T22:22:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:22.584+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe500cbc20 con 0x7fbe501599f0 2024-04-05T22:22:23.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.134+0000 7fbe47e00640 1 -- 172.21.15.73:0/4287946802 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe500cbc20 con 0x7fbe501599f0 2024-04-05T22:22:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:22:23.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.146+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 msgr2=0x7fbe2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.146+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 0x7fbe2c06d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40008040 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.146+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 msgr2=0x7fbe501489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.146+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c410 tx=0x7fbe4800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 shutdown_connections 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe2c06ac00 0x7fbe2c06d0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe501599f0 0x7fbe501489e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe500add00 0x7fbe50148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 --2- 172.21.15.73:0/4287946802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe5006cb80 0x7fbe50167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.147+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 >> 172.21.15.73:0/4287946802 conn(0x7fbe50069cb0 msgr2=0x7fbe50157170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:23.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.148+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 shutdown_connections 2024-04-05T22:22:23.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.148+0000 7fbe56c00640 1 -- 172.21.15.73:0/4287946802 wait complete. 2024-04-05T22:22:23.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:22:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.361+0000 7f0f62000640 1 Processor -- start 2024-04-05T22:22:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.361+0000 7f0f62000640 1 -- start start 2024-04-05T22:22:23.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f62000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15aa70 0x7f0f5c070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f62000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 0x7f0f5c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f62000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c15ae50 con 0x7f0f5c0714a0 2024-04-05T22:22:23.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f62000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c071930 con 0x7f0f5c15aa70 2024-04-05T22:22:23.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f62000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15e3b0 con 0x7f0f5c15dd90 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f5aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0f5c0714a0 0x7f0f5c15d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49664/0 (socket says 172.21.15.73:49664) 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f5aa00640 1 -- 172.21.15.73:0/581181407 learned_addr learned my addr 172.21.15.73:0/581181407 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f5be00640 1 --2- 172.21.15.73:0/581181407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 0x7f0f5c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.363+0000 7f0f5b400640 1 --2- 172.21.15.73:0/581181407 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15aa70 0x7f0f5c070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023473857 0 0) 0x7f0f5c15ae50 con 0x7f0f5c0714a0 2024-04-05T22:22:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f40003520 con 0x7f0f5c0714a0 2024-04-05T22:22:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5be00640 1 -- 172.21.15.73:0/581181407 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15aa70 msgr2=0x7f0f5c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5be00640 1 --2- 172.21.15.73:0/581181407 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15aa70 0x7f0f5c070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5be00640 1 -- 172.21.15.73:0/581181407 >> v1:172.21.15.73:6789/0 conn(0x7f0f5c0714a0 legacy=0x7f0f5c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1237074098 0 0) 0x7f0f40003520 con 0x7f0f5c0714a0 2024-04-05T22:22:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.364+0000 7f0f5be00640 1 -- 172.21.15.73:0/581181407 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0ac9e0 con 0x7f0f5c15dd90 2024-04-05T22:22:23.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.366+0000 7f0f5be00640 1 --2- 172.21.15.73:0/581181407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 0x7f0f5c06ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009d90 tx=0x7f0f5402bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1210e293dcd55a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.366+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48001670 con 0x7f0f5c15dd90 2024-04-05T22:22:23.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.366+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5402c8a0 con 0x7f0f5c15dd90 2024-04-05T22:22:23.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.366+0000 7f0f5a000640 1 -- 172.21.15.73:0/581181407 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5402cc30 con 0x7f0f5c15dd90 2024-04-05T22:22:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 -- 172.21.15.73:0/581181407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 msgr2=0x7f0f5c06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 --2- 172.21.15.73:0/581181407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 0x7f0f5c06ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009d90 tx=0x7f0f5402bd70 comp rx=0 tx=0).stop 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 -- 172.21.15.73:0/581181407 shutdown_connections 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 --2- 172.21.15.73:0/581181407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15dd90 0x7f0f5c06ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 --2- 172.21.15.73:0/581181407 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15aa70 0x7f0f5c070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 -- 172.21.15.73:0/581181407 >> 172.21.15.73:0/581181407 conn(0x7f0f5c069cb0 msgr2=0x7f0f5c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 -- 172.21.15.73:0/581181407 shutdown_connections 2024-04-05T22:22:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.367+0000 7f0f62000640 1 -- 172.21.15.73:0/581181407 wait complete. 2024-04-05T22:22:23.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.369+0000 7f0f62000640 1 Processor -- start 2024-04-05T22:22:23.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.369+0000 7f0f62000640 1 -- start start 2024-04-05T22:22:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0714a0 0x7f0f5c155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15dd90 0x7f0f5c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15c5e0 con 0x7f0f5c15aa70 2024-04-05T22:22:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15b9d0 con 0x7f0f5c15dd90 2024-04-05T22:22:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.370+0000 7f0f62000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15bcb0 con 0x7f0f5c0714a0 2024-04-05T22:22:23.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49486/0 (socket says 172.21.15.73:49486) 2024-04-05T22:22:23.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 -- 172.21.15.73:0/1558685045 learned_addr learned my addr 172.21.15.73:0/1558685045 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:23.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5b400640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0714a0 0x7f0f5c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5be00640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15dd90 0x7f0f5c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 -- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0714a0 msgr2=0x7f0f5c155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0714a0 0x7f0f5c155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 -- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15dd90 msgr2=0x7f0f5c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15dd90 0x7f0f5c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.371+0000 7f0f5aa00640 1 -- 172.21.15.73:0/1558685045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009a70 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f5aa00640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c710 tx=0x7f0f4800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48005ad0 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48005c30 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48007070 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c15bf90 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.372+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c152920 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.373+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c15fc50 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.375+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f480071f0 con 0x7f0f5c15aa70 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.375+0000 7f0f53400640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 0x7f0f3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.376+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0f480891c0 con 0x7f0f5c15aa70 2024-04-05T22:22:23.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.376+0000 7f0f5b400640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 0x7f0f3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.376+0000 7f0f5b400640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 0x7f0f3c06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ad0 tx=0x7f0f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.381+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0f48054410 con 0x7f0f5c15aa70 2024-04-05T22:22:23.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.612+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0f5c068160 con 0x7f0f5c15aa70 2024-04-05T22:22:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.630+0000 7f0f53400640 1 -- 172.21.15.73:0/1558685045 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0f48057a30 con 0x7f0f5c15aa70 2024-04-05T22:22:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 4 and data pool 5 2024-04-05T22:22:23.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 msgr2=0x7f0f3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 0x7f0f3c06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ad0 tx=0x7f0f4c008040 comp rx=0 tx=0).stop 2024-04-05T22:22:23.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 msgr2=0x7f0f5c151700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c710 tx=0x7f0f4800cbd0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 shutdown_connections 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0f3c06ac00 0x7f0f3c06d0c0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ad0 tx=0x7f0f4c008040 comp rx=0 tx=0).stop 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15dd90 0x7f0f5c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c15aa70 0x7f0f5c151700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 --2- 172.21.15.73:0/1558685045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0714a0 0x7f0f5c155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.639+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 >> 172.21.15.73:0/1558685045 conn(0x7f0f5c069cb0 msgr2=0x7f0f5c15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:23.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.640+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 shutdown_connections 2024-04-05T22:22:23.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.640+0000 7f0f62000640 1 -- 172.21.15.73:0/1558685045 wait complete. 2024-04-05T22:22:23.677 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:23.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.849+0000 7f9e4ce00640 1 Processor -- start 2024-04-05T22:22:23.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.849+0000 7f9e4ce00640 1 -- start start 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e4ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e4815ce20 0x7f9e480c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e4ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e480ca050 0x7f9e480acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e4ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806aa40 con 0x7f9e4815d200 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e4ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4809d110 con 0x7f9e4815ce20 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e4ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4806ad20 con 0x7f9e480ca050 2024-04-05T22:22:23.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e4815ce20 0x7f9e480c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e3d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9e4815d200 0x7f9e480ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49670/0 (socket says 172.21.15.73:49670) 2024-04-05T22:22:23.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e4815ce20 0x7f9e480c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49500/0 (socket says 172.21.15.73:49500) 2024-04-05T22:22:23.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e3d400640 1 -- 172.21.15.73:0/4290964448 learned_addr learned my addr 172.21.15.73:0/4290964448 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:23.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e46800640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e480ca050 0x7f9e480acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 -- 172.21.15.73:0/4290964448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e480ca050 msgr2=0x7f9e480acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:23.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e480ca050 0x7f9e480acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 -- 172.21.15.73:0/4290964448 >> v1:172.21.15.73:6789/0 conn(0x7f9e4815d200 legacy=0x7f9e480ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.851+0000 7f9e45e00640 1 -- 172.21.15.73:0/4290964448 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480cde30 con 0x7f9e4815ce20 2024-04-05T22:22:23.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e45e00640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815ce20 0x7f9e480c9b10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009d90 tx=0x7f9e3802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90873adffa3a048b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e45400640 1 -- 172.21.15.73:0/4290964448 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3802cc50 con 0x7f9e4815ce20 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e45400640 1 -- 172.21.15.73:0/4290964448 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3802cdb0 con 0x7f9e4815ce20 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e45400640 1 -- 172.21.15.73:0/4290964448 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38034660 con 0x7f9e4815ce20 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e4ce00640 1 -- 172.21.15.73:0/4290964448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815ce20 msgr2=0x7f9e480c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.853+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815ce20 0x7f9e480c9b10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009d90 tx=0x7f9e3802bea0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 -- 172.21.15.73:0/4290964448 shutdown_connections 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e480ca050 0x7f9e480acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/4290964448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815ce20 0x7f9e480c9b10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 -- 172.21.15.73:0/4290964448 >> 172.21.15.73:0/4290964448 conn(0x7f9e480cf190 msgr2=0x7f9e480cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:23.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 -- 172.21.15.73:0/4290964448 shutdown_connections 2024-04-05T22:22:23.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.854+0000 7f9e4ce00640 1 -- 172.21.15.73:0/4290964448 wait complete. 2024-04-05T22:22:23.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.855+0000 7f9e4ce00640 1 Processor -- start 2024-04-05T22:22:23.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.856+0000 7f9e4ce00640 1 -- start start 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e480ca050 0x7f9e4815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e4815ce20 0x7f9e481f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480b4760 con 0x7f9e4815d200 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4815c5c0 con 0x7f9e480ca050 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e4ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e481f2a40 con 0x7f9e4815ce20 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49504/0 (socket says 172.21.15.73:49504) 2024-04-05T22:22:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 -- 172.21.15.73:0/362468937 learned_addr learned my addr 172.21.15.73:0/362468937 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e45e00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e480ca050 0x7f9e4815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 -- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e4815ce20 msgr2=0x7f9e481f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e4815ce20 0x7f9e481f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 -- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e480ca050 msgr2=0x7f9e4815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e480ca050 0x7f9e4815c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.857+0000 7f9e46800640 1 -- 172.21.15.73:0/362468937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e38009a70 con 0x7f9e4815d200 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e46800640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c820 tx=0x7f9e4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40004020 con 0x7f9e4815d200 2024-04-05T22:22:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e40004180 con 0x7f9e4815d200 2024-04-05T22:22:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e480ad690 con 0x7f9e4815d200 2024-04-05T22:22:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40007830 con 0x7f9e4815d200 2024-04-05T22:22:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.858+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e481f6a60 con 0x7f9e4815d200 2024-04-05T22:22:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.859+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e48068160 con 0x7f9e4815d200 2024-04-05T22:22:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.860+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e40007990 con 0x7f9e4815d200 2024-04-05T22:22:23.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.861+0000 7f9e3ea00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 0x7f9e2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:23.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.861+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e400890b0 con 0x7f9e4815d200 2024-04-05T22:22:23.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.861+0000 7f9e45e00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 0x7f9e2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:23.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.862+0000 7f9e45e00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 0x7f9e2406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e38036000 tx=0x7f9e38030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:23.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:23.866+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9e40054290 con 0x7f9e4815d200 2024-04-05T22:22:24.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.054+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e48065e50 con 0x7f9e4815d200 2024-04-05T22:22:24.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.055+0000 7f9e3ea00640 1 -- 172.21.15.73:0/362468937 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f9e400578b0 con 0x7f9e4815d200 2024-04-05T22:22:24.056 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:24.056 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":35,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:23.614685+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 msgr2=0x7f9e2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 0x7f9e2406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e38036000 tx=0x7f9e38030000 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 msgr2=0x7f9e481f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c820 tx=0x7f9e4000cce0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 shutdown_connections 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9e2406ac00 0x7f9e2406d0c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e4815d200 0x7f9e481f6080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e4815ce20 0x7f9e481f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 --2- 172.21.15.73:0/362468937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e480ca050 0x7f9e4815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.058+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 >> 172.21.15.73:0/362468937 conn(0x7f9e480cf190 msgr2=0x7f9e480af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.059+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 shutdown_connections 2024-04-05T22:22:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.059+0000 7f9e4ce00640 1 -- 172.21.15.73:0/362468937 wait complete. 2024-04-05T22:22:24.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.277+0000 7f5f73200640 1 Processor -- start 2024-04-05T22:22:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.277+0000 7f5f73200640 1 -- start start 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c1587d0 0x7f5f6c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 0x7f5f6c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c06f5b0 con 0x7f5f6c0ccd20 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c070410 con 0x7f5f6c06d0c0 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.278+0000 7f5f73200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c158bb0 con 0x7f5f6c1587d0 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49520/0 (socket says 172.21.15.73:49520) 2024-04-05T22:22:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 -- 172.21.15.73:0/1145510056 learned_addr learned my addr 172.21.15.73:0/1145510056 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:24.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f71200640 1 --2- 172.21.15.73:0/1145510056 >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 0x7f5f6c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 -- 172.21.15.73:0/1145510056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c1587d0 msgr2=0x7f5f6c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:24.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 --2- 172.21.15.73:0/1145510056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c1587d0 0x7f5f6c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 -- 172.21.15.73:0/1145510056 >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 msgr2=0x7f5f6c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:24.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 --2- 172.21.15.73:0/1145510056 >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 0x7f5f6c15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f63400640 1 -- 172.21.15.73:0/1145510056 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c06f800 con 0x7f5f6c0ccd20 2024-04-05T22:22:24.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.279+0000 7f5f71200640 1 --2- 172.21.15.73:0/1145510056 >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 0x7f5f6c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:24.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.280+0000 7f5f63400640 1 --2- 172.21.15.73:0/1145510056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0ae4b0 tx=0x7f5f5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9061f95fdf19ded server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.280+0000 7f5f6b400640 1 -- 172.21.15.73:0/1145510056 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54031870 con 0x7f5f6c0ccd20 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f6b400640 1 -- 172.21.15.73:0/1145510056 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f540319d0 con 0x7f5f6c0ccd20 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f6b400640 1 -- 172.21.15.73:0/1145510056 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54031d60 con 0x7f5f6c0ccd20 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 -- 172.21.15.73:0/1145510056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c0ccd20 msgr2=0x7f5f6c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 --2- 172.21.15.73:0/1145510056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0ae4b0 tx=0x7f5f5402b910 comp rx=0 tx=0).stop 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 -- 172.21.15.73:0/1145510056 shutdown_connections 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 --2- 172.21.15.73:0/1145510056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c0ccd20 0x7f5f6c06cb80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 --2- 172.21.15.73:0/1145510056 >> v2:172.21.15.157:3300/0 conn(0x7f5f6c06d0c0 0x7f5f6c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 --2- 172.21.15.73:0/1145510056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c1587d0 0x7f5f6c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 -- 172.21.15.73:0/1145510056 >> 172.21.15.73:0/1145510056 conn(0x7f5f6c0ce8e0 msgr2=0x7f5f6c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:24.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 -- 172.21.15.73:0/1145510056 shutdown_connections 2024-04-05T22:22:24.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.281+0000 7f5f73200640 1 -- 172.21.15.73:0/1145510056 wait complete. 2024-04-05T22:22:24.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.283+0000 7f5f73200640 1 Processor -- start 2024-04-05T22:22:24.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.283+0000 7f5f73200640 1 -- start start 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c06d0c0 0x7f5f6c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 0x7f5f6c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c15a8a0 con 0x7f5f6c15a440 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0c3880 con 0x7f5f6c1587d0 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f73200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1f24a0 con 0x7f5f6c06d0c0 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f71200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f71200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49536/0 (socket says 172.21.15.73:49536) 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.284+0000 7f5f71200640 1 -- 172.21.15.73:0/3023439278 learned_addr learned my addr 172.21.15.73:0/3023439278 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f63400640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 0x7f5f6c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f6be00640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c06d0c0 0x7f5f6c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 -- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c06d0c0 msgr2=0x7f5f6c14dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c06d0c0 0x7f5f6c14dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 -- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 msgr2=0x7f5f6c159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 0x7f5f6c159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 -- 172.21.15.73:0/3023439278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540094d0 con 0x7f5f6c15a440 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f63400640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 0x7f5f6c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f71200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c820 tx=0x7f5f6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64007a30 con 0x7f5f6c15a440 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64007b90 con 0x7f5f6c15a440 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400ce10 con 0x7f5f6c15a440 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.285+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c0c3af0 con 0x7f5f6c15a440 2024-04-05T22:22:24.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.286+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0c40b0 con 0x7f5f6c15a440 2024-04-05T22:22:24.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.287+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c066af0 con 0x7f5f6c15a440 2024-04-05T22:22:24.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.288+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f640113f0 con 0x7f5f6c15a440 2024-04-05T22:22:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.288+0000 7f5f68c00640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 0x7f5f4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.289+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f64089490 con 0x7f5f6c15a440 2024-04-05T22:22:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.289+0000 7f5f6be00640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 0x7f5f4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.289+0000 7f5f6be00640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 0x7f5f4806d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007dc0 tx=0x7f5f5c0072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.293+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5f64054660 con 0x7f5f6c15a440 2024-04-05T22:22:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.488+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f6c0c42a0 con 0x7f5f6c15a440 2024-04-05T22:22:24.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.491+0000 7f5f68c00640 1 -- 172.21.15.73:0/3023439278 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f5f64057c80 con 0x7f5f6c15a440 2024-04-05T22:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":35,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:23.614685+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 msgr2=0x7f5f4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 0x7f5f4806d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007dc0 tx=0x7f5f5c0072f0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 msgr2=0x7f5f6c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c820 tx=0x7f5f6400cce0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 shutdown_connections 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5f4806ac00 0x7f5f4806d0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f6c15a440 0x7f5f6c0b21f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f6c1587d0 0x7f5f6c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 --2- 172.21.15.73:0/3023439278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f6c06d0c0 0x7f5f6c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.494+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 >> 172.21.15.73:0/3023439278 conn(0x7f5f6c0ce8e0 msgr2=0x7f5f6c15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:24.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.495+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 shutdown_connections 2024-04-05T22:22:24.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.495+0000 7f5f73200640 1 -- 172.21.15.73:0/3023439278 wait complete. 2024-04-05T22:22:24.512 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:22:24.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.705+0000 7f4aab200640 1 Processor -- start 2024-04-05T22:22:24.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.705+0000 7f4aab200640 1 -- start start 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aab200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4aac0d37d0 0x7f4aac0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aab200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 0x7f4aac0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aab200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0cfe40 con 0x7f4aac15cdd0 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aab200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0cd800 con 0x7f4aac0d37d0 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aab200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0cdae0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.706+0000 7f4aa9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4aac0d37d0 0x7f4aac0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4aac15cdd0 0x7f4aac0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49676/0 (socket says 172.21.15.73:49676) 2024-04-05T22:22:24.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9e00640 1 -- 172.21.15.73:0/1663674852 learned_addr learned my addr 172.21.15.73:0/1663674852 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:24.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa8a00640 1 -- 172.21.15.73:0/1663674852 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742170800 0 0) 0x7f4a9c001110 con 0x7f4aac15cdd0 2024-04-05T22:22:24.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa8a00640 1 -- 172.21.15.73:0/1663674852 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a90003520 con 0x7f4aac15cdd0 2024-04-05T22:22:24.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aaa800640 1 --2- 172.21.15.73:0/1663674852 >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 0x7f4aac0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9400640 1 -- 172.21.15.73:0/1663674852 >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 msgr2=0x7f4aac0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9400640 1 --2- 172.21.15.73:0/1663674852 >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 0x7f4aac0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9400640 1 -- 172.21.15.73:0/1663674852 >> v1:172.21.15.73:6789/0 conn(0x7f4aac15cdd0 legacy=0x7f4aac0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aa9400640 1 -- 172.21.15.73:0/1663674852 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0cddc0 con 0x7f4aac0d37d0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.707+0000 7f4aaa800640 1 --2- 172.21.15.73:0/1663674852 >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 0x7f4aac0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.708+0000 7f4aa9400640 1 --2- 172.21.15.73:0/1663674852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0d37d0 0x7f4aac0d1130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0cd130 tx=0x7f4a9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd5033196385157 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.708+0000 7f4aa8a00640 1 -- 172.21.15.73:0/1663674852 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9802e070 con 0x7f4aac0d37d0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.708+0000 7f4aa8a00640 1 -- 172.21.15.73:0/1663674852 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a980315a0 con 0x7f4aac0d37d0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aa8a00640 1 -- 172.21.15.73:0/1663674852 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98031930 con 0x7f4aac0d37d0 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 -- 172.21.15.73:0/1663674852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0d37d0 msgr2=0x7f4aac0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 --2- 172.21.15.73:0/1663674852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0d37d0 0x7f4aac0d1130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0cd130 tx=0x7f4a9802b840 comp rx=0 tx=0).stop 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 -- 172.21.15.73:0/1663674852 shutdown_connections 2024-04-05T22:22:24.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 --2- 172.21.15.73:0/1663674852 >> v2:172.21.15.157:3300/0 conn(0x7f4aac0cf4b0 0x7f4aac0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 --2- 172.21.15.73:0/1663674852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0d37d0 0x7f4aac0d1130 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 -- 172.21.15.73:0/1663674852 >> 172.21.15.73:0/1663674852 conn(0x7f4aac09be20 msgr2=0x7f4aac0d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 -- 172.21.15.73:0/1663674852 shutdown_connections 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.709+0000 7f4aab200640 1 -- 172.21.15.73:0/1663674852 wait complete. 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.711+0000 7f4aab200640 1 Processor -- start 2024-04-05T22:22:24.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.711+0000 7f4aab200640 1 -- start start 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac0d37d0 0x7f4aac0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac15cdd0 0x7f4aac06a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15a6a0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cc720 con 0x7f4aac15cdd0 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cca00 con 0x7f4aac0d37d0 2024-04-05T22:22:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49568/0 (socket says 172.21.15.73:49568) 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 -- 172.21.15.73:0/2232528919 learned_addr learned my addr 172.21.15.73:0/2232528919 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 -- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac0d37d0 msgr2=0x7f4aac0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac0d37d0 0x7f4aac0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 -- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac15cdd0 msgr2=0x7f4aac06a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac15cdd0 0x7f4aac06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.712+0000 7f4aa9e00640 1 -- 172.21.15.73:0/2232528919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98009400 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aa9e00640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c420 tx=0x7f4a9c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c005ce0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c005e40 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0072f0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac15d330 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.713+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac1f3c00 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.714+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac04e160 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.715+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a9c007510 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.716+0000 7f4aa2000640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 0x7f4a8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:24.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.716+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a9c0888b0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.716+0000 7f4aa9400640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 0x7f4a8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.717+0000 7f4aa9400640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 0x7f4a8c06d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a98036000 tx=0x7f4a98030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.721+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4a9c0539f0 con 0x7f4aac0cf4b0 2024-04-05T22:22:24.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:24.947+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4aac0cfe40 con 0x7f4aac0cf4b0 2024-04-05T22:22:25.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.638+0000 7f4aa2000640 1 -- 172.21.15.73:0/2232528919 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4aac0cfe40 con 0x7f4aac0cf4b0 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 msgr2=0x7f4a8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 0x7f4a8c06d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a98036000 tx=0x7f4a98030000 comp rx=0 tx=0).stop 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 msgr2=0x7f4aac15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c420 tx=0x7f4a9c00c8e0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 shutdown_connections 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4a8c06ac00 0x7f4a8c06d0c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac15cdd0 0x7f4aac06a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac0d37d0 0x7f4aac0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 --2- 172.21.15.73:0/2232528919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aac0cf4b0 0x7f4aac15a160 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 >> 172.21.15.73:0/2232528919 conn(0x7f4aac09be20 msgr2=0x7f4aac0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 shutdown_connections 2024-04-05T22:22:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.642+0000 7f4aab200640 1 -- 172.21.15.73:0/2232528919 wait complete. 2024-04-05T22:22:25.662 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:22:25.662 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:22:25.787 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-05T22:22:25.786+0000 7f2700e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:22:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.871+0000 7f5014800640 1 Processor -- start 2024-04-05T22:22:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.871+0000 7f5014800640 1 -- start start 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.872+0000 7f5014800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f500c06cd50 0x7f500c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.872+0000 7f5014800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 0x7f500c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.872+0000 7f5014800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500c0b27f0 con 0x7f500c1599f0 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.872+0000 7f5014800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c0b3890 con 0x7f500c06cd50 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.872+0000 7f5014800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c0b2ad0 con 0x7f500c0add00 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f500c06cd50 0x7f500c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f500c06cd50 0x7f500c070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46422/0 (socket says 172.21.15.73:46422) 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 -- 172.21.15.73:0/2382077021 learned_addr learned my addr 172.21.15.73:0/2382077021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:25.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5012800640 1 --2- 172.21.15.73:0/2382077021 >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 0x7f500c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 -- 172.21.15.73:0/2382077021 >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 msgr2=0x7f500c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:25.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 --2- 172.21.15.73:0/2382077021 >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 0x7f500c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 -- 172.21.15.73:0/2382077021 >> v1:172.21.15.157:6789/0 conn(0x7f500c1599f0 legacy=0x7f500c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:25.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5011400640 1 -- 172.21.15.73:0/2382077021 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c0b2db0 con 0x7f500c06cd50 2024-04-05T22:22:25.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.873+0000 7f5012800640 1 --2- 172.21.15.73:0/2382077021 >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 0x7f500c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5011400640 1 --2- 172.21.15.73:0/2382077021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c070a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f500c06d750 tx=0x7f4ffc02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fdeb6fcce1680b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5010a00640 1 -- 172.21.15.73:0/2382077021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc02eea0 con 0x7f500c06cd50 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5010a00640 1 -- 172.21.15.73:0/2382077021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc036440 con 0x7f500c06cd50 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5010a00640 1 -- 172.21.15.73:0/2382077021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0367d0 con 0x7f500c06cd50 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 -- 172.21.15.73:0/2382077021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 msgr2=0x7f500c070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 --2- 172.21.15.73:0/2382077021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c070a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f500c06d750 tx=0x7f4ffc02e0f0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 -- 172.21.15.73:0/2382077021 shutdown_connections 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 --2- 172.21.15.73:0/2382077021 >> v2:172.21.15.157:3300/0 conn(0x7f500c0add00 0x7f500c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 --2- 172.21.15.73:0/2382077021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c070a40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.875+0000 7f5014800640 1 -- 172.21.15.73:0/2382077021 >> 172.21.15.73:0/2382077021 conn(0x7f500c069cb0 msgr2=0x7f500c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:25.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.876+0000 7f5014800640 1 -- 172.21.15.73:0/2382077021 shutdown_connections 2024-04-05T22:22:25.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.876+0000 7f5014800640 1 -- 172.21.15.73:0/2382077021 wait complete. 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.877+0000 7f5014800640 1 Processor -- start 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.877+0000 7f5014800640 1 -- start start 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f500c0add00 0x7f500c1412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500c1599f0 0x7f500c1417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c141d30 con 0x7f500c06cd50 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c141fb0 con 0x7f500c1599f0 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5014800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c142200 con 0x7f500c0add00 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.878+0000 7f5011e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46426/0 (socket says 172.21.15.73:46426) 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 -- 172.21.15.73:0/1458729732 learned_addr learned my addr 172.21.15.73:0/1458729732 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011400640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f500c0add00 0x7f500c1412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 -- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f500c0add00 msgr2=0x7f500c1412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f500c0add00 0x7f500c1412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5012800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500c1599f0 0x7f500c1417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 -- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500c1599f0 msgr2=0x7f500c1417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500c1599f0 0x7f500c1417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 -- 172.21.15.73:0/1458729732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc00b770 con 0x7f500c06cd50 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5011e00640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f500000b3f0 tx=0x7f500000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000011830 con 0x7f500c06cd50 2024-04-05T22:22:25.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000011990 con 0x7f500c06cd50 2024-04-05T22:22:25.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.879+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c1424e0 con 0x7f500c06cd50 2024-04-05T22:22:25.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.880+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000011d40 con 0x7f500c06cd50 2024-04-05T22:22:25.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.880+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c142aa0 con 0x7f500c06cd50 2024-04-05T22:22:25.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.881+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c142c90 con 0x7f500c06cd50 2024-04-05T22:22:25.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.881+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f500001a400 con 0x7f500c06cd50 2024-04-05T22:22:25.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.883+0000 7f5006000640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 0x7f4ff006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:25.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.883+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50000892a0 con 0x7f500c06cd50 2024-04-05T22:22:25.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.883+0000 7f5011400640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 0x7f4ff006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:25.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.884+0000 7f5011400640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 0x7f4ff006d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc038000 tx=0x7f4ffc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:25.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:25.888+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f50000543e0 con 0x7f500c06cd50 2024-04-05T22:22:26.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.120+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f500c068160 con 0x7f500c06cd50 2024-04-05T22:22:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.641+0000 7f5006000640 1 -- 172.21.15.73:0/1458729732 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5000057a00 con 0x7f500c06cd50 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 msgr2=0x7f4ff006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 0x7f4ff006d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc038000 tx=0x7f4ffc008040 comp rx=0 tx=0).stop 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 msgr2=0x7f500c06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f500000b3f0 tx=0x7f500000b8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 shutdown_connections 2024-04-05T22:22:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4ff006ac00 0x7f4ff006d0c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500c1599f0 0x7f500c1417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f500c0add00 0x7f500c1412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 --2- 172.21.15.73:0/1458729732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f500c06cd50 0x7f500c06bb00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 >> 172.21.15.73:0/1458729732 conn(0x7f500c069cb0 msgr2=0x7f500c15c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 shutdown_connections 2024-04-05T22:22:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.644+0000 7f5014800640 1 -- 172.21.15.73:0/1458729732 wait complete. 2024-04-05T22:22:26.663 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:22:26.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:26.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.855+0000 7f21a8400640 1 Processor -- start 2024-04-05T22:22:26.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.855+0000 7f21a8400640 1 -- start start 2024-04-05T22:22:26.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f21a01541a0 0x7f21a00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21a006d710 0x7f21a00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a00cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a015aa60 con 0x7f21a006d710 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a00b3b60 con 0x7f21a01541a0 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.856+0000 7f21a8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00701c0 con 0x7f21a015aef0 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21a006d710 0x7f21a00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21a006d710 0x7f21a00cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46428/0 (socket says 172.21.15.73:46428) 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 -- 172.21.15.73:0/817953108 learned_addr learned my addr 172.21.15.73:0/817953108 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a6400640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a00cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 -- 172.21.15.73:0/817953108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 msgr2=0x7f21a00cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a00cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 -- 172.21.15.73:0/817953108 >> v2:172.21.15.157:3300/0 conn(0x7f21a01541a0 msgr2=0x7f21a00b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:26.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 --2- 172.21.15.73:0/817953108 >> v2:172.21.15.157:3300/0 conn(0x7f21a01541a0 0x7f21a00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.857+0000 7f21a5000640 1 -- 172.21.15.73:0/817953108 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00b3880 con 0x7f21a006d710 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.858+0000 7f21a5000640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a00cd8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f21a0066850 tx=0x7f218c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f95f5313e0d883 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.858+0000 7f219fe00640 1 -- 172.21.15.73:0/817953108 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c031870 con 0x7f21a006d710 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.858+0000 7f219fe00640 1 -- 172.21.15.73:0/817953108 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c0319d0 con 0x7f21a006d710 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.858+0000 7f219fe00640 1 -- 172.21.15.73:0/817953108 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c031d60 con 0x7f21a006d710 2024-04-05T22:22:26.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 -- 172.21.15.73:0/817953108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 msgr2=0x7f21a00cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a00cd8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f21a0066850 tx=0x7f218c02b910 comp rx=0 tx=0).stop 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 -- 172.21.15.73:0/817953108 shutdown_connections 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a00cd8f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 --2- 172.21.15.73:0/817953108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a00cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 --2- 172.21.15.73:0/817953108 >> v2:172.21.15.157:3300/0 conn(0x7f21a01541a0 0x7f21a00b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 -- 172.21.15.73:0/817953108 >> 172.21.15.73:0/817953108 conn(0x7f21a00ce8e0 msgr2=0x7f21a00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:26.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 -- 172.21.15.73:0/817953108 shutdown_connections 2024-04-05T22:22:26.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.859+0000 7f21a8400640 1 -- 172.21.15.73:0/817953108 wait complete. 2024-04-05T22:22:26.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.861+0000 7f21a8400640 1 Processor -- start 2024-04-05T22:22:26.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.861+0000 7f21a8400640 1 -- start start 2024-04-05T22:22:26.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 0x7f21a01452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a01edd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a006cce0 con 0x7f21a006d710 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a0145820 con 0x7f21a01541a0 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a0155190 con 0x7f21a015aef0 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46444/0 (socket says 172.21.15.73:46444) 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a5a00640 1 -- 172.21.15.73:0/4102382923 learned_addr learned my addr 172.21.15.73:0/4102382923 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a5000640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 0x7f21a01452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a5a00640 1 -- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 msgr2=0x7f21a01edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.862+0000 7f21a6400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a01edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5a00640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a01edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5a00640 1 -- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 msgr2=0x7f21a01452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5a00640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 0x7f21a01452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:26.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5a00640 1 -- 172.21.15.73:0/4102382923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0094d0 con 0x7f21a006d710 2024-04-05T22:22:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5000640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 0x7f21a01452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a6400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a01edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a5a00640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f219400b4a0 tx=0x7f219400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194010610 con 0x7f21a006d710 2024-04-05T22:22:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194010770 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194010b20 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a01ee2d0 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.863+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a006c720 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.864+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a0066af0 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.865+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2194010c80 con 0x7f21a006d710 2024-04-05T22:22:26.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.866+0000 7f219d600640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 0x7f218006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:26.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.867+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2194088ff0 con 0x7f21a006d710 2024-04-05T22:22:26.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.867+0000 7f21a5000640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 0x7f218006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:26.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.867+0000 7f21a5000640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 0x7f218006d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f218c036000 tx=0x7f218c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:26.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:26.871+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f21940bc650 con 0x7f21a006d710 2024-04-05T22:22:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.104+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a0154f60 con 0x7f21a006d710 2024-04-05T22:22:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.104+0000 7f219d600640 1 -- 172.21.15.73:0/4102382923 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f2194014090 con 0x7f21a006d710 2024-04-05T22:22:27.105 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:27.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:27.105 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 15 2024-04-05T22:22:27.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 msgr2=0x7f218006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 0x7f218006d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f218c036000 tx=0x7f218c03c040 comp rx=0 tx=0).stop 2024-04-05T22:22:27.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 msgr2=0x7f21a006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f219400b4a0 tx=0x7f219400b960 comp rx=0 tx=0).stop 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 shutdown_connections 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f218006ac00 0x7f218006d0c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21a015aef0 0x7f21a01edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21a01541a0 0x7f21a01452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 --2- 172.21.15.73:0/4102382923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21a006d710 0x7f21a006bd40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.107+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 >> 172.21.15.73:0/4102382923 conn(0x7f21a00ce8e0 msgr2=0x7f21a015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:27.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.108+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 shutdown_connections 2024-04-05T22:22:27.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.108+0000 7f21a8400640 1 -- 172.21.15.73:0/4102382923 wait complete. 2024-04-05T22:22:27.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:27.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.308+0000 7f6395200640 1 Processor -- start 2024-04-05T22:22:27.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.309+0000 7f6395200640 1 -- start start 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f6395200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63900d1290 0x7f63900d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f6395200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1670 0x7f63900cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f6395200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63900cd670 con 0x7f63900d0b70 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f6395200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63900cf460 con 0x7f63900d1290 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f6395200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63900cf740 con 0x7f63900d1670 2024-04-05T22:22:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63900d1290 0x7f63900d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f63900d1290 0x7f63900d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46460/0 (socket says 172.21.15.73:46460) 2024-04-05T22:22:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 -- 172.21.15.73:0/1826987475 learned_addr learned my addr 172.21.15.73:0/1826987475 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 -- 172.21.15.73:0/1826987475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1670 msgr2=0x7f63900cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 --2- 172.21.15.73:0/1826987475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1670 0x7f63900cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 -- 172.21.15.73:0/1826987475 >> v1:172.21.15.157:6789/0 conn(0x7f63900d0b70 legacy=0x7f639015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.310+0000 7f638f400640 1 -- 172.21.15.73:0/1826987475 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900cfa20 con 0x7f63900d1290 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.311+0000 7f638f400640 1 --2- 172.21.15.73:0/1826987475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d1290 0x7f63900d0630 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fc40221bbeb5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f638ea00640 1 -- 172.21.15.73:0/1826987475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800317c0 con 0x7f63900d1290 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f638ea00640 1 -- 172.21.15.73:0/1826987475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380031920 con 0x7f63900d1290 2024-04-05T22:22:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f638ea00640 1 -- 172.21.15.73:0/1826987475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380031cb0 con 0x7f63900d1290 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 -- 172.21.15.73:0/1826987475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d1290 msgr2=0x7f63900d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 --2- 172.21.15.73:0/1826987475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d1290 0x7f63900d0630 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638002b860 comp rx=0 tx=0).stop 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 -- 172.21.15.73:0/1826987475 shutdown_connections 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 --2- 172.21.15.73:0/1826987475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1670 0x7f63900cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 --2- 172.21.15.73:0/1826987475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d1290 0x7f63900d0630 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 -- 172.21.15.73:0/1826987475 >> 172.21.15.73:0/1826987475 conn(0x7f639009bdd0 msgr2=0x7f63900d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.312+0000 7f6395200640 1 -- 172.21.15.73:0/1826987475 shutdown_connections 2024-04-05T22:22:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.313+0000 7f6395200640 1 -- 172.21.15.73:0/1826987475 wait complete. 2024-04-05T22:22:27.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.314+0000 7f6395200640 1 Processor -- start 2024-04-05T22:22:27.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.314+0000 7f6395200640 1 -- start start 2024-04-05T22:22:27.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1290 0x7f63900c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 0x7f639015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900ca410 con 0x7f63900d0b70 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900ca6f0 con 0x7f63900d1670 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f6395200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63900ca9d0 con 0x7f63900d1290 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f638f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f638f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46476/0 (socket says 172.21.15.73:46476) 2024-04-05T22:22:27.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.315+0000 7f638f400640 1 -- 172.21.15.73:0/868502441 learned_addr learned my addr 172.21.15.73:0/868502441 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 -- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1290 msgr2=0x7f63900c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1290 0x7f63900c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638fe00640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 0x7f639015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 -- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 msgr2=0x7f639015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 0x7f639015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 -- 172.21.15.73:0/868502441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800094d0 con 0x7f63900d0b70 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638fe00640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 0x7f639015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f638f400640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6380000c00 tx=0x7f63800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380032860 con 0x7f63900d0b70 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63800329e0 con 0x7f63900d0b70 2024-04-05T22:22:27.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63900b3c30 con 0x7f63900d0b70 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.316+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380032d90 con 0x7f63900d0b70 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.317+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63900caf90 con 0x7f63900d0b70 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.318+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63900cb180 con 0x7f63900d0b70 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.319+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f638004e450 con 0x7f63900d0b70 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.320+0000 7f6387400640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 0x7f636c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.320+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63800ab650 con 0x7f63900d0b70 2024-04-05T22:22:27.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.320+0000 7f6387e00640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 0x7f636c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.321+0000 7f6387e00640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 0x7f636c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6378006fd0 tx=0x7f6378006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.324+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6380076830 con 0x7f63900d0b70 2024-04-05T22:22:27.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.506+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f639004e160 con 0x7f63900d0b70 2024-04-05T22:22:27.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.508+0000 7f6387400640 1 -- 172.21.15.73:0/868502441 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f6380079e50 con 0x7f63900d0b70 2024-04-05T22:22:27.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:27.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:27.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 msgr2=0x7f636c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 0x7f636c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6378006fd0 tx=0x7f6378006bc0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 msgr2=0x7f639014dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6380000c00 tx=0x7f63800046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 shutdown_connections 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f636c06ac00 0x7f636c06d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63900d1670 0x7f639015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63900d1290 0x7f63900c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 --2- 172.21.15.73:0/868502441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63900d0b70 0x7f639014dcf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 >> 172.21.15.73:0/868502441 conn(0x7f639009bdd0 msgr2=0x7f639015e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:27.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 shutdown_connections 2024-04-05T22:22:27.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.510+0000 7f6395200640 1 -- 172.21.15.73:0/868502441 wait complete. 2024-04-05T22:22:27.529 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:22:27.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:22:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.724+0000 7f9586800640 1 Processor -- start 2024-04-05T22:22:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.724+0000 7f9586800640 1 -- start start 2024-04-05T22:22:27.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.725+0000 7f9586800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9580160620 0x7f95800caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.725+0000 7f9586800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f95800ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.725+0000 7f9586800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95800ad6b0 0x7f95800cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f9586800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95800ce5d0 con 0x7f95800ad6b0 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f9586800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95800ce010 con 0x7f9580160620 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f9586800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800ce2f0 con 0x7f95800b1870 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95800ad6b0 0x7f95800cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95800ad6b0 0x7f95800cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46480/0 (socket says 172.21.15.73:46480) 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 -- 172.21.15.73:0/3989930052 learned_addr learned my addr 172.21.15.73:0/3989930052 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957f400640 1 --2- 172.21.15.73:0/3989930052 >> v2:172.21.15.157:3300/0 conn(0x7f9580160620 0x7f95800caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f9577e00640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f95800ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 -- 172.21.15.73:0/3989930052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 msgr2=0x7f95800ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f95800ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 -- 172.21.15.73:0/3989930052 >> v2:172.21.15.157:3300/0 conn(0x7f9580160620 msgr2=0x7f95800caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:27.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 --2- 172.21.15.73:0/3989930052 >> v2:172.21.15.157:3300/0 conn(0x7f9580160620 0x7f95800caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.726+0000 7f957fe00640 1 -- 172.21.15.73:0/3989930052 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95800ac970 con 0x7f95800ad6b0 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.727+0000 7f957fe00640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f95800cb2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95800667b0 tx=0x7f957802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea35d95341d5144 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f957ea00640 1 -- 172.21.15.73:0/3989930052 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957802ca70 con 0x7f95800ad6b0 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f957ea00640 1 -- 172.21.15.73:0/3989930052 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957802cbd0 con 0x7f95800ad6b0 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f957ea00640 1 -- 172.21.15.73:0/3989930052 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780343f0 con 0x7f95800ad6b0 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 -- 172.21.15.73:0/3989930052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 msgr2=0x7f95800cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f95800cb2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95800667b0 tx=0x7f957802bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 -- 172.21.15.73:0/3989930052 shutdown_connections 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f95800cb2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 --2- 172.21.15.73:0/3989930052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f95800ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 --2- 172.21.15.73:0/3989930052 >> v2:172.21.15.157:3300/0 conn(0x7f9580160620 0x7f95800caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 -- 172.21.15.73:0/3989930052 >> 172.21.15.73:0/3989930052 conn(0x7f95800ce8e0 msgr2=0x7f95800cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:27.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.728+0000 7f9586800640 1 -- 172.21.15.73:0/3989930052 shutdown_connections 2024-04-05T22:22:27.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.729+0000 7f9586800640 1 -- 172.21.15.73:0/3989930052 wait complete. 2024-04-05T22:22:27.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.730+0000 7f9586800640 1 Processor -- start 2024-04-05T22:22:27.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.730+0000 7f9586800640 1 -- start start 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f9580156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 0x7f9580157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958015c5c0 con 0x7f95800ad6b0 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9580157200 con 0x7f9580160620 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f9586800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958015c810 con 0x7f95800b1870 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f957f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f957f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46494/0 (socket says 172.21.15.73:46494) 2024-04-05T22:22:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.731+0000 7f957f400640 1 -- 172.21.15.73:0/3123910941 learned_addr learned my addr 172.21.15.73:0/3123910941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957fe00640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 0x7f9580157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f9577e00640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f9580156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 -- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 msgr2=0x7f9580156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f9580156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 -- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 msgr2=0x7f9580157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 0x7f9580157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 -- 172.21.15.73:0/3123910941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009b30 con 0x7f95800ad6b0 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957fe00640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 0x7f9580157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f957f400640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f957000d620 tx=0x7f957000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000b620 con 0x7f95800ad6b0 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000b780 con 0x7f95800ad6b0 2024-04-05T22:22:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000bb30 con 0x7f95800ad6b0 2024-04-05T22:22:27.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.732+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958015caf0 con 0x7f95800ad6b0 2024-04-05T22:22:27.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.733+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958015d0b0 con 0x7f95800ad6b0 2024-04-05T22:22:27.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.733+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9580066af0 con 0x7f95800ad6b0 2024-04-05T22:22:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.734+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f957000bc90 con 0x7f95800ad6b0 2024-04-05T22:22:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.735+0000 7f9577400640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.735+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9570089810 con 0x7f95800ad6b0 2024-04-05T22:22:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.735+0000 7f9577e00640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.736+0000 7f9577e00640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 0x7f955c06d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9580154ac0 tx=0x7f9568008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.739+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f95700549f0 con 0x7f95800ad6b0 2024-04-05T22:22:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.930+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95800cafa0 con 0x7f955c06ac00 2024-04-05T22:22:27.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.955+0000 7f9577400640 1 -- 172.21.15.73:0/3123910941 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276334 (secure 0 0 0) 0x7f956800c040 con 0x7f955c06ac00 2024-04-05T22:22:27.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:27.961 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 msgr2=0x7f955c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 0x7f955c06d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9580154ac0 tx=0x7f9568008040 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 msgr2=0x7f9580154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f957000d620 tx=0x7f957000dae0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 shutdown_connections 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f955c06ac00 0x7f955c06d0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9580160620 0x7f9580157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95800b1870 0x7f9580156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 --2- 172.21.15.73:0/3123910941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95800ad6b0 0x7f9580154580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 >> 172.21.15.73:0/3123910941 conn(0x7f95800ce8e0 msgr2=0x7f95800b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:27.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 shutdown_connections 2024-04-05T22:22:27.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:27.961+0000 7f9586800640 1 -- 172.21.15.73:0/3123910941 wait complete. 2024-04-05T22:22:27.980 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-05T22:22:26.510885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236588,"kb_used_data":4016,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738132,"statfs":{"total":773094113280,"available":772851847168,"internally_reserved":0,"allocated":4112384,"data_stored":2294486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.396348"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130660+0000","last_change":"2024-04-05T22:22:22.130660+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130660+0000","last_undegraded":"2024-04-05T22:22:22.130660+0000","last_fullsized":"2024-04-05T22:22:22.130660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:07:08.680338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236965+0000","last_change":"2024-04-05T22:22:22.236965+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236965+0000","last_undegraded":"2024-04-05T22:22:22.236965+0000","last_fullsized":"2024-04-05T22:22:22.236965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:54.000983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123876+0000","last_change":"2024-04-05T22:22:22.123876+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123876+0000","last_undegraded":"2024-04-05T22:22:22.123876+0000","last_fullsized":"2024-04-05T22:22:22.123876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.126329+0000","last_change":"2024-04-05T22:22:22.126329+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.126329+0000","last_undegraded":"2024-04-05T22:22:22.126329+0000","last_fullsized":"2024-04-05T22:22:22.126329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124221+0000","last_change":"2024-04-05T22:22:22.124221+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124221+0000","last_undegraded":"2024-04-05T22:22:22.124221+0000","last_fullsized":"2024-04-05T22:22:22.124221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:19:31.591205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120973+0000","last_change":"2024-04-05T22:22:22.120973+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120973+0000","last_undegraded":"2024-04-05T22:22:22.120973+0000","last_fullsized":"2024-04-05T22:22:22.120973+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:27:39.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.125717+0000","last_change":"2024-04-05T22:22:22.125717+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.125717+0000","last_undegraded":"2024-04-05T22:22:22.125717+0000","last_fullsized":"2024-04-05T22:22:22.125717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121292+0000","last_change":"2024-04-05T22:22:22.121292+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121292+0000","last_undegraded":"2024-04-05T22:22:22.121292+0000","last_fullsized":"2024-04-05T22:22:22.121292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:11.473072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.125556+0000","last_change":"2024-04-05T22:22:22.125556+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.125556+0000","last_undegraded":"2024-04-05T22:22:22.125556+0000","last_fullsized":"2024-04-05T22:22:22.125556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:18.168591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.235956+0000","last_change":"2024-04-05T22:22:22.235956+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.235956+0000","last_undegraded":"2024-04-05T22:22:22.235956+0000","last_fullsized":"2024-04-05T22:22:22.235956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:58:26.510282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.129848+0000","last_change":"2024-04-05T22:22:22.129848+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.129848+0000","last_undegraded":"2024-04-05T22:22:22.129848+0000","last_fullsized":"2024-04-05T22:22:22.129848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123267+0000","last_change":"2024-04-05T22:22:22.123267+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123267+0000","last_undegraded":"2024-04-05T22:22:22.123267+0000","last_fullsized":"2024-04-05T22:22:22.123267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121298+0000","last_change":"2024-04-05T22:22:22.121298+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121298+0000","last_undegraded":"2024-04-05T22:22:22.121298+0000","last_fullsized":"2024-04-05T22:22:22.121298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:51.706002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.122252+0000","last_change":"2024-04-05T22:22:22.122252+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.122252+0000","last_undegraded":"2024-04-05T22:22:22.122252+0000","last_fullsized":"2024-04-05T22:22:22.122252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124115+0000","last_change":"2024-04-05T22:22:22.124115+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124115+0000","last_undegraded":"2024-04-05T22:22:22.124115+0000","last_fullsized":"2024-04-05T22:22:22.124115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124595+0000","last_change":"2024-04-05T22:22:22.124595+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124595+0000","last_undegraded":"2024-04-05T22:22:22.124595+0000","last_fullsized":"2024-04-05T22:22:22.124595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123411+0000","last_change":"2024-04-05T22:22:22.123411+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123411+0000","last_undegraded":"2024-04-05T22:22:22.123411+0000","last_fullsized":"2024-04-05T22:22:22.123411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121389+0000","last_change":"2024-04-05T22:22:22.121389+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121389+0000","last_undegraded":"2024-04-05T22:22:22.121389+0000","last_fullsized":"2024-04-05T22:22:22.121389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121211+0000","last_change":"2024-04-05T22:22:22.121211+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121211+0000","last_undegraded":"2024-04-05T22:22:22.121211+0000","last_fullsized":"2024-04-05T22:22:22.121211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:58:42.971651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121330+0000","last_change":"2024-04-05T22:22:22.121330+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121330+0000","last_undegraded":"2024-04-05T22:22:22.121330+0000","last_fullsized":"2024-04-05T22:22:22.121330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:26.099736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.237666+0000","last_change":"2024-04-05T22:22:22.237666+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.237666+0000","last_undegraded":"2024-04-05T22:22:22.237666+0000","last_fullsized":"2024-04-05T22:22:22.237666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130295+0000","last_change":"2024-04-05T22:22:22.130295+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130295+0000","last_undegraded":"2024-04-05T22:22:22.130295+0000","last_fullsized":"2024-04-05T22:22:22.130295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:25.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.122968+0000","last_change":"2024-04-05T22:22:22.122968+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.122968+0000","last_undegraded":"2024-04-05T22:22:22.122968+0000","last_fullsized":"2024-04-05T22:22:22.122968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:13.978598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.128837+0000","last_change":"2024-04-05T22:22:22.128837+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.128837+0000","last_undegraded":"2024-04-05T22:22:22.128837+0000","last_fullsized":"2024-04-05T22:22:22.128837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121916+0000","last_change":"2024-04-05T22:22:22.121916+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121916+0000","last_undegraded":"2024-04-05T22:22:22.121916+0000","last_fullsized":"2024-04-05T22:22:22.121916+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236476+0000","last_change":"2024-04-05T22:22:22.236476+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236476+0000","last_undegraded":"2024-04-05T22:22:22.236476+0000","last_fullsized":"2024-04-05T22:22:22.236476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:23.826984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120956+0000","last_change":"2024-04-05T22:22:22.120956+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120956+0000","last_undegraded":"2024-04-05T22:22:22.120956+0000","last_fullsized":"2024-04-05T22:22:22.120956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236224+0000","last_change":"2024-04-05T22:22:22.236224+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236224+0000","last_undegraded":"2024-04-05T22:22:22.236224+0000","last_fullsized":"2024-04-05T22:22:22.236224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:58.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121093+0000","last_change":"2024-04-05T22:22:22.121093+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121093+0000","last_undegraded":"2024-04-05T22:22:22.121093+0000","last_fullsized":"2024-04-05T22:22:22.121093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121009+0000","last_change":"2024-04-05T22:22:22.121009+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121009+0000","last_undegraded":"2024-04-05T22:22:22.121009+0000","last_fullsized":"2024-04-05T22:22:22.121009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:14:22.438726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120979+0000","last_change":"2024-04-05T22:22:22.120979+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120979+0000","last_undegraded":"2024-04-05T22:22:22.120979+0000","last_fullsized":"2024-04-05T22:22:22.120979+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130600+0000","last_change":"2024-04-05T22:22:22.130600+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130600+0000","last_undegraded":"2024-04-05T22:22:22.130600+0000","last_fullsized":"2024-04-05T22:22:22.130600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:24:22.511159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-05T22:22:16.657875+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:22:16.657875+0000","last_peered":"2024-04-05T22:22:16.657875+0000","last_clean":"2024-04-05T22:22:16.657875+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:22:16.657875+0000","last_undegraded":"2024-04-05T22:22:16.657875+0000","last_fullsized":"2024-04-05T22:22:16.657875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":950272,"data_stored":725224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.747}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.415},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2470000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.29},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.319}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.591},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.1799999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.51}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.349}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.876}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.325}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.131},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.224}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.262},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.29}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4219999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.4710000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.92700000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.393},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.111}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.198},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.022},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4550000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.0429999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.95199999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.026}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.4139999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.6839999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.099}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.369},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.6020000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.87},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.218}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.367}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.244}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.429}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":134856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.629}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.056}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.222}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1559999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.438},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.054},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.96999999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.751}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":731041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.177}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.4710000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.107},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:22:27.985 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:22:28.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.229+0000 7fbd1e200640 1 Processor -- start 2024-04-05T22:22:28.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.229+0000 7fbd1e200640 1 -- start start 2024-04-05T22:22:28.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd1e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbd181597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd1e200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbd180b4650 0x7fbd18159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd1e200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1815b4f0 con 0x7fbd1806a930 2024-04-05T22:22:28.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd1e200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180b5180 con 0x7fbd180b4650 2024-04-05T22:22:28.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd1e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b4bc0 con 0x7fbd1815c990 2024-04-05T22:22:28.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd17400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbd1806a930 0x7fbd180b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52150/0 (socket says 172.21.15.73:52150) 2024-04-05T22:22:28.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.231+0000 7fbd17400640 1 -- 172.21.15.73:0/3035934747 learned_addr learned my addr 172.21.15.73:0/3035934747 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:28.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16a00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbd181597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118120680 0 0) 0x7fbd1815b4f0 con 0x7fbd1806a930 2024-04-05T22:22:28.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd17e00640 1 --2- 172.21.15.73:0/3035934747 >> v2:172.21.15.157:3300/0 conn(0x7fbd180b4650 0x7fbd18159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003520 con 0x7fbd1806a930 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 186780904 0 0) 0x7fbcfc003520 con 0x7fbd1806a930 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 msgr2=0x7fbd181597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbd181597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 >> v2:172.21.15.157:3300/0 conn(0x7fbd180b4650 msgr2=0x7fbd18159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 --2- 172.21.15.73:0/3035934747 >> v2:172.21.15.157:3300/0 conn(0x7fbd180b4650 0x7fbd18159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180b4ea0 con 0x7fbd1806a930 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbd08001670 con 0x7fbd1806a930 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 >> v1:172.21.15.73:6789/0 conn(0x7fbd1806a930 legacy=0x7fbd180b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcfc003970 0x7fbcfc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbcfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbcfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b5180 con 0x7fbcfc0047c0 2024-04-05T22:22:28.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b4bc0 con 0x7fbcfc003970 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.232+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc00dc40 con 0x7fbd1815c990 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbcfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd17e00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbcfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd17400640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcfc003970 0x7fbcfc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 -- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 msgr2=0x7fbcfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd1815c990 0x7fbcfc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 -- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcfc003970 msgr2=0x7fbcfc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcfc003970 0x7fbcfc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.233+0000 7fbd16a00640 1 -- 172.21.15.73:0/3035934747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc003520 con 0x7fbcfc0047c0 2024-04-05T22:22:28.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.234+0000 7fbd16a00640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbcfc004c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400ba00 tx=0x7fbcfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17316d18ff47fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.234+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040325f0 con 0x7fbcfc0047c0 2024-04-05T22:22:28.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.234+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd04032750 con 0x7fbcfc0047c0 2024-04-05T22:22:28.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.234+0000 7fbd16000640 1 -- 172.21.15.73:0/3035934747 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04032ae0 con 0x7fbcfc0047c0 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 -- 172.21.15.73:0/3035934747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 msgr2=0x7fbcfc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 --2- 172.21.15.73:0/3035934747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbcfc004c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400ba00 tx=0x7fbcfc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd17e00640 1 -- 172.21.15.73:0/3035934747 reap_dead start 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 -- 172.21.15.73:0/3035934747 shutdown_connections 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 -- 172.21.15.73:0/3035934747 >> 172.21.15.73:0/3035934747 conn(0x7fbd18071250 msgr2=0x7fbd18071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 -- 172.21.15.73:0/3035934747 shutdown_connections 2024-04-05T22:22:28.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.235+0000 7fbd1e200640 1 -- 172.21.15.73:0/3035934747 wait complete. 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.237+0000 7fbd1e200640 1 Processor -- start 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.237+0000 7fbd1e200640 1 -- start start 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd18159360 0x7fbd180b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd181597e0 0x7fbd180b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180af080 con 0x7fbcfc0047c0 2024-04-05T22:22:28.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180af360 con 0x7fbd181597e0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd1e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180af640 con 0x7fbd18159360 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46514/0 (socket says 172.21.15.73:46514) 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 -- 172.21.15.73:0/721451122 learned_addr learned my addr 172.21.15.73:0/721451122 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 -- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd18159360 msgr2=0x7fbd180b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd18159360 0x7fbd180b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 -- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd181597e0 msgr2=0x7fbd180b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd181597e0 0x7fbd180b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.238+0000 7fbd17400640 1 -- 172.21.15.73:0/721451122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180af920 con 0x7fbcfc0047c0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd17400640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d810 tx=0x7fbd0800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080074e0 con 0x7fbcfc0047c0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08007640 con 0x7fbcfc0047c0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08004a00 con 0x7fbcfc0047c0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd180afc00 con 0x7fbcfc0047c0 2024-04-05T22:22:28.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.239+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180b0660 con 0x7fbcfc0047c0 2024-04-05T22:22:28.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.240+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd180afdf0 con 0x7fbcfc0047c0 2024-04-05T22:22:28.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.241+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd08004b60 con 0x7fbcfc0047c0 2024-04-05T22:22:28.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.242+0000 7fbd0f400640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.242+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd08088d40 con 0x7fbcfc0047c0 2024-04-05T22:22:28.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.242+0000 7fbd16a00640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.243+0000 7fbd16a00640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 0x7fbcf806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0057a0 tx=0x7fbd04032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.246+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbd08053f10 con 0x7fbcfc0047c0 2024-04-05T22:22:28.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.422+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd18068160 con 0x7fbcf806ac30 2024-04-05T22:22:28.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.429+0000 7fbd0f400640 1 -- 172.21.15.73:0/721451122 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276334 (secure 0 0 0) 0x7fbd18068160 con 0x7fbcf806ac30 2024-04-05T22:22:28.433 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:22:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 msgr2=0x7fbcf806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 0x7fbcf806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0057a0 tx=0x7fbd04032000 comp rx=0 tx=0).stop 2024-04-05T22:22:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 msgr2=0x7fbd180b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d810 tx=0x7fbd0800dcd0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 shutdown_connections 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbcf806ac30 0x7fbcf806d0f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd181597e0 0x7fbd180b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd18159360 0x7fbd180b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 --2- 172.21.15.73:0/721451122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcfc0047c0 0x7fbd180b0120 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.434+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 >> 172.21.15.73:0/721451122 conn(0x7fbd18071250 msgr2=0x7fbd180cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.435+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 shutdown_connections 2024-04-05T22:22:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.435+0000 7fbd1e200640 1 -- 172.21.15.73:0/721451122 wait complete. 2024-04-05T22:22:28.452 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-05T22:22:26.510885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236588,"kb_used_data":4016,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738132,"statfs":{"total":773094113280,"available":772851847168,"internally_reserved":0,"allocated":4112384,"data_stored":2294486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.396348"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130660+0000","last_change":"2024-04-05T22:22:22.130660+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130660+0000","last_undegraded":"2024-04-05T22:22:22.130660+0000","last_fullsized":"2024-04-05T22:22:22.130660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:07:08.680338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236965+0000","last_change":"2024-04-05T22:22:22.236965+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236965+0000","last_undegraded":"2024-04-05T22:22:22.236965+0000","last_fullsized":"2024-04-05T22:22:22.236965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:54.000983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123876+0000","last_change":"2024-04-05T22:22:22.123876+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123876+0000","last_undegraded":"2024-04-05T22:22:22.123876+0000","last_fullsized":"2024-04-05T22:22:22.123876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.126329+0000","last_change":"2024-04-05T22:22:22.126329+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.126329+0000","last_undegraded":"2024-04-05T22:22:22.126329+0000","last_fullsized":"2024-04-05T22:22:22.126329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124221+0000","last_change":"2024-04-05T22:22:22.124221+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124221+0000","last_undegraded":"2024-04-05T22:22:22.124221+0000","last_fullsized":"2024-04-05T22:22:22.124221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:19:31.591205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120973+0000","last_change":"2024-04-05T22:22:22.120973+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120973+0000","last_undegraded":"2024-04-05T22:22:22.120973+0000","last_fullsized":"2024-04-05T22:22:22.120973+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:27:39.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.125717+0000","last_change":"2024-04-05T22:22:22.125717+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.125717+0000","last_undegraded":"2024-04-05T22:22:22.125717+0000","last_fullsized":"2024-04-05T22:22:22.125717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121292+0000","last_change":"2024-04-05T22:22:22.121292+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121292+0000","last_undegraded":"2024-04-05T22:22:22.121292+0000","last_fullsized":"2024-04-05T22:22:22.121292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:11.473072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.125556+0000","last_change":"2024-04-05T22:22:22.125556+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.125556+0000","last_undegraded":"2024-04-05T22:22:22.125556+0000","last_fullsized":"2024-04-05T22:22:22.125556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:18.168591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.235956+0000","last_change":"2024-04-05T22:22:22.235956+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.235956+0000","last_undegraded":"2024-04-05T22:22:22.235956+0000","last_fullsized":"2024-04-05T22:22:22.235956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:58:26.510282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.129848+0000","last_change":"2024-04-05T22:22:22.129848+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.129848+0000","last_undegraded":"2024-04-05T22:22:22.129848+0000","last_fullsized":"2024-04-05T22:22:22.129848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123267+0000","last_change":"2024-04-05T22:22:22.123267+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123267+0000","last_undegraded":"2024-04-05T22:22:22.123267+0000","last_fullsized":"2024-04-05T22:22:22.123267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121298+0000","last_change":"2024-04-05T22:22:22.121298+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121298+0000","last_undegraded":"2024-04-05T22:22:22.121298+0000","last_fullsized":"2024-04-05T22:22:22.121298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:51.706002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.122252+0000","last_change":"2024-04-05T22:22:22.122252+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.122252+0000","last_undegraded":"2024-04-05T22:22:22.122252+0000","last_fullsized":"2024-04-05T22:22:22.122252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124115+0000","last_change":"2024-04-05T22:22:22.124115+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124115+0000","last_undegraded":"2024-04-05T22:22:22.124115+0000","last_fullsized":"2024-04-05T22:22:22.124115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.124595+0000","last_change":"2024-04-05T22:22:22.124595+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.124595+0000","last_undegraded":"2024-04-05T22:22:22.124595+0000","last_fullsized":"2024-04-05T22:22:22.124595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.123411+0000","last_change":"2024-04-05T22:22:22.123411+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.123411+0000","last_undegraded":"2024-04-05T22:22:22.123411+0000","last_fullsized":"2024-04-05T22:22:22.123411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121389+0000","last_change":"2024-04-05T22:22:22.121389+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121389+0000","last_undegraded":"2024-04-05T22:22:22.121389+0000","last_fullsized":"2024-04-05T22:22:22.121389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121211+0000","last_change":"2024-04-05T22:22:22.121211+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121211+0000","last_undegraded":"2024-04-05T22:22:22.121211+0000","last_fullsized":"2024-04-05T22:22:22.121211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:58:42.971651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121330+0000","last_change":"2024-04-05T22:22:22.121330+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121330+0000","last_undegraded":"2024-04-05T22:22:22.121330+0000","last_fullsized":"2024-04-05T22:22:22.121330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:26.099736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.237666+0000","last_change":"2024-04-05T22:22:22.237666+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.237666+0000","last_undegraded":"2024-04-05T22:22:22.237666+0000","last_fullsized":"2024-04-05T22:22:22.237666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130295+0000","last_change":"2024-04-05T22:22:22.130295+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130295+0000","last_undegraded":"2024-04-05T22:22:22.130295+0000","last_fullsized":"2024-04-05T22:22:22.130295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:25.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.122968+0000","last_change":"2024-04-05T22:22:22.122968+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.122968+0000","last_undegraded":"2024-04-05T22:22:22.122968+0000","last_fullsized":"2024-04-05T22:22:22.122968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:13.978598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.128837+0000","last_change":"2024-04-05T22:22:22.128837+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.128837+0000","last_undegraded":"2024-04-05T22:22:22.128837+0000","last_fullsized":"2024-04-05T22:22:22.128837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121916+0000","last_change":"2024-04-05T22:22:22.121916+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121916+0000","last_undegraded":"2024-04-05T22:22:22.121916+0000","last_fullsized":"2024-04-05T22:22:22.121916+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236476+0000","last_change":"2024-04-05T22:22:22.236476+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236476+0000","last_undegraded":"2024-04-05T22:22:22.236476+0000","last_fullsized":"2024-04-05T22:22:22.236476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:23.826984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120956+0000","last_change":"2024-04-05T22:22:22.120956+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120956+0000","last_undegraded":"2024-04-05T22:22:22.120956+0000","last_fullsized":"2024-04-05T22:22:22.120956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.236224+0000","last_change":"2024-04-05T22:22:22.236224+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.236224+0000","last_undegraded":"2024-04-05T22:22:22.236224+0000","last_fullsized":"2024-04-05T22:22:22.236224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:58.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121093+0000","last_change":"2024-04-05T22:22:22.121093+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121093+0000","last_undegraded":"2024-04-05T22:22:22.121093+0000","last_fullsized":"2024-04-05T22:22:22.121093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.121009+0000","last_change":"2024-04-05T22:22:22.121009+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.121009+0000","last_undegraded":"2024-04-05T22:22:22.121009+0000","last_fullsized":"2024-04-05T22:22:22.121009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:14:22.438726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.120979+0000","last_change":"2024-04-05T22:22:22.120979+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.120979+0000","last_undegraded":"2024-04-05T22:22:22.120979+0000","last_fullsized":"2024-04-05T22:22:22.120979+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:22:23.114445+0000","last_change":"2024-04-05T22:22:23.114445+0000","last_active":"2024-04-05T22:22:23.114445+0000","last_peered":"2024-04-05T22:22:23.114445+0000","last_clean":"2024-04-05T22:22:23.114445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:23.114445+0000","last_undegraded":"2024-04-05T22:22:23.114445+0000","last_fullsized":"2024-04-05T22:22:23.114445+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-05T22:22:22.130600+0000","last_change":"2024-04-05T22:22:22.130600+0000","last_active":"2024-04-05T22:22:22.099510+0000","last_peered":"2024-04-05T22:22:22.099510+0000","last_clean":"2024-04-05T22:22:22.099510+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:22:22.130600+0000","last_undegraded":"2024-04-05T22:22:22.130600+0000","last_fullsized":"2024-04-05T22:22:22.130600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:24:22.511159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-05T22:22:16.657875+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:22:16.657875+0000","last_peered":"2024-04-05T22:22:16.657875+0000","last_clean":"2024-04-05T22:22:16.657875+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:22:16.657875+0000","last_undegraded":"2024-04-05T22:22:16.657875+0000","last_fullsized":"2024-04-05T22:22:16.657875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":950272,"data_stored":725224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.747}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.415},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2470000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.29},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.319}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.591},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.1799999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.51}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.349}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.876}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.325}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.131},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.224}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.262},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.29}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4219999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.4710000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.92700000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.393},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.111}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.198},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.022},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4550000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.0429999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.95199999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.026}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.4139999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.6839999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.099}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.369},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.6020000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.87},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.218}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.367}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.244}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.429}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":134856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.629}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.056}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.222}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1559999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.438},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.054},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.96999999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.751}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":731041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.177}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.4710000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.107},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:22:28.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:22:28.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.706+0000 7f60dbc00640 1 Processor -- start 2024-04-05T22:22:28.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.707+0000 7f60dbc00640 1 -- start start 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.708+0000 7f60dbc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60d415d200 0x7f60d40acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.708+0000 7f60dbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.708+0000 7f60dbc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d406aa40 con 0x7f60d415ce20 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.708+0000 7f60dbc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d409d110 con 0x7f60d415d200 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.708+0000 7f60dbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d406ad20 con 0x7f60d40ce630 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60d415d200 0x7f60d40acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60d415d200 0x7f60d40acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46526/0 (socket says 172.21.15.73:46526) 2024-04-05T22:22:28.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 -- 172.21.15.73:0/289177559 learned_addr learned my addr 172.21.15.73:0/289177559 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:28.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d9c00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 -- 172.21.15.73:0/289177559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 msgr2=0x7f60d40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d40ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 -- 172.21.15.73:0/289177559 >> v1:172.21.15.157:6789/0 conn(0x7f60d415ce20 legacy=0x7f60d40c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:28.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.709+0000 7f60d3e00640 1 -- 172.21.15.73:0/289177559 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40cde30 con 0x7f60d415d200 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.710+0000 7f60d3e00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415d200 0x7f60d40acfd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f60d4160ab0 tx=0x7f60c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd1ca936d4f619c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.710+0000 7f60d3400640 1 -- 172.21.15.73:0/289177559 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0031640 con 0x7f60d415d200 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.710+0000 7f60d3400640 1 -- 172.21.15.73:0/289177559 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c00317a0 con 0x7f60d415d200 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.710+0000 7f60d3400640 1 -- 172.21.15.73:0/289177559 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0031b30 con 0x7f60d415d200 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 -- 172.21.15.73:0/289177559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415d200 msgr2=0x7f60d40acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415d200 0x7f60d40acfd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f60d4160ab0 tx=0x7f60c002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 -- 172.21.15.73:0/289177559 shutdown_connections 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 --2- 172.21.15.73:0/289177559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415d200 0x7f60d40acfd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 -- 172.21.15.73:0/289177559 >> 172.21.15.73:0/289177559 conn(0x7f60d40cf190 msgr2=0x7f60d40cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 -- 172.21.15.73:0/289177559 shutdown_connections 2024-04-05T22:22:28.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.711+0000 7f60dbc00640 1 -- 172.21.15.73:0/289177559 wait complete. 2024-04-05T22:22:28.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.713+0000 7f60dbc00640 1 Processor -- start 2024-04-05T22:22:28.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.713+0000 7f60dbc00640 1 -- start start 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d409c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d415d200 0x7f60d415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b4240 con 0x7f60d415ce20 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b3630 con 0x7f60d415d200 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60dbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b3910 con 0x7f60d40ce630 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46534/0 (socket says 172.21.15.73:46534) 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.714+0000 7f60d3e00640 1 -- 172.21.15.73:0/2375409621 learned_addr learned my addr 172.21.15.73:0/2375409621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d9200640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d409c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 -- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 msgr2=0x7f60d409c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d409c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 -- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d415d200 msgr2=0x7f60d415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d415d200 0x7f60d415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 -- 172.21.15.73:0/2375409621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00094d0 con 0x7f60d415ce20 2024-04-05T22:22:28.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d9200640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d409c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d3e00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f60c002b930 tx=0x7f60c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c003ac60 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c003ae00 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0044630 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.715+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d40ca3a0 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.716+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d40b3ed0 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.717+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d40cda70 con 0x7f60d415ce20 2024-04-05T22:22:28.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.717+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f60c002f050 con 0x7f60d415ce20 2024-04-05T22:22:28.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.719+0000 7f60d0c00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 0x7f60b406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:28.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.719+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60c00ab430 con 0x7f60d415ce20 2024-04-05T22:22:28.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.719+0000 7f60d9200640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 0x7f60b406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:28.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.720+0000 7f60d9200640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 0x7f60b406d0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007fd0 tx=0x7f60c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:28.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.724+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f60c0076570 con 0x7f60d415ce20 2024-04-05T22:22:28.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.913+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60d40ca220 con 0x7f60b406ac30 2024-04-05T22:22:28.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.928+0000 7f60d0c00640 1 -- 172.21.15.73:0/2375409621 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287168 (secure 0 0 0) 0x7f60c8001370 con 0x7f60b406ac30 2024-04-05T22:22:28.932 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 msgr2=0x7f60b406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 0x7f60b406d0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007fd0 tx=0x7f60c8009040 comp rx=0 tx=0).stop 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 msgr2=0x7f60d409a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f60c002b930 tx=0x7f60c00046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 shutdown_connections 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f60b406ac30 0x7f60b406d0f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60d415d200 0x7f60d415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60d415ce20 0x7f60d409a470 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 --2- 172.21.15.73:0/2375409621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60d40ce630 0x7f60d409c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:28.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.934+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 >> 172.21.15.73:0/2375409621 conn(0x7f60d40cf190 msgr2=0x7f60d40afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:28.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.935+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 shutdown_connections 2024-04-05T22:22:28.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:28.935+0000 7f60dbc00640 1 -- 172.21.15.73:0/2375409621 wait complete. 2024-04-05T22:22:28.954 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-05T22:22:28.511523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236948,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737772,"statfs":{"total":773094113280,"available":772851478528,"internally_reserved":0,"allocated":4497408,"data_stored":2465580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001367"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.671468+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.671239+0000","last_became_peered":"2024-04-05T22:22:23.671239+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:17:31.785954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694516+0000","last_change":"2024-04-05T22:22:23.652316+0000","last_active":"2024-04-05T22:22:24.694516+0000","last_peered":"2024-04-05T22:22:24.694516+0000","last_clean":"2024-04-05T22:22:24.694516+0000","last_became_active":"2024-04-05T22:22:23.651989+0000","last_became_peered":"2024-04-05T22:22:23.651989+0000","last_unstale":"2024-04-05T22:22:24.694516+0000","last_undegraded":"2024-04-05T22:22:24.694516+0000","last_fullsized":"2024-04-05T22:22:24.694516+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639834+0000","last_change":"2024-04-05T22:22:23.672439+0000","last_active":"2024-04-05T22:22:24.639834+0000","last_peered":"2024-04-05T22:22:24.639834+0000","last_clean":"2024-04-05T22:22:24.639834+0000","last_became_active":"2024-04-05T22:22:23.672112+0000","last_became_peered":"2024-04-05T22:22:23.672112+0000","last_unstale":"2024-04-05T22:22:24.639834+0000","last_undegraded":"2024-04-05T22:22:24.639834+0000","last_fullsized":"2024-04-05T22:22:24.639834+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637508+0000","last_change":"2024-04-05T22:22:23.657494+0000","last_active":"2024-04-05T22:22:24.637508+0000","last_peered":"2024-04-05T22:22:24.637508+0000","last_clean":"2024-04-05T22:22:24.637508+0000","last_became_active":"2024-04-05T22:22:23.657164+0000","last_became_peered":"2024-04-05T22:22:23.657164+0000","last_unstale":"2024-04-05T22:22:24.637508+0000","last_undegraded":"2024-04-05T22:22:24.637508+0000","last_fullsized":"2024-04-05T22:22:24.637508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635155+0000","last_change":"2024-04-05T22:22:23.642932+0000","last_active":"2024-04-05T22:22:24.635155+0000","last_peered":"2024-04-05T22:22:24.635155+0000","last_clean":"2024-04-05T22:22:24.635155+0000","last_became_active":"2024-04-05T22:22:23.642640+0000","last_became_peered":"2024-04-05T22:22:23.642640+0000","last_unstale":"2024-04-05T22:22:24.635155+0000","last_undegraded":"2024-04-05T22:22:24.635155+0000","last_fullsized":"2024-04-05T22:22:24.635155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693992+0000","last_change":"2024-04-05T22:22:23.668411+0000","last_active":"2024-04-05T22:22:24.693992+0000","last_peered":"2024-04-05T22:22:24.693992+0000","last_clean":"2024-04-05T22:22:24.693992+0000","last_became_active":"2024-04-05T22:22:23.668135+0000","last_became_peered":"2024-04-05T22:22:23.668135+0000","last_unstale":"2024-04-05T22:22:24.693992+0000","last_undegraded":"2024-04-05T22:22:24.693992+0000","last_fullsized":"2024-04-05T22:22:24.693992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697247+0000","last_change":"2024-04-05T22:22:23.661512+0000","last_active":"2024-04-05T22:22:24.697247+0000","last_peered":"2024-04-05T22:22:24.697247+0000","last_clean":"2024-04-05T22:22:24.697247+0000","last_became_active":"2024-04-05T22:22:23.661235+0000","last_became_peered":"2024-04-05T22:22:23.661235+0000","last_unstale":"2024-04-05T22:22:24.697247+0000","last_undegraded":"2024-04-05T22:22:24.697247+0000","last_fullsized":"2024-04-05T22:22:24.697247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696214+0000","last_change":"2024-04-05T22:22:23.652995+0000","last_active":"2024-04-05T22:22:24.696214+0000","last_peered":"2024-04-05T22:22:24.696214+0000","last_clean":"2024-04-05T22:22:24.696214+0000","last_became_active":"2024-04-05T22:22:23.652715+0000","last_became_peered":"2024-04-05T22:22:23.652715+0000","last_unstale":"2024-04-05T22:22:24.696214+0000","last_undegraded":"2024-04-05T22:22:24.696214+0000","last_fullsized":"2024-04-05T22:22:24.696214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:27:23.769941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699562+0000","last_change":"2024-04-05T22:22:23.651001+0000","last_active":"2024-04-05T22:22:24.699562+0000","last_peered":"2024-04-05T22:22:24.699562+0000","last_clean":"2024-04-05T22:22:24.699562+0000","last_became_active":"2024-04-05T22:22:23.650633+0000","last_became_peered":"2024-04-05T22:22:23.650633+0000","last_unstale":"2024-04-05T22:22:24.699562+0000","last_undegraded":"2024-04-05T22:22:24.699562+0000","last_fullsized":"2024-04-05T22:22:24.699562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:23.999212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637115+0000","last_change":"2024-04-05T22:22:23.636389+0000","last_active":"2024-04-05T22:22:24.637115+0000","last_peered":"2024-04-05T22:22:24.637115+0000","last_clean":"2024-04-05T22:22:24.637115+0000","last_became_active":"2024-04-05T22:22:23.636028+0000","last_became_peered":"2024-04-05T22:22:23.636028+0000","last_unstale":"2024-04-05T22:22:24.637115+0000","last_undegraded":"2024-04-05T22:22:24.637115+0000","last_fullsized":"2024-04-05T22:22:24.637115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697848+0000","last_change":"2024-04-05T22:22:23.639648+0000","last_active":"2024-04-05T22:22:24.697848+0000","last_peered":"2024-04-05T22:22:24.697848+0000","last_clean":"2024-04-05T22:22:24.697848+0000","last_became_active":"2024-04-05T22:22:23.636042+0000","last_became_peered":"2024-04-05T22:22:23.636042+0000","last_unstale":"2024-04-05T22:22:24.697848+0000","last_undegraded":"2024-04-05T22:22:24.697848+0000","last_fullsized":"2024-04-05T22:22:24.697848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:19.468442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639272+0000","last_change":"2024-04-05T22:22:23.667880+0000","last_active":"2024-04-05T22:22:24.639272+0000","last_peered":"2024-04-05T22:22:24.639272+0000","last_clean":"2024-04-05T22:22:24.639272+0000","last_became_active":"2024-04-05T22:22:23.660507+0000","last_became_peered":"2024-04-05T22:22:23.660507+0000","last_unstale":"2024-04-05T22:22:24.639272+0000","last_undegraded":"2024-04-05T22:22:24.639272+0000","last_fullsized":"2024-04-05T22:22:24.639272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635308+0000","last_change":"2024-04-05T22:22:23.650060+0000","last_active":"2024-04-05T22:22:24.635308+0000","last_peered":"2024-04-05T22:22:24.635308+0000","last_clean":"2024-04-05T22:22:24.635308+0000","last_became_active":"2024-04-05T22:22:23.647699+0000","last_became_peered":"2024-04-05T22:22:23.647699+0000","last_unstale":"2024-04-05T22:22:24.635308+0000","last_undegraded":"2024-04-05T22:22:24.635308+0000","last_fullsized":"2024-04-05T22:22:24.635308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693997+0000","last_change":"2024-04-05T22:22:23.644515+0000","last_active":"2024-04-05T22:22:24.693997+0000","last_peered":"2024-04-05T22:22:24.693997+0000","last_clean":"2024-04-05T22:22:24.693997+0000","last_became_active":"2024-04-05T22:22:23.643820+0000","last_became_peered":"2024-04-05T22:22:23.643820+0000","last_unstale":"2024-04-05T22:22:24.693997+0000","last_undegraded":"2024-04-05T22:22:24.693997+0000","last_fullsized":"2024-04-05T22:22:24.693997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639487+0000","last_change":"2024-04-05T22:22:23.672903+0000","last_active":"2024-04-05T22:22:24.639487+0000","last_peered":"2024-04-05T22:22:24.639487+0000","last_clean":"2024-04-05T22:22:24.639487+0000","last_became_active":"2024-04-05T22:22:23.670922+0000","last_became_peered":"2024-04-05T22:22:23.670922+0000","last_unstale":"2024-04-05T22:22:24.639487+0000","last_undegraded":"2024-04-05T22:22:24.639487+0000","last_fullsized":"2024-04-05T22:22:24.639487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696950+0000","last_change":"2024-04-05T22:22:23.643442+0000","last_active":"2024-04-05T22:22:24.696950+0000","last_peered":"2024-04-05T22:22:24.696950+0000","last_clean":"2024-04-05T22:22:24.696950+0000","last_became_active":"2024-04-05T22:22:23.642787+0000","last_became_peered":"2024-04-05T22:22:23.642787+0000","last_unstale":"2024-04-05T22:22:24.696950+0000","last_undegraded":"2024-04-05T22:22:24.696950+0000","last_fullsized":"2024-04-05T22:22:24.696950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697924+0000","last_change":"2024-04-05T22:22:23.662150+0000","last_active":"2024-04-05T22:22:24.697924+0000","last_peered":"2024-04-05T22:22:24.697924+0000","last_clean":"2024-04-05T22:22:24.697924+0000","last_became_active":"2024-04-05T22:22:23.661297+0000","last_became_peered":"2024-04-05T22:22:23.661297+0000","last_unstale":"2024-04-05T22:22:24.697924+0000","last_undegraded":"2024-04-05T22:22:24.697924+0000","last_fullsized":"2024-04-05T22:22:24.697924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694661+0000","last_change":"2024-04-05T22:22:23.636014+0000","last_active":"2024-04-05T22:22:24.694661+0000","last_peered":"2024-04-05T22:22:24.694661+0000","last_clean":"2024-04-05T22:22:24.694661+0000","last_became_active":"2024-04-05T22:22:23.635758+0000","last_became_peered":"2024-04-05T22:22:23.635758+0000","last_unstale":"2024-04-05T22:22:24.694661+0000","last_undegraded":"2024-04-05T22:22:24.694661+0000","last_fullsized":"2024-04-05T22:22:24.694661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694137+0000","last_change":"2024-04-05T22:22:23.635424+0000","last_active":"2024-04-05T22:22:24.694137+0000","last_peered":"2024-04-05T22:22:24.694137+0000","last_clean":"2024-04-05T22:22:24.694137+0000","last_became_active":"2024-04-05T22:22:23.635166+0000","last_became_peered":"2024-04-05T22:22:23.635166+0000","last_unstale":"2024-04-05T22:22:24.694137+0000","last_undegraded":"2024-04-05T22:22:24.694137+0000","last_fullsized":"2024-04-05T22:22:24.694137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635608+0000","last_change":"2024-04-05T22:22:23.669393+0000","last_active":"2024-04-05T22:22:24.635608+0000","last_peered":"2024-04-05T22:22:24.635608+0000","last_clean":"2024-04-05T22:22:24.635608+0000","last_became_active":"2024-04-05T22:22:23.668797+0000","last_became_peered":"2024-04-05T22:22:23.668797+0000","last_unstale":"2024-04-05T22:22:24.635608+0000","last_undegraded":"2024-04-05T22:22:24.635608+0000","last_fullsized":"2024-04-05T22:22:24.635608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698740+0000","last_change":"2024-04-05T22:22:23.657832+0000","last_active":"2024-04-05T22:22:24.698740+0000","last_peered":"2024-04-05T22:22:24.698740+0000","last_clean":"2024-04-05T22:22:24.698740+0000","last_became_active":"2024-04-05T22:22:23.657507+0000","last_became_peered":"2024-04-05T22:22:23.657507+0000","last_unstale":"2024-04-05T22:22:24.698740+0000","last_undegraded":"2024-04-05T22:22:24.698740+0000","last_fullsized":"2024-04-05T22:22:24.698740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698901+0000","last_change":"2024-04-05T22:22:23.648571+0000","last_active":"2024-04-05T22:22:24.698901+0000","last_peered":"2024-04-05T22:22:24.698901+0000","last_clean":"2024-04-05T22:22:24.698901+0000","last_became_active":"2024-04-05T22:22:23.647134+0000","last_became_peered":"2024-04-05T22:22:23.647134+0000","last_unstale":"2024-04-05T22:22:24.698901+0000","last_undegraded":"2024-04-05T22:22:24.698901+0000","last_fullsized":"2024-04-05T22:22:24.698901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695306+0000","last_change":"2024-04-05T22:22:23.657788+0000","last_active":"2024-04-05T22:22:24.695306+0000","last_peered":"2024-04-05T22:22:24.695306+0000","last_clean":"2024-04-05T22:22:24.695306+0000","last_became_active":"2024-04-05T22:22:23.657531+0000","last_became_peered":"2024-04-05T22:22:23.657531+0000","last_unstale":"2024-04-05T22:22:24.695306+0000","last_undegraded":"2024-04-05T22:22:24.695306+0000","last_fullsized":"2024-04-05T22:22:24.695306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639005+0000","last_change":"2024-04-05T22:22:23.658718+0000","last_active":"2024-04-05T22:22:24.639005+0000","last_peered":"2024-04-05T22:22:24.639005+0000","last_clean":"2024-04-05T22:22:24.639005+0000","last_became_active":"2024-04-05T22:22:23.658443+0000","last_became_peered":"2024-04-05T22:22:23.658443+0000","last_unstale":"2024-04-05T22:22:24.639005+0000","last_undegraded":"2024-04-05T22:22:24.639005+0000","last_fullsized":"2024-04-05T22:22:24.639005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637646+0000","last_change":"2024-04-05T22:22:23.646258+0000","last_active":"2024-04-05T22:22:24.637646+0000","last_peered":"2024-04-05T22:22:24.637646+0000","last_clean":"2024-04-05T22:22:24.637646+0000","last_became_active":"2024-04-05T22:22:23.645504+0000","last_became_peered":"2024-04-05T22:22:23.645504+0000","last_unstale":"2024-04-05T22:22:24.637646+0000","last_undegraded":"2024-04-05T22:22:24.637646+0000","last_fullsized":"2024-04-05T22:22:24.637646+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694597+0000","last_change":"2024-04-05T22:22:23.661228+0000","last_active":"2024-04-05T22:22:24.694597+0000","last_peered":"2024-04-05T22:22:24.694597+0000","last_clean":"2024-04-05T22:22:24.694597+0000","last_became_active":"2024-04-05T22:22:23.660897+0000","last_became_peered":"2024-04-05T22:22:23.660897+0000","last_unstale":"2024-04-05T22:22:24.694597+0000","last_undegraded":"2024-04-05T22:22:24.694597+0000","last_fullsized":"2024-04-05T22:22:24.694597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:14.374696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637429+0000","last_change":"2024-04-05T22:22:23.658697+0000","last_active":"2024-04-05T22:22:24.637429+0000","last_peered":"2024-04-05T22:22:24.637429+0000","last_clean":"2024-04-05T22:22:24.637429+0000","last_became_active":"2024-04-05T22:22:23.658228+0000","last_became_peered":"2024-04-05T22:22:23.658228+0000","last_unstale":"2024-04-05T22:22:24.637429+0000","last_undegraded":"2024-04-05T22:22:24.637429+0000","last_fullsized":"2024-04-05T22:22:24.637429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639456+0000","last_change":"2024-04-05T22:22:23.660227+0000","last_active":"2024-04-05T22:22:24.639456+0000","last_peered":"2024-04-05T22:22:24.639456+0000","last_clean":"2024-04-05T22:22:24.639456+0000","last_became_active":"2024-04-05T22:22:23.659896+0000","last_became_peered":"2024-04-05T22:22:23.659896+0000","last_unstale":"2024-04-05T22:22:24.639456+0000","last_undegraded":"2024-04-05T22:22:24.639456+0000","last_fullsized":"2024-04-05T22:22:24.639456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698795+0000","last_change":"2024-04-05T22:22:23.649270+0000","last_active":"2024-04-05T22:22:24.698795+0000","last_peered":"2024-04-05T22:22:24.698795+0000","last_clean":"2024-04-05T22:22:24.698795+0000","last_became_active":"2024-04-05T22:22:23.648854+0000","last_became_peered":"2024-04-05T22:22:23.648854+0000","last_unstale":"2024-04-05T22:22:24.698795+0000","last_undegraded":"2024-04-05T22:22:24.698795+0000","last_fullsized":"2024-04-05T22:22:24.698795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697381+0000","last_change":"2024-04-05T22:22:23.656696+0000","last_active":"2024-04-05T22:22:24.697381+0000","last_peered":"2024-04-05T22:22:24.697381+0000","last_clean":"2024-04-05T22:22:24.697381+0000","last_became_active":"2024-04-05T22:22:23.656289+0000","last_became_peered":"2024-04-05T22:22:23.656289+0000","last_unstale":"2024-04-05T22:22:24.697381+0000","last_undegraded":"2024-04-05T22:22:24.697381+0000","last_fullsized":"2024-04-05T22:22:24.697381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:58.596322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696198+0000","last_change":"2024-04-05T22:22:23.661467+0000","last_active":"2024-04-05T22:22:24.696198+0000","last_peered":"2024-04-05T22:22:24.696198+0000","last_clean":"2024-04-05T22:22:24.696198+0000","last_became_active":"2024-04-05T22:22:23.661164+0000","last_became_peered":"2024-04-05T22:22:23.661164+0000","last_unstale":"2024-04-05T22:22:24.696198+0000","last_undegraded":"2024-04-05T22:22:24.696198+0000","last_fullsized":"2024-04-05T22:22:24.696198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:24.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696451+0000","last_change":"2024-04-05T22:22:23.654718+0000","last_active":"2024-04-05T22:22:24.696451+0000","last_peered":"2024-04-05T22:22:24.696451+0000","last_clean":"2024-04-05T22:22:24.696451+0000","last_became_active":"2024-04-05T22:22:23.654401+0000","last_became_peered":"2024-04-05T22:22:23.654401+0000","last_unstale":"2024-04-05T22:22:24.696451+0000","last_undegraded":"2024-04-05T22:22:24.696451+0000","last_fullsized":"2024-04-05T22:22:24.696451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:00:31.890854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695252+0000","last_change":"2024-04-05T22:22:23.656947+0000","last_active":"2024-04-05T22:22:24.695252+0000","last_peered":"2024-04-05T22:22:24.695252+0000","last_clean":"2024-04-05T22:22:24.695252+0000","last_became_active":"2024-04-05T22:22:23.656762+0000","last_became_peered":"2024-04-05T22:22:23.656762+0000","last_unstale":"2024-04-05T22:22:24.695252+0000","last_undegraded":"2024-04-05T22:22:24.695252+0000","last_fullsized":"2024-04-05T22:22:24.695252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673739+0000","last_change":"2024-04-05T22:22:23.167846+0000","last_active":"2024-04-05T22:22:23.673739+0000","last_peered":"2024-04-05T22:22:23.673739+0000","last_clean":"2024-04-05T22:22:23.673739+0000","last_became_active":"2024-04-05T22:22:23.167499+0000","last_became_peered":"2024-04-05T22:22:23.167499+0000","last_unstale":"2024-04-05T22:22:23.673739+0000","last_undegraded":"2024-04-05T22:22:23.673739+0000","last_fullsized":"2024-04-05T22:22:23.673739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:07:08.680338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696281+0000","last_change":"2024-04-05T22:22:23.665407+0000","last_active":"2024-04-05T22:22:24.696281+0000","last_peered":"2024-04-05T22:22:24.696281+0000","last_clean":"2024-04-05T22:22:24.696281+0000","last_became_active":"2024-04-05T22:22:23.661923+0000","last_became_peered":"2024-04-05T22:22:23.661923+0000","last_unstale":"2024-04-05T22:22:24.696281+0000","last_undegraded":"2024-04-05T22:22:24.696281+0000","last_fullsized":"2024-04-05T22:22:24.696281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:41:16.993667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656516+0000","last_change":"2024-04-05T22:22:23.173935+0000","last_active":"2024-04-05T22:22:23.656516+0000","last_peered":"2024-04-05T22:22:23.656516+0000","last_clean":"2024-04-05T22:22:23.656516+0000","last_became_active":"2024-04-05T22:22:23.173626+0000","last_became_peered":"2024-04-05T22:22:23.173626+0000","last_unstale":"2024-04-05T22:22:23.656516+0000","last_undegraded":"2024-04-05T22:22:23.656516+0000","last_fullsized":"2024-04-05T22:22:23.656516+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:54.000983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699314+0000","last_change":"2024-04-05T22:22:23.645196+0000","last_active":"2024-04-05T22:22:24.699314+0000","last_peered":"2024-04-05T22:22:24.699314+0000","last_clean":"2024-04-05T22:22:24.699314+0000","last_became_active":"2024-04-05T22:22:23.644571+0000","last_became_peered":"2024-04-05T22:22:23.644571+0000","last_unstale":"2024-04-05T22:22:24.699314+0000","last_undegraded":"2024-04-05T22:22:24.699314+0000","last_fullsized":"2024-04-05T22:22:24.699314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.663712+0000","last_change":"2024-04-05T22:22:23.167036+0000","last_active":"2024-04-05T22:22:23.663712+0000","last_peered":"2024-04-05T22:22:23.663712+0000","last_clean":"2024-04-05T22:22:23.663712+0000","last_became_active":"2024-04-05T22:22:23.166675+0000","last_became_peered":"2024-04-05T22:22:23.166675+0000","last_unstale":"2024-04-05T22:22:23.663712+0000","last_undegraded":"2024-04-05T22:22:23.663712+0000","last_fullsized":"2024-04-05T22:22:23.663712+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637547+0000","last_change":"2024-04-05T22:22:23.653873+0000","last_active":"2024-04-05T22:22:24.637547+0000","last_peered":"2024-04-05T22:22:24.637547+0000","last_clean":"2024-04-05T22:22:24.637547+0000","last_became_active":"2024-04-05T22:22:23.653328+0000","last_became_peered":"2024-04-05T22:22:23.653328+0000","last_unstale":"2024-04-05T22:22:24.637547+0000","last_undegraded":"2024-04-05T22:22:24.637547+0000","last_fullsized":"2024-04-05T22:22:24.637547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.659545+0000","last_change":"2024-04-05T22:22:23.165579+0000","last_active":"2024-04-05T22:22:23.659545+0000","last_peered":"2024-04-05T22:22:23.659545+0000","last_clean":"2024-04-05T22:22:23.659545+0000","last_became_active":"2024-04-05T22:22:23.164825+0000","last_became_peered":"2024-04-05T22:22:23.164825+0000","last_unstale":"2024-04-05T22:22:23.659545+0000","last_undegraded":"2024-04-05T22:22:23.659545+0000","last_fullsized":"2024-04-05T22:22:23.659545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697512+0000","last_change":"2024-04-05T22:22:23.651254+0000","last_active":"2024-04-05T22:22:24.697512+0000","last_peered":"2024-04-05T22:22:24.697512+0000","last_clean":"2024-04-05T22:22:24.697512+0000","last_became_active":"2024-04-05T22:22:23.650666+0000","last_became_peered":"2024-04-05T22:22:23.650666+0000","last_unstale":"2024-04-05T22:22:24.697512+0000","last_undegraded":"2024-04-05T22:22:24.697512+0000","last_fullsized":"2024-04-05T22:22:24.697512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668306+0000","last_change":"2024-04-05T22:22:23.161130+0000","last_active":"2024-04-05T22:22:23.668306+0000","last_peered":"2024-04-05T22:22:23.668306+0000","last_clean":"2024-04-05T22:22:23.668306+0000","last_became_active":"2024-04-05T22:22:23.160816+0000","last_became_peered":"2024-04-05T22:22:23.160816+0000","last_unstale":"2024-04-05T22:22:23.668306+0000","last_undegraded":"2024-04-05T22:22:23.668306+0000","last_fullsized":"2024-04-05T22:22:23.668306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:19:31.591205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.638992+0000","last_change":"2024-04-05T22:22:23.663030+0000","last_active":"2024-04-05T22:22:24.638992+0000","last_peered":"2024-04-05T22:22:24.638992+0000","last_clean":"2024-04-05T22:22:24.638992+0000","last_became_active":"2024-04-05T22:22:23.662634+0000","last_became_peered":"2024-04-05T22:22:23.662634+0000","last_unstale":"2024-04-05T22:22:24.638992+0000","last_undegraded":"2024-04-05T22:22:24.638992+0000","last_fullsized":"2024-04-05T22:22:24.638992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.669884+0000","last_change":"2024-04-05T22:22:23.171512+0000","last_active":"2024-04-05T22:22:23.669884+0000","last_peered":"2024-04-05T22:22:23.669884+0000","last_clean":"2024-04-05T22:22:23.669884+0000","last_became_active":"2024-04-05T22:22:23.171093+0000","last_became_peered":"2024-04-05T22:22:23.171093+0000","last_unstale":"2024-04-05T22:22:23.669884+0000","last_undegraded":"2024-04-05T22:22:23.669884+0000","last_fullsized":"2024-04-05T22:22:23.669884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:27:39.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.662602+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.662280+0000","last_became_peered":"2024-04-05T22:22:23.662280+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627380+0000","last_change":"2024-04-05T22:22:23.161421+0000","last_active":"2024-04-05T22:22:23.627380+0000","last_peered":"2024-04-05T22:22:23.627380+0000","last_clean":"2024-04-05T22:22:23.627380+0000","last_became_active":"2024-04-05T22:22:23.161106+0000","last_became_peered":"2024-04-05T22:22:23.161106+0000","last_unstale":"2024-04-05T22:22:23.627380+0000","last_undegraded":"2024-04-05T22:22:23.627380+0000","last_fullsized":"2024-04-05T22:22:23.627380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636847+0000","last_change":"2024-04-05T22:22:23.665869+0000","last_active":"2024-04-05T22:22:24.636847+0000","last_peered":"2024-04-05T22:22:24.636847+0000","last_clean":"2024-04-05T22:22:24.636847+0000","last_became_active":"2024-04-05T22:22:23.665635+0000","last_became_peered":"2024-04-05T22:22:23.665635+0000","last_unstale":"2024-04-05T22:22:24.636847+0000","last_undegraded":"2024-04-05T22:22:24.636847+0000","last_fullsized":"2024-04-05T22:22:24.636847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645193+0000","last_change":"2024-04-05T22:22:23.161442+0000","last_active":"2024-04-05T22:22:23.645193+0000","last_peered":"2024-04-05T22:22:23.645193+0000","last_clean":"2024-04-05T22:22:23.645193+0000","last_became_active":"2024-04-05T22:22:23.161144+0000","last_became_peered":"2024-04-05T22:22:23.161144+0000","last_unstale":"2024-04-05T22:22:23.645193+0000","last_undegraded":"2024-04-05T22:22:23.645193+0000","last_fullsized":"2024-04-05T22:22:23.645193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:11.473072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640122+0000","last_change":"2024-04-05T22:22:23.668335+0000","last_active":"2024-04-05T22:22:24.640122+0000","last_peered":"2024-04-05T22:22:24.640122+0000","last_clean":"2024-04-05T22:22:24.640122+0000","last_became_active":"2024-04-05T22:22:23.667779+0000","last_became_peered":"2024-04-05T22:22:23.667779+0000","last_unstale":"2024-04-05T22:22:24.640122+0000","last_undegraded":"2024-04-05T22:22:24.640122+0000","last_fullsized":"2024-04-05T22:22:24.640122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.630023+0000","last_change":"2024-04-05T22:22:23.170597+0000","last_active":"2024-04-05T22:22:23.630023+0000","last_peered":"2024-04-05T22:22:23.630023+0000","last_clean":"2024-04-05T22:22:23.630023+0000","last_became_active":"2024-04-05T22:22:23.170261+0000","last_became_peered":"2024-04-05T22:22:23.170261+0000","last_unstale":"2024-04-05T22:22:23.630023+0000","last_undegraded":"2024-04-05T22:22:23.630023+0000","last_fullsized":"2024-04-05T22:22:23.630023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:18.168591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694569+0000","last_change":"2024-04-05T22:22:23.637101+0000","last_active":"2024-04-05T22:22:24.694569+0000","last_peered":"2024-04-05T22:22:24.694569+0000","last_clean":"2024-04-05T22:22:24.694569+0000","last_became_active":"2024-04-05T22:22:23.636845+0000","last_became_peered":"2024-04-05T22:22:23.636845+0000","last_unstale":"2024-04-05T22:22:24.694569+0000","last_undegraded":"2024-04-05T22:22:24.694569+0000","last_fullsized":"2024-04-05T22:22:24.694569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646846+0000","last_change":"2024-04-05T22:22:23.159233+0000","last_active":"2024-04-05T22:22:23.646846+0000","last_peered":"2024-04-05T22:22:23.646846+0000","last_clean":"2024-04-05T22:22:23.646846+0000","last_became_active":"2024-04-05T22:22:23.158972+0000","last_became_peered":"2024-04-05T22:22:23.158972+0000","last_unstale":"2024-04-05T22:22:23.646846+0000","last_undegraded":"2024-04-05T22:22:23.646846+0000","last_fullsized":"2024-04-05T22:22:23.646846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:58:26.510282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694963+0000","last_change":"2024-04-05T22:22:23.660328+0000","last_active":"2024-04-05T22:22:24.694963+0000","last_peered":"2024-04-05T22:22:24.694963+0000","last_clean":"2024-04-05T22:22:24.694963+0000","last_became_active":"2024-04-05T22:22:23.659665+0000","last_became_peered":"2024-04-05T22:22:23.659665+0000","last_unstale":"2024-04-05T22:22:24.694963+0000","last_undegraded":"2024-04-05T22:22:24.694963+0000","last_fullsized":"2024-04-05T22:22:24.694963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:33:30.562929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.674054+0000","last_change":"2024-04-05T22:22:23.174204+0000","last_active":"2024-04-05T22:22:23.674054+0000","last_peered":"2024-04-05T22:22:23.674054+0000","last_clean":"2024-04-05T22:22:23.674054+0000","last_became_active":"2024-04-05T22:22:23.173693+0000","last_became_peered":"2024-04-05T22:22:23.173693+0000","last_unstale":"2024-04-05T22:22:23.674054+0000","last_undegraded":"2024-04-05T22:22:23.674054+0000","last_fullsized":"2024-04-05T22:22:23.674054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639858+0000","last_change":"2024-04-05T22:22:23.672617+0000","last_active":"2024-04-05T22:22:24.639858+0000","last_peered":"2024-04-05T22:22:24.639858+0000","last_clean":"2024-04-05T22:22:24.639858+0000","last_became_active":"2024-04-05T22:22:23.670917+0000","last_became_peered":"2024-04-05T22:22:23.670917+0000","last_unstale":"2024-04-05T22:22:24.639858+0000","last_undegraded":"2024-04-05T22:22:24.639858+0000","last_fullsized":"2024-04-05T22:22:24.639858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673235+0000","last_change":"2024-04-05T22:22:23.163943+0000","last_active":"2024-04-05T22:22:23.673235+0000","last_peered":"2024-04-05T22:22:23.673235+0000","last_clean":"2024-04-05T22:22:23.673235+0000","last_became_active":"2024-04-05T22:22:23.163784+0000","last_became_peered":"2024-04-05T22:22:23.163784+0000","last_unstale":"2024-04-05T22:22:23.673235+0000","last_undegraded":"2024-04-05T22:22:23.673235+0000","last_fullsized":"2024-04-05T22:22:23.673235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636948+0000","last_change":"2024-04-05T22:22:23.666108+0000","last_active":"2024-04-05T22:22:24.636948+0000","last_peered":"2024-04-05T22:22:24.636948+0000","last_clean":"2024-04-05T22:22:24.636948+0000","last_became_active":"2024-04-05T22:22:23.665839+0000","last_became_peered":"2024-04-05T22:22:23.665839+0000","last_unstale":"2024-04-05T22:22:24.636948+0000","last_undegraded":"2024-04-05T22:22:24.636948+0000","last_fullsized":"2024-04-05T22:22:24.636948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668765+0000","last_change":"2024-04-05T22:22:23.164105+0000","last_active":"2024-04-05T22:22:23.668765+0000","last_peered":"2024-04-05T22:22:23.668765+0000","last_clean":"2024-04-05T22:22:23.668765+0000","last_became_active":"2024-04-05T22:22:23.163812+0000","last_became_peered":"2024-04-05T22:22:23.163812+0000","last_unstale":"2024-04-05T22:22:23.668765+0000","last_undegraded":"2024-04-05T22:22:23.668765+0000","last_fullsized":"2024-04-05T22:22:23.668765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:51.706002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694350+0000","last_change":"2024-04-05T22:22:23.651615+0000","last_active":"2024-04-05T22:22:24.694350+0000","last_peered":"2024-04-05T22:22:24.694350+0000","last_clean":"2024-04-05T22:22:24.694350+0000","last_became_active":"2024-04-05T22:22:23.651372+0000","last_became_peered":"2024-04-05T22:22:23.651372+0000","last_unstale":"2024-04-05T22:22:24.694350+0000","last_undegraded":"2024-04-05T22:22:24.694350+0000","last_fullsized":"2024-04-05T22:22:24.694350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625635+0000","last_change":"2024-04-05T22:22:23.164204+0000","last_active":"2024-04-05T22:22:23.625635+0000","last_peered":"2024-04-05T22:22:23.625635+0000","last_clean":"2024-04-05T22:22:23.625635+0000","last_became_active":"2024-04-05T22:22:23.163969+0000","last_became_peered":"2024-04-05T22:22:23.163969+0000","last_unstale":"2024-04-05T22:22:23.625635+0000","last_undegraded":"2024-04-05T22:22:23.625635+0000","last_fullsized":"2024-04-05T22:22:23.625635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696698+0000","last_change":"2024-04-05T22:22:23.659000+0000","last_active":"2024-04-05T22:22:24.696698+0000","last_peered":"2024-04-05T22:22:24.696698+0000","last_clean":"2024-04-05T22:22:24.696698+0000","last_became_active":"2024-04-05T22:22:23.655679+0000","last_became_peered":"2024-04-05T22:22:23.655679+0000","last_unstale":"2024-04-05T22:22:24.696698+0000","last_undegraded":"2024-04-05T22:22:24.696698+0000","last_fullsized":"2024-04-05T22:22:24.696698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627490+0000","last_change":"2024-04-05T22:22:23.169398+0000","last_active":"2024-04-05T22:22:23.627490+0000","last_peered":"2024-04-05T22:22:23.627490+0000","last_clean":"2024-04-05T22:22:23.627490+0000","last_became_active":"2024-04-05T22:22:23.168976+0000","last_became_peered":"2024-04-05T22:22:23.168976+0000","last_unstale":"2024-04-05T22:22:23.627490+0000","last_undegraded":"2024-04-05T22:22:23.627490+0000","last_fullsized":"2024-04-05T22:22:23.627490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696514+0000","last_change":"2024-04-05T22:22:23.663801+0000","last_active":"2024-04-05T22:22:24.696514+0000","last_peered":"2024-04-05T22:22:24.696514+0000","last_clean":"2024-04-05T22:22:24.696514+0000","last_became_active":"2024-04-05T22:22:23.663458+0000","last_became_peered":"2024-04-05T22:22:23.663458+0000","last_unstale":"2024-04-05T22:22:24.696514+0000","last_undegraded":"2024-04-05T22:22:24.696514+0000","last_fullsized":"2024-04-05T22:22:24.696514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:57:49.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668534+0000","last_change":"2024-04-05T22:22:23.169360+0000","last_active":"2024-04-05T22:22:23.668534+0000","last_peered":"2024-04-05T22:22:23.668534+0000","last_clean":"2024-04-05T22:22:23.668534+0000","last_became_active":"2024-04-05T22:22:23.167009+0000","last_became_peered":"2024-04-05T22:22:23.167009+0000","last_unstale":"2024-04-05T22:22:23.668534+0000","last_undegraded":"2024-04-05T22:22:23.668534+0000","last_fullsized":"2024-04-05T22:22:23.668534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699135+0000","last_change":"2024-04-05T22:22:23.668712+0000","last_active":"2024-04-05T22:22:24.699135+0000","last_peered":"2024-04-05T22:22:24.699135+0000","last_clean":"2024-04-05T22:22:24.699135+0000","last_became_active":"2024-04-05T22:22:23.668435+0000","last_became_peered":"2024-04-05T22:22:23.668435+0000","last_unstale":"2024-04-05T22:22:24.699135+0000","last_undegraded":"2024-04-05T22:22:24.699135+0000","last_fullsized":"2024-04-05T22:22:24.699135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667960+0000","last_change":"2024-04-05T22:22:23.164411+0000","last_active":"2024-04-05T22:22:23.667960+0000","last_peered":"2024-04-05T22:22:23.667960+0000","last_clean":"2024-04-05T22:22:23.667960+0000","last_became_active":"2024-04-05T22:22:23.162250+0000","last_became_peered":"2024-04-05T22:22:23.162250+0000","last_unstale":"2024-04-05T22:22:23.667960+0000","last_undegraded":"2024-04-05T22:22:23.667960+0000","last_fullsized":"2024-04-05T22:22:23.667960+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640160+0000","last_change":"2024-04-05T22:22:23.673810+0000","last_active":"2024-04-05T22:22:24.640160+0000","last_peered":"2024-04-05T22:22:24.640160+0000","last_clean":"2024-04-05T22:22:24.640160+0000","last_became_active":"2024-04-05T22:22:23.673503+0000","last_became_peered":"2024-04-05T22:22:23.673503+0000","last_unstale":"2024-04-05T22:22:24.640160+0000","last_undegraded":"2024-04-05T22:22:24.640160+0000","last_fullsized":"2024-04-05T22:22:24.640160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.670057+0000","last_change":"2024-04-05T22:22:23.156737+0000","last_active":"2024-04-05T22:22:23.670057+0000","last_peered":"2024-04-05T22:22:23.670057+0000","last_clean":"2024-04-05T22:22:23.670057+0000","last_became_active":"2024-04-05T22:22:23.156286+0000","last_became_peered":"2024-04-05T22:22:23.156286+0000","last_unstale":"2024-04-05T22:22:23.670057+0000","last_undegraded":"2024-04-05T22:22:23.670057+0000","last_fullsized":"2024-04-05T22:22:23.670057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637263+0000","last_change":"2024-04-05T22:22:23.656662+0000","last_active":"2024-04-05T22:22:24.637263+0000","last_peered":"2024-04-05T22:22:24.637263+0000","last_clean":"2024-04-05T22:22:24.637263+0000","last_became_active":"2024-04-05T22:22:23.656406+0000","last_became_peered":"2024-04-05T22:22:23.656406+0000","last_unstale":"2024-04-05T22:22:24.637263+0000","last_undegraded":"2024-04-05T22:22:24.637263+0000","last_fullsized":"2024-04-05T22:22:24.637263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667743+0000","last_change":"2024-04-05T22:22:23.158044+0000","last_active":"2024-04-05T22:22:23.667743+0000","last_peered":"2024-04-05T22:22:23.667743+0000","last_clean":"2024-04-05T22:22:23.667743+0000","last_became_active":"2024-04-05T22:22:23.157699+0000","last_became_peered":"2024-04-05T22:22:23.157699+0000","last_unstale":"2024-04-05T22:22:23.667743+0000","last_undegraded":"2024-04-05T22:22:23.667743+0000","last_fullsized":"2024-04-05T22:22:23.667743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:58:42.971651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635512+0000","last_change":"2024-04-05T22:22:23.650874+0000","last_active":"2024-04-05T22:22:24.635512+0000","last_peered":"2024-04-05T22:22:24.635512+0000","last_clean":"2024-04-05T22:22:24.635512+0000","last_became_active":"2024-04-05T22:22:23.650617+0000","last_became_peered":"2024-04-05T22:22:23.650617+0000","last_unstale":"2024-04-05T22:22:24.635512+0000","last_undegraded":"2024-04-05T22:22:24.635512+0000","last_fullsized":"2024-04-05T22:22:24.635512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.642139+0000","last_change":"2024-04-05T22:22:23.171641+0000","last_active":"2024-04-05T22:22:23.642139+0000","last_peered":"2024-04-05T22:22:23.642139+0000","last_clean":"2024-04-05T22:22:23.642139+0000","last_became_active":"2024-04-05T22:22:23.171224+0000","last_became_peered":"2024-04-05T22:22:23.171224+0000","last_unstale":"2024-04-05T22:22:23.642139+0000","last_undegraded":"2024-04-05T22:22:23.642139+0000","last_fullsized":"2024-04-05T22:22:23.642139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:26.099736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635186+0000","last_change":"2024-04-05T22:22:23.649481+0000","last_active":"2024-04-05T22:22:24.635186+0000","last_peered":"2024-04-05T22:22:24.635186+0000","last_clean":"2024-04-05T22:22:24.635186+0000","last_became_active":"2024-04-05T22:22:23.649158+0000","last_became_peered":"2024-04-05T22:22:23.649158+0000","last_unstale":"2024-04-05T22:22:24.635186+0000","last_undegraded":"2024-04-05T22:22:24.635186+0000","last_fullsized":"2024-04-05T22:22:24.635186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.661505+0000","last_change":"2024-04-05T22:22:23.153928+0000","last_active":"2024-04-05T22:22:23.661505+0000","last_peered":"2024-04-05T22:22:23.661505+0000","last_clean":"2024-04-05T22:22:23.661505+0000","last_became_active":"2024-04-05T22:22:23.153301+0000","last_became_peered":"2024-04-05T22:22:23.153301+0000","last_unstale":"2024-04-05T22:22:23.661505+0000","last_undegraded":"2024-04-05T22:22:23.661505+0000","last_fullsized":"2024-04-05T22:22:23.661505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635294+0000","last_change":"2024-04-05T22:22:23.656526+0000","last_active":"2024-04-05T22:22:24.635294+0000","last_peered":"2024-04-05T22:22:24.635294+0000","last_clean":"2024-04-05T22:22:24.635294+0000","last_became_active":"2024-04-05T22:22:23.655430+0000","last_became_peered":"2024-04-05T22:22:23.655430+0000","last_unstale":"2024-04-05T22:22:24.635294+0000","last_undegraded":"2024-04-05T22:22:24.635294+0000","last_fullsized":"2024-04-05T22:22:24.635294+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:24.841919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.648674+0000","last_change":"2024-04-05T22:22:23.164487+0000","last_active":"2024-04-05T22:22:23.648674+0000","last_peered":"2024-04-05T22:22:23.648674+0000","last_clean":"2024-04-05T22:22:23.648674+0000","last_became_active":"2024-04-05T22:22:23.163704+0000","last_became_peered":"2024-04-05T22:22:23.163704+0000","last_unstale":"2024-04-05T22:22:23.648674+0000","last_undegraded":"2024-04-05T22:22:23.648674+0000","last_fullsized":"2024-04-05T22:22:23.648674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:25.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635044+0000","last_change":"2024-04-05T22:22:23.651556+0000","last_active":"2024-04-05T22:22:24.635044+0000","last_peered":"2024-04-05T22:22:24.635044+0000","last_clean":"2024-04-05T22:22:24.635044+0000","last_became_active":"2024-04-05T22:22:23.651314+0000","last_became_peered":"2024-04-05T22:22:23.651314+0000","last_unstale":"2024-04-05T22:22:24.635044+0000","last_undegraded":"2024-04-05T22:22:24.635044+0000","last_fullsized":"2024-04-05T22:22:24.635044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636186+0000","last_change":"2024-04-05T22:22:23.155981+0000","last_active":"2024-04-05T22:22:23.636186+0000","last_peered":"2024-04-05T22:22:23.636186+0000","last_clean":"2024-04-05T22:22:23.636186+0000","last_became_active":"2024-04-05T22:22:23.154363+0000","last_became_peered":"2024-04-05T22:22:23.154363+0000","last_unstale":"2024-04-05T22:22:23.636186+0000","last_undegraded":"2024-04-05T22:22:23.636186+0000","last_fullsized":"2024-04-05T22:22:23.636186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:13.978598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696889+0000","last_change":"2024-04-05T22:22:23.657509+0000","last_active":"2024-04-05T22:22:24.696889+0000","last_peered":"2024-04-05T22:22:24.696889+0000","last_clean":"2024-04-05T22:22:24.696889+0000","last_became_active":"2024-04-05T22:22:23.657035+0000","last_became_peered":"2024-04-05T22:22:23.657035+0000","last_unstale":"2024-04-05T22:22:24.696889+0000","last_undegraded":"2024-04-05T22:22:24.696889+0000","last_fullsized":"2024-04-05T22:22:24.696889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656814+0000","last_change":"2024-04-05T22:22:23.172774+0000","last_active":"2024-04-05T22:22:23.656814+0000","last_peered":"2024-04-05T22:22:23.656814+0000","last_clean":"2024-04-05T22:22:23.656814+0000","last_became_active":"2024-04-05T22:22:23.172492+0000","last_became_peered":"2024-04-05T22:22:23.172492+0000","last_unstale":"2024-04-05T22:22:23.656814+0000","last_undegraded":"2024-04-05T22:22:23.656814+0000","last_fullsized":"2024-04-05T22:22:23.656814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635485+0000","last_change":"2024-04-05T22:22:23.667039+0000","last_active":"2024-04-05T22:22:24.635485+0000","last_peered":"2024-04-05T22:22:24.635485+0000","last_clean":"2024-04-05T22:22:24.635485+0000","last_became_active":"2024-04-05T22:22:23.666665+0000","last_became_peered":"2024-04-05T22:22:23.666665+0000","last_unstale":"2024-04-05T22:22:24.635485+0000","last_undegraded":"2024-04-05T22:22:24.635485+0000","last_fullsized":"2024-04-05T22:22:24.635485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.665989+0000","last_change":"2024-04-05T22:22:23.165988+0000","last_active":"2024-04-05T22:22:23.665989+0000","last_peered":"2024-04-05T22:22:23.665989+0000","last_clean":"2024-04-05T22:22:23.665989+0000","last_became_active":"2024-04-05T22:22:23.165708+0000","last_became_peered":"2024-04-05T22:22:23.165708+0000","last_unstale":"2024-04-05T22:22:23.665989+0000","last_undegraded":"2024-04-05T22:22:23.665989+0000","last_fullsized":"2024-04-05T22:22:23.665989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640236+0000","last_change":"2024-04-05T22:22:23.664010+0000","last_active":"2024-04-05T22:22:24.640236+0000","last_peered":"2024-04-05T22:22:24.640236+0000","last_clean":"2024-04-05T22:22:24.640236+0000","last_became_active":"2024-04-05T22:22:23.663608+0000","last_became_peered":"2024-04-05T22:22:23.663608+0000","last_unstale":"2024-04-05T22:22:24.640236+0000","last_undegraded":"2024-04-05T22:22:24.640236+0000","last_fullsized":"2024-04-05T22:22:24.640236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695548+0000","last_change":"2024-04-05T22:22:23.154936+0000","last_active":"2024-04-05T22:22:24.695548+0000","last_peered":"2024-04-05T22:22:24.695548+0000","last_clean":"2024-04-05T22:22:24.695548+0000","last_became_active":"2024-04-05T22:22:23.154664+0000","last_became_peered":"2024-04-05T22:22:23.154664+0000","last_unstale":"2024-04-05T22:22:24.695548+0000","last_undegraded":"2024-04-05T22:22:24.695548+0000","last_fullsized":"2024-04-05T22:22:24.695548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:23.826984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695278+0000","last_change":"2024-04-05T22:22:23.649531+0000","last_active":"2024-04-05T22:22:24.695278+0000","last_peered":"2024-04-05T22:22:24.695278+0000","last_clean":"2024-04-05T22:22:24.695278+0000","last_became_active":"2024-04-05T22:22:23.649241+0000","last_became_peered":"2024-04-05T22:22:23.649241+0000","last_unstale":"2024-04-05T22:22:24.695278+0000","last_undegraded":"2024-04-05T22:22:24.695278+0000","last_fullsized":"2024-04-05T22:22:24.695278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:19.183526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.626116+0000","last_change":"2024-04-05T22:22:23.160628+0000","last_active":"2024-04-05T22:22:23.626116+0000","last_peered":"2024-04-05T22:22:23.626116+0000","last_clean":"2024-04-05T22:22:23.626116+0000","last_became_active":"2024-04-05T22:22:23.159150+0000","last_became_peered":"2024-04-05T22:22:23.159150+0000","last_unstale":"2024-04-05T22:22:23.626116+0000","last_undegraded":"2024-04-05T22:22:23.626116+0000","last_fullsized":"2024-04-05T22:22:23.626116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696531+0000","last_change":"2024-04-05T22:22:23.666678+0000","last_active":"2024-04-05T22:22:24.696531+0000","last_peered":"2024-04-05T22:22:24.696531+0000","last_clean":"2024-04-05T22:22:24.696531+0000","last_became_active":"2024-04-05T22:22:23.665881+0000","last_became_peered":"2024-04-05T22:22:23.665881+0000","last_unstale":"2024-04-05T22:22:24.696531+0000","last_undegraded":"2024-04-05T22:22:24.696531+0000","last_fullsized":"2024-04-05T22:22:24.696531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:36.143521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646709+0000","last_change":"2024-04-05T22:22:23.166020+0000","last_active":"2024-04-05T22:22:23.646709+0000","last_peered":"2024-04-05T22:22:23.646709+0000","last_clean":"2024-04-05T22:22:23.646709+0000","last_became_active":"2024-04-05T22:22:23.165719+0000","last_became_peered":"2024-04-05T22:22:23.165719+0000","last_unstale":"2024-04-05T22:22:23.646709+0000","last_undegraded":"2024-04-05T22:22:23.646709+0000","last_fullsized":"2024-04-05T22:22:23.646709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:58.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695379+0000","last_change":"2024-04-05T22:22:23.666126+0000","last_active":"2024-04-05T22:22:24.695379+0000","last_peered":"2024-04-05T22:22:24.695379+0000","last_clean":"2024-04-05T22:22:24.695379+0000","last_became_active":"2024-04-05T22:22:23.665889+0000","last_became_peered":"2024-04-05T22:22:23.665889+0000","last_unstale":"2024-04-05T22:22:24.695379+0000","last_undegraded":"2024-04-05T22:22:24.695379+0000","last_fullsized":"2024-04-05T22:22:24.695379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625723+0000","last_change":"2024-04-05T22:22:23.166747+0000","last_active":"2024-04-05T22:22:23.625723+0000","last_peered":"2024-04-05T22:22:23.625723+0000","last_clean":"2024-04-05T22:22:23.625723+0000","last_became_active":"2024-04-05T22:22:23.166495+0000","last_became_peered":"2024-04-05T22:22:23.166495+0000","last_unstale":"2024-04-05T22:22:23.625723+0000","last_undegraded":"2024-04-05T22:22:23.625723+0000","last_fullsized":"2024-04-05T22:22:23.625723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693701+0000","last_change":"2024-04-05T22:22:23.634863+0000","last_active":"2024-04-05T22:22:24.693701+0000","last_peered":"2024-04-05T22:22:24.693701+0000","last_clean":"2024-04-05T22:22:24.693701+0000","last_became_active":"2024-04-05T22:22:23.634615+0000","last_became_peered":"2024-04-05T22:22:23.634615+0000","last_unstale":"2024-04-05T22:22:24.693701+0000","last_undegraded":"2024-04-05T22:22:24.693701+0000","last_fullsized":"2024-04-05T22:22:24.693701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:32:16.541566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636456+0000","last_change":"2024-04-05T22:22:23.168895+0000","last_active":"2024-04-05T22:22:23.636456+0000","last_peered":"2024-04-05T22:22:23.636456+0000","last_clean":"2024-04-05T22:22:23.636456+0000","last_became_active":"2024-04-05T22:22:23.168395+0000","last_became_peered":"2024-04-05T22:22:23.168395+0000","last_unstale":"2024-04-05T22:22:23.636456+0000","last_undegraded":"2024-04-05T22:22:23.636456+0000","last_fullsized":"2024-04-05T22:22:23.636456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:14:22.438726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698300+0000","last_change":"2024-04-05T22:22:23.654099+0000","last_active":"2024-04-05T22:22:24.698300+0000","last_peered":"2024-04-05T22:22:24.698300+0000","last_clean":"2024-04-05T22:22:24.698300+0000","last_became_active":"2024-04-05T22:22:23.653171+0000","last_became_peered":"2024-04-05T22:22:23.653171+0000","last_unstale":"2024-04-05T22:22:24.698300+0000","last_undegraded":"2024-04-05T22:22:24.698300+0000","last_fullsized":"2024-04-05T22:22:24.698300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627663+0000","last_change":"2024-04-05T22:22:23.173853+0000","last_active":"2024-04-05T22:22:23.627663+0000","last_peered":"2024-04-05T22:22:23.627663+0000","last_clean":"2024-04-05T22:22:23.627663+0000","last_became_active":"2024-04-05T22:22:23.173561+0000","last_became_peered":"2024-04-05T22:22:23.173561+0000","last_unstale":"2024-04-05T22:22:23.627663+0000","last_undegraded":"2024-04-05T22:22:23.627663+0000","last_fullsized":"2024-04-05T22:22:23.627663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698265+0000","last_change":"2024-04-05T22:22:23.649429+0000","last_active":"2024-04-05T22:22:24.698265+0000","last_peered":"2024-04-05T22:22:24.698265+0000","last_clean":"2024-04-05T22:22:24.698265+0000","last_became_active":"2024-04-05T22:22:23.649091+0000","last_became_peered":"2024-04-05T22:22:23.649091+0000","last_unstale":"2024-04-05T22:22:24.698265+0000","last_undegraded":"2024-04-05T22:22:24.698265+0000","last_fullsized":"2024-04-05T22:22:24.698265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:51:19.346378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645441+0000","last_change":"2024-04-05T22:22:23.162280+0000","last_active":"2024-04-05T22:22:23.645441+0000","last_peered":"2024-04-05T22:22:23.645441+0000","last_clean":"2024-04-05T22:22:23.645441+0000","last_became_active":"2024-04-05T22:22:23.161975+0000","last_became_peered":"2024-04-05T22:22:23.161975+0000","last_unstale":"2024-04-05T22:22:23.645441+0000","last_undegraded":"2024-04-05T22:22:23.645441+0000","last_fullsized":"2024-04-05T22:22:23.645441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:24:22.511159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.631303+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:22:23.631303+0000","last_peered":"2024-04-05T22:22:23.631303+0000","last_clean":"2024-04-05T22:22:23.631303+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:22:23.631303+0000","last_undegraded":"2024-04-05T22:22:23.631303+0000","last_fullsized":"2024-04-05T22:22:23.631303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":1003520,"data_stored":750473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.577},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.073}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.179},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.038}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.329}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.274},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.7010000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.534}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.829}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":413696,"data_stored":160631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.579},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.242}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.524},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.7290000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.776},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.641}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.165},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.2809999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.26}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.159}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.359}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.196},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.361},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.232}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.3979999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.075}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.159},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.425}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.137},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":160085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.621},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.999}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.264}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.83699999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.875}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.419},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.534}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.841},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.3320000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":2.0619999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.663}]}]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":750383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.747},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.98699999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.101},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:22:28.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-05T22:22:29.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.203+0000 7f0e6d200640 1 Processor -- start 2024-04-05T22:22:29.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.204+0000 7f0e6d200640 1 -- start start 2024-04-05T22:22:29.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.205+0000 7f0e6d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e681587d0 0x7f0e680cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.205+0000 7f0e6d200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6806f5b0 con 0x7f0e680ccd20 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.205+0000 7f0e6d200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e68070410 con 0x7f0e6815e850 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.205+0000 7f0e6d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e68158bb0 con 0x7f0e681587d0 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.205+0000 7f0e65e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e681587d0 0x7f0e680cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e5fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e680ccd20 0x7f0e6806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52152/0 (socket says 172.21.15.73:52152) 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e65e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e681587d0 0x7f0e680cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46550/0 (socket says 172.21.15.73:46550) 2024-04-05T22:22:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e5fe00640 1 -- 172.21.15.73:0/3236472874 learned_addr learned my addr 172.21.15.73:0/3236472874 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e65e00640 1 -- 172.21.15.73:0/3236472874 >> v1:172.21.15.157:6789/0 conn(0x7f0e6815e850 legacy=0x7f0e680708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e65e00640 1 -- 172.21.15.73:0/3236472874 >> v1:172.21.15.73:6789/0 conn(0x7f0e680ccd20 legacy=0x7f0e6806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.206+0000 7f0e65e00640 1 -- 172.21.15.73:0/3236472874 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6806f800 con 0x7f0e681587d0 2024-04-05T22:22:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.207+0000 7f0e65e00640 1 --2- 172.21.15.73:0/3236472874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e681587d0 0x7f0e680cc7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e580097d0 tx=0x7f0e5802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a9f6582632f595 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.207+0000 7f0e65400640 1 -- 172.21.15.73:0/3236472874 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5802d040 con 0x7f0e681587d0 2024-04-05T22:22:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.207+0000 7f0e65400640 1 -- 172.21.15.73:0/3236472874 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58031810 con 0x7f0e681587d0 2024-04-05T22:22:29.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.207+0000 7f0e65400640 1 -- 172.21.15.73:0/3236472874 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58031ba0 con 0x7f0e681587d0 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 -- 172.21.15.73:0/3236472874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e681587d0 msgr2=0x7f0e680cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 --2- 172.21.15.73:0/3236472874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e681587d0 0x7f0e680cc7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e580097d0 tx=0x7f0e5802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 -- 172.21.15.73:0/3236472874 shutdown_connections 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 --2- 172.21.15.73:0/3236472874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e681587d0 0x7f0e680cc7e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 -- 172.21.15.73:0/3236472874 >> 172.21.15.73:0/3236472874 conn(0x7f0e680ce8e0 msgr2=0x7f0e680cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 -- 172.21.15.73:0/3236472874 shutdown_connections 2024-04-05T22:22:29.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.208+0000 7f0e6d200640 1 -- 172.21.15.73:0/3236472874 wait complete. 2024-04-05T22:22:29.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.210+0000 7f0e6d200640 1 Processor -- start 2024-04-05T22:22:29.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.210+0000 7f0e6d200640 1 -- start start 2024-04-05T22:22:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e681587d0 0x7f0e680b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e681464b0 0x7f0e6815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68146960 con 0x7f0e6815e850 2024-04-05T22:22:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68153e30 con 0x7f0e681464b0 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.211+0000 7f0e6d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68154110 con 0x7f0e681587d0 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46562/0 (socket says 172.21.15.73:46562) 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 -- 172.21.15.73:0/430835138 learned_addr learned my addr 172.21.15.73:0/430835138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e65e00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e681587d0 0x7f0e680b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 -- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e681587d0 msgr2=0x7f0e680b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e681587d0 0x7f0e680b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 -- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e681464b0 msgr2=0x7f0e6815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e681464b0 0x7f0e6815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 -- 172.21.15.73:0/430835138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009400 con 0x7f0e6815e850 2024-04-05T22:22:29.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c4a0 tx=0x7f0e5000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.212+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50005ce0 con 0x7f0e6815e850 2024-04-05T22:22:29.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.213+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50005e40 con 0x7f0e6815e850 2024-04-05T22:22:29.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.213+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50007270 con 0x7f0e6815e850 2024-04-05T22:22:29.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.213+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e681543f0 con 0x7f0e6815e850 2024-04-05T22:22:29.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.213+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e681549b0 con 0x7f0e6815e850 2024-04-05T22:22:29.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.214+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e68066af0 con 0x7f0e6815e850 2024-04-05T22:22:29.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.215+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e50007490 con 0x7f0e6815e850 2024-04-05T22:22:29.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.216+0000 7f0e5e000640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 0x7f0e4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.216+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e50088840 con 0x7f0e6815e850 2024-04-05T22:22:29.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.216+0000 7f0e65e00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 0x7f0e4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.217+0000 7f0e65e00640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 0x7f0e4406d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e58036000 tx=0x7f0e5803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.220+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0e50053a30 con 0x7f0e6815e850 2024-04-05T22:22:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.486+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0e68154ba0 con 0x7f0e6815e850 2024-04-05T22:22:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.486+0000 7f0e5e000640 1 -- 172.21.15.73:0/430835138 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7f0e50017410 con 0x7f0e6815e850 2024-04-05T22:22:29.487 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:29.487 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"38861c87-67cf-4180-a71f-97401becb77d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":93,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1712355667,"num_in_osds":8,"osd_in_since":1712355659,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592730,"bytes_used":242634752,"bytes_avail":772851478528,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5115}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-05T22:22:12.507686+0000","services":{}},"progress_events":{}} 2024-04-05T22:22:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 msgr2=0x7f0e4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 0x7f0e4406d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e58036000 tx=0x7f0e5803c040 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 msgr2=0x7f0e68145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c4a0 tx=0x7f0e5000c960 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 shutdown_connections 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e4406ac00 0x7f0e4406d0c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e681464b0 0x7f0e6815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6815e850 0x7f0e68145f70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 --2- 172.21.15.73:0/430835138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e681587d0 0x7f0e680b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 >> 172.21.15.73:0/430835138 conn(0x7f0e680ce8e0 msgr2=0x7f0e680cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 shutdown_connections 2024-04-05T22:22:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.489+0000 7f0e6d200640 1 -- 172.21.15.73:0/430835138 wait complete. 2024-04-05T22:22:29.506 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-05T22:22:29.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:22:29.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.699+0000 7fd87e800640 1 Processor -- start 2024-04-05T22:22:29.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.699+0000 7fd87e800640 1 -- start start 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.700+0000 7fd87e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.700+0000 7fd87e800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8780ccd20 0x7fd87806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.700+0000 7fd87e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd87806d0c0 0x7fd87815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.700+0000 7fd87e800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87806f5b0 con 0x7fd87806d0c0 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.700+0000 7fd87e800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd878070410 con 0x7fd8780ccd20 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd87e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd878158bb0 con 0x7fd8781587d0 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd87806d0c0 0x7fd87815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd87806d0c0 0x7fd87815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46574/0 (socket says 172.21.15.73:46574) 2024-04-05T22:22:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 -- 172.21.15.73:0/2957743936 learned_addr learned my addr 172.21.15.73:0/2957743936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:29.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877400640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 -- 172.21.15.73:0/2957743936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 msgr2=0x7fd8780cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:29.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd86ea00640 1 --2- 172.21.15.73:0/2957743936 >> v2:172.21.15.157:3300/0 conn(0x7fd8780ccd20 0x7fd87806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 -- 172.21.15.73:0/2957743936 >> v2:172.21.15.157:3300/0 conn(0x7fd8780ccd20 msgr2=0x7fd87806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:29.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 --2- 172.21.15.73:0/2957743936 >> v2:172.21.15.157:3300/0 conn(0x7fd8780ccd20 0x7fd87806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.701+0000 7fd877e00640 1 -- 172.21.15.73:0/2957743936 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87806f800 con 0x7fd87806d0c0 2024-04-05T22:22:29.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.702+0000 7fd877e00640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd87815e850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd878066850 tx=0x7fd87002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cd6df54ca98334 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd876a00640 1 -- 172.21.15.73:0/2957743936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87002ca70 con 0x7fd87806d0c0 2024-04-05T22:22:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd876a00640 1 -- 172.21.15.73:0/2957743936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87002cbd0 con 0x7fd87806d0c0 2024-04-05T22:22:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd876a00640 1 -- 172.21.15.73:0/2957743936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700343f0 con 0x7fd87806d0c0 2024-04-05T22:22:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 -- 172.21.15.73:0/2957743936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 msgr2=0x7fd87815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd87815e850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd878066850 tx=0x7fd87002bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 -- 172.21.15.73:0/2957743936 shutdown_connections 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd87815e850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 --2- 172.21.15.73:0/2957743936 >> v2:172.21.15.157:3300/0 conn(0x7fd8780ccd20 0x7fd87806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 --2- 172.21.15.73:0/2957743936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.703+0000 7fd87e800640 1 -- 172.21.15.73:0/2957743936 >> 172.21.15.73:0/2957743936 conn(0x7fd8780ce8e0 msgr2=0x7fd8780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.704+0000 7fd87e800640 1 -- 172.21.15.73:0/2957743936 shutdown_connections 2024-04-05T22:22:29.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.704+0000 7fd87e800640 1 -- 172.21.15.73:0/2957743936 wait complete. 2024-04-05T22:22:29.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.706+0000 7fd87e800640 1 Processor -- start 2024-04-05T22:22:29.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.706+0000 7fd87e800640 1 -- start start 2024-04-05T22:22:29.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8781f2530 0x7fd87815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b28b0 con 0x7fd87806d0c0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b1970 con 0x7fd8781f2530 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd87e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b1c30 con 0x7fd8781587d0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd877400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd877400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46588/0 (socket says 172.21.15.73:46588) 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.707+0000 7fd877400640 1 -- 172.21.15.73:0/918357369 learned_addr learned my addr 172.21.15.73:0/918357369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877e00640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8781f2530 0x7fd87815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 -- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 msgr2=0x7fd8780ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 -- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8781f2530 msgr2=0x7fd87815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8781f2530 0x7fd87815d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 -- 172.21.15.73:0/918357369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870009b30 con 0x7fd87806d0c0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd877400640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d5f0 tx=0x7fd86800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680125f0 con 0x7fd87806d0c0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868012750 con 0x7fd87806d0c0 2024-04-05T22:22:29.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868012b00 con 0x7fd87806d0c0 2024-04-05T22:22:29.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8780b1f10 con 0x7fd87806d0c0 2024-04-05T22:22:29.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.708+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8780b24d0 con 0x7fd87806d0c0 2024-04-05T22:22:29.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.710+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd878066af0 con 0x7fd87806d0c0 2024-04-05T22:22:29.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.710+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd868012c60 con 0x7fd87806d0c0 2024-04-05T22:22:29.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.711+0000 7fd86fe00640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 0x7fd85406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:29.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.711+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8680898c0 con 0x7fd87806d0c0 2024-04-05T22:22:29.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.711+0000 7fd86ea00640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 0x7fd85406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:29.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.712+0000 7fd86ea00640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 0x7fd85406d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd860006b10 tx=0x7fd860008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:29.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.717+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd868054aa0 con 0x7fd87806d0c0 2024-04-05T22:22:29.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.895+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8780700a0 con 0x7fd85406ac00 2024-04-05T22:22:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.904+0000 7fd86fe00640 1 -- 172.21.15.73:0/918357369 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287168 (secure 0 0 0) 0x7fd86000ae50 con 0x7fd85406ac00 2024-04-05T22:22:29.908 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:22:29.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 msgr2=0x7fd85406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 0x7fd85406d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd860006b10 tx=0x7fd860008040 comp rx=0 tx=0).stop 2024-04-05T22:22:29.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 msgr2=0x7fd878142010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d5f0 tx=0x7fd86800dab0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 shutdown_connections 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd85406ac00 0x7fd85406d0c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8781f2530 0x7fd87815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8781587d0 0x7fd8780ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 --2- 172.21.15.73:0/918357369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd87806d0c0 0x7fd878142010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 >> 172.21.15.73:0/918357369 conn(0x7fd8780ce8e0 msgr2=0x7fd8780af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 shutdown_connections 2024-04-05T22:22:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:29.909+0000 7fd87e800640 1 -- 172.21.15.73:0/918357369 wait complete. 2024-04-05T22:22:29.925 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-05T22:22:28.511523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236948,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737772,"statfs":{"total":773094113280,"available":772851478528,"internally_reserved":0,"allocated":4497408,"data_stored":2465580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001367"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.671468+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.671239+0000","last_became_peered":"2024-04-05T22:22:23.671239+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:17:31.785954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694516+0000","last_change":"2024-04-05T22:22:23.652316+0000","last_active":"2024-04-05T22:22:24.694516+0000","last_peered":"2024-04-05T22:22:24.694516+0000","last_clean":"2024-04-05T22:22:24.694516+0000","last_became_active":"2024-04-05T22:22:23.651989+0000","last_became_peered":"2024-04-05T22:22:23.651989+0000","last_unstale":"2024-04-05T22:22:24.694516+0000","last_undegraded":"2024-04-05T22:22:24.694516+0000","last_fullsized":"2024-04-05T22:22:24.694516+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639834+0000","last_change":"2024-04-05T22:22:23.672439+0000","last_active":"2024-04-05T22:22:24.639834+0000","last_peered":"2024-04-05T22:22:24.639834+0000","last_clean":"2024-04-05T22:22:24.639834+0000","last_became_active":"2024-04-05T22:22:23.672112+0000","last_became_peered":"2024-04-05T22:22:23.672112+0000","last_unstale":"2024-04-05T22:22:24.639834+0000","last_undegraded":"2024-04-05T22:22:24.639834+0000","last_fullsized":"2024-04-05T22:22:24.639834+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637508+0000","last_change":"2024-04-05T22:22:23.657494+0000","last_active":"2024-04-05T22:22:24.637508+0000","last_peered":"2024-04-05T22:22:24.637508+0000","last_clean":"2024-04-05T22:22:24.637508+0000","last_became_active":"2024-04-05T22:22:23.657164+0000","last_became_peered":"2024-04-05T22:22:23.657164+0000","last_unstale":"2024-04-05T22:22:24.637508+0000","last_undegraded":"2024-04-05T22:22:24.637508+0000","last_fullsized":"2024-04-05T22:22:24.637508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635155+0000","last_change":"2024-04-05T22:22:23.642932+0000","last_active":"2024-04-05T22:22:24.635155+0000","last_peered":"2024-04-05T22:22:24.635155+0000","last_clean":"2024-04-05T22:22:24.635155+0000","last_became_active":"2024-04-05T22:22:23.642640+0000","last_became_peered":"2024-04-05T22:22:23.642640+0000","last_unstale":"2024-04-05T22:22:24.635155+0000","last_undegraded":"2024-04-05T22:22:24.635155+0000","last_fullsized":"2024-04-05T22:22:24.635155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693992+0000","last_change":"2024-04-05T22:22:23.668411+0000","last_active":"2024-04-05T22:22:24.693992+0000","last_peered":"2024-04-05T22:22:24.693992+0000","last_clean":"2024-04-05T22:22:24.693992+0000","last_became_active":"2024-04-05T22:22:23.668135+0000","last_became_peered":"2024-04-05T22:22:23.668135+0000","last_unstale":"2024-04-05T22:22:24.693992+0000","last_undegraded":"2024-04-05T22:22:24.693992+0000","last_fullsized":"2024-04-05T22:22:24.693992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697247+0000","last_change":"2024-04-05T22:22:23.661512+0000","last_active":"2024-04-05T22:22:24.697247+0000","last_peered":"2024-04-05T22:22:24.697247+0000","last_clean":"2024-04-05T22:22:24.697247+0000","last_became_active":"2024-04-05T22:22:23.661235+0000","last_became_peered":"2024-04-05T22:22:23.661235+0000","last_unstale":"2024-04-05T22:22:24.697247+0000","last_undegraded":"2024-04-05T22:22:24.697247+0000","last_fullsized":"2024-04-05T22:22:24.697247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696214+0000","last_change":"2024-04-05T22:22:23.652995+0000","last_active":"2024-04-05T22:22:24.696214+0000","last_peered":"2024-04-05T22:22:24.696214+0000","last_clean":"2024-04-05T22:22:24.696214+0000","last_became_active":"2024-04-05T22:22:23.652715+0000","last_became_peered":"2024-04-05T22:22:23.652715+0000","last_unstale":"2024-04-05T22:22:24.696214+0000","last_undegraded":"2024-04-05T22:22:24.696214+0000","last_fullsized":"2024-04-05T22:22:24.696214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:27:23.769941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699562+0000","last_change":"2024-04-05T22:22:23.651001+0000","last_active":"2024-04-05T22:22:24.699562+0000","last_peered":"2024-04-05T22:22:24.699562+0000","last_clean":"2024-04-05T22:22:24.699562+0000","last_became_active":"2024-04-05T22:22:23.650633+0000","last_became_peered":"2024-04-05T22:22:23.650633+0000","last_unstale":"2024-04-05T22:22:24.699562+0000","last_undegraded":"2024-04-05T22:22:24.699562+0000","last_fullsized":"2024-04-05T22:22:24.699562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:23.999212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637115+0000","last_change":"2024-04-05T22:22:23.636389+0000","last_active":"2024-04-05T22:22:24.637115+0000","last_peered":"2024-04-05T22:22:24.637115+0000","last_clean":"2024-04-05T22:22:24.637115+0000","last_became_active":"2024-04-05T22:22:23.636028+0000","last_became_peered":"2024-04-05T22:22:23.636028+0000","last_unstale":"2024-04-05T22:22:24.637115+0000","last_undegraded":"2024-04-05T22:22:24.637115+0000","last_fullsized":"2024-04-05T22:22:24.637115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697848+0000","last_change":"2024-04-05T22:22:23.639648+0000","last_active":"2024-04-05T22:22:24.697848+0000","last_peered":"2024-04-05T22:22:24.697848+0000","last_clean":"2024-04-05T22:22:24.697848+0000","last_became_active":"2024-04-05T22:22:23.636042+0000","last_became_peered":"2024-04-05T22:22:23.636042+0000","last_unstale":"2024-04-05T22:22:24.697848+0000","last_undegraded":"2024-04-05T22:22:24.697848+0000","last_fullsized":"2024-04-05T22:22:24.697848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:19.468442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639272+0000","last_change":"2024-04-05T22:22:23.667880+0000","last_active":"2024-04-05T22:22:24.639272+0000","last_peered":"2024-04-05T22:22:24.639272+0000","last_clean":"2024-04-05T22:22:24.639272+0000","last_became_active":"2024-04-05T22:22:23.660507+0000","last_became_peered":"2024-04-05T22:22:23.660507+0000","last_unstale":"2024-04-05T22:22:24.639272+0000","last_undegraded":"2024-04-05T22:22:24.639272+0000","last_fullsized":"2024-04-05T22:22:24.639272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635308+0000","last_change":"2024-04-05T22:22:23.650060+0000","last_active":"2024-04-05T22:22:24.635308+0000","last_peered":"2024-04-05T22:22:24.635308+0000","last_clean":"2024-04-05T22:22:24.635308+0000","last_became_active":"2024-04-05T22:22:23.647699+0000","last_became_peered":"2024-04-05T22:22:23.647699+0000","last_unstale":"2024-04-05T22:22:24.635308+0000","last_undegraded":"2024-04-05T22:22:24.635308+0000","last_fullsized":"2024-04-05T22:22:24.635308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693997+0000","last_change":"2024-04-05T22:22:23.644515+0000","last_active":"2024-04-05T22:22:24.693997+0000","last_peered":"2024-04-05T22:22:24.693997+0000","last_clean":"2024-04-05T22:22:24.693997+0000","last_became_active":"2024-04-05T22:22:23.643820+0000","last_became_peered":"2024-04-05T22:22:23.643820+0000","last_unstale":"2024-04-05T22:22:24.693997+0000","last_undegraded":"2024-04-05T22:22:24.693997+0000","last_fullsized":"2024-04-05T22:22:24.693997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639487+0000","last_change":"2024-04-05T22:22:23.672903+0000","last_active":"2024-04-05T22:22:24.639487+0000","last_peered":"2024-04-05T22:22:24.639487+0000","last_clean":"2024-04-05T22:22:24.639487+0000","last_became_active":"2024-04-05T22:22:23.670922+0000","last_became_peered":"2024-04-05T22:22:23.670922+0000","last_unstale":"2024-04-05T22:22:24.639487+0000","last_undegraded":"2024-04-05T22:22:24.639487+0000","last_fullsized":"2024-04-05T22:22:24.639487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696950+0000","last_change":"2024-04-05T22:22:23.643442+0000","last_active":"2024-04-05T22:22:24.696950+0000","last_peered":"2024-04-05T22:22:24.696950+0000","last_clean":"2024-04-05T22:22:24.696950+0000","last_became_active":"2024-04-05T22:22:23.642787+0000","last_became_peered":"2024-04-05T22:22:23.642787+0000","last_unstale":"2024-04-05T22:22:24.696950+0000","last_undegraded":"2024-04-05T22:22:24.696950+0000","last_fullsized":"2024-04-05T22:22:24.696950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697924+0000","last_change":"2024-04-05T22:22:23.662150+0000","last_active":"2024-04-05T22:22:24.697924+0000","last_peered":"2024-04-05T22:22:24.697924+0000","last_clean":"2024-04-05T22:22:24.697924+0000","last_became_active":"2024-04-05T22:22:23.661297+0000","last_became_peered":"2024-04-05T22:22:23.661297+0000","last_unstale":"2024-04-05T22:22:24.697924+0000","last_undegraded":"2024-04-05T22:22:24.697924+0000","last_fullsized":"2024-04-05T22:22:24.697924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694661+0000","last_change":"2024-04-05T22:22:23.636014+0000","last_active":"2024-04-05T22:22:24.694661+0000","last_peered":"2024-04-05T22:22:24.694661+0000","last_clean":"2024-04-05T22:22:24.694661+0000","last_became_active":"2024-04-05T22:22:23.635758+0000","last_became_peered":"2024-04-05T22:22:23.635758+0000","last_unstale":"2024-04-05T22:22:24.694661+0000","last_undegraded":"2024-04-05T22:22:24.694661+0000","last_fullsized":"2024-04-05T22:22:24.694661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694137+0000","last_change":"2024-04-05T22:22:23.635424+0000","last_active":"2024-04-05T22:22:24.694137+0000","last_peered":"2024-04-05T22:22:24.694137+0000","last_clean":"2024-04-05T22:22:24.694137+0000","last_became_active":"2024-04-05T22:22:23.635166+0000","last_became_peered":"2024-04-05T22:22:23.635166+0000","last_unstale":"2024-04-05T22:22:24.694137+0000","last_undegraded":"2024-04-05T22:22:24.694137+0000","last_fullsized":"2024-04-05T22:22:24.694137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635608+0000","last_change":"2024-04-05T22:22:23.669393+0000","last_active":"2024-04-05T22:22:24.635608+0000","last_peered":"2024-04-05T22:22:24.635608+0000","last_clean":"2024-04-05T22:22:24.635608+0000","last_became_active":"2024-04-05T22:22:23.668797+0000","last_became_peered":"2024-04-05T22:22:23.668797+0000","last_unstale":"2024-04-05T22:22:24.635608+0000","last_undegraded":"2024-04-05T22:22:24.635608+0000","last_fullsized":"2024-04-05T22:22:24.635608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698740+0000","last_change":"2024-04-05T22:22:23.657832+0000","last_active":"2024-04-05T22:22:24.698740+0000","last_peered":"2024-04-05T22:22:24.698740+0000","last_clean":"2024-04-05T22:22:24.698740+0000","last_became_active":"2024-04-05T22:22:23.657507+0000","last_became_peered":"2024-04-05T22:22:23.657507+0000","last_unstale":"2024-04-05T22:22:24.698740+0000","last_undegraded":"2024-04-05T22:22:24.698740+0000","last_fullsized":"2024-04-05T22:22:24.698740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698901+0000","last_change":"2024-04-05T22:22:23.648571+0000","last_active":"2024-04-05T22:22:24.698901+0000","last_peered":"2024-04-05T22:22:24.698901+0000","last_clean":"2024-04-05T22:22:24.698901+0000","last_became_active":"2024-04-05T22:22:23.647134+0000","last_became_peered":"2024-04-05T22:22:23.647134+0000","last_unstale":"2024-04-05T22:22:24.698901+0000","last_undegraded":"2024-04-05T22:22:24.698901+0000","last_fullsized":"2024-04-05T22:22:24.698901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695306+0000","last_change":"2024-04-05T22:22:23.657788+0000","last_active":"2024-04-05T22:22:24.695306+0000","last_peered":"2024-04-05T22:22:24.695306+0000","last_clean":"2024-04-05T22:22:24.695306+0000","last_became_active":"2024-04-05T22:22:23.657531+0000","last_became_peered":"2024-04-05T22:22:23.657531+0000","last_unstale":"2024-04-05T22:22:24.695306+0000","last_undegraded":"2024-04-05T22:22:24.695306+0000","last_fullsized":"2024-04-05T22:22:24.695306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639005+0000","last_change":"2024-04-05T22:22:23.658718+0000","last_active":"2024-04-05T22:22:24.639005+0000","last_peered":"2024-04-05T22:22:24.639005+0000","last_clean":"2024-04-05T22:22:24.639005+0000","last_became_active":"2024-04-05T22:22:23.658443+0000","last_became_peered":"2024-04-05T22:22:23.658443+0000","last_unstale":"2024-04-05T22:22:24.639005+0000","last_undegraded":"2024-04-05T22:22:24.639005+0000","last_fullsized":"2024-04-05T22:22:24.639005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637646+0000","last_change":"2024-04-05T22:22:23.646258+0000","last_active":"2024-04-05T22:22:24.637646+0000","last_peered":"2024-04-05T22:22:24.637646+0000","last_clean":"2024-04-05T22:22:24.637646+0000","last_became_active":"2024-04-05T22:22:23.645504+0000","last_became_peered":"2024-04-05T22:22:23.645504+0000","last_unstale":"2024-04-05T22:22:24.637646+0000","last_undegraded":"2024-04-05T22:22:24.637646+0000","last_fullsized":"2024-04-05T22:22:24.637646+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694597+0000","last_change":"2024-04-05T22:22:23.661228+0000","last_active":"2024-04-05T22:22:24.694597+0000","last_peered":"2024-04-05T22:22:24.694597+0000","last_clean":"2024-04-05T22:22:24.694597+0000","last_became_active":"2024-04-05T22:22:23.660897+0000","last_became_peered":"2024-04-05T22:22:23.660897+0000","last_unstale":"2024-04-05T22:22:24.694597+0000","last_undegraded":"2024-04-05T22:22:24.694597+0000","last_fullsized":"2024-04-05T22:22:24.694597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:14.374696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637429+0000","last_change":"2024-04-05T22:22:23.658697+0000","last_active":"2024-04-05T22:22:24.637429+0000","last_peered":"2024-04-05T22:22:24.637429+0000","last_clean":"2024-04-05T22:22:24.637429+0000","last_became_active":"2024-04-05T22:22:23.658228+0000","last_became_peered":"2024-04-05T22:22:23.658228+0000","last_unstale":"2024-04-05T22:22:24.637429+0000","last_undegraded":"2024-04-05T22:22:24.637429+0000","last_fullsized":"2024-04-05T22:22:24.637429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639456+0000","last_change":"2024-04-05T22:22:23.660227+0000","last_active":"2024-04-05T22:22:24.639456+0000","last_peered":"2024-04-05T22:22:24.639456+0000","last_clean":"2024-04-05T22:22:24.639456+0000","last_became_active":"2024-04-05T22:22:23.659896+0000","last_became_peered":"2024-04-05T22:22:23.659896+0000","last_unstale":"2024-04-05T22:22:24.639456+0000","last_undegraded":"2024-04-05T22:22:24.639456+0000","last_fullsized":"2024-04-05T22:22:24.639456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698795+0000","last_change":"2024-04-05T22:22:23.649270+0000","last_active":"2024-04-05T22:22:24.698795+0000","last_peered":"2024-04-05T22:22:24.698795+0000","last_clean":"2024-04-05T22:22:24.698795+0000","last_became_active":"2024-04-05T22:22:23.648854+0000","last_became_peered":"2024-04-05T22:22:23.648854+0000","last_unstale":"2024-04-05T22:22:24.698795+0000","last_undegraded":"2024-04-05T22:22:24.698795+0000","last_fullsized":"2024-04-05T22:22:24.698795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697381+0000","last_change":"2024-04-05T22:22:23.656696+0000","last_active":"2024-04-05T22:22:24.697381+0000","last_peered":"2024-04-05T22:22:24.697381+0000","last_clean":"2024-04-05T22:22:24.697381+0000","last_became_active":"2024-04-05T22:22:23.656289+0000","last_became_peered":"2024-04-05T22:22:23.656289+0000","last_unstale":"2024-04-05T22:22:24.697381+0000","last_undegraded":"2024-04-05T22:22:24.697381+0000","last_fullsized":"2024-04-05T22:22:24.697381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:58.596322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696198+0000","last_change":"2024-04-05T22:22:23.661467+0000","last_active":"2024-04-05T22:22:24.696198+0000","last_peered":"2024-04-05T22:22:24.696198+0000","last_clean":"2024-04-05T22:22:24.696198+0000","last_became_active":"2024-04-05T22:22:23.661164+0000","last_became_peered":"2024-04-05T22:22:23.661164+0000","last_unstale":"2024-04-05T22:22:24.696198+0000","last_undegraded":"2024-04-05T22:22:24.696198+0000","last_fullsized":"2024-04-05T22:22:24.696198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:24.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696451+0000","last_change":"2024-04-05T22:22:23.654718+0000","last_active":"2024-04-05T22:22:24.696451+0000","last_peered":"2024-04-05T22:22:24.696451+0000","last_clean":"2024-04-05T22:22:24.696451+0000","last_became_active":"2024-04-05T22:22:23.654401+0000","last_became_peered":"2024-04-05T22:22:23.654401+0000","last_unstale":"2024-04-05T22:22:24.696451+0000","last_undegraded":"2024-04-05T22:22:24.696451+0000","last_fullsized":"2024-04-05T22:22:24.696451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:00:31.890854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695252+0000","last_change":"2024-04-05T22:22:23.656947+0000","last_active":"2024-04-05T22:22:24.695252+0000","last_peered":"2024-04-05T22:22:24.695252+0000","last_clean":"2024-04-05T22:22:24.695252+0000","last_became_active":"2024-04-05T22:22:23.656762+0000","last_became_peered":"2024-04-05T22:22:23.656762+0000","last_unstale":"2024-04-05T22:22:24.695252+0000","last_undegraded":"2024-04-05T22:22:24.695252+0000","last_fullsized":"2024-04-05T22:22:24.695252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673739+0000","last_change":"2024-04-05T22:22:23.167846+0000","last_active":"2024-04-05T22:22:23.673739+0000","last_peered":"2024-04-05T22:22:23.673739+0000","last_clean":"2024-04-05T22:22:23.673739+0000","last_became_active":"2024-04-05T22:22:23.167499+0000","last_became_peered":"2024-04-05T22:22:23.167499+0000","last_unstale":"2024-04-05T22:22:23.673739+0000","last_undegraded":"2024-04-05T22:22:23.673739+0000","last_fullsized":"2024-04-05T22:22:23.673739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:07:08.680338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696281+0000","last_change":"2024-04-05T22:22:23.665407+0000","last_active":"2024-04-05T22:22:24.696281+0000","last_peered":"2024-04-05T22:22:24.696281+0000","last_clean":"2024-04-05T22:22:24.696281+0000","last_became_active":"2024-04-05T22:22:23.661923+0000","last_became_peered":"2024-04-05T22:22:23.661923+0000","last_unstale":"2024-04-05T22:22:24.696281+0000","last_undegraded":"2024-04-05T22:22:24.696281+0000","last_fullsized":"2024-04-05T22:22:24.696281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:41:16.993667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656516+0000","last_change":"2024-04-05T22:22:23.173935+0000","last_active":"2024-04-05T22:22:23.656516+0000","last_peered":"2024-04-05T22:22:23.656516+0000","last_clean":"2024-04-05T22:22:23.656516+0000","last_became_active":"2024-04-05T22:22:23.173626+0000","last_became_peered":"2024-04-05T22:22:23.173626+0000","last_unstale":"2024-04-05T22:22:23.656516+0000","last_undegraded":"2024-04-05T22:22:23.656516+0000","last_fullsized":"2024-04-05T22:22:23.656516+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:54.000983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699314+0000","last_change":"2024-04-05T22:22:23.645196+0000","last_active":"2024-04-05T22:22:24.699314+0000","last_peered":"2024-04-05T22:22:24.699314+0000","last_clean":"2024-04-05T22:22:24.699314+0000","last_became_active":"2024-04-05T22:22:23.644571+0000","last_became_peered":"2024-04-05T22:22:23.644571+0000","last_unstale":"2024-04-05T22:22:24.699314+0000","last_undegraded":"2024-04-05T22:22:24.699314+0000","last_fullsized":"2024-04-05T22:22:24.699314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.663712+0000","last_change":"2024-04-05T22:22:23.167036+0000","last_active":"2024-04-05T22:22:23.663712+0000","last_peered":"2024-04-05T22:22:23.663712+0000","last_clean":"2024-04-05T22:22:23.663712+0000","last_became_active":"2024-04-05T22:22:23.166675+0000","last_became_peered":"2024-04-05T22:22:23.166675+0000","last_unstale":"2024-04-05T22:22:23.663712+0000","last_undegraded":"2024-04-05T22:22:23.663712+0000","last_fullsized":"2024-04-05T22:22:23.663712+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637547+0000","last_change":"2024-04-05T22:22:23.653873+0000","last_active":"2024-04-05T22:22:24.637547+0000","last_peered":"2024-04-05T22:22:24.637547+0000","last_clean":"2024-04-05T22:22:24.637547+0000","last_became_active":"2024-04-05T22:22:23.653328+0000","last_became_peered":"2024-04-05T22:22:23.653328+0000","last_unstale":"2024-04-05T22:22:24.637547+0000","last_undegraded":"2024-04-05T22:22:24.637547+0000","last_fullsized":"2024-04-05T22:22:24.637547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.659545+0000","last_change":"2024-04-05T22:22:23.165579+0000","last_active":"2024-04-05T22:22:23.659545+0000","last_peered":"2024-04-05T22:22:23.659545+0000","last_clean":"2024-04-05T22:22:23.659545+0000","last_became_active":"2024-04-05T22:22:23.164825+0000","last_became_peered":"2024-04-05T22:22:23.164825+0000","last_unstale":"2024-04-05T22:22:23.659545+0000","last_undegraded":"2024-04-05T22:22:23.659545+0000","last_fullsized":"2024-04-05T22:22:23.659545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697512+0000","last_change":"2024-04-05T22:22:23.651254+0000","last_active":"2024-04-05T22:22:24.697512+0000","last_peered":"2024-04-05T22:22:24.697512+0000","last_clean":"2024-04-05T22:22:24.697512+0000","last_became_active":"2024-04-05T22:22:23.650666+0000","last_became_peered":"2024-04-05T22:22:23.650666+0000","last_unstale":"2024-04-05T22:22:24.697512+0000","last_undegraded":"2024-04-05T22:22:24.697512+0000","last_fullsized":"2024-04-05T22:22:24.697512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668306+0000","last_change":"2024-04-05T22:22:23.161130+0000","last_active":"2024-04-05T22:22:23.668306+0000","last_peered":"2024-04-05T22:22:23.668306+0000","last_clean":"2024-04-05T22:22:23.668306+0000","last_became_active":"2024-04-05T22:22:23.160816+0000","last_became_peered":"2024-04-05T22:22:23.160816+0000","last_unstale":"2024-04-05T22:22:23.668306+0000","last_undegraded":"2024-04-05T22:22:23.668306+0000","last_fullsized":"2024-04-05T22:22:23.668306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:19:31.591205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.638992+0000","last_change":"2024-04-05T22:22:23.663030+0000","last_active":"2024-04-05T22:22:24.638992+0000","last_peered":"2024-04-05T22:22:24.638992+0000","last_clean":"2024-04-05T22:22:24.638992+0000","last_became_active":"2024-04-05T22:22:23.662634+0000","last_became_peered":"2024-04-05T22:22:23.662634+0000","last_unstale":"2024-04-05T22:22:24.638992+0000","last_undegraded":"2024-04-05T22:22:24.638992+0000","last_fullsized":"2024-04-05T22:22:24.638992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.669884+0000","last_change":"2024-04-05T22:22:23.171512+0000","last_active":"2024-04-05T22:22:23.669884+0000","last_peered":"2024-04-05T22:22:23.669884+0000","last_clean":"2024-04-05T22:22:23.669884+0000","last_became_active":"2024-04-05T22:22:23.171093+0000","last_became_peered":"2024-04-05T22:22:23.171093+0000","last_unstale":"2024-04-05T22:22:23.669884+0000","last_undegraded":"2024-04-05T22:22:23.669884+0000","last_fullsized":"2024-04-05T22:22:23.669884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:27:39.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.662602+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.662280+0000","last_became_peered":"2024-04-05T22:22:23.662280+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627380+0000","last_change":"2024-04-05T22:22:23.161421+0000","last_active":"2024-04-05T22:22:23.627380+0000","last_peered":"2024-04-05T22:22:23.627380+0000","last_clean":"2024-04-05T22:22:23.627380+0000","last_became_active":"2024-04-05T22:22:23.161106+0000","last_became_peered":"2024-04-05T22:22:23.161106+0000","last_unstale":"2024-04-05T22:22:23.627380+0000","last_undegraded":"2024-04-05T22:22:23.627380+0000","last_fullsized":"2024-04-05T22:22:23.627380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636847+0000","last_change":"2024-04-05T22:22:23.665869+0000","last_active":"2024-04-05T22:22:24.636847+0000","last_peered":"2024-04-05T22:22:24.636847+0000","last_clean":"2024-04-05T22:22:24.636847+0000","last_became_active":"2024-04-05T22:22:23.665635+0000","last_became_peered":"2024-04-05T22:22:23.665635+0000","last_unstale":"2024-04-05T22:22:24.636847+0000","last_undegraded":"2024-04-05T22:22:24.636847+0000","last_fullsized":"2024-04-05T22:22:24.636847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645193+0000","last_change":"2024-04-05T22:22:23.161442+0000","last_active":"2024-04-05T22:22:23.645193+0000","last_peered":"2024-04-05T22:22:23.645193+0000","last_clean":"2024-04-05T22:22:23.645193+0000","last_became_active":"2024-04-05T22:22:23.161144+0000","last_became_peered":"2024-04-05T22:22:23.161144+0000","last_unstale":"2024-04-05T22:22:23.645193+0000","last_undegraded":"2024-04-05T22:22:23.645193+0000","last_fullsized":"2024-04-05T22:22:23.645193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:11.473072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640122+0000","last_change":"2024-04-05T22:22:23.668335+0000","last_active":"2024-04-05T22:22:24.640122+0000","last_peered":"2024-04-05T22:22:24.640122+0000","last_clean":"2024-04-05T22:22:24.640122+0000","last_became_active":"2024-04-05T22:22:23.667779+0000","last_became_peered":"2024-04-05T22:22:23.667779+0000","last_unstale":"2024-04-05T22:22:24.640122+0000","last_undegraded":"2024-04-05T22:22:24.640122+0000","last_fullsized":"2024-04-05T22:22:24.640122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.630023+0000","last_change":"2024-04-05T22:22:23.170597+0000","last_active":"2024-04-05T22:22:23.630023+0000","last_peered":"2024-04-05T22:22:23.630023+0000","last_clean":"2024-04-05T22:22:23.630023+0000","last_became_active":"2024-04-05T22:22:23.170261+0000","last_became_peered":"2024-04-05T22:22:23.170261+0000","last_unstale":"2024-04-05T22:22:23.630023+0000","last_undegraded":"2024-04-05T22:22:23.630023+0000","last_fullsized":"2024-04-05T22:22:23.630023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:18.168591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694569+0000","last_change":"2024-04-05T22:22:23.637101+0000","last_active":"2024-04-05T22:22:24.694569+0000","last_peered":"2024-04-05T22:22:24.694569+0000","last_clean":"2024-04-05T22:22:24.694569+0000","last_became_active":"2024-04-05T22:22:23.636845+0000","last_became_peered":"2024-04-05T22:22:23.636845+0000","last_unstale":"2024-04-05T22:22:24.694569+0000","last_undegraded":"2024-04-05T22:22:24.694569+0000","last_fullsized":"2024-04-05T22:22:24.694569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646846+0000","last_change":"2024-04-05T22:22:23.159233+0000","last_active":"2024-04-05T22:22:23.646846+0000","last_peered":"2024-04-05T22:22:23.646846+0000","last_clean":"2024-04-05T22:22:23.646846+0000","last_became_active":"2024-04-05T22:22:23.158972+0000","last_became_peered":"2024-04-05T22:22:23.158972+0000","last_unstale":"2024-04-05T22:22:23.646846+0000","last_undegraded":"2024-04-05T22:22:23.646846+0000","last_fullsized":"2024-04-05T22:22:23.646846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:58:26.510282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694963+0000","last_change":"2024-04-05T22:22:23.660328+0000","last_active":"2024-04-05T22:22:24.694963+0000","last_peered":"2024-04-05T22:22:24.694963+0000","last_clean":"2024-04-05T22:22:24.694963+0000","last_became_active":"2024-04-05T22:22:23.659665+0000","last_became_peered":"2024-04-05T22:22:23.659665+0000","last_unstale":"2024-04-05T22:22:24.694963+0000","last_undegraded":"2024-04-05T22:22:24.694963+0000","last_fullsized":"2024-04-05T22:22:24.694963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:33:30.562929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.674054+0000","last_change":"2024-04-05T22:22:23.174204+0000","last_active":"2024-04-05T22:22:23.674054+0000","last_peered":"2024-04-05T22:22:23.674054+0000","last_clean":"2024-04-05T22:22:23.674054+0000","last_became_active":"2024-04-05T22:22:23.173693+0000","last_became_peered":"2024-04-05T22:22:23.173693+0000","last_unstale":"2024-04-05T22:22:23.674054+0000","last_undegraded":"2024-04-05T22:22:23.674054+0000","last_fullsized":"2024-04-05T22:22:23.674054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639858+0000","last_change":"2024-04-05T22:22:23.672617+0000","last_active":"2024-04-05T22:22:24.639858+0000","last_peered":"2024-04-05T22:22:24.639858+0000","last_clean":"2024-04-05T22:22:24.639858+0000","last_became_active":"2024-04-05T22:22:23.670917+0000","last_became_peered":"2024-04-05T22:22:23.670917+0000","last_unstale":"2024-04-05T22:22:24.639858+0000","last_undegraded":"2024-04-05T22:22:24.639858+0000","last_fullsized":"2024-04-05T22:22:24.639858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673235+0000","last_change":"2024-04-05T22:22:23.163943+0000","last_active":"2024-04-05T22:22:23.673235+0000","last_peered":"2024-04-05T22:22:23.673235+0000","last_clean":"2024-04-05T22:22:23.673235+0000","last_became_active":"2024-04-05T22:22:23.163784+0000","last_became_peered":"2024-04-05T22:22:23.163784+0000","last_unstale":"2024-04-05T22:22:23.673235+0000","last_undegraded":"2024-04-05T22:22:23.673235+0000","last_fullsized":"2024-04-05T22:22:23.673235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636948+0000","last_change":"2024-04-05T22:22:23.666108+0000","last_active":"2024-04-05T22:22:24.636948+0000","last_peered":"2024-04-05T22:22:24.636948+0000","last_clean":"2024-04-05T22:22:24.636948+0000","last_became_active":"2024-04-05T22:22:23.665839+0000","last_became_peered":"2024-04-05T22:22:23.665839+0000","last_unstale":"2024-04-05T22:22:24.636948+0000","last_undegraded":"2024-04-05T22:22:24.636948+0000","last_fullsized":"2024-04-05T22:22:24.636948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668765+0000","last_change":"2024-04-05T22:22:23.164105+0000","last_active":"2024-04-05T22:22:23.668765+0000","last_peered":"2024-04-05T22:22:23.668765+0000","last_clean":"2024-04-05T22:22:23.668765+0000","last_became_active":"2024-04-05T22:22:23.163812+0000","last_became_peered":"2024-04-05T22:22:23.163812+0000","last_unstale":"2024-04-05T22:22:23.668765+0000","last_undegraded":"2024-04-05T22:22:23.668765+0000","last_fullsized":"2024-04-05T22:22:23.668765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:51.706002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694350+0000","last_change":"2024-04-05T22:22:23.651615+0000","last_active":"2024-04-05T22:22:24.694350+0000","last_peered":"2024-04-05T22:22:24.694350+0000","last_clean":"2024-04-05T22:22:24.694350+0000","last_became_active":"2024-04-05T22:22:23.651372+0000","last_became_peered":"2024-04-05T22:22:23.651372+0000","last_unstale":"2024-04-05T22:22:24.694350+0000","last_undegraded":"2024-04-05T22:22:24.694350+0000","last_fullsized":"2024-04-05T22:22:24.694350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625635+0000","last_change":"2024-04-05T22:22:23.164204+0000","last_active":"2024-04-05T22:22:23.625635+0000","last_peered":"2024-04-05T22:22:23.625635+0000","last_clean":"2024-04-05T22:22:23.625635+0000","last_became_active":"2024-04-05T22:22:23.163969+0000","last_became_peered":"2024-04-05T22:22:23.163969+0000","last_unstale":"2024-04-05T22:22:23.625635+0000","last_undegraded":"2024-04-05T22:22:23.625635+0000","last_fullsized":"2024-04-05T22:22:23.625635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696698+0000","last_change":"2024-04-05T22:22:23.659000+0000","last_active":"2024-04-05T22:22:24.696698+0000","last_peered":"2024-04-05T22:22:24.696698+0000","last_clean":"2024-04-05T22:22:24.696698+0000","last_became_active":"2024-04-05T22:22:23.655679+0000","last_became_peered":"2024-04-05T22:22:23.655679+0000","last_unstale":"2024-04-05T22:22:24.696698+0000","last_undegraded":"2024-04-05T22:22:24.696698+0000","last_fullsized":"2024-04-05T22:22:24.696698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627490+0000","last_change":"2024-04-05T22:22:23.169398+0000","last_active":"2024-04-05T22:22:23.627490+0000","last_peered":"2024-04-05T22:22:23.627490+0000","last_clean":"2024-04-05T22:22:23.627490+0000","last_became_active":"2024-04-05T22:22:23.168976+0000","last_became_peered":"2024-04-05T22:22:23.168976+0000","last_unstale":"2024-04-05T22:22:23.627490+0000","last_undegraded":"2024-04-05T22:22:23.627490+0000","last_fullsized":"2024-04-05T22:22:23.627490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696514+0000","last_change":"2024-04-05T22:22:23.663801+0000","last_active":"2024-04-05T22:22:24.696514+0000","last_peered":"2024-04-05T22:22:24.696514+0000","last_clean":"2024-04-05T22:22:24.696514+0000","last_became_active":"2024-04-05T22:22:23.663458+0000","last_became_peered":"2024-04-05T22:22:23.663458+0000","last_unstale":"2024-04-05T22:22:24.696514+0000","last_undegraded":"2024-04-05T22:22:24.696514+0000","last_fullsized":"2024-04-05T22:22:24.696514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:57:49.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668534+0000","last_change":"2024-04-05T22:22:23.169360+0000","last_active":"2024-04-05T22:22:23.668534+0000","last_peered":"2024-04-05T22:22:23.668534+0000","last_clean":"2024-04-05T22:22:23.668534+0000","last_became_active":"2024-04-05T22:22:23.167009+0000","last_became_peered":"2024-04-05T22:22:23.167009+0000","last_unstale":"2024-04-05T22:22:23.668534+0000","last_undegraded":"2024-04-05T22:22:23.668534+0000","last_fullsized":"2024-04-05T22:22:23.668534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699135+0000","last_change":"2024-04-05T22:22:23.668712+0000","last_active":"2024-04-05T22:22:24.699135+0000","last_peered":"2024-04-05T22:22:24.699135+0000","last_clean":"2024-04-05T22:22:24.699135+0000","last_became_active":"2024-04-05T22:22:23.668435+0000","last_became_peered":"2024-04-05T22:22:23.668435+0000","last_unstale":"2024-04-05T22:22:24.699135+0000","last_undegraded":"2024-04-05T22:22:24.699135+0000","last_fullsized":"2024-04-05T22:22:24.699135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667960+0000","last_change":"2024-04-05T22:22:23.164411+0000","last_active":"2024-04-05T22:22:23.667960+0000","last_peered":"2024-04-05T22:22:23.667960+0000","last_clean":"2024-04-05T22:22:23.667960+0000","last_became_active":"2024-04-05T22:22:23.162250+0000","last_became_peered":"2024-04-05T22:22:23.162250+0000","last_unstale":"2024-04-05T22:22:23.667960+0000","last_undegraded":"2024-04-05T22:22:23.667960+0000","last_fullsized":"2024-04-05T22:22:23.667960+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640160+0000","last_change":"2024-04-05T22:22:23.673810+0000","last_active":"2024-04-05T22:22:24.640160+0000","last_peered":"2024-04-05T22:22:24.640160+0000","last_clean":"2024-04-05T22:22:24.640160+0000","last_became_active":"2024-04-05T22:22:23.673503+0000","last_became_peered":"2024-04-05T22:22:23.673503+0000","last_unstale":"2024-04-05T22:22:24.640160+0000","last_undegraded":"2024-04-05T22:22:24.640160+0000","last_fullsized":"2024-04-05T22:22:24.640160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.670057+0000","last_change":"2024-04-05T22:22:23.156737+0000","last_active":"2024-04-05T22:22:23.670057+0000","last_peered":"2024-04-05T22:22:23.670057+0000","last_clean":"2024-04-05T22:22:23.670057+0000","last_became_active":"2024-04-05T22:22:23.156286+0000","last_became_peered":"2024-04-05T22:22:23.156286+0000","last_unstale":"2024-04-05T22:22:23.670057+0000","last_undegraded":"2024-04-05T22:22:23.670057+0000","last_fullsized":"2024-04-05T22:22:23.670057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637263+0000","last_change":"2024-04-05T22:22:23.656662+0000","last_active":"2024-04-05T22:22:24.637263+0000","last_peered":"2024-04-05T22:22:24.637263+0000","last_clean":"2024-04-05T22:22:24.637263+0000","last_became_active":"2024-04-05T22:22:23.656406+0000","last_became_peered":"2024-04-05T22:22:23.656406+0000","last_unstale":"2024-04-05T22:22:24.637263+0000","last_undegraded":"2024-04-05T22:22:24.637263+0000","last_fullsized":"2024-04-05T22:22:24.637263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667743+0000","last_change":"2024-04-05T22:22:23.158044+0000","last_active":"2024-04-05T22:22:23.667743+0000","last_peered":"2024-04-05T22:22:23.667743+0000","last_clean":"2024-04-05T22:22:23.667743+0000","last_became_active":"2024-04-05T22:22:23.157699+0000","last_became_peered":"2024-04-05T22:22:23.157699+0000","last_unstale":"2024-04-05T22:22:23.667743+0000","last_undegraded":"2024-04-05T22:22:23.667743+0000","last_fullsized":"2024-04-05T22:22:23.667743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:58:42.971651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635512+0000","last_change":"2024-04-05T22:22:23.650874+0000","last_active":"2024-04-05T22:22:24.635512+0000","last_peered":"2024-04-05T22:22:24.635512+0000","last_clean":"2024-04-05T22:22:24.635512+0000","last_became_active":"2024-04-05T22:22:23.650617+0000","last_became_peered":"2024-04-05T22:22:23.650617+0000","last_unstale":"2024-04-05T22:22:24.635512+0000","last_undegraded":"2024-04-05T22:22:24.635512+0000","last_fullsized":"2024-04-05T22:22:24.635512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.642139+0000","last_change":"2024-04-05T22:22:23.171641+0000","last_active":"2024-04-05T22:22:23.642139+0000","last_peered":"2024-04-05T22:22:23.642139+0000","last_clean":"2024-04-05T22:22:23.642139+0000","last_became_active":"2024-04-05T22:22:23.171224+0000","last_became_peered":"2024-04-05T22:22:23.171224+0000","last_unstale":"2024-04-05T22:22:23.642139+0000","last_undegraded":"2024-04-05T22:22:23.642139+0000","last_fullsized":"2024-04-05T22:22:23.642139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:26.099736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635186+0000","last_change":"2024-04-05T22:22:23.649481+0000","last_active":"2024-04-05T22:22:24.635186+0000","last_peered":"2024-04-05T22:22:24.635186+0000","last_clean":"2024-04-05T22:22:24.635186+0000","last_became_active":"2024-04-05T22:22:23.649158+0000","last_became_peered":"2024-04-05T22:22:23.649158+0000","last_unstale":"2024-04-05T22:22:24.635186+0000","last_undegraded":"2024-04-05T22:22:24.635186+0000","last_fullsized":"2024-04-05T22:22:24.635186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.661505+0000","last_change":"2024-04-05T22:22:23.153928+0000","last_active":"2024-04-05T22:22:23.661505+0000","last_peered":"2024-04-05T22:22:23.661505+0000","last_clean":"2024-04-05T22:22:23.661505+0000","last_became_active":"2024-04-05T22:22:23.153301+0000","last_became_peered":"2024-04-05T22:22:23.153301+0000","last_unstale":"2024-04-05T22:22:23.661505+0000","last_undegraded":"2024-04-05T22:22:23.661505+0000","last_fullsized":"2024-04-05T22:22:23.661505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635294+0000","last_change":"2024-04-05T22:22:23.656526+0000","last_active":"2024-04-05T22:22:24.635294+0000","last_peered":"2024-04-05T22:22:24.635294+0000","last_clean":"2024-04-05T22:22:24.635294+0000","last_became_active":"2024-04-05T22:22:23.655430+0000","last_became_peered":"2024-04-05T22:22:23.655430+0000","last_unstale":"2024-04-05T22:22:24.635294+0000","last_undegraded":"2024-04-05T22:22:24.635294+0000","last_fullsized":"2024-04-05T22:22:24.635294+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:24.841919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.648674+0000","last_change":"2024-04-05T22:22:23.164487+0000","last_active":"2024-04-05T22:22:23.648674+0000","last_peered":"2024-04-05T22:22:23.648674+0000","last_clean":"2024-04-05T22:22:23.648674+0000","last_became_active":"2024-04-05T22:22:23.163704+0000","last_became_peered":"2024-04-05T22:22:23.163704+0000","last_unstale":"2024-04-05T22:22:23.648674+0000","last_undegraded":"2024-04-05T22:22:23.648674+0000","last_fullsized":"2024-04-05T22:22:23.648674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:25.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635044+0000","last_change":"2024-04-05T22:22:23.651556+0000","last_active":"2024-04-05T22:22:24.635044+0000","last_peered":"2024-04-05T22:22:24.635044+0000","last_clean":"2024-04-05T22:22:24.635044+0000","last_became_active":"2024-04-05T22:22:23.651314+0000","last_became_peered":"2024-04-05T22:22:23.651314+0000","last_unstale":"2024-04-05T22:22:24.635044+0000","last_undegraded":"2024-04-05T22:22:24.635044+0000","last_fullsized":"2024-04-05T22:22:24.635044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636186+0000","last_change":"2024-04-05T22:22:23.155981+0000","last_active":"2024-04-05T22:22:23.636186+0000","last_peered":"2024-04-05T22:22:23.636186+0000","last_clean":"2024-04-05T22:22:23.636186+0000","last_became_active":"2024-04-05T22:22:23.154363+0000","last_became_peered":"2024-04-05T22:22:23.154363+0000","last_unstale":"2024-04-05T22:22:23.636186+0000","last_undegraded":"2024-04-05T22:22:23.636186+0000","last_fullsized":"2024-04-05T22:22:23.636186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:13.978598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696889+0000","last_change":"2024-04-05T22:22:23.657509+0000","last_active":"2024-04-05T22:22:24.696889+0000","last_peered":"2024-04-05T22:22:24.696889+0000","last_clean":"2024-04-05T22:22:24.696889+0000","last_became_active":"2024-04-05T22:22:23.657035+0000","last_became_peered":"2024-04-05T22:22:23.657035+0000","last_unstale":"2024-04-05T22:22:24.696889+0000","last_undegraded":"2024-04-05T22:22:24.696889+0000","last_fullsized":"2024-04-05T22:22:24.696889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656814+0000","last_change":"2024-04-05T22:22:23.172774+0000","last_active":"2024-04-05T22:22:23.656814+0000","last_peered":"2024-04-05T22:22:23.656814+0000","last_clean":"2024-04-05T22:22:23.656814+0000","last_became_active":"2024-04-05T22:22:23.172492+0000","last_became_peered":"2024-04-05T22:22:23.172492+0000","last_unstale":"2024-04-05T22:22:23.656814+0000","last_undegraded":"2024-04-05T22:22:23.656814+0000","last_fullsized":"2024-04-05T22:22:23.656814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635485+0000","last_change":"2024-04-05T22:22:23.667039+0000","last_active":"2024-04-05T22:22:24.635485+0000","last_peered":"2024-04-05T22:22:24.635485+0000","last_clean":"2024-04-05T22:22:24.635485+0000","last_became_active":"2024-04-05T22:22:23.666665+0000","last_became_peered":"2024-04-05T22:22:23.666665+0000","last_unstale":"2024-04-05T22:22:24.635485+0000","last_undegraded":"2024-04-05T22:22:24.635485+0000","last_fullsized":"2024-04-05T22:22:24.635485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.665989+0000","last_change":"2024-04-05T22:22:23.165988+0000","last_active":"2024-04-05T22:22:23.665989+0000","last_peered":"2024-04-05T22:22:23.665989+0000","last_clean":"2024-04-05T22:22:23.665989+0000","last_became_active":"2024-04-05T22:22:23.165708+0000","last_became_peered":"2024-04-05T22:22:23.165708+0000","last_unstale":"2024-04-05T22:22:23.665989+0000","last_undegraded":"2024-04-05T22:22:23.665989+0000","last_fullsized":"2024-04-05T22:22:23.665989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640236+0000","last_change":"2024-04-05T22:22:23.664010+0000","last_active":"2024-04-05T22:22:24.640236+0000","last_peered":"2024-04-05T22:22:24.640236+0000","last_clean":"2024-04-05T22:22:24.640236+0000","last_became_active":"2024-04-05T22:22:23.663608+0000","last_became_peered":"2024-04-05T22:22:23.663608+0000","last_unstale":"2024-04-05T22:22:24.640236+0000","last_undegraded":"2024-04-05T22:22:24.640236+0000","last_fullsized":"2024-04-05T22:22:24.640236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695548+0000","last_change":"2024-04-05T22:22:23.154936+0000","last_active":"2024-04-05T22:22:24.695548+0000","last_peered":"2024-04-05T22:22:24.695548+0000","last_clean":"2024-04-05T22:22:24.695548+0000","last_became_active":"2024-04-05T22:22:23.154664+0000","last_became_peered":"2024-04-05T22:22:23.154664+0000","last_unstale":"2024-04-05T22:22:24.695548+0000","last_undegraded":"2024-04-05T22:22:24.695548+0000","last_fullsized":"2024-04-05T22:22:24.695548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:23.826984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695278+0000","last_change":"2024-04-05T22:22:23.649531+0000","last_active":"2024-04-05T22:22:24.695278+0000","last_peered":"2024-04-05T22:22:24.695278+0000","last_clean":"2024-04-05T22:22:24.695278+0000","last_became_active":"2024-04-05T22:22:23.649241+0000","last_became_peered":"2024-04-05T22:22:23.649241+0000","last_unstale":"2024-04-05T22:22:24.695278+0000","last_undegraded":"2024-04-05T22:22:24.695278+0000","last_fullsized":"2024-04-05T22:22:24.695278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:19.183526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.626116+0000","last_change":"2024-04-05T22:22:23.160628+0000","last_active":"2024-04-05T22:22:23.626116+0000","last_peered":"2024-04-05T22:22:23.626116+0000","last_clean":"2024-04-05T22:22:23.626116+0000","last_became_active":"2024-04-05T22:22:23.159150+0000","last_became_peered":"2024-04-05T22:22:23.159150+0000","last_unstale":"2024-04-05T22:22:23.626116+0000","last_undegraded":"2024-04-05T22:22:23.626116+0000","last_fullsized":"2024-04-05T22:22:23.626116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696531+0000","last_change":"2024-04-05T22:22:23.666678+0000","last_active":"2024-04-05T22:22:24.696531+0000","last_peered":"2024-04-05T22:22:24.696531+0000","last_clean":"2024-04-05T22:22:24.696531+0000","last_became_active":"2024-04-05T22:22:23.665881+0000","last_became_peered":"2024-04-05T22:22:23.665881+0000","last_unstale":"2024-04-05T22:22:24.696531+0000","last_undegraded":"2024-04-05T22:22:24.696531+0000","last_fullsized":"2024-04-05T22:22:24.696531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:36.143521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646709+0000","last_change":"2024-04-05T22:22:23.166020+0000","last_active":"2024-04-05T22:22:23.646709+0000","last_peered":"2024-04-05T22:22:23.646709+0000","last_clean":"2024-04-05T22:22:23.646709+0000","last_became_active":"2024-04-05T22:22:23.165719+0000","last_became_peered":"2024-04-05T22:22:23.165719+0000","last_unstale":"2024-04-05T22:22:23.646709+0000","last_undegraded":"2024-04-05T22:22:23.646709+0000","last_fullsized":"2024-04-05T22:22:23.646709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:58.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695379+0000","last_change":"2024-04-05T22:22:23.666126+0000","last_active":"2024-04-05T22:22:24.695379+0000","last_peered":"2024-04-05T22:22:24.695379+0000","last_clean":"2024-04-05T22:22:24.695379+0000","last_became_active":"2024-04-05T22:22:23.665889+0000","last_became_peered":"2024-04-05T22:22:23.665889+0000","last_unstale":"2024-04-05T22:22:24.695379+0000","last_undegraded":"2024-04-05T22:22:24.695379+0000","last_fullsized":"2024-04-05T22:22:24.695379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625723+0000","last_change":"2024-04-05T22:22:23.166747+0000","last_active":"2024-04-05T22:22:23.625723+0000","last_peered":"2024-04-05T22:22:23.625723+0000","last_clean":"2024-04-05T22:22:23.625723+0000","last_became_active":"2024-04-05T22:22:23.166495+0000","last_became_peered":"2024-04-05T22:22:23.166495+0000","last_unstale":"2024-04-05T22:22:23.625723+0000","last_undegraded":"2024-04-05T22:22:23.625723+0000","last_fullsized":"2024-04-05T22:22:23.625723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693701+0000","last_change":"2024-04-05T22:22:23.634863+0000","last_active":"2024-04-05T22:22:24.693701+0000","last_peered":"2024-04-05T22:22:24.693701+0000","last_clean":"2024-04-05T22:22:24.693701+0000","last_became_active":"2024-04-05T22:22:23.634615+0000","last_became_peered":"2024-04-05T22:22:23.634615+0000","last_unstale":"2024-04-05T22:22:24.693701+0000","last_undegraded":"2024-04-05T22:22:24.693701+0000","last_fullsized":"2024-04-05T22:22:24.693701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:32:16.541566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636456+0000","last_change":"2024-04-05T22:22:23.168895+0000","last_active":"2024-04-05T22:22:23.636456+0000","last_peered":"2024-04-05T22:22:23.636456+0000","last_clean":"2024-04-05T22:22:23.636456+0000","last_became_active":"2024-04-05T22:22:23.168395+0000","last_became_peered":"2024-04-05T22:22:23.168395+0000","last_unstale":"2024-04-05T22:22:23.636456+0000","last_undegraded":"2024-04-05T22:22:23.636456+0000","last_fullsized":"2024-04-05T22:22:23.636456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:14:22.438726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698300+0000","last_change":"2024-04-05T22:22:23.654099+0000","last_active":"2024-04-05T22:22:24.698300+0000","last_peered":"2024-04-05T22:22:24.698300+0000","last_clean":"2024-04-05T22:22:24.698300+0000","last_became_active":"2024-04-05T22:22:23.653171+0000","last_became_peered":"2024-04-05T22:22:23.653171+0000","last_unstale":"2024-04-05T22:22:24.698300+0000","last_undegraded":"2024-04-05T22:22:24.698300+0000","last_fullsized":"2024-04-05T22:22:24.698300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627663+0000","last_change":"2024-04-05T22:22:23.173853+0000","last_active":"2024-04-05T22:22:23.627663+0000","last_peered":"2024-04-05T22:22:23.627663+0000","last_clean":"2024-04-05T22:22:23.627663+0000","last_became_active":"2024-04-05T22:22:23.173561+0000","last_became_peered":"2024-04-05T22:22:23.173561+0000","last_unstale":"2024-04-05T22:22:23.627663+0000","last_undegraded":"2024-04-05T22:22:23.627663+0000","last_fullsized":"2024-04-05T22:22:23.627663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698265+0000","last_change":"2024-04-05T22:22:23.649429+0000","last_active":"2024-04-05T22:22:24.698265+0000","last_peered":"2024-04-05T22:22:24.698265+0000","last_clean":"2024-04-05T22:22:24.698265+0000","last_became_active":"2024-04-05T22:22:23.649091+0000","last_became_peered":"2024-04-05T22:22:23.649091+0000","last_unstale":"2024-04-05T22:22:24.698265+0000","last_undegraded":"2024-04-05T22:22:24.698265+0000","last_fullsized":"2024-04-05T22:22:24.698265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:51:19.346378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645441+0000","last_change":"2024-04-05T22:22:23.162280+0000","last_active":"2024-04-05T22:22:23.645441+0000","last_peered":"2024-04-05T22:22:23.645441+0000","last_clean":"2024-04-05T22:22:23.645441+0000","last_became_active":"2024-04-05T22:22:23.161975+0000","last_became_peered":"2024-04-05T22:22:23.161975+0000","last_unstale":"2024-04-05T22:22:23.645441+0000","last_undegraded":"2024-04-05T22:22:23.645441+0000","last_fullsized":"2024-04-05T22:22:23.645441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:24:22.511159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.631303+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:22:23.631303+0000","last_peered":"2024-04-05T22:22:23.631303+0000","last_clean":"2024-04-05T22:22:23.631303+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:22:23.631303+0000","last_undegraded":"2024-04-05T22:22:23.631303+0000","last_fullsized":"2024-04-05T22:22:23.631303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":1003520,"data_stored":750473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.577},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.073}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.179},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.038}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.329}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.274},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.7010000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.534}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.829}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":413696,"data_stored":160631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.579},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.242}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.524},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.7290000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.776},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.641}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.165},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.2809999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.26}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.159}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.359}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.196},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.361},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.232}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.3979999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.075}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.159},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.425}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.137},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":160085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.621},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.999}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.264}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.83699999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.875}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.419},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.534}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.841},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.3320000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":2.0619999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.663}]}]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":750383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.747},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.98699999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.101},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:22:32.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:22:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.148+0000 7f86d4c00640 1 Processor -- start 2024-04-05T22:22:33.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.148+0000 7f86d4c00640 1 -- start start 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86d4c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86d01587d0 0x7f86d00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86d4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00ccd20 0x7f86d006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86d4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d006f5b0 con 0x7f86d006d0c0 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86d4c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d0070410 con 0x7f86d01587d0 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86d4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0158bb0 con 0x7f86d00ccd20 2024-04-05T22:22:33.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86ce800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f86d006d0c0 0x7f86d015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52154/0 (socket says 172.21.15.73:52154) 2024-04-05T22:22:33.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.150+0000 7f86ce800640 1 -- 172.21.15.73:0/1012624798 learned_addr learned my addr 172.21.15.73:0/1012624798 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744277392 0 0) 0x7f86d006f5b0 con 0x7f86d006d0c0 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cd400640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00ccd20 0x7f86d006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b4003520 con 0x7f86d006d0c0 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cde00640 1 --2- 172.21.15.73:0/1012624798 >> v2:172.21.15.157:3300/0 conn(0x7f86d01587d0 0x7f86d00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 653583894 0 0) 0x7f86b4003520 con 0x7f86d006d0c0 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00ccd20 msgr2=0x7f86d006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00ccd20 0x7f86d006cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 >> v2:172.21.15.157:3300/0 conn(0x7f86d01587d0 msgr2=0x7f86d00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 --2- 172.21.15.73:0/1012624798 >> v2:172.21.15.157:3300/0 conn(0x7f86d01587d0 0x7f86d00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d006f800 con 0x7f86d006d0c0 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f86c80019f0 con 0x7f86d006d0c0 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 >> v1:172.21.15.73:6789/0 conn(0x7f86d006d0c0 legacy=0x7f86d015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b4003970 0x7f86b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86b40047c0 0x7f86b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86b4005180 0x7f86b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0070410 con 0x7f86b4003970 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0158bb0 con 0x7f86b40047c0 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b400bec0 con 0x7f86b4005180 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.151+0000 7f86cde00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b4003970 0x7f86b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cd400640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86b40047c0 0x7f86b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86ce800640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86b4005180 0x7f86b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cde00640 1 -- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86b4005180 msgr2=0x7f86b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cde00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86b4005180 0x7f86b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cde00640 1 -- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86b40047c0 msgr2=0x7f86b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cde00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86b40047c0 0x7f86b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.152+0000 7f86cde00640 1 -- 172.21.15.73:0/1012624798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4003520 con 0x7f86b4003970 2024-04-05T22:22:33.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.153+0000 7f86cde00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b4003970 0x7f86b4003d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ba30 tx=0x7f86b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307287d79c9f3fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0032680 con 0x7f86b4003970 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c00327e0 con 0x7f86b4003970 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86cca00640 1 -- 172.21.15.73:0/1012624798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0032b70 con 0x7f86b4003970 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 -- 172.21.15.73:0/1012624798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b4003970 msgr2=0x7f86b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 --2- 172.21.15.73:0/1012624798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b4003970 0x7f86b4003d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ba30 tx=0x7f86b40057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86ce800640 1 -- 172.21.15.73:0/1012624798 reap_dead start 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 -- 172.21.15.73:0/1012624798 shutdown_connections 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 -- 172.21.15.73:0/1012624798 >> 172.21.15.73:0/1012624798 conn(0x7f86d00ce8e0 msgr2=0x7f86d00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 -- 172.21.15.73:0/1012624798 shutdown_connections 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.154+0000 7f86d4c00640 1 -- 172.21.15.73:0/1012624798 wait complete. 2024-04-05T22:22:33.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.156+0000 7f86d4c00640 1 Processor -- start 2024-04-05T22:22:33.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.156+0000 7f86d4c00640 1 -- start start 2024-04-05T22:22:33.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00b2c30 0x7f86d013c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d00b3010 0x7f86d01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0157ca0 con 0x7f86d0157820 2024-04-05T22:22:33.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d013b250 con 0x7f86d00b3010 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.157+0000 7f86d4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d013b530 con 0x7f86d00b2c30 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46592/0 (socket says 172.21.15.73:46592) 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 -- 172.21.15.73:0/953078791 learned_addr learned my addr 172.21.15.73:0/953078791 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 -- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00b2c30 msgr2=0x7f86d013c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86cde00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00b2c30 0x7f86d013c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00b2c30 0x7f86d013c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 -- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d00b3010 msgr2=0x7f86d01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d00b3010 0x7f86d01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 -- 172.21.15.73:0/953078791 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d013b810 con 0x7f86d0157820 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.158+0000 7f86ce800640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f86c8002030 tx=0x7f86c800e220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.159+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800ec30 con 0x7f86d0157820 2024-04-05T22:22:33.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.159+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c800edb0 con 0x7f86d0157820 2024-04-05T22:22:33.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.159+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8013660 con 0x7f86d0157820 2024-04-05T22:22:33.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.159+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d013baf0 con 0x7f86d0157820 2024-04-05T22:22:33.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.159+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d013aa20 con 0x7f86d0157820 2024-04-05T22:22:33.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.160+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d0066af0 con 0x7f86d0157820 2024-04-05T22:22:33.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.161+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f86c80137c0 con 0x7f86d0157820 2024-04-05T22:22:33.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.162+0000 7f86c6000640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 0x7f86b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.162+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86c8089910 con 0x7f86d0157820 2024-04-05T22:22:33.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.162+0000 7f86cde00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 0x7f86b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.163+0000 7f86cde00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 0x7f86b006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ba00 tx=0x7f86c0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.166+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f86c80549c0 con 0x7f86d0157820 2024-04-05T22:22:33.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.355+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86d00700a0 con 0x7f86b006ac00 2024-04-05T22:22:33.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.376+0000 7f86c6000640 1 -- 172.21.15.73:0/953078791 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287168 (secure 0 0 0) 0x7f86c0032950 con 0x7f86b006ac00 2024-04-05T22:22:33.381 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:33.382 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:22:33.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.382+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 msgr2=0x7f86b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.382+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 0x7f86b006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ba00 tx=0x7f86c0030000 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 msgr2=0x7f86d013a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f86c8002030 tx=0x7f86c800e220 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 shutdown_connections 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f86b006ac00 0x7f86b006d0c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0157820 0x7f86d013a4e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86d00b3010 0x7f86d01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 --2- 172.21.15.73:0/953078791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86d00b2c30 0x7f86d013c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 >> 172.21.15.73:0/953078791 conn(0x7f86d00ce8e0 msgr2=0x7f86d0158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 shutdown_connections 2024-04-05T22:22:33.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.383+0000 7f86d4c00640 1 -- 172.21.15.73:0/953078791 wait complete. 2024-04-05T22:22:33.404 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-05T22:22:32.512716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236948,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737772,"statfs":{"total":773094113280,"available":772851478528,"internally_reserved":0,"allocated":4497408,"data_stored":2465580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.892159"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.671468+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.671239+0000","last_became_peered":"2024-04-05T22:22:23.671239+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:17:31.785954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694516+0000","last_change":"2024-04-05T22:22:23.652316+0000","last_active":"2024-04-05T22:22:24.694516+0000","last_peered":"2024-04-05T22:22:24.694516+0000","last_clean":"2024-04-05T22:22:24.694516+0000","last_became_active":"2024-04-05T22:22:23.651989+0000","last_became_peered":"2024-04-05T22:22:23.651989+0000","last_unstale":"2024-04-05T22:22:24.694516+0000","last_undegraded":"2024-04-05T22:22:24.694516+0000","last_fullsized":"2024-04-05T22:22:24.694516+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639834+0000","last_change":"2024-04-05T22:22:23.672439+0000","last_active":"2024-04-05T22:22:24.639834+0000","last_peered":"2024-04-05T22:22:24.639834+0000","last_clean":"2024-04-05T22:22:24.639834+0000","last_became_active":"2024-04-05T22:22:23.672112+0000","last_became_peered":"2024-04-05T22:22:23.672112+0000","last_unstale":"2024-04-05T22:22:24.639834+0000","last_undegraded":"2024-04-05T22:22:24.639834+0000","last_fullsized":"2024-04-05T22:22:24.639834+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637508+0000","last_change":"2024-04-05T22:22:23.657494+0000","last_active":"2024-04-05T22:22:24.637508+0000","last_peered":"2024-04-05T22:22:24.637508+0000","last_clean":"2024-04-05T22:22:24.637508+0000","last_became_active":"2024-04-05T22:22:23.657164+0000","last_became_peered":"2024-04-05T22:22:23.657164+0000","last_unstale":"2024-04-05T22:22:24.637508+0000","last_undegraded":"2024-04-05T22:22:24.637508+0000","last_fullsized":"2024-04-05T22:22:24.637508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635155+0000","last_change":"2024-04-05T22:22:23.642932+0000","last_active":"2024-04-05T22:22:24.635155+0000","last_peered":"2024-04-05T22:22:24.635155+0000","last_clean":"2024-04-05T22:22:24.635155+0000","last_became_active":"2024-04-05T22:22:23.642640+0000","last_became_peered":"2024-04-05T22:22:23.642640+0000","last_unstale":"2024-04-05T22:22:24.635155+0000","last_undegraded":"2024-04-05T22:22:24.635155+0000","last_fullsized":"2024-04-05T22:22:24.635155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693992+0000","last_change":"2024-04-05T22:22:23.668411+0000","last_active":"2024-04-05T22:22:24.693992+0000","last_peered":"2024-04-05T22:22:24.693992+0000","last_clean":"2024-04-05T22:22:24.693992+0000","last_became_active":"2024-04-05T22:22:23.668135+0000","last_became_peered":"2024-04-05T22:22:23.668135+0000","last_unstale":"2024-04-05T22:22:24.693992+0000","last_undegraded":"2024-04-05T22:22:24.693992+0000","last_fullsized":"2024-04-05T22:22:24.693992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697247+0000","last_change":"2024-04-05T22:22:23.661512+0000","last_active":"2024-04-05T22:22:24.697247+0000","last_peered":"2024-04-05T22:22:24.697247+0000","last_clean":"2024-04-05T22:22:24.697247+0000","last_became_active":"2024-04-05T22:22:23.661235+0000","last_became_peered":"2024-04-05T22:22:23.661235+0000","last_unstale":"2024-04-05T22:22:24.697247+0000","last_undegraded":"2024-04-05T22:22:24.697247+0000","last_fullsized":"2024-04-05T22:22:24.697247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696214+0000","last_change":"2024-04-05T22:22:23.652995+0000","last_active":"2024-04-05T22:22:24.696214+0000","last_peered":"2024-04-05T22:22:24.696214+0000","last_clean":"2024-04-05T22:22:24.696214+0000","last_became_active":"2024-04-05T22:22:23.652715+0000","last_became_peered":"2024-04-05T22:22:23.652715+0000","last_unstale":"2024-04-05T22:22:24.696214+0000","last_undegraded":"2024-04-05T22:22:24.696214+0000","last_fullsized":"2024-04-05T22:22:24.696214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:27:23.769941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699562+0000","last_change":"2024-04-05T22:22:23.651001+0000","last_active":"2024-04-05T22:22:24.699562+0000","last_peered":"2024-04-05T22:22:24.699562+0000","last_clean":"2024-04-05T22:22:24.699562+0000","last_became_active":"2024-04-05T22:22:23.650633+0000","last_became_peered":"2024-04-05T22:22:23.650633+0000","last_unstale":"2024-04-05T22:22:24.699562+0000","last_undegraded":"2024-04-05T22:22:24.699562+0000","last_fullsized":"2024-04-05T22:22:24.699562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:23.999212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637115+0000","last_change":"2024-04-05T22:22:23.636389+0000","last_active":"2024-04-05T22:22:24.637115+0000","last_peered":"2024-04-05T22:22:24.637115+0000","last_clean":"2024-04-05T22:22:24.637115+0000","last_became_active":"2024-04-05T22:22:23.636028+0000","last_became_peered":"2024-04-05T22:22:23.636028+0000","last_unstale":"2024-04-05T22:22:24.637115+0000","last_undegraded":"2024-04-05T22:22:24.637115+0000","last_fullsized":"2024-04-05T22:22:24.637115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697848+0000","last_change":"2024-04-05T22:22:23.639648+0000","last_active":"2024-04-05T22:22:24.697848+0000","last_peered":"2024-04-05T22:22:24.697848+0000","last_clean":"2024-04-05T22:22:24.697848+0000","last_became_active":"2024-04-05T22:22:23.636042+0000","last_became_peered":"2024-04-05T22:22:23.636042+0000","last_unstale":"2024-04-05T22:22:24.697848+0000","last_undegraded":"2024-04-05T22:22:24.697848+0000","last_fullsized":"2024-04-05T22:22:24.697848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:19.468442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639272+0000","last_change":"2024-04-05T22:22:23.667880+0000","last_active":"2024-04-05T22:22:24.639272+0000","last_peered":"2024-04-05T22:22:24.639272+0000","last_clean":"2024-04-05T22:22:24.639272+0000","last_became_active":"2024-04-05T22:22:23.660507+0000","last_became_peered":"2024-04-05T22:22:23.660507+0000","last_unstale":"2024-04-05T22:22:24.639272+0000","last_undegraded":"2024-04-05T22:22:24.639272+0000","last_fullsized":"2024-04-05T22:22:24.639272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635308+0000","last_change":"2024-04-05T22:22:23.650060+0000","last_active":"2024-04-05T22:22:24.635308+0000","last_peered":"2024-04-05T22:22:24.635308+0000","last_clean":"2024-04-05T22:22:24.635308+0000","last_became_active":"2024-04-05T22:22:23.647699+0000","last_became_peered":"2024-04-05T22:22:23.647699+0000","last_unstale":"2024-04-05T22:22:24.635308+0000","last_undegraded":"2024-04-05T22:22:24.635308+0000","last_fullsized":"2024-04-05T22:22:24.635308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693997+0000","last_change":"2024-04-05T22:22:23.644515+0000","last_active":"2024-04-05T22:22:24.693997+0000","last_peered":"2024-04-05T22:22:24.693997+0000","last_clean":"2024-04-05T22:22:24.693997+0000","last_became_active":"2024-04-05T22:22:23.643820+0000","last_became_peered":"2024-04-05T22:22:23.643820+0000","last_unstale":"2024-04-05T22:22:24.693997+0000","last_undegraded":"2024-04-05T22:22:24.693997+0000","last_fullsized":"2024-04-05T22:22:24.693997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639487+0000","last_change":"2024-04-05T22:22:23.672903+0000","last_active":"2024-04-05T22:22:24.639487+0000","last_peered":"2024-04-05T22:22:24.639487+0000","last_clean":"2024-04-05T22:22:24.639487+0000","last_became_active":"2024-04-05T22:22:23.670922+0000","last_became_peered":"2024-04-05T22:22:23.670922+0000","last_unstale":"2024-04-05T22:22:24.639487+0000","last_undegraded":"2024-04-05T22:22:24.639487+0000","last_fullsized":"2024-04-05T22:22:24.639487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696950+0000","last_change":"2024-04-05T22:22:23.643442+0000","last_active":"2024-04-05T22:22:24.696950+0000","last_peered":"2024-04-05T22:22:24.696950+0000","last_clean":"2024-04-05T22:22:24.696950+0000","last_became_active":"2024-04-05T22:22:23.642787+0000","last_became_peered":"2024-04-05T22:22:23.642787+0000","last_unstale":"2024-04-05T22:22:24.696950+0000","last_undegraded":"2024-04-05T22:22:24.696950+0000","last_fullsized":"2024-04-05T22:22:24.696950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697924+0000","last_change":"2024-04-05T22:22:23.662150+0000","last_active":"2024-04-05T22:22:24.697924+0000","last_peered":"2024-04-05T22:22:24.697924+0000","last_clean":"2024-04-05T22:22:24.697924+0000","last_became_active":"2024-04-05T22:22:23.661297+0000","last_became_peered":"2024-04-05T22:22:23.661297+0000","last_unstale":"2024-04-05T22:22:24.697924+0000","last_undegraded":"2024-04-05T22:22:24.697924+0000","last_fullsized":"2024-04-05T22:22:24.697924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694661+0000","last_change":"2024-04-05T22:22:23.636014+0000","last_active":"2024-04-05T22:22:24.694661+0000","last_peered":"2024-04-05T22:22:24.694661+0000","last_clean":"2024-04-05T22:22:24.694661+0000","last_became_active":"2024-04-05T22:22:23.635758+0000","last_became_peered":"2024-04-05T22:22:23.635758+0000","last_unstale":"2024-04-05T22:22:24.694661+0000","last_undegraded":"2024-04-05T22:22:24.694661+0000","last_fullsized":"2024-04-05T22:22:24.694661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694137+0000","last_change":"2024-04-05T22:22:23.635424+0000","last_active":"2024-04-05T22:22:24.694137+0000","last_peered":"2024-04-05T22:22:24.694137+0000","last_clean":"2024-04-05T22:22:24.694137+0000","last_became_active":"2024-04-05T22:22:23.635166+0000","last_became_peered":"2024-04-05T22:22:23.635166+0000","last_unstale":"2024-04-05T22:22:24.694137+0000","last_undegraded":"2024-04-05T22:22:24.694137+0000","last_fullsized":"2024-04-05T22:22:24.694137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635608+0000","last_change":"2024-04-05T22:22:23.669393+0000","last_active":"2024-04-05T22:22:24.635608+0000","last_peered":"2024-04-05T22:22:24.635608+0000","last_clean":"2024-04-05T22:22:24.635608+0000","last_became_active":"2024-04-05T22:22:23.668797+0000","last_became_peered":"2024-04-05T22:22:23.668797+0000","last_unstale":"2024-04-05T22:22:24.635608+0000","last_undegraded":"2024-04-05T22:22:24.635608+0000","last_fullsized":"2024-04-05T22:22:24.635608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698740+0000","last_change":"2024-04-05T22:22:23.657832+0000","last_active":"2024-04-05T22:22:24.698740+0000","last_peered":"2024-04-05T22:22:24.698740+0000","last_clean":"2024-04-05T22:22:24.698740+0000","last_became_active":"2024-04-05T22:22:23.657507+0000","last_became_peered":"2024-04-05T22:22:23.657507+0000","last_unstale":"2024-04-05T22:22:24.698740+0000","last_undegraded":"2024-04-05T22:22:24.698740+0000","last_fullsized":"2024-04-05T22:22:24.698740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698901+0000","last_change":"2024-04-05T22:22:23.648571+0000","last_active":"2024-04-05T22:22:24.698901+0000","last_peered":"2024-04-05T22:22:24.698901+0000","last_clean":"2024-04-05T22:22:24.698901+0000","last_became_active":"2024-04-05T22:22:23.647134+0000","last_became_peered":"2024-04-05T22:22:23.647134+0000","last_unstale":"2024-04-05T22:22:24.698901+0000","last_undegraded":"2024-04-05T22:22:24.698901+0000","last_fullsized":"2024-04-05T22:22:24.698901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695306+0000","last_change":"2024-04-05T22:22:23.657788+0000","last_active":"2024-04-05T22:22:24.695306+0000","last_peered":"2024-04-05T22:22:24.695306+0000","last_clean":"2024-04-05T22:22:24.695306+0000","last_became_active":"2024-04-05T22:22:23.657531+0000","last_became_peered":"2024-04-05T22:22:23.657531+0000","last_unstale":"2024-04-05T22:22:24.695306+0000","last_undegraded":"2024-04-05T22:22:24.695306+0000","last_fullsized":"2024-04-05T22:22:24.695306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639005+0000","last_change":"2024-04-05T22:22:23.658718+0000","last_active":"2024-04-05T22:22:24.639005+0000","last_peered":"2024-04-05T22:22:24.639005+0000","last_clean":"2024-04-05T22:22:24.639005+0000","last_became_active":"2024-04-05T22:22:23.658443+0000","last_became_peered":"2024-04-05T22:22:23.658443+0000","last_unstale":"2024-04-05T22:22:24.639005+0000","last_undegraded":"2024-04-05T22:22:24.639005+0000","last_fullsized":"2024-04-05T22:22:24.639005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637646+0000","last_change":"2024-04-05T22:22:23.646258+0000","last_active":"2024-04-05T22:22:24.637646+0000","last_peered":"2024-04-05T22:22:24.637646+0000","last_clean":"2024-04-05T22:22:24.637646+0000","last_became_active":"2024-04-05T22:22:23.645504+0000","last_became_peered":"2024-04-05T22:22:23.645504+0000","last_unstale":"2024-04-05T22:22:24.637646+0000","last_undegraded":"2024-04-05T22:22:24.637646+0000","last_fullsized":"2024-04-05T22:22:24.637646+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:27.290588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694597+0000","last_change":"2024-04-05T22:22:23.661228+0000","last_active":"2024-04-05T22:22:24.694597+0000","last_peered":"2024-04-05T22:22:24.694597+0000","last_clean":"2024-04-05T22:22:24.694597+0000","last_became_active":"2024-04-05T22:22:23.660897+0000","last_became_peered":"2024-04-05T22:22:23.660897+0000","last_unstale":"2024-04-05T22:22:24.694597+0000","last_undegraded":"2024-04-05T22:22:24.694597+0000","last_fullsized":"2024-04-05T22:22:24.694597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:14.374696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637429+0000","last_change":"2024-04-05T22:22:23.658697+0000","last_active":"2024-04-05T22:22:24.637429+0000","last_peered":"2024-04-05T22:22:24.637429+0000","last_clean":"2024-04-05T22:22:24.637429+0000","last_became_active":"2024-04-05T22:22:23.658228+0000","last_became_peered":"2024-04-05T22:22:23.658228+0000","last_unstale":"2024-04-05T22:22:24.637429+0000","last_undegraded":"2024-04-05T22:22:24.637429+0000","last_fullsized":"2024-04-05T22:22:24.637429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:13.143482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639456+0000","last_change":"2024-04-05T22:22:23.660227+0000","last_active":"2024-04-05T22:22:24.639456+0000","last_peered":"2024-04-05T22:22:24.639456+0000","last_clean":"2024-04-05T22:22:24.639456+0000","last_became_active":"2024-04-05T22:22:23.659896+0000","last_became_peered":"2024-04-05T22:22:23.659896+0000","last_unstale":"2024-04-05T22:22:24.639456+0000","last_undegraded":"2024-04-05T22:22:24.639456+0000","last_fullsized":"2024-04-05T22:22:24.639456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:15:49.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698795+0000","last_change":"2024-04-05T22:22:23.649270+0000","last_active":"2024-04-05T22:22:24.698795+0000","last_peered":"2024-04-05T22:22:24.698795+0000","last_clean":"2024-04-05T22:22:24.698795+0000","last_became_active":"2024-04-05T22:22:23.648854+0000","last_became_peered":"2024-04-05T22:22:23.648854+0000","last_unstale":"2024-04-05T22:22:24.698795+0000","last_undegraded":"2024-04-05T22:22:24.698795+0000","last_fullsized":"2024-04-05T22:22:24.698795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:59.790087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697381+0000","last_change":"2024-04-05T22:22:23.656696+0000","last_active":"2024-04-05T22:22:24.697381+0000","last_peered":"2024-04-05T22:22:24.697381+0000","last_clean":"2024-04-05T22:22:24.697381+0000","last_became_active":"2024-04-05T22:22:23.656289+0000","last_became_peered":"2024-04-05T22:22:23.656289+0000","last_unstale":"2024-04-05T22:22:24.697381+0000","last_undegraded":"2024-04-05T22:22:24.697381+0000","last_fullsized":"2024-04-05T22:22:24.697381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:58.596322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696198+0000","last_change":"2024-04-05T22:22:23.661467+0000","last_active":"2024-04-05T22:22:24.696198+0000","last_peered":"2024-04-05T22:22:24.696198+0000","last_clean":"2024-04-05T22:22:24.696198+0000","last_became_active":"2024-04-05T22:22:23.661164+0000","last_became_peered":"2024-04-05T22:22:23.661164+0000","last_unstale":"2024-04-05T22:22:24.696198+0000","last_undegraded":"2024-04-05T22:22:24.696198+0000","last_fullsized":"2024-04-05T22:22:24.696198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:24.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696451+0000","last_change":"2024-04-05T22:22:23.654718+0000","last_active":"2024-04-05T22:22:24.696451+0000","last_peered":"2024-04-05T22:22:24.696451+0000","last_clean":"2024-04-05T22:22:24.696451+0000","last_became_active":"2024-04-05T22:22:23.654401+0000","last_became_peered":"2024-04-05T22:22:23.654401+0000","last_unstale":"2024-04-05T22:22:24.696451+0000","last_undegraded":"2024-04-05T22:22:24.696451+0000","last_fullsized":"2024-04-05T22:22:24.696451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:00:31.890854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695252+0000","last_change":"2024-04-05T22:22:23.656947+0000","last_active":"2024-04-05T22:22:24.695252+0000","last_peered":"2024-04-05T22:22:24.695252+0000","last_clean":"2024-04-05T22:22:24.695252+0000","last_became_active":"2024-04-05T22:22:23.656762+0000","last_became_peered":"2024-04-05T22:22:23.656762+0000","last_unstale":"2024-04-05T22:22:24.695252+0000","last_undegraded":"2024-04-05T22:22:24.695252+0000","last_fullsized":"2024-04-05T22:22:24.695252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673739+0000","last_change":"2024-04-05T22:22:23.167846+0000","last_active":"2024-04-05T22:22:23.673739+0000","last_peered":"2024-04-05T22:22:23.673739+0000","last_clean":"2024-04-05T22:22:23.673739+0000","last_became_active":"2024-04-05T22:22:23.167499+0000","last_became_peered":"2024-04-05T22:22:23.167499+0000","last_unstale":"2024-04-05T22:22:23.673739+0000","last_undegraded":"2024-04-05T22:22:23.673739+0000","last_fullsized":"2024-04-05T22:22:23.673739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:07:08.680338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696281+0000","last_change":"2024-04-05T22:22:23.665407+0000","last_active":"2024-04-05T22:22:24.696281+0000","last_peered":"2024-04-05T22:22:24.696281+0000","last_clean":"2024-04-05T22:22:24.696281+0000","last_became_active":"2024-04-05T22:22:23.661923+0000","last_became_peered":"2024-04-05T22:22:23.661923+0000","last_unstale":"2024-04-05T22:22:24.696281+0000","last_undegraded":"2024-04-05T22:22:24.696281+0000","last_fullsized":"2024-04-05T22:22:24.696281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:41:16.993667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656516+0000","last_change":"2024-04-05T22:22:23.173935+0000","last_active":"2024-04-05T22:22:23.656516+0000","last_peered":"2024-04-05T22:22:23.656516+0000","last_clean":"2024-04-05T22:22:23.656516+0000","last_became_active":"2024-04-05T22:22:23.173626+0000","last_became_peered":"2024-04-05T22:22:23.173626+0000","last_unstale":"2024-04-05T22:22:23.656516+0000","last_undegraded":"2024-04-05T22:22:23.656516+0000","last_fullsized":"2024-04-05T22:22:23.656516+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:54.000983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699314+0000","last_change":"2024-04-05T22:22:23.645196+0000","last_active":"2024-04-05T22:22:24.699314+0000","last_peered":"2024-04-05T22:22:24.699314+0000","last_clean":"2024-04-05T22:22:24.699314+0000","last_became_active":"2024-04-05T22:22:23.644571+0000","last_became_peered":"2024-04-05T22:22:23.644571+0000","last_unstale":"2024-04-05T22:22:24.699314+0000","last_undegraded":"2024-04-05T22:22:24.699314+0000","last_fullsized":"2024-04-05T22:22:24.699314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:28.057266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.663712+0000","last_change":"2024-04-05T22:22:23.167036+0000","last_active":"2024-04-05T22:22:23.663712+0000","last_peered":"2024-04-05T22:22:23.663712+0000","last_clean":"2024-04-05T22:22:23.663712+0000","last_became_active":"2024-04-05T22:22:23.166675+0000","last_became_peered":"2024-04-05T22:22:23.166675+0000","last_unstale":"2024-04-05T22:22:23.663712+0000","last_undegraded":"2024-04-05T22:22:23.663712+0000","last_fullsized":"2024-04-05T22:22:23.663712+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637547+0000","last_change":"2024-04-05T22:22:23.653873+0000","last_active":"2024-04-05T22:22:24.637547+0000","last_peered":"2024-04-05T22:22:24.637547+0000","last_clean":"2024-04-05T22:22:24.637547+0000","last_became_active":"2024-04-05T22:22:23.653328+0000","last_became_peered":"2024-04-05T22:22:23.653328+0000","last_unstale":"2024-04-05T22:22:24.637547+0000","last_undegraded":"2024-04-05T22:22:24.637547+0000","last_fullsized":"2024-04-05T22:22:24.637547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:19.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.659545+0000","last_change":"2024-04-05T22:22:23.165579+0000","last_active":"2024-04-05T22:22:23.659545+0000","last_peered":"2024-04-05T22:22:23.659545+0000","last_clean":"2024-04-05T22:22:23.659545+0000","last_became_active":"2024-04-05T22:22:23.164825+0000","last_became_peered":"2024-04-05T22:22:23.164825+0000","last_unstale":"2024-04-05T22:22:23.659545+0000","last_undegraded":"2024-04-05T22:22:23.659545+0000","last_fullsized":"2024-04-05T22:22:23.659545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.697512+0000","last_change":"2024-04-05T22:22:23.651254+0000","last_active":"2024-04-05T22:22:24.697512+0000","last_peered":"2024-04-05T22:22:24.697512+0000","last_clean":"2024-04-05T22:22:24.697512+0000","last_became_active":"2024-04-05T22:22:23.650666+0000","last_became_peered":"2024-04-05T22:22:23.650666+0000","last_unstale":"2024-04-05T22:22:24.697512+0000","last_undegraded":"2024-04-05T22:22:24.697512+0000","last_fullsized":"2024-04-05T22:22:24.697512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:54.596096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668306+0000","last_change":"2024-04-05T22:22:23.161130+0000","last_active":"2024-04-05T22:22:23.668306+0000","last_peered":"2024-04-05T22:22:23.668306+0000","last_clean":"2024-04-05T22:22:23.668306+0000","last_became_active":"2024-04-05T22:22:23.160816+0000","last_became_peered":"2024-04-05T22:22:23.160816+0000","last_unstale":"2024-04-05T22:22:23.668306+0000","last_undegraded":"2024-04-05T22:22:23.668306+0000","last_fullsized":"2024-04-05T22:22:23.668306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:19:31.591205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.638992+0000","last_change":"2024-04-05T22:22:23.663030+0000","last_active":"2024-04-05T22:22:24.638992+0000","last_peered":"2024-04-05T22:22:24.638992+0000","last_clean":"2024-04-05T22:22:24.638992+0000","last_became_active":"2024-04-05T22:22:23.662634+0000","last_became_peered":"2024-04-05T22:22:23.662634+0000","last_unstale":"2024-04-05T22:22:24.638992+0000","last_undegraded":"2024-04-05T22:22:24.638992+0000","last_fullsized":"2024-04-05T22:22:24.638992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.669884+0000","last_change":"2024-04-05T22:22:23.171512+0000","last_active":"2024-04-05T22:22:23.669884+0000","last_peered":"2024-04-05T22:22:23.669884+0000","last_clean":"2024-04-05T22:22:23.669884+0000","last_became_active":"2024-04-05T22:22:23.171093+0000","last_became_peered":"2024-04-05T22:22:23.171093+0000","last_unstale":"2024-04-05T22:22:23.669884+0000","last_undegraded":"2024-04-05T22:22:23.669884+0000","last_fullsized":"2024-04-05T22:22:23.669884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:27:39.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696156+0000","last_change":"2024-04-05T22:22:23.662602+0000","last_active":"2024-04-05T22:22:24.696156+0000","last_peered":"2024-04-05T22:22:24.696156+0000","last_clean":"2024-04-05T22:22:24.696156+0000","last_became_active":"2024-04-05T22:22:23.662280+0000","last_became_peered":"2024-04-05T22:22:23.662280+0000","last_unstale":"2024-04-05T22:22:24.696156+0000","last_undegraded":"2024-04-05T22:22:24.696156+0000","last_fullsized":"2024-04-05T22:22:24.696156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627380+0000","last_change":"2024-04-05T22:22:23.161421+0000","last_active":"2024-04-05T22:22:23.627380+0000","last_peered":"2024-04-05T22:22:23.627380+0000","last_clean":"2024-04-05T22:22:23.627380+0000","last_became_active":"2024-04-05T22:22:23.161106+0000","last_became_peered":"2024-04-05T22:22:23.161106+0000","last_unstale":"2024-04-05T22:22:23.627380+0000","last_undegraded":"2024-04-05T22:22:23.627380+0000","last_fullsized":"2024-04-05T22:22:23.627380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636847+0000","last_change":"2024-04-05T22:22:23.665869+0000","last_active":"2024-04-05T22:22:24.636847+0000","last_peered":"2024-04-05T22:22:24.636847+0000","last_clean":"2024-04-05T22:22:24.636847+0000","last_became_active":"2024-04-05T22:22:23.665635+0000","last_became_peered":"2024-04-05T22:22:23.665635+0000","last_unstale":"2024-04-05T22:22:24.636847+0000","last_undegraded":"2024-04-05T22:22:24.636847+0000","last_fullsized":"2024-04-05T22:22:24.636847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:34.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645193+0000","last_change":"2024-04-05T22:22:23.161442+0000","last_active":"2024-04-05T22:22:23.645193+0000","last_peered":"2024-04-05T22:22:23.645193+0000","last_clean":"2024-04-05T22:22:23.645193+0000","last_became_active":"2024-04-05T22:22:23.161144+0000","last_became_peered":"2024-04-05T22:22:23.161144+0000","last_unstale":"2024-04-05T22:22:23.645193+0000","last_undegraded":"2024-04-05T22:22:23.645193+0000","last_fullsized":"2024-04-05T22:22:23.645193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:11.473072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640122+0000","last_change":"2024-04-05T22:22:23.668335+0000","last_active":"2024-04-05T22:22:24.640122+0000","last_peered":"2024-04-05T22:22:24.640122+0000","last_clean":"2024-04-05T22:22:24.640122+0000","last_became_active":"2024-04-05T22:22:23.667779+0000","last_became_peered":"2024-04-05T22:22:23.667779+0000","last_unstale":"2024-04-05T22:22:24.640122+0000","last_undegraded":"2024-04-05T22:22:24.640122+0000","last_fullsized":"2024-04-05T22:22:24.640122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.630023+0000","last_change":"2024-04-05T22:22:23.170597+0000","last_active":"2024-04-05T22:22:23.630023+0000","last_peered":"2024-04-05T22:22:23.630023+0000","last_clean":"2024-04-05T22:22:23.630023+0000","last_became_active":"2024-04-05T22:22:23.170261+0000","last_became_peered":"2024-04-05T22:22:23.170261+0000","last_unstale":"2024-04-05T22:22:23.630023+0000","last_undegraded":"2024-04-05T22:22:23.630023+0000","last_fullsized":"2024-04-05T22:22:23.630023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:18.168591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694569+0000","last_change":"2024-04-05T22:22:23.637101+0000","last_active":"2024-04-05T22:22:24.694569+0000","last_peered":"2024-04-05T22:22:24.694569+0000","last_clean":"2024-04-05T22:22:24.694569+0000","last_became_active":"2024-04-05T22:22:23.636845+0000","last_became_peered":"2024-04-05T22:22:23.636845+0000","last_unstale":"2024-04-05T22:22:24.694569+0000","last_undegraded":"2024-04-05T22:22:24.694569+0000","last_fullsized":"2024-04-05T22:22:24.694569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646846+0000","last_change":"2024-04-05T22:22:23.159233+0000","last_active":"2024-04-05T22:22:23.646846+0000","last_peered":"2024-04-05T22:22:23.646846+0000","last_clean":"2024-04-05T22:22:23.646846+0000","last_became_active":"2024-04-05T22:22:23.158972+0000","last_became_peered":"2024-04-05T22:22:23.158972+0000","last_unstale":"2024-04-05T22:22:23.646846+0000","last_undegraded":"2024-04-05T22:22:23.646846+0000","last_fullsized":"2024-04-05T22:22:23.646846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:58:26.510282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694963+0000","last_change":"2024-04-05T22:22:23.660328+0000","last_active":"2024-04-05T22:22:24.694963+0000","last_peered":"2024-04-05T22:22:24.694963+0000","last_clean":"2024-04-05T22:22:24.694963+0000","last_became_active":"2024-04-05T22:22:23.659665+0000","last_became_peered":"2024-04-05T22:22:23.659665+0000","last_unstale":"2024-04-05T22:22:24.694963+0000","last_undegraded":"2024-04-05T22:22:24.694963+0000","last_fullsized":"2024-04-05T22:22:24.694963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:33:30.562929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.674054+0000","last_change":"2024-04-05T22:22:23.174204+0000","last_active":"2024-04-05T22:22:23.674054+0000","last_peered":"2024-04-05T22:22:23.674054+0000","last_clean":"2024-04-05T22:22:23.674054+0000","last_became_active":"2024-04-05T22:22:23.173693+0000","last_became_peered":"2024-04-05T22:22:23.173693+0000","last_unstale":"2024-04-05T22:22:23.674054+0000","last_undegraded":"2024-04-05T22:22:23.674054+0000","last_fullsized":"2024-04-05T22:22:23.674054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.639858+0000","last_change":"2024-04-05T22:22:23.672617+0000","last_active":"2024-04-05T22:22:24.639858+0000","last_peered":"2024-04-05T22:22:24.639858+0000","last_clean":"2024-04-05T22:22:24.639858+0000","last_became_active":"2024-04-05T22:22:23.670917+0000","last_became_peered":"2024-04-05T22:22:23.670917+0000","last_unstale":"2024-04-05T22:22:24.639858+0000","last_undegraded":"2024-04-05T22:22:24.639858+0000","last_fullsized":"2024-04-05T22:22:24.639858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:51.008587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.673235+0000","last_change":"2024-04-05T22:22:23.163943+0000","last_active":"2024-04-05T22:22:23.673235+0000","last_peered":"2024-04-05T22:22:23.673235+0000","last_clean":"2024-04-05T22:22:23.673235+0000","last_became_active":"2024-04-05T22:22:23.163784+0000","last_became_peered":"2024-04-05T22:22:23.163784+0000","last_unstale":"2024-04-05T22:22:23.673235+0000","last_undegraded":"2024-04-05T22:22:23.673235+0000","last_fullsized":"2024-04-05T22:22:23.673235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.636948+0000","last_change":"2024-04-05T22:22:23.666108+0000","last_active":"2024-04-05T22:22:24.636948+0000","last_peered":"2024-04-05T22:22:24.636948+0000","last_clean":"2024-04-05T22:22:24.636948+0000","last_became_active":"2024-04-05T22:22:23.665839+0000","last_became_peered":"2024-04-05T22:22:23.665839+0000","last_unstale":"2024-04-05T22:22:24.636948+0000","last_undegraded":"2024-04-05T22:22:24.636948+0000","last_fullsized":"2024-04-05T22:22:24.636948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:14.357931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668765+0000","last_change":"2024-04-05T22:22:23.164105+0000","last_active":"2024-04-05T22:22:23.668765+0000","last_peered":"2024-04-05T22:22:23.668765+0000","last_clean":"2024-04-05T22:22:23.668765+0000","last_became_active":"2024-04-05T22:22:23.163812+0000","last_became_peered":"2024-04-05T22:22:23.163812+0000","last_unstale":"2024-04-05T22:22:23.668765+0000","last_undegraded":"2024-04-05T22:22:23.668765+0000","last_fullsized":"2024-04-05T22:22:23.668765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:51.706002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.694350+0000","last_change":"2024-04-05T22:22:23.651615+0000","last_active":"2024-04-05T22:22:24.694350+0000","last_peered":"2024-04-05T22:22:24.694350+0000","last_clean":"2024-04-05T22:22:24.694350+0000","last_became_active":"2024-04-05T22:22:23.651372+0000","last_became_peered":"2024-04-05T22:22:23.651372+0000","last_unstale":"2024-04-05T22:22:24.694350+0000","last_undegraded":"2024-04-05T22:22:24.694350+0000","last_fullsized":"2024-04-05T22:22:24.694350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625635+0000","last_change":"2024-04-05T22:22:23.164204+0000","last_active":"2024-04-05T22:22:23.625635+0000","last_peered":"2024-04-05T22:22:23.625635+0000","last_clean":"2024-04-05T22:22:23.625635+0000","last_became_active":"2024-04-05T22:22:23.163969+0000","last_became_peered":"2024-04-05T22:22:23.163969+0000","last_unstale":"2024-04-05T22:22:23.625635+0000","last_undegraded":"2024-04-05T22:22:23.625635+0000","last_fullsized":"2024-04-05T22:22:23.625635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:10.805035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696698+0000","last_change":"2024-04-05T22:22:23.659000+0000","last_active":"2024-04-05T22:22:24.696698+0000","last_peered":"2024-04-05T22:22:24.696698+0000","last_clean":"2024-04-05T22:22:24.696698+0000","last_became_active":"2024-04-05T22:22:23.655679+0000","last_became_peered":"2024-04-05T22:22:23.655679+0000","last_unstale":"2024-04-05T22:22:24.696698+0000","last_undegraded":"2024-04-05T22:22:24.696698+0000","last_fullsized":"2024-04-05T22:22:24.696698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:09.854704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627490+0000","last_change":"2024-04-05T22:22:23.169398+0000","last_active":"2024-04-05T22:22:23.627490+0000","last_peered":"2024-04-05T22:22:23.627490+0000","last_clean":"2024-04-05T22:22:23.627490+0000","last_became_active":"2024-04-05T22:22:23.168976+0000","last_became_peered":"2024-04-05T22:22:23.168976+0000","last_unstale":"2024-04-05T22:22:23.627490+0000","last_undegraded":"2024-04-05T22:22:23.627490+0000","last_fullsized":"2024-04-05T22:22:23.627490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696514+0000","last_change":"2024-04-05T22:22:23.663801+0000","last_active":"2024-04-05T22:22:24.696514+0000","last_peered":"2024-04-05T22:22:24.696514+0000","last_clean":"2024-04-05T22:22:24.696514+0000","last_became_active":"2024-04-05T22:22:23.663458+0000","last_became_peered":"2024-04-05T22:22:23.663458+0000","last_unstale":"2024-04-05T22:22:24.696514+0000","last_undegraded":"2024-04-05T22:22:24.696514+0000","last_fullsized":"2024-04-05T22:22:24.696514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:57:49.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.668534+0000","last_change":"2024-04-05T22:22:23.169360+0000","last_active":"2024-04-05T22:22:23.668534+0000","last_peered":"2024-04-05T22:22:23.668534+0000","last_clean":"2024-04-05T22:22:23.668534+0000","last_became_active":"2024-04-05T22:22:23.167009+0000","last_became_peered":"2024-04-05T22:22:23.167009+0000","last_unstale":"2024-04-05T22:22:23.668534+0000","last_undegraded":"2024-04-05T22:22:23.668534+0000","last_fullsized":"2024-04-05T22:22:23.668534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.699135+0000","last_change":"2024-04-05T22:22:23.668712+0000","last_active":"2024-04-05T22:22:24.699135+0000","last_peered":"2024-04-05T22:22:24.699135+0000","last_clean":"2024-04-05T22:22:24.699135+0000","last_became_active":"2024-04-05T22:22:23.668435+0000","last_became_peered":"2024-04-05T22:22:23.668435+0000","last_unstale":"2024-04-05T22:22:24.699135+0000","last_undegraded":"2024-04-05T22:22:24.699135+0000","last_fullsized":"2024-04-05T22:22:24.699135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667960+0000","last_change":"2024-04-05T22:22:23.164411+0000","last_active":"2024-04-05T22:22:23.667960+0000","last_peered":"2024-04-05T22:22:23.667960+0000","last_clean":"2024-04-05T22:22:23.667960+0000","last_became_active":"2024-04-05T22:22:23.162250+0000","last_became_peered":"2024-04-05T22:22:23.162250+0000","last_unstale":"2024-04-05T22:22:23.667960+0000","last_undegraded":"2024-04-05T22:22:23.667960+0000","last_fullsized":"2024-04-05T22:22:23.667960+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640160+0000","last_change":"2024-04-05T22:22:23.673810+0000","last_active":"2024-04-05T22:22:24.640160+0000","last_peered":"2024-04-05T22:22:24.640160+0000","last_clean":"2024-04-05T22:22:24.640160+0000","last_became_active":"2024-04-05T22:22:23.673503+0000","last_became_peered":"2024-04-05T22:22:23.673503+0000","last_unstale":"2024-04-05T22:22:24.640160+0000","last_undegraded":"2024-04-05T22:22:24.640160+0000","last_fullsized":"2024-04-05T22:22:24.640160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:45.264144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.670057+0000","last_change":"2024-04-05T22:22:23.156737+0000","last_active":"2024-04-05T22:22:23.670057+0000","last_peered":"2024-04-05T22:22:23.670057+0000","last_clean":"2024-04-05T22:22:23.670057+0000","last_became_active":"2024-04-05T22:22:23.156286+0000","last_became_peered":"2024-04-05T22:22:23.156286+0000","last_unstale":"2024-04-05T22:22:23.670057+0000","last_undegraded":"2024-04-05T22:22:23.670057+0000","last_fullsized":"2024-04-05T22:22:23.670057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:30.414736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.637263+0000","last_change":"2024-04-05T22:22:23.656662+0000","last_active":"2024-04-05T22:22:24.637263+0000","last_peered":"2024-04-05T22:22:24.637263+0000","last_clean":"2024-04-05T22:22:24.637263+0000","last_became_active":"2024-04-05T22:22:23.656406+0000","last_became_peered":"2024-04-05T22:22:23.656406+0000","last_unstale":"2024-04-05T22:22:24.637263+0000","last_undegraded":"2024-04-05T22:22:24.637263+0000","last_fullsized":"2024-04-05T22:22:24.637263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.667743+0000","last_change":"2024-04-05T22:22:23.158044+0000","last_active":"2024-04-05T22:22:23.667743+0000","last_peered":"2024-04-05T22:22:23.667743+0000","last_clean":"2024-04-05T22:22:23.667743+0000","last_became_active":"2024-04-05T22:22:23.157699+0000","last_became_peered":"2024-04-05T22:22:23.157699+0000","last_unstale":"2024-04-05T22:22:23.667743+0000","last_undegraded":"2024-04-05T22:22:23.667743+0000","last_fullsized":"2024-04-05T22:22:23.667743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:58:42.971651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635512+0000","last_change":"2024-04-05T22:22:23.650874+0000","last_active":"2024-04-05T22:22:24.635512+0000","last_peered":"2024-04-05T22:22:24.635512+0000","last_clean":"2024-04-05T22:22:24.635512+0000","last_became_active":"2024-04-05T22:22:23.650617+0000","last_became_peered":"2024-04-05T22:22:23.650617+0000","last_unstale":"2024-04-05T22:22:24.635512+0000","last_undegraded":"2024-04-05T22:22:24.635512+0000","last_fullsized":"2024-04-05T22:22:24.635512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:11.819970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.642139+0000","last_change":"2024-04-05T22:22:23.171641+0000","last_active":"2024-04-05T22:22:23.642139+0000","last_peered":"2024-04-05T22:22:23.642139+0000","last_clean":"2024-04-05T22:22:23.642139+0000","last_became_active":"2024-04-05T22:22:23.171224+0000","last_became_peered":"2024-04-05T22:22:23.171224+0000","last_unstale":"2024-04-05T22:22:23.642139+0000","last_undegraded":"2024-04-05T22:22:23.642139+0000","last_fullsized":"2024-04-05T22:22:23.642139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:26.099736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635186+0000","last_change":"2024-04-05T22:22:23.649481+0000","last_active":"2024-04-05T22:22:24.635186+0000","last_peered":"2024-04-05T22:22:24.635186+0000","last_clean":"2024-04-05T22:22:24.635186+0000","last_became_active":"2024-04-05T22:22:23.649158+0000","last_became_peered":"2024-04-05T22:22:23.649158+0000","last_unstale":"2024-04-05T22:22:24.635186+0000","last_undegraded":"2024-04-05T22:22:24.635186+0000","last_fullsized":"2024-04-05T22:22:24.635186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:55.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.661505+0000","last_change":"2024-04-05T22:22:23.153928+0000","last_active":"2024-04-05T22:22:23.661505+0000","last_peered":"2024-04-05T22:22:23.661505+0000","last_clean":"2024-04-05T22:22:23.661505+0000","last_became_active":"2024-04-05T22:22:23.153301+0000","last_became_peered":"2024-04-05T22:22:23.153301+0000","last_unstale":"2024-04-05T22:22:23.661505+0000","last_undegraded":"2024-04-05T22:22:23.661505+0000","last_fullsized":"2024-04-05T22:22:23.661505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:23.927189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635294+0000","last_change":"2024-04-05T22:22:23.656526+0000","last_active":"2024-04-05T22:22:24.635294+0000","last_peered":"2024-04-05T22:22:24.635294+0000","last_clean":"2024-04-05T22:22:24.635294+0000","last_became_active":"2024-04-05T22:22:23.655430+0000","last_became_peered":"2024-04-05T22:22:23.655430+0000","last_unstale":"2024-04-05T22:22:24.635294+0000","last_undegraded":"2024-04-05T22:22:24.635294+0000","last_fullsized":"2024-04-05T22:22:24.635294+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:24.841919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.648674+0000","last_change":"2024-04-05T22:22:23.164487+0000","last_active":"2024-04-05T22:22:23.648674+0000","last_peered":"2024-04-05T22:22:23.648674+0000","last_clean":"2024-04-05T22:22:23.648674+0000","last_became_active":"2024-04-05T22:22:23.163704+0000","last_became_peered":"2024-04-05T22:22:23.163704+0000","last_unstale":"2024-04-05T22:22:23.648674+0000","last_undegraded":"2024-04-05T22:22:23.648674+0000","last_fullsized":"2024-04-05T22:22:23.648674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:27:25.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635044+0000","last_change":"2024-04-05T22:22:23.651556+0000","last_active":"2024-04-05T22:22:24.635044+0000","last_peered":"2024-04-05T22:22:24.635044+0000","last_clean":"2024-04-05T22:22:24.635044+0000","last_became_active":"2024-04-05T22:22:23.651314+0000","last_became_peered":"2024-04-05T22:22:23.651314+0000","last_unstale":"2024-04-05T22:22:24.635044+0000","last_undegraded":"2024-04-05T22:22:24.635044+0000","last_fullsized":"2024-04-05T22:22:24.635044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:33:02.668787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636186+0000","last_change":"2024-04-05T22:22:23.155981+0000","last_active":"2024-04-05T22:22:23.636186+0000","last_peered":"2024-04-05T22:22:23.636186+0000","last_clean":"2024-04-05T22:22:23.636186+0000","last_became_active":"2024-04-05T22:22:23.154363+0000","last_became_peered":"2024-04-05T22:22:23.154363+0000","last_unstale":"2024-04-05T22:22:23.636186+0000","last_undegraded":"2024-04-05T22:22:23.636186+0000","last_fullsized":"2024-04-05T22:22:23.636186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:13.978598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696889+0000","last_change":"2024-04-05T22:22:23.657509+0000","last_active":"2024-04-05T22:22:24.696889+0000","last_peered":"2024-04-05T22:22:24.696889+0000","last_clean":"2024-04-05T22:22:24.696889+0000","last_became_active":"2024-04-05T22:22:23.657035+0000","last_became_peered":"2024-04-05T22:22:23.657035+0000","last_unstale":"2024-04-05T22:22:24.696889+0000","last_undegraded":"2024-04-05T22:22:24.696889+0000","last_fullsized":"2024-04-05T22:22:24.696889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:42:27.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.656814+0000","last_change":"2024-04-05T22:22:23.172774+0000","last_active":"2024-04-05T22:22:23.656814+0000","last_peered":"2024-04-05T22:22:23.656814+0000","last_clean":"2024-04-05T22:22:23.656814+0000","last_became_active":"2024-04-05T22:22:23.172492+0000","last_became_peered":"2024-04-05T22:22:23.172492+0000","last_unstale":"2024-04-05T22:22:23.656814+0000","last_undegraded":"2024-04-05T22:22:23.656814+0000","last_fullsized":"2024-04-05T22:22:23.656814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:27.042331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.635485+0000","last_change":"2024-04-05T22:22:23.667039+0000","last_active":"2024-04-05T22:22:24.635485+0000","last_peered":"2024-04-05T22:22:24.635485+0000","last_clean":"2024-04-05T22:22:24.635485+0000","last_became_active":"2024-04-05T22:22:23.666665+0000","last_became_peered":"2024-04-05T22:22:23.666665+0000","last_unstale":"2024-04-05T22:22:24.635485+0000","last_undegraded":"2024-04-05T22:22:24.635485+0000","last_fullsized":"2024-04-05T22:22:24.635485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:50.267711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.665989+0000","last_change":"2024-04-05T22:22:23.165988+0000","last_active":"2024-04-05T22:22:23.665989+0000","last_peered":"2024-04-05T22:22:23.665989+0000","last_clean":"2024-04-05T22:22:23.665989+0000","last_became_active":"2024-04-05T22:22:23.165708+0000","last_became_peered":"2024-04-05T22:22:23.165708+0000","last_unstale":"2024-04-05T22:22:23.665989+0000","last_undegraded":"2024-04-05T22:22:23.665989+0000","last_fullsized":"2024-04-05T22:22:23.665989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:40:30.394470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.640236+0000","last_change":"2024-04-05T22:22:23.664010+0000","last_active":"2024-04-05T22:22:24.640236+0000","last_peered":"2024-04-05T22:22:24.640236+0000","last_clean":"2024-04-05T22:22:24.640236+0000","last_became_active":"2024-04-05T22:22:23.663608+0000","last_became_peered":"2024-04-05T22:22:23.663608+0000","last_unstale":"2024-04-05T22:22:24.640236+0000","last_undegraded":"2024-04-05T22:22:24.640236+0000","last_fullsized":"2024-04-05T22:22:24.640236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:00:09.963924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695548+0000","last_change":"2024-04-05T22:22:23.154936+0000","last_active":"2024-04-05T22:22:24.695548+0000","last_peered":"2024-04-05T22:22:24.695548+0000","last_clean":"2024-04-05T22:22:24.695548+0000","last_became_active":"2024-04-05T22:22:23.154664+0000","last_became_peered":"2024-04-05T22:22:23.154664+0000","last_unstale":"2024-04-05T22:22:24.695548+0000","last_undegraded":"2024-04-05T22:22:24.695548+0000","last_fullsized":"2024-04-05T22:22:24.695548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:44:23.826984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695278+0000","last_change":"2024-04-05T22:22:23.649531+0000","last_active":"2024-04-05T22:22:24.695278+0000","last_peered":"2024-04-05T22:22:24.695278+0000","last_clean":"2024-04-05T22:22:24.695278+0000","last_became_active":"2024-04-05T22:22:23.649241+0000","last_became_peered":"2024-04-05T22:22:23.649241+0000","last_unstale":"2024-04-05T22:22:24.695278+0000","last_undegraded":"2024-04-05T22:22:24.695278+0000","last_fullsized":"2024-04-05T22:22:24.695278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:19.183526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.626116+0000","last_change":"2024-04-05T22:22:23.160628+0000","last_active":"2024-04-05T22:22:23.626116+0000","last_peered":"2024-04-05T22:22:23.626116+0000","last_clean":"2024-04-05T22:22:23.626116+0000","last_became_active":"2024-04-05T22:22:23.159150+0000","last_became_peered":"2024-04-05T22:22:23.159150+0000","last_unstale":"2024-04-05T22:22:23.626116+0000","last_undegraded":"2024-04-05T22:22:23.626116+0000","last_fullsized":"2024-04-05T22:22:23.626116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:49.252776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.696531+0000","last_change":"2024-04-05T22:22:23.666678+0000","last_active":"2024-04-05T22:22:24.696531+0000","last_peered":"2024-04-05T22:22:24.696531+0000","last_clean":"2024-04-05T22:22:24.696531+0000","last_became_active":"2024-04-05T22:22:23.665881+0000","last_became_peered":"2024-04-05T22:22:23.665881+0000","last_unstale":"2024-04-05T22:22:24.696531+0000","last_undegraded":"2024-04-05T22:22:24.696531+0000","last_fullsized":"2024-04-05T22:22:24.696531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:36.143521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.646709+0000","last_change":"2024-04-05T22:22:23.166020+0000","last_active":"2024-04-05T22:22:23.646709+0000","last_peered":"2024-04-05T22:22:23.646709+0000","last_clean":"2024-04-05T22:22:23.646709+0000","last_became_active":"2024-04-05T22:22:23.165719+0000","last_became_peered":"2024-04-05T22:22:23.165719+0000","last_unstale":"2024-04-05T22:22:23.646709+0000","last_undegraded":"2024-04-05T22:22:23.646709+0000","last_fullsized":"2024-04-05T22:22:23.646709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:58.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.695379+0000","last_change":"2024-04-05T22:22:23.666126+0000","last_active":"2024-04-05T22:22:24.695379+0000","last_peered":"2024-04-05T22:22:24.695379+0000","last_clean":"2024-04-05T22:22:24.695379+0000","last_became_active":"2024-04-05T22:22:23.665889+0000","last_became_peered":"2024-04-05T22:22:23.665889+0000","last_unstale":"2024-04-05T22:22:24.695379+0000","last_undegraded":"2024-04-05T22:22:24.695379+0000","last_fullsized":"2024-04-05T22:22:24.695379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:55:14.548181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.625723+0000","last_change":"2024-04-05T22:22:23.166747+0000","last_active":"2024-04-05T22:22:23.625723+0000","last_peered":"2024-04-05T22:22:23.625723+0000","last_clean":"2024-04-05T22:22:23.625723+0000","last_became_active":"2024-04-05T22:22:23.166495+0000","last_became_peered":"2024-04-05T22:22:23.166495+0000","last_unstale":"2024-04-05T22:22:23.625723+0000","last_undegraded":"2024-04-05T22:22:23.625723+0000","last_fullsized":"2024-04-05T22:22:23.625723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.693701+0000","last_change":"2024-04-05T22:22:23.634863+0000","last_active":"2024-04-05T22:22:24.693701+0000","last_peered":"2024-04-05T22:22:24.693701+0000","last_clean":"2024-04-05T22:22:24.693701+0000","last_became_active":"2024-04-05T22:22:23.634615+0000","last_became_peered":"2024-04-05T22:22:23.634615+0000","last_unstale":"2024-04-05T22:22:24.693701+0000","last_undegraded":"2024-04-05T22:22:24.693701+0000","last_fullsized":"2024-04-05T22:22:24.693701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:32:16.541566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.636456+0000","last_change":"2024-04-05T22:22:23.168895+0000","last_active":"2024-04-05T22:22:23.636456+0000","last_peered":"2024-04-05T22:22:23.636456+0000","last_clean":"2024-04-05T22:22:23.636456+0000","last_became_active":"2024-04-05T22:22:23.168395+0000","last_became_peered":"2024-04-05T22:22:23.168395+0000","last_unstale":"2024-04-05T22:22:23.636456+0000","last_undegraded":"2024-04-05T22:22:23.636456+0000","last_fullsized":"2024-04-05T22:22:23.636456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:14:22.438726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698300+0000","last_change":"2024-04-05T22:22:23.654099+0000","last_active":"2024-04-05T22:22:24.698300+0000","last_peered":"2024-04-05T22:22:24.698300+0000","last_clean":"2024-04-05T22:22:24.698300+0000","last_became_active":"2024-04-05T22:22:23.653171+0000","last_became_peered":"2024-04-05T22:22:23.653171+0000","last_unstale":"2024-04-05T22:22:24.698300+0000","last_undegraded":"2024-04-05T22:22:24.698300+0000","last_fullsized":"2024-04-05T22:22:24.698300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:24.942124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.627663+0000","last_change":"2024-04-05T22:22:23.173853+0000","last_active":"2024-04-05T22:22:23.627663+0000","last_peered":"2024-04-05T22:22:23.627663+0000","last_clean":"2024-04-05T22:22:23.627663+0000","last_became_active":"2024-04-05T22:22:23.173561+0000","last_became_peered":"2024-04-05T22:22:23.173561+0000","last_unstale":"2024-04-05T22:22:23.627663+0000","last_undegraded":"2024-04-05T22:22:23.627663+0000","last_fullsized":"2024-04-05T22:22:23.627663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:22.984277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-05T22:22:24.698265+0000","last_change":"2024-04-05T22:22:23.649429+0000","last_active":"2024-04-05T22:22:24.698265+0000","last_peered":"2024-04-05T22:22:24.698265+0000","last_clean":"2024-04-05T22:22:24.698265+0000","last_became_active":"2024-04-05T22:22:23.649091+0000","last_became_peered":"2024-04-05T22:22:23.649091+0000","last_unstale":"2024-04-05T22:22:24.698265+0000","last_undegraded":"2024-04-05T22:22:24.698265+0000","last_fullsized":"2024-04-05T22:22:24.698265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:23.114445+0000","last_clean_scrub_stamp":"2024-04-05T22:22:23.114445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:51:19.346378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.645441+0000","last_change":"2024-04-05T22:22:23.162280+0000","last_active":"2024-04-05T22:22:23.645441+0000","last_peered":"2024-04-05T22:22:23.645441+0000","last_clean":"2024-04-05T22:22:23.645441+0000","last_became_active":"2024-04-05T22:22:23.161975+0000","last_became_peered":"2024-04-05T22:22:23.161975+0000","last_unstale":"2024-04-05T22:22:23.645441+0000","last_undegraded":"2024-04-05T22:22:23.645441+0000","last_fullsized":"2024-04-05T22:22:23.645441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:22:22.099510+0000","last_clean_scrub_stamp":"2024-04-05T22:22:22.099510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:24:22.511159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-05T22:22:23.631303+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:22:23.631303+0000","last_peered":"2024-04-05T22:22:23.631303+0000","last_clean":"2024-04-05T22:22:23.631303+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:22:23.631303+0000","last_undegraded":"2024-04-05T22:22:23.631303+0000","last_fullsized":"2024-04-05T22:22:23.631303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":1003520,"data_stored":750473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.577},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.073}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.179},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.038}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.329}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.274},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.7010000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.534}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.829}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":413696,"data_stored":160631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.579},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.242}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.524},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.7290000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.776},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.641}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.165},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.2809999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.26}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.159}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.359}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":413696,"data_stored":160119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.196},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.361},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.232}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.3979999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.075}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.159},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.425}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.137},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":160085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.621},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.999}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.264}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.83699999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.875}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.419},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.534}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.841},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.3320000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":2.0619999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.663}]}]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":750383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.747},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.98699999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.101},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:22:33.412 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:22:33.412 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:22:33.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.663+0000 7faea7200640 1 Processor -- start 2024-04-05T22:22:33.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.663+0000 7faea7200640 1 -- start start 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 0x7faea80b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faea80cd740 0x7faea8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea7200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea81577f0 con 0x7faea815af80 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea80b1e90 con 0x7faea80cd740 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80cb630 con 0x7faea8157410 2024-04-05T22:22:33.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7fae9fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faea815af80 0x7faea80cd030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52160/0 (socket says 172.21.15.73:52160) 2024-04-05T22:22:33.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7fae9fe00640 1 -- 172.21.15.73:0/2282524281 learned_addr learned my addr 172.21.15.73:0/2282524281 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.665+0000 7faea5e00640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 0x7faea80b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> v2:172.21.15.157:3300/0 conn(0x7faea80cd740 0x7faea8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661781348 0 0) 0x7faea81577f0 con 0x7faea815af80 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae8c003520 con 0x7faea815af80 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500776596 0 0) 0x7fae8c003520 con 0x7faea815af80 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 msgr2=0x7faea80b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 0x7faea80b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 >> v2:172.21.15.157:3300/0 conn(0x7faea80cd740 msgr2=0x7faea8159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 --2- 172.21.15.73:0/2282524281 >> v2:172.21.15.157:3300/0 conn(0x7faea80cd740 0x7faea8159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80b20a0 con 0x7faea815af80 2024-04-05T22:22:33.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fae90001670 con 0x7faea815af80 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 >> v1:172.21.15.73:6789/0 conn(0x7faea815af80 legacy=0x7faea80cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae8c003970 0x7fae8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 0x7fae8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae8c004d00 0x7fae8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80b1e90 con 0x7fae8c004d00 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80cb630 con 0x7fae8c003970 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c00b980 con 0x7faea8157410 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.666+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> v2:172.21.15.157:3300/0 conn(0x7faea80cd740 0x7faea8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae8c004d00 0x7fae8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 -- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 msgr2=0x7fae8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea8157410 0x7fae8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 -- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae8c003970 msgr2=0x7fae8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae8c003970 0x7fae8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.667+0000 7faea6800640 1 -- 172.21.15.73:0/2282524281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c003520 con 0x7fae8c004d00 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.668+0000 7faea6800640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae8c004d00 0x7fae8c005180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faea0007410 tx=0x7fae8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261141d255ac270c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea002cdc0 con 0x7fae8c004d00 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0034440 con 0x7fae8c004d00 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea5400640 1 -- 172.21.15.73:0/2282524281 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea00347b0 con 0x7fae8c004d00 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 -- 172.21.15.73:0/2282524281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae8c004d00 msgr2=0x7fae8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 --2- 172.21.15.73:0/2282524281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae8c004d00 0x7fae8c005180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faea0007410 tx=0x7fae8c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea6800640 1 -- 172.21.15.73:0/2282524281 reap_dead start 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 -- 172.21.15.73:0/2282524281 shutdown_connections 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 -- 172.21.15.73:0/2282524281 >> 172.21.15.73:0/2282524281 conn(0x7faea80b0830 msgr2=0x7faea80b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 -- 172.21.15.73:0/2282524281 shutdown_connections 2024-04-05T22:22:33.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.669+0000 7faea7200640 1 -- 172.21.15.73:0/2282524281 wait complete. 2024-04-05T22:22:33.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.671+0000 7faea7200640 1 Processor -- start 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.671+0000 7faea7200640 1 -- start start 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 0x7faea81f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea81433f0 0x7faea8142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c9b40 con 0x7faea814c1e0 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c8f30 con 0x7faea814be00 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.672+0000 7faea7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c9210 con 0x7faea81433f0 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46608/0 (socket says 172.21.15.73:46608) 2024-04-05T22:22:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 -- 172.21.15.73:0/3116713927 learned_addr learned my addr 172.21.15.73:0/3116713927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7faea5e00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 0x7faea81f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 -- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea81433f0 msgr2=0x7faea8142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea81433f0 0x7faea8142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 -- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 msgr2=0x7faea81f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 0x7faea81f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 -- 172.21.15.73:0/3116713927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80c94f0 con 0x7faea814c1e0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7faea5e00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 0x7faea81f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.673+0000 7fae9fe00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fae9000ca20 tx=0x7fae9000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.674+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90004aa0 con 0x7faea814c1e0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.674+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90004c00 con 0x7faea814c1e0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.674+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae900070d0 con 0x7faea814c1e0 2024-04-05T22:22:33.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.674+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea80c97d0 con 0x7faea814c1e0 2024-04-05T22:22:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.674+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea81f28b0 con 0x7faea814c1e0 2024-04-05T22:22:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.675+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea80c9cb0 con 0x7faea814c1e0 2024-04-05T22:22:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.676+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae90007230 con 0x7faea814c1e0 2024-04-05T22:22:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.677+0000 7fae9e000640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 0x7fae8406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.677+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae90088780 con 0x7faea814c1e0 2024-04-05T22:22:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.677+0000 7faea5e00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 0x7fae8406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:33.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.678+0000 7faea5e00640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 0x7fae8406d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae98004770 tx=0x7fae98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:33.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.682+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae900538c0 con 0x7faea814c1e0 2024-04-05T22:22:33.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.948+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faea804e160 con 0x7faea814c1e0 2024-04-05T22:22:33.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.956+0000 7fae9e000640 1 -- 172.21.15.73:0/3116713927 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fae900180e0 con 0x7faea814c1e0 2024-04-05T22:22:33.956 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.958+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 msgr2=0x7fae8406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.958+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 0x7fae8406d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae98004770 tx=0x7fae98008040 comp rx=0 tx=0).stop 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.958+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 msgr2=0x7faea8142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.958+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fae9000ca20 tx=0x7fae9000cee0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 shutdown_connections 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae8406ac00 0x7fae8406d0c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea81433f0 0x7faea8142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea814c1e0 0x7faea8142eb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 --2- 172.21.15.73:0/3116713927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea814be00 0x7faea81f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 >> 172.21.15.73:0/3116713927 conn(0x7faea80b0830 msgr2=0x7faea8156640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 shutdown_connections 2024-04-05T22:22:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:33.959+0000 7faea7200640 1 -- 172.21.15.73:0/3116713927 wait complete. 2024-04-05T22:22:33.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:34.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.178+0000 7f6c7a400640 1 Processor -- start 2024-04-05T22:22:34.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.179+0000 7f6c7a400640 1 -- start start 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.180+0000 7f6c7a400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6c741587d0 0x7f6c740cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.180+0000 7f6c7a400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7406f5b0 con 0x7f6c740ccd20 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.180+0000 7f6c7a400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c74070410 con 0x7f6c7415e850 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.180+0000 7f6c7a400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c74158bb0 con 0x7f6c741587d0 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6c741587d0 0x7f6c740cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6c741587d0 0x7f6c740cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46618/0 (socket says 172.21.15.73:46618) 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 -- 172.21.15.73:0/1313692389 learned_addr learned my addr 172.21.15.73:0/1313692389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c72a00640 1 --1- 172.21.15.73:0/1313692389 >> v1:172.21.15.73:6789/0 conn(0x7f6c740ccd20 0x7f6c7406cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52168/0 (socket says 172.21.15.73:52168) 2024-04-05T22:22:34.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 -- 172.21.15.73:0/1313692389 >> v1:172.21.15.157:6789/0 conn(0x7f6c7415e850 legacy=0x7f6c740708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:34.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 -- 172.21.15.73:0/1313692389 >> v1:172.21.15.73:6789/0 conn(0x7f6c740ccd20 legacy=0x7f6c7406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c73400640 1 -- 172.21.15.73:0/1313692389 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7406f800 con 0x7f6c741587d0 2024-04-05T22:22:34.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.181+0000 7f6c72000640 1 -- 172.21.15.73:0/1313692389 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409846871 0 0) 0x7f6c7406f5b0 con 0x7f6c740ccd20 2024-04-05T22:22:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.182+0000 7f6c73400640 1 --2- 172.21.15.73:0/1313692389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c741587d0 0x7f6c740cc7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329033e7683f2e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.182+0000 7f6c72000640 1 -- 172.21.15.73:0/1313692389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64030840 con 0x7f6c741587d0 2024-04-05T22:22:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c72000640 1 -- 172.21.15.73:0/1313692389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c640309a0 con 0x7f6c741587d0 2024-04-05T22:22:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c72000640 1 -- 172.21.15.73:0/1313692389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64030d30 con 0x7f6c741587d0 2024-04-05T22:22:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 -- 172.21.15.73:0/1313692389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c741587d0 msgr2=0x7f6c740cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 --2- 172.21.15.73:0/1313692389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c741587d0 0x7f6c740cc7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c6402b980 comp rx=0 tx=0).stop 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 -- 172.21.15.73:0/1313692389 shutdown_connections 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 --2- 172.21.15.73:0/1313692389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c741587d0 0x7f6c740cc7e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 -- 172.21.15.73:0/1313692389 >> 172.21.15.73:0/1313692389 conn(0x7f6c740ce8e0 msgr2=0x7f6c740cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 -- 172.21.15.73:0/1313692389 shutdown_connections 2024-04-05T22:22:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.183+0000 7f6c7a400640 1 -- 172.21.15.73:0/1313692389 wait complete. 2024-04-05T22:22:34.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.185+0000 7f6c7a400640 1 Processor -- start 2024-04-05T22:22:34.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.185+0000 7f6c7a400640 1 -- start start 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c741587d0 0x7f6c741f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 0x7f6c740ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74149b70 con 0x7f6c7415e850 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74149e10 con 0x7f6c741587d0 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c7a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7414a0f0 con 0x7f6c741460e0 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46624/0 (socket says 172.21.15.73:46624) 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.186+0000 7f6c72a00640 1 -- 172.21.15.73:0/2507299055 learned_addr learned my addr 172.21.15.73:0/2507299055 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c73400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c741587d0 0x7f6c741f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:34.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c73e00640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 0x7f6c740ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:34.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 -- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 msgr2=0x7f6c740ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 0x7f6c740ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 -- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c741587d0 msgr2=0x7f6c741f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c741587d0 0x7f6c741f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 -- 172.21.15.73:0/2507299055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640094d0 con 0x7f6c7415e850 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c73e00640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 0x7f6c740ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c72a00640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c4d0 tx=0x7f6c6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600074e0 con 0x7f6c7415e850 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c60007660 con 0x7f6c7415e850 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004a20 con 0x7f6c7415e850 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.187+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7414a3d0 con 0x7f6c7415e850 2024-04-05T22:22:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.188+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c740acd70 con 0x7f6c7415e850 2024-04-05T22:22:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.189+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c74066af0 con 0x7f6c7415e850 2024-04-05T22:22:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.190+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6c60004b80 con 0x7f6c7415e850 2024-04-05T22:22:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.190+0000 7f6c6b400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 0x7f6c5406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.191+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c60088540 con 0x7f6c7415e850 2024-04-05T22:22:34.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.191+0000 7f6c73400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 0x7f6c5406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:34.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.192+0000 7f6c73400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 0x7f6c5406d0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c64036000 tx=0x7f6c64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.195+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6c60053720 con 0x7f6c7415e850 2024-04-05T22:22:34.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.424+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c74068160 con 0x7f6c7415e850 2024-04-05T22:22:34.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.425+0000 7f6c6b400640 1 -- 172.21.15.73:0/2507299055 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f6c60056d40 con 0x7f6c7415e850 2024-04-05T22:22:34.426 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:34.426 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 15 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 msgr2=0x7f6c5406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 0x7f6c5406d0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c64036000 tx=0x7f6c64008030 comp rx=0 tx=0).stop 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 msgr2=0x7f6c741f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c4d0 tx=0x7f6c6000c990 comp rx=0 tx=0).stop 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 shutdown_connections 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6c5406ac00 0x7f6c5406d0c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c741460e0 0x7f6c740ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c7415e850 0x7f6c741f4140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 --2- 172.21.15.73:0/2507299055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c741587d0 0x7f6c741f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 >> 172.21.15.73:0/2507299055 conn(0x7f6c740ce8e0 msgr2=0x7f6c7415d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 shutdown_connections 2024-04-05T22:22:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:34.428+0000 7f6c7a400640 1 -- 172.21.15.73:0/2507299055 wait complete. 2024-04-05T22:22:34.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:22:23.615118+0000', 'modified': '2024-04-05T22:22:26.635166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5115': {'gid': 5115, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6836/2312639511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2312639511}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 2312639511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:22:34.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:22:34.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-05T22:22:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.053+0000 7efe8a200640 1 Processor -- start 2024-04-05T22:22:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.054+0000 7efe8a200640 1 -- start start 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.055+0000 7efe8a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe8406cb80 0x7efe8406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.055+0000 7efe8a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe8415e850 0x7efe8415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.055+0000 7efe8a200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8406f5b0 con 0x7efe841587d0 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.055+0000 7efe8a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8406d500 con 0x7efe8406cb80 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.055+0000 7efe8a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8415f220 con 0x7efe8415e850 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe8406cb80 0x7efe8406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe8406cb80 0x7efe8406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46640/0 (socket says 172.21.15.73:46640) 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 -- 172.21.15.73:0/2869687073 learned_addr learned my addr 172.21.15.73:0/2869687073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 -- 172.21.15.73:0/2869687073 >> v2:172.21.15.157:3300/0 conn(0x7efe8415e850 msgr2=0x7efe8415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 --2- 172.21.15.73:0/2869687073 >> v2:172.21.15.157:3300/0 conn(0x7efe8415e850 0x7efe8415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 -- 172.21.15.73:0/2869687073 >> v1:172.21.15.157:6789/0 conn(0x7efe841587d0 legacy=0x7efe840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.056+0000 7efe82a00640 1 -- 172.21.15.73:0/2869687073 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84070410 con 0x7efe8406cb80 2024-04-05T22:22:35.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.057+0000 7efe82a00640 1 --2- 172.21.15.73:0/2869687073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe8406cfc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efe8415c270 tx=0x7efe7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a9c2d561c75327 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.057+0000 7efe82000640 1 -- 172.21.15.73:0/2869687073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70031870 con 0x7efe8406cb80 2024-04-05T22:22:35.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe82000640 1 -- 172.21.15.73:0/2869687073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700319d0 con 0x7efe8406cb80 2024-04-05T22:22:35.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe82000640 1 -- 172.21.15.73:0/2869687073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70031d60 con 0x7efe8406cb80 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 -- 172.21.15.73:0/2869687073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 msgr2=0x7efe8406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 --2- 172.21.15.73:0/2869687073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe8406cfc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efe8415c270 tx=0x7efe7002b910 comp rx=0 tx=0).stop 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 -- 172.21.15.73:0/2869687073 shutdown_connections 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 --2- 172.21.15.73:0/2869687073 >> v2:172.21.15.157:3300/0 conn(0x7efe8415e850 0x7efe8415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 --2- 172.21.15.73:0/2869687073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe8406cfc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 -- 172.21.15.73:0/2869687073 >> 172.21.15.73:0/2869687073 conn(0x7efe840ce8e0 msgr2=0x7efe840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 -- 172.21.15.73:0/2869687073 shutdown_connections 2024-04-05T22:22:35.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.058+0000 7efe8a200640 1 -- 172.21.15.73:0/2869687073 wait complete. 2024-04-05T22:22:35.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.060+0000 7efe8a200640 1 Processor -- start 2024-04-05T22:22:35.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.060+0000 7efe8a200640 1 -- start start 2024-04-05T22:22:35.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe841587d0 0x7efe841f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe8415e850 0x7efe841f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841e5e60 con 0x7efe8406cb80 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841e6140 con 0x7efe841587d0 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe8a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe841e6420 con 0x7efe8415e850 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46652/0 (socket says 172.21.15.73:46652) 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.061+0000 7efe83400640 1 -- 172.21.15.73:0/3944999662 learned_addr learned my addr 172.21.15.73:0/3944999662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83e00640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe8415e850 0x7efe841f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 -- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe8415e850 msgr2=0x7efe841f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe8415e850 0x7efe841f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 -- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe841587d0 msgr2=0x7efe841f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe841587d0 0x7efe841f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 -- 172.21.15.73:0/3944999662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700094d0 con 0x7efe8406cb80 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe83400640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe7400c410 tx=0x7efe7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe7b400640 1 -- 172.21.15.73:0/3944999662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740035f0 con 0x7efe8406cb80 2024-04-05T22:22:35.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe7b400640 1 -- 172.21.15.73:0/3944999662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74003750 con 0x7efe8406cb80 2024-04-05T22:22:35.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe7b400640 1 -- 172.21.15.73:0/3944999662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74003b40 con 0x7efe8406cb80 2024-04-05T22:22:35.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe841e6700 con 0x7efe8406cb80 2024-04-05T22:22:35.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.062+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe84157350 con 0x7efe8406cb80 2024-04-05T22:22:35.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.065+0000 7efe7b400640 1 -- 172.21.15.73:0/3944999662 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efe74003ca0 con 0x7efe8406cb80 2024-04-05T22:22:35.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.065+0000 7efe7b400640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 0x7efe6406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.066+0000 7efe7b400640 1 -- 172.21.15.73:0/3944999662 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe74088570 con 0x7efe8406cb80 2024-04-05T22:22:35.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.066+0000 7efe82a00640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 0x7efe6406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.066+0000 7efe8d054740 1 Processor -- start 2024-04-05T22:22:35.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.066+0000 7efe8d054740 1 -- start start 2024-04-05T22:22:35.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe82a00640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 0x7efe6406d0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe70036000 tx=0x7efe70030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e21f0 0x55ecf09b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09b56d0 0x55ecf09e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09258c0 con 0x55ecf09e25d0 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0923050 con 0x55ecf09e21f0 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.067+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecf091e940 con 0x55ecf09b56d0 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46664/0 (socket says 172.21.15.73:46664) 2024-04-05T22:22:35.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 -- 172.21.15.73:0/1447302932 learned_addr learned my addr 172.21.15.73:0/1447302932 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe83400640 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e21f0 0x55ecf09b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 -- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09b56d0 msgr2=0x55ecf09e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09b56d0 0x55ecf09e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 -- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e21f0 msgr2=0x55ecf09b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e21f0 0x55ecf09b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 -- 172.21.15.73:0/1447302932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf08e99e0 con 0x55ecf09e25d0 2024-04-05T22:22:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.068+0000 7efe82a00640 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe70031c40 tx=0x7efe70031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f98710ca42ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe6fe00640 1 -- 172.21.15.73:0/1447302932 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7003a9a0 con 0x55ecf09e25d0 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe6fe00640 1 -- 172.21.15.73:0/1447302932 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7004a070 con 0x55ecf09e25d0 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe6fe00640 1 -- 172.21.15.73:0/1447302932 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7003ad00 con 0x55ecf09e25d0 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 -- 172.21.15.73:0/1447302932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 msgr2=0x55ecf09e4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe70031c40 tx=0x7efe70031000 comp rx=0 tx=0).stop 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 -- 172.21.15.73:0/1447302932 shutdown_connections 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09b56d0 0x55ecf09e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e25d0 0x55ecf09e4f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 --2- 172.21.15.73:0/1447302932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e21f0 0x55ecf09b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.069+0000 7efe8d054740 1 -- 172.21.15.73:0/1447302932 >> 172.21.15.73:0/1447302932 conn(0x55ecf09b7400 msgr2=0x55ecf09fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.070+0000 7efe8d054740 1 -- 172.21.15.73:0/1447302932 shutdown_connections 2024-04-05T22:22:35.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.070+0000 7efe8d054740 1 -- 172.21.15.73:0/1447302932 wait complete. 2024-04-05T22:22:35.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.070+0000 7efe8d054740 1 -- start start 2024-04-05T22:22:35.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.071+0000 7efe8d054740 1 Processor -- start 2024-04-05T22:22:35.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e25d0 0x55ecf09d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09d2960 0x55ecf09d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf08c0070 con 0x55ecf09e21f0 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf08bfd10 con 0x55ecf09d2960 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0685f20 con 0x55ecf09e25d0 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe8d054740 10 client.0 _ll_register_callbacks cb 0x55ecf0917fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46674/0 (socket says 172.21.15.73:46674) 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 -- 172.21.15.73:0/2837752083 learned_addr learned my addr 172.21.15.73:0/2837752083 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe82a00640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e25d0 0x55ecf09d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 -- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e25d0 msgr2=0x55ecf09d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e25d0 0x55ecf09d33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 -- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09d2960 msgr2=0x55ecf09d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09d2960 0x55ecf09d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.072+0000 7efe83400640 1 -- 172.21.15.73:0/2837752083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe70040070 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe83400640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe7405f2c0 tx=0x7efe74057400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a242e84727cfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7405f560 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe7405f7f0 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecf0948ae0 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe740880c0 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 10 client.5325 fetch_fsmap learned FSMap version 15 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55ecf091cfc0 con 0x55ecf09e21f0 2024-04-05T22:22:35.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecf0948ae0 con 0x55ecf09e21f0 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7efe74088220 con 0x55ecf09e21f0 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 10 client.5325 fetch_fsmap finished waiting for FSMap version 15 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 10 client.5325 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.073+0000 7efe8d054740 20 client.5325 populate_metadata read hostname 'smithi073' 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09e6fc0 0x55ecf09e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe8d054740 4 client.5325 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 --> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecf06123c0 con 0x55ecf09e6fc0 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe82a00640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09e6fc0 0x55ecf09e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe82a00640 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09e6fc0 0x55ecf09e73a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8929412742462282 server_cookie=b3900d019e4e4c4f in_seq=0 out_seq=0 2024-04-05T22:22:35.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.074+0000 7efe6cc00640 10 client.5325 ms_handle_connect on v2:172.21.15.73:6834/2312639511 2024-04-05T22:22:35.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe6cc00640 1 -- 172.21.15.73:0/2837752083 <== mds.0 v2:172.21.15.73:6834/2312639511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ecf06123c0 con 0x55ecf09e6fc0 2024-04-05T22:22:35.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe6cc00640 10 client.5325 handle_command_reply: tid=0 2024-04-05T22:22:35.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 client.5325 shutdown 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 msgr2=0x55ecf0a001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe7405f2c0 tx=0x7efe74057400 comp rx=0 tx=0).stop 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 shutdown_connections 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09e6fc0 0x55ecf09e73a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09d2960 0x55ecf09d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e25d0 0x55ecf09d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 --2- 172.21.15.73:0/2837752083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e21f0 0x55ecf0a001b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe6cc00640 0 client.5325 ms_handle_reset on v2:172.21.15.73:6834/2312639511 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.075+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 >> 172.21.15.73:0/2837752083 conn(0x55ecf09b56d0 msgr2=0x55ecf09b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.076+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 shutdown_connections 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.076+0000 7efe8d054740 1 -- 172.21.15.73:0/2837752083 wait complete. 2024-04-05T22:22:35.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.076+0000 7efe8d054740 20 client.5325 trim_cache size 0 max 16384 2024-04-05T22:22:35.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.092+0000 7efe8d054740 1 Processor -- start 2024-04-05T22:22:35.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.092+0000 7efe8d054740 1 -- start start 2024-04-05T22:22:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf09d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09fb030 con 0x55ecf09d2960 2024-04-05T22:22:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09b8bf0 con 0x55ecf09e25d0 2024-04-05T22:22:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09d6270 con 0x55ecf09e21f0 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.093+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46686/0 (socket says 172.21.15.73:46686) 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 -- 172.21.15.73:0/1397500424 learned_addr learned my addr 172.21.15.73:0/1397500424 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83e00640 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf09d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe82a00640 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 -- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 msgr2=0x55ecf09ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 -- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 msgr2=0x55ecf09d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf09d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 -- 172.21.15.73:0/1397500424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf0951f80 con 0x55ecf09d2960 2024-04-05T22:22:35.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.094+0000 7efe83400640 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe74055750 tx=0x7efe7401d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a437c6a402ce34a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe78c00640 1 -- 172.21.15.73:0/1397500424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7400e040 con 0x55ecf09d2960 2024-04-05T22:22:35.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe78c00640 1 -- 172.21.15.73:0/1397500424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7405d260 con 0x55ecf09d2960 2024-04-05T22:22:35.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe78c00640 1 -- 172.21.15.73:0/1397500424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7405f560 con 0x55ecf09d2960 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 -- 172.21.15.73:0/1397500424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 msgr2=0x55ecf0a011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe74055750 tx=0x7efe7401d070 comp rx=0 tx=0).stop 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 -- 172.21.15.73:0/1397500424 shutdown_connections 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf09d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 --2- 172.21.15.73:0/1397500424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09d2960 0x55ecf0a011d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 -- 172.21.15.73:0/1397500424 >> 172.21.15.73:0/1397500424 conn(0x55ecf09b56d0 msgr2=0x55ecf09b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.095+0000 7efe8d054740 1 -- 172.21.15.73:0/1397500424 shutdown_connections 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.096+0000 7efe8d054740 1 -- 172.21.15.73:0/1397500424 wait complete. 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.096+0000 7efe8d054740 1 -- start start 2024-04-05T22:22:35.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.096+0000 7efe8d054740 1 Processor -- start 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf0a67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0a68c60 con 0x55ecf09e0b90 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09e5450 con 0x55ecf09e25d0 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecf09d34b0 con 0x55ecf09e21f0 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.097+0000 7efe8d054740 10 client.0 _ll_register_callbacks cb 0x7efe74005560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46692/0 (socket says 172.21.15.73:46692) 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 -- 172.21.15.73:0/3887236857 learned_addr learned my addr 172.21.15.73:0/3887236857 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 -- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 msgr2=0x55ecf09e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83400640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe82a00640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf0a67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 -- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 msgr2=0x55ecf0a67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf0a67e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 -- 172.21.15.73:0/3887236857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecf091cfc0 con 0x55ecf09e0b90 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe83e00640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00f8a0 tx=0x7efe7c00fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270bb40646824a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.098+0000 7efe6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00d900 con 0x55ecf09e0b90 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe7c00da60 con 0x55ecf09e0b90 2024-04-05T22:22:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe7004a070 con 0x55ecf09e0b90 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7c00dc20 con 0x55ecf09e0b90 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 10 client.5331 fetch_fsmap learned FSMap version 15 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55ecf08bf280 con 0x55ecf09e0b90 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7004a070 con 0x55ecf09e0b90 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7efe7c015450 con 0x55ecf09e0b90 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 10 client.5331 fetch_fsmap finished waiting for FSMap version 15 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 10 client.5331 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 20 client.5331 populate_metadata read hostname 'smithi073' 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09d2070 0x55ecf0a0e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 4 client.5331 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.099+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 --> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ecf06236c0 con 0x55ecf09d2070 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe83400640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09d2070 0x55ecf0a0e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe6cc00640 10 client.5331 ms_handle_connect on v2:172.21.15.73:6834/2312639511 2024-04-05T22:22:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe83400640 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09d2070 0x55ecf0a0e2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b389d90d6ff1c7d server_cookie=13521643171f0dd6 in_seq=0 out_seq=0 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe6cc00640 1 -- 172.21.15.73:0/3887236857 <== mds.0 v2:172.21.15.73:6834/2312639511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55ecf06236c0 con 0x55ecf09d2070 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe6cc00640 10 client.5331 handle_command_reply: tid=0 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe8d054740 1 client.5331 shutdown 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 msgr2=0x55ecf0a683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.100+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00f8a0 tx=0x7efe7c00fca0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 shutdown_connections 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:6834/2312639511,v1:172.21.15.73:6836/2312639511] conn(0x55ecf09d2070 0x55ecf0a0e2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ecf09e0b90 0x55ecf0a683a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecf09e25d0 0x55ecf0a67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 --2- 172.21.15.73:0/3887236857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ecf09e21f0 0x55ecf09e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe6cc00640 0 client.5331 ms_handle_reset on v2:172.21.15.73:6834/2312639511 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 >> 172.21.15.73:0/3887236857 conn(0x55ecf09b56d0 msgr2=0x55ecf09d5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 shutdown_connections 2024-04-05T22:22:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 1 -- 172.21.15.73:0/3887236857 wait complete. 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8d054740 20 client.5331 trim_cache size 0 max 16384 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5115, 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 11.480943045, 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 15, 2024-04-05T22:22:35.105 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 36, 2024-04-05T22:22:35.106 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 36, 2024-04-05T22:22:35.106 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 20.05583214 2024-04-05T22:22:35.106 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:22:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 msgr2=0x7efe6406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 0x7efe6406d0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe70036000 tx=0x7efe70030000 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 msgr2=0x7efe84156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.101+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe7400c410 tx=0x7efe7400c8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 shutdown_connections 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7efe6406ac30 0x7efe6406d0f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe8415e850 0x7efe841f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe841587d0 0x7efe841f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 --2- 172.21.15.73:0/3944999662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8406cb80 0x7efe84156e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 >> 172.21.15.73:0/3944999662 conn(0x7efe840ce8e0 msgr2=0x7efe840ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 shutdown_connections 2024-04-05T22:22:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.102+0000 7efe8a200640 1 -- 172.21.15.73:0/3944999662 wait complete. 2024-04-05T22:22:35.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:22:35.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.326+0000 7f3207000640 1 Processor -- start 2024-04-05T22:22:35.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.327+0000 7f3207000640 1 -- start start 2024-04-05T22:22:35.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3207000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f320015cef0 0x7f32000d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3207000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32000b7510 0x7f32000d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3207000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32000b6070 con 0x7f32000d0740 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3207000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000b5a60 con 0x7f320015cef0 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3207000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000d0bd0 con 0x7f32000b7510 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3205c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f320015cef0 0x7f32000d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3205c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f320015cef0 0x7f32000d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46706/0 (socket says 172.21.15.73:46706) 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.328+0000 7f3205c00640 1 -- 172.21.15.73:0/55792518 learned_addr learned my addr 172.21.15.73:0/55792518 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.329+0000 7f3205c00640 1 -- 172.21.15.73:0/55792518 >> v2:172.21.15.157:3300/0 conn(0x7f32000b7510 msgr2=0x7f32000d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.329+0000 7f3205c00640 1 --2- 172.21.15.73:0/55792518 >> v2:172.21.15.157:3300/0 conn(0x7f32000b7510 0x7f32000d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.329+0000 7f3205c00640 1 -- 172.21.15.73:0/55792518 >> v1:172.21.15.157:6789/0 conn(0x7f32000d0740 legacy=0x7f32000cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.329+0000 7f3205c00640 1 -- 172.21.15.73:0/55792518 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000b5d00 con 0x7f320015cef0 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.330+0000 7f3205c00640 1 --2- 172.21.15.73:0/55792518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f32000d1130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f32000cf150 tx=0x7f31f402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e69ef83a0314003 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.330+0000 7f31ffe00640 1 -- 172.21.15.73:0/55792518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f402c9f0 con 0x7f320015cef0 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.330+0000 7f31ffe00640 1 -- 172.21.15.73:0/55792518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f402cb50 con 0x7f320015cef0 2024-04-05T22:22:35.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.330+0000 7f31ffe00640 1 -- 172.21.15.73:0/55792518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40343f0 con 0x7f320015cef0 2024-04-05T22:22:35.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 -- 172.21.15.73:0/55792518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 msgr2=0x7f32000d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 --2- 172.21.15.73:0/55792518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f32000d1130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f32000cf150 tx=0x7f31f402be50 comp rx=0 tx=0).stop 2024-04-05T22:22:35.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 -- 172.21.15.73:0/55792518 shutdown_connections 2024-04-05T22:22:35.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 --2- 172.21.15.73:0/55792518 >> v2:172.21.15.157:3300/0 conn(0x7f32000b7510 0x7f32000d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 --2- 172.21.15.73:0/55792518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f32000d1130 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 -- 172.21.15.73:0/55792518 >> 172.21.15.73:0/55792518 conn(0x7f320009be40 msgr2=0x7f32000d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 -- 172.21.15.73:0/55792518 shutdown_connections 2024-04-05T22:22:35.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.331+0000 7f3207000640 1 -- 172.21.15.73:0/55792518 wait complete. 2024-04-05T22:22:35.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.333+0000 7f3207000640 1 Processor -- start 2024-04-05T22:22:35.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.333+0000 7f3207000640 1 -- start start 2024-04-05T22:22:35.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32000b7510 0x7f320015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 0x7f32000c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200157590 con 0x7f320015cef0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320015d890 con 0x7f320015d2d0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3207000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320015db70 con 0x7f32000b7510 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3205200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3205200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46710/0 (socket says 172.21.15.73:46710) 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3205200640 1 -- 172.21.15.73:0/1635170467 learned_addr learned my addr 172.21.15.73:0/1635170467 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3206600640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 0x7f32000c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.334+0000 7f3205c00640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32000b7510 0x7f320015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 -- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32000b7510 msgr2=0x7f320015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32000b7510 0x7f320015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 -- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 msgr2=0x7f32000c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 0x7f32000c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 -- 172.21.15.73:0/1635170467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4009b30 con 0x7f320015cef0 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3206600640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 0x7f32000c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:35.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3205200640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c410 tx=0x7f31ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0035f0 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec003750 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec003b00 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320015de50 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.335+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320015e410 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.336+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320004e160 con 0x7f320015cef0 2024-04-05T22:22:35.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.337+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f31ec003c60 con 0x7f320015cef0 2024-04-05T22:22:35.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.338+0000 7f31fd600640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 0x7f31e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:35.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.338+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31ec088630 con 0x7f320015cef0 2024-04-05T22:22:35.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.338+0000 7f3205c00640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 0x7f31e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:35.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.339+0000 7f3205c00640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 0x7f31e006d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31f4035000 tx=0x7f31f4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:35.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.343+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f31ec053810 con 0x7f320015cef0 2024-04-05T22:22:35.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.577+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f320006bc40 con 0x7f320015cef0 2024-04-05T22:22:35.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.577+0000 7f31fd600640 1 -- 172.21.15.73:0/1635170467 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31ec056e30 con 0x7f320015cef0 2024-04-05T22:22:35.578 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 msgr2=0x7f31e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 0x7f31e006d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31f4035000 tx=0x7f31f4030000 comp rx=0 tx=0).stop 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 msgr2=0x7f320014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c410 tx=0x7f31ec00c8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 shutdown_connections 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006ac00 0x7f31e006d0c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320015d2d0 0x7f32000c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f320015cef0 0x7f320014dfc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.580+0000 7f3207000640 1 --2- 172.21.15.73:0/1635170467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32000b7510 0x7f320015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:35.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.581+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 >> 172.21.15.73:0/1635170467 conn(0x7f320009be40 msgr2=0x7f32000ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:35.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.581+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 shutdown_connections 2024-04-05T22:22:35.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:35.581+0000 7f3207000640 1 -- 172.21.15.73:0/1635170467 wait complete. 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:22:35.600 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:22:35.601 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:22:35.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:35.601 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:22:35.654 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42507sec preferred_lft 42507sec 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:22:35.656 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:35.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:35.657 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:22:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:22:35.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:35.751 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:22:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:22:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:22:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:22:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:22:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:22:35.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:35.803 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:22:35.804 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:22:35.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:22:35.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:22:35.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:22:35.804 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:22:35.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:35.989 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:22:36.036 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:22:36.037 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:22:36.038 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:22:36.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:36.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:22:36.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:22:36.125 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:22:36.175 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:22:36.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:22:36.175 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:22:36.224 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:22:36.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:36.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:36.356 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:36.356 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:22:36.390 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:36.390 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:22:36.391 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:22:36.391 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:22:36.391 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:22:36.393 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:22:36.393 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:22:36.393 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:22:36.393 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:22:36.393 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:22:36.398 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:22:36.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:36.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:22:36.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:36.483 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout:client4744 2024-04-05T22:22:36.546 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:36.546 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:36.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:22:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.857+0000 7f8197200640 1 Processor -- start 2024-04-05T22:22:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.857+0000 7f8197200640 1 -- start start 2024-04-05T22:22:36.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.859+0000 7f8197200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81980cd9e0 0x7f81980b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:36.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.859+0000 7f8197200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980b21c0 con 0x7f81980b11e0 2024-04-05T22:22:36.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.859+0000 7f8197200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980cf4c0 con 0x7f81980b0e00 2024-04-05T22:22:36.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.859+0000 7f8197200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980cf7a0 con 0x7f81980cd9e0 2024-04-05T22:22:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81980cd9e0 0x7f81980b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81980cd9e0 0x7f81980b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36994/0 (socket says 172.21.15.73:36994) 2024-04-05T22:22:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 -- 172.21.15.73:0/1730516329 learned_addr learned my addr 172.21.15.73:0/1730516329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8196800640 1 --1- 172.21.15.73:0/1730516329 >> v1:172.21.15.73:6789/0 conn(0x7f81980b11e0 0x7f81980cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40746/0 (socket says 172.21.15.73:40746) 2024-04-05T22:22:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 -- 172.21.15.73:0/1730516329 >> v1:172.21.15.157:6789/0 conn(0x7f81980b0e00 legacy=0x7f81980cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 -- 172.21.15.73:0/1730516329 >> v1:172.21.15.73:6789/0 conn(0x7f81980b11e0 legacy=0x7f81980cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8195400640 1 -- 172.21.15.73:0/1730516329 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81980cfa80 con 0x7f81980cd9e0 2024-04-05T22:22:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.860+0000 7f8194a00640 1 -- 172.21.15.73:0/1730516329 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630514266 0 0) 0x7f81980b21c0 con 0x7f81980b11e0 2024-04-05T22:22:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.861+0000 7f8195400640 1 --2- 172.21.15.73:0/1730516329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980cd9e0 0x7f81980b2f50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8184009870 tx=0x7f818402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24376caed4db042 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:36.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8194a00640 1 -- 172.21.15.73:0/1730516329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184031870 con 0x7f81980cd9e0 2024-04-05T22:22:36.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8194a00640 1 -- 172.21.15.73:0/1730516329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840319d0 con 0x7f81980cd9e0 2024-04-05T22:22:36.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8194a00640 1 -- 172.21.15.73:0/1730516329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184031d60 con 0x7f81980cd9e0 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 -- 172.21.15.73:0/1730516329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980cd9e0 msgr2=0x7f81980b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 --2- 172.21.15.73:0/1730516329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980cd9e0 0x7f81980b2f50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8184009870 tx=0x7f818402b910 comp rx=0 tx=0).stop 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 -- 172.21.15.73:0/1730516329 shutdown_connections 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 --2- 172.21.15.73:0/1730516329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980cd9e0 0x7f81980b2f50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 -- 172.21.15.73:0/1730516329 >> 172.21.15.73:0/1730516329 conn(0x7f81980b2430 msgr2=0x7f81980b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 -- 172.21.15.73:0/1730516329 shutdown_connections 2024-04-05T22:22:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.862+0000 7f8197200640 1 -- 172.21.15.73:0/1730516329 wait complete. 2024-04-05T22:22:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.864+0000 7f8197200640 1 Processor -- start 2024-04-05T22:22:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.864+0000 7f8197200640 1 -- start start 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81980b11e0 0x7f81980b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81980cd9e0 0x7f81980b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980b67c0 con 0x7f81980b0e00 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c8c90 con 0x7f81980cd9e0 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.865+0000 7f8197200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c8f70 con 0x7f81980b11e0 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37008/0 (socket says 172.21.15.73:37008) 2024-04-05T22:22:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 -- 172.21.15.73:0/3874445316 learned_addr learned my addr 172.21.15.73:0/3874445316 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:36.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 -- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81980b11e0 msgr2=0x7f81980b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:36.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81980b11e0 0x7f81980b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:36.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 -- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81980cd9e0 msgr2=0x7f81980b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81980cd9e0 0x7f81980b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 -- 172.21.15.73:0/3874445316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840094d0 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.866+0000 7f8195e00640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f818800c430 tx=0x7f818800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.867+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188011790 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.867+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81880118f0 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.867+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188011ce0 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.867+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980c9250 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.867+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81980b6420 con 0x7f81980b0e00 2024-04-05T22:22:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.868+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81980c9770 con 0x7f81980b0e00 2024-04-05T22:22:36.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.869+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f818801c460 con 0x7f81980b0e00 2024-04-05T22:22:36.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.870+0000 7f818e000640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 0x7f817806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:36.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.870+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8188089390 con 0x7f81980b0e00 2024-04-05T22:22:36.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.870+0000 7f8195400640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 0x7f817806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:36.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.871+0000 7f8195400640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 0x7f817806d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8184036000 tx=0x7f8184030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:36.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:36.874+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f81980c9770 con 0x7f81980b0e00 2024-04-05T22:22:37.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.167+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81980c9770 con 0x7f81980b0e00 2024-04-05T22:22:37.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.168+0000 7f818e000640 1 -- 172.21.15.73:0/3874445316 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f81980c9770 con 0x7f81980b0e00 2024-04-05T22:22:37.169 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:22:37.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 msgr2=0x7f817806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:37.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 0x7f817806d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8184036000 tx=0x7f8184030000 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 msgr2=0x7f81981f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f818800c430 tx=0x7f818800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 shutdown_connections 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f817806ac00 0x7f817806d0c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81980cd9e0 0x7f81980b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81980b11e0 0x7f81980b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 --2- 172.21.15.73:0/3874445316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81980b0e00 0x7f81981f23f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 >> 172.21.15.73:0/3874445316 conn(0x7f81980b2430 msgr2=0x7f81980cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 shutdown_connections 2024-04-05T22:22:37.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:37.171+0000 7f8197200640 1 -- 172.21.15.73:0/3874445316 wait complete. 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:22:37.191 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:22:37.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.192 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.252 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.253 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:22:37.254 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:22:37.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.255 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:22:37.347 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:22:37.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.351 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:22:37.404 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:22:37.405 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:22:37.405 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:22:37.405 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:22:37.405 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:22:37.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.405 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:22:37.405 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:22:37.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:22:37.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:22:37.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:22:37.406 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:22:37.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.592 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:22:37.641 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:22:37.642 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:22:37.642 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:22:37.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.642 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:22:37.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:22:37.733 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:22:37.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:22:37.769 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:22:37.787 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:22:37.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:22:37.787 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:22:37.876 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:22:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:37.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:38.004 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:38.005 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:22:38.040 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:22:38.040 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:22:38.040 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:22:38.040 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:22:38.041 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:22:38.042 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:22:38.042 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:22:38.042 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:22:38.042 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:22:38.042 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:22:38.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:22:38.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:38.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:22:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:38.128 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:22:38.181 INFO:teuthology.orchestra.run.smithi157.stdout:client4747 2024-04-05T22:22:38.182 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:38.182 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:38.231 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:22:38.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.450+0000 7f0b55600640 1 Processor -- start 2024-04-05T22:22:38.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.451+0000 7f0b55600640 1 -- start start 2024-04-05T22:22:38.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.452+0000 7f0b55600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b5015ae50 0x7f0b500b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.452+0000 7f0b55600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b500b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.452+0000 7f0b55600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5015b230 con 0x7f0b500cce90 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.452+0000 7f0b55600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b500cd320 con 0x7f0b5015ae50 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.452+0000 7f0b55600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500ce2f0 con 0x7f0b500cf3e0 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4e000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b500cce90 0x7f0b500cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40748/0 (socket says 172.21.15.73:40748) 2024-04-05T22:22:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4e000640 1 -- 172.21.15.73:0/3298925203 learned_addr learned my addr 172.21.15.73:0/3298925203 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4f400640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b500b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> v2:172.21.15.157:3300/0 conn(0x7f0b5015ae50 0x7f0b500b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802953208 0 0) 0x7f0b5015b230 con 0x7f0b500cce90 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b34003520 con 0x7f0b500cce90 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1002077345 0 0) 0x7f0b34003520 con 0x7f0b500cce90 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 msgr2=0x7f0b500b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b500b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 >> v2:172.21.15.157:3300/0 conn(0x7f0b5015ae50 msgr2=0x7f0b500b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 --2- 172.21.15.73:0/3298925203 >> v2:172.21.15.157:3300/0 conn(0x7f0b5015ae50 0x7f0b500b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.453+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5015f4a0 con 0x7f0b500cce90 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0b38001670 con 0x7f0b500cce90 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 >> v1:172.21.15.73:6789/0 conn(0x7f0b500cce90 legacy=0x7f0b500cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b34003970 0x7f0b34003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34004d00 0x7f0b34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500ce2f0 con 0x7f0b34003970 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3400ba90 con 0x7f0b34004d00 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3400bc50 con 0x7f0b500cf3e0 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> v2:172.21.15.157:3300/0 conn(0x7f0b5015ae50 0x7f0b500b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b34003970 0x7f0b34003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4e000640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4f400640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34004d00 0x7f0b34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4ea00640 1 -- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 msgr2=0x7f0b340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.454+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.455+0000 7f0b4ea00640 1 -- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34004d00 msgr2=0x7f0b34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.455+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34004d00 0x7f0b34005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.455+0000 7f0b4ea00640 1 -- 172.21.15.73:0/3298925203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34003520 con 0x7f0b34003970 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.455+0000 7f0b4e000640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cf3e0 0x7f0b340047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b34003970 0x7f0b34003db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0b400098a0 tx=0x7f0b340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62e56aedfc363a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40037600 con 0x7f0b34003970 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40037760 con 0x7f0b34003970 2024-04-05T22:22:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b4d600640 1 -- 172.21.15.73:0/3298925203 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40037ad0 con 0x7f0b34003970 2024-04-05T22:22:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b55600640 1 -- 172.21.15.73:0/3298925203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b34003970 msgr2=0x7f0b34003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b55600640 1 --2- 172.21.15.73:0/3298925203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b34003970 0x7f0b34003db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0b400098a0 tx=0x7f0b340057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b4f400640 1 -- 172.21.15.73:0/3298925203 reap_dead start 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.456+0000 7f0b55600640 1 -- 172.21.15.73:0/3298925203 shutdown_connections 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.457+0000 7f0b55600640 1 -- 172.21.15.73:0/3298925203 >> 172.21.15.73:0/3298925203 conn(0x7f0b5006a0d0 msgr2=0x7f0b5006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.457+0000 7f0b55600640 1 -- 172.21.15.73:0/3298925203 shutdown_connections 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.457+0000 7f0b55600640 1 -- 172.21.15.73:0/3298925203 wait complete. 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.458+0000 7f0b55600640 1 Processor -- start 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.458+0000 7f0b55600640 1 -- start start 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34003970 0x7f0b500c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b501496b0 0x7f0b500ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5014ad60 con 0x7f0b500aead0 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500aef30 con 0x7f0b34003970 2024-04-05T22:22:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.459+0000 7f0b55600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5014a0e0 con 0x7f0b501496b0 2024-04-05T22:22:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37012/0 (socket says 172.21.15.73:37012) 2024-04-05T22:22:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 -- 172.21.15.73:0/1333357873 learned_addr learned my addr 172.21.15.73:0/1333357873 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34003970 0x7f0b500c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 -- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b501496b0 msgr2=0x7f0b500ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4e000640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b501496b0 0x7f0b500ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b501496b0 0x7f0b500ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 -- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34003970 msgr2=0x7f0b500c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34003970 0x7f0b500c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 -- 172.21.15.73:0/1333357873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5014a3c0 con 0x7f0b500aead0 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.460+0000 7f0b4f400640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0b500ce890 tx=0x7f0b48009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.461+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48007ca0 con 0x7f0b500aead0 2024-04-05T22:22:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.461+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b48007e20 con 0x7f0b500aead0 2024-04-05T22:22:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.461+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48014640 con 0x7f0b500aead0 2024-04-05T22:22:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.461+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5014a6a0 con 0x7f0b500aead0 2024-04-05T22:22:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.461+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b50149b10 con 0x7f0b500aead0 2024-04-05T22:22:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.462+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50149d30 con 0x7f0b500aead0 2024-04-05T22:22:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.463+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0b4801d3b0 con 0x7f0b500aead0 2024-04-05T22:22:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.464+0000 7f0b46a00640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 0x7f0b2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.464+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b48088f20 con 0x7f0b500aead0 2024-04-05T22:22:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.464+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 0x7f0b2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:38.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.465+0000 7f0b4ea00640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 0x7f0b2c06d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b40036000 tx=0x7f0b40031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.468+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0b50149d30 con 0x7f0b500aead0 2024-04-05T22:22:38.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:38.707+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0b50149d30 con 0x7f0b500aead0 2024-04-05T22:22:39.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.597+0000 7f0b46a00640 1 -- 172.21.15.73:0/1333357873 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0b50149d30 con 0x7f0b500aead0 2024-04-05T22:22:39.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.600+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 msgr2=0x7f0b2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 0x7f0b2c06d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b40036000 tx=0x7f0b40031000 comp rx=0 tx=0).stop 2024-04-05T22:22:39.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 msgr2=0x7f0b500c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0b500ce890 tx=0x7f0b48009cd0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 shutdown_connections 2024-04-05T22:22:39.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b2c06ac00 0x7f0b2c06d0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b500aead0 0x7f0b500c5ff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b501496b0 0x7f0b500ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 --2- 172.21.15.73:0/1333357873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34003970 0x7f0b500c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 >> 172.21.15.73:0/1333357873 conn(0x7f0b5006a0d0 msgr2=0x7f0b500ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:39.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 shutdown_connections 2024-04-05T22:22:39.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.601+0000 7f0b55600640 1 -- 172.21.15.73:0/1333357873 wait complete. 2024-04-05T22:22:39.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:22:39.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.824+0000 7fb8ad400640 1 Processor -- start 2024-04-05T22:22:39.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.825+0000 7fb8ad400640 1 -- start start 2024-04-05T22:22:39.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8ad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a815cf50 0x7fb8a806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8ad400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8ad400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80cb950 con 0x7fb8a80d09b0 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8ad400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a80ce990 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8ad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80d1350 con 0x7fb8a815cf50 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8a7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8a7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37026/0 (socket says 172.21.15.73:37026) 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8a7200640 1 -- 172.21.15.73:0/1172592411 learned_addr learned my addr 172.21.15.73:0/1172592411 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.826+0000 7fb8a5e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb8a80d09b0 0x7fb8a80cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40760/0 (socket says 172.21.15.73:40760) 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a6800640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a815cf50 0x7fb8a806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a7200640 1 -- 172.21.15.73:0/1172592411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a815cf50 msgr2=0x7fb8a806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a7200640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a815cf50 0x7fb8a806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a7200640 1 -- 172.21.15.73:0/1172592411 >> v1:172.21.15.73:6789/0 conn(0x7fb8a80d09b0 legacy=0x7fb8a80cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a7200640 1 -- 172.21.15.73:0/1172592411 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80cdce0 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.827+0000 7fb8a5400640 1 -- 172.21.15.73:0/1172592411 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596278082 0 0) 0x7fb8a80cb950 con 0x7fb8a80d09b0 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.828+0000 7fb8a7200640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e50 tx=0x7fb8a002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a3bd80b94f3404 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.828+0000 7fb8a5400640 1 -- 172.21.15.73:0/1172592411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a002cb00 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.828+0000 7fb8a5400640 1 -- 172.21.15.73:0/1172592411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a002cc60 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.828+0000 7fb8a5400640 1 -- 172.21.15.73:0/1172592411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00343f0 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 -- 172.21.15.73:0/1172592411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a80cd7a0 msgr2=0x7fb8a80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e50 tx=0x7fb8a002bf60 comp rx=0 tx=0).stop 2024-04-05T22:22:39.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 -- 172.21.15.73:0/1172592411 shutdown_connections 2024-04-05T22:22:39.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80d0e10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1172592411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a815cf50 0x7fb8a806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 -- 172.21.15.73:0/1172592411 >> 172.21.15.73:0/1172592411 conn(0x7fb8a8099a60 msgr2=0x7fb8a8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:39.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 -- 172.21.15.73:0/1172592411 shutdown_connections 2024-04-05T22:22:39.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.829+0000 7fb8ad400640 1 -- 172.21.15.73:0/1172592411 wait complete. 2024-04-05T22:22:39.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.831+0000 7fb8ad400640 1 Processor -- start 2024-04-05T22:22:39.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.831+0000 7fb8ad400640 1 -- start start 2024-04-05T22:22:39.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 0x7fb8a815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c58b0 con 0x7fb8a815cf50 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c4fa0 con 0x7fb8a80cd7a0 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8ad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a806efa0 con 0x7fb8a80d09b0 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8a7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8a7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37040/0 (socket says 172.21.15.73:37040) 2024-04-05T22:22:39.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8a7200640 1 -- 172.21.15.73:0/1587160534 learned_addr learned my addr 172.21.15.73:0/1587160534 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 -- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 msgr2=0x7fb8a815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.832+0000 7fb8a5e00640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 0x7fb8a815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a6800640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 0x7fb8a815b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 -- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 msgr2=0x7fb8a80b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:39.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 -- 172.21.15.73:0/1587160534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0009b30 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a6800640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a5e00640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 0x7fb8a815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8a7200640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e50 tx=0x7fb8a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0009510 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a003e450 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a80c51f0 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a003e5e0 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.833+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a80b6920 con 0x7fb8a815cf50 2024-04-05T22:22:39.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.835+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a80c59f0 con 0x7fb8a815cf50 2024-04-05T22:22:39.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.835+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb8a003e740 con 0x7fb8a815cf50 2024-04-05T22:22:39.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.836+0000 7fb89ea00640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 0x7fb88406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:39.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.836+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8a00abd20 con 0x7fb8a815cf50 2024-04-05T22:22:39.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.837+0000 7fb8a6800640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 0x7fb88406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:39.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.837+0000 7fb8a6800640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 0x7fb88406d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:39.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:39.841+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb8a80c59f0 con 0x7fb8a815cf50 2024-04-05T22:22:40.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.060+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a80cb5e0 con 0x7fb88406ac00 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.063+0000 7fb89ea00640 1 -- 172.21.15.73:0/1587160534 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb898009530 con 0x7fb88406ac00 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:22:40.063 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.065+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 msgr2=0x7fb88406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.065+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 0x7fb88406d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).stop 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.065+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 msgr2=0x7fb8a806b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.065+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e50 tx=0x7fb8a0004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 shutdown_connections 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb88406ac00 0x7fb88406d0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8a815cf50 0x7fb8a806b3d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8a80d09b0 0x7fb8a815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 --2- 172.21.15.73:0/1587160534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a80cd7a0 0x7fb8a80b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 >> 172.21.15.73:0/1587160534 conn(0x7fb8a8099a60 msgr2=0x7fb8a80ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 shutdown_connections 2024-04-05T22:22:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.066+0000 7fb8ad400640 1 -- 172.21.15.73:0/1587160534 wait complete. 2024-04-05T22:22:40.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:22:40.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.296+0000 7fc83d000640 1 Processor -- start 2024-04-05T22:22:40.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.296+0000 7fc83d000640 1 -- start start 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc83d000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 0x7fc8380d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc83d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8380d0bc0 0x7fc8380d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc83d000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380cd630 con 0x7fc83815d1a0 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc83d000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8380ceab0 con 0x7fc8380d0bc0 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc83d000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83815e280 con 0x7fc83815cdc0 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc82fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8380d0bc0 0x7fc8380d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc82fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8380d0bc0 0x7fc8380d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37048/0 (socket says 172.21.15.73:37048) 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc837e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc83815d1a0 0x7fc8380b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40772/0 (socket says 172.21.15.73:40772) 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.298+0000 7fc82fe00640 1 -- 172.21.15.73:0/2868475290 learned_addr learned my addr 172.21.15.73:0/2868475290 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc837400640 1 --2- 172.21.15.73:0/2868475290 >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 0x7fc8380d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:40.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc82fe00640 1 -- 172.21.15.73:0/2868475290 >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 msgr2=0x7fc8380d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:40.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc82fe00640 1 --2- 172.21.15.73:0/2868475290 >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 0x7fc8380d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc82fe00640 1 -- 172.21.15.73:0/2868475290 >> v1:172.21.15.73:6789/0 conn(0x7fc83815d1a0 legacy=0x7fc8380b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc82fe00640 1 -- 172.21.15.73:0/2868475290 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83815d700 con 0x7fc8380d0bc0 2024-04-05T22:22:40.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc836a00640 1 -- 172.21.15.73:0/2868475290 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385370453 0 0) 0x7fc8380cd630 con 0x7fc83815d1a0 2024-04-05T22:22:40.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.299+0000 7fc837400640 1 --2- 172.21.15.73:0/2868475290 >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 0x7fc8380d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.300+0000 7fc82fe00640 1 --2- 172.21.15.73:0/2868475290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380d0bc0 0x7fc8380d1070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8380d1c50 tx=0x7fc82002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ec9c476a89b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.300+0000 7fc836a00640 1 -- 172.21.15.73:0/2868475290 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820031870 con 0x7fc8380d0bc0 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.300+0000 7fc836a00640 1 -- 172.21.15.73:0/2868475290 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200319d0 con 0x7fc8380d0bc0 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.300+0000 7fc836a00640 1 -- 172.21.15.73:0/2868475290 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820031d60 con 0x7fc8380d0bc0 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 -- 172.21.15.73:0/2868475290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380d0bc0 msgr2=0x7fc8380d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 --2- 172.21.15.73:0/2868475290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380d0bc0 0x7fc8380d1070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8380d1c50 tx=0x7fc82002b910 comp rx=0 tx=0).stop 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 -- 172.21.15.73:0/2868475290 shutdown_connections 2024-04-05T22:22:40.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 --2- 172.21.15.73:0/2868475290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380d0bc0 0x7fc8380d1070 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 --2- 172.21.15.73:0/2868475290 >> v2:172.21.15.157:3300/0 conn(0x7fc83815cdc0 0x7fc8380d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 -- 172.21.15.73:0/2868475290 >> 172.21.15.73:0/2868475290 conn(0x7fc83809be20 msgr2=0x7fc8380d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:40.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 -- 172.21.15.73:0/2868475290 shutdown_connections 2024-04-05T22:22:40.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.301+0000 7fc83d000640 1 -- 172.21.15.73:0/2868475290 wait complete. 2024-04-05T22:22:40.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.303+0000 7fc83d000640 1 Processor -- start 2024-04-05T22:22:40.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.303+0000 7fc83d000640 1 -- start start 2024-04-05T22:22:40.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8380d0bc0 0x7fc8380cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83815d1a0 0x7fc838158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380b8810 con 0x7fc83815cdc0 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380c7670 con 0x7fc83815d1a0 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc83d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380c7950 con 0x7fc8380d0bc0 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37054/0 (socket says 172.21.15.73:37054) 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 -- 172.21.15.73:0/3621774305 learned_addr learned my addr 172.21.15.73:0/3621774305 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc837400640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8380d0bc0 0x7fc8380cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 -- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8380d0bc0 msgr2=0x7fc8380cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8380d0bc0 0x7fc8380cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 -- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83815d1a0 msgr2=0x7fc838158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83815d1a0 0x7fc838158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.304+0000 7fc82fe00640 1 -- 172.21.15.73:0/3621774305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200094d0 con 0x7fc83815cdc0 2024-04-05T22:22:40.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc82fe00640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc82002b950 tx=0x7fc8200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:40.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820031860 con 0x7fc83815cdc0 2024-04-05T22:22:40.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200443f0 con 0x7fc83815cdc0 2024-04-05T22:22:40.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200447a0 con 0x7fc83815cdc0 2024-04-05T22:22:40.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8380c7c30 con 0x7fc83815cdc0 2024-04-05T22:22:40.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.305+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8380b84a0 con 0x7fc83815cdc0 2024-04-05T22:22:40.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.306+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8380c8150 con 0x7fc83815cdc0 2024-04-05T22:22:40.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.307+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc82004d450 con 0x7fc83815cdc0 2024-04-05T22:22:40.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.308+0000 7fc82f400640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:40.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.308+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc82002d040 con 0x7fc83815cdc0 2024-04-05T22:22:40.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.308+0000 7fc837400640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:40.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.309+0000 7fc837400640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 0x7fc81406d0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc838160000 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:40.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.312+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc8380c8150 con 0x7fc83815cdc0 2024-04-05T22:22:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.504+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8380c8150 con 0x7fc83815cdc0 2024-04-05T22:22:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.505+0000 7fc82f400640 1 -- 172.21.15.73:0/3621774305 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8380c8150 con 0x7fc83815cdc0 2024-04-05T22:22:40.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 msgr2=0x7fc81406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 0x7fc81406d0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc838160000 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 msgr2=0x7fc8380cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc82002b950 tx=0x7fc8200046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 shutdown_connections 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc81406ac00 0x7fc81406d0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83815d1a0 0x7fc838158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83815cdc0 0x7fc8380cb0c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 --2- 172.21.15.73:0/3621774305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8380d0bc0 0x7fc8380cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 >> 172.21.15.73:0/3621774305 conn(0x7fc83809be20 msgr2=0x7fc83806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 shutdown_connections 2024-04-05T22:22:40.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:40.508+0000 7fc83d000640 1 -- 172.21.15.73:0/3621774305 wait complete. 2024-04-05T22:22:40.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:40.528 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp26820/subvol52580) 2024-04-05T22:22:40.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:40.633 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:22:40.687 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:22:40.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:40.688 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp26820/subvol52580) 2024-04-05T22:22:40.811 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:40.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp26820/subvol52580/.meta 2024-04-05T22:22:40.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol52580 subvol_grp26820 2024-04-05T22:22:41.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.173+0000 7f5251600640 1 Processor -- start 2024-04-05T22:22:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.173+0000 7f5251600640 1 -- start start 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.175+0000 7f5251600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0b36e0 0x7f524c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.175+0000 7f5251600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f524c0cf210 0x7f524c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.175+0000 7f5251600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c15afe0 con 0x7f524c15ac00 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.175+0000 7f5251600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c0cf6a0 con 0x7f524c0cf210 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.175+0000 7f5251600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c161760 con 0x7f524c0b36e0 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f524c0cf210 0x7f524c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f524c0cf210 0x7f524c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:51920/0 (socket says 172.21.15.73:51920) 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f5243e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0b36e0 0x7f524c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 -- 172.21.15.73:0/1556345551 learned_addr learned my addr 172.21.15.73:0/1556345551 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:41.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 -- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0b36e0 msgr2=0x7f524c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:41.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 --2- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0b36e0 0x7f524c0cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 -- 172.21.15.73:0/1556345551 >> v1:172.21.15.157:6789/0 conn(0x7f524c15ac00 legacy=0x7f524c06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.176+0000 7f524b400640 1 -- 172.21.15.73:0/1556345551 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c160b50 con 0x7f524c0cf210 2024-04-05T22:22:41.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.178+0000 7f524b400640 1 --2- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0cf210 0x7f524c15d560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5244009e50 tx=0x7f524402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e879b9e97f2208 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.178+0000 7f524a000640 1 -- 172.21.15.73:0/1556345551 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524402ca70 con 0x7f524c0cf210 2024-04-05T22:22:41.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.178+0000 7f524a000640 1 -- 172.21.15.73:0/1556345551 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524402cbd0 con 0x7f524c0cf210 2024-04-05T22:22:41.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.178+0000 7f524a000640 1 -- 172.21.15.73:0/1556345551 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440343f0 con 0x7f524c0cf210 2024-04-05T22:22:41.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 -- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0cf210 msgr2=0x7f524c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 --2- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0cf210 0x7f524c15d560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5244009e50 tx=0x7f524402bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 -- 172.21.15.73:0/1556345551 shutdown_connections 2024-04-05T22:22:41.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 --2- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0cf210 0x7f524c15d560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 --2- 172.21.15.73:0/1556345551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0b36e0 0x7f524c0cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 -- 172.21.15.73:0/1556345551 >> 172.21.15.73:0/1556345551 conn(0x7f524c06a0d0 msgr2=0x7f524c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:41.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 -- 172.21.15.73:0/1556345551 shutdown_connections 2024-04-05T22:22:41.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.179+0000 7f5251600640 1 -- 172.21.15.73:0/1556345551 wait complete. 2024-04-05T22:22:41.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.181+0000 7f5251600640 1 Processor -- start 2024-04-05T22:22:41.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.181+0000 7f5251600640 1 -- start start 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0b36e0 0x7f524c157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 0x7f524c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1589c0 con 0x7f524c15ac00 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c158ca0 con 0x7f524c0b36e0 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5251600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c158f80 con 0x7f524c0cf210 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f524b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f524b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37060/0 (socket says 172.21.15.73:37060) 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f524b400640 1 -- 172.21.15.73:0/4244564975 learned_addr learned my addr 172.21.15.73:0/4244564975 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f524aa00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0b36e0 0x7f524c157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.182+0000 7f5243e00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 0x7f524c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 -- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 msgr2=0x7f524c0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 0x7f524c0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 -- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0b36e0 msgr2=0x7f524c157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0b36e0 0x7f524c157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 -- 172.21.15.73:0/4244564975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244009b30 con 0x7f524c15ac00 2024-04-05T22:22:41.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5243e00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 0x7f524c0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f524b400640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5244009e50 tx=0x7f5244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524403e450 con 0x7f524c15ac00 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524403e5b0 con 0x7f524c15ac00 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524403e960 con 0x7f524c15ac00 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c159260 con 0x7f524c15ac00 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.183+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c15bf80 con 0x7f524c15ac00 2024-04-05T22:22:41.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.185+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c15c110 con 0x7f524c15ac00 2024-04-05T22:22:41.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.185+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f524403eac0 con 0x7f524c15ac00 2024-04-05T22:22:41.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.186+0000 7f5242a00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 0x7f522806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.186+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52440abfe0 con 0x7f524c15ac00 2024-04-05T22:22:41.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.187+0000 7f524aa00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 0x7f522806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.187+0000 7f524aa00640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 0x7f522806d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f523c007fd0 tx=0x7f523c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.191+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f52440771d0 con 0x7f524c15ac00 2024-04-05T22:22:41.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.420+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol52580", "group_name": "subvol_grp26820", "target": ["mon-mgr", ""]}) v1 -- 0x7f524c1613f0 con 0x7f522806ac00 2024-04-05T22:22:41.430 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:41.430+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:41.430 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:41.430+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:41.430 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:41.430+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:41.430 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:41.430+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:41.430 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:41.430+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.481+0000 7f5242a00640 1 -- 172.21.15.73:0/4244564975 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f523c001200 con 0x7f522806ac00 2024-04-05T22:22:41.482 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp26820/subvol52580 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 msgr2=0x7f522806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 0x7f522806d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f523c007fd0 tx=0x7f523c009040 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 msgr2=0x7f524c15de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5244009e50 tx=0x7f5244004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 shutdown_connections 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f522806ac00 0x7f522806d0c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f524c15ac00 0x7f524c15de20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f524c0cf210 0x7f524c0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 --2- 172.21.15.73:0/4244564975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524c0b36e0 0x7f524c157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 >> 172.21.15.73:0/4244564975 conn(0x7f524c06a0d0 msgr2=0x7f524c070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.484+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 shutdown_connections 2024-04-05T22:22:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.485+0000 7f5251600640 1 -- 172.21.15.73:0/4244564975 wait complete. 2024-04-05T22:22:41.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol52580 subvol_grp26820 2024-04-05T22:22:41.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.707+0000 7f6ba2e00640 1 Processor -- start 2024-04-05T22:22:41.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.707+0000 7f6ba2e00640 1 -- start start 2024-04-05T22:22:41.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b9c157370 0x7f6b9c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c070820 0x7f6b9c069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 0x7f6b9c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c15e2f0 con 0x7f6b9c157370 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c06a620 con 0x7f6b9c06a1f0 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6ba2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c070de0 con 0x7f6b9c070820 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6b9be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b9c157370 0x7f6b9c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6b9be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b9c157370 0x7f6b9c15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37094/0 (socket says 172.21.15.73:37094) 2024-04-05T22:22:41.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.709+0000 7f6b9be00640 1 -- 172.21.15.73:0/3872710074 learned_addr learned my addr 172.21.15.73:0/3872710074 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9b400640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c070820 0x7f6b9c069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6ba0e00640 1 --2- 172.21.15.73:0/3872710074 >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 0x7f6b9c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9be00640 1 -- 172.21.15.73:0/3872710074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c070820 msgr2=0x7f6b9c069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9be00640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c070820 0x7f6b9c069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9be00640 1 -- 172.21.15.73:0/3872710074 >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 msgr2=0x7f6b9c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9be00640 1 --2- 172.21.15.73:0/3872710074 >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 0x7f6b9c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6b9be00640 1 -- 172.21.15.73:0/3872710074 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0710c0 con 0x7f6b9c157370 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.710+0000 7f6ba0e00640 1 --2- 172.21.15.73:0/3872710074 >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 0x7f6b9c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:41.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.711+0000 7f6b9be00640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15ddb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c15af00 tx=0x7f6b8c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649097c50f775d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.711+0000 7f6b9aa00640 1 -- 172.21.15.73:0/3872710074 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c031860 con 0x7f6b9c157370 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.711+0000 7f6b9aa00640 1 -- 172.21.15.73:0/3872710074 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0319c0 con 0x7f6b9c157370 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.711+0000 7f6b9aa00640 1 -- 172.21.15.73:0/3872710074 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c031d50 con 0x7f6b9c157370 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 -- 172.21.15.73:0/3872710074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 msgr2=0x7f6b9c15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15ddb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c15af00 tx=0x7f6b8c02b900 comp rx=0 tx=0).stop 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 -- 172.21.15.73:0/3872710074 shutdown_connections 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/3872710074 >> v2:172.21.15.157:3300/0 conn(0x7f6b9c06a1f0 0x7f6b9c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c070820 0x7f6b9c069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/3872710074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15ddb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 -- 172.21.15.73:0/3872710074 >> 172.21.15.73:0/3872710074 conn(0x7f6b9c06adc0 msgr2=0x7f6b9c06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 -- 172.21.15.73:0/3872710074 shutdown_connections 2024-04-05T22:22:41.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.712+0000 7f6ba2e00640 1 -- 172.21.15.73:0/3872710074 wait complete. 2024-04-05T22:22:41.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.714+0000 7f6ba2e00640 1 Processor -- start 2024-04-05T22:22:41.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.714+0000 7f6ba2e00640 1 -- start start 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c06a1f0 0x7f6b9c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 0x7f6b9c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c1589a0 con 0x7f6b9c157370 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c157d90 con 0x7f6b9c070820 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c158070 con 0x7f6b9c06a1f0 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37096/0 (socket says 172.21.15.73:37096) 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.715+0000 7f6ba0e00640 1 -- 172.21.15.73:0/2026115862 learned_addr learned my addr 172.21.15.73:0/2026115862 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b9be00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c06a1f0 0x7f6b9c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b9b400640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 0x7f6b9c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 -- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c06a1f0 msgr2=0x7f6b9c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c06a1f0 0x7f6b9c159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 -- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 msgr2=0x7f6b9c150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 0x7f6b9c150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 -- 172.21.15.73:0/2026115862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0094d0 con 0x7f6b9c157370 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b9b400640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 0x7f6b9c150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba0e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400cef0 tx=0x7f6b9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400d840 con 0x7f6b9c157370 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9400d9a0 con 0x7f6b9c157370 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400dd50 con 0x7f6b9c157370 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.716+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c158350 con 0x7f6b9c157370 2024-04-05T22:22:41.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.717+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c155e10 con 0x7f6b9c157370 2024-04-05T22:22:41.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.718+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9c067290 con 0x7f6b9c157370 2024-04-05T22:22:41.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.719+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b9401e3b0 con 0x7f6b9c157370 2024-04-05T22:22:41.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.719+0000 7f6b93e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 0x7f6b7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:41.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.719+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b94089de0 con 0x7f6b9c157370 2024-04-05T22:22:41.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.720+0000 7f6b9be00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 0x7f6b7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:41.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.721+0000 7f6b9be00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 0x7f6b7806d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c036000 tx=0x7f6b8c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:41.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.724+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6b94054fb0 con 0x7f6b9c157370 2024-04-05T22:22:41.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.950+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol52580", "group_name": "subvol_grp26820", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b9c0b2960 con 0x7f6b7806ac00 2024-04-05T22:22:41.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.960+0000 7f6b93e00640 1 -- 172.21.15.73:0/2026115862 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6b8c031800 con 0x7f6b7806ac00 2024-04-05T22:22:41.960 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp26820/subvol52580 2024-04-05T22:22:41.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 msgr2=0x7f6b7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 0x7f6b7806d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c036000 tx=0x7f6b8c03c040 comp rx=0 tx=0).stop 2024-04-05T22:22:41.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 msgr2=0x7f6b9c15f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:41.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400cef0 tx=0x7f6b9400a710 comp rx=0 tx=0).stop 2024-04-05T22:22:41.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 shutdown_connections 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6b7806ac00 0x7f6b7806d0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b9c157370 0x7f6b9c15f520 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b9c070820 0x7f6b9c150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 --2- 172.21.15.73:0/2026115862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b9c06a1f0 0x7f6b9c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 >> 172.21.15.73:0/2026115862 conn(0x7f6b9c06adc0 msgr2=0x7f6b9c0ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 shutdown_connections 2024-04-05T22:22:41.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:41.963+0000 7f6ba2e00640 1 -- 172.21.15.73:0/2026115862 wait complete. 2024-04-05T22:22:41.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:41.983 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/9df49a2130b9456cdf0bb1957611c780.meta) 2024-04-05T22:22:45.061 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:22:45.061 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:22:45.062 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:22:45.062 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/subvol_grp26820/subvol52580 2024-04-05T22:22:45.062 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:22:45.062 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:45.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol52580 subvol_grp26820 2024-04-05T22:22:45.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.272+0000 7f1fbd400640 1 Processor -- start 2024-04-05T22:22:45.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.272+0000 7f1fbd400640 1 -- start start 2024-04-05T22:22:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fbd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1fb81541a0 0x7f1fb80cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fbd400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 0x7f1fb80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fbd400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb815d900 con 0x7f1fb80b2840 2024-04-05T22:22:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fbd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb815e1a0 con 0x7f1fb81541a0 2024-04-05T22:22:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fbd400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb80b3490 con 0x7f1fb8154580 2024-04-05T22:22:45.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fb6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1fb81541a0 0x7f1fb80cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fb6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1fb81541a0 0x7f1fb80cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37112/0 (socket says 172.21.15.73:37112) 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.274+0000 7f1fb6800640 1 -- 172.21.15.73:0/354500412 learned_addr learned my addr 172.21.15.73:0/354500412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fade00640 1 --2- 172.21.15.73:0/354500412 >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 0x7f1fb80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fb6800640 1 -- 172.21.15.73:0/354500412 >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 msgr2=0x7f1fb80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fb6800640 1 --2- 172.21.15.73:0/354500412 >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 0x7f1fb80cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fb6800640 1 -- 172.21.15.73:0/354500412 >> v1:172.21.15.73:6789/0 conn(0x7f1fb80b2840 legacy=0x7f1fb80b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fb6800640 1 -- 172.21.15.73:0/354500412 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb815db50 con 0x7f1fb81541a0 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fb5e00640 1 -- 172.21.15.73:0/354500412 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526064227 0 0) 0x7f1fb815d900 con 0x7f1fb80b2840 2024-04-05T22:22:45.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.275+0000 7f1fade00640 1 --2- 172.21.15.73:0/354500412 >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 0x7f1fb80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:45.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.276+0000 7f1fb6800640 1 --2- 172.21.15.73:0/354500412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb80cecd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ce3d0 tx=0x7f1fa802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347db95c023261bc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.276+0000 7f1fb5e00640 1 -- 172.21.15.73:0/354500412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa802cb20 con 0x7f1fb81541a0 2024-04-05T22:22:45.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.276+0000 7f1fb5e00640 1 -- 172.21.15.73:0/354500412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa802cc80 con 0x7f1fb81541a0 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.276+0000 7f1fb5e00640 1 -- 172.21.15.73:0/354500412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80343f0 con 0x7f1fb81541a0 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 -- 172.21.15.73:0/354500412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 msgr2=0x7f1fb80cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 --2- 172.21.15.73:0/354500412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb80cecd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ce3d0 tx=0x7f1fa802bf80 comp rx=0 tx=0).stop 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 -- 172.21.15.73:0/354500412 shutdown_connections 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 --2- 172.21.15.73:0/354500412 >> v2:172.21.15.157:3300/0 conn(0x7f1fb8154580 0x7f1fb80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 --2- 172.21.15.73:0/354500412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb80cecd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 -- 172.21.15.73:0/354500412 >> 172.21.15.73:0/354500412 conn(0x7f1fb806a0d0 msgr2=0x7f1fb806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 -- 172.21.15.73:0/354500412 shutdown_connections 2024-04-05T22:22:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.277+0000 7f1fbd400640 1 -- 172.21.15.73:0/354500412 wait complete. 2024-04-05T22:22:45.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.279+0000 7f1fbd400640 1 Processor -- start 2024-04-05T22:22:45.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.279+0000 7f1fbd400640 1 -- start start 2024-04-05T22:22:45.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb80b2840 0x7f1fb8150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8154580 0x7f1fb80b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80b12f0 con 0x7f1fb81541a0 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80ad830 con 0x7f1fb80b2840 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fbd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80ad550 con 0x7f1fb8154580 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37120/0 (socket says 172.21.15.73:37120) 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 -- 172.21.15.73:0/964343667 learned_addr learned my addr 172.21.15.73:0/964343667 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 -- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8154580 msgr2=0x7f1fb80b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8154580 0x7f1fb80b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 -- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb80b2840 msgr2=0x7f1fb8150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb80b2840 0x7f1fb8150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.280+0000 7f1fade00640 1 -- 172.21.15.73:0/964343667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8009b30 con 0x7f1fb81541a0 2024-04-05T22:22:45.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1fade00640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c7c0 tx=0x7f1fa000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007820 con 0x7f1fb81541a0 2024-04-05T22:22:45.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0007980 con 0x7f1fb81541a0 2024-04-05T22:22:45.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb80af780 con 0x7f1fb81541a0 2024-04-05T22:22:45.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007d10 con 0x7f1fb81541a0 2024-04-05T22:22:45.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.281+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb80afd40 con 0x7f1fb81541a0 2024-04-05T22:22:45.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.282+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb8068160 con 0x7f1fb81541a0 2024-04-05T22:22:45.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.283+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 6) v1 ==== 89196+0+0 (secure 0 0 0) 0x7f1fa00113f0 con 0x7f1fb81541a0 2024-04-05T22:22:45.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.284+0000 7f1faf400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 0x7f1f9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.284+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fa00895c0 con 0x7f1fb81541a0 2024-04-05T22:22:45.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.284+0000 7f1fb6800640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 0x7f1f9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.285+0000 7f1fb6800640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 0x7f1f9406d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8035000 tx=0x7f1fa802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.289+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1fa0054630 con 0x7f1fb81541a0 2024-04-05T22:22:45.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.504+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol52580", "group_name": "subvol_grp26820", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fb815a310 con 0x7f1f9406ac90 2024-04-05T22:22:45.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.530+0000 7f1faf400640 1 -- 172.21.15.73:0/964343667 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1fa8009510 con 0x7f1f9406ac90 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 msgr2=0x7f1f9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 0x7f1f9406d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8035000 tx=0x7f1fa802e000 comp rx=0 tx=0).stop 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 msgr2=0x7f1fb8157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c7c0 tx=0x7f1fa000cc80 comp rx=0 tx=0).stop 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 shutdown_connections 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1f9406ac90 0x7f1f9406d150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8154580 0x7f1fb80b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb81541a0 0x7f1fb8157f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 --2- 172.21.15.73:0/964343667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb80b2840 0x7f1fb8150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.532+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 >> 172.21.15.73:0/964343667 conn(0x7f1fb806a0d0 msgr2=0x7f1fb815bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.533+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 shutdown_connections 2024-04-05T22:22:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.533+0000 7f1fbd400640 1 -- 172.21.15.73:0/964343667 wait complete. 2024-04-05T22:22:45.535 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.536+0000 7f5502a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.536+0000 7f5502a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.536+0000 7f5502a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.536+0000 7f5502a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.536+0000 7f5502a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:45.551 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:22:45.555 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.556+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.555 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.556+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.556 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.556+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.556 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.556+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.556 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:22:45.556+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:22:45.592 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:22:45.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp26820 2024-04-05T22:22:45.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.822+0000 7f281b200640 1 Processor -- start 2024-04-05T22:22:45.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.822+0000 7f281b200640 1 -- start start 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f281415abe0 0x7f28140712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406c9a0 0x7f281406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f281406ba70 0x7f28140cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281406bec0 con 0x7f281415abe0 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281415afc0 con 0x7f281406ba70 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.823+0000 7f281b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281406f5b0 con 0x7f281406c9a0 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f281415abe0 0x7f28140712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f281415abe0 0x7f28140712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37166/0 (socket says 172.21.15.73:37166) 2024-04-05T22:22:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 -- 172.21.15.73:0/2840281449 learned_addr learned my addr 172.21.15.73:0/2840281449 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f280b400640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406c9a0 0x7f281406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2819200640 1 --2- 172.21.15.73:0/2840281449 >> v2:172.21.15.157:3300/0 conn(0x7f281406ba70 0x7f28140cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 -- 172.21.15.73:0/2840281449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406c9a0 msgr2=0x7f281406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406c9a0 0x7f281406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 -- 172.21.15.73:0/2840281449 >> v2:172.21.15.157:3300/0 conn(0x7f281406ba70 msgr2=0x7f28140cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 --2- 172.21.15.73:0/2840281449 >> v2:172.21.15.157:3300/0 conn(0x7f281406ba70 0x7f28140cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.824+0000 7f2813e00640 1 -- 172.21.15.73:0/2840281449 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281406f890 con 0x7f281415abe0 2024-04-05T22:22:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.826+0000 7f2813e00640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140712e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2814070bc0 tx=0x7f280402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb757f83504a9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.826+0000 7f2813400640 1 -- 172.21.15.73:0/2840281449 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804031750 con 0x7f281415abe0 2024-04-05T22:22:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.826+0000 7f2813400640 1 -- 172.21.15.73:0/2840281449 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28040318b0 con 0x7f281415abe0 2024-04-05T22:22:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.826+0000 7f2813400640 1 -- 172.21.15.73:0/2840281449 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804031c40 con 0x7f281415abe0 2024-04-05T22:22:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 -- 172.21.15.73:0/2840281449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 msgr2=0x7f28140712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140712e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2814070bc0 tx=0x7f280402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 -- 172.21.15.73:0/2840281449 shutdown_connections 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 --2- 172.21.15.73:0/2840281449 >> v2:172.21.15.157:3300/0 conn(0x7f281406ba70 0x7f28140cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406c9a0 0x7f281406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 --2- 172.21.15.73:0/2840281449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140712e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 -- 172.21.15.73:0/2840281449 >> 172.21.15.73:0/2840281449 conn(0x7f2814069cb0 msgr2=0x7f281406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 -- 172.21.15.73:0/2840281449 shutdown_connections 2024-04-05T22:22:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.827+0000 7f281b200640 1 -- 172.21.15.73:0/2840281449 wait complete. 2024-04-05T22:22:45.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.829+0000 7f281b200640 1 Processor -- start 2024-04-05T22:22:45.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.829+0000 7f281b200640 1 -- start start 2024-04-05T22:22:45.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406ba70 0x7f28141497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 0x7f28140acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141521f0 con 0x7f281415abe0 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140aa2f0 con 0x7f281406c9a0 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f281b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2814151850 con 0x7f281406ba70 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41984/0 (socket says 172.21.15.73:41984) 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 -- 172.21.15.73:0/1900544927 learned_addr learned my addr 172.21.15.73:0/1900544927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f280b400640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 0x7f28140acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2813e00640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406ba70 0x7f28141497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 -- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406ba70 msgr2=0x7f28141497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406ba70 0x7f28141497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 -- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 msgr2=0x7f28140acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 0x7f28140acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f2819200640 1 -- 172.21.15.73:0/1900544927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040094d0 con 0x7f281415abe0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.831+0000 7f280b400640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 0x7f28140acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f2819200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f280c007e40 tx=0x7f280c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c014620 con 0x7f281415abe0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c00ee90 con 0x7f281415abe0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814151b30 con 0x7f281415abe0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0149e0 con 0x7f281415abe0 2024-04-05T22:22:45.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.832+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2814149ce0 con 0x7f281415abe0 2024-04-05T22:22:45.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.833+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814152330 con 0x7f281415abe0 2024-04-05T22:22:45.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.834+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 6) v1 ==== 89196+0+0 (secure 0 0 0) 0x7f280c014b40 con 0x7f281415abe0 2024-04-05T22:22:45.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.835+0000 7f2810c00640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 0x7f27f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:45.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.835+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f280c089d00 con 0x7f281415abe0 2024-04-05T22:22:45.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.836+0000 7f2813e00640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 0x7f27f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:45.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.836+0000 7f2813e00640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 0x7f27f006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2804035000 tx=0x7f2804008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:45.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:45.840+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2814152330 con 0x7f281415abe0 2024-04-05T22:22:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.057+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp26820", "target": ["mon-mgr", ""]}) v1 -- 0x7f2814070130 con 0x7f27f006ac90 2024-04-05T22:22:46.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.068+0000 7f2810c00640 1 -- 172.21.15.73:0/1900544927 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f28040430c0 con 0x7f27f006ac90 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 msgr2=0x7f27f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 0x7f27f006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2804035000 tx=0x7f2804008030 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 msgr2=0x7f28140ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f280c007e40 tx=0x7f280c00e4b0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 shutdown_connections 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27f006ac90 0x7f27f006d150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281415abe0 0x7f28140ad2d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281406c9a0 0x7f28140acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 --2- 172.21.15.73:0/1900544927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f281406ba70 0x7f28141497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.070+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 >> 172.21.15.73:0/1900544927 conn(0x7f2814069cb0 msgr2=0x7f28140ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:46.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.071+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 shutdown_connections 2024-04-05T22:22:46.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.071+0000 7f281b200640 1 -- 172.21.15.73:0/1900544927 wait complete. 2024-04-05T22:22:46.088 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:22:46.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:22:46.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:22:46.328 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:22:46.329 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:46.329 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:46.388 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:46.388 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:46.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.656+0000 7f6d87a00640 1 Processor -- start 2024-04-05T22:22:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.657+0000 7f6d87a00640 1 -- start start 2024-04-05T22:22:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.658+0000 7f6d87a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d800ce630 0x7f6d800cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.658+0000 7f6d87a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8006aa40 con 0x7f6d8015d200 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.658+0000 7f6d87a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8006ad20 con 0x7f6d8015ce20 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.658+0000 7f6d87a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d800ad6e0 con 0x7f6d800ce630 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.659+0000 7f6d85a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d800ce630 0x7f6d800cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.659+0000 7f6d77e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d8015d200 0x7f6d800acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37386/0 (socket says 172.21.15.73:37386) 2024-04-05T22:22:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.659+0000 7f6d77e00640 1 -- 172.21.15.73:0/2833861277 learned_addr learned my addr 172.21.15.73:0/2833861277 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.660+0000 7f6d85a00640 1 -- 172.21.15.73:0/2833861277 >> v1:172.21.15.157:6789/0 conn(0x7f6d8015ce20 legacy=0x7f6d800c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.660+0000 7f6d85a00640 1 -- 172.21.15.73:0/2833861277 >> v1:172.21.15.73:6789/0 conn(0x7f6d8015d200 legacy=0x7f6d800acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.660+0000 7f6d85a00640 1 -- 172.21.15.73:0/2833861277 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800cde30 con 0x7f6d800ce630 2024-04-05T22:22:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.662+0000 7f6d85a00640 1 --2- 172.21.15.73:0/2833861277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800cea70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cb7c5d35773ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.664+0000 7f6d7fe00640 1 -- 172.21.15.73:0/2833861277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7802ccf0 con 0x7f6d800ce630 2024-04-05T22:22:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.664+0000 7f6d7fe00640 1 -- 172.21.15.73:0/2833861277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7802ce50 con 0x7f6d800ce630 2024-04-05T22:22:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.664+0000 7f6d7fe00640 1 -- 172.21.15.73:0/2833861277 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78034660 con 0x7f6d800ce630 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 -- 172.21.15.73:0/2833861277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 msgr2=0x7f6d800cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 --2- 172.21.15.73:0/2833861277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800cea70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7802bf40 comp rx=0 tx=0).stop 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 -- 172.21.15.73:0/2833861277 shutdown_connections 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 --2- 172.21.15.73:0/2833861277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800cea70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 -- 172.21.15.73:0/2833861277 >> 172.21.15.73:0/2833861277 conn(0x7f6d800cf190 msgr2=0x7f6d800cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 -- 172.21.15.73:0/2833861277 shutdown_connections 2024-04-05T22:22:46.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.665+0000 7f6d87a00640 1 -- 172.21.15.73:0/2833861277 wait complete. 2024-04-05T22:22:46.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.667+0000 7f6d87a00640 1 Processor -- start 2024-04-05T22:22:46.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.667+0000 7f6d87a00640 1 -- start start 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8015ce20 0x7f6d80159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8015d200 0x7f6d800b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8009c7e0 con 0x7f6d800ce630 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800b38a0 con 0x7f6d8015ce20 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d87a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d801597c0 con 0x7f6d8015d200 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d85000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.668+0000 7f6d85000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42006/0 (socket says 172.21.15.73:42006) 2024-04-05T22:22:46.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 -- 172.21.15.73:0/1471324358 learned_addr learned my addr 172.21.15.73:0/1471324358 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:46.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 -- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8015d200 msgr2=0x7f6d800b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8015d200 0x7f6d800b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 -- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8015ce20 msgr2=0x7f6d80159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8015ce20 0x7f6d80159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 -- 172.21.15.73:0/1471324358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009a70 con 0x7f6d800ce630 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d85000640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c450 tx=0x7f6d7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.669+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700052d0 con 0x7f6d800ce630 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.670+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d70005430 con 0x7f6d800ce630 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.670+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70012870 con 0x7f6d800ce630 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.670+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d800ca3a0 con 0x7f6d800ce630 2024-04-05T22:22:46.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.670+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800b3e20 con 0x7f6d800ce630 2024-04-05T22:22:46.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.671+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d80068160 con 0x7f6d800ce630 2024-04-05T22:22:46.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.672+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f6d700129d0 con 0x7f6d800ce630 2024-04-05T22:22:46.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.672+0000 7f6d7d600640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 0x7f6d5c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:46.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.673+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d70089200 con 0x7f6d800ce630 2024-04-05T22:22:46.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.673+0000 7f6d77e00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 0x7f6d5c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:46.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.673+0000 7f6d77e00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 0x7f6d5c06d270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d80069f10 tx=0x7f6d68007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:46.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.677+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d70054140 con 0x7f6d800ce630 2024-04-05T22:22:46.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.872+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d8006a690 con 0x7f6d800ce630 2024-04-05T22:22:46.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.874+0000 7f6d7d600640 1 -- 172.21.15.73:0/1471324358 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f6d70057760 con 0x7f6d800ce630 2024-04-05T22:22:46.873 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:46.873 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.876+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 msgr2=0x7f6d5c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.876+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 0x7f6d5c06d270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d80069f10 tx=0x7f6d68007a40 comp rx=0 tx=0).stop 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.876+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 msgr2=0x7f6d800b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.876+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c450 tx=0x7f6d7000c910 comp rx=0 tx=0).stop 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 shutdown_connections 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6d5c06adb0 0x7f6d5c06d270 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8015d200 0x7f6d800b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8015ce20 0x7f6d80159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 --2- 172.21.15.73:0/1471324358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d800ce630 0x7f6d800b4900 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 >> 172.21.15.73:0/1471324358 conn(0x7f6d800cf190 msgr2=0x7f6d800acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 shutdown_connections 2024-04-05T22:22:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:46.877+0000 7f6d87a00640 1 -- 172.21.15.73:0/1471324358 wait complete. 2024-04-05T22:22:46.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:22:21.488491+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:22:22.636776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:46.896 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:47.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.102+0000 7f4cade00640 1 Processor -- start 2024-04-05T22:22:47.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.102+0000 7f4cade00640 1 -- start start 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.103+0000 7f4cade00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4ca815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4cade00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca815ddd0 con 0x7f4ca815ce10 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4cade00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca815e0b0 con 0x7f4ca80b0990 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4cade00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80ce370 con 0x7f4ca815d1f0 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4ca6a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4ca815ce10 0x7f4ca80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37388/0 (socket says 172.21.15.73:37388) 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4ca6a00640 1 -- 172.21.15.73:0/1590842089 learned_addr learned my addr 172.21.15.73:0/1590842089 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:47.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4c9fe00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4ca815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882704440 0 0) 0x7f4ca815ddd0 con 0x7f4ca815ce10 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.104+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c8c003520 con 0x7f4ca815ce10 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631376641 0 0) 0x7f4c8c003520 con 0x7f4ca815ce10 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 msgr2=0x7f4ca815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4ca815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 >> v1:172.21.15.157:6789/0 conn(0x7f4ca80b0990 legacy=0x7f4ca80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80cd460 con 0x7f4ca815ce10 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4c98001670 con 0x7f4ca815ce10 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 >> v1:172.21.15.73:6789/0 conn(0x7f4ca815ce10 legacy=0x7f4ca80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4c8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c8c0047c0 0x7f4c8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4c8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80ce370 con 0x7f4c8c0039a0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c00ba30 con 0x7f4c8c0047c0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c00bbf0 con 0x7f4ca815d1f0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4c8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.105+0000 7f4ca7400640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4c8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca6a00640 1 -- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 msgr2=0x7f4c8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4c8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca6a00640 1 -- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c8c0047c0 msgr2=0x7f4c8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c8c0047c0 0x7f4c8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca6a00640 1 -- 172.21.15.73:0/1590842089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c003520 con 0x7f4c8c0039a0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.106+0000 7f4ca7400640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca815d1f0 0x7f4c8c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4c8c003dc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c210 tx=0x7f4c8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ed3c204a595c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98005ad0 con 0x7f4c8c0039a0 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98005c30 con 0x7f4c8c0039a0 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4ca6000640 1 -- 172.21.15.73:0/1590842089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9802c870 con 0x7f4c8c0039a0 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4cade00640 1 -- 172.21.15.73:0/1590842089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 msgr2=0x7f4c8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4cade00640 1 --2- 172.21.15.73:0/1590842089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4c8c003dc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c210 tx=0x7f4c8c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4ca7400640 1 -- 172.21.15.73:0/1590842089 reap_dead start 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4cade00640 1 -- 172.21.15.73:0/1590842089 shutdown_connections 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.107+0000 7f4cade00640 1 -- 172.21.15.73:0/1590842089 >> 172.21.15.73:0/1590842089 conn(0x7f4ca8099a60 msgr2=0x7f4ca8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.108+0000 7f4cade00640 1 -- 172.21.15.73:0/1590842089 shutdown_connections 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.108+0000 7f4cade00640 1 -- 172.21.15.73:0/1590842089 wait complete. 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.109+0000 7f4cade00640 1 Processor -- start 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.109+0000 7f4cade00640 1 -- start start 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca80c52a0 0x7f4ca80cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8155e30 0x7f4ca815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80c5760 con 0x7f4c8c0039a0 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80c5a40 con 0x7f4ca8155e30 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.110+0000 7f4cade00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80c5d20 con 0x7f4ca80c52a0 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42024/0 (socket says 172.21.15.73:42024) 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 -- 172.21.15.73:0/3670165696 learned_addr learned my addr 172.21.15.73:0/3670165696 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4c9fe00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca80c52a0 0x7f4ca80cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 -- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca80c52a0 msgr2=0x7f4ca80cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca80c52a0 0x7f4ca80cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 -- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8155e30 msgr2=0x7f4ca815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8155e30 0x7f4ca815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 -- 172.21.15.73:0/3670165696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80c6000 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.111+0000 7f4ca6a00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802c410 tx=0x7f4c98001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.112+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98039070 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.112+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9802dd10 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.112+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca80c62e0 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.112+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9803c040 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.112+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca80d0a60 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.113+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8067480 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.114+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f4c98032070 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.115+0000 7f4c9f400640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 0x7f4c8406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.115+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c980aabc0 con 0x7f4c8c0039a0 2024-04-05T22:22:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.115+0000 7f4c9fe00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 0x7f4c8406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.116+0000 7f4c9fe00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 0x7f4c8406d270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ca815dc20 tx=0x7f4c90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.119+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4c98075c30 con 0x7f4c8c0039a0 2024-04-05T22:22:47.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.313+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ca8068160 con 0x7f4c8c0039a0 2024-04-05T22:22:47.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.314+0000 7f4c9f400640 1 -- 172.21.15.73:0/3670165696 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c98046020 con 0x7f4c8c0039a0 2024-04-05T22:22:47.314 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:47.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 msgr2=0x7f4c8406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 0x7f4c8406d270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ca815dc20 tx=0x7f4c90008040 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 msgr2=0x7f4ca80c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802c410 tx=0x7f4c98001a00 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 shutdown_connections 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4c8406adb0 0x7f4c8406d270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8155e30 0x7f4ca815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca80c52a0 0x7f4ca80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 --2- 172.21.15.73:0/3670165696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c8c0039a0 0x7f4ca80c6800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 >> 172.21.15.73:0/3670165696 conn(0x7f4ca8099a60 msgr2=0x7f4ca815e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 shutdown_connections 2024-04-05T22:22:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.317+0000 7f4cade00640 1 -- 172.21.15.73:0/3670165696 wait complete. 2024-04-05T22:22:47.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:47.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.547+0000 7fd0e3800640 1 Processor -- start 2024-04-05T22:22:47.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.548+0000 7fd0e3800640 1 -- start start 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0dc15abe0 0x7fd0dc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc06c9a0 0x7fd0dc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc06bec0 con 0x7fd0dc06ba70 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc15afc0 con 0x7fd0dc15abe0 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.549+0000 7fd0e3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc06f5b0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42038/0 (socket says 172.21.15.73:42038) 2024-04-05T22:22:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 -- 172.21.15.73:0/2363969342 learned_addr learned my addr 172.21.15.73:0/2363969342 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0d3e00640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc06c9a0 0x7fd0dc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 -- 172.21.15.73:0/2363969342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc06c9a0 msgr2=0x7fd0dc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc06c9a0 0x7fd0dc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 -- 172.21.15.73:0/2363969342 >> v2:172.21.15.157:3300/0 conn(0x7fd0dc15abe0 msgr2=0x7fd0dc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e0e00640 1 --2- 172.21.15.73:0/2363969342 >> v2:172.21.15.157:3300/0 conn(0x7fd0dc15abe0 0x7fd0dc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 --2- 172.21.15.73:0/2363969342 >> v2:172.21.15.157:3300/0 conn(0x7fd0dc15abe0 0x7fd0dc0712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.550+0000 7fd0e1800640 1 -- 172.21.15.73:0/2363969342 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc06f890 con 0x7fd0dc06ba70 2024-04-05T22:22:47.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.551+0000 7fd0e1800640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc06d300 tx=0x7fd0d402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6596c31b036ce741 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.551+0000 7fd0dbe00640 1 -- 172.21.15.73:0/2363969342 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d402c950 con 0x7fd0dc06ba70 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0dbe00640 1 -- 172.21.15.73:0/2363969342 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d402cab0 con 0x7fd0dc06ba70 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0dbe00640 1 -- 172.21.15.73:0/2363969342 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d402ce40 con 0x7fd0dc06ba70 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 -- 172.21.15.73:0/2363969342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06ba70 msgr2=0x7fd0dc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc06d300 tx=0x7fd0d402bdb0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 -- 172.21.15.73:0/2363969342 shutdown_connections 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc0cccb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2363969342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc06c9a0 0x7fd0dc06b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2363969342 >> v2:172.21.15.157:3300/0 conn(0x7fd0dc15abe0 0x7fd0dc0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 -- 172.21.15.73:0/2363969342 >> 172.21.15.73:0/2363969342 conn(0x7fd0dc069cb0 msgr2=0x7fd0dc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 -- 172.21.15.73:0/2363969342 shutdown_connections 2024-04-05T22:22:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.552+0000 7fd0e3800640 1 -- 172.21.15.73:0/2363969342 wait complete. 2024-04-05T22:22:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.554+0000 7fd0e3800640 1 Processor -- start 2024-04-05T22:22:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.554+0000 7fd0e3800640 1 -- start start 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc15abe0 0x7fd0dc15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1520a0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc15ca90 con 0x7fd0dc06ba70 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0e3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc151770 con 0x7fd0dc15abe0 2024-04-05T22:22:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42054/0 (socket says 172.21.15.73:42054) 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.555+0000 7fd0d3e00640 1 -- 172.21.15.73:0/2264037743 learned_addr learned my addr 172.21.15.73:0/2264037743 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0e1800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc15abe0 0x7fd0dc15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0e0e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 -- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc15abe0 msgr2=0x7fd0dc15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc15abe0 0x7fd0dc15bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 -- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 msgr2=0x7fd0dc153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 -- 172.21.15.73:0/2264037743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d4009b30 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0e0e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d3e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c7f0 tx=0x7fd0cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc004220 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc0043a0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc151a50 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc007860 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.556+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc15c2a0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.557+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc152210 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.558+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fd0cc007bd0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.559+0000 7fd0d9600640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 0x7fd0b806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.559+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0cc089780 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.559+0000 7fd0e0e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 0x7fd0b806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.560+0000 7fd0e0e00640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 0x7fd0b806d270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc153490 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:47.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.564+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd0cc0545c0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.769+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0dc068160 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.770+0000 7fd0d9600640 1 -- 172.21.15.73:0/2264037743 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0cc057be0 con 0x7fd0dc06c9a0 2024-04-05T22:22:47.769 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 msgr2=0x7fd0b806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 0x7fd0b806d270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc153490 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2024-04-05T22:22:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 msgr2=0x7fd0dc153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c7f0 tx=0x7fd0cc00ccb0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 shutdown_connections 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd0b806adb0 0x7fd0b806d270 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0dc15abe0 0x7fd0dc15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0dc06c9a0 0x7fd0dc153d70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 --2- 172.21.15.73:0/2264037743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0dc06ba70 0x7fd0dc153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 >> 172.21.15.73:0/2264037743 conn(0x7fd0dc069cb0 msgr2=0x7fd0dc0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 shutdown_connections 2024-04-05T22:22:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.773+0000 7fd0e3800640 1 -- 172.21.15.73:0/2264037743 wait complete. 2024-04-05T22:22:47.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:47.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.997+0000 7fc9d7400640 1 Processor -- start 2024-04-05T22:22:47.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.997+0000 7fc9d7400640 1 -- start start 2024-04-05T22:22:47.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9d006c630 0x7fc9d006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00cbe00 0x7fc9d00ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d00b22a0 0x7fc9d00b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d00b1b30 con 0x7fc9d00b22a0 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d00b1e10 con 0x7fc9d006c630 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d006cf50 con 0x7fc9d00cbe00 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:47.999+0000 7fc9d5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d00b22a0 0x7fc9d00b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9d00b22a0 0x7fc9d00b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42070/0 (socket says 172.21.15.73:42070) 2024-04-05T22:22:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 -- 172.21.15.73:0/3318768102 learned_addr learned my addr 172.21.15.73:0/3318768102 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d4a00640 1 --2- 172.21.15.73:0/3318768102 >> v2:172.21.15.157:3300/0 conn(0x7fc9d006c630 0x7fc9d006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9c7e00640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00cbe00 0x7fc9d00ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 -- 172.21.15.73:0/3318768102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00cbe00 msgr2=0x7fc9d00ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00cbe00 0x7fc9d00ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 -- 172.21.15.73:0/3318768102 >> v2:172.21.15.157:3300/0 conn(0x7fc9d006c630 msgr2=0x7fc9d006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3318768102 >> v2:172.21.15.157:3300/0 conn(0x7fc9d006c630 0x7fc9d006ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.000+0000 7fc9d5400640 1 -- 172.21.15.73:0/3318768102 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d00cc300 con 0x7fc9d00b22a0 2024-04-05T22:22:48.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.001+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00b22a0 0x7fc9d00b2730 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9d006f360 tx=0x7fc9c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4796b1eed269f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.001+0000 7fc9cfe00640 1 -- 172.21.15.73:0/3318768102 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c802ca70 con 0x7fc9d00b22a0 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.001+0000 7fc9cfe00640 1 -- 172.21.15.73:0/3318768102 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c802cbd0 con 0x7fc9d00b22a0 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9cfe00640 1 -- 172.21.15.73:0/3318768102 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80343f0 con 0x7fc9d00b22a0 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 -- 172.21.15.73:0/3318768102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00b22a0 msgr2=0x7fc9d00b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00b22a0 0x7fc9d00b2730 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9d006f360 tx=0x7fc9c802bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 -- 172.21.15.73:0/3318768102 shutdown_connections 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00b22a0 0x7fc9d00b2730 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3318768102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00cbe00 0x7fc9d00ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3318768102 >> v2:172.21.15.157:3300/0 conn(0x7fc9d006c630 0x7fc9d006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 -- 172.21.15.73:0/3318768102 >> 172.21.15.73:0/3318768102 conn(0x7fc9d00b0830 msgr2=0x7fc9d00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:48.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 -- 172.21.15.73:0/3318768102 shutdown_connections 2024-04-05T22:22:48.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.002+0000 7fc9d7400640 1 -- 172.21.15.73:0/3318768102 wait complete. 2024-04-05T22:22:48.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.004+0000 7fc9d7400640 1 Processor -- start 2024-04-05T22:22:48.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.005+0000 7fc9d7400640 1 -- start start 2024-04-05T22:22:48.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d006c630 0x7fc9d00c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00b22a0 0x7fc9d0133390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00c3fe0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d015ddd0 con 0x7fc9d006c630 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d015d7c0 con 0x7fc9d00b22a0 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42072/0 (socket says 172.21.15.73:42072) 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 -- 172.21.15.73:0/3105810414 learned_addr learned my addr 172.21.15.73:0/3105810414 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 -- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00b22a0 msgr2=0x7fc9d0133390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00b22a0 0x7fc9d0133390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 -- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d006c630 msgr2=0x7fc9d00c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d006c630 0x7fc9d00c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.006+0000 7fc9d5400640 1 -- 172.21.15.73:0/3105810414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8009b30 con 0x7fc9d00cbe00 2024-04-05T22:22:48.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9d5400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8036040 tx=0x7fc9c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c803e450 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c803e5d0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d00b36d0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c803e980 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.007+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d00b3c90 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.008+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d00b0c10 con 0x7fc9d00cbe00 2024-04-05T22:22:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.009+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc9c803eae0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.010+0000 7fc9cd600640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 0x7fc9ac06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.010+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c80ac610 con 0x7fc9d00cbe00 2024-04-05T22:22:48.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.011+0000 7fc9d4a00640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 0x7fc9ac06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.011+0000 7fc9d4a00640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 0x7fc9ac06d270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006fd0 tx=0x7fc9b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.015+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc9c80774d0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.208+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9d0068160 con 0x7fc9d00cbe00 2024-04-05T22:22:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.208+0000 7fc9cd600640 1 -- 172.21.15.73:0/3105810414 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9c807aaf0 con 0x7fc9d00cbe00 2024-04-05T22:22:48.207 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 msgr2=0x7fc9ac06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 0x7fc9ac06d270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006fd0 tx=0x7fc9b8008040 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 msgr2=0x7fc9d01f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8036040 tx=0x7fc9c8004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 shutdown_connections 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc9ac06adb0 0x7fc9ac06d270 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9d00cbe00 0x7fc9d01f55b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9d00b22a0 0x7fc9d0133390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3105810414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d006c630 0x7fc9d00c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 >> 172.21.15.73:0/3105810414 conn(0x7fc9d00b0830 msgr2=0x7fc9d00af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 shutdown_connections 2024-04-05T22:22:48.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.211+0000 7fc9d7400640 1 -- 172.21.15.73:0/3105810414 wait complete. 2024-04-05T22:22:48.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:22:48.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.444+0000 7f0b39800640 1 Processor -- start 2024-04-05T22:22:48.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.444+0000 7f0b39800640 1 -- start start 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.445+0000 7f0b39800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b340b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.445+0000 7f0b39800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b3415aaa0 0x7f0b34156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b39800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b340b3ea0 con 0x7f0b3406d440 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b39800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3415e2c0 con 0x7f0b3415aaa0 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b39800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3415d500 con 0x7f0b3406d060 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b38e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b3406d440 0x7f0b3406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37404/0 (socket says 172.21.15.73:37404) 2024-04-05T22:22:48.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b38e00640 1 -- 172.21.15.73:0/2135601508 learned_addr learned my addr 172.21.15.73:0/2135601508 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276861872 0 0) 0x7f0b340b3ea0 con 0x7f0b3406d440 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b14003520 con 0x7f0b3406d440 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b33e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b340b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.446+0000 7f0b33400640 1 --2- 172.21.15.73:0/2135601508 >> v2:172.21.15.157:3300/0 conn(0x7f0b3415aaa0 0x7f0b34156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083041065 0 0) 0x7f0b14003520 con 0x7f0b3406d440 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 msgr2=0x7f0b340b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b340b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 >> v2:172.21.15.157:3300/0 conn(0x7f0b3415aaa0 msgr2=0x7f0b34156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 --2- 172.21.15.73:0/2135601508 >> v2:172.21.15.157:3300/0 conn(0x7f0b3415aaa0 0x7f0b34156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3415d7e0 con 0x7f0b3406d440 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0b2c001670 con 0x7f0b3406d440 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 >> v1:172.21.15.73:6789/0 conn(0x7f0b3406d440 legacy=0x7f0b3406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b14003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b14003fa0 0x7f0b140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b14004d00 0x7f0b14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3415e2c0 con 0x7f0b14004d00 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3415d500 con 0x7f0b14003fa0 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1400ba80 con 0x7f0b3406d060 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b38e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b14004d00 0x7f0b14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b33e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b14003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.447+0000 7f0b33400640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b14003fa0 0x7f0b140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b38e00640 1 -- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 msgr2=0x7f0b14003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b38e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b14003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b38e00640 1 -- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b14003fa0 msgr2=0x7f0b140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b38e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b14003fa0 0x7f0b140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b38e00640 1 -- 172.21.15.73:0/2135601508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14003520 con 0x7f0b14004d00 2024-04-05T22:22:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b33e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b3406d060 0x7f0b14003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.448+0000 7f0b33400640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b14003fa0 0x7f0b140047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.449+0000 7f0b38e00640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b14004d00 0x7f0b14005180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c210 tx=0x7f0b140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d54ee0cb06ac9c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.449+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c005ad0 con 0x7f0b14004d00 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.449+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c005c30 con 0x7f0b14004d00 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.449+0000 7f0b32a00640 1 -- 172.21.15.73:0/2135601508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c02c8d0 con 0x7f0b14004d00 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 -- 172.21.15.73:0/2135601508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b14004d00 msgr2=0x7f0b14005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 --2- 172.21.15.73:0/2135601508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b14004d00 0x7f0b14005180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c210 tx=0x7f0b140057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b38e00640 1 -- 172.21.15.73:0/2135601508 reap_dead start 2024-04-05T22:22:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 -- 172.21.15.73:0/2135601508 shutdown_connections 2024-04-05T22:22:48.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 -- 172.21.15.73:0/2135601508 >> 172.21.15.73:0/2135601508 conn(0x7f0b340b0830 msgr2=0x7f0b340b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:48.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 -- 172.21.15.73:0/2135601508 shutdown_connections 2024-04-05T22:22:48.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.450+0000 7f0b39800640 1 -- 172.21.15.73:0/2135601508 wait complete. 2024-04-05T22:22:48.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.452+0000 7f0b39800640 1 Processor -- start 2024-04-05T22:22:48.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.452+0000 7f0b39800640 1 -- start start 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34164a40 0x7f0b34164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b34144870 0x7f0b34142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34165360 con 0x7f0b3414b660 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b341436f0 con 0x7f0b34164a40 2024-04-05T22:22:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b39800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b341577b0 con 0x7f0b34144870 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42092/0 (socket says 172.21.15.73:42092) 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 -- 172.21.15.73:0/474336811 learned_addr learned my addr 172.21.15.73:0/474336811 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b33e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34164a40 0x7f0b34164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 -- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b34144870 msgr2=0x7f0b34142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b33400640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b34144870 0x7f0b34142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b34144870 0x7f0b34142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 -- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34164a40 msgr2=0x7f0b34164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.453+0000 7f0b38e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34164a40 0x7f0b34164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b38e00640 1 -- 172.21.15.73:0/474336811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34157a90 con 0x7f0b3414b660 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b38e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000c00 tx=0x7f0b3415d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004aa0 con 0x7f0b3414b660 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c004c00 con 0x7f0b3414b660 2024-04-05T22:22:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c001640 con 0x7f0b3414b660 2024-04-05T22:22:48.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b34157d70 con 0x7f0b3414b660 2024-04-05T22:22:48.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.454+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b341433e0 con 0x7f0b3414b660 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.455+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34158290 con 0x7f0b3414b660 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.456+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f0b2c032070 con 0x7f0b3414b660 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.457+0000 7f0b2be00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 0x7f0b1006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.457+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b2c03c070 con 0x7f0b3414b660 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.457+0000 7f0b33e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 0x7f0b1006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.458+0000 7f0b33e00640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 0x7f0b1006d270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006fd0 tx=0x7f0b24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:48.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.461+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0b34158290 con 0x7f0b3414b660 2024-04-05T22:22:48.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.672+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b34158290 con 0x7f0b3414b660 2024-04-05T22:22:48.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.673+0000 7f0b2be00640 1 -- 172.21.15.73:0/474336811 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b34158290 con 0x7f0b3414b660 2024-04-05T22:22:48.672 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 msgr2=0x7f0b1006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 0x7f0b1006d270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006fd0 tx=0x7f0b24008040 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 msgr2=0x7f0b340b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000c00 tx=0x7f0b3415d3f0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 shutdown_connections 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b1006adb0 0x7f0b1006d270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b3414b660 0x7f0b340b2ae0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b34144870 0x7f0b34142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 --2- 172.21.15.73:0/474336811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b34164a40 0x7f0b34164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 >> 172.21.15.73:0/474336811 conn(0x7f0b340b0830 msgr2=0x7f0b34070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 shutdown_connections 2024-04-05T22:22:48.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:48.676+0000 7f0b39800640 1 -- 172.21.15.73:0/474336811 wait complete. 2024-04-05T22:22:48.694 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:22:48.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:48.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:22:48.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:22:48.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:48.837 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:22:48.884 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:48.885 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:48.924 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:48.924 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:49.010 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:49.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.216+0000 7f772ce00640 1 Processor -- start 2024-04-05T22:22:49.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.216+0000 7f772ce00640 1 -- start start 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f772ce00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 0x7f772806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f772ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772815e850 0x7f772815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f772ce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772806f5b0 con 0x7f77281587d0 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f772ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772806d500 con 0x7f772815e850 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f772ce00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772815f220 con 0x7f772806cb80 2024-04-05T22:22:49.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f7726800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772815e850 0x7f772815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f7726800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f772815e850 0x7f772815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42102/0 (socket says 172.21.15.73:42102) 2024-04-05T22:22:49.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f7726800640 1 -- 172.21.15.73:0/592987854 learned_addr learned my addr 172.21.15.73:0/592987854 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:49.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.218+0000 7f7725400640 1 --2- 172.21.15.73:0/592987854 >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 0x7f772806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.219+0000 7f7726800640 1 -- 172.21.15.73:0/592987854 >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 msgr2=0x7f772806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:49.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.219+0000 7f7726800640 1 --2- 172.21.15.73:0/592987854 >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 0x7f772806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.219+0000 7f7726800640 1 -- 172.21.15.73:0/592987854 >> v1:172.21.15.157:6789/0 conn(0x7f77281587d0 legacy=0x7f77280cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:49.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.219+0000 7f7726800640 1 -- 172.21.15.73:0/592987854 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7728070410 con 0x7f772815e850 2024-04-05T22:22:49.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.219+0000 7f7725400640 1 --2- 172.21.15.73:0/592987854 >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 0x7f772806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:49.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.220+0000 7f7726800640 1 --2- 172.21.15.73:0/592987854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f772815ece0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f772806b560 tx=0x7f772002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f079e865652ce6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.220+0000 7f7724a00640 1 -- 172.21.15.73:0/592987854 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772002ca80 con 0x7f772815e850 2024-04-05T22:22:49.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.220+0000 7f7724a00640 1 -- 172.21.15.73:0/592987854 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772002cbe0 con 0x7f772815e850 2024-04-05T22:22:49.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.220+0000 7f7724a00640 1 -- 172.21.15.73:0/592987854 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200343f0 con 0x7f772815e850 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 -- 172.21.15.73:0/592987854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 msgr2=0x7f772815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 --2- 172.21.15.73:0/592987854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f772815ece0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f772806b560 tx=0x7f772002bee0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 -- 172.21.15.73:0/592987854 shutdown_connections 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 --2- 172.21.15.73:0/592987854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f772815ece0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 --2- 172.21.15.73:0/592987854 >> v2:172.21.15.157:3300/0 conn(0x7f772806cb80 0x7f772806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 -- 172.21.15.73:0/592987854 >> 172.21.15.73:0/592987854 conn(0x7f77280ce8e0 msgr2=0x7f77280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 -- 172.21.15.73:0/592987854 shutdown_connections 2024-04-05T22:22:49.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.221+0000 7f772ce00640 1 -- 172.21.15.73:0/592987854 wait complete. 2024-04-05T22:22:49.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.222+0000 7f772ce00640 1 Processor -- start 2024-04-05T22:22:49.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.223+0000 7f772ce00640 1 -- start start 2024-04-05T22:22:49.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.223+0000 7f772ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772806cb80 0x7f7728146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f772ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77281587d0 0x7f77280ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f772ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f772ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280adae0 con 0x7f772815e850 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f772ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728146fa0 con 0x7f772806cb80 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f772ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7728146700 con 0x7f77281587d0 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42106/0 (socket says 172.21.15.73:42106) 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 -- 172.21.15.73:0/2555879351 learned_addr learned my addr 172.21.15.73:0/2555879351 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7725e00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772806cb80 0x7f7728146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 -- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77281587d0 msgr2=0x7f77280ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77281587d0 0x7f77280ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 -- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772806cb80 msgr2=0x7f7728146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772806cb80 0x7f7728146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.224+0000 7f7726800640 1 -- 172.21.15.73:0/2555879351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720009b30 con 0x7f772815e850 2024-04-05T22:22:49.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f7726800640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7720036040 tx=0x7f7720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720040670 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77200407f0 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77281469e0 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720040ba0 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.225+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728144110 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.226+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7728066af0 con 0x7f772815e850 2024-04-05T22:22:49.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.227+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f772002cd50 con 0x7f772815e850 2024-04-05T22:22:49.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.228+0000 7f771e000640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 0x7f770406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.228+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77200ac590 con 0x7f772815e850 2024-04-05T22:22:49.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.228+0000 7f7725e00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 0x7f770406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.229+0000 7f7725e00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 0x7f770406d270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7718003d90 tx=0x7f7718009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.232+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f77200774e0 con 0x7f772815e850 2024-04-05T22:22:49.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.430+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7728144300 con 0x7f772815e850 2024-04-05T22:22:49.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.432+0000 7f771e000640 1 -- 172.21.15.73:0/2555879351 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f772007ab00 con 0x7f772815e850 2024-04-05T22:22:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:49.431 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.434+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 msgr2=0x7f770406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.434+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 0x7f770406d270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7718003d90 tx=0x7f7718009040 comp rx=0 tx=0).stop 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.434+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 msgr2=0x7f7728070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.434+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7720036040 tx=0x7f7720004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 shutdown_connections 2024-04-05T22:22:49.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f770406adb0 0x7f770406d270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f772815e850 0x7f7728070b10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77281587d0 0x7f77280ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 --2- 172.21.15.73:0/2555879351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f772806cb80 0x7f7728146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 >> 172.21.15.73:0/2555879351 conn(0x7f77280ce8e0 msgr2=0x7f77280ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 shutdown_connections 2024-04-05T22:22:49.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.435+0000 7f772ce00640 1 -- 172.21.15.73:0/2555879351 wait complete. 2024-04-05T22:22:49.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:22:21.488491+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:22:22.636776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:49.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:49.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.666+0000 7ff66a000640 1 Processor -- start 2024-04-05T22:22:49.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.666+0000 7ff66a000640 1 -- start start 2024-04-05T22:22:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.667+0000 7ff66a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff66415abe0 0x7ff6640712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff66a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff66a000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66415afc0 con 0x7ff66406c9a0 2024-04-05T22:22:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff66a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66406f5b0 con 0x7ff66415abe0 2024-04-05T22:22:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff66a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66406f890 con 0x7ff66406bc40 2024-04-05T22:22:49.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff663e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff663e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36092/0 (socket says 172.21.15.73:36092) 2024-04-05T22:22:49.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff663e00640 1 -- 172.21.15.73:0/849811961 learned_addr learned my addr 172.21.15.73:0/849811961 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:49.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.668+0000 7ff662a00640 1 --1- 172.21.15.73:0/849811961 >> v1:172.21.15.157:6789/0 conn(0x7ff66406c9a0 0x7ff66406b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:58658/0 (socket says 172.21.15.73:58658) 2024-04-05T22:22:49.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.669+0000 7ff663400640 1 --2- 172.21.15.73:0/849811961 >> v2:172.21.15.157:3300/0 conn(0x7ff66415abe0 0x7ff6640712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.669+0000 7ff663e00640 1 -- 172.21.15.73:0/849811961 >> v2:172.21.15.157:3300/0 conn(0x7ff66415abe0 msgr2=0x7ff6640712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:49.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.669+0000 7ff663e00640 1 --2- 172.21.15.73:0/849811961 >> v2:172.21.15.157:3300/0 conn(0x7ff66415abe0 0x7ff6640712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.669+0000 7ff663e00640 1 -- 172.21.15.73:0/849811961 >> v1:172.21.15.157:6789/0 conn(0x7ff66406c9a0 legacy=0x7ff66406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.669+0000 7ff663e00640 1 -- 172.21.15.73:0/849811961 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66406fb70 con 0x7ff66406bc40 2024-04-05T22:22:49.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.671+0000 7ff663e00640 1 --2- 172.21.15.73:0/849811961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009e50 tx=0x7ff65c02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6171a3ced1adb163 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.675+0000 7ff662000640 1 -- 172.21.15.73:0/849811961 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c02c9e0 con 0x7ff66406bc40 2024-04-05T22:22:49.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.675+0000 7ff662000640 1 -- 172.21.15.73:0/849811961 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c02cb40 con 0x7ff66406bc40 2024-04-05T22:22:49.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.675+0000 7ff662000640 1 -- 172.21.15.73:0/849811961 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c0343f0 con 0x7ff66406bc40 2024-04-05T22:22:49.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 -- 172.21.15.73:0/849811961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 msgr2=0x7ff664164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 --2- 172.21.15.73:0/849811961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009e50 tx=0x7ff65c02be40 comp rx=0 tx=0).stop 2024-04-05T22:22:49.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 -- 172.21.15.73:0/849811961 shutdown_connections 2024-04-05T22:22:49.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 --2- 172.21.15.73:0/849811961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66406bc40 0x7ff664164740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 --2- 172.21.15.73:0/849811961 >> v2:172.21.15.157:3300/0 conn(0x7ff66415abe0 0x7ff6640712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 -- 172.21.15.73:0/849811961 >> 172.21.15.73:0/849811961 conn(0x7ff664069cb0 msgr2=0x7ff66406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:49.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 -- 172.21.15.73:0/849811961 shutdown_connections 2024-04-05T22:22:49.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.676+0000 7ff66a000640 1 -- 172.21.15.73:0/849811961 wait complete. 2024-04-05T22:22:49.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.678+0000 7ff66a000640 1 Processor -- start 2024-04-05T22:22:49.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.678+0000 7ff66a000640 1 -- start start 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 0x7ff664156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66415abe0 0x7ff6641571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641498b0 con 0x7ff66406bc40 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66413e0c0 con 0x7ff66406c9a0 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff66a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66413e3a0 con 0x7ff66415abe0 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff663400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff663400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42114/0 (socket says 172.21.15.73:42114) 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff663400640 1 -- 172.21.15.73:0/660052208 learned_addr learned my addr 172.21.15.73:0/660052208 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.679+0000 7ff662a00640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 0x7ff664156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663e00640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66415abe0 0x7ff6641571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 -- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66415abe0 msgr2=0x7ff6641571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66415abe0 0x7ff6641571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 -- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 msgr2=0x7ff664156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 0x7ff664156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 -- 172.21.15.73:0/660052208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c009b30 con 0x7ff66406bc40 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff662a00640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 0x7ff664156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:49.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff663400640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9b0 tx=0x7ff65400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654009670 con 0x7ff66406bc40 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65400c9a0 con 0x7ff66406bc40 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400cd50 con 0x7ff66406bc40 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66413e680 con 0x7ff66406bc40 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.680+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66413e960 con 0x7ff66406bc40 2024-04-05T22:22:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.682+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664068160 con 0x7ff66406bc40 2024-04-05T22:22:49.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.682+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff654013470 con 0x7ff66406bc40 2024-04-05T22:22:49.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.683+0000 7ff65b400640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 0x7ff64406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:49.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.683+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff654089880 con 0x7ff66406bc40 2024-04-05T22:22:49.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.684+0000 7ff662a00640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 0x7ff64406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:49.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.684+0000 7ff662a00640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 0x7ff64406d270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6641f3ab0 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:49.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.688+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff654054870 con 0x7ff66406bc40 2024-04-05T22:22:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.881+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6641499f0 con 0x7ff66406bc40 2024-04-05T22:22:49.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.882+0000 7ff65b400640 1 -- 172.21.15.73:0/660052208 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6641499f0 con 0x7ff66406bc40 2024-04-05T22:22:49.881 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 msgr2=0x7ff64406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 0x7ff64406d270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6641f3ab0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 msgr2=0x7ff6640a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9b0 tx=0x7ff65400ee70 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 shutdown_connections 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff64406adb0 0x7ff64406d270 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff66415abe0 0x7ff6641571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66406c9a0 0x7ff664156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 --2- 172.21.15.73:0/660052208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66406bc40 0x7ff6640a8600 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.884+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 >> 172.21.15.73:0/660052208 conn(0x7ff664069cb0 msgr2=0x7ff6640ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.885+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 shutdown_connections 2024-04-05T22:22:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:49.885+0000 7ff66a000640 1 -- 172.21.15.73:0/660052208 wait complete. 2024-04-05T22:22:49.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:50.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.115+0000 7f0e7fc00640 1 Processor -- start 2024-04-05T22:22:50.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.116+0000 7f0e7fc00640 1 -- start start 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.117+0000 7f0e7fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e7815abe0 0x7f0e780712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.117+0000 7f0e7fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7815afc0 con 0x7f0e7806c9a0 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.117+0000 7f0e7fc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7806f5b0 con 0x7f0e7806bc40 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.117+0000 7f0e7fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7806f890 con 0x7f0e7815abe0 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.117+0000 7f0e7d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e7815abe0 0x7f0e780712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e6fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e7806c9a0 0x7f0e7806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37406/0 (socket says 172.21.15.73:37406) 2024-04-05T22:22:50.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e6fe00640 1 -- 172.21.15.73:0/2241812298 learned_addr learned my addr 172.21.15.73:0/2241812298 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:50.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e7d200640 1 -- 172.21.15.73:0/2241812298 >> v1:172.21.15.157:6789/0 conn(0x7f0e7806bc40 legacy=0x7f0e78164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e7d200640 1 -- 172.21.15.73:0/2241812298 >> v1:172.21.15.73:6789/0 conn(0x7f0e7806c9a0 legacy=0x7f0e7806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e7d200640 1 -- 172.21.15.73:0/2241812298 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7806fb70 con 0x7f0e7815abe0 2024-04-05T22:22:50.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.118+0000 7f0e77e00640 1 -- 172.21.15.73:0/2241812298 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965495404 0 0) 0x7f0e7815afc0 con 0x7f0e7806c9a0 2024-04-05T22:22:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.119+0000 7f0e7d200640 1 --2- 172.21.15.73:0/2241812298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7815abe0 0x7f0e780712e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76811e13d3b8af95 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e77e00640 1 -- 172.21.15.73:0/2241812298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6802e070 con 0x7f0e7815abe0 2024-04-05T22:22:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e77e00640 1 -- 172.21.15.73:0/2241812298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e680315a0 con 0x7f0e7815abe0 2024-04-05T22:22:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e77e00640 1 -- 172.21.15.73:0/2241812298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68031930 con 0x7f0e7815abe0 2024-04-05T22:22:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 -- 172.21.15.73:0/2241812298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7815abe0 msgr2=0x7f0e780712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/2241812298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7815abe0 0x7f0e780712e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b840 comp rx=0 tx=0).stop 2024-04-05T22:22:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 -- 172.21.15.73:0/2241812298 shutdown_connections 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/2241812298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7815abe0 0x7f0e780712e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 -- 172.21.15.73:0/2241812298 >> 172.21.15.73:0/2241812298 conn(0x7f0e78069cb0 msgr2=0x7f0e7806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 -- 172.21.15.73:0/2241812298 shutdown_connections 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.120+0000 7f0e7fc00640 1 -- 172.21.15.73:0/2241812298 wait complete. 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.122+0000 7f0e7fc00640 1 Processor -- start 2024-04-05T22:22:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.122+0000 7f0e7fc00640 1 -- start start 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 0x7f0e781448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 0x7f0e7814c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7814b2a0 con 0x7f0e7806bc40 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7814b580 con 0x7f0e7806c9a0 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7814b860 con 0x7f0e7815abe0 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42124/0 (socket says 172.21.15.73:42124) 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.123+0000 7f0e7d200640 1 -- 172.21.15.73:0/996994801 learned_addr learned my addr 172.21.15.73:0/996994801 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e6fe00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 0x7f0e781448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7dc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 0x7f0e7814c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 -- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 msgr2=0x7f0e7814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 0x7f0e7814c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 -- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 msgr2=0x7f0e781448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 0x7f0e781448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 -- 172.21.15.73:0/996994801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68009400 con 0x7f0e7806bc40 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e6fe00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 0x7f0e781448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7dc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 0x7f0e7814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e7d200640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.124+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004220 con 0x7f0e7806bc40 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.125+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e680043c0 con 0x7f0e7806bc40 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.125+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7814bb40 con 0x7f0e7806bc40 2024-04-05T22:22:50.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.125+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68032880 con 0x7f0e7806bc40 2024-04-05T22:22:50.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.125+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7814aec0 con 0x7f0e7806bc40 2024-04-05T22:22:50.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.126+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7814c060 con 0x7f0e7806bc40 2024-04-05T22:22:50.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.127+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f0e680329e0 con 0x7f0e7806bc40 2024-04-05T22:22:50.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.127+0000 7f0e75600640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 0x7f0e5406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.128+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e680ab830 con 0x7f0e7806bc40 2024-04-05T22:22:50.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.128+0000 7f0e6fe00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 0x7f0e5406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.128+0000 7f0e6fe00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 0x7f0e5406d2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e78148ab0 tx=0x7f0e60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.132+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0e7814c060 con 0x7f0e7806bc40 2024-04-05T22:22:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.324+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e7814c060 con 0x7f0e7806bc40 2024-04-05T22:22:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.325+0000 7f0e75600640 1 -- 172.21.15.73:0/996994801 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e7814c060 con 0x7f0e7806bc40 2024-04-05T22:22:50.325 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:50.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 msgr2=0x7f0e5406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 0x7f0e5406d2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e78148ab0 tx=0x7f0e60009040 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 msgr2=0x7f0e78145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e680046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 shutdown_connections 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0e5406ade0 0x7f0e5406d2a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e7815abe0 0x7f0e7814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e7806c9a0 0x7f0e781448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 --2- 172.21.15.73:0/996994801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e7806bc40 0x7f0e78145140 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 >> 172.21.15.73:0/996994801 conn(0x7f0e78069cb0 msgr2=0x7f0e78070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 shutdown_connections 2024-04-05T22:22:50.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.328+0000 7f0e7fc00640 1 -- 172.21.15.73:0/996994801 wait complete. 2024-04-05T22:22:50.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:50.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.554+0000 7f1e63200640 1 Processor -- start 2024-04-05T22:22:50.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.554+0000 7f1e63200640 1 -- start start 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.555+0000 7f1e63200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e5c06c630 0x7f1e5c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.555+0000 7f1e63200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.555+0000 7f1e63200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0b1b30 con 0x7f1e5c0cbe00 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.555+0000 7f1e63200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b1e10 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.555+0000 7f1e63200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c0ce7b0 con 0x7f1e5c06c630 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42134/0 (socket says 172.21.15.73:42134) 2024-04-05T22:22:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 -- 172.21.15.73:0/3470378747 learned_addr learned my addr 172.21.15.73:0/3470378747 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 -- 172.21.15.73:0/3470378747 >> v2:172.21.15.157:3300/0 conn(0x7f1e5c06c630 msgr2=0x7f1e5c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 --2- 172.21.15.73:0/3470378747 >> v2:172.21.15.157:3300/0 conn(0x7f1e5c06c630 0x7f1e5c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 -- 172.21.15.73:0/3470378747 >> v1:172.21.15.157:6789/0 conn(0x7f1e5c0cbe00 legacy=0x7f1e5c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.556+0000 7f1e61200640 1 -- 172.21.15.73:0/3470378747 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c06cf50 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.557+0000 7f1e61200640 1 --2- 172.21.15.73:0/3470378747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06f360 tx=0x7f1e5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b71fb9fc2dbee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.557+0000 7f1e5b400640 1 -- 172.21.15.73:0/3470378747 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54031870 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.557+0000 7f1e5b400640 1 -- 172.21.15.73:0/3470378747 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540319d0 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.557+0000 7f1e5b400640 1 -- 172.21.15.73:0/3470378747 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54031d60 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 -- 172.21.15.73:0/3470378747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c0b32b0 msgr2=0x7f1e5c1647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 --2- 172.21.15.73:0/3470378747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06f360 tx=0x7f1e5402b910 comp rx=0 tx=0).stop 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 -- 172.21.15.73:0/3470378747 shutdown_connections 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 --2- 172.21.15.73:0/3470378747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c1647c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 --2- 172.21.15.73:0/3470378747 >> v2:172.21.15.157:3300/0 conn(0x7f1e5c06c630 0x7f1e5c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 -- 172.21.15.73:0/3470378747 >> 172.21.15.73:0/3470378747 conn(0x7f1e5c0b0830 msgr2=0x7f1e5c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 -- 172.21.15.73:0/3470378747 shutdown_connections 2024-04-05T22:22:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.558+0000 7f1e63200640 1 -- 172.21.15.73:0/3470378747 wait complete. 2024-04-05T22:22:50.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.560+0000 7f1e63200640 1 Processor -- start 2024-04-05T22:22:50.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.560+0000 7f1e63200640 1 -- start start 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 0x7f1e5c13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c140e10 con 0x7f1e5c06c630 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c140ac0 con 0x7f1e5c0b32b0 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e63200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0a9410 con 0x7f1e5c0cbe00 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42150/0 (socket says 172.21.15.73:42150) 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.561+0000 7f1e5be00640 1 -- 172.21.15.73:0/1620319493 learned_addr learned my addr 172.21.15.73:0/1620319493 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e61200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 0x7f1e5c13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e53400640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 -- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 msgr2=0x7f1e5c13c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 0x7f1e5c13c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 -- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e5c0b32b0 msgr2=0x7f1e5c0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c0ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 -- 172.21.15.73:0/1620319493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540094d0 con 0x7f1e5c06c630 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e61200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 0x7f1e5c13c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e5be00640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b4d0 tx=0x7f1e4c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.562+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010870 con 0x7f1e5c06c630 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.563+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c0109d0 con 0x7f1e5c06c630 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.563+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0a96e0 con 0x7f1e5c06c630 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.563+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010d80 con 0x7f1e5c06c630 2024-04-05T22:22:50.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.563+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0a9ca0 con 0x7f1e5c06c630 2024-04-05T22:22:50.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.564+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c140f50 con 0x7f1e5c06c630 2024-04-05T22:22:50.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.565+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f1e4c00f3c0 con 0x7f1e5c06c630 2024-04-05T22:22:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.566+0000 7f1e58c00640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 0x7f1e3806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.566+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c089700 con 0x7f1e5c06c630 2024-04-05T22:22:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.566+0000 7f1e53400640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 0x7f1e3806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.567+0000 7f1e53400640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 0x7f1e3806d270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cbd70 tx=0x7f1e440072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:50.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.570+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1e5c140f50 con 0x7f1e5c06c630 2024-04-05T22:22:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.767+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e5c140f50 con 0x7f1e5c06c630 2024-04-05T22:22:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.768+0000 7f1e58c00640 1 -- 172.21.15.73:0/1620319493 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e5c140f50 con 0x7f1e5c06c630 2024-04-05T22:22:50.768 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:50.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 msgr2=0x7f1e3806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 0x7f1e3806d270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cbd70 tx=0x7f1e440072f0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 msgr2=0x7f1e5c0a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b4d0 tx=0x7f1e4c00b990 comp rx=0 tx=0).stop 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 shutdown_connections 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e3806adb0 0x7f1e3806d270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5c0cbe00 0x7f1e5c13c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e5c0b32b0 0x7f1e5c0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 --2- 172.21.15.73:0/1620319493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e5c06c630 0x7f1e5c0a8ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 >> 172.21.15.73:0/1620319493 conn(0x7f1e5c0b0830 msgr2=0x7f1e5c15b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 shutdown_connections 2024-04-05T22:22:50.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.771+0000 7f1e63200640 1 -- 172.21.15.73:0/1620319493 wait complete. 2024-04-05T22:22:50.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:22:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.997+0000 7f5e78800640 1 Processor -- start 2024-04-05T22:22:50.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.997+0000 7f5e78800640 1 -- start start 2024-04-05T22:22:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e78800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5e7015ef60 0x7f5e7006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e78800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7006f5b0 con 0x7f5e7006cb80 2024-04-05T22:22:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e78800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e70070410 con 0x7f5e701587d0 2024-04-05T22:22:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e78800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7006d500 con 0x7f5e7015ef60 2024-04-05T22:22:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e75400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5e7006cb80 0x7f5e7015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37416/0 (socket says 172.21.15.73:37416) 2024-04-05T22:22:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:50.999+0000 7f5e75400640 1 -- 172.21.15.73:0/656377569 learned_addr learned my addr 172.21.15.73:0/656377569 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> v2:172.21.15.157:3300/0 conn(0x7f5e7015ef60 0x7f5e7006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950188822 0 0) 0x7f5e7006f5b0 con 0x7f5e7006cb80 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e58003520 con 0x7f5e7006cb80 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873312857 0 0) 0x7f5e58003520 con 0x7f5e7006cb80 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 >> v2:172.21.15.157:3300/0 conn(0x7f5e7015ef60 msgr2=0x7f5e7006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 --2- 172.21.15.73:0/656377569 >> v2:172.21.15.157:3300/0 conn(0x7f5e7015ef60 0x7f5e7006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 >> v1:172.21.15.157:6789/0 conn(0x7f5e701587d0 legacy=0x7f5e700cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700ccef0 con 0x7f5e7006cb80 2024-04-05T22:22:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5e60001670 con 0x7f5e7006cb80 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> v2:172.21.15.157:3300/0 conn(0x7f5e7015ef60 0x7f5e7006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 >> v1:172.21.15.73:6789/0 conn(0x7f5e7006cb80 legacy=0x7f5e7015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e58003970 0x7f5e58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e580047c0 0x7f5e58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e5800fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7006d500 con 0x7f5e58005180 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580104e0 con 0x7f5e580047c0 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.000+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580106a0 con 0x7f5e58003970 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e5800fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e75e00640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e58003970 0x7f5e58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e75400640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e580047c0 0x7f5e58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 -- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e58003970 msgr2=0x7f5e58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e58003970 0x7f5e58003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 -- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e580047c0 msgr2=0x7f5e58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e580047c0 0x7f5e58004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e76800640 1 -- 172.21.15.73:0/656377569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58003520 con 0x7f5e58005180 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.001+0000 7f5e75400640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e580047c0 0x7f5e58004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.002+0000 7f5e76800640 1 -- 172.21.15.73:0/656377569 reap_dead start 2024-04-05T22:22:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.002+0000 7f5e76800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e5800fd90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0073e0 tx=0x7f5e580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c67e7e5e48014 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.002+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02cdc0 con 0x7f5e58005180 2024-04-05T22:22:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.002+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c034440 con 0x7f5e58005180 2024-04-05T22:22:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.002+0000 7f5e74a00640 1 -- 172.21.15.73:0/656377569 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0347d0 con 0x7f5e58005180 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 -- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 msgr2=0x7f5e5800fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e5800fd90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0073e0 tx=0x7f5e580057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 -- 172.21.15.73:0/656377569 shutdown_connections 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 --2- 172.21.15.73:0/656377569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e5800fd90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 -- 172.21.15.73:0/656377569 >> 172.21.15.73:0/656377569 conn(0x7f5e700ce8e0 msgr2=0x7f5e700cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 -- 172.21.15.73:0/656377569 shutdown_connections 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.003+0000 7f5e78800640 1 -- 172.21.15.73:0/656377569 wait complete. 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.005+0000 7f5e78800640 1 Processor -- start 2024-04-05T22:22:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.005+0000 7f5e78800640 1 -- start start 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 0x7f5e7015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7015b5a0 0x7f5e7015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015ba20 con 0x7f5e58005180 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015c390 con 0x7f5e7015b5a0 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e78800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e700ad180 con 0x7f5e7015a6a0 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e75e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e75e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42172/0 (socket says 172.21.15.73:42172) 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e75e00640 1 -- 172.21.15.73:0/1611111676 learned_addr learned my addr 172.21.15.73:0/1611111676 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.006+0000 7f5e76800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7015b5a0 0x7f5e7015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75400640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 0x7f5e7015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 -- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 msgr2=0x7f5e7015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 0x7f5e7015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 -- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7015b5a0 msgr2=0x7f5e7015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7015b5a0 0x7f5e7015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 -- 172.21.15.73:0/1611111676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7015c5e0 con 0x7f5e58005180 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75400640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 0x7f5e7015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e75e00640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400d570 tx=0x7f5e6400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64005480 con 0x7f5e58005180 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64003610 con 0x7f5e58005180 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64003980 con 0x7f5e58005180 2024-04-05T22:22:51.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.007+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7015c8c0 con 0x7f5e58005180 2024-04-05T22:22:51.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.008+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e70153900 con 0x7f5e58005180 2024-04-05T22:22:51.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.009+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e70066af0 con 0x7f5e58005180 2024-04-05T22:22:51.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.009+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f5e64003ae0 con 0x7f5e58005180 2024-04-05T22:22:51.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.010+0000 7f5e6a000640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 0x7f5e5406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.010+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e64089590 con 0x7f5e58005180 2024-04-05T22:22:51.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.011+0000 7f5e75400640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 0x7f5e5406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.011+0000 7f5e75400640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 0x7f5e5406d270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e70157b40 tx=0x7f5e60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.015+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5e640543e0 con 0x7f5e58005180 2024-04-05T22:22:51.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.223+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e70153a90 con 0x7f5e58005180 2024-04-05T22:22:51.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.224+0000 7f5e6a000640 1 -- 172.21.15.73:0/1611111676 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5e64057a00 con 0x7f5e58005180 2024-04-05T22:22:51.223 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:22:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 msgr2=0x7f5e5406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 0x7f5e5406d270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e70157b40 tx=0x7f5e60009040 comp rx=0 tx=0).stop 2024-04-05T22:22:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 msgr2=0x7f5e700acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400d570 tx=0x7f5e6400da30 comp rx=0 tx=0).stop 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 shutdown_connections 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e5406adb0 0x7f5e5406d270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7015b5a0 0x7f5e7015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e7015a6a0 0x7f5e7015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 --2- 172.21.15.73:0/1611111676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e58005180 0x7f5e700acba0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 >> 172.21.15.73:0/1611111676 conn(0x7f5e700ce8e0 msgr2=0x7f5e700ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 shutdown_connections 2024-04-05T22:22:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.227+0000 7f5e78800640 1 -- 172.21.15.73:0/1611111676 wait complete. 2024-04-05T22:22:51.246 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:22:51.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:51.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:22:51.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:22:51.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:22:51.362 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:22:51.412 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:22:51.413 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:51.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:22:51.469 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:22:51.469 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:22:51.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:22:51.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:51.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.743+0000 7fc60cc00640 1 Processor -- start 2024-04-05T22:22:51.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.743+0000 7fc60cc00640 1 -- start start 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.744+0000 7fc60cc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc60815eab0 0x7fc60806c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.744+0000 7fc60cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60806b500 con 0x7fc60815e620 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc60cc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080b20d0 con 0x7fc608158860 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc60cc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080ccaf0 con 0x7fc60815eab0 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc5ffe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc60815e620 0x7fc60806bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37420/0 (socket says 172.21.15.73:37420) 2024-04-05T22:22:51.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc5ffe00640 1 -- 172.21.15.73:0/2127303796 learned_addr learned my addr 172.21.15.73:0/2127303796 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc607e00640 1 --2- 172.21.15.73:0/2127303796 >> v2:172.21.15.157:3300/0 conn(0x7fc60815eab0 0x7fc60806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795785589 0 0) 0x7fc60806b500 con 0x7fc60815e620 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.745+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5ec003520 con 0x7fc60815e620 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134158391 0 0) 0x7fc5ec003520 con 0x7fc60815e620 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 >> v2:172.21.15.157:3300/0 conn(0x7fc60815eab0 msgr2=0x7fc60806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 --2- 172.21.15.73:0/2127303796 >> v2:172.21.15.157:3300/0 conn(0x7fc60815eab0 0x7fc60806c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 >> v1:172.21.15.157:6789/0 conn(0x7fc608158860 legacy=0x7fc6080b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080b23a0 con 0x7fc60815e620 2024-04-05T22:22:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc5f0001670 con 0x7fc60815e620 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 >> v1:172.21.15.73:6789/0 conn(0x7fc60815e620 legacy=0x7fc60806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 0x7fc5ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5ec0047c0 0x7fc5ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec005180 0x7fc5ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080ccaf0 con 0x7fc5ec003970 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec00c120 con 0x7fc5ec0047c0 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec00c2e0 con 0x7fc5ec005180 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.746+0000 7fc607400640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 0x7fc5ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc5ffe00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5ec0047c0 0x7fc5ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607400640 1 -- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec005180 msgr2=0x7fc5ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607400640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec005180 0x7fc5ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607400640 1 -- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5ec0047c0 msgr2=0x7fc5ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607400640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5ec0047c0 0x7fc5ec004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607400640 1 -- 172.21.15.73:0/2127303796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec003520 con 0x7fc5ec003970 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.747+0000 7fc607e00640 1 -- 172.21.15.73:0/2127303796 reap_dead start 2024-04-05T22:22:51.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc607400640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 0x7fc5ec003d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8001750 tx=0x7fc5ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa849ffa4847aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80052d0 con 0x7fc5ec003970 2024-04-05T22:22:51.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f802e050 con 0x7fc5ec003970 2024-04-05T22:22:51.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc606a00640 1 -- 172.21.15.73:0/2127303796 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003830 con 0x7fc5ec003970 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc60cc00640 1 -- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 msgr2=0x7fc5ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.748+0000 7fc60cc00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 0x7fc5ec003d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8001750 tx=0x7fc5ec0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.749+0000 7fc60cc00640 1 -- 172.21.15.73:0/2127303796 shutdown_connections 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.749+0000 7fc60cc00640 1 --2- 172.21.15.73:0/2127303796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ec003970 0x7fc5ec003d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.749+0000 7fc60cc00640 1 -- 172.21.15.73:0/2127303796 >> 172.21.15.73:0/2127303796 conn(0x7fc6080b06c0 msgr2=0x7fc6080b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.749+0000 7fc60cc00640 1 -- 172.21.15.73:0/2127303796 shutdown_connections 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.749+0000 7fc60cc00640 1 -- 172.21.15.73:0/2127303796 wait complete. 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.751+0000 7fc60cc00640 1 Processor -- start 2024-04-05T22:22:51.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.751+0000 7fc60cc00640 1 -- start start 2024-04-05T22:22:51.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 0x7fc60814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc608151d50 0x7fc6081521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc608152710 con 0x7fc6081546d0 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080ae8d0 con 0x7fc608151d50 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc60cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc608155240 con 0x7fc5ec003970 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42190/0 (socket says 172.21.15.73:42190) 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 -- 172.21.15.73:0/4192784828 learned_addr learned my addr 172.21.15.73:0/4192784828 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc607400640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 0x7fc60814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc607e00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc608151d50 0x7fc6081521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 -- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 msgr2=0x7fc60814efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 0x7fc60814efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 -- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc608151d50 msgr2=0x7fc6081521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc608151d50 0x7fc6081521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc5ffe00640 1 -- 172.21.15.73:0/4192784828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608155520 con 0x7fc6081546d0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.752+0000 7fc607400640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 0x7fc60814efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc5ffe00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0000ff0 tx=0x7fc5f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00074e0 con 0x7fc6081546d0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0007640 con 0x7fc6081546d0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0004a20 con 0x7fc6081546d0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc608155800 con 0x7fc6081546d0 2024-04-05T22:22:51.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.753+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6081508f0 con 0x7fc6081546d0 2024-04-05T22:22:51.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.754+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60804e160 con 0x7fc6081546d0 2024-04-05T22:22:51.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.755+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc5f0004b80 con 0x7fc6081546d0 2024-04-05T22:22:51.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.756+0000 7fc5ff400640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 0x7fc5e406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:51.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.757+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5f0088fa0 con 0x7fc6081546d0 2024-04-05T22:22:51.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.757+0000 7fc607400640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 0x7fc5e406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:51.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.758+0000 7fc607400640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 0x7fc5e406d270 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8033000 tx=0x7fc5f8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:51.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.761+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc5f00bc650 con 0x7fc6081546d0 2024-04-05T22:22:51.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.991+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6080b13f0 con 0x7fc6081546d0 2024-04-05T22:22:51.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.991+0000 7fc5ff400640 1 -- 172.21.15.73:0/4192784828 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fc5f0019020 con 0x7fc6081546d0 2024-04-05T22:22:51.991 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:51.991 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:51.991 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 16 2024-04-05T22:22:51.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 msgr2=0x7fc5e406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 0x7fc5e406d270 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8033000 tx=0x7fc5f8009030 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 msgr2=0x7fc608154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0000ff0 tx=0x7fc5f000dc00 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 shutdown_connections 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc5e406adb0 0x7fc5e406d270 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc608151d50 0x7fc6081521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6081546d0 0x7fc608154b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 --2- 172.21.15.73:0/4192784828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec003970 0x7fc60814efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 >> 172.21.15.73:0/4192784828 conn(0x7fc6080b06c0 msgr2=0x7fc60806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 shutdown_connections 2024-04-05T22:22:51.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:51.994+0000 7fc60cc00640 1 -- 172.21.15.73:0/4192784828 wait complete. 2024-04-05T22:22:52.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:52.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.218+0000 7f6e08c00640 1 Processor -- start 2024-04-05T22:22:52.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.218+0000 7f6e08c00640 1 -- start start 2024-04-05T22:22:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.219+0000 7f6e08c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e041587d0 0x7f6e040cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.219+0000 7f6e08c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.220+0000 7f6e08c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0406f5b0 con 0x7f6e0406d0c0 2024-04-05T22:22:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.220+0000 7f6e08c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e04070410 con 0x7f6e041587d0 2024-04-05T22:22:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.220+0000 7f6e08c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04158bb0 con 0x7f6e040ccd20 2024-04-05T22:22:52.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.221+0000 7f6e01400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.221+0000 7f6e01400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36174/0 (socket says 172.21.15.73:36174) 2024-04-05T22:22:52.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.221+0000 7f6e01400640 1 -- 172.21.15.73:0/2302395509 learned_addr learned my addr 172.21.15.73:0/2302395509 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:52.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.221+0000 7f6e01e00640 1 --2- 172.21.15.73:0/2302395509 >> v2:172.21.15.157:3300/0 conn(0x7f6e041587d0 0x7f6e040cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.221+0000 7f6e00a00640 1 -- 172.21.15.73:0/2302395509 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409050492 0 0) 0x7f6e0406f5b0 con 0x7f6e0406d0c0 2024-04-05T22:22:52.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.222+0000 7f6e00a00640 1 -- 172.21.15.73:0/2302395509 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de8003520 con 0x7f6e0406d0c0 2024-04-05T22:22:52.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.222+0000 7f6e01400640 1 -- 172.21.15.73:0/2302395509 >> v2:172.21.15.157:3300/0 conn(0x7f6e041587d0 msgr2=0x7f6e040cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:52.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.222+0000 7f6e01400640 1 --2- 172.21.15.73:0/2302395509 >> v2:172.21.15.157:3300/0 conn(0x7f6e041587d0 0x7f6e040cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.222+0000 7f6e01400640 1 -- 172.21.15.73:0/2302395509 >> v1:172.21.15.157:6789/0 conn(0x7f6e0406d0c0 legacy=0x7f6e0415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.222+0000 7f6e01400640 1 -- 172.21.15.73:0/2302395509 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0406f800 con 0x7f6e040ccd20 2024-04-05T22:22:52.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.223+0000 7f6e01400640 1 --2- 172.21.15.73:0/2302395509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009870 tx=0x7f6df002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ebe9037671f801 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e00a00640 1 -- 172.21.15.73:0/2302395509 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0030840 con 0x7f6e040ccd20 2024-04-05T22:22:52.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e00a00640 1 -- 172.21.15.73:0/2302395509 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df00309a0 con 0x7f6e040ccd20 2024-04-05T22:22:52.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e00a00640 1 -- 172.21.15.73:0/2302395509 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0030d10 con 0x7f6e040ccd20 2024-04-05T22:22:52.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 -- 172.21.15.73:0/2302395509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 msgr2=0x7f6e0406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 --2- 172.21.15.73:0/2302395509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009870 tx=0x7f6df002b980 comp rx=0 tx=0).stop 2024-04-05T22:22:52.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 -- 172.21.15.73:0/2302395509 shutdown_connections 2024-04-05T22:22:52.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 --2- 172.21.15.73:0/2302395509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040ccd20 0x7f6e0406cb80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 --2- 172.21.15.73:0/2302395509 >> v2:172.21.15.157:3300/0 conn(0x7f6e041587d0 0x7f6e040cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.224+0000 7f6e08c00640 1 -- 172.21.15.73:0/2302395509 >> 172.21.15.73:0/2302395509 conn(0x7f6e040ce8e0 msgr2=0x7f6e040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:52.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.225+0000 7f6e08c00640 1 -- 172.21.15.73:0/2302395509 shutdown_connections 2024-04-05T22:22:52.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.225+0000 7f6e08c00640 1 -- 172.21.15.73:0/2302395509 wait complete. 2024-04-05T22:22:52.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.227+0000 7f6e08c00640 1 Processor -- start 2024-04-05T22:22:52.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.227+0000 7f6e08c00640 1 -- start start 2024-04-05T22:22:52.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0406d0c0 0x7f6e0415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040a84c0 0x7f6e040ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04141f70 con 0x7f6e041587d0 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0413f8f0 con 0x7f6e0406d0c0 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e08c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040abea0 con 0x7f6e040a84c0 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e01400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e01400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42200/0 (socket says 172.21.15.73:42200) 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e01400640 1 -- 172.21.15.73:0/234301784 learned_addr learned my addr 172.21.15.73:0/234301784 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.228+0000 7f6e01e00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0406d0c0 0x7f6e0415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e02800640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040a84c0 0x7f6e040ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 -- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040a84c0 msgr2=0x7f6e040ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040a84c0 0x7f6e040ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 -- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0406d0c0 msgr2=0x7f6e0415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0406d0c0 0x7f6e0415cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 -- 172.21.15.73:0/234301784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00094d0 con 0x7f6e041587d0 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6e01400640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6df002b9c0 tx=0x7f6df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df003e070 con 0x7f6e041587d0 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df00043f0 con 0x7f6e041587d0 2024-04-05T22:22:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.229+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0003850 con 0x7f6e041587d0 2024-04-05T22:22:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.230+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0415d9b0 con 0x7f6e041587d0 2024-04-05T22:22:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.230+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040ab850 con 0x7f6e041587d0 2024-04-05T22:22:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.231+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04066af0 con 0x7f6e041587d0 2024-04-05T22:22:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.232+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f6df0042070 con 0x7f6e041587d0 2024-04-05T22:22:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.233+0000 7f6dfa000640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 0x7f6de406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.233+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df00ab2f0 con 0x7f6e041587d0 2024-04-05T22:22:52.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.233+0000 7f6e01e00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 0x7f6de406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.234+0000 7f6e01e00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 0x7f6de406d270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009a10 tx=0x7f6df4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.238+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6df00760a0 con 0x7f6e041587d0 2024-04-05T22:22:52.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.430+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e040ab6b0 con 0x7f6e041587d0 2024-04-05T22:22:52.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.433+0000 7f6dfa000640 1 -- 172.21.15.73:0/234301784 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f6e040ab6b0 con 0x7f6e041587d0 2024-04-05T22:22:52.432 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:52.432 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.435+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 msgr2=0x7f6de406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.435+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 0x7f6de406d270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009a10 tx=0x7f6df4008040 comp rx=0 tx=0).stop 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.435+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 msgr2=0x7f6e0415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.435+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6df002b9c0 tx=0x7f6df00046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 shutdown_connections 2024-04-05T22:22:52.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6de406adb0 0x7f6de406d270 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e040a84c0 0x7f6e040ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e041587d0 0x7f6e0415d2e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 --2- 172.21.15.73:0/234301784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0406d0c0 0x7f6e0415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 >> 172.21.15.73:0/234301784 conn(0x7f6e040ce8e0 msgr2=0x7f6e040adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 shutdown_connections 2024-04-05T22:22:52.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.436+0000 7f6e08c00640 1 -- 172.21.15.73:0/234301784 wait complete. 2024-04-05T22:22:52.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:22:21.488491+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:22:22.636776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:22:52.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:22:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.670+0000 7fdf66000640 1 Processor -- start 2024-04-05T22:22:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.670+0000 7fdf66000640 1 -- start start 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.671+0000 7fdf66000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf600ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf66000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf60157410 0x7fdf60160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf66000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf600cb0b0 0x7fdf600cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf66000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600cba60 con 0x7fdf600cb0b0 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf66000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6006d800 con 0x7fdf60157410 2024-04-05T22:22:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf66000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600b3090 con 0x7fdf60157030 2024-04-05T22:22:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf65600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf600cb0b0 0x7fdf600cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf65600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf600cb0b0 0x7fdf600cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42212/0 (socket says 172.21.15.73:42212) 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf65600640 1 -- 172.21.15.73:0/2294392710 learned_addr learned my addr 172.21.15.73:0/2294392710 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf64c00640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf600ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.672+0000 7fdf5fe00640 1 --2- 172.21.15.73:0/2294392710 >> v2:172.21.15.157:3300/0 conn(0x7fdf60157410 0x7fdf60160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.673+0000 7fdf65600640 1 -- 172.21.15.73:0/2294392710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 msgr2=0x7fdf600ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.673+0000 7fdf65600640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf600ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.673+0000 7fdf65600640 1 -- 172.21.15.73:0/2294392710 >> v2:172.21.15.157:3300/0 conn(0x7fdf60157410 msgr2=0x7fdf60160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.673+0000 7fdf65600640 1 --2- 172.21.15.73:0/2294392710 >> v2:172.21.15.157:3300/0 conn(0x7fdf60157410 0x7fdf60160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.673+0000 7fdf65600640 1 -- 172.21.15.73:0/2294392710 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600b3370 con 0x7fdf600cb0b0 2024-04-05T22:22:52.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.674+0000 7fdf65600640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf600cb520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cf740 tx=0x7fdf5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2f3fdd9368bf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.674+0000 7fdf5f400640 1 -- 172.21.15.73:0/2294392710 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5802ca70 con 0x7fdf600cb0b0 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.674+0000 7fdf5f400640 1 -- 172.21.15.73:0/2294392710 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5802cbd0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.674+0000 7fdf5f400640 1 -- 172.21.15.73:0/2294392710 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf580343f0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 -- 172.21.15.73:0/2294392710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 msgr2=0x7fdf600cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf600cb520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cf740 tx=0x7fdf5802bed0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 -- 172.21.15.73:0/2294392710 shutdown_connections 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf600cb520 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 --2- 172.21.15.73:0/2294392710 >> v2:172.21.15.157:3300/0 conn(0x7fdf60157410 0x7fdf60160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 --2- 172.21.15.73:0/2294392710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf600ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 -- 172.21.15.73:0/2294392710 >> 172.21.15.73:0/2294392710 conn(0x7fdf60069cb0 msgr2=0x7fdf6006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 -- 172.21.15.73:0/2294392710 shutdown_connections 2024-04-05T22:22:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.675+0000 7fdf66000640 1 -- 172.21.15.73:0/2294392710 wait complete. 2024-04-05T22:22:52.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.677+0000 7fdf66000640 1 Processor -- start 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.677+0000 7fdf66000640 1 -- start start 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf60158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf60157410 0x7fdf6015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60148ba0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60158560 con 0x7fdf60157410 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf66000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60147f90 con 0x7fdf60157030 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42218/0 (socket says 172.21.15.73:42218) 2024-04-05T22:22:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 -- 172.21.15.73:0/747009713 learned_addr learned my addr 172.21.15.73:0/747009713 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 -- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 msgr2=0x7fdf60158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf60158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 -- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf60157410 msgr2=0x7fdf6015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf60157410 0x7fdf6015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.678+0000 7fdf64c00640 1 -- 172.21.15.73:0/747009713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58009b30 con 0x7fdf600cb0b0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf64c00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400d800 tx=0x7fdf5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540096a0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54009800 con 0x7fdf600cb0b0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54009b90 con 0x7fdf600cb0b0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf60148270 con 0x7fdf600cb0b0 2024-04-05T22:22:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.679+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60148830 con 0x7fdf600cb0b0 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.680+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6004e160 con 0x7fdf600cb0b0 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.681+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdf54009cf0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.682+0000 7fdf5cc00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 0x7fdf4406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.682+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf54089d80 con 0x7fdf600cb0b0 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.682+0000 7fdf5fe00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 0x7fdf4406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.683+0000 7fdf5fe00640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 0x7fdf4406d2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:52.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.686+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdf54054ce0 con 0x7fdf600cb0b0 2024-04-05T22:22:52.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.880+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf600ad220 con 0x7fdf600cb0b0 2024-04-05T22:22:52.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.880+0000 7fdf5cc00640 1 -- 172.21.15.73:0/747009713 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf54058300 con 0x7fdf600cb0b0 2024-04-05T22:22:52.880 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:22:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 msgr2=0x7fdf4406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 0x7fdf4406d2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 msgr2=0x7fdf6015f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400d800 tx=0x7fdf5400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 shutdown_connections 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdf4406ade0 0x7fdf4406d2a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf60157410 0x7fdf6015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf60157030 0x7fdf60158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 --2- 172.21.15.73:0/747009713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf600cb0b0 0x7fdf6015f370 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 >> 172.21.15.73:0/747009713 conn(0x7fdf60069cb0 msgr2=0x7fdf600ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 shutdown_connections 2024-04-05T22:22:52.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:52.883+0000 7fdf66000640 1 -- 172.21.15.73:0/747009713 wait complete. 2024-04-05T22:22:52.902 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:22:53.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.116+0000 7f0477e00640 1 Processor -- start 2024-04-05T22:22:53.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.116+0000 7f0477e00640 1 -- start start 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.117+0000 7f0477e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f04700712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.117+0000 7f0477e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f047006c9a0 0x7f047006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0477e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047006bec0 con 0x7f047006ba70 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0477e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047015afc0 con 0x7f047006c9a0 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0477e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047006f5b0 con 0x7f047015abe0 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0475e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f047006ba70 0x7f04700cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37422/0 (socket says 172.21.15.73:37422) 2024-04-05T22:22:53.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0475e00640 1 -- 172.21.15.73:0/1958113415 learned_addr learned my addr 172.21.15.73:0/1958113415 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> v2:172.21.15.157:3300/0 conn(0x7f047006c9a0 0x7f047006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f0475400640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f04700712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029637805 0 0) 0x7f0468001490 con 0x7f047006ba70 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.118+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0458003520 con 0x7f047006ba70 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281734646 0 0) 0x7f0458003520 con 0x7f047006ba70 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 msgr2=0x7f04700712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f04700712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 >> v2:172.21.15.157:3300/0 conn(0x7f047006c9a0 msgr2=0x7f047006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 --2- 172.21.15.73:0/1958113415 >> v2:172.21.15.157:3300/0 conn(0x7f047006c9a0 0x7f047006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047006f890 con 0x7f047006ba70 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0468001b80 con 0x7f047006ba70 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> v2:172.21.15.157:3300/0 conn(0x7f047006c9a0 0x7f047006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 >> v1:172.21.15.73:6789/0 conn(0x7f047006ba70 legacy=0x7f04700cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f0458003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0458003ef0 0x7f04580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047015afc0 con 0x7f0458003ef0 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047006f5b0 con 0x7f0458004d00 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045800bb10 con 0x7f047015abe0 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.119+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0458003ef0 0x7f04580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0475400640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f0458003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0475e00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0474a00640 1 -- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 msgr2=0x7f0458003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f047015abe0 0x7f0458003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0474a00640 1 -- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458004d00 msgr2=0x7f0458005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0474a00640 1 -- 172.21.15.73:0/1958113415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458003520 con 0x7f0458003ef0 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.120+0000 7f0475e00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.121+0000 7f0474a00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0458003ef0 0x7f04580047c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f045c0098a0 tx=0x7f04580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a585c8654e5fc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.121+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c004230 con 0x7f0458003ef0 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.121+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c004390 con 0x7f0458003ef0 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.121+0000 7f046fe00640 1 -- 172.21.15.73:0/1958113415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c003830 con 0x7f0458003ef0 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 -- 172.21.15.73:0/1958113415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0458003ef0 msgr2=0x7f04580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 --2- 172.21.15.73:0/1958113415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0458003ef0 0x7f04580047c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f045c0098a0 tx=0x7f04580057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0475e00640 1 -- 172.21.15.73:0/1958113415 reap_dead start 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 -- 172.21.15.73:0/1958113415 shutdown_connections 2024-04-05T22:22:53.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 -- 172.21.15.73:0/1958113415 >> 172.21.15.73:0/1958113415 conn(0x7f0470069cb0 msgr2=0x7f047006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:53.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 -- 172.21.15.73:0/1958113415 shutdown_connections 2024-04-05T22:22:53.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.122+0000 7f0477e00640 1 -- 172.21.15.73:0/1958113415 wait complete. 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.124+0000 7f0477e00640 1 Processor -- start 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.124+0000 7f0477e00640 1 -- start start 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458003ef0 0x7f04700af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04700afb50 0x7f04701513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470154910 con 0x7f0470151900 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470150330 con 0x7f0458003ef0 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0477e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04700affb0 con 0x7f04700afb50 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42244/0 (socket says 172.21.15.73:42244) 2024-04-05T22:22:53.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 -- 172.21.15.73:0/1253537666 learned_addr learned my addr 172.21.15.73:0/1253537666 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0474a00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04700afb50 0x7f04701513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475400640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458003ef0 0x7f04700af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 -- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04700afb50 msgr2=0x7f04701513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04700afb50 0x7f04701513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.125+0000 7f0475e00640 1 -- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458003ef0 msgr2=0x7f04700af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f0475e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458003ef0 0x7f04700af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f0475e00640 1 -- 172.21.15.73:0/1253537666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470150750 con 0x7f0470151900 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f0475e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0468002230 tx=0x7f046800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468004d60 con 0x7f0470151900 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046800bb80 con 0x7f0470151900 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047014f6b0 con 0x7f0470151900 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468007070 con 0x7f0470151900 2024-04-05T22:22:53.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.126+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047014fc70 con 0x7f0470151900 2024-04-05T22:22:53.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.127+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0470150470 con 0x7f0470151900 2024-04-05T22:22:53.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.128+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f04680071d0 con 0x7f0470151900 2024-04-05T22:22:53.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.129+0000 7f046d600640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 0x7f045006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.129+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04680899e0 con 0x7f0470151900 2024-04-05T22:22:53.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.129+0000 7f0475400640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 0x7f045006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.130+0000 7f0475400640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 0x7f045006d2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0464009870 tx=0x7f0464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.134+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0470150470 con 0x7f0470151900 2024-04-05T22:22:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.328+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f047006bec0 con 0x7f0470151900 2024-04-05T22:22:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.329+0000 7f046d600640 1 -- 172.21.15.73:0/1253537666 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f047006bec0 con 0x7f0470151900 2024-04-05T22:22:53.328 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:22:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.331+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 msgr2=0x7f045006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.331+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 0x7f045006d2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0464009870 tx=0x7f0464008040 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.331+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 msgr2=0x7f0470152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.331+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0468002230 tx=0x7f046800b4a0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 shutdown_connections 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f045006ade0 0x7f045006d2a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0470151900 0x7f0470152c30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04700afb50 0x7f04701513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 --2- 172.21.15.73:0/1253537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0458003ef0 0x7f04700af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 >> 172.21.15.73:0/1253537666 conn(0x7f0470069cb0 msgr2=0x7f04700ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 shutdown_connections 2024-04-05T22:22:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.332+0000 7f0477e00640 1 -- 172.21.15.73:0/1253537666 wait complete. 2024-04-05T22:22:53.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:22:53.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.562+0000 7ff181e00640 1 Processor -- start 2024-04-05T22:22:53.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.562+0000 7ff181e00640 1 -- start start 2024-04-05T22:22:53.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff181e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff17c143660 0x7ff17c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff181e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0cceb0 con 0x7ff17c070660 2024-04-05T22:22:53.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff181e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0cd130 con 0x7ff17c06cd40 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff181e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0cb8c0 con 0x7ff17c143660 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff17aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff17c143660 0x7ff17c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff17a000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff17c070660 0x7ff17c06c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37434/0 (socket says 172.21.15.73:37434) 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.564+0000 7ff17a000640 1 -- 172.21.15.73:0/1872692269 learned_addr learned my addr 172.21.15.73:0/1872692269 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.565+0000 7ff17aa00640 1 -- 172.21.15.73:0/1872692269 >> v1:172.21.15.157:6789/0 conn(0x7ff17c06cd40 legacy=0x7ff17c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.565+0000 7ff17aa00640 1 -- 172.21.15.73:0/1872692269 >> v1:172.21.15.73:6789/0 conn(0x7ff17c070660 legacy=0x7ff17c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.565+0000 7ff17aa00640 1 -- 172.21.15.73:0/1872692269 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c070b90 con 0x7ff17c143660 2024-04-05T22:22:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.565+0000 7ff179600640 1 -- 172.21.15.73:0/1872692269 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517393450 0 0) 0x7ff17c0cceb0 con 0x7ff17c070660 2024-04-05T22:22:53.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.566+0000 7ff17aa00640 1 --2- 172.21.15.73:0/1872692269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c143660 0x7ff17c15a200 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0097a0 tx=0x7ff16c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea8c7783adec09f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.566+0000 7ff179600640 1 -- 172.21.15.73:0/1872692269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c031830 con 0x7ff17c143660 2024-04-05T22:22:53.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.566+0000 7ff179600640 1 -- 172.21.15.73:0/1872692269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c031990 con 0x7ff17c143660 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.566+0000 7ff179600640 1 -- 172.21.15.73:0/1872692269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c031d20 con 0x7ff17c143660 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 -- 172.21.15.73:0/1872692269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c143660 msgr2=0x7ff17c15a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 --2- 172.21.15.73:0/1872692269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c143660 0x7ff17c15a200 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0097a0 tx=0x7ff16c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 -- 172.21.15.73:0/1872692269 shutdown_connections 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 --2- 172.21.15.73:0/1872692269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c143660 0x7ff17c15a200 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 -- 172.21.15.73:0/1872692269 >> 172.21.15.73:0/1872692269 conn(0x7ff17c0b0830 msgr2=0x7ff17c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:53.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 -- 172.21.15.73:0/1872692269 shutdown_connections 2024-04-05T22:22:53.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.567+0000 7ff181e00640 1 -- 172.21.15.73:0/1872692269 wait complete. 2024-04-05T22:22:53.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.569+0000 7ff181e00640 1 Processor -- start 2024-04-05T22:22:53.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.569+0000 7ff181e00640 1 -- start start 2024-04-05T22:22:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 0x7ff17c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff17c143660 0x7ff17c141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c141160 con 0x7ff17c06cd40 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1426c0 con 0x7ff17c070660 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff181e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1429a0 con 0x7ff17c143660 2024-04-05T22:22:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff17aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff17aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42258/0 (socket says 172.21.15.73:42258) 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.570+0000 7ff17aa00640 1 -- 172.21.15.73:0/1407856 learned_addr learned my addr 172.21.15.73:0/1407856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17a000640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 0x7ff17c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 -- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff17c143660 msgr2=0x7ff17c141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff17c143660 0x7ff17c141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 -- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 msgr2=0x7ff17c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 0x7ff17c158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 -- 172.21.15.73:0/1407856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c009400 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17a000640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 0x7ff17c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff17aa00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000c00 tx=0x7ff16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c03aa60 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c03abc0 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0443f0 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.571+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c140b10 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.572+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c159020 con 0x7ff17c06cd40 2024-04-05T22:22:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.573+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c068160 con 0x7ff17c06cd40 2024-04-05T22:22:53.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.573+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff16c04d450 con 0x7ff17c06cd40 2024-04-05T22:22:53.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.575+0000 7ff173400640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 0x7ff15c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:53.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.575+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff16c0ab930 con 0x7ff17c06cd40 2024-04-05T22:22:53.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.575+0000 7ff17a000640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 0x7ff15c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:53.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.576+0000 7ff17a000640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 0x7ff15c06d270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0b0800 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:53.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.579+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff16c0768a0 con 0x7ff17c06cd40 2024-04-05T22:22:53.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.773+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff17c06d620 con 0x7ff17c06cd40 2024-04-05T22:22:53.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.773+0000 7ff173400640 1 -- 172.21.15.73:0/1407856 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff16c035070 con 0x7ff17c06cd40 2024-04-05T22:22:53.774 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 msgr2=0x7ff15c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 0x7ff15c06d270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0b0800 tx=0x7ff168009040 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 msgr2=0x7ff17c1415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000c00 tx=0x7ff16c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 shutdown_connections 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff15c06adb0 0x7ff15c06d270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff17c143660 0x7ff17c141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff17c070660 0x7ff17c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 --2- 172.21.15.73:0/1407856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff17c06cd40 0x7ff17c1415f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 >> 172.21.15.73:0/1407856 conn(0x7ff17c0b0830 msgr2=0x7ff17c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.776+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 shutdown_connections 2024-04-05T22:22:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:53.777+0000 7ff181e00640 1 -- 172.21.15.73:0/1407856 wait complete. 2024-04-05T22:22:53.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:54.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.002+0000 7fe37ee00640 1 Processor -- start 2024-04-05T22:22:54.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.002+0000 7fe37ee00640 1 -- start start 2024-04-05T22:22:54.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.003+0000 7fe37ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe37815e620 0x7fe37815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.003+0000 7fe37ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37806b500 con 0x7fe378158860 2024-04-05T22:22:54.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37815eff0 con 0x7fe37815e620 2024-04-05T22:22:54.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3780ccaf0 con 0x7fe37806bc70 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36238/0 (socket says 172.21.15.73:36238) 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37e400640 1 -- 172.21.15.73:0/4193356222 learned_addr learned my addr 172.21.15.73:0/4193356222 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37da00640 1 --1- 172.21.15.73:0/4193356222 >> v1:172.21.15.157:6789/0 conn(0x7fe378158860 0x7fe3780b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:58708/0 (socket says 172.21.15.73:58708) 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.004+0000 7fe37d000640 1 --2- 172.21.15.73:0/4193356222 >> v2:172.21.15.157:3300/0 conn(0x7fe37815e620 0x7fe37815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.005+0000 7fe37e400640 1 -- 172.21.15.73:0/4193356222 >> v2:172.21.15.157:3300/0 conn(0x7fe37815e620 msgr2=0x7fe37815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.005+0000 7fe37e400640 1 --2- 172.21.15.73:0/4193356222 >> v2:172.21.15.157:3300/0 conn(0x7fe37815e620 0x7fe37815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.005+0000 7fe37e400640 1 -- 172.21.15.73:0/4193356222 >> v1:172.21.15.157:6789/0 conn(0x7fe378158860 legacy=0x7fe3780b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.005+0000 7fe37e400640 1 -- 172.21.15.73:0/4193356222 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3780b20d0 con 0x7fe37806bc70 2024-04-05T22:22:54.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.005+0000 7fe377e00640 1 -- 172.21.15.73:0/4193356222 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353924825 0 0) 0x7fe37806b500 con 0x7fe378158860 2024-04-05T22:22:54.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.007+0000 7fe37e400640 1 --2- 172.21.15.73:0/4193356222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe370009e50 tx=0x7fe37002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e961e1d80ce091f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.007+0000 7fe377e00640 1 -- 172.21.15.73:0/4193356222 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37002ca80 con 0x7fe37806bc70 2024-04-05T22:22:54.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.007+0000 7fe377e00640 1 -- 172.21.15.73:0/4193356222 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37002cbe0 con 0x7fe37806bc70 2024-04-05T22:22:54.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.007+0000 7fe377e00640 1 -- 172.21.15.73:0/4193356222 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700343f0 con 0x7fe37806bc70 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 -- 172.21.15.73:0/4193356222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 msgr2=0x7fe37806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 --2- 172.21.15.73:0/4193356222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe370009e50 tx=0x7fe37002bee0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 -- 172.21.15.73:0/4193356222 shutdown_connections 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 --2- 172.21.15.73:0/4193356222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37806bc70 0x7fe37806c100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 --2- 172.21.15.73:0/4193356222 >> v2:172.21.15.157:3300/0 conn(0x7fe37815e620 0x7fe37815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 -- 172.21.15.73:0/4193356222 >> 172.21.15.73:0/4193356222 conn(0x7fe3780b06c0 msgr2=0x7fe3780b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 -- 172.21.15.73:0/4193356222 shutdown_connections 2024-04-05T22:22:54.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.008+0000 7fe37ee00640 1 -- 172.21.15.73:0/4193356222 wait complete. 2024-04-05T22:22:54.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.009+0000 7fe37ee00640 1 Processor -- start 2024-04-05T22:22:54.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.010+0000 7fe37ee00640 1 -- start start 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.010+0000 7fe37ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 0x7fe37815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 0x7fe37813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37815dc10 con 0x7fe3780cdde0 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3780ce7b0 con 0x7fe37806bc70 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37814ed30 con 0x7fe37815e620 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42266/0 (socket says 172.21.15.73:42266) 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 -- 172.21.15.73:0/2752578544 learned_addr learned my addr 172.21.15.73:0/2752578544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37d000640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 0x7fe37813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37da00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 0x7fe37815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 -- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 msgr2=0x7fe37813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 0x7fe37813c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 -- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 msgr2=0x7fe37815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 0x7fe37815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37e400640 1 -- 172.21.15.73:0/2752578544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe370009b30 con 0x7fe3780cdde0 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37d000640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 0x7fe37813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:54.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.011+0000 7fe37da00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 0x7fe37815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe37e400640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe370009e50 tx=0x7fe370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009510 con 0x7fe3780cdde0 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37003e450 con 0x7fe3780cdde0 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37815cfd0 con 0x7fe3780cdde0 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37003e5e0 con 0x7fe3780cdde0 2024-04-05T22:22:54.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.012+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37815d550 con 0x7fe3780cdde0 2024-04-05T22:22:54.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.013+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37815dd50 con 0x7fe3780cdde0 2024-04-05T22:22:54.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.014+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fe37002cd50 con 0x7fe3780cdde0 2024-04-05T22:22:54.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.016+0000 7fe375600640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 0x7fe35806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.016+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3700abba0 con 0x7fe3780cdde0 2024-04-05T22:22:54.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.016+0000 7fe37da00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 0x7fe35806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.016+0000 7fe37da00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 0x7fe35806d2a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000f60 tx=0x7fe36c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.020+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe37815dd50 con 0x7fe3780cdde0 2024-04-05T22:22:54.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.245+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe37815dd50 con 0x7fe3780cdde0 2024-04-05T22:22:54.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.246+0000 7fe375600640 1 -- 172.21.15.73:0/2752578544 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fe37815dd50 con 0x7fe3780cdde0 2024-04-05T22:22:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:54.247 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:54.247 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 16 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 msgr2=0x7fe35806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 0x7fe35806d2a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000f60 tx=0x7fe36c009040 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 msgr2=0x7fe3780ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe370009e50 tx=0x7fe370004a40 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 shutdown_connections 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe35806ade0 0x7fe35806d2a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3780cdde0 0x7fe3780ce270 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe37815e620 0x7fe37813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 --2- 172.21.15.73:0/2752578544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37806bc70 0x7fe37815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 >> 172.21.15.73:0/2752578544 conn(0x7fe3780b06c0 msgr2=0x7fe3780b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:54.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 shutdown_connections 2024-04-05T22:22:54.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.249+0000 7fe37ee00640 1 -- 172.21.15.73:0/2752578544 wait complete. 2024-04-05T22:22:54.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:54.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.470+0000 7f2ec3e00640 1 Processor -- start 2024-04-05T22:22:54.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.470+0000 7f2ec3e00640 1 -- start start 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec40b23e0 0x7f2ec406bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ec406c2d0 0x7f2ec406fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec406ca50 con 0x7f2ec413f3b0 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec406f920 con 0x7f2ec406c2d0 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec406cd30 con 0x7f2ec40b23e0 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ec406c2d0 0x7f2ec406fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec2a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ec413f3b0 0x7f2ec40b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37450/0 (socket says 172.21.15.73:37450) 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ec406c2d0 0x7f2ec406fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42282/0 (socket says 172.21.15.73:42282) 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec2a00640 1 -- 172.21.15.73:0/2909352118 learned_addr learned my addr 172.21.15.73:0/2909352118 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.472+0000 7f2ec2000640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec40b23e0 0x7f2ec406bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.473+0000 7f2ec3400640 1 -- 172.21.15.73:0/2909352118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec40b23e0 msgr2=0x7f2ec406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:54.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.473+0000 7f2ec3400640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec40b23e0 0x7f2ec406bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.473+0000 7f2ec3400640 1 -- 172.21.15.73:0/2909352118 >> v1:172.21.15.73:6789/0 conn(0x7f2ec413f3b0 legacy=0x7f2ec40b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.473+0000 7f2ec3400640 1 -- 172.21.15.73:0/2909352118 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec406d010 con 0x7f2ec406c2d0 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.474+0000 7f2ec3400640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec406fd40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40097a0 tx=0x7f2eb402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541eaadf824f1ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.474+0000 7f2ec1600640 1 -- 172.21.15.73:0/2909352118 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031780 con 0x7f2ec406c2d0 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.474+0000 7f2ec1600640 1 -- 172.21.15.73:0/2909352118 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40318e0 con 0x7f2ec406c2d0 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.474+0000 7f2ec1600640 1 -- 172.21.15.73:0/2909352118 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031c70 con 0x7f2ec406c2d0 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 -- 172.21.15.73:0/2909352118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 msgr2=0x7f2ec406fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec406fd40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40097a0 tx=0x7f2eb402b820 comp rx=0 tx=0).stop 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 -- 172.21.15.73:0/2909352118 shutdown_connections 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec40b23e0 0x7f2ec406bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/2909352118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec406fd40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 -- 172.21.15.73:0/2909352118 >> 172.21.15.73:0/2909352118 conn(0x7f2ec40b06c0 msgr2=0x7f2ec40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:54.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 -- 172.21.15.73:0/2909352118 shutdown_connections 2024-04-05T22:22:54.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.475+0000 7f2ec3e00640 1 -- 172.21.15.73:0/2909352118 wait complete. 2024-04-05T22:22:54.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.476+0000 7f2ec3e00640 1 Processor -- start 2024-04-05T22:22:54.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.477+0000 7f2ec3e00640 1 -- start start 2024-04-05T22:22:54.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.477+0000 7f2ec3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.477+0000 7f2ec3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ec40b23e0 0x7f2ec41648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec413f3b0 0x7f2ec413cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4162780 con 0x7f2ec406c2d0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4164fc0 con 0x7f2ec40b23e0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec41652a0 con 0x7f2ec413f3b0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42290/0 (socket says 172.21.15.73:42290) 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 -- 172.21.15.73:0/3381416585 learned_addr learned my addr 172.21.15.73:0/3381416585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2000640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ec40b23e0 0x7f2ec41648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 -- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec413f3b0 msgr2=0x7f2ec413cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec413f3b0 0x7f2ec413cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 -- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ec40b23e0 msgr2=0x7f2ec41648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ec40b23e0 0x7f2ec41648c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.478+0000 7f2ec2a00640 1 -- 172.21.15.73:0/3381416585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb4009400 con 0x7f2ec406c2d0 2024-04-05T22:22:54.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ec2a00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00cbc0 tx=0x7f2ebc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc00aca0 con 0x7f2ec406c2d0 2024-04-05T22:22:54.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc00ae00 con 0x7f2ec406c2d0 2024-04-05T22:22:54.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec41623a0 con 0x7f2ec406c2d0 2024-04-05T22:22:54.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc012600 con 0x7f2ec406c2d0 2024-04-05T22:22:54.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.479+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec40a84c0 con 0x7f2ec406c2d0 2024-04-05T22:22:54.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.480+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec4162200 con 0x7f2ec406c2d0 2024-04-05T22:22:54.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.481+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2ebc007070 con 0x7f2ec406c2d0 2024-04-05T22:22:54.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.482+0000 7f2ebb400640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 0x7f2ea006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.482+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ebc089470 con 0x7f2ec406c2d0 2024-04-05T22:22:54.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.482+0000 7f2ec2000640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 0x7f2ea006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.483+0000 7f2ec2000640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 0x7f2ea006d270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ec41f5870 tx=0x7f2eac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.487+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2ec4162200 con 0x7f2ec406c2d0 2024-04-05T22:22:54.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.682+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ec4162200 con 0x7f2ec406c2d0 2024-04-05T22:22:54.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.684+0000 7f2ebb400640 1 -- 172.21.15.73:0/3381416585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f2ec4162200 con 0x7f2ec406c2d0 2024-04-05T22:22:54.685 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:54.685 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 msgr2=0x7f2ea006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 0x7f2ea006d270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ec41f5870 tx=0x7f2eac008040 comp rx=0 tx=0).stop 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 msgr2=0x7f2ec4158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00cbc0 tx=0x7f2ebc00a060 comp rx=0 tx=0).stop 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 shutdown_connections 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2ea006adb0 0x7f2ea006d270 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ec413f3b0 0x7f2ec413cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ec40b23e0 0x7f2ec41648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 --2- 172.21.15.73:0/3381416585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec406c2d0 0x7f2ec4158f70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 >> 172.21.15.73:0/3381416585 conn(0x7f2ec40b06c0 msgr2=0x7f2ec4165680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:54.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 shutdown_connections 2024-04-05T22:22:54.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.687+0000 7f2ec3e00640 1 -- 172.21.15.73:0/3381416585 wait complete. 2024-04-05T22:22:54.707 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:22:54.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:54.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.902+0000 7fb022e00640 1 Processor -- start 2024-04-05T22:22:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.902+0000 7fb022e00640 1 -- start start 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb022e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb01c1541a0 0x7fb01c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb022e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 0x7fb01c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb022e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c15aa60 con 0x7fb01c06d710 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb022e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0b3b60 con 0x7fb01c1541a0 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb022e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0701c0 con 0x7fb01c0ce000 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb01b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb01c06d710 0x7fb01c0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37466/0 (socket says 172.21.15.73:37466) 2024-04-05T22:22:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb01b400640 1 -- 172.21.15.73:0/4030464729 learned_addr learned my addr 172.21.15.73:0/4030464729 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:54.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb01be00640 1 --2- 172.21.15.73:0/4030464729 >> v2:172.21.15.157:3300/0 conn(0x7fb01c1541a0 0x7fb01c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb020e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 0x7fb01c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.904+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809343749 0 0) 0x7fb01c15aa60 con 0x7fb01c06d710 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb000005730 con 0x7fb01c06d710 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990614022 0 0) 0x7fb000005730 con 0x7fb01c06d710 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 msgr2=0x7fb01c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 0x7fb01c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 >> v2:172.21.15.157:3300/0 conn(0x7fb01c1541a0 msgr2=0x7fb01c0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 --2- 172.21.15.73:0/4030464729 >> v2:172.21.15.157:3300/0 conn(0x7fb01c1541a0 0x7fb01c0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0b3880 con 0x7fb01c06d710 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb008001670 con 0x7fb01c06d710 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 >> v1:172.21.15.73:6789/0 conn(0x7fb01c06d710 legacy=0x7fb01c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb000005b80 0x7fb000005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 0x7fb0000069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb000006f10 0x7fb000007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b3b60 con 0x7fb000006f10 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0701c0 con 0x7fb000005b80 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000db90 con 0x7fb01c0ce000 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.905+0000 7fb020e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb000006f10 0x7fb000007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.906+0000 7fb020e00640 1 -- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 msgr2=0x7fb0000069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.906+0000 7fb020e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb01c0ce000 0x7fb0000069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.906+0000 7fb020e00640 1 -- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb000005b80 msgr2=0x7fb000005fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.906+0000 7fb020e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb000005b80 0x7fb000005fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.906+0000 7fb020e00640 1 -- 172.21.15.73:0/4030464729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000005730 con 0x7fb000006f10 2024-04-05T22:22:54.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb020e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb000006f10 0x7fb000007390 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb014009fc0 tx=0x7fb000007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d89025ae8a224 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01402cc60 con 0x7fb000006f10 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01400f040 con 0x7fb000006f10 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb01aa00640 1 -- 172.21.15.73:0/4030464729 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014034440 con 0x7fb000006f10 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb022e00640 1 -- 172.21.15.73:0/4030464729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb000006f10 msgr2=0x7fb000007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb022e00640 1 --2- 172.21.15.73:0/4030464729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb000006f10 0x7fb000007390 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb014009fc0 tx=0x7fb000007a00 comp rx=0 tx=0).stop 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb020e00640 1 -- 172.21.15.73:0/4030464729 reap_dead start 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb022e00640 1 -- 172.21.15.73:0/4030464729 shutdown_connections 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.907+0000 7fb022e00640 1 -- 172.21.15.73:0/4030464729 >> 172.21.15.73:0/4030464729 conn(0x7fb01c0ce8e0 msgr2=0x7fb01c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.908+0000 7fb022e00640 1 -- 172.21.15.73:0/4030464729 shutdown_connections 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.908+0000 7fb022e00640 1 -- 172.21.15.73:0/4030464729 wait complete. 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.909+0000 7fb022e00640 1 Processor -- start 2024-04-05T22:22:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.909+0000 7fb022e00640 1 -- start start 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb000006f10 0x7fb01c141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01c147b40 0x7fb01c06b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c141d10 con 0x7fb01c0ac900 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0ad2c0 con 0x7fb01c147b40 2024-04-05T22:22:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.910+0000 7fb022e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1419c0 con 0x7fb000006f10 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42302/0 (socket says 172.21.15.73:42302) 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 -- 172.21.15.73:0/1804109052 learned_addr learned my addr 172.21.15.73:0/1804109052 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 -- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb000006f10 msgr2=0x7fb01c141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb000006f10 0x7fb01c141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 -- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01c147b40 msgr2=0x7fb01c06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01c147b40 0x7fb01c06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 -- 172.21.15.73:0/1804109052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c06beb0 con 0x7fb01c0ac900 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.911+0000 7fb020e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb014037040 tx=0x7fb014009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.912+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014034d50 con 0x7fb01c0ac900 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.912+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014003990 con 0x7fb01c0ac900 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.912+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014003d40 con 0x7fb01c0ac900 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.912+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c06c190 con 0x7fb01c0ac900 2024-04-05T22:22:54.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.912+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c1415b0 con 0x7fb01c0ac900 2024-04-05T22:22:54.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.913+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c066af0 con 0x7fb01c0ac900 2024-04-05T22:22:54.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.914+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fb014004030 con 0x7fb01c0ac900 2024-04-05T22:22:54.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.914+0000 7fb013e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 0x7faffc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:54.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.914+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0140aae70 con 0x7fb01c0ac900 2024-04-05T22:22:54.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.915+0000 7fb01be00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 0x7faffc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:54.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.915+0000 7fb01be00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 0x7faffc06d2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006a90 tx=0x7fb00c006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:54.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:54.919+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb014075e70 con 0x7fb01c0ac900 2024-04-05T22:22:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.135+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb01c141e50 con 0x7fb01c0ac900 2024-04-05T22:22:55.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.136+0000 7fb013e00640 1 -- 172.21.15.73:0/1804109052 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fb01c141e50 con 0x7fb01c0ac900 2024-04-05T22:22:55.136 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:55.136 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:55.137 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 16 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 msgr2=0x7faffc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 0x7faffc06d2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006a90 tx=0x7fb00c006a20 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 msgr2=0x7fb01c0acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb014037040 tx=0x7fb014009750 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 shutdown_connections 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faffc06ade0 0x7faffc06d2a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c0ac900 0x7fb01c0acd80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01c147b40 0x7fb01c06b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 --2- 172.21.15.73:0/1804109052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb000006f10 0x7fb01c141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.138+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 >> 172.21.15.73:0/1804109052 conn(0x7fb01c0ce8e0 msgr2=0x7fb01c0ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.139+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 shutdown_connections 2024-04-05T22:22:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.139+0000 7fb022e00640 1 -- 172.21.15.73:0/1804109052 wait complete. 2024-04-05T22:22:55.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:22:55.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.359+0000 7f24bd000640 1 Processor -- start 2024-04-05T22:22:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.359+0000 7f24bd000640 1 -- start start 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24bd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f24b806cb80 0x7f24b806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24bd000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b806f5b0 con 0x7f24b81587d0 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24bd000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b8070410 con 0x7f24b815e850 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24bd000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b806d500 con 0x7f24b806cb80 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24b5e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f24b81587d0 0x7f24b80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37474/0 (socket says 172.21.15.73:37474) 2024-04-05T22:22:55.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24b5e00640 1 -- 172.21.15.73:0/3969784158 learned_addr learned my addr 172.21.15.73:0/3969784158 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:55.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.361+0000 7f24ad400640 1 --2- 172.21.15.73:0/3969784158 >> v2:172.21.15.73:3300/0 conn(0x7f24b806cb80 0x7f24b806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.362+0000 7f24b5400640 1 -- 172.21.15.73:0/3969784158 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753061547 0 0) 0x7f24b806f5b0 con 0x7f24b81587d0 2024-04-05T22:22:55.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.362+0000 7f24b5400640 1 -- 172.21.15.73:0/3969784158 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f249c003520 con 0x7f24b81587d0 2024-04-05T22:22:55.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.362+0000 7f24ad400640 1 -- 172.21.15.73:0/3969784158 >> v1:172.21.15.157:6789/0 conn(0x7f24b815e850 legacy=0x7f24b80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:55.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.362+0000 7f24ad400640 1 -- 172.21.15.73:0/3969784158 >> v1:172.21.15.73:6789/0 conn(0x7f24b81587d0 legacy=0x7f24b80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.362+0000 7f24ad400640 1 -- 172.21.15.73:0/3969784158 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80ccef0 con 0x7f24b806cb80 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24ad400640 1 --2- 172.21.15.73:0/3969784158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b806cfc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dab9b4de43c2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24b5400640 1 -- 172.21.15.73:0/3969784158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031830 con 0x7f24b806cb80 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24b5400640 1 -- 172.21.15.73:0/3969784158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0031990 con 0x7f24b806cb80 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24b5400640 1 -- 172.21.15.73:0/3969784158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031d20 con 0x7f24b806cb80 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24bd000640 1 -- 172.21.15.73:0/3969784158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 msgr2=0x7f24b806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.363+0000 7f24bd000640 1 --2- 172.21.15.73:0/3969784158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b806cfc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.364+0000 7f24bd000640 1 -- 172.21.15.73:0/3969784158 shutdown_connections 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.364+0000 7f24bd000640 1 --2- 172.21.15.73:0/3969784158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b806cfc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.364+0000 7f24bd000640 1 -- 172.21.15.73:0/3969784158 >> 172.21.15.73:0/3969784158 conn(0x7f24b80ce8e0 msgr2=0x7f24b80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:55.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.364+0000 7f24bd000640 1 -- 172.21.15.73:0/3969784158 shutdown_connections 2024-04-05T22:22:55.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.364+0000 7f24bd000640 1 -- 172.21.15.73:0/3969784158 wait complete. 2024-04-05T22:22:55.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.365+0000 7f24bd000640 1 Processor -- start 2024-04-05T22:22:55.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.365+0000 7f24bd000640 1 -- start start 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24b81587d0 0x7f24b8145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24b815e850 0x7f24b815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80aa5c0 con 0x7f24b806cb80 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b814c730 con 0x7f24b815e850 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24bd000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80a9940 con 0x7f24b81587d0 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.366+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42318/0 (socket says 172.21.15.73:42318) 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 -- 172.21.15.73:0/942036982 learned_addr learned my addr 172.21.15.73:0/942036982 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 -- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24b81587d0 msgr2=0x7f24b8145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24b81587d0 0x7f24b8145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 -- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24b815e850 msgr2=0x7f24b815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b6800640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24b815e850 0x7f24b815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24b815e850 0x7f24b815eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 -- 172.21.15.73:0/942036982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0009400 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.367+0000 7f24b5e00640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c530 tx=0x7f24a800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.368+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8005ce0 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.368+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8005e40 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.368+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80072b0 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.368+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b80a9c20 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.368+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b80aa1e0 con 0x7f24b806cb80 2024-04-05T22:22:55.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.369+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b8066af0 con 0x7f24b806cb80 2024-04-05T22:22:55.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.370+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f24a8007410 con 0x7f24b806cb80 2024-04-05T22:22:55.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.371+0000 7f24aea00640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 0x7f249406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.371+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24a8088c00 con 0x7f24b806cb80 2024-04-05T22:22:55.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.371+0000 7f24ad400640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 0x7f249406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.372+0000 7f24ad400640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 0x7f249406d270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f24a0036000 tx=0x7f24a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.375+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f24a808f080 con 0x7f24b806cb80 2024-04-05T22:22:55.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.589+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24b80aa700 con 0x7f24b806cb80 2024-04-05T22:22:55.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.590+0000 7f24aea00640 1 -- 172.21.15.73:0/942036982 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f24b80aa700 con 0x7f24b806cb80 2024-04-05T22:22:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:55.591 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:22:23.615118+0000","modified":"2024-04-05T22:22:26.635166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/2312639511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2312639511},{"type":"v1","addr":"172.21.15.73:6836","nonce":2312639511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-05T22:22:55.591 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 16 2024-04-05T22:22:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 msgr2=0x7f249406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 0x7f249406d270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f24a0036000 tx=0x7f24a0030000 comp rx=0 tx=0).stop 2024-04-05T22:22:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 msgr2=0x7f24b81492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c530 tx=0x7f24a800c9f0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 shutdown_connections 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f249406adb0 0x7f249406d270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24b815e850 0x7f24b815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24b81587d0 0x7f24b8145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 --2- 172.21.15.73:0/942036982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24b806cb80 0x7f24b81492c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 >> 172.21.15.73:0/942036982 conn(0x7f24b80ce8e0 msgr2=0x7f24b80cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 shutdown_connections 2024-04-05T22:22:55.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.593+0000 7f24bd000640 1 -- 172.21.15.73:0/942036982 wait complete. 2024-04-05T22:22:55.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:22:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.801+0000 7fbedda00640 1 Processor -- start 2024-04-05T22:22:55.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.801+0000 7fbedda00640 1 -- start start 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedda00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbed815e620 0x7fbed815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedda00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbed806bc70 0x7fbed806c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedda00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed806b500 con 0x7fbed8158860 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedda00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed815eff0 con 0x7fbed806bc70 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedda00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed80ccaf0 con 0x7fbed815e620 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbed806bc70 0x7fbed806c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbed806bc70 0x7fbed806c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42322/0 (socket says 172.21.15.73:42322) 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbedd000640 1 -- 172.21.15.73:0/216024284 learned_addr learned my addr 172.21.15.73:0/216024284 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbed7e00640 1 --1- 172.21.15.73:0/216024284 >> v1:172.21.15.73:6789/0 conn(0x7fbed8158860 0x7fbed80b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37490/0 (socket says 172.21.15.73:37490) 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.803+0000 7fbecf400640 1 --2- 172.21.15.73:0/216024284 >> v2:172.21.15.157:3300/0 conn(0x7fbed815e620 0x7fbed815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.804+0000 7fbedd000640 1 -- 172.21.15.73:0/216024284 >> v2:172.21.15.157:3300/0 conn(0x7fbed815e620 msgr2=0x7fbed815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.804+0000 7fbedd000640 1 --2- 172.21.15.73:0/216024284 >> v2:172.21.15.157:3300/0 conn(0x7fbed815e620 0x7fbed815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.804+0000 7fbedd000640 1 -- 172.21.15.73:0/216024284 >> v1:172.21.15.73:6789/0 conn(0x7fbed8158860 legacy=0x7fbed80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.804+0000 7fbedd000640 1 -- 172.21.15.73:0/216024284 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80b20d0 con 0x7fbed806bc70 2024-04-05T22:22:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.804+0000 7fbed7400640 1 -- 172.21.15.73:0/216024284 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471026276 0 0) 0x7fbed806b500 con 0x7fbed8158860 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedd000640 1 --2- 172.21.15.73:0/216024284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed806c0c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbed8072570 tx=0x7fbed002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1875c0c0d3acd588 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbed7400640 1 -- 172.21.15.73:0/216024284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed002ca80 con 0x7fbed806bc70 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbed7400640 1 -- 172.21.15.73:0/216024284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed002cbe0 con 0x7fbed806bc70 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbed7400640 1 -- 172.21.15.73:0/216024284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00343f0 con 0x7fbed806bc70 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 -- 172.21.15.73:0/216024284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 msgr2=0x7fbed806c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 --2- 172.21.15.73:0/216024284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed806c0c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbed8072570 tx=0x7fbed002bee0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 -- 172.21.15.73:0/216024284 shutdown_connections 2024-04-05T22:22:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 --2- 172.21.15.73:0/216024284 >> v2:172.21.15.157:3300/0 conn(0x7fbed815e620 0x7fbed815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 --2- 172.21.15.73:0/216024284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed806c0c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.805+0000 7fbedda00640 1 -- 172.21.15.73:0/216024284 >> 172.21.15.73:0/216024284 conn(0x7fbed80b06c0 msgr2=0x7fbed80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.806+0000 7fbedda00640 1 -- 172.21.15.73:0/216024284 shutdown_connections 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.806+0000 7fbedda00640 1 -- 172.21.15.73:0/216024284 wait complete. 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.807+0000 7fbedda00640 1 Processor -- start 2024-04-05T22:22:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.807+0000 7fbedda00640 1 -- start start 2024-04-05T22:22:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815e620 0x7fbed8148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 0x7fbed815d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815d5b0 con 0x7fbed806bc70 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c96f0 con 0x7fbed815cbc0 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbedda00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80cd0a0 con 0x7fbed815e620 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbed7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.808+0000 7fbed7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42330/0 (socket says 172.21.15.73:42330) 2024-04-05T22:22:55.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 -- 172.21.15.73:0/4108830856 learned_addr learned my addr 172.21.15.73:0/4108830856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbedd000640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 0x7fbed815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 -- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815e620 msgr2=0x7fbed8148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbecf400640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815e620 0x7fbed8148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815e620 0x7fbed8148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 -- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 msgr2=0x7fbed815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 0x7fbed815d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 -- 172.21.15.73:0/4108830856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0009b30 con 0x7fbed806bc70 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbedd000640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 0x7fbed815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.809+0000 7fbed7e00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c4d0 tx=0x7fbec800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.810+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80074e0 con 0x7fbed806bc70 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.810+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec8007660 con 0x7fbed806bc70 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.810+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8004a20 con 0x7fbed806bc70 2024-04-05T22:22:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.810+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed80cd380 con 0x7fbed806bc70 2024-04-05T22:22:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.810+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed80cd940 con 0x7fbed806bc70 2024-04-05T22:22:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.811+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed80cdb60 con 0x7fbed806bc70 2024-04-05T22:22:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.812+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fbec8004b80 con 0x7fbed806bc70 2024-04-05T22:22:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.813+0000 7fbed4c00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 0x7fbeb406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.813+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbec8088850 con 0x7fbed806bc70 2024-04-05T22:22:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.813+0000 7fbecf400640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 0x7fbeb406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.813+0000 7fbecf400640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 0x7fbeb406d270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:55.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:55.817+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbec80537a0 con 0x7fbed806bc70 2024-04-05T22:22:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.014+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbed804e160 con 0x7fbed806bc70 2024-04-05T22:22:56.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.017+0000 7fbed4c00640 1 -- 172.21.15.73:0/4108830856 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fbec8018090 con 0x7fbed806bc70 2024-04-05T22:22:56.017 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:22:56.017 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:24.619494+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:22:21.488491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-05T22:22:22.636776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:22:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 msgr2=0x7fbeb406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 0x7fbeb406d270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0008040 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 msgr2=0x7fbed8148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c4d0 tx=0x7fbec800c990 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 shutdown_connections 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb406adb0 0x7fbeb406d270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815cbc0 0x7fbed815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815e620 0x7fbed8148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 --2- 172.21.15.73:0/4108830856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed806bc70 0x7fbed8148d00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 >> 172.21.15.73:0/4108830856 conn(0x7fbed80b06c0 msgr2=0x7fbed8154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 shutdown_connections 2024-04-05T22:22:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.020+0000 7fbedda00640 1 -- 172.21.15.73:0/4108830856 wait complete. 2024-04-05T22:22:56.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:22:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.244+0000 7f275fe00640 1 Processor -- start 2024-04-05T22:22:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.245+0000 7f275fe00640 1 -- start start 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27601441e0 0x7f27600cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f276006c2d0 0x7f27600cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275fe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27600b1b60 con 0x7f27600b2c00 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600b1950 con 0x7f27601441e0 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275fe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600b1e40 con 0x7f276006c2d0 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27601441e0 0x7f27600cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27601441e0 0x7f27600cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36012/0 (socket says 172.21.15.73:36012) 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.246+0000 7f275ea00640 1 -- 172.21.15.73:0/3002951147 learned_addr learned my addr 172.21.15.73:0/3002951147 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.247+0000 7f275ea00640 1 -- 172.21.15.73:0/3002951147 >> v2:172.21.15.157:3300/0 conn(0x7f276006c2d0 msgr2=0x7f27600cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.247+0000 7f275ea00640 1 --2- 172.21.15.73:0/3002951147 >> v2:172.21.15.157:3300/0 conn(0x7f276006c2d0 0x7f27600cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.247+0000 7f275ea00640 1 -- 172.21.15.73:0/3002951147 >> v1:172.21.15.157:6789/0 conn(0x7f27600b2c00 legacy=0x7f276006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.247+0000 7f275ea00640 1 -- 172.21.15.73:0/3002951147 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600b2120 con 0x7f27601441e0 2024-04-05T22:22:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.248+0000 7f275ea00640 1 --2- 172.21.15.73:0/3002951147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27601441e0 0x7f27600cc050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f276006f540 tx=0x7f275002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ca4b4c4503fecc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.248+0000 7f275e000640 1 -- 172.21.15.73:0/3002951147 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750031870 con 0x7f27601441e0 2024-04-05T22:22:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.248+0000 7f275e000640 1 -- 172.21.15.73:0/3002951147 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500319d0 con 0x7f27601441e0 2024-04-05T22:22:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.248+0000 7f275e000640 1 -- 172.21.15.73:0/3002951147 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750031d60 con 0x7f27601441e0 2024-04-05T22:22:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 -- 172.21.15.73:0/3002951147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27601441e0 msgr2=0x7f27600cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 --2- 172.21.15.73:0/3002951147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27601441e0 0x7f27600cc050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f276006f540 tx=0x7f275002b910 comp rx=0 tx=0).stop 2024-04-05T22:22:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 -- 172.21.15.73:0/3002951147 shutdown_connections 2024-04-05T22:22:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 --2- 172.21.15.73:0/3002951147 >> v2:172.21.15.157:3300/0 conn(0x7f276006c2d0 0x7f27600cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 --2- 172.21.15.73:0/3002951147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27601441e0 0x7f27600cc050 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 -- 172.21.15.73:0/3002951147 >> 172.21.15.73:0/3002951147 conn(0x7f27600b0830 msgr2=0x7f27600b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:56.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 -- 172.21.15.73:0/3002951147 shutdown_connections 2024-04-05T22:22:56.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.249+0000 7f275fe00640 1 -- 172.21.15.73:0/3002951147 wait complete. 2024-04-05T22:22:56.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.250+0000 7f275fe00640 1 Processor -- start 2024-04-05T22:22:56.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 -- start start 2024-04-05T22:22:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27600b2c00 0x7f2760153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27601441e0 0x7f2760154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760132670 con 0x7f276006c2d0 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760132950 con 0x7f27601441e0 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.251+0000 7f275fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2760132c30 con 0x7f27600b2c00 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36026/0 (socket says 172.21.15.73:36026) 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 -- 172.21.15.73:0/4075098745 learned_addr learned my addr 172.21.15.73:0/4075098745 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 -- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27600b2c00 msgr2=0x7f2760153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27600b2c00 0x7f2760153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 -- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27601441e0 msgr2=0x7f2760154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27601441e0 0x7f2760154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 -- 172.21.15.73:0/4075098745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500094d0 con 0x7f276006c2d0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.252+0000 7f275ea00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f27500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.253+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275003ad10 con 0x7f276006c2d0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.253+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275003ae70 con 0x7f276006c2d0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.253+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750044610 con 0x7f276006c2d0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.253+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2760132f10 con 0x7f276006c2d0 2024-04-05T22:22:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.253+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2760132300 con 0x7f276006c2d0 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.254+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276004e160 con 0x7f276006c2d0 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.255+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f275004d450 con 0x7f276006c2d0 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.255+0000 7f2756a00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 0x7f273c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.256+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f275002d040 con 0x7f276006c2d0 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.256+0000 7f2757e00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 0x7f273c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.256+0000 7f2757e00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 0x7f273c06d270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27601f3720 tx=0x7f2748007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.260+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2750076cc0 con 0x7f276006c2d0 2024-04-05T22:22:56.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.490+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f27600b03c0 con 0x7f276006c2d0 2024-04-05T22:22:56.492 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:22:56.492+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:22:56.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.506+0000 7f2756a00640 1 -- 172.21.15.73:0/4075098745 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f275007a2e0 con 0x7f276006c2d0 2024-04-05T22:22:56.506 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:22:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 msgr2=0x7f273c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 0x7f273c06d270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27601f3720 tx=0x7f2748007990 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 msgr2=0x7f2760155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f27500046c0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 shutdown_connections 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f273c06adb0 0x7f273c06d270 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27601441e0 0x7f2760154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27600b2c00 0x7f2760153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 --2- 172.21.15.73:0/4075098745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f276006c2d0 0x7f2760155fa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 >> 172.21.15.73:0/4075098745 conn(0x7f27600b0830 msgr2=0x7f276006c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 shutdown_connections 2024-04-05T22:22:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.509+0000 7f275fe00640 1 -- 172.21.15.73:0/4075098745 wait complete. 2024-04-05T22:22:56.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:22:56.584 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:22:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.719+0000 7f167da00640 1 Processor -- start 2024-04-05T22:22:56.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.719+0000 7f167da00640 1 -- start start 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 0x7f16780b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167da00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16780ce940 0x7f16780cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167da00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780b35b0 con 0x7f1678158a30 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167da00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780b1b10 con 0x7f16780ce940 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780cf310 con 0x7f16780b1f30 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f1677e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1678158a30 0x7f16780b2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41038/0 (socket says 172.21.15.73:41038) 2024-04-05T22:22:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f1677e00640 1 -- 172.21.15.73:0/3438747155 learned_addr learned my addr 172.21.15.73:0/3438747155 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f167d000640 1 --2- 172.21.15.73:0/3438747155 >> v2:172.21.15.157:3300/0 conn(0x7f16780ce940 0x7f16780cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996312301 0 0) 0x7f16780b35b0 con 0x7f1678158a30 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f1677400640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 0x7f16780b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.721+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f165c003520 con 0x7f1678158a30 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1969597488 0 0) 0x7f165c003520 con 0x7f1678158a30 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 msgr2=0x7f16780b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 0x7f16780b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 >> v2:172.21.15.157:3300/0 conn(0x7f16780ce940 msgr2=0x7f16780cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 --2- 172.21.15.73:0/3438747155 >> v2:172.21.15.157:3300/0 conn(0x7f16780ce940 0x7f16780cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780b3890 con 0x7f1678158a30 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f16680019f0 con 0x7f1678158a30 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 >> v1:172.21.15.73:6789/0 conn(0x7f1678158a30 legacy=0x7f16780b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f165c003970 0x7f165c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f165c0047c0 0x7f165c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 0x7f165c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b1b10 con 0x7f165c003970 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780cf310 con 0x7f165c0047c0 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c00b980 con 0x7f16780b1f30 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f165c003970 0x7f165c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 -- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 msgr2=0x7f165c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16780b1f30 0x7f165c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 -- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f165c0047c0 msgr2=0x7f165c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f165c0047c0 0x7f165c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.722+0000 7f1677e00640 1 -- 172.21.15.73:0/3438747155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c003520 con 0x7f165c003970 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.723+0000 7f1677e00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f165c003970 0x7f165c003d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f166800a860 tx=0x7f165c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca28f0b9631f380 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668034630 con 0x7f165c003970 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166802f070 con 0x7f165c003970 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f1676a00640 1 -- 172.21.15.73:0/3438747155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680349d0 con 0x7f165c003970 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 -- 172.21.15.73:0/3438747155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f165c003970 msgr2=0x7f165c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 --2- 172.21.15.73:0/3438747155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f165c003970 0x7f165c003d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f166800a860 tx=0x7f165c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167d000640 1 -- 172.21.15.73:0/3438747155 reap_dead start 2024-04-05T22:22:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 -- 172.21.15.73:0/3438747155 shutdown_connections 2024-04-05T22:22:56.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 -- 172.21.15.73:0/3438747155 >> 172.21.15.73:0/3438747155 conn(0x7f1678069e90 msgr2=0x7f167806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:56.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 -- 172.21.15.73:0/3438747155 shutdown_connections 2024-04-05T22:22:56.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.724+0000 7f167da00640 1 -- 172.21.15.73:0/3438747155 wait complete. 2024-04-05T22:22:56.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.726+0000 7f167da00640 1 Processor -- start 2024-04-05T22:22:56.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.726+0000 7f167da00640 1 -- start start 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f165c003970 0x7f167815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16781529d0 0x7f1678158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678155440 con 0x7f167815a010 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781543a0 con 0x7f16781529d0 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f167da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678154680 con 0x7f165c003970 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f1677400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f1677400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36074/0 (socket says 172.21.15.73:36074) 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f1677400640 1 -- 172.21.15.73:0/3859538191 learned_addr learned my addr 172.21.15.73:0/3859538191 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.727+0000 7f1677e00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f165c003970 0x7f167815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f167d000640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16781529d0 0x7f1678158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 -- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f165c003970 msgr2=0x7f167815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f165c003970 0x7f167815d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 -- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16781529d0 msgr2=0x7f1678158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16781529d0 0x7f1678158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 -- 172.21.15.73:0/3859538191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678154960 con 0x7f167815a010 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f1677400640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f166000d800 tx=0x7f166000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000b6c0 con 0x7f167815a010 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166000b840 con 0x7f167815a010 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678154c40 con 0x7f167815a010 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000bbf0 con 0x7f167815a010 2024-04-05T22:22:56.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.728+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167815acf0 con 0x7f167815a010 2024-04-05T22:22:56.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.729+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167804e160 con 0x7f167815a010 2024-04-05T22:22:56.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.730+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f166000bd50 con 0x7f167815a010 2024-04-05T22:22:56.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.731+0000 7f166fe00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 0x7f165406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:56.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.731+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1660089b30 con 0x7f167815a010 2024-04-05T22:22:56.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.731+0000 7f1677e00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 0x7f165406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:56.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.732+0000 7f1677e00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 0x7f165406d2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1668034000 tx=0x7f1668009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:56.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.735+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1660054990 con 0x7f167815a010 2024-04-05T22:22:56.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:56.969+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f167815a780 con 0x7f167815a010 2024-04-05T22:22:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.510+0000 7f166fe00640 1 -- 172.21.15.73:0/3859538191 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1660057fb0 con 0x7f167815a010 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.514+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 msgr2=0x7f165406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.514+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 0x7f165406d2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1668034000 tx=0x7f1668009030 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.514+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 msgr2=0x7f167815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.514+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f166000d800 tx=0x7f166000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 shutdown_connections 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f165406ade0 0x7f165406d2a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16781529d0 0x7f1678158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f167815a010 0x7f167815d960 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 --2- 172.21.15.73:0/3859538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f165c003970 0x7f167815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 >> 172.21.15.73:0/3859538191 conn(0x7f1678069e90 msgr2=0x7f167806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 shutdown_connections 2024-04-05T22:22:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.515+0000 7f167da00640 1 -- 172.21.15.73:0/3859538191 wait complete. 2024-04-05T22:22:57.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:22:57.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.794+0000 7f38ac800640 1 Processor -- start 2024-04-05T22:22:57.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.794+0000 7f38ac800640 1 -- start start 2024-04-05T22:22:57.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38ac800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f38a40cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38ac800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415b230 con 0x7f38a415ae50 2024-04-05T22:22:57.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38ac800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415f4a0 con 0x7f38a40cf3e0 2024-04-05T22:22:57.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38ac800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b3f30 con 0x7f38a40b3aa0 2024-04-05T22:22:57.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38a9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38a415ae50 0x7f38a40cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41048/0 (socket says 172.21.15.73:41048) 2024-04-05T22:22:57.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.796+0000 7f38a9e00640 1 -- 172.21.15.73:0/630683397 learned_addr learned my addr 172.21.15.73:0/630683397 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:57.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38aa800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f38a40cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260716632 0 0) 0x7f38a415b230 con 0x7f38a415ae50 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f388c003520 con 0x7f38a415ae50 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3183243857 0 0) 0x7f388c003520 con 0x7f38a415ae50 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 msgr2=0x7f38a40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f38a40cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 >> v1:172.21.15.157:6789/0 conn(0x7f38a40cf3e0 legacy=0x7f38a40b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a415f780 con 0x7f38a415ae50 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3898001670 con 0x7f38a415ae50 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 >> v1:172.21.15.73:6789/0 conn(0x7f38a415ae50 legacy=0x7f38a40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f388c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c003f20 0x7f388c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f388c004d00 0x7f388c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b3f30 con 0x7f388c004d00 2024-04-05T22:22:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c00bb10 con 0x7f388c003f20 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.797+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c00bcd0 con 0x7f38a40b3aa0 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f388c004d00 0x7f388c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38a9400640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c003f20 0x7f388c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38a9e00640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f388c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 -- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 msgr2=0x7f388c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a40b3aa0 0x7f388c0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 -- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c003f20 msgr2=0x7f388c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c003f20 0x7f388c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38aa800640 1 -- 172.21.15.73:0/630683397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c003520 con 0x7f388c004d00 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.798+0000 7f38a9400640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c003f20 0x7f388c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.799+0000 7f38aa800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f388c004d00 0x7f388c005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009fc0 tx=0x7f388c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d3a690b36b6477 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.799+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a002cc20 con 0x7f388c004d00 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.799+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a000f040 con 0x7f388c004d00 2024-04-05T22:22:57.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38a8a00640 1 -- 172.21.15.73:0/630683397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0034440 con 0x7f388c004d00 2024-04-05T22:22:57.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 -- 172.21.15.73:0/630683397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f388c004d00 msgr2=0x7f388c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 --2- 172.21.15.73:0/630683397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f388c004d00 0x7f388c005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009fc0 tx=0x7f388c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38aa800640 1 -- 172.21.15.73:0/630683397 reap_dead start 2024-04-05T22:22:57.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 -- 172.21.15.73:0/630683397 shutdown_connections 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 -- 172.21.15.73:0/630683397 >> 172.21.15.73:0/630683397 conn(0x7f38a406a0d0 msgr2=0x7f38a406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 -- 172.21.15.73:0/630683397 shutdown_connections 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.800+0000 7f38ac800640 1 -- 172.21.15.73:0/630683397 wait complete. 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.802+0000 7f38ac800640 1 Processor -- start 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.802+0000 7f38ac800640 1 -- start start 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 0x7f38a4159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 0x7f38a40c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41f3860 con 0x7f38a40c3e00 2024-04-05T22:22:57.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41f2340 con 0x7f388c004d00 2024-04-05T22:22:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38ac800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41f2620 con 0x7f38a41f33e0 2024-04-05T22:22:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38aa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38aa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36102/0 (socket says 172.21.15.73:36102) 2024-04-05T22:22:57.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.803+0000 7f38aa800640 1 -- 172.21.15.73:0/3897185777 learned_addr learned my addr 172.21.15.73:0/3897185777 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38a9e00640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 0x7f38a4159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38a9400640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 0x7f38a40c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 -- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 msgr2=0x7f38a40c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 0x7f38a40c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 -- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 msgr2=0x7f38a4159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 0x7f38a4159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 -- 172.21.15.73:0/3897185777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a41f2900 con 0x7f38a40c3e00 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38a9400640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 0x7f38a40c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38a9e00640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 0x7f38a4159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38aa800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f38a0037040 tx=0x7f38a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0046070 con 0x7f38a40c3e00 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0034ad0 con 0x7f38a40c3e00 2024-04-05T22:22:57.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41f2be0 con 0x7f38a40c3e00 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.804+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0034e40 con 0x7f38a40c3e00 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.805+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41f1fd0 con 0x7f38a40c3e00 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.806+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a41f3100 con 0x7f38a40c3e00 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.807+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f38a003b050 con 0x7f38a40c3e00 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.808+0000 7f389e000640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 0x7f388806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:57.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.808+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38a00aadc0 con 0x7f38a40c3e00 2024-04-05T22:22:57.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.808+0000 7f38a9e00640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 0x7f388806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:57.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.809+0000 7f38a9e00640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 0x7f388806d270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3898001200 tx=0x7f38980014f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:57.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:57.813+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f38a41f3100 con 0x7f38a40c3e00 2024-04-05T22:22:58.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.006+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38a41f3100 con 0x7f38a40c3e00 2024-04-05T22:22:58.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.732+0000 7f389e000640 1 -- 172.21.15.73:0/3897185777 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38a41f3100 con 0x7f38a40c3e00 2024-04-05T22:22:58.736 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:22:58.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.740+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 msgr2=0x7f388806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.740+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 0x7f388806d270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3898001200 tx=0x7f38980014f0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.740+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 msgr2=0x7f38a40c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.740+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f38a0037040 tx=0x7f38a0009750 comp rx=0 tx=0).stop 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 shutdown_connections 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f388806adb0 0x7f388806d270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a40c3e00 0x7f38a40c43b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a41f33e0 0x7f38a40c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 --2- 172.21.15.73:0/3897185777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f388c004d00 0x7f38a4159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 >> 172.21.15.73:0/3897185777 conn(0x7f38a406a0d0 msgr2=0x7f38a406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:58.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 shutdown_connections 2024-04-05T22:22:58.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.742+0000 7f38ac800640 1 -- 172.21.15.73:0/3897185777 wait complete. 2024-04-05T22:22:58.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:22:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.962+0000 7f48cd400640 1 Processor -- start 2024-04-05T22:22:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.963+0000 7f48cd400640 1 -- start start 2024-04-05T22:22:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.964+0000 7f48cd400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f48c815aa80 0x7f48c806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.964+0000 7f48cd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.964+0000 7f48cd400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c80ce380 con 0x7f48c80714b0 2024-04-05T22:22:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.964+0000 7f48cd400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c80cd5f0 con 0x7f48c815aa80 2024-04-05T22:22:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.964+0000 7f48cd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80ccc50 con 0x7f48c80cda10 2024-04-05T22:22:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58124/0 (socket says 172.21.15.73:58124) 2024-04-05T22:22:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 -- 172.21.15.73:0/763458730 learned_addr learned my addr 172.21.15.73:0/763458730 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48c7e00640 1 --2- 172.21.15.73:0/763458730 >> v2:172.21.15.157:3300/0 conn(0x7f48c815aa80 0x7f48c806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 -- 172.21.15.73:0/763458730 >> v2:172.21.15.157:3300/0 conn(0x7f48c815aa80 msgr2=0x7f48c806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 --2- 172.21.15.73:0/763458730 >> v2:172.21.15.157:3300/0 conn(0x7f48c815aa80 0x7f48c806fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.965+0000 7f48cca00640 1 -- 172.21.15.73:0/763458730 >> v1:172.21.15.157:6789/0 conn(0x7f48c80714b0 legacy=0x7f48c806b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.966+0000 7f48cca00640 1 -- 172.21.15.73:0/763458730 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80ccf30 con 0x7f48c80cda10 2024-04-05T22:22:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.967+0000 7f48cca00640 1 --2- 172.21.15.73:0/763458730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e50 tx=0x7f48c002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671d4071c672749 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48c7400640 1 -- 172.21.15.73:0/763458730 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c002c9e0 con 0x7f48c80cda10 2024-04-05T22:22:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48c7400640 1 -- 172.21.15.73:0/763458730 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c002cb40 con 0x7f48c80cda10 2024-04-05T22:22:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48c7400640 1 -- 172.21.15.73:0/763458730 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00343f0 con 0x7f48c80cda10 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 -- 172.21.15.73:0/763458730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 msgr2=0x7f48c80cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 --2- 172.21.15.73:0/763458730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e50 tx=0x7f48c002be40 comp rx=0 tx=0).stop 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 -- 172.21.15.73:0/763458730 shutdown_connections 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 --2- 172.21.15.73:0/763458730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c80cda10 0x7f48c80cde40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 --2- 172.21.15.73:0/763458730 >> v2:172.21.15.157:3300/0 conn(0x7f48c815aa80 0x7f48c806fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.968+0000 7f48cd400640 1 -- 172.21.15.73:0/763458730 >> 172.21.15.73:0/763458730 conn(0x7f48c80b0360 msgr2=0x7f48c80b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.969+0000 7f48cd400640 1 -- 172.21.15.73:0/763458730 shutdown_connections 2024-04-05T22:22:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.969+0000 7f48cd400640 1 -- 172.21.15.73:0/763458730 wait complete. 2024-04-05T22:22:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.970+0000 7f48cd400640 1 Processor -- start 2024-04-05T22:22:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.970+0000 7f48cd400640 1 -- start start 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c80714b0 0x7f48c8152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c815aa80 0x7f48c80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81525c0 con 0x7f48c80cda10 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81dfcf0 con 0x7f48c80714b0 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.971+0000 7f48cd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80b2140 con 0x7f48c815aa80 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36104/0 (socket says 172.21.15.73:36104) 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 -- 172.21.15.73:0/4050657250 learned_addr learned my addr 172.21.15.73:0/4050657250 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48c7e00640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c80714b0 0x7f48c8152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48cca00640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c815aa80 0x7f48c80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 -- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c815aa80 msgr2=0x7f48c80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c815aa80 0x7f48c80ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 -- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c80714b0 msgr2=0x7f48c8152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c80714b0 0x7f48c8152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.972+0000 7f48bf400640 1 -- 172.21.15.73:0/4050657250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0009b30 con 0x7f48c80cda10 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48bf400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c3c0 tx=0x7f48b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b00052d0 con 0x7f48c80cda10 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0005450 con 0x7f48c80cda10 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80b23e0 con 0x7f48c80cda10 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003830 con 0x7f48c80cda10 2024-04-05T22:22:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.973+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c81dff00 con 0x7f48c80cda10 2024-04-05T22:22:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.974+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c81e0120 con 0x7f48c80cda10 2024-04-05T22:22:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.975+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f48b0003990 con 0x7f48c80cda10 2024-04-05T22:22:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.976+0000 7f48c4c00640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 0x7f48a406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.976+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f48b0088a80 con 0x7f48c80cda10 2024-04-05T22:22:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.976+0000 7f48c7e00640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 0x7f48a406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.977+0000 7f48c7e00640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 0x7f48a406d270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a30 tx=0x7f48b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:58.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:58.980+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f48b0053aa0 con 0x7f48c80cda10 2024-04-05T22:22:59.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.175+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48c804e160 con 0x7f48c80cda10 2024-04-05T22:22:59.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.738+0000 7f48c4c00640 1 -- 172.21.15.73:0/4050657250 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48b00170d0 con 0x7f48c80cda10 2024-04-05T22:22:59.739 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:22:59.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.743+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 msgr2=0x7f48a406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.743+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 0x7f48a406d270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a30 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2024-04-05T22:22:59.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.743+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 msgr2=0x7f48c81f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.743+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c3c0 tx=0x7f48b000c880 comp rx=0 tx=0).stop 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 shutdown_connections 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f48a406adb0 0x7f48a406d270 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a30 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48c815aa80 0x7f48c80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c80cda10 0x7f48c81f3220 secure :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c3c0 tx=0x7f48b000c880 comp rx=0 tx=0).stop 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 --2- 172.21.15.73:0/4050657250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c80714b0 0x7f48c8152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.744+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 >> 172.21.15.73:0/4050657250 conn(0x7f48c80b0360 msgr2=0x7f48c8158070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.749+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 shutdown_connections 2024-04-05T22:22:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.749+0000 7f48cd400640 1 -- 172.21.15.73:0/4050657250 wait complete. 2024-04-05T22:22:59.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:22:59.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.961+0000 7f1cc2600640 1 Processor -- start 2024-04-05T22:22:59.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.961+0000 7f1cc2600640 1 -- start start 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.962+0000 7f1cc2600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cbc143660 0x7f1cbc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.962+0000 7f1cc2600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0cceb0 con 0x7f1cbc06cd40 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.962+0000 7f1cc2600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0cd130 con 0x7f1cbc070660 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.962+0000 7f1cc2600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc0cb8c0 con 0x7f1cbc143660 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbbe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1cbc06cd40 0x7f1cbc0cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41050/0 (socket says 172.21.15.73:41050) 2024-04-05T22:22:59.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2257932221 learned_addr learned my addr 172.21.15.73:0/2257932221 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260320718 0 0) 0x7f1cbc0cceb0 con 0x7f1cbc06cd40 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca0003520 con 0x7f1cbc06cd40 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2257932221 >> v2:172.21.15.157:3300/0 conn(0x7f1cbc143660 0x7f1cbc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2541239125 0 0) 0x7f1ca0003520 con 0x7f1cbc06cd40 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 >> v2:172.21.15.157:3300/0 conn(0x7f1cbc143660 msgr2=0x7f1cbc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 --2- 172.21.15.73:0/2257932221 >> v2:172.21.15.157:3300/0 conn(0x7f1cbc143660 0x7f1cbc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 >> v1:172.21.15.157:6789/0 conn(0x7f1cbc070660 legacy=0x7f1cbc06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc070b90 con 0x7f1cbc06cd40 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f1cb40019f0 con 0x7f1cbc06cd40 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2257932221 >> v2:172.21.15.157:3300/0 conn(0x7f1cbc143660 0x7f1cbc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.963+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 >> v1:172.21.15.73:6789/0 conn(0x7f1cbc06cd40 legacy=0x7f1cbc0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca0003970 0x7f1ca0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca00047c0 0x7f1ca0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 0x7f1ca000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0cb8c0 con 0x7f1ca0005380 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca000c220 con 0x7f1ca0003970 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca000c3e0 con 0x7f1ca00047c0 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 0x7f1ca000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca0003970 0x7f1ca0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca00047c0 msgr2=0x7f1ca0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:59.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cb2a00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca00047c0 0x7f1ca0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca00047c0 0x7f1ca0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca0003970 msgr2=0x7f1ca0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca0003970 0x7f1ca0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2257932221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca0003520 con 0x7f1ca0005380 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.964+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca0003970 0x7f1ca0003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.965+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2257932221 reap_dead start 2024-04-05T22:22:59.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.965+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 0x7f1ca000b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a810 tx=0x7f1ca00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9510a8409d26807 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4034630 con 0x7f1ca0005380 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb402f070 con 0x7f1ca0005380 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cbaa00640 1 -- 172.21.15.73:0/2257932221 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40349d0 con 0x7f1ca0005380 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 -- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 msgr2=0x7f1ca000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 0x7f1ca000b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a810 tx=0x7f1ca00057f0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 -- 172.21.15.73:0/2257932221 shutdown_connections 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2257932221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca0005380 0x7f1ca000b980 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 -- 172.21.15.73:0/2257932221 >> 172.21.15.73:0/2257932221 conn(0x7f1cbc0b0830 msgr2=0x7f1cbc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:22:59.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 -- 172.21.15.73:0/2257932221 shutdown_connections 2024-04-05T22:22:59.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.966+0000 7f1cc2600640 1 -- 172.21.15.73:0/2257932221 wait complete. 2024-04-05T22:22:59.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.968+0000 7f1cc2600640 1 Processor -- start 2024-04-05T22:22:59.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.968+0000 7f1cc2600640 1 -- start start 2024-04-05T22:22:59.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 0x7f1cbc06b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cbc1588d0 0x7f1cbc0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15a590 con 0x7f1cbc158d50 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc159240 con 0x7f1cbc1588d0 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cc2600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc159520 con 0x7f1ca0005380 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36130/0 (socket says 172.21.15.73:36130) 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.969+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2215330026 learned_addr learned my addr 172.21.15.73:0/2215330026 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 0x7f1cbc06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cb2a00640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cbc1588d0 0x7f1cbc0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 msgr2=0x7f1cbc06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 0x7f1cbc06b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cbc1588d0 msgr2=0x7f1cbc0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:22:59.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cbc1588d0 0x7f1cbc0c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 -- 172.21.15.73:0/2215330026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc159800 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 0x7f1cbc06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cbbe00640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a930 tx=0x7f1cbc0678c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4001930 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb4001a90 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4034650 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.970+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc159ae0 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.971+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc06bfb0 con 0x7f1cbc158d50 2024-04-05T22:22:59.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.972+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc15a740 con 0x7f1cbc158d50 2024-04-05T22:22:59.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.972+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f1cb4031030 con 0x7f1cbc158d50 2024-04-05T22:22:59.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.973+0000 7f1cb3e00640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 0x7f1c9806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:22:59.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.973+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1cb40aadf0 con 0x7f1cbc158d50 2024-04-05T22:22:59.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.973+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 0x7f1c9806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:22:59.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.974+0000 7f1cbb400640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 0x7f1c9806d270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006fd0 tx=0x7f1cac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:22:59.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:22:59.977+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1cbc15a740 con 0x7f1cbc158d50 2024-04-05T22:23:00.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.171+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1cbc06bfb0 con 0x7f1cbc158d50 2024-04-05T22:23:00.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.179+0000 7f1cb3e00640 1 -- 172.21.15.73:0/2215330026 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1cbc06bfb0 con 0x7f1cbc158d50 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 msgr2=0x7f1c9806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 0x7f1c9806d270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006fd0 tx=0x7f1cac008040 comp rx=0 tx=0).stop 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 msgr2=0x7f1cbc0c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a930 tx=0x7f1cbc0678c0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 shutdown_connections 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1c9806adb0 0x7f1c9806d270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.181+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc158d50 0x7f1cbc0c4be0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.182+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cbc1588d0 0x7f1cbc0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.182+0000 7f1cc2600640 1 --2- 172.21.15.73:0/2215330026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca0005380 0x7f1cbc06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.182+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 >> 172.21.15.73:0/2215330026 conn(0x7f1cbc0b0830 msgr2=0x7f1cbc13d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:00.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.182+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 shutdown_connections 2024-04-05T22:23:00.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.182+0000 7f1cc2600640 1 -- 172.21.15.73:0/2215330026 wait complete. 2024-04-05T22:23:00.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:23:00.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.409+0000 7f9b0ea00640 1 Processor -- start 2024-04-05T22:23:00.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.409+0000 7f9b0ea00640 1 -- start start 2024-04-05T22:23:00.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.410+0000 7f9b0ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b080d0bd0 0x7f9b080d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.410+0000 7f9b0ea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b080ceee0 con 0x7f9b0815cf70 2024-04-05T22:23:00.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.410+0000 7f9b0ea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b080ce5b0 con 0x7f9b080d1930 2024-04-05T22:23:00.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.410+0000 7f9b0ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b080ce890 con 0x7f9b080d0bd0 2024-04-05T22:23:00.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b07e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9b0815cf70 0x7f9b080d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41066/0 (socket says 172.21.15.73:41066) 2024-04-05T22:23:00.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b07e00640 1 -- 172.21.15.73:0/1972698676 learned_addr learned my addr 172.21.15.73:0/1972698676 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:00.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767231066 0 0) 0x7f9b080ceee0 con 0x7f9b0815cf70 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aec003520 con 0x7f9b0815cf70 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b0ca00640 1 --2- 172.21.15.73:0/1972698676 >> v2:172.21.15.157:3300/0 conn(0x7f9b080d0bd0 0x7f9b080d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627360788 0 0) 0x7f9aec003520 con 0x7f9b0815cf70 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 >> v2:172.21.15.157:3300/0 conn(0x7f9b080d0bd0 msgr2=0x7f9b080d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 --2- 172.21.15.73:0/1972698676 >> v2:172.21.15.157:3300/0 conn(0x7f9b080d0bd0 0x7f9b080d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 >> v1:172.21.15.157:6789/0 conn(0x7f9b080d1930 legacy=0x7f9b080b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080ceb70 con 0x7f9b0815cf70 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9af80019f0 con 0x7f9b0815cf70 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.411+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 >> v1:172.21.15.73:6789/0 conn(0x7f9b0815cf70 legacy=0x7f9b080d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0047c0 0x7f9aec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec0052a0 0x7f9aec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b080ce890 con 0x7f9aec003970 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec00c170 con 0x7f9aec0052a0 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec00c330 con 0x7f9aec0047c0 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 -- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0047c0 msgr2=0x7f9aec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0047c0 0x7f9aec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 -- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec0052a0 msgr2=0x7f9aec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec0052a0 0x7f9aec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b07e00640 1 -- 172.21.15.73:0/1972698676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec003520 con 0x7f9aec003970 2024-04-05T22:23:00.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.412+0000 7f9b0ca00640 1 -- 172.21.15.73:0/1972698676 reap_dead start 2024-04-05T22:23:00.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.413+0000 7f9b07e00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 0x7f9aec003d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9af800a800 tx=0x7f9aec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc76c57441627faf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80072d0 con 0x7f9aec003970 2024-04-05T22:23:00.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af802f070 con 0x7f9aec003970 2024-04-05T22:23:00.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b06a00640 1 -- 172.21.15.73:0/1972698676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80076f0 con 0x7f9aec003970 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 -- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 msgr2=0x7f9aec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 0x7f9aec003d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9af800a800 tx=0x7f9aec0057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 -- 172.21.15.73:0/1972698676 shutdown_connections 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/1972698676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 -- 172.21.15.73:0/1972698676 >> 172.21.15.73:0/1972698676 conn(0x7f9b080d2df0 msgr2=0x7f9b080d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 -- 172.21.15.73:0/1972698676 shutdown_connections 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.414+0000 7f9b0ea00640 1 -- 172.21.15.73:0/1972698676 wait complete. 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.416+0000 7f9b0ea00640 1 Processor -- start 2024-04-05T22:23:00.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.416+0000 7f9b0ea00640 1 -- start start 2024-04-05T22:23:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 0x7f9b0815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 0x7f9b080b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815a590 con 0x7f9b08158250 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b080c9ec0 con 0x7f9aec003970 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b0ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b080caa40 con 0x7f9b080b80d0 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36142/0 (socket says 172.21.15.73:36142) 2024-04-05T22:23:00.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.417+0000 7f9b07400640 1 -- 172.21.15.73:0/2970953877 learned_addr learned my addr 172.21.15.73:0/2970953877 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:00.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07e00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 0x7f9b0815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b0ca00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 0x7f9b080b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 -- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 msgr2=0x7f9b080b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 0x7f9b080b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 -- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 msgr2=0x7f9b0815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 0x7f9b0815a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 -- 172.21.15.73:0/2970953877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080ca110 con 0x7f9b08158250 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07e00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 0x7f9b0815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b0ca00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 0x7f9b080b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b07400640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c3c0 tx=0x7f9af000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00052d0 con 0x7f9b08158250 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0005430 con 0x7f9b08158250 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0007260 con 0x7f9b08158250 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.418+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b080ca3f0 con 0x7f9b08158250 2024-04-05T22:23:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.419+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b080cc930 con 0x7f9b08158250 2024-04-05T22:23:00.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.420+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b08068160 con 0x7f9b08158250 2024-04-05T22:23:00.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.421+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f9af00073c0 con 0x7f9b08158250 2024-04-05T22:23:00.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.421+0000 7f9affe00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 0x7f9ae406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.422+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9af0088770 con 0x7f9b08158250 2024-04-05T22:23:00.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.422+0000 7f9b07e00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 0x7f9ae406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.422+0000 7f9b07e00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 0x7f9ae406d2a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.426+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9af00539c0 con 0x7f9b08158250 2024-04-05T22:23:00.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.692+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f9b080cab80 con 0x7f9b08158250 2024-04-05T22:23:00.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.752+0000 7f9affe00640 1 -- 172.21.15.73:0/2970953877 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v115) v1 ==== 158+0+0 (secure 0 0 0) 0x7f9af0056fe0 con 0x7f9b08158250 2024-04-05T22:23:00.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 msgr2=0x7f9ae406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 0x7f9ae406d2a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af8007000 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 msgr2=0x7f9b080b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c3c0 tx=0x7f9af000c880 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 shutdown_connections 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ae406ade0 0x7f9ae406d2a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b080b80d0 0x7f9b080b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08158250 0x7f9b080b7b90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 --2- 172.21.15.73:0/2970953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec003970 0x7f9b0815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 >> 172.21.15.73:0/2970953877 conn(0x7f9b080d2df0 msgr2=0x7f9b08158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 shutdown_connections 2024-04-05T22:23:00.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.755+0000 7f9b0ea00640 1 -- 172.21.15.73:0/2970953877 wait complete. 2024-04-05T22:23:00.774 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-04-05T22:23:00.775 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:23:00.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.982+0000 7fae0ee00640 1 Processor -- start 2024-04-05T22:23:00.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.982+0000 7fae0ee00640 1 -- start start 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 0x7fae080cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae0815e850 0x7fae0815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0806f5b0 con 0x7fae080ccd20 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0815f240 con 0x7fae0815e850 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae08070410 con 0x7fae081587d0 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae0815e850 0x7fae0815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae0815e850 0x7fae0815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36146/0 (socket says 172.21.15.73:36146) 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.984+0000 7fae0ce00640 1 -- 172.21.15.73:0/3253062077 learned_addr learned my addr 172.21.15.73:0/3253062077 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae07e00640 1 --2- 172.21.15.73:0/3253062077 >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 0x7fae080cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae0ce00640 1 -- 172.21.15.73:0/3253062077 >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 msgr2=0x7fae080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae0ce00640 1 --2- 172.21.15.73:0/3253062077 >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 0x7fae080cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae0ce00640 1 -- 172.21.15.73:0/3253062077 >> v1:172.21.15.157:6789/0 conn(0x7fae080ccd20 legacy=0x7fae0806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae0ce00640 1 -- 172.21.15.73:0/3253062077 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08158bb0 con 0x7fae0815e850 2024-04-05T22:23:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.985+0000 7fae07e00640 1 --2- 172.21.15.73:0/3253062077 >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 0x7fae080cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.986+0000 7fae0ce00640 1 --2- 172.21.15.73:0/3253062077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae0815e850 0x7fae0815ed00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae00009e50 tx=0x7fae0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d07708992f9350 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.986+0000 7fae07400640 1 -- 172.21.15.73:0/3253062077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0002ca70 con 0x7fae0815e850 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.986+0000 7fae07400640 1 -- 172.21.15.73:0/3253062077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0002cbd0 con 0x7fae0815e850 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.986+0000 7fae07400640 1 -- 172.21.15.73:0/3253062077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000343f0 con 0x7fae0815e850 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 -- 172.21.15.73:0/3253062077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae0815e850 msgr2=0x7fae0815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3253062077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae0815e850 0x7fae0815ed00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae00009e50 tx=0x7fae0002bed0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 -- 172.21.15.73:0/3253062077 shutdown_connections 2024-04-05T22:23:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3253062077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae0815e850 0x7fae0815ed00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3253062077 >> v2:172.21.15.157:3300/0 conn(0x7fae081587d0 0x7fae080cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 -- 172.21.15.73:0/3253062077 >> 172.21.15.73:0/3253062077 conn(0x7fae080ce8e0 msgr2=0x7fae080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 -- 172.21.15.73:0/3253062077 shutdown_connections 2024-04-05T22:23:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.987+0000 7fae0ee00640 1 -- 172.21.15.73:0/3253062077 wait complete. 2024-04-05T22:23:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 Processor -- start 2024-04-05T22:23:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 -- start start 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae081587d0 0x7fae081443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae0815e850 0x7fae08153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.989+0000 7fae0ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08144910 con 0x7fae08154140 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08154650 con 0x7fae0815e850 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae08154990 con 0x7fae081587d0 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36150/0 (socket says 172.21.15.73:36150) 2024-04-05T22:23:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 -- 172.21.15.73:0/3843153275 learned_addr learned my addr 172.21.15.73:0/3843153275 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:00.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 -- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae081587d0 msgr2=0x7fae081443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae081587d0 0x7fae081443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 -- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae0815e850 msgr2=0x7fae08153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae0815e850 0x7fae08153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.990+0000 7fae0ce00640 1 -- 172.21.15.73:0/3843153275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00009b30 con 0x7fae08154140 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae0ce00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fae00036040 tx=0x7fae00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0003e450 con 0x7fae08154140 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0003e5b0 con 0x7fae08154140 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0003e960 con 0x7fae08154140 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae08154c70 con 0x7fae08154140 2024-04-05T22:23:00.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.991+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae08155230 con 0x7fae08154140 2024-04-05T22:23:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.992+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08066af0 con 0x7fae08154140 2024-04-05T22:23:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.993+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fae0003eac0 con 0x7fae08154140 2024-04-05T22:23:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.994+0000 7fae04c00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 0x7fade406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.994+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae000abfe0 con 0x7fae08154140 2024-04-05T22:23:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.994+0000 7fae07e00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 0x7fade406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.995+0000 7fae07e00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 0x7fade406d2a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006fd0 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:00.999+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae00077200 con 0x7fae08154140 2024-04-05T22:23:01.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.192+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae08068160 con 0x7fae08154140 2024-04-05T22:23:01.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.194+0000 7fae04c00640 1 -- 172.21.15.73:0/3843153275 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12518 (secure 0 0 0) 0x7fae0007a820 con 0x7fae08154140 2024-04-05T22:23:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:01.195 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":39,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:22:59.727477+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6836/2312639511":"2024-04-06T22:22:56.492683+0000","172.21.15.73:6834/2312639511":"2024-04-06T22:22:56.492683+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:01.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 msgr2=0x7fade406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 0x7fade406d2a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006fd0 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2024-04-05T22:23:01.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 msgr2=0x7fae080b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fae00036040 tx=0x7fae00004a40 comp rx=0 tx=0).stop 2024-04-05T22:23:01.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 shutdown_connections 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fade406ade0 0x7fade406d2a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae08154140 0x7fae080b2650 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae0815e850 0x7fae08153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 --2- 172.21.15.73:0/3843153275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae081587d0 0x7fae081443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 >> 172.21.15.73:0/3843153275 conn(0x7fae080ce8e0 msgr2=0x7fae0806c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 shutdown_connections 2024-04-05T22:23:01.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.197+0000 7fae0ee00640 1 -- 172.21.15.73:0/3843153275 wait complete. 2024-04-05T22:23:01.218 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:23:01.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:23:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.429+0000 7fc795a00640 1 Processor -- start 2024-04-05T22:23:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.430+0000 7fc795a00640 1 -- start start 2024-04-05T22:23:01.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc79015ee80 0x7fc7900cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7900b1950 0x7fc790164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc790165310 0x7fc790161850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7901657c0 con 0x7fc79015ee80 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc790070d20 con 0x7fc7900b1950 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc795a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79015f260 con 0x7fc790165310 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc78ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc79015ee80 0x7fc7900cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc78ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc79015ee80 0x7fc7900cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36154/0 (socket says 172.21.15.73:36154) 2024-04-05T22:23:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.431+0000 7fc78ea00640 1 -- 172.21.15.73:0/1909290379 learned_addr learned my addr 172.21.15.73:0/1909290379 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:01.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.432+0000 7fc78ea00640 1 -- 172.21.15.73:0/1909290379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc790165310 msgr2=0x7fc790161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.432+0000 7fc78ea00640 1 --2- 172.21.15.73:0/1909290379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc790165310 0x7fc790161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.432+0000 7fc78ea00640 1 -- 172.21.15.73:0/1909290379 >> v2:172.21.15.157:3300/0 conn(0x7fc7900b1950 msgr2=0x7fc790164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.432+0000 7fc78ea00640 1 --2- 172.21.15.73:0/1909290379 >> v2:172.21.15.157:3300/0 conn(0x7fc7900b1950 0x7fc790164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.432+0000 7fc78ea00640 1 -- 172.21.15.73:0/1909290379 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79006f5b0 con 0x7fc79015ee80 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.433+0000 7fc78ea00640 1 --2- 172.21.15.73:0/1909290379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79015ee80 0x7fc7900cecc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc79006d4a0 tx=0x7fc78002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8610e04463a5f4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.433+0000 7fc78e000640 1 -- 172.21.15.73:0/1909290379 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780031870 con 0x7fc79015ee80 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.433+0000 7fc78e000640 1 -- 172.21.15.73:0/1909290379 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7800319d0 con 0x7fc79015ee80 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.433+0000 7fc78e000640 1 -- 172.21.15.73:0/1909290379 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780031d60 con 0x7fc79015ee80 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 -- 172.21.15.73:0/1909290379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79015ee80 msgr2=0x7fc7900cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 --2- 172.21.15.73:0/1909290379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79015ee80 0x7fc7900cecc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc79006d4a0 tx=0x7fc78002b910 comp rx=0 tx=0).stop 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 -- 172.21.15.73:0/1909290379 shutdown_connections 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 --2- 172.21.15.73:0/1909290379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc790165310 0x7fc790161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 --2- 172.21.15.73:0/1909290379 >> v2:172.21.15.157:3300/0 conn(0x7fc7900b1950 0x7fc790164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 --2- 172.21.15.73:0/1909290379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79015ee80 0x7fc7900cecc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 -- 172.21.15.73:0/1909290379 >> 172.21.15.73:0/1909290379 conn(0x7fc790069cb0 msgr2=0x7fc79006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:01.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 -- 172.21.15.73:0/1909290379 shutdown_connections 2024-04-05T22:23:01.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.434+0000 7fc795a00640 1 -- 172.21.15.73:0/1909290379 wait complete. 2024-04-05T22:23:01.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.436+0000 7fc795a00640 1 Processor -- start 2024-04-05T22:23:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.437+0000 7fc795a00640 1 -- start start 2024-04-05T22:23:01.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 0x7fc790145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 0x7fc79015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901485f0 con 0x7fc790165310 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901493f0 con 0x7fc7900b1950 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc795a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901487e0 con 0x7fc79015ee80 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc78f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc78f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36166/0 (socket says 172.21.15.73:36166) 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc78f400640 1 -- 172.21.15.73:0/3123601243 learned_addr learned my addr 172.21.15.73:0/3123601243 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc787e00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 0x7fc79015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.438+0000 7fc78ea00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 0x7fc790145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 -- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 msgr2=0x7fc79015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 0x7fc79015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 -- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 msgr2=0x7fc790145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 0x7fc790145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 -- 172.21.15.73:0/3123601243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800094d0 con 0x7fc790165310 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc787e00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 0x7fc79015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78ea00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 0x7fc790145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc78f400640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b970 tx=0x7fc78800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800cb40 con 0x7fc790165310 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800ccc0 con 0x7fc790165310 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc790148ac0 con 0x7fc790165310 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788014680 con 0x7fc790165310 2024-04-05T22:23:01.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.439+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc790149080 con 0x7fc790165310 2024-04-05T22:23:01.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.441+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7901495a0 con 0x7fc790165310 2024-04-05T22:23:01.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.441+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc78801e440 con 0x7fc790165310 2024-04-05T22:23:01.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.442+0000 7fc786a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 0x7fc76c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.442+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc7880892b0 con 0x7fc790165310 2024-04-05T22:23:01.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.442+0000 7fc78ea00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 0x7fc76c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.443+0000 7fc78ea00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 0x7fc76c06d270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc780036000 tx=0x7fc78003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.446+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc7901495a0 con 0x7fc790165310 2024-04-05T22:23:01.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.639+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc790149560 con 0x7fc790165310 2024-04-05T22:23:01.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.640+0000 7fc786a00640 1 -- 172.21.15.73:0/3123601243 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc788054510 con 0x7fc790165310 2024-04-05T22:23:01.641 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 msgr2=0x7fc76c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 0x7fc76c06d270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc780036000 tx=0x7fc78003c040 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 msgr2=0x7fc7900b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b970 tx=0x7fc78800be30 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 shutdown_connections 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc76c06adb0 0x7fc76c06d270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc790165310 0x7fc7900b1da0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ee80 0x7fc79015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 --2- 172.21.15.73:0/3123601243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7900b1950 0x7fc790145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.643+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 >> 172.21.15.73:0/3123601243 conn(0x7fc790069cb0 msgr2=0x7fc79015d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.644+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 shutdown_connections 2024-04-05T22:23:01.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.644+0000 7fc795a00640 1 -- 172.21.15.73:0/3123601243 wait complete. 2024-04-05T22:23:01.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-05T22:23:01.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.865+0000 7f6e88a00640 1 Processor -- start 2024-04-05T22:23:01.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.865+0000 7f6e88a00640 1 -- start start 2024-04-05T22:23:01.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e88a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e840caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e88a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e840b1870 0x7f6e840ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e88a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e840ce5d0 con 0x7f6e840ad6b0 2024-04-05T22:23:01.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e88a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e840ce010 con 0x7f6e840b1870 2024-04-05T22:23:01.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e88a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840ce2f0 con 0x7f6e84160620 2024-04-05T22:23:01.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e82800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6e840ad6b0 0x7f6e840cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41078/0 (socket says 172.21.15.73:41078) 2024-04-05T22:23:01.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.867+0000 7f6e82800640 1 -- 172.21.15.73:0/2727516463 learned_addr learned my addr 172.21.15.73:0/2727516463 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81e00640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e840caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e79400640 1 --2- 172.21.15.73:0/2727516463 >> v2:172.21.15.157:3300/0 conn(0x7f6e840b1870 0x7f6e840ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879650438 0 0) 0x7f6e840ce5d0 con 0x7f6e840ad6b0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003520 con 0x7f6e840ad6b0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1136523831 0 0) 0x7f6e68003520 con 0x7f6e840ad6b0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 msgr2=0x7f6e840caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e840caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 >> v2:172.21.15.157:3300/0 conn(0x7f6e840b1870 msgr2=0x7f6e840ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 --2- 172.21.15.73:0/2727516463 >> v2:172.21.15.157:3300/0 conn(0x7f6e840b1870 0x7f6e840ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840ac970 con 0x7f6e840ad6b0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.868+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6e7c0019f0 con 0x7f6e840ad6b0 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 >> v1:172.21.15.73:6789/0 conn(0x7f6e840ad6b0 legacy=0x7f6e840cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e68003970 0x7f6e68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e68004d00 0x7f6e68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840ce010 con 0x7f6e68004d00 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840ce2f0 con 0x7f6e68003970 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6800b980 con 0x7f6e84160620 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e68004d00 0x7f6e68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e81e00640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e68003970 0x7f6e68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e79400640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 -- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 msgr2=0x7f6e680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e84160620 0x7f6e680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 -- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e68003970 msgr2=0x7f6e68003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e68003970 0x7f6e68003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.869+0000 7f6e82800640 1 -- 172.21.15.73:0/2727516463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68003520 con 0x7f6e68004d00 2024-04-05T22:23:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.870+0000 7f6e82800640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e68004d00 0x7f6e68005180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00a8d0 tx=0x7f6e680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88943777536611c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c034630 con 0x7f6e68004d00 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c02f070 con 0x7f6e68004d00 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e81400640 1 -- 172.21.15.73:0/2727516463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0349d0 con 0x7f6e68004d00 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 -- 172.21.15.73:0/2727516463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e68004d00 msgr2=0x7f6e68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 --2- 172.21.15.73:0/2727516463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e68004d00 0x7f6e68005180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00a8d0 tx=0x7f6e680057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e82800640 1 -- 172.21.15.73:0/2727516463 reap_dead start 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 -- 172.21.15.73:0/2727516463 shutdown_connections 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 -- 172.21.15.73:0/2727516463 >> 172.21.15.73:0/2727516463 conn(0x7f6e840ce8e0 msgr2=0x7f6e840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 -- 172.21.15.73:0/2727516463 shutdown_connections 2024-04-05T22:23:01.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.871+0000 7f6e88a00640 1 -- 172.21.15.73:0/2727516463 wait complete. 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.873+0000 7f6e88a00640 1 Processor -- start 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.873+0000 7f6e88a00640 1 -- start start 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8414ef70 0x7f6e8415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8415d3f0 0x7f6e840ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84150b20 con 0x7f6e8414e690 2024-04-05T22:23:01.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8415d870 con 0x7f6e8414ef70 2024-04-05T22:23:01.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e88a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84155360 con 0x7f6e8415d3f0 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e81e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e81e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36182/0 (socket says 172.21.15.73:36182) 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.874+0000 7f6e81e00640 1 -- 172.21.15.73:0/3233495937 learned_addr learned my addr 172.21.15.73:0/3233495937 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e82800640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8415d3f0 0x7f6e840ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 -- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8415d3f0 msgr2=0x7f6e840ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8415d3f0 0x7f6e840ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 -- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8414ef70 msgr2=0x7f6e8415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e79400640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8414ef70 0x7f6e8415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8414ef70 0x7f6e8415ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 -- 172.21.15.73:0/3233495937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84153390 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e81e00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6e74010860 tx=0x7f6e74010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.875+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009610 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.876+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7400eb60 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.876+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7401c400 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.876+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e84153df0 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.876+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841543b0 con 0x7f6e8414e690 2024-04-05T22:23:01.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.877+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e84066af0 con 0x7f6e8414e690 2024-04-05T22:23:01.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.878+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f6e7401c560 con 0x7f6e8414e690 2024-04-05T22:23:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.879+0000 7f6e7aa00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 0x7f6e6006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.879+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e740898e0 con 0x7f6e8414e690 2024-04-05T22:23:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.879+0000 7f6e79400640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 0x7f6e6006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:01.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.880+0000 7f6e79400640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 0x7f6e6006d270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6e840b0540 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:01.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:01.884+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6e74054bf0 con 0x7f6e8414e690 2024-04-05T22:23:02.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.139+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f6e841545d0 con 0x7f6e8414e690 2024-04-05T22:23:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.763+0000 7f6e7aa00640 1 -- 172.21.15.73:0/3233495937 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v117) v1 ==== 143+0+0 (secure 0 0 0) 0x7f6e74058210 con 0x7f6e8414e690 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.766+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 msgr2=0x7f6e6006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.766+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 0x7f6e6006d270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6e840b0540 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.766+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 msgr2=0x7f6e84152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.766+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6e74010860 tx=0x7f6e74010d20 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 shutdown_connections 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6e6006adb0 0x7f6e6006d270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8415d3f0 0x7f6e840ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8414ef70 0x7f6e8415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 --2- 172.21.15.73:0/3233495937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8414e690 0x7f6e84152d40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 >> 172.21.15.73:0/3233495937 conn(0x7f6e840ce8e0 msgr2=0x7f6e840b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:02.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 shutdown_connections 2024-04-05T22:23:02.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.767+0000 7f6e88a00640 1 -- 172.21.15.73:0/3233495937 wait complete. 2024-04-05T22:23:02.791 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:23:02.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.999+0000 7fb1c5400640 1 Processor -- start 2024-04-05T22:23:02.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:02.999+0000 7fb1c5400640 1 -- start start 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.000+0000 7fb1c5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c015abe0 0x7fb1c00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.000+0000 7fb1c5400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1c006c9a0 0x7fb1c006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.000+0000 7fb1c5400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c006bea0 con 0x7fb1c006ba70 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.000+0000 7fb1c5400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c015afc0 con 0x7fb1c015abe0 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.000+0000 7fb1c5400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c006f5b0 con 0x7fb1c006c9a0 2024-04-05T22:23:03.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c015abe0 0x7fb1c00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1bf200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb1c006ba70 0x7fb1c00cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41092/0 (socket says 172.21.15.73:41092) 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c015abe0 0x7fb1c00712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36194/0 (socket says 172.21.15.73:36194) 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1bf200640 1 -- 172.21.15.73:0/982307469 learned_addr learned my addr 172.21.15.73:0/982307469 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1bde00640 1 --2- 172.21.15.73:0/982307469 >> v2:172.21.15.157:3300/0 conn(0x7fb1c006c9a0 0x7fb1c006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 -- 172.21.15.73:0/982307469 >> v2:172.21.15.157:3300/0 conn(0x7fb1c006c9a0 msgr2=0x7fb1c006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 --2- 172.21.15.73:0/982307469 >> v2:172.21.15.157:3300/0 conn(0x7fb1c006c9a0 0x7fb1c006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 -- 172.21.15.73:0/982307469 >> v1:172.21.15.73:6789/0 conn(0x7fb1c006ba70 legacy=0x7fb1c00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.001+0000 7fb1be800640 1 -- 172.21.15.73:0/982307469 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c006f890 con 0x7fb1c015abe0 2024-04-05T22:23:03.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.002+0000 7fb1be800640 1 --2- 172.21.15.73:0/982307469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c015abe0 0x7fb1c00712e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb1c006ce30 tx=0x7fb1b002bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e31eaa6058d969 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.002+0000 7fb1bd400640 1 -- 172.21.15.73:0/982307469 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b002cb50 con 0x7fb1c015abe0 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1bd400640 1 -- 172.21.15.73:0/982307469 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b002ccb0 con 0x7fb1c015abe0 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1bd400640 1 -- 172.21.15.73:0/982307469 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0034660 con 0x7fb1c015abe0 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 -- 172.21.15.73:0/982307469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c015abe0 msgr2=0x7fb1c00712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 --2- 172.21.15.73:0/982307469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c015abe0 0x7fb1c00712e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb1c006ce30 tx=0x7fb1b002bda0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 -- 172.21.15.73:0/982307469 shutdown_connections 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 --2- 172.21.15.73:0/982307469 >> v2:172.21.15.157:3300/0 conn(0x7fb1c006c9a0 0x7fb1c006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 --2- 172.21.15.73:0/982307469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c015abe0 0x7fb1c00712e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 -- 172.21.15.73:0/982307469 >> 172.21.15.73:0/982307469 conn(0x7fb1c0069cb0 msgr2=0x7fb1c006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 -- 172.21.15.73:0/982307469 shutdown_connections 2024-04-05T22:23:03.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.003+0000 7fb1c5400640 1 -- 172.21.15.73:0/982307469 wait complete. 2024-04-05T22:23:03.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.005+0000 7fb1c5400640 1 Processor -- start 2024-04-05T22:23:03.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.005+0000 7fb1c5400640 1 -- start start 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c006c9a0 0x7fb1c014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1c015abe0 0x7fb1c01f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c014da30 con 0x7fb1c006ba70 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01f1650 con 0x7fb1c006c9a0 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1c5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01f1930 con 0x7fb1c015abe0 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1be800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1be800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36204/0 (socket says 172.21.15.73:36204) 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1be800640 1 -- 172.21.15.73:0/2832735721 learned_addr learned my addr 172.21.15.73:0/2832735721 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1bde00640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c006c9a0 0x7fb1c014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1be800640 1 -- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1c015abe0 msgr2=0x7fb1c01f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.006+0000 7fb1be800640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1c015abe0 0x7fb1c01f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1be800640 1 -- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c006c9a0 msgr2=0x7fb1c014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1be800640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c006c9a0 0x7fb1c014d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1be800640 1 -- 172.21.15.73:0/2832735721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0009a70 con 0x7fb1c006ba70 2024-04-05T22:23:03.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1be800640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0037040 tx=0x7fb1b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0040650 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b00407b0 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c01f1c10 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0040b60 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.007+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01f0850 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.008+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c006fe10 con 0x7fb1c006ba70 2024-04-05T22:23:03.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.009+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fb1b0040cc0 con 0x7fb1c006ba70 2024-04-05T22:23:03.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.010+0000 7fb1b6a00640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 0x7fb1a006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.010+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1b00ac370 con 0x7fb1c006ba70 2024-04-05T22:23:03.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.010+0000 7fb1bde00640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 0x7fb1a006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.011+0000 7fb1bde00640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 0x7fb1a006d270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.015+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb1b0077650 con 0x7fb1c006ba70 2024-04-05T22:23:03.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.206+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1c0068160 con 0x7fb1c006ba70 2024-04-05T22:23:03.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.212+0000 7fb1b6a00640 1 -- 172.21.15.73:0/2832735721 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1b007ac70 con 0x7fb1c006ba70 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.214+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 msgr2=0x7fb1a006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.214+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 0x7fb1a006d270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac008040 comp rx=0 tx=0).stop 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.214+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 msgr2=0x7fb1c014cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.214+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0037040 tx=0x7fb1b0004a40 comp rx=0 tx=0).stop 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 shutdown_connections 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb1a006adb0 0x7fb1a006d270 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1c015abe0 0x7fb1c01f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c006c9a0 0x7fb1c014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 --2- 172.21.15.73:0/2832735721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c006ba70 0x7fb1c014cc90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 >> 172.21.15.73:0/2832735721 conn(0x7fb1c0069cb0 msgr2=0x7fb1c015b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:03.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 shutdown_connections 2024-04-05T22:23:03.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.215+0000 7fb1c5400640 1 -- 172.21.15.73:0/2832735721 wait complete. 2024-04-05T22:23:03.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:23:03.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:23:03.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:23:03.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:23:03.466 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:23:03.466 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:03.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:23:03.524 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:23:03.524 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:03.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:23:03.581 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:23:03.581 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:03.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:23:03.638 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:23:03.638 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:03.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:23:03.695 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:23:03.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:03.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.910+0000 7fb8d1800640 1 Processor -- start 2024-04-05T22:23:03.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.910+0000 7fb8d1800640 1 -- start start 2024-04-05T22:23:03.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.911+0000 7fb8d1800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8cc1587d0 0x7fb8cc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.911+0000 7fb8d1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 0x7fb8cc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8d1800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc06f5b0 con 0x7fb8cc06d0c0 2024-04-05T22:23:03.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8d1800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc070410 con 0x7fb8cc1587d0 2024-04-05T22:23:03.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8d1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc158bb0 con 0x7fb8cc0ccd20 2024-04-05T22:23:03.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8c2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 0x7fb8cc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8cb400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb8cc06d0c0 0x7fb8cc15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:50226/0 (socket says 172.21.15.73:50226) 2024-04-05T22:23:03.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8cb400640 1 -- 172.21.15.73:0/354426385 learned_addr learned my addr 172.21.15.73:0/354426385 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:03.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.912+0000 7fb8caa00640 1 --2- 172.21.15.73:0/354426385 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc1587d0 0x7fb8cc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.913+0000 7fb8c2000640 1 -- 172.21.15.73:0/354426385 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc1587d0 msgr2=0x7fb8cc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.913+0000 7fb8c2000640 1 --2- 172.21.15.73:0/354426385 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc1587d0 0x7fb8cc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.913+0000 7fb8c2000640 1 -- 172.21.15.73:0/354426385 >> v1:172.21.15.157:6789/0 conn(0x7fb8cc06d0c0 legacy=0x7fb8cc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.913+0000 7fb8c2000640 1 -- 172.21.15.73:0/354426385 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc06f800 con 0x7fb8cc0ccd20 2024-04-05T22:23:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.915+0000 7fb8c2000640 1 --2- 172.21.15.73:0/354426385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 0x7fb8cc06cb80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40097a0 tx=0x7fb8b402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53088d576103962 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.915+0000 7fb8ca000640 1 -- 172.21.15.73:0/354426385 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4031830 con 0x7fb8cc0ccd20 2024-04-05T22:23:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.915+0000 7fb8ca000640 1 -- 172.21.15.73:0/354426385 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b4031990 con 0x7fb8cc0ccd20 2024-04-05T22:23:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.915+0000 7fb8ca000640 1 -- 172.21.15.73:0/354426385 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4031d20 con 0x7fb8cc0ccd20 2024-04-05T22:23:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 -- 172.21.15.73:0/354426385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 msgr2=0x7fb8cc06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 --2- 172.21.15.73:0/354426385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 0x7fb8cc06cb80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40097a0 tx=0x7fb8b402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 -- 172.21.15.73:0/354426385 shutdown_connections 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 --2- 172.21.15.73:0/354426385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ccd20 0x7fb8cc06cb80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 --2- 172.21.15.73:0/354426385 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc1587d0 0x7fb8cc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 -- 172.21.15.73:0/354426385 >> 172.21.15.73:0/354426385 conn(0x7fb8cc0ce8e0 msgr2=0x7fb8cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 -- 172.21.15.73:0/354426385 shutdown_connections 2024-04-05T22:23:03.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.916+0000 7fb8d1800640 1 -- 172.21.15.73:0/354426385 wait complete. 2024-04-05T22:23:03.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.918+0000 7fb8d1800640 1 Processor -- start 2024-04-05T22:23:03.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.918+0000 7fb8d1800640 1 -- start start 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 0x7fb8cc142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0c41f0 0x7fb8cc0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1f2600 con 0x7fb8cc1587d0 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0c4cb0 con 0x7fb8cc06d0c0 2024-04-05T22:23:03.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8d1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1413c0 con 0x7fb8cc0c41f0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36212/0 (socket says 172.21.15.73:36212) 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 -- 172.21.15.73:0/445660322 learned_addr learned my addr 172.21.15.73:0/445660322 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8caa00640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 0x7fb8cc142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8cb400640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0c41f0 0x7fb8cc0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 -- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0c41f0 msgr2=0x7fb8cc0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0c41f0 0x7fb8cc0c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 -- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 msgr2=0x7fb8cc142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 0x7fb8cc142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8c2000640 1 -- 172.21.15.73:0/445660322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009400 con 0x7fb8cc1587d0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.920+0000 7fb8caa00640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 0x7fb8cc142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8c2000640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb8b402bee0 tx=0x7fb8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403a6f0 con 0x7fb8cc1587d0 2024-04-05T22:23:03.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b403a870 con 0x7fb8cc1587d0 2024-04-05T22:23:03.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc1416a0 con 0x7fb8cc1587d0 2024-04-05T22:23:03.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403ac20 con 0x7fb8cc1587d0 2024-04-05T22:23:03.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.921+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc141c60 con 0x7fb8cc1587d0 2024-04-05T22:23:03.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.922+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc066af0 con 0x7fb8cc1587d0 2024-04-05T22:23:03.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.923+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fb8b404c3b0 con 0x7fb8cc1587d0 2024-04-05T22:23:03.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.924+0000 7fb8c3400640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 0x7fb8a806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:03.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.925+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8b40ab190 con 0x7fb8cc1587d0 2024-04-05T22:23:03.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.925+0000 7fb8caa00640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 0x7fb8a806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:03.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.925+0000 7fb8caa00640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 0x7fb8a806d270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009a20 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:03.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:03.929+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb8b4076400 con 0x7fb8cc1587d0 2024-04-05T22:23:04.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.159+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8cc15c190 con 0x7fb8cc1587d0 2024-04-05T22:23:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.160+0000 7fb8c3400640 1 -- 172.21.15.73:0/445660322 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fb8b4048de0 con 0x7fb8cc1587d0 2024-04-05T22:23:04.160 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:04.161 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/944464613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":944464613},{"type":"v1","addr":"172.21.15.73:6837","nonce":944464613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3376516923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3376516923},{"type":"v1","addr":"172.21.15.157:6835","nonce":3376516923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1995061359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1995061359},{"type":"v1","addr":"172.21.15.157:6834","nonce":1995061359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/685393399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":685393399},{"type":"v1","addr":"172.21.15.73:6836","nonce":685393399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2024-04-05T22:23:04.161 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 18 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.162+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 msgr2=0x7fb8a806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.162+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 0x7fb8a806d270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009a20 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.162+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 msgr2=0x7fb8cc1f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.162+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb8b402bee0 tx=0x7fb8b40046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 shutdown_connections 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb8a806adb0 0x7fb8a806d270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0c41f0 0x7fb8cc0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8cc1587d0 0x7fb8cc1f20c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 --2- 172.21.15.73:0/445660322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc06d0c0 0x7fb8cc142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 >> 172.21.15.73:0/445660322 conn(0x7fb8cc0ce8e0 msgr2=0x7fb8cc0add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:04.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 shutdown_connections 2024-04-05T22:23:04.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.163+0000 7fb8d1800640 1 -- 172.21.15.73:0/445660322 wait complete. 2024-04-05T22:23:04.183 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:23:04.183 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:23:04.183 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:23:04.183 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:23:04.184 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:23:04.184 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:23:04.184 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:23:04.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:23:04.184 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:23:04.184 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:23:04.184 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:23:04.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:23:04.185 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:23:04.185 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:23:04.185 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:23:04.185 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:23:04.201 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:23:04.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:23:04.202 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:23:04.202 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:23:04.205 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:23:04.205 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:23:04.207 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:23:04.207 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:23:04.248 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:23:04.248 DEBUG:teuthology.parallel:result is None 2024-04-05T22:23:04.253 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:23:04.254 DEBUG:teuthology.parallel:result is None 2024-04-05T22:23:04.254 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:23:04.255 DEBUG:teuthology.parallel:result is None 2024-04-05T22:23:04.260 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:23:04.260 DEBUG:teuthology.parallel:result is None 2024-04-05T22:23:04.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:23:04.400 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:23:04.401 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:23:04.415 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:23:04.416 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:23:04.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.453+0000 7f248a600640 1 Processor -- start 2024-04-05T22:23:04.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.454+0000 7f248a600640 1 -- start start 2024-04-05T22:23:04.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f248a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f248406cb80 0x7f248406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f248a600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 0x7f248415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f248a600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248406f5b0 con 0x7f24841587d0 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f248a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248406d500 con 0x7f248406cb80 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f248a600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248415f220 con 0x7f248415e850 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2482a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f248406cb80 0x7f248406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2483400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f24841587d0 0x7f24840cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41100/0 (socket says 172.21.15.73:41100) 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2483400640 1 -- 172.21.15.73:0/1721642401 learned_addr learned my addr 172.21.15.73:0/1721642401 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2482a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f248406cb80 0x7f248406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36246/0 (socket says 172.21.15.73:36246) 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2483e00640 1 --2- 172.21.15.73:0/1721642401 >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 0x7f248415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2482a00640 1 -- 172.21.15.73:0/1721642401 >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 msgr2=0x7f248415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2482a00640 1 --2- 172.21.15.73:0/1721642401 >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 0x7f248415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.455+0000 7f2482a00640 1 -- 172.21.15.73:0/1721642401 >> v1:172.21.15.73:6789/0 conn(0x7f24841587d0 legacy=0x7f24840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.456+0000 7f2482a00640 1 -- 172.21.15.73:0/1721642401 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2484070410 con 0x7f248406cb80 2024-04-05T22:23:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.456+0000 7f2482000640 1 -- 172.21.15.73:0/1721642401 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759148698 0 0) 0x7f248406f5b0 con 0x7f24841587d0 2024-04-05T22:23:04.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.456+0000 7f2483e00640 1 --2- 172.21.15.73:0/1721642401 >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 0x7f248415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f2482a00640 1 --2- 172.21.15.73:0/1721642401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f248406cfc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f248415c270 tx=0x7f247002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4821a0c08efc49bb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f2482000640 1 -- 172.21.15.73:0/1721642401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470031830 con 0x7f248406cb80 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f2482000640 1 -- 172.21.15.73:0/1721642401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470031990 con 0x7f248406cb80 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f2482000640 1 -- 172.21.15.73:0/1721642401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470031d20 con 0x7f248406cb80 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f248a600640 1 -- 172.21.15.73:0/1721642401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 msgr2=0x7f248406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.457+0000 7f248a600640 1 --2- 172.21.15.73:0/1721642401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f248406cfc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f248415c270 tx=0x7f247002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 -- 172.21.15.73:0/1721642401 shutdown_connections 2024-04-05T22:23:04.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 --2- 172.21.15.73:0/1721642401 >> v2:172.21.15.157:3300/0 conn(0x7f248415e850 0x7f248415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 --2- 172.21.15.73:0/1721642401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f248406cfc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 -- 172.21.15.73:0/1721642401 >> 172.21.15.73:0/1721642401 conn(0x7f24840ce8e0 msgr2=0x7f24840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 -- 172.21.15.73:0/1721642401 shutdown_connections 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.458+0000 7f248a600640 1 -- 172.21.15.73:0/1721642401 wait complete. 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.459+0000 7f248a600640 1 Processor -- start 2024-04-05T22:23:04.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.459+0000 7f248a600640 1 -- start start 2024-04-05T22:23:04.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.460+0000 7f248a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.460+0000 7f248a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 0x7f24841f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.460+0000 7f248a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f248415e850 0x7f24841f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.460+0000 7f248a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24841f6ad0 con 0x7f248406cb80 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f248a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484156de0 con 0x7f24841587d0 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f248a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24841f2510 con 0x7f248415e850 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36260/0 (socket says 172.21.15.73:36260) 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 -- 172.21.15.73:0/1719226132 learned_addr learned my addr 172.21.15.73:0/1719226132 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483e00640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f248415e850 0x7f24841f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2482a00640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 0x7f24841f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 -- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f248415e850 msgr2=0x7f24841f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f248415e850 0x7f24841f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 -- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 msgr2=0x7f24841f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 0x7f24841f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 -- 172.21.15.73:0/1719226132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009400 con 0x7f248406cb80 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2482a00640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 0x7f24841f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:04.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.461+0000 7f2483400640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f247400c500 tx=0x7f247400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:04.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.462+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474007560 con 0x7f248406cb80 2024-04-05T22:23:04.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.462+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24740047e0 con 0x7f248406cb80 2024-04-05T22:23:04.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.462+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474004b90 con 0x7f248406cb80 2024-04-05T22:23:04.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.462+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24841f6510 con 0x7f248406cb80 2024-04-05T22:23:04.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.462+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24841f2750 con 0x7f248406cb80 2024-04-05T22:23:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.463+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484066af0 con 0x7f248406cb80 2024-04-05T22:23:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.464+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2474004cf0 con 0x7f248406cb80 2024-04-05T22:23:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.465+0000 7f247b400640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 0x7f246406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.465+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2474088460 con 0x7f248406cb80 2024-04-05T22:23:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.465+0000 7f2482a00640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 0x7f246406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:04.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.466+0000 7f2482a00640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 0x7f246406d270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2470036000 tx=0x7f2470008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:04.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.469+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2474053760 con 0x7f248406cb80 2024-04-05T22:23:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:04.667+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2484074210 con 0x7f248406cb80 2024-04-05T22:23:05.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.455+0000 7f247b400640 1 -- 172.21.15.73:0/1719226132 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2474018080 con 0x7f248406cb80 2024-04-05T22:23:05.456 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:23:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.458+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 msgr2=0x7f246406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.458+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 0x7f246406d270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2470036000 tx=0x7f2470008030 comp rx=0 tx=0).stop 2024-04-05T22:23:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.458+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 msgr2=0x7f24841568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.458+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f247400c500 tx=0x7f247400c9c0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 shutdown_connections 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f246406adb0 0x7f246406d270 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f248415e850 0x7f24841f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24841587d0 0x7f24841f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 --2- 172.21.15.73:0/1719226132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f248406cb80 0x7f24841568a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 >> 172.21.15.73:0/1719226132 conn(0x7f24840ce8e0 msgr2=0x7f24840708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 shutdown_connections 2024-04-05T22:23:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.459+0000 7f248a600640 1 -- 172.21.15.73:0/1719226132 wait complete. 2024-04-05T22:23:05.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:23:05.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.687+0000 7f64d0c00640 1 Processor -- start 2024-04-05T22:23:05.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.687+0000 7f64d0c00640 1 -- start start 2024-04-05T22:23:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64d0c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64cc15ccd0 0x7f64cc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64d0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64d0c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0b2420 con 0x7f64cc0d35e0 2024-04-05T22:23:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64d0c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc0b2770 con 0x7f64cc15ccd0 2024-04-05T22:23:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64d0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0ce380 con 0x7f64cc0d1600 2024-04-05T22:23:05.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.689+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58302/0 (socket says 172.21.15.73:58302) 2024-04-05T22:23:05.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 -- 172.21.15.73:0/3460126834 learned_addr learned my addr 172.21.15.73:0/3460126834 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:05.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cb400640 1 --2- 172.21.15.73:0/3460126834 >> v2:172.21.15.157:3300/0 conn(0x7f64cc15ccd0 0x7f64cc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 -- 172.21.15.73:0/3460126834 >> v2:172.21.15.157:3300/0 conn(0x7f64cc15ccd0 msgr2=0x7f64cc15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:05.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 --2- 172.21.15.73:0/3460126834 >> v2:172.21.15.157:3300/0 conn(0x7f64cc15ccd0 0x7f64cc15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 -- 172.21.15.73:0/3460126834 >> v1:172.21.15.157:6789/0 conn(0x7f64cc0d35e0 legacy=0x7f64cc0d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.690+0000 7f64cbe00640 1 -- 172.21.15.73:0/3460126834 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0ce660 con 0x7f64cc0d1600 2024-04-05T22:23:05.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.692+0000 7f64cbe00640 1 --2- 172.21.15.73:0/3460126834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009870 tx=0x7f64c402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e3297791e58eed server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:05.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.692+0000 7f64ca000640 1 -- 172.21.15.73:0/3460126834 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40317c0 con 0x7f64cc0d1600 2024-04-05T22:23:05.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.692+0000 7f64ca000640 1 -- 172.21.15.73:0/3460126834 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4031920 con 0x7f64cc0d1600 2024-04-05T22:23:05.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.692+0000 7f64ca000640 1 -- 172.21.15.73:0/3460126834 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4031cb0 con 0x7f64cc0d1600 2024-04-05T22:23:05.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 -- 172.21.15.73:0/3460126834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 msgr2=0x7f64cc0d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 --2- 172.21.15.73:0/3460126834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009870 tx=0x7f64c402b860 comp rx=0 tx=0).stop 2024-04-05T22:23:05.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 -- 172.21.15.73:0/3460126834 shutdown_connections 2024-04-05T22:23:05.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 --2- 172.21.15.73:0/3460126834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc0d08a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 --2- 172.21.15.73:0/3460126834 >> v2:172.21.15.157:3300/0 conn(0x7f64cc15ccd0 0x7f64cc15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 -- 172.21.15.73:0/3460126834 >> 172.21.15.73:0/3460126834 conn(0x7f64cc0d2ac0 msgr2=0x7f64cc0d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:05.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 -- 172.21.15.73:0/3460126834 shutdown_connections 2024-04-05T22:23:05.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.693+0000 7f64d0c00640 1 -- 172.21.15.73:0/3460126834 wait complete. 2024-04-05T22:23:05.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.695+0000 7f64d0c00640 1 Processor -- start 2024-04-05T22:23:05.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.695+0000 7f64d0c00640 1 -- start start 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc1f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64cc15ccd0 0x7f64cc09cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc14c2d0 con 0x7f64cc0d35e0 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc14c5b0 con 0x7f64cc15ccd0 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64d0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc14c890 con 0x7f64cc0d1600 2024-04-05T22:23:05.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36272/0 (socket says 172.21.15.73:36272) 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 -- 172.21.15.73:0/2042284867 learned_addr learned my addr 172.21.15.73:0/2042284867 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64cbe00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64cc15ccd0 0x7f64cc09cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64cb400640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc1f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 -- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 msgr2=0x7f64cc1f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc1f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 -- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64cc15ccd0 msgr2=0x7f64cc09cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64cc15ccd0 0x7f64cc09cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.696+0000 7f64caa00640 1 -- 172.21.15.73:0/2042284867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40094d0 con 0x7f64cc0d35e0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64caa00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c870 tx=0x7f64b400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4003b70 con 0x7f64cc0d35e0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4003cf0 con 0x7f64cc0d35e0 2024-04-05T22:23:05.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc14cb70 con 0x7f64cc0d35e0 2024-04-05T22:23:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b400aa90 con 0x7f64cc0d35e0 2024-04-05T22:23:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.697+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc15c160 con 0x7f64cc0d35e0 2024-04-05T22:23:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.698+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc14d090 con 0x7f64cc0d35e0 2024-04-05T22:23:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.699+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f64b400abf0 con 0x7f64cc0d35e0 2024-04-05T22:23:05.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.700+0000 7f64c3400640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 0x7f64a806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:05.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.700+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f64b4089210 con 0x7f64cc0d35e0 2024-04-05T22:23:05.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.700+0000 7f64cb400640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 0x7f64a806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:05.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.701+0000 7f64cb400640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 0x7f64a806d270 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009970 tx=0x7f64bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:05.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.705+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f64cc14d090 con 0x7f64cc0d35e0 2024-04-05T22:23:05.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.951+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f64cc14d090 con 0x7f64cc0d35e0 2024-04-05T22:23:05.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.952+0000 7f64c3400640 1 -- 172.21.15.73:0/2042284867 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f64cc14d090 con 0x7f64cc0d35e0 2024-04-05T22:23:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:05.952 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:23:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:23:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:23:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:23:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:23:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:23:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:23:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:23:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:23:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:23:05.965 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.954+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 msgr2=0x7f64a806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.954+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 0x7f64a806d270 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009970 tx=0x7f64bc008040 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.954+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 msgr2=0x7f64cc09c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.954+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c870 tx=0x7f64b400cd30 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 shutdown_connections 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f64a806adb0 0x7f64a806d270 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64cc15ccd0 0x7f64cc09cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc0d35e0 0x7f64cc09c7a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 --2- 172.21.15.73:0/2042284867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64cc0d1600 0x7f64cc1f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 >> 172.21.15.73:0/2042284867 conn(0x7f64cc0d2ac0 msgr2=0x7f64cc0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 shutdown_connections 2024-04-05T22:23:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:05.955+0000 7f64d0c00640 1 -- 172.21.15.73:0/2042284867 wait complete. 2024-04-05T22:23:05.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:06.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.162+0000 7fae98a00640 1 Processor -- start 2024-04-05T22:23:06.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.162+0000 7fae98a00640 1 -- start start 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.163+0000 7fae98a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae9415e620 0x7fae9415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.163+0000 7fae98a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae9406bc70 0x7fae9406c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.163+0000 7fae98a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9406b500 con 0x7fae94158860 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.163+0000 7fae98a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9415eff0 con 0x7fae9415e620 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.163+0000 7fae98a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940ccaf0 con 0x7fae9406bc70 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae9415e620 0x7fae9415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae9415e620 0x7fae9415eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51934/0 (socket says 172.21.15.73:51934) 2024-04-05T22:23:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 -- 172.21.15.73:0/2246768534 learned_addr learned my addr 172.21.15.73:0/2246768534 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 -- 172.21.15.73:0/2246768534 >> v2:172.21.15.157:3300/0 conn(0x7fae9406bc70 msgr2=0x7fae9406c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 --2- 172.21.15.73:0/2246768534 >> v2:172.21.15.157:3300/0 conn(0x7fae9406bc70 0x7fae9406c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 -- 172.21.15.73:0/2246768534 >> v1:172.21.15.157:6789/0 conn(0x7fae94158860 legacy=0x7fae940b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.164+0000 7fae91e00640 1 -- 172.21.15.73:0/2246768534 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940b20d0 con 0x7fae9415e620 2024-04-05T22:23:06.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.165+0000 7fae91e00640 1 --2- 172.21.15.73:0/2246768534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9415e620 0x7fae9415eab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae940b16a0 tx=0x7fae8002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1234d6e64d3e6675 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.165+0000 7fae91400640 1 -- 172.21.15.73:0/2246768534 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae800317e0 con 0x7fae9415e620 2024-04-05T22:23:06.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae91400640 1 -- 172.21.15.73:0/2246768534 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80031940 con 0x7fae9415e620 2024-04-05T22:23:06.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae91400640 1 -- 172.21.15.73:0/2246768534 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80031cd0 con 0x7fae9415e620 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 -- 172.21.15.73:0/2246768534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9415e620 msgr2=0x7fae9415eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 --2- 172.21.15.73:0/2246768534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9415e620 0x7fae9415eab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae940b16a0 tx=0x7fae8002b880 comp rx=0 tx=0).stop 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 -- 172.21.15.73:0/2246768534 shutdown_connections 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 --2- 172.21.15.73:0/2246768534 >> v2:172.21.15.157:3300/0 conn(0x7fae9406bc70 0x7fae9406c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 --2- 172.21.15.73:0/2246768534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9415e620 0x7fae9415eab0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 -- 172.21.15.73:0/2246768534 >> 172.21.15.73:0/2246768534 conn(0x7fae940b06c0 msgr2=0x7fae940b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 -- 172.21.15.73:0/2246768534 shutdown_connections 2024-04-05T22:23:06.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.166+0000 7fae98a00640 1 -- 172.21.15.73:0/2246768534 wait complete. 2024-04-05T22:23:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.168+0000 7fae98a00640 1 Processor -- start 2024-04-05T22:23:06.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.168+0000 7fae98a00640 1 -- start start 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9415e620 0x7fae941534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae94153a00 0x7fae94156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94153eb0 con 0x7fae9406bc70 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9406fad0 con 0x7fae94153a00 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae98a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940acbc0 con 0x7fae9415e620 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae92800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae92800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51938/0 (socket says 172.21.15.73:51938) 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.169+0000 7fae92800640 1 -- 172.21.15.73:0/17848118 learned_addr learned my addr 172.21.15.73:0/17848118 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 -- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9415e620 msgr2=0x7fae941534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae91e00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9415e620 0x7fae941534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9415e620 0x7fae941534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 -- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae94153a00 msgr2=0x7fae94156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae94153a00 0x7fae94156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 -- 172.21.15.73:0/17848118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae800094d0 con 0x7fae9406bc70 2024-04-05T22:23:06.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae92800640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c410 tx=0x7fae8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840035f0 con 0x7fae9406bc70 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae84003750 con 0x7fae9406bc70 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84003b40 con 0x7fae9406bc70 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae94157700 con 0x7fae9406bc70 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.170+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae94157cc0 con 0x7fae9406bc70 2024-04-05T22:23:06.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.171+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae940acde0 con 0x7fae9406bc70 2024-04-05T22:23:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.172+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fae84003ca0 con 0x7fae9406bc70 2024-04-05T22:23:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.173+0000 7fae8aa00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 0x7fae7406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.173+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae840885a0 con 0x7fae9406bc70 2024-04-05T22:23:06.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.173+0000 7fae91e00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 0x7fae7406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.174+0000 7fae91e00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 0x7fae7406d270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae80036000 tx=0x7fae8003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.178+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae840537f0 con 0x7fae9406bc70 2024-04-05T22:23:06.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.356+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae9404e160 con 0x7fae9406bc70 2024-04-05T22:23:06.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.358+0000 7fae8aa00640 1 -- 172.21.15.73:0/17848118 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12389 (secure 0 0 0) 0x7fae84056e10 con 0x7fae9406bc70 2024-04-05T22:23:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:06.359 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":44,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:05.440763+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 msgr2=0x7fae7406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 0x7fae7406d270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae80036000 tx=0x7fae8003d040 comp rx=0 tx=0).stop 2024-04-05T22:23:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 msgr2=0x7fae94148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c410 tx=0x7fae8400c8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 shutdown_connections 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406adb0 0x7fae7406d270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae94153a00 0x7fae94156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9415e620 0x7fae941534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 --2- 172.21.15.73:0/17848118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9406bc70 0x7fae94148440 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 >> 172.21.15.73:0/17848118 conn(0x7fae940b06c0 msgr2=0x7fae940ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 shutdown_connections 2024-04-05T22:23:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.361+0000 7fae98a00640 1 -- 172.21.15.73:0/17848118 wait complete. 2024-04-05T22:23:06.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:23:06.378 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:23:06.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.579+0000 7f5cb2000640 1 Processor -- start 2024-04-05T22:23:06.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.579+0000 7f5cb2000640 1 -- start start 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cb2000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5cac1599f0 0x7f5cac070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cb2000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5cac0add00 0x7f5cac06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cb2000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac06cfb0 con 0x7f5cac06cb80 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cb2000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0b3890 con 0x7f5cac1599f0 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cb2000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0ae190 con 0x7f5cac0add00 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cabe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5cac06cb80 0x7f5cac15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54222/0 (socket says 172.21.15.73:54222) 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cabe00640 1 -- 172.21.15.73:0/604938600 learned_addr learned my addr 172.21.15.73:0/604938600 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:06.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.580+0000 7f5cab400640 1 --2- 172.21.15.73:0/604938600 >> v2:172.21.15.73:3300/0 conn(0x7f5cac1599f0 0x7f5cac070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5ca2a00640 1 --2- 172.21.15.73:0/604938600 >> v2:172.21.15.157:3300/0 conn(0x7f5cac0add00 0x7f5cac06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5cab400640 1 -- 172.21.15.73:0/604938600 >> v2:172.21.15.157:3300/0 conn(0x7f5cac0add00 msgr2=0x7f5cac06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5cab400640 1 --2- 172.21.15.73:0/604938600 >> v2:172.21.15.157:3300/0 conn(0x7f5cac0add00 0x7f5cac06c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5cab400640 1 -- 172.21.15.73:0/604938600 >> v1:172.21.15.73:6789/0 conn(0x7f5cac06cb80 legacy=0x7f5cac15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5cab400640 1 -- 172.21.15.73:0/604938600 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0b27f0 con 0x7f5cac1599f0 2024-04-05T22:23:06.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.581+0000 7f5caaa00640 1 -- 172.21.15.73:0/604938600 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926286104 0 0) 0x7f5cac06cfb0 con 0x7f5cac06cb80 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.582+0000 7f5cab400640 1 --2- 172.21.15.73:0/604938600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac070a40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d90 tx=0x7f5c9c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7e0a8439d45df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.582+0000 7f5caaa00640 1 -- 172.21.15.73:0/604938600 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c02cc60 con 0x7f5cac1599f0 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.582+0000 7f5caaa00640 1 -- 172.21.15.73:0/604938600 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c02cdc0 con 0x7f5cac1599f0 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.582+0000 7f5caaa00640 1 -- 172.21.15.73:0/604938600 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c034660 con 0x7f5cac1599f0 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 -- 172.21.15.73:0/604938600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 msgr2=0x7f5cac070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 --2- 172.21.15.73:0/604938600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac070a40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d90 tx=0x7f5c9c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 -- 172.21.15.73:0/604938600 shutdown_connections 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 --2- 172.21.15.73:0/604938600 >> v2:172.21.15.157:3300/0 conn(0x7f5cac0add00 0x7f5cac06c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 --2- 172.21.15.73:0/604938600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac070a40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 -- 172.21.15.73:0/604938600 >> 172.21.15.73:0/604938600 conn(0x7f5cac069cb0 msgr2=0x7f5cac06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 -- 172.21.15.73:0/604938600 shutdown_connections 2024-04-05T22:23:06.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.583+0000 7f5cb2000640 1 -- 172.21.15.73:0/604938600 wait complete. 2024-04-05T22:23:06.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.585+0000 7f5cb2000640 1 Processor -- start 2024-04-05T22:23:06.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.585+0000 7f5cb2000640 1 -- start start 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac06cb80 0x7f5cac141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac0add00 0x7f5cac155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac165cd0 con 0x7f5cac1599f0 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac165100 con 0x7f5cac06cb80 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cb2000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac1653a0 con 0x7f5cac0add00 2024-04-05T22:23:06.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51950/0 (socket says 172.21.15.73:51950) 2024-04-05T22:23:06.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 -- 172.21.15.73:0/965409412 learned_addr learned my addr 172.21.15.73:0/965409412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5ca2a00640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac0add00 0x7f5cac155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 -- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac0add00 msgr2=0x7f5cac155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac0add00 0x7f5cac155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 -- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac06cb80 msgr2=0x7f5cac141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac06cb80 0x7f5cac141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.586+0000 7f5cabe00640 1 -- 172.21.15.73:0/965409412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009a70 con 0x7f5cac1599f0 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5cabe00640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c480 tx=0x7f5ca400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4005ce0 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4005e60 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40072b0 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac165680 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.587+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac155940 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.588+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac068160 con 0x7f5cac1599f0 2024-04-05T22:23:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.589+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f5ca4007410 con 0x7f5cac1599f0 2024-04-05T22:23:06.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.590+0000 7f5ca3e00640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 0x7f5c8806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:06.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.590+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ca4088900 con 0x7f5cac1599f0 2024-04-05T22:23:06.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.590+0000 7f5cab400640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 0x7f5c8806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.591+0000 7f5cab400640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 0x7f5c8806d270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c036000 tx=0x7f5c9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:06.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.595+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5ca4053ba0 con 0x7f5cac1599f0 2024-04-05T22:23:06.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.792+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cac165e10 con 0x7f5cac1599f0 2024-04-05T22:23:06.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.793+0000 7f5ca3e00640 1 -- 172.21.15.73:0/965409412 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ca4017020 con 0x7f5cac1599f0 2024-04-05T22:23:06.793 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:23:06.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 msgr2=0x7f5c8806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 0x7f5c8806d270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c036000 tx=0x7f5c9c02e000 comp rx=0 tx=0).stop 2024-04-05T22:23:06.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 msgr2=0x7f5cac14f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:06.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c480 tx=0x7f5ca400c940 comp rx=0 tx=0).stop 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 shutdown_connections 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c8806adb0 0x7f5c8806d270 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cac1599f0 0x7f5cac14f350 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac0add00 0x7f5cac155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 --2- 172.21.15.73:0/965409412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac06cb80 0x7f5cac141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.795+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 >> 172.21.15.73:0/965409412 conn(0x7f5cac069cb0 msgr2=0x7f5cac070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.796+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 shutdown_connections 2024-04-05T22:23:06.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:06.796+0000 7f5cb2000640 1 -- 172.21.15.73:0/965409412 wait complete. 2024-04-05T22:23:06.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:23:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.020+0000 7f3446800640 1 Processor -- start 2024-04-05T22:23:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.020+0000 7f3446800640 1 -- start start 2024-04-05T22:23:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.021+0000 7f3446800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34401541a0 0x7f34400b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3446800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 0x7f34400cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3446800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344015aa60 con 0x7f344015aef0 2024-04-05T22:23:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3446800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34400b3b60 con 0x7f34401541a0 2024-04-05T22:23:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3446800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34400701c0 con 0x7f344006d710 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34401541a0 0x7f34400b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34401541a0 0x7f34400b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51966/0 (socket says 172.21.15.73:51966) 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 -- 172.21.15.73:0/3090257744 learned_addr learned my addr 172.21.15.73:0/3090257744 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3436a00640 1 --2- 172.21.15.73:0/3090257744 >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 0x7f34400cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 -- 172.21.15.73:0/3090257744 >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 msgr2=0x7f34400cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 --2- 172.21.15.73:0/3090257744 >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 0x7f34400cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 -- 172.21.15.73:0/3090257744 >> v1:172.21.15.157:6789/0 conn(0x7f344015aef0 legacy=0x7f344015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:07.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f343f400640 1 -- 172.21.15.73:0/3090257744 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400b3880 con 0x7f34401541a0 2024-04-05T22:23:07.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.022+0000 7f3436a00640 1 --2- 172.21.15.73:0/3090257744 >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 0x7f34400cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f343f400640 1 --2- 172.21.15.73:0/3090257744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34401541a0 0x7f34400b3050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f344015e980 tx=0x7f343002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9bc6075b607709 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f343ea00640 1 -- 172.21.15.73:0/3090257744 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430031870 con 0x7f34401541a0 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f343ea00640 1 -- 172.21.15.73:0/3090257744 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300319d0 con 0x7f34401541a0 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f343ea00640 1 -- 172.21.15.73:0/3090257744 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430031d60 con 0x7f34401541a0 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f3446800640 1 -- 172.21.15.73:0/3090257744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34401541a0 msgr2=0x7f34400b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.024+0000 7f3446800640 1 --2- 172.21.15.73:0/3090257744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34401541a0 0x7f34400b3050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f344015e980 tx=0x7f343002b910 comp rx=0 tx=0).stop 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 -- 172.21.15.73:0/3090257744 shutdown_connections 2024-04-05T22:23:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 --2- 172.21.15.73:0/3090257744 >> v2:172.21.15.157:3300/0 conn(0x7f344006d710 0x7f34400cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 --2- 172.21.15.73:0/3090257744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34401541a0 0x7f34400b3050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 -- 172.21.15.73:0/3090257744 >> 172.21.15.73:0/3090257744 conn(0x7f34400ce8e0 msgr2=0x7f34400cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 -- 172.21.15.73:0/3090257744 shutdown_connections 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.025+0000 7f3446800640 1 -- 172.21.15.73:0/3090257744 wait complete. 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.026+0000 7f3446800640 1 Processor -- start 2024-04-05T22:23:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.026+0000 7f3446800640 1 -- start start 2024-04-05T22:23:07.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34401541a0 0x7f344014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f344015aef0 0x7f3440158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344014dfd0 con 0x7f344006d710 2024-04-05T22:23:07.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344014e2b0 con 0x7f34401541a0 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.027+0000 7f3446800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344014e590 con 0x7f344015aef0 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51968/0 (socket says 172.21.15.73:51968) 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 -- 172.21.15.73:0/2263448935 learned_addr learned my addr 172.21.15.73:0/2263448935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 -- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f344015aef0 msgr2=0x7f3440158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f344015aef0 0x7f3440158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 -- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34401541a0 msgr2=0x7f344014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34401541a0 0x7f344014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 -- 172.21.15.73:0/2263448935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300094d0 con 0x7f344006d710 2024-04-05T22:23:07.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.028+0000 7f343f400640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3430000c00 tx=0x7f34300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.029+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430031840 con 0x7f344006d710 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.029+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300443f0 con 0x7f344006d710 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.029+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300447a0 con 0x7f344006d710 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.029+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344014e870 con 0x7f344006d710 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.029+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344014bdf0 con 0x7f344006d710 2024-04-05T22:23:07.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.030+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3440066af0 con 0x7f344006d710 2024-04-05T22:23:07.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.031+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f343004d450 con 0x7f344006d710 2024-04-05T22:23:07.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.032+0000 7f3437e00640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 0x7f341c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.032+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f343002d040 con 0x7f344006d710 2024-04-05T22:23:07.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.032+0000 7f3436a00640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 0x7f341c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.033+0000 7f3436a00640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 0x7f341c06d270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3428006fd0 tx=0x7f3428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.036+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3430076de0 con 0x7f344006d710 2024-04-05T22:23:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.232+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f344014ed90 con 0x7f344006d710 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.233+0000 7f3437e00640 1 -- 172.21.15.73:0/2263448935 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+65 (secure 0 0 0) 0x7f344014ed90 con 0x7f344006d710 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:23:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:23:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:07.234 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.235+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 msgr2=0x7f341c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.235+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 0x7f341c06d270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3428006fd0 tx=0x7f3428008040 comp rx=0 tx=0).stop 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.235+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 msgr2=0x7f344014b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.235+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3430000c00 tx=0x7f34300046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 shutdown_connections 2024-04-05T22:23:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f341c06adb0 0x7f341c06d270 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f344015aef0 0x7f3440158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34401541a0 0x7f344014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 --2- 172.21.15.73:0/2263448935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344006d710 0x7f344014b8b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 >> 172.21.15.73:0/2263448935 conn(0x7f34400ce8e0 msgr2=0x7f34400ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 shutdown_connections 2024-04-05T22:23:07.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.236+0000 7f3446800640 1 -- 172.21.15.73:0/2263448935 wait complete. 2024-04-05T22:23:07.254 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:23:07.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:23:07.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.465+0000 7fe42dc00640 1 Processor -- start 2024-04-05T22:23:07.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.466+0000 7fe42dc00640 1 -- start start 2024-04-05T22:23:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe42815cf60 0x7fe42806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe42806ed20 0x7fe42815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 0x7fe4280aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4280af2d0 con 0x7fe42806ed20 2024-04-05T22:23:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42806a980 con 0x7fe4280ae900 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe42dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42806cf20 con 0x7fe42815cf60 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.467+0000 7fe426000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe42806ed20 0x7fe42815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe42806ed20 0x7fe42815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51974/0 (socket says 172.21.15.73:51974) 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 -- 172.21.15.73:0/623950073 learned_addr learned my addr 172.21.15.73:0/623950073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe427400640 1 --2- 172.21.15.73:0/623950073 >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 0x7fe4280aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426a00640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe42815cf60 0x7fe42806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 -- 172.21.15.73:0/623950073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe42815cf60 msgr2=0x7fe42806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe42815cf60 0x7fe42806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 -- 172.21.15.73:0/623950073 >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 msgr2=0x7fe4280aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 --2- 172.21.15.73:0/623950073 >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 0x7fe4280aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe426000640 1 -- 172.21.15.73:0/623950073 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42806d200 con 0x7fe42806ed20 2024-04-05T22:23:07.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.468+0000 7fe427400640 1 --2- 172.21.15.73:0/623950073 >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 0x7fe4280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:07.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.469+0000 7fe426000640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42806ed20 0x7fe42815dcd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4280af5e0 tx=0x7fe41002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352af19858257ff server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe425600640 1 -- 172.21.15.73:0/623950073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41002ca70 con 0x7fe42806ed20 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe425600640 1 -- 172.21.15.73:0/623950073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41002cbd0 con 0x7fe42806ed20 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe425600640 1 -- 172.21.15.73:0/623950073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100343f0 con 0x7fe42806ed20 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 -- 172.21.15.73:0/623950073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42806ed20 msgr2=0x7fe42815dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42806ed20 0x7fe42815dcd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4280af5e0 tx=0x7fe41002bed0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 -- 172.21.15.73:0/623950073 shutdown_connections 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42806ed20 0x7fe42815dcd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 --2- 172.21.15.73:0/623950073 >> v2:172.21.15.157:3300/0 conn(0x7fe4280ae900 0x7fe4280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 --2- 172.21.15.73:0/623950073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe42815cf60 0x7fe42806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 -- 172.21.15.73:0/623950073 >> 172.21.15.73:0/623950073 conn(0x7fe42806adc0 msgr2=0x7fe42806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 -- 172.21.15.73:0/623950073 shutdown_connections 2024-04-05T22:23:07.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.470+0000 7fe42dc00640 1 -- 172.21.15.73:0/623950073 wait complete. 2024-04-05T22:23:07.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.472+0000 7fe42dc00640 1 Processor -- start 2024-04-05T22:23:07.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.472+0000 7fe42dc00640 1 -- start start 2024-04-05T22:23:07.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 0x7fe428159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4280ae900 0x7fe4281f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280c63d0 con 0x7fe42815cf60 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280cd910 con 0x7fe42806ed20 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe42dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe428159e50 con 0x7fe4280ae900 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe427400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe427400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51976/0 (socket says 172.21.15.73:51976) 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.473+0000 7fe427400640 1 -- 172.21.15.73:0/830488372 learned_addr learned my addr 172.21.15.73:0/830488372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe426000640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4280ae900 0x7fe4281f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe426a00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 0x7fe428159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 -- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4280ae900 msgr2=0x7fe4281f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4280ae900 0x7fe4281f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 -- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 msgr2=0x7fe428159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 0x7fe428159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 -- 172.21.15.73:0/830488372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009b30 con 0x7fe42815cf60 2024-04-05T22:23:07.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe426a00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 0x7fe428159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe427400640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe42000e560 tx=0x7fe42000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420004230 con 0x7fe42815cf60 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420004390 con 0x7fe42815cf60 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4280cdb60 con 0x7fe42815cf60 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.474+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420003810 con 0x7fe42815cf60 2024-04-05T22:23:07.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.475+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4280ce120 con 0x7fe42815cf60 2024-04-05T22:23:07.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.475+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe428068160 con 0x7fe42815cf60 2024-04-05T22:23:07.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.477+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fe420003970 con 0x7fe42815cf60 2024-04-05T22:23:07.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.477+0000 7fe41ea00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 0x7fe40406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:07.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.478+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe420088fb0 con 0x7fe42815cf60 2024-04-05T22:23:07.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.478+0000 7fe426a00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 0x7fe40406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:07.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.478+0000 7fe426a00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 0x7fe40406d270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe418006fd0 tx=0x7fe418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:07.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.482+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe420054140 con 0x7fe42815cf60 2024-04-05T22:23:07.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.677+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe4280b4e30 con 0x7fe42815cf60 2024-04-05T22:23:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.801+0000 7fe41ea00640 1 -- 172.21.15.73:0/830488372 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe420057760 con 0x7fe42815cf60 2024-04-05T22:23:07.802 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:23:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.806+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 msgr2=0x7fe40406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.806+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 0x7fe40406d270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe418006fd0 tx=0x7fe418008040 comp rx=0 tx=0).stop 2024-04-05T22:23:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.806+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 msgr2=0x7fe4280ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:07.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.806+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe42000e560 tx=0x7fe42000ea20 comp rx=0 tx=0).stop 2024-04-05T22:23:07.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 shutdown_connections 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe40406adb0 0x7fe40406d270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42815cf60 0x7fe4280ce920 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4280ae900 0x7fe4281f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 --2- 172.21.15.73:0/830488372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42806ed20 0x7fe428159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 >> 172.21.15.73:0/830488372 conn(0x7fe42806adc0 msgr2=0x7fe42815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 shutdown_connections 2024-04-05T22:23:07.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:07.808+0000 7fe42dc00640 1 -- 172.21.15.73:0/830488372 wait complete. 2024-04-05T22:23:07.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:23:07.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:23:07.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:23:08.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.035+0000 7f938fa00640 1 Processor -- start 2024-04-05T22:23:08.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.035+0000 7f938fa00640 1 -- start start 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93880cef00 0x7f93880cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938fa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93880cf820 con 0x7f93880b1a70 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93880b2260 con 0x7f93880cef00 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880cb8c0 con 0x7f93880c4600 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93880cef00 0x7f93880cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93880cef00 0x7f93880cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51992/0 (socket says 172.21.15.73:51992) 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938d000640 1 -- 172.21.15.73:0/4163645743 learned_addr learned my addr 172.21.15.73:0/4163645743 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f938da00640 1 --1- 172.21.15.73:0/4163645743 >> v1:172.21.15.73:6789/0 conn(0x7f93880b1a70 0x7f9388163800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54238/0 (socket says 172.21.15.73:54238) 2024-04-05T22:23:08.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.037+0000 7f9387e00640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.038+0000 7f938d000640 1 -- 172.21.15.73:0/4163645743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 msgr2=0x7f93880b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.038+0000 7f938d000640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.038+0000 7f938d000640 1 -- 172.21.15.73:0/4163645743 >> v1:172.21.15.73:6789/0 conn(0x7f93880b1a70 legacy=0x7f9388163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.038+0000 7f938d000640 1 -- 172.21.15.73:0/4163645743 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880cbba0 con 0x7f93880cef00 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.039+0000 7f938d000640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880cef00 0x7f93880cf2e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f937c009870 tx=0x7f937c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70e9b17e399ec7e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.039+0000 7f9387400640 1 -- 172.21.15.73:0/4163645743 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c031640 con 0x7f93880cef00 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.039+0000 7f9387400640 1 -- 172.21.15.73:0/4163645743 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c0317a0 con 0x7f93880cef00 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.039+0000 7f9387400640 1 -- 172.21.15.73:0/4163645743 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c031b30 con 0x7f93880cef00 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 -- 172.21.15.73:0/4163645743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880cef00 msgr2=0x7f93880cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880cef00 0x7f93880cf2e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f937c009870 tx=0x7f937c02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 -- 172.21.15.73:0/4163645743 shutdown_connections 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 --2- 172.21.15.73:0/4163645743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880cef00 0x7f93880cf2e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 -- 172.21.15.73:0/4163645743 >> 172.21.15.73:0/4163645743 conn(0x7f9388071be0 msgr2=0x7f9388099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 -- 172.21.15.73:0/4163645743 shutdown_connections 2024-04-05T22:23:08.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.040+0000 7f938fa00640 1 -- 172.21.15.73:0/4163645743 wait complete. 2024-04-05T22:23:08.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.042+0000 7f938fa00640 1 Processor -- start 2024-04-05T22:23:08.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.042+0000 7f938fa00640 1 -- start start 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93880cef00 0x7f938806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880aecc0 con 0x7f93880b1a70 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880ae090 con 0x7f93880cef00 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880ae320 con 0x7f93880c4600 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52004/0 (socket says 172.21.15.73:52004) 2024-04-05T22:23:08.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938d000640 1 -- 172.21.15.73:0/3413297034 learned_addr learned my addr 172.21.15.73:0/3413297034 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f938da00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93880cef00 0x7f938806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.043+0000 7f9387e00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 -- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 msgr2=0x7f93880b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 -- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93880cef00 msgr2=0x7f938806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93880cef00 0x7f938806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 -- 172.21.15.73:0/3413297034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0094d0 con 0x7f93880b1a70 2024-04-05T22:23:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938d000640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f937c000c00 tx=0x7f937c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c03ac60 con 0x7f93880b1a70 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c03ade0 con 0x7f93880b1a70 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c044630 con 0x7f93880b1a70 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93880ae600 con 0x7f93880b1a70 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.044+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388154ba0 con 0x7f93880b1a70 2024-04-05T22:23:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.045+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93880aee00 con 0x7f93880b1a70 2024-04-05T22:23:08.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.047+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f937c02f050 con 0x7f93880b1a70 2024-04-05T22:23:08.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.048+0000 7f9384c00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 0x7f936806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:08.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.048+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f937c0ab5f0 con 0x7f93880b1a70 2024-04-05T22:23:08.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.048+0000 7f9387e00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 0x7f936806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:08.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.049+0000 7f9387e00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 0x7f936806d270 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93880b4e70 tx=0x7f9374008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.053+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f937c0766b0 con 0x7f93880b1a70 2024-04-05T22:23:08.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.247+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f93880ae460 con 0x7f93880b1a70 2024-04-05T22:23:08.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.811+0000 7f9384c00640 1 -- 172.21.15.73:0/3413297034 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7f93880ae460 con 0x7f93880b1a70 2024-04-05T22:23:08.812 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:23:08.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.818+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 msgr2=0x7f936806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.818+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 0x7f936806d270 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93880b4e70 tx=0x7f9374008040 comp rx=0 tx=0).stop 2024-04-05T22:23:08.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.818+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 msgr2=0x7f9388154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:08.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.818+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f937c000c00 tx=0x7f937c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 shutdown_connections 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f936806adb0 0x7f936806d270 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93880cef00 0x7f938806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93880c4600 0x7f93880b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 --2- 172.21.15.73:0/3413297034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93880b1a70 0x7f9388154660 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 >> 172.21.15.73:0/3413297034 conn(0x7f9388071be0 msgr2=0x7f93880cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 shutdown_connections 2024-04-05T22:23:08.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:08.822+0000 7f938fa00640 1 -- 172.21.15.73:0/3413297034 wait complete. 2024-04-05T22:23:08.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:23:09.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.084+0000 7f5c32800640 1 Processor -- start 2024-04-05T22:23:09.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.084+0000 7f5c32800640 1 -- start start 2024-04-05T22:23:09.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c32800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c32800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c070dd0 0x7f5c2c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c32800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2c15a530 con 0x7f5c2c0d0710 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c32800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c06cf90 con 0x7f5c2c0d0330 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c32800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b3e10 con 0x7f5c2c070dd0 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52018/0 (socket says 172.21.15.73:52018) 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2b400640 1 -- 172.21.15.73:0/4208497305 learned_addr learned my addr 172.21.15.73:0/4208497305 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2be00640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c070dd0 0x7f5c2c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2b400640 1 -- 172.21.15.73:0/4208497305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c070dd0 msgr2=0x7f5c2c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:09.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.086+0000 7f5c2b400640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c070dd0 0x7f5c2c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.087+0000 7f5c2b400640 1 -- 172.21.15.73:0/4208497305 >> v1:172.21.15.157:6789/0 conn(0x7f5c2c0d0710 legacy=0x7f5c2c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:09.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.087+0000 7f5c2b400640 1 -- 172.21.15.73:0/4208497305 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0b40f0 con 0x7f5c2c0d0330 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c2b400640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009870 tx=0x7f5c2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efe699e45985d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c2a000640 1 -- 172.21.15.73:0/4208497305 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24031640 con 0x7f5c2c0d0330 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c2a000640 1 -- 172.21.15.73:0/4208497305 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c240317a0 con 0x7f5c2c0d0330 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c2a000640 1 -- 172.21.15.73:0/4208497305 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24031b30 con 0x7f5c2c0d0330 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c32800640 1 -- 172.21.15.73:0/4208497305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0330 msgr2=0x7f5c2c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.088+0000 7f5c32800640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009870 tx=0x7f5c2402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 -- 172.21.15.73:0/4208497305 shutdown_connections 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c070dd0 0x7f5c2c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 --2- 172.21.15.73:0/4208497305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0330 0x7f5c2c06d9e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 -- 172.21.15.73:0/4208497305 >> 172.21.15.73:0/4208497305 conn(0x7f5c2c06adc0 msgr2=0x7f5c2c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:09.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 -- 172.21.15.73:0/4208497305 shutdown_connections 2024-04-05T22:23:09.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.089+0000 7f5c32800640 1 -- 172.21.15.73:0/4208497305 wait complete. 2024-04-05T22:23:09.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.090+0000 7f5c32800640 1 Processor -- start 2024-04-05T22:23:09.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.090+0000 7f5c32800640 1 -- start start 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 0x7f5c2c14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c0d0330 0x7f5c2c144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1313c0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c14acb0 con 0x7f5c2c070dd0 2024-04-05T22:23:09.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.091+0000 7f5c32800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c14b610 con 0x7f5c2c0d0330 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52030/0 (socket says 172.21.15.73:52030) 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 -- 172.21.15.73:0/1696072878 learned_addr learned my addr 172.21.15.73:0/1696072878 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2aa00640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c0d0330 0x7f5c2c144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2b400640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 0x7f5c2c14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 -- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c0d0330 msgr2=0x7f5c2c144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c0d0330 0x7f5c2c144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 -- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 msgr2=0x7f5c2c14b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 0x7f5c2c14b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 -- 172.21.15.73:0/1696072878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c240094d0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2b400640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 0x7f5c2c14b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:09.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.092+0000 7f5c2be00640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c510 tx=0x7f5c1c00c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.093+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004220 con 0x7f5c2c0d0710 2024-04-05T22:23:09.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.093+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c004380 con 0x7f5c2c0d0710 2024-04-05T22:23:09.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.093+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003830 con 0x7f5c2c0d0710 2024-04-05T22:23:09.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.093+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c14a350 con 0x7f5c2c0d0710 2024-04-05T22:23:09.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.093+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c14a5f0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.094+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c14adf0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.095+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f5c1c003990 con 0x7f5c2c0d0710 2024-04-05T22:23:09.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.096+0000 7f5c23400640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 0x7f5c0806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.096+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5c1c0893d0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.096+0000 7f5c2b400640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 0x7f5c0806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.097+0000 7f5c2b400640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 0x7f5c0806d270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c24036000 tx=0x7f5c2403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.101+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5c2c14adf0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.331+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5c2c14adf0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.347 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 6 and data pool 7 2024-04-05T22:23:09.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.343+0000 7f5c23400640 1 -- 172.21.15.73:0/1696072878 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5c2c14adf0 con 0x7f5c2c0d0710 2024-04-05T22:23:09.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.348+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 msgr2=0x7f5c0806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.348+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 0x7f5c0806d270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c24036000 tx=0x7f5c2403c040 comp rx=0 tx=0).stop 2024-04-05T22:23:09.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.349+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 msgr2=0x7f5c2c14fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.349+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c510 tx=0x7f5c1c00c9d0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 shutdown_connections 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5c0806adb0 0x7f5c0806d270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2c0d0710 0x7f5c2c14fc20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c2c0d0330 0x7f5c2c144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 --2- 172.21.15.73:0/1696072878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c070dd0 0x7f5c2c14b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.351+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 >> 172.21.15.73:0/1696072878 conn(0x7f5c2c06adc0 msgr2=0x7f5c2c0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:09.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.352+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 shutdown_connections 2024-04-05T22:23:09.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.352+0000 7f5c32800640 1 -- 172.21.15.73:0/1696072878 wait complete. 2024-04-05T22:23:09.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:09.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.549+0000 7fcd3f200640 1 Processor -- start 2024-04-05T22:23:09.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.549+0000 7fcd3f200640 1 -- start start 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd40158860 0x7fcd400b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd400b1f00 0x7fcd4015e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4006b500 con 0x7fcd4015eb60 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400ccaf0 con 0x7fcd40158860 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4015efd0 con 0x7fcd400b1f00 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd40158860 0x7fcd400b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd40158860 0x7fcd400b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52044/0 (socket says 172.21.15.73:52044) 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 -- 172.21.15.73:0/1462531181 learned_addr learned my addr 172.21.15.73:0/1462531181 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 -- 172.21.15.73:0/1462531181 >> v2:172.21.15.157:3300/0 conn(0x7fcd400b1f00 msgr2=0x7fcd4015e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 --2- 172.21.15.73:0/1462531181 >> v2:172.21.15.157:3300/0 conn(0x7fcd400b1f00 0x7fcd4015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.551+0000 7fcd3de00640 1 -- 172.21.15.73:0/1462531181 >> v1:172.21.15.157:6789/0 conn(0x7fcd4015eb60 legacy=0x7fcd4015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:09.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.552+0000 7fcd3de00640 1 -- 172.21.15.73:0/1462531181 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400b23a0 con 0x7fcd40158860 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3de00640 1 --2- 172.21.15.73:0/1462531181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd40158860 0x7fcd400b19c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd40072570 tx=0x7fcd3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9273dc6c9f4b9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3ca00640 1 -- 172.21.15.73:0/1462531181 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd300317e0 con 0x7fcd40158860 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3ca00640 1 -- 172.21.15.73:0/1462531181 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30031940 con 0x7fcd40158860 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3ca00640 1 -- 172.21.15.73:0/1462531181 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30031cd0 con 0x7fcd40158860 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 -- 172.21.15.73:0/1462531181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd40158860 msgr2=0x7fcd400b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 --2- 172.21.15.73:0/1462531181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd40158860 0x7fcd400b19c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd40072570 tx=0x7fcd3002b880 comp rx=0 tx=0).stop 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 -- 172.21.15.73:0/1462531181 shutdown_connections 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 --2- 172.21.15.73:0/1462531181 >> v2:172.21.15.157:3300/0 conn(0x7fcd400b1f00 0x7fcd4015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 --2- 172.21.15.73:0/1462531181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd40158860 0x7fcd400b19c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.553+0000 7fcd3f200640 1 -- 172.21.15.73:0/1462531181 >> 172.21.15.73:0/1462531181 conn(0x7fcd400b06c0 msgr2=0x7fcd400b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.554+0000 7fcd3f200640 1 -- 172.21.15.73:0/1462531181 shutdown_connections 2024-04-05T22:23:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.554+0000 7fcd3f200640 1 -- 172.21.15.73:0/1462531181 wait complete. 2024-04-05T22:23:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.555+0000 7fcd3f200640 1 Processor -- start 2024-04-05T22:23:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.555+0000 7fcd3f200640 1 -- start start 2024-04-05T22:23:09.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd400b1f00 0x7fcd40152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd401491c0 0x7fcd4013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40152760 con 0x7fcd4015eb60 2024-04-05T22:23:09.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd401530a0 con 0x7fcd401491c0 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40151ae0 con 0x7fcd400b1f00 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52056/0 (socket says 172.21.15.73:52056) 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.556+0000 7fcd3d400640 1 -- 172.21.15.73:0/4188774678 learned_addr learned my addr 172.21.15.73:0/4188774678 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 -- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd400b1f00 msgr2=0x7fcd40152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd400b1f00 0x7fcd40152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 -- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd401491c0 msgr2=0x7fcd4013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd401491c0 0x7fcd4013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 -- 172.21.15.73:0/4188774678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300094d0 con 0x7fcd4015eb60 2024-04-05T22:23:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3d400640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b4a0 tx=0x7fcd2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c010610 con 0x7fcd4015eb60 2024-04-05T22:23:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c010770 con 0x7fcd4015eb60 2024-04-05T22:23:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c010b20 con 0x7fcd4015eb60 2024-04-05T22:23:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.557+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd40151dc0 con 0x7fcd4015eb60 2024-04-05T22:23:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.558+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd40152380 con 0x7fcd4015eb60 2024-04-05T22:23:09.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.558+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4004e160 con 0x7fcd4015eb60 2024-04-05T22:23:09.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.560+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fcd2c010c80 con 0x7fcd4015eb60 2024-04-05T22:23:09.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.560+0000 7fcd36000640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 0x7fcd2006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.560+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd2c0892e0 con 0x7fcd4015eb60 2024-04-05T22:23:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.561+0000 7fcd3de00640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 0x7fcd2006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.561+0000 7fcd3de00640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 0x7fcd2006d270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd30036000 tx=0x7fcd30030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.565+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcd2c054240 con 0x7fcd4015eb60 2024-04-05T22:23:09.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.758+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd400b13f0 con 0x7fcd4015eb60 2024-04-05T22:23:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.760+0000 7fcd36000640 1 -- 172.21.15.73:0/4188774678 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16058 (secure 0 0 0) 0x7fcd2c057860 con 0x7fcd4015eb60 2024-04-05T22:23:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:09.761 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":47,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:09.333107+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 msgr2=0x7fcd2006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 0x7fcd2006d270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd30036000 tx=0x7fcd30030000 comp rx=0 tx=0).stop 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 msgr2=0x7fcd4015b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b4a0 tx=0x7fcd2c00b960 comp rx=0 tx=0).stop 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 shutdown_connections 2024-04-05T22:23:09.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd2006adb0 0x7fcd2006d270 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd401491c0 0x7fcd4013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4015eb60 0x7fcd4015b230 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 --2- 172.21.15.73:0/4188774678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd400b1f00 0x7fcd40152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.763+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 >> 172.21.15.73:0/4188774678 conn(0x7fcd400b06c0 msgr2=0x7fcd400cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.764+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 shutdown_connections 2024-04-05T22:23:09.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.764+0000 7fcd3f200640 1 -- 172.21.15.73:0/4188774678 wait complete. 2024-04-05T22:23:09.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:09.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.995+0000 7f31aac00640 1 Processor -- start 2024-04-05T22:23:09.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.996+0000 7f31aac00640 1 -- start start 2024-04-05T22:23:09.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.997+0000 7f31aac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31a41587d0 0x7f31a40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.997+0000 7f31aac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a40ccd20 0x7f31a406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.997+0000 7f31aac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a406f5b0 con 0x7f31a406d0c0 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.997+0000 7f31aac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a4070410 con 0x7f31a41587d0 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.997+0000 7f31aac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4158bb0 con 0x7f31a40ccd20 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a8c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f31a406d0c0 0x7f31a415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54242/0 (socket says 172.21.15.73:54242) 2024-04-05T22:23:09.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a8c00640 1 -- 172.21.15.73:0/2893589857 learned_addr learned my addr 172.21.15.73:0/2893589857 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a3400640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a40ccd20 0x7f31a406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2893589857 >> v2:172.21.15.157:3300/0 conn(0x7f31a41587d0 0x7f31a40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463047504 0 0) 0x7f31a406f5b0 con 0x7f31a406d0c0 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3184003520 con 0x7f31a406d0c0 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858168890 0 0) 0x7f3184003520 con 0x7f31a406d0c0 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a40ccd20 msgr2=0x7f31a406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a40ccd20 0x7f31a406cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 >> v2:172.21.15.157:3300/0 conn(0x7f31a41587d0 msgr2=0x7f31a40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 --2- 172.21.15.73:0/2893589857 >> v2:172.21.15.157:3300/0 conn(0x7f31a41587d0 0x7f31a40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a406f800 con 0x7f31a406d0c0 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.998+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f319c001670 con 0x7f31a406d0c0 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 >> v1:172.21.15.73:6789/0 conn(0x7f31a406d0c0 legacy=0x7f31a415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3184003970 0x7f3184003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31840047c0 0x7f3184004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3184005140 0x7f318400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4070410 con 0x7f3184003970 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4158bb0 con 0x7f3184005140 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840045c0 con 0x7f31840047c0 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3184003970 0x7f3184003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3400640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31840047c0 0x7f3184004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 -- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31840047c0 msgr2=0x7f3184004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31840047c0 0x7f3184004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 -- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3184005140 msgr2=0x7f318400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a8c00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3184005140 0x7f318400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3184005140 0x7f318400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:09.999+0000 7f31a3e00640 1 -- 172.21.15.73:0/2893589857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184003520 con 0x7f3184003970 2024-04-05T22:23:10.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.000+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3184003970 0x7f3184003d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3194009ac0 tx=0x7f31840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a7e973e5023b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319402c710 con 0x7f3184003970 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319402c870 con 0x7f3184003970 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31a2a00640 1 -- 172.21.15.73:0/2893589857 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319402cc00 con 0x7f3184003970 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 -- 172.21.15.73:0/2893589857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3184003970 msgr2=0x7f3184003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 --2- 172.21.15.73:0/2893589857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3184003970 0x7f3184003d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3194009ac0 tx=0x7f31840057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31a8c00640 1 -- 172.21.15.73:0/2893589857 reap_dead start 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 -- 172.21.15.73:0/2893589857 shutdown_connections 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 -- 172.21.15.73:0/2893589857 >> 172.21.15.73:0/2893589857 conn(0x7f31a40ce8e0 msgr2=0x7f31a40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 -- 172.21.15.73:0/2893589857 shutdown_connections 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.001+0000 7f31aac00640 1 -- 172.21.15.73:0/2893589857 wait complete. 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.003+0000 7f31aac00640 1 Processor -- start 2024-04-05T22:23:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.003+0000 7f31aac00640 1 -- start start 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3184003970 0x7f31a413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a40ad060 0x7f31a415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a415e1e0 con 0x7f31a415d600 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a415df00 con 0x7f31a40ad060 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31aac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40a8560 con 0x7f3184003970 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31a8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31a8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52084/0 (socket says 172.21.15.73:52084) 2024-04-05T22:23:10.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.004+0000 7f31a8c00640 1 -- 172.21.15.73:0/2001345585 learned_addr learned my addr 172.21.15.73:0/2001345585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3184003970 0x7f31a413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 -- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3184003970 msgr2=0x7f31a413cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3184003970 0x7f31a413cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a3400640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a40ad060 0x7f31a415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 -- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a40ad060 msgr2=0x7f31a415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a40ad060 0x7f31a415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 -- 172.21.15.73:0/2001345585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40a8840 con 0x7f31a415d600 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31a8c00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c007230 con 0x7f31a415d600 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c0073b0 con 0x7f31a415d600 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c0047e0 con 0x7f31a415d600 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a40a8b20 con 0x7f31a415d600 2024-04-05T22:23:10.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.005+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a415dc60 con 0x7f31a415d600 2024-04-05T22:23:10.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.007+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a4066af0 con 0x7f31a415d600 2024-04-05T22:23:10.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.008+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f319c004940 con 0x7f31a415d600 2024-04-05T22:23:10.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.008+0000 7f319be00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 0x7f318006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.008+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f319c0895b0 con 0x7f31a415d600 2024-04-05T22:23:10.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.009+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 0x7f318006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.009+0000 7f31a3e00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 0x7f318006d270 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3194036000 tx=0x7f3194030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.013+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f319c054400 con 0x7f31a415d600 2024-04-05T22:23:10.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.207+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a4068160 con 0x7f31a415d600 2024-04-05T22:23:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.209+0000 7f319be00640 1 -- 172.21.15.73:0/2001345585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16058 (secure 0 0 0) 0x7f319c057a20 con 0x7f31a415d600 2024-04-05T22:23:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:10.209 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":47,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:09.333107+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 msgr2=0x7f318006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 0x7f318006d270 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3194036000 tx=0x7f3194030000 comp rx=0 tx=0).stop 2024-04-05T22:23:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 msgr2=0x7f31a40aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c00dc00 comp rx=0 tx=0).stop 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 shutdown_connections 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f318006adb0 0x7f318006d270 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415d600 0x7f31a40aa090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a40ad060 0x7f31a415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 --2- 172.21.15.73:0/2001345585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3184003970 0x7f31a413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 >> 172.21.15.73:0/2001345585 conn(0x7f31a40ce8e0 msgr2=0x7f31a4143700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 shutdown_connections 2024-04-05T22:23:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.212+0000 7f31aac00640 1 -- 172.21.15.73:0/2001345585 wait complete. 2024-04-05T22:23:10.231 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:23:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.437+0000 7f2867800640 1 Processor -- start 2024-04-05T22:23:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.437+0000 7f2867800640 1 -- start start 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2867800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f286015aa70 0x7f2860070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2867800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28600714a0 0x7f286015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2867800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286015ae50 con 0x7f286015dbc0 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2867800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2860071930 con 0x7f286015aa70 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2867800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286015e3b0 con 0x7f28600714a0 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2865800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f286015dbc0 0x7f28601609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54246/0 (socket says 172.21.15.73:54246) 2024-04-05T22:23:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2865800640 1 -- 172.21.15.73:0/3518971865 learned_addr learned my addr 172.21.15.73:0/3518971865 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2864e00640 1 --2- 172.21.15.73:0/3518971865 >> v2:172.21.15.73:3300/0 conn(0x7f286015aa70 0x7f2860070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.439+0000 7f2857e00640 1 --2- 172.21.15.73:0/3518971865 >> v2:172.21.15.157:3300/0 conn(0x7f28600714a0 0x7f286015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.440+0000 7f285fe00640 1 -- 172.21.15.73:0/3518971865 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127269985 0 0) 0x7f286015ae50 con 0x7f286015dbc0 2024-04-05T22:23:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.440+0000 7f2864e00640 1 -- 172.21.15.73:0/3518971865 >> v2:172.21.15.157:3300/0 conn(0x7f28600714a0 msgr2=0x7f286015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.440+0000 7f2864e00640 1 --2- 172.21.15.73:0/3518971865 >> v2:172.21.15.157:3300/0 conn(0x7f28600714a0 0x7f286015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.440+0000 7f2864e00640 1 -- 172.21.15.73:0/3518971865 >> v1:172.21.15.73:6789/0 conn(0x7f286015dbc0 legacy=0x7f28601609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.440+0000 7f2864e00640 1 -- 172.21.15.73:0/3518971865 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286006ca50 con 0x7f286015aa70 2024-04-05T22:23:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.441+0000 7f2864e00640 1 --2- 172.21.15.73:0/3518971865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015aa70 0x7f2860070f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2860161a70 tx=0x7f285002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e967281f41652aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.442+0000 7f285fe00640 1 -- 172.21.15.73:0/3518971865 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850031830 con 0x7f286015aa70 2024-04-05T22:23:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.442+0000 7f285fe00640 1 -- 172.21.15.73:0/3518971865 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850031990 con 0x7f286015aa70 2024-04-05T22:23:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.442+0000 7f285fe00640 1 -- 172.21.15.73:0/3518971865 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850031d20 con 0x7f286015aa70 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.442+0000 7f2867800640 1 -- 172.21.15.73:0/3518971865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015aa70 msgr2=0x7f2860070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.442+0000 7f2867800640 1 --2- 172.21.15.73:0/3518971865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015aa70 0x7f2860070f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2860161a70 tx=0x7f285002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 -- 172.21.15.73:0/3518971865 shutdown_connections 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 --2- 172.21.15.73:0/3518971865 >> v2:172.21.15.157:3300/0 conn(0x7f28600714a0 0x7f286015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 --2- 172.21.15.73:0/3518971865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015aa70 0x7f2860070f60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 -- 172.21.15.73:0/3518971865 >> 172.21.15.73:0/3518971865 conn(0x7f2860069cb0 msgr2=0x7f286006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 -- 172.21.15.73:0/3518971865 shutdown_connections 2024-04-05T22:23:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.443+0000 7f2867800640 1 -- 172.21.15.73:0/3518971865 wait complete. 2024-04-05T22:23:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.444+0000 7f2867800640 1 Processor -- start 2024-04-05T22:23:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.445+0000 7f2867800640 1 -- start start 2024-04-05T22:23:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28600714a0 0x7f2860158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286015aa70 0x7f28601f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2860159780 con 0x7f286015dbc0 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601f58d0 con 0x7f28600714a0 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2867800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28601f5bb0 con 0x7f286015aa70 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52114/0 (socket says 172.21.15.73:52114) 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 -- 172.21.15.73:0/1774166560 learned_addr learned my addr 172.21.15.73:0/1774166560 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 -- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286015aa70 msgr2=0x7f28601f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286015aa70 0x7f28601f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 -- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28600714a0 msgr2=0x7f2860158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2864e00640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28600714a0 0x7f2860158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28600714a0 0x7f2860158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.446+0000 7f2865800640 1 -- 172.21.15.73:0/1774166560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2850009400 con 0x7f286015dbc0 2024-04-05T22:23:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f2865800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f285800c780 tx=0x7f285800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800ce90 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800a640 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800a9d0 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601f5e90 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.447+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28601f6690 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.448+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28601f63b0 con 0x7f286015dbc0 2024-04-05T22:23:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.449+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f285800ab30 con 0x7f286015dbc0 2024-04-05T22:23:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.450+0000 7f285d600640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 0x7f283c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.450+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2858088be0 con 0x7f286015dbc0 2024-04-05T22:23:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.450+0000 7f2864e00640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 0x7f283c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:10.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.451+0000 7f2864e00640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 0x7f283c06d270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2850036000 tx=0x7f2850008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:10.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.455+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f28601f63b0 con 0x7f286015dbc0 2024-04-05T22:23:10.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:10.689+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f28601f63b0 con 0x7f286015dbc0 2024-04-05T22:23:11.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.348+0000 7f285d600640 1 -- 172.21.15.73:0/1774166560 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f28601f63b0 con 0x7f286015dbc0 2024-04-05T22:23:11.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 msgr2=0x7f283c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 0x7f283c06d270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2850036000 tx=0x7f2850008030 comp rx=0 tx=0).stop 2024-04-05T22:23:11.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 msgr2=0x7f28601607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f285800c780 tx=0x7f285800cc40 comp rx=0 tx=0).stop 2024-04-05T22:23:11.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 shutdown_connections 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f283c06adb0 0x7f283c06d270 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f286015dbc0 0x7f28601607d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286015aa70 0x7f28601f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 --2- 172.21.15.73:0/1774166560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28600714a0 0x7f2860158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 >> 172.21.15.73:0/1774166560 conn(0x7f2860069cb0 msgr2=0x7f286006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 shutdown_connections 2024-04-05T22:23:11.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.352+0000 7f2867800640 1 -- 172.21.15.73:0/1774166560 wait complete. 2024-04-05T22:23:11.373 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:23:11.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:23:11.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.578+0000 7fefef800640 1 Processor -- start 2024-04-05T22:23:11.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.578+0000 7fefef800640 1 -- start start 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.579+0000 7fefef800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefe806bc40 0x7fefe8165750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.579+0000 7fefef800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe815afc0 con 0x7fefe815abe0 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefef800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe806f5b0 con 0x7fefe806c9a0 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefef800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe806f890 con 0x7fefe806bc40 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefece00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fefe815abe0 0x7fefe80712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54248/0 (socket says 172.21.15.73:54248) 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefece00640 1 -- 172.21.15.73:0/604829096 learned_addr learned my addr 172.21.15.73:0/604829096 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:11.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefed800640 1 --2- 172.21.15.73:0/604829096 >> v2:172.21.15.73:3300/0 conn(0x7fefe806bc40 0x7fefe8165750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefe7400640 1 -- 172.21.15.73:0/604829096 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687412399 0 0) 0x7fefe815afc0 con 0x7fefe815abe0 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefe7400640 1 -- 172.21.15.73:0/604829096 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd0003520 con 0x7fefe815abe0 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefed800640 1 -- 172.21.15.73:0/604829096 >> v1:172.21.15.157:6789/0 conn(0x7fefe806c9a0 legacy=0x7fefe806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefed800640 1 -- 172.21.15.73:0/604829096 >> v1:172.21.15.73:6789/0 conn(0x7fefe815abe0 legacy=0x7fefe80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.580+0000 7fefed800640 1 -- 172.21.15.73:0/604829096 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe806fb70 con 0x7fefe806bc40 2024-04-05T22:23:11.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.582+0000 7fefed800640 1 --2- 172.21.15.73:0/604829096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806bc40 0x7fefe8165750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009d90 tx=0x7fefe002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d27447a2589550 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:11.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.582+0000 7fefe7400640 1 -- 172.21.15.73:0/604829096 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe002cb40 con 0x7fefe806bc40 2024-04-05T22:23:11.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.582+0000 7fefe7400640 1 -- 172.21.15.73:0/604829096 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe002cca0 con 0x7fefe806bc40 2024-04-05T22:23:11.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.582+0000 7fefe7400640 1 -- 172.21.15.73:0/604829096 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0034660 con 0x7fefe806bc40 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 -- 172.21.15.73:0/604829096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806bc40 msgr2=0x7fefe8165750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 --2- 172.21.15.73:0/604829096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806bc40 0x7fefe8165750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009d90 tx=0x7fefe002bd90 comp rx=0 tx=0).stop 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 -- 172.21.15.73:0/604829096 shutdown_connections 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 --2- 172.21.15.73:0/604829096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806bc40 0x7fefe8165750 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 -- 172.21.15.73:0/604829096 >> 172.21.15.73:0/604829096 conn(0x7fefe8069cb0 msgr2=0x7fefe806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 -- 172.21.15.73:0/604829096 shutdown_connections 2024-04-05T22:23:11.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.583+0000 7fefef800640 1 -- 172.21.15.73:0/604829096 wait complete. 2024-04-05T22:23:11.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.585+0000 7fefef800640 1 Processor -- start 2024-04-05T22:23:11.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.585+0000 7fefef800640 1 -- start start 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 0x7fefe80ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 0x7fefe815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80acec0 con 0x7fefe806c9a0 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe815c010 con 0x7fefe815abe0 2024-04-05T22:23:11.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefef800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80ac8d0 con 0x7fefe806bc40 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52134/0 (socket says 172.21.15.73:52134) 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 -- 172.21.15.73:0/2481552182 learned_addr learned my addr 172.21.15.73:0/2481552182 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefece00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 0x7fefe80ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 -- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 msgr2=0x7fefe80ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefed800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 0x7fefe815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 0x7fefe80ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 -- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 msgr2=0x7fefe815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 0x7fefe815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefe7e00640 1 -- 172.21.15.73:0/2481552182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009a70 con 0x7fefe806c9a0 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.586+0000 7fefece00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 0x7fefe80ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefed800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 0x7fefe815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefe7e00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c430 tx=0x7fefd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:11.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011790 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd40118f0 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011ce0 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe815ba50 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.587+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe80acae0 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.588+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80acc70 con 0x7fefe806c9a0 2024-04-05T22:23:11.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.589+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fefd401c460 con 0x7fefe806c9a0 2024-04-05T22:23:11.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.590+0000 7fefe4c00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 0x7fefc806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:11.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.590+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefd40896b0 con 0x7fefe806c9a0 2024-04-05T22:23:11.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.590+0000 7fefece00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 0x7fefc806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:11.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.591+0000 7fefece00640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 0x7fefc806d270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fefdc007fd0 tx=0x7fefdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:11.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.595+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fefd4054570 con 0x7fefe806c9a0 2024-04-05T22:23:11.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:11.820+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fefe80ad000 con 0x7fefe806c9a0 2024-04-05T22:23:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.360+0000 7fefe4c00640 1 -- 172.21.15.73:0/2481552182 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fefe80ad000 con 0x7fefe806c9a0 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 msgr2=0x7fefc806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 0x7fefc806d270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fefdc007fd0 tx=0x7fefdc009040 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 msgr2=0x7fefe81f1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c430 tx=0x7fefd400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 shutdown_connections 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefc806adb0 0x7fefc806d270 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe815abe0 0x7fefe815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefe806c9a0 0x7fefe81f1c80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 --2- 172.21.15.73:0/2481552182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe806bc40 0x7fefe80ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.363+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 >> 172.21.15.73:0/2481552182 conn(0x7fefe8069cb0 msgr2=0x7fefe80af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.364+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 shutdown_connections 2024-04-05T22:23:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.364+0000 7fefef800640 1 -- 172.21.15.73:0/2481552182 wait complete. 2024-04-05T22:23:12.383 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:23:12.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.592+0000 7f6861a00640 1 Processor -- start 2024-04-05T22:23:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.592+0000 7f6861a00640 1 -- start start 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f685c160620 0x7f685c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 0x7f685c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 0x7f685c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0ce5d0 con 0x7f685c160620 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0ce010 con 0x7f685c0ad6b0 2024-04-05T22:23:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6861a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ce2f0 con 0x7f685c0b1870 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f685c160620 0x7f685c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f685c160620 0x7f685c0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52150/0 (socket says 172.21.15.73:52150) 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685aa00640 1 -- 172.21.15.73:0/2207421967 learned_addr learned my addr 172.21.15.73:0/2207421967 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f6853e00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 0x7f685c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685b400640 1 --2- 172.21.15.73:0/2207421967 >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 0x7f685c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685aa00640 1 -- 172.21.15.73:0/2207421967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 msgr2=0x7f685c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.594+0000 7f685aa00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 0x7f685c0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.595+0000 7f685aa00640 1 -- 172.21.15.73:0/2207421967 >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 msgr2=0x7f685c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.595+0000 7f685aa00640 1 --2- 172.21.15.73:0/2207421967 >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 0x7f685c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.595+0000 7f685aa00640 1 -- 172.21.15.73:0/2207421967 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0ac970 con 0x7f685c160620 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.595+0000 7f685b400640 1 --2- 172.21.15.73:0/2207421967 >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 0x7f685c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.595+0000 7f6853e00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 0x7f685c0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.596+0000 7f685aa00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c160620 0x7f685c0caa60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cdec0 tx=0x7f684c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6536966c3127bb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.596+0000 7f685a000640 1 -- 172.21.15.73:0/2207421967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c031870 con 0x7f685c160620 2024-04-05T22:23:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.596+0000 7f685a000640 1 -- 172.21.15.73:0/2207421967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c0319d0 con 0x7f685c160620 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.596+0000 7f685a000640 1 -- 172.21.15.73:0/2207421967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c031d60 con 0x7f685c160620 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 -- 172.21.15.73:0/2207421967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c160620 msgr2=0x7f685c0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c160620 0x7f685c0caa60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cdec0 tx=0x7f684c02b910 comp rx=0 tx=0).stop 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 -- 172.21.15.73:0/2207421967 shutdown_connections 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 --2- 172.21.15.73:0/2207421967 >> v2:172.21.15.157:3300/0 conn(0x7f685c0ad6b0 0x7f685c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c0b1870 0x7f685c0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 --2- 172.21.15.73:0/2207421967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c160620 0x7f685c0caa60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 -- 172.21.15.73:0/2207421967 >> 172.21.15.73:0/2207421967 conn(0x7f685c0ce8e0 msgr2=0x7f685c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 -- 172.21.15.73:0/2207421967 shutdown_connections 2024-04-05T22:23:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.597+0000 7f6861a00640 1 -- 172.21.15.73:0/2207421967 wait complete. 2024-04-05T22:23:12.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.599+0000 7f6861a00640 1 Processor -- start 2024-04-05T22:23:12.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.599+0000 7f6861a00640 1 -- start start 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c0b1870 0x7f685c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c160620 0x7f685c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c4840 con 0x7f685c0ad6b0 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c63e0 con 0x7f685c0b1870 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f6861a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c3bc0 con 0x7f685c160620 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52164/0 (socket says 172.21.15.73:52164) 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 -- 172.21.15.73:0/657541775 learned_addr learned my addr 172.21.15.73:0/657541775 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 -- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c160620 msgr2=0x7f685c14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c160620 0x7f685c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 -- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c0b1870 msgr2=0x7f685c15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c0b1870 0x7f685c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.600+0000 7f685aa00640 1 -- 172.21.15.73:0/657541775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0094d0 con 0x7f685c0ad6b0 2024-04-05T22:23:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f685aa00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f684c02b950 tx=0x7f684c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c031860 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c0443f0 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0447a0 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0c3ea0 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.601+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0c4460 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.602+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c066af0 con 0x7f685c0ad6b0 2024-04-05T22:23:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.603+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f684c044900 con 0x7f685c0ad6b0 2024-04-05T22:23:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.604+0000 7f6852a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 0x7f683806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.604+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f684c02d040 con 0x7f685c0ad6b0 2024-04-05T22:23:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.604+0000 7f6853e00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 0x7f683806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.605+0000 7f6853e00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 0x7f683806d270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.609+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f684c076e20 con 0x7f685c0ad6b0 2024-04-05T22:23:12.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.844+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685c0c4980 con 0x7f685c0ad6b0 2024-04-05T22:23:12.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.845+0000 7f6852a00640 1 -- 172.21.15.73:0/657541775 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f685c0c4980 con 0x7f685c0ad6b0 2024-04-05T22:23:12.846 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:12.846 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:12.846 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 23 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 msgr2=0x7f683806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 0x7f683806d270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844008040 comp rx=0 tx=0).stop 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 msgr2=0x7f685c149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f684c02b950 tx=0x7f684c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 shutdown_connections 2024-04-05T22:23:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f683806adb0 0x7f683806d270 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685c160620 0x7f685c14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c0b1870 0x7f685c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 --2- 172.21.15.73:0/657541775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685c0ad6b0 0x7f685c149320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 >> 172.21.15.73:0/657541775 conn(0x7f685c0ce8e0 msgr2=0x7f685c15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 shutdown_connections 2024-04-05T22:23:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:12.848+0000 7f6861a00640 1 -- 172.21.15.73:0/657541775 wait complete. 2024-04-05T22:23:12.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.064+0000 7f8754c00640 1 Processor -- start 2024-04-05T22:23:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.064+0000 7f8754c00640 1 -- start start 2024-04-05T22:23:13.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f8754c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f875015e620 0x7f875015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f8754c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875006b500 con 0x7f875006bc70 2024-04-05T22:23:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f8754c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87500b20d0 con 0x7f8750158860 2024-04-05T22:23:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f8754c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875015eff0 con 0x7f875015e620 2024-04-05T22:23:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f874fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f875006bc70 0x7f875006c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54254/0 (socket says 172.21.15.73:54254) 2024-04-05T22:23:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.066+0000 7f874fe00640 1 -- 172.21.15.73:0/3795148269 learned_addr learned my addr 172.21.15.73:0/3795148269 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f8746a00640 1 --2- 172.21.15.73:0/3795148269 >> v2:172.21.15.157:3300/0 conn(0x7f875015e620 0x7f875015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278694695 0 0) 0x7f875006b500 con 0x7f875006bc70 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8734003520 con 0x7f875006bc70 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118961086 0 0) 0x7f8734003520 con 0x7f875006bc70 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 >> v2:172.21.15.157:3300/0 conn(0x7f875015e620 msgr2=0x7f875015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 --2- 172.21.15.73:0/3795148269 >> v2:172.21.15.157:3300/0 conn(0x7f875015e620 0x7f875015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 >> v1:172.21.15.157:6789/0 conn(0x7f8750158860 legacy=0x7f87500b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500b23a0 con 0x7f875006bc70 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f87480019f0 con 0x7f875006bc70 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 >> v1:172.21.15.73:6789/0 conn(0x7f875006bc70 legacy=0x7f875006c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8734003970 0x7f8734003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 0x7f8734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87340052a0 0x7f873400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875015eff0 con 0x7f87340047c0 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873400c170 con 0x7f87340052a0 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.067+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873400c330 con 0x7f8734003970 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 0x7f8734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 -- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8734003970 msgr2=0x7f8734003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8734003970 0x7f8734003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 -- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87340052a0 msgr2=0x7f873400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87340052a0 0x7f873400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f8746a00640 1 -- 172.21.15.73:0/3795148269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734003520 con 0x7f87340047c0 2024-04-05T22:23:13.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.068+0000 7f874fe00640 1 -- 172.21.15.73:0/3795148269 reap_dead start 2024-04-05T22:23:13.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.069+0000 7f8746a00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 0x7f8734004c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f873800ba00 tx=0x7f87340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec5d31ee038a7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.069+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738032640 con 0x7f87340047c0 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.069+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380327a0 con 0x7f87340047c0 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.069+0000 7f874ea00640 1 -- 172.21.15.73:0/3795148269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738032b30 con 0x7f87340047c0 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 -- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 msgr2=0x7f8734004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 0x7f8734004c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f873800ba00 tx=0x7f87340057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 -- 172.21.15.73:0/3795148269 shutdown_connections 2024-04-05T22:23:13.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 --2- 172.21.15.73:0/3795148269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87340047c0 0x7f8734004c40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 -- 172.21.15.73:0/3795148269 >> 172.21.15.73:0/3795148269 conn(0x7f87500b06c0 msgr2=0x7f87500b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:13.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 -- 172.21.15.73:0/3795148269 shutdown_connections 2024-04-05T22:23:13.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.070+0000 7f8754c00640 1 -- 172.21.15.73:0/3795148269 wait complete. 2024-04-05T22:23:13.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.072+0000 7f8754c00640 1 Processor -- start 2024-04-05T22:23:13.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.072+0000 7f8754c00640 1 -- start start 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 0x7f87501454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750154ff0 0x7f87500712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8750156790 con 0x7f8750154b90 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87500ce410 con 0x7f8750154ff0 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8754c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8750156a50 con 0x7f87340047c0 2024-04-05T22:23:13.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8746a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8746a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52182/0 (socket says 172.21.15.73:52182) 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f8746a00640 1 -- 172.21.15.73:0/3335571126 learned_addr learned my addr 172.21.15.73:0/3335571126 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.073+0000 7f874f400640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 0x7f87501454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 -- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 msgr2=0x7f87501454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 0x7f87501454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f874fe00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750154ff0 0x7f87500712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 -- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750154ff0 msgr2=0x7f87500712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750154ff0 0x7f87500712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 -- 172.21.15.73:0/3335571126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500cdb70 con 0x7f8750154b90 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f874f400640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 0x7f87501454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8746a00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f873802bec0 tx=0x7f8738004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738004220 con 0x7f8750154b90 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380043a0 con 0x7f8750154b90 2024-04-05T22:23:13.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873803b9f0 con 0x7f8750154b90 2024-04-05T22:23:13.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87500cde50 con 0x7f8750154b90 2024-04-05T22:23:13.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.074+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875015e0e0 con 0x7f8750154b90 2024-04-05T22:23:13.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.076+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8750156bc0 con 0x7f8750154b90 2024-04-05T22:23:13.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.076+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f873803bb50 con 0x7f8750154b90 2024-04-05T22:23:13.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.078+0000 7f8747e00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 0x7f872c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.078+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87380ac120 con 0x7f8750154b90 2024-04-05T22:23:13.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.078+0000 7f874f400640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 0x7f872c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.079+0000 7f874f400640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 0x7f872c06d270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8750063c30 tx=0x7f8740009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.082+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8738076ee0 con 0x7f8750154b90 2024-04-05T22:23:13.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.275+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8750156bc0 con 0x7f8750154b90 2024-04-05T22:23:13.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.278+0000 7f8747e00640 1 -- 172.21.15.73:0/3335571126 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f873807a500 con 0x7f8750154b90 2024-04-05T22:23:13.279 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:13.279 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":48,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:10.337170+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 msgr2=0x7f872c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 0x7f872c06d270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8750063c30 tx=0x7f8740009040 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 msgr2=0x7f8750070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f873802bec0 tx=0x7f8738004730 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 shutdown_connections 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f872c06adb0 0x7f872c06d270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8750154ff0 0x7f87500712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8750154b90 0x7f8750070d70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 --2- 172.21.15.73:0/3335571126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87340047c0 0x7f87501454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.281+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 >> 172.21.15.73:0/3335571126 conn(0x7f87500b06c0 msgr2=0x7f875006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:13.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.282+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 shutdown_connections 2024-04-05T22:23:13.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.282+0000 7f8754c00640 1 -- 172.21.15.73:0/3335571126 wait complete. 2024-04-05T22:23:13.301 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:23:13.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:23:13.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.500+0000 7fd352600640 1 Processor -- start 2024-04-05T22:23:13.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.500+0000 7fd352600640 1 -- start start 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd352600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd34c161f90 0x7fd34c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd352600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c164e50 0x7fd34c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd352600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c165cf0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd352600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c158790 con 0x7fd34c161f90 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd352600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c165730 con 0x7fd34c164e50 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd34b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd34c161f90 0x7fd34c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd34b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd34c161f90 0x7fd34c162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52184/0 (socket says 172.21.15.73:52184) 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd34b400640 1 -- 172.21.15.73:0/3800956764 learned_addr learned my addr 172.21.15.73:0/3800956764 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd342a00640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c164e50 0x7fd34c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.502+0000 7fd34b400640 1 -- 172.21.15.73:0/3800956764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c164e50 msgr2=0x7fd34c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.503+0000 7fd34b400640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c164e50 0x7fd34c0b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.503+0000 7fd34b400640 1 -- 172.21.15.73:0/3800956764 >> v1:172.21.15.157:6789/0 conn(0x7fd34c0b2ba0 legacy=0x7fd34c070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.503+0000 7fd34b400640 1 -- 172.21.15.73:0/3800956764 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c165a10 con 0x7fd34c161f90 2024-04-05T22:23:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.504+0000 7fd34b400640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c161f90 0x7fd34c162370 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07df1bf171ff426 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.504+0000 7fd34aa00640 1 -- 172.21.15.73:0/3800956764 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c031640 con 0x7fd34c161f90 2024-04-05T22:23:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.504+0000 7fd34aa00640 1 -- 172.21.15.73:0/3800956764 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0317a0 con 0x7fd34c161f90 2024-04-05T22:23:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.504+0000 7fd34aa00640 1 -- 172.21.15.73:0/3800956764 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c031b30 con 0x7fd34c161f90 2024-04-05T22:23:13.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 -- 172.21.15.73:0/3800956764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c161f90 msgr2=0x7fd34c162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c161f90 0x7fd34c162370 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 -- 172.21.15.73:0/3800956764 shutdown_connections 2024-04-05T22:23:13.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c164e50 0x7fd34c0b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 --2- 172.21.15.73:0/3800956764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c161f90 0x7fd34c162370 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 -- 172.21.15.73:0/3800956764 >> 172.21.15.73:0/3800956764 conn(0x7fd34c0ce8e0 msgr2=0x7fd34c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:13.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 -- 172.21.15.73:0/3800956764 shutdown_connections 2024-04-05T22:23:13.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.505+0000 7fd352600640 1 -- 172.21.15.73:0/3800956764 wait complete. 2024-04-05T22:23:13.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.507+0000 7fd352600640 1 Processor -- start 2024-04-05T22:23:13.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.507+0000 7fd352600640 1 -- start start 2024-04-05T22:23:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c161f90 0x7fd34c150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c164e50 0x7fd34c151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c14f000 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c14e380 con 0x7fd34c164e50 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd352600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c14e660 con 0x7fd34c161f90 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52190/0 (socket says 172.21.15.73:52190) 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 -- 172.21.15.73:0/4246715572 learned_addr learned my addr 172.21.15.73:0/4246715572 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 -- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c161f90 msgr2=0x7fd34c150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c161f90 0x7fd34c150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 -- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c164e50 msgr2=0x7fd34c151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c164e50 0x7fd34c151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.508+0000 7fd34b400640 1 -- 172.21.15.73:0/4246715572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0094d0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd34b400640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c03ac60 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c03ade0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c14e940 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c044630 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.509+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c14d1d0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.510+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c066af0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.511+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fd33c02f050 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.512+0000 7fd343e00640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 0x7fd32806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.512+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd33c0ab9c0 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.512+0000 7fd342a00640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 0x7fd32806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.513+0000 7fd342a00640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 0x7fd32806d270 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd34c150d40 tx=0x7fd3340072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.516+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd33c076770 con 0x7fd34c0b2ba0 2024-04-05T22:23:13.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.691+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd34c0b1bf0 con 0x7fd32806adb0 2024-04-05T22:23:13.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.718+0000 7fd343e00640 1 -- 172.21.15.73:0/4246715572 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272355 (secure 0 0 0) 0x7fd3340079b0 con 0x7fd32806adb0 2024-04-05T22:23:13.722 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:13.723 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:23:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 msgr2=0x7fd32806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 0x7fd32806d270 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd34c150d40 tx=0x7fd3340072f0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 msgr2=0x7fd34c14cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 shutdown_connections 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd32806adb0 0x7fd32806d270 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c164e50 0x7fd34c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c161f90 0x7fd34c150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 --2- 172.21.15.73:0/4246715572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd34c0b2ba0 0x7fd34c14cc90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 >> 172.21.15.73:0/4246715572 conn(0x7fd34c0ce8e0 msgr2=0x7fd34c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.724+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 shutdown_connections 2024-04-05T22:23:13.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.725+0000 7fd352600640 1 -- 172.21.15.73:0/4246715572 wait complete. 2024-04-05T22:23:13.742 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-04-05T22:23:12.525549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237772,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736948,"statfs":{"total":773094113280,"available":772850634752,"internally_reserved":0,"allocated":5283840,"data_stored":2807640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.731115"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-05T22:23:05.268776+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:23:05.268776+0000","last_peered":"2024-04-05T22:23:05.268776+0000","last_clean":"2024-04-05T22:23:05.268776+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:23:05.268776+0000","last_undegraded":"2024-04-05T22:23:05.268776+0000","last_fullsized":"2024-04-05T22:23:05.268776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1105920,"data_stored":793731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.302},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.141}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.395},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.585}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.248},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.476}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.7869999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.145},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.014}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.238},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.6220000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.389},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.772}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":2.2599999999999998},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.95599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.286},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.159}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.855},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.389}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.339}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.508}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.952}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.554},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":2.0609999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31628,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.135},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.214}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.27}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.0429999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.302}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.435},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.0169999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.028},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.258}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.385},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.194},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.486}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.91200000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.036},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.76100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":793731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.077},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.024},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.161},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.28}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.383}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.413},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.208}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.24}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:23:13.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:23:13.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.978+0000 7f320a200640 1 Processor -- start 2024-04-05T22:23:13.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.979+0000 7f320a200640 1 -- start start 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f320a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3204161f90 0x7f3204162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f320a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3204164e50 0x7f32040b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f320a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3204165cf0 con 0x7f32040b2ba0 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f320a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3204158790 con 0x7f3204161f90 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f320a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3204165730 con 0x7f3204164e50 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.980+0000 7f3203400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3204161f90 0x7f3204162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f32040b2ba0 0x7f3204070510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54262/0 (socket says 172.21.15.73:54262) 2024-04-05T22:23:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203e00640 1 -- 172.21.15.73:0/2516507585 learned_addr learned my addr 172.21.15.73:0/2516507585 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3204161f90 0x7f3204162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52196/0 (socket says 172.21.15.73:52196) 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f31faa00640 1 --2- 172.21.15.73:0/2516507585 >> v2:172.21.15.157:3300/0 conn(0x7f3204164e50 0x7f32040b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203400640 1 -- 172.21.15.73:0/2516507585 >> v2:172.21.15.157:3300/0 conn(0x7f3204164e50 msgr2=0x7f32040b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203400640 1 --2- 172.21.15.73:0/2516507585 >> v2:172.21.15.157:3300/0 conn(0x7f3204164e50 0x7f32040b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203400640 1 -- 172.21.15.73:0/2516507585 >> v1:172.21.15.73:6789/0 conn(0x7f32040b2ba0 legacy=0x7f3204070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.981+0000 7f3203400640 1 -- 172.21.15.73:0/2516507585 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204165a10 con 0x7f3204161f90 2024-04-05T22:23:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f3203400640 1 --2- 172.21.15.73:0/2516507585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f3204162370 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f320406af00 tx=0x7f31f402bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12cfa9d9b42c323 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f3202a00640 1 -- 172.21.15.73:0/2516507585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f402ca30 con 0x7f3204161f90 2024-04-05T22:23:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f3202a00640 1 -- 172.21.15.73:0/2516507585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f402cb90 con 0x7f3204161f90 2024-04-05T22:23:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f3202a00640 1 -- 172.21.15.73:0/2516507585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4034440 con 0x7f3204161f90 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f320a200640 1 -- 172.21.15.73:0/2516507585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 msgr2=0x7f3204162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.983+0000 7f320a200640 1 --2- 172.21.15.73:0/2516507585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f3204162370 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f320406af00 tx=0x7f31f402bd00 comp rx=0 tx=0).stop 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 -- 172.21.15.73:0/2516507585 shutdown_connections 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 --2- 172.21.15.73:0/2516507585 >> v2:172.21.15.157:3300/0 conn(0x7f3204164e50 0x7f32040b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 --2- 172.21.15.73:0/2516507585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f3204162370 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 -- 172.21.15.73:0/2516507585 >> 172.21.15.73:0/2516507585 conn(0x7f32040ce8e0 msgr2=0x7f32040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 -- 172.21.15.73:0/2516507585 shutdown_connections 2024-04-05T22:23:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.984+0000 7f320a200640 1 -- 172.21.15.73:0/2516507585 wait complete. 2024-04-05T22:23:13.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.985+0000 7f320a200640 1 Processor -- start 2024-04-05T22:23:13.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 -- start start 2024-04-05T22:23:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32040b2ba0 0x7f32041588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3204164e50 0x7f32041566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204152540 con 0x7f3204161f90 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204158e10 con 0x7f32040b2ba0 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.986+0000 7f320a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320414e630 con 0x7f3204164e50 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52200/0 (socket says 172.21.15.73:52200) 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 -- 172.21.15.73:0/3179316928 learned_addr learned my addr 172.21.15.73:0/3179316928 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 -- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3204164e50 msgr2=0x7f32041566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3204164e50 0x7f32041566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 -- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32040b2ba0 msgr2=0x7f32041588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32040b2ba0 0x7f32041588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 -- 172.21.15.73:0/3179316928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4009a70 con 0x7f3204161f90 2024-04-05T22:23:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.987+0000 7f31faa00640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00d620 tx=0x7f31ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.988+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00b620 con 0x7f3204161f90 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.988+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00b780 con 0x7f3204161f90 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.988+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00bb30 con 0x7f3204161f90 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.988+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320414e910 con 0x7f3204161f90 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.988+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3204152160 con 0x7f3204161f90 2024-04-05T22:23:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.989+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32041527b0 con 0x7f3204161f90 2024-04-05T22:23:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.990+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f31ec00bc90 con 0x7f3204161f90 2024-04-05T22:23:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.991+0000 7f31fbe00640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 0x7f31e006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.991+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31ec089bc0 con 0x7f3204161f90 2024-04-05T22:23:13.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.991+0000 7f3203400640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 0x7f31e006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.992+0000 7f3203400640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 0x7f31e006d270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31f4036000 tx=0x7f31f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:13.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:13.995+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f31ec054a80 con 0x7f3204161f90 2024-04-05T22:23:14.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.178+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32040b1bf0 con 0x7f31e006adb0 2024-04-05T22:23:14.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.189+0000 7f31fbe00640 1 -- 172.21.15.73:0/3179316928 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272355 (secure 0 0 0) 0x7f31f4034440 con 0x7f31e006adb0 2024-04-05T22:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:14.197 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:23:14.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 msgr2=0x7f31e006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 0x7f31e006d270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31f4036000 tx=0x7f31f402e000 comp rx=0 tx=0).stop 2024-04-05T22:23:14.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 msgr2=0x7f320414cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00d620 tx=0x7f31ec00dae0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 shutdown_connections 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f31e006adb0 0x7f31e006d270 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3204164e50 0x7f32041566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3204161f90 0x7f320414cce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 --2- 172.21.15.73:0/3179316928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32040b2ba0 0x7f32041588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 >> 172.21.15.73:0/3179316928 conn(0x7f32040ce8e0 msgr2=0x7f3204164350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 shutdown_connections 2024-04-05T22:23:14.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.195+0000 7f320a200640 1 -- 172.21.15.73:0/3179316928 wait complete. 2024-04-05T22:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-04-05T22:23:12.525549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237772,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736948,"statfs":{"total":773094113280,"available":772850634752,"internally_reserved":0,"allocated":5283840,"data_stored":2807640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.731115"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:08.792633+0000","last_change":"2024-04-05T22:23:08.792633+0000","last_active":"2024-04-05T22:23:08.792633+0000","last_peered":"2024-04-05T22:23:08.792633+0000","last_clean":"2024-04-05T22:23:08.792633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:08.792633+0000","last_undegraded":"2024-04-05T22:23:08.792633+0000","last_fullsized":"2024-04-05T22:23:08.792633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:23:07.787650+0000","last_change":"2024-04-05T22:23:07.787650+0000","last_active":"2024-04-05T22:23:07.787650+0000","last_peered":"2024-04-05T22:23:07.787650+0000","last_clean":"2024-04-05T22:23:07.787650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:23:07.787650+0000","last_undegraded":"2024-04-05T22:23:07.787650+0000","last_fullsized":"2024-04-05T22:23:07.787650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-05T22:23:05.268776+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:23:05.268776+0000","last_peered":"2024-04-05T22:23:05.268776+0000","last_clean":"2024-04-05T22:23:05.268776+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:23:05.268776+0000","last_undegraded":"2024-04-05T22:23:05.268776+0000","last_fullsized":"2024-04-05T22:23:05.268776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1105920,"data_stored":793731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.302},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.141}]},{"osd":1,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.395},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.585}]},{"osd":3,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.248},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.476}]},{"osd":5,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:22:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.7869999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.145},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.014}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.238},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.6220000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.389},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.772}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":2.2599999999999998},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.95599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.286},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.159}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.855},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.389}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.339}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.508}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.952}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.554},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":2.0609999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31628,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.135},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.214}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.27}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.0429999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.302}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.435},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.0169999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.028},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.258}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.385},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.194},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.486}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":512000,"data_stored":203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.91200000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.036},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.76100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":793731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.077},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.024},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.161},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.28}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.383}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.413},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.208}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.24}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:23:14.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:23:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.465+0000 7fbfbba00640 1 Processor -- start 2024-04-05T22:23:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.465+0000 7fbfbba00640 1 -- start start 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.466+0000 7fbfbba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 0x7fbfb40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.466+0000 7fbfbba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbfb40ccd20 0x7fbfb406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.466+0000 7fbfbba00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfb406d0c0 0x7fbfb415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.466+0000 7fbfbba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb406f5b0 con 0x7fbfb40ccd20 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfbba00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb4070410 con 0x7fbfb406d0c0 2024-04-05T22:23:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfbba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4158bb0 con 0x7fbfb41587d0 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfb3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbfb40ccd20 0x7fbfb406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfb3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbfb40ccd20 0x7fbfb406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52204/0 (socket says 172.21.15.73:52204) 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfb3e00640 1 -- 172.21.15.73:0/2353590641 learned_addr learned my addr 172.21.15.73:0/2353590641 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfb9000640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 0x7fbfb40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.467+0000 7fbfb9a00640 1 --2- 172.21.15.73:0/2353590641 >> v2:172.21.15.157:3300/0 conn(0x7fbfb406d0c0 0x7fbfb415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb3e00640 1 -- 172.21.15.73:0/2353590641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 msgr2=0x7fbfb40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb3e00640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 0x7fbfb40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb3e00640 1 -- 172.21.15.73:0/2353590641 >> v2:172.21.15.157:3300/0 conn(0x7fbfb406d0c0 msgr2=0x7fbfb415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb3e00640 1 --2- 172.21.15.73:0/2353590641 >> v2:172.21.15.157:3300/0 conn(0x7fbfb406d0c0 0x7fbfb415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb3e00640 1 -- 172.21.15.73:0/2353590641 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb406f800 con 0x7fbfb40ccd20 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.468+0000 7fbfb9000640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 0x7fbfb40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:14.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.469+0000 7fbfb3e00640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb40ccd20 0x7fbfb406cb80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40ae4b0 tx=0x7fbfa002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca90495e636d3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.469+0000 7fbfb3400640 1 -- 172.21.15.73:0/2353590641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0031870 con 0x7fbfb40ccd20 2024-04-05T22:23:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.469+0000 7fbfb3400640 1 -- 172.21.15.73:0/2353590641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa00319d0 con 0x7fbfb40ccd20 2024-04-05T22:23:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.469+0000 7fbfb3400640 1 -- 172.21.15.73:0/2353590641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0031d60 con 0x7fbfb40ccd20 2024-04-05T22:23:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 -- 172.21.15.73:0/2353590641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb40ccd20 msgr2=0x7fbfb406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb40ccd20 0x7fbfb406cb80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40ae4b0 tx=0x7fbfa002b910 comp rx=0 tx=0).stop 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 -- 172.21.15.73:0/2353590641 shutdown_connections 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb40ccd20 0x7fbfb406cb80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2353590641 >> v2:172.21.15.157:3300/0 conn(0x7fbfb406d0c0 0x7fbfb415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2353590641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb41587d0 0x7fbfb40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 -- 172.21.15.73:0/2353590641 >> 172.21.15.73:0/2353590641 conn(0x7fbfb40ce8e0 msgr2=0x7fbfb40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 -- 172.21.15.73:0/2353590641 shutdown_connections 2024-04-05T22:23:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.470+0000 7fbfbba00640 1 -- 172.21.15.73:0/2353590641 wait complete. 2024-04-05T22:23:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.472+0000 7fbfbba00640 1 Processor -- start 2024-04-05T22:23:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.472+0000 7fbfbba00640 1 -- start start 2024-04-05T22:23:14.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb406d0c0 0x7fbfb4141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 0x7fbfb41429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb415dea0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4142540 con 0x7fbfb41587d0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfbba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4141660 con 0x7fbfb406d0c0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52208/0 (socket says 172.21.15.73:52208) 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfb9a00640 1 -- 172.21.15.73:0/2880213271 learned_addr learned my addr 172.21.15.73:0/2880213271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.473+0000 7fbfb9000640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb406d0c0 0x7fbfb4141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb3e00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 0x7fbfb41429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 -- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb406d0c0 msgr2=0x7fbfb4141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb406d0c0 0x7fbfb4141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 -- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 msgr2=0x7fbfb41429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 0x7fbfb41429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 -- 172.21.15.73:0/2880213271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00094d0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb3e00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 0x7fbfb41429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb9a00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007e90 tx=0x7fbfac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac014420 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac0145a0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac014950 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4142f10 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.474+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4141c10 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.475+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4066af0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.476+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fbfac014ab0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.477+0000 7fbfb0c00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 0x7fbf9406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.477+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfac089fe0 con 0x7fbfb415d4d0 2024-04-05T22:23:14.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.477+0000 7fbfb9000640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 0x7fbf9406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.478+0000 7fbfb9000640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 0x7fbf9406d270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.482+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbfac054f30 con 0x7fbfb415d4d0 2024-04-05T22:23:14.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.659+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb40700a0 con 0x7fbf9406adb0 2024-04-05T22:23:14.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.668+0000 7fbfb0c00640 1 -- 172.21.15.73:0/2880213271 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287170 (secure 0 0 0) 0x7fbfa800b6b0 con 0x7fbf9406adb0 2024-04-05T22:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 msgr2=0x7fbf9406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 0x7fbf9406d270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 msgr2=0x7fbfb415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007e90 tx=0x7fbfac00e4b0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 shutdown_connections 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf9406adb0 0x7fbf9406d270 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb415d4d0 0x7fbfb415d960 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb41587d0 0x7fbfb41429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 --2- 172.21.15.73:0/2880213271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb406d0c0 0x7fbfb4141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 >> 172.21.15.73:0/2880213271 conn(0x7fbfb40ce8e0 msgr2=0x7fbfb415b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 shutdown_connections 2024-04-05T22:23:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.675+0000 7fbfbba00640 1 -- 172.21.15.73:0/2880213271 wait complete. 2024-04-05T22:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-04-05T22:23:14.526223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242280,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754732440,"statfs":{"total":773094113280,"available":772846018560,"internally_reserved":0,"allocated":5742592,"data_stored":3013636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":242208315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001842"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347585+0000","last_change":"2024-04-05T22:23:09.381025+0000","last_active":"2024-04-05T22:23:10.347585+0000","last_peered":"2024-04-05T22:23:10.347585+0000","last_clean":"2024-04-05T22:23:10.347585+0000","last_became_active":"2024-04-05T22:23:09.379621+0000","last_became_peered":"2024-04-05T22:23:09.379621+0000","last_unstale":"2024-04-05T22:23:10.347585+0000","last_undegraded":"2024-04-05T22:23:10.347585+0000","last_fullsized":"2024-04-05T22:23:10.347585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487095+0000","last_change":"2024-04-05T22:23:09.358874+0000","last_active":"2024-04-05T22:23:10.487095+0000","last_peered":"2024-04-05T22:23:10.487095+0000","last_clean":"2024-04-05T22:23:10.487095+0000","last_became_active":"2024-04-05T22:23:09.358109+0000","last_became_peered":"2024-04-05T22:23:09.358109+0000","last_unstale":"2024-04-05T22:23:10.487095+0000","last_undegraded":"2024-04-05T22:23:10.487095+0000","last_fullsized":"2024-04-05T22:23:10.487095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073685+0000","last_change":"2024-04-05T22:23:09.365855+0000","last_active":"2024-04-05T22:23:11.073685+0000","last_peered":"2024-04-05T22:23:11.073685+0000","last_clean":"2024-04-05T22:23:11.073685+0000","last_became_active":"2024-04-05T22:23:09.365109+0000","last_became_peered":"2024-04-05T22:23:09.365109+0000","last_unstale":"2024-04-05T22:23:11.073685+0000","last_undegraded":"2024-04-05T22:23:11.073685+0000","last_fullsized":"2024-04-05T22:23:11.073685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:51.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072301+0000","last_change":"2024-04-05T22:23:09.360571+0000","last_active":"2024-04-05T22:23:11.072301+0000","last_peered":"2024-04-05T22:23:11.072301+0000","last_clean":"2024-04-05T22:23:11.072301+0000","last_became_active":"2024-04-05T22:23:09.360249+0000","last_became_peered":"2024-04-05T22:23:09.360249+0000","last_unstale":"2024-04-05T22:23:11.072301+0000","last_undegraded":"2024-04-05T22:23:11.072301+0000","last_fullsized":"2024-04-05T22:23:11.072301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:02.342545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486869+0000","last_change":"2024-04-05T22:23:09.363323+0000","last_active":"2024-04-05T22:23:10.486869+0000","last_peered":"2024-04-05T22:23:10.486869+0000","last_clean":"2024-04-05T22:23:10.486869+0000","last_became_active":"2024-04-05T22:23:09.363021+0000","last_became_peered":"2024-04-05T22:23:09.363021+0000","last_unstale":"2024-04-05T22:23:10.486869+0000","last_undegraded":"2024-04-05T22:23:10.486869+0000","last_fullsized":"2024-04-05T22:23:10.486869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349441+0000","last_change":"2024-04-05T22:23:09.376486+0000","last_active":"2024-04-05T22:23:10.349441+0000","last_peered":"2024-04-05T22:23:10.349441+0000","last_clean":"2024-04-05T22:23:10.349441+0000","last_became_active":"2024-04-05T22:23:09.376214+0000","last_became_peered":"2024-04-05T22:23:09.376214+0000","last_unstale":"2024-04-05T22:23:10.349441+0000","last_undegraded":"2024-04-05T22:23:10.349441+0000","last_fullsized":"2024-04-05T22:23:10.349441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:56:38.205365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347409+0000","last_change":"2024-04-05T22:23:09.374598+0000","last_active":"2024-04-05T22:23:10.347409+0000","last_peered":"2024-04-05T22:23:10.347409+0000","last_clean":"2024-04-05T22:23:10.347409+0000","last_became_active":"2024-04-05T22:23:09.374273+0000","last_became_peered":"2024-04-05T22:23:09.374273+0000","last_unstale":"2024-04-05T22:23:10.347409+0000","last_undegraded":"2024-04-05T22:23:10.347409+0000","last_fullsized":"2024-04-05T22:23:10.347409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071571+0000","last_change":"2024-04-05T22:23:09.362559+0000","last_active":"2024-04-05T22:23:11.071571+0000","last_peered":"2024-04-05T22:23:11.071571+0000","last_clean":"2024-04-05T22:23:11.071571+0000","last_became_active":"2024-04-05T22:23:09.362193+0000","last_became_peered":"2024-04-05T22:23:09.362193+0000","last_unstale":"2024-04-05T22:23:11.071571+0000","last_undegraded":"2024-04-05T22:23:11.071571+0000","last_fullsized":"2024-04-05T22:23:11.071571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:17.066127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072891+0000","last_change":"2024-04-05T22:23:09.381934+0000","last_active":"2024-04-05T22:23:11.072891+0000","last_peered":"2024-04-05T22:23:11.072891+0000","last_clean":"2024-04-05T22:23:11.072891+0000","last_became_active":"2024-04-05T22:23:09.380359+0000","last_became_peered":"2024-04-05T22:23:09.380359+0000","last_unstale":"2024-04-05T22:23:11.072891+0000","last_undegraded":"2024-04-05T22:23:11.072891+0000","last_fullsized":"2024-04-05T22:23:11.072891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073130+0000","last_change":"2024-04-05T22:23:09.360338+0000","last_active":"2024-04-05T22:23:11.073130+0000","last_peered":"2024-04-05T22:23:11.073130+0000","last_clean":"2024-04-05T22:23:11.073130+0000","last_became_active":"2024-04-05T22:23:09.360037+0000","last_became_peered":"2024-04-05T22:23:09.360037+0000","last_unstale":"2024-04-05T22:23:11.073130+0000","last_undegraded":"2024-04-05T22:23:11.073130+0000","last_fullsized":"2024-04-05T22:23:11.073130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:00.708491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071876+0000","last_change":"2024-04-05T22:23:09.363867+0000","last_active":"2024-04-05T22:23:11.071876+0000","last_peered":"2024-04-05T22:23:11.071876+0000","last_clean":"2024-04-05T22:23:11.071876+0000","last_became_active":"2024-04-05T22:23:09.363604+0000","last_became_peered":"2024-04-05T22:23:09.363604+0000","last_unstale":"2024-04-05T22:23:11.071876+0000","last_undegraded":"2024-04-05T22:23:11.071876+0000","last_fullsized":"2024-04-05T22:23:11.071876+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487272+0000","last_change":"2024-04-05T22:23:09.361574+0000","last_active":"2024-04-05T22:23:10.487272+0000","last_peered":"2024-04-05T22:23:10.487272+0000","last_clean":"2024-04-05T22:23:10.487272+0000","last_became_active":"2024-04-05T22:23:09.361302+0000","last_became_peered":"2024-04-05T22:23:09.361302+0000","last_unstale":"2024-04-05T22:23:10.487272+0000","last_undegraded":"2024-04-05T22:23:10.487272+0000","last_fullsized":"2024-04-05T22:23:10.487272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070969+0000","last_change":"2024-04-05T22:23:09.374114+0000","last_active":"2024-04-05T22:23:11.070969+0000","last_peered":"2024-04-05T22:23:11.070969+0000","last_clean":"2024-04-05T22:23:11.070969+0000","last_became_active":"2024-04-05T22:23:09.373743+0000","last_became_peered":"2024-04-05T22:23:09.373743+0000","last_unstale":"2024-04-05T22:23:11.070969+0000","last_undegraded":"2024-04-05T22:23:11.070969+0000","last_fullsized":"2024-04-05T22:23:11.070969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:25.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072456+0000","last_change":"2024-04-05T22:23:09.360964+0000","last_active":"2024-04-05T22:23:11.072456+0000","last_peered":"2024-04-05T22:23:11.072456+0000","last_clean":"2024-04-05T22:23:11.072456+0000","last_became_active":"2024-04-05T22:23:09.360707+0000","last_became_peered":"2024-04-05T22:23:09.360707+0000","last_unstale":"2024-04-05T22:23:11.072456+0000","last_undegraded":"2024-04-05T22:23:11.072456+0000","last_fullsized":"2024-04-05T22:23:11.072456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349236+0000","last_change":"2024-04-05T22:23:09.374250+0000","last_active":"2024-04-05T22:23:10.349236+0000","last_peered":"2024-04-05T22:23:10.349236+0000","last_clean":"2024-04-05T22:23:10.349236+0000","last_became_active":"2024-04-05T22:23:09.373960+0000","last_became_peered":"2024-04-05T22:23:09.373960+0000","last_unstale":"2024-04-05T22:23:10.349236+0000","last_undegraded":"2024-04-05T22:23:10.349236+0000","last_fullsized":"2024-04-05T22:23:10.349236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:59.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071101+0000","last_change":"2024-04-05T22:23:09.359204+0000","last_active":"2024-04-05T22:23:11.071101+0000","last_peered":"2024-04-05T22:23:11.071101+0000","last_clean":"2024-04-05T22:23:11.071101+0000","last_became_active":"2024-04-05T22:23:09.358942+0000","last_became_peered":"2024-04-05T22:23:09.358942+0000","last_unstale":"2024-04-05T22:23:11.071101+0000","last_undegraded":"2024-04-05T22:23:11.071101+0000","last_fullsized":"2024-04-05T22:23:11.071101+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:02.659494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072261+0000","last_change":"2024-04-05T22:23:09.376368+0000","last_active":"2024-04-05T22:23:11.072261+0000","last_peered":"2024-04-05T22:23:11.072261+0000","last_clean":"2024-04-05T22:23:11.072261+0000","last_became_active":"2024-04-05T22:23:09.376102+0000","last_became_peered":"2024-04-05T22:23:09.376102+0000","last_unstale":"2024-04-05T22:23:11.072261+0000","last_undegraded":"2024-04-05T22:23:11.072261+0000","last_fullsized":"2024-04-05T22:23:11.072261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:33.661018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073167+0000","last_change":"2024-04-05T22:23:09.377480+0000","last_active":"2024-04-05T22:23:11.073167+0000","last_peered":"2024-04-05T22:23:11.073167+0000","last_clean":"2024-04-05T22:23:11.073167+0000","last_became_active":"2024-04-05T22:23:09.377201+0000","last_became_peered":"2024-04-05T22:23:09.377201+0000","last_unstale":"2024-04-05T22:23:11.073167+0000","last_undegraded":"2024-04-05T22:23:11.073167+0000","last_fullsized":"2024-04-05T22:23:11.073167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:09.415421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072444+0000","last_change":"2024-04-05T22:23:09.361412+0000","last_active":"2024-04-05T22:23:11.072444+0000","last_peered":"2024-04-05T22:23:11.072444+0000","last_clean":"2024-04-05T22:23:11.072444+0000","last_became_active":"2024-04-05T22:23:09.361098+0000","last_became_peered":"2024-04-05T22:23:09.361098+0000","last_unstale":"2024-04-05T22:23:11.072444+0000","last_undegraded":"2024-04-05T22:23:11.072444+0000","last_fullsized":"2024-04-05T22:23:11.072444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:40.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071327+0000","last_change":"2024-04-05T22:23:09.355237+0000","last_active":"2024-04-05T22:23:11.071327+0000","last_peered":"2024-04-05T22:23:11.071327+0000","last_clean":"2024-04-05T22:23:11.071327+0000","last_became_active":"2024-04-05T22:23:09.354907+0000","last_became_peered":"2024-04-05T22:23:09.354907+0000","last_unstale":"2024-04-05T22:23:11.071327+0000","last_undegraded":"2024-04-05T22:23:11.071327+0000","last_fullsized":"2024-04-05T22:23:11.071327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:11:07.814340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350119+0000","last_change":"2024-04-05T22:23:09.373051+0000","last_active":"2024-04-05T22:23:10.350119+0000","last_peered":"2024-04-05T22:23:10.350119+0000","last_clean":"2024-04-05T22:23:10.350119+0000","last_became_active":"2024-04-05T22:23:09.372014+0000","last_became_peered":"2024-04-05T22:23:09.372014+0000","last_unstale":"2024-04-05T22:23:10.350119+0000","last_undegraded":"2024-04-05T22:23:10.350119+0000","last_fullsized":"2024-04-05T22:23:10.350119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:58.535697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072702+0000","last_change":"2024-04-05T22:23:09.360582+0000","last_active":"2024-04-05T22:23:11.072702+0000","last_peered":"2024-04-05T22:23:11.072702+0000","last_clean":"2024-04-05T22:23:11.072702+0000","last_became_active":"2024-04-05T22:23:09.360346+0000","last_became_peered":"2024-04-05T22:23:09.360346+0000","last_unstale":"2024-04-05T22:23:11.072702+0000","last_undegraded":"2024-04-05T22:23:11.072702+0000","last_fullsized":"2024-04-05T22:23:11.072702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071233+0000","last_change":"2024-04-05T22:23:09.365234+0000","last_active":"2024-04-05T22:23:11.071233+0000","last_peered":"2024-04-05T22:23:11.071233+0000","last_clean":"2024-04-05T22:23:11.071233+0000","last_became_active":"2024-04-05T22:23:09.364930+0000","last_became_peered":"2024-04-05T22:23:09.364930+0000","last_unstale":"2024-04-05T22:23:11.071233+0000","last_undegraded":"2024-04-05T22:23:11.071233+0000","last_fullsized":"2024-04-05T22:23:11.071233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:33:24.735249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071336+0000","last_change":"2024-04-05T22:23:09.367889+0000","last_active":"2024-04-05T22:23:11.071336+0000","last_peered":"2024-04-05T22:23:11.071336+0000","last_clean":"2024-04-05T22:23:11.071336+0000","last_became_active":"2024-04-05T22:23:09.367652+0000","last_became_peered":"2024-04-05T22:23:09.367652+0000","last_unstale":"2024-04-05T22:23:11.071336+0000","last_undegraded":"2024-04-05T22:23:11.071336+0000","last_fullsized":"2024-04-05T22:23:11.071336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:03.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073053+0000","last_change":"2024-04-05T22:23:09.365048+0000","last_active":"2024-04-05T22:23:11.073053+0000","last_peered":"2024-04-05T22:23:11.073053+0000","last_clean":"2024-04-05T22:23:11.073053+0000","last_became_active":"2024-04-05T22:23:09.364707+0000","last_became_peered":"2024-04-05T22:23:09.364707+0000","last_unstale":"2024-04-05T22:23:11.073053+0000","last_undegraded":"2024-04-05T22:23:11.073053+0000","last_fullsized":"2024-04-05T22:23:11.073053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:43.155641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071908+0000","last_change":"2024-04-05T22:23:09.375125+0000","last_active":"2024-04-05T22:23:11.071908+0000","last_peered":"2024-04-05T22:23:11.071908+0000","last_clean":"2024-04-05T22:23:11.071908+0000","last_became_active":"2024-04-05T22:23:09.373382+0000","last_became_peered":"2024-04-05T22:23:09.373382+0000","last_unstale":"2024-04-05T22:23:11.071908+0000","last_undegraded":"2024-04-05T22:23:11.071908+0000","last_fullsized":"2024-04-05T22:23:11.071908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:53:45.868945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073020+0000","last_change":"2024-04-05T22:23:09.376021+0000","last_active":"2024-04-05T22:23:11.073020+0000","last_peered":"2024-04-05T22:23:11.073020+0000","last_clean":"2024-04-05T22:23:11.073020+0000","last_became_active":"2024-04-05T22:23:09.375702+0000","last_became_peered":"2024-04-05T22:23:09.375702+0000","last_unstale":"2024-04-05T22:23:11.073020+0000","last_undegraded":"2024-04-05T22:23:11.073020+0000","last_fullsized":"2024-04-05T22:23:11.073020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:55.351589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487322+0000","last_change":"2024-04-05T22:23:09.356705+0000","last_active":"2024-04-05T22:23:10.487322+0000","last_peered":"2024-04-05T22:23:10.487322+0000","last_clean":"2024-04-05T22:23:10.487322+0000","last_became_active":"2024-04-05T22:23:09.356429+0000","last_became_peered":"2024-04-05T22:23:09.356429+0000","last_unstale":"2024-04-05T22:23:10.487322+0000","last_undegraded":"2024-04-05T22:23:10.487322+0000","last_fullsized":"2024-04-05T22:23:10.487322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071142+0000","last_change":"2024-04-05T22:23:09.364089+0000","last_active":"2024-04-05T22:23:11.071142+0000","last_peered":"2024-04-05T22:23:11.071142+0000","last_clean":"2024-04-05T22:23:11.071142+0000","last_became_active":"2024-04-05T22:23:09.363793+0000","last_became_peered":"2024-04-05T22:23:09.363793+0000","last_unstale":"2024-04-05T22:23:11.071142+0000","last_undegraded":"2024-04-05T22:23:11.071142+0000","last_fullsized":"2024-04-05T22:23:11.071142+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:18:38.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071969+0000","last_change":"2024-04-05T22:23:09.365957+0000","last_active":"2024-04-05T22:23:11.071969+0000","last_peered":"2024-04-05T22:23:11.071969+0000","last_clean":"2024-04-05T22:23:11.071969+0000","last_became_active":"2024-04-05T22:23:09.365640+0000","last_became_peered":"2024-04-05T22:23:09.365640+0000","last_unstale":"2024-04-05T22:23:11.071969+0000","last_undegraded":"2024-04-05T22:23:11.071969+0000","last_fullsized":"2024-04-05T22:23:11.071969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:48.736221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349506+0000","last_change":"2024-04-05T22:23:09.377291+0000","last_active":"2024-04-05T22:23:10.349506+0000","last_peered":"2024-04-05T22:23:10.349506+0000","last_clean":"2024-04-05T22:23:10.349506+0000","last_became_active":"2024-04-05T22:23:09.377044+0000","last_became_peered":"2024-04-05T22:23:09.377044+0000","last_unstale":"2024-04-05T22:23:10.349506+0000","last_undegraded":"2024-04-05T22:23:10.349506+0000","last_fullsized":"2024-04-05T22:23:10.349506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:13.822332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070568+0000","last_change":"2024-04-05T22:23:09.348727+0000","last_active":"2024-04-05T22:23:11.070568+0000","last_peered":"2024-04-05T22:23:11.070568+0000","last_clean":"2024-04-05T22:23:11.070568+0000","last_became_active":"2024-04-05T22:23:09.348346+0000","last_became_peered":"2024-04-05T22:23:09.348346+0000","last_unstale":"2024-04-05T22:23:11.070568+0000","last_undegraded":"2024-04-05T22:23:11.070568+0000","last_fullsized":"2024-04-05T22:23:11.070568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:18.309908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347754+0000","last_change":"2024-04-05T22:23:09.379081+0000","last_active":"2024-04-05T22:23:10.347754+0000","last_peered":"2024-04-05T22:23:10.347754+0000","last_clean":"2024-04-05T22:23:10.347754+0000","last_became_active":"2024-04-05T22:23:09.378754+0000","last_became_peered":"2024-04-05T22:23:09.378754+0000","last_unstale":"2024-04-05T22:23:10.347754+0000","last_undegraded":"2024-04-05T22:23:10.347754+0000","last_fullsized":"2024-04-05T22:23:10.347754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:07.603359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350430+0000","last_change":"2024-04-05T22:23:08.841250+0000","last_active":"2024-04-05T22:23:10.350430+0000","last_peered":"2024-04-05T22:23:10.350430+0000","last_clean":"2024-04-05T22:23:10.350430+0000","last_became_active":"2024-04-05T22:23:08.841032+0000","last_became_peered":"2024-04-05T22:23:08.841032+0000","last_unstale":"2024-04-05T22:23:10.350430+0000","last_undegraded":"2024-04-05T22:23:10.350430+0000","last_fullsized":"2024-04-05T22:23:10.350430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:06.598376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073172+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.073172+0000","last_peered":"2024-04-05T22:23:11.073172+0000","last_clean":"2024-04-05T22:23:11.073172+0000","last_became_active":"2024-04-05T22:23:09.371747+0000","last_became_peered":"2024-04-05T22:23:09.371747+0000","last_unstale":"2024-04-05T22:23:11.073172+0000","last_undegraded":"2024-04-05T22:23:11.073172+0000","last_fullsized":"2024-04-05T22:23:11.073172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:48:19.797618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073337+0000","last_change":"2024-04-05T22:23:08.839262+0000","last_active":"2024-04-05T22:23:11.073337+0000","last_peered":"2024-04-05T22:23:11.073337+0000","last_clean":"2024-04-05T22:23:11.073337+0000","last_became_active":"2024-04-05T22:23:08.838980+0000","last_became_peered":"2024-04-05T22:23:08.838980+0000","last_unstale":"2024-04-05T22:23:11.073337+0000","last_undegraded":"2024-04-05T22:23:11.073337+0000","last_fullsized":"2024-04-05T22:23:11.073337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:24:35.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487575+0000","last_change":"2024-04-05T22:23:09.372679+0000","last_active":"2024-04-05T22:23:10.487575+0000","last_peered":"2024-04-05T22:23:10.487575+0000","last_clean":"2024-04-05T22:23:10.487575+0000","last_became_active":"2024-04-05T22:23:09.372413+0000","last_became_peered":"2024-04-05T22:23:09.372413+0000","last_unstale":"2024-04-05T22:23:10.487575+0000","last_undegraded":"2024-04-05T22:23:10.487575+0000","last_fullsized":"2024-04-05T22:23:10.487575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347122+0000","last_change":"2024-04-05T22:23:08.831526+0000","last_active":"2024-04-05T22:23:10.347122+0000","last_peered":"2024-04-05T22:23:10.347122+0000","last_clean":"2024-04-05T22:23:10.347122+0000","last_became_active":"2024-04-05T22:23:08.831156+0000","last_became_peered":"2024-04-05T22:23:08.831156+0000","last_unstale":"2024-04-05T22:23:10.347122+0000","last_undegraded":"2024-04-05T22:23:10.347122+0000","last_fullsized":"2024-04-05T22:23:10.347122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347002+0000","last_change":"2024-04-05T22:23:09.371601+0000","last_active":"2024-04-05T22:23:10.347002+0000","last_peered":"2024-04-05T22:23:10.347002+0000","last_clean":"2024-04-05T22:23:10.347002+0000","last_became_active":"2024-04-05T22:23:09.371235+0000","last_became_peered":"2024-04-05T22:23:09.371235+0000","last_unstale":"2024-04-05T22:23:10.347002+0000","last_undegraded":"2024-04-05T22:23:10.347002+0000","last_fullsized":"2024-04-05T22:23:10.347002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347223+0000","last_change":"2024-04-05T22:23:08.834593+0000","last_active":"2024-04-05T22:23:10.347223+0000","last_peered":"2024-04-05T22:23:10.347223+0000","last_clean":"2024-04-05T22:23:10.347223+0000","last_became_active":"2024-04-05T22:23:08.833796+0000","last_became_peered":"2024-04-05T22:23:08.833796+0000","last_unstale":"2024-04-05T22:23:10.347223+0000","last_undegraded":"2024-04-05T22:23:10.347223+0000","last_fullsized":"2024-04-05T22:23:10.347223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072191+0000","last_change":"2024-04-05T22:23:09.364226+0000","last_active":"2024-04-05T22:23:11.072191+0000","last_peered":"2024-04-05T22:23:11.072191+0000","last_clean":"2024-04-05T22:23:11.072191+0000","last_became_active":"2024-04-05T22:23:09.363915+0000","last_became_peered":"2024-04-05T22:23:09.363915+0000","last_unstale":"2024-04-05T22:23:11.072191+0000","last_undegraded":"2024-04-05T22:23:11.072191+0000","last_fullsized":"2024-04-05T22:23:11.072191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:41:08.452502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350046+0000","last_change":"2024-04-05T22:23:08.837746+0000","last_active":"2024-04-05T22:23:10.350046+0000","last_peered":"2024-04-05T22:23:10.350046+0000","last_clean":"2024-04-05T22:23:10.350046+0000","last_became_active":"2024-04-05T22:23:08.837489+0000","last_became_peered":"2024-04-05T22:23:08.837489+0000","last_unstale":"2024-04-05T22:23:10.350046+0000","last_undegraded":"2024-04-05T22:23:10.350046+0000","last_fullsized":"2024-04-05T22:23:10.350046+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:34.404596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071930+0000","last_change":"2024-04-05T22:23:09.357300+0000","last_active":"2024-04-05T22:23:11.071930+0000","last_peered":"2024-04-05T22:23:11.071930+0000","last_clean":"2024-04-05T22:23:11.071930+0000","last_became_active":"2024-04-05T22:23:09.357047+0000","last_became_peered":"2024-04-05T22:23:09.357047+0000","last_unstale":"2024-04-05T22:23:11.071930+0000","last_undegraded":"2024-04-05T22:23:11.071930+0000","last_fullsized":"2024-04-05T22:23:11.071930+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072432+0000","last_change":"2024-04-05T22:23:08.837128+0000","last_active":"2024-04-05T22:23:11.072432+0000","last_peered":"2024-04-05T22:23:11.072432+0000","last_clean":"2024-04-05T22:23:11.072432+0000","last_became_active":"2024-04-05T22:23:08.836444+0000","last_became_peered":"2024-04-05T22:23:08.836444+0000","last_unstale":"2024-04-05T22:23:11.072432+0000","last_undegraded":"2024-04-05T22:23:11.072432+0000","last_fullsized":"2024-04-05T22:23:11.072432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:27.879231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071573+0000","last_change":"2024-04-05T22:23:09.362416+0000","last_active":"2024-04-05T22:23:11.071573+0000","last_peered":"2024-04-05T22:23:11.071573+0000","last_clean":"2024-04-05T22:23:11.071573+0000","last_became_active":"2024-04-05T22:23:09.361962+0000","last_became_peered":"2024-04-05T22:23:09.361962+0000","last_unstale":"2024-04-05T22:23:11.071573+0000","last_undegraded":"2024-04-05T22:23:11.071573+0000","last_fullsized":"2024-04-05T22:23:11.071573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:52.825291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486775+0000","last_change":"2024-04-05T22:23:08.831328+0000","last_active":"2024-04-05T22:23:10.486775+0000","last_peered":"2024-04-05T22:23:10.486775+0000","last_clean":"2024-04-05T22:23:10.486775+0000","last_became_active":"2024-04-05T22:23:08.830959+0000","last_became_peered":"2024-04-05T22:23:08.830959+0000","last_unstale":"2024-04-05T22:23:10.486775+0000","last_undegraded":"2024-04-05T22:23:10.486775+0000","last_fullsized":"2024-04-05T22:23:10.486775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070939+0000","last_change":"2024-04-05T22:23:09.365286+0000","last_active":"2024-04-05T22:23:11.070939+0000","last_peered":"2024-04-05T22:23:11.070939+0000","last_clean":"2024-04-05T22:23:11.070939+0000","last_became_active":"2024-04-05T22:23:09.364944+0000","last_became_peered":"2024-04-05T22:23:09.364944+0000","last_unstale":"2024-04-05T22:23:11.070939+0000","last_undegraded":"2024-04-05T22:23:11.070939+0000","last_fullsized":"2024-04-05T22:23:11.070939+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:39:32.120858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071812+0000","last_change":"2024-04-05T22:23:08.833902+0000","last_active":"2024-04-05T22:23:11.071812+0000","last_peered":"2024-04-05T22:23:11.071812+0000","last_clean":"2024-04-05T22:23:11.071812+0000","last_became_active":"2024-04-05T22:23:08.833179+0000","last_became_peered":"2024-04-05T22:23:08.833179+0000","last_unstale":"2024-04-05T22:23:11.071812+0000","last_undegraded":"2024-04-05T22:23:11.071812+0000","last_fullsized":"2024-04-05T22:23:11.071812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:08:00.232153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073368+0000","last_change":"2024-04-05T22:23:09.379153+0000","last_active":"2024-04-05T22:23:11.073368+0000","last_peered":"2024-04-05T22:23:11.073368+0000","last_clean":"2024-04-05T22:23:11.073368+0000","last_became_active":"2024-04-05T22:23:09.378897+0000","last_became_peered":"2024-04-05T22:23:09.378897+0000","last_unstale":"2024-04-05T22:23:11.073368+0000","last_undegraded":"2024-04-05T22:23:11.073368+0000","last_fullsized":"2024-04-05T22:23:11.073368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:08.612559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072706+0000","last_change":"2024-04-05T22:23:08.828406+0000","last_active":"2024-04-05T22:23:11.072706+0000","last_peered":"2024-04-05T22:23:11.072706+0000","last_clean":"2024-04-05T22:23:11.072706+0000","last_became_active":"2024-04-05T22:23:08.828116+0000","last_became_peered":"2024-04-05T22:23:08.828116+0000","last_unstale":"2024-04-05T22:23:11.072706+0000","last_undegraded":"2024-04-05T22:23:11.072706+0000","last_fullsized":"2024-04-05T22:23:11.072706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347731+0000","last_change":"2024-04-05T22:23:09.370788+0000","last_active":"2024-04-05T22:23:10.347731+0000","last_peered":"2024-04-05T22:23:10.347731+0000","last_clean":"2024-04-05T22:23:10.347731+0000","last_became_active":"2024-04-05T22:23:09.370475+0000","last_became_peered":"2024-04-05T22:23:09.370475+0000","last_unstale":"2024-04-05T22:23:10.347731+0000","last_undegraded":"2024-04-05T22:23:10.347731+0000","last_fullsized":"2024-04-05T22:23:10.347731+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073323+0000","last_change":"2024-04-05T22:23:08.833154+0000","last_active":"2024-04-05T22:23:11.073323+0000","last_peered":"2024-04-05T22:23:11.073323+0000","last_clean":"2024-04-05T22:23:11.073323+0000","last_became_active":"2024-04-05T22:23:08.831734+0000","last_became_peered":"2024-04-05T22:23:08.831734+0000","last_unstale":"2024-04-05T22:23:11.073323+0000","last_undegraded":"2024-04-05T22:23:11.073323+0000","last_fullsized":"2024-04-05T22:23:11.073323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:21.532272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349928+0000","last_change":"2024-04-05T22:23:09.368222+0000","last_active":"2024-04-05T22:23:10.349928+0000","last_peered":"2024-04-05T22:23:10.349928+0000","last_clean":"2024-04-05T22:23:10.349928+0000","last_became_active":"2024-04-05T22:23:09.367924+0000","last_became_peered":"2024-04-05T22:23:09.367924+0000","last_unstale":"2024-04-05T22:23:10.349928+0000","last_undegraded":"2024-04-05T22:23:10.349928+0000","last_fullsized":"2024-04-05T22:23:10.349928+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347148+0000","last_change":"2024-04-05T22:23:08.839971+0000","last_active":"2024-04-05T22:23:10.347148+0000","last_peered":"2024-04-05T22:23:10.347148+0000","last_clean":"2024-04-05T22:23:10.347148+0000","last_became_active":"2024-04-05T22:23:08.839707+0000","last_became_peered":"2024-04-05T22:23:08.839707+0000","last_unstale":"2024-04-05T22:23:10.347148+0000","last_undegraded":"2024-04-05T22:23:10.347148+0000","last_fullsized":"2024-04-05T22:23:10.347148+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347367+0000","last_change":"2024-04-05T22:23:09.368953+0000","last_active":"2024-04-05T22:23:10.347367+0000","last_peered":"2024-04-05T22:23:10.347367+0000","last_clean":"2024-04-05T22:23:10.347367+0000","last_became_active":"2024-04-05T22:23:09.367313+0000","last_became_peered":"2024-04-05T22:23:09.367313+0000","last_unstale":"2024-04-05T22:23:10.347367+0000","last_undegraded":"2024-04-05T22:23:10.347367+0000","last_fullsized":"2024-04-05T22:23:10.347367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072503+0000","last_change":"2024-04-05T22:23:08.825046+0000","last_active":"2024-04-05T22:23:11.072503+0000","last_peered":"2024-04-05T22:23:11.072503+0000","last_clean":"2024-04-05T22:23:11.072503+0000","last_became_active":"2024-04-05T22:23:08.824750+0000","last_became_peered":"2024-04-05T22:23:08.824750+0000","last_unstale":"2024-04-05T22:23:11.072503+0000","last_undegraded":"2024-04-05T22:23:11.072503+0000","last_fullsized":"2024-04-05T22:23:11.072503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:07.006045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071685+0000","last_change":"2024-04-05T22:23:09.367072+0000","last_active":"2024-04-05T22:23:11.071685+0000","last_peered":"2024-04-05T22:23:11.071685+0000","last_clean":"2024-04-05T22:23:11.071685+0000","last_became_active":"2024-04-05T22:23:09.366737+0000","last_became_peered":"2024-04-05T22:23:09.366737+0000","last_unstale":"2024-04-05T22:23:11.071685+0000","last_undegraded":"2024-04-05T22:23:11.071685+0000","last_fullsized":"2024-04-05T22:23:11.071685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:13.331331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350234+0000","last_change":"2024-04-05T22:23:08.840066+0000","last_active":"2024-04-05T22:23:10.350234+0000","last_peered":"2024-04-05T22:23:10.350234+0000","last_clean":"2024-04-05T22:23:10.350234+0000","last_became_active":"2024-04-05T22:23:08.839794+0000","last_became_peered":"2024-04-05T22:23:08.839794+0000","last_unstale":"2024-04-05T22:23:10.350234+0000","last_undegraded":"2024-04-05T22:23:10.350234+0000","last_fullsized":"2024-04-05T22:23:10.350234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:39.221256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072213+0000","last_change":"2024-04-05T22:23:09.370990+0000","last_active":"2024-04-05T22:23:11.072213+0000","last_peered":"2024-04-05T22:23:11.072213+0000","last_clean":"2024-04-05T22:23:11.072213+0000","last_became_active":"2024-04-05T22:23:09.370739+0000","last_became_peered":"2024-04-05T22:23:09.370739+0000","last_unstale":"2024-04-05T22:23:11.072213+0000","last_undegraded":"2024-04-05T22:23:11.072213+0000","last_fullsized":"2024-04-05T22:23:11.072213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347073+0000","last_change":"2024-04-05T22:23:08.831972+0000","last_active":"2024-04-05T22:23:10.347073+0000","last_peered":"2024-04-05T22:23:10.347073+0000","last_clean":"2024-04-05T22:23:10.347073+0000","last_became_active":"2024-04-05T22:23:08.831650+0000","last_became_peered":"2024-04-05T22:23:08.831650+0000","last_unstale":"2024-04-05T22:23:10.347073+0000","last_undegraded":"2024-04-05T22:23:10.347073+0000","last_fullsized":"2024-04-05T22:23:10.347073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:52:24.125437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.348862+0000","last_change":"2024-04-05T22:23:09.364012+0000","last_active":"2024-04-05T22:23:10.348862+0000","last_peered":"2024-04-05T22:23:10.348862+0000","last_clean":"2024-04-05T22:23:10.348862+0000","last_became_active":"2024-04-05T22:23:09.363768+0000","last_became_peered":"2024-04-05T22:23:09.363768+0000","last_unstale":"2024-04-05T22:23:10.348862+0000","last_undegraded":"2024-04-05T22:23:10.348862+0000","last_fullsized":"2024-04-05T22:23:10.348862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:30.810750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349521+0000","last_change":"2024-04-05T22:23:08.840058+0000","last_active":"2024-04-05T22:23:10.349521+0000","last_peered":"2024-04-05T22:23:10.349521+0000","last_clean":"2024-04-05T22:23:10.349521+0000","last_became_active":"2024-04-05T22:23:08.839727+0000","last_became_peered":"2024-04-05T22:23:08.839727+0000","last_unstale":"2024-04-05T22:23:10.349521+0000","last_undegraded":"2024-04-05T22:23:10.349521+0000","last_fullsized":"2024-04-05T22:23:10.349521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:56.777874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347343+0000","last_change":"2024-04-05T22:23:09.362618+0000","last_active":"2024-04-05T22:23:10.347343+0000","last_peered":"2024-04-05T22:23:10.347343+0000","last_clean":"2024-04-05T22:23:10.347343+0000","last_became_active":"2024-04-05T22:23:09.362358+0000","last_became_peered":"2024-04-05T22:23:09.362358+0000","last_unstale":"2024-04-05T22:23:10.347343+0000","last_undegraded":"2024-04-05T22:23:10.347343+0000","last_fullsized":"2024-04-05T22:23:10.347343+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072045+0000","last_change":"2024-04-05T22:23:08.839690+0000","last_active":"2024-04-05T22:23:11.072045+0000","last_peered":"2024-04-05T22:23:11.072045+0000","last_clean":"2024-04-05T22:23:11.072045+0000","last_became_active":"2024-04-05T22:23:08.838945+0000","last_became_peered":"2024-04-05T22:23:08.838945+0000","last_unstale":"2024-04-05T22:23:11.072045+0000","last_undegraded":"2024-04-05T22:23:11.072045+0000","last_fullsized":"2024-04-05T22:23:11.072045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:19:04.891763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072339+0000","last_change":"2024-04-05T22:23:09.352156+0000","last_active":"2024-04-05T22:23:11.072339+0000","last_peered":"2024-04-05T22:23:11.072339+0000","last_clean":"2024-04-05T22:23:11.072339+0000","last_became_active":"2024-04-05T22:23:09.351439+0000","last_became_peered":"2024-04-05T22:23:09.351439+0000","last_unstale":"2024-04-05T22:23:11.072339+0000","last_undegraded":"2024-04-05T22:23:11.072339+0000","last_fullsized":"2024-04-05T22:23:11.072339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:22.537256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072786+0000","last_change":"2024-04-05T22:23:08.828030+0000","last_active":"2024-04-05T22:23:11.072786+0000","last_peered":"2024-04-05T22:23:11.072786+0000","last_clean":"2024-04-05T22:23:11.072786+0000","last_became_active":"2024-04-05T22:23:08.827689+0000","last_became_peered":"2024-04-05T22:23:08.827689+0000","last_unstale":"2024-04-05T22:23:11.072786+0000","last_undegraded":"2024-04-05T22:23:11.072786+0000","last_fullsized":"2024-04-05T22:23:11.072786+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:36.825290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072998+0000","last_change":"2024-04-05T22:23:09.374497+0000","last_active":"2024-04-05T22:23:11.072998+0000","last_peered":"2024-04-05T22:23:11.072998+0000","last_clean":"2024-04-05T22:23:11.072998+0000","last_became_active":"2024-04-05T22:23:09.374239+0000","last_became_peered":"2024-04-05T22:23:09.374239+0000","last_unstale":"2024-04-05T22:23:11.072998+0000","last_undegraded":"2024-04-05T22:23:11.072998+0000","last_fullsized":"2024-04-05T22:23:11.072998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:07:24.621644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073462+0000","last_change":"2024-04-05T22:23:08.822162+0000","last_active":"2024-04-05T22:23:11.073462+0000","last_peered":"2024-04-05T22:23:11.073462+0000","last_clean":"2024-04-05T22:23:11.073462+0000","last_became_active":"2024-04-05T22:23:08.821738+0000","last_became_peered":"2024-04-05T22:23:08.821738+0000","last_unstale":"2024-04-05T22:23:11.073462+0000","last_undegraded":"2024-04-05T22:23:11.073462+0000","last_fullsized":"2024-04-05T22:23:11.073462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:35:05.837398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071741+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.071741+0000","last_peered":"2024-04-05T22:23:11.071741+0000","last_clean":"2024-04-05T22:23:11.071741+0000","last_became_active":"2024-04-05T22:23:09.372140+0000","last_became_peered":"2024-04-05T22:23:09.372140+0000","last_unstale":"2024-04-05T22:23:11.071741+0000","last_undegraded":"2024-04-05T22:23:11.071741+0000","last_fullsized":"2024-04-05T22:23:11.071741+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:56:39.781215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487427+0000","last_change":"2024-04-05T22:23:08.833929+0000","last_active":"2024-04-05T22:23:10.487427+0000","last_peered":"2024-04-05T22:23:10.487427+0000","last_clean":"2024-04-05T22:23:10.487427+0000","last_became_active":"2024-04-05T22:23:08.831111+0000","last_became_peered":"2024-04-05T22:23:08.831111+0000","last_unstale":"2024-04-05T22:23:10.487427+0000","last_undegraded":"2024-04-05T22:23:10.487427+0000","last_fullsized":"2024-04-05T22:23:10.487427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072277+0000","last_change":"2024-04-05T22:23:09.365652+0000","last_active":"2024-04-05T22:23:11.072277+0000","last_peered":"2024-04-05T22:23:11.072277+0000","last_clean":"2024-04-05T22:23:11.072277+0000","last_became_active":"2024-04-05T22:23:09.364565+0000","last_became_peered":"2024-04-05T22:23:09.364565+0000","last_unstale":"2024-04-05T22:23:11.072277+0000","last_undegraded":"2024-04-05T22:23:11.072277+0000","last_fullsized":"2024-04-05T22:23:11.072277+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:51.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350248+0000","last_change":"2024-04-05T22:23:08.841687+0000","last_active":"2024-04-05T22:23:10.350248+0000","last_peered":"2024-04-05T22:23:10.350248+0000","last_clean":"2024-04-05T22:23:10.350248+0000","last_became_active":"2024-04-05T22:23:08.841487+0000","last_became_peered":"2024-04-05T22:23:08.841487+0000","last_unstale":"2024-04-05T22:23:10.350248+0000","last_undegraded":"2024-04-05T22:23:10.350248+0000","last_fullsized":"2024-04-05T22:23:10.350248+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:35:48.140463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349783+0000","last_change":"2024-04-05T22:23:09.368685+0000","last_active":"2024-04-05T22:23:10.349783+0000","last_peered":"2024-04-05T22:23:10.349783+0000","last_clean":"2024-04-05T22:23:10.349783+0000","last_became_active":"2024-04-05T22:23:09.368451+0000","last_became_peered":"2024-04-05T22:23:09.368451+0000","last_unstale":"2024-04-05T22:23:10.349783+0000","last_undegraded":"2024-04-05T22:23:10.349783+0000","last_fullsized":"2024-04-05T22:23:10.349783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:11.976720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350456+0000","last_change":"2024-04-05T22:23:08.838367+0000","last_active":"2024-04-05T22:23:10.350456+0000","last_peered":"2024-04-05T22:23:10.350456+0000","last_clean":"2024-04-05T22:23:10.350456+0000","last_became_active":"2024-04-05T22:23:08.837504+0000","last_became_peered":"2024-04-05T22:23:08.837504+0000","last_unstale":"2024-04-05T22:23:10.350456+0000","last_undegraded":"2024-04-05T22:23:10.350456+0000","last_fullsized":"2024-04-05T22:23:10.350456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:56.682813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346632+0000","last_change":"2024-04-05T22:23:09.371368+0000","last_active":"2024-04-05T22:23:10.346632+0000","last_peered":"2024-04-05T22:23:10.346632+0000","last_clean":"2024-04-05T22:23:10.346632+0000","last_became_active":"2024-04-05T22:23:09.369500+0000","last_became_peered":"2024-04-05T22:23:09.369500+0000","last_unstale":"2024-04-05T22:23:10.346632+0000","last_undegraded":"2024-04-05T22:23:10.346632+0000","last_fullsized":"2024-04-05T22:23:10.346632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:40.226240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487033+0000","last_change":"2024-04-05T22:23:08.838608+0000","last_active":"2024-04-05T22:23:10.487033+0000","last_peered":"2024-04-05T22:23:10.487033+0000","last_clean":"2024-04-05T22:23:10.487033+0000","last_became_active":"2024-04-05T22:23:08.838196+0000","last_became_peered":"2024-04-05T22:23:08.838196+0000","last_unstale":"2024-04-05T22:23:10.487033+0000","last_undegraded":"2024-04-05T22:23:10.487033+0000","last_fullsized":"2024-04-05T22:23:10.487033+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070603+0000","last_change":"2024-04-05T22:23:09.368050+0000","last_active":"2024-04-05T22:23:11.070603+0000","last_peered":"2024-04-05T22:23:11.070603+0000","last_clean":"2024-04-05T22:23:11.070603+0000","last_became_active":"2024-04-05T22:23:09.367340+0000","last_became_peered":"2024-04-05T22:23:09.367340+0000","last_unstale":"2024-04-05T22:23:11.070603+0000","last_undegraded":"2024-04-05T22:23:11.070603+0000","last_fullsized":"2024-04-05T22:23:11.070603+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:10.015015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071791+0000","last_change":"2024-04-05T22:23:08.832053+0000","last_active":"2024-04-05T22:23:11.071791+0000","last_peered":"2024-04-05T22:23:11.071791+0000","last_clean":"2024-04-05T22:23:11.071791+0000","last_became_active":"2024-04-05T22:23:08.831669+0000","last_became_peered":"2024-04-05T22:23:08.831669+0000","last_unstale":"2024-04-05T22:23:11.071791+0000","last_undegraded":"2024-04-05T22:23:11.071791+0000","last_fullsized":"2024-04-05T22:23:11.071791+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:17.304924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071638+0000","last_change":"2024-04-05T22:23:09.362024+0000","last_active":"2024-04-05T22:23:11.071638+0000","last_peered":"2024-04-05T22:23:11.071638+0000","last_clean":"2024-04-05T22:23:11.071638+0000","last_became_active":"2024-04-05T22:23:09.361762+0000","last_became_peered":"2024-04-05T22:23:09.361762+0000","last_unstale":"2024-04-05T22:23:11.071638+0000","last_undegraded":"2024-04-05T22:23:11.071638+0000","last_fullsized":"2024-04-05T22:23:11.071638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346491+0000","last_change":"2024-04-05T22:23:08.831835+0000","last_active":"2024-04-05T22:23:10.346491+0000","last_peered":"2024-04-05T22:23:10.346491+0000","last_clean":"2024-04-05T22:23:10.346491+0000","last_became_active":"2024-04-05T22:23:08.831526+0000","last_became_peered":"2024-04-05T22:23:08.831526+0000","last_unstale":"2024-04-05T22:23:10.346491+0000","last_undegraded":"2024-04-05T22:23:10.346491+0000","last_fullsized":"2024-04-05T22:23:10.346491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:09.010032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073313+0000","last_change":"2024-04-05T22:23:09.361600+0000","last_active":"2024-04-05T22:23:11.073313+0000","last_peered":"2024-04-05T22:23:11.073313+0000","last_clean":"2024-04-05T22:23:11.073313+0000","last_became_active":"2024-04-05T22:23:09.361317+0000","last_became_peered":"2024-04-05T22:23:09.361317+0000","last_unstale":"2024-04-05T22:23:11.073313+0000","last_undegraded":"2024-04-05T22:23:11.073313+0000","last_fullsized":"2024-04-05T22:23:11.073313+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:07.080539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350095+0000","last_change":"2024-04-05T22:23:08.835573+0000","last_active":"2024-04-05T22:23:10.350095+0000","last_peered":"2024-04-05T22:23:10.350095+0000","last_clean":"2024-04-05T22:23:10.350095+0000","last_became_active":"2024-04-05T22:23:08.835334+0000","last_became_peered":"2024-04-05T22:23:08.835334+0000","last_unstale":"2024-04-05T22:23:10.350095+0000","last_undegraded":"2024-04-05T22:23:10.350095+0000","last_fullsized":"2024-04-05T22:23:10.350095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:16.063665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072567+0000","last_change":"2024-04-05T22:23:09.369695+0000","last_active":"2024-04-05T22:23:11.072567+0000","last_peered":"2024-04-05T22:23:11.072567+0000","last_clean":"2024-04-05T22:23:11.072567+0000","last_became_active":"2024-04-05T22:23:09.366626+0000","last_became_peered":"2024-04-05T22:23:09.366626+0000","last_unstale":"2024-04-05T22:23:11.072567+0000","last_undegraded":"2024-04-05T22:23:11.072567+0000","last_fullsized":"2024-04-05T22:23:11.072567+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347294+0000","last_change":"2024-04-05T22:23:08.834972+0000","last_active":"2024-04-05T22:23:10.347294+0000","last_peered":"2024-04-05T22:23:10.347294+0000","last_clean":"2024-04-05T22:23:10.347294+0000","last_became_active":"2024-04-05T22:23:08.834348+0000","last_became_peered":"2024-04-05T22:23:08.834348+0000","last_unstale":"2024-04-05T22:23:10.347294+0000","last_undegraded":"2024-04-05T22:23:10.347294+0000","last_fullsized":"2024-04-05T22:23:10.347294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:47.731238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346985+0000","last_change":"2024-04-05T22:23:09.362042+0000","last_active":"2024-04-05T22:23:10.346985+0000","last_peered":"2024-04-05T22:23:10.346985+0000","last_clean":"2024-04-05T22:23:10.346985+0000","last_became_active":"2024-04-05T22:23:09.360332+0000","last_became_peered":"2024-04-05T22:23:09.360332+0000","last_unstale":"2024-04-05T22:23:10.346985+0000","last_undegraded":"2024-04-05T22:23:10.346985+0000","last_fullsized":"2024-04-05T22:23:10.346985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346736+0000","last_change":"2024-04-05T22:23:08.834808+0000","last_active":"2024-04-05T22:23:10.346736+0000","last_peered":"2024-04-05T22:23:10.346736+0000","last_clean":"2024-04-05T22:23:10.346736+0000","last_became_active":"2024-04-05T22:23:08.834540+0000","last_became_peered":"2024-04-05T22:23:08.834540+0000","last_unstale":"2024-04-05T22:23:10.346736+0000","last_undegraded":"2024-04-05T22:23:10.346736+0000","last_fullsized":"2024-04-05T22:23:10.346736+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349819+0000","last_change":"2024-04-05T22:23:09.368779+0000","last_active":"2024-04-05T22:23:10.349819+0000","last_peered":"2024-04-05T22:23:10.349819+0000","last_clean":"2024-04-05T22:23:10.349819+0000","last_became_active":"2024-04-05T22:23:09.368525+0000","last_became_peered":"2024-04-05T22:23:09.368525+0000","last_unstale":"2024-04-05T22:23:10.349819+0000","last_undegraded":"2024-04-05T22:23:10.349819+0000","last_fullsized":"2024-04-05T22:23:10.349819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:55.756624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347079+0000","last_change":"2024-04-05T22:23:08.837346+0000","last_active":"2024-04-05T22:23:10.347079+0000","last_peered":"2024-04-05T22:23:10.347079+0000","last_clean":"2024-04-05T22:23:10.347079+0000","last_became_active":"2024-04-05T22:23:08.834887+0000","last_became_peered":"2024-04-05T22:23:08.834887+0000","last_unstale":"2024-04-05T22:23:10.347079+0000","last_undegraded":"2024-04-05T22:23:10.347079+0000","last_fullsized":"2024-04-05T22:23:10.347079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:53.019125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349798+0000","last_change":"2024-04-05T22:23:09.372281+0000","last_active":"2024-04-05T22:23:10.349798+0000","last_peered":"2024-04-05T22:23:10.349798+0000","last_clean":"2024-04-05T22:23:10.349798+0000","last_became_active":"2024-04-05T22:23:09.371720+0000","last_became_peered":"2024-04-05T22:23:09.371720+0000","last_unstale":"2024-04-05T22:23:10.349798+0000","last_undegraded":"2024-04-05T22:23:10.349798+0000","last_fullsized":"2024-04-05T22:23:10.349798+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:16:32.033132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072619+0000","last_change":"2024-04-05T22:23:08.831235+0000","last_active":"2024-04-05T22:23:11.072619+0000","last_peered":"2024-04-05T22:23:11.072619+0000","last_clean":"2024-04-05T22:23:11.072619+0000","last_became_active":"2024-04-05T22:23:08.830890+0000","last_became_peered":"2024-04-05T22:23:08.830890+0000","last_unstale":"2024-04-05T22:23:11.072619+0000","last_undegraded":"2024-04-05T22:23:11.072619+0000","last_fullsized":"2024-04-05T22:23:11.072619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:28:35.503122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487316+0000","last_change":"2024-04-05T22:23:09.361959+0000","last_active":"2024-04-05T22:23:10.487316+0000","last_peered":"2024-04-05T22:23:10.487316+0000","last_clean":"2024-04-05T22:23:10.487316+0000","last_became_active":"2024-04-05T22:23:09.357075+0000","last_became_peered":"2024-04-05T22:23:09.357075+0000","last_unstale":"2024-04-05T22:23:10.487316+0000","last_undegraded":"2024-04-05T22:23:10.487316+0000","last_fullsized":"2024-04-05T22:23:10.487316+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346877+0000","last_change":"2024-04-05T22:23:08.838081+0000","last_active":"2024-04-05T22:23:10.346877+0000","last_peered":"2024-04-05T22:23:10.346877+0000","last_clean":"2024-04-05T22:23:10.346877+0000","last_became_active":"2024-04-05T22:23:08.837732+0000","last_became_peered":"2024-04-05T22:23:08.837732+0000","last_unstale":"2024-04-05T22:23:10.346877+0000","last_undegraded":"2024-04-05T22:23:10.346877+0000","last_fullsized":"2024-04-05T22:23:10.346877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071714+0000","last_change":"2024-04-05T22:23:09.374296+0000","last_active":"2024-04-05T22:23:11.071714+0000","last_peered":"2024-04-05T22:23:11.071714+0000","last_clean":"2024-04-05T22:23:11.071714+0000","last_became_active":"2024-04-05T22:23:09.374031+0000","last_became_peered":"2024-04-05T22:23:09.374031+0000","last_unstale":"2024-04-05T22:23:11.071714+0000","last_undegraded":"2024-04-05T22:23:11.071714+0000","last_fullsized":"2024-04-05T22:23:11.071714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:54.024108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072320+0000","last_change":"2024-04-05T22:23:08.831395+0000","last_active":"2024-04-05T22:23:11.072320+0000","last_peered":"2024-04-05T22:23:11.072320+0000","last_clean":"2024-04-05T22:23:11.072320+0000","last_became_active":"2024-04-05T22:23:08.831124+0000","last_became_peered":"2024-04-05T22:23:08.831124+0000","last_unstale":"2024-04-05T22:23:11.072320+0000","last_undegraded":"2024-04-05T22:23:11.072320+0000","last_fullsized":"2024-04-05T22:23:11.072320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073173+0000","last_change":"2024-04-05T22:23:09.375678+0000","last_active":"2024-04-05T22:23:11.073173+0000","last_peered":"2024-04-05T22:23:11.073173+0000","last_clean":"2024-04-05T22:23:11.073173+0000","last_became_active":"2024-04-05T22:23:09.375368+0000","last_became_peered":"2024-04-05T22:23:09.375368+0000","last_unstale":"2024-04-05T22:23:11.073173+0000","last_undegraded":"2024-04-05T22:23:11.073173+0000","last_fullsized":"2024-04-05T22:23:11.073173+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487546+0000","last_change":"2024-04-05T22:23:08.836443+0000","last_active":"2024-04-05T22:23:10.487546+0000","last_peered":"2024-04-05T22:23:10.487546+0000","last_clean":"2024-04-05T22:23:10.487546+0000","last_became_active":"2024-04-05T22:23:08.836065+0000","last_became_peered":"2024-04-05T22:23:08.836065+0000","last_unstale":"2024-04-05T22:23:10.487546+0000","last_undegraded":"2024-04-05T22:23:10.487546+0000","last_fullsized":"2024-04-05T22:23:10.487546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:07.607575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346513+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:23:10.346513+0000","last_peered":"2024-04-05T22:23:10.346513+0000","last_clean":"2024-04-05T22:23:10.346513+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:23:10.346513+0000","last_undegraded":"2024-04-05T22:23:10.346513+0000","last_fullsized":"2024-04-05T22:23:10.346513+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1159168,"data_stored":818980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.02,"15min":1.018},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.0069999999999999,"15min":0.99299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.5129999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":1.0149999999999999},"min":{"1min":0.77700000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.054},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1879999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.179,"15min":1.173},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.252},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2110000000000001,"15min":1.2010000000000001},"min":{"1min":0.93200000000000005,"5min":0.75,"15min":0.75},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.133,"15min":1.1140000000000001},"min":{"1min":0.95699999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5409999999999999,"5min":1.712,"15min":1.712},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2170000000000001,"15min":1.2150000000000001},"min":{"1min":0.90400000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4810000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.4810000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.232,"15min":1.218},"min":{"1min":1.0309999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6950000000000001,"5min":1.865,"15min":1.865},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.325,"15min":1.323},"min":{"1min":0.98699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.7050000000000001,"5min":1.788,"15min":1.788},"last":1.7050000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.2410000000000001,"15min":1.218},"min":{"1min":0.95999999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.8440000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.772},{"interface":"front","average":{"1min":1.46,"5min":1.3600000000000001,"15min":1.343},"min":{"1min":1.1140000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.802,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.47,"15min":1.448},"min":{"1min":1.3680000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.51,"15min":1.492},"min":{"1min":1.296,"5min":0.755,"15min":0.755},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.8160000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":229194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.131},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.002},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.036},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.579}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.6439999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.698},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.248}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.179}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.444},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.95599999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.605}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.28},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.5620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":573440,"data_stored":230086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.034}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.103},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.198}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.159},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.306},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.083}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.3600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":229160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.435,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4119999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.505},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.90100000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4390000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.27,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.5720000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.999,"15min":1.0249999999999999},"min":{"1min":0.57399999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.1679999999999999,"5min":1.827,"15min":1.827},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.075,"15min":1.083},"min":{"1min":0.67000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6559999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1020000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2310000000000001,"15min":1.2549999999999999},"min":{"1min":0.82299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.496,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.306,"15min":1.3340000000000001},"min":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.573,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.361,"15min":1.391},"min":{"1min":0.87,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4219999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.476,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9390000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.399,"15min":1.4370000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4890000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.0900000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.4590000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7050000000000001,"5min":2.468,"15min":2.468},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.294,"5min":1.4179999999999999,"15min":1.4390000000000001},"min":{"1min":0.89400000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.0089999999999999,"5min":2.798,"15min":2.798},"last":1.1899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":230108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.028},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.258}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.385},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.194},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.486}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.97699999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.113}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.2889999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.3120000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1159168,"data_stored":818924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.179,"5min":1.367,"15min":1.367},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.39500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4590000000000001,"5min":1.669,"15min":1.669},"last":1.1579999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.514,"15min":1.514},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.49399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.6040000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.90900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.506,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0840000000000001,"15min":1.1020000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.202}]},{"osd":4,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1579999999999999,"15min":1.1539999999999999},"min":{"1min":0.747,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5960000000000001,"5min":1.929,"15min":1.929},"last":1.389},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6519999999999999,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.266}]},{"osd":5,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6559999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.042},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.196,"15min":1.1899999999999999},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.526,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.325}]},{"osd":6,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.784,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7110000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.2829999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.339,"15min":1.339},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7509999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.3120000000000001,"15min":1.331},"min":{"1min":0.75,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6619999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.2370000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:23:14.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-05T22:23:14.784 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-04-05T22:23:14.785+0000 7f35b2c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:23:14.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.936+0000 7f7c93600640 1 Processor -- start 2024-04-05T22:23:14.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.936+0000 7f7c93600640 1 -- start start 2024-04-05T22:23:14.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c93600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c8c1587d0 0x7f7c8c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c93600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c06f5b0 con 0x7f7c8c15e850 2024-04-05T22:23:14.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c93600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c070410 con 0x7f7c8c0ccd20 2024-04-05T22:23:14.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c93600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c158bb0 con 0x7f7c8c1587d0 2024-04-05T22:23:14.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c91600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7c8c15e850 0x7f7c8c0708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54270/0 (socket says 172.21.15.73:54270) 2024-04-05T22:23:14.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.938+0000 7f7c91600640 1 -- 172.21.15.73:0/563438497 learned_addr learned my addr 172.21.15.73:0/563438497 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c90c00640 1 --2- 172.21.15.73:0/563438497 >> v2:172.21.15.157:3300/0 conn(0x7f7c8c1587d0 0x7f7c8c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881002879 0 0) 0x7f7c8c06f5b0 con 0x7f7c8c15e850 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c70003520 con 0x7f7c8c15e850 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252182390 0 0) 0x7f7c70003520 con 0x7f7c8c15e850 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 >> v2:172.21.15.157:3300/0 conn(0x7f7c8c1587d0 msgr2=0x7f7c8c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 --2- 172.21.15.73:0/563438497 >> v2:172.21.15.157:3300/0 conn(0x7f7c8c1587d0 0x7f7c8c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 >> v1:172.21.15.157:6789/0 conn(0x7f7c8c0ccd20 legacy=0x7f7c8c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c06f800 con 0x7f7c8c15e850 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7c840019f0 con 0x7f7c8c15e850 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 >> v1:172.21.15.73:6789/0 conn(0x7f7c8c15e850 legacy=0x7f7c8c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.939+0000 7f7c8be00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c8be00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c700047c0 0x7f7c70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c8be00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c70005180 0x7f7c7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c158bb0 con 0x7f7c70003970 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7000c020 con 0x7f7c700047c0 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7000c1e0 con 0x7f7c70005180 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 -- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c70005180 msgr2=0x7f7c7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:14.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c70005180 0x7f7c7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c83e00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c700047c0 0x7f7c70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 -- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c700047c0 msgr2=0x7f7c70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c700047c0 0x7f7c70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c90c00640 1 -- 172.21.15.73:0/563438497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70003520 con 0x7f7c70003970 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c91600640 1 -- 172.21.15.73:0/563438497 reap_dead start 2024-04-05T22:23:14.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.940+0000 7f7c83e00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c700047c0 0x7f7c70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:14.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.941+0000 7f7c90c00640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c70003d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009ac0 tx=0x7f7c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee306deba269865 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.941+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c02c6b0 con 0x7f7c70003970 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.941+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c02c810 con 0x7f7c70003970 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.941+0000 7f7c8be00640 1 -- 172.21.15.73:0/563438497 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c02cba0 con 0x7f7c70003970 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 -- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 msgr2=0x7f7c70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c70003d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009ac0 tx=0x7f7c700057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 -- 172.21.15.73:0/563438497 shutdown_connections 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 --2- 172.21.15.73:0/563438497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c70003d90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 -- 172.21.15.73:0/563438497 >> 172.21.15.73:0/563438497 conn(0x7f7c8c0ce8e0 msgr2=0x7f7c8c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 -- 172.21.15.73:0/563438497 shutdown_connections 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.942+0000 7f7c93600640 1 -- 172.21.15.73:0/563438497 wait complete. 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.944+0000 7f7c93600640 1 Processor -- start 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.944+0000 7f7c93600640 1 -- start start 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c8c1f3620 0x7f7c8c1efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c8c1f0390 0x7f7c8c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c14df60 con 0x7f7c70003970 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1f1180 con 0x7f7c8c1f3620 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c93600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1f0850 con 0x7f7c8c1f0390 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52222/0 (socket says 172.21.15.73:52222) 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 -- 172.21.15.73:0/1088166565 learned_addr learned my addr 172.21.15.73:0/1088166565 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 -- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c8c1f0390 msgr2=0x7f7c8c156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c8c1f0390 0x7f7c8c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 -- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c8c1f3620 msgr2=0x7f7c8c1efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c8c1f3620 0x7f7c8c1efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.945+0000 7f7c90c00640 1 -- 172.21.15.73:0/1088166565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c1f0b30 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c90c00640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02c250 tx=0x7f7c7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c003810 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c003970 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c003d00 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c1f0e10 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.946+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c0ad570 con 0x7f7c70003970 2024-04-05T22:23:14.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.947+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c066af0 con 0x7f7c70003970 2024-04-05T22:23:14.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.948+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f7c7c034440 con 0x7f7c70003970 2024-04-05T22:23:14.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.949+0000 7f7c89600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 0x7f7c6806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:14.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.949+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c7c0aba70 con 0x7f7c70003970 2024-04-05T22:23:14.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.949+0000 7f7c83e00640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 0x7f7c6806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:14.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.950+0000 7f7c83e00640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 0x7f7c6806d270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007fd0 tx=0x7f7c74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:14.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:14.953+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7c7c076b00 con 0x7f7c70003970 2024-04-05T22:23:15.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.202+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7c8c06a770 con 0x7f7c70003970 2024-04-05T22:23:15.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.203+0000 7f7c89600640 1 -- 172.21.15.73:0/1088166565 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1431 (secure 0 0 0) 0x7f7c7c04c5b0 con 0x7f7c70003970 2024-04-05T22:23:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:15.204 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"38861c87-67cf-4180-a71f-97401becb77d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1712355667,"num_in_osds":8,"osd_in_since":1712355659,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":248094720,"bytes_avail":772846018560,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5634}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-05T22:22:56.520747+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-05T22:23:15.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.205+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 msgr2=0x7f7c6806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 0x7f7c6806d270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007fd0 tx=0x7f7c74009040 comp rx=0 tx=0).stop 2024-04-05T22:23:15.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 msgr2=0x7f7c8c0ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02c250 tx=0x7f7c7c004730 comp rx=0 tx=0).stop 2024-04-05T22:23:15.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 shutdown_connections 2024-04-05T22:23:15.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7c6806adb0 0x7f7c6806d270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c8c1f0390 0x7f7c8c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c8c1f3620 0x7f7c8c1efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 --2- 172.21.15.73:0/1088166565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70003970 0x7f7c8c0ace50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 >> 172.21.15.73:0/1088166565 conn(0x7f7c8c0ce8e0 msgr2=0x7f7c8c0cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:15.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 shutdown_connections 2024-04-05T22:23:15.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.206+0000 7f7c93600640 1 -- 172.21.15.73:0/1088166565 wait complete. 2024-04-05T22:23:15.225 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-05T22:23:15.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:23:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.422+0000 7f26ee000640 1 Processor -- start 2024-04-05T22:23:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.423+0000 7f26ee000640 1 -- start start 2024-04-05T22:23:15.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.424+0000 7f26ee000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26e80b20a0 0x7f26e8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.424+0000 7f26ee000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 0x7f26e806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.424+0000 7f26ee000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e806d3e0 con 0x7f26e80b2480 2024-04-05T22:23:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.424+0000 7f26ee000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e806d610 con 0x7f26e80b20a0 2024-04-05T22:23:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.424+0000 7f26ee000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e8070230 con 0x7f26e80b1b50 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26e80b20a0 0x7f26e8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26e80b20a0 0x7f26e8070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52226/0 (socket says 172.21.15.73:52226) 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 -- 172.21.15.73:0/3452611426 learned_addr learned my addr 172.21.15.73:0/3452611426 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26dfe00640 1 --2- 172.21.15.73:0/3452611426 >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 0x7f26e806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 -- 172.21.15.73:0/3452611426 >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 msgr2=0x7f26e806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 --2- 172.21.15.73:0/3452611426 >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 0x7f26e806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 -- 172.21.15.73:0/3452611426 >> v1:172.21.15.157:6789/0 conn(0x7f26e80b2480 legacy=0x7f26e80b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26ecc00640 1 -- 172.21.15.73:0/3452611426 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8070510 con 0x7f26e80b20a0 2024-04-05T22:23:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.425+0000 7f26dfe00640 1 --2- 172.21.15.73:0/3452611426 >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 0x7f26e806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.426+0000 7f26ecc00640 1 --2- 172.21.15.73:0/3452611426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b20a0 0x7f26e8070ff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f26e806f380 tx=0x7f26d002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d8f56dc24e91cf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26e7e00640 1 -- 172.21.15.73:0/3452611426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0031750 con 0x7f26e80b20a0 2024-04-05T22:23:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26e7e00640 1 -- 172.21.15.73:0/3452611426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d00318b0 con 0x7f26e80b20a0 2024-04-05T22:23:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26e7e00640 1 -- 172.21.15.73:0/3452611426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0031c40 con 0x7f26e80b20a0 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 -- 172.21.15.73:0/3452611426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b20a0 msgr2=0x7f26e8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 --2- 172.21.15.73:0/3452611426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b20a0 0x7f26e8070ff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f26e806f380 tx=0x7f26d002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 -- 172.21.15.73:0/3452611426 shutdown_connections 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 --2- 172.21.15.73:0/3452611426 >> v2:172.21.15.157:3300/0 conn(0x7f26e80b1b50 0x7f26e806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 --2- 172.21.15.73:0/3452611426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b20a0 0x7f26e8070ff0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.427+0000 7f26ee000640 1 -- 172.21.15.73:0/3452611426 >> 172.21.15.73:0/3452611426 conn(0x7f26e80b06c0 msgr2=0x7f26e80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.428+0000 7f26ee000640 1 -- 172.21.15.73:0/3452611426 shutdown_connections 2024-04-05T22:23:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.428+0000 7f26ee000640 1 -- 172.21.15.73:0/3452611426 wait complete. 2024-04-05T22:23:15.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.430+0000 7f26ee000640 1 Processor -- start 2024-04-05T22:23:15.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.430+0000 7f26ee000640 1 -- start start 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e80b20a0 0x7f26e80b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e80b2480 0x7f26e814a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e814eb30 con 0x7f26e80b1b50 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8151ca0 con 0x7f26e80b20a0 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ee000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e814bb90 con 0x7f26e80b2480 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ecc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ecc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52236/0 (socket says 172.21.15.73:52236) 2024-04-05T22:23:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26ecc00640 1 -- 172.21.15.73:0/1862508200 learned_addr learned my addr 172.21.15.73:0/1862508200 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.431+0000 7f26dfe00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e80b20a0 0x7f26e80b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ed600640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e80b2480 0x7f26e814a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 -- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e80b2480 msgr2=0x7f26e814a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e80b2480 0x7f26e814a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 -- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e80b20a0 msgr2=0x7f26e80b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e80b20a0 0x7f26e80b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 -- 172.21.15.73:0/1862508200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00094d0 con 0x7f26e80b1b50 2024-04-05T22:23:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ecc00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0033c80 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0033de0 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d003c620 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e814e570 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.432+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e814b840 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.434+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e804e160 con 0x7f26e80b1b50 2024-04-05T22:23:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.434+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f26d003a040 con 0x7f26e80b1b50 2024-04-05T22:23:15.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.435+0000 7f26e5600640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 0x7f26c406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.435+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26d00ab520 con 0x7f26e80b1b50 2024-04-05T22:23:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.436+0000 7f26dfe00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 0x7f26c406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.436+0000 7f26dfe00640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 0x7f26c406d270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f26e815c290 tx=0x7f26d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:15.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.440+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f26d0076480 con 0x7f26e80b1b50 2024-04-05T22:23:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.633+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26e8070ad0 con 0x7f26c406adb0 2024-04-05T22:23:15.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.656+0000 7f26e5600640 1 -- 172.21.15.73:0/1862508200 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287170 (secure 0 0 0) 0x7f26d800c050 con 0x7f26c406adb0 2024-04-05T22:23:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:15.662 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 msgr2=0x7f26c406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 0x7f26c406d270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f26e815c290 tx=0x7f26d8008040 comp rx=0 tx=0).stop 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 msgr2=0x7f26e8151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d00046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 shutdown_connections 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26c406adb0 0x7f26c406d270 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e80b2480 0x7f26e814a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e80b20a0 0x7f26e80b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 --2- 172.21.15.73:0/1862508200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e80b1b50 0x7f26e8151760 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:15.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 >> 172.21.15.73:0/1862508200 conn(0x7f26e80b06c0 msgr2=0x7f26e8070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:15.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 shutdown_connections 2024-04-05T22:23:15.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:15.662+0000 7f26ee000640 1 -- 172.21.15.73:0/1862508200 wait complete. 2024-04-05T22:23:15.681 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-04-05T22:23:14.526223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242280,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754732440,"statfs":{"total":773094113280,"available":772846018560,"internally_reserved":0,"allocated":5742592,"data_stored":3013636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":242208315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001842"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347585+0000","last_change":"2024-04-05T22:23:09.381025+0000","last_active":"2024-04-05T22:23:10.347585+0000","last_peered":"2024-04-05T22:23:10.347585+0000","last_clean":"2024-04-05T22:23:10.347585+0000","last_became_active":"2024-04-05T22:23:09.379621+0000","last_became_peered":"2024-04-05T22:23:09.379621+0000","last_unstale":"2024-04-05T22:23:10.347585+0000","last_undegraded":"2024-04-05T22:23:10.347585+0000","last_fullsized":"2024-04-05T22:23:10.347585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487095+0000","last_change":"2024-04-05T22:23:09.358874+0000","last_active":"2024-04-05T22:23:10.487095+0000","last_peered":"2024-04-05T22:23:10.487095+0000","last_clean":"2024-04-05T22:23:10.487095+0000","last_became_active":"2024-04-05T22:23:09.358109+0000","last_became_peered":"2024-04-05T22:23:09.358109+0000","last_unstale":"2024-04-05T22:23:10.487095+0000","last_undegraded":"2024-04-05T22:23:10.487095+0000","last_fullsized":"2024-04-05T22:23:10.487095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073685+0000","last_change":"2024-04-05T22:23:09.365855+0000","last_active":"2024-04-05T22:23:11.073685+0000","last_peered":"2024-04-05T22:23:11.073685+0000","last_clean":"2024-04-05T22:23:11.073685+0000","last_became_active":"2024-04-05T22:23:09.365109+0000","last_became_peered":"2024-04-05T22:23:09.365109+0000","last_unstale":"2024-04-05T22:23:11.073685+0000","last_undegraded":"2024-04-05T22:23:11.073685+0000","last_fullsized":"2024-04-05T22:23:11.073685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:51.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072301+0000","last_change":"2024-04-05T22:23:09.360571+0000","last_active":"2024-04-05T22:23:11.072301+0000","last_peered":"2024-04-05T22:23:11.072301+0000","last_clean":"2024-04-05T22:23:11.072301+0000","last_became_active":"2024-04-05T22:23:09.360249+0000","last_became_peered":"2024-04-05T22:23:09.360249+0000","last_unstale":"2024-04-05T22:23:11.072301+0000","last_undegraded":"2024-04-05T22:23:11.072301+0000","last_fullsized":"2024-04-05T22:23:11.072301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:02.342545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486869+0000","last_change":"2024-04-05T22:23:09.363323+0000","last_active":"2024-04-05T22:23:10.486869+0000","last_peered":"2024-04-05T22:23:10.486869+0000","last_clean":"2024-04-05T22:23:10.486869+0000","last_became_active":"2024-04-05T22:23:09.363021+0000","last_became_peered":"2024-04-05T22:23:09.363021+0000","last_unstale":"2024-04-05T22:23:10.486869+0000","last_undegraded":"2024-04-05T22:23:10.486869+0000","last_fullsized":"2024-04-05T22:23:10.486869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349441+0000","last_change":"2024-04-05T22:23:09.376486+0000","last_active":"2024-04-05T22:23:10.349441+0000","last_peered":"2024-04-05T22:23:10.349441+0000","last_clean":"2024-04-05T22:23:10.349441+0000","last_became_active":"2024-04-05T22:23:09.376214+0000","last_became_peered":"2024-04-05T22:23:09.376214+0000","last_unstale":"2024-04-05T22:23:10.349441+0000","last_undegraded":"2024-04-05T22:23:10.349441+0000","last_fullsized":"2024-04-05T22:23:10.349441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:56:38.205365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347409+0000","last_change":"2024-04-05T22:23:09.374598+0000","last_active":"2024-04-05T22:23:10.347409+0000","last_peered":"2024-04-05T22:23:10.347409+0000","last_clean":"2024-04-05T22:23:10.347409+0000","last_became_active":"2024-04-05T22:23:09.374273+0000","last_became_peered":"2024-04-05T22:23:09.374273+0000","last_unstale":"2024-04-05T22:23:10.347409+0000","last_undegraded":"2024-04-05T22:23:10.347409+0000","last_fullsized":"2024-04-05T22:23:10.347409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071571+0000","last_change":"2024-04-05T22:23:09.362559+0000","last_active":"2024-04-05T22:23:11.071571+0000","last_peered":"2024-04-05T22:23:11.071571+0000","last_clean":"2024-04-05T22:23:11.071571+0000","last_became_active":"2024-04-05T22:23:09.362193+0000","last_became_peered":"2024-04-05T22:23:09.362193+0000","last_unstale":"2024-04-05T22:23:11.071571+0000","last_undegraded":"2024-04-05T22:23:11.071571+0000","last_fullsized":"2024-04-05T22:23:11.071571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:17.066127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072891+0000","last_change":"2024-04-05T22:23:09.381934+0000","last_active":"2024-04-05T22:23:11.072891+0000","last_peered":"2024-04-05T22:23:11.072891+0000","last_clean":"2024-04-05T22:23:11.072891+0000","last_became_active":"2024-04-05T22:23:09.380359+0000","last_became_peered":"2024-04-05T22:23:09.380359+0000","last_unstale":"2024-04-05T22:23:11.072891+0000","last_undegraded":"2024-04-05T22:23:11.072891+0000","last_fullsized":"2024-04-05T22:23:11.072891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073130+0000","last_change":"2024-04-05T22:23:09.360338+0000","last_active":"2024-04-05T22:23:11.073130+0000","last_peered":"2024-04-05T22:23:11.073130+0000","last_clean":"2024-04-05T22:23:11.073130+0000","last_became_active":"2024-04-05T22:23:09.360037+0000","last_became_peered":"2024-04-05T22:23:09.360037+0000","last_unstale":"2024-04-05T22:23:11.073130+0000","last_undegraded":"2024-04-05T22:23:11.073130+0000","last_fullsized":"2024-04-05T22:23:11.073130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:00.708491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071876+0000","last_change":"2024-04-05T22:23:09.363867+0000","last_active":"2024-04-05T22:23:11.071876+0000","last_peered":"2024-04-05T22:23:11.071876+0000","last_clean":"2024-04-05T22:23:11.071876+0000","last_became_active":"2024-04-05T22:23:09.363604+0000","last_became_peered":"2024-04-05T22:23:09.363604+0000","last_unstale":"2024-04-05T22:23:11.071876+0000","last_undegraded":"2024-04-05T22:23:11.071876+0000","last_fullsized":"2024-04-05T22:23:11.071876+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487272+0000","last_change":"2024-04-05T22:23:09.361574+0000","last_active":"2024-04-05T22:23:10.487272+0000","last_peered":"2024-04-05T22:23:10.487272+0000","last_clean":"2024-04-05T22:23:10.487272+0000","last_became_active":"2024-04-05T22:23:09.361302+0000","last_became_peered":"2024-04-05T22:23:09.361302+0000","last_unstale":"2024-04-05T22:23:10.487272+0000","last_undegraded":"2024-04-05T22:23:10.487272+0000","last_fullsized":"2024-04-05T22:23:10.487272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070969+0000","last_change":"2024-04-05T22:23:09.374114+0000","last_active":"2024-04-05T22:23:11.070969+0000","last_peered":"2024-04-05T22:23:11.070969+0000","last_clean":"2024-04-05T22:23:11.070969+0000","last_became_active":"2024-04-05T22:23:09.373743+0000","last_became_peered":"2024-04-05T22:23:09.373743+0000","last_unstale":"2024-04-05T22:23:11.070969+0000","last_undegraded":"2024-04-05T22:23:11.070969+0000","last_fullsized":"2024-04-05T22:23:11.070969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:25.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072456+0000","last_change":"2024-04-05T22:23:09.360964+0000","last_active":"2024-04-05T22:23:11.072456+0000","last_peered":"2024-04-05T22:23:11.072456+0000","last_clean":"2024-04-05T22:23:11.072456+0000","last_became_active":"2024-04-05T22:23:09.360707+0000","last_became_peered":"2024-04-05T22:23:09.360707+0000","last_unstale":"2024-04-05T22:23:11.072456+0000","last_undegraded":"2024-04-05T22:23:11.072456+0000","last_fullsized":"2024-04-05T22:23:11.072456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349236+0000","last_change":"2024-04-05T22:23:09.374250+0000","last_active":"2024-04-05T22:23:10.349236+0000","last_peered":"2024-04-05T22:23:10.349236+0000","last_clean":"2024-04-05T22:23:10.349236+0000","last_became_active":"2024-04-05T22:23:09.373960+0000","last_became_peered":"2024-04-05T22:23:09.373960+0000","last_unstale":"2024-04-05T22:23:10.349236+0000","last_undegraded":"2024-04-05T22:23:10.349236+0000","last_fullsized":"2024-04-05T22:23:10.349236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:59.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071101+0000","last_change":"2024-04-05T22:23:09.359204+0000","last_active":"2024-04-05T22:23:11.071101+0000","last_peered":"2024-04-05T22:23:11.071101+0000","last_clean":"2024-04-05T22:23:11.071101+0000","last_became_active":"2024-04-05T22:23:09.358942+0000","last_became_peered":"2024-04-05T22:23:09.358942+0000","last_unstale":"2024-04-05T22:23:11.071101+0000","last_undegraded":"2024-04-05T22:23:11.071101+0000","last_fullsized":"2024-04-05T22:23:11.071101+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:02.659494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072261+0000","last_change":"2024-04-05T22:23:09.376368+0000","last_active":"2024-04-05T22:23:11.072261+0000","last_peered":"2024-04-05T22:23:11.072261+0000","last_clean":"2024-04-05T22:23:11.072261+0000","last_became_active":"2024-04-05T22:23:09.376102+0000","last_became_peered":"2024-04-05T22:23:09.376102+0000","last_unstale":"2024-04-05T22:23:11.072261+0000","last_undegraded":"2024-04-05T22:23:11.072261+0000","last_fullsized":"2024-04-05T22:23:11.072261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:33.661018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073167+0000","last_change":"2024-04-05T22:23:09.377480+0000","last_active":"2024-04-05T22:23:11.073167+0000","last_peered":"2024-04-05T22:23:11.073167+0000","last_clean":"2024-04-05T22:23:11.073167+0000","last_became_active":"2024-04-05T22:23:09.377201+0000","last_became_peered":"2024-04-05T22:23:09.377201+0000","last_unstale":"2024-04-05T22:23:11.073167+0000","last_undegraded":"2024-04-05T22:23:11.073167+0000","last_fullsized":"2024-04-05T22:23:11.073167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:09.415421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072444+0000","last_change":"2024-04-05T22:23:09.361412+0000","last_active":"2024-04-05T22:23:11.072444+0000","last_peered":"2024-04-05T22:23:11.072444+0000","last_clean":"2024-04-05T22:23:11.072444+0000","last_became_active":"2024-04-05T22:23:09.361098+0000","last_became_peered":"2024-04-05T22:23:09.361098+0000","last_unstale":"2024-04-05T22:23:11.072444+0000","last_undegraded":"2024-04-05T22:23:11.072444+0000","last_fullsized":"2024-04-05T22:23:11.072444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:40.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071327+0000","last_change":"2024-04-05T22:23:09.355237+0000","last_active":"2024-04-05T22:23:11.071327+0000","last_peered":"2024-04-05T22:23:11.071327+0000","last_clean":"2024-04-05T22:23:11.071327+0000","last_became_active":"2024-04-05T22:23:09.354907+0000","last_became_peered":"2024-04-05T22:23:09.354907+0000","last_unstale":"2024-04-05T22:23:11.071327+0000","last_undegraded":"2024-04-05T22:23:11.071327+0000","last_fullsized":"2024-04-05T22:23:11.071327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:11:07.814340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350119+0000","last_change":"2024-04-05T22:23:09.373051+0000","last_active":"2024-04-05T22:23:10.350119+0000","last_peered":"2024-04-05T22:23:10.350119+0000","last_clean":"2024-04-05T22:23:10.350119+0000","last_became_active":"2024-04-05T22:23:09.372014+0000","last_became_peered":"2024-04-05T22:23:09.372014+0000","last_unstale":"2024-04-05T22:23:10.350119+0000","last_undegraded":"2024-04-05T22:23:10.350119+0000","last_fullsized":"2024-04-05T22:23:10.350119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:58.535697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072702+0000","last_change":"2024-04-05T22:23:09.360582+0000","last_active":"2024-04-05T22:23:11.072702+0000","last_peered":"2024-04-05T22:23:11.072702+0000","last_clean":"2024-04-05T22:23:11.072702+0000","last_became_active":"2024-04-05T22:23:09.360346+0000","last_became_peered":"2024-04-05T22:23:09.360346+0000","last_unstale":"2024-04-05T22:23:11.072702+0000","last_undegraded":"2024-04-05T22:23:11.072702+0000","last_fullsized":"2024-04-05T22:23:11.072702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071233+0000","last_change":"2024-04-05T22:23:09.365234+0000","last_active":"2024-04-05T22:23:11.071233+0000","last_peered":"2024-04-05T22:23:11.071233+0000","last_clean":"2024-04-05T22:23:11.071233+0000","last_became_active":"2024-04-05T22:23:09.364930+0000","last_became_peered":"2024-04-05T22:23:09.364930+0000","last_unstale":"2024-04-05T22:23:11.071233+0000","last_undegraded":"2024-04-05T22:23:11.071233+0000","last_fullsized":"2024-04-05T22:23:11.071233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:33:24.735249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071336+0000","last_change":"2024-04-05T22:23:09.367889+0000","last_active":"2024-04-05T22:23:11.071336+0000","last_peered":"2024-04-05T22:23:11.071336+0000","last_clean":"2024-04-05T22:23:11.071336+0000","last_became_active":"2024-04-05T22:23:09.367652+0000","last_became_peered":"2024-04-05T22:23:09.367652+0000","last_unstale":"2024-04-05T22:23:11.071336+0000","last_undegraded":"2024-04-05T22:23:11.071336+0000","last_fullsized":"2024-04-05T22:23:11.071336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:03.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073053+0000","last_change":"2024-04-05T22:23:09.365048+0000","last_active":"2024-04-05T22:23:11.073053+0000","last_peered":"2024-04-05T22:23:11.073053+0000","last_clean":"2024-04-05T22:23:11.073053+0000","last_became_active":"2024-04-05T22:23:09.364707+0000","last_became_peered":"2024-04-05T22:23:09.364707+0000","last_unstale":"2024-04-05T22:23:11.073053+0000","last_undegraded":"2024-04-05T22:23:11.073053+0000","last_fullsized":"2024-04-05T22:23:11.073053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:43.155641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071908+0000","last_change":"2024-04-05T22:23:09.375125+0000","last_active":"2024-04-05T22:23:11.071908+0000","last_peered":"2024-04-05T22:23:11.071908+0000","last_clean":"2024-04-05T22:23:11.071908+0000","last_became_active":"2024-04-05T22:23:09.373382+0000","last_became_peered":"2024-04-05T22:23:09.373382+0000","last_unstale":"2024-04-05T22:23:11.071908+0000","last_undegraded":"2024-04-05T22:23:11.071908+0000","last_fullsized":"2024-04-05T22:23:11.071908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:53:45.868945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073020+0000","last_change":"2024-04-05T22:23:09.376021+0000","last_active":"2024-04-05T22:23:11.073020+0000","last_peered":"2024-04-05T22:23:11.073020+0000","last_clean":"2024-04-05T22:23:11.073020+0000","last_became_active":"2024-04-05T22:23:09.375702+0000","last_became_peered":"2024-04-05T22:23:09.375702+0000","last_unstale":"2024-04-05T22:23:11.073020+0000","last_undegraded":"2024-04-05T22:23:11.073020+0000","last_fullsized":"2024-04-05T22:23:11.073020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:55.351589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487322+0000","last_change":"2024-04-05T22:23:09.356705+0000","last_active":"2024-04-05T22:23:10.487322+0000","last_peered":"2024-04-05T22:23:10.487322+0000","last_clean":"2024-04-05T22:23:10.487322+0000","last_became_active":"2024-04-05T22:23:09.356429+0000","last_became_peered":"2024-04-05T22:23:09.356429+0000","last_unstale":"2024-04-05T22:23:10.487322+0000","last_undegraded":"2024-04-05T22:23:10.487322+0000","last_fullsized":"2024-04-05T22:23:10.487322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071142+0000","last_change":"2024-04-05T22:23:09.364089+0000","last_active":"2024-04-05T22:23:11.071142+0000","last_peered":"2024-04-05T22:23:11.071142+0000","last_clean":"2024-04-05T22:23:11.071142+0000","last_became_active":"2024-04-05T22:23:09.363793+0000","last_became_peered":"2024-04-05T22:23:09.363793+0000","last_unstale":"2024-04-05T22:23:11.071142+0000","last_undegraded":"2024-04-05T22:23:11.071142+0000","last_fullsized":"2024-04-05T22:23:11.071142+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:18:38.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071969+0000","last_change":"2024-04-05T22:23:09.365957+0000","last_active":"2024-04-05T22:23:11.071969+0000","last_peered":"2024-04-05T22:23:11.071969+0000","last_clean":"2024-04-05T22:23:11.071969+0000","last_became_active":"2024-04-05T22:23:09.365640+0000","last_became_peered":"2024-04-05T22:23:09.365640+0000","last_unstale":"2024-04-05T22:23:11.071969+0000","last_undegraded":"2024-04-05T22:23:11.071969+0000","last_fullsized":"2024-04-05T22:23:11.071969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:48.736221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349506+0000","last_change":"2024-04-05T22:23:09.377291+0000","last_active":"2024-04-05T22:23:10.349506+0000","last_peered":"2024-04-05T22:23:10.349506+0000","last_clean":"2024-04-05T22:23:10.349506+0000","last_became_active":"2024-04-05T22:23:09.377044+0000","last_became_peered":"2024-04-05T22:23:09.377044+0000","last_unstale":"2024-04-05T22:23:10.349506+0000","last_undegraded":"2024-04-05T22:23:10.349506+0000","last_fullsized":"2024-04-05T22:23:10.349506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:13.822332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070568+0000","last_change":"2024-04-05T22:23:09.348727+0000","last_active":"2024-04-05T22:23:11.070568+0000","last_peered":"2024-04-05T22:23:11.070568+0000","last_clean":"2024-04-05T22:23:11.070568+0000","last_became_active":"2024-04-05T22:23:09.348346+0000","last_became_peered":"2024-04-05T22:23:09.348346+0000","last_unstale":"2024-04-05T22:23:11.070568+0000","last_undegraded":"2024-04-05T22:23:11.070568+0000","last_fullsized":"2024-04-05T22:23:11.070568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:18.309908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347754+0000","last_change":"2024-04-05T22:23:09.379081+0000","last_active":"2024-04-05T22:23:10.347754+0000","last_peered":"2024-04-05T22:23:10.347754+0000","last_clean":"2024-04-05T22:23:10.347754+0000","last_became_active":"2024-04-05T22:23:09.378754+0000","last_became_peered":"2024-04-05T22:23:09.378754+0000","last_unstale":"2024-04-05T22:23:10.347754+0000","last_undegraded":"2024-04-05T22:23:10.347754+0000","last_fullsized":"2024-04-05T22:23:10.347754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:07.603359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350430+0000","last_change":"2024-04-05T22:23:08.841250+0000","last_active":"2024-04-05T22:23:10.350430+0000","last_peered":"2024-04-05T22:23:10.350430+0000","last_clean":"2024-04-05T22:23:10.350430+0000","last_became_active":"2024-04-05T22:23:08.841032+0000","last_became_peered":"2024-04-05T22:23:08.841032+0000","last_unstale":"2024-04-05T22:23:10.350430+0000","last_undegraded":"2024-04-05T22:23:10.350430+0000","last_fullsized":"2024-04-05T22:23:10.350430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:06.598376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073172+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.073172+0000","last_peered":"2024-04-05T22:23:11.073172+0000","last_clean":"2024-04-05T22:23:11.073172+0000","last_became_active":"2024-04-05T22:23:09.371747+0000","last_became_peered":"2024-04-05T22:23:09.371747+0000","last_unstale":"2024-04-05T22:23:11.073172+0000","last_undegraded":"2024-04-05T22:23:11.073172+0000","last_fullsized":"2024-04-05T22:23:11.073172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:48:19.797618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073337+0000","last_change":"2024-04-05T22:23:08.839262+0000","last_active":"2024-04-05T22:23:11.073337+0000","last_peered":"2024-04-05T22:23:11.073337+0000","last_clean":"2024-04-05T22:23:11.073337+0000","last_became_active":"2024-04-05T22:23:08.838980+0000","last_became_peered":"2024-04-05T22:23:08.838980+0000","last_unstale":"2024-04-05T22:23:11.073337+0000","last_undegraded":"2024-04-05T22:23:11.073337+0000","last_fullsized":"2024-04-05T22:23:11.073337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:24:35.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487575+0000","last_change":"2024-04-05T22:23:09.372679+0000","last_active":"2024-04-05T22:23:10.487575+0000","last_peered":"2024-04-05T22:23:10.487575+0000","last_clean":"2024-04-05T22:23:10.487575+0000","last_became_active":"2024-04-05T22:23:09.372413+0000","last_became_peered":"2024-04-05T22:23:09.372413+0000","last_unstale":"2024-04-05T22:23:10.487575+0000","last_undegraded":"2024-04-05T22:23:10.487575+0000","last_fullsized":"2024-04-05T22:23:10.487575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347122+0000","last_change":"2024-04-05T22:23:08.831526+0000","last_active":"2024-04-05T22:23:10.347122+0000","last_peered":"2024-04-05T22:23:10.347122+0000","last_clean":"2024-04-05T22:23:10.347122+0000","last_became_active":"2024-04-05T22:23:08.831156+0000","last_became_peered":"2024-04-05T22:23:08.831156+0000","last_unstale":"2024-04-05T22:23:10.347122+0000","last_undegraded":"2024-04-05T22:23:10.347122+0000","last_fullsized":"2024-04-05T22:23:10.347122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347002+0000","last_change":"2024-04-05T22:23:09.371601+0000","last_active":"2024-04-05T22:23:10.347002+0000","last_peered":"2024-04-05T22:23:10.347002+0000","last_clean":"2024-04-05T22:23:10.347002+0000","last_became_active":"2024-04-05T22:23:09.371235+0000","last_became_peered":"2024-04-05T22:23:09.371235+0000","last_unstale":"2024-04-05T22:23:10.347002+0000","last_undegraded":"2024-04-05T22:23:10.347002+0000","last_fullsized":"2024-04-05T22:23:10.347002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347223+0000","last_change":"2024-04-05T22:23:08.834593+0000","last_active":"2024-04-05T22:23:10.347223+0000","last_peered":"2024-04-05T22:23:10.347223+0000","last_clean":"2024-04-05T22:23:10.347223+0000","last_became_active":"2024-04-05T22:23:08.833796+0000","last_became_peered":"2024-04-05T22:23:08.833796+0000","last_unstale":"2024-04-05T22:23:10.347223+0000","last_undegraded":"2024-04-05T22:23:10.347223+0000","last_fullsized":"2024-04-05T22:23:10.347223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072191+0000","last_change":"2024-04-05T22:23:09.364226+0000","last_active":"2024-04-05T22:23:11.072191+0000","last_peered":"2024-04-05T22:23:11.072191+0000","last_clean":"2024-04-05T22:23:11.072191+0000","last_became_active":"2024-04-05T22:23:09.363915+0000","last_became_peered":"2024-04-05T22:23:09.363915+0000","last_unstale":"2024-04-05T22:23:11.072191+0000","last_undegraded":"2024-04-05T22:23:11.072191+0000","last_fullsized":"2024-04-05T22:23:11.072191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:41:08.452502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350046+0000","last_change":"2024-04-05T22:23:08.837746+0000","last_active":"2024-04-05T22:23:10.350046+0000","last_peered":"2024-04-05T22:23:10.350046+0000","last_clean":"2024-04-05T22:23:10.350046+0000","last_became_active":"2024-04-05T22:23:08.837489+0000","last_became_peered":"2024-04-05T22:23:08.837489+0000","last_unstale":"2024-04-05T22:23:10.350046+0000","last_undegraded":"2024-04-05T22:23:10.350046+0000","last_fullsized":"2024-04-05T22:23:10.350046+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:34.404596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071930+0000","last_change":"2024-04-05T22:23:09.357300+0000","last_active":"2024-04-05T22:23:11.071930+0000","last_peered":"2024-04-05T22:23:11.071930+0000","last_clean":"2024-04-05T22:23:11.071930+0000","last_became_active":"2024-04-05T22:23:09.357047+0000","last_became_peered":"2024-04-05T22:23:09.357047+0000","last_unstale":"2024-04-05T22:23:11.071930+0000","last_undegraded":"2024-04-05T22:23:11.071930+0000","last_fullsized":"2024-04-05T22:23:11.071930+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072432+0000","last_change":"2024-04-05T22:23:08.837128+0000","last_active":"2024-04-05T22:23:11.072432+0000","last_peered":"2024-04-05T22:23:11.072432+0000","last_clean":"2024-04-05T22:23:11.072432+0000","last_became_active":"2024-04-05T22:23:08.836444+0000","last_became_peered":"2024-04-05T22:23:08.836444+0000","last_unstale":"2024-04-05T22:23:11.072432+0000","last_undegraded":"2024-04-05T22:23:11.072432+0000","last_fullsized":"2024-04-05T22:23:11.072432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:27.879231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071573+0000","last_change":"2024-04-05T22:23:09.362416+0000","last_active":"2024-04-05T22:23:11.071573+0000","last_peered":"2024-04-05T22:23:11.071573+0000","last_clean":"2024-04-05T22:23:11.071573+0000","last_became_active":"2024-04-05T22:23:09.361962+0000","last_became_peered":"2024-04-05T22:23:09.361962+0000","last_unstale":"2024-04-05T22:23:11.071573+0000","last_undegraded":"2024-04-05T22:23:11.071573+0000","last_fullsized":"2024-04-05T22:23:11.071573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:52.825291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486775+0000","last_change":"2024-04-05T22:23:08.831328+0000","last_active":"2024-04-05T22:23:10.486775+0000","last_peered":"2024-04-05T22:23:10.486775+0000","last_clean":"2024-04-05T22:23:10.486775+0000","last_became_active":"2024-04-05T22:23:08.830959+0000","last_became_peered":"2024-04-05T22:23:08.830959+0000","last_unstale":"2024-04-05T22:23:10.486775+0000","last_undegraded":"2024-04-05T22:23:10.486775+0000","last_fullsized":"2024-04-05T22:23:10.486775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070939+0000","last_change":"2024-04-05T22:23:09.365286+0000","last_active":"2024-04-05T22:23:11.070939+0000","last_peered":"2024-04-05T22:23:11.070939+0000","last_clean":"2024-04-05T22:23:11.070939+0000","last_became_active":"2024-04-05T22:23:09.364944+0000","last_became_peered":"2024-04-05T22:23:09.364944+0000","last_unstale":"2024-04-05T22:23:11.070939+0000","last_undegraded":"2024-04-05T22:23:11.070939+0000","last_fullsized":"2024-04-05T22:23:11.070939+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:39:32.120858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071812+0000","last_change":"2024-04-05T22:23:08.833902+0000","last_active":"2024-04-05T22:23:11.071812+0000","last_peered":"2024-04-05T22:23:11.071812+0000","last_clean":"2024-04-05T22:23:11.071812+0000","last_became_active":"2024-04-05T22:23:08.833179+0000","last_became_peered":"2024-04-05T22:23:08.833179+0000","last_unstale":"2024-04-05T22:23:11.071812+0000","last_undegraded":"2024-04-05T22:23:11.071812+0000","last_fullsized":"2024-04-05T22:23:11.071812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:08:00.232153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073368+0000","last_change":"2024-04-05T22:23:09.379153+0000","last_active":"2024-04-05T22:23:11.073368+0000","last_peered":"2024-04-05T22:23:11.073368+0000","last_clean":"2024-04-05T22:23:11.073368+0000","last_became_active":"2024-04-05T22:23:09.378897+0000","last_became_peered":"2024-04-05T22:23:09.378897+0000","last_unstale":"2024-04-05T22:23:11.073368+0000","last_undegraded":"2024-04-05T22:23:11.073368+0000","last_fullsized":"2024-04-05T22:23:11.073368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:08.612559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072706+0000","last_change":"2024-04-05T22:23:08.828406+0000","last_active":"2024-04-05T22:23:11.072706+0000","last_peered":"2024-04-05T22:23:11.072706+0000","last_clean":"2024-04-05T22:23:11.072706+0000","last_became_active":"2024-04-05T22:23:08.828116+0000","last_became_peered":"2024-04-05T22:23:08.828116+0000","last_unstale":"2024-04-05T22:23:11.072706+0000","last_undegraded":"2024-04-05T22:23:11.072706+0000","last_fullsized":"2024-04-05T22:23:11.072706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347731+0000","last_change":"2024-04-05T22:23:09.370788+0000","last_active":"2024-04-05T22:23:10.347731+0000","last_peered":"2024-04-05T22:23:10.347731+0000","last_clean":"2024-04-05T22:23:10.347731+0000","last_became_active":"2024-04-05T22:23:09.370475+0000","last_became_peered":"2024-04-05T22:23:09.370475+0000","last_unstale":"2024-04-05T22:23:10.347731+0000","last_undegraded":"2024-04-05T22:23:10.347731+0000","last_fullsized":"2024-04-05T22:23:10.347731+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073323+0000","last_change":"2024-04-05T22:23:08.833154+0000","last_active":"2024-04-05T22:23:11.073323+0000","last_peered":"2024-04-05T22:23:11.073323+0000","last_clean":"2024-04-05T22:23:11.073323+0000","last_became_active":"2024-04-05T22:23:08.831734+0000","last_became_peered":"2024-04-05T22:23:08.831734+0000","last_unstale":"2024-04-05T22:23:11.073323+0000","last_undegraded":"2024-04-05T22:23:11.073323+0000","last_fullsized":"2024-04-05T22:23:11.073323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:21.532272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349928+0000","last_change":"2024-04-05T22:23:09.368222+0000","last_active":"2024-04-05T22:23:10.349928+0000","last_peered":"2024-04-05T22:23:10.349928+0000","last_clean":"2024-04-05T22:23:10.349928+0000","last_became_active":"2024-04-05T22:23:09.367924+0000","last_became_peered":"2024-04-05T22:23:09.367924+0000","last_unstale":"2024-04-05T22:23:10.349928+0000","last_undegraded":"2024-04-05T22:23:10.349928+0000","last_fullsized":"2024-04-05T22:23:10.349928+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347148+0000","last_change":"2024-04-05T22:23:08.839971+0000","last_active":"2024-04-05T22:23:10.347148+0000","last_peered":"2024-04-05T22:23:10.347148+0000","last_clean":"2024-04-05T22:23:10.347148+0000","last_became_active":"2024-04-05T22:23:08.839707+0000","last_became_peered":"2024-04-05T22:23:08.839707+0000","last_unstale":"2024-04-05T22:23:10.347148+0000","last_undegraded":"2024-04-05T22:23:10.347148+0000","last_fullsized":"2024-04-05T22:23:10.347148+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347367+0000","last_change":"2024-04-05T22:23:09.368953+0000","last_active":"2024-04-05T22:23:10.347367+0000","last_peered":"2024-04-05T22:23:10.347367+0000","last_clean":"2024-04-05T22:23:10.347367+0000","last_became_active":"2024-04-05T22:23:09.367313+0000","last_became_peered":"2024-04-05T22:23:09.367313+0000","last_unstale":"2024-04-05T22:23:10.347367+0000","last_undegraded":"2024-04-05T22:23:10.347367+0000","last_fullsized":"2024-04-05T22:23:10.347367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072503+0000","last_change":"2024-04-05T22:23:08.825046+0000","last_active":"2024-04-05T22:23:11.072503+0000","last_peered":"2024-04-05T22:23:11.072503+0000","last_clean":"2024-04-05T22:23:11.072503+0000","last_became_active":"2024-04-05T22:23:08.824750+0000","last_became_peered":"2024-04-05T22:23:08.824750+0000","last_unstale":"2024-04-05T22:23:11.072503+0000","last_undegraded":"2024-04-05T22:23:11.072503+0000","last_fullsized":"2024-04-05T22:23:11.072503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:07.006045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071685+0000","last_change":"2024-04-05T22:23:09.367072+0000","last_active":"2024-04-05T22:23:11.071685+0000","last_peered":"2024-04-05T22:23:11.071685+0000","last_clean":"2024-04-05T22:23:11.071685+0000","last_became_active":"2024-04-05T22:23:09.366737+0000","last_became_peered":"2024-04-05T22:23:09.366737+0000","last_unstale":"2024-04-05T22:23:11.071685+0000","last_undegraded":"2024-04-05T22:23:11.071685+0000","last_fullsized":"2024-04-05T22:23:11.071685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:13.331331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350234+0000","last_change":"2024-04-05T22:23:08.840066+0000","last_active":"2024-04-05T22:23:10.350234+0000","last_peered":"2024-04-05T22:23:10.350234+0000","last_clean":"2024-04-05T22:23:10.350234+0000","last_became_active":"2024-04-05T22:23:08.839794+0000","last_became_peered":"2024-04-05T22:23:08.839794+0000","last_unstale":"2024-04-05T22:23:10.350234+0000","last_undegraded":"2024-04-05T22:23:10.350234+0000","last_fullsized":"2024-04-05T22:23:10.350234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:39.221256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072213+0000","last_change":"2024-04-05T22:23:09.370990+0000","last_active":"2024-04-05T22:23:11.072213+0000","last_peered":"2024-04-05T22:23:11.072213+0000","last_clean":"2024-04-05T22:23:11.072213+0000","last_became_active":"2024-04-05T22:23:09.370739+0000","last_became_peered":"2024-04-05T22:23:09.370739+0000","last_unstale":"2024-04-05T22:23:11.072213+0000","last_undegraded":"2024-04-05T22:23:11.072213+0000","last_fullsized":"2024-04-05T22:23:11.072213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347073+0000","last_change":"2024-04-05T22:23:08.831972+0000","last_active":"2024-04-05T22:23:10.347073+0000","last_peered":"2024-04-05T22:23:10.347073+0000","last_clean":"2024-04-05T22:23:10.347073+0000","last_became_active":"2024-04-05T22:23:08.831650+0000","last_became_peered":"2024-04-05T22:23:08.831650+0000","last_unstale":"2024-04-05T22:23:10.347073+0000","last_undegraded":"2024-04-05T22:23:10.347073+0000","last_fullsized":"2024-04-05T22:23:10.347073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:52:24.125437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.348862+0000","last_change":"2024-04-05T22:23:09.364012+0000","last_active":"2024-04-05T22:23:10.348862+0000","last_peered":"2024-04-05T22:23:10.348862+0000","last_clean":"2024-04-05T22:23:10.348862+0000","last_became_active":"2024-04-05T22:23:09.363768+0000","last_became_peered":"2024-04-05T22:23:09.363768+0000","last_unstale":"2024-04-05T22:23:10.348862+0000","last_undegraded":"2024-04-05T22:23:10.348862+0000","last_fullsized":"2024-04-05T22:23:10.348862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:30.810750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349521+0000","last_change":"2024-04-05T22:23:08.840058+0000","last_active":"2024-04-05T22:23:10.349521+0000","last_peered":"2024-04-05T22:23:10.349521+0000","last_clean":"2024-04-05T22:23:10.349521+0000","last_became_active":"2024-04-05T22:23:08.839727+0000","last_became_peered":"2024-04-05T22:23:08.839727+0000","last_unstale":"2024-04-05T22:23:10.349521+0000","last_undegraded":"2024-04-05T22:23:10.349521+0000","last_fullsized":"2024-04-05T22:23:10.349521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:56.777874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347343+0000","last_change":"2024-04-05T22:23:09.362618+0000","last_active":"2024-04-05T22:23:10.347343+0000","last_peered":"2024-04-05T22:23:10.347343+0000","last_clean":"2024-04-05T22:23:10.347343+0000","last_became_active":"2024-04-05T22:23:09.362358+0000","last_became_peered":"2024-04-05T22:23:09.362358+0000","last_unstale":"2024-04-05T22:23:10.347343+0000","last_undegraded":"2024-04-05T22:23:10.347343+0000","last_fullsized":"2024-04-05T22:23:10.347343+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072045+0000","last_change":"2024-04-05T22:23:08.839690+0000","last_active":"2024-04-05T22:23:11.072045+0000","last_peered":"2024-04-05T22:23:11.072045+0000","last_clean":"2024-04-05T22:23:11.072045+0000","last_became_active":"2024-04-05T22:23:08.838945+0000","last_became_peered":"2024-04-05T22:23:08.838945+0000","last_unstale":"2024-04-05T22:23:11.072045+0000","last_undegraded":"2024-04-05T22:23:11.072045+0000","last_fullsized":"2024-04-05T22:23:11.072045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:19:04.891763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072339+0000","last_change":"2024-04-05T22:23:09.352156+0000","last_active":"2024-04-05T22:23:11.072339+0000","last_peered":"2024-04-05T22:23:11.072339+0000","last_clean":"2024-04-05T22:23:11.072339+0000","last_became_active":"2024-04-05T22:23:09.351439+0000","last_became_peered":"2024-04-05T22:23:09.351439+0000","last_unstale":"2024-04-05T22:23:11.072339+0000","last_undegraded":"2024-04-05T22:23:11.072339+0000","last_fullsized":"2024-04-05T22:23:11.072339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:22.537256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072786+0000","last_change":"2024-04-05T22:23:08.828030+0000","last_active":"2024-04-05T22:23:11.072786+0000","last_peered":"2024-04-05T22:23:11.072786+0000","last_clean":"2024-04-05T22:23:11.072786+0000","last_became_active":"2024-04-05T22:23:08.827689+0000","last_became_peered":"2024-04-05T22:23:08.827689+0000","last_unstale":"2024-04-05T22:23:11.072786+0000","last_undegraded":"2024-04-05T22:23:11.072786+0000","last_fullsized":"2024-04-05T22:23:11.072786+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:36.825290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072998+0000","last_change":"2024-04-05T22:23:09.374497+0000","last_active":"2024-04-05T22:23:11.072998+0000","last_peered":"2024-04-05T22:23:11.072998+0000","last_clean":"2024-04-05T22:23:11.072998+0000","last_became_active":"2024-04-05T22:23:09.374239+0000","last_became_peered":"2024-04-05T22:23:09.374239+0000","last_unstale":"2024-04-05T22:23:11.072998+0000","last_undegraded":"2024-04-05T22:23:11.072998+0000","last_fullsized":"2024-04-05T22:23:11.072998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:07:24.621644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073462+0000","last_change":"2024-04-05T22:23:08.822162+0000","last_active":"2024-04-05T22:23:11.073462+0000","last_peered":"2024-04-05T22:23:11.073462+0000","last_clean":"2024-04-05T22:23:11.073462+0000","last_became_active":"2024-04-05T22:23:08.821738+0000","last_became_peered":"2024-04-05T22:23:08.821738+0000","last_unstale":"2024-04-05T22:23:11.073462+0000","last_undegraded":"2024-04-05T22:23:11.073462+0000","last_fullsized":"2024-04-05T22:23:11.073462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:35:05.837398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071741+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.071741+0000","last_peered":"2024-04-05T22:23:11.071741+0000","last_clean":"2024-04-05T22:23:11.071741+0000","last_became_active":"2024-04-05T22:23:09.372140+0000","last_became_peered":"2024-04-05T22:23:09.372140+0000","last_unstale":"2024-04-05T22:23:11.071741+0000","last_undegraded":"2024-04-05T22:23:11.071741+0000","last_fullsized":"2024-04-05T22:23:11.071741+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:56:39.781215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487427+0000","last_change":"2024-04-05T22:23:08.833929+0000","last_active":"2024-04-05T22:23:10.487427+0000","last_peered":"2024-04-05T22:23:10.487427+0000","last_clean":"2024-04-05T22:23:10.487427+0000","last_became_active":"2024-04-05T22:23:08.831111+0000","last_became_peered":"2024-04-05T22:23:08.831111+0000","last_unstale":"2024-04-05T22:23:10.487427+0000","last_undegraded":"2024-04-05T22:23:10.487427+0000","last_fullsized":"2024-04-05T22:23:10.487427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072277+0000","last_change":"2024-04-05T22:23:09.365652+0000","last_active":"2024-04-05T22:23:11.072277+0000","last_peered":"2024-04-05T22:23:11.072277+0000","last_clean":"2024-04-05T22:23:11.072277+0000","last_became_active":"2024-04-05T22:23:09.364565+0000","last_became_peered":"2024-04-05T22:23:09.364565+0000","last_unstale":"2024-04-05T22:23:11.072277+0000","last_undegraded":"2024-04-05T22:23:11.072277+0000","last_fullsized":"2024-04-05T22:23:11.072277+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:51.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350248+0000","last_change":"2024-04-05T22:23:08.841687+0000","last_active":"2024-04-05T22:23:10.350248+0000","last_peered":"2024-04-05T22:23:10.350248+0000","last_clean":"2024-04-05T22:23:10.350248+0000","last_became_active":"2024-04-05T22:23:08.841487+0000","last_became_peered":"2024-04-05T22:23:08.841487+0000","last_unstale":"2024-04-05T22:23:10.350248+0000","last_undegraded":"2024-04-05T22:23:10.350248+0000","last_fullsized":"2024-04-05T22:23:10.350248+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:35:48.140463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349783+0000","last_change":"2024-04-05T22:23:09.368685+0000","last_active":"2024-04-05T22:23:10.349783+0000","last_peered":"2024-04-05T22:23:10.349783+0000","last_clean":"2024-04-05T22:23:10.349783+0000","last_became_active":"2024-04-05T22:23:09.368451+0000","last_became_peered":"2024-04-05T22:23:09.368451+0000","last_unstale":"2024-04-05T22:23:10.349783+0000","last_undegraded":"2024-04-05T22:23:10.349783+0000","last_fullsized":"2024-04-05T22:23:10.349783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:11.976720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350456+0000","last_change":"2024-04-05T22:23:08.838367+0000","last_active":"2024-04-05T22:23:10.350456+0000","last_peered":"2024-04-05T22:23:10.350456+0000","last_clean":"2024-04-05T22:23:10.350456+0000","last_became_active":"2024-04-05T22:23:08.837504+0000","last_became_peered":"2024-04-05T22:23:08.837504+0000","last_unstale":"2024-04-05T22:23:10.350456+0000","last_undegraded":"2024-04-05T22:23:10.350456+0000","last_fullsized":"2024-04-05T22:23:10.350456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:56.682813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346632+0000","last_change":"2024-04-05T22:23:09.371368+0000","last_active":"2024-04-05T22:23:10.346632+0000","last_peered":"2024-04-05T22:23:10.346632+0000","last_clean":"2024-04-05T22:23:10.346632+0000","last_became_active":"2024-04-05T22:23:09.369500+0000","last_became_peered":"2024-04-05T22:23:09.369500+0000","last_unstale":"2024-04-05T22:23:10.346632+0000","last_undegraded":"2024-04-05T22:23:10.346632+0000","last_fullsized":"2024-04-05T22:23:10.346632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:40.226240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487033+0000","last_change":"2024-04-05T22:23:08.838608+0000","last_active":"2024-04-05T22:23:10.487033+0000","last_peered":"2024-04-05T22:23:10.487033+0000","last_clean":"2024-04-05T22:23:10.487033+0000","last_became_active":"2024-04-05T22:23:08.838196+0000","last_became_peered":"2024-04-05T22:23:08.838196+0000","last_unstale":"2024-04-05T22:23:10.487033+0000","last_undegraded":"2024-04-05T22:23:10.487033+0000","last_fullsized":"2024-04-05T22:23:10.487033+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070603+0000","last_change":"2024-04-05T22:23:09.368050+0000","last_active":"2024-04-05T22:23:11.070603+0000","last_peered":"2024-04-05T22:23:11.070603+0000","last_clean":"2024-04-05T22:23:11.070603+0000","last_became_active":"2024-04-05T22:23:09.367340+0000","last_became_peered":"2024-04-05T22:23:09.367340+0000","last_unstale":"2024-04-05T22:23:11.070603+0000","last_undegraded":"2024-04-05T22:23:11.070603+0000","last_fullsized":"2024-04-05T22:23:11.070603+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:10.015015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071791+0000","last_change":"2024-04-05T22:23:08.832053+0000","last_active":"2024-04-05T22:23:11.071791+0000","last_peered":"2024-04-05T22:23:11.071791+0000","last_clean":"2024-04-05T22:23:11.071791+0000","last_became_active":"2024-04-05T22:23:08.831669+0000","last_became_peered":"2024-04-05T22:23:08.831669+0000","last_unstale":"2024-04-05T22:23:11.071791+0000","last_undegraded":"2024-04-05T22:23:11.071791+0000","last_fullsized":"2024-04-05T22:23:11.071791+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:17.304924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071638+0000","last_change":"2024-04-05T22:23:09.362024+0000","last_active":"2024-04-05T22:23:11.071638+0000","last_peered":"2024-04-05T22:23:11.071638+0000","last_clean":"2024-04-05T22:23:11.071638+0000","last_became_active":"2024-04-05T22:23:09.361762+0000","last_became_peered":"2024-04-05T22:23:09.361762+0000","last_unstale":"2024-04-05T22:23:11.071638+0000","last_undegraded":"2024-04-05T22:23:11.071638+0000","last_fullsized":"2024-04-05T22:23:11.071638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346491+0000","last_change":"2024-04-05T22:23:08.831835+0000","last_active":"2024-04-05T22:23:10.346491+0000","last_peered":"2024-04-05T22:23:10.346491+0000","last_clean":"2024-04-05T22:23:10.346491+0000","last_became_active":"2024-04-05T22:23:08.831526+0000","last_became_peered":"2024-04-05T22:23:08.831526+0000","last_unstale":"2024-04-05T22:23:10.346491+0000","last_undegraded":"2024-04-05T22:23:10.346491+0000","last_fullsized":"2024-04-05T22:23:10.346491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:09.010032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073313+0000","last_change":"2024-04-05T22:23:09.361600+0000","last_active":"2024-04-05T22:23:11.073313+0000","last_peered":"2024-04-05T22:23:11.073313+0000","last_clean":"2024-04-05T22:23:11.073313+0000","last_became_active":"2024-04-05T22:23:09.361317+0000","last_became_peered":"2024-04-05T22:23:09.361317+0000","last_unstale":"2024-04-05T22:23:11.073313+0000","last_undegraded":"2024-04-05T22:23:11.073313+0000","last_fullsized":"2024-04-05T22:23:11.073313+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:07.080539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350095+0000","last_change":"2024-04-05T22:23:08.835573+0000","last_active":"2024-04-05T22:23:10.350095+0000","last_peered":"2024-04-05T22:23:10.350095+0000","last_clean":"2024-04-05T22:23:10.350095+0000","last_became_active":"2024-04-05T22:23:08.835334+0000","last_became_peered":"2024-04-05T22:23:08.835334+0000","last_unstale":"2024-04-05T22:23:10.350095+0000","last_undegraded":"2024-04-05T22:23:10.350095+0000","last_fullsized":"2024-04-05T22:23:10.350095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:16.063665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072567+0000","last_change":"2024-04-05T22:23:09.369695+0000","last_active":"2024-04-05T22:23:11.072567+0000","last_peered":"2024-04-05T22:23:11.072567+0000","last_clean":"2024-04-05T22:23:11.072567+0000","last_became_active":"2024-04-05T22:23:09.366626+0000","last_became_peered":"2024-04-05T22:23:09.366626+0000","last_unstale":"2024-04-05T22:23:11.072567+0000","last_undegraded":"2024-04-05T22:23:11.072567+0000","last_fullsized":"2024-04-05T22:23:11.072567+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347294+0000","last_change":"2024-04-05T22:23:08.834972+0000","last_active":"2024-04-05T22:23:10.347294+0000","last_peered":"2024-04-05T22:23:10.347294+0000","last_clean":"2024-04-05T22:23:10.347294+0000","last_became_active":"2024-04-05T22:23:08.834348+0000","last_became_peered":"2024-04-05T22:23:08.834348+0000","last_unstale":"2024-04-05T22:23:10.347294+0000","last_undegraded":"2024-04-05T22:23:10.347294+0000","last_fullsized":"2024-04-05T22:23:10.347294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:47.731238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346985+0000","last_change":"2024-04-05T22:23:09.362042+0000","last_active":"2024-04-05T22:23:10.346985+0000","last_peered":"2024-04-05T22:23:10.346985+0000","last_clean":"2024-04-05T22:23:10.346985+0000","last_became_active":"2024-04-05T22:23:09.360332+0000","last_became_peered":"2024-04-05T22:23:09.360332+0000","last_unstale":"2024-04-05T22:23:10.346985+0000","last_undegraded":"2024-04-05T22:23:10.346985+0000","last_fullsized":"2024-04-05T22:23:10.346985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346736+0000","last_change":"2024-04-05T22:23:08.834808+0000","last_active":"2024-04-05T22:23:10.346736+0000","last_peered":"2024-04-05T22:23:10.346736+0000","last_clean":"2024-04-05T22:23:10.346736+0000","last_became_active":"2024-04-05T22:23:08.834540+0000","last_became_peered":"2024-04-05T22:23:08.834540+0000","last_unstale":"2024-04-05T22:23:10.346736+0000","last_undegraded":"2024-04-05T22:23:10.346736+0000","last_fullsized":"2024-04-05T22:23:10.346736+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349819+0000","last_change":"2024-04-05T22:23:09.368779+0000","last_active":"2024-04-05T22:23:10.349819+0000","last_peered":"2024-04-05T22:23:10.349819+0000","last_clean":"2024-04-05T22:23:10.349819+0000","last_became_active":"2024-04-05T22:23:09.368525+0000","last_became_peered":"2024-04-05T22:23:09.368525+0000","last_unstale":"2024-04-05T22:23:10.349819+0000","last_undegraded":"2024-04-05T22:23:10.349819+0000","last_fullsized":"2024-04-05T22:23:10.349819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:55.756624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347079+0000","last_change":"2024-04-05T22:23:08.837346+0000","last_active":"2024-04-05T22:23:10.347079+0000","last_peered":"2024-04-05T22:23:10.347079+0000","last_clean":"2024-04-05T22:23:10.347079+0000","last_became_active":"2024-04-05T22:23:08.834887+0000","last_became_peered":"2024-04-05T22:23:08.834887+0000","last_unstale":"2024-04-05T22:23:10.347079+0000","last_undegraded":"2024-04-05T22:23:10.347079+0000","last_fullsized":"2024-04-05T22:23:10.347079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:53.019125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349798+0000","last_change":"2024-04-05T22:23:09.372281+0000","last_active":"2024-04-05T22:23:10.349798+0000","last_peered":"2024-04-05T22:23:10.349798+0000","last_clean":"2024-04-05T22:23:10.349798+0000","last_became_active":"2024-04-05T22:23:09.371720+0000","last_became_peered":"2024-04-05T22:23:09.371720+0000","last_unstale":"2024-04-05T22:23:10.349798+0000","last_undegraded":"2024-04-05T22:23:10.349798+0000","last_fullsized":"2024-04-05T22:23:10.349798+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:16:32.033132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072619+0000","last_change":"2024-04-05T22:23:08.831235+0000","last_active":"2024-04-05T22:23:11.072619+0000","last_peered":"2024-04-05T22:23:11.072619+0000","last_clean":"2024-04-05T22:23:11.072619+0000","last_became_active":"2024-04-05T22:23:08.830890+0000","last_became_peered":"2024-04-05T22:23:08.830890+0000","last_unstale":"2024-04-05T22:23:11.072619+0000","last_undegraded":"2024-04-05T22:23:11.072619+0000","last_fullsized":"2024-04-05T22:23:11.072619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:28:35.503122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487316+0000","last_change":"2024-04-05T22:23:09.361959+0000","last_active":"2024-04-05T22:23:10.487316+0000","last_peered":"2024-04-05T22:23:10.487316+0000","last_clean":"2024-04-05T22:23:10.487316+0000","last_became_active":"2024-04-05T22:23:09.357075+0000","last_became_peered":"2024-04-05T22:23:09.357075+0000","last_unstale":"2024-04-05T22:23:10.487316+0000","last_undegraded":"2024-04-05T22:23:10.487316+0000","last_fullsized":"2024-04-05T22:23:10.487316+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346877+0000","last_change":"2024-04-05T22:23:08.838081+0000","last_active":"2024-04-05T22:23:10.346877+0000","last_peered":"2024-04-05T22:23:10.346877+0000","last_clean":"2024-04-05T22:23:10.346877+0000","last_became_active":"2024-04-05T22:23:08.837732+0000","last_became_peered":"2024-04-05T22:23:08.837732+0000","last_unstale":"2024-04-05T22:23:10.346877+0000","last_undegraded":"2024-04-05T22:23:10.346877+0000","last_fullsized":"2024-04-05T22:23:10.346877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071714+0000","last_change":"2024-04-05T22:23:09.374296+0000","last_active":"2024-04-05T22:23:11.071714+0000","last_peered":"2024-04-05T22:23:11.071714+0000","last_clean":"2024-04-05T22:23:11.071714+0000","last_became_active":"2024-04-05T22:23:09.374031+0000","last_became_peered":"2024-04-05T22:23:09.374031+0000","last_unstale":"2024-04-05T22:23:11.071714+0000","last_undegraded":"2024-04-05T22:23:11.071714+0000","last_fullsized":"2024-04-05T22:23:11.071714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:54.024108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072320+0000","last_change":"2024-04-05T22:23:08.831395+0000","last_active":"2024-04-05T22:23:11.072320+0000","last_peered":"2024-04-05T22:23:11.072320+0000","last_clean":"2024-04-05T22:23:11.072320+0000","last_became_active":"2024-04-05T22:23:08.831124+0000","last_became_peered":"2024-04-05T22:23:08.831124+0000","last_unstale":"2024-04-05T22:23:11.072320+0000","last_undegraded":"2024-04-05T22:23:11.072320+0000","last_fullsized":"2024-04-05T22:23:11.072320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073173+0000","last_change":"2024-04-05T22:23:09.375678+0000","last_active":"2024-04-05T22:23:11.073173+0000","last_peered":"2024-04-05T22:23:11.073173+0000","last_clean":"2024-04-05T22:23:11.073173+0000","last_became_active":"2024-04-05T22:23:09.375368+0000","last_became_peered":"2024-04-05T22:23:09.375368+0000","last_unstale":"2024-04-05T22:23:11.073173+0000","last_undegraded":"2024-04-05T22:23:11.073173+0000","last_fullsized":"2024-04-05T22:23:11.073173+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487546+0000","last_change":"2024-04-05T22:23:08.836443+0000","last_active":"2024-04-05T22:23:10.487546+0000","last_peered":"2024-04-05T22:23:10.487546+0000","last_clean":"2024-04-05T22:23:10.487546+0000","last_became_active":"2024-04-05T22:23:08.836065+0000","last_became_peered":"2024-04-05T22:23:08.836065+0000","last_unstale":"2024-04-05T22:23:10.487546+0000","last_undegraded":"2024-04-05T22:23:10.487546+0000","last_fullsized":"2024-04-05T22:23:10.487546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:07.607575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346513+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:23:10.346513+0000","last_peered":"2024-04-05T22:23:10.346513+0000","last_clean":"2024-04-05T22:23:10.346513+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:23:10.346513+0000","last_undegraded":"2024-04-05T22:23:10.346513+0000","last_fullsized":"2024-04-05T22:23:10.346513+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1159168,"data_stored":818980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.02,"15min":1.018},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.0069999999999999,"15min":0.99299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.5129999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":1.0149999999999999},"min":{"1min":0.77700000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.054},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1879999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.179,"15min":1.173},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.252},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2110000000000001,"15min":1.2010000000000001},"min":{"1min":0.93200000000000005,"5min":0.75,"15min":0.75},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.133,"15min":1.1140000000000001},"min":{"1min":0.95699999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5409999999999999,"5min":1.712,"15min":1.712},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2170000000000001,"15min":1.2150000000000001},"min":{"1min":0.90400000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4810000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.4810000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.232,"15min":1.218},"min":{"1min":1.0309999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6950000000000001,"5min":1.865,"15min":1.865},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.325,"15min":1.323},"min":{"1min":0.98699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.7050000000000001,"5min":1.788,"15min":1.788},"last":1.7050000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.2410000000000001,"15min":1.218},"min":{"1min":0.95999999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.8440000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.772},{"interface":"front","average":{"1min":1.46,"5min":1.3600000000000001,"15min":1.343},"min":{"1min":1.1140000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.802,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.47,"15min":1.448},"min":{"1min":1.3680000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.51,"15min":1.492},"min":{"1min":1.296,"5min":0.755,"15min":0.755},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.8160000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":229194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.131},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.002},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.036},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.579}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":1.6439999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.698},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.248}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.179}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.444},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.95599999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.605}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.28},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.5620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":573440,"data_stored":230086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.034}]},{"osd":1,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.103},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.198}]},{"osd":3,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.159},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.306},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.083}]},{"osd":7,"last update":"Fri Apr 5 22:22:10 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.3600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":229160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.435,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4119999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.505},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.90100000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4390000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.27,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.5720000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.999,"15min":1.0249999999999999},"min":{"1min":0.57399999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.1679999999999999,"5min":1.827,"15min":1.827},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.075,"15min":1.083},"min":{"1min":0.67000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6559999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1020000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2310000000000001,"15min":1.2549999999999999},"min":{"1min":0.82299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.496,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.306,"15min":1.3340000000000001},"min":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.573,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.361,"15min":1.391},"min":{"1min":0.87,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4219999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.476,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9390000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.399,"15min":1.4370000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4890000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.0900000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.4590000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7050000000000001,"5min":2.468,"15min":2.468},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.294,"5min":1.4179999999999999,"15min":1.4390000000000001},"min":{"1min":0.89400000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.0089999999999999,"5min":2.798,"15min":2.798},"last":1.1899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":230108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.028},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.258}]},{"osd":6,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.385},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.194},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.486}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.97699999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.113}]},{"osd":6,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.2889999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:22:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.3120000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1159168,"data_stored":818924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.179,"5min":1.367,"15min":1.367},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.39500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4590000000000001,"5min":1.669,"15min":1.669},"last":1.1579999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.514,"15min":1.514},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.49399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.6040000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.90900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.506,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0840000000000001,"15min":1.1020000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.202}]},{"osd":4,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1579999999999999,"15min":1.1539999999999999},"min":{"1min":0.747,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5960000000000001,"5min":1.929,"15min":1.929},"last":1.389},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6519999999999999,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.266}]},{"osd":5,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6559999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.042},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.196,"15min":1.1899999999999999},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.526,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.325}]},{"osd":6,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.784,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7110000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.2829999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.339,"15min":1.339},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7509999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.3120000000000001,"15min":1.331},"min":{"1min":0.75,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6619999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.2370000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:23:18.688 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:23:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.014+0000 7fc59a800640 1 Processor -- start 2024-04-05T22:23:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.014+0000 7fc59a800640 1 -- start start 2024-04-05T22:23:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc59a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 0x7fc5940cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc59a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc59415e850 0x7fc59415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc59a800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59406f5b0 con 0x7fc5940ccd20 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc59a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59415f240 con 0x7fc59415e850 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc59a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594070410 con 0x7fc5941587d0 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc593e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc59415e850 0x7fc59415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc593e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc59415e850 0x7fc59415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34276/0 (socket says 172.21.15.73:34276) 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc593e00640 1 -- 172.21.15.73:0/3761268641 learned_addr learned my addr 172.21.15.73:0/3761268641 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.016+0000 7fc593400640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 0x7fc5940cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.017+0000 7fc593e00640 1 -- 172.21.15.73:0/3761268641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 msgr2=0x7fc5940cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.017+0000 7fc593e00640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 0x7fc5940cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.017+0000 7fc593e00640 1 -- 172.21.15.73:0/3761268641 >> v1:172.21.15.157:6789/0 conn(0x7fc5940ccd20 legacy=0x7fc59406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.017+0000 7fc593e00640 1 -- 172.21.15.73:0/3761268641 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594158bb0 con 0x7fc59415e850 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.017+0000 7fc593400640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 0x7fc5940cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.018+0000 7fc593e00640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59415e850 0x7fc59415ed00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009e50 tx=0x7fc58c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44a707c7517a3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.018+0000 7fc592000640 1 -- 172.21.15.73:0/3761268641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c02cb00 con 0x7fc59415e850 2024-04-05T22:23:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.018+0000 7fc592000640 1 -- 172.21.15.73:0/3761268641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c02cc60 con 0x7fc59415e850 2024-04-05T22:23:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.018+0000 7fc592000640 1 -- 172.21.15.73:0/3761268641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0343f0 con 0x7fc59415e850 2024-04-05T22:23:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 -- 172.21.15.73:0/3761268641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59415e850 msgr2=0x7fc59415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59415e850 0x7fc59415ed00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009e50 tx=0x7fc58c02bf60 comp rx=0 tx=0).stop 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 -- 172.21.15.73:0/3761268641 shutdown_connections 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59415e850 0x7fc59415ed00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 --2- 172.21.15.73:0/3761268641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5941587d0 0x7fc5940cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 -- 172.21.15.73:0/3761268641 >> 172.21.15.73:0/3761268641 conn(0x7fc5940ce8e0 msgr2=0x7fc5940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 -- 172.21.15.73:0/3761268641 shutdown_connections 2024-04-05T22:23:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.019+0000 7fc59a800640 1 -- 172.21.15.73:0/3761268641 wait complete. 2024-04-05T22:23:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.021+0000 7fc59a800640 1 Processor -- start 2024-04-05T22:23:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.021+0000 7fc59a800640 1 -- start start 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59415e850 0x7fc594156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc59415a1a0 0x7fc5940aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940ad010 con 0x7fc5941587d0 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940ad2f0 con 0x7fc59415e850 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc59a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940ad5d0 con 0x7fc59415a1a0 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc593400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc593400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34278/0 (socket says 172.21.15.73:34278) 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc593400640 1 -- 172.21.15.73:0/644324960 learned_addr learned my addr 172.21.15.73:0/644324960 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.022+0000 7fc592a00640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59415e850 0x7fc594156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593e00640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc59415a1a0 0x7fc5940aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 -- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc59415a1a0 msgr2=0x7fc5940aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc59415a1a0 0x7fc5940aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 -- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59415e850 msgr2=0x7fc594156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59415e850 0x7fc594156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 -- 172.21.15.73:0/644324960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c009b30 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc593400640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e7e0 tx=0x7fc58400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584004030 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584004190 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584004520 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5940ad8b0 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.023+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc594156d70 con 0x7fc5941587d0 2024-04-05T22:23:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.024+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc594066af0 con 0x7fc5941587d0 2024-04-05T22:23:19.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.026+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc58400c6d0 con 0x7fc5941587d0 2024-04-05T22:23:19.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.026+0000 7fc58b400640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 0x7fc57406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.026+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5840894d0 con 0x7fc5941587d0 2024-04-05T22:23:19.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.027+0000 7fc592a00640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 0x7fc57406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.027+0000 7fc592a00640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 0x7fc57406d270 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc594157150 tx=0x7fc580009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.031+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc584054280 con 0x7fc5941587d0 2024-04-05T22:23:19.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.221+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5940ad460 con 0x7fc57406adb0 2024-04-05T22:23:19.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.243+0000 7fc58b400640 1 -- 172.21.15.73:0/644324960 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286648 (secure 0 0 0) 0x7fc5940ad460 con 0x7fc57406adb0 2024-04-05T22:23:19.246 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:19.248 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:23:19.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.249+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 msgr2=0x7fc57406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.249+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 0x7fc57406d270 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc594157150 tx=0x7fc580009040 comp rx=0 tx=0).stop 2024-04-05T22:23:19.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 msgr2=0x7fc5940aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e7e0 tx=0x7fc58400eca0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 shutdown_connections 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc57406adb0 0x7fc57406d270 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc59415a1a0 0x7fc5940aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59415e850 0x7fc594156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 --2- 172.21.15.73:0/644324960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5941587d0 0x7fc5940aca10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 >> 172.21.15.73:0/644324960 conn(0x7fc5940ce8e0 msgr2=0x7fc5940cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 shutdown_connections 2024-04-05T22:23:19.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.250+0000 7fc59a800640 1 -- 172.21.15.73:0/644324960 wait complete. 2024-04-05T22:23:19.270 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-05T22:23:18.527730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242280,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754732440,"statfs":{"total":773094113280,"available":772846018560,"internally_reserved":0,"allocated":5742592,"data_stored":3013636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":242208315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.189042"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347585+0000","last_change":"2024-04-05T22:23:09.381025+0000","last_active":"2024-04-05T22:23:10.347585+0000","last_peered":"2024-04-05T22:23:10.347585+0000","last_clean":"2024-04-05T22:23:10.347585+0000","last_became_active":"2024-04-05T22:23:09.379621+0000","last_became_peered":"2024-04-05T22:23:09.379621+0000","last_unstale":"2024-04-05T22:23:10.347585+0000","last_undegraded":"2024-04-05T22:23:10.347585+0000","last_fullsized":"2024-04-05T22:23:10.347585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487095+0000","last_change":"2024-04-05T22:23:09.358874+0000","last_active":"2024-04-05T22:23:10.487095+0000","last_peered":"2024-04-05T22:23:10.487095+0000","last_clean":"2024-04-05T22:23:10.487095+0000","last_became_active":"2024-04-05T22:23:09.358109+0000","last_became_peered":"2024-04-05T22:23:09.358109+0000","last_unstale":"2024-04-05T22:23:10.487095+0000","last_undegraded":"2024-04-05T22:23:10.487095+0000","last_fullsized":"2024-04-05T22:23:10.487095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073685+0000","last_change":"2024-04-05T22:23:09.365855+0000","last_active":"2024-04-05T22:23:11.073685+0000","last_peered":"2024-04-05T22:23:11.073685+0000","last_clean":"2024-04-05T22:23:11.073685+0000","last_became_active":"2024-04-05T22:23:09.365109+0000","last_became_peered":"2024-04-05T22:23:09.365109+0000","last_unstale":"2024-04-05T22:23:11.073685+0000","last_undegraded":"2024-04-05T22:23:11.073685+0000","last_fullsized":"2024-04-05T22:23:11.073685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:51.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072301+0000","last_change":"2024-04-05T22:23:09.360571+0000","last_active":"2024-04-05T22:23:11.072301+0000","last_peered":"2024-04-05T22:23:11.072301+0000","last_clean":"2024-04-05T22:23:11.072301+0000","last_became_active":"2024-04-05T22:23:09.360249+0000","last_became_peered":"2024-04-05T22:23:09.360249+0000","last_unstale":"2024-04-05T22:23:11.072301+0000","last_undegraded":"2024-04-05T22:23:11.072301+0000","last_fullsized":"2024-04-05T22:23:11.072301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:02.342545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486869+0000","last_change":"2024-04-05T22:23:09.363323+0000","last_active":"2024-04-05T22:23:10.486869+0000","last_peered":"2024-04-05T22:23:10.486869+0000","last_clean":"2024-04-05T22:23:10.486869+0000","last_became_active":"2024-04-05T22:23:09.363021+0000","last_became_peered":"2024-04-05T22:23:09.363021+0000","last_unstale":"2024-04-05T22:23:10.486869+0000","last_undegraded":"2024-04-05T22:23:10.486869+0000","last_fullsized":"2024-04-05T22:23:10.486869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349441+0000","last_change":"2024-04-05T22:23:09.376486+0000","last_active":"2024-04-05T22:23:10.349441+0000","last_peered":"2024-04-05T22:23:10.349441+0000","last_clean":"2024-04-05T22:23:10.349441+0000","last_became_active":"2024-04-05T22:23:09.376214+0000","last_became_peered":"2024-04-05T22:23:09.376214+0000","last_unstale":"2024-04-05T22:23:10.349441+0000","last_undegraded":"2024-04-05T22:23:10.349441+0000","last_fullsized":"2024-04-05T22:23:10.349441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:56:38.205365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347409+0000","last_change":"2024-04-05T22:23:09.374598+0000","last_active":"2024-04-05T22:23:10.347409+0000","last_peered":"2024-04-05T22:23:10.347409+0000","last_clean":"2024-04-05T22:23:10.347409+0000","last_became_active":"2024-04-05T22:23:09.374273+0000","last_became_peered":"2024-04-05T22:23:09.374273+0000","last_unstale":"2024-04-05T22:23:10.347409+0000","last_undegraded":"2024-04-05T22:23:10.347409+0000","last_fullsized":"2024-04-05T22:23:10.347409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071571+0000","last_change":"2024-04-05T22:23:09.362559+0000","last_active":"2024-04-05T22:23:11.071571+0000","last_peered":"2024-04-05T22:23:11.071571+0000","last_clean":"2024-04-05T22:23:11.071571+0000","last_became_active":"2024-04-05T22:23:09.362193+0000","last_became_peered":"2024-04-05T22:23:09.362193+0000","last_unstale":"2024-04-05T22:23:11.071571+0000","last_undegraded":"2024-04-05T22:23:11.071571+0000","last_fullsized":"2024-04-05T22:23:11.071571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:17.066127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072891+0000","last_change":"2024-04-05T22:23:09.381934+0000","last_active":"2024-04-05T22:23:11.072891+0000","last_peered":"2024-04-05T22:23:11.072891+0000","last_clean":"2024-04-05T22:23:11.072891+0000","last_became_active":"2024-04-05T22:23:09.380359+0000","last_became_peered":"2024-04-05T22:23:09.380359+0000","last_unstale":"2024-04-05T22:23:11.072891+0000","last_undegraded":"2024-04-05T22:23:11.072891+0000","last_fullsized":"2024-04-05T22:23:11.072891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073130+0000","last_change":"2024-04-05T22:23:09.360338+0000","last_active":"2024-04-05T22:23:11.073130+0000","last_peered":"2024-04-05T22:23:11.073130+0000","last_clean":"2024-04-05T22:23:11.073130+0000","last_became_active":"2024-04-05T22:23:09.360037+0000","last_became_peered":"2024-04-05T22:23:09.360037+0000","last_unstale":"2024-04-05T22:23:11.073130+0000","last_undegraded":"2024-04-05T22:23:11.073130+0000","last_fullsized":"2024-04-05T22:23:11.073130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:00.708491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071876+0000","last_change":"2024-04-05T22:23:09.363867+0000","last_active":"2024-04-05T22:23:11.071876+0000","last_peered":"2024-04-05T22:23:11.071876+0000","last_clean":"2024-04-05T22:23:11.071876+0000","last_became_active":"2024-04-05T22:23:09.363604+0000","last_became_peered":"2024-04-05T22:23:09.363604+0000","last_unstale":"2024-04-05T22:23:11.071876+0000","last_undegraded":"2024-04-05T22:23:11.071876+0000","last_fullsized":"2024-04-05T22:23:11.071876+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:35.409579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487272+0000","last_change":"2024-04-05T22:23:09.361574+0000","last_active":"2024-04-05T22:23:10.487272+0000","last_peered":"2024-04-05T22:23:10.487272+0000","last_clean":"2024-04-05T22:23:10.487272+0000","last_became_active":"2024-04-05T22:23:09.361302+0000","last_became_peered":"2024-04-05T22:23:09.361302+0000","last_unstale":"2024-04-05T22:23:10.487272+0000","last_undegraded":"2024-04-05T22:23:10.487272+0000","last_fullsized":"2024-04-05T22:23:10.487272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070969+0000","last_change":"2024-04-05T22:23:09.374114+0000","last_active":"2024-04-05T22:23:11.070969+0000","last_peered":"2024-04-05T22:23:11.070969+0000","last_clean":"2024-04-05T22:23:11.070969+0000","last_became_active":"2024-04-05T22:23:09.373743+0000","last_became_peered":"2024-04-05T22:23:09.373743+0000","last_unstale":"2024-04-05T22:23:11.070969+0000","last_undegraded":"2024-04-05T22:23:11.070969+0000","last_fullsized":"2024-04-05T22:23:11.070969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:25.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072456+0000","last_change":"2024-04-05T22:23:09.360964+0000","last_active":"2024-04-05T22:23:11.072456+0000","last_peered":"2024-04-05T22:23:11.072456+0000","last_clean":"2024-04-05T22:23:11.072456+0000","last_became_active":"2024-04-05T22:23:09.360707+0000","last_became_peered":"2024-04-05T22:23:09.360707+0000","last_unstale":"2024-04-05T22:23:11.072456+0000","last_undegraded":"2024-04-05T22:23:11.072456+0000","last_fullsized":"2024-04-05T22:23:11.072456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349236+0000","last_change":"2024-04-05T22:23:09.374250+0000","last_active":"2024-04-05T22:23:10.349236+0000","last_peered":"2024-04-05T22:23:10.349236+0000","last_clean":"2024-04-05T22:23:10.349236+0000","last_became_active":"2024-04-05T22:23:09.373960+0000","last_became_peered":"2024-04-05T22:23:09.373960+0000","last_unstale":"2024-04-05T22:23:10.349236+0000","last_undegraded":"2024-04-05T22:23:10.349236+0000","last_fullsized":"2024-04-05T22:23:10.349236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:59.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071101+0000","last_change":"2024-04-05T22:23:09.359204+0000","last_active":"2024-04-05T22:23:11.071101+0000","last_peered":"2024-04-05T22:23:11.071101+0000","last_clean":"2024-04-05T22:23:11.071101+0000","last_became_active":"2024-04-05T22:23:09.358942+0000","last_became_peered":"2024-04-05T22:23:09.358942+0000","last_unstale":"2024-04-05T22:23:11.071101+0000","last_undegraded":"2024-04-05T22:23:11.071101+0000","last_fullsized":"2024-04-05T22:23:11.071101+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:02.659494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072261+0000","last_change":"2024-04-05T22:23:09.376368+0000","last_active":"2024-04-05T22:23:11.072261+0000","last_peered":"2024-04-05T22:23:11.072261+0000","last_clean":"2024-04-05T22:23:11.072261+0000","last_became_active":"2024-04-05T22:23:09.376102+0000","last_became_peered":"2024-04-05T22:23:09.376102+0000","last_unstale":"2024-04-05T22:23:11.072261+0000","last_undegraded":"2024-04-05T22:23:11.072261+0000","last_fullsized":"2024-04-05T22:23:11.072261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:33.661018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073167+0000","last_change":"2024-04-05T22:23:09.377480+0000","last_active":"2024-04-05T22:23:11.073167+0000","last_peered":"2024-04-05T22:23:11.073167+0000","last_clean":"2024-04-05T22:23:11.073167+0000","last_became_active":"2024-04-05T22:23:09.377201+0000","last_became_peered":"2024-04-05T22:23:09.377201+0000","last_unstale":"2024-04-05T22:23:11.073167+0000","last_undegraded":"2024-04-05T22:23:11.073167+0000","last_fullsized":"2024-04-05T22:23:11.073167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:09.415421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072444+0000","last_change":"2024-04-05T22:23:09.361412+0000","last_active":"2024-04-05T22:23:11.072444+0000","last_peered":"2024-04-05T22:23:11.072444+0000","last_clean":"2024-04-05T22:23:11.072444+0000","last_became_active":"2024-04-05T22:23:09.361098+0000","last_became_peered":"2024-04-05T22:23:09.361098+0000","last_unstale":"2024-04-05T22:23:11.072444+0000","last_undegraded":"2024-04-05T22:23:11.072444+0000","last_fullsized":"2024-04-05T22:23:11.072444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:40.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071327+0000","last_change":"2024-04-05T22:23:09.355237+0000","last_active":"2024-04-05T22:23:11.071327+0000","last_peered":"2024-04-05T22:23:11.071327+0000","last_clean":"2024-04-05T22:23:11.071327+0000","last_became_active":"2024-04-05T22:23:09.354907+0000","last_became_peered":"2024-04-05T22:23:09.354907+0000","last_unstale":"2024-04-05T22:23:11.071327+0000","last_undegraded":"2024-04-05T22:23:11.071327+0000","last_fullsized":"2024-04-05T22:23:11.071327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:11:07.814340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350119+0000","last_change":"2024-04-05T22:23:09.373051+0000","last_active":"2024-04-05T22:23:10.350119+0000","last_peered":"2024-04-05T22:23:10.350119+0000","last_clean":"2024-04-05T22:23:10.350119+0000","last_became_active":"2024-04-05T22:23:09.372014+0000","last_became_peered":"2024-04-05T22:23:09.372014+0000","last_unstale":"2024-04-05T22:23:10.350119+0000","last_undegraded":"2024-04-05T22:23:10.350119+0000","last_fullsized":"2024-04-05T22:23:10.350119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:58.535697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072702+0000","last_change":"2024-04-05T22:23:09.360582+0000","last_active":"2024-04-05T22:23:11.072702+0000","last_peered":"2024-04-05T22:23:11.072702+0000","last_clean":"2024-04-05T22:23:11.072702+0000","last_became_active":"2024-04-05T22:23:09.360346+0000","last_became_peered":"2024-04-05T22:23:09.360346+0000","last_unstale":"2024-04-05T22:23:11.072702+0000","last_undegraded":"2024-04-05T22:23:11.072702+0000","last_fullsized":"2024-04-05T22:23:11.072702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071233+0000","last_change":"2024-04-05T22:23:09.365234+0000","last_active":"2024-04-05T22:23:11.071233+0000","last_peered":"2024-04-05T22:23:11.071233+0000","last_clean":"2024-04-05T22:23:11.071233+0000","last_became_active":"2024-04-05T22:23:09.364930+0000","last_became_peered":"2024-04-05T22:23:09.364930+0000","last_unstale":"2024-04-05T22:23:11.071233+0000","last_undegraded":"2024-04-05T22:23:11.071233+0000","last_fullsized":"2024-04-05T22:23:11.071233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:33:24.735249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071336+0000","last_change":"2024-04-05T22:23:09.367889+0000","last_active":"2024-04-05T22:23:11.071336+0000","last_peered":"2024-04-05T22:23:11.071336+0000","last_clean":"2024-04-05T22:23:11.071336+0000","last_became_active":"2024-04-05T22:23:09.367652+0000","last_became_peered":"2024-04-05T22:23:09.367652+0000","last_unstale":"2024-04-05T22:23:11.071336+0000","last_undegraded":"2024-04-05T22:23:11.071336+0000","last_fullsized":"2024-04-05T22:23:11.071336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:03.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073053+0000","last_change":"2024-04-05T22:23:09.365048+0000","last_active":"2024-04-05T22:23:11.073053+0000","last_peered":"2024-04-05T22:23:11.073053+0000","last_clean":"2024-04-05T22:23:11.073053+0000","last_became_active":"2024-04-05T22:23:09.364707+0000","last_became_peered":"2024-04-05T22:23:09.364707+0000","last_unstale":"2024-04-05T22:23:11.073053+0000","last_undegraded":"2024-04-05T22:23:11.073053+0000","last_fullsized":"2024-04-05T22:23:11.073053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:43.155641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071908+0000","last_change":"2024-04-05T22:23:09.375125+0000","last_active":"2024-04-05T22:23:11.071908+0000","last_peered":"2024-04-05T22:23:11.071908+0000","last_clean":"2024-04-05T22:23:11.071908+0000","last_became_active":"2024-04-05T22:23:09.373382+0000","last_became_peered":"2024-04-05T22:23:09.373382+0000","last_unstale":"2024-04-05T22:23:11.071908+0000","last_undegraded":"2024-04-05T22:23:11.071908+0000","last_fullsized":"2024-04-05T22:23:11.071908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:53:45.868945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073020+0000","last_change":"2024-04-05T22:23:09.376021+0000","last_active":"2024-04-05T22:23:11.073020+0000","last_peered":"2024-04-05T22:23:11.073020+0000","last_clean":"2024-04-05T22:23:11.073020+0000","last_became_active":"2024-04-05T22:23:09.375702+0000","last_became_peered":"2024-04-05T22:23:09.375702+0000","last_unstale":"2024-04-05T22:23:11.073020+0000","last_undegraded":"2024-04-05T22:23:11.073020+0000","last_fullsized":"2024-04-05T22:23:11.073020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:55.351589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487322+0000","last_change":"2024-04-05T22:23:09.356705+0000","last_active":"2024-04-05T22:23:10.487322+0000","last_peered":"2024-04-05T22:23:10.487322+0000","last_clean":"2024-04-05T22:23:10.487322+0000","last_became_active":"2024-04-05T22:23:09.356429+0000","last_became_peered":"2024-04-05T22:23:09.356429+0000","last_unstale":"2024-04-05T22:23:10.487322+0000","last_undegraded":"2024-04-05T22:23:10.487322+0000","last_fullsized":"2024-04-05T22:23:10.487322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071142+0000","last_change":"2024-04-05T22:23:09.364089+0000","last_active":"2024-04-05T22:23:11.071142+0000","last_peered":"2024-04-05T22:23:11.071142+0000","last_clean":"2024-04-05T22:23:11.071142+0000","last_became_active":"2024-04-05T22:23:09.363793+0000","last_became_peered":"2024-04-05T22:23:09.363793+0000","last_unstale":"2024-04-05T22:23:11.071142+0000","last_undegraded":"2024-04-05T22:23:11.071142+0000","last_fullsized":"2024-04-05T22:23:11.071142+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:18:38.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071969+0000","last_change":"2024-04-05T22:23:09.365957+0000","last_active":"2024-04-05T22:23:11.071969+0000","last_peered":"2024-04-05T22:23:11.071969+0000","last_clean":"2024-04-05T22:23:11.071969+0000","last_became_active":"2024-04-05T22:23:09.365640+0000","last_became_peered":"2024-04-05T22:23:09.365640+0000","last_unstale":"2024-04-05T22:23:11.071969+0000","last_undegraded":"2024-04-05T22:23:11.071969+0000","last_fullsized":"2024-04-05T22:23:11.071969+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:48.736221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349506+0000","last_change":"2024-04-05T22:23:09.377291+0000","last_active":"2024-04-05T22:23:10.349506+0000","last_peered":"2024-04-05T22:23:10.349506+0000","last_clean":"2024-04-05T22:23:10.349506+0000","last_became_active":"2024-04-05T22:23:09.377044+0000","last_became_peered":"2024-04-05T22:23:09.377044+0000","last_unstale":"2024-04-05T22:23:10.349506+0000","last_undegraded":"2024-04-05T22:23:10.349506+0000","last_fullsized":"2024-04-05T22:23:10.349506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:39:13.822332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070568+0000","last_change":"2024-04-05T22:23:09.348727+0000","last_active":"2024-04-05T22:23:11.070568+0000","last_peered":"2024-04-05T22:23:11.070568+0000","last_clean":"2024-04-05T22:23:11.070568+0000","last_became_active":"2024-04-05T22:23:09.348346+0000","last_became_peered":"2024-04-05T22:23:09.348346+0000","last_unstale":"2024-04-05T22:23:11.070568+0000","last_undegraded":"2024-04-05T22:23:11.070568+0000","last_fullsized":"2024-04-05T22:23:11.070568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:18.309908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347754+0000","last_change":"2024-04-05T22:23:09.379081+0000","last_active":"2024-04-05T22:23:10.347754+0000","last_peered":"2024-04-05T22:23:10.347754+0000","last_clean":"2024-04-05T22:23:10.347754+0000","last_became_active":"2024-04-05T22:23:09.378754+0000","last_became_peered":"2024-04-05T22:23:09.378754+0000","last_unstale":"2024-04-05T22:23:10.347754+0000","last_undegraded":"2024-04-05T22:23:10.347754+0000","last_fullsized":"2024-04-05T22:23:10.347754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:07.603359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350430+0000","last_change":"2024-04-05T22:23:08.841250+0000","last_active":"2024-04-05T22:23:10.350430+0000","last_peered":"2024-04-05T22:23:10.350430+0000","last_clean":"2024-04-05T22:23:10.350430+0000","last_became_active":"2024-04-05T22:23:08.841032+0000","last_became_peered":"2024-04-05T22:23:08.841032+0000","last_unstale":"2024-04-05T22:23:10.350430+0000","last_undegraded":"2024-04-05T22:23:10.350430+0000","last_fullsized":"2024-04-05T22:23:10.350430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:06.598376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073172+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.073172+0000","last_peered":"2024-04-05T22:23:11.073172+0000","last_clean":"2024-04-05T22:23:11.073172+0000","last_became_active":"2024-04-05T22:23:09.371747+0000","last_became_peered":"2024-04-05T22:23:09.371747+0000","last_unstale":"2024-04-05T22:23:11.073172+0000","last_undegraded":"2024-04-05T22:23:11.073172+0000","last_fullsized":"2024-04-05T22:23:11.073172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:48:19.797618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073337+0000","last_change":"2024-04-05T22:23:08.839262+0000","last_active":"2024-04-05T22:23:11.073337+0000","last_peered":"2024-04-05T22:23:11.073337+0000","last_clean":"2024-04-05T22:23:11.073337+0000","last_became_active":"2024-04-05T22:23:08.838980+0000","last_became_peered":"2024-04-05T22:23:08.838980+0000","last_unstale":"2024-04-05T22:23:11.073337+0000","last_undegraded":"2024-04-05T22:23:11.073337+0000","last_fullsized":"2024-04-05T22:23:11.073337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:24:35.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487575+0000","last_change":"2024-04-05T22:23:09.372679+0000","last_active":"2024-04-05T22:23:10.487575+0000","last_peered":"2024-04-05T22:23:10.487575+0000","last_clean":"2024-04-05T22:23:10.487575+0000","last_became_active":"2024-04-05T22:23:09.372413+0000","last_became_peered":"2024-04-05T22:23:09.372413+0000","last_unstale":"2024-04-05T22:23:10.487575+0000","last_undegraded":"2024-04-05T22:23:10.487575+0000","last_fullsized":"2024-04-05T22:23:10.487575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347122+0000","last_change":"2024-04-05T22:23:08.831526+0000","last_active":"2024-04-05T22:23:10.347122+0000","last_peered":"2024-04-05T22:23:10.347122+0000","last_clean":"2024-04-05T22:23:10.347122+0000","last_became_active":"2024-04-05T22:23:08.831156+0000","last_became_peered":"2024-04-05T22:23:08.831156+0000","last_unstale":"2024-04-05T22:23:10.347122+0000","last_undegraded":"2024-04-05T22:23:10.347122+0000","last_fullsized":"2024-04-05T22:23:10.347122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347002+0000","last_change":"2024-04-05T22:23:09.371601+0000","last_active":"2024-04-05T22:23:10.347002+0000","last_peered":"2024-04-05T22:23:10.347002+0000","last_clean":"2024-04-05T22:23:10.347002+0000","last_became_active":"2024-04-05T22:23:09.371235+0000","last_became_peered":"2024-04-05T22:23:09.371235+0000","last_unstale":"2024-04-05T22:23:10.347002+0000","last_undegraded":"2024-04-05T22:23:10.347002+0000","last_fullsized":"2024-04-05T22:23:10.347002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347223+0000","last_change":"2024-04-05T22:23:08.834593+0000","last_active":"2024-04-05T22:23:10.347223+0000","last_peered":"2024-04-05T22:23:10.347223+0000","last_clean":"2024-04-05T22:23:10.347223+0000","last_became_active":"2024-04-05T22:23:08.833796+0000","last_became_peered":"2024-04-05T22:23:08.833796+0000","last_unstale":"2024-04-05T22:23:10.347223+0000","last_undegraded":"2024-04-05T22:23:10.347223+0000","last_fullsized":"2024-04-05T22:23:10.347223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072191+0000","last_change":"2024-04-05T22:23:09.364226+0000","last_active":"2024-04-05T22:23:11.072191+0000","last_peered":"2024-04-05T22:23:11.072191+0000","last_clean":"2024-04-05T22:23:11.072191+0000","last_became_active":"2024-04-05T22:23:09.363915+0000","last_became_peered":"2024-04-05T22:23:09.363915+0000","last_unstale":"2024-04-05T22:23:11.072191+0000","last_undegraded":"2024-04-05T22:23:11.072191+0000","last_fullsized":"2024-04-05T22:23:11.072191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:41:08.452502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350046+0000","last_change":"2024-04-05T22:23:08.837746+0000","last_active":"2024-04-05T22:23:10.350046+0000","last_peered":"2024-04-05T22:23:10.350046+0000","last_clean":"2024-04-05T22:23:10.350046+0000","last_became_active":"2024-04-05T22:23:08.837489+0000","last_became_peered":"2024-04-05T22:23:08.837489+0000","last_unstale":"2024-04-05T22:23:10.350046+0000","last_undegraded":"2024-04-05T22:23:10.350046+0000","last_fullsized":"2024-04-05T22:23:10.350046+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:58:34.404596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071930+0000","last_change":"2024-04-05T22:23:09.357300+0000","last_active":"2024-04-05T22:23:11.071930+0000","last_peered":"2024-04-05T22:23:11.071930+0000","last_clean":"2024-04-05T22:23:11.071930+0000","last_became_active":"2024-04-05T22:23:09.357047+0000","last_became_peered":"2024-04-05T22:23:09.357047+0000","last_unstale":"2024-04-05T22:23:11.071930+0000","last_undegraded":"2024-04-05T22:23:11.071930+0000","last_fullsized":"2024-04-05T22:23:11.071930+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:06.197871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072432+0000","last_change":"2024-04-05T22:23:08.837128+0000","last_active":"2024-04-05T22:23:11.072432+0000","last_peered":"2024-04-05T22:23:11.072432+0000","last_clean":"2024-04-05T22:23:11.072432+0000","last_became_active":"2024-04-05T22:23:08.836444+0000","last_became_peered":"2024-04-05T22:23:08.836444+0000","last_unstale":"2024-04-05T22:23:11.072432+0000","last_undegraded":"2024-04-05T22:23:11.072432+0000","last_fullsized":"2024-04-05T22:23:11.072432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:27.879231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071573+0000","last_change":"2024-04-05T22:23:09.362416+0000","last_active":"2024-04-05T22:23:11.071573+0000","last_peered":"2024-04-05T22:23:11.071573+0000","last_clean":"2024-04-05T22:23:11.071573+0000","last_became_active":"2024-04-05T22:23:09.361962+0000","last_became_peered":"2024-04-05T22:23:09.361962+0000","last_unstale":"2024-04-05T22:23:11.071573+0000","last_undegraded":"2024-04-05T22:23:11.071573+0000","last_fullsized":"2024-04-05T22:23:11.071573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:52.825291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.486775+0000","last_change":"2024-04-05T22:23:08.831328+0000","last_active":"2024-04-05T22:23:10.486775+0000","last_peered":"2024-04-05T22:23:10.486775+0000","last_clean":"2024-04-05T22:23:10.486775+0000","last_became_active":"2024-04-05T22:23:08.830959+0000","last_became_peered":"2024-04-05T22:23:08.830959+0000","last_unstale":"2024-04-05T22:23:10.486775+0000","last_undegraded":"2024-04-05T22:23:10.486775+0000","last_fullsized":"2024-04-05T22:23:10.486775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070939+0000","last_change":"2024-04-05T22:23:09.365286+0000","last_active":"2024-04-05T22:23:11.070939+0000","last_peered":"2024-04-05T22:23:11.070939+0000","last_clean":"2024-04-05T22:23:11.070939+0000","last_became_active":"2024-04-05T22:23:09.364944+0000","last_became_peered":"2024-04-05T22:23:09.364944+0000","last_unstale":"2024-04-05T22:23:11.070939+0000","last_undegraded":"2024-04-05T22:23:11.070939+0000","last_fullsized":"2024-04-05T22:23:11.070939+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:39:32.120858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071812+0000","last_change":"2024-04-05T22:23:08.833902+0000","last_active":"2024-04-05T22:23:11.071812+0000","last_peered":"2024-04-05T22:23:11.071812+0000","last_clean":"2024-04-05T22:23:11.071812+0000","last_became_active":"2024-04-05T22:23:08.833179+0000","last_became_peered":"2024-04-05T22:23:08.833179+0000","last_unstale":"2024-04-05T22:23:11.071812+0000","last_undegraded":"2024-04-05T22:23:11.071812+0000","last_fullsized":"2024-04-05T22:23:11.071812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:08:00.232153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073368+0000","last_change":"2024-04-05T22:23:09.379153+0000","last_active":"2024-04-05T22:23:11.073368+0000","last_peered":"2024-04-05T22:23:11.073368+0000","last_clean":"2024-04-05T22:23:11.073368+0000","last_became_active":"2024-04-05T22:23:09.378897+0000","last_became_peered":"2024-04-05T22:23:09.378897+0000","last_unstale":"2024-04-05T22:23:11.073368+0000","last_undegraded":"2024-04-05T22:23:11.073368+0000","last_fullsized":"2024-04-05T22:23:11.073368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:08.612559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072706+0000","last_change":"2024-04-05T22:23:08.828406+0000","last_active":"2024-04-05T22:23:11.072706+0000","last_peered":"2024-04-05T22:23:11.072706+0000","last_clean":"2024-04-05T22:23:11.072706+0000","last_became_active":"2024-04-05T22:23:08.828116+0000","last_became_peered":"2024-04-05T22:23:08.828116+0000","last_unstale":"2024-04-05T22:23:11.072706+0000","last_undegraded":"2024-04-05T22:23:11.072706+0000","last_fullsized":"2024-04-05T22:23:11.072706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347731+0000","last_change":"2024-04-05T22:23:09.370788+0000","last_active":"2024-04-05T22:23:10.347731+0000","last_peered":"2024-04-05T22:23:10.347731+0000","last_clean":"2024-04-05T22:23:10.347731+0000","last_became_active":"2024-04-05T22:23:09.370475+0000","last_became_peered":"2024-04-05T22:23:09.370475+0000","last_unstale":"2024-04-05T22:23:10.347731+0000","last_undegraded":"2024-04-05T22:23:10.347731+0000","last_fullsized":"2024-04-05T22:23:10.347731+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073323+0000","last_change":"2024-04-05T22:23:08.833154+0000","last_active":"2024-04-05T22:23:11.073323+0000","last_peered":"2024-04-05T22:23:11.073323+0000","last_clean":"2024-04-05T22:23:11.073323+0000","last_became_active":"2024-04-05T22:23:08.831734+0000","last_became_peered":"2024-04-05T22:23:08.831734+0000","last_unstale":"2024-04-05T22:23:11.073323+0000","last_undegraded":"2024-04-05T22:23:11.073323+0000","last_fullsized":"2024-04-05T22:23:11.073323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:21.532272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349928+0000","last_change":"2024-04-05T22:23:09.368222+0000","last_active":"2024-04-05T22:23:10.349928+0000","last_peered":"2024-04-05T22:23:10.349928+0000","last_clean":"2024-04-05T22:23:10.349928+0000","last_became_active":"2024-04-05T22:23:09.367924+0000","last_became_peered":"2024-04-05T22:23:09.367924+0000","last_unstale":"2024-04-05T22:23:10.349928+0000","last_undegraded":"2024-04-05T22:23:10.349928+0000","last_fullsized":"2024-04-05T22:23:10.349928+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:57.812584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347148+0000","last_change":"2024-04-05T22:23:08.839971+0000","last_active":"2024-04-05T22:23:10.347148+0000","last_peered":"2024-04-05T22:23:10.347148+0000","last_clean":"2024-04-05T22:23:10.347148+0000","last_became_active":"2024-04-05T22:23:08.839707+0000","last_became_peered":"2024-04-05T22:23:08.839707+0000","last_unstale":"2024-04-05T22:23:10.347148+0000","last_undegraded":"2024-04-05T22:23:10.347148+0000","last_fullsized":"2024-04-05T22:23:10.347148+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347367+0000","last_change":"2024-04-05T22:23:09.368953+0000","last_active":"2024-04-05T22:23:10.347367+0000","last_peered":"2024-04-05T22:23:10.347367+0000","last_clean":"2024-04-05T22:23:10.347367+0000","last_became_active":"2024-04-05T22:23:09.367313+0000","last_became_peered":"2024-04-05T22:23:09.367313+0000","last_unstale":"2024-04-05T22:23:10.347367+0000","last_undegraded":"2024-04-05T22:23:10.347367+0000","last_fullsized":"2024-04-05T22:23:10.347367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:57.687796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072503+0000","last_change":"2024-04-05T22:23:08.825046+0000","last_active":"2024-04-05T22:23:11.072503+0000","last_peered":"2024-04-05T22:23:11.072503+0000","last_clean":"2024-04-05T22:23:11.072503+0000","last_became_active":"2024-04-05T22:23:08.824750+0000","last_became_peered":"2024-04-05T22:23:08.824750+0000","last_unstale":"2024-04-05T22:23:11.072503+0000","last_undegraded":"2024-04-05T22:23:11.072503+0000","last_fullsized":"2024-04-05T22:23:11.072503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:07.006045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071685+0000","last_change":"2024-04-05T22:23:09.367072+0000","last_active":"2024-04-05T22:23:11.071685+0000","last_peered":"2024-04-05T22:23:11.071685+0000","last_clean":"2024-04-05T22:23:11.071685+0000","last_became_active":"2024-04-05T22:23:09.366737+0000","last_became_peered":"2024-04-05T22:23:09.366737+0000","last_unstale":"2024-04-05T22:23:11.071685+0000","last_undegraded":"2024-04-05T22:23:11.071685+0000","last_fullsized":"2024-04-05T22:23:11.071685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:13.331331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350234+0000","last_change":"2024-04-05T22:23:08.840066+0000","last_active":"2024-04-05T22:23:10.350234+0000","last_peered":"2024-04-05T22:23:10.350234+0000","last_clean":"2024-04-05T22:23:10.350234+0000","last_became_active":"2024-04-05T22:23:08.839794+0000","last_became_peered":"2024-04-05T22:23:08.839794+0000","last_unstale":"2024-04-05T22:23:10.350234+0000","last_undegraded":"2024-04-05T22:23:10.350234+0000","last_fullsized":"2024-04-05T22:23:10.350234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:39.221256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072213+0000","last_change":"2024-04-05T22:23:09.370990+0000","last_active":"2024-04-05T22:23:11.072213+0000","last_peered":"2024-04-05T22:23:11.072213+0000","last_clean":"2024-04-05T22:23:11.072213+0000","last_became_active":"2024-04-05T22:23:09.370739+0000","last_became_peered":"2024-04-05T22:23:09.370739+0000","last_unstale":"2024-04-05T22:23:11.072213+0000","last_undegraded":"2024-04-05T22:23:11.072213+0000","last_fullsized":"2024-04-05T22:23:11.072213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:10:33.459328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347073+0000","last_change":"2024-04-05T22:23:08.831972+0000","last_active":"2024-04-05T22:23:10.347073+0000","last_peered":"2024-04-05T22:23:10.347073+0000","last_clean":"2024-04-05T22:23:10.347073+0000","last_became_active":"2024-04-05T22:23:08.831650+0000","last_became_peered":"2024-04-05T22:23:08.831650+0000","last_unstale":"2024-04-05T22:23:10.347073+0000","last_undegraded":"2024-04-05T22:23:10.347073+0000","last_fullsized":"2024-04-05T22:23:10.347073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:52:24.125437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.348862+0000","last_change":"2024-04-05T22:23:09.364012+0000","last_active":"2024-04-05T22:23:10.348862+0000","last_peered":"2024-04-05T22:23:10.348862+0000","last_clean":"2024-04-05T22:23:10.348862+0000","last_became_active":"2024-04-05T22:23:09.363768+0000","last_became_peered":"2024-04-05T22:23:09.363768+0000","last_unstale":"2024-04-05T22:23:10.348862+0000","last_undegraded":"2024-04-05T22:23:10.348862+0000","last_fullsized":"2024-04-05T22:23:10.348862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:30.810750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349521+0000","last_change":"2024-04-05T22:23:08.840058+0000","last_active":"2024-04-05T22:23:10.349521+0000","last_peered":"2024-04-05T22:23:10.349521+0000","last_clean":"2024-04-05T22:23:10.349521+0000","last_became_active":"2024-04-05T22:23:08.839727+0000","last_became_peered":"2024-04-05T22:23:08.839727+0000","last_unstale":"2024-04-05T22:23:10.349521+0000","last_undegraded":"2024-04-05T22:23:10.349521+0000","last_fullsized":"2024-04-05T22:23:10.349521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:56.777874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347343+0000","last_change":"2024-04-05T22:23:09.362618+0000","last_active":"2024-04-05T22:23:10.347343+0000","last_peered":"2024-04-05T22:23:10.347343+0000","last_clean":"2024-04-05T22:23:10.347343+0000","last_became_active":"2024-04-05T22:23:09.362358+0000","last_became_peered":"2024-04-05T22:23:09.362358+0000","last_unstale":"2024-04-05T22:23:10.347343+0000","last_undegraded":"2024-04-05T22:23:10.347343+0000","last_fullsized":"2024-04-05T22:23:10.347343+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072045+0000","last_change":"2024-04-05T22:23:08.839690+0000","last_active":"2024-04-05T22:23:11.072045+0000","last_peered":"2024-04-05T22:23:11.072045+0000","last_clean":"2024-04-05T22:23:11.072045+0000","last_became_active":"2024-04-05T22:23:08.838945+0000","last_became_peered":"2024-04-05T22:23:08.838945+0000","last_unstale":"2024-04-05T22:23:11.072045+0000","last_undegraded":"2024-04-05T22:23:11.072045+0000","last_fullsized":"2024-04-05T22:23:11.072045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:19:04.891763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072339+0000","last_change":"2024-04-05T22:23:09.352156+0000","last_active":"2024-04-05T22:23:11.072339+0000","last_peered":"2024-04-05T22:23:11.072339+0000","last_clean":"2024-04-05T22:23:11.072339+0000","last_became_active":"2024-04-05T22:23:09.351439+0000","last_became_peered":"2024-04-05T22:23:09.351439+0000","last_unstale":"2024-04-05T22:23:11.072339+0000","last_undegraded":"2024-04-05T22:23:11.072339+0000","last_fullsized":"2024-04-05T22:23:11.072339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:08:22.537256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072786+0000","last_change":"2024-04-05T22:23:08.828030+0000","last_active":"2024-04-05T22:23:11.072786+0000","last_peered":"2024-04-05T22:23:11.072786+0000","last_clean":"2024-04-05T22:23:11.072786+0000","last_became_active":"2024-04-05T22:23:08.827689+0000","last_became_peered":"2024-04-05T22:23:08.827689+0000","last_unstale":"2024-04-05T22:23:11.072786+0000","last_undegraded":"2024-04-05T22:23:11.072786+0000","last_fullsized":"2024-04-05T22:23:11.072786+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:38:36.825290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072998+0000","last_change":"2024-04-05T22:23:09.374497+0000","last_active":"2024-04-05T22:23:11.072998+0000","last_peered":"2024-04-05T22:23:11.072998+0000","last_clean":"2024-04-05T22:23:11.072998+0000","last_became_active":"2024-04-05T22:23:09.374239+0000","last_became_peered":"2024-04-05T22:23:09.374239+0000","last_unstale":"2024-04-05T22:23:11.072998+0000","last_undegraded":"2024-04-05T22:23:11.072998+0000","last_fullsized":"2024-04-05T22:23:11.072998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:07:24.621644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073462+0000","last_change":"2024-04-05T22:23:08.822162+0000","last_active":"2024-04-05T22:23:11.073462+0000","last_peered":"2024-04-05T22:23:11.073462+0000","last_clean":"2024-04-05T22:23:11.073462+0000","last_became_active":"2024-04-05T22:23:08.821738+0000","last_became_peered":"2024-04-05T22:23:08.821738+0000","last_unstale":"2024-04-05T22:23:11.073462+0000","last_undegraded":"2024-04-05T22:23:11.073462+0000","last_fullsized":"2024-04-05T22:23:11.073462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:35:05.837398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071741+0000","last_change":"2024-04-05T22:23:09.372463+0000","last_active":"2024-04-05T22:23:11.071741+0000","last_peered":"2024-04-05T22:23:11.071741+0000","last_clean":"2024-04-05T22:23:11.071741+0000","last_became_active":"2024-04-05T22:23:09.372140+0000","last_became_peered":"2024-04-05T22:23:09.372140+0000","last_unstale":"2024-04-05T22:23:11.071741+0000","last_undegraded":"2024-04-05T22:23:11.071741+0000","last_fullsized":"2024-04-05T22:23:11.071741+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:56:39.781215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487427+0000","last_change":"2024-04-05T22:23:08.833929+0000","last_active":"2024-04-05T22:23:10.487427+0000","last_peered":"2024-04-05T22:23:10.487427+0000","last_clean":"2024-04-05T22:23:10.487427+0000","last_became_active":"2024-04-05T22:23:08.831111+0000","last_became_peered":"2024-04-05T22:23:08.831111+0000","last_unstale":"2024-04-05T22:23:10.487427+0000","last_undegraded":"2024-04-05T22:23:10.487427+0000","last_fullsized":"2024-04-05T22:23:10.487427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:32.656034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072277+0000","last_change":"2024-04-05T22:23:09.365652+0000","last_active":"2024-04-05T22:23:11.072277+0000","last_peered":"2024-04-05T22:23:11.072277+0000","last_clean":"2024-04-05T22:23:11.072277+0000","last_became_active":"2024-04-05T22:23:09.364565+0000","last_became_peered":"2024-04-05T22:23:09.364565+0000","last_unstale":"2024-04-05T22:23:11.072277+0000","last_undegraded":"2024-04-05T22:23:11.072277+0000","last_fullsized":"2024-04-05T22:23:11.072277+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:51.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350248+0000","last_change":"2024-04-05T22:23:08.841687+0000","last_active":"2024-04-05T22:23:10.350248+0000","last_peered":"2024-04-05T22:23:10.350248+0000","last_clean":"2024-04-05T22:23:10.350248+0000","last_became_active":"2024-04-05T22:23:08.841487+0000","last_became_peered":"2024-04-05T22:23:08.841487+0000","last_unstale":"2024-04-05T22:23:10.350248+0000","last_undegraded":"2024-04-05T22:23:10.350248+0000","last_fullsized":"2024-04-05T22:23:10.350248+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:35:48.140463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349783+0000","last_change":"2024-04-05T22:23:09.368685+0000","last_active":"2024-04-05T22:23:10.349783+0000","last_peered":"2024-04-05T22:23:10.349783+0000","last_clean":"2024-04-05T22:23:10.349783+0000","last_became_active":"2024-04-05T22:23:09.368451+0000","last_became_peered":"2024-04-05T22:23:09.368451+0000","last_unstale":"2024-04-05T22:23:10.349783+0000","last_undegraded":"2024-04-05T22:23:10.349783+0000","last_fullsized":"2024-04-05T22:23:10.349783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:11.976720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.350456+0000","last_change":"2024-04-05T22:23:08.838367+0000","last_active":"2024-04-05T22:23:10.350456+0000","last_peered":"2024-04-05T22:23:10.350456+0000","last_clean":"2024-04-05T22:23:10.350456+0000","last_became_active":"2024-04-05T22:23:08.837504+0000","last_became_peered":"2024-04-05T22:23:08.837504+0000","last_unstale":"2024-04-05T22:23:10.350456+0000","last_undegraded":"2024-04-05T22:23:10.350456+0000","last_fullsized":"2024-04-05T22:23:10.350456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:06:56.682813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346632+0000","last_change":"2024-04-05T22:23:09.371368+0000","last_active":"2024-04-05T22:23:10.346632+0000","last_peered":"2024-04-05T22:23:10.346632+0000","last_clean":"2024-04-05T22:23:10.346632+0000","last_became_active":"2024-04-05T22:23:09.369500+0000","last_became_peered":"2024-04-05T22:23:09.369500+0000","last_unstale":"2024-04-05T22:23:10.346632+0000","last_undegraded":"2024-04-05T22:23:10.346632+0000","last_fullsized":"2024-04-05T22:23:10.346632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:50:40.226240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487033+0000","last_change":"2024-04-05T22:23:08.838608+0000","last_active":"2024-04-05T22:23:10.487033+0000","last_peered":"2024-04-05T22:23:10.487033+0000","last_clean":"2024-04-05T22:23:10.487033+0000","last_became_active":"2024-04-05T22:23:08.838196+0000","last_became_peered":"2024-04-05T22:23:08.838196+0000","last_unstale":"2024-04-05T22:23:10.487033+0000","last_undegraded":"2024-04-05T22:23:10.487033+0000","last_fullsized":"2024-04-05T22:23:10.487033+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.070603+0000","last_change":"2024-04-05T22:23:09.368050+0000","last_active":"2024-04-05T22:23:11.070603+0000","last_peered":"2024-04-05T22:23:11.070603+0000","last_clean":"2024-04-05T22:23:11.070603+0000","last_became_active":"2024-04-05T22:23:09.367340+0000","last_became_peered":"2024-04-05T22:23:09.367340+0000","last_unstale":"2024-04-05T22:23:11.070603+0000","last_undegraded":"2024-04-05T22:23:11.070603+0000","last_fullsized":"2024-04-05T22:23:11.070603+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:10.015015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071791+0000","last_change":"2024-04-05T22:23:08.832053+0000","last_active":"2024-04-05T22:23:11.071791+0000","last_peered":"2024-04-05T22:23:11.071791+0000","last_clean":"2024-04-05T22:23:11.071791+0000","last_became_active":"2024-04-05T22:23:08.831669+0000","last_became_peered":"2024-04-05T22:23:08.831669+0000","last_unstale":"2024-04-05T22:23:11.071791+0000","last_undegraded":"2024-04-05T22:23:11.071791+0000","last_fullsized":"2024-04-05T22:23:11.071791+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:10:17.304924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071638+0000","last_change":"2024-04-05T22:23:09.362024+0000","last_active":"2024-04-05T22:23:11.071638+0000","last_peered":"2024-04-05T22:23:11.071638+0000","last_clean":"2024-04-05T22:23:11.071638+0000","last_became_active":"2024-04-05T22:23:09.361762+0000","last_became_peered":"2024-04-05T22:23:09.361762+0000","last_unstale":"2024-04-05T22:23:11.071638+0000","last_undegraded":"2024-04-05T22:23:11.071638+0000","last_fullsized":"2024-04-05T22:23:11.071638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:17.068648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346491+0000","last_change":"2024-04-05T22:23:08.831835+0000","last_active":"2024-04-05T22:23:10.346491+0000","last_peered":"2024-04-05T22:23:10.346491+0000","last_clean":"2024-04-05T22:23:10.346491+0000","last_became_active":"2024-04-05T22:23:08.831526+0000","last_became_peered":"2024-04-05T22:23:08.831526+0000","last_unstale":"2024-04-05T22:23:10.346491+0000","last_undegraded":"2024-04-05T22:23:10.346491+0000","last_fullsized":"2024-04-05T22:23:10.346491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:02:09.010032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073313+0000","last_change":"2024-04-05T22:23:09.361600+0000","last_active":"2024-04-05T22:23:11.073313+0000","last_peered":"2024-04-05T22:23:11.073313+0000","last_clean":"2024-04-05T22:23:11.073313+0000","last_became_active":"2024-04-05T22:23:09.361317+0000","last_became_peered":"2024-04-05T22:23:09.361317+0000","last_unstale":"2024-04-05T22:23:11.073313+0000","last_undegraded":"2024-04-05T22:23:11.073313+0000","last_fullsized":"2024-04-05T22:23:11.073313+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:07.080539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:12.649115+0000","last_change":"2024-04-05T22:23:08.835573+0000","last_active":"2024-04-05T22:23:12.649115+0000","last_peered":"2024-04-05T22:23:12.649115+0000","last_clean":"2024-04-05T22:23:12.649115+0000","last_became_active":"2024-04-05T22:23:08.835334+0000","last_became_peered":"2024-04-05T22:23:08.835334+0000","last_unstale":"2024-04-05T22:23:12.649115+0000","last_undegraded":"2024-04-05T22:23:12.649115+0000","last_fullsized":"2024-04-05T22:23:12.649115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:16.063665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072567+0000","last_change":"2024-04-05T22:23:09.369695+0000","last_active":"2024-04-05T22:23:11.072567+0000","last_peered":"2024-04-05T22:23:11.072567+0000","last_clean":"2024-04-05T22:23:11.072567+0000","last_became_active":"2024-04-05T22:23:09.366626+0000","last_became_peered":"2024-04-05T22:23:09.366626+0000","last_unstale":"2024-04-05T22:23:11.072567+0000","last_undegraded":"2024-04-05T22:23:11.072567+0000","last_fullsized":"2024-04-05T22:23:11.072567+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.906171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347294+0000","last_change":"2024-04-05T22:23:08.834972+0000","last_active":"2024-04-05T22:23:10.347294+0000","last_peered":"2024-04-05T22:23:10.347294+0000","last_clean":"2024-04-05T22:23:10.347294+0000","last_became_active":"2024-04-05T22:23:08.834348+0000","last_became_peered":"2024-04-05T22:23:08.834348+0000","last_unstale":"2024-04-05T22:23:10.347294+0000","last_undegraded":"2024-04-05T22:23:10.347294+0000","last_fullsized":"2024-04-05T22:23:10.347294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:47.731238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346985+0000","last_change":"2024-04-05T22:23:09.362042+0000","last_active":"2024-04-05T22:23:10.346985+0000","last_peered":"2024-04-05T22:23:10.346985+0000","last_clean":"2024-04-05T22:23:10.346985+0000","last_became_active":"2024-04-05T22:23:09.360332+0000","last_became_peered":"2024-04-05T22:23:09.360332+0000","last_unstale":"2024-04-05T22:23:10.346985+0000","last_undegraded":"2024-04-05T22:23:10.346985+0000","last_fullsized":"2024-04-05T22:23:10.346985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346736+0000","last_change":"2024-04-05T22:23:08.834808+0000","last_active":"2024-04-05T22:23:10.346736+0000","last_peered":"2024-04-05T22:23:10.346736+0000","last_clean":"2024-04-05T22:23:10.346736+0000","last_became_active":"2024-04-05T22:23:08.834540+0000","last_became_peered":"2024-04-05T22:23:08.834540+0000","last_unstale":"2024-04-05T22:23:10.346736+0000","last_undegraded":"2024-04-05T22:23:10.346736+0000","last_fullsized":"2024-04-05T22:23:10.346736+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:55:54.346606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349819+0000","last_change":"2024-04-05T22:23:09.368779+0000","last_active":"2024-04-05T22:23:10.349819+0000","last_peered":"2024-04-05T22:23:10.349819+0000","last_clean":"2024-04-05T22:23:10.349819+0000","last_became_active":"2024-04-05T22:23:09.368525+0000","last_became_peered":"2024-04-05T22:23:09.368525+0000","last_unstale":"2024-04-05T22:23:10.349819+0000","last_undegraded":"2024-04-05T22:23:10.349819+0000","last_fullsized":"2024-04-05T22:23:10.349819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:55.756624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.347079+0000","last_change":"2024-04-05T22:23:08.837346+0000","last_active":"2024-04-05T22:23:10.347079+0000","last_peered":"2024-04-05T22:23:10.347079+0000","last_clean":"2024-04-05T22:23:10.347079+0000","last_became_active":"2024-04-05T22:23:08.834887+0000","last_became_peered":"2024-04-05T22:23:08.834887+0000","last_unstale":"2024-04-05T22:23:10.347079+0000","last_undegraded":"2024-04-05T22:23:10.347079+0000","last_fullsized":"2024-04-05T22:23:10.347079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:53.019125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.349798+0000","last_change":"2024-04-05T22:23:09.372281+0000","last_active":"2024-04-05T22:23:10.349798+0000","last_peered":"2024-04-05T22:23:10.349798+0000","last_clean":"2024-04-05T22:23:10.349798+0000","last_became_active":"2024-04-05T22:23:09.371720+0000","last_became_peered":"2024-04-05T22:23:09.371720+0000","last_unstale":"2024-04-05T22:23:10.349798+0000","last_undegraded":"2024-04-05T22:23:10.349798+0000","last_fullsized":"2024-04-05T22:23:10.349798+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:16:32.033132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072619+0000","last_change":"2024-04-05T22:23:08.831235+0000","last_active":"2024-04-05T22:23:11.072619+0000","last_peered":"2024-04-05T22:23:11.072619+0000","last_clean":"2024-04-05T22:23:11.072619+0000","last_became_active":"2024-04-05T22:23:08.830890+0000","last_became_peered":"2024-04-05T22:23:08.830890+0000","last_unstale":"2024-04-05T22:23:11.072619+0000","last_undegraded":"2024-04-05T22:23:11.072619+0000","last_fullsized":"2024-04-05T22:23:11.072619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:28:35.503122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487316+0000","last_change":"2024-04-05T22:23:09.361959+0000","last_active":"2024-04-05T22:23:10.487316+0000","last_peered":"2024-04-05T22:23:10.487316+0000","last_clean":"2024-04-05T22:23:10.487316+0000","last_became_active":"2024-04-05T22:23:09.357075+0000","last_became_peered":"2024-04-05T22:23:09.357075+0000","last_unstale":"2024-04-05T22:23:10.487316+0000","last_undegraded":"2024-04-05T22:23:10.487316+0000","last_fullsized":"2024-04-05T22:23:10.487316+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:34:46.106269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346877+0000","last_change":"2024-04-05T22:23:08.838081+0000","last_active":"2024-04-05T22:23:10.346877+0000","last_peered":"2024-04-05T22:23:10.346877+0000","last_clean":"2024-04-05T22:23:10.346877+0000","last_became_active":"2024-04-05T22:23:08.837732+0000","last_became_peered":"2024-04-05T22:23:08.837732+0000","last_unstale":"2024-04-05T22:23:10.346877+0000","last_undegraded":"2024-04-05T22:23:10.346877+0000","last_fullsized":"2024-04-05T22:23:10.346877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:28:50.236391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.071714+0000","last_change":"2024-04-05T22:23:09.374296+0000","last_active":"2024-04-05T22:23:11.071714+0000","last_peered":"2024-04-05T22:23:11.071714+0000","last_clean":"2024-04-05T22:23:11.071714+0000","last_became_active":"2024-04-05T22:23:09.374031+0000","last_became_peered":"2024-04-05T22:23:09.374031+0000","last_unstale":"2024-04-05T22:23:11.071714+0000","last_undegraded":"2024-04-05T22:23:11.071714+0000","last_fullsized":"2024-04-05T22:23:11.071714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:23:54.024108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.072320+0000","last_change":"2024-04-05T22:23:08.831395+0000","last_active":"2024-04-05T22:23:11.072320+0000","last_peered":"2024-04-05T22:23:11.072320+0000","last_clean":"2024-04-05T22:23:11.072320+0000","last_became_active":"2024-04-05T22:23:08.831124+0000","last_became_peered":"2024-04-05T22:23:08.831124+0000","last_unstale":"2024-04-05T22:23:11.072320+0000","last_undegraded":"2024-04-05T22:23:11.072320+0000","last_fullsized":"2024-04-05T22:23:11.072320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:54.415069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:11.073173+0000","last_change":"2024-04-05T22:23:09.375678+0000","last_active":"2024-04-05T22:23:11.073173+0000","last_peered":"2024-04-05T22:23:11.073173+0000","last_clean":"2024-04-05T22:23:11.073173+0000","last_became_active":"2024-04-05T22:23:09.375368+0000","last_became_peered":"2024-04-05T22:23:09.375368+0000","last_unstale":"2024-04-05T22:23:11.073173+0000","last_undegraded":"2024-04-05T22:23:11.073173+0000","last_fullsized":"2024-04-05T22:23:11.073173+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:08.792633+0000","last_clean_scrub_stamp":"2024-04-05T22:23:08.792633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:57.782857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.487546+0000","last_change":"2024-04-05T22:23:08.836443+0000","last_active":"2024-04-05T22:23:10.487546+0000","last_peered":"2024-04-05T22:23:10.487546+0000","last_clean":"2024-04-05T22:23:10.487546+0000","last_became_active":"2024-04-05T22:23:08.836065+0000","last_became_peered":"2024-04-05T22:23:08.836065+0000","last_unstale":"2024-04-05T22:23:10.487546+0000","last_undegraded":"2024-04-05T22:23:10.487546+0000","last_fullsized":"2024-04-05T22:23:10.487546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:23:07.787650+0000","last_clean_scrub_stamp":"2024-04-05T22:23:07.787650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:19:07.607575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-05T22:23:10.346513+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:23:10.346513+0000","last_peered":"2024-04-05T22:23:10.346513+0000","last_clean":"2024-04-05T22:23:10.346513+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:23:10.346513+0000","last_undegraded":"2024-04-05T22:23:10.346513+0000","last_fullsized":"2024-04-05T22:23:10.346513+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1159168,"data_stored":818980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.02,"15min":1.018},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.0069999999999999,"15min":0.99299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":1.0149999999999999},"min":{"1min":0.77700000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.054},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.179,"15min":1.173},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.224},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2110000000000001,"15min":1.2010000000000001},"min":{"1min":0.93200000000000005,"5min":0.75,"15min":0.75},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.462}]},{"osd":3,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.133,"15min":1.1140000000000001},"min":{"1min":0.95699999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5409999999999999,"5min":1.712,"15min":1.712},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2170000000000001,"15min":1.2150000000000001},"min":{"1min":0.90400000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4810000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.365}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.232,"15min":1.218},"min":{"1min":1.0309999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6950000000000001,"5min":1.865,"15min":1.865},"last":1.179},{"interface":"front","average":{"1min":1.341,"5min":1.325,"15min":1.323},"min":{"1min":0.98699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.7050000000000001,"5min":1.788,"15min":1.788},"last":1.504}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.2410000000000001,"15min":1.218},"min":{"1min":0.95999999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.8440000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.3600000000000001,"15min":1.343},"min":{"1min":1.1140000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.802,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.4019999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.47,"15min":1.448},"min":{"1min":1.3680000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.51,"15min":1.492},"min":{"1min":1.296,"5min":0.755,"15min":0.755},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.5720000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":229194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0529999999999999,"15min":1.0629999999999999},"min":{"1min":0.68100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.579,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.72099999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.242,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.6479999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.151,"15min":1.175},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.524,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.472},{"interface":"front","average":{"1min":1.081,"5min":1.1479999999999999,"15min":1.159},"min":{"1min":0.70799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4330000000000001,"5min":1.722,"15min":1.722},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.208,"15min":1.21},"min":{"1min":0.747,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.524,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.2070000000000001,"15min":1.218},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.484,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.181}]},{"osd":3,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.335,"15min":1.3540000000000001},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.52,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.411,"15min":1.427},"min":{"1min":0.89300000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7290000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.718}]},{"osd":4,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3320000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.776,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.5780000000000001,"15min":1.6160000000000001},"min":{"1min":0.80900000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.772,"5min":3.27,"15min":3.27},"last":1.157}]},{"osd":5,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.4550000000000001,"15min":1.498},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.665,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5640000000000001,"15min":1.591},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8380000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.571}]},{"osd":7,"last update":"Fri Apr 5 22:23:13 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.5680000000000001,"15min":1.603},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2599999999999998,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.51},{"interface":"front","average":{"1min":1.325,"5min":1.569,"15min":1.609},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6639999999999999,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.611}]}]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.73599999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.381,"5min":1.419,"15min":1.419},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5649999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.3660000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.149,"15min":1.161},"min":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.462,"5min":1.488,"15min":1.488},"last":1.397},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1419999999999999,"15min":1.1459999999999999},"min":{"1min":0.77400000000000002,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.633,"15min":1.633},"last":1.0940000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.161,"15min":1.149},"min":{"1min":0.77200000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.855,"5min":1.988,"15min":1.988},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.2350000000000001,"15min":1.2549999999999999},"min":{"1min":0.78000000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.403,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.327}]},{"osd":3,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.1879999999999999,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.016},{"interface":"front","average":{"1min":1.123,"5min":1.3460000000000001,"15min":1.383},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5629999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.474}]},{"osd":4,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":0.626,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0099999999999998,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.163},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6279999999999999,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.65700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7290000000000001,"5min":1.972,"15min":1.972},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.464,"5min":1.49,"15min":1.494},"min":{"1min":0.81799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.952,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.423}]},{"osd":7,"last update":"Fri Apr 5 22:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.4279999999999999,"15min":1.46},"min":{"1min":0.75800000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.554,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.28},{"interface":"front","average":{"1min":1.389,"5min":1.617,"15min":1.655},"min":{"1min":0.72099999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0609999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.2989999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":573440,"data_stored":230086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.99099999999999999,"15min":0.97899999999999998},"min":{"1min":0.746,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3260000000000001,"5min":1.405,"15min":1.405},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.153,"5min":1.0880000000000001,"15min":1.077},"min":{"1min":0.77900000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.01}]},{"osd":1,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1020000000000001,"15min":1.1160000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6379999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1499999999999999,"15min":1.169},"min":{"1min":0.68300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.425,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6930000000000001,"5min":1.974,"15min":1.974},"last":1.27},{"interface":"front","average":{"1min":1.145,"5min":1.1739999999999999,"15min":1.179},"min":{"1min":0.68200000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6439999999999999,"5min":1.673,"15min":1.673},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.383,"15min":1.387},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.863,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.7569999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2949999999999999,"15min":1.3129999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.579,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.393},{"interface":"front","average":{"1min":1.329,"5min":1.3999999999999999,"15min":1.4119999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.75,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.51}]},{"osd":6,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.4510000000000001,"15min":1.4850000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.7430000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.5129999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.986,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.605}]},{"osd":7,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5489999999999999},"min":{"1min":1.0600000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.911,"5min":2.54,"15min":2.54},"last":1.5680000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":229160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.435,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4119999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.90100000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4390000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.044}]},{"osd":1,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.27,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.88},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.5720000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.085}]},{"osd":2,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.999,"15min":1.0249999999999999},"min":{"1min":0.57399999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.1679999999999999,"5min":1.827,"15min":1.827},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.075,"15min":1.083},"min":{"1min":0.67000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6559999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.327}]},{"osd":4,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2310000000000001,"15min":1.2549999999999999},"min":{"1min":0.82299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.496,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.274},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.306,"15min":1.3340000000000001},"min":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.573,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.2330000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.361,"15min":1.391},"min":{"1min":0.87,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4219999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.476,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.397}]},{"osd":6,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9390000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.399,"15min":1.4370000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4890000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:23:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.4590000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7050000000000001,"5min":2.468,"15min":2.468},"last":1.502},{"interface":"front","average":{"1min":1.294,"5min":1.4179999999999999,"15min":1.4390000000000001},"min":{"1min":0.89400000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.0089999999999999,"5min":2.798,"15min":2.798},"last":1.607}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":230108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80900000000000005,"15min":0.82399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.147,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90000000000000002,"15min":0.90800000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.95599999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.85199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3340000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.877,"15min":0.876},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2130000000000001,"5min":1.292,"15min":1.292},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.03,"15min":1.03},"min":{"1min":0.621,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.468,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.113,"15min":1.119},"min":{"1min":0.629,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":2.694,"15min":2.694},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":1.0069999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.115},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2569999999999999,"15min":1.244},"min":{"1min":0.84999999999999998,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.726,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.0800000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.266,"15min":1.272},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.8879999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.272,"5min":1.284,"15min":1.286},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6699999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.048}]},{"osd":6,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2789999999999999,"15min":1.272},"min":{"1min":0.86399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6779999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.226},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2709999999999999,"15min":1.2669999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.764,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Apr 5 22:23:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.754,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.962,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.409,"5min":1.397,"15min":1.395},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.8,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1459999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":1.242,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82999999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.98399999999999999,"15min":0.995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.093,"15min":1.105},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":1.5509999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.94199999999999995,"15min":0.95799999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.4770000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.179}]},{"osd":4,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1220000000000001,"15min":1.1259999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.698,"5min":1.855,"15min":1.855},"last":1.349},{"interface":"front","average":{"1min":1.198,"5min":1.2689999999999999,"15min":1.2809999999999999},"min":{"1min":0.747,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.534,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2230000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2569999999999999,"15min":1.2669999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.841,"5min":1.887,"15min":1.887},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.70499999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.849,"5min":2.173,"15min":2.173},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.389},{"interface":"front","average":{"1min":1.175,"5min":1.339,"15min":1.367},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.736}]},{"osd":7,"last update":"Fri Apr 5 22:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.341,"15min":1.357},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9359999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.5}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1159168,"data_stored":818924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.179,"5min":1.367,"15min":1.367},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.39500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4590000000000001,"5min":1.669,"15min":1.669},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.514,"15min":1.514},"last":1.115},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96099999999999997,"15min":0.95899999999999996},"min":{"1min":0.49399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.6040000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2949999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.90900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.506,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0840000000000001,"15min":1.1020000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1579999999999999,"15min":1.1539999999999999},"min":{"1min":0.747,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5960000000000001,"5min":1.929,"15min":1.929},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6519999999999999,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.5669999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6559999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.169},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.196,"15min":1.1899999999999999},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.526,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2270000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.784,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.663},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7110000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.494}]},{"osd":7,"last update":"Fri Apr 5 22:23:08 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.339,"15min":1.339},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7509999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.3120000000000001,"15min":1.331},"min":{"1min":0.75,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6619999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.534}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:23:19.274 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:23:19.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:23:19.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.531+0000 7fef18000640 1 Processor -- start 2024-04-05T22:23:19.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.532+0000 7fef18000640 1 -- start start 2024-04-05T22:23:19.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.533+0000 7fef18000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 0x7fef1006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.533+0000 7fef18000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef100b32b0 0x7fef101647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.533+0000 7fef18000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100b1b30 con 0x7fef100cbe00 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.533+0000 7fef18000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef100b1e10 con 0x7fef100b32b0 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.533+0000 7fef18000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100ce7b0 con 0x7fef1006c630 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.534+0000 7fef15600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 0x7fef1006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.534+0000 7fef14c00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fef100cbe00 0x7fef100ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:36476/0 (socket says 172.21.15.73:36476) 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.534+0000 7fef14c00640 1 -- 172.21.15.73:0/3400928441 learned_addr learned my addr 172.21.15.73:0/3400928441 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.534+0000 7fef16000640 1 --2- 172.21.15.73:0/3400928441 >> v2:172.21.15.157:3300/0 conn(0x7fef100b32b0 0x7fef101647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.535+0000 7fef15600640 1 -- 172.21.15.73:0/3400928441 >> v2:172.21.15.157:3300/0 conn(0x7fef100b32b0 msgr2=0x7fef101647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.535+0000 7fef15600640 1 --2- 172.21.15.73:0/3400928441 >> v2:172.21.15.157:3300/0 conn(0x7fef100b32b0 0x7fef101647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.535+0000 7fef15600640 1 -- 172.21.15.73:0/3400928441 >> v1:172.21.15.157:6789/0 conn(0x7fef100cbe00 legacy=0x7fef100ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.535+0000 7fef15600640 1 -- 172.21.15.73:0/3400928441 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1006cf50 con 0x7fef1006c630 2024-04-05T22:23:19.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.537+0000 7fef15600640 1 --2- 172.21.15.73:0/3400928441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 0x7fef1006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef040097a0 tx=0x7fef0402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56762632f3b48307 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.537+0000 7fef0fe00640 1 -- 172.21.15.73:0/3400928441 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04031830 con 0x7fef1006c630 2024-04-05T22:23:19.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.537+0000 7fef0fe00640 1 -- 172.21.15.73:0/3400928441 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef04031990 con 0x7fef1006c630 2024-04-05T22:23:19.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.537+0000 7fef0fe00640 1 -- 172.21.15.73:0/3400928441 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04031d20 con 0x7fef1006c630 2024-04-05T22:23:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 -- 172.21.15.73:0/3400928441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 msgr2=0x7fef1006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 --2- 172.21.15.73:0/3400928441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 0x7fef1006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef040097a0 tx=0x7fef0402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 -- 172.21.15.73:0/3400928441 shutdown_connections 2024-04-05T22:23:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 --2- 172.21.15.73:0/3400928441 >> v2:172.21.15.157:3300/0 conn(0x7fef100b32b0 0x7fef101647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 --2- 172.21.15.73:0/3400928441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef1006c630 0x7fef1006ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 -- 172.21.15.73:0/3400928441 >> 172.21.15.73:0/3400928441 conn(0x7fef100b0830 msgr2=0x7fef100b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 -- 172.21.15.73:0/3400928441 shutdown_connections 2024-04-05T22:23:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.538+0000 7fef18000640 1 -- 172.21.15.73:0/3400928441 wait complete. 2024-04-05T22:23:19.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.540+0000 7fef18000640 1 Processor -- start 2024-04-05T22:23:19.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.540+0000 7fef18000640 1 -- start start 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef100b32b0 0x7fef100ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 0x7fef100b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10130370 con 0x7fef1006c630 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101305c0 con 0x7fef100cbe00 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef18000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100ad100 con 0x7fef100b32b0 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef15600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef15600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34290/0 (socket says 172.21.15.73:34290) 2024-04-05T22:23:19.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef15600640 1 -- 172.21.15.73:0/193224606 learned_addr learned my addr 172.21.15.73:0/193224606 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef16000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 0x7fef100b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.541+0000 7fef14c00640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef100b32b0 0x7fef100ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 -- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef100b32b0 msgr2=0x7fef100ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef100b32b0 0x7fef100ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 -- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 msgr2=0x7fef100b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 0x7fef100b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 -- 172.21.15.73:0/193224606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04009400 con 0x7fef1006c630 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef16000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 0x7fef100b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:19.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef15600640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fef0402bee0 tx=0x7fef040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0403b9f0 con 0x7fef1006c630 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0403bb70 con 0x7fef1006c630 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040453f0 con 0x7fef1006c630 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef100ad3e0 con 0x7fef1006c630 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.542+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef10130810 con 0x7fef1006c630 2024-04-05T22:23:19.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.544+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef100b0c10 con 0x7fef1006c630 2024-04-05T22:23:19.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.545+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fef0404e450 con 0x7fef1006c630 2024-04-05T22:23:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.545+0000 7fef0d600640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 0x7feef006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.546+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef040ab950 con 0x7fef1006c630 2024-04-05T22:23:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.546+0000 7fef14c00640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 0x7feef006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:19.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.546+0000 7fef14c00640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 0x7feef006d270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef1013fb70 tx=0x7feefc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:19.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.550+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fef04076790 con 0x7fef1006c630 2024-04-05T22:23:19.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.812+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fef10068160 con 0x7fef1006c630 2024-04-05T22:23:19.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.819+0000 7fef0d600640 1 -- 172.21.15.73:0/193224606 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fef040350e0 con 0x7fef1006c630 2024-04-05T22:23:19.820 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 msgr2=0x7feef006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 0x7feef006d270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef1013fb70 tx=0x7feefc009040 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 msgr2=0x7fef1012fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fef0402bee0 tx=0x7fef040046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 shutdown_connections 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006adb0 0x7feef006d270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef100cbe00 0x7fef100b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef100b32b0 0x7fef100ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 --2- 172.21.15.73:0/193224606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef1006c630 0x7fef1012fe30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.822+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 >> 172.21.15.73:0/193224606 conn(0x7fef100b0830 msgr2=0x7fef10072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.823+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 shutdown_connections 2024-04-05T22:23:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:19.823+0000 7fef18000640 1 -- 172.21.15.73:0/193224606 wait complete. 2024-04-05T22:23:19.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:20.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.046+0000 7f38fa600640 1 Processor -- start 2024-04-05T22:23:20.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.047+0000 7f38fa600640 1 -- start start 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.048+0000 7f38fa600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38f41587d0 0x7f38f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.048+0000 7f38fa600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f406f5b0 con 0x7f38f415e850 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.048+0000 7f38fa600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f4070410 con 0x7f38f40ccd20 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.048+0000 7f38fa600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f4158bb0 con 0x7f38f41587d0 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38f41587d0 0x7f38f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38f41587d0 0x7f38f40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34296/0 (socket says 172.21.15.73:34296) 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 -- 172.21.15.73:0/1899221780 learned_addr learned my addr 172.21.15.73:0/1899221780 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3e00640 1 --1- 172.21.15.73:0/1899221780 >> v1:172.21.15.73:6789/0 conn(0x7f38f415e850 0x7f38f40708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33112/0 (socket says 172.21.15.73:33112) 2024-04-05T22:23:20.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 -- 172.21.15.73:0/1899221780 >> v1:172.21.15.157:6789/0 conn(0x7f38f40ccd20 legacy=0x7f38f406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 -- 172.21.15.73:0/1899221780 >> v1:172.21.15.73:6789/0 conn(0x7f38f415e850 legacy=0x7f38f40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.049+0000 7f38f3400640 1 -- 172.21.15.73:0/1899221780 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f406f800 con 0x7f38f41587d0 2024-04-05T22:23:20.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.050+0000 7f38f3400640 1 --2- 172.21.15.73:0/1899221780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f41587d0 0x7f38f40cc7e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f38e40098a0 tx=0x7f38e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b25d7c2168d418 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38f2a00640 1 -- 172.21.15.73:0/1899221780 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4030840 con 0x7f38f41587d0 2024-04-05T22:23:20.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38f2a00640 1 -- 172.21.15.73:0/1899221780 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e40309a0 con 0x7f38f41587d0 2024-04-05T22:23:20.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38f2a00640 1 -- 172.21.15.73:0/1899221780 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4030d30 con 0x7f38f41587d0 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38fa600640 1 -- 172.21.15.73:0/1899221780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f41587d0 msgr2=0x7f38f40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38fa600640 1 --2- 172.21.15.73:0/1899221780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f41587d0 0x7f38f40cc7e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f38e40098a0 tx=0x7f38e402b980 comp rx=0 tx=0).stop 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38fa600640 1 -- 172.21.15.73:0/1899221780 shutdown_connections 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38fa600640 1 --2- 172.21.15.73:0/1899221780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f41587d0 0x7f38f40cc7e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.051+0000 7f38fa600640 1 -- 172.21.15.73:0/1899221780 >> 172.21.15.73:0/1899221780 conn(0x7f38f40ce8e0 msgr2=0x7f38f40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.052+0000 7f38fa600640 1 -- 172.21.15.73:0/1899221780 shutdown_connections 2024-04-05T22:23:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.052+0000 7f38fa600640 1 -- 172.21.15.73:0/1899221780 wait complete. 2024-04-05T22:23:20.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.053+0000 7f38fa600640 1 Processor -- start 2024-04-05T22:23:20.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.053+0000 7f38fa600640 1 -- start start 2024-04-05T22:23:20.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f41587d0 0x7f38f4143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 0x7f38f41465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40afa50 con 0x7f38f415e850 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4144530 con 0x7f38f40af5a0 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.054+0000 7f38fa600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40a84c0 con 0x7f38f41587d0 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34310/0 (socket says 172.21.15.73:34310) 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 -- 172.21.15.73:0/3570502271 learned_addr learned my addr 172.21.15.73:0/3570502271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38f3e00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 0x7f38f41465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38f3400640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f41587d0 0x7f38f4143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 -- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f41587d0 msgr2=0x7f38f4143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f41587d0 0x7f38f4143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 -- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 msgr2=0x7f38f41465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 0x7f38f41465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 -- 172.21.15.73:0/3570502271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40094d0 con 0x7f38f415e850 2024-04-05T22:23:20.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38f3e00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 0x7f38f41465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.055+0000 7f38eaa00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c460 tx=0x7f38dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.056+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc012830 con 0x7f38f415e850 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.056+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc0129d0 con 0x7f38f415e850 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.056+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc012d80 con 0x7f38f415e850 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.056+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4144200 con 0x7f38f415e850 2024-04-05T22:23:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.056+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f4144740 con 0x7f38f415e850 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.057+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f4066af0 con 0x7f38f415e850 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.058+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f38dc01b400 con 0x7f38f415e850 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.059+0000 7f38ebe00640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 0x7f38d006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.059+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38dc089680 con 0x7f38f415e850 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.059+0000 7f38f3400640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 0x7f38d006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.060+0000 7f38f3400640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 0x7f38d006d2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f38e4036000 tx=0x7f38e403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.063+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f38dc054540 con 0x7f38f415e850 2024-04-05T22:23:20.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.289+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38f4068160 con 0x7f38f415e850 2024-04-05T22:23:20.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.290+0000 7f38ebe00640 1 -- 172.21.15.73:0/3570502271 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f38dc057b60 con 0x7f38f415e850 2024-04-05T22:23:20.291 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:20.291 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:20.291 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 23 2024-04-05T22:23:20.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.292+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 msgr2=0x7f38d006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.292+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 0x7f38d006d2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f38e4036000 tx=0x7f38e403b040 comp rx=0 tx=0).stop 2024-04-05T22:23:20.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.292+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 msgr2=0x7f38f40af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.292+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c460 tx=0x7f38dc00c920 comp rx=0 tx=0).stop 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 shutdown_connections 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ade0 0x7f38d006d2a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f40af5a0 0x7f38f41465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415e850 0x7f38f40af060 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 --2- 172.21.15.73:0/3570502271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f41587d0 0x7f38f4143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 >> 172.21.15.73:0/3570502271 conn(0x7f38f40ce8e0 msgr2=0x7f38f415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 shutdown_connections 2024-04-05T22:23:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.293+0000 7f38fa600640 1 -- 172.21.15.73:0/3570502271 wait complete. 2024-04-05T22:23:20.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:23:09.333638+0000', 'modified': '2024-04-05T22:23:12.354447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5634': {'gid': 5634, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/166143460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6833', 'nonce': 166143460}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 166143460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:23:20.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:23:20.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-05T22:23:20.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.934+0000 7fdb2f000640 1 Processor -- start 2024-04-05T22:23:20.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.934+0000 7fdb2f000640 1 -- start start 2024-04-05T22:23:20.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 0x7fdb280cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb280ccd20 0x7fdb2806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2806f5b0 con 0x7fdb2806d0c0 2024-04-05T22:23:20.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb28070410 con 0x7fdb280ccd20 2024-04-05T22:23:20.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28158bb0 con 0x7fdb281587d0 2024-04-05T22:23:20.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2d000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdb2806d0c0 0x7fdb2815e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33116/0 (socket says 172.21.15.73:33116) 2024-04-05T22:23:20.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.936+0000 7fdb2d000640 1 -- 172.21.15.73:0/2555035295 learned_addr learned my addr 172.21.15.73:0/2555035295 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2555035295 >> v2:172.21.15.157:3300/0 conn(0x7fdb280ccd20 0x7fdb2806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 0x7fdb280cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808238334 0 0) 0x7fdb2806f5b0 con 0x7fdb2806d0c0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb0c003520 con 0x7fdb2806d0c0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331833870 0 0) 0x7fdb0c003520 con 0x7fdb2806d0c0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 msgr2=0x7fdb280cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 0x7fdb280cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 >> v2:172.21.15.157:3300/0 conn(0x7fdb280ccd20 msgr2=0x7fdb2806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 --2- 172.21.15.73:0/2555035295 >> v2:172.21.15.157:3300/0 conn(0x7fdb280ccd20 0x7fdb2806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2806f800 con 0x7fdb2806d0c0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fdb200019f0 con 0x7fdb2806d0c0 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 >> v1:172.21.15.73:6789/0 conn(0x7fdb2806d0c0 legacy=0x7fdb2815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c003970 0x7fdb0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb0c0047c0 0x7fdb0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 0x7fdb0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28070410 con 0x7fdb0c0047c0 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28158bb0 con 0x7fdb0c003970 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.937+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c00ba30 con 0x7fdb281587d0 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb0c0047c0 0x7fdb0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c003970 0x7fdb0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 -- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 msgr2=0x7fdb0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb281587d0 0x7fdb0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 -- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c003970 msgr2=0x7fdb0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c003970 0x7fdb0c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.938+0000 7fdb1f400640 1 -- 172.21.15.73:0/2555035295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c003520 con 0x7fdb0c0047c0 2024-04-05T22:23:20.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.939+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb0c0047c0 0x7fdb0c004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000ba00 tx=0x7fdb0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4224dccb7ba53fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.939+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10032630 con 0x7fdb0c0047c0 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.939+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb10032790 con 0x7fdb0c0047c0 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.939+0000 7fdb27400640 1 -- 172.21.15.73:0/2555035295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10032b20 con 0x7fdb0c0047c0 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 -- 172.21.15.73:0/2555035295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb0c0047c0 msgr2=0x7fdb0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2555035295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb0c0047c0 0x7fdb0c004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000ba00 tx=0x7fdb0c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2d000640 1 -- 172.21.15.73:0/2555035295 reap_dead start 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 -- 172.21.15.73:0/2555035295 shutdown_connections 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 -- 172.21.15.73:0/2555035295 >> 172.21.15.73:0/2555035295 conn(0x7fdb280ce8e0 msgr2=0x7fdb280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 -- 172.21.15.73:0/2555035295 shutdown_connections 2024-04-05T22:23:20.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.940+0000 7fdb2f000640 1 -- 172.21.15.73:0/2555035295 wait complete. 2024-04-05T22:23:20.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.942+0000 7fdb2f000640 1 Processor -- start 2024-04-05T22:23:20.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.942+0000 7fdb2f000640 1 -- start start 2024-04-05T22:23:20.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 0x7fdb28148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 0x7fdb2815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2815dd50 con 0x7fdb2815b060 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2815bb20 con 0x7fdb0c0047c0 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2815be00 con 0x7fdb2815d390 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34324/0 (socket says 172.21.15.73:34324) 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb1f400640 1 -- 172.21.15.73:0/2070339458 learned_addr learned my addr 172.21.15.73:0/2070339458 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 0x7fdb28148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.943+0000 7fdb2d000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 0x7fdb2815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 -- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 msgr2=0x7fdb2815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 0x7fdb2815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 -- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 msgr2=0x7fdb28148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 0x7fdb28148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 -- 172.21.15.73:0/2070339458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2815c0e0 con 0x7fdb2815b060 2024-04-05T22:23:20.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 0x7fdb28148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb2d000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 0x7fdb2815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb1f400640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdb1002beb0 tx=0x7fdb10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb24c00640 1 -- 172.21.15.73:0/2070339458 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004220 con 0x7fdb2815b060 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb24c00640 1 -- 172.21.15.73:0/2070339458 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb100043a0 con 0x7fdb2815b060 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb24c00640 1 -- 172.21.15.73:0/2070339458 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1003ba70 con 0x7fdb2815b060 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2815c3c0 con 0x7fdb2815b060 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.944+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb28155e30 con 0x7fdb2815b060 2024-04-05T22:23:20.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.946+0000 7fdb24c00640 1 -- 172.21.15.73:0/2070339458 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdb1003bbd0 con 0x7fdb2815b060 2024-04-05T22:23:20.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.947+0000 7fdb24c00640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 0x7fdb0406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.948+0000 7fdb24c00640 1 -- 172.21.15.73:0/2070339458 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb100abff0 con 0x7fdb2815b060 2024-04-05T22:23:20.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.948+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 0x7fdb0406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.948+0000 7fdb31e66740 1 Processor -- start 2024-04-05T22:23:20.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.948+0000 7fdb27e00640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 0x7fdb0406d2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006fd0 tx=0x7fdb18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 -- start start 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8986d0 0x555aec8c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec8088c0 con 0x555aec8c51f0 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec806050 con 0x555aec8986d0 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.949+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x555aec801940 con 0x555aec8c55d0 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34332/0 (socket says 172.21.15.73:34332) 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 -- 172.21.15.73:0/4162074779 learned_addr learned my addr 172.21.15.73:0/4162074779 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 -- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 msgr2=0x555aec8c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 -- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8986d0 msgr2=0x555aec8c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb2d000640 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8986d0 0x555aec8c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8986d0 0x555aec8c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 -- 172.21.15.73:0/4162074779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aec7cc9e0 con 0x555aec8c51f0 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.950+0000 7fdb27e00640 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800f990 tx=0x7fdb18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645f9cc170dc03a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb1cc00640 1 -- 172.21.15.73:0/4162074779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800ddf0 con 0x555aec8c51f0 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb1cc00640 1 -- 172.21.15.73:0/4162074779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18019070 con 0x555aec8c51f0 2024-04-05T22:23:20.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb1cc00640 1 -- 172.21.15.73:0/4162074779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18014de0 con 0x555aec8c51f0 2024-04-05T22:23:20.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 -- 172.21.15.73:0/4162074779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 msgr2=0x555aec898190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800f990 tx=0x7fdb18012000 comp rx=0 tx=0).stop 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 -- 172.21.15.73:0/4162074779 shutdown_connections 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8986d0 0x555aec8c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 --2- 172.21.15.73:0/4162074779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c51f0 0x555aec898190 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.951+0000 7fdb31e66740 1 -- 172.21.15.73:0/4162074779 >> 172.21.15.73:0/4162074779 conn(0x555aec89a400 msgr2=0x555aec8e2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.952+0000 7fdb31e66740 1 -- 172.21.15.73:0/4162074779 shutdown_connections 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.952+0000 7fdb31e66740 1 -- 172.21.15.73:0/4162074779 wait complete. 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.952+0000 7fdb31e66740 1 -- start start 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.952+0000 7fdb31e66740 1 Processor -- start 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c51f0 0x555aec8dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec7a3070 con 0x555aec8c8f70 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec7a2d10 con 0x555aec8c51f0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x555aec568f20 con 0x555aec8c55d0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.953+0000 7fdb31e66740 10 client.0 _ll_register_callbacks cb 0x555aec7fafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34336/0 (socket says 172.21.15.73:34336) 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 -- 172.21.15.73:0/732880632 learned_addr learned my addr 172.21.15.73:0/732880632 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb1f400640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb27e00640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c51f0 0x555aec8dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 -- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 msgr2=0x555aec8cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 -- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c51f0 msgr2=0x555aec8dabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c51f0 0x555aec8dabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 -- 172.21.15.73:0/732880632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1800e040 con 0x555aec8c8f70 2024-04-05T22:23:20.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb2d000640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdb280667b0 tx=0x7fdb2000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94021937dbb8e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.954+0000 7fdb15600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20004d60 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb2000b680 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aec82bae0 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb20007ca0 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 10 client.5793 fetch_fsmap learned FSMap version 23 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555aec7fffc0 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555aec82bae0 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdb2000b840 con 0x555aec8c8f70 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 10 client.5793 fetch_fsmap finished waiting for FSMap version 23 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 10 client.5793 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 20 client.5793 populate_metadata read hostname 'smithi073' 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec8f5b50 0x555aec8f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 4 client.5793 mds_command: new command op to 5634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:23:20.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.955+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 --> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aec4f53c0 con 0x555aec8f5b50 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.956+0000 7fdb27e00640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec8f5b50 0x555aec8f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.957+0000 7fdb27e00640 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec8f5b50 0x555aec8f5f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e534ff305ff280f server_cookie=305f63134b46530e in_seq=0 out_seq=0 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.957+0000 7fdb15600640 10 client.5793 ms_handle_connect on v2:172.21.15.157:6833/166143460 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.958+0000 7fdb15600640 1 -- 172.21.15.73:0/732880632 <== mds.0 v2:172.21.15.157:6833/166143460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555aec4f53c0 con 0x555aec8f5b50 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.958+0000 7fdb15600640 10 client.5793 handle_command_reply: tid=0 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 client.5793 shutdown 2024-04-05T22:23:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 msgr2=0x555aec8c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdb280667b0 tx=0x7fdb2000db60 comp rx=0 tx=0).stop 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 shutdown_connections 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec8f5b50 0x555aec8f5f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8c93d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c55d0 0x555aec8cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 --2- 172.21.15.73:0/732880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c51f0 0x555aec8dabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb15600640 0 client.5793 ms_handle_reset on v2:172.21.15.157:6833/166143460 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 >> 172.21.15.73:0/732880632 conn(0x555aec8986d0 msgr2=0x555aec89c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 shutdown_connections 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 1 -- 172.21.15.73:0/732880632 wait complete. 2024-04-05T22:23:20.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.959+0000 7fdb31e66740 20 client.5793 trim_cache size 0 max 16384 2024-04-05T22:23:20.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.975+0000 7fdb31e66740 1 Processor -- start 2024-04-05T22:23:20.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.975+0000 7fdb31e66740 1 -- start start 2024-04-05T22:23:20.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec8c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 0x555aec8fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec8fc9e0 con 0x555aec8c8f70 2024-04-05T22:23:20.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec807a30 con 0x555aec8c55d0 2024-04-05T22:23:20.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.976+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x555aec8c82c0 con 0x555aec8c51f0 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34352/0 (socket says 172.21.15.73:34352) 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 -- 172.21.15.73:0/633148980 learned_addr learned my addr 172.21.15.73:0/633148980 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb27e00640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec8c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb1f400640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 0x555aec8fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 -- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 msgr2=0x555aec8c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec8c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 -- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 msgr2=0x555aec8fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 0x555aec8fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb2d000640 1 -- 172.21.15.73:0/633148980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aec834f80 con 0x555aec8c8f70 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.977+0000 7fdb1f400640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 0x555aec8fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb2d000640 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009010 tx=0x7fdb20005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd964ceb36fe2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb1d600640 1 -- 172.21.15.73:0/633148980 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000dea0 con 0x555aec8c8f70 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb1d600640 1 -- 172.21.15.73:0/633148980 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000bd50 con 0x555aec8c8f70 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb1d600640 1 -- 172.21.15.73:0/633148980 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20004b60 con 0x555aec8c8f70 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb31e66740 1 -- 172.21.15.73:0/633148980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 msgr2=0x555aec8fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb31e66740 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009010 tx=0x7fdb20005000 comp rx=0 tx=0).stop 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.978+0000 7fdb31e66740 1 -- 172.21.15.73:0/633148980 shutdown_connections 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c8f70 0x555aec8fc4a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c55d0 0x555aec8fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 --2- 172.21.15.73:0/633148980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec8c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 -- 172.21.15.73:0/633148980 >> 172.21.15.73:0/633148980 conn(0x555aec8986d0 msgr2=0x555aec8dc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 -- 172.21.15.73:0/633148980 shutdown_connections 2024-04-05T22:23:20.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 -- 172.21.15.73:0/633148980 wait complete. 2024-04-05T22:23:20.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 -- start start 2024-04-05T22:23:20.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.979+0000 7fdb31e66740 1 Processor -- start 2024-04-05T22:23:20.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec904cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c8f70 0x555aec8c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec8c22f0 con 0x555aec8c55d0 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aec897330 con 0x555aec8c8f70 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x555aec897610 con 0x555aec8c51f0 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.980+0000 7fdb31e66740 10 client.0 _ll_register_callbacks cb 0x7fdb2000d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34360/0 (socket says 172.21.15.73:34360) 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 -- 172.21.15.73:0/1639691143 learned_addr learned my addr 172.21.15.73:0/1639691143 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 -- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 msgr2=0x555aec904cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec904cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 -- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c8f70 msgr2=0x555aec8c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c8f70 0x555aec8c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 -- 172.21.15.73:0/1639691143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555aec7fffc0 con 0x555aec8c55d0 2024-04-05T22:23:20.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb1f400640 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000ba00 tx=0x7fdb1007a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e73055c4b80f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb15600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100aa640 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb1007abe0 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.981+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb18019070 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb1007ea10 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 10 client.5799 fetch_fsmap learned FSMap version 23 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555aec7a2280 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18019070 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdb1004c060 con 0x555aec8c55d0 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 10 client.5799 fetch_fsmap finished waiting for FSMap version 23 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 10 client.5799 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 20 client.5799 populate_metadata read hostname 'smithi073' 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec94aae0 0x555aec94aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:20.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 4 client.5799 mds_command: new command op to 5634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:23:20.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.982+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 --> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555aec5066c0 con 0x555aec94aae0 2024-04-05T22:23:20.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.983+0000 7fdb27e00640 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec94aae0 0x555aec94aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.984+0000 7fdb27e00640 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec94aae0 0x555aec94aee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c3ab73a94cfb1f server_cookie=3fa13660636dddf6 in_seq=0 out_seq=0 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.984+0000 7fdb15600640 10 client.5799 ms_handle_connect on v2:172.21.15.157:6833/166143460 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.984+0000 7fdb15600640 1 -- 172.21.15.73:0/1639691143 <== mds.0 v2:172.21.15.157:6833/166143460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555aec5066c0 con 0x555aec94aae0 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.984+0000 7fdb15600640 10 client.5799 handle_command_reply: tid=0 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.984+0000 7fdb31e66740 1 client.5799 shutdown 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 msgr2=0x555aec8c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000ba00 tx=0x7fdb1007a540 comp rx=0 tx=0).stop 2024-04-05T22:23:20.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 shutdown_connections 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555aec8c8f70 0x555aec8c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:6833/166143460,v1:172.21.15.157:6835/166143460] conn(0x555aec94aae0 0x555aec94aee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555aec8c55d0 0x555aec8c2780 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 --2- 172.21.15.73:0/1639691143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x555aec8c51f0 0x555aec904cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb15600640 0 client.5799 ms_handle_reset on v2:172.21.15.157:6833/166143460 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 >> 172.21.15.73:0/1639691143 conn(0x555aec8986d0 msgr2=0x555aec89bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 shutdown_connections 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 1 -- 172.21.15.73:0/1639691143 wait complete. 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.985+0000 7fdb31e66740 20 client.5799 trim_cache size 0 max 16384 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:23:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5634, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 11.646500378000001, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 23, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 48, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 48, 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 16.562826873999999 2024-04-05T22:23:20.989 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 msgr2=0x7fdb0406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 0x7fdb0406d2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006fd0 tx=0x7fdb18008040 comp rx=0 tx=0).stop 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 msgr2=0x7fdb2815b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdb1002beb0 tx=0x7fdb10004730 comp rx=0 tx=0).stop 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 shutdown_connections 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb0406ade0 0x7fdb0406d2a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb2815d390 0x7fdb2815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb2815b060 0x7fdb2815b4e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 --2- 172.21.15.73:0/2070339458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0047c0 0x7fdb28148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:20.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 >> 172.21.15.73:0/2070339458 conn(0x7fdb280ce8e0 msgr2=0x7fdb280afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:20.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 shutdown_connections 2024-04-05T22:23:20.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:20.986+0000 7fdb2f000640 1 -- 172.21.15.73:0/2070339458 wait complete. 2024-04-05T22:23:21.002 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:23:21.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.204+0000 7fa17f000640 1 Processor -- start 2024-04-05T22:23:21.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.204+0000 7fa17f000640 1 -- start start 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa178160620 0x7fa1780caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1780cb2f0 0x7fa1780cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780cbc70 con 0x7fa1780b1870 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780ad880 con 0x7fa178160620 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780ce5d0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa177e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa178160620 0x7fa1780caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa177400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa1780b1870 0x7fa1780ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33132/0 (socket says 172.21.15.73:33132) 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa177400640 1 -- 172.21.15.73:0/1123349518 learned_addr learned my addr 172.21.15.73:0/1123349518 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.206+0000 7fa17d000640 1 --2- 172.21.15.73:0/1123349518 >> v2:172.21.15.157:3300/0 conn(0x7fa1780cb2f0 0x7fa1780cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.207+0000 7fa177e00640 1 -- 172.21.15.73:0/1123349518 >> v2:172.21.15.157:3300/0 conn(0x7fa1780cb2f0 msgr2=0x7fa1780cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.207+0000 7fa177e00640 1 --2- 172.21.15.73:0/1123349518 >> v2:172.21.15.157:3300/0 conn(0x7fa1780cb2f0 0x7fa1780cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.207+0000 7fa177e00640 1 -- 172.21.15.73:0/1123349518 >> v1:172.21.15.73:6789/0 conn(0x7fa1780b1870 legacy=0x7fa1780ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.207+0000 7fa177e00640 1 -- 172.21.15.73:0/1123349518 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780ce010 con 0x7fa178160620 2024-04-05T22:23:21.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa177e00640 1 --2- 172.21.15.73:0/1123349518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa178160620 0x7fa1780caa60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa17806f1b0 tx=0x7fa16802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371243be43f30e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:21.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa176a00640 1 -- 172.21.15.73:0/1123349518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168031830 con 0x7fa178160620 2024-04-05T22:23:21.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa176a00640 1 -- 172.21.15.73:0/1123349518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168031990 con 0x7fa178160620 2024-04-05T22:23:21.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa176a00640 1 -- 172.21.15.73:0/1123349518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168031d20 con 0x7fa178160620 2024-04-05T22:23:21.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa17f000640 1 -- 172.21.15.73:0/1123349518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa178160620 msgr2=0x7fa1780caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.208+0000 7fa17f000640 1 --2- 172.21.15.73:0/1123349518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa178160620 0x7fa1780caa60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa17806f1b0 tx=0x7fa16802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 -- 172.21.15.73:0/1123349518 shutdown_connections 2024-04-05T22:23:21.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 --2- 172.21.15.73:0/1123349518 >> v2:172.21.15.157:3300/0 conn(0x7fa1780cb2f0 0x7fa1780cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 --2- 172.21.15.73:0/1123349518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa178160620 0x7fa1780caa60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 -- 172.21.15.73:0/1123349518 >> 172.21.15.73:0/1123349518 conn(0x7fa1780ce8e0 msgr2=0x7fa1780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:21.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 -- 172.21.15.73:0/1123349518 shutdown_connections 2024-04-05T22:23:21.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.209+0000 7fa17f000640 1 -- 172.21.15.73:0/1123349518 wait complete. 2024-04-05T22:23:21.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.210+0000 7fa17f000640 1 Processor -- start 2024-04-05T22:23:21.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.210+0000 7fa17f000640 1 -- start start 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1780b1870 0x7fa1780b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 0x7fa1781494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781f3850 con 0x7fa1780cb2f0 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780c3f40 con 0x7fa178160620 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.211+0000 7fa17f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781f3660 con 0x7fa1780b1870 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34364/0 (socket says 172.21.15.73:34364) 2024-04-05T22:23:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 -- 172.21.15.73:0/1568564916 learned_addr learned my addr 172.21.15.73:0/1568564916 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:21.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 -- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1780b1870 msgr2=0x7fa1780b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa17d000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 0x7fa1781494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177e00640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1780b1870 0x7fa1780b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1780b1870 0x7fa1780b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 -- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 msgr2=0x7fa1781494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 0x7fa1781494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 -- 172.21.15.73:0/1568564916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009400 con 0x7fa1780cb2f0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa17d000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 0x7fa1781494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.212+0000 7fa177400640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c4d0 tx=0x7fa16000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.213+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160005ce0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.213+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160005e40 con 0x7fa1780cb2f0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.213+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160007270 con 0x7fa1780cb2f0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.213+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1781f3380 con 0x7fa1780cb2f0 2024-04-05T22:23:21.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.213+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780c3bd0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.214+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa178066af0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.215+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fa1600073d0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.215+0000 7fa16fe00640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 0x7fa15406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:21.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.216+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa160088a80 con 0x7fa1780cb2f0 2024-04-05T22:23:21.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.216+0000 7fa177e00640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 0x7fa15406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:21.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.216+0000 7fa177e00640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 0x7fa15406d270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa168036000 tx=0x7fa16803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:21.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.223+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa160053a90 con 0x7fa1780cb2f0 2024-04-05T22:23:21.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.453+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1780c40f0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.454+0000 7fa16fe00640 1 -- 172.21.15.73:0/1568564916 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa1780c40f0 con 0x7fa1780cb2f0 2024-04-05T22:23:21.454 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.456+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 msgr2=0x7fa15406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.456+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 0x7fa15406d270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa168036000 tx=0x7fa16803c040 comp rx=0 tx=0).stop 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.456+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 msgr2=0x7fa178148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.456+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c4d0 tx=0x7fa16000c990 comp rx=0 tx=0).stop 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 shutdown_connections 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa15406adb0 0x7fa15406d270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa178160620 0x7fa1781494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1780cb2f0 0x7fa178148f80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 --2- 172.21.15.73:0/1568564916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1780b1870 0x7fa1780b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:21.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 >> 172.21.15.73:0/1568564916 conn(0x7fa1780ce8e0 msgr2=0x7fa17815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:21.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 shutdown_connections 2024-04-05T22:23:21.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:21.457+0000 7fa17f000640 1 -- 172.21.15.73:0/1568564916 wait complete. 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:23:21.476 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:23:21.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.477 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:23:21.530 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:21.531 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.532 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:23:21.533 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:23:21.533 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:23:21.533 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:21.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.534 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:23:21.624 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:23:21.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.628 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:23:21.681 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:23:21.681 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:23:21.681 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:23:21.682 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:23:21.682 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:23:21.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:23:21.682 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:23:21.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.857 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:23:21.907 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:23:21.907 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:23:21.907 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:23:21.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:21.907 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:23:22.002 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:23:22.003 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:23:22.053 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:23:22.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:23:22.054 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:23:22.101 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:23:22.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:22.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:22.234 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:22.234 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:23:22.271 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:22.271 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:23:22.272 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:23:22.272 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:23:22.272 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:23:22.274 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:23:22.274 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:23:22.274 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:23:22.274 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:23:22.274 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:23:22.278 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:23:22.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:22.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:23:22.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:22.367 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:23:22.440 INFO:teuthology.orchestra.run.smithi073.stdout:client4679 2024-04-05T22:23:22.441 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:23:22.441 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:22.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:23:22.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.713+0000 7fd673e00640 1 Processor -- start 2024-04-05T22:23:22.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.714+0000 7fd673e00640 1 -- start start 2024-04-05T22:23:22.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.715+0000 7fd673e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd674158860 0x7fd6740b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.715+0000 7fd673e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd67415ed30 0x7fd67406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.715+0000 7fd673e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406b500 con 0x7fd6740b1f00 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.715+0000 7fd673e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6740ccaf0 con 0x7fd67415ed30 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.715+0000 7fd673e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6740b23a0 con 0x7fd674158860 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd673400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd67415ed30 0x7fd67406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd66be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd6740b1f00 0x7fd67415e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33142/0 (socket says 172.21.15.73:33142) 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd66be00640 1 -- 172.21.15.73:0/2532031358 learned_addr learned my addr 172.21.15.73:0/2532031358 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd672a00640 1 --2- 172.21.15.73:0/2532031358 >> v2:172.21.15.157:3300/0 conn(0x7fd674158860 0x7fd6740b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd673400640 1 -- 172.21.15.73:0/2532031358 >> v2:172.21.15.157:3300/0 conn(0x7fd674158860 msgr2=0x7fd6740b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd673400640 1 --2- 172.21.15.73:0/2532031358 >> v2:172.21.15.157:3300/0 conn(0x7fd674158860 0x7fd6740b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd673400640 1 -- 172.21.15.73:0/2532031358 >> v1:172.21.15.73:6789/0 conn(0x7fd6740b1f00 legacy=0x7fd67415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd673400640 1 -- 172.21.15.73:0/2532031358 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740b2680 con 0x7fd67415ed30 2024-04-05T22:23:22.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.716+0000 7fd672000640 1 -- 172.21.15.73:0/2532031358 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224177058 0 0) 0x7fd67406b500 con 0x7fd6740b1f00 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.717+0000 7fd673400640 1 --2- 172.21.15.73:0/2532031358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd67406bc70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd67406f380 tx=0x7fd66c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1038cc25c11af05 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd672000640 1 -- 172.21.15.73:0/2532031358 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c02cbf0 con 0x7fd67415ed30 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd672000640 1 -- 172.21.15.73:0/2532031358 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c02cd50 con 0x7fd67415ed30 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd672000640 1 -- 172.21.15.73:0/2532031358 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c034660 con 0x7fd67415ed30 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 -- 172.21.15.73:0/2532031358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 msgr2=0x7fd67406bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 --2- 172.21.15.73:0/2532031358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd67406bc70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd67406f380 tx=0x7fd66c02be40 comp rx=0 tx=0).stop 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 -- 172.21.15.73:0/2532031358 shutdown_connections 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 --2- 172.21.15.73:0/2532031358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd67406bc70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 --2- 172.21.15.73:0/2532031358 >> v2:172.21.15.157:3300/0 conn(0x7fd674158860 0x7fd6740b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 -- 172.21.15.73:0/2532031358 >> 172.21.15.73:0/2532031358 conn(0x7fd6740b06c0 msgr2=0x7fd6740b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:22.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 -- 172.21.15.73:0/2532031358 shutdown_connections 2024-04-05T22:23:22.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.718+0000 7fd673e00640 1 -- 172.21.15.73:0/2532031358 wait complete. 2024-04-05T22:23:22.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.720+0000 7fd673e00640 1 Processor -- start 2024-04-05T22:23:22.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.720+0000 7fd673e00640 1 -- start start 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 0x7fd67415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674150220 0x7fd6740ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c7d40 con 0x7fd67415ed30 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6741f2c40 con 0x7fd6740b1f00 2024-04-05T22:23:22.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd673e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6741f2380 con 0x7fd674150220 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd66be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd66be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34388/0 (socket says 172.21.15.73:34388) 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.721+0000 7fd66be00640 1 -- 172.21.15.73:0/3774973413 learned_addr learned my addr 172.21.15.73:0/3774973413 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd672a00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 0x7fd67415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 -- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674150220 msgr2=0x7fd6740ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674150220 0x7fd6740ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 -- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 msgr2=0x7fd67415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 0x7fd67415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 -- 172.21.15.73:0/3774973413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c009a70 con 0x7fd67415ed30 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd672a00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 0x7fd67415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:22.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66be00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c460 tx=0x7fd65c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0074e0 con 0x7fd67415ed30 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c007660 con 0x7fd67415ed30 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c004a20 con 0x7fd67415ed30 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.722+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6741f25f0 con 0x7fd67415ed30 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.723+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd674071710 con 0x7fd67415ed30 2024-04-05T22:23:22.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.723+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6741f2dd0 con 0x7fd67415ed30 2024-04-05T22:23:22.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.725+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fd65c004b80 con 0x7fd67415ed30 2024-04-05T22:23:22.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.725+0000 7fd66aa00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 0x7fd65006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:22.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.726+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd65c088910 con 0x7fd67415ed30 2024-04-05T22:23:22.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.726+0000 7fd672a00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 0x7fd65006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:22.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.726+0000 7fd672a00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 0x7fd65006d270 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd664006fd0 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.730+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd65c0537d0 con 0x7fd67415ed30 2024-04-05T22:23:22.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.964+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd67404e160 con 0x7fd67415ed30 2024-04-05T22:23:22.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.965+0000 7fd66aa00640 1 -- 172.21.15.73:0/3774973413 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd65c018070 con 0x7fd67415ed30 2024-04-05T22:23:22.965 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:23:22.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 msgr2=0x7fd65006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 0x7fd65006d270 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd664006fd0 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 msgr2=0x7fd6740711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c460 tx=0x7fd65c00c920 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 shutdown_connections 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd65006adb0 0x7fd65006d270 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674150220 0x7fd6740ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd67415ed30 0x7fd6740711d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 --2- 172.21.15.73:0/3774973413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740b1f00 0x7fd67415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 >> 172.21.15.73:0/3774973413 conn(0x7fd6740b06c0 msgr2=0x7fd6740ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 shutdown_connections 2024-04-05T22:23:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:22.968+0000 7fd673e00640 1 -- 172.21.15.73:0/3774973413 wait complete. 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:23:22.987 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:23:22.988 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:23:22.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:22.988 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:23:23.047 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.048 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:23:23.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:23:23.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.050 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:23:23.143 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:23:23.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.147 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:23:23.201 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:23:23.201 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:23:23.201 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:23:23.201 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:23:23.201 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:23:23.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:23:23.202 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:23:23.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.382 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:23:23.432 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:23:23.432 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:23:23.433 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:23:23.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:23:23.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:23:23.523 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:23:23.577 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:23:23.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:23:23.577 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:23:23.628 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:23:23.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.628 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:23.762 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:23.762 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:23:23.801 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:23:23.801 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:23:23.801 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:23:23.801 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:23:23.801 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:23:23.804 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:23:23.804 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:23:23.804 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:23:23.805 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:23:23.805 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:23:23.808 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:23:23.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:23:23.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:23.892 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:23:23.946 INFO:teuthology.orchestra.run.smithi157.stdout:client5820 2024-04-05T22:23:23.946 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:23:23.947 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:23.997 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:23:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.220+0000 7f1adae00640 1 Processor -- start 2024-04-05T22:23:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.220+0000 7f1adae00640 1 -- start start 2024-04-05T22:23:24.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.221+0000 7f1adae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad40714b0 0x7f1ad406b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.221+0000 7f1adae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 0x7f1ad40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1adae00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40ccc50 con 0x7f1ad415aa80 2024-04-05T22:23:24.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1adae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40cd5f0 con 0x7f1ad40714b0 2024-04-05T22:23:24.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1adae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40705b0 con 0x7f1ad406b9d0 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad40714b0 0x7f1ad406b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad40714b0 0x7f1ad406b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34404/0 (socket says 172.21.15.73:34404) 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 -- 172.21.15.73:0/915529918 learned_addr learned my addr 172.21.15.73:0/915529918 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ada400640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 0x7f1ad40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 -- 172.21.15.73:0/915529918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 msgr2=0x7f1ad40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 0x7f1ad40cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.222+0000 7f1ad9000640 1 -- 172.21.15.73:0/915529918 >> v1:172.21.15.73:6789/0 conn(0x7f1ad415aa80 legacy=0x7f1ad406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.223+0000 7f1ad9000640 1 -- 172.21.15.73:0/915529918 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40ccf30 con 0x7f1ad40714b0 2024-04-05T22:23:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.223+0000 7f1ada400640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 0x7f1ad40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1ad9000640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad40714b0 0x7f1ad406b490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e142828c264af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1ad3e00640 1 -- 172.21.15.73:0/915529918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00317c0 con 0x7f1ad40714b0 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1ad3e00640 1 -- 172.21.15.73:0/915529918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0031920 con 0x7f1ad40714b0 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1ad3e00640 1 -- 172.21.15.73:0/915529918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0031cb0 con 0x7f1ad40714b0 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1adae00640 1 -- 172.21.15.73:0/915529918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad40714b0 msgr2=0x7f1ad406b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.224+0000 7f1adae00640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad40714b0 0x7f1ad406b490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac002b860 comp rx=0 tx=0).stop 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 -- 172.21.15.73:0/915529918 shutdown_connections 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad406b9d0 0x7f1ad40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 --2- 172.21.15.73:0/915529918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad40714b0 0x7f1ad406b490 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 -- 172.21.15.73:0/915529918 >> 172.21.15.73:0/915529918 conn(0x7f1ad40b0360 msgr2=0x7f1ad40b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 -- 172.21.15.73:0/915529918 shutdown_connections 2024-04-05T22:23:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.225+0000 7f1adae00640 1 -- 172.21.15.73:0/915529918 wait complete. 2024-04-05T22:23:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.226+0000 7f1adae00640 1 Processor -- start 2024-04-05T22:23:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.226+0000 7f1adae00640 1 -- start start 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad406b9d0 0x7f1ad40c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad40714b0 0x7f1ad41463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41468e0 con 0x7f1ad415aa80 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c50d0 con 0x7f1ad406b9d0 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.227+0000 7f1adae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c44c0 con 0x7f1ad40714b0 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34418/0 (socket says 172.21.15.73:34418) 2024-04-05T22:23:24.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 -- 172.21.15.73:0/248067317 learned_addr learned my addr 172.21.15.73:0/248067317 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ad9000640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad40714b0 0x7f1ad41463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 -- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad40714b0 msgr2=0x7f1ad41463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad40714b0 0x7f1ad41463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 -- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad406b9d0 msgr2=0x7f1ad40c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad406b9d0 0x7f1ad40c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 -- 172.21.15.73:0/248067317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00094d0 con 0x7f1ad415aa80 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.228+0000 7f1ada400640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cd50 tx=0x7f1acc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.229+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00f680 con 0x7f1ad415aa80 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.229+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc00f800 con 0x7f1ad415aa80 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.229+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00fbb0 con 0x7f1ad415aa80 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.229+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad40c47a0 con 0x7f1ad415aa80 2024-04-05T22:23:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.229+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40c4d60 con 0x7f1ad415aa80 2024-04-05T22:23:24.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.230+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad404e160 con 0x7f1ad415aa80 2024-04-05T22:23:24.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.231+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f1acc00b040 con 0x7f1ad415aa80 2024-04-05T22:23:24.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.232+0000 7f1ad1600640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 0x7f1ab406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.232+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1acc08a290 con 0x7f1ad415aa80 2024-04-05T22:23:24.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.233+0000 7f1ad9a00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 0x7f1ab406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.233+0000 7f1ad9a00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 0x7f1ab406d270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8000f60 tx=0x7f1ac80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.237+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1acc054fb0 con 0x7f1ad415aa80 2024-04-05T22:23:24.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.469+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ad40ad560 con 0x7f1ad415aa80 2024-04-05T22:23:24.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.627+0000 7f1ad1600640 1 -- 172.21.15.73:0/248067317 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1acc0585d0 con 0x7f1ad415aa80 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 msgr2=0x7f1ab406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 0x7f1ab406d270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8000f60 tx=0x7f1ac80079b0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 msgr2=0x7f1ad4156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cd50 tx=0x7f1acc00a650 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 shutdown_connections 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1ab406adb0 0x7f1ab406d270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415aa80 0x7f1ad4156c50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad40714b0 0x7f1ad41463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 --2- 172.21.15.73:0/248067317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad406b9d0 0x7f1ad40c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 >> 172.21.15.73:0/248067317 conn(0x7f1ad40b0360 msgr2=0x7f1ad40ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 shutdown_connections 2024-04-05T22:23:24.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.630+0000 7f1adae00640 1 -- 172.21.15.73:0/248067317 wait complete. 2024-04-05T22:23:24.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:23:24.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.868+0000 7f1d99400640 1 Processor -- start 2024-04-05T22:23:24.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.868+0000 7f1d99400640 1 -- start start 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d99400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 0x7f1d9406bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d99400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d940b3070 0x7f1d9406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d99400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d940cc760 con 0x7f1d941441e0 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d99400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940b1950 con 0x7f1d940b3070 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d99400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940b1b60 con 0x7f1d940b2c00 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d93e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1d941441e0 0x7f1d940cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33148/0 (socket says 172.21.15.73:33148) 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d93e00640 1 -- 172.21.15.73:0/1333239657 learned_addr learned my addr 172.21.15.73:0/1333239657 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:24.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d98a00640 1 --2- 172.21.15.73:0/1333239657 >> v2:172.21.15.73:3300/0 conn(0x7f1d940b3070 0x7f1d9406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d98a00640 1 -- 172.21.15.73:0/1333239657 >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 msgr2=0x7f1d9406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:24.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d8b400640 1 --2- 172.21.15.73:0/1333239657 >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 0x7f1d9406bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.870+0000 7f1d93400640 1 -- 172.21.15.73:0/1333239657 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197955478 0 0) 0x7f1d940cc760 con 0x7f1d941441e0 2024-04-05T22:23:24.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.871+0000 7f1d98a00640 1 --2- 172.21.15.73:0/1333239657 >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 0x7f1d9406bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.871+0000 7f1d98a00640 1 -- 172.21.15.73:0/1333239657 >> v1:172.21.15.73:6789/0 conn(0x7f1d941441e0 legacy=0x7f1d940cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.871+0000 7f1d98a00640 1 -- 172.21.15.73:0/1333239657 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940b1e40 con 0x7f1d940b3070 2024-04-05T22:23:24.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.871+0000 7f1d8b400640 1 --2- 172.21.15.73:0/1333239657 >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 0x7f1d9406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d98a00640 1 --2- 172.21.15.73:0/1333239657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b3070 0x7f1d9406c100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1d94071930 tx=0x7f1d8c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6755082062b03e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d93400640 1 -- 172.21.15.73:0/1333239657 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c02cc80 con 0x7f1d940b3070 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d93400640 1 -- 172.21.15.73:0/1333239657 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c02cde0 con 0x7f1d940b3070 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d93400640 1 -- 172.21.15.73:0/1333239657 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c034660 con 0x7f1d940b3070 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d99400640 1 -- 172.21.15.73:0/1333239657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b3070 msgr2=0x7f1d9406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.872+0000 7f1d99400640 1 --2- 172.21.15.73:0/1333239657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b3070 0x7f1d9406c100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1d94071930 tx=0x7f1d8c02bed0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 -- 172.21.15.73:0/1333239657 shutdown_connections 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 --2- 172.21.15.73:0/1333239657 >> v2:172.21.15.157:3300/0 conn(0x7f1d940b2c00 0x7f1d9406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 --2- 172.21.15.73:0/1333239657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b3070 0x7f1d9406c100 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 -- 172.21.15.73:0/1333239657 >> 172.21.15.73:0/1333239657 conn(0x7f1d940b0830 msgr2=0x7f1d940b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:24.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 -- 172.21.15.73:0/1333239657 shutdown_connections 2024-04-05T22:23:24.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.873+0000 7f1d99400640 1 -- 172.21.15.73:0/1333239657 wait complete. 2024-04-05T22:23:24.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.874+0000 7f1d99400640 1 Processor -- start 2024-04-05T22:23:24.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.874+0000 7f1d99400640 1 -- start start 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 0x7f1d941524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d941441e0 0x7f1d94159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941538a0 con 0x7f1d940b2c00 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94152a30 con 0x7f1d941441e0 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.875+0000 7f1d99400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941f25a0 con 0x7f1d940b3070 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34434/0 (socket says 172.21.15.73:34434) 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 -- 172.21.15.73:0/2318144064 learned_addr learned my addr 172.21.15.73:0/2318144064 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d8b400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 0x7f1d941524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d98a00640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d941441e0 0x7f1d94159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 -- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 msgr2=0x7f1d941524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 0x7f1d941524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 -- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d941441e0 msgr2=0x7f1d94159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d941441e0 0x7f1d94159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 -- 172.21.15.73:0/2318144064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009a70 con 0x7f1d940b2c00 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d8b400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 0x7f1d941524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.876+0000 7f1d93e00640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400c430 tx=0x7f1d8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.877+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840074e0 con 0x7f1d940b2c00 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.877+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84007660 con 0x7f1d940b2c00 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.877+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d94152c90 con 0x7f1d940b2c00 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.877+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84004a20 con 0x7f1d940b2c00 2024-04-05T22:23:24.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.877+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d94153250 con 0x7f1d940b2c00 2024-04-05T22:23:24.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.878+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9415ef60 con 0x7f1d940b2c00 2024-04-05T22:23:24.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.879+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f1d84004b80 con 0x7f1d940b2c00 2024-04-05T22:23:24.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.880+0000 7f1d90c00640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 0x7f1d7006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:24.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.880+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d84089550 con 0x7f1d940b2c00 2024-04-05T22:23:24.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.880+0000 7f1d8b400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 0x7f1d7006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:24.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.881+0000 7f1d8b400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 0x7f1d7006d270 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d940b3f90 tx=0x7f1d7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:24.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:24.885+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1d84054410 con 0x7f1d940b2c00 2024-04-05T22:23:25.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.101+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d940b26e0 con 0x7f1d7006adb0 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.104+0000 7f1d90c00640 1 -- 172.21.15.73:0/2318144064 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1d7c00c040 con 0x7f1d7006adb0 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:23:25.103 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:23:25.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 msgr2=0x7f1d7006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 0x7f1d7006d270 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d940b3f90 tx=0x7f1d7c008040 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 msgr2=0x7f1d9415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400c430 tx=0x7f1d8400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 shutdown_connections 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1d7006adb0 0x7f1d7006d270 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d941441e0 0x7f1d94159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940b3070 0x7f1d941524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 --2- 172.21.15.73:0/2318144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d940b2c00 0x7f1d9415d400 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 >> 172.21.15.73:0/2318144064 conn(0x7f1d940b0830 msgr2=0x7f1d9406bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 shutdown_connections 2024-04-05T22:23:25.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.106+0000 7f1d99400640 1 -- 172.21.15.73:0/2318144064 wait complete. 2024-04-05T22:23:25.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:23:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.334+0000 7fbedd600640 1 Processor -- start 2024-04-05T22:23:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.334+0000 7fbedd600640 1 -- start start 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbedd600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbed806c9a0 0x7fbed806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbedd600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed806bea0 con 0x7fbed815abe0 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbedd600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed815afc0 con 0x7fbed806ba70 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbedd600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed806f5b0 con 0x7fbed806c9a0 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbed6a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbed815abe0 0x7fbed80712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33152/0 (socket says 172.21.15.73:33152) 2024-04-05T22:23:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbed6a00640 1 -- 172.21.15.73:0/2756081928 learned_addr learned my addr 172.21.15.73:0/2756081928 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:25.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110957441 0 0) 0x7fbed806bea0 con 0x7fbed815abe0 2024-04-05T22:23:25.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.336+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbebc003520 con 0x7fbed815abe0 2024-04-05T22:23:25.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed6000640 1 --2- 172.21.15.73:0/2756081928 >> v2:172.21.15.157:3300/0 conn(0x7fbed806c9a0 0x7fbed806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089431310 0 0) 0x7fbebc003520 con 0x7fbed815abe0 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 >> v2:172.21.15.157:3300/0 conn(0x7fbed806c9a0 msgr2=0x7fbed806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 --2- 172.21.15.73:0/2756081928 >> v2:172.21.15.157:3300/0 conn(0x7fbed806c9a0 0x7fbed806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 >> v1:172.21.15.157:6789/0 conn(0x7fbed806ba70 legacy=0x7fbed8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed806f890 con 0x7fbed815abe0 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbec8001690 con 0x7fbed815abe0 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 >> v1:172.21.15.73:6789/0 conn(0x7fbed815abe0 legacy=0x7fbed80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbebc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0047c0 0x7fbebc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc005380 0x7fbebc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed806f5b0 con 0x7fbebc003970 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00c210 con 0x7fbebc005380 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00c3d0 con 0x7fbebc0047c0 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.337+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbebc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed6a00640 1 -- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0047c0 msgr2=0x7fbebc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0047c0 0x7fbebc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed7400640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc005380 0x7fbebc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed6a00640 1 -- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc005380 msgr2=0x7fbebc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc005380 0x7fbebc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed7400640 1 -- 172.21.15.73:0/2756081928 reap_dead start 2024-04-05T22:23:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.338+0000 7fbed6a00640 1 -- 172.21.15.73:0/2756081928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc003520 con 0x7fbebc003970 2024-04-05T22:23:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.339+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbebc003d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbed806c030 tx=0x7fbebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7024184162da42b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.339+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8005ad0 con 0x7fbebc003970 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.339+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec8005c30 con 0x7fbebc003970 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.339+0000 7fbed5600640 1 -- 172.21.15.73:0/2756081928 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec802c8a0 con 0x7fbebc003970 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 -- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 msgr2=0x7fbebc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbebc003d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbed806c030 tx=0x7fbebc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 -- 172.21.15.73:0/2756081928 shutdown_connections 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 --2- 172.21.15.73:0/2756081928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbebc003d90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 -- 172.21.15.73:0/2756081928 >> 172.21.15.73:0/2756081928 conn(0x7fbed8069cb0 msgr2=0x7fbed806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 -- 172.21.15.73:0/2756081928 shutdown_connections 2024-04-05T22:23:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.340+0000 7fbedd600640 1 -- 172.21.15.73:0/2756081928 wait complete. 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.342+0000 7fbedd600640 1 Processor -- start 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.342+0000 7fbedd600640 1 -- start start 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815da60 0x7fbed8149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815c210 0x7fbed815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815bdf0 con 0x7fbebc003970 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8158450 con 0x7fbed815da60 2024-04-05T22:23:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbedd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815b170 con 0x7fbed815c210 2024-04-05T22:23:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbed6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbed6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34458/0 (socket says 172.21.15.73:34458) 2024-04-05T22:23:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.343+0000 7fbed6a00640 1 -- 172.21.15.73:0/2009040545 learned_addr learned my addr 172.21.15.73:0/2009040545 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed7400640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815c210 0x7fbed815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6000640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815da60 0x7fbed8149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 -- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815c210 msgr2=0x7fbed815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815c210 0x7fbed815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 -- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815da60 msgr2=0x7fbed8149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815da60 0x7fbed8149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 -- 172.21.15.73:0/2009040545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed815b450 con 0x7fbebc003970 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbed6a00640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c300 tx=0x7fbec8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec802ccb0 con 0x7fbebc003970 2024-04-05T22:23:25.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec802ce10 con 0x7fbebc003970 2024-04-05T22:23:25.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80072b0 con 0x7fbebc003970 2024-04-05T22:23:25.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.344+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed815b730 con 0x7fbebc003970 2024-04-05T22:23:25.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.345+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed8149a50 con 0x7fbebc003970 2024-04-05T22:23:25.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.346+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed8068160 con 0x7fbebc003970 2024-04-05T22:23:25.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.346+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fbec8032070 con 0x7fbebc003970 2024-04-05T22:23:25.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.347+0000 7fbecea00640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 0x7fbeb806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.347+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbec803c040 con 0x7fbebc003970 2024-04-05T22:23:25.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.348+0000 7fbed6000640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 0x7fbeb806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.348+0000 7fbed6000640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 0x7fbeb806d270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.352+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbec8075750 con 0x7fbebc003970 2024-04-05T22:23:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.555+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbed80b3890 con 0x7fbebc003970 2024-04-05T22:23:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.556+0000 7fbecea00640 1 -- 172.21.15.73:0/2009040545 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbec80450c0 con 0x7fbebc003970 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 msgr2=0x7fbeb806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 0x7fbeb806d270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4008040 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 msgr2=0x7fbed8157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c300 tx=0x7fbec8001a20 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 shutdown_connections 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbeb806adb0 0x7fbeb806d270 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed815c210 0x7fbed815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed815da60 0x7fbed8149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 --2- 172.21.15.73:0/2009040545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbebc003970 0x7fbed8157f10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.559+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 >> 172.21.15.73:0/2009040545 conn(0x7fbed8069cb0 msgr2=0x7fbed80b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.560+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 shutdown_connections 2024-04-05T22:23:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.560+0000 7fbedd600640 1 -- 172.21.15.73:0/2009040545 wait complete. 2024-04-05T22:23:25.578 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:23:25.578 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:25.624 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:23:25.624 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:25.720 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:25.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.931+0000 7f328be00640 1 Processor -- start 2024-04-05T22:23:25.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.931+0000 7f328be00640 1 -- start start 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f328be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f328406cb80 0x7f328406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f328be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f328415e850 0x7f328415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f328be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328406f5b0 con 0x7f32841587d0 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f328be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328406d500 con 0x7f328406cb80 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f328be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328415f220 con 0x7f328415e850 2024-04-05T22:23:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f3288a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f328406cb80 0x7f328406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f3288a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f328406cb80 0x7f328406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41734/0 (socket says 172.21.15.73:41734) 2024-04-05T22:23:25.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f3288a00640 1 -- 172.21.15.73:0/851925423 learned_addr learned my addr 172.21.15.73:0/851925423 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:25.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.933+0000 7f3289e00640 1 --2- 172.21.15.73:0/851925423 >> v2:172.21.15.157:3300/0 conn(0x7f328415e850 0x7f328415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.934+0000 7f3288a00640 1 -- 172.21.15.73:0/851925423 >> v2:172.21.15.157:3300/0 conn(0x7f328415e850 msgr2=0x7f328415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:25.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.934+0000 7f3288a00640 1 --2- 172.21.15.73:0/851925423 >> v2:172.21.15.157:3300/0 conn(0x7f328415e850 0x7f328415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.934+0000 7f3288a00640 1 -- 172.21.15.73:0/851925423 >> v1:172.21.15.157:6789/0 conn(0x7f32841587d0 legacy=0x7f32840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:25.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.934+0000 7f3288a00640 1 -- 172.21.15.73:0/851925423 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284070410 con 0x7f328406cb80 2024-04-05T22:23:25.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.935+0000 7f3288a00640 1 --2- 172.21.15.73:0/851925423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f328406cfc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f328415c270 tx=0x7f327402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba75f0023feb74ad server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.935+0000 7f3283e00640 1 -- 172.21.15.73:0/851925423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274031870 con 0x7f328406cb80 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.935+0000 7f3283e00640 1 -- 172.21.15.73:0/851925423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32740319d0 con 0x7f328406cb80 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.935+0000 7f3283e00640 1 -- 172.21.15.73:0/851925423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274031d60 con 0x7f328406cb80 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 -- 172.21.15.73:0/851925423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 msgr2=0x7f328406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 --2- 172.21.15.73:0/851925423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f328406cfc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f328415c270 tx=0x7f327402b910 comp rx=0 tx=0).stop 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 -- 172.21.15.73:0/851925423 shutdown_connections 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 --2- 172.21.15.73:0/851925423 >> v2:172.21.15.157:3300/0 conn(0x7f328415e850 0x7f328415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 --2- 172.21.15.73:0/851925423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f328406cfc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 -- 172.21.15.73:0/851925423 >> 172.21.15.73:0/851925423 conn(0x7f32840ce8e0 msgr2=0x7f32840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 -- 172.21.15.73:0/851925423 shutdown_connections 2024-04-05T22:23:25.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.936+0000 7f328be00640 1 -- 172.21.15.73:0/851925423 wait complete. 2024-04-05T22:23:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.937+0000 7f328be00640 1 Processor -- start 2024-04-05T22:23:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.938+0000 7f328be00640 1 -- start start 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32841587d0 0x7f32841e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f328415e850 0x7f3284070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841e6830 con 0x7f328406cb80 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840ad560 con 0x7f32841587d0 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f328be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32841e5c20 con 0x7f328415e850 2024-04-05T22:23:25.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41736/0 (socket says 172.21.15.73:41736) 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 -- 172.21.15.73:0/3068899093 learned_addr learned my addr 172.21.15.73:0/3068899093 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 -- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f328415e850 msgr2=0x7f3284070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f328415e850 0x7f3284070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 -- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32841587d0 msgr2=0x7f32841e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32841587d0 0x7f32841e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.939+0000 7f3289400640 1 -- 172.21.15.73:0/3068899093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740094d0 con 0x7f328406cb80 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f3289400640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f327800c430 tx=0x7f327800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278011790 con 0x7f328406cb80 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780118f0 con 0x7f328406cb80 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32841e5f00 con 0x7f328406cb80 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278011ca0 con 0x7f328406cb80 2024-04-05T22:23:25.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.940+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32841e64c0 con 0x7f328406cb80 2024-04-05T22:23:25.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.941+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284066af0 con 0x7f328406cb80 2024-04-05T22:23:25.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.942+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f327801c460 con 0x7f328406cb80 2024-04-05T22:23:25.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.943+0000 7f3281600640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 0x7f326806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:25.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.944+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327808a0e0 con 0x7f328406cb80 2024-04-05T22:23:25.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.944+0000 7f3288a00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 0x7f326806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:25.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.944+0000 7f3288a00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 0x7f326806d270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3274036000 tx=0x7f3274030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:25.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:25.947+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3278055020 con 0x7f328406cb80 2024-04-05T22:23:26.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.141+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3284068160 con 0x7f328406cb80 2024-04-05T22:23:26.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.144+0000 7f3281600640 1 -- 172.21.15.73:0/3068899093 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f3278058640 con 0x7f328406cb80 2024-04-05T22:23:26.143 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:26.144 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":48,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:10.337170+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.146+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 msgr2=0x7f326806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.146+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 0x7f326806d270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3274036000 tx=0x7f3274030000 comp rx=0 tx=0).stop 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.146+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 msgr2=0x7f32840ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.146+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f327800c430 tx=0x7f327800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 shutdown_connections 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f326806adb0 0x7f326806d270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f328415e850 0x7f3284070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32841587d0 0x7f32841e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 --2- 172.21.15.73:0/3068899093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f328406cb80 0x7f32840ad020 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 >> 172.21.15.73:0/3068899093 conn(0x7f32840ce8e0 msgr2=0x7f32840cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 shutdown_connections 2024-04-05T22:23:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.147+0000 7f328be00640 1 -- 172.21.15.73:0/3068899093 wait complete. 2024-04-05T22:23:26.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:23:07.732446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:23:08.298551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:23:26.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:23:26.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.377+0000 7f391b200640 1 Processor -- start 2024-04-05T22:23:26.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.377+0000 7f391b200640 1 -- start start 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f391415ac30 0x7f391406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f391406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391406bc50 con 0x7f391415a850 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391415f530 con 0x7f391415ac30 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140708e0 con 0x7f3914070470 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f391415ac30 0x7f391406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f391415ac30 0x7f391406ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41748/0 (socket says 172.21.15.73:41748) 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 -- 172.21.15.73:0/3321807665 learned_addr learned my addr 172.21.15.73:0/3321807665 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919e00640 1 --1- 172.21.15.73:0/3321807665 >> v1:172.21.15.73:6789/0 conn(0x7f391415a850 0x7f391415e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37756/0 (socket says 172.21.15.73:37756) 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f391a800640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f391406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 -- 172.21.15.73:0/3321807665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 msgr2=0x7f391406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:26.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f391406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3919400640 1 -- 172.21.15.73:0/3321807665 >> v1:172.21.15.73:6789/0 conn(0x7f391415a850 legacy=0x7f391415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.380+0000 7f3919400640 1 -- 172.21.15.73:0/3321807665 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391415ef70 con 0x7f391415ac30 2024-04-05T22:23:26.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.379+0000 7f3918a00640 1 -- 172.21.15.73:0/3321807665 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913285719 0 0) 0x7f391406bc50 con 0x7f391415a850 2024-04-05T22:23:26.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.380+0000 7f391a800640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f391406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f3919400640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f391406ff30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f39040098a0 tx=0x7f390402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd238942db85c591 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f3918a00640 1 -- 172.21.15.73:0/3321807665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904030840 con 0x7f391415ac30 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f3918a00640 1 -- 172.21.15.73:0/3321807665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39040309a0 con 0x7f391415ac30 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f3918a00640 1 -- 172.21.15.73:0/3321807665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904030d30 con 0x7f391415ac30 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f391b200640 1 -- 172.21.15.73:0/3321807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 msgr2=0x7f391406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.381+0000 7f391b200640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f391406ff30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f39040098a0 tx=0x7f390402b980 comp rx=0 tx=0).stop 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 -- 172.21.15.73:0/3321807665 shutdown_connections 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f391406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 --2- 172.21.15.73:0/3321807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f391406ff30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 -- 172.21.15.73:0/3321807665 >> 172.21.15.73:0/3321807665 conn(0x7f3914069cb0 msgr2=0x7f391406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:26.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 -- 172.21.15.73:0/3321807665 shutdown_connections 2024-04-05T22:23:26.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.382+0000 7f391b200640 1 -- 172.21.15.73:0/3321807665 wait complete. 2024-04-05T22:23:26.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.383+0000 7f391b200640 1 Processor -- start 2024-04-05T22:23:26.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.383+0000 7f391b200640 1 -- start start 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f3914153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415a850 0x7f3914157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140cc490 con 0x7f391415ac30 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39141541f0 con 0x7f391415a850 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.384+0000 7f391b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140acec0 con 0x7f3914070470 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41760/0 (socket says 172.21.15.73:41760) 2024-04-05T22:23:26.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 -- 172.21.15.73:0/3683879421 learned_addr learned my addr 172.21.15.73:0/3683879421 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:26.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f3919e00640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f3914153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 -- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 msgr2=0x7f3914153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f3919400640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415a850 0x7f3914157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f3914153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 -- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415a850 msgr2=0x7f3914157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415a850 0x7f3914157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.385+0000 7f391a800640 1 -- 172.21.15.73:0/3683879421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040094d0 con 0x7f391415ac30 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f391a800640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f391000c7f0 tx=0x7f391000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910007820 con 0x7f391415ac30 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910007980 con 0x7f391415ac30 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39140ad1a0 con 0x7f391415ac30 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910007d70 con 0x7f391415ac30 2024-04-05T22:23:26.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.386+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3914157980 con 0x7f391415ac30 2024-04-05T22:23:26.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.387+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140ad3e0 con 0x7f391415ac30 2024-04-05T22:23:26.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.388+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f39100113f0 con 0x7f391415ac30 2024-04-05T22:23:26.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.389+0000 7f390a000640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 0x7f38f806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.389+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3910089af0 con 0x7f391415ac30 2024-04-05T22:23:26.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.389+0000 7f3919e00640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 0x7f38f806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.390+0000 7f3919e00640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 0x7f38f806d270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f390c0012e0 tx=0x7f390c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.394+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f39100549d0 con 0x7f391415ac30 2024-04-05T22:23:26.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.587+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f391404e160 con 0x7f391415ac30 2024-04-05T22:23:26.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.588+0000 7f390a000640 1 -- 172.21.15.73:0/3683879421 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f391001f0a0 con 0x7f391415ac30 2024-04-05T22:23:26.588 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:23:26.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 msgr2=0x7f38f806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 0x7f38f806d270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f390c0012e0 tx=0x7f390c009040 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 msgr2=0x7f3914142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f391000c7f0 tx=0x7f391000ccb0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 shutdown_connections 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38f806adb0 0x7f38f806d270 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f391415ac30 0x7f3914142bf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415a850 0x7f3914157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 --2- 172.21.15.73:0/3683879421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3914070470 0x7f3914153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 >> 172.21.15.73:0/3683879421 conn(0x7f3914069cb0 msgr2=0x7f391406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 shutdown_connections 2024-04-05T22:23:26.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.591+0000 7f391b200640 1 -- 172.21.15.73:0/3683879421 wait complete. 2024-04-05T22:23:26.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:23:26.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.823+0000 7f3419a00640 1 Processor -- start 2024-04-05T22:23:26.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.823+0000 7f3419a00640 1 -- start start 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.824+0000 7f3419a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f341415abe0 0x7f34140712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3419a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341415afc0 con 0x7f341406bc40 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3419a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341406f5b0 con 0x7f341406c9a0 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3419a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341406f890 con 0x7f341415abe0 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3413400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f341406bc40 0x7f3414164740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37768/0 (socket says 172.21.15.73:37768) 2024-04-05T22:23:26.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3413400640 1 -- 172.21.15.73:0/2752680897 learned_addr learned my addr 172.21.15.73:0/2752680897 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:26.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3412a00640 1 --2- 172.21.15.73:0/2752680897 >> v2:172.21.15.157:3300/0 conn(0x7f341415abe0 0x7f34140712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708807231 0 0) 0x7f341415afc0 con 0x7f341406bc40 2024-04-05T22:23:26.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.825+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f8003520 con 0x7f341406bc40 2024-04-05T22:23:26.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800941497 0 0) 0x7f33f8003520 con 0x7f341406bc40 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 >> v2:172.21.15.157:3300/0 conn(0x7f341415abe0 msgr2=0x7f34140712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 --2- 172.21.15.73:0/2752680897 >> v2:172.21.15.157:3300/0 conn(0x7f341415abe0 0x7f34140712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 >> v1:172.21.15.157:6789/0 conn(0x7f341406c9a0 legacy=0x7f341406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341406fb70 con 0x7f341406bc40 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f340c001670 con 0x7f341406bc40 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412a00640 1 --2- 172.21.15.73:0/2752680897 >> v2:172.21.15.157:3300/0 conn(0x7f341415abe0 0x7f34140712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 >> v1:172.21.15.73:6789/0 conn(0x7f341406bc40 legacy=0x7f3414164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8003970 0x7f33f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33f80047c0 0x7f33f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 0x7f33f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341406f890 con 0x7f33f8005380 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800c210 con 0x7f33f80047c0 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800c3d0 con 0x7f33f8003970 2024-04-05T22:23:26.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.826+0000 7f3413400640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 0x7f33f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f340a000640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33f80047c0 0x7f33f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3412a00640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8003970 0x7f33f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3413400640 1 -- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8003970 msgr2=0x7f33f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3413400640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8003970 0x7f33f8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3413400640 1 -- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33f80047c0 msgr2=0x7f33f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3413400640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33f80047c0 0x7f33f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f3413400640 1 -- 172.21.15.73:0/2752680897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8003520 con 0x7f33f8005380 2024-04-05T22:23:26.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.827+0000 7f340a000640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33f80047c0 0x7f33f8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3413400640 1 -- 172.21.15.73:0/2752680897 reap_dead start 2024-04-05T22:23:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3413400640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 0x7f33f800b980 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c240 tx=0x7f33f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afea2733abc7da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c005ad0 con 0x7f33f8005380 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c005c70 con 0x7f33f8005380 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3412000640 1 -- 172.21.15.73:0/2752680897 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c02c8b0 con 0x7f33f8005380 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3419a00640 1 -- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 msgr2=0x7f33f800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.828+0000 7f3419a00640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 0x7f33f800b980 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c240 tx=0x7f33f80057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.829+0000 7f3419a00640 1 -- 172.21.15.73:0/2752680897 shutdown_connections 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.829+0000 7f3419a00640 1 --2- 172.21.15.73:0/2752680897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33f8005380 0x7f33f800b980 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.829+0000 7f3419a00640 1 -- 172.21.15.73:0/2752680897 >> 172.21.15.73:0/2752680897 conn(0x7f3414069cb0 msgr2=0x7f341406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.829+0000 7f3419a00640 1 -- 172.21.15.73:0/2752680897 shutdown_connections 2024-04-05T22:23:26.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.829+0000 7f3419a00640 1 -- 172.21.15.73:0/2752680897 wait complete. 2024-04-05T22:23:26.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.831+0000 7f3419a00640 1 Processor -- start 2024-04-05T22:23:26.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.831+0000 7f3419a00640 1 -- start start 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8005380 0x7f341413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3414154890 0x7f341413f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414154d40 con 0x7f34141ee010 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34141407a0 con 0x7f3414154890 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f3419a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34141404c0 con 0x7f33f8005380 2024-04-05T22:23:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f340a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f340a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41780/0 (socket says 172.21.15.73:41780) 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.832+0000 7f340a000640 1 -- 172.21.15.73:0/1524292557 learned_addr learned my addr 172.21.15.73:0/1524292557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f3412a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8005380 0x7f341413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 -- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8005380 msgr2=0x7f341413ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8005380 0x7f341413ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 -- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3414154890 msgr2=0x7f341413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3414154890 0x7f341413f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 -- 172.21.15.73:0/1524292557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414151d20 con 0x7f34141ee010 2024-04-05T22:23:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340a000640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d770 tx=0x7f33fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.833+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc005270 con 0x7f34141ee010 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.834+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc0053d0 con 0x7f34141ee010 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.834+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00b8f0 con 0x7f34141ee010 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.834+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3414152000 con 0x7f34141ee010 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.834+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341413f1a0 con 0x7f34141ee010 2024-04-05T22:23:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.835+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414068160 con 0x7f34141ee010 2024-04-05T22:23:26.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.836+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f33fc00ba50 con 0x7f34141ee010 2024-04-05T22:23:26.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.836+0000 7f340b400640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 0x7f33f006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.837+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33fc0896c0 con 0x7f34141ee010 2024-04-05T22:23:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.837+0000 7f3412a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 0x7f33f006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.837+0000 7f3412a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 0x7f33f006d270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34040099c0 tx=0x7f340400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:26.841+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f33fc054500 con 0x7f34141ee010 2024-04-05T22:23:27.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.034+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34140b3310 con 0x7f34141ee010 2024-04-05T22:23:27.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.035+0000 7f340b400640 1 -- 172.21.15.73:0/1524292557 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33fc019310 con 0x7f34141ee010 2024-04-05T22:23:27.035 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.037+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 msgr2=0x7f33f006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.037+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 0x7f33f006d270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34040099c0 tx=0x7f340400b040 comp rx=0 tx=0).stop 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.037+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 msgr2=0x7f3414154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.037+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d770 tx=0x7f33fc00dc30 comp rx=0 tx=0).stop 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 shutdown_connections 2024-04-05T22:23:27.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f33f006adb0 0x7f33f006d270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3414154890 0x7f341413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34141ee010 0x7f3414154350 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 --2- 172.21.15.73:0/1524292557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33f8005380 0x7f341413ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 >> 172.21.15.73:0/1524292557 conn(0x7f3414069cb0 msgr2=0x7f34140b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 shutdown_connections 2024-04-05T22:23:27.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.038+0000 7f3419a00640 1 -- 172.21.15.73:0/1524292557 wait complete. 2024-04-05T22:23:27.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:23:27.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.270+0000 7f5670800640 1 Processor -- start 2024-04-05T22:23:27.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.270+0000 7f5670800640 1 -- start start 2024-04-05T22:23:27.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.272+0000 7f5670800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f566806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.272+0000 7f5670800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56680b1b30 con 0x7f56680cbe00 2024-04-05T22:23:27.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.272+0000 7f5670800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56680b1e10 con 0x7f56680b32b0 2024-04-05T22:23:27.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.272+0000 7f5670800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ce7b0 con 0x7f566806c630 2024-04-05T22:23:27.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.272+0000 7f566d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f56680cbe00 0x7f56680ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37776/0 (socket says 172.21.15.73:37776) 2024-04-05T22:23:27.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566d400640 1 -- 172.21.15.73:0/174469136 learned_addr learned my addr 172.21.15.73:0/174469136 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566de00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f566806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945745631 0 0) 0x7f56680b1b30 con 0x7f56680cbe00 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5650003520 con 0x7f56680cbe00 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 656593781 0 0) 0x7f5650003520 con 0x7f56680cbe00 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 msgr2=0x7f566806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f566806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 >> v1:172.21.15.157:6789/0 conn(0x7f56680b32b0 legacy=0x7f56681647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566806cf50 con 0x7f56680cbe00 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.273+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5658001670 con 0x7f56680cbe00 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 >> v1:172.21.15.73:6789/0 conn(0x7f56680cbe00 legacy=0x7f56680ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56500039a0 0x7f5650003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500047c0 0x7f5650004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f5650005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ce7b0 con 0x7f56500039a0 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565000ba30 con 0x7f56500047c0 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565000bbf0 con 0x7f566806c630 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56500039a0 0x7f5650003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566d400640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500047c0 0x7f5650004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566e800640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f5650005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 -- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 msgr2=0x7f5650005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f566806c630 0x7f5650005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 -- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500047c0 msgr2=0x7f5650004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:27.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500047c0 0x7f5650004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.274+0000 7f566de00640 1 -- 172.21.15.73:0/174469136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650003520 con 0x7f56500039a0 2024-04-05T22:23:27.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.275+0000 7f566de00640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56500039a0 0x7f5650003dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f565c009a90 tx=0x7f56500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d548d1b3ea5f254c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c02c680 con 0x7f56500039a0 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c02c7e0 con 0x7f56500039a0 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f566ca00640 1 -- 172.21.15.73:0/174469136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c02cb70 con 0x7f56500039a0 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f5670800640 1 -- 172.21.15.73:0/174469136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56500039a0 msgr2=0x7f5650003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f5670800640 1 --2- 172.21.15.73:0/174469136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56500039a0 0x7f5650003dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f565c009a90 tx=0x7f56500057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f566e800640 1 -- 172.21.15.73:0/174469136 reap_dead start 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f5670800640 1 -- 172.21.15.73:0/174469136 shutdown_connections 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.276+0000 7f5670800640 1 -- 172.21.15.73:0/174469136 >> 172.21.15.73:0/174469136 conn(0x7f56680b0830 msgr2=0x7f56680b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.277+0000 7f5670800640 1 -- 172.21.15.73:0/174469136 shutdown_connections 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.277+0000 7f5670800640 1 -- 172.21.15.73:0/174469136 wait complete. 2024-04-05T22:23:27.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.278+0000 7f5670800640 1 Processor -- start 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.278+0000 7f5670800640 1 -- start start 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 0x7f566815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56680c54b0 0x7f56680707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680c5090 con 0x7f5668142df0 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680c4410 con 0x7f56680c54b0 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.279+0000 7f5670800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680c46f0 con 0x7f56500039a0 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41784/0 (socket says 172.21.15.73:41784) 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 -- 172.21.15.73:0/4171148677 learned_addr learned my addr 172.21.15.73:0/4171148677 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:27.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566de00640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 0x7f566815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 -- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 msgr2=0x7f566815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 0x7f566815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566e800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56680c54b0 0x7f56680707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 -- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56680c54b0 msgr2=0x7f56680707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56680c54b0 0x7f56680707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 -- 172.21.15.73:0/4171148677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680c49d0 con 0x7f5668142df0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566de00640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 0x7f566815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.280+0000 7f566d400640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f565800e8c0 tx=0x7f565800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.281+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800c8d0 con 0x7f5668142df0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.281+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800ca30 con 0x7f5668142df0 2024-04-05T22:23:27.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.281+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800cde0 con 0x7f5668142df0 2024-04-05T22:23:27.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.281+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56680c4cb0 con 0x7f5668142df0 2024-04-05T22:23:27.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.281+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56680c3fe0 con 0x7f5668142df0 2024-04-05T22:23:27.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.282+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56680c41d0 con 0x7f5668142df0 2024-04-05T22:23:27.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.283+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f565801d460 con 0x7f5668142df0 2024-04-05T22:23:27.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.284+0000 7f5662000640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 0x7f564c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.284+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5658089700 con 0x7f5668142df0 2024-04-05T22:23:27.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.284+0000 7f566de00640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 0x7f564c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.285+0000 7f566de00640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 0x7f564c06d270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.289+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f56680c41d0 con 0x7f5668142df0 2024-04-05T22:23:27.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.475+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56680c51d0 con 0x7f5668142df0 2024-04-05T22:23:27.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.475+0000 7f5662000640 1 -- 172.21.15.73:0/4171148677 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56680c51d0 con 0x7f5668142df0 2024-04-05T22:23:27.475 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 msgr2=0x7f564c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 0x7f564c06d270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c030000 comp rx=0 tx=0).stop 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 msgr2=0x7f56680c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f565800e8c0 tx=0x7f565800ed80 comp rx=0 tx=0).stop 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 shutdown_connections 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f564c06adb0 0x7f564c06d270 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56680c54b0 0x7f56680707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5668142df0 0x7f56680c3aa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 --2- 172.21.15.73:0/4171148677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56500039a0 0x7f566815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 >> 172.21.15.73:0/4171148677 conn(0x7f56680b0830 msgr2=0x7f56680ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:27.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.478+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 shutdown_connections 2024-04-05T22:23:27.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.479+0000 7f5670800640 1 -- 172.21.15.73:0/4171148677 wait complete. 2024-04-05T22:23:27.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:23:27.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.703+0000 7f2a35800640 1 Processor -- start 2024-04-05T22:23:27.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.703+0000 7f2a35800640 1 -- start start 2024-04-05T22:23:27.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.705+0000 7f2a35800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a3015ce50 0x7f2a300d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.705+0000 7f2a35800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 0x7f2a300cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.705+0000 7f2a35800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300cfdf0 con 0x7f2a300b2c30 2024-04-05T22:23:27.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.705+0000 7f2a35800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a300d0d60 con 0x7f2a3015ce50 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.705+0000 7f2a35800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cef00 con 0x7f2a300cf450 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 0x7f2a300cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a3015ce50 0x7f2a300d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 0x7f2a300cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:46564/0 (socket says 172.21.15.73:46564) 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2f400640 1 -- 172.21.15.73:0/1868511648 learned_addr learned my addr 172.21.15.73:0/1868511648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a34e00640 1 --1- 172.21.15.73:0/1868511648 >> v1:172.21.15.157:6789/0 conn(0x7f2a300b2c30 0x7f2a300b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:52426/0 (socket says 172.21.15.73:52426) 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2fe00640 1 -- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 msgr2=0x7f2a300cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2fe00640 1 --2- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 0x7f2a300cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2fe00640 1 -- 172.21.15.73:0/1868511648 >> v1:172.21.15.157:6789/0 conn(0x7f2a300b2c30 legacy=0x7f2a300b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.706+0000 7f2a2fe00640 1 -- 172.21.15.73:0/1868511648 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300b2150 con 0x7f2a3015ce50 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.707+0000 7f2a2fe00640 1 --2- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300d0820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a200097a0 tx=0x7f2a2002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85bd53d20aa1056e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.708+0000 7f2a2ea00640 1 -- 172.21.15.73:0/1868511648 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2002d040 con 0x7f2a3015ce50 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.708+0000 7f2a2ea00640 1 -- 172.21.15.73:0/1868511648 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20031810 con 0x7f2a3015ce50 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.708+0000 7f2a2ea00640 1 -- 172.21.15.73:0/1868511648 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031ba0 con 0x7f2a3015ce50 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.708+0000 7f2a35800640 1 -- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 msgr2=0x7f2a300d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.708+0000 7f2a35800640 1 --2- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300d0820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a200097a0 tx=0x7f2a2002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 -- 172.21.15.73:0/1868511648 shutdown_connections 2024-04-05T22:23:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 --2- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300cf450 0x7f2a300cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 --2- 172.21.15.73:0/1868511648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300d0820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 -- 172.21.15.73:0/1868511648 >> 172.21.15.73:0/1868511648 conn(0x7f2a300d2a40 msgr2=0x7f2a300d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 -- 172.21.15.73:0/1868511648 shutdown_connections 2024-04-05T22:23:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.709+0000 7f2a35800640 1 -- 172.21.15.73:0/1868511648 wait complete. 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.710+0000 7f2a35800640 1 Processor -- start 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.710+0000 7f2a35800640 1 -- start start 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300b2c30 0x7f2a3015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cc1f0 con 0x7f2a300cf450 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cabc0 con 0x7f2a3015ce50 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a35800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cb850 con 0x7f2a300b2c30 2024-04-05T22:23:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.711+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41796/0 (socket says 172.21.15.73:41796) 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 -- 172.21.15.73:0/1123080820 learned_addr learned my addr 172.21.15.73:0/1123080820 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a34e00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 -- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300b2c30 msgr2=0x7f2a3015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2fe00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300b2c30 0x7f2a3015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300b2c30 0x7f2a3015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 -- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 msgr2=0x7f2a300b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 -- 172.21.15.73:0/1123080820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a20009400 con 0x7f2a300cf450 2024-04-05T22:23:27.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a34e00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a2f400640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800efc0 tx=0x7f2a1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.712+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800cda0 con 0x7f2a300cf450 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.713+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1800f040 con 0x7f2a300cf450 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.713+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a300cbb30 con 0x7f2a300cf450 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.713+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18003bd0 con 0x7f2a300cf450 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.713+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3015b7c0 con 0x7f2a300cf450 2024-04-05T22:23:27.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.714+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a300cc330 con 0x7f2a300cf450 2024-04-05T22:23:27.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.715+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2a18003d30 con 0x7f2a300cf450 2024-04-05T22:23:27.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.715+0000 7f2a27e00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 0x7f2a0c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.716+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a18089e20 con 0x7f2a300cf450 2024-04-05T22:23:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.716+0000 7f2a2fe00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 0x7f2a0c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.717+0000 7f2a2fe00640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 0x7f2a0c06d270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a20036000 tx=0x7f2a2003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.720+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2a300cc330 con 0x7f2a300cf450 2024-04-05T22:23:27.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.926+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a300cc330 con 0x7f2a300cf450 2024-04-05T22:23:27.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.926+0000 7f2a27e00640 1 -- 172.21.15.73:0/1123080820 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2a300cc330 con 0x7f2a300cf450 2024-04-05T22:23:27.925 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:23:27.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 msgr2=0x7f2a0c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 0x7f2a0c06d270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a20036000 tx=0x7f2a2003c040 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 msgr2=0x7f2a300b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800efc0 tx=0x7f2a1800c490 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 shutdown_connections 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2a0c06adb0 0x7f2a0c06d270 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3015ce50 0x7f2a300b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a300cf450 0x7f2a300b4310 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 --2- 172.21.15.73:0/1123080820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a300b2c30 0x7f2a3015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 >> 172.21.15.73:0/1123080820 conn(0x7f2a300d2a40 msgr2=0x7f2a300ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:27.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 shutdown_connections 2024-04-05T22:23:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:27.929+0000 7f2a35800640 1 -- 172.21.15.73:0/1123080820 wait complete. 2024-04-05T22:23:27.947 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:23:27.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:27.947 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:23:28.047 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:23:28.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:28.048 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:23:28.103 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:23:28.103 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:28.161 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:23:28.161 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:23:28.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:23:28.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.432+0000 7f3e55200640 1 Processor -- start 2024-04-05T22:23:28.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.432+0000 7f3e55200640 1 -- start start 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e50160620 0x7f3e500caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e500b1870 0x7f3e500ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 0x7f3e500cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500ce5d0 con 0x7f3e50160620 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500ce010 con 0x7f3e500ad6b0 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e55200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500ce2f0 con 0x7f3e500b1870 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e4de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e50160620 0x7f3e500caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e4de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e50160620 0x7f3e500caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41806/0 (socket says 172.21.15.73:41806) 2024-04-05T22:23:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.434+0000 7f3e4de00640 1 -- 172.21.15.73:0/1759575887 learned_addr learned my addr 172.21.15.73:0/1759575887 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4de00640 1 -- 172.21.15.73:0/1759575887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e500b1870 msgr2=0x7f3e500ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4d400640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e500b1870 0x7f3e500ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4e800640 1 --2- 172.21.15.73:0/1759575887 >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 0x7f3e500cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e500b1870 0x7f3e500ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4de00640 1 -- 172.21.15.73:0/1759575887 >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 msgr2=0x7f3e500cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1759575887 >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 0x7f3e500cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4de00640 1 -- 172.21.15.73:0/1759575887 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500ac970 con 0x7f3e50160620 2024-04-05T22:23:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.435+0000 7f3e4e800640 1 --2- 172.21.15.73:0/1759575887 >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 0x7f3e500cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.436+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e50160620 0x7f3e500caa60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cdec0 tx=0x7f3e4002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe8e17d24b738d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.436+0000 7f3e4ca00640 1 -- 172.21.15.73:0/1759575887 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40031860 con 0x7f3e50160620 2024-04-05T22:23:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.436+0000 7f3e4ca00640 1 -- 172.21.15.73:0/1759575887 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e400319c0 con 0x7f3e50160620 2024-04-05T22:23:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.436+0000 7f3e4ca00640 1 -- 172.21.15.73:0/1759575887 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40031d50 con 0x7f3e50160620 2024-04-05T22:23:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 -- 172.21.15.73:0/1759575887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e50160620 msgr2=0x7f3e500caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e50160620 0x7f3e500caa60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cdec0 tx=0x7f3e4002b900 comp rx=0 tx=0).stop 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 -- 172.21.15.73:0/1759575887 shutdown_connections 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 --2- 172.21.15.73:0/1759575887 >> v2:172.21.15.157:3300/0 conn(0x7f3e500ad6b0 0x7f3e500cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e500b1870 0x7f3e500ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 --2- 172.21.15.73:0/1759575887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e50160620 0x7f3e500caa60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 -- 172.21.15.73:0/1759575887 >> 172.21.15.73:0/1759575887 conn(0x7f3e500ce8e0 msgr2=0x7f3e500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 -- 172.21.15.73:0/1759575887 shutdown_connections 2024-04-05T22:23:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.437+0000 7f3e55200640 1 -- 172.21.15.73:0/1759575887 wait complete. 2024-04-05T22:23:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.439+0000 7f3e55200640 1 Processor -- start 2024-04-05T22:23:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.439+0000 7f3e55200640 1 -- start start 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.440+0000 7f3e55200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.440+0000 7f3e55200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e500b1870 0x7f3e5006fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.440+0000 7f3e55200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e50160620 0x7f3e50156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e55200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500c6730 con 0x7f3e500ad6b0 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e55200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50070000 con 0x7f3e500b1870 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e55200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50157350 con 0x7f3e50160620 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41808/0 (socket says 172.21.15.73:41808) 2024-04-05T22:23:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 -- 172.21.15.73:0/1051439018 learned_addr learned my addr 172.21.15.73:0/1051439018 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:28.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4e800640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e50160620 0x7f3e50156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4d400640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e500b1870 0x7f3e5006fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 -- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e50160620 msgr2=0x7f3e50156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e50160620 0x7f3e50156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 -- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e500b1870 msgr2=0x7f3e5006fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e500b1870 0x7f3e5006fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.441+0000 7f3e4de00640 1 -- 172.21.15.73:0/1051439018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400094d0 con 0x7f3e500ad6b0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e4de00640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000c00 tx=0x7f3e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40031850 con 0x7f3e500ad6b0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40044400 con 0x7f3e500ad6b0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e500c3dc0 con 0x7f3e500ad6b0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400447b0 con 0x7f3e500ad6b0 2024-04-05T22:23:28.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.442+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e500c4380 con 0x7f3e500ad6b0 2024-04-05T22:23:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.443+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50066af0 con 0x7f3e500ad6b0 2024-04-05T22:23:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.445+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f3e4004d430 con 0x7f3e500ad6b0 2024-04-05T22:23:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.445+0000 7f3e46000640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 0x7f3e2c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.446+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e400abeb0 con 0x7f3e500ad6b0 2024-04-05T22:23:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.446+0000 7f3e4d400640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 0x7f3e2c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.447+0000 7f3e4d400640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 0x7f3e2c06d270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e380070b0 tx=0x7f3e38007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.450+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3e40076d90 con 0x7f3e500ad6b0 2024-04-05T22:23:28.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.647+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e50068160 con 0x7f3e500ad6b0 2024-04-05T22:23:28.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.650+0000 7f3e46000640 1 -- 172.21.15.73:0/1051439018 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f3e40035090 con 0x7f3e500ad6b0 2024-04-05T22:23:28.650 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:28.650 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":48,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:23:10.337170+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:23:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 msgr2=0x7f3e2c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 0x7f3e2c06d270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e380070b0 tx=0x7f3e38007040 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 msgr2=0x7f3e500c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000c00 tx=0x7f3e400046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 shutdown_connections 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3e2c06adb0 0x7f3e2c06d270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e50160620 0x7f3e50156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e500b1870 0x7f3e5006fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 --2- 172.21.15.73:0/1051439018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500ad6b0 0x7f3e500c37e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 >> 172.21.15.73:0/1051439018 conn(0x7f3e500ce8e0 msgr2=0x7f3e500ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 shutdown_connections 2024-04-05T22:23:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.653+0000 7f3e55200640 1 -- 172.21.15.73:0/1051439018 wait complete. 2024-04-05T22:23:28.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:23:07.732446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:23:08.298551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:23:28.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:23:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.886+0000 7fe56ee00640 1 Processor -- start 2024-04-05T22:23:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.886+0000 7fe56ee00640 1 -- start start 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.888+0000 7fe56ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 0x7fe56806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.888+0000 7fe56ee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe56806ba70 0x7fe568164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.888+0000 7fe56ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56806bea0 con 0x7fe56815abe0 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.888+0000 7fe56ee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56815afc0 con 0x7fe56806ba70 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.888+0000 7fe56ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56806f5b0 con 0x7fe56806c9a0 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe56806ba70 0x7fe568164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe56806ba70 0x7fe568164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41818/0 (socket says 172.21.15.73:41818) 2024-04-05T22:23:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 -- 172.21.15.73:0/823307309 learned_addr learned my addr 172.21.15.73:0/823307309 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe567400640 1 --2- 172.21.15.73:0/823307309 >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 0x7fe56806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 -- 172.21.15.73:0/823307309 >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 msgr2=0x7fe56806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 --2- 172.21.15.73:0/823307309 >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 0x7fe56806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 -- 172.21.15.73:0/823307309 >> v1:172.21.15.157:6789/0 conn(0x7fe56815abe0 legacy=0x7fe5680712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe56ce00640 1 -- 172.21.15.73:0/823307309 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56806f890 con 0x7fe56806ba70 2024-04-05T22:23:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.889+0000 7fe567400640 1 --2- 172.21.15.73:0/823307309 >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 0x7fe56806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.891+0000 7fe56ce00640 1 --2- 172.21.15.73:0/823307309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56806ba70 0x7fe568164740 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe56806c900 tx=0x7fe56002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252add33ad95f4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.891+0000 7fe566a00640 1 -- 172.21.15.73:0/823307309 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56002c930 con 0x7fe56806ba70 2024-04-05T22:23:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.891+0000 7fe566a00640 1 -- 172.21.15.73:0/823307309 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56002ca90 con 0x7fe56806ba70 2024-04-05T22:23:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.891+0000 7fe566a00640 1 -- 172.21.15.73:0/823307309 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56002ce20 con 0x7fe56806ba70 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 -- 172.21.15.73:0/823307309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56806ba70 msgr2=0x7fe568164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 --2- 172.21.15.73:0/823307309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56806ba70 0x7fe568164740 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe56806c900 tx=0x7fe56002bd90 comp rx=0 tx=0).stop 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 -- 172.21.15.73:0/823307309 shutdown_connections 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 --2- 172.21.15.73:0/823307309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56806ba70 0x7fe568164740 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 --2- 172.21.15.73:0/823307309 >> v2:172.21.15.157:3300/0 conn(0x7fe56806c9a0 0x7fe56806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 -- 172.21.15.73:0/823307309 >> 172.21.15.73:0/823307309 conn(0x7fe568069cb0 msgr2=0x7fe56806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 -- 172.21.15.73:0/823307309 shutdown_connections 2024-04-05T22:23:28.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.892+0000 7fe56ee00640 1 -- 172.21.15.73:0/823307309 wait complete. 2024-04-05T22:23:28.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.893+0000 7fe56ee00640 1 Processor -- start 2024-04-05T22:23:28.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.894+0000 7fe56ee00640 1 -- start start 2024-04-05T22:23:28.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.894+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56806ba70 0x7fe5680b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.894+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 0x7fe5681532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56814c820 con 0x7fe56815abe0 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b3890 con 0x7fe56806c9a0 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56814bc10 con 0x7fe56806ba70 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41828/0 (socket says 172.21.15.73:41828) 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 -- 172.21.15.73:0/3362247730 learned_addr learned my addr 172.21.15.73:0/3362247730 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe567400640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 0x7fe5681532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe567e00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56806ba70 0x7fe5680b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 -- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56806ba70 msgr2=0x7fe5680b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56806ba70 0x7fe5680b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 -- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 msgr2=0x7fe5681532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:28.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 0x7fe5681532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:28.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe56ce00640 1 -- 172.21.15.73:0/3362247730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009b30 con 0x7fe56815abe0 2024-04-05T22:23:28.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.895+0000 7fe567400640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 0x7fe5681532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:28.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe56ce00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe560036040 tx=0x7fe560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56003e450 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56003e5d0 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56814bef0 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56003e980 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.896+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56814c4b0 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.897+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56814c990 con 0x7fe56815abe0 2024-04-05T22:23:28.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.898+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fe56003eae0 con 0x7fe56815abe0 2024-04-05T22:23:28.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.899+0000 7fe55fe00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 0x7fe54806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:28.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.899+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5600ac5b0 con 0x7fe56815abe0 2024-04-05T22:23:28.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.899+0000 7fe567e00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 0x7fe54806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:28.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.900+0000 7fe567e00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 0x7fe54806d270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5681f3920 tx=0x7fe558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:28.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:28.904+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe560077520 con 0x7fe56815abe0 2024-04-05T22:23:29.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.098+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe568068160 con 0x7fe56815abe0 2024-04-05T22:23:29.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.098+0000 7fe55fe00640 1 -- 172.21.15.73:0/3362247730 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5600aae60 con 0x7fe56815abe0 2024-04-05T22:23:29.098 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:23:29.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 msgr2=0x7fe54806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 0x7fe54806d270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5681f3920 tx=0x7fe558009040 comp rx=0 tx=0).stop 2024-04-05T22:23:29.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 msgr2=0x7fe5681537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe560036040 tx=0x7fe560004a40 comp rx=0 tx=0).stop 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 shutdown_connections 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe54806adb0 0x7fe54806d270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe56815abe0 0x7fe5681537f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe56806c9a0 0x7fe5681532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 --2- 172.21.15.73:0/3362247730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56806ba70 0x7fe5680b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 >> 172.21.15.73:0/3362247730 conn(0x7fe568069cb0 msgr2=0x7fe5680af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 shutdown_connections 2024-04-05T22:23:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.101+0000 7fe56ee00640 1 -- 172.21.15.73:0/3362247730 wait complete. 2024-04-05T22:23:29.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:23:29.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.329+0000 7ff8bfa00640 1 Processor -- start 2024-04-05T22:23:29.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.329+0000 7ff8bfa00640 1 -- start start 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.330+0000 7ff8bfa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8b806c9a0 0x7ff8b806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.330+0000 7ff8bfa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8b806ba70 0x7ff8b8164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.330+0000 7ff8bfa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b806bea0 con 0x7ff8b815abe0 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.330+0000 7ff8bfa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b815afc0 con 0x7ff8b806c9a0 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.330+0000 7ff8bfa00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b806f5b0 con 0x7ff8b806ba70 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8b806c9a0 0x7ff8b806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8b806c9a0 0x7ff8b806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41842/0 (socket says 172.21.15.73:41842) 2024-04-05T22:23:29.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 -- 172.21.15.73:0/1883200465 learned_addr learned my addr 172.21.15.73:0/1883200465 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:29.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8bda00640 1 --2- 172.21.15.73:0/1883200465 >> v2:172.21.15.157:3300/0 conn(0x7ff8b806ba70 0x7ff8b8164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 -- 172.21.15.73:0/1883200465 >> v2:172.21.15.157:3300/0 conn(0x7ff8b806ba70 msgr2=0x7ff8b8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:29.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 --2- 172.21.15.73:0/1883200465 >> v2:172.21.15.157:3300/0 conn(0x7ff8b806ba70 0x7ff8b8164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 -- 172.21.15.73:0/1883200465 >> v1:172.21.15.157:6789/0 conn(0x7ff8b815abe0 legacy=0x7ff8b80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:29.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.331+0000 7ff8b7e00640 1 -- 172.21.15.73:0/1883200465 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b806f890 con 0x7ff8b806c9a0 2024-04-05T22:23:29.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.332+0000 7ff8b7e00640 1 --2- 172.21.15.73:0/1883200465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806c9a0 0x7ff8b806b530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff8b806ce30 tx=0x7ff8a402b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2629ae075c1a8d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8b7400640 1 -- 172.21.15.73:0/1883200465 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4031520 con 0x7ff8b806c9a0 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8b7400640 1 -- 172.21.15.73:0/1883200465 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a4031680 con 0x7ff8b806c9a0 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8b7400640 1 -- 172.21.15.73:0/1883200465 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4031a10 con 0x7ff8b806c9a0 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1883200465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806c9a0 msgr2=0x7ff8b806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1883200465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806c9a0 0x7ff8b806b530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff8b806ce30 tx=0x7ff8a402b7c0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1883200465 shutdown_connections 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1883200465 >> v2:172.21.15.157:3300/0 conn(0x7ff8b806ba70 0x7ff8b8164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1883200465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806c9a0 0x7ff8b806b530 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1883200465 >> 172.21.15.73:0/1883200465 conn(0x7ff8b8069cb0 msgr2=0x7ff8b806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1883200465 shutdown_connections 2024-04-05T22:23:29.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.333+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1883200465 wait complete. 2024-04-05T22:23:29.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.335+0000 7ff8bfa00640 1 Processor -- start 2024-04-05T22:23:29.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.335+0000 7ff8bfa00640 1 -- start start 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8b806c9a0 0x7ff8b81ed860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8b815abe0 0x7ff8b80aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8153910 con 0x7ff8b806ba70 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8153bf0 con 0x7ff8b806c9a0 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bfa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8153ed0 con 0x7ff8b815abe0 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41856/0 (socket says 172.21.15.73:41856) 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.336+0000 7ff8bd000640 1 -- 172.21.15.73:0/1819660456 learned_addr learned my addr 172.21.15.73:0/1819660456 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bda00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8b815abe0 0x7ff8b80aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8b7e00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8b806c9a0 0x7ff8b81ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 -- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8b815abe0 msgr2=0x7ff8b80aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8b815abe0 0x7ff8b80aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 -- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8b806c9a0 msgr2=0x7ff8b81ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8b806c9a0 0x7ff8b81ed860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 -- 172.21.15.73:0/1819660456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40094d0 con 0x7ff8b806ba70 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bd000640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b3f0 tx=0x7ff8ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011830 con 0x7ff8b806ba70 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac011990 con 0x7ff8b806ba70 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b81541b0 con 0x7ff8b806ba70 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.337+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011d40 con 0x7ff8b806ba70 2024-04-05T22:23:29.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.338+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b81edda0 con 0x7ff8b806ba70 2024-04-05T22:23:29.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.339+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b806b890 con 0x7ff8b806ba70 2024-04-05T22:23:29.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.340+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff8ac01a400 con 0x7ff8b806ba70 2024-04-05T22:23:29.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.340+0000 7ff8b4c00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 0x7ff89806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.341+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8ac0894c0 con 0x7ff8b806ba70 2024-04-05T22:23:29.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.341+0000 7ff8b7e00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 0x7ff89806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.341+0000 7ff8b7e00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 0x7ff89806d270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4035000 tx=0x7ff8a4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.345+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff8ac054420 con 0x7ff8b806ba70 2024-04-05T22:23:29.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.544+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8b8068160 con 0x7ff8b806ba70 2024-04-05T22:23:29.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.545+0000 7ff8b4c00640 1 -- 172.21.15.73:0/1819660456 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8ac057a40 con 0x7ff8b806ba70 2024-04-05T22:23:29.544 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:23:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.547+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 msgr2=0x7ff89806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.547+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 0x7ff89806d270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4035000 tx=0x7ff8a4030000 comp rx=0 tx=0).stop 2024-04-05T22:23:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.547+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 msgr2=0x7ff8b8150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.547+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b3f0 tx=0x7ff8ac00b8b0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 shutdown_connections 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff89806adb0 0x7ff89806d270 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8b815abe0 0x7ff8b80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8b806c9a0 0x7ff8b81ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 --2- 172.21.15.73:0/1819660456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b806ba70 0x7ff8b8150030 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 >> 172.21.15.73:0/1819660456 conn(0x7ff8b8069cb0 msgr2=0x7ff8b80afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 shutdown_connections 2024-04-05T22:23:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.548+0000 7ff8bfa00640 1 -- 172.21.15.73:0/1819660456 wait complete. 2024-04-05T22:23:29.567 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:23:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.788+0000 7fbf4e800640 1 Processor -- start 2024-04-05T22:23:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.788+0000 7fbf4e800640 1 -- start start 2024-04-05T22:23:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.790+0000 7fbf4e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 0x7fbf480ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.790+0000 7fbf4e800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf48146ac0 con 0x7fbf48070c50 2024-04-05T22:23:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.790+0000 7fbf4e800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf48070720 con 0x7fbf4815dd90 2024-04-05T22:23:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.790+0000 7fbf4e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480acf60 con 0x7fbf4806c630 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf47e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbf48070c50 0x7fbf480710e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37792/0 (socket says 172.21.15.73:37792) 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf47e00640 1 -- 172.21.15.73:0/4132204196 learned_addr learned my addr 172.21.15.73:0/4132204196 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf47400640 1 --2- 172.21.15.73:0/4132204196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 0x7fbf480ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089565902 0 0) 0x7fbf48146ac0 con 0x7fbf48070c50 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf2c003520 con 0x7fbf48070c50 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055020462 0 0) 0x7fbf48070720 con 0x7fbf4815dd90 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf48146ac0 con 0x7fbf4815dd90 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf47400640 1 -- 172.21.15.73:0/4132204196 >> v1:172.21.15.157:6789/0 conn(0x7fbf4815dd90 legacy=0x7fbf4806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.791+0000 7fbf47400640 1 -- 172.21.15.73:0/4132204196 >> v1:172.21.15.73:6789/0 conn(0x7fbf48070c50 legacy=0x7fbf480710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.792+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3486927258 0 0) 0x7fbf2c003520 con 0x7fbf48070c50 2024-04-05T22:23:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.792+0000 7fbf47400640 1 -- 172.21.15.73:0/4132204196 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480ad240 con 0x7fbf4806c630 2024-04-05T22:23:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.793+0000 7fbf47400640 1 --2- 172.21.15.73:0/4132204196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 0x7fbf480ac8d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf380097d0 tx=0x7fbf3802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d18aac4e8868838 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400019f0 con 0x7fbf4806c630 2024-04-05T22:23:29.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3802e050 con 0x7fbf4806c630 2024-04-05T22:23:29.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf46a00640 1 -- 172.21.15.73:0/4132204196 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38030710 con 0x7fbf4806c630 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf4e800640 1 -- 172.21.15.73:0/4132204196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 msgr2=0x7fbf480ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf4e800640 1 --2- 172.21.15.73:0/4132204196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 0x7fbf480ac8d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf380097d0 tx=0x7fbf3802b800 comp rx=0 tx=0).stop 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf4e800640 1 -- 172.21.15.73:0/4132204196 shutdown_connections 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf4e800640 1 --2- 172.21.15.73:0/4132204196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf4806c630 0x7fbf480ac8d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.794+0000 7fbf4e800640 1 -- 172.21.15.73:0/4132204196 >> 172.21.15.73:0/4132204196 conn(0x7fbf480b0830 msgr2=0x7fbf480b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.795+0000 7fbf4e800640 1 -- 172.21.15.73:0/4132204196 shutdown_connections 2024-04-05T22:23:29.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.795+0000 7fbf4e800640 1 -- 172.21.15.73:0/4132204196 wait complete. 2024-04-05T22:23:29.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.796+0000 7fbf4e800640 1 Processor -- start 2024-04-05T22:23:29.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.796+0000 7fbf4e800640 1 -- start start 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48070c50 0x7fbf48145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4815dd90 0x7fbf4813d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf481462f0 con 0x7fbf4806c630 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4813df90 con 0x7fbf4815dd90 2024-04-05T22:23:29.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.797+0000 7fbf4e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4813f220 con 0x7fbf48070c50 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41866/0 (socket says 172.21.15.73:41866) 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 -- 172.21.15.73:0/862468167 learned_addr learned my addr 172.21.15.73:0/862468167 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 -- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48070c50 msgr2=0x7fbf48145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf3fe00640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48070c50 0x7fbf48145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48070c50 0x7fbf48145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47e00640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4815dd90 0x7fbf4813d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 -- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4815dd90 msgr2=0x7fbf4813d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4815dd90 0x7fbf4813d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 -- 172.21.15.73:0/862468167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf38009400 con 0x7fbf4806c630 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.798+0000 7fbf47400640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802b840 tx=0x7fbf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.799+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38003ea0 con 0x7fbf4806c630 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.799+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf380044c0 con 0x7fbf4806c630 2024-04-05T22:23:29.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.799+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf48142a10 con 0x7fbf4806c630 2024-04-05T22:23:29.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.799+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38030730 con 0x7fbf4806c630 2024-04-05T22:23:29.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.799+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4813f470 con 0x7fbf4806c630 2024-04-05T22:23:29.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.800+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf48068160 con 0x7fbf4806c630 2024-04-05T22:23:29.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.801+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fbf38039070 con 0x7fbf4806c630 2024-04-05T22:23:29.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.802+0000 7fbf3f400640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 0x7fbf2406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:29.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.802+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf380aaf10 con 0x7fbf4806c630 2024-04-05T22:23:29.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.802+0000 7fbf3fe00640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 0x7fbf2406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:29.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.803+0000 7fbf3fe00640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 0x7fbf2406d270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf300010e0 tx=0x7fbf30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:29.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.806+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbf38075d50 con 0x7fbf4806c630 2024-04-05T22:23:29.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.998+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf480adb90 con 0x7fbf4806c630 2024-04-05T22:23:29.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:29.999+0000 7fbf3f400640 1 -- 172.21.15.73:0/862468167 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf380370a0 con 0x7fbf4806c630 2024-04-05T22:23:29.999 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 msgr2=0x7fbf2406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 0x7fbf2406d270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf300010e0 tx=0x7fbf30009040 comp rx=0 tx=0).stop 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 msgr2=0x7fbf480ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802b840 tx=0x7fbf380046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 shutdown_connections 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbf2406adb0 0x7fbf2406d270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4815dd90 0x7fbf4813d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf48070c50 0x7fbf48145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 --2- 172.21.15.73:0/862468167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf4806c630 0x7fbf480ca7a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.002+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 >> 172.21.15.73:0/862468167 conn(0x7fbf480b0830 msgr2=0x7fbf480afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.003+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 shutdown_connections 2024-04-05T22:23:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.003+0000 7fbf4e800640 1 -- 172.21.15.73:0/862468167 wait complete. 2024-04-05T22:23:30.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:23:30.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.233+0000 7f3bd9c00640 1 Processor -- start 2024-04-05T22:23:30.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.233+0000 7f3bd9c00640 1 -- start start 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.234+0000 7f3bd9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd4160620 0x7f3bd40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd9c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 0x7f3bd40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd9c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd40cbc70 con 0x7f3bd40b1870 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd9c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40ad880 con 0x7f3bd4160620 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd9c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40ce5d0 con 0x7f3bd40cb2f0 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd4160620 0x7f3bd40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd4160620 0x7f3bd40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41870/0 (socket says 172.21.15.73:41870) 2024-04-05T22:23:30.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd2a00640 1 -- 172.21.15.73:0/169634804 learned_addr learned my addr 172.21.15.73:0/169634804 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:30.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.235+0000 7f3bd3400640 1 --2- 172.21.15.73:0/169634804 >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 0x7f3bd40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.236+0000 7f3bd2a00640 1 -- 172.21.15.73:0/169634804 >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 msgr2=0x7f3bd40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:30.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.236+0000 7f3bd2a00640 1 --2- 172.21.15.73:0/169634804 >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 0x7f3bd40cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.236+0000 7f3bd2a00640 1 -- 172.21.15.73:0/169634804 >> v1:172.21.15.157:6789/0 conn(0x7f3bd40b1870 legacy=0x7f3bd40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:30.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.236+0000 7f3bd2a00640 1 -- 172.21.15.73:0/169634804 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd40ce010 con 0x7f3bd4160620 2024-04-05T22:23:30.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.236+0000 7f3bd3400640 1 --2- 172.21.15.73:0/169634804 >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 0x7f3bd40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:30.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.237+0000 7f3bd2a00640 1 --2- 172.21.15.73:0/169634804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd4160620 0x7f3bd40caa60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3bd406f1b0 tx=0x7f3bc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c2383014e8a033 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.237+0000 7f3bd2000640 1 -- 172.21.15.73:0/169634804 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4031870 con 0x7f3bd4160620 2024-04-05T22:23:30.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd2000640 1 -- 172.21.15.73:0/169634804 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc40319d0 con 0x7f3bd4160620 2024-04-05T22:23:30.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd2000640 1 -- 172.21.15.73:0/169634804 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4031d60 con 0x7f3bd4160620 2024-04-05T22:23:30.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 -- 172.21.15.73:0/169634804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd4160620 msgr2=0x7f3bd40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/169634804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd4160620 0x7f3bd40caa60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3bd406f1b0 tx=0x7f3bc402b910 comp rx=0 tx=0).stop 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 -- 172.21.15.73:0/169634804 shutdown_connections 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/169634804 >> v2:172.21.15.157:3300/0 conn(0x7f3bd40cb2f0 0x7f3bd40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/169634804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd4160620 0x7f3bd40caa60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 -- 172.21.15.73:0/169634804 >> 172.21.15.73:0/169634804 conn(0x7f3bd40ce8e0 msgr2=0x7f3bd40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 -- 172.21.15.73:0/169634804 shutdown_connections 2024-04-05T22:23:30.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.238+0000 7f3bd9c00640 1 -- 172.21.15.73:0/169634804 wait complete. 2024-04-05T22:23:30.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.240+0000 7f3bd9c00640 1 Processor -- start 2024-04-05T22:23:30.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.240+0000 7f3bd9c00640 1 -- start start 2024-04-05T22:23:30.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 0x7f3bd415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 0x7f3bd415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40ae3e0 con 0x7f3bd40b1870 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd414cbc0 con 0x7f3bd4160620 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40ae6a0 con 0x7f3bd40cb2f0 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41872/0 (socket says 172.21.15.73:41872) 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.241+0000 7f3bd2a00640 1 -- 172.21.15.73:0/425801728 learned_addr learned my addr 172.21.15.73:0/425801728 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bca000640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 0x7f3bd415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd3400640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 0x7f3bd415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 -- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 msgr2=0x7f3bd415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 0x7f3bd415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 -- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 msgr2=0x7f3bd415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 0x7f3bd415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 -- 172.21.15.73:0/425801728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40094d0 con 0x7f3bd40b1870 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bca000640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 0x7f3bd415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:30.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd3400640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 0x7f3bd415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:30.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.242+0000 7f3bd2a00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000c00 tx=0x7f3bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.243+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4031840 con 0x7f3bd40b1870 2024-04-05T22:23:30.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.243+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc40443f0 con 0x7f3bd40b1870 2024-04-05T22:23:30.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.243+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40447a0 con 0x7f3bd40b1870 2024-04-05T22:23:30.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.243+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd40ae980 con 0x7f3bd40b1870 2024-04-05T22:23:30.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.243+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd40c5350 con 0x7f3bd40b1870 2024-04-05T22:23:30.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.244+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd4066af0 con 0x7f3bd40b1870 2024-04-05T22:23:30.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.245+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f3bc404d450 con 0x7f3bd40b1870 2024-04-05T22:23:30.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.246+0000 7f3bcb400640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 0x7f3bb006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.246+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc402d040 con 0x7f3bd40b1870 2024-04-05T22:23:30.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.246+0000 7f3bca000640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 0x7f3bb006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.247+0000 7f3bca000640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 0x7f3bb006d270 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bd415f390 tx=0x7f3bbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.250+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3bc4076d70 con 0x7f3bd40b1870 2024-04-05T22:23:30.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.455+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3bd40ae7e0 con 0x7f3bd40b1870 2024-04-05T22:23:30.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.456+0000 7f3bcb400640 1 -- 172.21.15.73:0/425801728 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3bd40ae7e0 con 0x7f3bd40b1870 2024-04-05T22:23:30.455 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:23:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.458+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 msgr2=0x7f3bb006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.458+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 0x7f3bb006d270 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bd415f390 tx=0x7f3bbc009040 comp rx=0 tx=0).stop 2024-04-05T22:23:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.458+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 msgr2=0x7f3bd40c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.458+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000c00 tx=0x7f3bc40046c0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 shutdown_connections 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bb006adb0 0x7f3bb006d270 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd4160620 0x7f3bd415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd40cb2f0 0x7f3bd415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 --2- 172.21.15.73:0/425801728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd40b1870 0x7f3bd40c4e10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 >> 172.21.15.73:0/425801728 conn(0x7f3bd40ce8e0 msgr2=0x7f3bd4070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 shutdown_connections 2024-04-05T22:23:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.459+0000 7f3bd9c00640 1 -- 172.21.15.73:0/425801728 wait complete. 2024-04-05T22:23:30.476 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:23:30.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:30.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:23:30.588 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:23:30.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:23:30.588 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:23:30.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.847+0000 7f5ec4e00640 1 Processor -- start 2024-04-05T22:23:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.847+0000 7f5ec4e00640 1 -- start start 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ec0158860 0x7f5ec00b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec015e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 0x7f5ec015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec015eff0 con 0x7f5ec0158860 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec00ccaf0 con 0x7f5ec015eb60 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.848+0000 7f5ec4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec006b500 con 0x7f5ec00b1f00 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ec0158860 0x7f5ec00b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ec0158860 0x7f5ec00b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41888/0 (socket says 172.21.15.73:41888) 2024-04-05T22:23:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 -- 172.21.15.73:0/2804881164 learned_addr learned my addr 172.21.15.73:0/2804881164 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 -- 172.21.15.73:0/2804881164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 msgr2=0x7f5ec015e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5eb7e00640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebfe00640 1 --2- 172.21.15.73:0/2804881164 >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 0x7f5ec015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec015e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 -- 172.21.15.73:0/2804881164 >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 msgr2=0x7f5ec015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:30.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 --2- 172.21.15.73:0/2804881164 >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 0x7f5ec015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebf400640 1 -- 172.21.15.73:0/2804881164 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00b23a0 con 0x7f5ec0158860 2024-04-05T22:23:30.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.849+0000 7f5ebfe00640 1 --2- 172.21.15.73:0/2804881164 >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 0x7f5ec015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:30.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.850+0000 7f5ebf400640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec0158860 0x7f5ec00b19c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00cf9d0 tx=0x7f5eb002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82979e39d5cd7a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.851+0000 7f5ebea00640 1 -- 172.21.15.73:0/2804881164 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00317e0 con 0x7f5ec0158860 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.851+0000 7f5ebea00640 1 -- 172.21.15.73:0/2804881164 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0031940 con 0x7f5ec0158860 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.851+0000 7f5ebea00640 1 -- 172.21.15.73:0/2804881164 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0031cd0 con 0x7f5ec0158860 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.851+0000 7f5ec4e00640 1 -- 172.21.15.73:0/2804881164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec0158860 msgr2=0x7f5ec00b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.851+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec0158860 0x7f5ec00b19c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00cf9d0 tx=0x7f5eb002b880 comp rx=0 tx=0).stop 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 -- 172.21.15.73:0/2804881164 shutdown_connections 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/2804881164 >> v2:172.21.15.157:3300/0 conn(0x7f5ec015eb60 0x7f5ec015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/2804881164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec0158860 0x7f5ec00b19c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 -- 172.21.15.73:0/2804881164 >> 172.21.15.73:0/2804881164 conn(0x7f5ec00b06c0 msgr2=0x7f5ec00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 -- 172.21.15.73:0/2804881164 shutdown_connections 2024-04-05T22:23:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.852+0000 7f5ec4e00640 1 -- 172.21.15.73:0/2804881164 wait complete. 2024-04-05T22:23:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.854+0000 7f5ec4e00640 1 Processor -- start 2024-04-05T22:23:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.854+0000 7f5ec4e00640 1 -- start start 2024-04-05T22:23:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 0x7f5ec00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00ce220 con 0x7f5ec015eb60 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00cd610 con 0x7f5ec00cd170 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5ec4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00cd8f0 con 0x7f5ec00b1f00 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41894/0 (socket says 172.21.15.73:41894) 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.855+0000 7f5eb7e00640 1 -- 172.21.15.73:0/3335717881 learned_addr learned my addr 172.21.15.73:0/3335717881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5ebf400640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5ebfe00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 0x7f5ec00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 -- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 msgr2=0x7f5ec00ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec00ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 -- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 msgr2=0x7f5ec00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 0x7f5ec00c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 -- 172.21.15.73:0/3335717881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00094d0 con 0x7f5ec015eb60 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5ebfe00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 0x7f5ec00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c7f0 tx=0x7f5ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004220 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8004380 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8007860 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.856+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec00cdbd0 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.857+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec0155140 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.858+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec004e160 con 0x7f5ec015eb60 2024-04-05T22:23:30.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.859+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f5ea8007bd0 con 0x7f5ec015eb60 2024-04-05T22:23:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.860+0000 7f5eb7400640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 0x7f5e9c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.860+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ea80897d0 con 0x7f5ec015eb60 2024-04-05T22:23:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.860+0000 7f5ebf400640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 0x7f5e9c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:30.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.861+0000 7f5ebf400640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 0x7f5e9c06d270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036000 tx=0x7f5eb003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:30.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:30.865+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5ea80544f0 con 0x7f5ec015eb60 2024-04-05T22:23:31.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.094+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ec00b3470 con 0x7f5ec015eb60 2024-04-05T22:23:31.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.095+0000 7f5eb7400640 1 -- 172.21.15.73:0/3335717881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f5ea801e020 con 0x7f5ec015eb60 2024-04-05T22:23:31.094 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:31.095 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:31.095 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 msgr2=0x7f5e9c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 0x7f5e9c06d270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036000 tx=0x7f5eb003d040 comp rx=0 tx=0).stop 2024-04-05T22:23:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 msgr2=0x7f5ec00c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c7f0 tx=0x7f5ea800ccb0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 shutdown_connections 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5e9c06adb0 0x7f5e9c06d270 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec00cd170 0x7f5ec00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ec015eb60 0x7f5ec00c7d40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 --2- 172.21.15.73:0/3335717881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec00b1f00 0x7f5ec00ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 >> 172.21.15.73:0/3335717881 conn(0x7f5ec00b06c0 msgr2=0x7f5ec00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 shutdown_connections 2024-04-05T22:23:31.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.098+0000 7f5ec4e00640 1 -- 172.21.15.73:0/3335717881 wait complete. 2024-04-05T22:23:31.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:31.354 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2024-04-05T22:23:31.354 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-05T22:23:31.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol69600 2024-04-05T22:23:31.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.575+0000 7f8153e00640 1 Processor -- start 2024-04-05T22:23:31.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.575+0000 7f8153e00640 1 -- start start 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8153e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c15abe0 0x7f814c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8153e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f814c06bc40 0x7f814c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8153e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c15afc0 con 0x7f814c06c9a0 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8153e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c06f5b0 con 0x7f814c06bc40 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8153e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c06f890 con 0x7f814c15abe0 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8150a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f814c06c9a0 0x7f814c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37808/0 (socket says 172.21.15.73:37808) 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8150a00640 1 -- 172.21.15.73:0/494021382 learned_addr learned my addr 172.21.15.73:0/494021382 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:31.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8151e00640 1 --2- 172.21.15.73:0/494021382 >> v2:172.21.15.73:3300/0 conn(0x7f814c06bc40 0x7f814c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.577+0000 7f8151400640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c15abe0 0x7f814c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f814be00640 1 -- 172.21.15.73:0/494021382 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802507592 0 0) 0x7f814c15afc0 con 0x7f814c06c9a0 2024-04-05T22:23:31.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f814be00640 1 -- 172.21.15.73:0/494021382 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8134003520 con 0x7f814c06c9a0 2024-04-05T22:23:31.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f8151e00640 1 -- 172.21.15.73:0/494021382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c15abe0 msgr2=0x7f814c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:31.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f8151e00640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c15abe0 0x7f814c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f8151e00640 1 -- 172.21.15.73:0/494021382 >> v1:172.21.15.73:6789/0 conn(0x7f814c06c9a0 legacy=0x7f814c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.578+0000 7f8151e00640 1 -- 172.21.15.73:0/494021382 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c06fb70 con 0x7f814c06bc40 2024-04-05T22:23:31.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.579+0000 7f8151e00640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c06bc40 0x7f814c164740 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f814402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61657593128129f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.579+0000 7f814be00640 1 -- 172.21.15.73:0/494021382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814402cb30 con 0x7f814c06bc40 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.579+0000 7f814be00640 1 -- 172.21.15.73:0/494021382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814402cc90 con 0x7f814c06bc40 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.579+0000 7f814be00640 1 -- 172.21.15.73:0/494021382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144034440 con 0x7f814c06bc40 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 -- 172.21.15.73:0/494021382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c06bc40 msgr2=0x7f814c164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c06bc40 0x7f814c164740 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f814402bd80 comp rx=0 tx=0).stop 2024-04-05T22:23:31.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 -- 172.21.15.73:0/494021382 shutdown_connections 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c06bc40 0x7f814c164740 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 --2- 172.21.15.73:0/494021382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c15abe0 0x7f814c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 -- 172.21.15.73:0/494021382 >> 172.21.15.73:0/494021382 conn(0x7f814c069cb0 msgr2=0x7f814c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 -- 172.21.15.73:0/494021382 shutdown_connections 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.580+0000 7f8153e00640 1 -- 172.21.15.73:0/494021382 wait complete. 2024-04-05T22:23:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.582+0000 7f8153e00640 1 Processor -- start 2024-04-05T22:23:31.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.582+0000 7f8153e00640 1 -- start start 2024-04-05T22:23:31.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c06bc40 0x7f814c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 0x7f814c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c146870 con 0x7f814c15abe0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c145c60 con 0x7f814c06c9a0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8153e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c145f40 con 0x7f814c06bc40 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8151e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8151e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41912/0 (socket says 172.21.15.73:41912) 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8151e00640 1 -- 172.21.15.73:0/4259992638 learned_addr learned my addr 172.21.15.73:0/4259992638 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.583+0000 7f8151400640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c06bc40 0x7f814c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8150a00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 0x7f814c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 -- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c06bc40 msgr2=0x7f814c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c06bc40 0x7f814c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 -- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 msgr2=0x7f814c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 0x7f814c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 -- 172.21.15.73:0/4259992638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009a70 con 0x7f814c15abe0 2024-04-05T22:23:31.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8150a00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 0x7f814c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8151e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f8144004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144009450 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8144040450 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144040800 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c146220 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.584+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c1570f0 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.585+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c068160 con 0x7f814c15abe0 2024-04-05T22:23:31.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.586+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f8144040960 con 0x7f814c15abe0 2024-04-05T22:23:31.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.587+0000 7f8149600640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 0x7f812c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:31.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.587+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81440ac350 con 0x7f814c15abe0 2024-04-05T22:23:31.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.587+0000 7f8151400640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 0x7f812c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:31.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.588+0000 7f8151400640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 0x7f812c06d270 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ad0 tx=0x7f8140008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:31.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.592+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8144077330 con 0x7f814c15abe0 2024-04-05T22:23:31.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.811+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol69600", "target": ["mon-mgr", ""]}) v1 -- 0x7f814c071820 con 0x7f812c06adb0 2024-04-05T22:23:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:23:31.819+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:23:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:23:31.819+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:23:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:23:31.819+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:23:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:23:31.819+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:23:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:23:31.819+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:23:31.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.897+0000 7f8149600640 1 -- 172.21.15.73:0/4259992638 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8140009800 con 0x7f812c06adb0 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.900+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 msgr2=0x7f812c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.900+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 0x7f812c06d270 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ad0 tx=0x7f8140008040 comp rx=0 tx=0).stop 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.900+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 msgr2=0x7f814c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.900+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f8144004a40 comp rx=0 tx=0).stop 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 shutdown_connections 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f812c06adb0 0x7f812c06d270 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814c15abe0 0x7f814c15d310 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f814c06c9a0 0x7f814c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 --2- 172.21.15.73:0/4259992638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f814c06bc40 0x7f814c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:31.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 >> 172.21.15.73:0/4259992638 conn(0x7f814c069cb0 msgr2=0x7f814c0ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:31.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 shutdown_connections 2024-04-05T22:23:31.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:31.901+0000 7f8153e00640 1 -- 172.21.15.73:0/4259992638 wait complete. 2024-04-05T22:23:31.920 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.110+0000 7fb037e00640 1 Processor -- start 2024-04-05T22:23:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.111+0000 7fb037e00640 1 -- start start 2024-04-05T22:23:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb038158860 0x7fb0380b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 0x7fb03815e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb03815eb60 0x7fb03815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03815eff0 con 0x7fb03815eb60 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0380ccaf0 con 0x7fb038158860 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03806b500 con 0x7fb0380b1f00 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.112+0000 7fb037400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb03815eb60 0x7fb03815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb03815eb60 0x7fb03815afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41942/0 (socket says 172.21.15.73:41942) 2024-04-05T22:23:32.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 -- 172.21.15.73:0/1276802243 learned_addr learned my addr 172.21.15.73:0/1276802243 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb036a00640 1 --2- 172.21.15.73:0/1276802243 >> v2:172.21.15.157:3300/0 conn(0x7fb038158860 0x7fb0380b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb02fe00640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 0x7fb03815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 -- 172.21.15.73:0/1276802243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 msgr2=0x7fb03815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 0x7fb03815e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 -- 172.21.15.73:0/1276802243 >> v2:172.21.15.157:3300/0 conn(0x7fb038158860 msgr2=0x7fb0380b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 --2- 172.21.15.73:0/1276802243 >> v2:172.21.15.157:3300/0 conn(0x7fb038158860 0x7fb0380b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb037400640 1 -- 172.21.15.73:0/1276802243 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380b23a0 con 0x7fb03815eb60 2024-04-05T22:23:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.113+0000 7fb02fe00640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 0x7fb03815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.114+0000 7fb037400640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03815eb60 0x7fb03815afe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb03806f380 tx=0x7fb03002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e5f31821dfef29 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb036000640 1 -- 172.21.15.73:0/1276802243 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03002c9e0 con 0x7fb03815eb60 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb036000640 1 -- 172.21.15.73:0/1276802243 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03002cb40 con 0x7fb03815eb60 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb036000640 1 -- 172.21.15.73:0/1276802243 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0300343f0 con 0x7fb03815eb60 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 -- 172.21.15.73:0/1276802243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03815eb60 msgr2=0x7fb03815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03815eb60 0x7fb03815afe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb03806f380 tx=0x7fb03002be40 comp rx=0 tx=0).stop 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 -- 172.21.15.73:0/1276802243 shutdown_connections 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03815eb60 0x7fb03815afe0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 --2- 172.21.15.73:0/1276802243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0380b1f00 0x7fb03815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 --2- 172.21.15.73:0/1276802243 >> v2:172.21.15.157:3300/0 conn(0x7fb038158860 0x7fb0380b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 -- 172.21.15.73:0/1276802243 >> 172.21.15.73:0/1276802243 conn(0x7fb0380b06c0 msgr2=0x7fb0380b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.115+0000 7fb037e00640 1 -- 172.21.15.73:0/1276802243 shutdown_connections 2024-04-05T22:23:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.116+0000 7fb037e00640 1 -- 172.21.15.73:0/1276802243 wait complete. 2024-04-05T22:23:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.117+0000 7fb037e00640 1 Processor -- start 2024-04-05T22:23:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.117+0000 7fb037e00640 1 -- start start 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0380b1f00 0x7fb0381f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb03815eb60 0x7fb03813b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb038155a40 con 0x7fb03813b640 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03815dca0 con 0x7fb0380b1f00 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03815d9c0 con 0x7fb03815eb60 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41958/0 (socket says 172.21.15.73:41958) 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.118+0000 7fb037400640 1 -- 172.21.15.73:0/909093177 learned_addr learned my addr 172.21.15.73:0/909093177 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb036a00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0380b1f00 0x7fb0381f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 -- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb03815eb60 msgr2=0x7fb03813b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb03815eb60 0x7fb03813b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 -- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0380b1f00 msgr2=0x7fb0381f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0380b1f00 0x7fb0381f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 -- 172.21.15.73:0/909093177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030009b30 con 0x7fb03813b640 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037400640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb030036040 tx=0x7fb030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030009510 con 0x7fb03813b640 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03003e450 con 0x7fb03813b640 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03003e800 con 0x7fb03813b640 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.119+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03813bf10 con 0x7fb03813b640 2024-04-05T22:23:32.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.120+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03813c4d0 con 0x7fb03813b640 2024-04-05T22:23:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.120+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03813c6c0 con 0x7fb03813b640 2024-04-05T22:23:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.121+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fb03003e960 con 0x7fb03813b640 2024-04-05T22:23:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.122+0000 7fb02f400640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 0x7fb01406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.122+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0300ac520 con 0x7fb03813b640 2024-04-05T22:23:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.122+0000 7fb036a00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 0x7fb01406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.123+0000 7fb036a00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 0x7fb01406d270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb028006fd0 tx=0x7fb028008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.126+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb0300773e0 con 0x7fb03813b640 2024-04-05T22:23:32.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.337+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb03804e160 con 0x7fb03813b640 2024-04-05T22:23:32.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.338+0000 7fb02f400640 1 -- 172.21.15.73:0/909093177 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fb03007aa00 con 0x7fb03813b640 2024-04-05T22:23:32.337 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:32.338 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:32.338 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 msgr2=0x7fb01406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 0x7fb01406d270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb028006fd0 tx=0x7fb028008040 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 msgr2=0x7fb03815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb030036040 tx=0x7fb030004a40 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 shutdown_connections 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb01406adb0 0x7fb01406d270 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb03813b640 0x7fb03815cc50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb03815eb60 0x7fb03813b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 --2- 172.21.15.73:0/909093177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0380b1f00 0x7fb0381f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 >> 172.21.15.73:0/909093177 conn(0x7fb0380b06c0 msgr2=0x7fb0380b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 shutdown_connections 2024-04-05T22:23:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.341+0000 7fb037e00640 1 -- 172.21.15.73:0/909093177 wait complete. 2024-04-05T22:23:32.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:32.589 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.75098665899999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.963294529120066,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:32.590 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.963294529120066 }, "state": "open", "uptime": 0.750986659 } ] 2024-04-05T22:23:32.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:32.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.807+0000 7f7702c00640 1 Processor -- start 2024-04-05T22:23:32.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.807+0000 7f7702c00640 1 -- start start 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.808+0000 7f7702c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76fc158860 0x7f76fc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.808+0000 7f7702c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc15ed30 0x7f76fc06bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.808+0000 7f7702c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc06b500 con 0x7f76fc0b1f00 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.808+0000 7f7702c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc0ccaf0 con 0x7f76fc15ed30 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.808+0000 7f7702c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc0b23a0 con 0x7f76fc158860 2024-04-05T22:23:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc15ed30 0x7f76fc06bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc15ed30 0x7f76fc06bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41964/0 (socket says 172.21.15.73:41964) 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 -- 172.21.15.73:0/116175681 learned_addr learned my addr 172.21.15.73:0/116175681 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 -- 172.21.15.73:0/116175681 >> v2:172.21.15.157:3300/0 conn(0x7f76fc158860 msgr2=0x7f76fc0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 --2- 172.21.15.73:0/116175681 >> v2:172.21.15.157:3300/0 conn(0x7f76fc158860 0x7f76fc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 -- 172.21.15.73:0/116175681 >> v1:172.21.15.157:6789/0 conn(0x7f76fc0b1f00 legacy=0x7f76fc15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:32.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.809+0000 7f7702200640 1 -- 172.21.15.73:0/116175681 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0b2680 con 0x7f76fc15ed30 2024-04-05T22:23:32.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.810+0000 7f7702200640 1 --2- 172.21.15.73:0/116175681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc06bc70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f76fc06f380 tx=0x7f76f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2d42b5386ffd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f76fbe00640 1 -- 172.21.15.73:0/116175681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40317e0 con 0x7f76fc15ed30 2024-04-05T22:23:32.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f76fbe00640 1 -- 172.21.15.73:0/116175681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4031940 con 0x7f76fc15ed30 2024-04-05T22:23:32.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f76fbe00640 1 -- 172.21.15.73:0/116175681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4031cd0 con 0x7f76fc15ed30 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 -- 172.21.15.73:0/116175681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 msgr2=0x7f76fc06bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 --2- 172.21.15.73:0/116175681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc06bc70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f76fc06f380 tx=0x7f76f402b880 comp rx=0 tx=0).stop 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 -- 172.21.15.73:0/116175681 shutdown_connections 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 --2- 172.21.15.73:0/116175681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc06bc70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 --2- 172.21.15.73:0/116175681 >> v2:172.21.15.157:3300/0 conn(0x7f76fc158860 0x7f76fc0b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 -- 172.21.15.73:0/116175681 >> 172.21.15.73:0/116175681 conn(0x7f76fc0b06c0 msgr2=0x7f76fc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 -- 172.21.15.73:0/116175681 shutdown_connections 2024-04-05T22:23:32.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.811+0000 7f7702c00640 1 -- 172.21.15.73:0/116175681 wait complete. 2024-04-05T22:23:32.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.813+0000 7f7702c00640 1 Processor -- start 2024-04-05T22:23:32.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.813+0000 7f7702c00640 1 -- start start 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 0x7f76fc15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc159dd0 0x7f76fc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc15a7c0 con 0x7f76fc15ed30 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc14da20 con 0x7f76fc0b1f00 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7702c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0ad1b0 con 0x7f76fc159dd0 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7700e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7700e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41978/0 (socket says 172.21.15.73:41978) 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.814+0000 7f7700e00640 1 -- 172.21.15.73:0/108154182 learned_addr learned my addr 172.21.15.73:0/108154182 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7701800640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 0x7f76fc15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 -- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc159dd0 msgr2=0x7f76fc15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7702200640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc159dd0 0x7f76fc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc159dd0 0x7f76fc15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 -- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 msgr2=0x7f76fc15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 0x7f76fc15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 -- 172.21.15.73:0/108154182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f40094d0 con 0x7f76fc15ed30 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7701800640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 0x7f76fc15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7700e00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b410 tx=0x7f76e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8011830 con 0x7f76fc15ed30 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e8011990 con 0x7f76fc15ed30 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc152f80 con 0x7f76fc15ed30 2024-04-05T22:23:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.815+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8011d40 con 0x7f76fc15ed30 2024-04-05T22:23:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.816+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc0acb60 con 0x7f76fc15ed30 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.817+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc0ad340 con 0x7f76fc15ed30 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.818+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f76e801a400 con 0x7f76fc15ed30 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.818+0000 7f76f9600640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.819+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e8089980 con 0x7f76fc15ed30 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.819+0000 7f7701800640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.819+0000 7f7701800640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 0x7f76dc06d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007eb0 tx=0x7f76f00073c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:32.823+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f76e8054760 con 0x7f76fc15ed30 2024-04-05T22:23:33.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.055+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76fc04e160 con 0x7f76fc15ed30 2024-04-05T22:23:33.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.057+0000 7f76f9600640 1 -- 172.21.15.73:0/108154182 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f76e8015020 con 0x7f76fc15ed30 2024-04-05T22:23:33.056 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:33.057 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:33.057 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.059+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 msgr2=0x7f76dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.059+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 0x7f76dc06d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007eb0 tx=0x7f76f00073c0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.059+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 msgr2=0x7f76fc14d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.059+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b410 tx=0x7f76e800b8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 shutdown_connections 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc159dd0 0x7f76fc15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc15ed30 0x7f76fc14d4e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 --2- 172.21.15.73:0/108154182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc0b1f00 0x7f76fc15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:33.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 >> 172.21.15.73:0/108154182 conn(0x7f76fc0b06c0 msgr2=0x7f76fc0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:33.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 shutdown_connections 2024-04-05T22:23:33.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:33.060+0000 7f7702c00640 1 -- 172.21.15.73:0/108154182 wait complete. 2024-04-05T22:23:33.077 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:33.313 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.4749737970000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.926585799351397,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:33.313 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.926585799351397 }, "state": "open", "uptime": 1.474973797 } ] 2024-04-05T22:23:33.313 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:23:38.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:23:38.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:23:38.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:38.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.527+0000 7f8c04800640 1 Processor -- start 2024-04-05T22:23:38.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.527+0000 7f8c04800640 1 -- start start 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.528+0000 7f8c04800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c04800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc06f5b0 con 0x7f8bfc15e850 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c04800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc070410 con 0x7f8bfc0ccd20 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c04800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc158bb0 con 0x7f8bfc1587d0 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c02800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8bfc15e850 0x7f8bfc0708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59502/0 (socket says 172.21.15.73:59502) 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48050/0 (socket says 172.21.15.73:48050) 2024-04-05T22:23:38.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 -- 172.21.15.73:0/2295290498 learned_addr learned my addr 172.21.15.73:0/2295290498 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:38.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 -- 172.21.15.73:0/2295290498 >> v1:172.21.15.157:6789/0 conn(0x7f8bfc0ccd20 legacy=0x7f8bfc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:38.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 -- 172.21.15.73:0/2295290498 >> v1:172.21.15.73:6789/0 conn(0x7f8bfc15e850 legacy=0x7f8bfc0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c01e00640 1 -- 172.21.15.73:0/2295290498 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc06f800 con 0x7f8bfc1587d0 2024-04-05T22:23:38.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.529+0000 7f8c00a00640 1 -- 172.21.15.73:0/2295290498 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669470628 0 0) 0x7f8bfc06f5b0 con 0x7f8bfc15e850 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.530+0000 7f8c01e00640 1 --2- 172.21.15.73:0/2295290498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00098a0 tx=0x7f8bf002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba2702d7c998ee server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c00a00640 1 -- 172.21.15.73:0/2295290498 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0030840 con 0x7f8bfc1587d0 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c00a00640 1 -- 172.21.15.73:0/2295290498 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf00309a0 con 0x7f8bfc1587d0 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c00a00640 1 -- 172.21.15.73:0/2295290498 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0030d30 con 0x7f8bfc1587d0 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c04800640 1 -- 172.21.15.73:0/2295290498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc1587d0 msgr2=0x7f8bfc0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c04800640 1 --2- 172.21.15.73:0/2295290498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00098a0 tx=0x7f8bf002b980 comp rx=0 tx=0).stop 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c04800640 1 -- 172.21.15.73:0/2295290498 shutdown_connections 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c04800640 1 --2- 172.21.15.73:0/2295290498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc1587d0 0x7f8bfc0cc7e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.531+0000 7f8c04800640 1 -- 172.21.15.73:0/2295290498 >> 172.21.15.73:0/2295290498 conn(0x7f8bfc0ce8e0 msgr2=0x7f8bfc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.532+0000 7f8c04800640 1 -- 172.21.15.73:0/2295290498 shutdown_connections 2024-04-05T22:23:38.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.532+0000 7f8c04800640 1 -- 172.21.15.73:0/2295290498 wait complete. 2024-04-05T22:23:38.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.533+0000 7f8c04800640 1 Processor -- start 2024-04-05T22:23:38.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.533+0000 7f8c04800640 1 -- start start 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 0x7f8bfc0ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 0x7f8bfc0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b0020 con 0x7f8bfc14a270 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0af780 con 0x7f8bfc15e850 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c04800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0afa60 con 0x7f8bfc1587d0 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.534+0000 7f8c02800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48056/0 (socket says 172.21.15.73:48056) 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 -- 172.21.15.73:0/1490383204 learned_addr learned my addr 172.21.15.73:0/1490383204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:38.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c01e00640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 0x7f8bfc0ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 -- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 msgr2=0x7f8bfc0ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 0x7f8bfc0ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 -- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 msgr2=0x7f8bfc0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c01400640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 0x7f8bfc0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 0x7f8bfc0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 -- 172.21.15.73:0/1490383204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00094d0 con 0x7f8bfc14a270 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c01e00640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 0x7f8bfc0ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c01400640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 0x7f8bfc0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.535+0000 7f8c02800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c700 tx=0x7f8bf800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:38.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.536+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800ce10 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.536+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf800a640 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.536+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800aa70 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.536+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc0afd40 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.536+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc0af470 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.537+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc066af0 con 0x7f8bfc14a270 2024-04-05T22:23:38.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.538+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f8bf800abd0 con 0x7f8bfc14a270 2024-04-05T22:23:38.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.539+0000 7f8bf6000640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 0x7f8be006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:38.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.539+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bf8088d60 con 0x7f8bfc14a270 2024-04-05T22:23:38.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.539+0000 7f8c01e00640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 0x7f8be006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:38.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.540+0000 7f8c01e00640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 0x7f8be006d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0036000 tx=0x7f8bf003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:38.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.544+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8bf8053b40 con 0x7f8bfc14a270 2024-04-05T22:23:38.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.775+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bfc0a85e0 con 0x7f8bfc14a270 2024-04-05T22:23:38.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.776+0000 7f8bf6000640 1 -- 172.21.15.73:0/1490383204 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f8bfc0a85e0 con 0x7f8bfc14a270 2024-04-05T22:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:38.777 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 msgr2=0x7f8be006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 0x7f8be006d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0036000 tx=0x7f8bf003b040 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 msgr2=0x7f8bfc15c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c700 tx=0x7f8bf800cbc0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 shutdown_connections 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8be006ae40 0x7f8be006d300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc14a270 0x7f8bfc15c420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc15e850 0x7f8bfc0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 --2- 172.21.15.73:0/1490383204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc1587d0 0x7f8bfc0ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:38.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.779+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 >> 172.21.15.73:0/1490383204 conn(0x7f8bfc0ce8e0 msgr2=0x7f8bfc0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:38.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.780+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 shutdown_connections 2024-04-05T22:23:38.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:38.780+0000 7f8c04800640 1 -- 172.21.15.73:0/1490383204 wait complete. 2024-04-05T22:23:38.799 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:39.040 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":7.2018720549999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.638365493509284,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:39.040 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.638365493509284 }, "state": "open", "uptime": 7.201872055 } ] 2024-04-05T22:23:39.041 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:23:44.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:44.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.249+0000 7fc401600640 1 Processor -- start 2024-04-05T22:23:44.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.249+0000 7fc401600640 1 -- start start 2024-04-05T22:23:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc401600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3fc160620 0x7fc3fc0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc401600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc401600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0ad880 con 0x7fc3fc0b1870 2024-04-05T22:23:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc401600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0cbcc0 con 0x7fc3fc160620 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc401600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0ce5d0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc3faa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3fc160620 0x7fc3fc0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc3faa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3fc160620 0x7fc3fc0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48058/0 (socket says 172.21.15.73:48058) 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.251+0000 7fc3faa00640 1 -- 172.21.15.73:0/3432733808 learned_addr learned my addr 172.21.15.73:0/3432733808 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.252+0000 7fc3faa00640 1 -- 172.21.15.73:0/3432733808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc0cb2f0 msgr2=0x7fc3fc0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.252+0000 7fc3faa00640 1 --2- 172.21.15.73:0/3432733808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.252+0000 7fc3faa00640 1 -- 172.21.15.73:0/3432733808 >> v1:172.21.15.157:6789/0 conn(0x7fc3fc0b1870 legacy=0x7fc3fc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:44.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.252+0000 7fc3faa00640 1 -- 172.21.15.73:0/3432733808 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0ce010 con 0x7fc3fc160620 2024-04-05T22:23:44.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.253+0000 7fc3faa00640 1 --2- 172.21.15.73:0/3432733808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc160620 0x7fc3fc0caa60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0667b0 tx=0x7fc3ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb728cd9ad5a3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:44.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.253+0000 7fc3f9600640 1 -- 172.21.15.73:0/3432733808 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec031640 con 0x7fc3fc160620 2024-04-05T22:23:44.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.253+0000 7fc3f9600640 1 -- 172.21.15.73:0/3432733808 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0317a0 con 0x7fc3fc160620 2024-04-05T22:23:44.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.253+0000 7fc3f9600640 1 -- 172.21.15.73:0/3432733808 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec031b30 con 0x7fc3fc160620 2024-04-05T22:23:44.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 -- 172.21.15.73:0/3432733808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc160620 msgr2=0x7fc3fc0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 --2- 172.21.15.73:0/3432733808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc160620 0x7fc3fc0caa60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0667b0 tx=0x7fc3ec02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 -- 172.21.15.73:0/3432733808 shutdown_connections 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 --2- 172.21.15.73:0/3432733808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 --2- 172.21.15.73:0/3432733808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc160620 0x7fc3fc0caa60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 -- 172.21.15.73:0/3432733808 >> 172.21.15.73:0/3432733808 conn(0x7fc3fc0ce8e0 msgr2=0x7fc3fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 -- 172.21.15.73:0/3432733808 shutdown_connections 2024-04-05T22:23:44.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.254+0000 7fc401600640 1 -- 172.21.15.73:0/3432733808 wait complete. 2024-04-05T22:23:44.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.255+0000 7fc401600640 1 Processor -- start 2024-04-05T22:23:44.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 -- start start 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3fc0b1870 0x7fc3fc147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc160620 0x7fc3fc0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc15c1b0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc147860 con 0x7fc3fc0b1870 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.256+0000 7fc401600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc147cc0 con 0x7fc3fc160620 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48064/0 (socket says 172.21.15.73:48064) 2024-04-05T22:23:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 -- 172.21.15.73:0/1437833718 learned_addr learned my addr 172.21.15.73:0/1437833718 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:44.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 -- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc160620 msgr2=0x7fc3fc0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:44.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc160620 0x7fc3fc0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 -- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3fc0b1870 msgr2=0x7fc3fc147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:44.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3fc0b1870 0x7fc3fc147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 -- 172.21.15.73:0/1437833718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0094d0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3fa000640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c410 tx=0x7fc3e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.257+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80035f0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.258+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8003750 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.258+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8003b40 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.258+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc147fa0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.258+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc148560 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.259+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc066af0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.260+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fc3e8003ca0 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.261+0000 7fc3f2a00640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 0x7fc3dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:44.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.261+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3e8088a00 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.261+0000 7fc3faa00640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 0x7fc3dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:44.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.262+0000 7fc3faa00640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 0x7fc3dc06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec036000 tx=0x7fc3ec030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:44.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.266+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc3e8053870 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.496+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3fc068160 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.498+0000 7fc3f2a00640 1 -- 172.21.15.73:0/1437833718 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fc3e8056e90 con 0x7fc3fc0cb2f0 2024-04-05T22:23:44.503 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:44.503 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 msgr2=0x7fc3dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 0x7fc3dc06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec036000 tx=0x7fc3ec030000 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 msgr2=0x7fc3fc0c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c410 tx=0x7fc3e800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 shutdown_connections 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3dc06ae40 0x7fc3dc06d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3fc160620 0x7fc3fc0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3fc0cb2f0 0x7fc3fc0c37e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 --2- 172.21.15.73:0/1437833718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3fc0b1870 0x7fc3fc147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 >> 172.21.15.73:0/1437833718 conn(0x7fc3fc0ce8e0 msgr2=0x7fc3fc070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:44.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 shutdown_connections 2024-04-05T22:23:44.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:44.501+0000 7fc401600640 1 -- 172.21.15.73:0/1437833718 wait complete. 2024-04-05T22:23:44.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:44.756 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":12.917770505,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.354476498393836,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:44.756 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.354476498393836 }, "state": "open", "uptime": 12.917770505 } ] 2024-04-05T22:23:44.757 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:23:49.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.972+0000 7f7285400640 1 Processor -- start 2024-04-05T22:23:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.972+0000 7f7285400640 1 -- start start 2024-04-05T22:23:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7285400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72800cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7285400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f728015e850 0x7f728015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7285400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728006f5b0 con 0x7f72800ccd20 2024-04-05T22:23:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7285400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728015f240 con 0x7f728015e850 2024-04-05T22:23:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7285400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280070410 con 0x7f72801587d0 2024-04-05T22:23:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7275e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f72800ccd20 0x7f728006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38662/0 (socket says 172.21.15.73:38662) 2024-04-05T22:23:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.974+0000 7f7275e00640 1 -- 172.21.15.73:0/4254088580 learned_addr learned my addr 172.21.15.73:0/4254088580 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727f200640 1 --2- 172.21.15.73:0/4254088580 >> v2:172.21.15.157:3300/0 conn(0x7f728015e850 0x7f728015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727e800640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72800cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454266389 0 0) 0x7f728006f5b0 con 0x7f72800ccd20 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7264003520 con 0x7f72800ccd20 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780188957 0 0) 0x7f7264003520 con 0x7f72800ccd20 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 msgr2=0x7f72800cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72800cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 >> v2:172.21.15.157:3300/0 conn(0x7f728015e850 msgr2=0x7f728015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 --2- 172.21.15.73:0/4254088580 >> v2:172.21.15.157:3300/0 conn(0x7f728015e850 0x7f728015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280158bb0 con 0x7f72800ccd20 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7268001670 con 0x7f72800ccd20 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 >> v1:172.21.15.73:6789/0 conn(0x7f72800ccd20 legacy=0x7f728006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7264003970 0x7f7264003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264004d00 0x7f7264005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728015f240 con 0x7f7264004d00 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280070410 con 0x7f7264003970 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.975+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726400b980 con 0x7f72801587d0 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264004d00 0x7f7264005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727e800640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7264003970 0x7f7264003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f7275e00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 -- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 msgr2=0x7f72640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 -- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7264003970 msgr2=0x7f7264003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7264003970 0x7f7264003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727f200640 1 -- 172.21.15.73:0/4254088580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7264003520 con 0x7f7264004d00 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f727e800640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7264003970 0x7f7264003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.976+0000 7f7275e00640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801587d0 0x7f72640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.977+0000 7f727f200640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264004d00 0x7f7264005180 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fc0 tx=0x7f72640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff11c66a53afb20f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727802cc30 con 0x7f7264004d00 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800f040 con 0x7f7264004d00 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f727de00640 1 -- 172.21.15.73:0/4254088580 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278034440 con 0x7f7264004d00 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 -- 172.21.15.73:0/4254088580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264004d00 msgr2=0x7f7264005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 --2- 172.21.15.73:0/4254088580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264004d00 0x7f7264005180 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fc0 tx=0x7f72640057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f727f200640 1 -- 172.21.15.73:0/4254088580 reap_dead start 2024-04-05T22:23:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 -- 172.21.15.73:0/4254088580 shutdown_connections 2024-04-05T22:23:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 -- 172.21.15.73:0/4254088580 >> 172.21.15.73:0/4254088580 conn(0x7f72800ce8e0 msgr2=0x7f72800cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 -- 172.21.15.73:0/4254088580 shutdown_connections 2024-04-05T22:23:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.978+0000 7f7285400640 1 -- 172.21.15.73:0/4254088580 wait complete. 2024-04-05T22:23:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.980+0000 7f7285400640 1 Processor -- start 2024-04-05T22:23:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.980+0000 7f7285400640 1 -- start start 2024-04-05T22:23:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801f5b80 0x7f72800ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72800aee20 0x7f72800c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800af2a0 con 0x7f72800c4df0 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800b3870 con 0x7f72800aee20 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f7285400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800c5250 con 0x7f72801f5b80 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f727f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f727f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41328/0 (socket says 172.21.15.73:41328) 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.981+0000 7f727f200640 1 -- 172.21.15.73:0/1344111606 learned_addr learned my addr 172.21.15.73:0/1344111606 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f7275e00640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72800aee20 0x7f72800c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727e800640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801f5b80 0x7f72800ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 -- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801f5b80 msgr2=0x7f72800ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801f5b80 0x7f72800ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 -- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72800aee20 msgr2=0x7f72800c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72800aee20 0x7f72800c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 -- 172.21.15.73:0/1344111606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72801f5f60 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f727f200640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7278037040 tx=0x7f7278009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278043070 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.982+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727802ce20 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.983+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72800b29c0 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.983+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278034d90 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.983+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72800b2f40 con 0x7f72800c4df0 2024-04-05T22:23:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.984+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280066af0 con 0x7f72800c4df0 2024-04-05T22:23:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.985+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f727803e440 con 0x7f72800c4df0 2024-04-05T22:23:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.985+0000 7f7277400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 0x7f725c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.986+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72780ab1c0 con 0x7f72800c4df0 2024-04-05T22:23:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.986+0000 7f727e800640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 0x7f725c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.986+0000 7f727e800640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 0x7f725c06d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7270009aa0 tx=0x7f7270008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:49.991+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7278075fa0 con 0x7f72800c4df0 2024-04-05T22:23:50.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.219+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f728015efb0 con 0x7f72800c4df0 2024-04-05T22:23:50.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.221+0000 7f7277400640 1 -- 172.21.15.73:0/1344111606 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f7278039090 con 0x7f72800c4df0 2024-04-05T22:23:50.221 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:50.221 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:50.221 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.223+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 msgr2=0x7f725c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.223+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 0x7f725c06d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7270009aa0 tx=0x7f7270008040 comp rx=0 tx=0).stop 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.223+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 msgr2=0x7f72801399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.223+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7278037040 tx=0x7f7278009750 comp rx=0 tx=0).stop 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 shutdown_connections 2024-04-05T22:23:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725c06ae40 0x7f725c06d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800c4df0 0x7f72801399d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72800aee20 0x7f72800c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 --2- 172.21.15.73:0/1344111606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72801f5b80 0x7f72800ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 >> 172.21.15.73:0/1344111606 conn(0x7f72800ce8e0 msgr2=0x7f728015a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 shutdown_connections 2024-04-05T22:23:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:50.224+0000 7f7285400640 1 -- 172.21.15.73:0/1344111606 wait complete. 2024-04-05T22:23:50.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:50.480 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.639668856,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.074019898296854,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:50.480 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.074019898296854 }, "state": "open", "uptime": 18.639668856 } ] 2024-04-05T22:23:50.481 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:23:55.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:23:55.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.697+0000 7f468ca00640 1 Processor -- start 2024-04-05T22:23:55.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.698+0000 7f468ca00640 1 -- start start 2024-04-05T22:23:55.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f468ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f468806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f468ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880cb950 con 0x7f46880cd7a0 2024-04-05T22:23:55.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f468ca00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880cdce0 con 0x7f46880d09b0 2024-04-05T22:23:55.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f468ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880ce990 con 0x7f468815cf50 2024-04-05T22:23:55.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f4686800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46880cd7a0 0x7f46880cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38668/0 (socket says 172.21.15.73:38668) 2024-04-05T22:23:55.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.699+0000 7f4686800640 1 -- 172.21.15.73:0/2741364780 learned_addr learned my addr 172.21.15.73:0/2741364780 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:23:55.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778289064 0 0) 0x7f46880cb950 con 0x7f46880cd7a0 2024-04-05T22:23:55.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f466c003520 con 0x7f46880cd7a0 2024-04-05T22:23:55.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685e00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f468806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3838684107 0 0) 0x7f466c003520 con 0x7f46880cd7a0 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 msgr2=0x7f468806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f468806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 >> v1:172.21.15.157:6789/0 conn(0x7f46880d09b0 legacy=0x7f46880cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880cdfc0 con 0x7f46880cd7a0 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f46800019f0 con 0x7f46880cd7a0 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 >> v1:172.21.15.73:6789/0 conn(0x7f46880cd7a0 legacy=0x7f46880cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f466c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f466c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c004d00 0x7f466c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880ce990 con 0x7f466c0039a0 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00bb10 con 0x7f466c004d00 2024-04-05T22:23:55.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.700+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00bca0 con 0x7f468815cf50 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f466c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f467fe00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f466c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4686800640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c004d00 0x7f466c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 -- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 msgr2=0x7f466c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f468815cf50 0x7f466c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 -- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c004d00 msgr2=0x7f466c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c004d00 0x7f466c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4685e00640 1 -- 172.21.15.73:0/2741364780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c003520 con 0x7f466c0039a0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.701+0000 7f4686800640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c004d00 0x7f466c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.702+0000 7f4685e00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f466c003dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f467800ba60 tx=0x7f466c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d637dbd401a67381 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.702+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678032630 con 0x7f466c0039a0 2024-04-05T22:23:55.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678032790 con 0x7f466c0039a0 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f4685400640 1 -- 172.21.15.73:0/2741364780 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678032b20 con 0x7f466c0039a0 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 -- 172.21.15.73:0/2741364780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 msgr2=0x7f466c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 --2- 172.21.15.73:0/2741364780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f466c003dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f467800ba60 tx=0x7f466c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f4686800640 1 -- 172.21.15.73:0/2741364780 reap_dead start 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 -- 172.21.15.73:0/2741364780 shutdown_connections 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 -- 172.21.15.73:0/2741364780 >> 172.21.15.73:0/2741364780 conn(0x7f4688099a60 msgr2=0x7f4688099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 -- 172.21.15.73:0/2741364780 shutdown_connections 2024-04-05T22:23:55.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.703+0000 7f468ca00640 1 -- 172.21.15.73:0/2741364780 wait complete. 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.705+0000 7f468ca00640 1 Processor -- start 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.705+0000 7f468ca00640 1 -- start start 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4688150f80 0x7f46880b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880b63c0 0x7f46880b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880ca2b0 con 0x7f466c0039a0 2024-04-05T22:23:55.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880b6840 con 0x7f4688150f80 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f468ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c96e0 con 0x7f46880b63c0 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f4685e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f4685e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41336/0 (socket says 172.21.15.73:41336) 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.706+0000 7f4685e00640 1 -- 172.21.15.73:0/2866473505 learned_addr learned my addr 172.21.15.73:0/2866473505 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f467fe00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4688150f80 0x7f46880b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 -- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880b63c0 msgr2=0x7f46880b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880b63c0 0x7f46880b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 -- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4688150f80 msgr2=0x7f46880b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4688150f80 0x7f46880b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 -- 172.21.15.73:0/2866473505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880c9980 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f4685e00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f467802beb0 tx=0x7f4678004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678004220 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678004380 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46880c9c60 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467803ba90 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.707+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46881474d0 con 0x7f466c0039a0 2024-04-05T22:23:55.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.709+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688068160 con 0x7f466c0039a0 2024-04-05T22:23:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.709+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f467803bbf0 con 0x7f466c0039a0 2024-04-05T22:23:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.710+0000 7f467e000640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:23:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.710+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46780ac0e0 con 0x7f466c0039a0 2024-04-05T22:23:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.711+0000 7f467fe00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:23:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.711+0000 7f467fe00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 0x7f466406d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4670007fd0 tx=0x7f4670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:23:55.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.715+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4678076ec0 con 0x7f466c0039a0 2024-04-05T22:23:55.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.944+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4688068160 con 0x7f466c0039a0 2024-04-05T22:23:55.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.946+0000 7f467e000640 1 -- 172.21.15.73:0/2866473505 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f467807a4e0 con 0x7f466c0039a0 2024-04-05T22:23:55.946 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:23:55.946 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:23:55.947 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 msgr2=0x7f466406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 0x7f466406d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4670007fd0 tx=0x7f4670009040 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 msgr2=0x7f4688146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f467802beb0 tx=0x7f4678004730 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 shutdown_connections 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880b63c0 0x7f46880b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4688150f80 0x7f46880b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 --2- 172.21.15.73:0/2866473505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466c0039a0 0x7f4688146f90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 >> 172.21.15.73:0/2866473505 conn(0x7f4688099a60 msgr2=0x7f46880ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:23:55.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 shutdown_connections 2024-04-05T22:23:55.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:23:55.948+0000 7f468ca00640 1 -- 172.21.15.73:0/2866473505 wait complete. 2024-04-05T22:23:55.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:23:56.201 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":24.363567163999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.797150541816357,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:23:56.201 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.797150541816357 }, "state": "open", "uptime": 24.363567164 } ] 2024-04-05T22:23:56.201 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:01.202 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:01.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.419+0000 7ff8c3000640 1 Processor -- start 2024-04-05T22:24:01.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.420+0000 7ff8c3000640 1 -- start start 2024-04-05T22:24:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.421+0000 7ff8c3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.421+0000 7ff8c3000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8bc0b1f00 0x7ff8bc15e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.421+0000 7ff8c3000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc06b500 con 0x7ff8bc15eb60 2024-04-05T22:24:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.421+0000 7ff8c3000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc15eff0 con 0x7ff8bc0b1f00 2024-04-05T22:24:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.421+0000 7ff8c3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0ccaf0 con 0x7ff8bc158860 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.422+0000 7ff8c1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.422+0000 7ff8c1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:39332/0 (socket says 172.21.15.73:39332) 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.422+0000 7ff8c1c00640 1 -- 172.21.15.73:0/3657703974 learned_addr learned my addr 172.21.15.73:0/3657703974 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.422+0000 7ff8c1200640 1 --2- 172.21.15.73:0/3657703974 >> v2:172.21.15.157:3300/0 conn(0x7ff8bc0b1f00 0x7ff8bc15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.423+0000 7ff8c1c00640 1 -- 172.21.15.73:0/3657703974 >> v2:172.21.15.157:3300/0 conn(0x7ff8bc0b1f00 msgr2=0x7ff8bc15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.423+0000 7ff8c1c00640 1 --2- 172.21.15.73:0/3657703974 >> v2:172.21.15.157:3300/0 conn(0x7ff8bc0b1f00 0x7ff8bc15e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.423+0000 7ff8c1c00640 1 -- 172.21.15.73:0/3657703974 >> v1:172.21.15.157:6789/0 conn(0x7ff8bc15eb60 legacy=0x7ff8bc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.423+0000 7ff8c1c00640 1 -- 172.21.15.73:0/3657703974 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0b23a0 con 0x7ff8bc158860 2024-04-05T22:24:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.424+0000 7ff8c1c00640 1 --2- 172.21.15.73:0/3657703974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009870 tx=0x7ff8b002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4b37cae14d6c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:01.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.425+0000 7ff8bbe00640 1 -- 172.21.15.73:0/3657703974 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0031640 con 0x7ff8bc158860 2024-04-05T22:24:01.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.425+0000 7ff8bbe00640 1 -- 172.21.15.73:0/3657703974 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b00317a0 con 0x7ff8bc158860 2024-04-05T22:24:01.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.425+0000 7ff8bbe00640 1 -- 172.21.15.73:0/3657703974 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0031b30 con 0x7ff8bc158860 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.425+0000 7ff8c3000640 1 -- 172.21.15.73:0/3657703974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 msgr2=0x7ff8bc0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.425+0000 7ff8c3000640 1 --2- 172.21.15.73:0/3657703974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009870 tx=0x7ff8b002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 -- 172.21.15.73:0/3657703974 shutdown_connections 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 --2- 172.21.15.73:0/3657703974 >> v2:172.21.15.157:3300/0 conn(0x7ff8bc0b1f00 0x7ff8bc15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 --2- 172.21.15.73:0/3657703974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc158860 0x7ff8bc0b19c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 -- 172.21.15.73:0/3657703974 >> 172.21.15.73:0/3657703974 conn(0x7ff8bc0b06c0 msgr2=0x7ff8bc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 -- 172.21.15.73:0/3657703974 shutdown_connections 2024-04-05T22:24:01.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.426+0000 7ff8c3000640 1 -- 172.21.15.73:0/3657703974 wait complete. 2024-04-05T22:24:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.427+0000 7ff8c3000640 1 Processor -- start 2024-04-05T22:24:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.427+0000 7ff8c3000640 1 -- start start 2024-04-05T22:24:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc0b1f00 0x7ff8bc140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc15eb60 0x7ff8bc070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc1413c0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc1416a0 con 0x7ff8bc15eb60 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.428+0000 7ff8c3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc141980 con 0x7ff8bc0b1f00 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54900/0 (socket says 172.21.15.73:54900) 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 -- 172.21.15.73:0/2603330472 learned_addr learned my addr 172.21.15.73:0/2603330472 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c1c00640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc0b1f00 0x7ff8bc140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c1200640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc15eb60 0x7ff8bc070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 -- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc0b1f00 msgr2=0x7ff8bc140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc0b1f00 0x7ff8bc140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 -- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc15eb60 msgr2=0x7ff8bc070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc15eb60 0x7ff8bc070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.429+0000 7ff8c2600640 1 -- 172.21.15.73:0/2603330472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00094d0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8c2600640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4001eb0 tx=0x7ff8b400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:01.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40052d0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4005430 con 0x7ff8bc0706f0 2024-04-05T22:24:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003810 con 0x7ff8bc0706f0 2024-04-05T22:24:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc141c60 con 0x7ff8bc0706f0 2024-04-05T22:24:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.430+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc141020 con 0x7ff8bc0706f0 2024-04-05T22:24:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.431+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc148cf0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.432+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7ff8b4003970 con 0x7ff8bc0706f0 2024-04-05T22:24:01.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.434+0000 7ff8b9600640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 0x7ff8a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:01.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.434+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b4088d00 con 0x7ff8bc0706f0 2024-04-05T22:24:01.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.434+0000 7ff8c1c00640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 0x7ff8a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:01.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.435+0000 7ff8c1c00640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 0x7ff8a006d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0036000 tx=0x7ff8b003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:01.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.439+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff8bc148cf0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.675+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8bc148cf0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.675+0000 7ff8b9600640 1 -- 172.21.15.73:0/2603330472 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7ff8bc148cf0 con 0x7ff8bc0706f0 2024-04-05T22:24:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:01.676 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:01.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 msgr2=0x7ff8a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 0x7ff8a006d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0036000 tx=0x7ff8b003c040 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 msgr2=0x7ff8bc13ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4001eb0 tx=0x7ff8b400c890 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 shutdown_connections 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff8a006ae70 0x7ff8a006d330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc0706f0 0x7ff8bc13ff40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc15eb60 0x7ff8bc070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 --2- 172.21.15.73:0/2603330472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc0b1f00 0x7ff8bc140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 >> 172.21.15.73:0/2603330472 conn(0x7ff8bc0b06c0 msgr2=0x7ff8bc14e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 shutdown_connections 2024-04-05T22:24:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:01.678+0000 7ff8c3000640 1 -- 172.21.15.73:0/2603330472 wait complete. 2024-04-05T22:24:01.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:01.930 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":30.092465383,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.523634165338247,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:01.930 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.523634165338247 }, "state": "open", "uptime": 30.092465383 } ] 2024-04-05T22:24:01.930 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:06.932 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:07.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.146+0000 7fa4e2e00640 1 Processor -- start 2024-04-05T22:24:07.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.146+0000 7fa4e2e00640 1 -- start start 2024-04-05T22:24:07.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4dc1541a0 0x7fa4dc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc06d710 0x7fa4dc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4dc15aef0 0x7fa4dc0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc15aa60 con 0x7fa4dc15aef0 2024-04-05T22:24:07.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b3b60 con 0x7fa4dc1541a0 2024-04-05T22:24:07.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.147+0000 7fa4e2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0701c0 con 0x7fa4dc06d710 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4dc15aef0 0x7fa4dc0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4dc15aef0 0x7fa4dc0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59000/0 (socket says 172.21.15.73:59000) 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 -- 172.21.15.73:0/603169378 learned_addr learned my addr 172.21.15.73:0/603169378 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4d3400640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc06d710 0x7fa4dc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4dbe00640 1 --2- 172.21.15.73:0/603169378 >> v2:172.21.15.157:3300/0 conn(0x7fa4dc1541a0 0x7fa4dc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 -- 172.21.15.73:0/603169378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc06d710 msgr2=0x7fa4dc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc06d710 0x7fa4dc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 -- 172.21.15.73:0/603169378 >> v2:172.21.15.157:3300/0 conn(0x7fa4dc1541a0 msgr2=0x7fa4dc0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 --2- 172.21.15.73:0/603169378 >> v2:172.21.15.157:3300/0 conn(0x7fa4dc1541a0 0x7fa4dc0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.148+0000 7fa4e0e00640 1 -- 172.21.15.73:0/603169378 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc0b3880 con 0x7fa4dc15aef0 2024-04-05T22:24:07.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.150+0000 7fa4e0e00640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc0cde30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc066850 tx=0x7fa4d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f21a57c3769f3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:07.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.150+0000 7fa4db400640 1 -- 172.21.15.73:0/603169378 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d402ca70 con 0x7fa4dc15aef0 2024-04-05T22:24:07.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.150+0000 7fa4db400640 1 -- 172.21.15.73:0/603169378 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d402cbd0 con 0x7fa4dc15aef0 2024-04-05T22:24:07.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.150+0000 7fa4db400640 1 -- 172.21.15.73:0/603169378 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40343f0 con 0x7fa4dc15aef0 2024-04-05T22:24:07.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 -- 172.21.15.73:0/603169378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc15aef0 msgr2=0x7fa4dc0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:07.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc0cde30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc066850 tx=0x7fa4d402bed0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 -- 172.21.15.73:0/603169378 shutdown_connections 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc0cde30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/603169378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc06d710 0x7fa4dc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/603169378 >> v2:172.21.15.157:3300/0 conn(0x7fa4dc1541a0 0x7fa4dc0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 -- 172.21.15.73:0/603169378 >> 172.21.15.73:0/603169378 conn(0x7fa4dc0ce8e0 msgr2=0x7fa4dc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 -- 172.21.15.73:0/603169378 shutdown_connections 2024-04-05T22:24:07.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.151+0000 7fa4e2e00640 1 -- 172.21.15.73:0/603169378 wait complete. 2024-04-05T22:24:07.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.153+0000 7fa4e2e00640 1 Processor -- start 2024-04-05T22:24:07.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.153+0000 7fa4e2e00640 1 -- start start 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc1541a0 0x7fa4dc1412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0cc7e0 con 0x7fa4dc06d710 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc15b820 con 0x7fa4dc15aef0 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.154+0000 7fa4e2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc159510 con 0x7fa4dc1541a0 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59006/0 (socket says 172.21.15.73:59006) 2024-04-05T22:24:07.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 -- 172.21.15.73:0/3486210527 learned_addr learned my addr 172.21.15.73:0/3486210527 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4d3400640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc1541a0 0x7fa4dc1412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4e0e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 -- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc1541a0 msgr2=0x7fa4dc1412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc1541a0 0x7fa4dc1412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 -- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 msgr2=0x7fa4dc151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 -- 172.21.15.73:0/3486210527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009b30 con 0x7fa4dc06d710 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4e0e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.155+0000 7fa4dbe00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00d800 tx=0x7fa4cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.156+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00b6c0 con 0x7fa4dc06d710 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.156+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc00b820 con 0x7fa4dc06d710 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.156+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc1597f0 con 0x7fa4dc06d710 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.156+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00bbb0 con 0x7fa4dc06d710 2024-04-05T22:24:07.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.156+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc0ccd10 con 0x7fa4dc06d710 2024-04-05T22:24:07.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.157+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc159a30 con 0x7fa4dc06d710 2024-04-05T22:24:07.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.158+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fa4cc00bd10 con 0x7fa4dc06d710 2024-04-05T22:24:07.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.159+0000 7fa4d8c00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 0x7fa4b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:07.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.159+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4cc089e10 con 0x7fa4dc06d710 2024-04-05T22:24:07.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.159+0000 7fa4d3400640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 0x7fa4b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:07.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.160+0000 7fa4d3400640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 0x7fa4b806d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0b0380 tx=0x7fa4c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:07.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.164+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa4cc08d030 con 0x7fa4dc06d710 2024-04-05T22:24:07.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.398+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4dc066af0 con 0x7fa4dc06d710 2024-04-05T22:24:07.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.399+0000 7fa4d8c00640 1 -- 172.21.15.73:0/3486210527 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fa4cc054bb0 con 0x7fa4dc06d710 2024-04-05T22:24:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:07.399 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:07.399 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 msgr2=0x7fa4b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 0x7fa4b806d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0b0380 tx=0x7fa4c4008040 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 msgr2=0x7fa4dc0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00d800 tx=0x7fa4cc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 shutdown_connections 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa4b806ae40 0x7fa4b806d300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15aef0 0x7fa4dc151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc1541a0 0x7fa4dc1412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 --2- 172.21.15.73:0/3486210527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4dc06d710 0x7fa4dc0cd230 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 >> 172.21.15.73:0/3486210527 conn(0x7fa4dc0ce8e0 msgr2=0x7fa4dc159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.402+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 shutdown_connections 2024-04-05T22:24:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:07.403+0000 7fa4e2e00640 1 -- 172.21.15.73:0/3486210527 wait complete. 2024-04-05T22:24:07.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:07.660 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":35.823363567000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.253714974398825,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:07.661 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.253714974398825 }, "state": "open", "uptime": 35.823363567 } ] 2024-04-05T22:24:07.661 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:08.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:24:08.194 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:24:12.662 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.871+0000 7f2358a00640 1 Processor -- start 2024-04-05T22:24:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.871+0000 7f2358a00640 1 -- start start 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2358a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23541541a0 0x7f23540b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2358a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23540ce000 0x7f235415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2358a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235415aa60 con 0x7f235406d710 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2358a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23540b3b60 con 0x7f23540ce000 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2358a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23540701c0 con 0x7f23541541a0 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2352800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23540ce000 0x7f235415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2352800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23540ce000 0x7f235415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59010/0 (socket says 172.21.15.73:59010) 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.873+0000 7f2352800640 1 -- 172.21.15.73:0/2150942813 learned_addr learned my addr 172.21.15.73:0/2150942813 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.874+0000 7f2351e00640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23541541a0 0x7f23540b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.874+0000 7f2352800640 1 -- 172.21.15.73:0/2150942813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23541541a0 msgr2=0x7f23540b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.874+0000 7f2352800640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23541541a0 0x7f23540b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.874+0000 7f2352800640 1 -- 172.21.15.73:0/2150942813 >> v1:172.21.15.157:6789/0 conn(0x7f235406d710 legacy=0x7f23540cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.874+0000 7f2352800640 1 -- 172.21.15.73:0/2150942813 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23540b3880 con 0x7f23540ce000 2024-04-05T22:24:12.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.875+0000 7f2352800640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23540ce000 0x7f235415d860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f234c009e80 tx=0x7f234c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ce7b16d4adbb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:12.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.875+0000 7f2351400640 1 -- 172.21.15.73:0/2150942813 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c02ca50 con 0x7f23540ce000 2024-04-05T22:24:12.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.875+0000 7f2351400640 1 -- 172.21.15.73:0/2150942813 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c02cbb0 con 0x7f23540ce000 2024-04-05T22:24:12.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.875+0000 7f2351400640 1 -- 172.21.15.73:0/2150942813 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c0343f0 con 0x7f23540ce000 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 -- 172.21.15.73:0/2150942813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23540ce000 msgr2=0x7f235415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23540ce000 0x7f235415d860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f234c009e80 tx=0x7f234c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 -- 172.21.15.73:0/2150942813 shutdown_connections 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23540ce000 0x7f235415d860 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 --2- 172.21.15.73:0/2150942813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23541541a0 0x7f23540b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 -- 172.21.15.73:0/2150942813 >> 172.21.15.73:0/2150942813 conn(0x7f23540ce8e0 msgr2=0x7f23540cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 -- 172.21.15.73:0/2150942813 shutdown_connections 2024-04-05T22:24:12.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.876+0000 7f2358a00640 1 -- 172.21.15.73:0/2150942813 wait complete. 2024-04-05T22:24:12.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.878+0000 7f2358a00640 1 Processor -- start 2024-04-05T22:24:12.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.878+0000 7f2358a00640 1 -- start start 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f235406d710 0x7f235415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23540ce000 0x7f23541440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235414baf0 con 0x7f23541541a0 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235415d230 con 0x7f23540ce000 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2358a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235414b5c0 con 0x7f235406d710 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2352800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2352800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59020/0 (socket says 172.21.15.73:59020) 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.879+0000 7f2352800640 1 -- 172.21.15.73:0/251184033 learned_addr learned my addr 172.21.15.73:0/251184033 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2351e00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f235406d710 0x7f235415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2349400640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23540ce000 0x7f23541440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 -- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f235406d710 msgr2=0x7f235415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f235406d710 0x7f235415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 -- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23540ce000 msgr2=0x7f23541440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23540ce000 0x7f23541440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 -- 172.21.15.73:0/251184033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c009b30 con 0x7f23541541a0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2352800640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f234c02c4c0 tx=0x7f234c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c009510 con 0x7f23541541a0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c03e450 con 0x7f23541541a0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.880+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235414b180 con 0x7f23541541a0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.881+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c03e7e0 con 0x7f23541541a0 2024-04-05T22:24:12.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.881+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235414a6a0 con 0x7f23541541a0 2024-04-05T22:24:12.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.882+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2354066af0 con 0x7f23541541a0 2024-04-05T22:24:12.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.883+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f234c03e940 con 0x7f23541541a0 2024-04-05T22:24:12.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.884+0000 7f234aa00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 0x7f233006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:12.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.884+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f234c0ac830 con 0x7f23541541a0 2024-04-05T22:24:12.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.884+0000 7f2351e00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 0x7f233006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:12.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.885+0000 7f2351e00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 0x7f233006d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2344006fd0 tx=0x7f2344008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:12.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:12.889+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f234c077780 con 0x7f23541541a0 2024-04-05T22:24:13.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.119+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2354068160 con 0x7f23541541a0 2024-04-05T22:24:13.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.120+0000 7f234aa00640 1 -- 172.21.15.73:0/251184033 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f234c033e80 con 0x7f23541541a0 2024-04-05T22:24:13.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:13.119 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:13.120 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:13.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 msgr2=0x7f233006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:13.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 0x7f233006d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2344006fd0 tx=0x7f2344008040 comp rx=0 tx=0).stop 2024-04-05T22:24:13.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 msgr2=0x7f235414bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:13.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f234c02c4c0 tx=0x7f234c004a40 comp rx=0 tx=0).stop 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 shutdown_connections 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f233006ae40 0x7f233006d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23541541a0 0x7f235414bcf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23540ce000 0x7f23541440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 --2- 172.21.15.73:0/251184033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f235406d710 0x7f235415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 >> 172.21.15.73:0/251184033 conn(0x7f23540ce8e0 msgr2=0x7f23540cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:13.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 shutdown_connections 2024-04-05T22:24:13.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:13.122+0000 7f2358a00640 1 -- 172.21.15.73:0/251184033 wait complete. 2024-04-05T22:24:13.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:13.381 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.541261984000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.987899824705334,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:13.382 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.987899824705334 }, "state": "open", "uptime": 41.541261984 } ] 2024-04-05T22:24:13.382 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:18.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:18.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.599+0000 7f5796200640 1 Processor -- start 2024-04-05T22:24:18.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.599+0000 7f5796200640 1 -- start start 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f5796200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57901541a0 0x7f57900b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f5796200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f579006d710 0x7f57900cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f5796200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579015aa60 con 0x7f579015aef0 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f5796200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900b3b60 con 0x7f579006d710 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f5796200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900701c0 con 0x7f57901541a0 2024-04-05T22:24:18.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f578ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f579006d710 0x7f57900cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f578ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f579006d710 0x7f57900cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36004/0 (socket says 172.21.15.73:36004) 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.601+0000 7f578ea00640 1 -- 172.21.15.73:0/1592955319 learned_addr learned my addr 172.21.15.73:0/1592955319 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.602+0000 7f578ea00640 1 -- 172.21.15.73:0/1592955319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57901541a0 msgr2=0x7f57900b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.602+0000 7f578ea00640 1 --2- 172.21.15.73:0/1592955319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57901541a0 0x7f57900b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.602+0000 7f578ea00640 1 -- 172.21.15.73:0/1592955319 >> v1:172.21.15.73:6789/0 conn(0x7f579015aef0 legacy=0x7f579015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.602+0000 7f578ea00640 1 -- 172.21.15.73:0/1592955319 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900b3880 con 0x7f579006d710 2024-04-05T22:24:18.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.603+0000 7f578ea00640 1 --2- 172.21.15.73:0/1592955319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579006d710 0x7f57900cd8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f577c009870 tx=0x7f577c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9808415a2f5b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:18.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.603+0000 7f578e000640 1 -- 172.21.15.73:0/1592955319 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c030840 con 0x7f579006d710 2024-04-05T22:24:18.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.603+0000 7f578e000640 1 -- 172.21.15.73:0/1592955319 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c0309a0 con 0x7f579006d710 2024-04-05T22:24:18.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.603+0000 7f578e000640 1 -- 172.21.15.73:0/1592955319 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c030d30 con 0x7f579006d710 2024-04-05T22:24:18.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 -- 172.21.15.73:0/1592955319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579006d710 msgr2=0x7f57900cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 --2- 172.21.15.73:0/1592955319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579006d710 0x7f57900cd8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f577c009870 tx=0x7f577c02b980 comp rx=0 tx=0).stop 2024-04-05T22:24:18.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 -- 172.21.15.73:0/1592955319 shutdown_connections 2024-04-05T22:24:18.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 --2- 172.21.15.73:0/1592955319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579006d710 0x7f57900cd8f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 --2- 172.21.15.73:0/1592955319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57901541a0 0x7f57900b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 -- 172.21.15.73:0/1592955319 >> 172.21.15.73:0/1592955319 conn(0x7f57900ce8e0 msgr2=0x7f57900cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:18.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 -- 172.21.15.73:0/1592955319 shutdown_connections 2024-04-05T22:24:18.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.604+0000 7f5796200640 1 -- 172.21.15.73:0/1592955319 wait complete. 2024-04-05T22:24:18.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.605+0000 7f5796200640 1 Processor -- start 2024-04-05T22:24:18.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.606+0000 7f5796200640 1 -- start start 2024-04-05T22:24:18.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.606+0000 7f5796200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579006d710 0x7f57901493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.606+0000 7f5796200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 0x7f57900c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f5796200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f5796200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900aba80 con 0x7f579015aef0 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f5796200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900abcc0 con 0x7f57901541a0 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f5796200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900abfa0 con 0x7f579006d710 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36010/0 (socket says 172.21.15.73:36010) 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 -- 172.21.15.73:0/3792562817 learned_addr learned my addr 172.21.15.73:0/3792562817 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578ea00640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 0x7f57900c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578f400640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579006d710 0x7f57901493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 -- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579006d710 msgr2=0x7f57901493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579006d710 0x7f57901493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 -- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 msgr2=0x7f57900c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 0x7f57900c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578fe00640 1 -- 172.21.15.73:0/3792562817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0094d0 con 0x7f579015aef0 2024-04-05T22:24:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.607+0000 7f578ea00640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 0x7f57900c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f578fe00640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f578800c6b0 tx=0x7f578800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788003b70 con 0x7f579015aef0 2024-04-05T22:24:18.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788003cf0 con 0x7f579015aef0 2024-04-05T22:24:18.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788003ea0 con 0x7f579015aef0 2024-04-05T22:24:18.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57900ac280 con 0x7f579015aef0 2024-04-05T22:24:18.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.608+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57900ac840 con 0x7f579015aef0 2024-04-05T22:24:18.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.609+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790066af0 con 0x7f579015aef0 2024-04-05T22:24:18.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.610+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f5788005310 con 0x7f579015aef0 2024-04-05T22:24:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.611+0000 7f5787400640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 0x7f577006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.611+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f57880895f0 con 0x7f579015aef0 2024-04-05T22:24:18.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.611+0000 7f578f400640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 0x7f577006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:18.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.612+0000 7f578f400640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 0x7f577006d350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f578000a810 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:18.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.616+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5788054410 con 0x7f579015aef0 2024-04-05T22:24:18.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.848+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5790068160 con 0x7f579015aef0 2024-04-05T22:24:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.849+0000 7f5787400640 1 -- 172.21.15.73:0/3792562817 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f5788057a30 con 0x7f579015aef0 2024-04-05T22:24:18.850 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:18.850 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.852+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 msgr2=0x7f577006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.852+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 0x7f577006d350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f578000a810 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-04-05T22:24:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.852+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 msgr2=0x7f5790149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.852+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f578800c6b0 tx=0x7f578800cb70 comp rx=0 tx=0).stop 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.852+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 shutdown_connections 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f577006ae90 0x7f577006d350 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f579015aef0 0x7f5790149e90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57901541a0 0x7f57900c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 --2- 172.21.15.73:0/3792562817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579006d710 0x7f57901493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:18.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 >> 172.21.15.73:0/3792562817 conn(0x7f57900ce8e0 msgr2=0x7f579015b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:18.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 shutdown_connections 2024-04-05T22:24:18.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:18.853+0000 7f5796200640 1 -- 172.21.15.73:0/3792562817 wait complete. 2024-04-05T22:24:18.873 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:19.110 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":47.272160169999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.724980838486644,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:19.110 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.724980838486644 }, "state": "open", "uptime": 47.27216017 } ] 2024-04-05T22:24:19.110 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:24.111 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.329+0000 7f1066c00640 1 Processor -- start 2024-04-05T22:24:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.329+0000 7f1066c00640 1 -- start start 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10601587d0 0x7f10600cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10600ccd20 0x7f106006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f106006d0c0 0x7f106015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106006f5b0 con 0x7f10601587d0 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1060070410 con 0x7f106006d0c0 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.330+0000 7f1066c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1060158bb0 con 0x7f10600ccd20 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10601587d0 0x7f10600cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10601587d0 0x7f10600cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36024/0 (socket says 172.21.15.73:36024) 2024-04-05T22:24:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 -- 172.21.15.73:0/2068413898 learned_addr learned my addr 172.21.15.73:0/2068413898 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 -- 172.21.15.73:0/2068413898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10600ccd20 msgr2=0x7f106006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 --2- 172.21.15.73:0/2068413898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10600ccd20 0x7f106006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 -- 172.21.15.73:0/2068413898 >> v2:172.21.15.157:3300/0 conn(0x7f106006d0c0 msgr2=0x7f106015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 --2- 172.21.15.73:0/2068413898 >> v2:172.21.15.157:3300/0 conn(0x7f106006d0c0 0x7f106015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.331+0000 7f105fe00640 1 -- 172.21.15.73:0/2068413898 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106006f800 con 0x7f10601587d0 2024-04-05T22:24:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.332+0000 7f105fe00640 1 --2- 172.21.15.73:0/2068413898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600cc7e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f10600ae4b0 tx=0x7f105002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ddc79c175a154 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.332+0000 7f105ea00640 1 -- 172.21.15.73:0/2068413898 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050031860 con 0x7f10601587d0 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f105ea00640 1 -- 172.21.15.73:0/2068413898 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10500319c0 con 0x7f10601587d0 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f105ea00640 1 -- 172.21.15.73:0/2068413898 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050031d50 con 0x7f10601587d0 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 -- 172.21.15.73:0/2068413898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 msgr2=0x7f10600cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 --2- 172.21.15.73:0/2068413898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600cc7e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f10600ae4b0 tx=0x7f105002b900 comp rx=0 tx=0).stop 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 -- 172.21.15.73:0/2068413898 shutdown_connections 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 --2- 172.21.15.73:0/2068413898 >> v2:172.21.15.157:3300/0 conn(0x7f106006d0c0 0x7f106015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 --2- 172.21.15.73:0/2068413898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10600ccd20 0x7f106006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 --2- 172.21.15.73:0/2068413898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600cc7e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 -- 172.21.15.73:0/2068413898 >> 172.21.15.73:0/2068413898 conn(0x7f10600ce8e0 msgr2=0x7f10600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 -- 172.21.15.73:0/2068413898 shutdown_connections 2024-04-05T22:24:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.333+0000 7f1066c00640 1 -- 172.21.15.73:0/2068413898 wait complete. 2024-04-05T22:24:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.335+0000 7f1066c00640 1 Processor -- start 2024-04-05T22:24:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.335+0000 7f1066c00640 1 -- start start 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106006d0c0 0x7f1060149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015c4d0 0x7f106015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106015cec0 con 0x7f10601587d0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1060154280 con 0x7f106006d0c0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f1066c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1060154560 con 0x7f106015c4d0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.336+0000 7f105f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36032/0 (socket says 172.21.15.73:36032) 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 -- 172.21.15.73:0/2786022156 learned_addr learned my addr 172.21.15.73:0/2786022156 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f1064c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015c4d0 0x7f106015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105fe00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106006d0c0 0x7f1060149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 -- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015c4d0 msgr2=0x7f106015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015c4d0 0x7f106015c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 -- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106006d0c0 msgr2=0x7f1060149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106006d0c0 0x7f1060149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 -- 172.21.15.73:0/2786022156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500094d0 con 0x7f10601587d0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f105f400640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f104800b4a0 tx=0x7f104800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.337+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048010610 con 0x7f10601587d0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.338+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1048010770 con 0x7f10601587d0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.338+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048010b20 con 0x7f10601587d0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.338+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1060154840 con 0x7f10601587d0 2024-04-05T22:24:24.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.338+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1060154e00 con 0x7f10601587d0 2024-04-05T22:24:24.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.339+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1060066af0 con 0x7f10601587d0 2024-04-05T22:24:24.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.340+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f1048010c80 con 0x7f10601587d0 2024-04-05T22:24:24.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.341+0000 7f1057e00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 0x7f103c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:24.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.341+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f10480893e0 con 0x7f10601587d0 2024-04-05T22:24:24.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.341+0000 7f105fe00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 0x7f103c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:24.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.342+0000 7f105fe00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 0x7f103c06d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1050036000 tx=0x7f105003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.345+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1048054210 con 0x7f10601587d0 2024-04-05T22:24:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.580+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1060068160 con 0x7f10601587d0 2024-04-05T22:24:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.581+0000 7f1057e00640 1 -- 172.21.15.73:0/2786022156 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f1048057830 con 0x7f10601587d0 2024-04-05T22:24:24.582 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:24.582 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 msgr2=0x7f103c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 0x7f103c06d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1050036000 tx=0x7f105003c040 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 msgr2=0x7f10600acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f104800b4a0 tx=0x7f104800b960 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 shutdown_connections 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f103c06ae70 0x7f103c06d330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015c4d0 0x7f106015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10601587d0 0x7f10600acc60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 --2- 172.21.15.73:0/2786022156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106006d0c0 0x7f1060149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 >> 172.21.15.73:0/2786022156 conn(0x7f10600ce8e0 msgr2=0x7f106006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 shutdown_connections 2024-04-05T22:24:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:24.584+0000 7f1066c00640 1 -- 172.21.15.73:0/2786022156 wait complete. 2024-04-05T22:24:24.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:24.842 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":53.004058157999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.465475298358601,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:24.842 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.4654752983586 }, "state": "open", "uptime": 53.004058158 } ] 2024-04-05T22:24:24.843 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:29.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:30.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.056+0000 7f2647400640 1 Processor -- start 2024-04-05T22:24:30.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.056+0000 7f2647400640 1 -- start start 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26401587d0 0x7f26400cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 0x7f264006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f264006d0c0 0x7f264015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264006f5b0 con 0x7f26401587d0 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2640070410 con 0x7f264006d0c0 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2647400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2640158bb0 con 0x7f26400ccd20 2024-04-05T22:24:30.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2644a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26401587d0 0x7f26400cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2644a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26401587d0 0x7f26400cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52312/0 (socket says 172.21.15.73:52312) 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.058+0000 7f2644a00640 1 -- 172.21.15.73:0/1038512713 learned_addr learned my addr 172.21.15.73:0/1038512713 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f263fe00640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 0x7f264006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2645400640 1 --2- 172.21.15.73:0/1038512713 >> v2:172.21.15.157:3300/0 conn(0x7f264006d0c0 0x7f264015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2644a00640 1 -- 172.21.15.73:0/1038512713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 msgr2=0x7f264006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2644a00640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 0x7f264006cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2644a00640 1 -- 172.21.15.73:0/1038512713 >> v2:172.21.15.157:3300/0 conn(0x7f264006d0c0 msgr2=0x7f264015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2644a00640 1 --2- 172.21.15.73:0/1038512713 >> v2:172.21.15.157:3300/0 conn(0x7f264006d0c0 0x7f264015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f2644a00640 1 -- 172.21.15.73:0/1038512713 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264006f800 con 0x7f26401587d0 2024-04-05T22:24:30.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.059+0000 7f263fe00640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 0x7f264006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.060+0000 7f2644a00640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400cc7e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f26400ae4b0 tx=0x7f263402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ed4872ceab3757 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.060+0000 7f263f400640 1 -- 172.21.15.73:0/1038512713 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634031870 con 0x7f26401587d0 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.060+0000 7f263f400640 1 -- 172.21.15.73:0/1038512713 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26340319d0 con 0x7f26401587d0 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.060+0000 7f263f400640 1 -- 172.21.15.73:0/1038512713 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634031d60 con 0x7f26401587d0 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 -- 172.21.15.73:0/1038512713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 msgr2=0x7f26400cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400cc7e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f26400ae4b0 tx=0x7f263402b910 comp rx=0 tx=0).stop 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 -- 172.21.15.73:0/1038512713 shutdown_connections 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 --2- 172.21.15.73:0/1038512713 >> v2:172.21.15.157:3300/0 conn(0x7f264006d0c0 0x7f264015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26400ccd20 0x7f264006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 --2- 172.21.15.73:0/1038512713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400cc7e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 -- 172.21.15.73:0/1038512713 >> 172.21.15.73:0/1038512713 conn(0x7f26400ce8e0 msgr2=0x7f26400cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:30.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 -- 172.21.15.73:0/1038512713 shutdown_connections 2024-04-05T22:24:30.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.061+0000 7f2647400640 1 -- 172.21.15.73:0/1038512713 wait complete. 2024-04-05T22:24:30.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.062+0000 7f2647400640 1 Processor -- start 2024-04-05T22:24:30.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.063+0000 7f2647400640 1 -- start start 2024-04-05T22:24:30.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.063+0000 7f2647400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264006d0c0 0x7f26400b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.063+0000 7f2647400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2647400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26401f27b0 0x7f26400b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2647400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26401f5e80 con 0x7f26401587d0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2647400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26400c3b30 con 0x7f26401f27b0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2647400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26400b3960 con 0x7f264006d0c0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52322/0 (socket says 172.21.15.73:52322) 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 -- 172.21.15.73:0/616615332 learned_addr learned my addr 172.21.15.73:0/616615332 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2644a00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264006d0c0 0x7f26400b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 -- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264006d0c0 msgr2=0x7f26400b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264006d0c0 0x7f26400b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 -- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26401f27b0 msgr2=0x7f26400b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f2645400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26401f27b0 0x7f26400b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26401f27b0 0x7f26400b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.064+0000 7f263fe00640 1 -- 172.21.15.73:0/616615332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340094d0 con 0x7f26401587d0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f263fe00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c820 tx=0x7f262c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c007a30 con 0x7f26401587d0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c007b90 con 0x7f26401587d0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26401f2d20 con 0x7f26401587d0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00ce70 con 0x7f26401587d0 2024-04-05T22:24:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.065+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26401f32e0 con 0x7f26401587d0 2024-04-05T22:24:30.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.066+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2640066af0 con 0x7f26401587d0 2024-04-05T22:24:30.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.067+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f262c0113f0 con 0x7f26401587d0 2024-04-05T22:24:30.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.068+0000 7f263cc00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 0x7f262006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:30.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.068+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f262c089aa0 con 0x7f26401587d0 2024-04-05T22:24:30.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.068+0000 7f2644a00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 0x7f262006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:30.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.069+0000 7f2644a00640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 0x7f262006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2634036000 tx=0x7f263403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:30.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.075+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f262c054730 con 0x7f26401587d0 2024-04-05T22:24:30.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.305+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26401f5fc0 con 0x7f26401587d0 2024-04-05T22:24:30.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.306+0000 7f263cc00640 1 -- 172.21.15.73:0/616615332 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f262c01f020 con 0x7f26401587d0 2024-04-05T22:24:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:30.306 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:30.307 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.308+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 msgr2=0x7f262006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.308+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 0x7f262006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2634036000 tx=0x7f263403c040 comp rx=0 tx=0).stop 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 msgr2=0x7f26400acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c820 tx=0x7f262c00cc20 comp rx=0 tx=0).stop 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 shutdown_connections 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f262006ae40 0x7f262006d300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26401f27b0 0x7f26400b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26401587d0 0x7f26400acfb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 --2- 172.21.15.73:0/616615332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264006d0c0 0x7f26400b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:30.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 >> 172.21.15.73:0/616615332 conn(0x7f26400ce8e0 msgr2=0x7f264015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:30.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 shutdown_connections 2024-04-05T22:24:30.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:30.309+0000 7f2647400640 1 -- 172.21.15.73:0/616615332 wait complete. 2024-04-05T22:24:30.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:30.565 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":58.726955808,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.20978332875487,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:30.566 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.20978332875487 }, "state": "open", "uptime": 58.726955808 } ] 2024-04-05T22:24:30.566 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:35.567 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:35.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.787+0000 7f38c0000640 1 Processor -- start 2024-04-05T22:24:35.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.787+0000 7f38c0000640 1 -- start start 2024-04-05T22:24:35.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38c0000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38b81587d0 0x7f38b80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38c0000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b806f5b0 con 0x7f38b815e850 2024-04-05T22:24:35.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38c0000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b8070410 con 0x7f38b80ccd20 2024-04-05T22:24:35.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38c0000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b8158bb0 con 0x7f38b81587d0 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38be000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38b815e850 0x7f38b80708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53774/0 (socket says 172.21.15.73:53774) 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.789+0000 7f38be000640 1 -- 172.21.15.73:0/1626455540 learned_addr learned my addr 172.21.15.73:0/1626455540 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38bd600640 1 --2- 172.21.15.73:0/1626455540 >> v2:172.21.15.157:3300/0 conn(0x7f38b81587d0 0x7f38b80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401832653 0 0) 0x7f38b806f5b0 con 0x7f38b815e850 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a0003520 con 0x7f38b815e850 2024-04-05T22:24:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486955240 0 0) 0x7f38a0003520 con 0x7f38b815e850 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 >> v2:172.21.15.157:3300/0 conn(0x7f38b81587d0 msgr2=0x7f38b80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 --2- 172.21.15.73:0/1626455540 >> v2:172.21.15.157:3300/0 conn(0x7f38b81587d0 0x7f38b80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 >> v1:172.21.15.157:6789/0 conn(0x7f38b80ccd20 legacy=0x7f38b806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b806f800 con 0x7f38b815e850 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f38b00019f0 con 0x7f38b815e850 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 >> v1:172.21.15.73:6789/0 conn(0x7f38b815e850 legacy=0x7f38b80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a0003970 0x7f38a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a00047c0 0x7f38a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8158bb0 con 0x7f38a0005180 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a000c020 con 0x7f38a0003970 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.790+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a000c1e0 con 0x7f38a00047c0 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 -- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a00047c0 msgr2=0x7f38a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a00047c0 0x7f38a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 -- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a0003970 msgr2=0x7f38a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a0003970 0x7f38a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.791+0000 7f38be000640 1 -- 172.21.15.73:0/1626455540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0003520 con 0x7f38a0005180 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.792+0000 7f38be000640 1 -- 172.21.15.73:0/1626455540 reap_dead start 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.792+0000 7f38be000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38a000b980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38b000a820 tx=0x7f38a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286a228afe5bc9ef server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.792+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00072d0 con 0x7f38a0005180 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.792+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b002f070 con 0x7f38a0005180 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.792+0000 7f38b7e00640 1 -- 172.21.15.73:0/1626455540 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00076b0 con 0x7f38a0005180 2024-04-05T22:24:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 -- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 msgr2=0x7f38a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38a000b980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38b000a820 tx=0x7f38a00057f0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 -- 172.21.15.73:0/1626455540 shutdown_connections 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 --2- 172.21.15.73:0/1626455540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38a000b980 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 -- 172.21.15.73:0/1626455540 >> 172.21.15.73:0/1626455540 conn(0x7f38b80ce8e0 msgr2=0x7f38b80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 -- 172.21.15.73:0/1626455540 shutdown_connections 2024-04-05T22:24:35.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.793+0000 7f38c0000640 1 -- 172.21.15.73:0/1626455540 wait complete. 2024-04-05T22:24:35.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.795+0000 7f38c0000640 1 Processor -- start 2024-04-05T22:24:35.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.795+0000 7f38c0000640 1 -- start start 2024-04-05T22:24:35.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38b8139e20 0x7f38b813a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b8158cf0 0x7f38b8159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b813b7d0 con 0x7f38a0005180 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b813ab50 con 0x7f38b8158cf0 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38c0000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b813ae30 con 0x7f38b8139e20 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38bd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38bd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52348/0 (socket says 172.21.15.73:52348) 2024-04-05T22:24:35.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.796+0000 7f38bd600640 1 -- 172.21.15.73:0/615792352 learned_addr learned my addr 172.21.15.73:0/615792352 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:35.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 -- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38b8139e20 msgr2=0x7f38b813a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bcc00640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38b8139e20 0x7f38b813a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38b8139e20 0x7f38b813a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 -- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b8158cf0 msgr2=0x7f38b8159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b8158cf0 0x7f38b8159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 -- 172.21.15.73:0/615792352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b813b110 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38bd600640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e7a0 tx=0x7f38ac00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00eeb0 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00c940 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00cd10 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b813b3f0 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.797+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b8141aa0 con 0x7f38a0005180 2024-04-05T22:24:35.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.798+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b8066af0 con 0x7f38a0005180 2024-04-05T22:24:35.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.799+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f38ac013470 con 0x7f38a0005180 2024-04-05T22:24:35.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.800+0000 7f38b5600640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 0x7f389806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:35.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.800+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38ac089880 con 0x7f38a0005180 2024-04-05T22:24:35.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.800+0000 7f38bcc00640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 0x7f389806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:35.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.801+0000 7f38bcc00640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 0x7f389806d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b815a9e0 tx=0x7f38a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:35.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:35.805+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f38ac0546b0 con 0x7f38a0005180 2024-04-05T22:24:36.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.042+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38b8068160 con 0x7f38a0005180 2024-04-05T22:24:36.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.043+0000 7f38b5600640 1 -- 172.21.15.73:0/615792352 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f38ac057cd0 con 0x7f38a0005180 2024-04-05T22:24:36.044 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:36.044 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:36.044 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 msgr2=0x7f389806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 0x7f389806d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b815a9e0 tx=0x7f38a4009040 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 msgr2=0x7f38b8141560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e7a0 tx=0x7f38ac00ec60 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 shutdown_connections 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f389806ae40 0x7f389806d300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b8158cf0 0x7f38b8159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38b8139e20 0x7f38b813a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 --2- 172.21.15.73:0/615792352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0005180 0x7f38b8141560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:36.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.046+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 >> 172.21.15.73:0/615792352 conn(0x7f38b80ce8e0 msgr2=0x7f38b8142d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:36.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.047+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 shutdown_connections 2024-04-05T22:24:36.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:36.047+0000 7f38c0000640 1 -- 172.21.15.73:0/615792352 wait complete. 2024-04-05T22:24:36.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:36.306 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":64.467853134999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.956661655657964,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:36.307 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.956661655657964 }, "state": "open", "uptime": 64.467853135 } ] 2024-04-05T22:24:36.307 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:38.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:24:38.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:24:41.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:41.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.525+0000 7fb421a00640 1 Processor -- start 2024-04-05T22:24:41.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.526+0000 7fb421a00640 1 -- start start 2024-04-05T22:24:41.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb421a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb41c1587d0 0x7fb41c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb421a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c0ccd20 0x7fb41c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb421a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c06f5b0 con 0x7fb41c06d0c0 2024-04-05T22:24:41.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb421a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c070410 con 0x7fb41c1587d0 2024-04-05T22:24:41.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb421a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c158bb0 con 0x7fb41c0ccd20 2024-04-05T22:24:41.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb41b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb41c06d0c0 0x7fb41c15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41934/0 (socket says 172.21.15.73:41934) 2024-04-05T22:24:41.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb41b400640 1 -- 172.21.15.73:0/470147070 learned_addr learned my addr 172.21.15.73:0/470147070 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:24:41.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.527+0000 7fb41aa00640 1 --2- 172.21.15.73:0/470147070 >> v2:172.21.15.73:3300/0 conn(0x7fb41c1587d0 0x7fb41c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.528+0000 7fb41a000640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c0ccd20 0x7fb41c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.528+0000 7fb41aa00640 1 -- 172.21.15.73:0/470147070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c0ccd20 msgr2=0x7fb41c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.528+0000 7fb41aa00640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c0ccd20 0x7fb41c06cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.528+0000 7fb41aa00640 1 -- 172.21.15.73:0/470147070 >> v1:172.21.15.73:6789/0 conn(0x7fb41c06d0c0 legacy=0x7fb41c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.528+0000 7fb41aa00640 1 -- 172.21.15.73:0/470147070 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c06f800 con 0x7fb41c1587d0 2024-04-05T22:24:41.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.529+0000 7fb41aa00640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c1587d0 0x7fb41c0cc7e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e623b51fcc0537ce server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:41.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb419600640 1 -- 172.21.15.73:0/470147070 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402d040 con 0x7fb41c1587d0 2024-04-05T22:24:41.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb419600640 1 -- 172.21.15.73:0/470147070 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404031810 con 0x7fb41c1587d0 2024-04-05T22:24:41.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb419600640 1 -- 172.21.15.73:0/470147070 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404031ba0 con 0x7fb41c1587d0 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 -- 172.21.15.73:0/470147070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c1587d0 msgr2=0x7fb41c0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c1587d0 0x7fb41c0cc7e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 -- 172.21.15.73:0/470147070 shutdown_connections 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c0ccd20 0x7fb41c06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 --2- 172.21.15.73:0/470147070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c1587d0 0x7fb41c0cc7e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.530+0000 7fb421a00640 1 -- 172.21.15.73:0/470147070 >> 172.21.15.73:0/470147070 conn(0x7fb41c0ce8e0 msgr2=0x7fb41c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.531+0000 7fb421a00640 1 -- 172.21.15.73:0/470147070 shutdown_connections 2024-04-05T22:24:41.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.531+0000 7fb421a00640 1 -- 172.21.15.73:0/470147070 wait complete. 2024-04-05T22:24:41.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.532+0000 7fb421a00640 1 Processor -- start 2024-04-05T22:24:41.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.532+0000 7fb421a00640 1 -- start start 2024-04-05T22:24:41.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c1587d0 0x7fb41c1f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41c13f210 0x7fb41c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1f1240 con 0x7fb41c06d0c0 2024-04-05T22:24:41.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c13d510 con 0x7fb41c13f210 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.533+0000 7fb421a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c15c530 con 0x7fb41c1587d0 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57582/0 (socket says 172.21.15.73:57582) 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 -- 172.21.15.73:0/1038376326 learned_addr learned my addr 172.21.15.73:0/1038376326 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 -- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c1587d0 msgr2=0x7fb41c1f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c1587d0 0x7fb41c1f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 -- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41c13f210 msgr2=0x7fb41c13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41c13f210 0x7fb41c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 -- 172.21.15.73:0/1038376326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009400 con 0x7fb41c06d0c0 2024-04-05T22:24:41.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.534+0000 7fb41aa00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb404000c00 tx=0x7fb4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.535+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40403ad60 con 0x7fb41c06d0c0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.535+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4040443f0 con 0x7fb41c06d0c0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.535+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040447a0 con 0x7fb41c06d0c0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.535+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c1f1440 con 0x7fb41c06d0c0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.535+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c13d7b0 con 0x7fb41c06d0c0 2024-04-05T22:24:41.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.536+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c066af0 con 0x7fb41c06d0c0 2024-04-05T22:24:41.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.537+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fb40404d450 con 0x7fb41c06d0c0 2024-04-05T22:24:41.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.538+0000 7fb412a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 0x7fb3f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:41.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.538+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb4040abb30 con 0x7fb41c06d0c0 2024-04-05T22:24:41.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.538+0000 7fb41a000640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 0x7fb3f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:41.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.539+0000 7fb41a000640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 0x7fb3f806d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0070b0 tx=0x7fb40c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.543+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb404076860 con 0x7fb41c06d0c0 2024-04-05T22:24:41.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.773+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb41c15c390 con 0x7fb41c06d0c0 2024-04-05T22:24:41.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.774+0000 7fb412a00640 1 -- 172.21.15.73:0/1038376326 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fb41c15c390 con 0x7fb41c06d0c0 2024-04-05T22:24:41.774 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:41.775 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:41.775 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:41.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 msgr2=0x7fb3f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 0x7fb3f806d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0070b0 tx=0x7fb40c007040 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 msgr2=0x7fb41c13c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb404000c00 tx=0x7fb4040046c0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 shutdown_connections 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3f806ae70 0x7fb3f806d330 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41c13f210 0x7fb41c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41c1587d0 0x7fb41c1f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 --2- 172.21.15.73:0/1038376326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41c06d0c0 0x7fb41c13c570 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 >> 172.21.15.73:0/1038376326 conn(0x7fb41c0ce8e0 msgr2=0x7fb41c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 shutdown_connections 2024-04-05T22:24:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:41.777+0000 7fb421a00640 1 -- 172.21.15.73:0/1038376326 wait complete. 2024-04-05T22:24:41.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:42.037 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":70.198750641000004,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.707307510358621,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:42.037 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.70730751035862 }, "state": "open", "uptime": 70.198750641 } ] 2024-04-05T22:24:42.037 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:47.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.251+0000 7fafb3800640 1 Processor -- start 2024-04-05T22:24:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.251+0000 7fafb3800640 1 -- start start 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.252+0000 7fafb3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7fafac0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafb3800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac0ad880 con 0x7fafac0b1870 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafb3800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac0ce5d0 con 0x7fafac0cb2f0 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafb3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0ce010 con 0x7fafac160620 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafabe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fafac0b1870 0x7fafac0ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35186/0 (socket says 172.21.15.73:35186) 2024-04-05T22:24:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafabe00640 1 -- 172.21.15.73:0/767198421 learned_addr learned my addr 172.21.15.73:0/767198421 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:24:47.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149123833 0 0) 0x7fafac0ad880 con 0x7fafac0b1870 2024-04-05T22:24:47.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf90003520 con 0x7fafac0b1870 2024-04-05T22:24:47.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.253+0000 7fafb0e00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7fafac0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520279828 0 0) 0x7faf90003520 con 0x7fafac0b1870 2024-04-05T22:24:47.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 msgr2=0x7fafac0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7fafac0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 >> v1:172.21.15.157:6789/0 conn(0x7fafac0cb2f0 legacy=0x7fafac0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0ce2f0 con 0x7fafac0b1870 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7faf94001670 con 0x7fafac0b1870 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 >> v1:172.21.15.73:6789/0 conn(0x7fafac0b1870 legacy=0x7fafac0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900039a0 0x7faf90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7faf900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf90004d00 0x7faf90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0ce010 con 0x7faf900039a0 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9000bb10 con 0x7faf90004d00 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9000bca0 con 0x7fafac160620 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.254+0000 7fafb0e00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900039a0 0x7faf90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafabe00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7faf900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb0e00640 1 -- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 msgr2=0x7faf900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb0e00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac160620 0x7faf900047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb0e00640 1 -- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf90004d00 msgr2=0x7faf90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb1800640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf90004d00 0x7faf90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb0e00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf90004d00 0x7faf90005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb0e00640 1 -- 172.21.15.73:0/767198421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90003520 con 0x7faf900039a0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.255+0000 7fafb1800640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf90004d00 0x7faf90005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.256+0000 7fafb0e00640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900039a0 0x7faf90003dc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00ba60 tx=0x7faf900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8371cd6f3b3254c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.256+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c032630 con 0x7faf900039a0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.256+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c032790 con 0x7faf900039a0 2024-04-05T22:24:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.256+0000 7fafab400640 1 -- 172.21.15.73:0/767198421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c032b20 con 0x7faf900039a0 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 -- 172.21.15.73:0/767198421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900039a0 msgr2=0x7faf90003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 --2- 172.21.15.73:0/767198421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900039a0 0x7faf90003dc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00ba60 tx=0x7faf900057f0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb1800640 1 -- 172.21.15.73:0/767198421 reap_dead start 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 -- 172.21.15.73:0/767198421 shutdown_connections 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 -- 172.21.15.73:0/767198421 >> 172.21.15.73:0/767198421 conn(0x7fafac0ce8e0 msgr2=0x7fafac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:47.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 -- 172.21.15.73:0/767198421 shutdown_connections 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.257+0000 7fafb3800640 1 -- 172.21.15.73:0/767198421 wait complete. 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.258+0000 7fafb3800640 1 Processor -- start 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.259+0000 7fafb3800640 1 -- start start 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.259+0000 7fafb3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 0x7fafac149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac14a680 0x7fafac070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac14ab90 con 0x7fafac14bc30 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac14ae70 con 0x7faf900039a0 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac14b150 con 0x7fafac14a680 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47562/0 (socket says 172.21.15.73:47562) 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 -- 172.21.15.73:0/3288167959 learned_addr learned my addr 172.21.15.73:0/3288167959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb1800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac14a680 0x7fafac070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb0e00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 0x7fafac149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 -- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac14a680 msgr2=0x7fafac070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac14a680 0x7fafac070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 -- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 msgr2=0x7fafac149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 0x7fafac149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafabe00640 1 -- 172.21.15.73:0/3288167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac14b430 con 0x7fafac14bc30 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.260+0000 7fafb0e00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 0x7fafac149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafabe00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf9400d7e0 tx=0x7faf9400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940074e0 con 0x7fafac14bc30 2024-04-05T22:24:47.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94007640 con 0x7fafac14bc30 2024-04-05T22:24:47.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940049e0 con 0x7fafac14bc30 2024-04-05T22:24:47.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac14b710 con 0x7fafac14bc30 2024-04-05T22:24:47.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.261+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac0707a0 con 0x7fafac14bc30 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.262+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac066af0 con 0x7fafac14bc30 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.263+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7faf94004b40 con 0x7fafac14bc30 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.264+0000 7fafa8c00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.264+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faf94089200 con 0x7fafac14bc30 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.264+0000 7fafb0e00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:47.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.265+0000 7fafb0e00640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 0x7faf8806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00ba30 tx=0x7faf9c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:47.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.270+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7faf94053f40 con 0x7fafac14bc30 2024-04-05T22:24:47.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.498+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafac14b900 con 0x7fafac14bc30 2024-04-05T22:24:47.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.500+0000 7fafa8c00640 1 -- 172.21.15.73:0/3288167959 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7faf94057560 con 0x7fafac14bc30 2024-04-05T22:24:47.500 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:47.500 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:47.500 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 msgr2=0x7faf8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 0x7faf8806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00ba30 tx=0x7faf9c030000 comp rx=0 tx=0).stop 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 msgr2=0x7fafac14a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf9400d7e0 tx=0x7faf9400dca0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 shutdown_connections 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafac14a680 0x7fafac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac14bc30 0x7fafac14a140 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 --2- 172.21.15.73:0/3288167959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf900039a0 0x7fafac149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 >> 172.21.15.73:0/3288167959 conn(0x7fafac0ce8e0 msgr2=0x7fafac15f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 shutdown_connections 2024-04-05T22:24:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:47.503+0000 7fafb3800640 1 -- 172.21.15.73:0/3288167959 wait complete. 2024-04-05T22:24:47.522 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:47.762 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":75.922648272999993,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.461531924627735,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:47.763 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.461531924627735 }, "state": "open", "uptime": 75.922648273 } ] 2024-04-05T22:24:47.763 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:52.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:52.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.982+0000 7ff006a00640 1 Processor -- start 2024-04-05T22:24:52.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.982+0000 7ff006a00640 1 -- start start 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.983+0000 7ff006a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff0000b20a0 0x7ff000070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.983+0000 7ff006a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b1b50 0x7ff00006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.983+0000 7ff006a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00006d3e0 con 0x7ff0000b2480 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.983+0000 7ff006a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00006d610 con 0x7ff0000b20a0 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.983+0000 7ff006a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff000070230 con 0x7ff0000b1b50 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff0000b20a0 0x7ff000070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff0000b20a0 0x7ff000070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47570/0 (socket says 172.21.15.73:47570) 2024-04-05T22:24:52.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 -- 172.21.15.73:0/1925519042 learned_addr learned my addr 172.21.15.73:0/1925519042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:52.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 -- 172.21.15.73:0/1925519042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b1b50 msgr2=0x7ff00006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:52.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 --2- 172.21.15.73:0/1925519042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b1b50 0x7ff00006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:52.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 -- 172.21.15.73:0/1925519042 >> v1:172.21.15.73:6789/0 conn(0x7ff0000b2480 legacy=0x7ff0000b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:52.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.984+0000 7ff005600640 1 -- 172.21.15.73:0/1925519042 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000070510 con 0x7ff0000b20a0 2024-04-05T22:24:52.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.985+0000 7ff005600640 1 --2- 172.21.15.73:0/1925519042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b20a0 0x7ff000070ff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feff000bb10 tx=0x7feff002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b188b9964c2bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:52.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.985+0000 7fefffe00640 1 -- 172.21.15.73:0/1925519042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff002ede0 con 0x7ff0000b20a0 2024-04-05T22:24:52.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7fefffe00640 1 -- 172.21.15.73:0/1925519042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0036440 con 0x7ff0000b20a0 2024-04-05T22:24:52.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7fefffe00640 1 -- 172.21.15.73:0/1925519042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff00367d0 con 0x7ff0000b20a0 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 -- 172.21.15.73:0/1925519042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b20a0 msgr2=0x7ff000070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 --2- 172.21.15.73:0/1925519042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b20a0 0x7ff000070ff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feff000bb10 tx=0x7feff002e030 comp rx=0 tx=0).stop 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 -- 172.21.15.73:0/1925519042 shutdown_connections 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 --2- 172.21.15.73:0/1925519042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b1b50 0x7ff00006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 --2- 172.21.15.73:0/1925519042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b20a0 0x7ff000070ff0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 -- 172.21.15.73:0/1925519042 >> 172.21.15.73:0/1925519042 conn(0x7ff0000b06c0 msgr2=0x7ff0000b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 -- 172.21.15.73:0/1925519042 shutdown_connections 2024-04-05T22:24:52.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.986+0000 7ff006a00640 1 -- 172.21.15.73:0/1925519042 wait complete. 2024-04-05T22:24:52.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.988+0000 7ff006a00640 1 Processor -- start 2024-04-05T22:24:52.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.988+0000 7ff006a00640 1 -- start start 2024-04-05T22:24:52.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b20a0 0x7ff00013e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000b2480 0x7ff000150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001509d0 con 0x7ff0000b1b50 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00014fdc0 con 0x7ff0000b2480 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff006a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001500a0 con 0x7ff0000b20a0 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff005600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff005600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47574/0 (socket says 172.21.15.73:47574) 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.989+0000 7ff005600640 1 -- 172.21.15.73:0/1790667351 learned_addr learned my addr 172.21.15.73:0/1790667351 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 -- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b20a0 msgr2=0x7ff00013e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b20a0 0x7ff00013e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 -- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000b2480 msgr2=0x7ff000150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000b2480 0x7ff000150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 -- 172.21.15.73:0/1790667351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff000b770 con 0x7ff0000b1b50 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff005600640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feff000b2c0 tx=0x7feff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff003f840 con 0x7ff0000b1b50 2024-04-05T22:24:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff003f9e0 con 0x7ff0000b1b50 2024-04-05T22:24:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff003fd90 con 0x7ff0000b1b50 2024-04-05T22:24:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff000150380 con 0x7ff0000b1b50 2024-04-05T22:24:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.990+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00014e280 con 0x7ff0000b1b50 2024-04-05T22:24:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.991+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00014e4d0 con 0x7ff0000b1b50 2024-04-05T22:24:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.992+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7feff003d040 con 0x7ff0000b1b50 2024-04-05T22:24:52.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.994+0000 7feffd600640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 0x7fefe006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:52.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.994+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feff00ae320 con 0x7ff0000b1b50 2024-04-05T22:24:52.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.994+0000 7ff004c00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 0x7fefe006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:52.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.995+0000 7ff004c00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 0x7fefe006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff00015a500 tx=0x7fefec009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:52.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:52.999+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feff0079150 con 0x7ff0000b1b50 2024-04-05T22:24:53.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.226+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff00004e160 con 0x7ff0000b1b50 2024-04-05T22:24:53.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.227+0000 7feffd600640 1 -- 172.21.15.73:0/1790667351 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7feff007c770 con 0x7ff0000b1b50 2024-04-05T22:24:53.227 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:53.227 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 msgr2=0x7fefe006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 0x7fefe006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff00015a500 tx=0x7fefec009250 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 msgr2=0x7ff00013e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feff000b2c0 tx=0x7feff00046c0 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 shutdown_connections 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fefe006ae40 0x7fefe006d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000b2480 0x7ff000150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0000b20a0 0x7ff00013e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 --2- 172.21.15.73:0/1790667351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0000b1b50 0x7ff00013e0b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.230+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 >> 172.21.15.73:0/1790667351 conn(0x7ff0000b06c0 msgr2=0x7ff000072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.231+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 shutdown_connections 2024-04-05T22:24:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:53.231+0000 7ff006a00640 1 -- 172.21.15.73:0/1790667351 wait complete. 2024-04-05T22:24:53.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:53.490 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":81.653545782999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.218690669094759,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:53.491 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.21869066909476 }, "state": "open", "uptime": 81.653545783 } ] 2024-04-05T22:24:53.491 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:24:58.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:24:58.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.708+0000 7f6701600640 1 Processor -- start 2024-04-05T22:24:58.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.709+0000 7f6701600640 1 -- start start 2024-04-05T22:24:58.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66fc0b20a0 0x7f66fc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66fc0b1b50 0x7f66fc06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b2480 0x7f66fc0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc06d3e0 con 0x7f66fc0b1b50 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc06d610 con 0x7f66fc0b20a0 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f6701600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc070230 con 0x7f66fc0b2480 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.710+0000 7f66fb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66fc0b1b50 0x7f66fc06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66fc0b1b50 0x7f66fc06cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56550/0 (socket says 172.21.15.73:56550) 2024-04-05T22:24:58.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 -- 172.21.15.73:0/2399018823 learned_addr learned my addr 172.21.15.73:0/2399018823 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:58.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 -- 172.21.15.73:0/2399018823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b2480 msgr2=0x7f66fc0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:58.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 --2- 172.21.15.73:0/2399018823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b2480 0x7f66fc0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 -- 172.21.15.73:0/2399018823 >> v2:172.21.15.157:3300/0 conn(0x7f66fc0b20a0 msgr2=0x7f66fc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:24:58.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 --2- 172.21.15.73:0/2399018823 >> v2:172.21.15.157:3300/0 conn(0x7f66fc0b20a0 0x7f66fc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.711+0000 7f66fb400640 1 -- 172.21.15.73:0/2399018823 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc070510 con 0x7f66fc0b1b50 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.712+0000 7f66fb400640 1 --2- 172.21.15.73:0/2399018823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b1b50 0x7f66fc06cea0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0b40b0 tx=0x7f66e402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c480def8bf5f464 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.712+0000 7f66faa00640 1 -- 172.21.15.73:0/2399018823 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4031750 con 0x7f66fc0b1b50 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.712+0000 7f66faa00640 1 -- 172.21.15.73:0/2399018823 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e40318b0 con 0x7f66fc0b1b50 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.712+0000 7f66faa00640 1 -- 172.21.15.73:0/2399018823 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4031c40 con 0x7f66fc0b1b50 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 -- 172.21.15.73:0/2399018823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b1b50 msgr2=0x7f66fc06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:58.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 --2- 172.21.15.73:0/2399018823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b1b50 0x7f66fc06cea0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0b40b0 tx=0x7f66e402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 -- 172.21.15.73:0/2399018823 shutdown_connections 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 --2- 172.21.15.73:0/2399018823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b1b50 0x7f66fc06cea0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 --2- 172.21.15.73:0/2399018823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b2480 0x7f66fc0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 --2- 172.21.15.73:0/2399018823 >> v2:172.21.15.157:3300/0 conn(0x7f66fc0b20a0 0x7f66fc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 -- 172.21.15.73:0/2399018823 >> 172.21.15.73:0/2399018823 conn(0x7f66fc0b06c0 msgr2=0x7f66fc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 -- 172.21.15.73:0/2399018823 shutdown_connections 2024-04-05T22:24:58.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.713+0000 7f6701600640 1 -- 172.21.15.73:0/2399018823 wait complete. 2024-04-05T22:24:58.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.715+0000 7f6701600640 1 Processor -- start 2024-04-05T22:24:58.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.715+0000 7f6701600640 1 -- start start 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66fc0b1b50 0x7f66fc1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 0x7f66fc06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1550e0 con 0x7f66fc0b2480 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1553c0 con 0x7f66fc0b1b50 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6701600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1556a0 con 0x7f66fc0b20a0 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6700c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6700c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56554/0 (socket says 172.21.15.73:56554) 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.716+0000 7f6700c00640 1 -- 172.21.15.73:0/271420198 learned_addr learned my addr 172.21.15.73:0/271420198 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:24:58.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66fbe00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66fc0b1b50 0x7f66fc1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66fb400640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 0x7f66fc06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 -- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 msgr2=0x7f66fc06bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 0x7f66fc06bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 -- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66fc0b1b50 msgr2=0x7f66fc1434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66fc0b1b50 0x7f66fc1434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 -- 172.21.15.73:0/271420198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40094d0 con 0x7f66fc0b2480 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66fb400640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 0x7f66fc06bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:24:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6700c00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009fd0 tx=0x7f66f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4011830 con 0x7f66fc0b2480 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f4011990 con 0x7f66fc0b2480 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4011d20 con 0x7f66fc0b2480 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.717+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc155980 con 0x7f66fc0b2480 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.718+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc1f3a40 con 0x7f66fc0b2480 2024-04-05T22:24:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.719+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc04e160 con 0x7f66fc0b2480 2024-04-05T22:24:58.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.719+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f66f401b460 con 0x7f66fc0b2480 2024-04-05T22:24:58.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.721+0000 7f66f3e00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 0x7f66d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:24:58.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.721+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f66f4089680 con 0x7f66fc0b2480 2024-04-05T22:24:58.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.721+0000 7f66fbe00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 0x7f66d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:24:58.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.722+0000 7f66fbe00640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 0x7f66d806d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f20 tx=0x7f66ec0094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:24:58.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.726+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f66f4054420 con 0x7f66fc0b2480 2024-04-05T22:24:58.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.952+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66fc155220 con 0x7f66fc0b2480 2024-04-05T22:24:58.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.953+0000 7f66f3e00640 1 -- 172.21.15.73:0/271420198 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f66fc155220 con 0x7f66fc0b2480 2024-04-05T22:24:58.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:24:58.954 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:24:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:24:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 msgr2=0x7f66d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 0x7f66d806d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f20 tx=0x7f66ec0094f0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 msgr2=0x7f66fc06c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:24:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009fd0 tx=0x7f66f400b8e0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 shutdown_connections 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f66d806ae70 0x7f66d806d330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66fc0b2480 0x7f66fc06c2d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66fc0b20a0 0x7f66fc06bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 --2- 172.21.15.73:0/271420198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66fc0b1b50 0x7f66fc1434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 >> 172.21.15.73:0/271420198 conn(0x7f66fc0b06c0 msgr2=0x7f66fc06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.956+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 shutdown_connections 2024-04-05T22:24:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:24:58.957+0000 7f6701600640 1 -- 172.21.15.73:0/271420198 wait complete. 2024-04-05T22:24:58.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:24:59.217 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5925,"entity":{"name":{"type":"client","num":5925},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":37213423}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":87.378443400999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.979292964320262,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5925 172.21.15.73:0/37213423","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:24:59.218 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 37213423, "type": "any" }, "name": { "num": 5925, "type": "client" } }, "id": 5925, "inst": "client.5925 172.21.15.73:0/37213423", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.979292964320262 }, "state": "open", "uptime": 87.378443401 } ] 2024-04-05T22:24:59.218 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-05T22:25:04.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:04.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.437+0000 7f038b200640 1 Processor -- start 2024-04-05T22:25:04.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.438+0000 7f038b200640 1 -- start start 2024-04-05T22:25:04.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.439+0000 7f038b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f038c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.439+0000 7f038b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c143330 con 0x7f038c13fa60 2024-04-05T22:25:04.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.439+0000 7f038b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c143050 con 0x7f038c13fe40 2024-04-05T22:25:04.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.439+0000 7f038b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c06d3a0 con 0x7f038c142ba0 2024-04-05T22:25:04.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0389e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f038c13fa60 0x7f038c142490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55048/0 (socket says 172.21.15.73:55048) 2024-04-05T22:25:04.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0389e00640 1 -- 172.21.15.73:0/1007684980 learned_addr learned my addr 172.21.15.73:0/1007684980 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:04.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460467875 0 0) 0x7f038c143330 con 0x7f038c13fa60 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f036c003520 con 0x7f038c13fa60 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0389400640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f038c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 593186112 0 0) 0x7f036c003520 con 0x7f038c13fa60 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 msgr2=0x7f038c06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f038c06f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 >> v1:172.21.15.157:6789/0 conn(0x7f038c13fe40 legacy=0x7f038c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0ac970 con 0x7f038c13fa60 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f037c001670 con 0x7f038c13fa60 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 >> v1:172.21.15.73:6789/0 conn(0x7f038c13fa60 legacy=0x7f038c142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036c0039a0 0x7f036c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f036c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f036c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c06d3a0 con 0x7f036c004d00 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c00bab0 con 0x7f036c0039a0 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.440+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c00bc70 con 0x7f038c142ba0 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f036c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f0389e00640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036c0039a0 0x7f036c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f0389400640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f036c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 -- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 msgr2=0x7f036c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f036c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 -- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036c0039a0 msgr2=0x7f036c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036c0039a0 0x7f036c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f038a800640 1 -- 172.21.15.73:0/1007684980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c003520 con 0x7f036c004d00 2024-04-05T22:25:04.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.441+0000 7f0389400640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c142ba0 0x7f036c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.443+0000 7f038a800640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f036c005180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0384003db0 tx=0x7f036c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dea4fa5cc7ec163 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.443+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384032600 con 0x7f036c004d00 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.443+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038402e050 con 0x7f036c004d00 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.443+0000 7f0388a00640 1 -- 172.21.15.73:0/1007684980 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03840329a0 con 0x7f036c004d00 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 -- 172.21.15.73:0/1007684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 msgr2=0x7f036c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 --2- 172.21.15.73:0/1007684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f036c005180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0384003db0 tx=0x7f036c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038a800640 1 -- 172.21.15.73:0/1007684980 reap_dead start 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 -- 172.21.15.73:0/1007684980 shutdown_connections 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 -- 172.21.15.73:0/1007684980 >> 172.21.15.73:0/1007684980 conn(0x7f038c0b06c0 msgr2=0x7f038c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 -- 172.21.15.73:0/1007684980 shutdown_connections 2024-04-05T22:25:04.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.444+0000 7f038b200640 1 -- 172.21.15.73:0/1007684980 wait complete. 2024-04-05T22:25:04.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.446+0000 7f038b200640 1 Processor -- start 2024-04-05T22:25:04.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.446+0000 7f038b200640 1 -- start start 2024-04-05T22:25:04.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 0x7f038c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c1f5b60 0x7f038c1f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c15d030 con 0x7f036c004d00 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c15c730 con 0x7f038c1f2620 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c1f6520 con 0x7f038c1f5b60 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f0389e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f0389e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56576/0 (socket says 172.21.15.73:56576) 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f0389e00640 1 -- 172.21.15.73:0/3066482950 learned_addr learned my addr 172.21.15.73:0/3066482950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f0389400640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 0x7f038c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.447+0000 7f038a800640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c1f5b60 0x7f038c1f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 -- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c1f5b60 msgr2=0x7f038c1f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c1f5b60 0x7f038c1f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 -- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 msgr2=0x7f038c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 0x7f038c0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 -- 172.21.15.73:0/3066482950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c15d210 con 0x7f036c004d00 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389400640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 0x7f038c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0389e00640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f037c00e7f0 tx=0x7f037c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00c870 con 0x7f036c004d00 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00c9d0 con 0x7f036c004d00 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00cd40 con 0x7f036c004d00 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c15d4f0 con 0x7f036c004d00 2024-04-05T22:25:04.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.448+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c15dab0 con 0x7f036c004d00 2024-04-05T22:25:04.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.449+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c04e160 con 0x7f036c004d00 2024-04-05T22:25:04.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.450+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f037c013420 con 0x7f036c004d00 2024-04-05T22:25:04.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.451+0000 7f0382000640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 0x7f036806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:04.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.451+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f037c089490 con 0x7f036c004d00 2024-04-05T22:25:04.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.451+0000 7f0389400640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 0x7f036806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:04.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.452+0000 7f0389400640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 0x7f036806d2a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0374009540 tx=0x7f037400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:04.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.456+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f037c054280 con 0x7f036c004d00 2024-04-05T22:25:04.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.688+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f038c160d70 con 0x7f036c004d00 2024-04-05T22:25:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.689+0000 7f0382000640 1 -- 172.21.15.73:0/3066482950 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f037c0578a0 con 0x7f036c004d00 2024-04-05T22:25:04.690 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:04.690 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:25:04.690 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 msgr2=0x7f036806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 0x7f036806d2a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0374009540 tx=0x7f037400b040 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 msgr2=0x7f038c15c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f037c00e7f0 tx=0x7f037c00ecb0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 shutdown_connections 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f036806ade0 0x7f036806d2a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038c1f5b60 0x7f038c1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038c1f2620 0x7f038c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 --2- 172.21.15.73:0/3066482950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f036c004d00 0x7f038c15c1f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 >> 172.21.15.73:0/3066482950 conn(0x7f038c0b06c0 msgr2=0x7f038c0afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:04.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 shutdown_connections 2024-04-05T22:25:04.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:04.692+0000 7f038b200640 1 -- 172.21.15.73:0/3066482950 wait complete. 2024-04-05T22:25:04.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:25:04.952 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2024-04-05T22:25:04.952 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-05T22:25:04.952 DEBUG:tasks.ceph_test_case:wait_until_true: success in 80s and 0 retries 2024-04-05T22:25:04.952 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:25:04.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:05.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:05.187 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:25:05.187 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:05.187 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:05.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:25:05.244 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:05.244 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:05.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:25:05.301 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:05.302 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:05.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:25:05.357 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:05.357 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:05.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:25:05.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.626+0000 7f91d3600640 1 Processor -- start 2024-04-05T22:25:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.627+0000 7f91d3600640 1 -- start start 2024-04-05T22:25:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.628+0000 7f91d3600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91cc0b1870 0x7f91cc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.628+0000 7f91d3600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91cc0ad6b0 0x7f91cc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.628+0000 7f91d3600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0ce5d0 con 0x7f91cc160620 2024-04-05T22:25:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.628+0000 7f91d3600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0ce010 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.628+0000 7f91d3600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0ce2f0 con 0x7f91cc0b1870 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d0c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f91cc160620 0x7f91cc0caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55056/0 (socket says 172.21.15.73:55056) 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d0c00640 1 -- 172.21.15.73:0/1697263103 learned_addr learned my addr 172.21.15.73:0/1697263103 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d1600640 1 --2- 172.21.15.73:0/1697263103 >> v2:172.21.15.73:3300/0 conn(0x7f91cc0ad6b0 0x7f91cc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91c3e00640 1 --2- 172.21.15.73:0/1697263103 >> v2:172.21.15.157:3300/0 conn(0x7f91cc0b1870 0x7f91cc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91cbe00640 1 -- 172.21.15.73:0/1697263103 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441786984 0 0) 0x7f91cc0ce5d0 con 0x7f91cc160620 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91cbe00640 1 -- 172.21.15.73:0/1697263103 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ac003520 con 0x7f91cc160620 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d1600640 1 -- 172.21.15.73:0/1697263103 >> v2:172.21.15.157:3300/0 conn(0x7f91cc0b1870 msgr2=0x7f91cc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d1600640 1 --2- 172.21.15.73:0/1697263103 >> v2:172.21.15.157:3300/0 conn(0x7f91cc0b1870 0x7f91cc0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d1600640 1 -- 172.21.15.73:0/1697263103 >> v1:172.21.15.73:6789/0 conn(0x7f91cc160620 legacy=0x7f91cc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.629+0000 7f91d1600640 1 -- 172.21.15.73:0/1697263103 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0ac970 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.630+0000 7f91d1600640 1 --2- 172.21.15.73:0/1697263103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0cb2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91cc06af00 tx=0x7f91c402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bffa8706b19a6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.630+0000 7f91cbe00640 1 -- 172.21.15.73:0/1697263103 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c402cc70 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91cbe00640 1 -- 172.21.15.73:0/1697263103 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c402cdd0 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91cbe00640 1 -- 172.21.15.73:0/1697263103 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4034660 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 -- 172.21.15.73:0/1697263103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0ad6b0 msgr2=0x7f91cc0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 --2- 172.21.15.73:0/1697263103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0cb2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91cc06af00 tx=0x7f91c402bec0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 -- 172.21.15.73:0/1697263103 shutdown_connections 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 --2- 172.21.15.73:0/1697263103 >> v2:172.21.15.157:3300/0 conn(0x7f91cc0b1870 0x7f91cc0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 --2- 172.21.15.73:0/1697263103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0cb2f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 -- 172.21.15.73:0/1697263103 >> 172.21.15.73:0/1697263103 conn(0x7f91cc0ce8e0 msgr2=0x7f91cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 -- 172.21.15.73:0/1697263103 shutdown_connections 2024-04-05T22:25:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.631+0000 7f91d3600640 1 -- 172.21.15.73:0/1697263103 wait complete. 2024-04-05T22:25:05.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.633+0000 7f91d3600640 1 Processor -- start 2024-04-05T22:25:05.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.633+0000 7f91d3600640 1 -- start start 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91cc160620 0x7f91cc15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1f6500 con 0x7f91cc0b1870 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0caf20 con 0x7f91cc0ad6b0 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91d3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1f5bd0 con 0x7f91cc160620 2024-04-05T22:25:05.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.634+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56584/0 (socket says 172.21.15.73:56584) 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 -- 172.21.15.73:0/3275429902 learned_addr learned my addr 172.21.15.73:0/3275429902 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 -- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91cc160620 msgr2=0x7f91cc15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91cc160620 0x7f91cc15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 -- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91cc0ad6b0 msgr2=0x7f91cc0c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 -- 172.21.15.73:0/3275429902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c4009a70 con 0x7f91cc0b1870 2024-04-05T22:25:05.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.635+0000 7f91c3e00640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00d800 tx=0x7f91bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.636+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00b6c0 con 0x7f91cc0b1870 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.636+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc00b820 con 0x7f91cc0b1870 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.636+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00bbb0 con 0x7f91cc0b1870 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.636+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc1f5eb0 con 0x7f91cc0b1870 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.636+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc0c8a90 con 0x7f91cc0b1870 2024-04-05T22:25:05.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.637+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc066af0 con 0x7f91cc0b1870 2024-04-05T22:25:05.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.638+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f91bc00bd10 con 0x7f91cc0b1870 2024-04-05T22:25:05.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.639+0000 7f91c9600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 0x7f91a806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:05.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.639+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91bc089d00 con 0x7f91cc0b1870 2024-04-05T22:25:05.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.639+0000 7f91d0c00640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 0x7f91a806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:05.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.639+0000 7f91d0c00640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 0x7f91a806d270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001570 tx=0x7f91b4007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:05.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.643+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f91bc054b80 con 0x7f91cc0b1870 2024-04-05T22:25:05.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.874+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91cc0b1fc0 con 0x7f91cc0b1870 2024-04-05T22:25:05.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.876+0000 7f91c9600640 1 -- 172.21.15.73:0/3275429902 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f91bc0581a0 con 0x7f91cc0b1870 2024-04-05T22:25:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:05.876 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:25:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:25:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 msgr2=0x7f91a806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 0x7f91a806d270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001570 tx=0x7f91b4007930 comp rx=0 tx=0).stop 2024-04-05T22:25:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 msgr2=0x7f91cc15e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00d800 tx=0x7f91bc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 shutdown_connections 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91a806adb0 0x7f91a806d270 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91cc160620 0x7f91cc15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc0b1870 0x7f91cc15e550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 --2- 172.21.15.73:0/3275429902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91cc0ad6b0 0x7f91cc0c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 >> 172.21.15.73:0/3275429902 conn(0x7f91cc0ce8e0 msgr2=0x7f91cc0cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 shutdown_connections 2024-04-05T22:25:05.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:05.879+0000 7f91d3600640 1 -- 172.21.15.73:0/3275429902 wait complete. 2024-04-05T22:25:05.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:06.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.096+0000 7eff9e400640 1 Processor -- start 2024-04-05T22:25:06.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.096+0000 7eff9e400640 1 -- start start 2024-04-05T22:25:06.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.097+0000 7eff9e400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 0x7eff980d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.097+0000 7eff9e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.097+0000 7eff9e400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980ceee0 con 0x7eff9815cf70 2024-04-05T22:25:06.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.097+0000 7eff9e400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980ce5b0 con 0x7eff980d1930 2024-04-05T22:25:06.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.097+0000 7eff9e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff980ce890 con 0x7eff980d0bd0 2024-04-05T22:25:06.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:53378/0 (socket says 172.21.15.73:53378) 2024-04-05T22:25:06.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 -- 172.21.15.73:0/484916620 learned_addr learned my addr 172.21.15.73:0/484916620 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff96a00640 1 --2- 172.21.15.73:0/484916620 >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 0x7eff980d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 -- 172.21.15.73:0/484916620 >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 msgr2=0x7eff980d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:06.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 --2- 172.21.15.73:0/484916620 >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 0x7eff980d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 -- 172.21.15.73:0/484916620 >> v1:172.21.15.157:6789/0 conn(0x7eff9815cf70 legacy=0x7eff980d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.098+0000 7eff97e00640 1 -- 172.21.15.73:0/484916620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980ceb70 con 0x7eff980d0bd0 2024-04-05T22:25:06.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.099+0000 7eff96a00640 1 --2- 172.21.15.73:0/484916620 >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 0x7eff980d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:25:06.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.100+0000 7eff97e00640 1 --2- 172.21.15.73:0/484916620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff90009870 tx=0x7eff9002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad1b046cdd51cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.100+0000 7eff96000640 1 -- 172.21.15.73:0/484916620 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90031640 con 0x7eff980d0bd0 2024-04-05T22:25:06.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.100+0000 7eff96000640 1 -- 172.21.15.73:0/484916620 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff900317a0 con 0x7eff980d0bd0 2024-04-05T22:25:06.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.100+0000 7eff96000640 1 -- 172.21.15.73:0/484916620 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90031b30 con 0x7eff980d0bd0 2024-04-05T22:25:06.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 -- 172.21.15.73:0/484916620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 msgr2=0x7eff980b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 --2- 172.21.15.73:0/484916620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff90009870 tx=0x7eff9002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 -- 172.21.15.73:0/484916620 shutdown_connections 2024-04-05T22:25:06.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 --2- 172.21.15.73:0/484916620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d0bd0 0x7eff980b2900 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 --2- 172.21.15.73:0/484916620 >> v2:172.21.15.157:3300/0 conn(0x7eff980d1930 0x7eff980d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 -- 172.21.15.73:0/484916620 >> 172.21.15.73:0/484916620 conn(0x7eff980d2df0 msgr2=0x7eff980d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:06.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 -- 172.21.15.73:0/484916620 shutdown_connections 2024-04-05T22:25:06.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.101+0000 7eff9e400640 1 -- 172.21.15.73:0/484916620 wait complete. 2024-04-05T22:25:06.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.103+0000 7eff9e400640 1 Processor -- start 2024-04-05T22:25:06.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.103+0000 7eff9e400640 1 -- start start 2024-04-05T22:25:06.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d1930 0x7eff9815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 0x7eff9815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b0210 con 0x7eff980d0bd0 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b04f0 con 0x7eff9815cf70 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff9e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b5bb0 con 0x7eff980d1930 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50780/0 (socket says 172.21.15.73:50780) 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.104+0000 7eff97400640 1 -- 172.21.15.73:0/723215095 learned_addr learned my addr 172.21.15.73:0/723215095 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff96a00640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d1930 0x7eff9815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97e00640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 0x7eff9815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 -- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d1930 msgr2=0x7eff9815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d1930 0x7eff9815ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 -- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 msgr2=0x7eff9815b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 0x7eff9815b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 -- 172.21.15.73:0/723215095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009a70 con 0x7eff980d0bd0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97e00640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 0x7eff9815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff97400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7eff8800c400 tx=0x7eff8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880052d0 con 0x7eff980d0bd0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff88005430 con 0x7eff980d0bd0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88003830 con 0x7eff980d0bd0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.105+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff900094d0 con 0x7eff980d0bd0 2024-04-05T22:25:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.106+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9815b8d0 con 0x7eff980d0bd0 2024-04-05T22:25:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.107+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff980b5a10 con 0x7eff980d0bd0 2024-04-05T22:25:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.108+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7eff88003990 con 0x7eff980d0bd0 2024-04-05T22:25:06.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.109+0000 7eff8f400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 0x7eff7406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.109+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7eff88088c40 con 0x7eff980d0bd0 2024-04-05T22:25:06.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.109+0000 7eff96a00640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 0x7eff7406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.110+0000 7eff96a00640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 0x7eff7406d270 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff981f5a10 tx=0x7eff8000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.113+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7eff980b5a10 con 0x7eff980d0bd0 2024-04-05T22:25:06.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.306+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff980b60d0 con 0x7eff980d0bd0 2024-04-05T22:25:06.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.308+0000 7eff8f400640 1 -- 172.21.15.73:0/723215095 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16197 (secure 0 0 0) 0x7eff980b60d0 con 0x7eff980d0bd0 2024-04-05T22:25:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:06.309 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:24:03.627819+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 msgr2=0x7eff7406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 0x7eff7406d270 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff981f5a10 tx=0x7eff8000b040 comp rx=0 tx=0).stop 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 msgr2=0x7eff980c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7eff8800c400 tx=0x7eff8800c8c0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 shutdown_connections 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7eff7406adb0 0x7eff7406d270 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815cf70 0x7eff9815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980d1930 0x7eff9815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 --2- 172.21.15.73:0/723215095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff980d0bd0 0x7eff980c69c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.311+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 >> 172.21.15.73:0/723215095 conn(0x7eff980d2df0 msgr2=0x7eff980cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:06.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.312+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 shutdown_connections 2024-04-05T22:25:06.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.312+0000 7eff9e400640 1 -- 172.21.15.73:0/723215095 wait complete. 2024-04-05T22:25:06.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:23:07.732446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:23:08.298551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:06.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:06.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.540+0000 7f8b22a00640 1 Processor -- start 2024-04-05T22:25:06.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.540+0000 7f8b22a00640 1 -- start start 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b22a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1c06cd50 0x7f8b1c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b22a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c0add00 0x7f8b1c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b22a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c0b27f0 con 0x7f8b1c1599f0 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b22a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c0b3890 con 0x7f8b1c06cd50 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b22a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0b2ad0 con 0x7f8b1c0add00 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1c06cd50 0x7f8b1c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1c06cd50 0x7f8b1c070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50790/0 (socket says 172.21.15.73:50790) 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 -- 172.21.15.73:0/3304456093 learned_addr learned my addr 172.21.15.73:0/3304456093 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b20a00640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c0add00 0x7f8b1c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 -- 172.21.15.73:0/3304456093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c0add00 msgr2=0x7f8b1c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c0add00 0x7f8b1c159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 -- 172.21.15.73:0/3304456093 >> v1:172.21.15.157:6789/0 conn(0x7f8b1c1599f0 legacy=0x7f8b1c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:06.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.542+0000 7f8b1b400640 1 -- 172.21.15.73:0/3304456093 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0b2db0 con 0x7f8b1c06cd50 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.543+0000 7f8b1b400640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c070a40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c071ad0 tx=0x7f8b0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5852676fb2c19792 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.544+0000 7f8b1aa00640 1 -- 172.21.15.73:0/3304456093 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04031830 con 0x7f8b1c06cd50 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.544+0000 7f8b1aa00640 1 -- 172.21.15.73:0/3304456093 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b04031990 con 0x7f8b1c06cd50 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.544+0000 7f8b1aa00640 1 -- 172.21.15.73:0/3304456093 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04031d20 con 0x7f8b1c06cd50 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.544+0000 7f8b22a00640 1 -- 172.21.15.73:0/3304456093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c06cd50 msgr2=0x7f8b1c070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.544+0000 7f8b22a00640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c070a40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c071ad0 tx=0x7f8b0402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 -- 172.21.15.73:0/3304456093 shutdown_connections 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c0add00 0x7f8b1c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 --2- 172.21.15.73:0/3304456093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c070a40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 -- 172.21.15.73:0/3304456093 >> 172.21.15.73:0/3304456093 conn(0x7f8b1c069cb0 msgr2=0x7f8b1c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:06.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 -- 172.21.15.73:0/3304456093 shutdown_connections 2024-04-05T22:25:06.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.545+0000 7f8b22a00640 1 -- 172.21.15.73:0/3304456093 wait complete. 2024-04-05T22:25:06.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.546+0000 7f8b22a00640 1 Processor -- start 2024-04-05T22:25:06.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.546+0000 7f8b22a00640 1 -- start start 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c1599f0 0x7f8b1c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c153a80 con 0x7f8b1c0add00 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c155220 con 0x7f8b1c06cd50 2024-04-05T22:25:06.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.547+0000 7f8b22a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c152b30 con 0x7f8b1c1599f0 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50798/0 (socket says 172.21.15.73:50798) 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 -- 172.21.15.73:0/1591114720 learned_addr learned my addr 172.21.15.73:0/1591114720 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 -- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c1599f0 msgr2=0x7f8b1c14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c1599f0 0x7f8b1c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 -- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b1c06cd50 msgr2=0x7f8b1c153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 -- 172.21.15.73:0/1591114720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040094d0 con 0x7f8b1c0add00 2024-04-05T22:25:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.548+0000 7f8b1b400640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8b04000c00 tx=0x7f8b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.549+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0403aa60 con 0x7f8b1c0add00 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.549+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0403abc0 con 0x7f8b1c0add00 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.549+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040443f0 con 0x7f8b1c0add00 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.549+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c154c60 con 0x7f8b1c0add00 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.549+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c152d40 con 0x7f8b1c0add00 2024-04-05T22:25:06.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.550+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c066f20 con 0x7f8b1c0add00 2024-04-05T22:25:06.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.551+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f8b0404d450 con 0x7f8b1c0add00 2024-04-05T22:25:06.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.551+0000 7f8b13e00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 0x7f8af806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.552+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b040abac0 con 0x7f8b1c0add00 2024-04-05T22:25:06.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.552+0000 7f8b1be00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 0x7f8af806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.552+0000 7f8b1be00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 0x7f8af806d270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c006fd0 tx=0x7f8b0c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.556+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8b040768d0 con 0x7f8b1c0add00 2024-04-05T22:25:06.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.749+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b1c068160 con 0x7f8b1c0add00 2024-04-05T22:25:06.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.750+0000 7f8b13e00640 1 -- 172.21.15.73:0/1591114720 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b040abe60 con 0x7f8b1c0add00 2024-04-05T22:25:06.750 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 msgr2=0x7f8af806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 0x7f8af806d270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c006fd0 tx=0x7f8b0c006bc0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 msgr2=0x7f8b1c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8b04000c00 tx=0x7f8b040046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 shutdown_connections 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8af806adb0 0x7f8af806d270 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b1c1599f0 0x7f8b1c14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1c0add00 0x7f8b1c154330 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 --2- 172.21.15.73:0/1591114720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b1c06cd50 0x7f8b1c153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 >> 172.21.15.73:0/1591114720 conn(0x7f8b1c069cb0 msgr2=0x7f8b1c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 shutdown_connections 2024-04-05T22:25:06.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.752+0000 7f8b22a00640 1 -- 172.21.15.73:0/1591114720 wait complete. 2024-04-05T22:25:06.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:25:06.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.970+0000 7f8d34800640 1 Processor -- start 2024-04-05T22:25:06.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.970+0000 7f8d34800640 1 -- start start 2024-04-05T22:25:06.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d34800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d34800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d2c0add00 0x7f8d2c06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d34800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b27f0 con 0x7f8d2c06cb80 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d34800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c06d010 con 0x7f8d2c0add00 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d34800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b3890 con 0x7f8d2c1599f0 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d31400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d2c0add00 0x7f8d2c06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d31400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d2c0add00 0x7f8d2c06c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50812/0 (socket says 172.21.15.73:50812) 2024-04-05T22:25:06.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.971+0000 7f8d31400640 1 -- 172.21.15.73:0/1022929580 learned_addr learned my addr 172.21.15.73:0/1022929580 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d31e00640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d31400640 1 -- 172.21.15.73:0/1022929580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 msgr2=0x7f8d2c070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d31400640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d31400640 1 -- 172.21.15.73:0/1022929580 >> v1:172.21.15.73:6789/0 conn(0x7f8d2c06cb80 legacy=0x7f8d2c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d31400640 1 -- 172.21.15.73:0/1022929580 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0b2ad0 con 0x7f8d2c0add00 2024-04-05T22:25:06.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.972+0000 7f8d30a00640 1 -- 172.21.15.73:0/1022929580 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238648451 0 0) 0x7f8d2c0b27f0 con 0x7f8d2c06cb80 2024-04-05T22:25:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.973+0000 7f8d31400640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c06c640 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009870 tx=0x7f8d1c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ffdda17e7acb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.973+0000 7f8d30a00640 1 -- 172.21.15.73:0/1022929580 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c030840 con 0x7f8d2c0add00 2024-04-05T22:25:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.973+0000 7f8d30a00640 1 -- 172.21.15.73:0/1022929580 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0309a0 con 0x7f8d2c0add00 2024-04-05T22:25:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.973+0000 7f8d30a00640 1 -- 172.21.15.73:0/1022929580 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c030d30 con 0x7f8d2c0add00 2024-04-05T22:25:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 -- 172.21.15.73:0/1022929580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 msgr2=0x7f8d2c06c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c06c640 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009870 tx=0x7f8d1c02b980 comp rx=0 tx=0).stop 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 -- 172.21.15.73:0/1022929580 shutdown_connections 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c06c640 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 --2- 172.21.15.73:0/1022929580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 -- 172.21.15.73:0/1022929580 >> 172.21.15.73:0/1022929580 conn(0x7f8d2c069cb0 msgr2=0x7f8d2c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 -- 172.21.15.73:0/1022929580 shutdown_connections 2024-04-05T22:25:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.974+0000 7f8d34800640 1 -- 172.21.15.73:0/1022929580 wait complete. 2024-04-05T22:25:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.976+0000 7f8d34800640 1 Processor -- start 2024-04-05T22:25:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.976+0000 7f8d34800640 1 -- start start 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d2c06cb80 0x7f8d2c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c152a50 con 0x7f8d2c0add00 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15ce80 con 0x7f8d2c06cb80 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d34800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15d700 con 0x7f8d2c1599f0 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50820/0 (socket says 172.21.15.73:50820) 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d31400640 1 -- 172.21.15.73:0/2888576142 learned_addr learned my addr 172.21.15.73:0/2888576142 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.977+0000 7f8d32800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31e00640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d2c06cb80 0x7f8d2c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 -- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 msgr2=0x7f8d2c153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 -- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d2c06cb80 msgr2=0x7f8d2c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d2c06cb80 0x7f8d2c152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 -- 172.21.15.73:0/2888576142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0094d0 con 0x7f8d2c0add00 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d31400640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0090b0 tx=0x7f8d1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c03e040 con 0x7f8d2c0add00 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c039d80 con 0x7f8d2c0add00 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c004270 con 0x7f8d2c0add00 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c157ea0 con 0x7f8d2c0add00 2024-04-05T22:25:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.978+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c15d0b0 con 0x7f8d2c0add00 2024-04-05T22:25:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.980+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c067270 con 0x7f8d2c0add00 2024-04-05T22:25:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.980+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f8d1c0043d0 con 0x7f8d2c0add00 2024-04-05T22:25:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.981+0000 7f8d22000640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 0x7f8d1006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.981+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8d1c0ac330 con 0x7f8d2c0add00 2024-04-05T22:25:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.982+0000 7f8d31e00640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 0x7f8d1006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.982+0000 7f8d31e00640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 0x7f8d1006d270 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d24008e50 tx=0x7f8d240083a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:06.986+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8d1c077250 con 0x7f8d2c0add00 2024-04-05T22:25:07.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.178+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d2c06c640 con 0x7f8d2c0add00 2024-04-05T22:25:07.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.179+0000 7f8d22000640 1 -- 172.21.15.73:0/2888576142 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d1c07a870 con 0x7f8d2c0add00 2024-04-05T22:25:07.179 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:25:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 msgr2=0x7f8d1006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 0x7f8d1006d270 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d24008e50 tx=0x7f8d240083a0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 msgr2=0x7f8d2c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0090b0 tx=0x7f8d1c004730 comp rx=0 tx=0).stop 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 shutdown_connections 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8d1006adb0 0x7f8d1006d270 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d2c1599f0 0x7f8d2c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2c0add00 0x7f8d2c15ae20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 --2- 172.21.15.73:0/2888576142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d2c06cb80 0x7f8d2c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 >> 172.21.15.73:0/2888576142 conn(0x7f8d2c069cb0 msgr2=0x7f8d2c158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 shutdown_connections 2024-04-05T22:25:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.181+0000 7f8d34800640 1 -- 172.21.15.73:0/2888576142 wait complete. 2024-04-05T22:25:07.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:25:07.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.399+0000 7ff334200640 1 Processor -- start 2024-04-05T22:25:07.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.400+0000 7ff334200640 1 -- start start 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.401+0000 7ff334200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c06c630 0x7ff32c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.401+0000 7ff334200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff32c15dd90 0x7ff32c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.401+0000 7ff334200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c146ac0 con 0x7ff32c160930 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.401+0000 7ff334200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c06cf50 con 0x7ff32c15dd90 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.401+0000 7ff334200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c070720 con 0x7ff32c06c630 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff330e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff32c15dd90 0x7ff32c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff332200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff32c160930 0x7ff32c160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48184/0 (socket says 172.21.15.73:48184) 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff332200640 1 -- 172.21.15.73:0/3689956364 learned_addr learned my addr 172.21.15.73:0/3689956364 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff331800640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c06c630 0x7ff32c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff330e00640 1 -- 172.21.15.73:0/3689956364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c06c630 msgr2=0x7ff32c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff330e00640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c06c630 0x7ff32c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff330e00640 1 -- 172.21.15.73:0/3689956364 >> v1:172.21.15.73:6789/0 conn(0x7ff32c160930 legacy=0x7ff32c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff330e00640 1 -- 172.21.15.73:0/3689956364 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0acf60 con 0x7ff32c15dd90 2024-04-05T22:25:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.402+0000 7ff32be00640 1 -- 172.21.15.73:0/3689956364 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107007169 0 0) 0x7ff32c146ac0 con 0x7ff32c160930 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.403+0000 7ff330e00640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c15dd90 0x7ff32c06ca10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097a0 tx=0x7ff31802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85e4437682ad870 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff32be00640 1 -- 172.21.15.73:0/3689956364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318031830 con 0x7ff32c15dd90 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff32be00640 1 -- 172.21.15.73:0/3689956364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318031990 con 0x7ff32c15dd90 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff32be00640 1 -- 172.21.15.73:0/3689956364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318031d20 con 0x7ff32c15dd90 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 -- 172.21.15.73:0/3689956364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c15dd90 msgr2=0x7ff32c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c15dd90 0x7ff32c06ca10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097a0 tx=0x7ff31802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 -- 172.21.15.73:0/3689956364 shutdown_connections 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c15dd90 0x7ff32c06ca10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 --2- 172.21.15.73:0/3689956364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c06c630 0x7ff32c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 -- 172.21.15.73:0/3689956364 >> 172.21.15.73:0/3689956364 conn(0x7ff32c0b0830 msgr2=0x7ff32c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 -- 172.21.15.73:0/3689956364 shutdown_connections 2024-04-05T22:25:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.404+0000 7ff334200640 1 -- 172.21.15.73:0/3689956364 wait complete. 2024-04-05T22:25:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.406+0000 7ff334200640 1 Processor -- start 2024-04-05T22:25:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.406+0000 7ff334200640 1 -- start start 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff32c15dd90 0x7ff32c13f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c160930 0x7ff32c13f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c46e0 con 0x7ff32c06c630 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c3b40 con 0x7ff32c15dd90 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff334200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c3db0 con 0x7ff32c160930 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff331800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff331800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50830/0 (socket says 172.21.15.73:50830) 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.407+0000 7ff331800640 1 -- 172.21.15.73:0/1658289830 learned_addr learned my addr 172.21.15.73:0/1658289830 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff330e00640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff32c15dd90 0x7ff32c13f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff332200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c160930 0x7ff32c13f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 -- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c160930 msgr2=0x7ff32c13f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c160930 0x7ff32c13f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 -- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff32c15dd90 msgr2=0x7ff32c13f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff32c15dd90 0x7ff32c13f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 -- 172.21.15.73:0/1658289830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009400 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.408+0000 7ff331800640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff32000d620 tx=0x7ff32000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.409+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000b620 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.409+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32000b780 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.409+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000bb30 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.409+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c0c4090 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.409+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c0c53a0 con 0x7ff32c06c630 2024-04-05T22:25:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.410+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0c5590 con 0x7ff32c06c630 2024-04-05T22:25:07.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.411+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff32000bc90 con 0x7ff32c06c630 2024-04-05T22:25:07.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.412+0000 7ff329600640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 0x7ff30c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.412+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff320089ac0 con 0x7ff32c06c630 2024-04-05T22:25:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.413+0000 7ff330e00640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 0x7ff30c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.413+0000 7ff330e00640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 0x7ff30c06d270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.417+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff3200549f0 con 0x7ff32c06c630 2024-04-05T22:25:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.605+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff32c068160 con 0x7ff32c06c630 2024-04-05T22:25:07.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.606+0000 7ff329600640 1 -- 172.21.15.73:0/1658289830 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff320058010 con 0x7ff32c06c630 2024-04-05T22:25:07.606 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 msgr2=0x7ff30c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 0x7ff30c06d270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff318036000 tx=0x7ff31803d040 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 msgr2=0x7ff32c0c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff32000d620 tx=0x7ff32000dae0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 shutdown_connections 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff30c06adb0 0x7ff30c06d270 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32c160930 0x7ff32c13f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff32c15dd90 0x7ff32c13f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 --2- 172.21.15.73:0/1658289830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c06c630 0x7ff32c0c4e60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.608+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 >> 172.21.15.73:0/1658289830 conn(0x7ff32c0b0830 msgr2=0x7ff32c0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.609+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 shutdown_connections 2024-04-05T22:25:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.609+0000 7ff334200640 1 -- 172.21.15.73:0/1658289830 wait complete. 2024-04-05T22:25:07.625 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:07.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.821+0000 7ff3b9c00640 1 Processor -- start 2024-04-05T22:25:07.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.821+0000 7ff3b9c00640 1 -- start start 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3b4158860 0x7ff3b40b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b40b1f00 0x7ff3b415e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b406b500 con 0x7ff3b415eb60 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b415eff0 con 0x7ff3b4158860 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40ccaf0 con 0x7ff3b40b1f00 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3b4158860 0x7ff3b40b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff3b415eb60 0x7ff3b415afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48186/0 (socket says 172.21.15.73:48186) 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.822+0000 7ff3b9200640 1 -- 172.21.15.73:0/2700196573 learned_addr learned my addr 172.21.15.73:0/2700196573 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b3400640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b40b1f00 0x7ff3b415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b3e00640 1 -- 172.21.15.73:0/2700196573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b40b1f00 msgr2=0x7ff3b415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b3e00640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b40b1f00 0x7ff3b415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b2a00640 1 -- 172.21.15.73:0/2700196573 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192047601 0 0) 0x7ff3b406b500 con 0x7ff3b415eb60 2024-04-05T22:25:07.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b3e00640 1 -- 172.21.15.73:0/2700196573 >> v1:172.21.15.73:6789/0 conn(0x7ff3b415eb60 legacy=0x7ff3b415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.823+0000 7ff3b3e00640 1 -- 172.21.15.73:0/2700196573 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40b23a0 con 0x7ff3b4158860 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b3e00640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b4158860 0x7ff3b40b19c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40097a0 tx=0x7ff3a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f178d1361ffee56f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b2a00640 1 -- 172.21.15.73:0/2700196573 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a402e070 con 0x7ff3b4158860 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b2a00640 1 -- 172.21.15.73:0/2700196573 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a40315a0 con 0x7ff3b4158860 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b2a00640 1 -- 172.21.15.73:0/2700196573 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4031930 con 0x7ff3b4158860 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b9c00640 1 -- 172.21.15.73:0/2700196573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b4158860 msgr2=0x7ff3b40b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.824+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b4158860 0x7ff3b40b19c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40097a0 tx=0x7ff3a402b840 comp rx=0 tx=0).stop 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 -- 172.21.15.73:0/2700196573 shutdown_connections 2024-04-05T22:25:07.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b40b1f00 0x7ff3b415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/2700196573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b4158860 0x7ff3b40b19c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 -- 172.21.15.73:0/2700196573 >> 172.21.15.73:0/2700196573 conn(0x7ff3b40b06c0 msgr2=0x7ff3b40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 -- 172.21.15.73:0/2700196573 shutdown_connections 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.825+0000 7ff3b9c00640 1 -- 172.21.15.73:0/2700196573 wait complete. 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.826+0000 7ff3b9c00640 1 Processor -- start 2024-04-05T22:25:07.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 -- start start 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3b40b1f00 0x7ff3b4155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b415eb60 0x7ff3b41f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b414a770 con 0x7ff3b414a2c0 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4155a90 con 0x7ff3b40b1f00 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.827+0000 7ff3b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4155d70 con 0x7ff3b415eb60 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50844/0 (socket says 172.21.15.73:50844) 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 -- 172.21.15.73:0/568596959 learned_addr learned my addr 172.21.15.73:0/568596959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:07.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 -- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b415eb60 msgr2=0x7ff3b41f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b415eb60 0x7ff3b41f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 -- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3b40b1f00 msgr2=0x7ff3b4155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3b40b1f00 0x7ff3b4155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 -- 172.21.15.73:0/568596959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4009400 con 0x7ff3b414a2c0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.828+0000 7ff3b9200640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c7e0 tx=0x7ff3ac00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.829+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00a640 con 0x7ff3b414a2c0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.829+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac00a7c0 con 0x7ff3b414a2c0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.829+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00ab70 con 0x7ff3b414a2c0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.829+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b4156050 con 0x7ff3b414a2c0 2024-04-05T22:25:07.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.829+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b4156610 con 0x7ff3b414a2c0 2024-04-05T22:25:07.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.830+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4156800 con 0x7ff3b414a2c0 2024-04-05T22:25:07.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.831+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff3ac00acd0 con 0x7ff3b414a2c0 2024-04-05T22:25:07.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.831+0000 7ff3abe00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 0x7ff39006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:07.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.832+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff3ac088c00 con 0x7ff3b414a2c0 2024-04-05T22:25:07.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.832+0000 7ff3b3e00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 0x7ff39006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:07.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.832+0000 7ff3b3e00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 0x7ff39006d2a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4036000 tx=0x7ff3a4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:07.836+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff3ac053a80 con 0x7ff3b414a2c0 2024-04-05T22:25:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.066+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3b404e160 con 0x7ff3b414a2c0 2024-04-05T22:25:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.067+0000 7ff3abe00640 1 -- 172.21.15.73:0/568596959 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7ff3ac018020 con 0x7ff3b414a2c0 2024-04-05T22:25:08.068 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:08.068 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:25:08.068 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 msgr2=0x7ff39006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 0x7ff39006d2a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4036000 tx=0x7ff3a4030000 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 msgr2=0x7ff3b40c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c7e0 tx=0x7ff3ac00cca0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 shutdown_connections 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39006ade0 0x7ff39006d2a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b414a2c0 0x7ff3b40c8680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3b415eb60 0x7ff3b41f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 --2- 172.21.15.73:0/568596959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3b40b1f00 0x7ff3b4155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.070+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 >> 172.21.15.73:0/568596959 conn(0x7ff3b40b06c0 msgr2=0x7ff3b406f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.071+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 shutdown_connections 2024-04-05T22:25:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.071+0000 7ff3b9c00640 1 -- 172.21.15.73:0/568596959 wait complete. 2024-04-05T22:25:08.090 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:08.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.296+0000 7f1611200640 1 Processor -- start 2024-04-05T22:25:08.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.296+0000 7f1611200640 1 -- start start 2024-04-05T22:25:08.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.297+0000 7f1611200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f160c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.297+0000 7f1611200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f160c0ce000 0x7f160c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1611200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c15aa60 con 0x7f160c06d710 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1611200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c0b3b60 con 0x7f160c0ce000 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1611200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0701c0 con 0x7f160c1541a0 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1603e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f160c06d710 0x7f160c0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48192/0 (socket says 172.21.15.73:48192) 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1603e00640 1 -- 172.21.15.73:0/3126608373 learned_addr learned my addr 172.21.15.73:0/3126608373 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1609e00640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f160c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f160a800640 1 --2- 172.21.15.73:0/3126608373 >> v2:172.21.15.157:3300/0 conn(0x7f160c0ce000 0x7f160c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.298+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903235093 0 0) 0x7f160c15aa60 con 0x7f160c06d710 2024-04-05T22:25:08.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f0003520 con 0x7f160c06d710 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139066687 0 0) 0x7f15f0003520 con 0x7f160c06d710 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 msgr2=0x7f160c0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f160c0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 >> v2:172.21.15.157:3300/0 conn(0x7f160c0ce000 msgr2=0x7f160c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 --2- 172.21.15.73:0/3126608373 >> v2:172.21.15.157:3300/0 conn(0x7f160c0ce000 0x7f160c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c0b3880 con 0x7f160c06d710 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f15f4001670 con 0x7f160c06d710 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 >> v1:172.21.15.73:6789/0 conn(0x7f160c06d710 legacy=0x7f160c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0003970 0x7f15f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f15f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f0004d00 0x7f15f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0b3b60 con 0x7f15f0004d00 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0701c0 con 0x7f15f0003970 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f000b980 con 0x7f160c1541a0 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.299+0000 7f160a800640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f0004d00 0x7f15f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f1603e00640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f15f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f1609e00640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0003970 0x7f15f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f160a800640 1 -- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 msgr2=0x7f15f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f160a800640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c1541a0 0x7f15f00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f160a800640 1 -- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0003970 msgr2=0x7f15f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f160a800640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0003970 0x7f15f0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.300+0000 7f160a800640 1 -- 172.21.15.73:0/3126608373 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0003520 con 0x7f15f0004d00 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f160a800640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f0004d00 0x7f15f0005180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1604007390 tx=0x7f15f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8729e26e00fc13fc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160402ccf0 con 0x7f15f0004d00 2024-04-05T22:25:08.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160402ce50 con 0x7f15f0004d00 2024-04-05T22:25:08.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f1609400640 1 -- 172.21.15.73:0/3126608373 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604034660 con 0x7f15f0004d00 2024-04-05T22:25:08.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f1611200640 1 -- 172.21.15.73:0/3126608373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f0004d00 msgr2=0x7f15f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.301+0000 7f1611200640 1 --2- 172.21.15.73:0/3126608373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f0004d00 0x7f15f0005180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1604007390 tx=0x7f15f00057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.302+0000 7f160a800640 1 -- 172.21.15.73:0/3126608373 reap_dead start 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.302+0000 7f1611200640 1 -- 172.21.15.73:0/3126608373 shutdown_connections 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.302+0000 7f1611200640 1 -- 172.21.15.73:0/3126608373 >> 172.21.15.73:0/3126608373 conn(0x7f160c0ce8e0 msgr2=0x7f160c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.302+0000 7f1611200640 1 -- 172.21.15.73:0/3126608373 shutdown_connections 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.302+0000 7f1611200640 1 -- 172.21.15.73:0/3126608373 wait complete. 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.303+0000 7f1611200640 1 Processor -- start 2024-04-05T22:25:08.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.304+0000 7f1611200640 1 -- start start 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.304+0000 7f1611200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0004d00 0x7f160c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1611200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1611200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 0x7f160c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1611200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0aedd0 con 0x7f160c155570 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1611200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c156c00 con 0x7f15f0004d00 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1611200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c156ee0 con 0x7f160c0ae3e0 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50866/0 (socket says 172.21.15.73:50866) 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 -- 172.21.15.73:0/1651664845 learned_addr learned my addr 172.21.15.73:0/1651664845 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1609e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0004d00 0x7f160c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f160a800640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 0x7f160c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 -- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 msgr2=0x7f160c0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 0x7f160c0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 -- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0004d00 msgr2=0x7f160c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0004d00 0x7f160c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f1603e00640 1 -- 172.21.15.73:0/1651664845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c1571c0 con 0x7f160c155570 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.305+0000 7f160a800640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 0x7f160c0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1603e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15f400d800 tx=0x7f15f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40074e0 con 0x7f160c155570 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4007640 con 0x7f160c155570 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40049e0 con 0x7f160c155570 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c1574a0 con 0x7f160c155570 2024-04-05T22:25:08.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.306+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c157ca0 con 0x7f160c155570 2024-04-05T22:25:08.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.307+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c066af0 con 0x7f160c155570 2024-04-05T22:25:08.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.308+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f15f4004b40 con 0x7f160c155570 2024-04-05T22:25:08.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.309+0000 7f1602000640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 0x7f15e806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.309+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f15f4089210 con 0x7f160c155570 2024-04-05T22:25:08.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.310+0000 7f1609e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 0x7f15e806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.310+0000 7f1609e00640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 0x7f15e806d270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00bb90 tx=0x7f15fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.314+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f15f4054020 con 0x7f160c155570 2024-04-05T22:25:08.488 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:25:08.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:25:08.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.507+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f160c1579c0 con 0x7f160c155570 2024-04-05T22:25:08.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.509+0000 7f1602000640 1 -- 172.21.15.73:0/1651664845 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16197 (secure 0 0 0) 0x7f160c1579c0 con 0x7f160c155570 2024-04-05T22:25:08.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:08.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:24:03.627819+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 msgr2=0x7f15e806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 0x7f15e806d270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00bb90 tx=0x7f15fc008040 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 msgr2=0x7f160c1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15f400d800 tx=0x7f15f400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 shutdown_connections 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f15e806adb0 0x7f15e806d270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f160c0ae3e0 0x7f160c0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c155570 0x7f160c1562f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 --2- 172.21.15.73:0/1651664845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0004d00 0x7f160c151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 >> 172.21.15.73:0/1651664845 conn(0x7f160c0ce8e0 msgr2=0x7f160c0b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 shutdown_connections 2024-04-05T22:25:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.511+0000 7f1611200640 1 -- 172.21.15.73:0/1651664845 wait complete. 2024-04-05T22:25:08.529 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:25:08.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.774+0000 7f7512400640 1 Processor -- start 2024-04-05T22:25:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.774+0000 7f7512400640 1 -- start start 2024-04-05T22:25:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f7512400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f750c1541a0 0x7f750c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f7512400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c15aa60 con 0x7f750c0ce000 2024-04-05T22:25:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f7512400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c0b3b60 con 0x7f750c06d710 2024-04-05T22:25:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f7512400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0701c0 con 0x7f750c1541a0 2024-04-05T22:25:08.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f750be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f750c0ce000 0x7f750c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48206/0 (socket says 172.21.15.73:48206) 2024-04-05T22:25:08.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.775+0000 7f750be00640 1 -- 172.21.15.73:0/444099558 learned_addr learned my addr 172.21.15.73:0/444099558 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:08.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033897895 0 0) 0x7f750c15aa60 con 0x7f750c0ce000 2024-04-05T22:25:08.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750b400640 1 --2- 172.21.15.73:0/444099558 >> v2:172.21.15.157:3300/0 conn(0x7f750c1541a0 0x7f750c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f0003520 con 0x7f750c0ce000 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649968750 0 0) 0x7f74f0003520 con 0x7f750c0ce000 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 >> v2:172.21.15.157:3300/0 conn(0x7f750c1541a0 msgr2=0x7f750c0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 --2- 172.21.15.73:0/444099558 >> v2:172.21.15.157:3300/0 conn(0x7f750c1541a0 0x7f750c0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 >> v1:172.21.15.157:6789/0 conn(0x7f750c06d710 legacy=0x7f750c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0b3880 con 0x7f750c0ce000 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f75040019f0 con 0x7f750c0ce000 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 >> v1:172.21.15.73:6789/0 conn(0x7f750c0ce000 legacy=0x7f750c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f0003970 0x7f74f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.776+0000 7f750aa00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f00047c0 0x7f74f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750aa00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 0x7f74f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0701c0 con 0x7f74f0005180 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f000c120 con 0x7f74f0003970 2024-04-05T22:25:08.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f000c2e0 con 0x7f74f00047c0 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 0x7f74f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f7503e00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f00047c0 0x7f74f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 -- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f00047c0 msgr2=0x7f74f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750b400640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f0003970 0x7f74f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f00047c0 0x7f74f0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 -- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f0003970 msgr2=0x7f74f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f0003970 0x7f74f0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.777+0000 7f750be00640 1 -- 172.21.15.73:0/444099558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0003520 con 0x7f74f0005180 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.778+0000 7f750be00640 1 -- 172.21.15.73:0/444099558 reap_dead start 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.778+0000 7f750be00640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 0x7f74f000b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f750400a820 tx=0x7f74f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57efef2932f24ab server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.778+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040072d0 con 0x7f74f0005180 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750402f070 con 0x7f74f0005180 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f750aa00640 1 -- 172.21.15.73:0/444099558 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040076b0 con 0x7f74f0005180 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 -- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 msgr2=0x7f74f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 0x7f74f000b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f750400a820 tx=0x7f74f00057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 -- 172.21.15.73:0/444099558 shutdown_connections 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 --2- 172.21.15.73:0/444099558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74f0005180 0x7f74f000b980 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 -- 172.21.15.73:0/444099558 >> 172.21.15.73:0/444099558 conn(0x7f750c0ce8e0 msgr2=0x7f750c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:08.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 -- 172.21.15.73:0/444099558 shutdown_connections 2024-04-05T22:25:08.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.779+0000 7f7512400640 1 -- 172.21.15.73:0/444099558 wait complete. 2024-04-05T22:25:08.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.781+0000 7f7512400640 1 Processor -- start 2024-04-05T22:25:08.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.781+0000 7f7512400640 1 -- start start 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0005180 0x7f750c168b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f750c14a4c0 0x7f750c149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0afec0 con 0x7f750c0af4d0 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c14aa60 con 0x7f750c14a4c0 2024-04-05T22:25:08.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f7512400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750c14ad40 con 0x7f74f0005180 2024-04-05T22:25:08.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f750be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f750be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50886/0 (socket says 172.21.15.73:50886) 2024-04-05T22:25:08.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.782+0000 7f750be00640 1 -- 172.21.15.73:0/687247578 learned_addr learned my addr 172.21.15.73:0/687247578 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750b400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0005180 0x7f750c168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7503e00640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f750c14a4c0 0x7f750c149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 -- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0005180 msgr2=0x7f750c168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0005180 0x7f750c168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 -- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f750c14a4c0 msgr2=0x7f750c149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f750c14a4c0 0x7f750c149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 -- 172.21.15.73:0/687247578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c14b020 con 0x7f750c0af4d0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f750be00640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f750402cc40 tx=0x7f750c070950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040019f0 con 0x7f750c0af4d0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504004b60 con 0x7f750c0af4d0 2024-04-05T22:25:08.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504007450 con 0x7f750c0af4d0 2024-04-05T22:25:08.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c14b300 con 0x7f750c0af4d0 2024-04-05T22:25:08.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.783+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c1690b0 con 0x7f750c0af4d0 2024-04-05T22:25:08.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.784+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c066af0 con 0x7f750c0af4d0 2024-04-05T22:25:08.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.785+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f7504031030 con 0x7f750c0af4d0 2024-04-05T22:25:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.786+0000 7f7503400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 0x7f74e806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.786+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f75040ab4e0 con 0x7f750c0af4d0 2024-04-05T22:25:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.786+0000 7f750b400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 0x7f74e806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:08.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.787+0000 7f750b400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 0x7f74e806d270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009a10 tx=0x7f74fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:08.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:08.791+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7504076250 con 0x7f750c0af4d0 2024-04-05T22:25:09.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.028+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f750c066af0 con 0x7f750c0af4d0 2024-04-05T22:25:09.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.029+0000 7f7503400640 1 -- 172.21.15.73:0/687247578 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f7504079870 con 0x7f750c0af4d0 2024-04-05T22:25:09.029 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:09.029 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:25:09.029 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:25:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 msgr2=0x7f74e806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 0x7f74e806d270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009a10 tx=0x7f74fc008040 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 msgr2=0x7f750c0af980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f750402cc40 tx=0x7f750c070950 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 shutdown_connections 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f74e806adb0 0x7f74e806d270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f750c0af4d0 0x7f750c0af980 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f750c14a4c0 0x7f750c149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 --2- 172.21.15.73:0/687247578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74f0005180 0x7f750c168b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 >> 172.21.15.73:0/687247578 conn(0x7f750c0ce8e0 msgr2=0x7f750c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 shutdown_connections 2024-04-05T22:25:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.032+0000 7f7512400640 1 -- 172.21.15.73:0/687247578 wait complete. 2024-04-05T22:25:09.051 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:09.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.249+0000 7feda5c00640 1 Processor -- start 2024-04-05T22:25:09.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.249+0000 7feda5c00640 1 -- start start 2024-04-05T22:25:09.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7feda5c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda0158860 0x7feda00b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7feda5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 0x7feda015e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7feda5c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda006b500 con 0x7feda015eb60 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7feda5c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda015eff0 con 0x7feda0158860 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7feda5c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00ccaf0 con 0x7feda00b1f00 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7fed9fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda0158860 0x7feda00b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.250+0000 7fed9fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda0158860 0x7feda00b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50890/0 (socket says 172.21.15.73:50890) 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9fe00640 1 -- 172.21.15.73:0/2809248207 learned_addr learned my addr 172.21.15.73:0/2809248207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:09.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9f400640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 0x7feda015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9fe00640 1 -- 172.21.15.73:0/2809248207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 msgr2=0x7feda015e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:09.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9fe00640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 0x7feda015e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9fe00640 1 -- 172.21.15.73:0/2809248207 >> v1:172.21.15.157:6789/0 conn(0x7feda015eb60 legacy=0x7feda015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:09.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9fe00640 1 -- 172.21.15.73:0/2809248207 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00b23a0 con 0x7feda0158860 2024-04-05T22:25:09.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.251+0000 7fed9f400640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 0x7feda015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:09.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.252+0000 7fed9fe00640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda0158860 0x7feda00b19c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cfb0a53770227a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.253+0000 7fed9ea00640 1 -- 172.21.15.73:0/2809248207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9002c9e0 con 0x7feda0158860 2024-04-05T22:25:09.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.253+0000 7fed9ea00640 1 -- 172.21.15.73:0/2809248207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9002cb40 con 0x7feda0158860 2024-04-05T22:25:09.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.253+0000 7fed9ea00640 1 -- 172.21.15.73:0/2809248207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900343f0 con 0x7feda0158860 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.253+0000 7feda5c00640 1 -- 172.21.15.73:0/2809248207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda0158860 msgr2=0x7feda00b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.253+0000 7feda5c00640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda0158860 0x7feda00b19c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed9002be40 comp rx=0 tx=0).stop 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 -- 172.21.15.73:0/2809248207 shutdown_connections 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda00b1f00 0x7feda015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 --2- 172.21.15.73:0/2809248207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda0158860 0x7feda00b19c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 -- 172.21.15.73:0/2809248207 >> 172.21.15.73:0/2809248207 conn(0x7feda00b06c0 msgr2=0x7feda00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 -- 172.21.15.73:0/2809248207 shutdown_connections 2024-04-05T22:25:09.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.254+0000 7feda5c00640 1 -- 172.21.15.73:0/2809248207 wait complete. 2024-04-05T22:25:09.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.255+0000 7feda5c00640 1 Processor -- start 2024-04-05T22:25:09.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.255+0000 7feda5c00640 1 -- start start 2024-04-05T22:25:09.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda015eb60 0x7feda013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feda014dd80 0x7feda0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda0158110 con 0x7feda00b1f00 2024-04-05T22:25:09.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda013c370 con 0x7feda014dd80 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.256+0000 7feda5c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda014e230 con 0x7feda015eb60 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50902/0 (socket says 172.21.15.73:50902) 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 -- 172.21.15.73:0/567785781 learned_addr learned my addr 172.21.15.73:0/567785781 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 -- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda015eb60 msgr2=0x7feda013be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda015eb60 0x7feda013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 -- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feda014dd80 msgr2=0x7feda0154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feda014dd80 0x7feda0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 -- 172.21.15.73:0/567785781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009b30 con 0x7feda00b1f00 2024-04-05T22:25:09.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.257+0000 7fed9fe00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.258+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9003e450 con 0x7feda00b1f00 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.258+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9003e5f0 con 0x7feda00b1f00 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.258+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9003e9a0 con 0x7feda00b1f00 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.258+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda0157b50 con 0x7feda00b1f00 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.258+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda0150a40 con 0x7feda00b1f00 2024-04-05T22:25:09.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.259+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda01508a0 con 0x7feda00b1f00 2024-04-05T22:25:09.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.260+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fed9003eb00 con 0x7feda00b1f00 2024-04-05T22:25:09.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.261+0000 7fed97e00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 0x7fed8006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.261+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed900ac620 con 0x7feda00b1f00 2024-04-05T22:25:09.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.261+0000 7fed9f400640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 0x7fed8006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.262+0000 7fed9f400640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 0x7fed8006d270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006fd0 tx=0x7fed8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.265+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feda01508a0 con 0x7feda00b1f00 2024-04-05T22:25:09.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.503+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda01508a0 con 0x7feda00b1f00 2024-04-05T22:25:09.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.504+0000 7fed97e00640 1 -- 172.21.15.73:0/567785781 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4734 (secure 0 0 0) 0x7feda01508a0 con 0x7feda00b1f00 2024-04-05T22:25:09.505 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:09.505 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:23:09.333638+0000","modified":"2024-04-05T22:23:12.354447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5634":{"gid":5634,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/166143460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":166143460},{"type":"v1","addr":"172.21.15.157:6835","nonce":166143460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-05T22:25:09.505 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 24 2024-04-05T22:25:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.507+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 msgr2=0x7fed8006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.507+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 0x7fed8006d270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006fd0 tx=0x7fed8c008040 comp rx=0 tx=0).stop 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.507+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 msgr2=0x7feda01574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.507+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed90004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 shutdown_connections 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fed8006adb0 0x7fed8006d270 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feda014dd80 0x7feda0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feda015eb60 0x7feda013be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 --2- 172.21.15.73:0/567785781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda00b1f00 0x7feda01574c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 >> 172.21.15.73:0/567785781 conn(0x7feda00b06c0 msgr2=0x7feda015c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:09.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 shutdown_connections 2024-04-05T22:25:09.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.508+0000 7feda5c00640 1 -- 172.21.15.73:0/567785781 wait complete. 2024-04-05T22:25:09.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:09.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.733+0000 7fee0a800640 1 Processor -- start 2024-04-05T22:25:09.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.733+0000 7fee0a800640 1 -- start start 2024-04-05T22:25:09.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee0a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee0415d200 0x7fee040acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee0a800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 0x7fee040cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee0a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0406aa40 con 0x7fee0415ce20 2024-04-05T22:25:09.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee0a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0409d110 con 0x7fee0415d200 2024-04-05T22:25:09.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee0a800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0406ad20 con 0x7fee040ce630 2024-04-05T22:25:09.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee03400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fee0415ce20 0x7fee040c9b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48218/0 (socket says 172.21.15.73:48218) 2024-04-05T22:25:09.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee03400640 1 -- 172.21.15.73:0/3647691166 learned_addr learned my addr 172.21.15.73:0/3647691166 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:09.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.735+0000 7fee02a00640 1 --2- 172.21.15.73:0/3647691166 >> v2:172.21.15.73:3300/0 conn(0x7fee0415d200 0x7fee040acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02000640 1 -- 172.21.15.73:0/3647691166 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449817534 0 0) 0x7fee0406aa40 con 0x7fee0415ce20 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02000640 1 -- 172.21.15.73:0/3647691166 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede8003520 con 0x7fee0415ce20 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee03e00640 1 --2- 172.21.15.73:0/3647691166 >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 0x7fee040cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02a00640 1 -- 172.21.15.73:0/3647691166 >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 msgr2=0x7fee040cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02a00640 1 --2- 172.21.15.73:0/3647691166 >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 0x7fee040cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02a00640 1 -- 172.21.15.73:0/3647691166 >> v1:172.21.15.73:6789/0 conn(0x7fee0415ce20 legacy=0x7fee040c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee02a00640 1 -- 172.21.15.73:0/3647691166 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040cde30 con 0x7fee0415d200 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.736+0000 7fee03e00640 1 --2- 172.21.15.73:0/3647691166 >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 0x7fee040cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:09.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.737+0000 7fee02a00640 1 --2- 172.21.15.73:0/3647691166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee040acfd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fee04065df0 tx=0x7fedec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a951b194bf7f0d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.737+0000 7fee02000640 1 -- 172.21.15.73:0/3647691166 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec02cc80 con 0x7fee0415d200 2024-04-05T22:25:09.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.737+0000 7fee02000640 1 -- 172.21.15.73:0/3647691166 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec02cde0 con 0x7fee0415d200 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.737+0000 7fee02000640 1 -- 172.21.15.73:0/3647691166 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec034640 con 0x7fee0415d200 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 -- 172.21.15.73:0/3647691166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 msgr2=0x7fee040acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 --2- 172.21.15.73:0/3647691166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee040acfd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fee04065df0 tx=0x7fedec02bed0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 -- 172.21.15.73:0/3647691166 shutdown_connections 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 --2- 172.21.15.73:0/3647691166 >> v2:172.21.15.157:3300/0 conn(0x7fee040ce630 0x7fee040cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 --2- 172.21.15.73:0/3647691166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee040acfd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 -- 172.21.15.73:0/3647691166 >> 172.21.15.73:0/3647691166 conn(0x7fee040cf190 msgr2=0x7fee040cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 -- 172.21.15.73:0/3647691166 shutdown_connections 2024-04-05T22:25:09.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.738+0000 7fee0a800640 1 -- 172.21.15.73:0/3647691166 wait complete. 2024-04-05T22:25:09.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.740+0000 7fee0a800640 1 Processor -- start 2024-04-05T22:25:09.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.740+0000 7fee0a800640 1 -- start start 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee040ce630 0x7fee04099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee0415ce20 0x7fee0409b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0415a1b0 con 0x7fee0415d200 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0409a4b0 con 0x7fee0415ce20 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee0a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0409a790 con 0x7fee040ce630 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50912/0 (socket says 172.21.15.73:50912) 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.741+0000 7fee03e00640 1 -- 172.21.15.73:0/1569778648 learned_addr learned my addr 172.21.15.73:0/1569778648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03400640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee040ce630 0x7fee04099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee02a00640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee0415ce20 0x7fee0409b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 -- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee040ce630 msgr2=0x7fee04099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee040ce630 0x7fee04099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 -- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee0415ce20 msgr2=0x7fee0409b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee0415ce20 0x7fee0409b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 -- 172.21.15.73:0/1569778648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009a70 con 0x7fee0415d200 2024-04-05T22:25:09.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee03e00640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c7f0 tx=0x7fedf400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4007820 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf4007980 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4007d70 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040ca3a0 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.742+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee040b2230 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.744+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee040b1db0 con 0x7fee0415d200 2024-04-05T22:25:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.744+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fedf40113f0 con 0x7fee0415d200 2024-04-05T22:25:09.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.745+0000 7fedfb400640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 0x7fede006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:09.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.745+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fedf4089b30 con 0x7fee0415d200 2024-04-05T22:25:09.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.745+0000 7fee03400640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 0x7fede006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:09.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.746+0000 7fee03400640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 0x7fede006d270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f60 tx=0x7fedfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.750+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fee040b1db0 con 0x7fee0415d200 2024-04-05T22:25:09.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.947+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee040b1db0 con 0x7fee0415d200 2024-04-05T22:25:09.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.948+0000 7fedfb400640 1 -- 172.21.15.73:0/1569778648 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16197 (secure 0 0 0) 0x7fee040b1db0 con 0x7fee0415d200 2024-04-05T22:25:09.949 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:09.949 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:24:03.627819+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:23:07.732446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-05T22:23:08.298551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:09.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 msgr2=0x7fede006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 0x7fede006d270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f60 tx=0x7fedfc009040 comp rx=0 tx=0).stop 2024-04-05T22:25:09.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 msgr2=0x7fee0409b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:09.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c7f0 tx=0x7fedf400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 shutdown_connections 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fede006adb0 0x7fede006d270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0415d200 0x7fee0409b810 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee0415ce20 0x7fee0409b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 --2- 172.21.15.73:0/1569778648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee040ce630 0x7fee04099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 >> 172.21.15.73:0/1569778648 conn(0x7fee040cf190 msgr2=0x7fee040cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:09.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.951+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 shutdown_connections 2024-04-05T22:25:09.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:09.952+0000 7fee0a800640 1 -- 172.21.15.73:0/1569778648 wait complete. 2024-04-05T22:25:09.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:25:10.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.179+0000 7f2e48e00640 1 Processor -- start 2024-04-05T22:25:10.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.179+0000 7f2e48e00640 1 -- start start 2024-04-05T22:25:10.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e48e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e4406d710 0x7f2e440cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e48e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4415aa60 con 0x7f2e441541a0 2024-04-05T22:25:10.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e48e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e440b3b60 con 0x7f2e4415aef0 2024-04-05T22:25:10.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e48e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440701c0 con 0x7f2e4406d710 2024-04-05T22:25:10.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e3be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e4406d710 0x7f2e440cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e41e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2e441541a0 0x7f2e440b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48220/0 (socket says 172.21.15.73:48220) 2024-04-05T22:25:10.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.181+0000 7f2e41e00640 1 -- 172.21.15.73:0/2911465244 learned_addr learned my addr 172.21.15.73:0/2911465244 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:10.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.182+0000 7f2e41400640 1 -- 172.21.15.73:0/2911465244 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900612184 0 0) 0x7f2e4415aa60 con 0x7f2e441541a0 2024-04-05T22:25:10.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.182+0000 7f2e41400640 1 -- 172.21.15.73:0/2911465244 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e28003520 con 0x7f2e441541a0 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.182+0000 7f2e3be00640 1 -- 172.21.15.73:0/2911465244 >> v1:172.21.15.157:6789/0 conn(0x7f2e4415aef0 legacy=0x7f2e4415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.182+0000 7f2e3be00640 1 -- 172.21.15.73:0/2911465244 >> v1:172.21.15.73:6789/0 conn(0x7f2e441541a0 legacy=0x7f2e440b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.182+0000 7f2e3be00640 1 -- 172.21.15.73:0/2911465244 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e440b3880 con 0x7f2e4406d710 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.183+0000 7f2e3be00640 1 --2- 172.21.15.73:0/2911465244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e4406d710 0x7f2e440cd8f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097a0 tx=0x7f2e2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a49bfab369f22a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.183+0000 7f2e41400640 1 -- 172.21.15.73:0/2911465244 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c031830 con 0x7f2e4406d710 2024-04-05T22:25:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.183+0000 7f2e41400640 1 -- 172.21.15.73:0/2911465244 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c031990 con 0x7f2e4406d710 2024-04-05T22:25:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.183+0000 7f2e41400640 1 -- 172.21.15.73:0/2911465244 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c031d20 con 0x7f2e4406d710 2024-04-05T22:25:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 -- 172.21.15.73:0/2911465244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e4406d710 msgr2=0x7f2e440cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 --2- 172.21.15.73:0/2911465244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e4406d710 0x7f2e440cd8f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097a0 tx=0x7f2e2c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 -- 172.21.15.73:0/2911465244 shutdown_connections 2024-04-05T22:25:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 --2- 172.21.15.73:0/2911465244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e4406d710 0x7f2e440cd8f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 -- 172.21.15.73:0/2911465244 >> 172.21.15.73:0/2911465244 conn(0x7f2e440ce8e0 msgr2=0x7f2e440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:10.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 -- 172.21.15.73:0/2911465244 shutdown_connections 2024-04-05T22:25:10.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.184+0000 7f2e48e00640 1 -- 172.21.15.73:0/2911465244 wait complete. 2024-04-05T22:25:10.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.186+0000 7f2e48e00640 1 Processor -- start 2024-04-05T22:25:10.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.186+0000 7f2e48e00640 1 -- start start 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4406d710 0x7f2e441f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4415aef0 0x7f2e440b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440b2d30 con 0x7f2e441541a0 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441f6d50 con 0x7f2e4406d710 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e48e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441503f0 con 0x7f2e4415aef0 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50934/0 (socket says 172.21.15.73:50934) 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.187+0000 7f2e3be00640 1 -- 172.21.15.73:0/296474134 learned_addr learned my addr 172.21.15.73:0/296474134 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e41e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4406d710 0x7f2e441f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 -- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4415aef0 msgr2=0x7f2e440b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4415aef0 0x7f2e440b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 -- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4406d710 msgr2=0x7f2e441f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4406d710 0x7f2e441f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 -- 172.21.15.73:0/296474134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c009400 con 0x7f2e441541a0 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3be00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c03a8f0 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c03aa50 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c03ae00 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e44158180 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.188+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e441f5f90 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.189+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44066af0 con 0x7f2e441541a0 2024-04-05T22:25:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.190+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2e2c04d450 con 0x7f2e441541a0 2024-04-05T22:25:10.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.191+0000 7f2e3a000640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 0x7f2e2006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.191+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2e2c0ab920 con 0x7f2e441541a0 2024-04-05T22:25:10.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.191+0000 7f2e41e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 0x7f2e2006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.192+0000 7f2e41e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 0x7f2e2006d270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2e340011e0 tx=0x7f2e34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.195+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2e2c076690 con 0x7f2e441541a0 2024-04-05T22:25:10.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.421+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2e44068160 con 0x7f2e441541a0 2024-04-05T22:25:10.422 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:25:10.422+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:25:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.436+0000 7f2e3a000640 1 -- 172.21.15.73:0/296474134 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2e2c035070 con 0x7f2e441541a0 2024-04-05T22:25:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 msgr2=0x7f2e2006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 0x7f2e2006d270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2e340011e0 tx=0x7f2e34009040 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 msgr2=0x7f2e4414feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 shutdown_connections 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2e2006adb0 0x7f2e2006d270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4415aef0 0x7f2e440b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e441541a0 0x7f2e4414feb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 --2- 172.21.15.73:0/296474134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4406d710 0x7f2e441f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.438+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 >> 172.21.15.73:0/296474134 conn(0x7f2e440ce8e0 msgr2=0x7f2e4406f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.439+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 shutdown_connections 2024-04-05T22:25:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.439+0000 7f2e48e00640 1 -- 172.21.15.73:0/296474134 wait complete. 2024-04-05T22:25:10.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:25:10.560 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:25:10.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.654+0000 7f72a1400640 1 Processor -- start 2024-04-05T22:25:10.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.654+0000 7f72a1400640 1 -- start start 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.655+0000 7f72a1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f729c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f72a1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0cceb0 con 0x7f729c070660 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f72a1400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0cd130 con 0x7f729c06cd40 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f72a1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0cb8c0 con 0x7f729c143660 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f7291e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f729c070660 0x7f729c06c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48236/0 (socket says 172.21.15.73:48236) 2024-04-05T22:25:10.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f7291e00640 1 -- 172.21.15.73:0/37094783 learned_addr learned my addr 172.21.15.73:0/37094783 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f729a800640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f729c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182701232 0 0) 0x7f729c0cceb0 con 0x7f729c070660 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.656+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7280003520 con 0x7f729c070660 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431713455 0 0) 0x7f7280003520 con 0x7f729c070660 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 msgr2=0x7f729c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f729c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 >> v1:172.21.15.157:6789/0 conn(0x7f729c06cd40 legacy=0x7f729c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c070b90 con 0x7f729c070660 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7284001670 con 0x7f729c070660 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 >> v1:172.21.15.73:6789/0 conn(0x7f729c070660 legacy=0x7f729c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f7280003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f72800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7280004d00 0x7f7280005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0cb8c0 con 0x7f72800039a0 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728000bb60 con 0x7f7280004d00 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728000df30 con 0x7f729c143660 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f7280003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7291e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f72800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 -- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 msgr2=0x7f72800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f72800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729b200640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7280004d00 0x7f7280005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 -- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7280004d00 msgr2=0x7f7280005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:10.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7280004d00 0x7f7280005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f729a800640 1 -- 172.21.15.73:0/37094783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280003520 con 0x7f72800039a0 2024-04-05T22:25:10.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.657+0000 7f7291e00640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c143660 0x7f72800047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:10.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.658+0000 7f729a800640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f7280003dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f728c00ba30 tx=0x7f72800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870c611b5f4738df server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c032640 con 0x7f72800039a0 2024-04-05T22:25:10.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c0327a0 con 0x7f72800039a0 2024-04-05T22:25:10.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f7299e00640 1 -- 172.21.15.73:0/37094783 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c032b30 con 0x7f72800039a0 2024-04-05T22:25:10.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 -- 172.21.15.73:0/37094783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 msgr2=0x7f7280003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 --2- 172.21.15.73:0/37094783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f7280003dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f728c00ba30 tx=0x7f72800057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f729b200640 1 -- 172.21.15.73:0/37094783 reap_dead start 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 -- 172.21.15.73:0/37094783 shutdown_connections 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 -- 172.21.15.73:0/37094783 >> 172.21.15.73:0/37094783 conn(0x7f729c0b0830 msgr2=0x7f729c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 -- 172.21.15.73:0/37094783 shutdown_connections 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.659+0000 7f72a1400640 1 -- 172.21.15.73:0/37094783 wait complete. 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.661+0000 7f72a1400640 1 Processor -- start 2024-04-05T22:25:10.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.661+0000 7f72a1400640 1 -- start start 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 0x7f729c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c15be60 0x7f729c1392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0c83f0 con 0x7f72800039a0 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0c86c0 con 0x7f729c13bee0 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f72a1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0c89a0 con 0x7f729c15be60 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f729a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f729a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50964/0 (socket says 172.21.15.73:50964) 2024-04-05T22:25:10.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.662+0000 7f729a800640 1 -- 172.21.15.73:0/1465535684 learned_addr learned my addr 172.21.15.73:0/1465535684 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 -- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c15be60 msgr2=0x7f729c1392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729b200640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c15be60 0x7f729c1392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f7291e00640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 0x7f729c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c15be60 0x7f729c1392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 -- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 msgr2=0x7f729c0c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 0x7f729c0c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 -- 172.21.15.73:0/1465535684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0c8c80 con 0x7f72800039a0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f7291e00640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 0x7f729c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f729a800640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f728c035040 tx=0x7f728c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c03b860 con 0x7f72800039a0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c03b9c0 con 0x7f72800039a0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c03bd50 con 0x7f72800039a0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.663+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c0c8f60 con 0x7f72800039a0 2024-04-05T22:25:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.664+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c15c2e0 con 0x7f72800039a0 2024-04-05T22:25:10.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.665+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c068160 con 0x7f72800039a0 2024-04-05T22:25:10.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.666+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f728c043070 con 0x7f72800039a0 2024-04-05T22:25:10.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.666+0000 7f7293400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 0x7f727806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:10.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.667+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f728c0aba60 con 0x7f72800039a0 2024-04-05T22:25:10.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.667+0000 7f7291e00640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 0x7f727806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:10.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.667+0000 7f7291e00640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 0x7f727806d2a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7284001200 tx=0x7f7284009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:10.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.672+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f728c0766f0 con 0x7f72800039a0 2024-04-05T22:25:10.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:10.908+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f729c06d620 con 0x7f72800039a0 2024-04-05T22:25:11.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.447+0000 7f7293400640 1 -- 172.21.15.73:0/1465535684 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f728c04a0a0 con 0x7f72800039a0 2024-04-05T22:25:11.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 msgr2=0x7f727806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 0x7f727806d2a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7284001200 tx=0x7f7284009040 comp rx=0 tx=0).stop 2024-04-05T22:25:11.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 msgr2=0x7f729c0c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f728c035040 tx=0x7f728c004730 comp rx=0 tx=0).stop 2024-04-05T22:25:11.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 shutdown_connections 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f727806ade0 0x7f727806d2a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f729c15be60 0x7f729c1392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f729c13bee0 0x7f729c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 --2- 172.21.15.73:0/1465535684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72800039a0 0x7f729c0c9760 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 >> 172.21.15.73:0/1465535684 conn(0x7f729c0b0830 msgr2=0x7f729c140df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 shutdown_connections 2024-04-05T22:25:11.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.451+0000 7f72a1400640 1 -- 172.21.15.73:0/1465535684 wait complete. 2024-04-05T22:25:11.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:25:11.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.678+0000 7f427f600640 1 Processor -- start 2024-04-05T22:25:11.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.678+0000 7f427f600640 1 -- start start 2024-04-05T22:25:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427f600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42780cf3e0 0x7f4278167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427f600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427815e570 con 0x7f427815ac00 2024-04-05T22:25:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427f600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427815afe0 con 0x7f42780b36e0 2024-04-05T22:25:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427f600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4278161760 con 0x7f42780cf3e0 2024-04-05T22:25:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427cc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f427815ac00 0x7f427806cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48248/0 (socket says 172.21.15.73:48248) 2024-04-05T22:25:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427cc00640 1 -- 172.21.15.73:0/3445888002 learned_addr learned my addr 172.21.15.73:0/3445888002 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.680+0000 7f427d600640 1 --2- 172.21.15.73:0/3445888002 >> v2:172.21.15.157:3300/0 conn(0x7f42780cf3e0 0x7f4278167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.681+0000 7f427d600640 1 -- 172.21.15.73:0/3445888002 >> v1:172.21.15.157:6789/0 conn(0x7f42780b36e0 legacy=0x7f42780cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.681+0000 7f427d600640 1 -- 172.21.15.73:0/3445888002 >> v1:172.21.15.73:6789/0 conn(0x7f427815ac00 legacy=0x7f427806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.681+0000 7f427d600640 1 -- 172.21.15.73:0/3445888002 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278160b50 con 0x7f42780cf3e0 2024-04-05T22:25:11.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.683+0000 7f427d600640 1 --2- 172.21.15.73:0/3445888002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780cf3e0 0x7f4278167ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c82ae725d0f1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.687+0000 7f4277e00640 1 -- 172.21.15.73:0/3445888002 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427002cc60 con 0x7f42780cf3e0 2024-04-05T22:25:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.687+0000 7f4277e00640 1 -- 172.21.15.73:0/3445888002 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427002cdc0 con 0x7f42780cf3e0 2024-04-05T22:25:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.687+0000 7f4277e00640 1 -- 172.21.15.73:0/3445888002 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270034660 con 0x7f42780cf3e0 2024-04-05T22:25:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 -- 172.21.15.73:0/3445888002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780cf3e0 msgr2=0x7f4278167ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 --2- 172.21.15.73:0/3445888002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780cf3e0 0x7f4278167ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427002beb0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 -- 172.21.15.73:0/3445888002 shutdown_connections 2024-04-05T22:25:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 --2- 172.21.15.73:0/3445888002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780cf3e0 0x7f4278167ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 -- 172.21.15.73:0/3445888002 >> 172.21.15.73:0/3445888002 conn(0x7f427806a0d0 msgr2=0x7f427806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 -- 172.21.15.73:0/3445888002 shutdown_connections 2024-04-05T22:25:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.688+0000 7f427f600640 1 -- 172.21.15.73:0/3445888002 wait complete. 2024-04-05T22:25:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.690+0000 7f427f600640 1 Processor -- start 2024-04-05T22:25:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.690+0000 7f427f600640 1 -- start start 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 0x7f427815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 0x7f42781536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427815b180 con 0x7f427815ac00 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427815b460 con 0x7f42780b36e0 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427815b740 con 0x7f42780cf3e0 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50966/0 (socket says 172.21.15.73:50966) 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427d600640 1 -- 172.21.15.73:0/2097636291 learned_addr learned my addr 172.21.15.73:0/2097636291 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f427cc00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 0x7f427815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.691+0000 7f426fe00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 0x7f42781536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 -- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 msgr2=0x7f42781536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 0x7f42781536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 -- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 msgr2=0x7f427815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 0x7f427815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 -- 172.21.15.73:0/2097636291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009a70 con 0x7f427815ac00 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427cc00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 0x7f427815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f426fe00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 0x7f42781536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427d600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f4270004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270040650 con 0x7f427815ac00 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700407d0 con 0x7f427815ac00 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270040b80 con 0x7f427815ac00 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427815ba20 con 0x7f427815ac00 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.692+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427815c760 con 0x7f427815ac00 2024-04-05T22:25:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.693+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427815bee0 con 0x7f427815ac00 2024-04-05T22:25:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.694+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f4270040ce0 con 0x7f427815ac00 2024-04-05T22:25:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.695+0000 7f4275600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 0x7f425406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.695+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f42700ac6d0 con 0x7f427815ac00 2024-04-05T22:25:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.695+0000 7f427cc00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 0x7f425406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.696+0000 7f427cc00640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 0x7f425406d270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4268005880 tx=0x7f4268009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.699+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4270077500 con 0x7f427815ac00 2024-04-05T22:25:11.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:11.896+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4278068160 con 0x7f427815ac00 2024-04-05T22:25:12.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.698+0000 7f4275600640 1 -- 172.21.15.73:0/2097636291 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f427007ab20 con 0x7f427815ac00 2024-04-05T22:25:12.699 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:25:12.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 msgr2=0x7f425406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 0x7f425406d270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4268005880 tx=0x7f4268009040 comp rx=0 tx=0).stop 2024-04-05T22:25:12.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 msgr2=0x7f42780b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f4270004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 shutdown_connections 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f425406adb0 0x7f425406d270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427815ac00 0x7f42780b05f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42780cf3e0 0x7f42781536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 --2- 172.21.15.73:0/2097636291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780b36e0 0x7f427815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.704+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 >> 172.21.15.73:0/2097636291 conn(0x7f427806a0d0 msgr2=0x7f427815ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.705+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 shutdown_connections 2024-04-05T22:25:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.705+0000 7f427f600640 1 -- 172.21.15.73:0/2097636291 wait complete. 2024-04-05T22:25:12.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:25:12.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.919+0000 7f0b6f000640 1 Processor -- start 2024-04-05T22:25:12.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.920+0000 7f0b6f000640 1 -- start start 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.921+0000 7f0b6f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b6815ac00 0x7f0b6806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.921+0000 7f0b6f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 0x7f0b6815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.921+0000 7f0b6f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6815afe0 con 0x7f0b680b36e0 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.921+0000 7f0b6f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b68161760 con 0x7f0b6815ac00 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.921+0000 7f0b6f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6806d4c0 con 0x7f0b680cf3e0 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b6815ac00 0x7f0b6806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b680b36e0 0x7f0b680cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48262/0 (socket says 172.21.15.73:48262) 2024-04-05T22:25:12.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67400640 1 -- 172.21.15.73:0/702002248 learned_addr learned my addr 172.21.15.73:0/702002248 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:12.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67e00640 1 -- 172.21.15.73:0/702002248 >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 msgr2=0x7f0b6815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b6d000640 1 --2- 172.21.15.73:0/702002248 >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 0x7f0b6815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67e00640 1 --2- 172.21.15.73:0/702002248 >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 0x7f0b6815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67e00640 1 -- 172.21.15.73:0/702002248 >> v1:172.21.15.73:6789/0 conn(0x7f0b680b36e0 legacy=0x7f0b680cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b67e00640 1 -- 172.21.15.73:0/702002248 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68160b50 con 0x7f0b6815ac00 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b66a00640 1 -- 172.21.15.73:0/702002248 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309904981 0 0) 0x7f0b50001490 con 0x7f0b680b36e0 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.922+0000 7f0b6d000640 1 --2- 172.21.15.73:0/702002248 >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 0x7f0b6815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.923+0000 7f0b67e00640 1 --2- 172.21.15.73:0/702002248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b6815ac00 0x7f0b6806cf80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806fb80 tx=0x7f0b5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bca8de30f2a878d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b66a00640 1 -- 172.21.15.73:0/702002248 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802e070 con 0x7f0b6815ac00 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b66a00640 1 -- 172.21.15.73:0/702002248 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b580315a0 con 0x7f0b6815ac00 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b66a00640 1 -- 172.21.15.73:0/702002248 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58031930 con 0x7f0b6815ac00 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 -- 172.21.15.73:0/702002248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b6815ac00 msgr2=0x7f0b6806cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 --2- 172.21.15.73:0/702002248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b6815ac00 0x7f0b6806cf80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806fb80 tx=0x7f0b5802b840 comp rx=0 tx=0).stop 2024-04-05T22:25:12.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 -- 172.21.15.73:0/702002248 shutdown_connections 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 --2- 172.21.15.73:0/702002248 >> v2:172.21.15.157:3300/0 conn(0x7f0b680cf3e0 0x7f0b6815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 --2- 172.21.15.73:0/702002248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b6815ac00 0x7f0b6806cf80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 -- 172.21.15.73:0/702002248 >> 172.21.15.73:0/702002248 conn(0x7f0b6806a0d0 msgr2=0x7f0b6806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 -- 172.21.15.73:0/702002248 shutdown_connections 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.924+0000 7f0b6f000640 1 -- 172.21.15.73:0/702002248 wait complete. 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.926+0000 7f0b6f000640 1 Processor -- start 2024-04-05T22:25:12.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.926+0000 7f0b6f000640 1 -- start start 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680b36e0 0x7f0b68153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6815ac00 0x7f0b6815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68160040 con 0x7f0b680cf3e0 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6815efa0 con 0x7f0b680b36e0 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b6f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6815f280 con 0x7f0b6815ac00 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50982/0 (socket says 172.21.15.73:50982) 2024-04-05T22:25:12.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.927+0000 7f0b67400640 1 -- 172.21.15.73:0/2107174789 learned_addr learned my addr 172.21.15.73:0/2107174789 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67e00640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680b36e0 0x7f0b68153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 -- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6815ac00 msgr2=0x7f0b6815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6815ac00 0x7f0b6815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 -- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680b36e0 msgr2=0x7f0b68153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680b36e0 0x7f0b68153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 -- 172.21.15.73:0/2107174789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58009400 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b67400640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c890 tx=0x7f0b5000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000a5e0 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5000a740 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6815f560 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.928+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000aad0 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.929+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6815fb20 con 0x7f0b680cf3e0 2024-04-05T22:25:12.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.930+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6815fd40 con 0x7f0b680cf3e0 2024-04-05T22:25:12.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.931+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f0b5000ac30 con 0x7f0b680cf3e0 2024-04-05T22:25:12.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.931+0000 7f0b5fe00640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 0x7f0b4406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:12.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.932+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f0b50088d00 con 0x7f0b680cf3e0 2024-04-05T22:25:12.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.932+0000 7f0b67e00640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 0x7f0b4406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:12.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.932+0000 7f0b67e00640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 0x7f0b4406d270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b58036000 tx=0x7f0b58008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:12.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:12.936+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0b50053ce0 con 0x7f0b680cf3e0 2024-04-05T22:25:13.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.128+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b6815afe0 con 0x7f0b680cf3e0 2024-04-05T22:25:13.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.704+0000 7f0b5fe00640 1 -- 172.21.15.73:0/2107174789 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b6815afe0 con 0x7f0b680cf3e0 2024-04-05T22:25:13.706 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:25:13.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.710+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 msgr2=0x7f0b4406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.710+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 0x7f0b4406d270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b58036000 tx=0x7f0b58008030 comp rx=0 tx=0).stop 2024-04-05T22:25:13.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.710+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 msgr2=0x7f0b6815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.710+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c890 tx=0x7f0b5000cd50 comp rx=0 tx=0).stop 2024-04-05T22:25:13.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 shutdown_connections 2024-04-05T22:25:13.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0b4406adb0 0x7f0b4406d270 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6815ac00 0x7f0b6815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b680cf3e0 0x7f0b6815b6b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 --2- 172.21.15.73:0/2107174789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680b36e0 0x7f0b68153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 >> 172.21.15.73:0/2107174789 conn(0x7f0b6806a0d0 msgr2=0x7f0b6806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 shutdown_connections 2024-04-05T22:25:13.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.712+0000 7f0b6f000640 1 -- 172.21.15.73:0/2107174789 wait complete. 2024-04-05T22:25:13.737 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:25:13.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.923+0000 7f47ecc00640 1 Processor -- start 2024-04-05T22:25:13.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.923+0000 7f47ecc00640 1 -- start start 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.924+0000 7f47ecc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47e80cbe00 0x7f47e80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47ecc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e80b1b30 con 0x7f47e80b32b0 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47ecc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e80b1e10 con 0x7f47e806c630 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47ecc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80cc300 con 0x7f47e80cbe00 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47e6800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f47e80b32b0 0x7f47e81647c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48278/0 (socket says 172.21.15.73:48278) 2024-04-05T22:25:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47e6800640 1 -- 172.21.15.73:0/3817710662 learned_addr learned my addr 172.21.15.73:0/3817710662 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224326007 0 0) 0x7f47e80b1b30 con 0x7f47e80b32b0 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47cc003520 con 0x7f47e80b32b0 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.925+0000 7f47dd400640 1 --2- 172.21.15.73:0/3817710662 >> v2:172.21.15.157:3300/0 conn(0x7f47e80cbe00 0x7f47e80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968537584 0 0) 0x7f47cc003520 con 0x7f47e80b32b0 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 >> v2:172.21.15.157:3300/0 conn(0x7f47e80cbe00 msgr2=0x7f47e80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 --2- 172.21.15.73:0/3817710662 >> v2:172.21.15.157:3300/0 conn(0x7f47e80cbe00 0x7f47e80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 >> v1:172.21.15.157:6789/0 conn(0x7f47e806c630 legacy=0x7f47e806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e80cc5e0 con 0x7f47e80b32b0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f47e00019f0 con 0x7f47e80b32b0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 >> v1:172.21.15.73:6789/0 conn(0x7f47e80b32b0 legacy=0x7f47e81647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 0x7f47cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc0047c0 0x7f47cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc005180 0x7f47cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80cc300 con 0x7f47cc003970 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00c0d0 con 0x7f47cc0047c0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00c290 con 0x7f47cc005180 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 0x7f47cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.926+0000 7f47dd400640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc0047c0 0x7f47cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e6800640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc005180 0x7f47cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e5e00640 1 -- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc005180 msgr2=0x7f47cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:13.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc005180 0x7f47cc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e5e00640 1 -- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc0047c0 msgr2=0x7f47cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc0047c0 0x7f47cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e5e00640 1 -- 172.21.15.73:0/3817710662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc003520 con 0x7f47cc003970 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e6800640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc005180 0x7f47cc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.927+0000 7f47e6800640 1 -- 172.21.15.73:0/3817710662 reap_dead start 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.928+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 0x7f47cc003d90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001750 tx=0x7f47cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4a32ba91697c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.928+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80052d0 con 0x7f47cc003970 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.928+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d802e050 con 0x7f47cc003970 2024-04-05T22:25:13.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.928+0000 7f47e5400640 1 -- 172.21.15.73:0/3817710662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8003830 con 0x7f47cc003970 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 -- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 msgr2=0x7f47cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 0x7f47cc003d90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001750 tx=0x7f47cc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 -- 172.21.15.73:0/3817710662 shutdown_connections 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3817710662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47cc003970 0x7f47cc003d90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 -- 172.21.15.73:0/3817710662 >> 172.21.15.73:0/3817710662 conn(0x7f47e80b0830 msgr2=0x7f47e80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 -- 172.21.15.73:0/3817710662 shutdown_connections 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.929+0000 7f47ecc00640 1 -- 172.21.15.73:0/3817710662 wait complete. 2024-04-05T22:25:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.931+0000 7f47ecc00640 1 Processor -- start 2024-04-05T22:25:13.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.931+0000 7f47ecc00640 1 -- start start 2024-04-05T22:25:13.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc003970 0x7f47e815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e81f5d60 0x7f47e81f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80b04f0 con 0x7f47e80afb30 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e815ab60 con 0x7f47e81f5d60 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47ecc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e81f6ad0 con 0x7f47cc003970 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50998/0 (socket says 172.21.15.73:50998) 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 -- 172.21.15.73:0/3309503444 learned_addr learned my addr 172.21.15.73:0/3309503444 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc003970 0x7f47e815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47e6800640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e81f5d60 0x7f47e81f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 -- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc003970 msgr2=0x7f47e815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc003970 0x7f47e815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 -- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e81f5d60 msgr2=0x7f47e81f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e81f5d60 0x7f47e81f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:13.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.932+0000 7f47dd400640 1 -- 172.21.15.73:0/3309503444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e81f2730 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47dd400640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f47d000f600 tx=0x7f47d000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0014870 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d00149d0 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0014d40 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e81f2a10 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.933+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e81f2fd0 con 0x7f47e80afb30 2024-04-05T22:25:13.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.934+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e81f3160 con 0x7f47e80afb30 2024-04-05T22:25:13.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.935+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f47d000d490 con 0x7f47e80afb30 2024-04-05T22:25:13.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.936+0000 7f47dea00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 0x7f47c406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:13.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.936+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f47d0089dc0 con 0x7f47e80afb30 2024-04-05T22:25:13.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.936+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 0x7f47c406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:13.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.937+0000 7f47e5e00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 0x7f47c406d270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f47d8033000 tx=0x7f47d8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:13.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:13.940+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f47d0054f80 con 0x7f47e80afb30 2024-04-05T22:25:14.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.150+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f47e80ccf80 con 0x7f47e80afb30 2024-04-05T22:25:14.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.156+0000 7f47dea00640 1 -- 172.21.15.73:0/3309503444 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f47d00585a0 con 0x7f47e80afb30 2024-04-05T22:25:14.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 msgr2=0x7f47c406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 0x7f47c406d270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f47d8033000 tx=0x7f47d8009030 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 msgr2=0x7f47e80affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f47d000f600 tx=0x7f47d000fac0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 shutdown_connections 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f47c406adb0 0x7f47c406d270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e81f5d60 0x7f47e81f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47e80afb30 0x7f47e80affb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 --2- 172.21.15.73:0/3309503444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47cc003970 0x7f47e815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 >> 172.21.15.73:0/3309503444 conn(0x7f47e80b0830 msgr2=0x7f47e80b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 shutdown_connections 2024-04-05T22:25:14.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.159+0000 7f47ecc00640 1 -- 172.21.15.73:0/3309503444 wait complete. 2024-04-05T22:25:14.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-05T22:25:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.374+0000 7f517fe00640 1 Processor -- start 2024-04-05T22:25:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.374+0000 7f517fe00640 1 -- start start 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518006fe90 0x7f518015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5180143330 con 0x7f5180142ba0 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517fe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5180143050 con 0x7f518013fa60 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517fe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518006d3a0 con 0x7f518006fe90 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517e000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5180142ba0 0x7f518006f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48292/0 (socket says 172.21.15.73:48292) 2024-04-05T22:25:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517e000640 1 -- 172.21.15.73:0/1731662595 learned_addr learned my addr 172.21.15.73:0/1731662595 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970689015 0 0) 0x7f5180143330 con 0x7f5180142ba0 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5164003520 con 0x7f5180142ba0 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.376+0000 7f517f400640 1 --2- 172.21.15.73:0/1731662595 >> v2:172.21.15.157:3300/0 conn(0x7f518006fe90 0x7f518015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006378531 0 0) 0x7f5164003520 con 0x7f5180142ba0 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 >> v2:172.21.15.157:3300/0 conn(0x7f518006fe90 msgr2=0x7f518015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 --2- 172.21.15.73:0/1731662595 >> v2:172.21.15.157:3300/0 conn(0x7f518006fe90 0x7f518015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 >> v1:172.21.15.157:6789/0 conn(0x7f518013fa60 legacy=0x7f5180142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800ac970 con 0x7f5180142ba0 2024-04-05T22:25:14.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f516c001670 con 0x7f5180142ba0 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 >> v1:172.21.15.73:6789/0 conn(0x7f5180142ba0 legacy=0x7f518006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164003970 0x7f5164003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 0x7f5164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164005180 0x7f516400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518006d3a0 con 0x7f51640047c0 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516400c0d0 con 0x7f5164005180 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516400c290 con 0x7f5164003970 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.377+0000 7f517e000640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 0x7f5164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517ea00640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164003970 0x7f5164003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517f400640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164005180 0x7f516400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517e000640 1 -- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164003970 msgr2=0x7f5164003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517e000640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164003970 0x7f5164003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517e000640 1 -- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164005180 msgr2=0x7f516400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517e000640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164005180 0x7f516400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517e000640 1 -- 172.21.15.73:0/1731662595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164003520 con 0x7f51640047c0 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517ea00640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164003970 0x7f5164003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517f400640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164005180 0x7f516400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.378+0000 7f517f400640 1 -- 172.21.15.73:0/1731662595 reap_dead start 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517e000640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 0x7f5164004c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c240 tx=0x7f51640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a9650785ab6b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c005ad0 con 0x7f51640047c0 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c005c30 con 0x7f51640047c0 2024-04-05T22:25:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517d600640 1 -- 172.21.15.73:0/1731662595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02caf0 con 0x7f51640047c0 2024-04-05T22:25:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517fe00640 1 -- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 msgr2=0x7f5164004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.379+0000 7f517fe00640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 0x7f5164004c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c240 tx=0x7f51640057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.380+0000 7f517fe00640 1 -- 172.21.15.73:0/1731662595 shutdown_connections 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.380+0000 7f517fe00640 1 --2- 172.21.15.73:0/1731662595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51640047c0 0x7f5164004c40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.380+0000 7f517fe00640 1 -- 172.21.15.73:0/1731662595 >> 172.21.15.73:0/1731662595 conn(0x7f51800b06c0 msgr2=0x7f51800b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.380+0000 7f517fe00640 1 -- 172.21.15.73:0/1731662595 shutdown_connections 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.380+0000 7f517fe00640 1 -- 172.21.15.73:0/1731662595 wait complete. 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.381+0000 7f517fe00640 1 Processor -- start 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.381+0000 7f517fe00640 1 -- start start 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640047c0 0x7f518015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015f120 0x7f51801f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800cb640 con 0x7f518015df80 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518015e400 con 0x7f518015f120 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.382+0000 7f517fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015e6e0 con 0x7f51640047c0 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51016/0 (socket says 172.21.15.73:51016) 2024-04-05T22:25:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 -- 172.21.15.73:0/1538465426 learned_addr learned my addr 172.21.15.73:0/1538465426 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517ea00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640047c0 0x7f518015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517f400640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015f120 0x7f51801f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 -- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640047c0 msgr2=0x7f518015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640047c0 0x7f518015da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 -- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015f120 msgr2=0x7f51801f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015f120 0x7f51801f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 -- 172.21.15.73:0/1538465426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518015e9c0 con 0x7f518015df80 2024-04-05T22:25:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.383+0000 7f517e000640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f516c001930 tx=0x7f516c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.384+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c03c040 con 0x7f518015df80 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.384+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c02cd80 con 0x7f518015df80 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.384+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c0072b0 con 0x7f518015df80 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.384+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51800cb2d0 con 0x7f518015df80 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.384+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801f2ef0 con 0x7f518015df80 2024-04-05T22:25:14.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.385+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51800cb7b0 con 0x7f518015df80 2024-04-05T22:25:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.386+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f516c032070 con 0x7f518015df80 2024-04-05T22:25:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.387+0000 7f5176a00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 0x7f516006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.387+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f516c0aa960 con 0x7f518015df80 2024-04-05T22:25:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.387+0000 7f517ea00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 0x7f516006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.388+0000 7f517ea00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 0x7f516006d270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5180063c30 tx=0x7f5170009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.391+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f516c075bb0 con 0x7f518015df80 2024-04-05T22:25:14.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.657+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f518004e160 con 0x7f518015df80 2024-04-05T22:25:14.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.719+0000 7f5176a00640 1 -- 172.21.15.73:0/1538465426 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v209) v1 ==== 140+0+0 (secure 0 0 0) 0x7f516c0791d0 con 0x7f518015df80 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.722+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 msgr2=0x7f516006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.722+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 0x7f516006d270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5180063c30 tx=0x7f5170009040 comp rx=0 tx=0).stop 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.722+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 msgr2=0x7f51801f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.722+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f516c001930 tx=0x7f516c02e070 comp rx=0 tx=0).stop 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 shutdown_connections 2024-04-05T22:25:14.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f516006adb0 0x7f516006d270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015f120 0x7f51801f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f518015df80 0x7f51801f2290 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 --2- 172.21.15.73:0/1538465426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640047c0 0x7f518015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 >> 172.21.15.73:0/1538465426 conn(0x7f51800b06c0 msgr2=0x7f518006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 shutdown_connections 2024-04-05T22:25:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.723+0000 7f517fe00640 1 -- 172.21.15.73:0/1538465426 wait complete. 2024-04-05T22:25:14.743 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-05T22:25:14.743 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-04-05T22:25:14.743 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:25:14.743 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:14.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.953+0000 7f89db200640 1 Processor -- start 2024-04-05T22:25:14.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.953+0000 7f89db200640 1 -- start start 2024-04-05T22:25:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.955+0000 7f89db200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89d415cf50 0x7f89d406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.955+0000 7f89db200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.955+0000 7f89db200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d40cb950 con 0x7f89d40cd090 2024-04-05T22:25:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.955+0000 7f89db200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d40ce990 con 0x7f89d415cf50 2024-04-05T22:25:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.955+0000 7f89db200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40d1350 con 0x7f89d40d09b0 2024-04-05T22:25:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.956+0000 7f89cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.956+0000 7f89cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:53654/0 (socket says 172.21.15.73:53654) 2024-04-05T22:25:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.956+0000 7f89cb400640 1 -- 172.21.15.73:0/3601682506 learned_addr learned my addr 172.21.15.73:0/3601682506 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.956+0000 7f89d3e00640 1 --2- 172.21.15.73:0/3601682506 >> v2:172.21.15.157:3300/0 conn(0x7f89d415cf50 0x7f89d406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89d3400640 1 -- 172.21.15.73:0/3601682506 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385886742 0 0) 0x7f89d40cb950 con 0x7f89d40cd090 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89d3400640 1 -- 172.21.15.73:0/3601682506 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89b8003520 con 0x7f89d40cd090 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89cb400640 1 -- 172.21.15.73:0/3601682506 >> v2:172.21.15.157:3300/0 conn(0x7f89d415cf50 msgr2=0x7f89d406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89cb400640 1 --2- 172.21.15.73:0/3601682506 >> v2:172.21.15.157:3300/0 conn(0x7f89d415cf50 0x7f89d406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89cb400640 1 -- 172.21.15.73:0/3601682506 >> v1:172.21.15.157:6789/0 conn(0x7f89d40cd090 legacy=0x7f89d40cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.957+0000 7f89cb400640 1 -- 172.21.15.73:0/3601682506 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40cdce0 con 0x7f89d40d09b0 2024-04-05T22:25:14.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.958+0000 7f89cb400640 1 --2- 172.21.15.73:0/3601682506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009870 tx=0x7f89bc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb6beb70e6c1e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.958+0000 7f89d3400640 1 -- 172.21.15.73:0/3601682506 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc030840 con 0x7f89d40d09b0 2024-04-05T22:25:14.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89d3400640 1 -- 172.21.15.73:0/3601682506 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc0309a0 con 0x7f89d40d09b0 2024-04-05T22:25:14.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89d3400640 1 -- 172.21.15.73:0/3601682506 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc030d30 con 0x7f89d40d09b0 2024-04-05T22:25:14.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 -- 172.21.15.73:0/3601682506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 msgr2=0x7f89d40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 --2- 172.21.15.73:0/3601682506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009870 tx=0x7f89bc02b980 comp rx=0 tx=0).stop 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 -- 172.21.15.73:0/3601682506 shutdown_connections 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 --2- 172.21.15.73:0/3601682506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d40d09b0 0x7f89d40d0e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 --2- 172.21.15.73:0/3601682506 >> v2:172.21.15.157:3300/0 conn(0x7f89d415cf50 0x7f89d406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.959+0000 7f89db200640 1 -- 172.21.15.73:0/3601682506 >> 172.21.15.73:0/3601682506 conn(0x7f89d4099a60 msgr2=0x7f89d4099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.960+0000 7f89db200640 1 -- 172.21.15.73:0/3601682506 shutdown_connections 2024-04-05T22:25:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.960+0000 7f89db200640 1 -- 172.21.15.73:0/3601682506 wait complete. 2024-04-05T22:25:14.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.962+0000 7f89db200640 1 Processor -- start 2024-04-05T22:25:14.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.962+0000 7f89db200640 1 -- start start 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 0x7f89d41f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d415cf50 0x7f89d40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d414b9a0 con 0x7f89d40d09b0 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40b3180 con 0x7f89d40cd090 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89db200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4151ed0 con 0x7f89d415cf50 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.963+0000 7f89cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51026/0 (socket says 172.21.15.73:51026) 2024-04-05T22:25:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 -- 172.21.15.73:0/265275937 learned_addr learned my addr 172.21.15.73:0/265275937 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89d9200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d415cf50 0x7f89d40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89d3e00640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 0x7f89d41f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 -- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d415cf50 msgr2=0x7f89d40b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d415cf50 0x7f89d40b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 -- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 msgr2=0x7f89d41f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 0x7f89d41f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 -- 172.21.15.73:0/265275937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0094d0 con 0x7f89d40d09b0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89d3e00640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 0x7f89d41f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.964+0000 7f89cb400640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.965+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc047070 con 0x7f89d40d09b0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.965+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc039a90 con 0x7f89d40d09b0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.965+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc039e40 con 0x7f89d40d09b0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.965+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d41521b0 con 0x7f89d40d09b0 2024-04-05T22:25:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.965+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d4152770 con 0x7f89d40d09b0 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.966+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4068160 con 0x7f89d40d09b0 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.967+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f89bc04c410 con 0x7f89d40d09b0 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.968+0000 7f89d0c00640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 0x7f89b006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.968+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f89bc03e070 con 0x7f89d40d09b0 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.968+0000 7f89d3e00640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 0x7f89b006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.969+0000 7f89d3e00640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 0x7f89b006d2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89d4151420 tx=0x7f89c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:14.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:14.972+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f89bc076ce0 con 0x7f89d40d09b0 2024-04-05T22:25:15.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.166+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89d406cdb0 con 0x7f89d40d09b0 2024-04-05T22:25:15.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.169+0000 7f89d0c00640 1 -- 172.21.15.73:0/265275937 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12518 (secure 0 0 0) 0x7f89bc07a300 con 0x7f89d40d09b0 2024-04-05T22:25:15.169 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:15.169 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":53,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:13.694349+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/166143460":"2024-04-06T22:25:10.422903+0000","172.21.15.157:6833/166143460":"2024-04-06T22:25:10.422903+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:15.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 msgr2=0x7f89b006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 0x7f89b006d2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89d4151420 tx=0x7f89c4008040 comp rx=0 tx=0).stop 2024-04-05T22:25:15.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 msgr2=0x7f89d40b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc004730 comp rx=0 tx=0).stop 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 shutdown_connections 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f89b006ade0 0x7f89b006d2a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89d415cf50 0x7f89d40b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89d40d09b0 0x7f89d40b20b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 --2- 172.21.15.73:0/265275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d40cd090 0x7f89d41f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.171+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 >> 172.21.15.73:0/265275937 conn(0x7f89d4099a60 msgr2=0x7f89d4099e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.172+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 shutdown_connections 2024-04-05T22:25:15.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.172+0000 7f89db200640 1 -- 172.21.15.73:0/265275937 wait complete. 2024-04-05T22:25:15.191 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:15.191 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.402+0000 7fb3fd400640 1 Processor -- start 2024-04-05T22:25:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.402+0000 7fb3fd400640 1 -- start start 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 0x7fb3f806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3fd400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f806bec0 con 0x7fb3f806ba70 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3fd400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f815afc0 con 0x7fb3f815abe0 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3fd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f806f5b0 con 0x7fb3f806c9a0 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3f7200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb3f806ba70 0x7fb3f8164740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48300/0 (socket says 172.21.15.73:48300) 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3f7200640 1 -- 172.21.15.73:0/3354952531 learned_addr learned my addr 172.21.15.73:0/3354952531 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3f5e00640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 0x7fb3f806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162596198 0 0) 0x7fb3f0001490 con 0x7fb3f806ba70 2024-04-05T22:25:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.404+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3dc003520 con 0x7fb3f806ba70 2024-04-05T22:25:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1778419913 0 0) 0x7fb3dc003520 con 0x7fb3f806ba70 2024-04-05T22:25:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 msgr2=0x7fb3f806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 0x7fb3f806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 >> v1:172.21.15.157:6789/0 conn(0x7fb3f815abe0 legacy=0x7fb3f80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f806f890 con 0x7fb3f806ba70 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb3f0001b80 con 0x7fb3f806ba70 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 >> v1:172.21.15.73:6789/0 conn(0x7fb3f806ba70 legacy=0x7fb3f8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc0047c0 0x7fb3dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 0x7fb3dc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f806f5b0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc00bb70 con 0x7fb3dc0047c0 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc00bd30 con 0x7fb3f806c9a0 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.405+0000 7fb3f6800640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.406+0000 7fb3f6800640 1 -- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 msgr2=0x7fb3dc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.406+0000 7fb3f6800640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f806c9a0 0x7fb3dc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.406+0000 7fb3f6800640 1 -- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc0047c0 msgr2=0x7fb3dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.406+0000 7fb3f6800640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc0047c0 0x7fb3dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.406+0000 7fb3f6800640 1 -- 172.21.15.73:0/3354952531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc003520 con 0x7fb3dc0039a0 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.407+0000 7fb3f6800640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3dc003dc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001750 tx=0x7fb3dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73eaf1eebbf3b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.407+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80052d0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.407+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e802e050 con 0x7fb3dc0039a0 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.407+0000 7fb3f5400640 1 -- 172.21.15.73:0/3354952531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80035f0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 -- 172.21.15.73:0/3354952531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 msgr2=0x7fb3dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 --2- 172.21.15.73:0/3354952531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3dc003dc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001750 tx=0x7fb3dc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3f7200640 1 -- 172.21.15.73:0/3354952531 reap_dead start 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 -- 172.21.15.73:0/3354952531 shutdown_connections 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 -- 172.21.15.73:0/3354952531 >> 172.21.15.73:0/3354952531 conn(0x7fb3f8069cb0 msgr2=0x7fb3f806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 -- 172.21.15.73:0/3354952531 shutdown_connections 2024-04-05T22:25:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.408+0000 7fb3fd400640 1 -- 172.21.15.73:0/3354952531 wait complete. 2024-04-05T22:25:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.409+0000 7fb3fd400640 1 Processor -- start 2024-04-05T22:25:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.410+0000 7fb3fd400640 1 -- start start 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.410+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.410+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f80cdf30 0x7fb3f80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f814daa0 0x7fb3f814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3fd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81f38d0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3fd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f80ce8f0 con 0x7fb3f80cdf30 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3fd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81f3390 con 0x7fb3f814daa0 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51038/0 (socket says 172.21.15.73:51038) 2024-04-05T22:25:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 -- 172.21.15.73:0/1847833137 learned_addr learned my addr 172.21.15.73:0/1847833137 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:15.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 -- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f814daa0 msgr2=0x7fb3f814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f814daa0 0x7fb3f814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 -- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f80cdf30 msgr2=0x7fb3f80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f80cdf30 0x7fb3f80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 -- 172.21.15.73:0/1847833137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f81f2a60 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.411+0000 7fb3f6800640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8008fd0 tx=0x7fb3e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.412+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80076c0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.412+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8007a30 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.412+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f81f2d40 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.412+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8006be0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.412+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81f2720 con 0x7fb3dc0039a0 2024-04-05T22:25:15.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.413+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f80700f0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.414+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fb3e8006d40 con 0x7fb3dc0039a0 2024-04-05T22:25:15.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.415+0000 7fb3eea00640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 0x7fb3d806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.415+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3e80ab0b0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.415+0000 7fb3f5e00640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 0x7fb3d806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.416+0000 7fb3f5e00640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 0x7fb3d806d2a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006e80 tx=0x7fb3e4006dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.421+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb3e8076290 con 0x7fb3dc0039a0 2024-04-05T22:25:15.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.613+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3f8068160 con 0x7fb3dc0039a0 2024-04-05T22:25:15.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.613+0000 7fb3eea00640 1 -- 172.21.15.73:0/1847833137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3e80798b0 con 0x7fb3dc0039a0 2024-04-05T22:25:15.614 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 msgr2=0x7fb3d806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 0x7fb3d806d2a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006e80 tx=0x7fb3e4006dd0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 msgr2=0x7fb3f81ec7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8008fd0 tx=0x7fb3e8001120 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 shutdown_connections 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb3d806ade0 0x7fb3d806d2a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f814daa0 0x7fb3f814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f80cdf30 0x7fb3f80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 --2- 172.21.15.73:0/1847833137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc0039a0 0x7fb3f81ec7a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.616+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 >> 172.21.15.73:0/1847833137 conn(0x7fb3f8069cb0 msgr2=0x7fb3f80aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:15.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.617+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 shutdown_connections 2024-04-05T22:25:15.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.617+0000 7fb3fd400640 1 -- 172.21.15.73:0/1847833137 wait complete. 2024-04-05T22:25:15.635 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:25:15.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.846+0000 7fa91f400640 1 Processor -- start 2024-04-05T22:25:15.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.846+0000 7fa91f400640 1 -- start start 2024-04-05T22:25:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.847+0000 7fa91f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa91806d710 0x7fa9180cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.847+0000 7fa91f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa91815aef0 0x7fa9180cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.847+0000 7fa91f400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91815aa60 con 0x7fa9181541a0 2024-04-05T22:25:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa91f400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180b3b60 con 0x7fa91806d710 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa91f400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180701c0 con 0x7fa91815aef0 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa91806d710 0x7fa9180cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa91806d710 0x7fa9180cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43780/0 (socket says 172.21.15.73:43780) 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 -- 172.21.15.73:0/46231848 learned_addr learned my addr 172.21.15.73:0/46231848 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 -- 172.21.15.73:0/46231848 >> v2:172.21.15.157:3300/0 conn(0x7fa91815aef0 msgr2=0x7fa9180cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 --2- 172.21.15.73:0/46231848 >> v2:172.21.15.157:3300/0 conn(0x7fa91815aef0 0x7fa9180cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 -- 172.21.15.73:0/46231848 >> v1:172.21.15.157:6789/0 conn(0x7fa9181541a0 legacy=0x7fa9180b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.848+0000 7fa90fe00640 1 -- 172.21.15.73:0/46231848 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180b3880 con 0x7fa91806d710 2024-04-05T22:25:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.849+0000 7fa90fe00640 1 --2- 172.21.15.73:0/46231848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa9180cd8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa91815a9d0 tx=0x7fa90002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d83386bedcf5c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa917e00640 1 -- 172.21.15.73:0/46231848 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90002f8e0 con 0x7fa91806d710 2024-04-05T22:25:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa917e00640 1 -- 172.21.15.73:0/46231848 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90002fa40 con 0x7fa91806d710 2024-04-05T22:25:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa917e00640 1 -- 172.21.15.73:0/46231848 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90002fdd0 con 0x7fa91806d710 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 -- 172.21.15.73:0/46231848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 msgr2=0x7fa9180cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 --2- 172.21.15.73:0/46231848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa9180cd8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa91815a9d0 tx=0x7fa90002db20 comp rx=0 tx=0).stop 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 -- 172.21.15.73:0/46231848 shutdown_connections 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 --2- 172.21.15.73:0/46231848 >> v2:172.21.15.157:3300/0 conn(0x7fa91815aef0 0x7fa9180cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 --2- 172.21.15.73:0/46231848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa9180cd8f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 -- 172.21.15.73:0/46231848 >> 172.21.15.73:0/46231848 conn(0x7fa9180ce8e0 msgr2=0x7fa9180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 -- 172.21.15.73:0/46231848 shutdown_connections 2024-04-05T22:25:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.850+0000 7fa91f400640 1 -- 172.21.15.73:0/46231848 wait complete. 2024-04-05T22:25:15.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.852+0000 7fa91f400640 1 Processor -- start 2024-04-05T22:25:15.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.852+0000 7fa91f400640 1 -- start start 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181541a0 0x7fa918154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa91815aef0 0x7fa91806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181422c0 con 0x7fa91806d710 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918155240 con 0x7fa9181541a0 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa918155830 con 0x7fa91815aef0 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.853+0000 7fa91ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43784/0 (socket says 172.21.15.73:43784) 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 -- 172.21.15.73:0/1247350017 learned_addr learned my addr 172.21.15.73:0/1247350017 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 -- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa91815aef0 msgr2=0x7fa91806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa91815aef0 0x7fa91806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 -- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181541a0 msgr2=0x7fa918154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:15.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181541a0 0x7fa918154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 -- 172.21.15.73:0/1247350017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000094d0 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa91ca00640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c410 tx=0x7fa90800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080035f0 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908003750 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.854+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908003b40 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.855+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918155b10 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.855+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9181560d0 con 0x7fa91806d710 2024-04-05T22:25:15.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.856+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa918066af0 con 0x7fa91806d710 2024-04-05T22:25:15.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.857+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fa908003ca0 con 0x7fa91806d710 2024-04-05T22:25:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.857+0000 7fa915600640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 0x7fa8f406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.858+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9080886f0 con 0x7fa91806d710 2024-04-05T22:25:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.858+0000 7fa90fe00640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 0x7fa8f406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.858+0000 7fa90fe00640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 0x7fa8f406d270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa900038000 tx=0x7fa900032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:15.862+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9080538f0 con 0x7fa91806d710 2024-04-05T22:25:16.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.125+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fa918068160 con 0x7fa91806d710 2024-04-05T22:25:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.732+0000 7fa915600640 1 -- 172.21.15.73:0/1247350017 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v211) v1 ==== 164+0+0 (secure 0 0 0) 0x7fa908056f10 con 0x7fa91806d710 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 msgr2=0x7fa8f406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 0x7fa8f406d270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa900038000 tx=0x7fa900032000 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 msgr2=0x7fa918141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c410 tx=0x7fa90800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 shutdown_connections 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa8f406adb0 0x7fa8f406d270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa91815aef0 0x7fa91806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9181541a0 0x7fa918154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 --2- 172.21.15.73:0/1247350017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa91806d710 0x7fa918141d80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.735+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 >> 172.21.15.73:0/1247350017 conn(0x7fa9180ce8e0 msgr2=0x7fa9180b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.736+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 shutdown_connections 2024-04-05T22:25:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.736+0000 7fa91f400640 1 -- 172.21.15.73:0/1247350017 wait complete. 2024-04-05T22:25:16.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:25:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.972+0000 7f82af600640 1 Processor -- start 2024-04-05T22:25:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.972+0000 7f82af600640 1 -- start start 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82af600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f82a8070a40 0x7f82a80add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82af600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a8070e20 con 0x7f82a81599f0 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82af600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a80b27f0 con 0x7f82a806cd50 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82af600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a80ae240 con 0x7f82a8070a40 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82acc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f82a81599f0 0x7f82a806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44674/0 (socket says 172.21.15.73:44674) 2024-04-05T22:25:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82acc00640 1 -- 172.21.15.73:0/2380955253 learned_addr learned my addr 172.21.15.73:0/2380955253 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.974+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110626322 0 0) 0x7f82a8070e20 con 0x7f82a81599f0 2024-04-05T22:25:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8290005730 con 0x7f82a81599f0 2024-04-05T22:25:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82ad600640 1 --2- 172.21.15.73:0/2380955253 >> v2:172.21.15.157:3300/0 conn(0x7f82a8070a40 0x7f82a80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1382672418 0 0) 0x7f8290005730 con 0x7f82a81599f0 2024-04-05T22:25:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 >> v2:172.21.15.157:3300/0 conn(0x7f82a8070a40 msgr2=0x7f82a80add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 --2- 172.21.15.73:0/2380955253 >> v2:172.21.15.157:3300/0 conn(0x7f82a8070a40 0x7f82a80add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 >> v1:172.21.15.157:6789/0 conn(0x7f82a806cd50 legacy=0x7f82a8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80b2ad0 con 0x7f82a81599f0 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f829c001670 con 0x7f82a81599f0 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 >> v1:172.21.15.73:6789/0 conn(0x7f82a81599f0 legacy=0x7f82a806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 0x7f8290005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82900069d0 0x7f8290006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8290007390 0x7f829000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80ae240 con 0x7f8290005b80 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829000e2e0 con 0x7f8290007390 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829000e4a0 con 0x7f82900069d0 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.975+0000 7f82acc00640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 0x7f8290005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82acc00640 1 -- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82900069d0 msgr2=0x7f8290006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82acc00640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82900069d0 0x7f8290006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82acc00640 1 -- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8290007390 msgr2=0x7f829000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82acc00640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8290007390 0x7f829000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82acc00640 1 -- 172.21.15.73:0/2380955253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290005730 con 0x7f8290005b80 2024-04-05T22:25:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.976+0000 7f82ad600640 1 -- 172.21.15.73:0/2380955253 reap_dead start 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.977+0000 7f82acc00640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 0x7f8290005fa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c190 tx=0x7f8290007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff235f42da225c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.977+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c005ad0 con 0x7f8290005b80 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.977+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c005c30 con 0x7f8290005b80 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.977+0000 7f82a7400640 1 -- 172.21.15.73:0/2380955253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c02cae0 con 0x7f8290005b80 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 -- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 msgr2=0x7f8290005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 0x7f8290005fa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c190 tx=0x7f8290007a00 comp rx=0 tx=0).stop 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 -- 172.21.15.73:0/2380955253 shutdown_connections 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 --2- 172.21.15.73:0/2380955253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8290005b80 0x7f8290005fa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 -- 172.21.15.73:0/2380955253 >> 172.21.15.73:0/2380955253 conn(0x7f82a8069cb0 msgr2=0x7f82a806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 -- 172.21.15.73:0/2380955253 shutdown_connections 2024-04-05T22:25:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.978+0000 7f82af600640 1 -- 172.21.15.73:0/2380955253 wait complete. 2024-04-05T22:25:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.980+0000 7f82af600640 1 Processor -- start 2024-04-05T22:25:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.980+0000 7f82af600640 1 -- start start 2024-04-05T22:25:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8290005b80 0x7f82a814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a814ead0 0x7f82a80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a814d960 con 0x7f82a814ef30 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a814bf10 con 0x7f82a814ead0 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82af600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a814c1f0 con 0x7f8290005b80 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43804/0 (socket says 172.21.15.73:43804) 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 -- 172.21.15.73:0/743456716 learned_addr learned my addr 172.21.15.73:0/743456716 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 -- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8290005b80 msgr2=0x7f82a814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8290005b80 0x7f82a814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 -- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a814ead0 msgr2=0x7f82a80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a814ead0 0x7f82a80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.981+0000 7f82ad600640 1 -- 172.21.15.73:0/743456716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a814c4d0 con 0x7f82a814ef30 2024-04-05T22:25:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82ad600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f82a80710a0 tx=0x7f82a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000ed90 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0014420 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a814c7b0 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00147d0 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.982+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a814cd70 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.983+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a8068160 con 0x7f82a814ef30 2024-04-05T22:25:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.984+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f82a0014930 con 0x7f82a814ef30 2024-04-05T22:25:16.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.985+0000 7f82a4c00640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 0x7f828806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:16.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.985+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82a0089940 con 0x7f82a814ef30 2024-04-05T22:25:16.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.985+0000 7f82acc00640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 0x7f828806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:16.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.986+0000 7f82acc00640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 0x7f828806d2a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:16.989+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f82a0054a70 con 0x7f82a814ef30 2024-04-05T22:25:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.179+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f82a81568d0 con 0x7f82a814ef30 2024-04-05T22:25:17.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.185+0000 7f82a4c00640 1 -- 172.21.15.73:0/743456716 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82a0058090 con 0x7f82a814ef30 2024-04-05T22:25:17.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.187+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 msgr2=0x7f828806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:17.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 0x7f828806d2a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c039040 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 msgr2=0x7f82a80ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f82a80710a0 tx=0x7f82a000e4b0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 shutdown_connections 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f828806ade0 0x7f828806d2a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82a814ef30 0x7f82a80ace10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a814ead0 0x7f82a80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 --2- 172.21.15.73:0/743456716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8290005b80 0x7f82a814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 >> 172.21.15.73:0/743456716 conn(0x7f82a8069cb0 msgr2=0x7f82a81561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 shutdown_connections 2024-04-05T22:25:17.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.188+0000 7f82af600640 1 -- 172.21.15.73:0/743456716 wait complete. 2024-04-05T22:25:17.207 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:25:17.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:17.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:17.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:25:17.435 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:17.435 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:17.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:25:17.494 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:17.494 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:17.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:25:17.552 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:17.552 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:17.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:25:17.611 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:17.611 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:17.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:25:17.667 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:25:17.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:17.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.884+0000 7f8ab1400640 1 Processor -- start 2024-04-05T22:25:17.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.884+0000 7f8ab1400640 1 -- start start 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.885+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.885+0000 7f8ab1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aac06d3a0 0x7f8aac070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.885+0000 7f8ab1400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac15a830 con 0x7f8aac071340 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8ab1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0b2280 con 0x7f8aac06d3a0 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8ab1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b3a20 con 0x7f8aac159440 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aac06d3a0 0x7f8aac070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aac06d3a0 0x7f8aac070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43806/0 (socket says 172.21.15.73:43806) 2024-04-05T22:25:17.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 -- 172.21.15.73:0/3683935233 learned_addr learned my addr 172.21.15.73:0/3683935233 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:17.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 -- 172.21.15.73:0/3683935233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 msgr2=0x7f8aac0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:17.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 --2- 172.21.15.73:0/3683935233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 -- 172.21.15.73:0/3683935233 >> v1:172.21.15.157:6789/0 conn(0x7f8aac071340 legacy=0x7f8aac0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:17.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.886+0000 7f8aa1e00640 1 -- 172.21.15.73:0/3683935233 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0b2520 con 0x7f8aac06d3a0 2024-04-05T22:25:17.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.887+0000 7f8aa1e00640 1 --2- 172.21.15.73:0/3683935233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac070e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400aa10 tx=0x7f8a9402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7401c4f6f04f13a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:17.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8aa9e00640 1 -- 172.21.15.73:0/3683935233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402db80 con 0x7f8aac06d3a0 2024-04-05T22:25:17.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8aa9e00640 1 -- 172.21.15.73:0/3683935233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9402dce0 con 0x7f8aac06d3a0 2024-04-05T22:25:17.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8aa9e00640 1 -- 172.21.15.73:0/3683935233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94035660 con 0x7f8aac06d3a0 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 -- 172.21.15.73:0/3683935233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 msgr2=0x7f8aac070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 --2- 172.21.15.73:0/3683935233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac070e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400aa10 tx=0x7f8a9402cdd0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 -- 172.21.15.73:0/3683935233 shutdown_connections 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 --2- 172.21.15.73:0/3683935233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac070e00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 --2- 172.21.15.73:0/3683935233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 -- 172.21.15.73:0/3683935233 >> 172.21.15.73:0/3683935233 conn(0x7f8aac0ce8e0 msgr2=0x7f8aac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 -- 172.21.15.73:0/3683935233 shutdown_connections 2024-04-05T22:25:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.888+0000 7f8ab1400640 1 -- 172.21.15.73:0/3683935233 wait complete. 2024-04-05T22:25:17.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.890+0000 7f8ab1400640 1 Processor -- start 2024-04-05T22:25:17.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.890+0000 7f8ab1400640 1 -- start start 2024-04-05T22:25:17.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac071340 0x7f8aac1463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac146910 con 0x7f8aac06d3a0 2024-04-05T22:25:17.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac15af10 con 0x7f8aac071340 2024-04-05T22:25:17.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.891+0000 7f8ab1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac15b1b0 con 0x7f8aac159440 2024-04-05T22:25:17.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:17.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43810/0 (socket says 172.21.15.73:43810) 2024-04-05T22:25:17.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 -- 172.21.15.73:0/1260389610 learned_addr learned my addr 172.21.15.73:0/1260389610 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aab200640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aa1e00640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac071340 0x7f8aac1463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 -- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 msgr2=0x7f8aac15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 -- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac071340 msgr2=0x7f8aac1463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac071340 0x7f8aac1463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 -- 172.21.15.73:0/1260389610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9400a670 con 0x7f8aac06d3a0 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.892+0000 7f8aaa800640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b410 tx=0x7f8a9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.893+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c011830 con 0x7f8aac06d3a0 2024-04-05T22:25:17.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.893+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c0119b0 con 0x7f8aac06d3a0 2024-04-05T22:25:17.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.893+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c011d60 con 0x7f8aac06d3a0 2024-04-05T22:25:17.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.893+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac15b490 con 0x7f8aac06d3a0 2024-04-05T22:25:17.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.893+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac15ba50 con 0x7f8aac06d3a0 2024-04-05T22:25:17.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.894+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac066af0 con 0x7f8aac06d3a0 2024-04-05T22:25:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.895+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f8a9c01a400 con 0x7f8aac06d3a0 2024-04-05T22:25:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.896+0000 7f8aa3400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 0x7f8a8806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.896+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a9c089530 con 0x7f8aac06d3a0 2024-04-05T22:25:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.896+0000 7f8aa1e00640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 0x7f8a8806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.897+0000 7f8aa1e00640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 0x7f8a8806d2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8a94037000 tx=0x7f8a94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:17.900+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8a9c0546f0 con 0x7f8aac06d3a0 2024-04-05T22:25:18.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.136+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aac068160 con 0x7f8aac06d3a0 2024-04-05T22:25:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.137+0000 7f8aa3400640 1 -- 172.21.15.73:0/1260389610 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8a9c015090 con 0x7f8aac06d3a0 2024-04-05T22:25:18.138 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:18.138 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3900723030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3900723030},{"type":"v1","addr":"172.21.15.73:6837","nonce":3900723030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4772,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2050745096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2050745096},{"type":"v1","addr":"172.21.15.157:6835","nonce":2050745096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2149615180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2149615180},{"type":"v1","addr":"172.21.15.73:6836","nonce":2149615180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/758260240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":758260240},{"type":"v1","addr":"172.21.15.157:6834","nonce":758260240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-04-05T22:25:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 msgr2=0x7f8a8806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 0x7f8a8806d2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8a94037000 tx=0x7f8a94008040 comp rx=0 tx=0).stop 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 msgr2=0x7f8aac06bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b410 tx=0x7f8a9c00b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 shutdown_connections 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8a8806ade0 0x7f8a8806d2a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aac159440 0x7f8aac15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac071340 0x7f8aac1463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 --2- 172.21.15.73:0/1260389610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aac06d3a0 0x7f8aac06bdb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 >> 172.21.15.73:0/1260389610 conn(0x7f8aac0ce8e0 msgr2=0x7f8aac06c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 shutdown_connections 2024-04-05T22:25:18.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.140+0000 7f8ab1400640 1 -- 172.21.15.73:0/1260389610 wait complete. 2024-04-05T22:25:18.159 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:25:18.160 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:25:18.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:25:18.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:25:18.160 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:25:18.160 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:25:18.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:25:18.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:25:18.161 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:25:18.161 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:25:18.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:25:18.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:25:18.161 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:25:18.161 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:25:18.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:25:18.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:25:18.182 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:25:18.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:25:18.183 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:25:18.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:25:18.186 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:25:18.186 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:25:18.190 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:25:18.190 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:25:18.234 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:25:18.234 DEBUG:teuthology.parallel:result is None 2024-04-05T22:25:18.236 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:25:18.237 DEBUG:teuthology.parallel:result is None 2024-04-05T22:25:18.237 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:25:18.237 DEBUG:teuthology.parallel:result is None 2024-04-05T22:25:18.243 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:25:18.243 DEBUG:teuthology.parallel:result is None 2024-04-05T22:25:18.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:25:18.383 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:25:18.384 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:25:18.399 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:25:18.399 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:25:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.438+0000 7fa54e000640 1 Processor -- start 2024-04-05T22:25:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.439+0000 7fa54e000640 1 -- start start 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480cfb60 0x7fa5480d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5480b3c40 0x7fa5480cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54e000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480cef00 con 0x7fa54815ce50 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54e000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480d0d60 con 0x7fa5480b3c40 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b2150 con 0x7fa5480cfb60 2024-04-05T22:25:18.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5480b3c40 0x7fa5480cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5480b3c40 0x7fa5480cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43856/0 (socket says 172.21.15.73:43856) 2024-04-05T22:25:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 -- 172.21.15.73:0/3523537307 learned_addr learned my addr 172.21.15.73:0/3523537307 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 -- 172.21.15.73:0/3523537307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480cfb60 msgr2=0x7fa5480d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 --2- 172.21.15.73:0/3523537307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480cfb60 0x7fa5480d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 -- 172.21.15.73:0/3523537307 >> v1:172.21.15.157:6789/0 conn(0x7fa54815ce50 legacy=0x7fa5480cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.440+0000 7fa54d600640 1 -- 172.21.15.73:0/3523537307 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480b2430 con 0x7fa5480b3c40 2024-04-05T22:25:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.441+0000 7fa54d600640 1 --2- 172.21.15.73:0/3523537307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5480b3c40 0x7fa5480cd830 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa540009e50 tx=0x7fa54002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8fcadd2c617175 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa547400640 1 -- 172.21.15.73:0/3523537307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54002c9e0 con 0x7fa5480b3c40 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa547400640 1 -- 172.21.15.73:0/3523537307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54002cb40 con 0x7fa5480b3c40 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa547400640 1 -- 172.21.15.73:0/3523537307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5400343f0 con 0x7fa5480b3c40 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 -- 172.21.15.73:0/3523537307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5480b3c40 msgr2=0x7fa5480cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 --2- 172.21.15.73:0/3523537307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5480b3c40 0x7fa5480cd830 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa540009e50 tx=0x7fa54002be40 comp rx=0 tx=0).stop 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 -- 172.21.15.73:0/3523537307 shutdown_connections 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 --2- 172.21.15.73:0/3523537307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5480b3c40 0x7fa5480cd830 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 --2- 172.21.15.73:0/3523537307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480cfb60 0x7fa5480d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.442+0000 7fa54e000640 1 -- 172.21.15.73:0/3523537307 >> 172.21.15.73:0/3523537307 conn(0x7fa5480d2a40 msgr2=0x7fa5480d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.443+0000 7fa54e000640 1 -- 172.21.15.73:0/3523537307 shutdown_connections 2024-04-05T22:25:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.443+0000 7fa54e000640 1 -- 172.21.15.73:0/3523537307 wait complete. 2024-04-05T22:25:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.444+0000 7fa54e000640 1 Processor -- start 2024-04-05T22:25:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.444+0000 7fa54e000640 1 -- start start 2024-04-05T22:25:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480b3c40 0x7fa54815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5480cfb60 0x7fa54815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815b3f0 con 0x7fa54815ce50 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5481f2aa0 con 0x7fa5480cfb60 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815d5d0 con 0x7fa5480b3c40 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.445+0000 7fa54cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480b3c40 0x7fa54815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43868/0 (socket says 172.21.15.73:43868) 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 -- 172.21.15.73:0/69298348 learned_addr learned my addr 172.21.15.73:0/69298348 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:18.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa547e00640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5480cfb60 0x7fa54815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 -- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480b3c40 msgr2=0x7fa54815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480b3c40 0x7fa54815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 -- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5480cfb60 msgr2=0x7fa54815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5480cfb60 0x7fa54815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 -- 172.21.15.73:0/69298348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540009b30 con 0x7fa54815ce50 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.446+0000 7fa54d600640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa540036040 tx=0x7fa540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.447+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540009510 con 0x7fa54815ce50 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.447+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54003e450 con 0x7fa54815ce50 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.447+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54003e7e0 con 0x7fa54815ce50 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.447+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54815d8b0 con 0x7fa54815ce50 2024-04-05T22:25:18.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.447+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54815b780 con 0x7fa54815ce50 2024-04-05T22:25:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.448+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548067250 con 0x7fa54815ce50 2024-04-05T22:25:18.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.449+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fa54003e940 con 0x7fa54815ce50 2024-04-05T22:25:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.450+0000 7fa544c00640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 0x7fa52806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.450+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5400ac100 con 0x7fa54815ce50 2024-04-05T22:25:18.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.450+0000 7fa54cc00640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 0x7fa52806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:18.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.451+0000 7fa54cc00640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 0x7fa52806d270 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006fd0 tx=0x7fa53c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:18.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.454+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa5400772c0 con 0x7fa54815ce50 2024-04-05T22:25:18.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:18.658+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa54815dd70 con 0x7fa54815ce50 2024-04-05T22:25:19.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.434+0000 7fa544c00640 1 -- 172.21.15.73:0/69298348 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa54007a8e0 con 0x7fa54815ce50 2024-04-05T22:25:19.435 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 msgr2=0x7fa52806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 0x7fa52806d270 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006fd0 tx=0x7fa53c009060 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 msgr2=0x7fa5481f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa540036040 tx=0x7fa540004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 shutdown_connections 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa52806adb0 0x7fa52806d270 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa54815ce50 0x7fa5481f1ee0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5480cfb60 0x7fa54815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.438+0000 7fa54e000640 1 --2- 172.21.15.73:0/69298348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5480b3c40 0x7fa54815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.439+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 >> 172.21.15.73:0/69298348 conn(0x7fa5480d2a40 msgr2=0x7fa5480b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.439+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 shutdown_connections 2024-04-05T22:25:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.439+0000 7fa54e000640 1 -- 172.21.15.73:0/69298348 wait complete. 2024-04-05T22:25:19.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.663+0000 7f666fc00640 1 Processor -- start 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.664+0000 7f666fc00640 1 -- start start 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66680cb2f0 0x7f66680cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680cbc70 con 0x7f6668160620 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666fc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680ad880 con 0x7f66680b1870 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680ce5d0 con 0x7f66680cb2f0 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666d200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6668160620 0x7f66680caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44694/0 (socket says 172.21.15.73:44694) 2024-04-05T22:25:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666d200640 1 -- 172.21.15.73:0/494102197 learned_addr learned my addr 172.21.15.73:0/494102197 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.665+0000 7f666dc00640 1 --2- 172.21.15.73:0/494102197 >> v2:172.21.15.73:3300/0 conn(0x7f66680cb2f0 0x7f66680cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.666+0000 7f6667400640 1 -- 172.21.15.73:0/494102197 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616360621 0 0) 0x7f66680cbc70 con 0x7f6668160620 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.666+0000 7f6667400640 1 -- 172.21.15.73:0/494102197 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6650003520 con 0x7f6668160620 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.666+0000 7f666dc00640 1 -- 172.21.15.73:0/494102197 >> v1:172.21.15.157:6789/0 conn(0x7f66680b1870 legacy=0x7f66680ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.666+0000 7f666dc00640 1 -- 172.21.15.73:0/494102197 >> v1:172.21.15.73:6789/0 conn(0x7f6668160620 legacy=0x7f66680caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.666+0000 7f666dc00640 1 -- 172.21.15.73:0/494102197 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680ce010 con 0x7f66680cb2f0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.667+0000 7f666dc00640 1 --2- 172.21.15.73:0/494102197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66680cb2f0 0x7f66680cb730 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f666002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5cd389bd1899ea server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f6667400640 1 -- 172.21.15.73:0/494102197 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666002cc60 con 0x7f66680cb2f0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f6667400640 1 -- 172.21.15.73:0/494102197 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666002cdc0 con 0x7f66680cb2f0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f6667400640 1 -- 172.21.15.73:0/494102197 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660034660 con 0x7f66680cb2f0 2024-04-05T22:25:19.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f666fc00640 1 -- 172.21.15.73:0/494102197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66680cb2f0 msgr2=0x7f66680cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f666fc00640 1 --2- 172.21.15.73:0/494102197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66680cb2f0 0x7f66680cb730 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f666002beb0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f666fc00640 1 -- 172.21.15.73:0/494102197 shutdown_connections 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f666fc00640 1 --2- 172.21.15.73:0/494102197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66680cb2f0 0x7f66680cb730 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.668+0000 7f666fc00640 1 -- 172.21.15.73:0/494102197 >> 172.21.15.73:0/494102197 conn(0x7f66680ce8e0 msgr2=0x7f66680cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.669+0000 7f666fc00640 1 -- 172.21.15.73:0/494102197 shutdown_connections 2024-04-05T22:25:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.669+0000 7f666fc00640 1 -- 172.21.15.73:0/494102197 wait complete. 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.670+0000 7f666fc00640 1 Processor -- start 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.670+0000 7f666fc00640 1 -- start start 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66680b1870 0x7f6668140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66680cb2f0 0x7f6668143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681561c0 con 0x7f6668160620 2024-04-05T22:25:19.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680afba0 con 0x7f66680cb2f0 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.671+0000 7f666fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66681575b0 con 0x7f66680b1870 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43876/0 (socket says 172.21.15.73:43876) 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 -- 172.21.15.73:0/2588588727 learned_addr learned my addr 172.21.15.73:0/2588588727 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f6667e00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66680cb2f0 0x7f6668143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666d200640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66680b1870 0x7f6668140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 -- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66680b1870 msgr2=0x7f6668140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66680b1870 0x7f6668140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 -- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66680cb2f0 msgr2=0x7f6668143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66680cb2f0 0x7f6668143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.672+0000 7f666dc00640 1 -- 172.21.15.73:0/2588588727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660009a70 con 0x7f6668160620 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f666dc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f6660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660040650 con 0x7f6668160620 2024-04-05T22:25:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66600407b0 con 0x7f6668160620 2024-04-05T22:25:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660040b60 con 0x7f6668160620 2024-04-05T22:25:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66681f26d0 con 0x7f6668160620 2024-04-05T22:25:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.673+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680b0370 con 0x7f6668160620 2024-04-05T22:25:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.674+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6668066af0 con 0x7f6668160620 2024-04-05T22:25:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.676+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f6660040cc0 con 0x7f6668160620 2024-04-05T22:25:19.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.677+0000 7f6664c00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 0x7f664806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:19.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.677+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66600abf70 con 0x7f6668160620 2024-04-05T22:25:19.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.677+0000 7f666d200640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 0x7f664806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:19.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.678+0000 7f666d200640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 0x7f664806d270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f665c0011e0 tx=0x7f665c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:19.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.682+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f66600df650 con 0x7f6668160620 2024-04-05T22:25:19.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.943+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f66681f26d0 con 0x7f6668160620 2024-04-05T22:25:19.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.944+0000 7f6664c00640 1 -- 172.21.15.73:0/2588588727 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f66681f26d0 con 0x7f6668160620 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.945 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:25:19.946 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:25:19.947 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.949 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:25:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.953 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:25:19.955 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:25:19.956 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:25:19.957 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:25:19.958 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:25:19.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:19.960 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:25:19.960 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.946+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 msgr2=0x7f664806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.946+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 0x7f664806d270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f665c0011e0 tx=0x7f665c009040 comp rx=0 tx=0).stop 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.946+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 msgr2=0x7f666815cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.946+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6660009d90 tx=0x7f6660004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 shutdown_connections 2024-04-05T22:25:19.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f664806adb0 0x7f664806d270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668160620 0x7f666815cf00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66680cb2f0 0x7f6668143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 --2- 172.21.15.73:0/2588588727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66680b1870 0x7f6668140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 >> 172.21.15.73:0/2588588727 conn(0x7f66680ce8e0 msgr2=0x7f66680b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 shutdown_connections 2024-04-05T22:25:19.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:19.947+0000 7f666fc00640 1 -- 172.21.15.73:0/2588588727 wait complete. 2024-04-05T22:25:19.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.159+0000 7fd533000640 1 Processor -- start 2024-04-05T22:25:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.159+0000 7fd533000640 1 -- start start 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd533000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd52c15ef60 0x7fd52c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd533000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c06f5b0 con 0x7fd52c06cb80 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd533000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c070410 con 0x7fd52c1587d0 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd533000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c06d500 con 0x7fd52c15ef60 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd523400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd52c06cb80 0x7fd52c15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44706/0 (socket says 172.21.15.73:44706) 2024-04-05T22:25:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd523400640 1 -- 172.21.15.73:0/2677888322 learned_addr learned my addr 172.21.15.73:0/2677888322 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.161+0000 7fd531000640 1 --2- 172.21.15.73:0/2677888322 >> v2:172.21.15.157:3300/0 conn(0x7fd52c15ef60 0x7fd52c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345096567 0 0) 0x7fd52c06f5b0 con 0x7fd52c06cb80 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd510003520 con 0x7fd52c06cb80 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704877075 0 0) 0x7fd510003520 con 0x7fd52c06cb80 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 >> v2:172.21.15.157:3300/0 conn(0x7fd52c15ef60 msgr2=0x7fd52c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 --2- 172.21.15.73:0/2677888322 >> v2:172.21.15.157:3300/0 conn(0x7fd52c15ef60 0x7fd52c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 >> v1:172.21.15.157:6789/0 conn(0x7fd52c1587d0 legacy=0x7fd52c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0ccef0 con 0x7fd52c06cb80 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fd5140019f0 con 0x7fd52c06cb80 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 >> v1:172.21.15.73:6789/0 conn(0x7fd52c06cb80 legacy=0x7fd52c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 0x7fd510003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c06d500 con 0x7fd5100039c0 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000c080 con 0x7fd5100047c0 2024-04-05T22:25:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000c240 con 0x7fd510005180 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.162+0000 7fd52be00640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 0x7fd510003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd523400640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd531000640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd52be00640 1 -- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd510005180 msgr2=0x7fd51000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd52be00640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd52be00640 1 -- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100047c0 msgr2=0x7fd510004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd52be00640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd52be00640 1 -- 172.21.15.73:0/2677888322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510003520 con 0x7fd5100039c0 2024-04-05T22:25:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.163+0000 7fd531000640 1 -- 172.21.15.73:0/2677888322 reap_dead start 2024-04-05T22:25:20.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.164+0000 7fd52be00640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 0x7fd510003de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001750 tx=0x7fd5100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcca34ae0b6b1323 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.164+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0052d0 con 0x7fd5100039c0 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.164+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c02e050 con 0x7fd5100039c0 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.164+0000 7fd52b400640 1 -- 172.21.15.73:0/2677888322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c003830 con 0x7fd5100039c0 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 -- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 msgr2=0x7fd510003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 0x7fd510003de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001750 tx=0x7fd5100057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 -- 172.21.15.73:0/2677888322 shutdown_connections 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 --2- 172.21.15.73:0/2677888322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5100039c0 0x7fd510003de0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 -- 172.21.15.73:0/2677888322 >> 172.21.15.73:0/2677888322 conn(0x7fd52c0ce8e0 msgr2=0x7fd52c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 -- 172.21.15.73:0/2677888322 shutdown_connections 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.165+0000 7fd533000640 1 -- 172.21.15.73:0/2677888322 wait complete. 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.167+0000 7fd533000640 1 Processor -- start 2024-04-05T22:25:20.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.167+0000 7fd533000640 1 -- start start 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100039c0 0x7fd52c0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c0b2b90 0x7fd52c15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c142350 con 0x7fd52c0b2730 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1431b0 con 0x7fd5100039c0 2024-04-05T22:25:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd533000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1f6e10 con 0x7fd52c0b2b90 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43892/0 (socket says 172.21.15.73:43892) 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 -- 172.21.15.73:0/1462176078 learned_addr learned my addr 172.21.15.73:0/1462176078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd52be00640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100039c0 0x7fd52c0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd531000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c0b2b90 0x7fd52c15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 -- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c0b2b90 msgr2=0x7fd52c15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c0b2b90 0x7fd52c15d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 -- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100039c0 msgr2=0x7fd52c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100039c0 0x7fd52c0c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.168+0000 7fd523400640 1 -- 172.21.15.73:0/1462176078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c1425a0 con 0x7fd52c0b2730 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd523400640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd514007a10 tx=0x7fd51400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140019d0 con 0x7fd52c0b2730 2024-04-05T22:25:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400ee50 con 0x7fd52c0b2730 2024-04-05T22:25:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514013620 con 0x7fd52c0b2730 2024-04-05T22:25:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c142880 con 0x7fd52c0b2730 2024-04-05T22:25:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.169+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c142e40 con 0x7fd52c0b2730 2024-04-05T22:25:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.170+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c066af0 con 0x7fd52c0b2730 2024-04-05T22:25:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.171+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fd514013780 con 0x7fd52c0b2730 2024-04-05T22:25:20.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.172+0000 7fd528c00640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 0x7fd50806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.172+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5140898d0 con 0x7fd52c0b2730 2024-04-05T22:25:20.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.172+0000 7fd52be00640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 0x7fd50806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.173+0000 7fd52be00640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 0x7fd50806d2a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd51c033000 tx=0x7fd51c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.177+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd514054c10 con 0x7fd52c0b2730 2024-04-05T22:25:20.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.368+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd52c142fd0 con 0x7fd52c0b2730 2024-04-05T22:25:20.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.370+0000 7fd528c00640 1 -- 172.21.15.73:0/1462176078 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12389 (secure 0 0 0) 0x7fd514081e60 con 0x7fd52c0b2730 2024-04-05T22:25:20.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:20.370 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:19.417405+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:20.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 msgr2=0x7fd50806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 0x7fd50806d2a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd51c033000 tx=0x7fd51c009030 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 msgr2=0x7fd52c15d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd514007a10 tx=0x7fd51400e530 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 shutdown_connections 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd50806ade0 0x7fd50806d2a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd52c0b2b90 0x7fd52c15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0b2730 0x7fd52c15d3e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 --2- 172.21.15.73:0/1462176078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100039c0 0x7fd52c0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 >> 172.21.15.73:0/1462176078 conn(0x7fd52c0ce8e0 msgr2=0x7fd52c0cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:20.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 shutdown_connections 2024-04-05T22:25:20.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.373+0000 7fd533000640 1 -- 172.21.15.73:0/1462176078 wait complete. 2024-04-05T22:25:20.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:20.392 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:20.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.606+0000 7f71d4400640 1 Processor -- start 2024-04-05T22:25:20.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.606+0000 7f71d4400640 1 -- start start 2024-04-05T22:25:20.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.607+0000 7f71d4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc1599f0 0x7f71cc070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.607+0000 7f71d4400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71cc0add00 0x7f71cc06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.607+0000 7f71d4400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71cc06cb80 0x7f71cc15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.607+0000 7f71d4400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc06d010 con 0x7f71cc0add00 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d4400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc0b3890 con 0x7f71cc06cb80 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0b27f0 con 0x7f71cc1599f0 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71cc0add00 0x7f71cc06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71cc0add00 0x7f71cc06c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43908/0 (socket says 172.21.15.73:43908) 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 -- 172.21.15.73:0/3174793668 learned_addr learned my addr 172.21.15.73:0/3174793668 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1a00640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc1599f0 0x7f71cc070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d2400640 1 --2- 172.21.15.73:0/3174793668 >> v2:172.21.15.157:3300/0 conn(0x7f71cc06cb80 0x7f71cc15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 -- 172.21.15.73:0/3174793668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc1599f0 msgr2=0x7f71cc070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc1599f0 0x7f71cc070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 -- 172.21.15.73:0/3174793668 >> v2:172.21.15.157:3300/0 conn(0x7f71cc06cb80 msgr2=0x7f71cc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:20.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 --2- 172.21.15.73:0/3174793668 >> v2:172.21.15.157:3300/0 conn(0x7f71cc06cb80 0x7f71cc15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.608+0000 7f71d1000640 1 -- 172.21.15.73:0/3174793668 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0b2ad0 con 0x7f71cc0add00 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.609+0000 7f71d1000640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc0add00 0x7f71cc06c640 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ae450 tx=0x7f71b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67357b4e338fff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.610+0000 7f71cbe00640 1 -- 172.21.15.73:0/3174793668 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8031870 con 0x7f71cc0add00 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.610+0000 7f71cbe00640 1 -- 172.21.15.73:0/3174793668 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80319d0 con 0x7f71cc0add00 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.610+0000 7f71cbe00640 1 -- 172.21.15.73:0/3174793668 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8031d60 con 0x7f71cc0add00 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.610+0000 7f71d4400640 1 -- 172.21.15.73:0/3174793668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc0add00 msgr2=0x7f71cc06c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.610+0000 7f71d4400640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc0add00 0x7f71cc06c640 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ae450 tx=0x7f71b802b910 comp rx=0 tx=0).stop 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 -- 172.21.15.73:0/3174793668 shutdown_connections 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc0add00 0x7f71cc06c640 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 --2- 172.21.15.73:0/3174793668 >> v2:172.21.15.157:3300/0 conn(0x7f71cc06cb80 0x7f71cc15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 --2- 172.21.15.73:0/3174793668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc1599f0 0x7f71cc070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 -- 172.21.15.73:0/3174793668 >> 172.21.15.73:0/3174793668 conn(0x7f71cc069cb0 msgr2=0x7f71cc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 -- 172.21.15.73:0/3174793668 shutdown_connections 2024-04-05T22:25:20.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.611+0000 7f71d4400640 1 -- 172.21.15.73:0/3174793668 wait complete. 2024-04-05T22:25:20.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.612+0000 7f71d4400640 1 Processor -- start 2024-04-05T22:25:20.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.612+0000 7f71d4400640 1 -- start start 2024-04-05T22:25:20.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc06cb80 0x7f71cc1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc0add00 0x7f71cc0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0aaf40 con 0x7f71cc1599f0 2024-04-05T22:25:20.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0a84c0 con 0x7f71cc0add00 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.613+0000 7f71d4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc14c880 con 0x7f71cc06cb80 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43924/0 (socket says 172.21.15.73:43924) 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 -- 172.21.15.73:0/788231251 learned_addr learned my addr 172.21.15.73:0/788231251 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d1000640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc0add00 0x7f71cc0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d1a00640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc06cb80 0x7f71cc1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 -- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc06cb80 msgr2=0x7f71cc1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc06cb80 0x7f71cc1489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 -- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc0add00 msgr2=0x7f71cc0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc0add00 0x7f71cc0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.614+0000 7f71d2400640 1 -- 172.21.15.73:0/788231251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80094d0 con 0x7f71cc1599f0 2024-04-05T22:25:20.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71d2400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f71c4007e90 tx=0x7f71c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4014420 con 0x7f71cc1599f0 2024-04-05T22:25:20.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4014580 con 0x7f71cc1599f0 2024-04-05T22:25:20.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc0a94d0 con 0x7f71cc1599f0 2024-04-05T22:25:20.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4014930 con 0x7f71cc1599f0 2024-04-05T22:25:20.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.615+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc0a89f0 con 0x7f71cc1599f0 2024-04-05T22:25:20.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.616+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc068160 con 0x7f71cc1599f0 2024-04-05T22:25:20.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.617+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f71c4014a90 con 0x7f71cc1599f0 2024-04-05T22:25:20.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.618+0000 7f71c9600640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 0x7f71ac06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:20.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.618+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f71c4089d30 con 0x7f71cc1599f0 2024-04-05T22:25:20.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.618+0000 7f71d1a00640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 0x7f71ac06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:20.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.619+0000 7f71d1a00640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 0x7f71ac06d270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006fd0 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:20.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.623+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f71c4054f40 con 0x7f71cc1599f0 2024-04-05T22:25:20.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.804+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71cc163e70 con 0x7f71cc1599f0 2024-04-05T22:25:20.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.805+0000 7f71c9600640 1 -- 172.21.15.73:0/788231251 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71c4058560 con 0x7f71cc1599f0 2024-04-05T22:25:20.806 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:20.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 msgr2=0x7f71ac06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 0x7f71ac06d270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006fd0 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 msgr2=0x7f71cc0af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f71c4007e90 tx=0x7f71c400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 shutdown_connections 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f71ac06adb0 0x7f71ac06d270 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1599f0 0x7f71cc0af080 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc0add00 0x7f71cc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 --2- 172.21.15.73:0/788231251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71cc06cb80 0x7f71cc1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 >> 172.21.15.73:0/788231251 conn(0x7f71cc069cb0 msgr2=0x7f71cc14ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:20.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 shutdown_connections 2024-04-05T22:25:20.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:20.808+0000 7f71d4400640 1 -- 172.21.15.73:0/788231251 wait complete. 2024-04-05T22:25:20.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:25:21.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.022+0000 7f266a400640 1 Processor -- start 2024-04-05T22:25:21.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.023+0000 7f266a400640 1 -- start start 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f266a400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26640b2c00 0x7f266406bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f266a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f266a400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26640cc760 con 0x7f26641441e0 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f266a400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640b1950 con 0x7f26640b2c00 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f266a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b1b60 con 0x7f26640b3070 2024-04-05T22:25:21.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.024+0000 7f2663e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f26640b2c00 0x7f266406bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2669000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f26641441e0 0x7f26640cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44718/0 (socket says 172.21.15.73:44718) 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2669000640 1 -- 172.21.15.73:0/865771542 learned_addr learned my addr 172.21.15.73:0/865771542 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2663e00640 1 -- 172.21.15.73:0/865771542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 msgr2=0x7f266406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2663e00640 1 --2- 172.21.15.73:0/865771542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2663e00640 1 -- 172.21.15.73:0/865771542 >> v1:172.21.15.73:6789/0 conn(0x7f26641441e0 legacy=0x7f26640cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2663e00640 1 -- 172.21.15.73:0/865771542 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640b1e40 con 0x7f26640b2c00 2024-04-05T22:25:21.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.025+0000 7f2663400640 1 -- 172.21.15.73:0/865771542 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758473725 0 0) 0x7f26640cc760 con 0x7f26641441e0 2024-04-05T22:25:21.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.026+0000 7f2663e00640 1 --2- 172.21.15.73:0/865771542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f266406bbc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2650009d90 tx=0x7f265002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5074e5f49377ef4d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.026+0000 7f2663400640 1 -- 172.21.15.73:0/865771542 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265002cc50 con 0x7f26640b2c00 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f2663400640 1 -- 172.21.15.73:0/865771542 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265002cdb0 con 0x7f26640b2c00 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f2663400640 1 -- 172.21.15.73:0/865771542 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650034660 con 0x7f26640b2c00 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 -- 172.21.15.73:0/865771542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 msgr2=0x7f266406bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 --2- 172.21.15.73:0/865771542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f266406bbc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2650009d90 tx=0x7f265002bea0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 -- 172.21.15.73:0/865771542 shutdown_connections 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 --2- 172.21.15.73:0/865771542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 --2- 172.21.15.73:0/865771542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f266406bbc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 -- 172.21.15.73:0/865771542 >> 172.21.15.73:0/865771542 conn(0x7f26640b0830 msgr2=0x7f26640b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 -- 172.21.15.73:0/865771542 shutdown_connections 2024-04-05T22:25:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.027+0000 7f266a400640 1 -- 172.21.15.73:0/865771542 wait complete. 2024-04-05T22:25:21.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.029+0000 7f266a400640 1 Processor -- start 2024-04-05T22:25:21.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.029+0000 7f266a400640 1 -- start start 2024-04-05T22:25:21.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266413a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26641441e0 0x7f2664152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641537f0 con 0x7f26640b2c00 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640aedb0 con 0x7f26641441e0 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f266a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640af000 con 0x7f26640b3070 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f2669000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f2669000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43938/0 (socket says 172.21.15.73:43938) 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.030+0000 7f2669000640 1 -- 172.21.15.73:0/2972434172 learned_addr learned my addr 172.21.15.73:0/2972434172 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 -- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 msgr2=0x7f266413a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266413a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 -- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26641441e0 msgr2=0x7f2664152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669a00640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26641441e0 0x7f2664152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26641441e0 0x7f2664152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 -- 172.21.15.73:0/2972434172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650009a70 con 0x7f26640b2c00 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2669000640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f265400c4d0 tx=0x7f265400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540074e0 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654007660 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654004a20 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640af2e0 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.031+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640af8a0 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.033+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26640cbb80 con 0x7f26640b2c00 2024-04-05T22:25:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.033+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2654004b80 con 0x7f26640b2c00 2024-04-05T22:25:21.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.034+0000 7f2660c00640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 0x7f264406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.034+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2654088640 con 0x7f26640b2c00 2024-04-05T22:25:21.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.034+0000 7f2663e00640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 0x7f264406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.035+0000 7f2663e00640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 0x7f264406d270 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2650036000 tx=0x7f265002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.039+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f26540537c0 con 0x7f26640b2c00 2024-04-05T22:25:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.233+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f266404e160 con 0x7f26640b2c00 2024-04-05T22:25:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.234+0000 7f2660c00640 1 -- 172.21.15.73:0/2972434172 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2654018020 con 0x7f26640b2c00 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:21.235 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:25:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 msgr2=0x7f264406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 0x7f264406d270 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2650036000 tx=0x7f265002e000 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 msgr2=0x7f26640b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f265400c4d0 tx=0x7f265400c990 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 shutdown_connections 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f264406adb0 0x7f264406d270 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26641441e0 0x7f2664152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26640b3070 0x7f266413a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 --2- 172.21.15.73:0/2972434172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26640b2c00 0x7f26640b00a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 >> 172.21.15.73:0/2972434172 conn(0x7f26640b0830 msgr2=0x7f26640b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 shutdown_connections 2024-04-05T22:25:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.237+0000 7f266a400640 1 -- 172.21.15.73:0/2972434172 wait complete. 2024-04-05T22:25:21.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:25:21.257 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:25:21.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.464+0000 7fc7b1400640 1 Processor -- start 2024-04-05T22:25:21.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.464+0000 7fc7b1400640 1 -- start start 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac15ae50 0x7fc7ac0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7b1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7b1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac15b230 con 0x7fc7ac0cecd0 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7b1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac0ce2f0 con 0x7fc7ac0cce90 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0cd2f0 con 0x7fc7ac15ae50 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7a1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7a1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43942/0 (socket says 172.21.15.73:43942) 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7ab200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc7ac0cecd0 0x7fc7ac0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44724/0 (socket says 172.21.15.73:44724) 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.466+0000 7fc7a1e00640 1 -- 172.21.15.73:0/1324326566 learned_addr learned my addr 172.21.15.73:0/1324326566 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7aa800640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac15ae50 0x7fc7ac0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7a1e00640 1 -- 172.21.15.73:0/1324326566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac15ae50 msgr2=0x7fc7ac0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7a1e00640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac15ae50 0x7fc7ac0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7a1e00640 1 -- 172.21.15.73:0/1324326566 >> v1:172.21.15.73:6789/0 conn(0x7fc7ac0cecd0 legacy=0x7fc7ac0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7a1e00640 1 -- 172.21.15.73:0/1324326566 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac15f4a0 con 0x7fc7ac0cce90 2024-04-05T22:25:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.467+0000 7fc7a9e00640 1 -- 172.21.15.73:0/1324326566 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809634982 0 0) 0x7fc7ac15b230 con 0x7fc7ac0cecd0 2024-04-05T22:25:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.468+0000 7fc7a1e00640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0ce430 tx=0x7fc79402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f518628e5397d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7a9e00640 1 -- 172.21.15.73:0/1324326566 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794031640 con 0x7fc7ac0cce90 2024-04-05T22:25:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7a9e00640 1 -- 172.21.15.73:0/1324326566 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7940317a0 con 0x7fc7ac0cce90 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7a9e00640 1 -- 172.21.15.73:0/1324326566 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794031b30 con 0x7fc7ac0cce90 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 -- 172.21.15.73:0/1324326566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac0cce90 msgr2=0x7fc7ac0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0ce430 tx=0x7fc79402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 -- 172.21.15.73:0/1324326566 shutdown_connections 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac0cce90 0x7fc7ac0b2e00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 --2- 172.21.15.73:0/1324326566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac15ae50 0x7fc7ac0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 -- 172.21.15.73:0/1324326566 >> 172.21.15.73:0/1324326566 conn(0x7fc7ac06a0d0 msgr2=0x7fc7ac06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 -- 172.21.15.73:0/1324326566 shutdown_connections 2024-04-05T22:25:21.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.469+0000 7fc7b1400640 1 -- 172.21.15.73:0/1324326566 wait complete. 2024-04-05T22:25:21.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.471+0000 7fc7b1400640 1 Processor -- start 2024-04-05T22:25:21.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.471+0000 7fc7b1400640 1 -- start start 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 0x7fc7ac14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 0x7fc7ac0af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac15d290 con 0x7fc7ac15ae50 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac15c430 con 0x7fc7ac0cecd0 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.472+0000 7fc7b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac14f790 con 0x7fc7ac0cce90 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43952/0 (socket says 172.21.15.73:43952) 2024-04-05T22:25:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 -- 172.21.15.73:0/2705382942 learned_addr learned my addr 172.21.15.73:0/2705382942 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7aa800640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 0x7fc7ac14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7a1e00640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 0x7fc7ac0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 -- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 msgr2=0x7fc7ac14f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 0x7fc7ac14f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 -- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 msgr2=0x7fc7ac0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 0x7fc7ac0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 -- 172.21.15.73:0/2705382942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940094d0 con 0x7fc7ac15ae50 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7aa800640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 0x7fc7ac14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7a1e00640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 0x7fc7ac0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.473+0000 7fc7ab200640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c7a0 tx=0x7fc7a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.474+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400a640 con 0x7fc7ac15ae50 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.474+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a400a7a0 con 0x7fc7ac15ae50 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.474+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400abd0 con 0x7fc7ac15ae50 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.474+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac15c680 con 0x7fc7ac15ae50 2024-04-05T22:25:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.474+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac15cc40 con 0x7fc7ac15ae50 2024-04-05T22:25:21.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.475+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac15ed90 con 0x7fc7ac15ae50 2024-04-05T22:25:21.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.476+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc7a400ad30 con 0x7fc7ac15ae50 2024-04-05T22:25:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.477+0000 7fc7a3400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 0x7fc78806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:21.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.478+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7a40888e0 con 0x7fc7ac15ae50 2024-04-05T22:25:21.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.478+0000 7fc7aa800640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 0x7fc78806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:21.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.478+0000 7fc7aa800640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 0x7fc78806d270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.482+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc7a4053b90 con 0x7fc7ac15ae50 2024-04-05T22:25:21.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.676+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc7ac15d440 con 0x7fc7ac15ae50 2024-04-05T22:25:21.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.771+0000 7fc7a3400640 1 -- 172.21.15.73:0/2705382942 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7a4018070 con 0x7fc7ac15ae50 2024-04-05T22:25:21.771 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 msgr2=0x7fc78806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 0x7fc78806d270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c008040 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 msgr2=0x7fc7ac0afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c7a0 tx=0x7fc7a400cc60 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 shutdown_connections 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc78806adb0 0x7fc78806d270 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7ac15ae50 0x7fc7ac0afe50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7ac0cecd0 0x7fc7ac0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 --2- 172.21.15.73:0/2705382942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7ac0cce90 0x7fc7ac14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.775+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 >> 172.21.15.73:0/2705382942 conn(0x7fc7ac06a0d0 msgr2=0x7fc7ac06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.776+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 shutdown_connections 2024-04-05T22:25:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:21.776+0000 7fc7b1400640 1 -- 172.21.15.73:0/2705382942 wait complete. 2024-04-05T22:25:21.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:25:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.000+0000 7f29f6800640 1 Processor -- start 2024-04-05T22:25:22.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.001+0000 7f29f6800640 1 -- start start 2024-04-05T22:25:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.002+0000 7f29f6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.002+0000 7f29f6800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29f00add00 0x7f29f006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.002+0000 7f29f6800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00b27f0 con 0x7f29f006cb80 2024-04-05T22:25:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.002+0000 7f29f6800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f006d010 con 0x7f29f00add00 2024-04-05T22:25:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.002+0000 7f29f6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00b3890 con 0x7f29f01599f0 2024-04-05T22:25:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.003+0000 7f29ef400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.003+0000 7f29ef400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:39726/0 (socket says 172.21.15.73:39726) 2024-04-05T22:25:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.003+0000 7f29ef400640 1 -- 172.21.15.73:0/1582270156 learned_addr learned my addr 172.21.15.73:0/1582270156 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.003+0000 7f29e7e00640 1 --2- 172.21.15.73:0/1582270156 >> v2:172.21.15.157:3300/0 conn(0x7f29f00add00 0x7f29f006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.004+0000 7f29ef400640 1 -- 172.21.15.73:0/1582270156 >> v2:172.21.15.157:3300/0 conn(0x7f29f00add00 msgr2=0x7f29f006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.004+0000 7f29ef400640 1 --2- 172.21.15.73:0/1582270156 >> v2:172.21.15.157:3300/0 conn(0x7f29f00add00 0x7f29f006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.004+0000 7f29ef400640 1 -- 172.21.15.73:0/1582270156 >> v1:172.21.15.157:6789/0 conn(0x7f29f006cb80 legacy=0x7f29f015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.004+0000 7f29ef400640 1 -- 172.21.15.73:0/1582270156 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f00b2ad0 con 0x7f29f01599f0 2024-04-05T22:25:22.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.005+0000 7f29ef400640 1 --2- 172.21.15.73:0/1582270156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009870 tx=0x7f29e002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbca258bb9a84c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.006+0000 7f29eea00640 1 -- 172.21.15.73:0/1582270156 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0030840 con 0x7f29f01599f0 2024-04-05T22:25:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.006+0000 7f29eea00640 1 -- 172.21.15.73:0/1582270156 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e00309a0 con 0x7f29f01599f0 2024-04-05T22:25:22.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.006+0000 7f29eea00640 1 -- 172.21.15.73:0/1582270156 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0030d30 con 0x7f29f01599f0 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 -- 172.21.15.73:0/1582270156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 msgr2=0x7f29f0070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 --2- 172.21.15.73:0/1582270156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009870 tx=0x7f29e002b980 comp rx=0 tx=0).stop 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 -- 172.21.15.73:0/1582270156 shutdown_connections 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 --2- 172.21.15.73:0/1582270156 >> v2:172.21.15.157:3300/0 conn(0x7f29f00add00 0x7f29f006c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 --2- 172.21.15.73:0/1582270156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f01599f0 0x7f29f0070a40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 -- 172.21.15.73:0/1582270156 >> 172.21.15.73:0/1582270156 conn(0x7f29f0069cb0 msgr2=0x7f29f006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 -- 172.21.15.73:0/1582270156 shutdown_connections 2024-04-05T22:25:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.007+0000 7f29f6800640 1 -- 172.21.15.73:0/1582270156 wait complete. 2024-04-05T22:25:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.009+0000 7f29f6800640 1 Processor -- start 2024-04-05T22:25:22.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.009+0000 7f29f6800640 1 -- start start 2024-04-05T22:25:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f006cb80 0x7f29f0145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f00add00 0x7f29f0146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0145ca0 con 0x7f29f01599f0 2024-04-05T22:25:22.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0146500 con 0x7f29f00add00 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.010+0000 7f29f6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00ccbf0 con 0x7f29f006cb80 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43956/0 (socket says 172.21.15.73:43956) 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 -- 172.21.15.73:0/2675633368 learned_addr learned my addr 172.21.15.73:0/2675633368 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29ef400640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f006cb80 0x7f29f0145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29e7e00640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f00add00 0x7f29f0146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 -- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f006cb80 msgr2=0x7f29f0145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f006cb80 0x7f29f0145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 -- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f00add00 msgr2=0x7f29f0146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f00add00 0x7f29f0146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.011+0000 7f29efe00640 1 -- 172.21.15.73:0/2675633368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00094d0 con 0x7f29f01599f0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29efe00640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c7a0 tx=0x7f29e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8003b70 con 0x7f29f01599f0 2024-04-05T22:25:22.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8003cf0 con 0x7f29f01599f0 2024-04-05T22:25:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8003ea0 con 0x7f29f01599f0 2024-04-05T22:25:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f00cced0 con 0x7f29f01599f0 2024-04-05T22:25:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.012+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f0146190 con 0x7f29f01599f0 2024-04-05T22:25:22.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.013+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f01466b0 con 0x7f29f01599f0 2024-04-05T22:25:22.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.015+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f29e800aa90 con 0x7f29f01599f0 2024-04-05T22:25:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.016+0000 7f29e7400640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 0x7f29cc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:22.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.016+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f29e8089ad0 con 0x7f29f01599f0 2024-04-05T22:25:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.017+0000 7f29ef400640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 0x7f29cc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:22.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.017+0000 7f29ef400640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 0x7f29cc06d270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f29e0036000 tx=0x7f29e003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:22.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.020+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f29e8054cf0 con 0x7f29f01599f0 2024-04-05T22:25:22.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.215+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f29f0146670 con 0x7f29f01599f0 2024-04-05T22:25:22.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.790+0000 7f29e7400640 1 -- 172.21.15.73:0/2675633368 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f29e8058310 con 0x7f29f01599f0 2024-04-05T22:25:22.793 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 msgr2=0x7f29cc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 0x7f29cc06d270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f29e0036000 tx=0x7f29e003b040 comp rx=0 tx=0).stop 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 msgr2=0x7f29f00cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c7a0 tx=0x7f29e800cc60 comp rx=0 tx=0).stop 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 shutdown_connections 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f29cc06adb0 0x7f29cc06d270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f01599f0 0x7f29f00cd350 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f00add00 0x7f29f0146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 --2- 172.21.15.73:0/2675633368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29f006cb80 0x7f29f0145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:22.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.799+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 >> 172.21.15.73:0/2675633368 conn(0x7f29f0069cb0 msgr2=0x7f29f0070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:22.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.802+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 shutdown_connections 2024-04-05T22:25:22.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:22.802+0000 7f29f6800640 1 -- 172.21.15.73:0/2675633368 wait complete. 2024-04-05T22:25:22.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:25:23.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.015+0000 7f0e00c00640 1 Processor -- start 2024-04-05T22:25:23.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.015+0000 7f0e00c00640 1 -- start start 2024-04-05T22:25:23.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.017+0000 7f0e00c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.017+0000 7f0e00c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 0x7f0dfc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.017+0000 7f0e00c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0ae190 con 0x7f0dfc1599f0 2024-04-05T22:25:23.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.017+0000 7f0e00c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0b3890 con 0x7f0dfc0add00 2024-04-05T22:25:23.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.017+0000 7f0e00c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0b27f0 con 0x7f0dfc06cd50 2024-04-05T22:25:23.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.018+0000 7f0df1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.018+0000 7f0df1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:39740/0 (socket says 172.21.15.73:39740) 2024-04-05T22:25:23.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.018+0000 7f0df1400640 1 -- 172.21.15.73:0/152105565 learned_addr learned my addr 172.21.15.73:0/152105565 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:23.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.018+0000 7f0dfa800640 1 --2- 172.21.15.73:0/152105565 >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 0x7f0dfc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0df1400640 1 -- 172.21.15.73:0/152105565 >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 msgr2=0x7f0dfc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0df1400640 1 --2- 172.21.15.73:0/152105565 >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 0x7f0dfc159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0df1400640 1 -- 172.21.15.73:0/152105565 >> v1:172.21.15.157:6789/0 conn(0x7f0dfc1599f0 legacy=0x7f0dfc06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0df1400640 1 -- 172.21.15.73:0/152105565 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0b2ad0 con 0x7f0dfc06cd50 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0df9400640 1 -- 172.21.15.73:0/152105565 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167137645 0 0) 0x7f0dfc0ae190 con 0x7f0dfc1599f0 2024-04-05T22:25:23.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.019+0000 7f0dfa800640 1 --2- 172.21.15.73:0/152105565 >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 0x7f0dfc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:25:23.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.020+0000 7f0df1400640 1 --2- 172.21.15.73:0/152105565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d9c1ecbf2fb42b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.021+0000 7f0df9400640 1 -- 172.21.15.73:0/152105565 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4030840 con 0x7f0dfc06cd50 2024-04-05T22:25:23.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.021+0000 7f0df9400640 1 -- 172.21.15.73:0/152105565 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de40309a0 con 0x7f0dfc06cd50 2024-04-05T22:25:23.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.021+0000 7f0df9400640 1 -- 172.21.15.73:0/152105565 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4030d30 con 0x7f0dfc06cd50 2024-04-05T22:25:23.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 -- 172.21.15.73:0/152105565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 msgr2=0x7f0dfc070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 --2- 172.21.15.73:0/152105565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402b980 comp rx=0 tx=0).stop 2024-04-05T22:25:23.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 -- 172.21.15.73:0/152105565 shutdown_connections 2024-04-05T22:25:23.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 --2- 172.21.15.73:0/152105565 >> v2:172.21.15.157:3300/0 conn(0x7f0dfc0add00 0x7f0dfc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 --2- 172.21.15.73:0/152105565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc06cd50 0x7f0dfc070a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 -- 172.21.15.73:0/152105565 >> 172.21.15.73:0/152105565 conn(0x7f0dfc069cb0 msgr2=0x7f0dfc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:23.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 -- 172.21.15.73:0/152105565 shutdown_connections 2024-04-05T22:25:23.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.022+0000 7f0e00c00640 1 -- 172.21.15.73:0/152105565 wait complete. 2024-04-05T22:25:23.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.024+0000 7f0e00c00640 1 Processor -- start 2024-04-05T22:25:23.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.024+0000 7f0e00c00640 1 -- start start 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc06cd50 0x7f0dfc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc1599f0 0x7f0dfc1f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc1f1560 con 0x7f0dfc0add00 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0cdf10 con 0x7f0dfc06cd50 2024-04-05T22:25:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0e00c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc144d00 con 0x7f0dfc1599f0 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0df1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0df1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43962/0 (socket says 172.21.15.73:43962) 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0df1400640 1 -- 172.21.15.73:0/1490090048 learned_addr learned my addr 172.21.15.73:0/1490090048 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0df9e00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc06cd50 0x7f0dfc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.025+0000 7f0dfa800640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc1599f0 0x7f0dfc1f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 -- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc1599f0 msgr2=0x7f0dfc1f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc1599f0 0x7f0dfc1f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 -- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc06cd50 msgr2=0x7f0dfc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc06cd50 0x7f0dfc0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 -- 172.21.15.73:0/1490090048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4009400 con 0x7f0dfc0add00 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.026+0000 7f0df1400640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000c00 tx=0x7f0de4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.027+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4039c90 con 0x7f0dfc0add00 2024-04-05T22:25:23.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.027+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4039e10 con 0x7f0dfc0add00 2024-04-05T22:25:23.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.027+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4043640 con 0x7f0dfc0add00 2024-04-05T22:25:23.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.027+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de40094d0 con 0x7f0dfc0add00 2024-04-05T22:25:23.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.027+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc0cd5e0 con 0x7f0dfc0add00 2024-04-05T22:25:23.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.028+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc0b3cc0 con 0x7f0dfc0add00 2024-04-05T22:25:23.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.029+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f0de40437a0 con 0x7f0dfc0add00 2024-04-05T22:25:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.030+0000 7f0df2a00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 0x7f0dd8071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.030+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0de403e070 con 0x7f0dfc0add00 2024-04-05T22:25:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.031+0000 7f0df9e00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 0x7f0dd8071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.031+0000 7f0df9e00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 0x7f0dd8071680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0012e0 tx=0x7f0dec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.036+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0de40774b0 con 0x7f0dfc0add00 2024-04-05T22:25:23.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.268+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0dfc0ce080 con 0x7f0dfc0add00 2024-04-05T22:25:23.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.280+0000 7f0df2a00640 1 -- 172.21.15.73:0/1490090048 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0de407aad0 con 0x7f0dfc0add00 2024-04-05T22:25:23.281 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 8 and data pool 9 2024-04-05T22:25:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.286+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 msgr2=0x7f0dd8071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.286+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 0x7f0dd8071680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0012e0 tx=0x7f0dec009040 comp rx=0 tx=0).stop 2024-04-05T22:25:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.286+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 msgr2=0x7f0dfc0ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.286+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000c00 tx=0x7f0de4004730 comp rx=0 tx=0).stop 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 shutdown_connections 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f0dd806f1c0 0x7f0dd8071680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dfc1599f0 0x7f0dfc1f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dfc0add00 0x7f0dfc0ccc70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 --2- 172.21.15.73:0/1490090048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc06cd50 0x7f0dfc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.287+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 >> 172.21.15.73:0/1490090048 conn(0x7f0dfc069cb0 msgr2=0x7f0dfc06f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.288+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 shutdown_connections 2024-04-05T22:25:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.288+0000 7f0e00c00640 1 -- 172.21.15.73:0/1490090048 wait complete. 2024-04-05T22:25:23.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:23.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.490+0000 7ff79f400640 1 Processor -- start 2024-04-05T22:25:23.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.490+0000 7ff79f400640 1 -- start start 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7980b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7980b2ba0 0x7ff7980b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7981627f0 con 0x7ff798161f90 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79f400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff798158790 con 0x7ff7980b2ba0 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980b3570 con 0x7ff798162370 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79ca00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff798161f90 0x7ff798164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44738/0 (socket says 172.21.15.73:44738) 2024-04-05T22:25:23.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff79ca00640 1 -- 172.21.15.73:0/2762993738 learned_addr learned my addr 172.21.15.73:0/2762993738 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163303195 0 0) 0x7ff780001110 con 0x7ff798161f90 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778003520 con 0x7ff798161f90 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.492+0000 7ff78fe00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7980b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488240133 0 0) 0x7ff778003520 con 0x7ff798161f90 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 msgr2=0x7ff7980b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7980b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b2ba0 msgr2=0x7ff7980b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff79d400640 1 --2- 172.21.15.73:0/2762993738 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b2ba0 0x7ff7980b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 --2- 172.21.15.73:0/2762993738 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b2ba0 0x7ff7980b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798165cf0 con 0x7ff798161f90 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff780001800 con 0x7ff798161f90 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 >> v1:172.21.15.73:6789/0 conn(0x7ff798161f90 legacy=0x7ff798164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff778003970 0x7ff778003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778004d00 0x7ff778005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff798158790 con 0x7ff778003970 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980b3570 con 0x7ff778004d00 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800bb10 con 0x7ff798162370 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff79d400640 1 --2- 172.21.15.73:0/2762993738 >> v2:172.21.15.157:3300/0 conn(0x7ff7980b2ba0 0x7ff7980b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.493+0000 7ff79ca00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff778003970 0x7ff778003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79d400640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778004d00 0x7ff778005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff78fe00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79ca00640 1 -- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 msgr2=0x7ff7780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79ca00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff798162370 0x7ff7780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79ca00640 1 -- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778004d00 msgr2=0x7ff778005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79ca00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778004d00 0x7ff778005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79ca00640 1 -- 172.21.15.73:0/2762993738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778003520 con 0x7ff778003970 2024-04-05T22:25:23.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.494+0000 7ff79d400640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778004d00 0x7ff778005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.495+0000 7ff79ca00640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff778003970 0x7ff778003d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c210 tx=0x7ff7780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bdd10d72dbb5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.495+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780005ad0 con 0x7ff778003970 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.495+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780005c30 con 0x7ff778003970 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.495+0000 7ff797e00640 1 -- 172.21.15.73:0/2762993738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78002c810 con 0x7ff778003970 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.495+0000 7ff79f400640 1 -- 172.21.15.73:0/2762993738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff778003970 msgr2=0x7ff778003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79f400640 1 --2- 172.21.15.73:0/2762993738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff778003970 0x7ff778003d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c210 tx=0x7ff7780057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79d400640 1 -- 172.21.15.73:0/2762993738 reap_dead start 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79f400640 1 -- 172.21.15.73:0/2762993738 shutdown_connections 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79f400640 1 -- 172.21.15.73:0/2762993738 >> 172.21.15.73:0/2762993738 conn(0x7ff7980ce8e0 msgr2=0x7ff7980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:23.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79f400640 1 -- 172.21.15.73:0/2762993738 shutdown_connections 2024-04-05T22:25:23.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.496+0000 7ff79f400640 1 -- 172.21.15.73:0/2762993738 wait complete. 2024-04-05T22:25:23.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.497+0000 7ff79f400640 1 Processor -- start 2024-04-05T22:25:23.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.498+0000 7ff79f400640 1 -- start start 2024-04-05T22:25:23.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.498+0000 7ff79f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778003970 0x7ff79814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.498+0000 7ff79f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806cec0 0x7ff798153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.498+0000 7ff79f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79814d3d0 con 0x7ff79814d870 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cc190 con 0x7ff778003970 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff798151f80 con 0x7ff79806cec0 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43974/0 (socket says 172.21.15.73:43974) 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 -- 172.21.15.73:0/1696242189 learned_addr learned my addr 172.21.15.73:0/1696242189 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79ca00640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778003970 0x7ff79814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff78fe00640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806cec0 0x7ff798153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 -- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806cec0 msgr2=0x7ff798153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806cec0 0x7ff798153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 -- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778003970 msgr2=0x7ff79814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778003970 0x7ff79814ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 -- 172.21.15.73:0/1696242189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798152260 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.499+0000 7ff79d400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff7980cf9e0 tx=0x7ff79000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.500+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900097c0 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.500+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790009920 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.500+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790009cd0 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.500+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff798152540 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.500+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff798152b00 con 0x7ff79814d870 2024-04-05T22:25:23.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.501+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798066af0 con 0x7ff79814d870 2024-04-05T22:25:23.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.502+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7ff79001e3b0 con 0x7ff79814d870 2024-04-05T22:25:23.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.503+0000 7ff795600640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 0x7ff77406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.503+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff79008a050 con 0x7ff79814d870 2024-04-05T22:25:23.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.503+0000 7ff79ca00640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 0x7ff77406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.504+0000 7ff79ca00640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 0x7ff77406d270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff780034000 tx=0x7ff780007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.507+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff790054f30 con 0x7ff79814d870 2024-04-05T22:25:23.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.691+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff798068160 con 0x7ff79814d870 2024-04-05T22:25:23.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.694+0000 7ff795600640 1 -- 172.21.15.73:0/1696242189 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16110 (secure 0 0 0) 0x7ff790058550 con 0x7ff79814d870 2024-04-05T22:25:23.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:23.695 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":61,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:23.270390+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:23.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 msgr2=0x7ff77406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 0x7ff77406d270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff780034000 tx=0x7ff780007000 comp rx=0 tx=0).stop 2024-04-05T22:25:23.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 msgr2=0x7ff79814e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff7980cf9e0 tx=0x7ff79000b7a0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 shutdown_connections 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff77406adb0 0x7ff77406d270 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff79814d870 0x7ff79814e280 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79806cec0 0x7ff798153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 --2- 172.21.15.73:0/1696242189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff778003970 0x7ff79814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 >> 172.21.15.73:0/1696242189 conn(0x7ff7980ce8e0 msgr2=0x7ff7980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 shutdown_connections 2024-04-05T22:25:23.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.697+0000 7ff79f400640 1 -- 172.21.15.73:0/1696242189 wait complete. 2024-04-05T22:25:23.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:23.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.915+0000 7f9f0c000640 1 Processor -- start 2024-04-05T22:25:23.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.915+0000 7f9f0c000640 1 -- start start 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0c000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 0x7f9f040cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0c000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f040ccd20 0x7f9f0406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0c000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0406f5b0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0c000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f04070410 con 0x7f9f040ccd20 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0c000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f04158bb0 con 0x7f9f041587d0 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f08c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f040ccd20 0x7f9f0406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f08c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f040ccd20 0x7f9f0406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43980/0 (socket says 172.21.15.73:43980) 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f08c00640 1 -- 172.21.15.73:0/3165348698 learned_addr learned my addr 172.21.15.73:0/3165348698 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.917+0000 7f9f0a000640 1 --1- 172.21.15.73:0/3165348698 >> v1:172.21.15.73:6789/0 conn(0x7f9f0406d0c0 0x7f9f0415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44740/0 (socket says 172.21.15.73:44740) 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f09600640 1 --2- 172.21.15.73:0/3165348698 >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 0x7f9f040cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f08c00640 1 -- 172.21.15.73:0/3165348698 >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 msgr2=0x7f9f040cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:23.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f08c00640 1 --2- 172.21.15.73:0/3165348698 >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 0x7f9f040cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f08c00640 1 -- 172.21.15.73:0/3165348698 >> v1:172.21.15.73:6789/0 conn(0x7f9f0406d0c0 legacy=0x7f9f0415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f08c00640 1 -- 172.21.15.73:0/3165348698 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0406f800 con 0x7f9f040ccd20 2024-04-05T22:25:23.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f03e00640 1 -- 172.21.15.73:0/3165348698 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980826688 0 0) 0x7f9f0406f5b0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.918+0000 7f9f09600640 1 --2- 172.21.15.73:0/3165348698 >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 0x7f9f040cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.919+0000 7f9f08c00640 1 --2- 172.21.15.73:0/3165348698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f040ccd20 0x7f9f0406cb80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009870 tx=0x7f9ef002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45750772f6de3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.919+0000 7f9f03e00640 1 -- 172.21.15.73:0/3165348698 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0030850 con 0x7f9f040ccd20 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.919+0000 7f9f03e00640 1 -- 172.21.15.73:0/3165348698 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00309b0 con 0x7f9f040ccd20 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.919+0000 7f9f03e00640 1 -- 172.21.15.73:0/3165348698 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0030d40 con 0x7f9f040ccd20 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 -- 172.21.15.73:0/3165348698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f040ccd20 msgr2=0x7f9f0406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 --2- 172.21.15.73:0/3165348698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f040ccd20 0x7f9f0406cb80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009870 tx=0x7f9ef002b990 comp rx=0 tx=0).stop 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 -- 172.21.15.73:0/3165348698 shutdown_connections 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 --2- 172.21.15.73:0/3165348698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f040ccd20 0x7f9f0406cb80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 --2- 172.21.15.73:0/3165348698 >> v2:172.21.15.157:3300/0 conn(0x7f9f041587d0 0x7f9f040cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 -- 172.21.15.73:0/3165348698 >> 172.21.15.73:0/3165348698 conn(0x7f9f040ce8e0 msgr2=0x7f9f040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 -- 172.21.15.73:0/3165348698 shutdown_connections 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.920+0000 7f9f0c000640 1 -- 172.21.15.73:0/3165348698 wait complete. 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.921+0000 7f9f0c000640 1 Processor -- start 2024-04-05T22:25:23.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 -- start start 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f041587d0 0x7f9f0415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0415c8d0 0x7f9f04154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0415ddf0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04152dc0 con 0x7f9f0415c8d0 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.922+0000 7f9f0c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0415cd50 con 0x7f9f041587d0 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43984/0 (socket says 172.21.15.73:43984) 2024-04-05T22:25:23.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 -- 172.21.15.73:0/1170599706 learned_addr learned my addr 172.21.15.73:0/1170599706 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:23.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 -- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f041587d0 msgr2=0x7f9f0415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f041587d0 0x7f9f0415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 -- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0415c8d0 msgr2=0x7f9f04154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0415c8d0 0x7f9f04154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 -- 172.21.15.73:0/1170599706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00094d0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.923+0000 7f9f09600640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800ce00 tx=0x7f9ef800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.924+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800f890 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.924+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800f9f0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.924+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800fd80 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.924+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0415d030 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.924+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0415d5f0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.925+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f04066af0 con 0x7f9f0406d0c0 2024-04-05T22:25:23.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.926+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f9ef800b040 con 0x7f9f0406d0c0 2024-04-05T22:25:23.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.927+0000 7f9f01600640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 0x7f9ee406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:23.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.927+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef808a260 con 0x7f9f0406d0c0 2024-04-05T22:25:23.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.927+0000 7f9f08c00640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 0x7f9ee406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:23.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.928+0000 7f9f08c00640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 0x7f9ee406d270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0036000 tx=0x7f9ef003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:23.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:23.931+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9ef8055010 con 0x7f9f0406d0c0 2024-04-05T22:25:24.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.110+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f04068160 con 0x7f9f0406d0c0 2024-04-05T22:25:24.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.112+0000 7f9f01600640 1 -- 172.21.15.73:0/1170599706 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f9ef8058630 con 0x7f9f0406d0c0 2024-04-05T22:25:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:24.112 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":61,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:23.270390+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.115+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 msgr2=0x7f9ee406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.115+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 0x7f9ee406d270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0036000 tx=0x7f9ef003b040 comp rx=0 tx=0).stop 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.115+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 msgr2=0x7f9f04152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.115+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800ce00 tx=0x7f9ef800a650 comp rx=0 tx=0).stop 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 shutdown_connections 2024-04-05T22:25:24.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9ee406adb0 0x7f9ee406d270 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0415c8d0 0x7f9f04154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f041587d0 0x7f9f0415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 --2- 172.21.15.73:0/1170599706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f0406d0c0 0x7f9f04152880 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 >> 172.21.15.73:0/1170599706 conn(0x7f9f040ce8e0 msgr2=0x7f9f0406c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 shutdown_connections 2024-04-05T22:25:24.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.116+0000 7f9f0c000640 1 -- 172.21.15.73:0/1170599706 wait complete. 2024-04-05T22:25:24.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:25:24.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.342+0000 7f25eee00640 1 Processor -- start 2024-04-05T22:25:24.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.342+0000 7f25eee00640 1 -- start start 2024-04-05T22:25:24.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.343+0000 7f25eee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25e815ce10 0x7f25e80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.343+0000 7f25eee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25e815d1f0 0x7f25e815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.343+0000 7f25eee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e815ddd0 con 0x7f25e80b0990 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.343+0000 7f25eee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e815dbb0 con 0x7f25e815d1f0 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.343+0000 7f25eee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e815e0b0 con 0x7f25e815ce10 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25e815d1f0 0x7f25e815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25ece00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25e80b0990 0x7f25e80b0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44742/0 (socket says 172.21.15.73:44742) 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25e815d1f0 0x7f25e815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44000/0 (socket says 172.21.15.73:44000) 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25ece00640 1 -- 172.21.15.73:0/2601109748 learned_addr learned my addr 172.21.15.73:0/2601109748 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7e00640 1 --2- 172.21.15.73:0/2601109748 >> v2:172.21.15.157:3300/0 conn(0x7f25e815ce10 0x7f25e80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 -- 172.21.15.73:0/2601109748 >> v2:172.21.15.157:3300/0 conn(0x7f25e815ce10 msgr2=0x7f25e80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 --2- 172.21.15.73:0/2601109748 >> v2:172.21.15.157:3300/0 conn(0x7f25e815ce10 0x7f25e80cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 -- 172.21.15.73:0/2601109748 >> v1:172.21.15.73:6789/0 conn(0x7f25e80b0990 legacy=0x7f25e80b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e7400640 1 -- 172.21.15.73:0/2601109748 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80cd460 con 0x7f25e815d1f0 2024-04-05T22:25:24.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.344+0000 7f25e6a00640 1 -- 172.21.15.73:0/2601109748 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694379735 0 0) 0x7f25e815ddd0 con 0x7f25e80b0990 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.345+0000 7f25e7400640 1 --2- 172.21.15.73:0/2601109748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e815d670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f25e806cfa0 tx=0x7f25d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d25752c97cb4c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25e6a00640 1 -- 172.21.15.73:0/2601109748 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4031830 con 0x7f25e815d1f0 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25e6a00640 1 -- 172.21.15.73:0/2601109748 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4031990 con 0x7f25e815d1f0 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25e6a00640 1 -- 172.21.15.73:0/2601109748 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4031d20 con 0x7f25e815d1f0 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 -- 172.21.15.73:0/2601109748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 msgr2=0x7f25e815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 --2- 172.21.15.73:0/2601109748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e815d670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f25e806cfa0 tx=0x7f25d402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 -- 172.21.15.73:0/2601109748 shutdown_connections 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 --2- 172.21.15.73:0/2601109748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e815d670 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 --2- 172.21.15.73:0/2601109748 >> v2:172.21.15.157:3300/0 conn(0x7f25e815ce10 0x7f25e80cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 -- 172.21.15.73:0/2601109748 >> 172.21.15.73:0/2601109748 conn(0x7f25e8099a60 msgr2=0x7f25e8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 -- 172.21.15.73:0/2601109748 shutdown_connections 2024-04-05T22:25:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.346+0000 7f25eee00640 1 -- 172.21.15.73:0/2601109748 wait complete. 2024-04-05T22:25:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.348+0000 7f25eee00640 1 Processor -- start 2024-04-05T22:25:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.348+0000 7f25eee00640 1 -- start start 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 0x7f25e809e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 0x7f25e806e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81f69d0 con 0x7f25e815d1f0 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809e7b0 con 0x7f25e815ce10 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25eee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81f5dc0 con 0x7f25e80b0990 2024-04-05T22:25:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44012/0 (socket says 172.21.15.73:44012) 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.349+0000 7f25ece00640 1 -- 172.21.15.73:0/4059867535 learned_addr learned my addr 172.21.15.73:0/4059867535 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25e7e00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 0x7f25e809e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25e7400640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 0x7f25e806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 -- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 msgr2=0x7f25e809e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 0x7f25e809e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 -- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 msgr2=0x7f25e806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 0x7f25e806e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 -- 172.21.15.73:0/4059867535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d4009400 con 0x7f25e815d1f0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25e7e00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 0x7f25e809e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25e7400640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 0x7f25e806e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25ece00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c7e0 tx=0x7f25e000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000a640 con 0x7f25e815d1f0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e000a7c0 con 0x7f25e815d1f0 2024-04-05T22:25:24.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e81f60a0 con 0x7f25e815d1f0 2024-04-05T22:25:24.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.350+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ab70 con 0x7f25e815d1f0 2024-04-05T22:25:24.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.351+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81f6660 con 0x7f25e815d1f0 2024-04-05T22:25:24.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.352+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8067480 con 0x7f25e815d1f0 2024-04-05T22:25:24.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.352+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f25e000acd0 con 0x7f25e815d1f0 2024-04-05T22:25:24.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.354+0000 7f25dfe00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 0x7f25c806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:24.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.354+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25e0088ee0 con 0x7f25e815d1f0 2024-04-05T22:25:24.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.354+0000 7f25e7e00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 0x7f25c806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:24.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.355+0000 7f25e7e00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 0x7f25c806d270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f25d8008890 tx=0x7f25d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.358+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f25e0053e40 con 0x7f25e815d1f0 2024-04-05T22:25:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:24.591+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f25e8068160 con 0x7f25e815d1f0 2024-04-05T22:25:25.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.300+0000 7f25dfe00640 1 -- 172.21.15.73:0/4059867535 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f25e0018020 con 0x7f25e815d1f0 2024-04-05T22:25:25.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 msgr2=0x7f25c806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 0x7f25c806d270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f25d8008890 tx=0x7f25d8008040 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 msgr2=0x7f25e806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c7e0 tx=0x7f25e000cca0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 shutdown_connections 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25c806adb0 0x7f25c806d270 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25e815d1f0 0x7f25e806e680 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e815ce10 0x7f25e806e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 --2- 172.21.15.73:0/4059867535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25e80b0990 0x7f25e809e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 >> 172.21.15.73:0/4059867535 conn(0x7f25e8099a60 msgr2=0x7f25e80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 shutdown_connections 2024-04-05T22:25:25.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.304+0000 7f25eee00640 1 -- 172.21.15.73:0/4059867535 wait complete. 2024-04-05T22:25:25.326 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:25:25.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:25:25.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.533+0000 7f5fe8200640 1 Processor -- start 2024-04-05T22:25:25.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.533+0000 7f5fe8200640 1 -- start start 2024-04-05T22:25:25.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe8200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5fe006c9a0 0x7f5fe006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe8200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 0x7f5fe0164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe8200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe006bea0 con 0x7f5fe015abe0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe8200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe015afc0 con 0x7f5fe006c9a0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe8200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe006f5b0 con 0x7f5fe006ba70 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe5800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5fe015abe0 0x7f5fe00712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44748/0 (socket says 172.21.15.73:44748) 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe5800640 1 -- 172.21.15.73:0/228406420 learned_addr learned my addr 172.21.15.73:0/228406420 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe4e00640 1 --2- 172.21.15.73:0/228406420 >> v2:172.21.15.73:3300/0 conn(0x7f5fe006c9a0 0x7f5fe006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe6200640 1 --2- 172.21.15.73:0/228406420 >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 0x7f5fe0164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.535+0000 7f5fe4e00640 1 -- 172.21.15.73:0/228406420 >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 msgr2=0x7f5fe0164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.536+0000 7f5fe4e00640 1 --2- 172.21.15.73:0/228406420 >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 0x7f5fe0164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.536+0000 7f5fe4e00640 1 -- 172.21.15.73:0/228406420 >> v1:172.21.15.73:6789/0 conn(0x7f5fe015abe0 legacy=0x7f5fe00712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.536+0000 7f5fe4e00640 1 -- 172.21.15.73:0/228406420 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe006f890 con 0x7f5fe006c9a0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.536+0000 7f5fdfe00640 1 -- 172.21.15.73:0/228406420 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696655349 0 0) 0x7f5fe006bea0 con 0x7f5fe015abe0 2024-04-05T22:25:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.536+0000 7f5fe6200640 1 --2- 172.21.15.73:0/228406420 >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 0x7f5fe0164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:25.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.537+0000 7f5fe4e00640 1 --2- 172.21.15.73:0/228406420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006c9a0 0x7f5fe006b530 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5fe006c900 tx=0x7f5fcc02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c927b4fb78da62 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:25.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.537+0000 7f5fdfe00640 1 -- 172.21.15.73:0/228406420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc031710 con 0x7f5fe006c9a0 2024-04-05T22:25:25.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.537+0000 7f5fdfe00640 1 -- 172.21.15.73:0/228406420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc02f050 con 0x7f5fe006c9a0 2024-04-05T22:25:25.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.537+0000 7f5fdfe00640 1 -- 172.21.15.73:0/228406420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc031a90 con 0x7f5fe006c9a0 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 -- 172.21.15.73:0/228406420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006c9a0 msgr2=0x7f5fe006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 --2- 172.21.15.73:0/228406420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006c9a0 0x7f5fe006b530 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5fe006c900 tx=0x7f5fcc02b7b0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 -- 172.21.15.73:0/228406420 shutdown_connections 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 --2- 172.21.15.73:0/228406420 >> v2:172.21.15.157:3300/0 conn(0x7f5fe006ba70 0x7f5fe0164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 --2- 172.21.15.73:0/228406420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006c9a0 0x7f5fe006b530 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 -- 172.21.15.73:0/228406420 >> 172.21.15.73:0/228406420 conn(0x7f5fe0069cb0 msgr2=0x7f5fe006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 -- 172.21.15.73:0/228406420 shutdown_connections 2024-04-05T22:25:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.538+0000 7f5fe8200640 1 -- 172.21.15.73:0/228406420 wait complete. 2024-04-05T22:25:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.540+0000 7f5fe8200640 1 Processor -- start 2024-04-05T22:25:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.540+0000 7f5fe8200640 1 -- start start 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe006c9a0 0x7f5fe0150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe015abe0 0x7f5fe01f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe015b190 con 0x7f5fe006ba70 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01f3090 con 0x7f5fe015abe0 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.541+0000 7f5fe8200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01f3370 con 0x7f5fe006c9a0 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44040/0 (socket says 172.21.15.73:44040) 2024-04-05T22:25:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 -- 172.21.15.73:0/2141126180 learned_addr learned my addr 172.21.15.73:0/2141126180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 -- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe006c9a0 msgr2=0x7f5fe0150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe006c9a0 0x7f5fe0150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 -- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe015abe0 msgr2=0x7f5fe01f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe015abe0 0x7f5fe01f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 -- 172.21.15.73:0/2141126180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009400 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.542+0000 7f5fe5800640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c400 tx=0x7f5fd400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.543+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4005ce0 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.543+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4005e40 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.543+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe01f3650 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.543+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4007310 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.543+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe0150ac0 con 0x7f5fe006ba70 2024-04-05T22:25:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.544+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe015b5e0 con 0x7f5fe006ba70 2024-04-05T22:25:25.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.545+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f5fd4007530 con 0x7f5fe006ba70 2024-04-05T22:25:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.546+0000 7f5fdd600640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 0x7f5fc006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.546+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fd4088ca0 con 0x7f5fe006ba70 2024-04-05T22:25:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.546+0000 7f5fe4e00640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 0x7f5fc006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:25.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.547+0000 7f5fe4e00640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 0x7f5fc006d270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc035000 tx=0x7f5fcc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.550+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5fe015b5e0 con 0x7f5fe006ba70 2024-04-05T22:25:25.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:25.769+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5fe0067270 con 0x7f5fe006ba70 2024-04-05T22:25:26.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.304+0000 7f5fdd600640 1 -- 172.21.15.73:0/2141126180 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5fd40170c0 con 0x7f5fe006ba70 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.307+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 msgr2=0x7f5fc006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.307+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 0x7f5fc006d270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc035000 tx=0x7f5fcc030000 comp rx=0 tx=0).stop 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.307+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 msgr2=0x7f5fe0149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.307+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c400 tx=0x7f5fd400c8c0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 shutdown_connections 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5fc006adb0 0x7f5fc006d270 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe015abe0 0x7f5fe01f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe006c9a0 0x7f5fe0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 --2- 172.21.15.73:0/2141126180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe006ba70 0x7f5fe0149710 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 >> 172.21.15.73:0/2141126180 conn(0x7f5fe0069cb0 msgr2=0x7f5fe00ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:26.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 shutdown_connections 2024-04-05T22:25:26.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.308+0000 7f5fe8200640 1 -- 172.21.15.73:0/2141126180 wait complete. 2024-04-05T22:25:26.328 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:25:26.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:26.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.540+0000 7fd4b4e00640 1 Processor -- start 2024-04-05T22:25:26.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.540+0000 7fd4b4e00640 1 -- start start 2024-04-05T22:25:26.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.541+0000 7fd4b4e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4b0158860 0x7fd4b00b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.541+0000 7fd4b4e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4b015ed30 0x7fd4b006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.541+0000 7fd4b4e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b006b500 con 0x7fd4b00b1f00 2024-04-05T22:25:26.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.541+0000 7fd4b4e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00ccaf0 con 0x7fd4b015ed30 2024-04-05T22:25:26.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.541+0000 7fd4b4e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00b23a0 con 0x7fd4b0158860 2024-04-05T22:25:26.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4b015ed30 0x7fd4b006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:26.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4b015ed30 0x7fd4b006bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51380/0 (socket says 172.21.15.73:51380) 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 -- 172.21.15.73:0/681244787 learned_addr learned my addr 172.21.15.73:0/681244787 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 -- 172.21.15.73:0/681244787 >> v2:172.21.15.157:3300/0 conn(0x7fd4b0158860 msgr2=0x7fd4b00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 --2- 172.21.15.73:0/681244787 >> v2:172.21.15.157:3300/0 conn(0x7fd4b0158860 0x7fd4b00b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 -- 172.21.15.73:0/681244787 >> v1:172.21.15.157:6789/0 conn(0x7fd4b00b1f00 legacy=0x7fd4b015e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.542+0000 7fd4afe00640 1 -- 172.21.15.73:0/681244787 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00b2680 con 0x7fd4b015ed30 2024-04-05T22:25:26.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.543+0000 7fd4afe00640 1 --2- 172.21.15.73:0/681244787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b006bc70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006f380 tx=0x7fd4a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65d8f7fd77ef15 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:26.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4aea00640 1 -- 172.21.15.73:0/681244787 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a802c9f0 con 0x7fd4b015ed30 2024-04-05T22:25:26.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4aea00640 1 -- 172.21.15.73:0/681244787 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a802cb50 con 0x7fd4b015ed30 2024-04-05T22:25:26.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4aea00640 1 -- 172.21.15.73:0/681244787 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80343f0 con 0x7fd4b015ed30 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 -- 172.21.15.73:0/681244787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 msgr2=0x7fd4b006bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/681244787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b006bc70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006f380 tx=0x7fd4a802be50 comp rx=0 tx=0).stop 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 -- 172.21.15.73:0/681244787 shutdown_connections 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/681244787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b006bc70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/681244787 >> v2:172.21.15.157:3300/0 conn(0x7fd4b0158860 0x7fd4b00b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 -- 172.21.15.73:0/681244787 >> 172.21.15.73:0/681244787 conn(0x7fd4b00b06c0 msgr2=0x7fd4b00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 -- 172.21.15.73:0/681244787 shutdown_connections 2024-04-05T22:25:26.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.544+0000 7fd4b4e00640 1 -- 172.21.15.73:0/681244787 wait complete. 2024-04-05T22:25:26.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.546+0000 7fd4b4e00640 1 Processor -- start 2024-04-05T22:25:26.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.546+0000 7fd4b4e00640 1 -- start start 2024-04-05T22:25:26.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 0x7fd4b00af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b0156920 0x7fd4b0142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015d4f0 con 0x7fd4b015ed30 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015d7d0 con 0x7fd4b00b1f00 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4b4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015dab0 con 0x7fd4b0156920 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51390/0 (socket says 172.21.15.73:51390) 2024-04-05T22:25:26.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.547+0000 7fd4a6a00640 1 -- 172.21.15.73:0/1042784836 learned_addr learned my addr 172.21.15.73:0/1042784836 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4af400640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 0x7fd4b00af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4afe00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b0156920 0x7fd4b0142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 -- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b0156920 msgr2=0x7fd4b0142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b0156920 0x7fd4b0142e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 -- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 msgr2=0x7fd4b00af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 0x7fd4b00af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 -- 172.21.15.73:0/1042784836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009b30 con 0x7fd4b015ed30 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4af400640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 0x7fd4b00af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a6a00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c410 tx=0x7fd49800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980035f0 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498003750 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498003b40 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.548+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b015dd90 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.549+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b00afe30 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.550+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b015e2b0 con 0x7fd4b015ed30 2024-04-05T22:25:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.551+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fd498003ca0 con 0x7fd4b015ed30 2024-04-05T22:25:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.552+0000 7fd4a7e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 0x7fd48c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.552+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4980888e0 con 0x7fd4b015ed30 2024-04-05T22:25:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.552+0000 7fd4af400640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 0x7fd48c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.553+0000 7fd4af400640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 0x7fd48c06d270 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0006fd0 tx=0x7fd4a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:26.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.556+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd4b015e2b0 con 0x7fd4b015ed30 2024-04-05T22:25:26.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.789+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4b015e2b0 con 0x7fd4b015ed30 2024-04-05T22:25:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.790+0000 7fd4a7e00640 1 -- 172.21.15.73:0/1042784836 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fd4b015e2b0 con 0x7fd4b015ed30 2024-04-05T22:25:26.790 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:26.790 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 32 2024-04-05T22:25:26.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 msgr2=0x7fd48c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 0x7fd48c06d270 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0006fd0 tx=0x7fd4a0008040 comp rx=0 tx=0).stop 2024-04-05T22:25:26.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 msgr2=0x7fd4b015cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c410 tx=0x7fd49800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 shutdown_connections 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fd48c06adb0 0x7fd48c06d270 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b0156920 0x7fd4b0142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4b015ed30 0x7fd4b015cb80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 --2- 172.21.15.73:0/1042784836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b00b1f00 0x7fd4b00af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.792+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 >> 172.21.15.73:0/1042784836 conn(0x7fd4b00b06c0 msgr2=0x7fd4b006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.793+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 shutdown_connections 2024-04-05T22:25:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:26.793+0000 7fd4b4e00640 1 -- 172.21.15.73:0/1042784836 wait complete. 2024-04-05T22:25:26.812 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.012+0000 7f28ef800640 1 Processor -- start 2024-04-05T22:25:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.012+0000 7f28ef800640 1 -- start start 2024-04-05T22:25:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.013+0000 7f28ef800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28e806cb80 0x7f28e806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.013+0000 7f28ef800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28e815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.013+0000 7f28ef800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e806f5b0 con 0x7f28e81587d0 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.013+0000 7f28ef800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e806d500 con 0x7f28e806cb80 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.013+0000 7f28ef800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815f220 con 0x7f28e815e850 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28ece00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f28e81587d0 0x7f28e80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38082/0 (socket says 172.21.15.73:38082) 2024-04-05T22:25:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28ece00640 1 -- 172.21.15.73:0/3995360577 learned_addr learned my addr 172.21.15.73:0/3995360577 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3995360577 >> v2:172.21.15.157:3300/0 conn(0x7f28e806cb80 0x7f28e806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364786861 0 0) 0x7f28e806f5b0 con 0x7f28e81587d0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28cc003520 con 0x7f28e81587d0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28ed800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28e815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 321580729 0 0) 0x7f28cc003520 con 0x7f28e81587d0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 msgr2=0x7f28e815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28e815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 >> v2:172.21.15.157:3300/0 conn(0x7f28e806cb80 msgr2=0x7f28e806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 --2- 172.21.15.73:0/3995360577 >> v2:172.21.15.157:3300/0 conn(0x7f28e806cb80 0x7f28e806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8070410 con 0x7f28e81587d0 2024-04-05T22:25:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f28d8001670 con 0x7f28e81587d0 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3995360577 >> v2:172.21.15.157:3300/0 conn(0x7f28e806cb80 0x7f28e806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.014+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 >> v1:172.21.15.73:6789/0 conn(0x7f28e81587d0 legacy=0x7f28e80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc0039a0 0x7f28cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e806d500 con 0x7f28cc004d00 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815f220 con 0x7f28cc0039a0 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc00ba80 con 0x7f28e815e850 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ece00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc0039a0 0x7f28cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 -- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 msgr2=0x7f28cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 -- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc0039a0 msgr2=0x7f28cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28cc0039a0 0x7f28cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28ed800640 1 -- 172.21.15.73:0/3995360577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc003520 con 0x7f28cc004d00 2024-04-05T22:25:27.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.015+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e815e850 0x7f28cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:27.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.016+0000 7f28ed800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28cc005180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f28e0007360 tx=0x7f28cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8a7c05737f5a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.016+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e002ccb0 con 0x7f28cc004d00 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.016+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e002ce10 con 0x7f28cc004d00 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28e7e00640 1 -- 172.21.15.73:0/3995360577 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0034660 con 0x7f28cc004d00 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 -- 172.21.15.73:0/3995360577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 msgr2=0x7f28cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 --2- 172.21.15.73:0/3995360577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28cc005180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f28e0007360 tx=0x7f28cc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ed800640 1 -- 172.21.15.73:0/3995360577 reap_dead start 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 -- 172.21.15.73:0/3995360577 shutdown_connections 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 -- 172.21.15.73:0/3995360577 >> 172.21.15.73:0/3995360577 conn(0x7f28e80ce8e0 msgr2=0x7f28e80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 -- 172.21.15.73:0/3995360577 shutdown_connections 2024-04-05T22:25:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.017+0000 7f28ef800640 1 -- 172.21.15.73:0/3995360577 wait complete. 2024-04-05T22:25:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.019+0000 7f28ef800640 1 Processor -- start 2024-04-05T22:25:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.020+0000 7f28ef800640 1 -- start start 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e815d330 0x7f28e815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e80c4cc0 0x7f28e80c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815dcf0 con 0x7f28cc004d00 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80c3880 con 0x7f28e815d330 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ef800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80c3b60 con 0x7f28e80c4cc0 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51396/0 (socket says 172.21.15.73:51396) 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28ece00640 1 -- 172.21.15.73:0/3726612188 learned_addr learned my addr 172.21.15.73:0/3726612188 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.021+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e815d330 0x7f28e815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ed800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e80c4cc0 0x7f28e80c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 -- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e80c4cc0 msgr2=0x7f28e80c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e80c4cc0 0x7f28e80c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 -- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e815d330 msgr2=0x7f28e815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e815d330 0x7f28e815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 -- 172.21.15.73:0/3726612188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80c3e40 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ece00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e7d0 tx=0x7f28d800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800eee0 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d800c8d0 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800cc60 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80c4120 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.022+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e8156bc0 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.023+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8066af0 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.024+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f28d801d460 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.025+0000 7f28e5600640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 0x7f28c406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.025+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28d80896d0 con 0x7f28cc004d00 2024-04-05T22:25:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.025+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 0x7f28c406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.026+0000 7f28dfe00640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 0x7f28c406d2a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28e806aec0 tx=0x7f28d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.030+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f28d8054590 con 0x7f28cc004d00 2024-04-05T22:25:27.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.221+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28e8068160 con 0x7f28cc004d00 2024-04-05T22:25:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.224+0000 7f28e5600640 1 -- 172.21.15.73:0/3726612188 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f28d8057bb0 con 0x7f28cc004d00 2024-04-05T22:25:27.224 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:27.224 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 msgr2=0x7f28c406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 0x7f28c406d2a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28e806aec0 tx=0x7f28d0008040 comp rx=0 tx=0).stop 2024-04-05T22:25:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 msgr2=0x7f28e8156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e7d0 tx=0x7f28d800ec90 comp rx=0 tx=0).stop 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 shutdown_connections 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f28c406ade0 0x7f28c406d2a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28e80c4cc0 0x7f28e80c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e815d330 0x7f28e815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 --2- 172.21.15.73:0/3726612188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28cc004d00 0x7f28e8156680 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.226+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 >> 172.21.15.73:0/3726612188 conn(0x7f28e80ce8e0 msgr2=0x7f28e80cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.227+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 shutdown_connections 2024-04-05T22:25:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.227+0000 7f28ef800640 1 -- 172.21.15.73:0/3726612188 wait complete. 2024-04-05T22:25:27.246 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:25:27.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:25:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.454+0000 7fdcfb200640 1 Processor -- start 2024-04-05T22:25:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.454+0000 7fdcfb200640 1 -- start start 2024-04-05T22:25:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcfb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcfc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcfb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc06d610 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcfb200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc070230 con 0x7fdcfc0b2b90 2024-04-05T22:25:27.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcfb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc070510 con 0x7fdcfc0b1b50 2024-04-05T22:25:27.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcf9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdcfc0b20a0 0x7fdcfc0b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38090/0 (socket says 172.21.15.73:38090) 2024-04-05T22:25:27.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.455+0000 7fdcf9e00640 1 -- 172.21.15.73:0/4181034487 learned_addr learned my addr 172.21.15.73:0/4181034487 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846104752 0 0) 0x7fdcfc06d610 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcdc003520 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcfa800640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcfc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252016725 0 0) 0x7fdcdc003520 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 msgr2=0x7fdcfc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcfc070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 >> v1:172.21.15.157:6789/0 conn(0x7fdcfc0b2b90 legacy=0x7fdcfc0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0707f0 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fdcec001670 con 0x7fdcfc0b20a0 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 >> v1:172.21.15.73:6789/0 conn(0x7fdcfc0b20a0 legacy=0x7fdcfc0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcdc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.456+0000 7fdcf9400640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc003fd0 0x7fdcdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9400640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc004d50 0x7fdcdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc070510 con 0x7fdcdc004d50 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc00bc60 con 0x7fdcdc003fd0 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc00e030 con 0x7fdcfc0b1b50 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc004d50 0x7fdcdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9e00640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcdc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 -- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 msgr2=0x7fdcdc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcdc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 -- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc003fd0 msgr2=0x7fdcdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf1400640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc003fd0 0x7fdcdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc003fd0 0x7fdcdc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcfa800640 1 -- 172.21.15.73:0/4181034487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc003520 con 0x7fdcdc004d50 2024-04-05T22:25:27.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.457+0000 7fdcf9e00640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc0b1b50 0x7fdcdc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.458+0000 7fdcfa800640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc004d50 0x7fdcdc0051d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009fc0 tx=0x7fdcdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b549a6b31a287c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf402cc80 con 0x7fdcdc004d50 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf402ce00 con 0x7fdcdc004d50 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcf9400640 1 -- 172.21.15.73:0/4181034487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4034680 con 0x7fdcdc004d50 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 -- 172.21.15.73:0/4181034487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc004d50 msgr2=0x7fdcdc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 --2- 172.21.15.73:0/4181034487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc004d50 0x7fdcdc0051d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009fc0 tx=0x7fdcdc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfa800640 1 -- 172.21.15.73:0/4181034487 reap_dead start 2024-04-05T22:25:27.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 -- 172.21.15.73:0/4181034487 shutdown_connections 2024-04-05T22:25:27.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 -- 172.21.15.73:0/4181034487 >> 172.21.15.73:0/4181034487 conn(0x7fdcfc0b06c0 msgr2=0x7fdcfc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:27.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 -- 172.21.15.73:0/4181034487 shutdown_connections 2024-04-05T22:25:27.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.459+0000 7fdcfb200640 1 -- 172.21.15.73:0/4181034487 wait complete. 2024-04-05T22:25:27.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.461+0000 7fdcfb200640 1 Processor -- start 2024-04-05T22:25:27.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.461+0000 7fdcfb200640 1 -- start start 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc14b6b0 0x7fdcfc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1523e0 0x7fdcfc152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc152da0 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15bf00 con 0x7fdcfc14b6b0 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcfb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc159fa0 con 0x7fdcfc1523e0 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcf1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcf1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51416/0 (socket says 172.21.15.73:51416) 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.462+0000 7fdcf1400640 1 -- 172.21.15.73:0/840448865 learned_addr learned my addr 172.21.15.73:0/840448865 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf9e00640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc14b6b0 0x7fdcfc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcfa800640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1523e0 0x7fdcfc152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 -- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1523e0 msgr2=0x7fdcfc152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1523e0 0x7fdcfc152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 -- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc14b6b0 msgr2=0x7fdcfc15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc14b6b0 0x7fdcfc15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 -- 172.21.15.73:0/840448865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc15a280 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf1400640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdce400d570 tx=0x7fdce400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4005480 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4003610 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.463+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003980 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.464+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc15a560 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.464+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc15c1e0 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.465+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc15a0e0 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.466+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdce4003ae0 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.467+0000 7fdcf2a00640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 0x7fdcd806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.467+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdce4089700 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.467+0000 7fdcf9e00640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 0x7fdcd806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.468+0000 7fdcf9e00640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 0x7fdcd806d270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdcec001eb0 tx=0x7fdcec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.471+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdcfc15a0e0 con 0x7fdcfc15b3f0 2024-04-05T22:25:27.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.663+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcfc15a0e0 con 0x7fdcd806adb0 2024-04-05T22:25:27.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.690+0000 7fdcf2a00640 1 -- 172.21.15.73:0/840448865 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275820 (secure 0 0 0) 0x7fdcfc15a0e0 con 0x7fdcd806adb0 2024-04-05T22:25:27.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:27.696 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 msgr2=0x7fdcd806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 0x7fdcd806d270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdcec001eb0 tx=0x7fdcec009040 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 msgr2=0x7fdcfc15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdce400d570 tx=0x7fdce400da30 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 shutdown_connections 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcd806adb0 0x7fdcd806d270 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1523e0 0x7fdcfc152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc15b3f0 0x7fdcfc15b870 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 --2- 172.21.15.73:0/840448865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc14b6b0 0x7fdcfc15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 >> 172.21.15.73:0/840448865 conn(0x7fdcfc0b06c0 msgr2=0x7fdcfc0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:27.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 shutdown_connections 2024-04-05T22:25:27.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.696+0000 7fdcfb200640 1 -- 172.21.15.73:0/840448865 wait complete. 2024-04-05T22:25:27.716 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-04-05T22:25:26.575999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":247396,"kb_used_data":6652,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727324,"statfs":{"total":773094113280,"available":772840779776,"internally_reserved":0,"allocated":6811648,"data_stored":3493679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.807599"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779501+0000","last_change":"2024-04-05T22:25:21.779501+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779501+0000","last_undegraded":"2024-04-05T22:25:21.779501+0000","last_fullsized":"2024-04-05T22:25:21.779501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:54.998254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779341+0000","last_change":"2024-04-05T22:25:21.779341+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779341+0000","last_undegraded":"2024-04-05T22:25:21.779341+0000","last_fullsized":"2024-04-05T22:25:21.779341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:54.794155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778136+0000","last_change":"2024-04-05T22:25:21.778136+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778136+0000","last_undegraded":"2024-04-05T22:25:21.778136+0000","last_fullsized":"2024-04-05T22:25:21.778136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.784643+0000","last_change":"2024-04-05T22:25:21.784643+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.784643+0000","last_undegraded":"2024-04-05T22:25:21.784643+0000","last_fullsized":"2024-04-05T22:25:21.784643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.786024+0000","last_change":"2024-04-05T22:25:21.786024+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.786024+0000","last_undegraded":"2024-04-05T22:25:21.786024+0000","last_fullsized":"2024-04-05T22:25:21.786024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.777895+0000","last_change":"2024-04-05T22:25:21.777895+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.777895+0000","last_undegraded":"2024-04-05T22:25:21.777895+0000","last_fullsized":"2024-04-05T22:25:21.777895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.782478+0000","last_change":"2024-04-05T22:25:21.782478+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.782478+0000","last_undegraded":"2024-04-05T22:25:21.782478+0000","last_fullsized":"2024-04-05T22:25:21.782478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:47.138497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.783270+0000","last_change":"2024-04-05T22:25:21.783270+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.783270+0000","last_undegraded":"2024-04-05T22:25:21.783270+0000","last_fullsized":"2024-04-05T22:25:21.783270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.788295+0000","last_change":"2024-04-05T22:25:21.788295+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.788295+0000","last_undegraded":"2024-04-05T22:25:21.788295+0000","last_fullsized":"2024-04-05T22:25:21.788295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778799+0000","last_change":"2024-04-05T22:25:21.778799+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778799+0000","last_undegraded":"2024-04-05T22:25:21.778799+0000","last_fullsized":"2024-04-05T22:25:21.778799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:36.447234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.786866+0000","last_change":"2024-04-05T22:25:21.786866+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.786866+0000","last_undegraded":"2024-04-05T22:25:21.786866+0000","last_fullsized":"2024-04-05T22:25:21.786866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.787007+0000","last_change":"2024-04-05T22:25:21.787007+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.787007+0000","last_undegraded":"2024-04-05T22:25:21.787007+0000","last_fullsized":"2024-04-05T22:25:21.787007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:04.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.781201+0000","last_change":"2024-04-05T22:25:21.781201+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.781201+0000","last_undegraded":"2024-04-05T22:25:21.781201+0000","last_fullsized":"2024-04-05T22:25:21.781201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:36.497236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775715+0000","last_change":"2024-04-05T22:25:21.775715+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775715+0000","last_undegraded":"2024-04-05T22:25:21.775715+0000","last_fullsized":"2024-04-05T22:25:21.775715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:02.494750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780915+0000","last_change":"2024-04-05T22:25:21.780915+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780915+0000","last_undegraded":"2024-04-05T22:25:21.780915+0000","last_fullsized":"2024-04-05T22:25:21.780915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.785511+0000","last_change":"2024-04-05T22:25:21.785511+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.785511+0000","last_undegraded":"2024-04-05T22:25:21.785511+0000","last_fullsized":"2024-04-05T22:25:21.785511+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778945+0000","last_change":"2024-04-05T22:25:21.778945+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778945+0000","last_undegraded":"2024-04-05T22:25:21.778945+0000","last_fullsized":"2024-04-05T22:25:21.778945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:26.685831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779796+0000","last_change":"2024-04-05T22:25:21.779796+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779796+0000","last_undegraded":"2024-04-05T22:25:21.779796+0000","last_fullsized":"2024-04-05T22:25:21.779796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.781515+0000","last_change":"2024-04-05T22:25:21.781515+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.781515+0000","last_undegraded":"2024-04-05T22:25:21.781515+0000","last_fullsized":"2024-04-05T22:25:21.781515+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.777890+0000","last_change":"2024-04-05T22:25:21.777890+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.777890+0000","last_undegraded":"2024-04-05T22:25:21.777890+0000","last_fullsized":"2024-04-05T22:25:21.777890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780854+0000","last_change":"2024-04-05T22:25:21.780854+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780854+0000","last_undegraded":"2024-04-05T22:25:21.780854+0000","last_fullsized":"2024-04-05T22:25:21.780854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:19.370558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774031+0000","last_change":"2024-04-05T22:25:21.774031+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774031+0000","last_undegraded":"2024-04-05T22:25:21.774031+0000","last_fullsized":"2024-04-05T22:25:21.774031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780961+0000","last_change":"2024-04-05T22:25:21.780961+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780961+0000","last_undegraded":"2024-04-05T22:25:21.780961+0000","last_fullsized":"2024-04-05T22:25:21.780961+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774722+0000","last_change":"2024-04-05T22:25:21.774722+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774722+0000","last_undegraded":"2024-04-05T22:25:21.774722+0000","last_fullsized":"2024-04-05T22:25:21.774722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778707+0000","last_change":"2024-04-05T22:25:21.778707+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778707+0000","last_undegraded":"2024-04-05T22:25:21.778707+0000","last_fullsized":"2024-04-05T22:25:21.778707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775634+0000","last_change":"2024-04-05T22:25:21.775634+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775634+0000","last_undegraded":"2024-04-05T22:25:21.775634+0000","last_fullsized":"2024-04-05T22:25:21.775634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:13.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780991+0000","last_change":"2024-04-05T22:25:21.780991+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780991+0000","last_undegraded":"2024-04-05T22:25:21.780991+0000","last_fullsized":"2024-04-05T22:25:21.780991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774615+0000","last_change":"2024-04-05T22:25:21.774615+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774615+0000","last_undegraded":"2024-04-05T22:25:21.774615+0000","last_fullsized":"2024-04-05T22:25:21.774615+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.783603+0000","last_change":"2024-04-05T22:25:21.783603+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.783603+0000","last_undegraded":"2024-04-05T22:25:21.783603+0000","last_fullsized":"2024-04-05T22:25:21.783603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774146+0000","last_change":"2024-04-05T22:25:21.774146+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774146+0000","last_undegraded":"2024-04-05T22:25:21.774146+0000","last_fullsized":"2024-04-05T22:25:21.774146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779757+0000","last_change":"2024-04-05T22:25:21.779757+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779757+0000","last_undegraded":"2024-04-05T22:25:21.779757+0000","last_fullsized":"2024-04-05T22:25:21.779757+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:11.859438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775559+0000","last_change":"2024-04-05T22:25:21.775559+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775559+0000","last_undegraded":"2024-04-05T22:25:21.775559+0000","last_fullsized":"2024-04-05T22:25:21.775559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:59.775684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":196,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-05T22:25:20.519635+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:25:20.519635+0000","last_peered":"2024-04-05T22:25:20.519635+0000","last_clean":"2024-04-05T22:25:20.519635+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:25:20.519635+0000","last_undegraded":"2024-04-05T22:25:20.519635+0000","last_fullsized":"2024-04-05T22:25:20.519635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542198,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1298432,"data_stored":880213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.375}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.038},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.002}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.274}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.234}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.482},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.343}]}]},{"osd":6,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.044}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.099}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.3740000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.226},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.5589999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.4870000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.84599999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.702},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.595}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.252}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.8300000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":1.641},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.732}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.796},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.446}]}]},{"osd":4,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.012,"15min":0.98599999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.01},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.077,"15min":1.073},"min":{"1min":0.80800000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2889999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78800000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6259999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.278},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.151,"15min":1.169},"min":{"1min":0.872,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.552,"5min":1.974,"15min":1.974},"last":1.083},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1839999999999999,"15min":1.1819999999999999},"min":{"1min":0.80300000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3720000000000001,"15min":1.383},"min":{"1min":0.999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8799999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3779999999999999,"15min":1.4099999999999999},"min":{"1min":0.94399999999999995,"5min":0.874,"15min":0.874},"max":{"1min":1.9810000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.274,"15min":1.306},"min":{"1min":0.77200000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.462},{"interface":"front","average":{"1min":1.24,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.747,"5min":0.625,"15min":0.625},"max":{"1min":1.7629999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4179999999999999,"15min":1.474},"min":{"1min":0.69899999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.8100000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.468,"15min":1.504},"min":{"1min":0.82199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.569}]},{"osd":7,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5149999999999999,"15min":1.54},"min":{"1min":0.90400000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9450000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.91300000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.044,"5min":2.54,"15min":2.54},"last":1.641}]}]},{"osd":3,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":284028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.88}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.752},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.363}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.073}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2889999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.522}]}]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27708,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.448}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.395}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.258},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.087},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.161}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.458},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.415},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":1.696},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.208}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.494}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.5449999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1298432,"data_stored":880213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":1.252},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":1.012},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.331},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.847}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2869999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.734},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.363}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:25:27.721 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:25:27.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.970+0000 7f2723000640 1 Processor -- start 2024-04-05T22:25:27.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.970+0000 7f2723000640 1 -- start start 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f271c15d120 0x7f271c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06ede0 0x7f271c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f271c06e920 0x7f271c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c15d500 con 0x7f271c06e920 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c0b4ac0 con 0x7f271c15d120 2024-04-05T22:25:27.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2723000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c06bbc0 con 0x7f271c06ede0 2024-04-05T22:25:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2721000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f271c06e920 0x7f271c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2721000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f271c06e920 0x7f271c0b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51422/0 (socket says 172.21.15.73:51422) 2024-04-05T22:25:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2721000640 1 -- 172.21.15.73:0/52646377 learned_addr learned my addr 172.21.15.73:0/52646377 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f271be00640 1 --2- 172.21.15.73:0/52646377 >> v2:172.21.15.157:3300/0 conn(0x7f271c15d120 0x7f271c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f271b400640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06ede0 0x7f271c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.972+0000 7f2721000640 1 -- 172.21.15.73:0/52646377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06ede0 msgr2=0x7f271c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.973+0000 7f2721000640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06ede0 0x7f271c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.973+0000 7f2721000640 1 -- 172.21.15.73:0/52646377 >> v2:172.21.15.157:3300/0 conn(0x7f271c15d120 msgr2=0x7f271c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.973+0000 7f2721000640 1 --2- 172.21.15.73:0/52646377 >> v2:172.21.15.157:3300/0 conn(0x7f271c15d120 0x7f271c071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.973+0000 7f2721000640 1 -- 172.21.15.73:0/52646377 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c06b8e0 con 0x7f271c06e920 2024-04-05T22:25:27.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.974+0000 7f2721000640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c06e920 0x7f271c0b4580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f271c04e2e0 tx=0x7f271402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2ebfaf9b9b3aef server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.974+0000 7f271aa00640 1 -- 172.21.15.73:0/52646377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271402ca70 con 0x7f271c06e920 2024-04-05T22:25:27.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.974+0000 7f271aa00640 1 -- 172.21.15.73:0/52646377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271402cbd0 con 0x7f271c06e920 2024-04-05T22:25:27.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.974+0000 7f271aa00640 1 -- 172.21.15.73:0/52646377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27140343f0 con 0x7f271c06e920 2024-04-05T22:25:27.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 -- 172.21.15.73:0/52646377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c06e920 msgr2=0x7f271c0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c06e920 0x7f271c0b4580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f271c04e2e0 tx=0x7f271402bed0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 -- 172.21.15.73:0/52646377 shutdown_connections 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c06e920 0x7f271c0b4580 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 --2- 172.21.15.73:0/52646377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06ede0 0x7f271c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 --2- 172.21.15.73:0/52646377 >> v2:172.21.15.157:3300/0 conn(0x7f271c15d120 0x7f271c071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 -- 172.21.15.73:0/52646377 >> 172.21.15.73:0/52646377 conn(0x7f271c06adc0 msgr2=0x7f271c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 -- 172.21.15.73:0/52646377 shutdown_connections 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.975+0000 7f2723000640 1 -- 172.21.15.73:0/52646377 wait complete. 2024-04-05T22:25:27.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.976+0000 7f2723000640 1 Processor -- start 2024-04-05T22:25:27.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.977+0000 7f2723000640 1 -- start start 2024-04-05T22:25:27.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 0x7f271c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c15d120 0x7f271c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c1502e0 con 0x7f271c14e890 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0cb8a0 con 0x7f271c06e920 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2723000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c14f540 con 0x7f271c15d120 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51430/0 (socket says 172.21.15.73:51430) 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 -- 172.21.15.73:0/738364242 learned_addr learned my addr 172.21.15.73:0/738364242 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f271be00640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 0x7f271c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f271b400640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c15d120 0x7f271c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 -- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c15d120 msgr2=0x7f271c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c15d120 0x7f271c15bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 -- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 msgr2=0x7f271c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 0x7f271c15b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:27.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f2721000640 1 -- 172.21.15.73:0/738364242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714009b30 con 0x7f271c14e890 2024-04-05T22:25:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.978+0000 7f271be00640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 0x7f271c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2721000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2714036040 tx=0x7f2714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714009510 con 0x7f271c14e890 2024-04-05T22:25:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271403e450 con 0x7f271c14e890 2024-04-05T22:25:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271403e7e0 con 0x7f271c14e890 2024-04-05T22:25:27.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c14f820 con 0x7f271c14e890 2024-04-05T22:25:27.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.979+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c14fde0 con 0x7f271c14e890 2024-04-05T22:25:27.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.980+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c066af0 con 0x7f271c14e890 2024-04-05T22:25:27.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.981+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f271403e940 con 0x7f271c14e890 2024-04-05T22:25:27.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.982+0000 7f2713e00640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 0x7f26fc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:27.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.982+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27140ac760 con 0x7f271c14e890 2024-04-05T22:25:27.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.982+0000 7f271be00640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 0x7f26fc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:27.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.983+0000 7f271be00640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 0x7f26fc06d270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:27.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:27.987+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f27140777e0 con 0x7f271c14e890 2024-04-05T22:25:28.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.175+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f271c070d20 con 0x7f26fc06adb0 2024-04-05T22:25:28.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.197+0000 7f2713e00640 1 -- 172.21.15.73:0/738364242 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275820 (secure 0 0 0) 0x7f270c009530 con 0x7f26fc06adb0 2024-04-05T22:25:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:28.203 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.204+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 msgr2=0x7f26fc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.204+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 0x7f26fc06d270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.204+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 msgr2=0x7f271c14ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.204+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2714036040 tx=0x7f2714004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 shutdown_connections 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f26fc06adb0 0x7f26fc06d270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271c14e890 0x7f271c14ed10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c15d120 0x7f271c15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 --2- 172.21.15.73:0/738364242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c06e920 0x7f271c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 >> 172.21.15.73:0/738364242 conn(0x7f271c06adc0 msgr2=0x7f271c06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:28.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 shutdown_connections 2024-04-05T22:25:28.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.205+0000 7f2723000640 1 -- 172.21.15.73:0/738364242 wait complete. 2024-04-05T22:25:28.224 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-04-05T22:25:26.575999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":247396,"kb_used_data":6652,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727324,"statfs":{"total":773094113280,"available":772840779776,"internally_reserved":0,"allocated":6811648,"data_stored":3493679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.807599"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779501+0000","last_change":"2024-04-05T22:25:21.779501+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779501+0000","last_undegraded":"2024-04-05T22:25:21.779501+0000","last_fullsized":"2024-04-05T22:25:21.779501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:54.998254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779341+0000","last_change":"2024-04-05T22:25:21.779341+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779341+0000","last_undegraded":"2024-04-05T22:25:21.779341+0000","last_fullsized":"2024-04-05T22:25:21.779341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:54.794155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778136+0000","last_change":"2024-04-05T22:25:21.778136+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778136+0000","last_undegraded":"2024-04-05T22:25:21.778136+0000","last_fullsized":"2024-04-05T22:25:21.778136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.784643+0000","last_change":"2024-04-05T22:25:21.784643+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.784643+0000","last_undegraded":"2024-04-05T22:25:21.784643+0000","last_fullsized":"2024-04-05T22:25:21.784643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.786024+0000","last_change":"2024-04-05T22:25:21.786024+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.786024+0000","last_undegraded":"2024-04-05T22:25:21.786024+0000","last_fullsized":"2024-04-05T22:25:21.786024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.777895+0000","last_change":"2024-04-05T22:25:21.777895+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.777895+0000","last_undegraded":"2024-04-05T22:25:21.777895+0000","last_fullsized":"2024-04-05T22:25:21.777895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.782478+0000","last_change":"2024-04-05T22:25:21.782478+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.782478+0000","last_undegraded":"2024-04-05T22:25:21.782478+0000","last_fullsized":"2024-04-05T22:25:21.782478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:47.138497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.783270+0000","last_change":"2024-04-05T22:25:21.783270+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.783270+0000","last_undegraded":"2024-04-05T22:25:21.783270+0000","last_fullsized":"2024-04-05T22:25:21.783270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.788295+0000","last_change":"2024-04-05T22:25:21.788295+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.788295+0000","last_undegraded":"2024-04-05T22:25:21.788295+0000","last_fullsized":"2024-04-05T22:25:21.788295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778799+0000","last_change":"2024-04-05T22:25:21.778799+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778799+0000","last_undegraded":"2024-04-05T22:25:21.778799+0000","last_fullsized":"2024-04-05T22:25:21.778799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:36.447234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.786866+0000","last_change":"2024-04-05T22:25:21.786866+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.786866+0000","last_undegraded":"2024-04-05T22:25:21.786866+0000","last_fullsized":"2024-04-05T22:25:21.786866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.787007+0000","last_change":"2024-04-05T22:25:21.787007+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.787007+0000","last_undegraded":"2024-04-05T22:25:21.787007+0000","last_fullsized":"2024-04-05T22:25:21.787007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:04.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.781201+0000","last_change":"2024-04-05T22:25:21.781201+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.781201+0000","last_undegraded":"2024-04-05T22:25:21.781201+0000","last_fullsized":"2024-04-05T22:25:21.781201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:36.497236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775715+0000","last_change":"2024-04-05T22:25:21.775715+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775715+0000","last_undegraded":"2024-04-05T22:25:21.775715+0000","last_fullsized":"2024-04-05T22:25:21.775715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:02.494750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780915+0000","last_change":"2024-04-05T22:25:21.780915+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780915+0000","last_undegraded":"2024-04-05T22:25:21.780915+0000","last_fullsized":"2024-04-05T22:25:21.780915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.785511+0000","last_change":"2024-04-05T22:25:21.785511+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.785511+0000","last_undegraded":"2024-04-05T22:25:21.785511+0000","last_fullsized":"2024-04-05T22:25:21.785511+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778945+0000","last_change":"2024-04-05T22:25:21.778945+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778945+0000","last_undegraded":"2024-04-05T22:25:21.778945+0000","last_fullsized":"2024-04-05T22:25:21.778945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:26.685831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779796+0000","last_change":"2024-04-05T22:25:21.779796+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779796+0000","last_undegraded":"2024-04-05T22:25:21.779796+0000","last_fullsized":"2024-04-05T22:25:21.779796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.781515+0000","last_change":"2024-04-05T22:25:21.781515+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.781515+0000","last_undegraded":"2024-04-05T22:25:21.781515+0000","last_fullsized":"2024-04-05T22:25:21.781515+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.777890+0000","last_change":"2024-04-05T22:25:21.777890+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.777890+0000","last_undegraded":"2024-04-05T22:25:21.777890+0000","last_fullsized":"2024-04-05T22:25:21.777890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780854+0000","last_change":"2024-04-05T22:25:21.780854+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780854+0000","last_undegraded":"2024-04-05T22:25:21.780854+0000","last_fullsized":"2024-04-05T22:25:21.780854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:19.370558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774031+0000","last_change":"2024-04-05T22:25:21.774031+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774031+0000","last_undegraded":"2024-04-05T22:25:21.774031+0000","last_fullsized":"2024-04-05T22:25:21.774031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780961+0000","last_change":"2024-04-05T22:25:21.780961+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780961+0000","last_undegraded":"2024-04-05T22:25:21.780961+0000","last_fullsized":"2024-04-05T22:25:21.780961+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774722+0000","last_change":"2024-04-05T22:25:21.774722+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774722+0000","last_undegraded":"2024-04-05T22:25:21.774722+0000","last_fullsized":"2024-04-05T22:25:21.774722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.778707+0000","last_change":"2024-04-05T22:25:21.778707+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.778707+0000","last_undegraded":"2024-04-05T22:25:21.778707+0000","last_fullsized":"2024-04-05T22:25:21.778707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775634+0000","last_change":"2024-04-05T22:25:21.775634+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775634+0000","last_undegraded":"2024-04-05T22:25:21.775634+0000","last_fullsized":"2024-04-05T22:25:21.775634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:13.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.780991+0000","last_change":"2024-04-05T22:25:21.780991+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.780991+0000","last_undegraded":"2024-04-05T22:25:21.780991+0000","last_fullsized":"2024-04-05T22:25:21.780991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774615+0000","last_change":"2024-04-05T22:25:21.774615+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774615+0000","last_undegraded":"2024-04-05T22:25:21.774615+0000","last_fullsized":"2024-04-05T22:25:21.774615+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.783603+0000","last_change":"2024-04-05T22:25:21.783603+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.783603+0000","last_undegraded":"2024-04-05T22:25:21.783603+0000","last_fullsized":"2024-04-05T22:25:21.783603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.774146+0000","last_change":"2024-04-05T22:25:21.774146+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.774146+0000","last_undegraded":"2024-04-05T22:25:21.774146+0000","last_fullsized":"2024-04-05T22:25:21.774146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.779757+0000","last_change":"2024-04-05T22:25:21.779757+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.779757+0000","last_undegraded":"2024-04-05T22:25:21.779757+0000","last_fullsized":"2024-04-05T22:25:21.779757+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:11.859438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:25:22.768595+0000","last_change":"2024-04-05T22:25:22.768595+0000","last_active":"2024-04-05T22:25:22.768595+0000","last_peered":"2024-04-05T22:25:22.768595+0000","last_clean":"2024-04-05T22:25:22.768595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:22.768595+0000","last_undegraded":"2024-04-05T22:25:22.768595+0000","last_fullsized":"2024-04-05T22:25:22.768595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-05T22:25:21.775559+0000","last_change":"2024-04-05T22:25:21.775559+0000","last_active":"2024-04-05T22:25:21.753939+0000","last_peered":"2024-04-05T22:25:21.753939+0000","last_clean":"2024-04-05T22:25:21.753939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:25:21.775559+0000","last_undegraded":"2024-04-05T22:25:21.775559+0000","last_fullsized":"2024-04-05T22:25:21.775559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:59.775684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":196,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-05T22:25:20.519635+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:25:20.519635+0000","last_peered":"2024-04-05T22:25:20.519635+0000","last_clean":"2024-04-05T22:25:20.519635+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:25:20.519635+0000","last_undegraded":"2024-04-05T22:25:20.519635+0000","last_fullsized":"2024-04-05T22:25:20.519635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542198,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1298432,"data_stored":880213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.375}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.038},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.002}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.274}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.234}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.482},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.343}]}]},{"osd":6,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.044}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.099}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.3740000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.226},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.5589999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.4870000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.84599999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.702},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.595}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.252}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.8300000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":1.641},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.732}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.796},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.446}]}]},{"osd":4,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.012,"15min":0.98599999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.01},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.077,"15min":1.073},"min":{"1min":0.80800000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2889999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78800000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6259999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.278},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.151,"15min":1.169},"min":{"1min":0.872,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.552,"5min":1.974,"15min":1.974},"last":1.083},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1839999999999999,"15min":1.1819999999999999},"min":{"1min":0.80300000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3720000000000001,"15min":1.383},"min":{"1min":0.999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8799999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3779999999999999,"15min":1.4099999999999999},"min":{"1min":0.94399999999999995,"5min":0.874,"15min":0.874},"max":{"1min":1.9810000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.274,"15min":1.306},"min":{"1min":0.77200000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.462},{"interface":"front","average":{"1min":1.24,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.747,"5min":0.625,"15min":0.625},"max":{"1min":1.7629999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4179999999999999,"15min":1.474},"min":{"1min":0.69899999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.8100000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.468,"15min":1.504},"min":{"1min":0.82199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.569}]},{"osd":7,"last update":"Fri Apr 5 22:24:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5149999999999999,"15min":1.54},"min":{"1min":0.90400000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9450000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.91300000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.044,"5min":2.54,"15min":2.54},"last":1.641}]}]},{"osd":3,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":284028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.88}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.752},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.363}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.073}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2889999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.522}]}]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27708,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.448}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.395}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.258},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":289845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.087},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.161}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.458},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.415},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":1.696},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.208}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.494}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.5449999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1298432,"data_stored":880213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":1.252},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":1.012},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.331},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.847}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2869999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.734},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.363}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:25:28.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:25:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 Processor -- start 2024-04-05T22:25:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 -- start start 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f28300b23e0 0x7f283006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f283006c2d0 0x7f283006fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283006ca50 con 0x7f283013f3b0 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283006f920 con 0x7f28300b23e0 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.476+0000 7f2835200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283006cd30 con 0x7f283006c2d0 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f28300b23e0 0x7f283006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f28300b23e0 0x7f283006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51434/0 (socket says 172.21.15.73:51434) 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 -- 172.21.15.73:0/2798522127 learned_addr learned my addr 172.21.15.73:0/2798522127 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 -- 172.21.15.73:0/2798522127 >> v2:172.21.15.157:3300/0 conn(0x7f283006c2d0 msgr2=0x7f283006fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:28.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 --2- 172.21.15.73:0/2798522127 >> v2:172.21.15.157:3300/0 conn(0x7f283006c2d0 0x7f283006fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 -- 172.21.15.73:0/2798522127 >> v1:172.21.15.157:6789/0 conn(0x7f283013f3b0 legacy=0x7f28300b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:28.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.477+0000 7f282ea00640 1 -- 172.21.15.73:0/2798522127 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283006d010 con 0x7f28300b23e0 2024-04-05T22:25:28.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.478+0000 7f282ea00640 1 --2- 172.21.15.73:0/2798522127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28300b23e0 0x7f283006bd90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f28300b3a40 tx=0x7f281c02cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450c1db428b4ff74 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.478+0000 7f282e000640 1 -- 172.21.15.73:0/2798522127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c02db00 con 0x7f28300b23e0 2024-04-05T22:25:28.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f282e000640 1 -- 172.21.15.73:0/2798522127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c02dc60 con 0x7f28300b23e0 2024-04-05T22:25:28.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f282e000640 1 -- 172.21.15.73:0/2798522127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c035440 con 0x7f28300b23e0 2024-04-05T22:25:28.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 -- 172.21.15.73:0/2798522127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28300b23e0 msgr2=0x7f283006bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 --2- 172.21.15.73:0/2798522127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28300b23e0 0x7f283006bd90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f28300b3a40 tx=0x7f281c02cd50 comp rx=0 tx=0).stop 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 -- 172.21.15.73:0/2798522127 shutdown_connections 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 --2- 172.21.15.73:0/2798522127 >> v2:172.21.15.157:3300/0 conn(0x7f283006c2d0 0x7f283006fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 --2- 172.21.15.73:0/2798522127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28300b23e0 0x7f283006bd90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 -- 172.21.15.73:0/2798522127 >> 172.21.15.73:0/2798522127 conn(0x7f28300b06c0 msgr2=0x7f28300b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 -- 172.21.15.73:0/2798522127 shutdown_connections 2024-04-05T22:25:28.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.479+0000 7f2835200640 1 -- 172.21.15.73:0/2798522127 wait complete. 2024-04-05T22:25:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.481+0000 7f2835200640 1 Processor -- start 2024-04-05T22:25:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.481+0000 7f2835200640 1 -- start start 2024-04-05T22:25:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f283006c2d0 0x7f28301560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 0x7f2830155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830155cb0 con 0x7f283013f3b0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830156740 con 0x7f28300b23e0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f2835200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2830156a20 con 0x7f283006c2d0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f282fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f282fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51446/0 (socket says 172.21.15.73:51446) 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.482+0000 7f282fe00640 1 -- 172.21.15.73:0/2523183197 learned_addr learned my addr 172.21.15.73:0/2523183197 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282ea00640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 0x7f2830155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282f400640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f283006c2d0 0x7f28301560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 -- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f283006c2d0 msgr2=0x7f28301560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f283006c2d0 0x7f28301560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 -- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 msgr2=0x7f2830155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 0x7f2830155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 -- 172.21.15.73:0/2523183197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c00a670 con 0x7f283013f3b0 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282ea00640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 0x7f2830155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f282fe00640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f282800b970 tx=0x7f282800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800cb40 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800ccc0 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2830156d00 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.483+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828014680 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.484+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28301572c0 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.485+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28301574e0 con 0x7f283013f3b0 2024-04-05T22:25:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.486+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f282801d3b0 con 0x7f283013f3b0 2024-04-05T22:25:28.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.487+0000 7f2827400640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 0x7f281006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.487+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28280895a0 con 0x7f283013f3b0 2024-04-05T22:25:28.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.487+0000 7f282f400640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 0x7f281006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.488+0000 7f282f400640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 0x7f281006d270 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.491+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2828054570 con 0x7f283013f3b0 2024-04-05T22:25:28.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.682+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f283013f790 con 0x7f281006adb0 2024-04-05T22:25:28.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.695+0000 7f2827400640 1 -- 172.21.15.73:0/2523183197 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286681 (secure 0 0 0) 0x7f282000be30 con 0x7f281006adb0 2024-04-05T22:25:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:28.701 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 msgr2=0x7f281006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 0x7f281006d270 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820008040 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 msgr2=0x7f283013e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f282800b970 tx=0x7f282800be30 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 shutdown_connections 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f281006adb0 0x7f281006d270 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f283013f3b0 0x7f283013e200 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28300b23e0 0x7f2830155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 --2- 172.21.15.73:0/2523183197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f283006c2d0 0x7f28301560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 >> 172.21.15.73:0/2523183197 conn(0x7f28300b06c0 msgr2=0x7f28300aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.702+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 shutdown_connections 2024-04-05T22:25:28.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.703+0000 7f2835200640 1 -- 172.21.15.73:0/2523183197 wait complete. 2024-04-05T22:25:28.722 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-04-05T22:25:28.576483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251892,"kb_used_data":7016,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754722828,"statfs":{"total":773094113280,"available":772836175872,"internally_reserved":0,"allocated":7184384,"data_stored":3658956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":250596923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290720+0000","last_change":"2024-04-05T22:25:23.299156+0000","last_active":"2024-04-05T22:25:24.290720+0000","last_peered":"2024-04-05T22:25:24.290720+0000","last_clean":"2024-04-05T22:25:24.290720+0000","last_became_active":"2024-04-05T22:25:23.298427+0000","last_became_peered":"2024-04-05T22:25:23.298427+0000","last_unstale":"2024-04-05T22:25:24.290720+0000","last_undegraded":"2024-04-05T22:25:24.290720+0000","last_fullsized":"2024-04-05T22:25:24.290720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:45:44.913900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308802+0000","last_change":"2024-04-05T22:25:23.291712+0000","last_active":"2024-04-05T22:25:24.308802+0000","last_peered":"2024-04-05T22:25:24.308802+0000","last_clean":"2024-04-05T22:25:24.308802+0000","last_became_active":"2024-04-05T22:25:23.290786+0000","last_became_peered":"2024-04-05T22:25:23.290786+0000","last_unstale":"2024-04-05T22:25:24.308802+0000","last_undegraded":"2024-04-05T22:25:24.308802+0000","last_fullsized":"2024-04-05T22:25:24.308802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:02:03.184140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312076+0000","last_change":"2024-04-05T22:25:23.322464+0000","last_active":"2024-04-05T22:25:24.312076+0000","last_peered":"2024-04-05T22:25:24.312076+0000","last_clean":"2024-04-05T22:25:24.312076+0000","last_became_active":"2024-04-05T22:25:23.316842+0000","last_became_peered":"2024-04-05T22:25:23.316842+0000","last_unstale":"2024-04-05T22:25:24.312076+0000","last_undegraded":"2024-04-05T22:25:24.312076+0000","last_fullsized":"2024-04-05T22:25:24.312076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:12:17.928701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312038+0000","last_change":"2024-04-05T22:25:23.324761+0000","last_active":"2024-04-05T22:25:24.312038+0000","last_peered":"2024-04-05T22:25:24.312038+0000","last_clean":"2024-04-05T22:25:24.312038+0000","last_became_active":"2024-04-05T22:25:23.323869+0000","last_became_peered":"2024-04-05T22:25:23.323869+0000","last_unstale":"2024-04-05T22:25:24.312038+0000","last_undegraded":"2024-04-05T22:25:24.312038+0000","last_fullsized":"2024-04-05T22:25:24.312038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290172+0000","last_change":"2024-04-05T22:25:23.318738+0000","last_active":"2024-04-05T22:25:24.290172+0000","last_peered":"2024-04-05T22:25:24.290172+0000","last_clean":"2024-04-05T22:25:24.290172+0000","last_became_active":"2024-04-05T22:25:23.318429+0000","last_became_peered":"2024-04-05T22:25:23.318429+0000","last_unstale":"2024-04-05T22:25:24.290172+0000","last_undegraded":"2024-04-05T22:25:24.290172+0000","last_fullsized":"2024-04-05T22:25:24.290172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:40.840026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307914+0000","last_change":"2024-04-05T22:25:23.296999+0000","last_active":"2024-04-05T22:25:24.307914+0000","last_peered":"2024-04-05T22:25:24.307914+0000","last_clean":"2024-04-05T22:25:24.307914+0000","last_became_active":"2024-04-05T22:25:23.296625+0000","last_became_peered":"2024-04-05T22:25:23.296625+0000","last_unstale":"2024-04-05T22:25:24.307914+0000","last_undegraded":"2024-04-05T22:25:24.307914+0000","last_fullsized":"2024-04-05T22:25:24.307914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293145+0000","last_change":"2024-04-05T22:25:23.312117+0000","last_active":"2024-04-05T22:25:24.293145+0000","last_peered":"2024-04-05T22:25:24.293145+0000","last_clean":"2024-04-05T22:25:24.293145+0000","last_became_active":"2024-04-05T22:25:23.310600+0000","last_became_peered":"2024-04-05T22:25:23.310600+0000","last_unstale":"2024-04-05T22:25:24.293145+0000","last_undegraded":"2024-04-05T22:25:24.293145+0000","last_fullsized":"2024-04-05T22:25:24.293145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:43.287781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306787+0000","last_change":"2024-04-05T22:25:23.296496+0000","last_active":"2024-04-05T22:25:24.306787+0000","last_peered":"2024-04-05T22:25:24.306787+0000","last_clean":"2024-04-05T22:25:24.306787+0000","last_became_active":"2024-04-05T22:25:23.296222+0000","last_became_peered":"2024-04-05T22:25:23.296222+0000","last_unstale":"2024-04-05T22:25:24.306787+0000","last_undegraded":"2024-04-05T22:25:24.306787+0000","last_fullsized":"2024-04-05T22:25:24.306787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:36:43.010960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294145+0000","last_change":"2024-04-05T22:25:23.303841+0000","last_active":"2024-04-05T22:25:24.294145+0000","last_peered":"2024-04-05T22:25:24.294145+0000","last_clean":"2024-04-05T22:25:24.294145+0000","last_became_active":"2024-04-05T22:25:23.301443+0000","last_became_peered":"2024-04-05T22:25:23.301443+0000","last_unstale":"2024-04-05T22:25:24.294145+0000","last_undegraded":"2024-04-05T22:25:24.294145+0000","last_fullsized":"2024-04-05T22:25:24.294145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291335+0000","last_change":"2024-04-05T22:25:23.314986+0000","last_active":"2024-04-05T22:25:24.291335+0000","last_peered":"2024-04-05T22:25:24.291335+0000","last_clean":"2024-04-05T22:25:24.291335+0000","last_became_active":"2024-04-05T22:25:23.314711+0000","last_became_peered":"2024-04-05T22:25:23.314711+0000","last_unstale":"2024-04-05T22:25:24.291335+0000","last_undegraded":"2024-04-05T22:25:24.291335+0000","last_fullsized":"2024-04-05T22:25:24.291335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291109+0000","last_change":"2024-04-05T22:25:23.290822+0000","last_active":"2024-04-05T22:25:24.291109+0000","last_peered":"2024-04-05T22:25:24.291109+0000","last_clean":"2024-04-05T22:25:24.291109+0000","last_became_active":"2024-04-05T22:25:23.290573+0000","last_became_peered":"2024-04-05T22:25:23.290573+0000","last_unstale":"2024-04-05T22:25:24.291109+0000","last_undegraded":"2024-04-05T22:25:24.291109+0000","last_fullsized":"2024-04-05T22:25:24.291109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312290+0000","last_change":"2024-04-05T22:25:23.314201+0000","last_active":"2024-04-05T22:25:24.312290+0000","last_peered":"2024-04-05T22:25:24.312290+0000","last_clean":"2024-04-05T22:25:24.312290+0000","last_became_active":"2024-04-05T22:25:23.307100+0000","last_became_peered":"2024-04-05T22:25:23.307100+0000","last_unstale":"2024-04-05T22:25:24.312290+0000","last_undegraded":"2024-04-05T22:25:24.312290+0000","last_fullsized":"2024-04-05T22:25:24.312290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309603+0000","last_change":"2024-04-05T22:25:23.315571+0000","last_active":"2024-04-05T22:25:24.309603+0000","last_peered":"2024-04-05T22:25:24.309603+0000","last_clean":"2024-04-05T22:25:24.309603+0000","last_became_active":"2024-04-05T22:25:23.315249+0000","last_became_peered":"2024-04-05T22:25:23.315249+0000","last_unstale":"2024-04-05T22:25:24.309603+0000","last_undegraded":"2024-04-05T22:25:24.309603+0000","last_fullsized":"2024-04-05T22:25:24.309603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:41.924668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293201+0000","last_change":"2024-04-05T22:25:23.303074+0000","last_active":"2024-04-05T22:25:24.293201+0000","last_peered":"2024-04-05T22:25:24.293201+0000","last_clean":"2024-04-05T22:25:24.293201+0000","last_became_active":"2024-04-05T22:25:23.302615+0000","last_became_peered":"2024-04-05T22:25:23.302615+0000","last_unstale":"2024-04-05T22:25:24.293201+0000","last_undegraded":"2024-04-05T22:25:24.293201+0000","last_fullsized":"2024-04-05T22:25:24.293201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:56.012910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293307+0000","last_change":"2024-04-05T22:25:23.297181+0000","last_active":"2024-04-05T22:25:24.293307+0000","last_peered":"2024-04-05T22:25:24.293307+0000","last_clean":"2024-04-05T22:25:24.293307+0000","last_became_active":"2024-04-05T22:25:23.293820+0000","last_became_peered":"2024-04-05T22:25:23.293820+0000","last_unstale":"2024-04-05T22:25:24.293307+0000","last_undegraded":"2024-04-05T22:25:24.293307+0000","last_fullsized":"2024-04-05T22:25:24.293307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307255+0000","last_change":"2024-04-05T22:25:23.313239+0000","last_active":"2024-04-05T22:25:24.307255+0000","last_peered":"2024-04-05T22:25:24.307255+0000","last_clean":"2024-04-05T22:25:24.307255+0000","last_became_active":"2024-04-05T22:25:23.312991+0000","last_became_peered":"2024-04-05T22:25:23.312991+0000","last_unstale":"2024-04-05T22:25:24.307255+0000","last_undegraded":"2024-04-05T22:25:24.307255+0000","last_fullsized":"2024-04-05T22:25:24.307255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:55.808811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308785+0000","last_change":"2024-04-05T22:25:23.317075+0000","last_active":"2024-04-05T22:25:24.308785+0000","last_peered":"2024-04-05T22:25:24.308785+0000","last_clean":"2024-04-05T22:25:24.308785+0000","last_became_active":"2024-04-05T22:25:23.314962+0000","last_became_peered":"2024-04-05T22:25:23.314962+0000","last_unstale":"2024-04-05T22:25:24.308785+0000","last_undegraded":"2024-04-05T22:25:24.308785+0000","last_fullsized":"2024-04-05T22:25:24.308785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:48.490866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311239+0000","last_change":"2024-04-05T22:25:23.318987+0000","last_active":"2024-04-05T22:25:24.311239+0000","last_peered":"2024-04-05T22:25:24.311239+0000","last_clean":"2024-04-05T22:25:24.311239+0000","last_became_active":"2024-04-05T22:25:23.317905+0000","last_became_peered":"2024-04-05T22:25:23.317905+0000","last_unstale":"2024-04-05T22:25:24.311239+0000","last_undegraded":"2024-04-05T22:25:24.311239+0000","last_fullsized":"2024-04-05T22:25:24.311239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:03:19.914501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290417+0000","last_change":"2024-04-05T22:25:23.297860+0000","last_active":"2024-04-05T22:25:24.290417+0000","last_peered":"2024-04-05T22:25:24.290417+0000","last_clean":"2024-04-05T22:25:24.290417+0000","last_became_active":"2024-04-05T22:25:23.297615+0000","last_became_peered":"2024-04-05T22:25:23.297615+0000","last_unstale":"2024-04-05T22:25:24.290417+0000","last_undegraded":"2024-04-05T22:25:24.290417+0000","last_fullsized":"2024-04-05T22:25:24.290417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:19:55.287884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308585+0000","last_change":"2024-04-05T22:25:23.294183+0000","last_active":"2024-04-05T22:25:24.308585+0000","last_peered":"2024-04-05T22:25:24.308585+0000","last_clean":"2024-04-05T22:25:24.308585+0000","last_became_active":"2024-04-05T22:25:23.293894+0000","last_became_peered":"2024-04-05T22:25:23.293894+0000","last_unstale":"2024-04-05T22:25:24.308585+0000","last_undegraded":"2024-04-05T22:25:24.308585+0000","last_fullsized":"2024-04-05T22:25:24.308585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:06:40.304684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311574+0000","last_change":"2024-04-05T22:25:23.323578+0000","last_active":"2024-04-05T22:25:24.311574+0000","last_peered":"2024-04-05T22:25:24.311574+0000","last_clean":"2024-04-05T22:25:24.311574+0000","last_became_active":"2024-04-05T22:25:23.323250+0000","last_became_peered":"2024-04-05T22:25:23.323250+0000","last_unstale":"2024-04-05T22:25:24.311574+0000","last_undegraded":"2024-04-05T22:25:24.311574+0000","last_fullsized":"2024-04-05T22:25:24.311574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308475+0000","last_change":"2024-04-05T22:25:23.297418+0000","last_active":"2024-04-05T22:25:24.308475+0000","last_peered":"2024-04-05T22:25:24.308475+0000","last_clean":"2024-04-05T22:25:24.308475+0000","last_became_active":"2024-04-05T22:25:23.297075+0000","last_became_peered":"2024-04-05T22:25:23.297075+0000","last_unstale":"2024-04-05T22:25:24.308475+0000","last_undegraded":"2024-04-05T22:25:24.308475+0000","last_fullsized":"2024-04-05T22:25:24.308475+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:45.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310394+0000","last_change":"2024-04-05T22:25:23.320793+0000","last_active":"2024-04-05T22:25:24.310394+0000","last_peered":"2024-04-05T22:25:24.310394+0000","last_clean":"2024-04-05T22:25:24.310394+0000","last_became_active":"2024-04-05T22:25:23.316015+0000","last_became_peered":"2024-04-05T22:25:23.316015+0000","last_unstale":"2024-04-05T22:25:24.310394+0000","last_undegraded":"2024-04-05T22:25:24.310394+0000","last_fullsized":"2024-04-05T22:25:24.310394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309223+0000","last_change":"2024-04-05T22:25:23.315728+0000","last_active":"2024-04-05T22:25:24.309223+0000","last_peered":"2024-04-05T22:25:24.309223+0000","last_clean":"2024-04-05T22:25:24.309223+0000","last_became_active":"2024-04-05T22:25:23.314345+0000","last_became_peered":"2024-04-05T22:25:23.314345+0000","last_unstale":"2024-04-05T22:25:24.309223+0000","last_undegraded":"2024-04-05T22:25:24.309223+0000","last_fullsized":"2024-04-05T22:25:24.309223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:50.046379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290866+0000","last_change":"2024-04-05T22:25:23.302044+0000","last_active":"2024-04-05T22:25:24.290866+0000","last_peered":"2024-04-05T22:25:24.290866+0000","last_clean":"2024-04-05T22:25:24.290866+0000","last_became_active":"2024-04-05T22:25:23.301336+0000","last_became_peered":"2024-04-05T22:25:23.301336+0000","last_unstale":"2024-04-05T22:25:24.290866+0000","last_undegraded":"2024-04-05T22:25:24.290866+0000","last_fullsized":"2024-04-05T22:25:24.290866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:42.782908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311640+0000","last_change":"2024-04-05T22:25:23.321078+0000","last_active":"2024-04-05T22:25:24.311640+0000","last_peered":"2024-04-05T22:25:24.311640+0000","last_clean":"2024-04-05T22:25:24.311640+0000","last_became_active":"2024-04-05T22:25:23.315835+0000","last_became_peered":"2024-04-05T22:25:23.315835+0000","last_unstale":"2024-04-05T22:25:24.311640+0000","last_undegraded":"2024-04-05T22:25:24.311640+0000","last_fullsized":"2024-04-05T22:25:24.311640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308033+0000","last_change":"2024-04-05T22:25:23.316796+0000","last_active":"2024-04-05T22:25:24.308033+0000","last_peered":"2024-04-05T22:25:24.308033+0000","last_clean":"2024-04-05T22:25:24.308033+0000","last_became_active":"2024-04-05T22:25:23.316540+0000","last_became_peered":"2024-04-05T22:25:23.316540+0000","last_unstale":"2024-04-05T22:25:24.308033+0000","last_undegraded":"2024-04-05T22:25:24.308033+0000","last_fullsized":"2024-04-05T22:25:24.308033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290359+0000","last_change":"2024-04-05T22:25:23.318867+0000","last_active":"2024-04-05T22:25:24.290359+0000","last_peered":"2024-04-05T22:25:24.290359+0000","last_clean":"2024-04-05T22:25:24.290359+0000","last_became_active":"2024-04-05T22:25:23.318583+0000","last_became_peered":"2024-04-05T22:25:23.318583+0000","last_unstale":"2024-04-05T22:25:24.290359+0000","last_undegraded":"2024-04-05T22:25:24.290359+0000","last_fullsized":"2024-04-05T22:25:24.290359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:20.782795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308872+0000","last_change":"2024-04-05T22:25:23.309759+0000","last_active":"2024-04-05T22:25:24.308872+0000","last_peered":"2024-04-05T22:25:24.308872+0000","last_clean":"2024-04-05T22:25:24.308872+0000","last_became_active":"2024-04-05T22:25:23.309475+0000","last_became_peered":"2024-04-05T22:25:23.309475+0000","last_unstale":"2024-04-05T22:25:24.308872+0000","last_undegraded":"2024-04-05T22:25:24.308872+0000","last_fullsized":"2024-04-05T22:25:24.308872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:40.089255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310131+0000","last_change":"2024-04-05T22:25:23.318074+0000","last_active":"2024-04-05T22:25:24.310131+0000","last_peered":"2024-04-05T22:25:24.310131+0000","last_clean":"2024-04-05T22:25:24.310131+0000","last_became_active":"2024-04-05T22:25:23.317792+0000","last_became_peered":"2024-04-05T22:25:23.317792+0000","last_unstale":"2024-04-05T22:25:24.310131+0000","last_undegraded":"2024-04-05T22:25:24.310131+0000","last_fullsized":"2024-04-05T22:25:24.310131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290117+0000","last_change":"2024-04-05T22:25:23.301201+0000","last_active":"2024-04-05T22:25:24.290117+0000","last_peered":"2024-04-05T22:25:24.290117+0000","last_clean":"2024-04-05T22:25:24.290117+0000","last_became_active":"2024-04-05T22:25:23.300963+0000","last_became_peered":"2024-04-05T22:25:23.300963+0000","last_unstale":"2024-04-05T22:25:24.290117+0000","last_undegraded":"2024-04-05T22:25:24.290117+0000","last_fullsized":"2024-04-05T22:25:24.290117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:29.929763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307353+0000","last_change":"2024-04-05T22:25:23.298905+0000","last_active":"2024-04-05T22:25:24.307353+0000","last_peered":"2024-04-05T22:25:24.307353+0000","last_clean":"2024-04-05T22:25:24.307353+0000","last_became_active":"2024-04-05T22:25:23.297849+0000","last_became_peered":"2024-04-05T22:25:23.297849+0000","last_unstale":"2024-04-05T22:25:24.307353+0000","last_undegraded":"2024-04-05T22:25:24.307353+0000","last_fullsized":"2024-04-05T22:25:24.307353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:12.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307777+0000","last_change":"2024-04-05T22:25:23.316032+0000","last_active":"2024-04-05T22:25:24.307777+0000","last_peered":"2024-04-05T22:25:24.307777+0000","last_clean":"2024-04-05T22:25:24.307777+0000","last_became_active":"2024-04-05T22:25:23.315784+0000","last_became_peered":"2024-04-05T22:25:23.315784+0000","last_unstale":"2024-04-05T22:25:24.307777+0000","last_undegraded":"2024-04-05T22:25:24.307777+0000","last_fullsized":"2024-04-05T22:25:24.307777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:36:12.120029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.321362+0000","last_change":"2024-04-05T22:25:22.820217+0000","last_active":"2024-04-05T22:25:23.321362+0000","last_peered":"2024-04-05T22:25:23.321362+0000","last_clean":"2024-04-05T22:25:23.321362+0000","last_became_active":"2024-04-05T22:25:22.819880+0000","last_became_peered":"2024-04-05T22:25:22.819880+0000","last_unstale":"2024-04-05T22:25:23.321362+0000","last_undegraded":"2024-04-05T22:25:23.321362+0000","last_fullsized":"2024-04-05T22:25:23.321362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:54.998254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291479+0000","last_change":"2024-04-05T22:25:23.308426+0000","last_active":"2024-04-05T22:25:24.291479+0000","last_peered":"2024-04-05T22:25:24.291479+0000","last_clean":"2024-04-05T22:25:24.291479+0000","last_became_active":"2024-04-05T22:25:23.308165+0000","last_became_peered":"2024-04-05T22:25:23.308165+0000","last_unstale":"2024-04-05T22:25:24.291479+0000","last_undegraded":"2024-04-05T22:25:24.291479+0000","last_fullsized":"2024-04-05T22:25:24.291479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314544+0000","last_change":"2024-04-05T22:25:22.818905+0000","last_active":"2024-04-05T22:25:23.314544+0000","last_peered":"2024-04-05T22:25:23.314544+0000","last_clean":"2024-04-05T22:25:23.314544+0000","last_became_active":"2024-04-05T22:25:22.818565+0000","last_became_peered":"2024-04-05T22:25:22.818565+0000","last_unstale":"2024-04-05T22:25:23.314544+0000","last_undegraded":"2024-04-05T22:25:23.314544+0000","last_fullsized":"2024-04-05T22:25:23.314544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:54.794155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308212+0000","last_change":"2024-04-05T22:25:23.299031+0000","last_active":"2024-04-05T22:25:24.308212+0000","last_peered":"2024-04-05T22:25:24.308212+0000","last_clean":"2024-04-05T22:25:24.308212+0000","last_became_active":"2024-04-05T22:25:23.298753+0000","last_became_peered":"2024-04-05T22:25:23.298753+0000","last_unstale":"2024-04-05T22:25:24.308212+0000","last_undegraded":"2024-04-05T22:25:24.308212+0000","last_fullsized":"2024-04-05T22:25:24.308212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306956+0000","last_change":"2024-04-05T22:25:22.807855+0000","last_active":"2024-04-05T22:25:23.306956+0000","last_peered":"2024-04-05T22:25:23.306956+0000","last_clean":"2024-04-05T22:25:23.306956+0000","last_became_active":"2024-04-05T22:25:22.807584+0000","last_became_peered":"2024-04-05T22:25:22.807584+0000","last_unstale":"2024-04-05T22:25:23.306956+0000","last_undegraded":"2024-04-05T22:25:23.306956+0000","last_fullsized":"2024-04-05T22:25:23.306956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307780+0000","last_change":"2024-04-05T22:25:23.321058+0000","last_active":"2024-04-05T22:25:24.307780+0000","last_peered":"2024-04-05T22:25:24.307780+0000","last_clean":"2024-04-05T22:25:24.307780+0000","last_became_active":"2024-04-05T22:25:23.320468+0000","last_became_peered":"2024-04-05T22:25:23.320468+0000","last_unstale":"2024-04-05T22:25:24.307780+0000","last_undegraded":"2024-04-05T22:25:24.307780+0000","last_fullsized":"2024-04-05T22:25:24.307780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:42:19.724742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.326384+0000","last_change":"2024-04-05T22:25:22.822056+0000","last_active":"2024-04-05T22:25:23.326384+0000","last_peered":"2024-04-05T22:25:23.326384+0000","last_clean":"2024-04-05T22:25:23.326384+0000","last_became_active":"2024-04-05T22:25:22.821071+0000","last_became_peered":"2024-04-05T22:25:22.821071+0000","last_unstale":"2024-04-05T22:25:23.326384+0000","last_undegraded":"2024-04-05T22:25:23.326384+0000","last_fullsized":"2024-04-05T22:25:23.326384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308559+0000","last_change":"2024-04-05T22:25:23.294038+0000","last_active":"2024-04-05T22:25:24.308559+0000","last_peered":"2024-04-05T22:25:24.308559+0000","last_clean":"2024-04-05T22:25:24.308559+0000","last_became_active":"2024-04-05T22:25:23.293856+0000","last_became_peered":"2024-04-05T22:25:23.293856+0000","last_unstale":"2024-04-05T22:25:24.308559+0000","last_undegraded":"2024-04-05T22:25:24.308559+0000","last_fullsized":"2024-04-05T22:25:24.308559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:23:59.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311539+0000","last_change":"2024-04-05T22:25:22.813362+0000","last_active":"2024-04-05T22:25:23.311539+0000","last_peered":"2024-04-05T22:25:23.311539+0000","last_clean":"2024-04-05T22:25:23.311539+0000","last_became_active":"2024-04-05T22:25:22.812956+0000","last_became_peered":"2024-04-05T22:25:22.812956+0000","last_unstale":"2024-04-05T22:25:23.311539+0000","last_undegraded":"2024-04-05T22:25:23.311539+0000","last_fullsized":"2024-04-05T22:25:23.311539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308570+0000","last_change":"2024-04-05T22:25:23.297447+0000","last_active":"2024-04-05T22:25:24.308570+0000","last_peered":"2024-04-05T22:25:24.308570+0000","last_clean":"2024-04-05T22:25:24.308570+0000","last_became_active":"2024-04-05T22:25:23.294666+0000","last_became_peered":"2024-04-05T22:25:23.294666+0000","last_unstale":"2024-04-05T22:25:24.308570+0000","last_undegraded":"2024-04-05T22:25:24.308570+0000","last_fullsized":"2024-04-05T22:25:24.308570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:33.835122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306675+0000","last_change":"2024-04-05T22:25:22.812184+0000","last_active":"2024-04-05T22:25:23.306675+0000","last_peered":"2024-04-05T22:25:23.306675+0000","last_clean":"2024-04-05T22:25:23.306675+0000","last_became_active":"2024-04-05T22:25:22.811919+0000","last_became_peered":"2024-04-05T22:25:22.811919+0000","last_unstale":"2024-04-05T22:25:23.306675+0000","last_undegraded":"2024-04-05T22:25:23.306675+0000","last_fullsized":"2024-04-05T22:25:23.306675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309323+0000","last_change":"2024-04-05T22:25:23.312968+0000","last_active":"2024-04-05T22:25:24.309323+0000","last_peered":"2024-04-05T22:25:24.309323+0000","last_clean":"2024-04-05T22:25:24.309323+0000","last_became_active":"2024-04-05T22:25:23.312697+0000","last_became_peered":"2024-04-05T22:25:23.312697+0000","last_unstale":"2024-04-05T22:25:24.309323+0000","last_undegraded":"2024-04-05T22:25:24.309323+0000","last_fullsized":"2024-04-05T22:25:24.309323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:43:47.570184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305456+0000","last_change":"2024-04-05T22:25:22.824792+0000","last_active":"2024-04-05T22:25:23.305456+0000","last_peered":"2024-04-05T22:25:23.305456+0000","last_clean":"2024-04-05T22:25:23.305456+0000","last_became_active":"2024-04-05T22:25:22.822520+0000","last_became_peered":"2024-04-05T22:25:22.822520+0000","last_unstale":"2024-04-05T22:25:23.305456+0000","last_undegraded":"2024-04-05T22:25:23.305456+0000","last_fullsized":"2024-04-05T22:25:23.305456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:47.138497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306877+0000","last_change":"2024-04-05T22:25:23.315314+0000","last_active":"2024-04-05T22:25:24.306877+0000","last_peered":"2024-04-05T22:25:24.306877+0000","last_clean":"2024-04-05T22:25:24.306877+0000","last_became_active":"2024-04-05T22:25:23.314998+0000","last_became_peered":"2024-04-05T22:25:23.314998+0000","last_unstale":"2024-04-05T22:25:24.306877+0000","last_undegraded":"2024-04-05T22:25:24.306877+0000","last_fullsized":"2024-04-05T22:25:24.306877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283790+0000","last_change":"2024-04-05T22:25:22.813243+0000","last_active":"2024-04-05T22:25:23.283790+0000","last_peered":"2024-04-05T22:25:23.283790+0000","last_clean":"2024-04-05T22:25:23.283790+0000","last_became_active":"2024-04-05T22:25:22.810600+0000","last_became_peered":"2024-04-05T22:25:22.810600+0000","last_unstale":"2024-04-05T22:25:23.283790+0000","last_undegraded":"2024-04-05T22:25:23.283790+0000","last_fullsized":"2024-04-05T22:25:23.283790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294078+0000","last_change":"2024-04-05T22:25:23.298671+0000","last_active":"2024-04-05T22:25:24.294078+0000","last_peered":"2024-04-05T22:25:24.294078+0000","last_clean":"2024-04-05T22:25:24.294078+0000","last_became_active":"2024-04-05T22:25:23.294595+0000","last_became_peered":"2024-04-05T22:25:23.294595+0000","last_unstale":"2024-04-05T22:25:24.294078+0000","last_undegraded":"2024-04-05T22:25:24.294078+0000","last_fullsized":"2024-04-05T22:25:24.294078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304213+0000","last_change":"2024-04-05T22:25:22.823584+0000","last_active":"2024-04-05T22:25:23.304213+0000","last_peered":"2024-04-05T22:25:23.304213+0000","last_clean":"2024-04-05T22:25:23.304213+0000","last_became_active":"2024-04-05T22:25:22.823316+0000","last_became_peered":"2024-04-05T22:25:22.823316+0000","last_unstale":"2024-04-05T22:25:23.304213+0000","last_undegraded":"2024-04-05T22:25:23.304213+0000","last_fullsized":"2024-04-05T22:25:23.304213+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312394+0000","last_change":"2024-04-05T22:25:23.314057+0000","last_active":"2024-04-05T22:25:24.312394+0000","last_peered":"2024-04-05T22:25:24.312394+0000","last_clean":"2024-04-05T22:25:24.312394+0000","last_became_active":"2024-04-05T22:25:23.312904+0000","last_became_peered":"2024-04-05T22:25:23.312904+0000","last_unstale":"2024-04-05T22:25:24.312394+0000","last_undegraded":"2024-04-05T22:25:24.312394+0000","last_fullsized":"2024-04-05T22:25:24.312394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283719+0000","last_change":"2024-04-05T22:25:22.812954+0000","last_active":"2024-04-05T22:25:23.283719+0000","last_peered":"2024-04-05T22:25:23.283719+0000","last_clean":"2024-04-05T22:25:23.283719+0000","last_became_active":"2024-04-05T22:25:22.812685+0000","last_became_peered":"2024-04-05T22:25:22.812685+0000","last_unstale":"2024-04-05T22:25:23.283719+0000","last_undegraded":"2024-04-05T22:25:23.283719+0000","last_fullsized":"2024-04-05T22:25:23.283719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:36.447234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291127+0000","last_change":"2024-04-05T22:25:23.307204+0000","last_active":"2024-04-05T22:25:24.291127+0000","last_peered":"2024-04-05T22:25:24.291127+0000","last_clean":"2024-04-05T22:25:24.291127+0000","last_became_active":"2024-04-05T22:25:23.306766+0000","last_became_peered":"2024-04-05T22:25:23.306766+0000","last_unstale":"2024-04-05T22:25:24.291127+0000","last_undegraded":"2024-04-05T22:25:24.291127+0000","last_fullsized":"2024-04-05T22:25:24.291127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:27.638305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311719+0000","last_change":"2024-04-05T22:25:22.820936+0000","last_active":"2024-04-05T22:25:23.311719+0000","last_peered":"2024-04-05T22:25:23.311719+0000","last_clean":"2024-04-05T22:25:23.311719+0000","last_became_active":"2024-04-05T22:25:22.820271+0000","last_became_peered":"2024-04-05T22:25:22.820271+0000","last_unstale":"2024-04-05T22:25:23.311719+0000","last_undegraded":"2024-04-05T22:25:23.311719+0000","last_fullsized":"2024-04-05T22:25:23.311719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312194+0000","last_change":"2024-04-05T22:25:23.309672+0000","last_active":"2024-04-05T22:25:24.312194+0000","last_peered":"2024-04-05T22:25:24.312194+0000","last_clean":"2024-04-05T22:25:24.312194+0000","last_became_active":"2024-04-05T22:25:23.308313+0000","last_became_peered":"2024-04-05T22:25:23.308313+0000","last_unstale":"2024-04-05T22:25:24.312194+0000","last_undegraded":"2024-04-05T22:25:24.312194+0000","last_fullsized":"2024-04-05T22:25:24.312194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.322355+0000","last_change":"2024-04-05T22:25:22.812066+0000","last_active":"2024-04-05T22:25:23.322355+0000","last_peered":"2024-04-05T22:25:23.322355+0000","last_clean":"2024-04-05T22:25:23.322355+0000","last_became_active":"2024-04-05T22:25:22.811662+0000","last_became_peered":"2024-04-05T22:25:22.811662+0000","last_unstale":"2024-04-05T22:25:23.322355+0000","last_undegraded":"2024-04-05T22:25:23.322355+0000","last_fullsized":"2024-04-05T22:25:23.322355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:04.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311269+0000","last_change":"2024-04-05T22:25:23.321510+0000","last_active":"2024-04-05T22:25:24.311269+0000","last_peered":"2024-04-05T22:25:24.311269+0000","last_clean":"2024-04-05T22:25:24.311269+0000","last_became_active":"2024-04-05T22:25:23.319576+0000","last_became_peered":"2024-04-05T22:25:23.319576+0000","last_unstale":"2024-04-05T22:25:24.311269+0000","last_undegraded":"2024-04-05T22:25:24.311269+0000","last_fullsized":"2024-04-05T22:25:24.311269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302829+0000","last_change":"2024-04-05T22:25:22.815327+0000","last_active":"2024-04-05T22:25:23.302829+0000","last_peered":"2024-04-05T22:25:23.302829+0000","last_clean":"2024-04-05T22:25:23.302829+0000","last_became_active":"2024-04-05T22:25:22.814733+0000","last_became_peered":"2024-04-05T22:25:22.814733+0000","last_unstale":"2024-04-05T22:25:23.302829+0000","last_undegraded":"2024-04-05T22:25:23.302829+0000","last_fullsized":"2024-04-05T22:25:23.302829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:36.497236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311892+0000","last_change":"2024-04-05T22:25:23.316894+0000","last_active":"2024-04-05T22:25:24.311892+0000","last_peered":"2024-04-05T22:25:24.311892+0000","last_clean":"2024-04-05T22:25:24.311892+0000","last_became_active":"2024-04-05T22:25:23.316563+0000","last_became_peered":"2024-04-05T22:25:23.316563+0000","last_unstale":"2024-04-05T22:25:24.311892+0000","last_undegraded":"2024-04-05T22:25:24.311892+0000","last_fullsized":"2024-04-05T22:25:24.311892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:28.048899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309891+0000","last_change":"2024-04-05T22:25:22.817554+0000","last_active":"2024-04-05T22:25:23.309891+0000","last_peered":"2024-04-05T22:25:23.309891+0000","last_clean":"2024-04-05T22:25:23.309891+0000","last_became_active":"2024-04-05T22:25:22.816462+0000","last_became_peered":"2024-04-05T22:25:22.816462+0000","last_unstale":"2024-04-05T22:25:23.309891+0000","last_undegraded":"2024-04-05T22:25:23.309891+0000","last_fullsized":"2024-04-05T22:25:23.309891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:02.494750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293141+0000","last_change":"2024-04-05T22:25:23.309583+0000","last_active":"2024-04-05T22:25:24.293141+0000","last_peered":"2024-04-05T22:25:24.293141+0000","last_clean":"2024-04-05T22:25:24.293141+0000","last_became_active":"2024-04-05T22:25:23.307782+0000","last_became_peered":"2024-04-05T22:25:23.307782+0000","last_unstale":"2024-04-05T22:25:24.293141+0000","last_undegraded":"2024-04-05T22:25:24.293141+0000","last_fullsized":"2024-04-05T22:25:24.293141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305585+0000","last_change":"2024-04-05T22:25:22.821958+0000","last_active":"2024-04-05T22:25:23.305585+0000","last_peered":"2024-04-05T22:25:23.305585+0000","last_clean":"2024-04-05T22:25:23.305585+0000","last_became_active":"2024-04-05T22:25:22.821425+0000","last_became_peered":"2024-04-05T22:25:22.821425+0000","last_unstale":"2024-04-05T22:25:23.305585+0000","last_undegraded":"2024-04-05T22:25:23.305585+0000","last_fullsized":"2024-04-05T22:25:23.305585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310387+0000","last_change":"2024-04-05T22:25:23.310520+0000","last_active":"2024-04-05T22:25:24.310387+0000","last_peered":"2024-04-05T22:25:24.310387+0000","last_clean":"2024-04-05T22:25:24.310387+0000","last_became_active":"2024-04-05T22:25:23.310234+0000","last_became_peered":"2024-04-05T22:25:23.310234+0000","last_unstale":"2024-04-05T22:25:24.310387+0000","last_undegraded":"2024-04-05T22:25:24.310387+0000","last_fullsized":"2024-04-05T22:25:24.310387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.325150+0000","last_change":"2024-04-05T22:25:22.822654+0000","last_active":"2024-04-05T22:25:23.325150+0000","last_peered":"2024-04-05T22:25:23.325150+0000","last_clean":"2024-04-05T22:25:23.325150+0000","last_became_active":"2024-04-05T22:25:22.822260+0000","last_became_peered":"2024-04-05T22:25:22.822260+0000","last_unstale":"2024-04-05T22:25:23.325150+0000","last_undegraded":"2024-04-05T22:25:23.325150+0000","last_fullsized":"2024-04-05T22:25:23.325150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312013+0000","last_change":"2024-04-05T22:25:23.318981+0000","last_active":"2024-04-05T22:25:24.312013+0000","last_peered":"2024-04-05T22:25:24.312013+0000","last_clean":"2024-04-05T22:25:24.312013+0000","last_became_active":"2024-04-05T22:25:23.318382+0000","last_became_peered":"2024-04-05T22:25:23.318382+0000","last_unstale":"2024-04-05T22:25:24.312013+0000","last_undegraded":"2024-04-05T22:25:24.312013+0000","last_fullsized":"2024-04-05T22:25:24.312013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:42:57.769217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302992+0000","last_change":"2024-04-05T22:25:22.817215+0000","last_active":"2024-04-05T22:25:23.302992+0000","last_peered":"2024-04-05T22:25:23.302992+0000","last_clean":"2024-04-05T22:25:23.302992+0000","last_became_active":"2024-04-05T22:25:22.816814+0000","last_became_peered":"2024-04-05T22:25:22.816814+0000","last_unstale":"2024-04-05T22:25:23.302992+0000","last_undegraded":"2024-04-05T22:25:23.302992+0000","last_fullsized":"2024-04-05T22:25:23.302992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:26.685831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311184+0000","last_change":"2024-04-05T22:25:23.321785+0000","last_active":"2024-04-05T22:25:24.311184+0000","last_peered":"2024-04-05T22:25:24.311184+0000","last_clean":"2024-04-05T22:25:24.311184+0000","last_became_active":"2024-04-05T22:25:23.318293+0000","last_became_peered":"2024-04-05T22:25:23.318293+0000","last_unstale":"2024-04-05T22:25:24.311184+0000","last_undegraded":"2024-04-05T22:25:24.311184+0000","last_fullsized":"2024-04-05T22:25:24.311184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.303053+0000","last_change":"2024-04-05T22:25:22.810298+0000","last_active":"2024-04-05T22:25:23.303053+0000","last_peered":"2024-04-05T22:25:23.303053+0000","last_clean":"2024-04-05T22:25:23.303053+0000","last_became_active":"2024-04-05T22:25:22.809980+0000","last_became_peered":"2024-04-05T22:25:22.809980+0000","last_unstale":"2024-04-05T22:25:23.303053+0000","last_undegraded":"2024-04-05T22:25:23.303053+0000","last_fullsized":"2024-04-05T22:25:23.303053+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308777+0000","last_change":"2024-04-05T22:25:23.310722+0000","last_active":"2024-04-05T22:25:24.308777+0000","last_peered":"2024-04-05T22:25:24.308777+0000","last_clean":"2024-04-05T22:25:24.308777+0000","last_became_active":"2024-04-05T22:25:23.310419+0000","last_became_peered":"2024-04-05T22:25:23.310419+0000","last_unstale":"2024-04-05T22:25:24.308777+0000","last_undegraded":"2024-04-05T22:25:24.308777+0000","last_fullsized":"2024-04-05T22:25:24.308777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:29:46.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.320593+0000","last_change":"2024-04-05T22:25:22.822964+0000","last_active":"2024-04-05T22:25:23.320593+0000","last_peered":"2024-04-05T22:25:23.320593+0000","last_clean":"2024-04-05T22:25:23.320593+0000","last_became_active":"2024-04-05T22:25:22.822645+0000","last_became_peered":"2024-04-05T22:25:22.822645+0000","last_unstale":"2024-04-05T22:25:23.320593+0000","last_undegraded":"2024-04-05T22:25:23.320593+0000","last_fullsized":"2024-04-05T22:25:23.320593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312362+0000","last_change":"2024-04-05T22:25:23.313098+0000","last_active":"2024-04-05T22:25:24.312362+0000","last_peered":"2024-04-05T22:25:24.312362+0000","last_clean":"2024-04-05T22:25:24.312362+0000","last_became_active":"2024-04-05T22:25:23.312388+0000","last_became_peered":"2024-04-05T22:25:23.312388+0000","last_unstale":"2024-04-05T22:25:24.312362+0000","last_undegraded":"2024-04-05T22:25:24.312362+0000","last_fullsized":"2024-04-05T22:25:24.312362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.287654+0000","last_change":"2024-04-05T22:25:22.810231+0000","last_active":"2024-04-05T22:25:23.287654+0000","last_peered":"2024-04-05T22:25:23.287654+0000","last_clean":"2024-04-05T22:25:23.287654+0000","last_became_active":"2024-04-05T22:25:22.809955+0000","last_became_peered":"2024-04-05T22:25:22.809955+0000","last_unstale":"2024-04-05T22:25:23.287654+0000","last_undegraded":"2024-04-05T22:25:23.287654+0000","last_fullsized":"2024-04-05T22:25:23.287654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311586+0000","last_change":"2024-04-05T22:25:23.308720+0000","last_active":"2024-04-05T22:25:24.311586+0000","last_peered":"2024-04-05T22:25:24.311586+0000","last_clean":"2024-04-05T22:25:24.311586+0000","last_became_active":"2024-04-05T22:25:23.308461+0000","last_became_peered":"2024-04-05T22:25:23.308461+0000","last_unstale":"2024-04-05T22:25:24.311586+0000","last_undegraded":"2024-04-05T22:25:24.311586+0000","last_fullsized":"2024-04-05T22:25:24.311586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314414+0000","last_change":"2024-04-05T22:25:22.808626+0000","last_active":"2024-04-05T22:25:23.314414+0000","last_peered":"2024-04-05T22:25:23.314414+0000","last_clean":"2024-04-05T22:25:23.314414+0000","last_became_active":"2024-04-05T22:25:22.808159+0000","last_became_peered":"2024-04-05T22:25:22.808159+0000","last_unstale":"2024-04-05T22:25:23.314414+0000","last_undegraded":"2024-04-05T22:25:23.314414+0000","last_fullsized":"2024-04-05T22:25:23.314414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:19.370558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307477+0000","last_change":"2024-04-05T22:25:23.310723+0000","last_active":"2024-04-05T22:25:24.307477+0000","last_peered":"2024-04-05T22:25:24.307477+0000","last_clean":"2024-04-05T22:25:24.307477+0000","last_became_active":"2024-04-05T22:25:23.310431+0000","last_became_peered":"2024-04-05T22:25:23.310431+0000","last_unstale":"2024-04-05T22:25:24.307477+0000","last_undegraded":"2024-04-05T22:25:24.307477+0000","last_fullsized":"2024-04-05T22:25:24.307477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:20.385215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279422+0000","last_change":"2024-04-05T22:25:22.809387+0000","last_active":"2024-04-05T22:25:23.279422+0000","last_peered":"2024-04-05T22:25:23.279422+0000","last_clean":"2024-04-05T22:25:23.279422+0000","last_became_active":"2024-04-05T22:25:22.809109+0000","last_became_peered":"2024-04-05T22:25:22.809109+0000","last_unstale":"2024-04-05T22:25:23.279422+0000","last_undegraded":"2024-04-05T22:25:23.279422+0000","last_fullsized":"2024-04-05T22:25:23.279422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307167+0000","last_change":"2024-04-05T22:25:23.316165+0000","last_active":"2024-04-05T22:25:24.307167+0000","last_peered":"2024-04-05T22:25:24.307167+0000","last_clean":"2024-04-05T22:25:24.307167+0000","last_became_active":"2024-04-05T22:25:23.315655+0000","last_became_peered":"2024-04-05T22:25:23.315655+0000","last_unstale":"2024-04-05T22:25:24.307167+0000","last_undegraded":"2024-04-05T22:25:24.307167+0000","last_fullsized":"2024-04-05T22:25:24.307167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304412+0000","last_change":"2024-04-05T22:25:22.820823+0000","last_active":"2024-04-05T22:25:23.304412+0000","last_peered":"2024-04-05T22:25:23.304412+0000","last_clean":"2024-04-05T22:25:23.304412+0000","last_became_active":"2024-04-05T22:25:22.819543+0000","last_became_peered":"2024-04-05T22:25:22.819543+0000","last_unstale":"2024-04-05T22:25:23.304412+0000","last_undegraded":"2024-04-05T22:25:23.304412+0000","last_fullsized":"2024-04-05T22:25:23.304412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312774+0000","last_change":"2024-04-05T22:25:23.304349+0000","last_active":"2024-04-05T22:25:24.312774+0000","last_peered":"2024-04-05T22:25:24.312774+0000","last_clean":"2024-04-05T22:25:24.312774+0000","last_became_active":"2024-04-05T22:25:23.304014+0000","last_became_peered":"2024-04-05T22:25:23.304014+0000","last_unstale":"2024-04-05T22:25:24.312774+0000","last_undegraded":"2024-04-05T22:25:24.312774+0000","last_fullsized":"2024-04-05T22:25:24.312774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305751+0000","last_change":"2024-04-05T22:25:22.813364+0000","last_active":"2024-04-05T22:25:23.305751+0000","last_peered":"2024-04-05T22:25:23.305751+0000","last_clean":"2024-04-05T22:25:23.305751+0000","last_became_active":"2024-04-05T22:25:22.813068+0000","last_became_peered":"2024-04-05T22:25:22.813068+0000","last_unstale":"2024-04-05T22:25:23.305751+0000","last_undegraded":"2024-04-05T22:25:23.305751+0000","last_fullsized":"2024-04-05T22:25:23.305751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311439+0000","last_change":"2024-04-05T22:25:23.321716+0000","last_active":"2024-04-05T22:25:24.311439+0000","last_peered":"2024-04-05T22:25:24.311439+0000","last_clean":"2024-04-05T22:25:24.311439+0000","last_became_active":"2024-04-05T22:25:23.321428+0000","last_became_peered":"2024-04-05T22:25:23.321428+0000","last_unstale":"2024-04-05T22:25:24.311439+0000","last_undegraded":"2024-04-05T22:25:24.311439+0000","last_fullsized":"2024-04-05T22:25:24.311439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.288133+0000","last_change":"2024-04-05T22:25:22.813803+0000","last_active":"2024-04-05T22:25:23.288133+0000","last_peered":"2024-04-05T22:25:23.288133+0000","last_clean":"2024-04-05T22:25:23.288133+0000","last_became_active":"2024-04-05T22:25:22.813365+0000","last_became_peered":"2024-04-05T22:25:22.813365+0000","last_unstale":"2024-04-05T22:25:23.288133+0000","last_undegraded":"2024-04-05T22:25:23.288133+0000","last_fullsized":"2024-04-05T22:25:23.288133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291236+0000","last_change":"2024-04-05T22:25:23.302602+0000","last_active":"2024-04-05T22:25:24.291236+0000","last_peered":"2024-04-05T22:25:24.291236+0000","last_clean":"2024-04-05T22:25:24.291236+0000","last_became_active":"2024-04-05T22:25:23.302356+0000","last_became_peered":"2024-04-05T22:25:23.302356+0000","last_unstale":"2024-04-05T22:25:24.291236+0000","last_undegraded":"2024-04-05T22:25:24.291236+0000","last_fullsized":"2024-04-05T22:25:24.291236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:27.824520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.312000+0000","last_change":"2024-04-05T22:25:22.812466+0000","last_active":"2024-04-05T22:25:23.312000+0000","last_peered":"2024-04-05T22:25:23.312000+0000","last_clean":"2024-04-05T22:25:23.312000+0000","last_became_active":"2024-04-05T22:25:22.812117+0000","last_became_peered":"2024-04-05T22:25:22.812117+0000","last_unstale":"2024-04-05T22:25:23.312000+0000","last_undegraded":"2024-04-05T22:25:23.312000+0000","last_fullsized":"2024-04-05T22:25:23.312000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:13.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307935+0000","last_change":"2024-04-05T22:25:23.321029+0000","last_active":"2024-04-05T22:25:24.307935+0000","last_peered":"2024-04-05T22:25:24.307935+0000","last_clean":"2024-04-05T22:25:24.307935+0000","last_became_active":"2024-04-05T22:25:23.320742+0000","last_became_peered":"2024-04-05T22:25:23.320742+0000","last_unstale":"2024-04-05T22:25:24.307935+0000","last_undegraded":"2024-04-05T22:25:24.307935+0000","last_fullsized":"2024-04-05T22:25:24.307935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:26.184734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.316800+0000","last_change":"2024-04-05T22:25:22.819850+0000","last_active":"2024-04-05T22:25:23.316800+0000","last_peered":"2024-04-05T22:25:23.316800+0000","last_clean":"2024-04-05T22:25:23.316800+0000","last_became_active":"2024-04-05T22:25:22.819495+0000","last_became_peered":"2024-04-05T22:25:22.819495+0000","last_unstale":"2024-04-05T22:25:23.316800+0000","last_undegraded":"2024-04-05T22:25:23.316800+0000","last_fullsized":"2024-04-05T22:25:23.316800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308979+0000","last_change":"2024-04-05T22:25:23.290975+0000","last_active":"2024-04-05T22:25:24.308979+0000","last_peered":"2024-04-05T22:25:24.308979+0000","last_clean":"2024-04-05T22:25:24.308979+0000","last_became_active":"2024-04-05T22:25:23.290733+0000","last_became_peered":"2024-04-05T22:25:23.290733+0000","last_unstale":"2024-04-05T22:25:24.308979+0000","last_undegraded":"2024-04-05T22:25:24.308979+0000","last_fullsized":"2024-04-05T22:25:24.308979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:29.453299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279621+0000","last_change":"2024-04-05T22:25:22.821995+0000","last_active":"2024-04-05T22:25:23.279621+0000","last_peered":"2024-04-05T22:25:23.279621+0000","last_clean":"2024-04-05T22:25:23.279621+0000","last_became_active":"2024-04-05T22:25:22.821657+0000","last_became_peered":"2024-04-05T22:25:22.821657+0000","last_unstale":"2024-04-05T22:25:23.279621+0000","last_undegraded":"2024-04-05T22:25:23.279621+0000","last_fullsized":"2024-04-05T22:25:23.279621+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312062+0000","last_change":"2024-04-05T22:25:23.307838+0000","last_active":"2024-04-05T22:25:24.312062+0000","last_peered":"2024-04-05T22:25:24.312062+0000","last_clean":"2024-04-05T22:25:24.312062+0000","last_became_active":"2024-04-05T22:25:23.307552+0000","last_became_peered":"2024-04-05T22:25:23.307552+0000","last_unstale":"2024-04-05T22:25:24.312062+0000","last_undegraded":"2024-04-05T22:25:24.312062+0000","last_fullsized":"2024-04-05T22:25:24.312062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:15.194717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283982+0000","last_change":"2024-04-05T22:25:22.809727+0000","last_active":"2024-04-05T22:25:23.283982+0000","last_peered":"2024-04-05T22:25:23.283982+0000","last_clean":"2024-04-05T22:25:23.283982+0000","last_became_active":"2024-04-05T22:25:22.809281+0000","last_became_peered":"2024-04-05T22:25:22.809281+0000","last_unstale":"2024-04-05T22:25:23.283982+0000","last_undegraded":"2024-04-05T22:25:23.283982+0000","last_fullsized":"2024-04-05T22:25:23.283982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290369+0000","last_change":"2024-04-05T22:25:23.312476+0000","last_active":"2024-04-05T22:25:24.290369+0000","last_peered":"2024-04-05T22:25:24.290369+0000","last_clean":"2024-04-05T22:25:24.290369+0000","last_became_active":"2024-04-05T22:25:23.312245+0000","last_became_peered":"2024-04-05T22:25:23.312245+0000","last_unstale":"2024-04-05T22:25:24.290369+0000","last_undegraded":"2024-04-05T22:25:24.290369+0000","last_fullsized":"2024-04-05T22:25:24.290369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279493+0000","last_change":"2024-04-05T22:25:22.805657+0000","last_active":"2024-04-05T22:25:23.279493+0000","last_peered":"2024-04-05T22:25:23.279493+0000","last_clean":"2024-04-05T22:25:23.279493+0000","last_became_active":"2024-04-05T22:25:22.805398+0000","last_became_peered":"2024-04-05T22:25:22.805398+0000","last_unstale":"2024-04-05T22:25:23.279493+0000","last_undegraded":"2024-04-05T22:25:23.279493+0000","last_fullsized":"2024-04-05T22:25:23.279493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308351+0000","last_change":"2024-04-05T22:25:23.315180+0000","last_active":"2024-04-05T22:25:24.308351+0000","last_peered":"2024-04-05T22:25:24.308351+0000","last_clean":"2024-04-05T22:25:24.308351+0000","last_became_active":"2024-04-05T22:25:23.314678+0000","last_became_peered":"2024-04-05T22:25:23.314678+0000","last_unstale":"2024-04-05T22:25:24.308351+0000","last_undegraded":"2024-04-05T22:25:24.308351+0000","last_fullsized":"2024-04-05T22:25:24.308351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:43.210562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.282244+0000","last_change":"2024-04-05T22:25:22.821960+0000","last_active":"2024-04-05T22:25:23.282244+0000","last_peered":"2024-04-05T22:25:23.282244+0000","last_clean":"2024-04-05T22:25:23.282244+0000","last_became_active":"2024-04-05T22:25:22.821679+0000","last_became_peered":"2024-04-05T22:25:22.821679+0000","last_unstale":"2024-04-05T22:25:23.282244+0000","last_undegraded":"2024-04-05T22:25:23.282244+0000","last_fullsized":"2024-04-05T22:25:23.282244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:11.859438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306832+0000","last_change":"2024-04-05T22:25:23.292675+0000","last_active":"2024-04-05T22:25:24.306832+0000","last_peered":"2024-04-05T22:25:24.306832+0000","last_clean":"2024-04-05T22:25:24.306832+0000","last_became_active":"2024-04-05T22:25:23.292226+0000","last_became_peered":"2024-04-05T22:25:23.292226+0000","last_unstale":"2024-04-05T22:25:24.306832+0000","last_undegraded":"2024-04-05T22:25:24.306832+0000","last_fullsized":"2024-04-05T22:25:24.306832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:45.144578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309603+0000","last_change":"2024-04-05T22:25:22.823366+0000","last_active":"2024-04-05T22:25:23.309603+0000","last_peered":"2024-04-05T22:25:23.309603+0000","last_clean":"2024-04-05T22:25:23.309603+0000","last_became_active":"2024-04-05T22:25:22.823082+0000","last_became_peered":"2024-04-05T22:25:22.823082+0000","last_unstale":"2024-04-05T22:25:23.309603+0000","last_undegraded":"2024-04-05T22:25:23.309603+0000","last_fullsized":"2024-04-05T22:25:23.309603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:59.775684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.296039+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:25:23.296039+0000","last_peered":"2024-04-05T22:25:23.296039+0000","last_clean":"2024-04-05T22:25:23.296039+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:25:23.296039+0000","last_undegraded":"2024-04-05T22:25:23.296039+0000","last_fullsized":"2024-04-05T22:25:23.296039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1343488,"data_stored":899715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":1.137},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.321},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.073}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.391}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.2370000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.544}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.623},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":741376,"data_stored":309187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":0.96899999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.508}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.248},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.681}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.796},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.365}]}]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":749568,"data_stored":309839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.042}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":1.256},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.331},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.52}]}]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":753664,"data_stored":310751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0509999999999999,"15min":0.999},"min":{"1min":0.84699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":1.673,"15min":1.673},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0980000000000001,"15min":1.0800000000000001},"min":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.343}]},{"osd":1,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1439999999999999,"15min":1.1299999999999999},"min":{"1min":0.875,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.236},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1539999999999999,"15min":1.1699999999999999},"min":{"1min":0.83399999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4450000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.95099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.482,"5min":1.974,"15min":1.974},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.89000000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.379},"min":{"1min":0.95999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.843,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.369,"15min":1.407},"min":{"1min":1.107,"5min":0.874,"15min":0.874},"max":{"1min":1.754,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.575},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3440000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.625,"15min":0.625},"max":{"1min":1.786,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.49}]},{"osd":6,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4610000000000001},"min":{"1min":0.95599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6379999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.706},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4410000000000001,"15min":1.4950000000000001},"min":{"1min":1.0149999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.655,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8819999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5029999999999999,"15min":1.536},"min":{"1min":1.1639999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.895,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.825},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5549999999999999,"15min":1.5760000000000001},"min":{"1min":1.05,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":2.54,"15min":2.54},"last":1.9279999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":1.23},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":1.121},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95999999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.044},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.204}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.464}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.377},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.2909999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":1},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.876}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.21}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.621},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.518}]}]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":745472,"data_stored":309277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4970000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.127}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.282}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":1.25},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.665}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.573},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.536}]}]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1351680,"data_stored":901701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.875}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:25:28.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-05T22:25:28.746 INFO:tasks.ceph.mds.a.smithi073.stderr:2024-04-05T22:25:28.747+0000 7f129aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:25:28.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.954+0000 7fa9e7c00640 1 Processor -- start 2024-04-05T22:25:28.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.954+0000 7fa9e7c00640 1 -- start start 2024-04-05T22:25:28.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e0160620 0x7fa9e00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e7c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9e00cb2f0 0x7fa9e00cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e7c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e00ad880 con 0x7fa9e00b1870 2024-04-05T22:25:28.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e7c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00cbcc0 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00ce5d0 con 0x7fa9e0160620 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e5c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9e00cb2f0 0x7fa9e00cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e5c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9e00cb2f0 0x7fa9e00cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51458/0 (socket says 172.21.15.73:51458) 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e5c00640 1 -- 172.21.15.73:0/1217899607 learned_addr learned my addr 172.21.15.73:0/1217899607 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.956+0000 7fa9e5200640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e0160620 0x7fa9e00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.957+0000 7fa9e5c00640 1 -- 172.21.15.73:0/1217899607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e0160620 msgr2=0x7fa9e00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.957+0000 7fa9e5c00640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e0160620 0x7fa9e00caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.957+0000 7fa9e5c00640 1 -- 172.21.15.73:0/1217899607 >> v1:172.21.15.157:6789/0 conn(0x7fa9e00b1870 legacy=0x7fa9e00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:28.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.957+0000 7fa9e5c00640 1 -- 172.21.15.73:0/1217899607 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00ce010 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.958+0000 7fa9e5c00640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e00cb2f0 0x7fa9e00cb780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e50 tx=0x7fa9d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21495eefaacaa52c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.958+0000 7fa9df400640 1 -- 172.21.15.73:0/1217899607 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d802ca70 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.958+0000 7fa9df400640 1 -- 172.21.15.73:0/1217899607 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d802cbd0 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.958+0000 7fa9df400640 1 -- 172.21.15.73:0/1217899607 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80343f0 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 -- 172.21.15.73:0/1217899607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e00cb2f0 msgr2=0x7fa9e00cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e00cb2f0 0x7fa9e00cb780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e50 tx=0x7fa9d802bed0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 -- 172.21.15.73:0/1217899607 shutdown_connections 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e00cb2f0 0x7fa9e00cb780 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/1217899607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e0160620 0x7fa9e00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 -- 172.21.15.73:0/1217899607 >> 172.21.15.73:0/1217899607 conn(0x7fa9e00ce8e0 msgr2=0x7fa9e00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 -- 172.21.15.73:0/1217899607 shutdown_connections 2024-04-05T22:25:28.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.959+0000 7fa9e7c00640 1 -- 172.21.15.73:0/1217899607 wait complete. 2024-04-05T22:25:28.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.960+0000 7fa9e7c00640 1 Processor -- start 2024-04-05T22:25:28.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 -- start start 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 0x7fa9e01f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e00cb2f0 0x7fa9e014baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01f69f0 con 0x7fa9e0160620 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e014f450 con 0x7fa9e00b1870 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.961+0000 7fa9e7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e014e8d0 con 0x7fa9e00cb2f0 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51470/0 (socket says 172.21.15.73:51470) 2024-04-05T22:25:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 -- 172.21.15.73:0/890792134 learned_addr learned my addr 172.21.15.73:0/890792134 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5200640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 0x7fa9e01f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 -- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e00cb2f0 msgr2=0x7fa9e014baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9dfe00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e00cb2f0 0x7fa9e014baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e00cb2f0 0x7fa9e014baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 -- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 msgr2=0x7fa9e01f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 0x7fa9e01f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5c00640 1 -- 172.21.15.73:0/890792134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8009b30 con 0x7fa9e0160620 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.962+0000 7fa9e5200640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 0x7fa9e01f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9e5c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8036040 tx=0x7fa9d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8009510 con 0x7fa9e0160620 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d803e450 con 0x7fa9e0160620 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d803e7e0 con 0x7fa9e0160620 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e014ebb0 con 0x7fa9e0160620 2024-04-05T22:25:28.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.963+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e014f170 con 0x7fa9e0160620 2024-04-05T22:25:28.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.964+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0066af0 con 0x7fa9e0160620 2024-04-05T22:25:28.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.965+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fa9d803e940 con 0x7fa9e0160620 2024-04-05T22:25:28.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.966+0000 7fa9dcc00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 0x7fa9bc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:28.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.966+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9d80ac440 con 0x7fa9e0160620 2024-04-05T22:25:28.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.966+0000 7fa9e5200640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 0x7fa9bc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:28.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.967+0000 7fa9e5200640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 0x7fa9bc06d270 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:28.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:28.971+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9d8077390 con 0x7fa9e0160620 2024-04-05T22:25:29.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.232+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa9e00b1690 con 0x7fa9e0160620 2024-04-05T22:25:29.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.233+0000 7fa9dcc00640 1 -- 172.21.15.73:0/890792134 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7fa9d807a9b0 con 0x7fa9e0160620 2024-04-05T22:25:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:29.233 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"38861c87-67cf-4180-a71f-97401becb77d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":273,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1712355667,"num_in_osds":8,"osd_in_since":1712355659,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":257937408,"bytes_avail":772836175872,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6195}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-05T22:24:16.550018+0000","services":{}},"progress_events":{}} 2024-04-05T22:25:29.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 msgr2=0x7fa9bc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 0x7fa9bc06d270 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0008040 comp rx=0 tx=0).stop 2024-04-05T22:25:29.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 msgr2=0x7fa9e015e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8036040 tx=0x7fa9d8004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 shutdown_connections 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa9bc06adb0 0x7fa9bc06d270 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e0160620 0x7fa9e015e3f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e00cb2f0 0x7fa9e014baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 --2- 172.21.15.73:0/890792134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e00b1870 0x7fa9e01f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 >> 172.21.15.73:0/890792134 conn(0x7fa9e00ce8e0 msgr2=0x7fa9e0071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:29.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 shutdown_connections 2024-04-05T22:25:29.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.236+0000 7fa9e7c00640 1 -- 172.21.15.73:0/890792134 wait complete. 2024-04-05T22:25:29.254 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-05T22:25:29.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:25:29.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.450+0000 7f9acf400640 1 Processor -- start 2024-04-05T22:25:29.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.450+0000 7f9acf400640 1 -- start start 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 0x7f9ac80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80ccd20 0x7f9ac806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac806d0c0 0x7f9ac815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac806f5b0 con 0x7f9ac806d0c0 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac8070410 con 0x7f9ac81587d0 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acf400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8158bb0 con 0x7f9ac80ccd20 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac806d0c0 0x7f9ac815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac806d0c0 0x7f9ac815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51478/0 (socket says 172.21.15.73:51478) 2024-04-05T22:25:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 -- 172.21.15.73:0/66774204 learned_addr learned my addr 172.21.15.73:0/66774204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:29.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acca00640 1 --2- 172.21.15.73:0/66774204 >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 0x7f9ac80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9ac7e00640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80ccd20 0x7f9ac806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 -- 172.21.15.73:0/66774204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80ccd20 msgr2=0x7f9ac806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80ccd20 0x7f9ac806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 -- 172.21.15.73:0/66774204 >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 msgr2=0x7f9ac80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 --2- 172.21.15.73:0/66774204 >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 0x7f9ac80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acd400640 1 -- 172.21.15.73:0/66774204 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac806f800 con 0x7f9ac806d0c0 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.452+0000 7f9acca00640 1 --2- 172.21.15.73:0/66774204 >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 0x7f9ac80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.453+0000 7f9acd400640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac806d0c0 0x7f9ac815e850 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80667b0 tx=0x7f9ac002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875f602fefccb447 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9ac7400640 1 -- 172.21.15.73:0/66774204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac002ca80 con 0x7f9ac806d0c0 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9ac7400640 1 -- 172.21.15.73:0/66774204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac002cbe0 con 0x7f9ac806d0c0 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9ac7400640 1 -- 172.21.15.73:0/66774204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00343f0 con 0x7f9ac806d0c0 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 -- 172.21.15.73:0/66774204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac806d0c0 msgr2=0x7f9ac815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac806d0c0 0x7f9ac815e850 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80667b0 tx=0x7f9ac002bee0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 -- 172.21.15.73:0/66774204 shutdown_connections 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac806d0c0 0x7f9ac815e850 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 --2- 172.21.15.73:0/66774204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80ccd20 0x7f9ac806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 --2- 172.21.15.73:0/66774204 >> v2:172.21.15.157:3300/0 conn(0x7f9ac81587d0 0x7f9ac80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.454+0000 7f9acf400640 1 -- 172.21.15.73:0/66774204 >> 172.21.15.73:0/66774204 conn(0x7f9ac80ce8e0 msgr2=0x7f9ac80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.455+0000 7f9acf400640 1 -- 172.21.15.73:0/66774204 shutdown_connections 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.455+0000 7f9acf400640 1 -- 172.21.15.73:0/66774204 wait complete. 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.456+0000 7f9acf400640 1 Processor -- start 2024-04-05T22:25:29.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.456+0000 7f9acf400640 1 -- start start 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac806d0c0 0x7f9ac8133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 0x7f9ac815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8139b00 con 0x7f9ac81357c0 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8135c50 con 0x7f9ac81587d0 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.457+0000 7f9acf400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8135f30 con 0x7f9ac806d0c0 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51494/0 (socket says 172.21.15.73:51494) 2024-04-05T22:25:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 -- 172.21.15.73:0/2945668219 learned_addr learned my addr 172.21.15.73:0/2945668219 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:29.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9ac7e00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 0x7f9ac815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acca00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac806d0c0 0x7f9ac8133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 -- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac806d0c0 msgr2=0x7f9ac8133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac806d0c0 0x7f9ac8133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 -- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 msgr2=0x7f9ac815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 0x7f9ac815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 -- 172.21.15.73:0/2945668219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0009b30 con 0x7f9ac81357c0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9ac7e00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 0x7f9ac815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.458+0000 7f9acd400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0036040 tx=0x7f9ac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.459+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0009510 con 0x7f9ac81357c0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.459+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac003e450 con 0x7f9ac81357c0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.459+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac003e800 con 0x7f9ac81357c0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.459+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac8136210 con 0x7f9ac81357c0 2024-04-05T22:25:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.459+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac815dfb0 con 0x7f9ac81357c0 2024-04-05T22:25:29.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.460+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac8066af0 con 0x7f9ac81357c0 2024-04-05T22:25:29.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.461+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f9ac002cd50 con 0x7f9ac81357c0 2024-04-05T22:25:29.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.462+0000 7f9ac4c00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 0x7f9aa806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:29.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.462+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ac00ac250 con 0x7f9ac81357c0 2024-04-05T22:25:29.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.462+0000 7f9acca00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 0x7f9aa806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:29.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.462+0000 7f9acca00640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 0x7f9aa806d270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b840 tx=0x7f9abc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:29.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.466+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9ac00772f0 con 0x7f9ac81357c0 2024-04-05T22:25:29.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.655+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ac80700a0 con 0x7f9aa806adb0 2024-04-05T22:25:29.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.663+0000 7f9ac4c00640 1 -- 172.21.15.73:0/2945668219 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286681 (secure 0 0 0) 0x7f9abc00c040 con 0x7f9aa806adb0 2024-04-05T22:25:29.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.669+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 msgr2=0x7f9aa806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.669+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 0x7f9aa806d270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b840 tx=0x7f9abc008040 comp rx=0 tx=0).stop 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.669+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 msgr2=0x7f9ac80c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.669+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0036040 tx=0x7f9ac0004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 shutdown_connections 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9aa806adb0 0x7f9aa806d270 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac81357c0 0x7f9ac80c3a50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac81587d0 0x7f9ac815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 --2- 172.21.15.73:0/2945668219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac806d0c0 0x7f9ac8133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 >> 172.21.15.73:0/2945668219 conn(0x7f9ac80ce8e0 msgr2=0x7f9ac80cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 shutdown_connections 2024-04-05T22:25:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:29.670+0000 7f9acf400640 1 -- 172.21.15.73:0/2945668219 wait complete. 2024-04-05T22:25:29.685 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-04-05T22:25:28.576483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251892,"kb_used_data":7016,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754722828,"statfs":{"total":773094113280,"available":772836175872,"internally_reserved":0,"allocated":7184384,"data_stored":3658956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":250596923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290720+0000","last_change":"2024-04-05T22:25:23.299156+0000","last_active":"2024-04-05T22:25:24.290720+0000","last_peered":"2024-04-05T22:25:24.290720+0000","last_clean":"2024-04-05T22:25:24.290720+0000","last_became_active":"2024-04-05T22:25:23.298427+0000","last_became_peered":"2024-04-05T22:25:23.298427+0000","last_unstale":"2024-04-05T22:25:24.290720+0000","last_undegraded":"2024-04-05T22:25:24.290720+0000","last_fullsized":"2024-04-05T22:25:24.290720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:45:44.913900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308802+0000","last_change":"2024-04-05T22:25:23.291712+0000","last_active":"2024-04-05T22:25:24.308802+0000","last_peered":"2024-04-05T22:25:24.308802+0000","last_clean":"2024-04-05T22:25:24.308802+0000","last_became_active":"2024-04-05T22:25:23.290786+0000","last_became_peered":"2024-04-05T22:25:23.290786+0000","last_unstale":"2024-04-05T22:25:24.308802+0000","last_undegraded":"2024-04-05T22:25:24.308802+0000","last_fullsized":"2024-04-05T22:25:24.308802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:02:03.184140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312076+0000","last_change":"2024-04-05T22:25:23.322464+0000","last_active":"2024-04-05T22:25:24.312076+0000","last_peered":"2024-04-05T22:25:24.312076+0000","last_clean":"2024-04-05T22:25:24.312076+0000","last_became_active":"2024-04-05T22:25:23.316842+0000","last_became_peered":"2024-04-05T22:25:23.316842+0000","last_unstale":"2024-04-05T22:25:24.312076+0000","last_undegraded":"2024-04-05T22:25:24.312076+0000","last_fullsized":"2024-04-05T22:25:24.312076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:12:17.928701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312038+0000","last_change":"2024-04-05T22:25:23.324761+0000","last_active":"2024-04-05T22:25:24.312038+0000","last_peered":"2024-04-05T22:25:24.312038+0000","last_clean":"2024-04-05T22:25:24.312038+0000","last_became_active":"2024-04-05T22:25:23.323869+0000","last_became_peered":"2024-04-05T22:25:23.323869+0000","last_unstale":"2024-04-05T22:25:24.312038+0000","last_undegraded":"2024-04-05T22:25:24.312038+0000","last_fullsized":"2024-04-05T22:25:24.312038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290172+0000","last_change":"2024-04-05T22:25:23.318738+0000","last_active":"2024-04-05T22:25:24.290172+0000","last_peered":"2024-04-05T22:25:24.290172+0000","last_clean":"2024-04-05T22:25:24.290172+0000","last_became_active":"2024-04-05T22:25:23.318429+0000","last_became_peered":"2024-04-05T22:25:23.318429+0000","last_unstale":"2024-04-05T22:25:24.290172+0000","last_undegraded":"2024-04-05T22:25:24.290172+0000","last_fullsized":"2024-04-05T22:25:24.290172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:40.840026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307914+0000","last_change":"2024-04-05T22:25:23.296999+0000","last_active":"2024-04-05T22:25:24.307914+0000","last_peered":"2024-04-05T22:25:24.307914+0000","last_clean":"2024-04-05T22:25:24.307914+0000","last_became_active":"2024-04-05T22:25:23.296625+0000","last_became_peered":"2024-04-05T22:25:23.296625+0000","last_unstale":"2024-04-05T22:25:24.307914+0000","last_undegraded":"2024-04-05T22:25:24.307914+0000","last_fullsized":"2024-04-05T22:25:24.307914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293145+0000","last_change":"2024-04-05T22:25:23.312117+0000","last_active":"2024-04-05T22:25:24.293145+0000","last_peered":"2024-04-05T22:25:24.293145+0000","last_clean":"2024-04-05T22:25:24.293145+0000","last_became_active":"2024-04-05T22:25:23.310600+0000","last_became_peered":"2024-04-05T22:25:23.310600+0000","last_unstale":"2024-04-05T22:25:24.293145+0000","last_undegraded":"2024-04-05T22:25:24.293145+0000","last_fullsized":"2024-04-05T22:25:24.293145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:43.287781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306787+0000","last_change":"2024-04-05T22:25:23.296496+0000","last_active":"2024-04-05T22:25:24.306787+0000","last_peered":"2024-04-05T22:25:24.306787+0000","last_clean":"2024-04-05T22:25:24.306787+0000","last_became_active":"2024-04-05T22:25:23.296222+0000","last_became_peered":"2024-04-05T22:25:23.296222+0000","last_unstale":"2024-04-05T22:25:24.306787+0000","last_undegraded":"2024-04-05T22:25:24.306787+0000","last_fullsized":"2024-04-05T22:25:24.306787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:36:43.010960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294145+0000","last_change":"2024-04-05T22:25:23.303841+0000","last_active":"2024-04-05T22:25:24.294145+0000","last_peered":"2024-04-05T22:25:24.294145+0000","last_clean":"2024-04-05T22:25:24.294145+0000","last_became_active":"2024-04-05T22:25:23.301443+0000","last_became_peered":"2024-04-05T22:25:23.301443+0000","last_unstale":"2024-04-05T22:25:24.294145+0000","last_undegraded":"2024-04-05T22:25:24.294145+0000","last_fullsized":"2024-04-05T22:25:24.294145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291335+0000","last_change":"2024-04-05T22:25:23.314986+0000","last_active":"2024-04-05T22:25:24.291335+0000","last_peered":"2024-04-05T22:25:24.291335+0000","last_clean":"2024-04-05T22:25:24.291335+0000","last_became_active":"2024-04-05T22:25:23.314711+0000","last_became_peered":"2024-04-05T22:25:23.314711+0000","last_unstale":"2024-04-05T22:25:24.291335+0000","last_undegraded":"2024-04-05T22:25:24.291335+0000","last_fullsized":"2024-04-05T22:25:24.291335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291109+0000","last_change":"2024-04-05T22:25:23.290822+0000","last_active":"2024-04-05T22:25:24.291109+0000","last_peered":"2024-04-05T22:25:24.291109+0000","last_clean":"2024-04-05T22:25:24.291109+0000","last_became_active":"2024-04-05T22:25:23.290573+0000","last_became_peered":"2024-04-05T22:25:23.290573+0000","last_unstale":"2024-04-05T22:25:24.291109+0000","last_undegraded":"2024-04-05T22:25:24.291109+0000","last_fullsized":"2024-04-05T22:25:24.291109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312290+0000","last_change":"2024-04-05T22:25:23.314201+0000","last_active":"2024-04-05T22:25:24.312290+0000","last_peered":"2024-04-05T22:25:24.312290+0000","last_clean":"2024-04-05T22:25:24.312290+0000","last_became_active":"2024-04-05T22:25:23.307100+0000","last_became_peered":"2024-04-05T22:25:23.307100+0000","last_unstale":"2024-04-05T22:25:24.312290+0000","last_undegraded":"2024-04-05T22:25:24.312290+0000","last_fullsized":"2024-04-05T22:25:24.312290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309603+0000","last_change":"2024-04-05T22:25:23.315571+0000","last_active":"2024-04-05T22:25:24.309603+0000","last_peered":"2024-04-05T22:25:24.309603+0000","last_clean":"2024-04-05T22:25:24.309603+0000","last_became_active":"2024-04-05T22:25:23.315249+0000","last_became_peered":"2024-04-05T22:25:23.315249+0000","last_unstale":"2024-04-05T22:25:24.309603+0000","last_undegraded":"2024-04-05T22:25:24.309603+0000","last_fullsized":"2024-04-05T22:25:24.309603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:41.924668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293201+0000","last_change":"2024-04-05T22:25:23.303074+0000","last_active":"2024-04-05T22:25:24.293201+0000","last_peered":"2024-04-05T22:25:24.293201+0000","last_clean":"2024-04-05T22:25:24.293201+0000","last_became_active":"2024-04-05T22:25:23.302615+0000","last_became_peered":"2024-04-05T22:25:23.302615+0000","last_unstale":"2024-04-05T22:25:24.293201+0000","last_undegraded":"2024-04-05T22:25:24.293201+0000","last_fullsized":"2024-04-05T22:25:24.293201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:56.012910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293307+0000","last_change":"2024-04-05T22:25:23.297181+0000","last_active":"2024-04-05T22:25:24.293307+0000","last_peered":"2024-04-05T22:25:24.293307+0000","last_clean":"2024-04-05T22:25:24.293307+0000","last_became_active":"2024-04-05T22:25:23.293820+0000","last_became_peered":"2024-04-05T22:25:23.293820+0000","last_unstale":"2024-04-05T22:25:24.293307+0000","last_undegraded":"2024-04-05T22:25:24.293307+0000","last_fullsized":"2024-04-05T22:25:24.293307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307255+0000","last_change":"2024-04-05T22:25:23.313239+0000","last_active":"2024-04-05T22:25:24.307255+0000","last_peered":"2024-04-05T22:25:24.307255+0000","last_clean":"2024-04-05T22:25:24.307255+0000","last_became_active":"2024-04-05T22:25:23.312991+0000","last_became_peered":"2024-04-05T22:25:23.312991+0000","last_unstale":"2024-04-05T22:25:24.307255+0000","last_undegraded":"2024-04-05T22:25:24.307255+0000","last_fullsized":"2024-04-05T22:25:24.307255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:55.808811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308785+0000","last_change":"2024-04-05T22:25:23.317075+0000","last_active":"2024-04-05T22:25:24.308785+0000","last_peered":"2024-04-05T22:25:24.308785+0000","last_clean":"2024-04-05T22:25:24.308785+0000","last_became_active":"2024-04-05T22:25:23.314962+0000","last_became_peered":"2024-04-05T22:25:23.314962+0000","last_unstale":"2024-04-05T22:25:24.308785+0000","last_undegraded":"2024-04-05T22:25:24.308785+0000","last_fullsized":"2024-04-05T22:25:24.308785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:48.490866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311239+0000","last_change":"2024-04-05T22:25:23.318987+0000","last_active":"2024-04-05T22:25:24.311239+0000","last_peered":"2024-04-05T22:25:24.311239+0000","last_clean":"2024-04-05T22:25:24.311239+0000","last_became_active":"2024-04-05T22:25:23.317905+0000","last_became_peered":"2024-04-05T22:25:23.317905+0000","last_unstale":"2024-04-05T22:25:24.311239+0000","last_undegraded":"2024-04-05T22:25:24.311239+0000","last_fullsized":"2024-04-05T22:25:24.311239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:03:19.914501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290417+0000","last_change":"2024-04-05T22:25:23.297860+0000","last_active":"2024-04-05T22:25:24.290417+0000","last_peered":"2024-04-05T22:25:24.290417+0000","last_clean":"2024-04-05T22:25:24.290417+0000","last_became_active":"2024-04-05T22:25:23.297615+0000","last_became_peered":"2024-04-05T22:25:23.297615+0000","last_unstale":"2024-04-05T22:25:24.290417+0000","last_undegraded":"2024-04-05T22:25:24.290417+0000","last_fullsized":"2024-04-05T22:25:24.290417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:19:55.287884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308585+0000","last_change":"2024-04-05T22:25:23.294183+0000","last_active":"2024-04-05T22:25:24.308585+0000","last_peered":"2024-04-05T22:25:24.308585+0000","last_clean":"2024-04-05T22:25:24.308585+0000","last_became_active":"2024-04-05T22:25:23.293894+0000","last_became_peered":"2024-04-05T22:25:23.293894+0000","last_unstale":"2024-04-05T22:25:24.308585+0000","last_undegraded":"2024-04-05T22:25:24.308585+0000","last_fullsized":"2024-04-05T22:25:24.308585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:06:40.304684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311574+0000","last_change":"2024-04-05T22:25:23.323578+0000","last_active":"2024-04-05T22:25:24.311574+0000","last_peered":"2024-04-05T22:25:24.311574+0000","last_clean":"2024-04-05T22:25:24.311574+0000","last_became_active":"2024-04-05T22:25:23.323250+0000","last_became_peered":"2024-04-05T22:25:23.323250+0000","last_unstale":"2024-04-05T22:25:24.311574+0000","last_undegraded":"2024-04-05T22:25:24.311574+0000","last_fullsized":"2024-04-05T22:25:24.311574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308475+0000","last_change":"2024-04-05T22:25:23.297418+0000","last_active":"2024-04-05T22:25:24.308475+0000","last_peered":"2024-04-05T22:25:24.308475+0000","last_clean":"2024-04-05T22:25:24.308475+0000","last_became_active":"2024-04-05T22:25:23.297075+0000","last_became_peered":"2024-04-05T22:25:23.297075+0000","last_unstale":"2024-04-05T22:25:24.308475+0000","last_undegraded":"2024-04-05T22:25:24.308475+0000","last_fullsized":"2024-04-05T22:25:24.308475+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:45.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310394+0000","last_change":"2024-04-05T22:25:23.320793+0000","last_active":"2024-04-05T22:25:24.310394+0000","last_peered":"2024-04-05T22:25:24.310394+0000","last_clean":"2024-04-05T22:25:24.310394+0000","last_became_active":"2024-04-05T22:25:23.316015+0000","last_became_peered":"2024-04-05T22:25:23.316015+0000","last_unstale":"2024-04-05T22:25:24.310394+0000","last_undegraded":"2024-04-05T22:25:24.310394+0000","last_fullsized":"2024-04-05T22:25:24.310394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309223+0000","last_change":"2024-04-05T22:25:23.315728+0000","last_active":"2024-04-05T22:25:24.309223+0000","last_peered":"2024-04-05T22:25:24.309223+0000","last_clean":"2024-04-05T22:25:24.309223+0000","last_became_active":"2024-04-05T22:25:23.314345+0000","last_became_peered":"2024-04-05T22:25:23.314345+0000","last_unstale":"2024-04-05T22:25:24.309223+0000","last_undegraded":"2024-04-05T22:25:24.309223+0000","last_fullsized":"2024-04-05T22:25:24.309223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:50.046379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290866+0000","last_change":"2024-04-05T22:25:23.302044+0000","last_active":"2024-04-05T22:25:24.290866+0000","last_peered":"2024-04-05T22:25:24.290866+0000","last_clean":"2024-04-05T22:25:24.290866+0000","last_became_active":"2024-04-05T22:25:23.301336+0000","last_became_peered":"2024-04-05T22:25:23.301336+0000","last_unstale":"2024-04-05T22:25:24.290866+0000","last_undegraded":"2024-04-05T22:25:24.290866+0000","last_fullsized":"2024-04-05T22:25:24.290866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:42.782908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311640+0000","last_change":"2024-04-05T22:25:23.321078+0000","last_active":"2024-04-05T22:25:24.311640+0000","last_peered":"2024-04-05T22:25:24.311640+0000","last_clean":"2024-04-05T22:25:24.311640+0000","last_became_active":"2024-04-05T22:25:23.315835+0000","last_became_peered":"2024-04-05T22:25:23.315835+0000","last_unstale":"2024-04-05T22:25:24.311640+0000","last_undegraded":"2024-04-05T22:25:24.311640+0000","last_fullsized":"2024-04-05T22:25:24.311640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308033+0000","last_change":"2024-04-05T22:25:23.316796+0000","last_active":"2024-04-05T22:25:24.308033+0000","last_peered":"2024-04-05T22:25:24.308033+0000","last_clean":"2024-04-05T22:25:24.308033+0000","last_became_active":"2024-04-05T22:25:23.316540+0000","last_became_peered":"2024-04-05T22:25:23.316540+0000","last_unstale":"2024-04-05T22:25:24.308033+0000","last_undegraded":"2024-04-05T22:25:24.308033+0000","last_fullsized":"2024-04-05T22:25:24.308033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290359+0000","last_change":"2024-04-05T22:25:23.318867+0000","last_active":"2024-04-05T22:25:24.290359+0000","last_peered":"2024-04-05T22:25:24.290359+0000","last_clean":"2024-04-05T22:25:24.290359+0000","last_became_active":"2024-04-05T22:25:23.318583+0000","last_became_peered":"2024-04-05T22:25:23.318583+0000","last_unstale":"2024-04-05T22:25:24.290359+0000","last_undegraded":"2024-04-05T22:25:24.290359+0000","last_fullsized":"2024-04-05T22:25:24.290359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:20.782795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308872+0000","last_change":"2024-04-05T22:25:23.309759+0000","last_active":"2024-04-05T22:25:24.308872+0000","last_peered":"2024-04-05T22:25:24.308872+0000","last_clean":"2024-04-05T22:25:24.308872+0000","last_became_active":"2024-04-05T22:25:23.309475+0000","last_became_peered":"2024-04-05T22:25:23.309475+0000","last_unstale":"2024-04-05T22:25:24.308872+0000","last_undegraded":"2024-04-05T22:25:24.308872+0000","last_fullsized":"2024-04-05T22:25:24.308872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:40.089255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310131+0000","last_change":"2024-04-05T22:25:23.318074+0000","last_active":"2024-04-05T22:25:24.310131+0000","last_peered":"2024-04-05T22:25:24.310131+0000","last_clean":"2024-04-05T22:25:24.310131+0000","last_became_active":"2024-04-05T22:25:23.317792+0000","last_became_peered":"2024-04-05T22:25:23.317792+0000","last_unstale":"2024-04-05T22:25:24.310131+0000","last_undegraded":"2024-04-05T22:25:24.310131+0000","last_fullsized":"2024-04-05T22:25:24.310131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290117+0000","last_change":"2024-04-05T22:25:23.301201+0000","last_active":"2024-04-05T22:25:24.290117+0000","last_peered":"2024-04-05T22:25:24.290117+0000","last_clean":"2024-04-05T22:25:24.290117+0000","last_became_active":"2024-04-05T22:25:23.300963+0000","last_became_peered":"2024-04-05T22:25:23.300963+0000","last_unstale":"2024-04-05T22:25:24.290117+0000","last_undegraded":"2024-04-05T22:25:24.290117+0000","last_fullsized":"2024-04-05T22:25:24.290117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:29.929763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307353+0000","last_change":"2024-04-05T22:25:23.298905+0000","last_active":"2024-04-05T22:25:24.307353+0000","last_peered":"2024-04-05T22:25:24.307353+0000","last_clean":"2024-04-05T22:25:24.307353+0000","last_became_active":"2024-04-05T22:25:23.297849+0000","last_became_peered":"2024-04-05T22:25:23.297849+0000","last_unstale":"2024-04-05T22:25:24.307353+0000","last_undegraded":"2024-04-05T22:25:24.307353+0000","last_fullsized":"2024-04-05T22:25:24.307353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:12.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307777+0000","last_change":"2024-04-05T22:25:23.316032+0000","last_active":"2024-04-05T22:25:24.307777+0000","last_peered":"2024-04-05T22:25:24.307777+0000","last_clean":"2024-04-05T22:25:24.307777+0000","last_became_active":"2024-04-05T22:25:23.315784+0000","last_became_peered":"2024-04-05T22:25:23.315784+0000","last_unstale":"2024-04-05T22:25:24.307777+0000","last_undegraded":"2024-04-05T22:25:24.307777+0000","last_fullsized":"2024-04-05T22:25:24.307777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:36:12.120029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.321362+0000","last_change":"2024-04-05T22:25:22.820217+0000","last_active":"2024-04-05T22:25:23.321362+0000","last_peered":"2024-04-05T22:25:23.321362+0000","last_clean":"2024-04-05T22:25:23.321362+0000","last_became_active":"2024-04-05T22:25:22.819880+0000","last_became_peered":"2024-04-05T22:25:22.819880+0000","last_unstale":"2024-04-05T22:25:23.321362+0000","last_undegraded":"2024-04-05T22:25:23.321362+0000","last_fullsized":"2024-04-05T22:25:23.321362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:54.998254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291479+0000","last_change":"2024-04-05T22:25:23.308426+0000","last_active":"2024-04-05T22:25:24.291479+0000","last_peered":"2024-04-05T22:25:24.291479+0000","last_clean":"2024-04-05T22:25:24.291479+0000","last_became_active":"2024-04-05T22:25:23.308165+0000","last_became_peered":"2024-04-05T22:25:23.308165+0000","last_unstale":"2024-04-05T22:25:24.291479+0000","last_undegraded":"2024-04-05T22:25:24.291479+0000","last_fullsized":"2024-04-05T22:25:24.291479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314544+0000","last_change":"2024-04-05T22:25:22.818905+0000","last_active":"2024-04-05T22:25:23.314544+0000","last_peered":"2024-04-05T22:25:23.314544+0000","last_clean":"2024-04-05T22:25:23.314544+0000","last_became_active":"2024-04-05T22:25:22.818565+0000","last_became_peered":"2024-04-05T22:25:22.818565+0000","last_unstale":"2024-04-05T22:25:23.314544+0000","last_undegraded":"2024-04-05T22:25:23.314544+0000","last_fullsized":"2024-04-05T22:25:23.314544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:54.794155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308212+0000","last_change":"2024-04-05T22:25:23.299031+0000","last_active":"2024-04-05T22:25:24.308212+0000","last_peered":"2024-04-05T22:25:24.308212+0000","last_clean":"2024-04-05T22:25:24.308212+0000","last_became_active":"2024-04-05T22:25:23.298753+0000","last_became_peered":"2024-04-05T22:25:23.298753+0000","last_unstale":"2024-04-05T22:25:24.308212+0000","last_undegraded":"2024-04-05T22:25:24.308212+0000","last_fullsized":"2024-04-05T22:25:24.308212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306956+0000","last_change":"2024-04-05T22:25:22.807855+0000","last_active":"2024-04-05T22:25:23.306956+0000","last_peered":"2024-04-05T22:25:23.306956+0000","last_clean":"2024-04-05T22:25:23.306956+0000","last_became_active":"2024-04-05T22:25:22.807584+0000","last_became_peered":"2024-04-05T22:25:22.807584+0000","last_unstale":"2024-04-05T22:25:23.306956+0000","last_undegraded":"2024-04-05T22:25:23.306956+0000","last_fullsized":"2024-04-05T22:25:23.306956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307780+0000","last_change":"2024-04-05T22:25:23.321058+0000","last_active":"2024-04-05T22:25:24.307780+0000","last_peered":"2024-04-05T22:25:24.307780+0000","last_clean":"2024-04-05T22:25:24.307780+0000","last_became_active":"2024-04-05T22:25:23.320468+0000","last_became_peered":"2024-04-05T22:25:23.320468+0000","last_unstale":"2024-04-05T22:25:24.307780+0000","last_undegraded":"2024-04-05T22:25:24.307780+0000","last_fullsized":"2024-04-05T22:25:24.307780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:42:19.724742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.326384+0000","last_change":"2024-04-05T22:25:22.822056+0000","last_active":"2024-04-05T22:25:23.326384+0000","last_peered":"2024-04-05T22:25:23.326384+0000","last_clean":"2024-04-05T22:25:23.326384+0000","last_became_active":"2024-04-05T22:25:22.821071+0000","last_became_peered":"2024-04-05T22:25:22.821071+0000","last_unstale":"2024-04-05T22:25:23.326384+0000","last_undegraded":"2024-04-05T22:25:23.326384+0000","last_fullsized":"2024-04-05T22:25:23.326384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308559+0000","last_change":"2024-04-05T22:25:23.294038+0000","last_active":"2024-04-05T22:25:24.308559+0000","last_peered":"2024-04-05T22:25:24.308559+0000","last_clean":"2024-04-05T22:25:24.308559+0000","last_became_active":"2024-04-05T22:25:23.293856+0000","last_became_peered":"2024-04-05T22:25:23.293856+0000","last_unstale":"2024-04-05T22:25:24.308559+0000","last_undegraded":"2024-04-05T22:25:24.308559+0000","last_fullsized":"2024-04-05T22:25:24.308559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:23:59.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311539+0000","last_change":"2024-04-05T22:25:22.813362+0000","last_active":"2024-04-05T22:25:23.311539+0000","last_peered":"2024-04-05T22:25:23.311539+0000","last_clean":"2024-04-05T22:25:23.311539+0000","last_became_active":"2024-04-05T22:25:22.812956+0000","last_became_peered":"2024-04-05T22:25:22.812956+0000","last_unstale":"2024-04-05T22:25:23.311539+0000","last_undegraded":"2024-04-05T22:25:23.311539+0000","last_fullsized":"2024-04-05T22:25:23.311539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308570+0000","last_change":"2024-04-05T22:25:23.297447+0000","last_active":"2024-04-05T22:25:24.308570+0000","last_peered":"2024-04-05T22:25:24.308570+0000","last_clean":"2024-04-05T22:25:24.308570+0000","last_became_active":"2024-04-05T22:25:23.294666+0000","last_became_peered":"2024-04-05T22:25:23.294666+0000","last_unstale":"2024-04-05T22:25:24.308570+0000","last_undegraded":"2024-04-05T22:25:24.308570+0000","last_fullsized":"2024-04-05T22:25:24.308570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:33.835122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306675+0000","last_change":"2024-04-05T22:25:22.812184+0000","last_active":"2024-04-05T22:25:23.306675+0000","last_peered":"2024-04-05T22:25:23.306675+0000","last_clean":"2024-04-05T22:25:23.306675+0000","last_became_active":"2024-04-05T22:25:22.811919+0000","last_became_peered":"2024-04-05T22:25:22.811919+0000","last_unstale":"2024-04-05T22:25:23.306675+0000","last_undegraded":"2024-04-05T22:25:23.306675+0000","last_fullsized":"2024-04-05T22:25:23.306675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309323+0000","last_change":"2024-04-05T22:25:23.312968+0000","last_active":"2024-04-05T22:25:24.309323+0000","last_peered":"2024-04-05T22:25:24.309323+0000","last_clean":"2024-04-05T22:25:24.309323+0000","last_became_active":"2024-04-05T22:25:23.312697+0000","last_became_peered":"2024-04-05T22:25:23.312697+0000","last_unstale":"2024-04-05T22:25:24.309323+0000","last_undegraded":"2024-04-05T22:25:24.309323+0000","last_fullsized":"2024-04-05T22:25:24.309323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:43:47.570184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305456+0000","last_change":"2024-04-05T22:25:22.824792+0000","last_active":"2024-04-05T22:25:23.305456+0000","last_peered":"2024-04-05T22:25:23.305456+0000","last_clean":"2024-04-05T22:25:23.305456+0000","last_became_active":"2024-04-05T22:25:22.822520+0000","last_became_peered":"2024-04-05T22:25:22.822520+0000","last_unstale":"2024-04-05T22:25:23.305456+0000","last_undegraded":"2024-04-05T22:25:23.305456+0000","last_fullsized":"2024-04-05T22:25:23.305456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:47.138497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306877+0000","last_change":"2024-04-05T22:25:23.315314+0000","last_active":"2024-04-05T22:25:24.306877+0000","last_peered":"2024-04-05T22:25:24.306877+0000","last_clean":"2024-04-05T22:25:24.306877+0000","last_became_active":"2024-04-05T22:25:23.314998+0000","last_became_peered":"2024-04-05T22:25:23.314998+0000","last_unstale":"2024-04-05T22:25:24.306877+0000","last_undegraded":"2024-04-05T22:25:24.306877+0000","last_fullsized":"2024-04-05T22:25:24.306877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283790+0000","last_change":"2024-04-05T22:25:22.813243+0000","last_active":"2024-04-05T22:25:23.283790+0000","last_peered":"2024-04-05T22:25:23.283790+0000","last_clean":"2024-04-05T22:25:23.283790+0000","last_became_active":"2024-04-05T22:25:22.810600+0000","last_became_peered":"2024-04-05T22:25:22.810600+0000","last_unstale":"2024-04-05T22:25:23.283790+0000","last_undegraded":"2024-04-05T22:25:23.283790+0000","last_fullsized":"2024-04-05T22:25:23.283790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294078+0000","last_change":"2024-04-05T22:25:23.298671+0000","last_active":"2024-04-05T22:25:24.294078+0000","last_peered":"2024-04-05T22:25:24.294078+0000","last_clean":"2024-04-05T22:25:24.294078+0000","last_became_active":"2024-04-05T22:25:23.294595+0000","last_became_peered":"2024-04-05T22:25:23.294595+0000","last_unstale":"2024-04-05T22:25:24.294078+0000","last_undegraded":"2024-04-05T22:25:24.294078+0000","last_fullsized":"2024-04-05T22:25:24.294078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304213+0000","last_change":"2024-04-05T22:25:22.823584+0000","last_active":"2024-04-05T22:25:23.304213+0000","last_peered":"2024-04-05T22:25:23.304213+0000","last_clean":"2024-04-05T22:25:23.304213+0000","last_became_active":"2024-04-05T22:25:22.823316+0000","last_became_peered":"2024-04-05T22:25:22.823316+0000","last_unstale":"2024-04-05T22:25:23.304213+0000","last_undegraded":"2024-04-05T22:25:23.304213+0000","last_fullsized":"2024-04-05T22:25:23.304213+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312394+0000","last_change":"2024-04-05T22:25:23.314057+0000","last_active":"2024-04-05T22:25:24.312394+0000","last_peered":"2024-04-05T22:25:24.312394+0000","last_clean":"2024-04-05T22:25:24.312394+0000","last_became_active":"2024-04-05T22:25:23.312904+0000","last_became_peered":"2024-04-05T22:25:23.312904+0000","last_unstale":"2024-04-05T22:25:24.312394+0000","last_undegraded":"2024-04-05T22:25:24.312394+0000","last_fullsized":"2024-04-05T22:25:24.312394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283719+0000","last_change":"2024-04-05T22:25:22.812954+0000","last_active":"2024-04-05T22:25:23.283719+0000","last_peered":"2024-04-05T22:25:23.283719+0000","last_clean":"2024-04-05T22:25:23.283719+0000","last_became_active":"2024-04-05T22:25:22.812685+0000","last_became_peered":"2024-04-05T22:25:22.812685+0000","last_unstale":"2024-04-05T22:25:23.283719+0000","last_undegraded":"2024-04-05T22:25:23.283719+0000","last_fullsized":"2024-04-05T22:25:23.283719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:36.447234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291127+0000","last_change":"2024-04-05T22:25:23.307204+0000","last_active":"2024-04-05T22:25:24.291127+0000","last_peered":"2024-04-05T22:25:24.291127+0000","last_clean":"2024-04-05T22:25:24.291127+0000","last_became_active":"2024-04-05T22:25:23.306766+0000","last_became_peered":"2024-04-05T22:25:23.306766+0000","last_unstale":"2024-04-05T22:25:24.291127+0000","last_undegraded":"2024-04-05T22:25:24.291127+0000","last_fullsized":"2024-04-05T22:25:24.291127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:27.638305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311719+0000","last_change":"2024-04-05T22:25:22.820936+0000","last_active":"2024-04-05T22:25:23.311719+0000","last_peered":"2024-04-05T22:25:23.311719+0000","last_clean":"2024-04-05T22:25:23.311719+0000","last_became_active":"2024-04-05T22:25:22.820271+0000","last_became_peered":"2024-04-05T22:25:22.820271+0000","last_unstale":"2024-04-05T22:25:23.311719+0000","last_undegraded":"2024-04-05T22:25:23.311719+0000","last_fullsized":"2024-04-05T22:25:23.311719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312194+0000","last_change":"2024-04-05T22:25:23.309672+0000","last_active":"2024-04-05T22:25:24.312194+0000","last_peered":"2024-04-05T22:25:24.312194+0000","last_clean":"2024-04-05T22:25:24.312194+0000","last_became_active":"2024-04-05T22:25:23.308313+0000","last_became_peered":"2024-04-05T22:25:23.308313+0000","last_unstale":"2024-04-05T22:25:24.312194+0000","last_undegraded":"2024-04-05T22:25:24.312194+0000","last_fullsized":"2024-04-05T22:25:24.312194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.322355+0000","last_change":"2024-04-05T22:25:22.812066+0000","last_active":"2024-04-05T22:25:23.322355+0000","last_peered":"2024-04-05T22:25:23.322355+0000","last_clean":"2024-04-05T22:25:23.322355+0000","last_became_active":"2024-04-05T22:25:22.811662+0000","last_became_peered":"2024-04-05T22:25:22.811662+0000","last_unstale":"2024-04-05T22:25:23.322355+0000","last_undegraded":"2024-04-05T22:25:23.322355+0000","last_fullsized":"2024-04-05T22:25:23.322355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:04.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311269+0000","last_change":"2024-04-05T22:25:23.321510+0000","last_active":"2024-04-05T22:25:24.311269+0000","last_peered":"2024-04-05T22:25:24.311269+0000","last_clean":"2024-04-05T22:25:24.311269+0000","last_became_active":"2024-04-05T22:25:23.319576+0000","last_became_peered":"2024-04-05T22:25:23.319576+0000","last_unstale":"2024-04-05T22:25:24.311269+0000","last_undegraded":"2024-04-05T22:25:24.311269+0000","last_fullsized":"2024-04-05T22:25:24.311269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302829+0000","last_change":"2024-04-05T22:25:22.815327+0000","last_active":"2024-04-05T22:25:23.302829+0000","last_peered":"2024-04-05T22:25:23.302829+0000","last_clean":"2024-04-05T22:25:23.302829+0000","last_became_active":"2024-04-05T22:25:22.814733+0000","last_became_peered":"2024-04-05T22:25:22.814733+0000","last_unstale":"2024-04-05T22:25:23.302829+0000","last_undegraded":"2024-04-05T22:25:23.302829+0000","last_fullsized":"2024-04-05T22:25:23.302829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:36.497236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311892+0000","last_change":"2024-04-05T22:25:23.316894+0000","last_active":"2024-04-05T22:25:24.311892+0000","last_peered":"2024-04-05T22:25:24.311892+0000","last_clean":"2024-04-05T22:25:24.311892+0000","last_became_active":"2024-04-05T22:25:23.316563+0000","last_became_peered":"2024-04-05T22:25:23.316563+0000","last_unstale":"2024-04-05T22:25:24.311892+0000","last_undegraded":"2024-04-05T22:25:24.311892+0000","last_fullsized":"2024-04-05T22:25:24.311892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:28.048899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309891+0000","last_change":"2024-04-05T22:25:22.817554+0000","last_active":"2024-04-05T22:25:23.309891+0000","last_peered":"2024-04-05T22:25:23.309891+0000","last_clean":"2024-04-05T22:25:23.309891+0000","last_became_active":"2024-04-05T22:25:22.816462+0000","last_became_peered":"2024-04-05T22:25:22.816462+0000","last_unstale":"2024-04-05T22:25:23.309891+0000","last_undegraded":"2024-04-05T22:25:23.309891+0000","last_fullsized":"2024-04-05T22:25:23.309891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:02.494750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293141+0000","last_change":"2024-04-05T22:25:23.309583+0000","last_active":"2024-04-05T22:25:24.293141+0000","last_peered":"2024-04-05T22:25:24.293141+0000","last_clean":"2024-04-05T22:25:24.293141+0000","last_became_active":"2024-04-05T22:25:23.307782+0000","last_became_peered":"2024-04-05T22:25:23.307782+0000","last_unstale":"2024-04-05T22:25:24.293141+0000","last_undegraded":"2024-04-05T22:25:24.293141+0000","last_fullsized":"2024-04-05T22:25:24.293141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305585+0000","last_change":"2024-04-05T22:25:22.821958+0000","last_active":"2024-04-05T22:25:23.305585+0000","last_peered":"2024-04-05T22:25:23.305585+0000","last_clean":"2024-04-05T22:25:23.305585+0000","last_became_active":"2024-04-05T22:25:22.821425+0000","last_became_peered":"2024-04-05T22:25:22.821425+0000","last_unstale":"2024-04-05T22:25:23.305585+0000","last_undegraded":"2024-04-05T22:25:23.305585+0000","last_fullsized":"2024-04-05T22:25:23.305585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310387+0000","last_change":"2024-04-05T22:25:23.310520+0000","last_active":"2024-04-05T22:25:24.310387+0000","last_peered":"2024-04-05T22:25:24.310387+0000","last_clean":"2024-04-05T22:25:24.310387+0000","last_became_active":"2024-04-05T22:25:23.310234+0000","last_became_peered":"2024-04-05T22:25:23.310234+0000","last_unstale":"2024-04-05T22:25:24.310387+0000","last_undegraded":"2024-04-05T22:25:24.310387+0000","last_fullsized":"2024-04-05T22:25:24.310387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.325150+0000","last_change":"2024-04-05T22:25:22.822654+0000","last_active":"2024-04-05T22:25:23.325150+0000","last_peered":"2024-04-05T22:25:23.325150+0000","last_clean":"2024-04-05T22:25:23.325150+0000","last_became_active":"2024-04-05T22:25:22.822260+0000","last_became_peered":"2024-04-05T22:25:22.822260+0000","last_unstale":"2024-04-05T22:25:23.325150+0000","last_undegraded":"2024-04-05T22:25:23.325150+0000","last_fullsized":"2024-04-05T22:25:23.325150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312013+0000","last_change":"2024-04-05T22:25:23.318981+0000","last_active":"2024-04-05T22:25:24.312013+0000","last_peered":"2024-04-05T22:25:24.312013+0000","last_clean":"2024-04-05T22:25:24.312013+0000","last_became_active":"2024-04-05T22:25:23.318382+0000","last_became_peered":"2024-04-05T22:25:23.318382+0000","last_unstale":"2024-04-05T22:25:24.312013+0000","last_undegraded":"2024-04-05T22:25:24.312013+0000","last_fullsized":"2024-04-05T22:25:24.312013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:42:57.769217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302992+0000","last_change":"2024-04-05T22:25:22.817215+0000","last_active":"2024-04-05T22:25:23.302992+0000","last_peered":"2024-04-05T22:25:23.302992+0000","last_clean":"2024-04-05T22:25:23.302992+0000","last_became_active":"2024-04-05T22:25:22.816814+0000","last_became_peered":"2024-04-05T22:25:22.816814+0000","last_unstale":"2024-04-05T22:25:23.302992+0000","last_undegraded":"2024-04-05T22:25:23.302992+0000","last_fullsized":"2024-04-05T22:25:23.302992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:26.685831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311184+0000","last_change":"2024-04-05T22:25:23.321785+0000","last_active":"2024-04-05T22:25:24.311184+0000","last_peered":"2024-04-05T22:25:24.311184+0000","last_clean":"2024-04-05T22:25:24.311184+0000","last_became_active":"2024-04-05T22:25:23.318293+0000","last_became_peered":"2024-04-05T22:25:23.318293+0000","last_unstale":"2024-04-05T22:25:24.311184+0000","last_undegraded":"2024-04-05T22:25:24.311184+0000","last_fullsized":"2024-04-05T22:25:24.311184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.303053+0000","last_change":"2024-04-05T22:25:22.810298+0000","last_active":"2024-04-05T22:25:23.303053+0000","last_peered":"2024-04-05T22:25:23.303053+0000","last_clean":"2024-04-05T22:25:23.303053+0000","last_became_active":"2024-04-05T22:25:22.809980+0000","last_became_peered":"2024-04-05T22:25:22.809980+0000","last_unstale":"2024-04-05T22:25:23.303053+0000","last_undegraded":"2024-04-05T22:25:23.303053+0000","last_fullsized":"2024-04-05T22:25:23.303053+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308777+0000","last_change":"2024-04-05T22:25:23.310722+0000","last_active":"2024-04-05T22:25:24.308777+0000","last_peered":"2024-04-05T22:25:24.308777+0000","last_clean":"2024-04-05T22:25:24.308777+0000","last_became_active":"2024-04-05T22:25:23.310419+0000","last_became_peered":"2024-04-05T22:25:23.310419+0000","last_unstale":"2024-04-05T22:25:24.308777+0000","last_undegraded":"2024-04-05T22:25:24.308777+0000","last_fullsized":"2024-04-05T22:25:24.308777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:29:46.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.320593+0000","last_change":"2024-04-05T22:25:22.822964+0000","last_active":"2024-04-05T22:25:23.320593+0000","last_peered":"2024-04-05T22:25:23.320593+0000","last_clean":"2024-04-05T22:25:23.320593+0000","last_became_active":"2024-04-05T22:25:22.822645+0000","last_became_peered":"2024-04-05T22:25:22.822645+0000","last_unstale":"2024-04-05T22:25:23.320593+0000","last_undegraded":"2024-04-05T22:25:23.320593+0000","last_fullsized":"2024-04-05T22:25:23.320593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312362+0000","last_change":"2024-04-05T22:25:23.313098+0000","last_active":"2024-04-05T22:25:24.312362+0000","last_peered":"2024-04-05T22:25:24.312362+0000","last_clean":"2024-04-05T22:25:24.312362+0000","last_became_active":"2024-04-05T22:25:23.312388+0000","last_became_peered":"2024-04-05T22:25:23.312388+0000","last_unstale":"2024-04-05T22:25:24.312362+0000","last_undegraded":"2024-04-05T22:25:24.312362+0000","last_fullsized":"2024-04-05T22:25:24.312362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.287654+0000","last_change":"2024-04-05T22:25:22.810231+0000","last_active":"2024-04-05T22:25:23.287654+0000","last_peered":"2024-04-05T22:25:23.287654+0000","last_clean":"2024-04-05T22:25:23.287654+0000","last_became_active":"2024-04-05T22:25:22.809955+0000","last_became_peered":"2024-04-05T22:25:22.809955+0000","last_unstale":"2024-04-05T22:25:23.287654+0000","last_undegraded":"2024-04-05T22:25:23.287654+0000","last_fullsized":"2024-04-05T22:25:23.287654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311586+0000","last_change":"2024-04-05T22:25:23.308720+0000","last_active":"2024-04-05T22:25:24.311586+0000","last_peered":"2024-04-05T22:25:24.311586+0000","last_clean":"2024-04-05T22:25:24.311586+0000","last_became_active":"2024-04-05T22:25:23.308461+0000","last_became_peered":"2024-04-05T22:25:23.308461+0000","last_unstale":"2024-04-05T22:25:24.311586+0000","last_undegraded":"2024-04-05T22:25:24.311586+0000","last_fullsized":"2024-04-05T22:25:24.311586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314414+0000","last_change":"2024-04-05T22:25:22.808626+0000","last_active":"2024-04-05T22:25:23.314414+0000","last_peered":"2024-04-05T22:25:23.314414+0000","last_clean":"2024-04-05T22:25:23.314414+0000","last_became_active":"2024-04-05T22:25:22.808159+0000","last_became_peered":"2024-04-05T22:25:22.808159+0000","last_unstale":"2024-04-05T22:25:23.314414+0000","last_undegraded":"2024-04-05T22:25:23.314414+0000","last_fullsized":"2024-04-05T22:25:23.314414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:19.370558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307477+0000","last_change":"2024-04-05T22:25:23.310723+0000","last_active":"2024-04-05T22:25:24.307477+0000","last_peered":"2024-04-05T22:25:24.307477+0000","last_clean":"2024-04-05T22:25:24.307477+0000","last_became_active":"2024-04-05T22:25:23.310431+0000","last_became_peered":"2024-04-05T22:25:23.310431+0000","last_unstale":"2024-04-05T22:25:24.307477+0000","last_undegraded":"2024-04-05T22:25:24.307477+0000","last_fullsized":"2024-04-05T22:25:24.307477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:20.385215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279422+0000","last_change":"2024-04-05T22:25:22.809387+0000","last_active":"2024-04-05T22:25:23.279422+0000","last_peered":"2024-04-05T22:25:23.279422+0000","last_clean":"2024-04-05T22:25:23.279422+0000","last_became_active":"2024-04-05T22:25:22.809109+0000","last_became_peered":"2024-04-05T22:25:22.809109+0000","last_unstale":"2024-04-05T22:25:23.279422+0000","last_undegraded":"2024-04-05T22:25:23.279422+0000","last_fullsized":"2024-04-05T22:25:23.279422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307167+0000","last_change":"2024-04-05T22:25:23.316165+0000","last_active":"2024-04-05T22:25:24.307167+0000","last_peered":"2024-04-05T22:25:24.307167+0000","last_clean":"2024-04-05T22:25:24.307167+0000","last_became_active":"2024-04-05T22:25:23.315655+0000","last_became_peered":"2024-04-05T22:25:23.315655+0000","last_unstale":"2024-04-05T22:25:24.307167+0000","last_undegraded":"2024-04-05T22:25:24.307167+0000","last_fullsized":"2024-04-05T22:25:24.307167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304412+0000","last_change":"2024-04-05T22:25:22.820823+0000","last_active":"2024-04-05T22:25:23.304412+0000","last_peered":"2024-04-05T22:25:23.304412+0000","last_clean":"2024-04-05T22:25:23.304412+0000","last_became_active":"2024-04-05T22:25:22.819543+0000","last_became_peered":"2024-04-05T22:25:22.819543+0000","last_unstale":"2024-04-05T22:25:23.304412+0000","last_undegraded":"2024-04-05T22:25:23.304412+0000","last_fullsized":"2024-04-05T22:25:23.304412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312774+0000","last_change":"2024-04-05T22:25:23.304349+0000","last_active":"2024-04-05T22:25:24.312774+0000","last_peered":"2024-04-05T22:25:24.312774+0000","last_clean":"2024-04-05T22:25:24.312774+0000","last_became_active":"2024-04-05T22:25:23.304014+0000","last_became_peered":"2024-04-05T22:25:23.304014+0000","last_unstale":"2024-04-05T22:25:24.312774+0000","last_undegraded":"2024-04-05T22:25:24.312774+0000","last_fullsized":"2024-04-05T22:25:24.312774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305751+0000","last_change":"2024-04-05T22:25:22.813364+0000","last_active":"2024-04-05T22:25:23.305751+0000","last_peered":"2024-04-05T22:25:23.305751+0000","last_clean":"2024-04-05T22:25:23.305751+0000","last_became_active":"2024-04-05T22:25:22.813068+0000","last_became_peered":"2024-04-05T22:25:22.813068+0000","last_unstale":"2024-04-05T22:25:23.305751+0000","last_undegraded":"2024-04-05T22:25:23.305751+0000","last_fullsized":"2024-04-05T22:25:23.305751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311439+0000","last_change":"2024-04-05T22:25:23.321716+0000","last_active":"2024-04-05T22:25:24.311439+0000","last_peered":"2024-04-05T22:25:24.311439+0000","last_clean":"2024-04-05T22:25:24.311439+0000","last_became_active":"2024-04-05T22:25:23.321428+0000","last_became_peered":"2024-04-05T22:25:23.321428+0000","last_unstale":"2024-04-05T22:25:24.311439+0000","last_undegraded":"2024-04-05T22:25:24.311439+0000","last_fullsized":"2024-04-05T22:25:24.311439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.288133+0000","last_change":"2024-04-05T22:25:22.813803+0000","last_active":"2024-04-05T22:25:23.288133+0000","last_peered":"2024-04-05T22:25:23.288133+0000","last_clean":"2024-04-05T22:25:23.288133+0000","last_became_active":"2024-04-05T22:25:22.813365+0000","last_became_peered":"2024-04-05T22:25:22.813365+0000","last_unstale":"2024-04-05T22:25:23.288133+0000","last_undegraded":"2024-04-05T22:25:23.288133+0000","last_fullsized":"2024-04-05T22:25:23.288133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291236+0000","last_change":"2024-04-05T22:25:23.302602+0000","last_active":"2024-04-05T22:25:24.291236+0000","last_peered":"2024-04-05T22:25:24.291236+0000","last_clean":"2024-04-05T22:25:24.291236+0000","last_became_active":"2024-04-05T22:25:23.302356+0000","last_became_peered":"2024-04-05T22:25:23.302356+0000","last_unstale":"2024-04-05T22:25:24.291236+0000","last_undegraded":"2024-04-05T22:25:24.291236+0000","last_fullsized":"2024-04-05T22:25:24.291236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:27.824520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.312000+0000","last_change":"2024-04-05T22:25:22.812466+0000","last_active":"2024-04-05T22:25:23.312000+0000","last_peered":"2024-04-05T22:25:23.312000+0000","last_clean":"2024-04-05T22:25:23.312000+0000","last_became_active":"2024-04-05T22:25:22.812117+0000","last_became_peered":"2024-04-05T22:25:22.812117+0000","last_unstale":"2024-04-05T22:25:23.312000+0000","last_undegraded":"2024-04-05T22:25:23.312000+0000","last_fullsized":"2024-04-05T22:25:23.312000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:13.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307935+0000","last_change":"2024-04-05T22:25:23.321029+0000","last_active":"2024-04-05T22:25:24.307935+0000","last_peered":"2024-04-05T22:25:24.307935+0000","last_clean":"2024-04-05T22:25:24.307935+0000","last_became_active":"2024-04-05T22:25:23.320742+0000","last_became_peered":"2024-04-05T22:25:23.320742+0000","last_unstale":"2024-04-05T22:25:24.307935+0000","last_undegraded":"2024-04-05T22:25:24.307935+0000","last_fullsized":"2024-04-05T22:25:24.307935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:26.184734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.316800+0000","last_change":"2024-04-05T22:25:22.819850+0000","last_active":"2024-04-05T22:25:23.316800+0000","last_peered":"2024-04-05T22:25:23.316800+0000","last_clean":"2024-04-05T22:25:23.316800+0000","last_became_active":"2024-04-05T22:25:22.819495+0000","last_became_peered":"2024-04-05T22:25:22.819495+0000","last_unstale":"2024-04-05T22:25:23.316800+0000","last_undegraded":"2024-04-05T22:25:23.316800+0000","last_fullsized":"2024-04-05T22:25:23.316800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308979+0000","last_change":"2024-04-05T22:25:23.290975+0000","last_active":"2024-04-05T22:25:24.308979+0000","last_peered":"2024-04-05T22:25:24.308979+0000","last_clean":"2024-04-05T22:25:24.308979+0000","last_became_active":"2024-04-05T22:25:23.290733+0000","last_became_peered":"2024-04-05T22:25:23.290733+0000","last_unstale":"2024-04-05T22:25:24.308979+0000","last_undegraded":"2024-04-05T22:25:24.308979+0000","last_fullsized":"2024-04-05T22:25:24.308979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:29.453299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279621+0000","last_change":"2024-04-05T22:25:22.821995+0000","last_active":"2024-04-05T22:25:23.279621+0000","last_peered":"2024-04-05T22:25:23.279621+0000","last_clean":"2024-04-05T22:25:23.279621+0000","last_became_active":"2024-04-05T22:25:22.821657+0000","last_became_peered":"2024-04-05T22:25:22.821657+0000","last_unstale":"2024-04-05T22:25:23.279621+0000","last_undegraded":"2024-04-05T22:25:23.279621+0000","last_fullsized":"2024-04-05T22:25:23.279621+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312062+0000","last_change":"2024-04-05T22:25:23.307838+0000","last_active":"2024-04-05T22:25:24.312062+0000","last_peered":"2024-04-05T22:25:24.312062+0000","last_clean":"2024-04-05T22:25:24.312062+0000","last_became_active":"2024-04-05T22:25:23.307552+0000","last_became_peered":"2024-04-05T22:25:23.307552+0000","last_unstale":"2024-04-05T22:25:24.312062+0000","last_undegraded":"2024-04-05T22:25:24.312062+0000","last_fullsized":"2024-04-05T22:25:24.312062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:15.194717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283982+0000","last_change":"2024-04-05T22:25:22.809727+0000","last_active":"2024-04-05T22:25:23.283982+0000","last_peered":"2024-04-05T22:25:23.283982+0000","last_clean":"2024-04-05T22:25:23.283982+0000","last_became_active":"2024-04-05T22:25:22.809281+0000","last_became_peered":"2024-04-05T22:25:22.809281+0000","last_unstale":"2024-04-05T22:25:23.283982+0000","last_undegraded":"2024-04-05T22:25:23.283982+0000","last_fullsized":"2024-04-05T22:25:23.283982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290369+0000","last_change":"2024-04-05T22:25:23.312476+0000","last_active":"2024-04-05T22:25:24.290369+0000","last_peered":"2024-04-05T22:25:24.290369+0000","last_clean":"2024-04-05T22:25:24.290369+0000","last_became_active":"2024-04-05T22:25:23.312245+0000","last_became_peered":"2024-04-05T22:25:23.312245+0000","last_unstale":"2024-04-05T22:25:24.290369+0000","last_undegraded":"2024-04-05T22:25:24.290369+0000","last_fullsized":"2024-04-05T22:25:24.290369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279493+0000","last_change":"2024-04-05T22:25:22.805657+0000","last_active":"2024-04-05T22:25:23.279493+0000","last_peered":"2024-04-05T22:25:23.279493+0000","last_clean":"2024-04-05T22:25:23.279493+0000","last_became_active":"2024-04-05T22:25:22.805398+0000","last_became_peered":"2024-04-05T22:25:22.805398+0000","last_unstale":"2024-04-05T22:25:23.279493+0000","last_undegraded":"2024-04-05T22:25:23.279493+0000","last_fullsized":"2024-04-05T22:25:23.279493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308351+0000","last_change":"2024-04-05T22:25:23.315180+0000","last_active":"2024-04-05T22:25:24.308351+0000","last_peered":"2024-04-05T22:25:24.308351+0000","last_clean":"2024-04-05T22:25:24.308351+0000","last_became_active":"2024-04-05T22:25:23.314678+0000","last_became_peered":"2024-04-05T22:25:23.314678+0000","last_unstale":"2024-04-05T22:25:24.308351+0000","last_undegraded":"2024-04-05T22:25:24.308351+0000","last_fullsized":"2024-04-05T22:25:24.308351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:43.210562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.282244+0000","last_change":"2024-04-05T22:25:22.821960+0000","last_active":"2024-04-05T22:25:23.282244+0000","last_peered":"2024-04-05T22:25:23.282244+0000","last_clean":"2024-04-05T22:25:23.282244+0000","last_became_active":"2024-04-05T22:25:22.821679+0000","last_became_peered":"2024-04-05T22:25:22.821679+0000","last_unstale":"2024-04-05T22:25:23.282244+0000","last_undegraded":"2024-04-05T22:25:23.282244+0000","last_fullsized":"2024-04-05T22:25:23.282244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:11.859438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306832+0000","last_change":"2024-04-05T22:25:23.292675+0000","last_active":"2024-04-05T22:25:24.306832+0000","last_peered":"2024-04-05T22:25:24.306832+0000","last_clean":"2024-04-05T22:25:24.306832+0000","last_became_active":"2024-04-05T22:25:23.292226+0000","last_became_peered":"2024-04-05T22:25:23.292226+0000","last_unstale":"2024-04-05T22:25:24.306832+0000","last_undegraded":"2024-04-05T22:25:24.306832+0000","last_fullsized":"2024-04-05T22:25:24.306832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:45.144578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309603+0000","last_change":"2024-04-05T22:25:22.823366+0000","last_active":"2024-04-05T22:25:23.309603+0000","last_peered":"2024-04-05T22:25:23.309603+0000","last_clean":"2024-04-05T22:25:23.309603+0000","last_became_active":"2024-04-05T22:25:22.823082+0000","last_became_peered":"2024-04-05T22:25:22.823082+0000","last_unstale":"2024-04-05T22:25:23.309603+0000","last_undegraded":"2024-04-05T22:25:23.309603+0000","last_fullsized":"2024-04-05T22:25:23.309603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:59.775684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.296039+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:25:23.296039+0000","last_peered":"2024-04-05T22:25:23.296039+0000","last_clean":"2024-04-05T22:25:23.296039+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:25:23.296039+0000","last_undegraded":"2024-04-05T22:25:23.296039+0000","last_fullsized":"2024-04-05T22:25:23.296039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1343488,"data_stored":899715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":1.137},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.321},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.073}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.391}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.2370000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.544}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.623},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":741376,"data_stored":309187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":0.96899999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.508}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.248},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.681}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.796},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.365}]}]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":749568,"data_stored":309839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.042}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":1.256},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.331},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.52}]}]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":753664,"data_stored":310751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0509999999999999,"15min":0.999},"min":{"1min":0.84699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":1.673,"15min":1.673},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0980000000000001,"15min":1.0800000000000001},"min":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.343}]},{"osd":1,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1439999999999999,"15min":1.1299999999999999},"min":{"1min":0.875,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.236},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1539999999999999,"15min":1.1699999999999999},"min":{"1min":0.83399999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4450000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.95099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.482,"5min":1.974,"15min":1.974},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.89000000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.379},"min":{"1min":0.95999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.843,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.369,"15min":1.407},"min":{"1min":1.107,"5min":0.874,"15min":0.874},"max":{"1min":1.754,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.575},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3440000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.625,"15min":0.625},"max":{"1min":1.786,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.49}]},{"osd":6,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4610000000000001},"min":{"1min":0.95599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6379999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.706},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4410000000000001,"15min":1.4950000000000001},"min":{"1min":1.0149999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.655,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8819999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5029999999999999,"15min":1.536},"min":{"1min":1.1639999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.895,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.825},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5549999999999999,"15min":1.5760000000000001},"min":{"1min":1.05,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":2.54,"15min":2.54},"last":1.9279999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":1.23},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":1.121},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95999999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.044},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.204}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.464}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.377},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.2909999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":1},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.876}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.21}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.621},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.518}]}]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":745472,"data_stored":309277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4970000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.127}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.282}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":1.25},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.665}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.573},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.536}]}]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1351680,"data_stored":901701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.875}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:25:32.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:25:32.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.912+0000 7f457ee00640 1 Processor -- start 2024-04-05T22:25:32.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.912+0000 7f457ee00640 1 -- start start 2024-04-05T22:25:32.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457806cb80 0x7f457806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 0x7f457815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457806f5b0 con 0x7f45781587d0 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457806d500 con 0x7f457806cb80 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457815f220 con 0x7f457815e850 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f4577400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457806cb80 0x7f457806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f4577400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457806cb80 0x7f457806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51510/0 (socket says 172.21.15.73:51510) 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f4577400640 1 -- 172.21.15.73:0/2500249961 learned_addr learned my addr 172.21.15.73:0/2500249961 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.914+0000 7f457ce00640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 0x7f457815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.915+0000 7f4577400640 1 -- 172.21.15.73:0/2500249961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 msgr2=0x7f457815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.915+0000 7f4577400640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 0x7f457815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.915+0000 7f4577400640 1 -- 172.21.15.73:0/2500249961 >> v1:172.21.15.157:6789/0 conn(0x7f45781587d0 legacy=0x7f45780cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.915+0000 7f4577400640 1 -- 172.21.15.73:0/2500249961 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4578070410 con 0x7f457806cb80 2024-04-05T22:25:32.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.915+0000 7f457ce00640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 0x7f457815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:32.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.916+0000 7f4577400640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f457806cfc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f45640098a0 tx=0x7f456402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8df69d9a3c2a660 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:32.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.916+0000 7f4576a00640 1 -- 172.21.15.73:0/2500249961 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564031640 con 0x7f457806cb80 2024-04-05T22:25:32.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.916+0000 7f4576a00640 1 -- 172.21.15.73:0/2500249961 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45640317a0 con 0x7f457806cb80 2024-04-05T22:25:32.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.916+0000 7f4576a00640 1 -- 172.21.15.73:0/2500249961 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564031b30 con 0x7f457806cb80 2024-04-05T22:25:32.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 -- 172.21.15.73:0/2500249961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 msgr2=0x7f457806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:32.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f457806cfc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f45640098a0 tx=0x7f456402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 -- 172.21.15.73:0/2500249961 shutdown_connections 2024-04-05T22:25:32.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f457815e850 0x7f457815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 --2- 172.21.15.73:0/2500249961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f457806cfc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 -- 172.21.15.73:0/2500249961 >> 172.21.15.73:0/2500249961 conn(0x7f45780ce8e0 msgr2=0x7f45780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:32.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 -- 172.21.15.73:0/2500249961 shutdown_connections 2024-04-05T22:25:32.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.917+0000 7f457ee00640 1 -- 172.21.15.73:0/2500249961 wait complete. 2024-04-05T22:25:32.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.919+0000 7f457ee00640 1 Processor -- start 2024-04-05T22:25:32.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.919+0000 7f457ee00640 1 -- start start 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45781587d0 0x7f4578156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 0x7f45781567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457815e170 con 0x7f457806cb80 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457815c700 con 0x7f457815e850 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457815de90 con 0x7f45781587d0 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51520/0 (socket says 172.21.15.73:51520) 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 -- 172.21.15.73:0/2180650306 learned_addr learned my addr 172.21.15.73:0/2180650306 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ce00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 0x7f45781567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577400640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45781587d0 0x7f4578156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 -- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45781587d0 msgr2=0x7f4578156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45781587d0 0x7f4578156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 -- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 msgr2=0x7f45781567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 0x7f45781567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f4577e00640 1 -- 172.21.15.73:0/2180650306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640094d0 con 0x7f457806cb80 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.920+0000 7f457ce00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 0x7f45781567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:32.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f4577e00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f456800c430 tx=0x7f456800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568011790 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45680118f0 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568011ce0 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45780c3980 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.921+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45780c3f40 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.922+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4578066af0 con 0x7f457806cb80 2024-04-05T22:25:32.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.923+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f456801c460 con 0x7f457806cb80 2024-04-05T22:25:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.924+0000 7f456fe00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 0x7f455806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.924+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4568089620 con 0x7f457806cb80 2024-04-05T22:25:32.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.924+0000 7f4577400640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 0x7f455806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:32.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.925+0000 7f4577400640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 0x7f455806d2a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f456403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:32.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:32.929+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f45680544e0 con 0x7f457806cb80 2024-04-05T22:25:33.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.118+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f457806f800 con 0x7f455806ade0 2024-04-05T22:25:33.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.128+0000 7f456fe00640 1 -- 172.21.15.73:0/2180650306 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286681 (secure 0 0 0) 0x7f4564031a90 con 0x7f455806ade0 2024-04-05T22:25:33.131 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:33.133 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 msgr2=0x7f455806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 0x7f455806d2a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f456403c040 comp rx=0 tx=0).stop 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 msgr2=0x7f45780b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f456800c430 tx=0x7f456800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 shutdown_connections 2024-04-05T22:25:33.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f455806ade0 0x7f455806d2a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f457815e850 0x7f45781567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45781587d0 0x7f4578156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 --2- 172.21.15.73:0/2180650306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457806cb80 0x7f45780b28c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.134+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 >> 172.21.15.73:0/2180650306 conn(0x7f45780ce8e0 msgr2=0x7f45780708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.135+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 shutdown_connections 2024-04-05T22:25:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.135+0000 7f457ee00640 1 -- 172.21.15.73:0/2180650306 wait complete. 2024-04-05T22:25:33.154 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-04-05T22:25:32.577918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251892,"kb_used_data":7016,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754722828,"statfs":{"total":773094113280,"available":772836175872,"internally_reserved":0,"allocated":7184384,"data_stored":3658956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":250596923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.300431"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290720+0000","last_change":"2024-04-05T22:25:23.299156+0000","last_active":"2024-04-05T22:25:24.290720+0000","last_peered":"2024-04-05T22:25:24.290720+0000","last_clean":"2024-04-05T22:25:24.290720+0000","last_became_active":"2024-04-05T22:25:23.298427+0000","last_became_peered":"2024-04-05T22:25:23.298427+0000","last_unstale":"2024-04-05T22:25:24.290720+0000","last_undegraded":"2024-04-05T22:25:24.290720+0000","last_fullsized":"2024-04-05T22:25:24.290720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:45:44.913900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308802+0000","last_change":"2024-04-05T22:25:23.291712+0000","last_active":"2024-04-05T22:25:24.308802+0000","last_peered":"2024-04-05T22:25:24.308802+0000","last_clean":"2024-04-05T22:25:24.308802+0000","last_became_active":"2024-04-05T22:25:23.290786+0000","last_became_peered":"2024-04-05T22:25:23.290786+0000","last_unstale":"2024-04-05T22:25:24.308802+0000","last_undegraded":"2024-04-05T22:25:24.308802+0000","last_fullsized":"2024-04-05T22:25:24.308802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:02:03.184140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312076+0000","last_change":"2024-04-05T22:25:23.322464+0000","last_active":"2024-04-05T22:25:24.312076+0000","last_peered":"2024-04-05T22:25:24.312076+0000","last_clean":"2024-04-05T22:25:24.312076+0000","last_became_active":"2024-04-05T22:25:23.316842+0000","last_became_peered":"2024-04-05T22:25:23.316842+0000","last_unstale":"2024-04-05T22:25:24.312076+0000","last_undegraded":"2024-04-05T22:25:24.312076+0000","last_fullsized":"2024-04-05T22:25:24.312076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:12:17.928701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312038+0000","last_change":"2024-04-05T22:25:23.324761+0000","last_active":"2024-04-05T22:25:24.312038+0000","last_peered":"2024-04-05T22:25:24.312038+0000","last_clean":"2024-04-05T22:25:24.312038+0000","last_became_active":"2024-04-05T22:25:23.323869+0000","last_became_peered":"2024-04-05T22:25:23.323869+0000","last_unstale":"2024-04-05T22:25:24.312038+0000","last_undegraded":"2024-04-05T22:25:24.312038+0000","last_fullsized":"2024-04-05T22:25:24.312038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290172+0000","last_change":"2024-04-05T22:25:23.318738+0000","last_active":"2024-04-05T22:25:24.290172+0000","last_peered":"2024-04-05T22:25:24.290172+0000","last_clean":"2024-04-05T22:25:24.290172+0000","last_became_active":"2024-04-05T22:25:23.318429+0000","last_became_peered":"2024-04-05T22:25:23.318429+0000","last_unstale":"2024-04-05T22:25:24.290172+0000","last_undegraded":"2024-04-05T22:25:24.290172+0000","last_fullsized":"2024-04-05T22:25:24.290172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:40.840026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307914+0000","last_change":"2024-04-05T22:25:23.296999+0000","last_active":"2024-04-05T22:25:24.307914+0000","last_peered":"2024-04-05T22:25:24.307914+0000","last_clean":"2024-04-05T22:25:24.307914+0000","last_became_active":"2024-04-05T22:25:23.296625+0000","last_became_peered":"2024-04-05T22:25:23.296625+0000","last_unstale":"2024-04-05T22:25:24.307914+0000","last_undegraded":"2024-04-05T22:25:24.307914+0000","last_fullsized":"2024-04-05T22:25:24.307914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293145+0000","last_change":"2024-04-05T22:25:23.312117+0000","last_active":"2024-04-05T22:25:24.293145+0000","last_peered":"2024-04-05T22:25:24.293145+0000","last_clean":"2024-04-05T22:25:24.293145+0000","last_became_active":"2024-04-05T22:25:23.310600+0000","last_became_peered":"2024-04-05T22:25:23.310600+0000","last_unstale":"2024-04-05T22:25:24.293145+0000","last_undegraded":"2024-04-05T22:25:24.293145+0000","last_fullsized":"2024-04-05T22:25:24.293145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:43.287781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306787+0000","last_change":"2024-04-05T22:25:23.296496+0000","last_active":"2024-04-05T22:25:24.306787+0000","last_peered":"2024-04-05T22:25:24.306787+0000","last_clean":"2024-04-05T22:25:24.306787+0000","last_became_active":"2024-04-05T22:25:23.296222+0000","last_became_peered":"2024-04-05T22:25:23.296222+0000","last_unstale":"2024-04-05T22:25:24.306787+0000","last_undegraded":"2024-04-05T22:25:24.306787+0000","last_fullsized":"2024-04-05T22:25:24.306787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:36:43.010960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294145+0000","last_change":"2024-04-05T22:25:23.303841+0000","last_active":"2024-04-05T22:25:24.294145+0000","last_peered":"2024-04-05T22:25:24.294145+0000","last_clean":"2024-04-05T22:25:24.294145+0000","last_became_active":"2024-04-05T22:25:23.301443+0000","last_became_peered":"2024-04-05T22:25:23.301443+0000","last_unstale":"2024-04-05T22:25:24.294145+0000","last_undegraded":"2024-04-05T22:25:24.294145+0000","last_fullsized":"2024-04-05T22:25:24.294145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291335+0000","last_change":"2024-04-05T22:25:23.314986+0000","last_active":"2024-04-05T22:25:24.291335+0000","last_peered":"2024-04-05T22:25:24.291335+0000","last_clean":"2024-04-05T22:25:24.291335+0000","last_became_active":"2024-04-05T22:25:23.314711+0000","last_became_peered":"2024-04-05T22:25:23.314711+0000","last_unstale":"2024-04-05T22:25:24.291335+0000","last_undegraded":"2024-04-05T22:25:24.291335+0000","last_fullsized":"2024-04-05T22:25:24.291335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291109+0000","last_change":"2024-04-05T22:25:23.290822+0000","last_active":"2024-04-05T22:25:24.291109+0000","last_peered":"2024-04-05T22:25:24.291109+0000","last_clean":"2024-04-05T22:25:24.291109+0000","last_became_active":"2024-04-05T22:25:23.290573+0000","last_became_peered":"2024-04-05T22:25:23.290573+0000","last_unstale":"2024-04-05T22:25:24.291109+0000","last_undegraded":"2024-04-05T22:25:24.291109+0000","last_fullsized":"2024-04-05T22:25:24.291109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312290+0000","last_change":"2024-04-05T22:25:23.314201+0000","last_active":"2024-04-05T22:25:24.312290+0000","last_peered":"2024-04-05T22:25:24.312290+0000","last_clean":"2024-04-05T22:25:24.312290+0000","last_became_active":"2024-04-05T22:25:23.307100+0000","last_became_peered":"2024-04-05T22:25:23.307100+0000","last_unstale":"2024-04-05T22:25:24.312290+0000","last_undegraded":"2024-04-05T22:25:24.312290+0000","last_fullsized":"2024-04-05T22:25:24.312290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309603+0000","last_change":"2024-04-05T22:25:23.315571+0000","last_active":"2024-04-05T22:25:24.309603+0000","last_peered":"2024-04-05T22:25:24.309603+0000","last_clean":"2024-04-05T22:25:24.309603+0000","last_became_active":"2024-04-05T22:25:23.315249+0000","last_became_peered":"2024-04-05T22:25:23.315249+0000","last_unstale":"2024-04-05T22:25:24.309603+0000","last_undegraded":"2024-04-05T22:25:24.309603+0000","last_fullsized":"2024-04-05T22:25:24.309603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:41.924668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293201+0000","last_change":"2024-04-05T22:25:23.303074+0000","last_active":"2024-04-05T22:25:24.293201+0000","last_peered":"2024-04-05T22:25:24.293201+0000","last_clean":"2024-04-05T22:25:24.293201+0000","last_became_active":"2024-04-05T22:25:23.302615+0000","last_became_peered":"2024-04-05T22:25:23.302615+0000","last_unstale":"2024-04-05T22:25:24.293201+0000","last_undegraded":"2024-04-05T22:25:24.293201+0000","last_fullsized":"2024-04-05T22:25:24.293201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:56.012910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293307+0000","last_change":"2024-04-05T22:25:23.297181+0000","last_active":"2024-04-05T22:25:24.293307+0000","last_peered":"2024-04-05T22:25:24.293307+0000","last_clean":"2024-04-05T22:25:24.293307+0000","last_became_active":"2024-04-05T22:25:23.293820+0000","last_became_peered":"2024-04-05T22:25:23.293820+0000","last_unstale":"2024-04-05T22:25:24.293307+0000","last_undegraded":"2024-04-05T22:25:24.293307+0000","last_fullsized":"2024-04-05T22:25:24.293307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307255+0000","last_change":"2024-04-05T22:25:23.313239+0000","last_active":"2024-04-05T22:25:24.307255+0000","last_peered":"2024-04-05T22:25:24.307255+0000","last_clean":"2024-04-05T22:25:24.307255+0000","last_became_active":"2024-04-05T22:25:23.312991+0000","last_became_peered":"2024-04-05T22:25:23.312991+0000","last_unstale":"2024-04-05T22:25:24.307255+0000","last_undegraded":"2024-04-05T22:25:24.307255+0000","last_fullsized":"2024-04-05T22:25:24.307255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:55.808811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308785+0000","last_change":"2024-04-05T22:25:23.317075+0000","last_active":"2024-04-05T22:25:24.308785+0000","last_peered":"2024-04-05T22:25:24.308785+0000","last_clean":"2024-04-05T22:25:24.308785+0000","last_became_active":"2024-04-05T22:25:23.314962+0000","last_became_peered":"2024-04-05T22:25:23.314962+0000","last_unstale":"2024-04-05T22:25:24.308785+0000","last_undegraded":"2024-04-05T22:25:24.308785+0000","last_fullsized":"2024-04-05T22:25:24.308785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:48.490866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311239+0000","last_change":"2024-04-05T22:25:23.318987+0000","last_active":"2024-04-05T22:25:24.311239+0000","last_peered":"2024-04-05T22:25:24.311239+0000","last_clean":"2024-04-05T22:25:24.311239+0000","last_became_active":"2024-04-05T22:25:23.317905+0000","last_became_peered":"2024-04-05T22:25:23.317905+0000","last_unstale":"2024-04-05T22:25:24.311239+0000","last_undegraded":"2024-04-05T22:25:24.311239+0000","last_fullsized":"2024-04-05T22:25:24.311239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:03:19.914501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290417+0000","last_change":"2024-04-05T22:25:23.297860+0000","last_active":"2024-04-05T22:25:24.290417+0000","last_peered":"2024-04-05T22:25:24.290417+0000","last_clean":"2024-04-05T22:25:24.290417+0000","last_became_active":"2024-04-05T22:25:23.297615+0000","last_became_peered":"2024-04-05T22:25:23.297615+0000","last_unstale":"2024-04-05T22:25:24.290417+0000","last_undegraded":"2024-04-05T22:25:24.290417+0000","last_fullsized":"2024-04-05T22:25:24.290417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:19:55.287884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308585+0000","last_change":"2024-04-05T22:25:23.294183+0000","last_active":"2024-04-05T22:25:24.308585+0000","last_peered":"2024-04-05T22:25:24.308585+0000","last_clean":"2024-04-05T22:25:24.308585+0000","last_became_active":"2024-04-05T22:25:23.293894+0000","last_became_peered":"2024-04-05T22:25:23.293894+0000","last_unstale":"2024-04-05T22:25:24.308585+0000","last_undegraded":"2024-04-05T22:25:24.308585+0000","last_fullsized":"2024-04-05T22:25:24.308585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:06:40.304684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311574+0000","last_change":"2024-04-05T22:25:23.323578+0000","last_active":"2024-04-05T22:25:24.311574+0000","last_peered":"2024-04-05T22:25:24.311574+0000","last_clean":"2024-04-05T22:25:24.311574+0000","last_became_active":"2024-04-05T22:25:23.323250+0000","last_became_peered":"2024-04-05T22:25:23.323250+0000","last_unstale":"2024-04-05T22:25:24.311574+0000","last_undegraded":"2024-04-05T22:25:24.311574+0000","last_fullsized":"2024-04-05T22:25:24.311574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:27.700487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308475+0000","last_change":"2024-04-05T22:25:23.297418+0000","last_active":"2024-04-05T22:25:24.308475+0000","last_peered":"2024-04-05T22:25:24.308475+0000","last_clean":"2024-04-05T22:25:24.308475+0000","last_became_active":"2024-04-05T22:25:23.297075+0000","last_became_peered":"2024-04-05T22:25:23.297075+0000","last_unstale":"2024-04-05T22:25:24.308475+0000","last_undegraded":"2024-04-05T22:25:24.308475+0000","last_fullsized":"2024-04-05T22:25:24.308475+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:45.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310394+0000","last_change":"2024-04-05T22:25:23.320793+0000","last_active":"2024-04-05T22:25:24.310394+0000","last_peered":"2024-04-05T22:25:24.310394+0000","last_clean":"2024-04-05T22:25:24.310394+0000","last_became_active":"2024-04-05T22:25:23.316015+0000","last_became_peered":"2024-04-05T22:25:23.316015+0000","last_unstale":"2024-04-05T22:25:24.310394+0000","last_undegraded":"2024-04-05T22:25:24.310394+0000","last_fullsized":"2024-04-05T22:25:24.310394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309223+0000","last_change":"2024-04-05T22:25:23.315728+0000","last_active":"2024-04-05T22:25:24.309223+0000","last_peered":"2024-04-05T22:25:24.309223+0000","last_clean":"2024-04-05T22:25:24.309223+0000","last_became_active":"2024-04-05T22:25:23.314345+0000","last_became_peered":"2024-04-05T22:25:23.314345+0000","last_unstale":"2024-04-05T22:25:24.309223+0000","last_undegraded":"2024-04-05T22:25:24.309223+0000","last_fullsized":"2024-04-05T22:25:24.309223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:50.046379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290866+0000","last_change":"2024-04-05T22:25:23.302044+0000","last_active":"2024-04-05T22:25:24.290866+0000","last_peered":"2024-04-05T22:25:24.290866+0000","last_clean":"2024-04-05T22:25:24.290866+0000","last_became_active":"2024-04-05T22:25:23.301336+0000","last_became_peered":"2024-04-05T22:25:23.301336+0000","last_unstale":"2024-04-05T22:25:24.290866+0000","last_undegraded":"2024-04-05T22:25:24.290866+0000","last_fullsized":"2024-04-05T22:25:24.290866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:42.782908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311640+0000","last_change":"2024-04-05T22:25:23.321078+0000","last_active":"2024-04-05T22:25:24.311640+0000","last_peered":"2024-04-05T22:25:24.311640+0000","last_clean":"2024-04-05T22:25:24.311640+0000","last_became_active":"2024-04-05T22:25:23.315835+0000","last_became_peered":"2024-04-05T22:25:23.315835+0000","last_unstale":"2024-04-05T22:25:24.311640+0000","last_undegraded":"2024-04-05T22:25:24.311640+0000","last_fullsized":"2024-04-05T22:25:24.311640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:17.621275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308033+0000","last_change":"2024-04-05T22:25:23.316796+0000","last_active":"2024-04-05T22:25:24.308033+0000","last_peered":"2024-04-05T22:25:24.308033+0000","last_clean":"2024-04-05T22:25:24.308033+0000","last_became_active":"2024-04-05T22:25:23.316540+0000","last_became_peered":"2024-04-05T22:25:23.316540+0000","last_unstale":"2024-04-05T22:25:24.308033+0000","last_undegraded":"2024-04-05T22:25:24.308033+0000","last_fullsized":"2024-04-05T22:25:24.308033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290359+0000","last_change":"2024-04-05T22:25:23.318867+0000","last_active":"2024-04-05T22:25:24.290359+0000","last_peered":"2024-04-05T22:25:24.290359+0000","last_clean":"2024-04-05T22:25:24.290359+0000","last_became_active":"2024-04-05T22:25:23.318583+0000","last_became_peered":"2024-04-05T22:25:23.318583+0000","last_unstale":"2024-04-05T22:25:24.290359+0000","last_undegraded":"2024-04-05T22:25:24.290359+0000","last_fullsized":"2024-04-05T22:25:24.290359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:34:20.782795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308872+0000","last_change":"2024-04-05T22:25:23.309759+0000","last_active":"2024-04-05T22:25:24.308872+0000","last_peered":"2024-04-05T22:25:24.308872+0000","last_clean":"2024-04-05T22:25:24.308872+0000","last_became_active":"2024-04-05T22:25:23.309475+0000","last_became_peered":"2024-04-05T22:25:23.309475+0000","last_unstale":"2024-04-05T22:25:24.308872+0000","last_undegraded":"2024-04-05T22:25:24.308872+0000","last_fullsized":"2024-04-05T22:25:24.308872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:40.089255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310131+0000","last_change":"2024-04-05T22:25:23.318074+0000","last_active":"2024-04-05T22:25:24.310131+0000","last_peered":"2024-04-05T22:25:24.310131+0000","last_clean":"2024-04-05T22:25:24.310131+0000","last_became_active":"2024-04-05T22:25:23.317792+0000","last_became_peered":"2024-04-05T22:25:23.317792+0000","last_unstale":"2024-04-05T22:25:24.310131+0000","last_undegraded":"2024-04-05T22:25:24.310131+0000","last_fullsized":"2024-04-05T22:25:24.310131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:48.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290117+0000","last_change":"2024-04-05T22:25:23.301201+0000","last_active":"2024-04-05T22:25:24.290117+0000","last_peered":"2024-04-05T22:25:24.290117+0000","last_clean":"2024-04-05T22:25:24.290117+0000","last_became_active":"2024-04-05T22:25:23.300963+0000","last_became_peered":"2024-04-05T22:25:23.300963+0000","last_unstale":"2024-04-05T22:25:24.290117+0000","last_undegraded":"2024-04-05T22:25:24.290117+0000","last_fullsized":"2024-04-05T22:25:24.290117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:43:29.929763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307353+0000","last_change":"2024-04-05T22:25:23.298905+0000","last_active":"2024-04-05T22:25:24.307353+0000","last_peered":"2024-04-05T22:25:24.307353+0000","last_clean":"2024-04-05T22:25:24.307353+0000","last_became_active":"2024-04-05T22:25:23.297849+0000","last_became_peered":"2024-04-05T22:25:23.297849+0000","last_unstale":"2024-04-05T22:25:24.307353+0000","last_undegraded":"2024-04-05T22:25:24.307353+0000","last_fullsized":"2024-04-05T22:25:24.307353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:12.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307777+0000","last_change":"2024-04-05T22:25:23.316032+0000","last_active":"2024-04-05T22:25:24.307777+0000","last_peered":"2024-04-05T22:25:24.307777+0000","last_clean":"2024-04-05T22:25:24.307777+0000","last_became_active":"2024-04-05T22:25:23.315784+0000","last_became_peered":"2024-04-05T22:25:23.315784+0000","last_unstale":"2024-04-05T22:25:24.307777+0000","last_undegraded":"2024-04-05T22:25:24.307777+0000","last_fullsized":"2024-04-05T22:25:24.307777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:36:12.120029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.321362+0000","last_change":"2024-04-05T22:25:22.820217+0000","last_active":"2024-04-05T22:25:23.321362+0000","last_peered":"2024-04-05T22:25:23.321362+0000","last_clean":"2024-04-05T22:25:23.321362+0000","last_became_active":"2024-04-05T22:25:22.819880+0000","last_became_peered":"2024-04-05T22:25:22.819880+0000","last_unstale":"2024-04-05T22:25:23.321362+0000","last_undegraded":"2024-04-05T22:25:23.321362+0000","last_fullsized":"2024-04-05T22:25:23.321362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:39:54.998254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291479+0000","last_change":"2024-04-05T22:25:23.308426+0000","last_active":"2024-04-05T22:25:24.291479+0000","last_peered":"2024-04-05T22:25:24.291479+0000","last_clean":"2024-04-05T22:25:24.291479+0000","last_became_active":"2024-04-05T22:25:23.308165+0000","last_became_peered":"2024-04-05T22:25:23.308165+0000","last_unstale":"2024-04-05T22:25:24.291479+0000","last_undegraded":"2024-04-05T22:25:24.291479+0000","last_fullsized":"2024-04-05T22:25:24.291479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314544+0000","last_change":"2024-04-05T22:25:22.818905+0000","last_active":"2024-04-05T22:25:23.314544+0000","last_peered":"2024-04-05T22:25:23.314544+0000","last_clean":"2024-04-05T22:25:23.314544+0000","last_became_active":"2024-04-05T22:25:22.818565+0000","last_became_peered":"2024-04-05T22:25:22.818565+0000","last_unstale":"2024-04-05T22:25:23.314544+0000","last_undegraded":"2024-04-05T22:25:23.314544+0000","last_fullsized":"2024-04-05T22:25:23.314544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:55:54.794155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308212+0000","last_change":"2024-04-05T22:25:23.299031+0000","last_active":"2024-04-05T22:25:24.308212+0000","last_peered":"2024-04-05T22:25:24.308212+0000","last_clean":"2024-04-05T22:25:24.308212+0000","last_became_active":"2024-04-05T22:25:23.298753+0000","last_became_peered":"2024-04-05T22:25:23.298753+0000","last_unstale":"2024-04-05T22:25:24.308212+0000","last_undegraded":"2024-04-05T22:25:24.308212+0000","last_fullsized":"2024-04-05T22:25:24.308212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306956+0000","last_change":"2024-04-05T22:25:22.807855+0000","last_active":"2024-04-05T22:25:23.306956+0000","last_peered":"2024-04-05T22:25:23.306956+0000","last_clean":"2024-04-05T22:25:23.306956+0000","last_became_active":"2024-04-05T22:25:22.807584+0000","last_became_peered":"2024-04-05T22:25:22.807584+0000","last_unstale":"2024-04-05T22:25:23.306956+0000","last_undegraded":"2024-04-05T22:25:23.306956+0000","last_fullsized":"2024-04-05T22:25:23.306956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307780+0000","last_change":"2024-04-05T22:25:23.321058+0000","last_active":"2024-04-05T22:25:24.307780+0000","last_peered":"2024-04-05T22:25:24.307780+0000","last_clean":"2024-04-05T22:25:24.307780+0000","last_became_active":"2024-04-05T22:25:23.320468+0000","last_became_peered":"2024-04-05T22:25:23.320468+0000","last_unstale":"2024-04-05T22:25:24.307780+0000","last_undegraded":"2024-04-05T22:25:24.307780+0000","last_fullsized":"2024-04-05T22:25:24.307780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:42:19.724742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.326384+0000","last_change":"2024-04-05T22:25:22.822056+0000","last_active":"2024-04-05T22:25:23.326384+0000","last_peered":"2024-04-05T22:25:23.326384+0000","last_clean":"2024-04-05T22:25:23.326384+0000","last_became_active":"2024-04-05T22:25:22.821071+0000","last_became_peered":"2024-04-05T22:25:22.821071+0000","last_unstale":"2024-04-05T22:25:23.326384+0000","last_undegraded":"2024-04-05T22:25:23.326384+0000","last_fullsized":"2024-04-05T22:25:23.326384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:36.908908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308559+0000","last_change":"2024-04-05T22:25:23.294038+0000","last_active":"2024-04-05T22:25:24.308559+0000","last_peered":"2024-04-05T22:25:24.308559+0000","last_clean":"2024-04-05T22:25:24.308559+0000","last_became_active":"2024-04-05T22:25:23.293856+0000","last_became_peered":"2024-04-05T22:25:23.293856+0000","last_unstale":"2024-04-05T22:25:24.308559+0000","last_undegraded":"2024-04-05T22:25:24.308559+0000","last_fullsized":"2024-04-05T22:25:24.308559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:23:59.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311539+0000","last_change":"2024-04-05T22:25:22.813362+0000","last_active":"2024-04-05T22:25:23.311539+0000","last_peered":"2024-04-05T22:25:23.311539+0000","last_clean":"2024-04-05T22:25:23.311539+0000","last_became_active":"2024-04-05T22:25:22.812956+0000","last_became_peered":"2024-04-05T22:25:22.812956+0000","last_unstale":"2024-04-05T22:25:23.311539+0000","last_undegraded":"2024-04-05T22:25:23.311539+0000","last_fullsized":"2024-04-05T22:25:23.311539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308570+0000","last_change":"2024-04-05T22:25:23.297447+0000","last_active":"2024-04-05T22:25:24.308570+0000","last_peered":"2024-04-05T22:25:24.308570+0000","last_clean":"2024-04-05T22:25:24.308570+0000","last_became_active":"2024-04-05T22:25:23.294666+0000","last_became_peered":"2024-04-05T22:25:23.294666+0000","last_unstale":"2024-04-05T22:25:24.308570+0000","last_undegraded":"2024-04-05T22:25:24.308570+0000","last_fullsized":"2024-04-05T22:25:24.308570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:33.835122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.306675+0000","last_change":"2024-04-05T22:25:22.812184+0000","last_active":"2024-04-05T22:25:23.306675+0000","last_peered":"2024-04-05T22:25:23.306675+0000","last_clean":"2024-04-05T22:25:23.306675+0000","last_became_active":"2024-04-05T22:25:22.811919+0000","last_became_peered":"2024-04-05T22:25:22.811919+0000","last_unstale":"2024-04-05T22:25:23.306675+0000","last_undegraded":"2024-04-05T22:25:23.306675+0000","last_fullsized":"2024-04-05T22:25:23.306675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.309323+0000","last_change":"2024-04-05T22:25:23.312968+0000","last_active":"2024-04-05T22:25:24.309323+0000","last_peered":"2024-04-05T22:25:24.309323+0000","last_clean":"2024-04-05T22:25:24.309323+0000","last_became_active":"2024-04-05T22:25:23.312697+0000","last_became_peered":"2024-04-05T22:25:23.312697+0000","last_unstale":"2024-04-05T22:25:24.309323+0000","last_undegraded":"2024-04-05T22:25:24.309323+0000","last_fullsized":"2024-04-05T22:25:24.309323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:43:47.570184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305456+0000","last_change":"2024-04-05T22:25:22.824792+0000","last_active":"2024-04-05T22:25:23.305456+0000","last_peered":"2024-04-05T22:25:23.305456+0000","last_clean":"2024-04-05T22:25:23.305456+0000","last_became_active":"2024-04-05T22:25:22.822520+0000","last_became_peered":"2024-04-05T22:25:22.822520+0000","last_unstale":"2024-04-05T22:25:23.305456+0000","last_undegraded":"2024-04-05T22:25:23.305456+0000","last_fullsized":"2024-04-05T22:25:23.305456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:47.138497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306877+0000","last_change":"2024-04-05T22:25:23.315314+0000","last_active":"2024-04-05T22:25:24.306877+0000","last_peered":"2024-04-05T22:25:24.306877+0000","last_clean":"2024-04-05T22:25:24.306877+0000","last_became_active":"2024-04-05T22:25:23.314998+0000","last_became_peered":"2024-04-05T22:25:23.314998+0000","last_unstale":"2024-04-05T22:25:24.306877+0000","last_undegraded":"2024-04-05T22:25:24.306877+0000","last_fullsized":"2024-04-05T22:25:24.306877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:56.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283790+0000","last_change":"2024-04-05T22:25:22.813243+0000","last_active":"2024-04-05T22:25:23.283790+0000","last_peered":"2024-04-05T22:25:23.283790+0000","last_clean":"2024-04-05T22:25:23.283790+0000","last_became_active":"2024-04-05T22:25:22.810600+0000","last_became_peered":"2024-04-05T22:25:22.810600+0000","last_unstale":"2024-04-05T22:25:23.283790+0000","last_undegraded":"2024-04-05T22:25:23.283790+0000","last_fullsized":"2024-04-05T22:25:23.283790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.294078+0000","last_change":"2024-04-05T22:25:23.298671+0000","last_active":"2024-04-05T22:25:24.294078+0000","last_peered":"2024-04-05T22:25:24.294078+0000","last_clean":"2024-04-05T22:25:24.294078+0000","last_became_active":"2024-04-05T22:25:23.294595+0000","last_became_peered":"2024-04-05T22:25:23.294595+0000","last_unstale":"2024-04-05T22:25:24.294078+0000","last_undegraded":"2024-04-05T22:25:24.294078+0000","last_fullsized":"2024-04-05T22:25:24.294078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304213+0000","last_change":"2024-04-05T22:25:22.823584+0000","last_active":"2024-04-05T22:25:23.304213+0000","last_peered":"2024-04-05T22:25:23.304213+0000","last_clean":"2024-04-05T22:25:23.304213+0000","last_became_active":"2024-04-05T22:25:22.823316+0000","last_became_peered":"2024-04-05T22:25:22.823316+0000","last_unstale":"2024-04-05T22:25:23.304213+0000","last_undegraded":"2024-04-05T22:25:23.304213+0000","last_fullsized":"2024-04-05T22:25:23.304213+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312394+0000","last_change":"2024-04-05T22:25:23.314057+0000","last_active":"2024-04-05T22:25:24.312394+0000","last_peered":"2024-04-05T22:25:24.312394+0000","last_clean":"2024-04-05T22:25:24.312394+0000","last_became_active":"2024-04-05T22:25:23.312904+0000","last_became_peered":"2024-04-05T22:25:23.312904+0000","last_unstale":"2024-04-05T22:25:24.312394+0000","last_undegraded":"2024-04-05T22:25:24.312394+0000","last_fullsized":"2024-04-05T22:25:24.312394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:25.196366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283719+0000","last_change":"2024-04-05T22:25:22.812954+0000","last_active":"2024-04-05T22:25:23.283719+0000","last_peered":"2024-04-05T22:25:23.283719+0000","last_clean":"2024-04-05T22:25:23.283719+0000","last_became_active":"2024-04-05T22:25:22.812685+0000","last_became_peered":"2024-04-05T22:25:22.812685+0000","last_unstale":"2024-04-05T22:25:23.283719+0000","last_undegraded":"2024-04-05T22:25:23.283719+0000","last_fullsized":"2024-04-05T22:25:23.283719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:36.447234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291127+0000","last_change":"2024-04-05T22:25:23.307204+0000","last_active":"2024-04-05T22:25:24.291127+0000","last_peered":"2024-04-05T22:25:24.291127+0000","last_clean":"2024-04-05T22:25:24.291127+0000","last_became_active":"2024-04-05T22:25:23.306766+0000","last_became_peered":"2024-04-05T22:25:23.306766+0000","last_unstale":"2024-04-05T22:25:24.291127+0000","last_undegraded":"2024-04-05T22:25:24.291127+0000","last_fullsized":"2024-04-05T22:25:24.291127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:27.638305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.311719+0000","last_change":"2024-04-05T22:25:22.820936+0000","last_active":"2024-04-05T22:25:23.311719+0000","last_peered":"2024-04-05T22:25:23.311719+0000","last_clean":"2024-04-05T22:25:23.311719+0000","last_became_active":"2024-04-05T22:25:22.820271+0000","last_became_peered":"2024-04-05T22:25:22.820271+0000","last_unstale":"2024-04-05T22:25:23.311719+0000","last_undegraded":"2024-04-05T22:25:23.311719+0000","last_fullsized":"2024-04-05T22:25:23.311719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312194+0000","last_change":"2024-04-05T22:25:23.309672+0000","last_active":"2024-04-05T22:25:24.312194+0000","last_peered":"2024-04-05T22:25:24.312194+0000","last_clean":"2024-04-05T22:25:24.312194+0000","last_became_active":"2024-04-05T22:25:23.308313+0000","last_became_peered":"2024-04-05T22:25:23.308313+0000","last_unstale":"2024-04-05T22:25:24.312194+0000","last_undegraded":"2024-04-05T22:25:24.312194+0000","last_fullsized":"2024-04-05T22:25:24.312194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.322355+0000","last_change":"2024-04-05T22:25:22.812066+0000","last_active":"2024-04-05T22:25:23.322355+0000","last_peered":"2024-04-05T22:25:23.322355+0000","last_clean":"2024-04-05T22:25:23.322355+0000","last_became_active":"2024-04-05T22:25:22.811662+0000","last_became_peered":"2024-04-05T22:25:22.811662+0000","last_unstale":"2024-04-05T22:25:23.322355+0000","last_undegraded":"2024-04-05T22:25:23.322355+0000","last_fullsized":"2024-04-05T22:25:23.322355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:04.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311269+0000","last_change":"2024-04-05T22:25:23.321510+0000","last_active":"2024-04-05T22:25:24.311269+0000","last_peered":"2024-04-05T22:25:24.311269+0000","last_clean":"2024-04-05T22:25:24.311269+0000","last_became_active":"2024-04-05T22:25:23.319576+0000","last_became_peered":"2024-04-05T22:25:23.319576+0000","last_unstale":"2024-04-05T22:25:24.311269+0000","last_undegraded":"2024-04-05T22:25:24.311269+0000","last_fullsized":"2024-04-05T22:25:24.311269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:50:33.266168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302829+0000","last_change":"2024-04-05T22:25:22.815327+0000","last_active":"2024-04-05T22:25:23.302829+0000","last_peered":"2024-04-05T22:25:23.302829+0000","last_clean":"2024-04-05T22:25:23.302829+0000","last_became_active":"2024-04-05T22:25:22.814733+0000","last_became_peered":"2024-04-05T22:25:22.814733+0000","last_unstale":"2024-04-05T22:25:23.302829+0000","last_undegraded":"2024-04-05T22:25:23.302829+0000","last_fullsized":"2024-04-05T22:25:23.302829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:36.497236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311892+0000","last_change":"2024-04-05T22:25:23.316894+0000","last_active":"2024-04-05T22:25:24.311892+0000","last_peered":"2024-04-05T22:25:24.311892+0000","last_clean":"2024-04-05T22:25:24.311892+0000","last_became_active":"2024-04-05T22:25:23.316563+0000","last_became_peered":"2024-04-05T22:25:23.316563+0000","last_unstale":"2024-04-05T22:25:24.311892+0000","last_undegraded":"2024-04-05T22:25:24.311892+0000","last_fullsized":"2024-04-05T22:25:24.311892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:28.048899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309891+0000","last_change":"2024-04-05T22:25:22.817554+0000","last_active":"2024-04-05T22:25:23.309891+0000","last_peered":"2024-04-05T22:25:23.309891+0000","last_clean":"2024-04-05T22:25:23.309891+0000","last_became_active":"2024-04-05T22:25:22.816462+0000","last_became_peered":"2024-04-05T22:25:22.816462+0000","last_unstale":"2024-04-05T22:25:23.309891+0000","last_undegraded":"2024-04-05T22:25:23.309891+0000","last_fullsized":"2024-04-05T22:25:23.309891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:02.494750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.293141+0000","last_change":"2024-04-05T22:25:23.309583+0000","last_active":"2024-04-05T22:25:24.293141+0000","last_peered":"2024-04-05T22:25:24.293141+0000","last_clean":"2024-04-05T22:25:24.293141+0000","last_became_active":"2024-04-05T22:25:23.307782+0000","last_became_peered":"2024-04-05T22:25:23.307782+0000","last_unstale":"2024-04-05T22:25:24.293141+0000","last_undegraded":"2024-04-05T22:25:24.293141+0000","last_fullsized":"2024-04-05T22:25:24.293141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:52.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305585+0000","last_change":"2024-04-05T22:25:22.821958+0000","last_active":"2024-04-05T22:25:23.305585+0000","last_peered":"2024-04-05T22:25:23.305585+0000","last_clean":"2024-04-05T22:25:23.305585+0000","last_became_active":"2024-04-05T22:25:22.821425+0000","last_became_peered":"2024-04-05T22:25:22.821425+0000","last_unstale":"2024-04-05T22:25:23.305585+0000","last_undegraded":"2024-04-05T22:25:23.305585+0000","last_fullsized":"2024-04-05T22:25:23.305585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.310387+0000","last_change":"2024-04-05T22:25:23.310520+0000","last_active":"2024-04-05T22:25:24.310387+0000","last_peered":"2024-04-05T22:25:24.310387+0000","last_clean":"2024-04-05T22:25:24.310387+0000","last_became_active":"2024-04-05T22:25:23.310234+0000","last_became_peered":"2024-04-05T22:25:23.310234+0000","last_unstale":"2024-04-05T22:25:24.310387+0000","last_undegraded":"2024-04-05T22:25:24.310387+0000","last_fullsized":"2024-04-05T22:25:24.310387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:37.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.325150+0000","last_change":"2024-04-05T22:25:22.822654+0000","last_active":"2024-04-05T22:25:23.325150+0000","last_peered":"2024-04-05T22:25:23.325150+0000","last_clean":"2024-04-05T22:25:23.325150+0000","last_became_active":"2024-04-05T22:25:22.822260+0000","last_became_peered":"2024-04-05T22:25:22.822260+0000","last_unstale":"2024-04-05T22:25:23.325150+0000","last_undegraded":"2024-04-05T22:25:23.325150+0000","last_fullsized":"2024-04-05T22:25:23.325150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:03:59.941475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312013+0000","last_change":"2024-04-05T22:25:23.318981+0000","last_active":"2024-04-05T22:25:24.312013+0000","last_peered":"2024-04-05T22:25:24.312013+0000","last_clean":"2024-04-05T22:25:24.312013+0000","last_became_active":"2024-04-05T22:25:23.318382+0000","last_became_peered":"2024-04-05T22:25:23.318382+0000","last_unstale":"2024-04-05T22:25:24.312013+0000","last_undegraded":"2024-04-05T22:25:24.312013+0000","last_fullsized":"2024-04-05T22:25:24.312013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:42:57.769217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.302992+0000","last_change":"2024-04-05T22:25:22.817215+0000","last_active":"2024-04-05T22:25:23.302992+0000","last_peered":"2024-04-05T22:25:23.302992+0000","last_clean":"2024-04-05T22:25:23.302992+0000","last_became_active":"2024-04-05T22:25:22.816814+0000","last_became_peered":"2024-04-05T22:25:22.816814+0000","last_unstale":"2024-04-05T22:25:23.302992+0000","last_undegraded":"2024-04-05T22:25:23.302992+0000","last_fullsized":"2024-04-05T22:25:23.302992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:26.685831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311184+0000","last_change":"2024-04-05T22:25:23.321785+0000","last_active":"2024-04-05T22:25:24.311184+0000","last_peered":"2024-04-05T22:25:24.311184+0000","last_clean":"2024-04-05T22:25:24.311184+0000","last_became_active":"2024-04-05T22:25:23.318293+0000","last_became_peered":"2024-04-05T22:25:23.318293+0000","last_unstale":"2024-04-05T22:25:24.311184+0000","last_undegraded":"2024-04-05T22:25:24.311184+0000","last_fullsized":"2024-04-05T22:25:24.311184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.303053+0000","last_change":"2024-04-05T22:25:22.810298+0000","last_active":"2024-04-05T22:25:23.303053+0000","last_peered":"2024-04-05T22:25:23.303053+0000","last_clean":"2024-04-05T22:25:23.303053+0000","last_became_active":"2024-04-05T22:25:22.809980+0000","last_became_peered":"2024-04-05T22:25:22.809980+0000","last_unstale":"2024-04-05T22:25:23.303053+0000","last_undegraded":"2024-04-05T22:25:23.303053+0000","last_fullsized":"2024-04-05T22:25:23.303053+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308777+0000","last_change":"2024-04-05T22:25:23.310722+0000","last_active":"2024-04-05T22:25:24.308777+0000","last_peered":"2024-04-05T22:25:24.308777+0000","last_clean":"2024-04-05T22:25:24.308777+0000","last_became_active":"2024-04-05T22:25:23.310419+0000","last_became_peered":"2024-04-05T22:25:23.310419+0000","last_unstale":"2024-04-05T22:25:24.308777+0000","last_undegraded":"2024-04-05T22:25:24.308777+0000","last_fullsized":"2024-04-05T22:25:24.308777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:29:46.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.320593+0000","last_change":"2024-04-05T22:25:22.822964+0000","last_active":"2024-04-05T22:25:23.320593+0000","last_peered":"2024-04-05T22:25:23.320593+0000","last_clean":"2024-04-05T22:25:23.320593+0000","last_became_active":"2024-04-05T22:25:22.822645+0000","last_became_peered":"2024-04-05T22:25:22.822645+0000","last_unstale":"2024-04-05T22:25:23.320593+0000","last_undegraded":"2024-04-05T22:25:23.320593+0000","last_fullsized":"2024-04-05T22:25:23.320593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:48:51.332791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312362+0000","last_change":"2024-04-05T22:25:23.313098+0000","last_active":"2024-04-05T22:25:24.312362+0000","last_peered":"2024-04-05T22:25:24.312362+0000","last_clean":"2024-04-05T22:25:24.312362+0000","last_became_active":"2024-04-05T22:25:23.312388+0000","last_became_peered":"2024-04-05T22:25:23.312388+0000","last_unstale":"2024-04-05T22:25:24.312362+0000","last_undegraded":"2024-04-05T22:25:24.312362+0000","last_fullsized":"2024-04-05T22:25:24.312362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.287654+0000","last_change":"2024-04-05T22:25:22.810231+0000","last_active":"2024-04-05T22:25:23.287654+0000","last_peered":"2024-04-05T22:25:23.287654+0000","last_clean":"2024-04-05T22:25:23.287654+0000","last_became_active":"2024-04-05T22:25:22.809955+0000","last_became_peered":"2024-04-05T22:25:22.809955+0000","last_unstale":"2024-04-05T22:25:23.287654+0000","last_undegraded":"2024-04-05T22:25:23.287654+0000","last_fullsized":"2024-04-05T22:25:23.287654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:26.623649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311586+0000","last_change":"2024-04-05T22:25:23.308720+0000","last_active":"2024-04-05T22:25:24.311586+0000","last_peered":"2024-04-05T22:25:24.311586+0000","last_clean":"2024-04-05T22:25:24.311586+0000","last_became_active":"2024-04-05T22:25:23.308461+0000","last_became_peered":"2024-04-05T22:25:23.308461+0000","last_unstale":"2024-04-05T22:25:24.311586+0000","last_undegraded":"2024-04-05T22:25:24.311586+0000","last_fullsized":"2024-04-05T22:25:24.311586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:22.968198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.314414+0000","last_change":"2024-04-05T22:25:22.808626+0000","last_active":"2024-04-05T22:25:23.314414+0000","last_peered":"2024-04-05T22:25:23.314414+0000","last_clean":"2024-04-05T22:25:23.314414+0000","last_became_active":"2024-04-05T22:25:22.808159+0000","last_became_peered":"2024-04-05T22:25:22.808159+0000","last_unstale":"2024-04-05T22:25:23.314414+0000","last_undegraded":"2024-04-05T22:25:23.314414+0000","last_fullsized":"2024-04-05T22:25:23.314414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:19.370558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307477+0000","last_change":"2024-04-05T22:25:23.310723+0000","last_active":"2024-04-05T22:25:24.307477+0000","last_peered":"2024-04-05T22:25:24.307477+0000","last_clean":"2024-04-05T22:25:24.307477+0000","last_became_active":"2024-04-05T22:25:23.310431+0000","last_became_peered":"2024-04-05T22:25:23.310431+0000","last_unstale":"2024-04-05T22:25:24.307477+0000","last_undegraded":"2024-04-05T22:25:24.307477+0000","last_fullsized":"2024-04-05T22:25:24.307477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:20.385215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279422+0000","last_change":"2024-04-05T22:25:22.809387+0000","last_active":"2024-04-05T22:25:23.279422+0000","last_peered":"2024-04-05T22:25:23.279422+0000","last_clean":"2024-04-05T22:25:23.279422+0000","last_became_active":"2024-04-05T22:25:22.809109+0000","last_became_peered":"2024-04-05T22:25:22.809109+0000","last_unstale":"2024-04-05T22:25:23.279422+0000","last_undegraded":"2024-04-05T22:25:23.279422+0000","last_fullsized":"2024-04-05T22:25:23.279422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307167+0000","last_change":"2024-04-05T22:25:23.316165+0000","last_active":"2024-04-05T22:25:24.307167+0000","last_peered":"2024-04-05T22:25:24.307167+0000","last_clean":"2024-04-05T22:25:24.307167+0000","last_became_active":"2024-04-05T22:25:23.315655+0000","last_became_peered":"2024-04-05T22:25:23.315655+0000","last_unstale":"2024-04-05T22:25:24.307167+0000","last_undegraded":"2024-04-05T22:25:24.307167+0000","last_fullsized":"2024-04-05T22:25:24.307167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:04:37.511892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.304412+0000","last_change":"2024-04-05T22:25:22.820823+0000","last_active":"2024-04-05T22:25:23.304412+0000","last_peered":"2024-04-05T22:25:23.304412+0000","last_clean":"2024-04-05T22:25:23.304412+0000","last_became_active":"2024-04-05T22:25:22.819543+0000","last_became_peered":"2024-04-05T22:25:22.819543+0000","last_unstale":"2024-04-05T22:25:23.304412+0000","last_undegraded":"2024-04-05T22:25:23.304412+0000","last_fullsized":"2024-04-05T22:25:23.304412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:36.021049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312774+0000","last_change":"2024-04-05T22:25:23.304349+0000","last_active":"2024-04-05T22:25:24.312774+0000","last_peered":"2024-04-05T22:25:24.312774+0000","last_clean":"2024-04-05T22:25:24.312774+0000","last_became_active":"2024-04-05T22:25:23.304014+0000","last_became_peered":"2024-04-05T22:25:23.304014+0000","last_unstale":"2024-04-05T22:25:24.312774+0000","last_undegraded":"2024-04-05T22:25:24.312774+0000","last_fullsized":"2024-04-05T22:25:24.312774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:37.461890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.305751+0000","last_change":"2024-04-05T22:25:22.813364+0000","last_active":"2024-04-05T22:25:23.305751+0000","last_peered":"2024-04-05T22:25:23.305751+0000","last_clean":"2024-04-05T22:25:23.305751+0000","last_became_active":"2024-04-05T22:25:22.813068+0000","last_became_peered":"2024-04-05T22:25:22.813068+0000","last_unstale":"2024-04-05T22:25:23.305751+0000","last_undegraded":"2024-04-05T22:25:23.305751+0000","last_fullsized":"2024-04-05T22:25:23.305751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:21.803962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.311439+0000","last_change":"2024-04-05T22:25:23.321716+0000","last_active":"2024-04-05T22:25:24.311439+0000","last_peered":"2024-04-05T22:25:24.311439+0000","last_clean":"2024-04-05T22:25:24.311439+0000","last_became_active":"2024-04-05T22:25:23.321428+0000","last_became_peered":"2024-04-05T22:25:23.321428+0000","last_unstale":"2024-04-05T22:25:24.311439+0000","last_undegraded":"2024-04-05T22:25:24.311439+0000","last_fullsized":"2024-04-05T22:25:24.311439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:57.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.288133+0000","last_change":"2024-04-05T22:25:22.813803+0000","last_active":"2024-04-05T22:25:23.288133+0000","last_peered":"2024-04-05T22:25:23.288133+0000","last_clean":"2024-04-05T22:25:23.288133+0000","last_became_active":"2024-04-05T22:25:22.813365+0000","last_became_peered":"2024-04-05T22:25:22.813365+0000","last_unstale":"2024-04-05T22:25:23.288133+0000","last_undegraded":"2024-04-05T22:25:23.288133+0000","last_fullsized":"2024-04-05T22:25:23.288133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.291236+0000","last_change":"2024-04-05T22:25:23.302602+0000","last_active":"2024-04-05T22:25:24.291236+0000","last_peered":"2024-04-05T22:25:24.291236+0000","last_clean":"2024-04-05T22:25:24.291236+0000","last_became_active":"2024-04-05T22:25:23.302356+0000","last_became_peered":"2024-04-05T22:25:23.302356+0000","last_unstale":"2024-04-05T22:25:24.291236+0000","last_undegraded":"2024-04-05T22:25:24.291236+0000","last_fullsized":"2024-04-05T22:25:24.291236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:27.824520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.312000+0000","last_change":"2024-04-05T22:25:22.812466+0000","last_active":"2024-04-05T22:25:23.312000+0000","last_peered":"2024-04-05T22:25:23.312000+0000","last_clean":"2024-04-05T22:25:23.312000+0000","last_became_active":"2024-04-05T22:25:22.812117+0000","last_became_peered":"2024-04-05T22:25:22.812117+0000","last_unstale":"2024-04-05T22:25:23.312000+0000","last_undegraded":"2024-04-05T22:25:23.312000+0000","last_fullsized":"2024-04-05T22:25:23.312000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:13.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.307935+0000","last_change":"2024-04-05T22:25:23.321029+0000","last_active":"2024-04-05T22:25:24.307935+0000","last_peered":"2024-04-05T22:25:24.307935+0000","last_clean":"2024-04-05T22:25:24.307935+0000","last_became_active":"2024-04-05T22:25:23.320742+0000","last_became_peered":"2024-04-05T22:25:23.320742+0000","last_unstale":"2024-04-05T22:25:24.307935+0000","last_undegraded":"2024-04-05T22:25:24.307935+0000","last_fullsized":"2024-04-05T22:25:24.307935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:26.184734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.316800+0000","last_change":"2024-04-05T22:25:22.819850+0000","last_active":"2024-04-05T22:25:23.316800+0000","last_peered":"2024-04-05T22:25:23.316800+0000","last_clean":"2024-04-05T22:25:23.316800+0000","last_became_active":"2024-04-05T22:25:22.819495+0000","last_became_peered":"2024-04-05T22:25:22.819495+0000","last_unstale":"2024-04-05T22:25:23.316800+0000","last_undegraded":"2024-04-05T22:25:23.316800+0000","last_fullsized":"2024-04-05T22:25:23.316800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:25:36.727027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308979+0000","last_change":"2024-04-05T22:25:23.290975+0000","last_active":"2024-04-05T22:25:24.308979+0000","last_peered":"2024-04-05T22:25:24.308979+0000","last_clean":"2024-04-05T22:25:24.308979+0000","last_became_active":"2024-04-05T22:25:23.290733+0000","last_became_peered":"2024-04-05T22:25:23.290733+0000","last_unstale":"2024-04-05T22:25:24.308979+0000","last_undegraded":"2024-04-05T22:25:24.308979+0000","last_fullsized":"2024-04-05T22:25:24.308979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:29.453299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279621+0000","last_change":"2024-04-05T22:25:22.821995+0000","last_active":"2024-04-05T22:25:23.279621+0000","last_peered":"2024-04-05T22:25:23.279621+0000","last_clean":"2024-04-05T22:25:23.279621+0000","last_became_active":"2024-04-05T22:25:22.821657+0000","last_became_peered":"2024-04-05T22:25:22.821657+0000","last_unstale":"2024-04-05T22:25:23.279621+0000","last_undegraded":"2024-04-05T22:25:23.279621+0000","last_fullsized":"2024-04-05T22:25:23.279621+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:37.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.312062+0000","last_change":"2024-04-05T22:25:23.307838+0000","last_active":"2024-04-05T22:25:24.312062+0000","last_peered":"2024-04-05T22:25:24.312062+0000","last_clean":"2024-04-05T22:25:24.312062+0000","last_became_active":"2024-04-05T22:25:23.307552+0000","last_became_peered":"2024-04-05T22:25:23.307552+0000","last_unstale":"2024-04-05T22:25:24.312062+0000","last_undegraded":"2024-04-05T22:25:24.312062+0000","last_fullsized":"2024-04-05T22:25:24.312062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:15.194717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.283982+0000","last_change":"2024-04-05T22:25:22.809727+0000","last_active":"2024-04-05T22:25:23.283982+0000","last_peered":"2024-04-05T22:25:23.283982+0000","last_clean":"2024-04-05T22:25:23.283982+0000","last_became_active":"2024-04-05T22:25:22.809281+0000","last_became_peered":"2024-04-05T22:25:22.809281+0000","last_unstale":"2024-04-05T22:25:23.283982+0000","last_undegraded":"2024-04-05T22:25:23.283982+0000","last_fullsized":"2024-04-05T22:25:23.283982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:54.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.290369+0000","last_change":"2024-04-05T22:25:23.312476+0000","last_active":"2024-04-05T22:25:24.290369+0000","last_peered":"2024-04-05T22:25:24.290369+0000","last_clean":"2024-04-05T22:25:24.290369+0000","last_became_active":"2024-04-05T22:25:23.312245+0000","last_became_peered":"2024-04-05T22:25:23.312245+0000","last_unstale":"2024-04-05T22:25:24.290369+0000","last_undegraded":"2024-04-05T22:25:24.290369+0000","last_fullsized":"2024-04-05T22:25:24.290369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:37.923564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.279493+0000","last_change":"2024-04-05T22:25:22.805657+0000","last_active":"2024-04-05T22:25:23.279493+0000","last_peered":"2024-04-05T22:25:23.279493+0000","last_clean":"2024-04-05T22:25:23.279493+0000","last_became_active":"2024-04-05T22:25:22.805398+0000","last_became_peered":"2024-04-05T22:25:22.805398+0000","last_unstale":"2024-04-05T22:25:23.279493+0000","last_undegraded":"2024-04-05T22:25:23.279493+0000","last_fullsized":"2024-04-05T22:25:23.279493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:01:11.674727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.308351+0000","last_change":"2024-04-05T22:25:23.315180+0000","last_active":"2024-04-05T22:25:24.308351+0000","last_peered":"2024-04-05T22:25:24.308351+0000","last_clean":"2024-04-05T22:25:24.308351+0000","last_became_active":"2024-04-05T22:25:23.314678+0000","last_became_peered":"2024-04-05T22:25:23.314678+0000","last_unstale":"2024-04-05T22:25:24.308351+0000","last_undegraded":"2024-04-05T22:25:24.308351+0000","last_fullsized":"2024-04-05T22:25:24.308351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:43.210562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.282244+0000","last_change":"2024-04-05T22:25:22.821960+0000","last_active":"2024-04-05T22:25:23.282244+0000","last_peered":"2024-04-05T22:25:23.282244+0000","last_clean":"2024-04-05T22:25:23.282244+0000","last_became_active":"2024-04-05T22:25:22.821679+0000","last_became_peered":"2024-04-05T22:25:22.821679+0000","last_unstale":"2024-04-05T22:25:23.282244+0000","last_undegraded":"2024-04-05T22:25:23.282244+0000","last_fullsized":"2024-04-05T22:25:23.282244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:11.859438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-05T22:25:24.306832+0000","last_change":"2024-04-05T22:25:23.292675+0000","last_active":"2024-04-05T22:25:24.306832+0000","last_peered":"2024-04-05T22:25:24.306832+0000","last_clean":"2024-04-05T22:25:24.306832+0000","last_became_active":"2024-04-05T22:25:23.292226+0000","last_became_peered":"2024-04-05T22:25:23.292226+0000","last_unstale":"2024-04-05T22:25:24.306832+0000","last_undegraded":"2024-04-05T22:25:24.306832+0000","last_fullsized":"2024-04-05T22:25:24.306832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:22.768595+0000","last_clean_scrub_stamp":"2024-04-05T22:25:22.768595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:45.144578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.309603+0000","last_change":"2024-04-05T22:25:22.823366+0000","last_active":"2024-04-05T22:25:23.309603+0000","last_peered":"2024-04-05T22:25:23.309603+0000","last_clean":"2024-04-05T22:25:23.309603+0000","last_became_active":"2024-04-05T22:25:22.823082+0000","last_became_peered":"2024-04-05T22:25:22.823082+0000","last_unstale":"2024-04-05T22:25:23.309603+0000","last_undegraded":"2024-04-05T22:25:23.309603+0000","last_fullsized":"2024-04-05T22:25:23.309603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:25:21.753939+0000","last_clean_scrub_stamp":"2024-04-05T22:25:21.753939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:59.775684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-05T22:25:23.296039+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:25:23.296039+0000","last_peered":"2024-04-05T22:25:23.296039+0000","last_clean":"2024-04-05T22:25:23.296039+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:25:23.296039+0000","last_undegraded":"2024-04-05T22:25:23.296039+0000","last_fullsized":"2024-04-05T22:25:23.296039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1343488,"data_stored":899715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":1.137},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.321},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.073}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.391}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.2370000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.544}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.623},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":741376,"data_stored":309187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":0.96899999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.508}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.248},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.681}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.796},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.365}]}]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":749568,"data_stored":309839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.042}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":1.256},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.331},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.52}]}]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":753664,"data_stored":310751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0509999999999999,"15min":0.999},"min":{"1min":0.84699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":1.673,"15min":1.673},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0980000000000001,"15min":1.0800000000000001},"min":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.343}]},{"osd":1,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1439999999999999,"15min":1.1299999999999999},"min":{"1min":0.875,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.236},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1539999999999999,"15min":1.1699999999999999},"min":{"1min":0.83399999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4450000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.3009999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.95099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.482,"5min":1.974,"15min":1.974},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.89000000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.379},"min":{"1min":0.95999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.843,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.369,"15min":1.407},"min":{"1min":1.107,"5min":0.874,"15min":0.874},"max":{"1min":1.754,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.575},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3440000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.625,"15min":0.625},"max":{"1min":1.786,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.49}]},{"osd":6,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4610000000000001},"min":{"1min":0.95599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6379999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.706},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4410000000000001,"15min":1.4950000000000001},"min":{"1min":1.0149999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.655,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8819999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5029999999999999,"15min":1.536},"min":{"1min":1.1639999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.895,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.825},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5549999999999999,"15min":1.5760000000000001},"min":{"1min":1.05,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":2.54,"15min":2.54},"last":1.9279999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":1.23},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":1.121},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95999999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.044},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.204}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.464}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.377},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.2909999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":309243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":1},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.876}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.21}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.621},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.518}]}]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":745472,"data_stored":309277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4970000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.127}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.282}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":1.25},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.665}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.573},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.536}]}]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1351680,"data_stored":901701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.875}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:25:33.158 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:25:33.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:25:33.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.419+0000 7fa4b9200640 1 Processor -- start 2024-04-05T22:25:33.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.420+0000 7fa4b9200640 1 -- start start 2024-04-05T22:25:33.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b4143660 0x7fa4b415a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 0x7fa4b4071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b40cb8c0 con 0x7fa4b4143660 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b406d000 con 0x7fa4b4070660 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.421+0000 7fa4b9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4070e70 con 0x7fa4b406c630 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b4143660 0x7fa4b415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b4143660 0x7fa4b415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51530/0 (socket says 172.21.15.73:51530) 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 -- 172.21.15.73:0/3112288958 learned_addr learned my addr 172.21.15.73:0/3112288958 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1400640 1 --2- 172.21.15.73:0/3112288958 >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 0x7fa4b4071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 -- 172.21.15.73:0/3112288958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 msgr2=0x7fa4b406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/3112288958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 -- 172.21.15.73:0/3112288958 >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 msgr2=0x7fa4b4071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/3112288958 >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 0x7fa4b4071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1e00640 1 -- 172.21.15.73:0/3112288958 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b4071b80 con 0x7fa4b4143660 2024-04-05T22:25:33.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.422+0000 7fa4b1400640 1 --2- 172.21.15.73:0/3112288958 >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 0x7fa4b4071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.423+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/3112288958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4143660 0x7fa4b415a200 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406fba0 tx=0x7fa4a402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b98ddf373c48351 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b0a00640 1 -- 172.21.15.73:0/3112288958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a402ede0 con 0x7fa4b4143660 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b0a00640 1 -- 172.21.15.73:0/3112288958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a4036440 con 0x7fa4b4143660 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b0a00640 1 -- 172.21.15.73:0/3112288958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40367d0 con 0x7fa4b4143660 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 -- 172.21.15.73:0/3112288958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4143660 msgr2=0x7fa4b415a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 --2- 172.21.15.73:0/3112288958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4143660 0x7fa4b415a200 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406fba0 tx=0x7fa4a402e030 comp rx=0 tx=0).stop 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 -- 172.21.15.73:0/3112288958 shutdown_connections 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 --2- 172.21.15.73:0/3112288958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 --2- 172.21.15.73:0/3112288958 >> v2:172.21.15.157:3300/0 conn(0x7fa4b4070660 0x7fa4b4071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 --2- 172.21.15.73:0/3112288958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4143660 0x7fa4b415a200 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 -- 172.21.15.73:0/3112288958 >> 172.21.15.73:0/3112288958 conn(0x7fa4b40b0830 msgr2=0x7fa4b40b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 -- 172.21.15.73:0/3112288958 shutdown_connections 2024-04-05T22:25:33.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.424+0000 7fa4b9200640 1 -- 172.21.15.73:0/3112288958 wait complete. 2024-04-05T22:25:33.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.426+0000 7fa4b9200640 1 Processor -- start 2024-04-05T22:25:33.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.426+0000 7fa4b9200640 1 -- start start 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b4139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4143660 0x7fa4b4141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b41f28e0 con 0x7fa4b4070660 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b40c8030 con 0x7fa4b4143660 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b40c8310 con 0x7fa4b406c630 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.427+0000 7fa4b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51540/0 (socket says 172.21.15.73:51540) 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 -- 172.21.15.73:0/1254982968 learned_addr learned my addr 172.21.15.73:0/1254982968 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b4139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 -- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 msgr2=0x7fa4b4139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b4139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 -- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4143660 msgr2=0x7fa4b4141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b2800640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4143660 0x7fa4b4141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4143660 0x7fa4b4141240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 -- 172.21.15.73:0/1254982968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a400b770 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b4139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.428+0000 7fa4b1400640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00c7f0 tx=0x7fa49c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.429+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c007a30 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.429+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c007b90 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.429+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00ce40 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.429+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b40c85f0 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.429+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b40c8bb0 con 0x7fa4b4070660 2024-04-05T22:25:33.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.430+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b4157d80 con 0x7fa4b4070660 2024-04-05T22:25:33.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.431+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fa49c0113f0 con 0x7fa4b4070660 2024-04-05T22:25:33.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.432+0000 7fa4aa000640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 0x7fa49006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.432+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa49c0899e0 con 0x7fa4b4070660 2024-04-05T22:25:33.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.432+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 0x7fa49006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.433+0000 7fa4b1e00640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 0x7fa49006d270 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4038000 tx=0x7fa4a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.436+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa49c054790 con 0x7fa4b4070660 2024-04-05T22:25:33.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.704+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa4b4068160 con 0x7fa4b4070660 2024-04-05T22:25:33.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.711+0000 7fa4aa000640 1 -- 172.21.15.73:0/1254982968 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa49c01f020 con 0x7fa4b4070660 2024-04-05T22:25:33.710 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:25:33.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 msgr2=0x7fa49006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 0x7fa49006d270 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4038000 tx=0x7fa4a4008040 comp rx=0 tx=0).stop 2024-04-05T22:25:33.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 msgr2=0x7fa4b41f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00c7f0 tx=0x7fa49c00cbf0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 shutdown_connections 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa49006adb0 0x7fa49006d270 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4143660 0x7fa4b4141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b4070660 0x7fa4b41f23a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 --2- 172.21.15.73:0/1254982968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4b406c630 0x7fa4b4139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 >> 172.21.15.73:0/1254982968 conn(0x7fa4b40b0830 msgr2=0x7fa4b40b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 shutdown_connections 2024-04-05T22:25:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.714+0000 7fa4b9200640 1 -- 172.21.15.73:0/1254982968 wait complete. 2024-04-05T22:25:33.732 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:33.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.937+0000 7f3af8800640 1 Processor -- start 2024-04-05T22:25:33.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.937+0000 7f3af8800640 1 -- start start 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af8800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3af006cb80 0x7f3af006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af8800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af015e850 0x7f3af015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af8800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af006f5b0 con 0x7f3af01587d0 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af8800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af006d500 con 0x7f3af006cb80 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af8800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af015f220 con 0x7f3af015e850 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3af006cb80 0x7f3af006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3af01587d0 0x7f3af00cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38102/0 (socket says 172.21.15.73:38102) 2024-04-05T22:25:33.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5e00640 1 -- 172.21.15.73:0/2572351706 learned_addr learned my addr 172.21.15.73:0/2572351706 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:33.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5400640 1 -- 172.21.15.73:0/2572351706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af015e850 msgr2=0x7f3af015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:33.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5400640 1 --2- 172.21.15.73:0/2572351706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af015e850 0x7f3af015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5400640 1 -- 172.21.15.73:0/2572351706 >> v1:172.21.15.73:6789/0 conn(0x7f3af01587d0 legacy=0x7f3af00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af5400640 1 -- 172.21.15.73:0/2572351706 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0070410 con 0x7f3af006cb80 2024-04-05T22:25:33.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.939+0000 7f3af4a00640 1 -- 172.21.15.73:0/2572351706 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878173177 0 0) 0x7f3af006f5b0 con 0x7f3af01587d0 2024-04-05T22:25:33.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.941+0000 7f3af5400640 1 --2- 172.21.15.73:0/2572351706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af006cb80 0x7f3af006cfc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00097d0 tx=0x7f3ae002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7b0f835e841c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.941+0000 7f3af4a00640 1 -- 172.21.15.73:0/2572351706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0031800 con 0x7f3af006cb80 2024-04-05T22:25:33.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.941+0000 7f3af4a00640 1 -- 172.21.15.73:0/2572351706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae0031960 con 0x7f3af006cb80 2024-04-05T22:25:33.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.941+0000 7f3af4a00640 1 -- 172.21.15.73:0/2572351706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0031cf0 con 0x7f3af006cb80 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 -- 172.21.15.73:0/2572351706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af006cb80 msgr2=0x7f3af006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 --2- 172.21.15.73:0/2572351706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af006cb80 0x7f3af006cfc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00097d0 tx=0x7f3ae002b8a0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 -- 172.21.15.73:0/2572351706 shutdown_connections 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 --2- 172.21.15.73:0/2572351706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af015e850 0x7f3af015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 --2- 172.21.15.73:0/2572351706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af006cb80 0x7f3af006cfc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 -- 172.21.15.73:0/2572351706 >> 172.21.15.73:0/2572351706 conn(0x7f3af00ce8e0 msgr2=0x7f3af00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 -- 172.21.15.73:0/2572351706 shutdown_connections 2024-04-05T22:25:33.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.942+0000 7f3af8800640 1 -- 172.21.15.73:0/2572351706 wait complete. 2024-04-05T22:25:33.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.944+0000 7f3af8800640 1 Processor -- start 2024-04-05T22:25:33.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.944+0000 7f3af8800640 1 -- start start 2024-04-05T22:25:33.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af006cb80 0x7f3af01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af01587d0 0x7f3af015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00ad950 con 0x7f3af015e850 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00b2120 con 0x7f3af006cb80 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af8800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00b1b10 con 0x7f3af01587d0 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51566/0 (socket says 172.21.15.73:51566) 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 -- 172.21.15.73:0/2361937263 learned_addr learned my addr 172.21.15.73:0/2361937263 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af5400640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af01587d0 0x7f3af015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 -- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af01587d0 msgr2=0x7f3af015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af5e00640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af006cb80 0x7f3af01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af01587d0 0x7f3af015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 -- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af006cb80 msgr2=0x7f3af01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af006cb80 0x7f3af01543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.945+0000 7f3af6800640 1 -- 172.21.15.73:0/2361937263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009400 con 0x7f3af015e850 2024-04-05T22:25:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3af6800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b970 tx=0x7f3aec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00cb40 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec00ccc0 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec014680 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af00ad670 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.946+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af0154910 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.947+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af0066af0 con 0x7f3af015e850 2024-04-05T22:25:33.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.948+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f3aec01d3b0 con 0x7f3af015e850 2024-04-05T22:25:33.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.949+0000 7f3aea000640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 0x7f3ad406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:33.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.949+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aec089630 con 0x7f3af015e850 2024-04-05T22:25:33.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.949+0000 7f3af5e00640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 0x7f3ad406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:33.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.950+0000 7f3af5e00640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 0x7f3ad406d2a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4007fd0 tx=0x7f3ae4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:33.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:33.954+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3aec0544f0 con 0x7f3af015e850 2024-04-05T22:25:34.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.187+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3af00b1970 con 0x7f3af015e850 2024-04-05T22:25:34.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.188+0000 7f3aea000640 1 -- 172.21.15.73:0/2361937263 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f3af00b1970 con 0x7f3af015e850 2024-04-05T22:25:34.187 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:34.187 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:34.188 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 32 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.190+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 msgr2=0x7f3ad406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 0x7f3ad406d2a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4007fd0 tx=0x7f3ae4009040 comp rx=0 tx=0).stop 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 msgr2=0x7f3af015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b970 tx=0x7f3aec00be30 comp rx=0 tx=0).stop 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 shutdown_connections 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3ad406ade0 0x7f3ad406d2a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af015e850 0x7f3af015a610 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af01587d0 0x7f3af015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 --2- 172.21.15.73:0/2361937263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af006cb80 0x7f3af01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 >> 172.21.15.73:0/2361937263 conn(0x7f3af00ce8e0 msgr2=0x7f3af00cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 shutdown_connections 2024-04-05T22:25:34.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.191+0000 7f3af8800640 1 -- 172.21.15.73:0/2361937263 wait complete. 2024-04-05T22:25:34.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:25:23.270861+0000', 'modified': '2024-04-05T22:25:26.297974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6195': {'gid': 6195, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/2831205947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 2831205947}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2831205947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:25:34.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:25:34.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-05T22:25:34.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.831+0000 7fcf29600640 1 Processor -- start 2024-04-05T22:25:34.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.831+0000 7fcf29600640 1 -- start start 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf29600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf2406c2d0 0x7fcf240cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf29600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf240cc760 con 0x7fcf241441e0 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf29600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf240b1b60 con 0x7fcf240b2c00 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf29600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf240b1950 con 0x7fcf2406c2d0 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf28c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf2406c2d0 0x7fcf240cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf28c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf2406c2d0 0x7fcf240cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51578/0 (socket says 172.21.15.73:51578) 2024-04-05T22:25:34.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf28c00640 1 -- 172.21.15.73:0/1918579047 learned_addr learned my addr 172.21.15.73:0/1918579047 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.833+0000 7fcf23e00640 1 --1- 172.21.15.73:0/1918579047 >> v1:172.21.15.73:6789/0 conn(0x7fcf241441e0 0x7fcf240cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38104/0 (socket says 172.21.15.73:38104) 2024-04-05T22:25:34.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.834+0000 7fcf28c00640 1 -- 172.21.15.73:0/1918579047 >> v1:172.21.15.157:6789/0 conn(0x7fcf240b2c00 legacy=0x7fcf2406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.834+0000 7fcf28c00640 1 -- 172.21.15.73:0/1918579047 >> v1:172.21.15.73:6789/0 conn(0x7fcf241441e0 legacy=0x7fcf240cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.834+0000 7fcf28c00640 1 -- 172.21.15.73:0/1918579047 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240b1e40 con 0x7fcf2406c2d0 2024-04-05T22:25:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.835+0000 7fcf28c00640 1 --2- 172.21.15.73:0/1918579047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf240cdb90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009870 tx=0x7fcf1c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1ab39af5a0fab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.835+0000 7fcf23400640 1 -- 172.21.15.73:0/1918579047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c030850 con 0x7fcf2406c2d0 2024-04-05T22:25:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.835+0000 7fcf23400640 1 -- 172.21.15.73:0/1918579047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0309b0 con 0x7fcf2406c2d0 2024-04-05T22:25:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.835+0000 7fcf23400640 1 -- 172.21.15.73:0/1918579047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c030d40 con 0x7fcf2406c2d0 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 -- 172.21.15.73:0/1918579047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 msgr2=0x7fcf240cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 --2- 172.21.15.73:0/1918579047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf240cdb90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009870 tx=0x7fcf1c02b990 comp rx=0 tx=0).stop 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 -- 172.21.15.73:0/1918579047 shutdown_connections 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 --2- 172.21.15.73:0/1918579047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf240cdb90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 -- 172.21.15.73:0/1918579047 >> 172.21.15.73:0/1918579047 conn(0x7fcf240b0830 msgr2=0x7fcf240b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 -- 172.21.15.73:0/1918579047 shutdown_connections 2024-04-05T22:25:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.836+0000 7fcf29600640 1 -- 172.21.15.73:0/1918579047 wait complete. 2024-04-05T22:25:34.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.837+0000 7fcf29600640 1 Processor -- start 2024-04-05T22:25:34.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.838+0000 7fcf29600640 1 -- start start 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.838+0000 7fcf29600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf29600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 0x7fcf2413da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf29600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf241441e0 0x7fcf2413df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf29600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2413f390 con 0x7fcf2406c2d0 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf29600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24154e20 con 0x7fcf240b2c00 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf29600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2413e620 con 0x7fcf241441e0 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51580/0 (socket says 172.21.15.73:51580) 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 -- 172.21.15.73:0/3502535025 learned_addr learned my addr 172.21.15.73:0/3502535025 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 -- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf241441e0 msgr2=0x7fcf2413df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf241441e0 0x7fcf2413df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 0x7fcf2413da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 -- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 msgr2=0x7fcf2413da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 0x7fcf2413da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf23e00640 1 -- 172.21.15.73:0/3502535025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0094d0 con 0x7fcf2406c2d0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.839+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 0x7fcf2413da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf23e00640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c7a0 tx=0x7fcf1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf20c00640 1 -- 172.21.15.73:0/3502535025 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400a640 con 0x7fcf2406c2d0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf20c00640 1 -- 172.21.15.73:0/3502535025 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1400a7a0 con 0x7fcf2406c2d0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2413e900 con 0x7fcf2406c2d0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf20c00640 1 -- 172.21.15.73:0/3502535025 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400ab10 con 0x7fcf2406c2d0 2024-04-05T22:25:34.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.840+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2413f8c0 con 0x7fcf2406c2d0 2024-04-05T22:25:34.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.842+0000 7fcf20c00640 1 -- 172.21.15.73:0/3502535025 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fcf1400ac70 con 0x7fcf2406c2d0 2024-04-05T22:25:34.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.843+0000 7fcf20c00640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 0x7fcf0006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.843+0000 7fcf20c00640 1 -- 172.21.15.73:0/3502535025 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf14088d50 con 0x7fcf2406c2d0 2024-04-05T22:25:34.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.843+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 0x7fcf0006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.844+0000 7fcf2dba0740 1 Processor -- start 2024-04-05T22:25:34.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.844+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 0x7fcf0006d270 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcf24152650 tx=0x7fcf0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.844+0000 7fcf2dba0740 1 -- start start 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd428fee0 0x562dd428daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 0x562dd42720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd41b28c0 con 0x562dd426f720 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd41b0050 con 0x562dd4247950 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd41ab940 con 0x562dd428fee0 2024-04-05T22:25:34.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51592/0 (socket says 172.21.15.73:51592) 2024-04-05T22:25:34.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf1b400640 1 -- 172.21.15.73:0/3734213154 learned_addr learned my addr 172.21.15.73:0/3734213154 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.845+0000 7fcf28c00640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 0x562dd42720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf23e00640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd428fee0 0x562dd428daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 -- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd428fee0 msgr2=0x562dd428daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd428fee0 0x562dd428daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 -- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 msgr2=0x562dd42720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 0x562dd42720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 -- 172.21.15.73:0/3734213154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd41769e0 con 0x562dd426f720 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf28c00640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 0x562dd42720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf1b400640 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00da90 tx=0x7fcf0c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00875f53721c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf18c00640 1 -- 172.21.15.73:0/3734213154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0149a0 con 0x562dd426f720 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf18c00640 1 -- 172.21.15.73:0/3734213154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c014b00 con 0x562dd426f720 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.846+0000 7fcf18c00640 1 -- 172.21.15.73:0/3734213154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01f400 con 0x562dd426f720 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 -- 172.21.15.73:0/3734213154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 msgr2=0x562dd4247410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00da90 tx=0x7fcf0c010000 comp rx=0 tx=0).stop 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 -- 172.21.15.73:0/3734213154 shutdown_connections 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4247950 0x562dd42720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd426f720 0x562dd4247410 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/3734213154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd428fee0 0x562dd428daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 -- 172.21.15.73:0/3734213154 >> 172.21.15.73:0/3734213154 conn(0x562dd4244570 msgr2=0x562dd428d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 -- 172.21.15.73:0/3734213154 shutdown_connections 2024-04-05T22:25:34.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.847+0000 7fcf2dba0740 1 -- 172.21.15.73:0/3734213154 wait complete. 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.848+0000 7fcf2dba0740 1 -- start start 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.848+0000 7fcf2dba0740 1 Processor -- start 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 0x562dd4273770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd4271740 0x562dd42a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd414d070 con 0x562dd428fee0 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd414cd10 con 0x562dd426f720 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd3f12f20 con 0x562dd4271740 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf2dba0740 10 client.0 _ll_register_callbacks cb 0x562dd41a4fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51604/0 (socket says 172.21.15.73:51604) 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.849+0000 7fcf1b400640 1 -- 172.21.15.73:0/2375729516 learned_addr learned my addr 172.21.15.73:0/2375729516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf28c00640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd4271740 0x562dd42a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 0x562dd4273770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 -- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd4271740 msgr2=0x562dd42a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd4271740 0x562dd42a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 -- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 msgr2=0x562dd4273770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 0x562dd4273770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 -- 172.21.15.73:0/2375729516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c00d7a0 con 0x562dd428fee0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 0x562dd4273770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf1b400640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c01e000 tx=0x7fcf0c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b411a7a315e5a403 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf11600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007440 con 0x562dd428fee0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf0c0075a0 con 0x562dd428fee0 2024-04-05T22:25:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.850+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dd41d5ae0 con 0x562dd428fee0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0c0067d0 con 0x562dd428fee0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 10 client.6369 fetch_fsmap learned FSMap version 32 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562dd41a9fc0 con 0x562dd428fee0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562dd41d5ae0 con 0x562dd428fee0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf0c006930 con 0x562dd428fee0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 10 client.6369 fetch_fsmap finished waiting for FSMap version 32 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 10 client.6369 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 20 client.6369 populate_metadata read hostname 'smithi073' 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4274710 0x562dd4274af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 4 client.6369 mds_command: new command op to 6195 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 --> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dd3e9f3c0 con 0x562dd4274710 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.851+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4274710 0x562dd4274af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.852+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4274710 0x562dd4274af0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3459d4f699f26bc0 server_cookie=37aab00f6f6f8f61 in_seq=0 out_seq=0 2024-04-05T22:25:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.852+0000 7fcf11600640 10 client.6369 ms_handle_connect on v2:172.21.15.73:6835/2831205947 2024-04-05T22:25:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf11600640 1 -- 172.21.15.73:0/2375729516 <== mds.0 v2:172.21.15.73:6835/2831205947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562dd3e9f3c0 con 0x562dd4274710 2024-04-05T22:25:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf11600640 10 client.6369 handle_command_reply: tid=0 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 client.6369 shutdown 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 msgr2=0x562dd426afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c01e000 tx=0x7fcf0c0036f0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 shutdown_connections 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd4271740 0x562dd42a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd426afa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4274710 0x562dd4274af0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2375729516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd426f720 0x562dd4273770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 >> 172.21.15.73:0/2375729516 conn(0x562dd4247950 msgr2=0x562dd428d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf11600640 0 client.6369 ms_handle_reset on v2:172.21.15.73:6835/2831205947 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 shutdown_connections 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2375729516 wait complete. 2024-04-05T22:25:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.853+0000 7fcf2dba0740 20 client.6369 trim_cache size 0 max 16384 2024-04-05T22:25:34.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.870+0000 7fcf2dba0740 1 Processor -- start 2024-04-05T22:25:34.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.870+0000 7fcf2dba0740 1 -- start start 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd42b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd423dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd4275cc0 con 0x562dd428fee0 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd428fd80 con 0x562dd4271740 2024-04-05T22:25:34.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd4274af0 con 0x562dd426f720 2024-04-05T22:25:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf28c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf28c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51618/0 (socket says 172.21.15.73:51618) 2024-04-05T22:25:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.871+0000 7fcf28c00640 1 -- 172.21.15.73:0/254722446 learned_addr learned my addr 172.21.15.73:0/254722446 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf1b400640 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd423dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 -- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 msgr2=0x562dd42b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd42b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 -- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 msgr2=0x562dd423dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd423dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 -- 172.21.15.73:0/254722446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd41def80 con 0x562dd428fee0 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf28c00640 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c02bfa0 tx=0x7fcf1c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34301db9351f3cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf19600640 1 -- 172.21.15.73:0/254722446 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c030b20 con 0x562dd428fee0 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf19600640 1 -- 172.21.15.73:0/254722446 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c030ca0 con 0x562dd428fee0 2024-04-05T22:25:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.872+0000 7fcf19600640 1 -- 172.21.15.73:0/254722446 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c039690 con 0x562dd428fee0 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 -- 172.21.15.73:0/254722446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 msgr2=0x562dd4246a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c02bfa0 tx=0x7fcf1c030000 comp rx=0 tx=0).stop 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 -- 172.21.15.73:0/254722446 shutdown_connections 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd4246a70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd423dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/254722446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd42b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 -- 172.21.15.73:0/254722446 >> 172.21.15.73:0/254722446 conn(0x562dd4247950 msgr2=0x562dd426adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 -- 172.21.15.73:0/254722446 shutdown_connections 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.873+0000 7fcf2dba0740 1 -- 172.21.15.73:0/254722446 wait complete. 2024-04-05T22:25:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.874+0000 7fcf2dba0740 1 -- start start 2024-04-05T22:25:34.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.874+0000 7fcf2dba0740 1 Processor -- start 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd4270a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd42b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd4290a20 con 0x562dd428fee0 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd42b3c40 con 0x562dd4271740 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd42fd9b0 con 0x562dd426f720 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf2dba0740 10 client.0 _ll_register_callbacks cb 0x7fcf0c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf28c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf28c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51632/0 (socket says 172.21.15.73:51632) 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.875+0000 7fcf28c00640 1 -- 172.21.15.73:0/2366763939 learned_addr learned my addr 172.21.15.73:0/2366763939 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf1b400640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd42b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 -- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 msgr2=0x562dd4270a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd4270a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 -- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 msgr2=0x562dd42b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd42b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 -- 172.21.15.73:0/2366763939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf240b1e40 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf28c00640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009870 tx=0x7fcf1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0b43a594e2f129 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf11600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0044f0 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf1c03e040 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.876+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf0c014b00 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c030e10 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 10 client.6375 fetch_fsmap learned FSMap version 32 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562dd414c280 con 0x562dd428fee0 2024-04-05T22:25:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c014b00 con 0x562dd428fee0 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf1c003820 con 0x562dd428fee0 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 10 client.6375 fetch_fsmap finished waiting for FSMap version 32 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 10 client.6375 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 20 client.6375 populate_metadata read hostname 'smithi073' 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4246a70 0x562dd4246e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 4 client.6375 mds_command: new command op to 6195 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:25:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 --> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562dd3eb06c0 con 0x562dd4246a70 2024-04-05T22:25:34.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.877+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4246a70 0x562dd4246e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:34.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf23e00640 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4246a70 0x562dd4246e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb6ce9dd6965bb2 server_cookie=53d5b93a1991955b in_seq=0 out_seq=0 2024-04-05T22:25:34.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf11600640 10 client.6375 ms_handle_connect on v2:172.21.15.73:6835/2831205947 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf11600640 1 -- 172.21.15.73:0/2366763939 <== mds.0 v2:172.21.15.73:6835/2831205947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562dd3eb06c0 con 0x562dd4246a70 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf11600640 10 client.6375 handle_command_reply: tid=0 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 client.6375 shutdown 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 msgr2=0x562dd42fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009870 tx=0x7fcf1c004730 comp rx=0 tx=0).stop 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 shutdown_connections 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562dd428fee0 0x562dd42fcc90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.73:6835/2831205947,v1:172.21.15.73:6837/2831205947] conn(0x562dd4246a70 0x562dd4246e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562dd4271740 0x562dd42b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 --2- 172.21.15.73:0/2366763939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562dd426f720 0x562dd4270a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf11600640 0 client.6375 ms_handle_reset on v2:172.21.15.73:6835/2831205947 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.878+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 >> 172.21.15.73:0/2366763939 conn(0x562dd4247950 msgr2=0x562dd42483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 shutdown_connections 2024-04-05T22:25:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf2dba0740 1 -- 172.21.15.73:0/2366763939 wait complete. 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf2dba0740 20 client.6375 trim_cache size 0 max 16384 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 6195, 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 11.602945499, 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 32, 2024-04-05T22:25:34.881 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 62, 2024-04-05T22:25:34.882 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 62, 2024-04-05T22:25:34.882 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 16.489768952999999 2024-04-05T22:25:34.882 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:25:34.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 msgr2=0x7fcf0006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 0x7fcf0006d270 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcf24152650 tx=0x7fcf0c009040 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 msgr2=0x7fcf2415c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c7a0 tx=0x7fcf1400cc60 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 shutdown_connections 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcf0006adb0 0x7fcf0006d270 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf241441e0 0x7fcf2413df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf240b2c00 0x7fcf2413da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 --2- 172.21.15.73:0/3502535025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf2406c2d0 0x7fcf2415c810 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.879+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 >> 172.21.15.73:0/3502535025 conn(0x7fcf240b0830 msgr2=0x7fcf2406bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.880+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 shutdown_connections 2024-04-05T22:25:34.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:34.880+0000 7fcf29600640 1 -- 172.21.15.73:0/3502535025 wait complete. 2024-04-05T22:25:34.898 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:25:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.102+0000 7f6666000640 1 Processor -- start 2024-04-05T22:25:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.102+0000 7f6666000640 1 -- start start 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6666000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 0x7f66600b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6666000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f666006d710 0x7f66600cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6666000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666015aa60 con 0x7f666015aef0 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6666000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600b3b60 con 0x7f666006d710 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6666000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600701c0 con 0x7f66601541a0 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f665fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f666015aef0 0x7f666015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38108/0 (socket says 172.21.15.73:38108) 2024-04-05T22:25:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f665fe00640 1 -- 172.21.15.73:0/1180990611 learned_addr learned my addr 172.21.15.73:0/1180990611 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f6656a00640 1 --2- 172.21.15.73:0/1180990611 >> v2:172.21.15.157:3300/0 conn(0x7f666006d710 0x7f66600cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.104+0000 7f665f400640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 0x7f66600b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947117876 0 0) 0x7f666015aa60 con 0x7f666015aef0 2024-04-05T22:25:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6644003520 con 0x7f666015aef0 2024-04-05T22:25:35.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335002773 0 0) 0x7f6644003520 con 0x7f666015aef0 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 msgr2=0x7f66600b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 0x7f66600b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 >> v2:172.21.15.157:3300/0 conn(0x7f666006d710 msgr2=0x7f66600cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 --2- 172.21.15.73:0/1180990611 >> v2:172.21.15.157:3300/0 conn(0x7f666006d710 0x7f66600cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600b3880 con 0x7f666015aef0 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6658001670 con 0x7f666015aef0 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 >> v1:172.21.15.73:6789/0 conn(0x7f666015aef0 legacy=0x7f666015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644003970 0x7f6644003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66440047c0 0x7f6644004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 0x7f6644005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600b3b60 con 0x7f66440047c0 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600701c0 con 0x7f6644003970 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664400b980 con 0x7f66601541a0 2024-04-05T22:25:35.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.105+0000 7f6656a00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66440047c0 0x7f6644004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.106+0000 7f6656a00640 1 -- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 msgr2=0x7f6644005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.106+0000 7f6656a00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66601541a0 0x7f6644005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.106+0000 7f6656a00640 1 -- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644003970 msgr2=0x7f6644003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.106+0000 7f6656a00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644003970 0x7f6644003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.106+0000 7f6656a00640 1 -- 172.21.15.73:0/1180990611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644003520 con 0x7f66440047c0 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f6656a00640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66440047c0 0x7f6644004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6648009fc0 tx=0x7f66440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c6786be4ffd54c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664802cc60 con 0x7f66440047c0 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664800f040 con 0x7f66440047c0 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f665ea00640 1 -- 172.21.15.73:0/1180990611 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648034440 con 0x7f66440047c0 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f6666000640 1 -- 172.21.15.73:0/1180990611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66440047c0 msgr2=0x7f6644004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f6666000640 1 --2- 172.21.15.73:0/1180990611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66440047c0 0x7f6644004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6648009fc0 tx=0x7f66440057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.107+0000 7f665fe00640 1 -- 172.21.15.73:0/1180990611 reap_dead start 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.108+0000 7f6666000640 1 -- 172.21.15.73:0/1180990611 shutdown_connections 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.108+0000 7f6666000640 1 -- 172.21.15.73:0/1180990611 >> 172.21.15.73:0/1180990611 conn(0x7f66600ce8e0 msgr2=0x7f66600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.108+0000 7f6666000640 1 -- 172.21.15.73:0/1180990611 shutdown_connections 2024-04-05T22:25:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.108+0000 7f6666000640 1 -- 172.21.15.73:0/1180990611 wait complete. 2024-04-05T22:25:35.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.109+0000 7f6666000640 1 Processor -- start 2024-04-05T22:25:35.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.109+0000 7f6666000640 1 -- start start 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440047c0 0x7f6660140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600af670 0x7f6660141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660144e60 con 0x7f666013fe80 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600afaf0 con 0x7f66600af670 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.110+0000 7f6666000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6660144220 con 0x7f66440047c0 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51650/0 (socket says 172.21.15.73:51650) 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 -- 172.21.15.73:0/3436687277 learned_addr learned my addr 172.21.15.73:0/3436687277 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 -- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440047c0 msgr2=0x7f6660140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f665f400640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440047c0 0x7f6660140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440047c0 0x7f6660140f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f665fe00640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600af670 0x7f6660141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 -- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600af670 msgr2=0x7f6660141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600af670 0x7f6660141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.111+0000 7f6656a00640 1 -- 172.21.15.73:0/3436687277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66601444c0 con 0x7f666013fe80 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6656a00640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f664800e230 tx=0x7f6648009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648034c50 con 0x7f666013fe80 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648034db0 con 0x7f666013fe80 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648003b90 con 0x7f666013fe80 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66601447a0 con 0x7f666013fe80 2024-04-05T22:25:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.112+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6660140300 con 0x7f666013fe80 2024-04-05T22:25:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.113+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660066af0 con 0x7f666013fe80 2024-04-05T22:25:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.114+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f6648040450 con 0x7f666013fe80 2024-04-05T22:25:35.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.115+0000 7f6657e00640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 0x7f663c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:35.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.115+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66480abfd0 con 0x7f666013fe80 2024-04-05T22:25:35.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.115+0000 7f665f400640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 0x7f663c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:35.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.116+0000 7f665f400640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 0x7f663c06d2a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6650006a90 tx=0x7f6650008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.119+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6648077050 con 0x7f666013fe80 2024-04-05T22:25:35.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.351+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f666014cda0 con 0x7f666013fe80 2024-04-05T22:25:35.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.352+0000 7f6657e00640 1 -- 172.21.15.73:0/3436687277 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f664807a670 con 0x7f666013fe80 2024-04-05T22:25:35.351 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.354+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 msgr2=0x7f663c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.354+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 0x7f663c06d2a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6650006a90 tx=0x7f6650008040 comp rx=0 tx=0).stop 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.354+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 msgr2=0x7f66600af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.354+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f664800e230 tx=0x7f6648009750 comp rx=0 tx=0).stop 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 shutdown_connections 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f663c06ade0 0x7f663c06d2a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600af670 0x7f6660141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f666013fe80 0x7f66600af130 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 --2- 172.21.15.73:0/3436687277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440047c0 0x7f6660140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:35.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 >> 172.21.15.73:0/3436687277 conn(0x7f66600ce8e0 msgr2=0x7f666006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:35.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 shutdown_connections 2024-04-05T22:25:35.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:35.355+0000 7f6666000640 1 -- 172.21.15.73:0/3436687277 wait complete. 2024-04-05T22:25:35.373 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:25:35.373 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:25:35.373 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:25:35.373 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:25:35.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:25:35.374 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:25:35.374 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:25:35.374 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:25:35.374 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:25:35.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.374 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:25:35.423 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:25:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:25:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:25:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:25:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42328sec preferred_lft 42328sec 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:25:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:25:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:25:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:35.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.427 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:25:35.516 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:25:35.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.520 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:25:35.572 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:25:35.572 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:25:35.573 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:25:35.573 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:25:35.573 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:25:35.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:25:35.574 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:25:35.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.746 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:25:35.794 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:25:35.794 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:25:35.795 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:25:35.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:25:35.883 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:25:35.883 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:25:35.934 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:25:35.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:25:35.935 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:25:35.984 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:25:35.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:35.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:36.117 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:36.117 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:25:36.152 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:36.152 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:25:36.152 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:25:36.152 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:25:36.152 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:25:36.154 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:25:36.155 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:25:36.155 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:25:36.155 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:25:36.155 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:25:36.159 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:25:36.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:36.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:25:36.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:36.242 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:25:36.306 INFO:teuthology.orchestra.run.smithi073.stdout:client5140 2024-04-05T22:25:36.307 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:36.307 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:36.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:25:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.611+0000 7fdbad400640 1 Processor -- start 2024-04-05T22:25:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.611+0000 7fdbad400640 1 -- start start 2024-04-05T22:25:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.612+0000 7fdbad400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdba81587d0 0x7fdba80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdbad400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdba80ccd20 0x7fdba806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdbad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 0x7fdba815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdbad400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba806f5b0 con 0x7fdba80ccd20 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdbad400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba8070410 con 0x7fdba81587d0 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdbad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8158bb0 con 0x7fdba806d0c0 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdba80ccd20 0x7fdba806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdba80ccd20 0x7fdba806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36558/0 (socket says 172.21.15.73:36558) 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 -- 172.21.15.73:0/2827258674 learned_addr learned my addr 172.21.15.73:0/2827258674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdba7200640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 0x7fdba815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 -- 172.21.15.73:0/2827258674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 msgr2=0x7fdba815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdba6800640 1 --2- 172.21.15.73:0/2827258674 >> v2:172.21.15.157:3300/0 conn(0x7fdba81587d0 0x7fdba80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 0x7fdba815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 -- 172.21.15.73:0/2827258674 >> v2:172.21.15.157:3300/0 conn(0x7fdba81587d0 msgr2=0x7fdba80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 --2- 172.21.15.73:0/2827258674 >> v2:172.21.15.157:3300/0 conn(0x7fdba81587d0 0x7fdba80cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdb9fe00640 1 -- 172.21.15.73:0/2827258674 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba806f800 con 0x7fdba80ccd20 2024-04-05T22:25:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.613+0000 7fdba7200640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 0x7fdba815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.614+0000 7fdb9fe00640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80ccd20 0x7fdba806cb80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdba80667b0 tx=0x7fdb9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea87f07d5e2f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.615+0000 7fdba5e00640 1 -- 172.21.15.73:0/2827258674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90031870 con 0x7fdba80ccd20 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.615+0000 7fdba5e00640 1 -- 172.21.15.73:0/2827258674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb900319d0 con 0x7fdba80ccd20 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.615+0000 7fdba5e00640 1 -- 172.21.15.73:0/2827258674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90031d60 con 0x7fdba80ccd20 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.615+0000 7fdbad400640 1 -- 172.21.15.73:0/2827258674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80ccd20 msgr2=0x7fdba806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.615+0000 7fdbad400640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80ccd20 0x7fdba806cb80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdba80667b0 tx=0x7fdb9002b910 comp rx=0 tx=0).stop 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 -- 172.21.15.73:0/2827258674 shutdown_connections 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80ccd20 0x7fdba806cb80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 --2- 172.21.15.73:0/2827258674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba806d0c0 0x7fdba815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 --2- 172.21.15.73:0/2827258674 >> v2:172.21.15.157:3300/0 conn(0x7fdba81587d0 0x7fdba80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 -- 172.21.15.73:0/2827258674 >> 172.21.15.73:0/2827258674 conn(0x7fdba80ce8e0 msgr2=0x7fdba80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 -- 172.21.15.73:0/2827258674 shutdown_connections 2024-04-05T22:25:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.616+0000 7fdbad400640 1 -- 172.21.15.73:0/2827258674 wait complete. 2024-04-05T22:25:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.618+0000 7fdbad400640 1 Processor -- start 2024-04-05T22:25:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.618+0000 7fdbad400640 1 -- start start 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 0x7fdba81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba81587d0 0x7fdba8154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8154dc0 con 0x7fdba80b2380 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b2b40 con 0x7fdba806d0c0 2024-04-05T22:25:36.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdbad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b2e20 con 0x7fdba81587d0 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdba7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.619+0000 7fdba7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36568/0 (socket says 172.21.15.73:36568) 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 -- 172.21.15.73:0/3150792485 learned_addr learned my addr 172.21.15.73:0/3150792485 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdb9fe00640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba81587d0 0x7fdba8154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba6800640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 0x7fdba81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 -- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba81587d0 msgr2=0x7fdba8154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba81587d0 0x7fdba8154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 -- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 msgr2=0x7fdba81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 0x7fdba81562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 -- 172.21.15.73:0/3150792485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900094d0 con 0x7fdba80b2380 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba6800640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 0x7fdba81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdba7200640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdba000cef0 tx=0x7fdba000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.620+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba000d840 con 0x7fdba80b2380 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.621+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba000d9a0 con 0x7fdba80b2380 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.621+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba000dd50 con 0x7fdba80b2380 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.621+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba80b3100 con 0x7fdba80b2380 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.621+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba80b2830 con 0x7fdba80b2380 2024-04-05T22:25:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.622+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba8066af0 con 0x7fdba80b2380 2024-04-05T22:25:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.623+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdba001e3b0 con 0x7fdba80b2380 2024-04-05T22:25:36.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.624+0000 7fdb9ea00640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 0x7fdb8406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:36.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.624+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdba008a150 con 0x7fdba80b2380 2024-04-05T22:25:36.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.624+0000 7fdba6800640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 0x7fdb8406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:36.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.625+0000 7fdba6800640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 0x7fdb8406d2a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b820 tx=0x7fdb98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.629+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdba0055010 con 0x7fdba80b2380 2024-04-05T22:25:36.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.863+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdba8068160 con 0x7fdba80b2380 2024-04-05T22:25:36.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.863+0000 7fdb9ea00640 1 -- 172.21.15.73:0/3150792485 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdba001a060 con 0x7fdba80b2380 2024-04-05T22:25:36.863 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:25:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 msgr2=0x7fdb8406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 0x7fdb8406d2a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b820 tx=0x7fdb98008040 comp rx=0 tx=0).stop 2024-04-05T22:25:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 msgr2=0x7fdba80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdba000cef0 tx=0x7fdba000a710 comp rx=0 tx=0).stop 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 shutdown_connections 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdb8406ade0 0x7fdb8406d2a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdba80b2380 0x7fdba80ac8d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba81587d0 0x7fdba8154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 --2- 172.21.15.73:0/3150792485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba806d0c0 0x7fdba81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 >> 172.21.15.73:0/3150792485 conn(0x7fdba80ce8e0 msgr2=0x7fdba815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 shutdown_connections 2024-04-05T22:25:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:36.866+0000 7fdbad400640 1 -- 172.21.15.73:0/3150792485 wait complete. 2024-04-05T22:25:36.882 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:25:36.883 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:25:36.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:36.883 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:25:36.941 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:25:36.942 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42350sec preferred_lft 42350sec 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.943 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:25:36.944 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:25:36.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:36.945 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:25:37.029 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:25:37.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.033 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:25:37.085 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:25:37.085 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:25:37.085 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:25:37.086 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:25:37.086 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:25:37.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:25:37.087 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:25:37.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.268 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:25:37.319 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:25:37.319 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:25:37.320 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:25:37.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:25:37.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:25:37.410 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:25:37.463 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:25:37.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:25:37.464 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:25:37.515 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:25:37.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:37.655 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:37.655 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:25:37.691 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:25:37.691 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:25:37.691 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:25:37.691 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:25:37.691 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:25:37.693 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:25:37.693 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:25:37.693 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:25:37.694 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:25:37.694 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:25:37.698 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:25:37.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.698 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:25:37.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:37.782 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:25:37.836 INFO:teuthology.orchestra.run.smithi157.stdout:client6390 2024-04-05T22:25:37.837 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:37.837 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:37.887 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:25:38.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.107+0000 7f7565a00640 1 Processor -- start 2024-04-05T22:25:38.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.107+0000 7f7565a00640 1 -- start start 2024-04-05T22:25:38.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7565a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f756015ae50 0x7f75600b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7565a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75600cce90 0x7f75600b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7565a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756015b230 con 0x7f75600cecd0 2024-04-05T22:25:38.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7565a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600ce2f0 con 0x7f75600cce90 2024-04-05T22:25:38.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7565a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cd2f0 con 0x7f756015ae50 2024-04-05T22:25:38.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7557e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75600cce90 0x7f75600b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7557e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75600cce90 0x7f75600b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36572/0 (socket says 172.21.15.73:36572) 2024-04-05T22:25:38.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.109+0000 7f7557e00640 1 -- 172.21.15.73:0/1120898574 learned_addr learned my addr 172.21.15.73:0/1120898574 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:38.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f755ea00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f756015ae50 0x7f75600b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f7557e00640 1 -- 172.21.15.73:0/1120898574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f756015ae50 msgr2=0x7f75600b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:38.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f7557e00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f756015ae50 0x7f75600b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f7557e00640 1 -- 172.21.15.73:0/1120898574 >> v1:172.21.15.73:6789/0 conn(0x7f75600cecd0 legacy=0x7f75600cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f755e000640 1 -- 172.21.15.73:0/1120898574 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596701687 0 0) 0x7f756015b230 con 0x7f75600cecd0 2024-04-05T22:25:38.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.110+0000 7f7557e00640 1 -- 172.21.15.73:0/1120898574 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756015f4a0 con 0x7f75600cce90 2024-04-05T22:25:38.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.111+0000 7f7557e00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cce90 0x7f75600b2e00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f75600ce430 tx=0x7f754802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d924fb9cb20831 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.111+0000 7f755e000640 1 -- 172.21.15.73:0/1120898574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031640 con 0x7f75600cce90 2024-04-05T22:25:38.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f755e000640 1 -- 172.21.15.73:0/1120898574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480317a0 con 0x7f75600cce90 2024-04-05T22:25:38.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f755e000640 1 -- 172.21.15.73:0/1120898574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031b30 con 0x7f75600cce90 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 -- 172.21.15.73:0/1120898574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cce90 msgr2=0x7f75600b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cce90 0x7f75600b2e00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f75600ce430 tx=0x7f754802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 -- 172.21.15.73:0/1120898574 shutdown_connections 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cce90 0x7f75600b2e00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 --2- 172.21.15.73:0/1120898574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f756015ae50 0x7f75600b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 -- 172.21.15.73:0/1120898574 >> 172.21.15.73:0/1120898574 conn(0x7f756006a0d0 msgr2=0x7f756006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 -- 172.21.15.73:0/1120898574 shutdown_connections 2024-04-05T22:25:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.112+0000 7f7565a00640 1 -- 172.21.15.73:0/1120898574 wait complete. 2024-04-05T22:25:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.114+0000 7f7565a00640 1 Processor -- start 2024-04-05T22:25:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.114+0000 7f7565a00640 1 -- start start 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600cce90 0x7f7560156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756015ae50 0x7f75600ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600b0ac0 con 0x7f75600cecd0 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560070850 con 0x7f756015ae50 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7565a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7560070b30 con 0x7f75600cce90 2024-04-05T22:25:38.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36588/0 (socket says 172.21.15.73:36588) 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.115+0000 7f7557e00640 1 -- 172.21.15.73:0/3434191591 learned_addr learned my addr 172.21.15.73:0/3434191591 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 -- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600cce90 msgr2=0x7f7560156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600cce90 0x7f7560156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 -- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756015ae50 msgr2=0x7f75600ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756015ae50 0x7f75600ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 -- 172.21.15.73:0/3434191591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480094d0 con 0x7f75600cecd0 2024-04-05T22:25:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7557e00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f75480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.116+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548004230 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.117+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480043b0 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.117+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548004520 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.117+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7560070e10 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.117+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75601569d0 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.118+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75600c4f10 con 0x7f75600cecd0 2024-04-05T22:25:38.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.119+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f7548043070 con 0x7f75600cecd0 2024-04-05T22:25:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.120+0000 7f7556a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 0x7f753c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.120+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75480ab4c0 con 0x7f75600cecd0 2024-04-05T22:25:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.120+0000 7f755ea00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 0x7f753c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.121+0000 7f755ea00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 0x7f753c06d270 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.124+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f75600c4f10 con 0x7f75600cecd0 2024-04-05T22:25:38.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.357+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f75600c4f10 con 0x7f75600cecd0 2024-04-05T22:25:38.627 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:25:38.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:25:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.673+0000 7f7556a00640 1 -- 172.21.15.73:0/3434191591 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 136+0+0 (secure 0 0 0) 0x7f75600c4f10 con 0x7f75600cecd0 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 msgr2=0x7f753c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 0x7f753c06d270 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 msgr2=0x7f75600ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f75480046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 shutdown_connections 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f753c06adb0 0x7f753c06d270 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756015ae50 0x7f75600ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75600cecd0 0x7f75600ac8d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 --2- 172.21.15.73:0/3434191591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600cce90 0x7f7560156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.676+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 >> 172.21.15.73:0/3434191591 conn(0x7f756006a0d0 msgr2=0x7f75600ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.677+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 shutdown_connections 2024-04-05T22:25:38.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.677+0000 7f7565a00640 1 -- 172.21.15.73:0/3434191591 wait complete. 2024-04-05T22:25:38.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:25:38.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.946+0000 7f3aa0600640 1 Processor -- start 2024-04-05T22:25:38.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.946+0000 7f3aa0600640 1 -- start start 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.947+0000 7f3aa0600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a981587d0 0x7f3a980cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.947+0000 7f3aa0600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 0x7f3a9806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.947+0000 7f3aa0600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9806f5b0 con 0x7f3a9806d0c0 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.947+0000 7f3aa0600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a98070410 con 0x7f3a981587d0 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.947+0000 7f3aa0600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a98158bb0 con 0x7f3a980ccd20 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a981587d0 0x7f3a980cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9e600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3a9806d0c0 0x7f3a9815e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60146/0 (socket says 172.21.15.73:60146) 2024-04-05T22:25:38.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9e600640 1 -- 172.21.15.73:0/4250643166 learned_addr learned my addr 172.21.15.73:0/4250643166 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9dc00640 1 -- 172.21.15.73:0/4250643166 >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 msgr2=0x7f3a9806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9d200640 1 --2- 172.21.15.73:0/4250643166 >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 0x7f3a9806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a97e00640 1 -- 172.21.15.73:0/4250643166 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939822814 0 0) 0x7f3a9806f5b0 con 0x7f3a9806d0c0 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9dc00640 1 --2- 172.21.15.73:0/4250643166 >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 0x7f3a9806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9dc00640 1 -- 172.21.15.73:0/4250643166 >> v1:172.21.15.73:6789/0 conn(0x7f3a9806d0c0 legacy=0x7f3a9815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9dc00640 1 -- 172.21.15.73:0/4250643166 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9806f800 con 0x7f3a981587d0 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.948+0000 7f3a9d200640 1 --2- 172.21.15.73:0/4250643166 >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 0x7f3a9806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.949+0000 7f3a9dc00640 1 --2- 172.21.15.73:0/4250643166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a980cc7e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009d90 tx=0x7f3a8c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44846a5090af39a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3a97e00640 1 -- 172.21.15.73:0/4250643166 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c02cc60 con 0x7f3a981587d0 2024-04-05T22:25:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3a97e00640 1 -- 172.21.15.73:0/4250643166 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c02cdc0 con 0x7f3a981587d0 2024-04-05T22:25:38.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3a97e00640 1 -- 172.21.15.73:0/4250643166 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c034660 con 0x7f3a981587d0 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 -- 172.21.15.73:0/4250643166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 msgr2=0x7f3a980cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 --2- 172.21.15.73:0/4250643166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a980cc7e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009d90 tx=0x7f3a8c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 -- 172.21.15.73:0/4250643166 shutdown_connections 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 --2- 172.21.15.73:0/4250643166 >> v2:172.21.15.157:3300/0 conn(0x7f3a980ccd20 0x7f3a9806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 --2- 172.21.15.73:0/4250643166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a980cc7e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.950+0000 7f3aa0600640 1 -- 172.21.15.73:0/4250643166 >> 172.21.15.73:0/4250643166 conn(0x7f3a980ce8e0 msgr2=0x7f3a980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.951+0000 7f3aa0600640 1 -- 172.21.15.73:0/4250643166 shutdown_connections 2024-04-05T22:25:38.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.951+0000 7f3aa0600640 1 -- 172.21.15.73:0/4250643166 wait complete. 2024-04-05T22:25:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.952+0000 7f3aa0600640 1 Processor -- start 2024-04-05T22:25:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.952+0000 7f3aa0600640 1 -- start start 2024-04-05T22:25:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a9806d0c0 0x7f3a980aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 0x7f3a98150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980af7a0 con 0x7f3a981587d0 2024-04-05T22:25:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980aeb20 con 0x7f3a98152c70 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.953+0000 7f3aa0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980aee00 con 0x7f3a9806d0c0 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36608/0 (socket says 172.21.15.73:36608) 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 -- 172.21.15.73:0/3537906884 learned_addr learned my addr 172.21.15.73:0/3537906884 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 -- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a9806d0c0 msgr2=0x7f3a980aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a9806d0c0 0x7f3a980aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9e600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 0x7f3a98150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 -- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 msgr2=0x7f3a98150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 0x7f3a98150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 -- 172.21.15.73:0/3537906884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c009a70 con 0x7f3a981587d0 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9e600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 0x7f3a98150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.954+0000 7f3a9d200640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c7f0 tx=0x7f3a8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.955+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007a30 con 0x7f3a981587d0 2024-04-05T22:25:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.955+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84007b90 con 0x7f3a981587d0 2024-04-05T22:25:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.955+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a980af0e0 con 0x7f3a981587d0 2024-04-05T22:25:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.955+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400ce40 con 0x7f3a981587d0 2024-04-05T22:25:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.955+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a980ae780 con 0x7f3a981587d0 2024-04-05T22:25:38.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.956+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a98066af0 con 0x7f3a981587d0 2024-04-05T22:25:38.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.957+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f3a840113f0 con 0x7f3a981587d0 2024-04-05T22:25:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.958+0000 7f3a95600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 0x7f3a7806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.958+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a84089980 con 0x7f3a981587d0 2024-04-05T22:25:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.958+0000 7f3a9dc00640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 0x7f3a7806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.959+0000 7f3a9dc00640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 0x7f3a7806d270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c036000 tx=0x7f3a8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:38.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:38.962+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3a840547c0 con 0x7f3a981587d0 2024-04-05T22:25:39.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.181+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a980700a0 con 0x7f3a7806adb0 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.185+0000 7f3a95600640 1 -- 172.21.15.73:0/3537906884 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3a8c02ca60 con 0x7f3a7806adb0 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:39.184 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 msgr2=0x7f3a7806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 0x7f3a7806d270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c036000 tx=0x7f3a8c02e000 comp rx=0 tx=0).stop 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 msgr2=0x7f3a98152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c7f0 tx=0x7f3a8400cbf0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 shutdown_connections 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3a7806adb0 0x7f3a7806d270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a98152c70 0x7f3a98150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a981587d0 0x7f3a98152320 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 --2- 172.21.15.73:0/3537906884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a9806d0c0 0x7f3a980aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 >> 172.21.15.73:0/3537906884 conn(0x7f3a980ce8e0 msgr2=0x7f3a9815dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:39.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.187+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 shutdown_connections 2024-04-05T22:25:39.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.188+0000 7f3aa0600640 1 -- 172.21.15.73:0/3537906884 wait complete. 2024-04-05T22:25:39.204 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:25:39.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.413+0000 7f3be1000640 1 Processor -- start 2024-04-05T22:25:39.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.413+0000 7f3be1000640 1 -- start start 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3be1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc15abe0 0x7f3bdc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3be1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bdc06c9a0 0x7f3bdc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3be1000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc06bec0 con 0x7f3bdc06ba70 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3be1000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc15afc0 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3be1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc06f5b0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3bd9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bdc06c9a0 0x7f3bdc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3bd9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bdc06c9a0 0x7f3bdc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36616/0 (socket says 172.21.15.73:36616) 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3bda800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3bdc06ba70 0x7f3bdc0cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60160/0 (socket says 172.21.15.73:60160) 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.415+0000 7f3bd9400640 1 -- 172.21.15.73:0/1237728078 learned_addr learned my addr 172.21.15.73:0/1237728078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd9e00640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc15abe0 0x7f3bdc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd9400640 1 -- 172.21.15.73:0/1237728078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc15abe0 msgr2=0x7f3bdc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd9400640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc15abe0 0x7f3bdc0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd9400640 1 -- 172.21.15.73:0/1237728078 >> v1:172.21.15.73:6789/0 conn(0x7f3bdc06ba70 legacy=0x7f3bdc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd9400640 1 -- 172.21.15.73:0/1237728078 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc06f890 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.416+0000 7f3bd8a00640 1 -- 172.21.15.73:0/1237728078 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061311661 0 0) 0x7f3bd4001490 con 0x7f3bdc06ba70 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.417+0000 7f3bd9400640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc06b530 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009870 tx=0x7f3bc802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e49c8ec4337d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.417+0000 7f3bd8a00640 1 -- 172.21.15.73:0/1237728078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80317c0 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.417+0000 7f3bd8a00640 1 -- 172.21.15.73:0/1237728078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8031920 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.417+0000 7f3bd8a00640 1 -- 172.21.15.73:0/1237728078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8031cb0 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 -- 172.21.15.73:0/1237728078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc06c9a0 msgr2=0x7f3bdc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc06b530 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009870 tx=0x7f3bc802b860 comp rx=0 tx=0).stop 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 -- 172.21.15.73:0/1237728078 shutdown_connections 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc06b530 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 --2- 172.21.15.73:0/1237728078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc15abe0 0x7f3bdc0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 -- 172.21.15.73:0/1237728078 >> 172.21.15.73:0/1237728078 conn(0x7f3bdc069cb0 msgr2=0x7f3bdc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 -- 172.21.15.73:0/1237728078 shutdown_connections 2024-04-05T22:25:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.418+0000 7f3be1000640 1 -- 172.21.15.73:0/1237728078 wait complete. 2024-04-05T22:25:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.420+0000 7f3be1000640 1 Processor -- start 2024-04-05T22:25:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.420+0000 7f3be1000640 1 -- start start 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc06ba70 0x7f3bdc13f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc155e40 con 0x7f3bdc15abe0 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc1f69a0 con 0x7f3bdc06c9a0 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3be1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc13fad0 con 0x7f3bdc06ba70 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36618/0 (socket says 172.21.15.73:36618) 2024-04-05T22:25:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 -- 172.21.15.73:0/3407907255 learned_addr learned my addr 172.21.15.73:0/3407907255 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bd9400640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bd9e00640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc06ba70 0x7f3bdc13f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 -- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc06ba70 msgr2=0x7f3bdc13f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc06ba70 0x7f3bdc13f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 -- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 msgr2=0x7f3bdc1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc1427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bda800640 1 -- 172.21.15.73:0/3407907255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc80094d0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.421+0000 7f3bd9400640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc1427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3bda800640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c780 tx=0x7f3bd400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400ce90 con 0x7f3bdc15abe0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd400a640 con 0x7f3bdc15abe0 2024-04-05T22:25:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400a9d0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc142d20 con 0x7f3bdc15abe0 2024-04-05T22:25:39.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.422+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc1554a0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.423+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc155f80 con 0x7f3bdc15abe0 2024-04-05T22:25:39.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.424+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f3bd400ab30 con 0x7f3bdc15abe0 2024-04-05T22:25:39.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.425+0000 7f3bd2000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 0x7f3bbc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:39.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.425+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bd4088950 con 0x7f3bdc15abe0 2024-04-05T22:25:39.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.425+0000 7f3bd9e00640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 0x7f3bbc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:39.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.426+0000 7f3bd9e00640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 0x7f3bbc06d270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc006fd0 tx=0x7f3bcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:39.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.429+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3bd4053940 con 0x7f3bdc15abe0 2024-04-05T22:25:39.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.622+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3bdc06bec0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.623+0000 7f3bd2000640 1 -- 172.21.15.73:0/3407907255 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3bdc06bec0 con 0x7f3bdc15abe0 2024-04-05T22:25:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 msgr2=0x7f3bbc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 0x7f3bbc06d270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc006fd0 tx=0x7f3bcc008040 comp rx=0 tx=0).stop 2024-04-05T22:25:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 msgr2=0x7f3bdc1f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c780 tx=0x7f3bd400cc40 comp rx=0 tx=0).stop 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 shutdown_connections 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3bbc06adb0 0x7f3bbc06d270 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bdc15abe0 0x7f3bdc1f5b40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bdc06c9a0 0x7f3bdc1427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 --2- 172.21.15.73:0/3407907255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bdc06ba70 0x7f3bdc13f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 >> 172.21.15.73:0/3407907255 conn(0x7f3bdc069cb0 msgr2=0x7f3bdc0af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 shutdown_connections 2024-04-05T22:25:39.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:39.626+0000 7f3be1000640 1 -- 172.21.15.73:0/3407907255 wait complete. 2024-04-05T22:25:39.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:39.645 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol43130) 2024-04-05T22:25:39.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:39.740 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:25:39.794 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:25:39.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:39.795 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol43130) 2024-04-05T22:25:39.895 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol20041 2024-04-05T22:25:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.100+0000 7fcc3f200640 1 Processor -- start 2024-04-05T22:25:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.101+0000 7fcc3f200640 1 -- start start 2024-04-05T22:25:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3815abe0 0x7fcc380712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcc3806bc40 0x7fcc38164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3815afc0 con 0x7fcc3806c9a0 2024-04-05T22:25:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3806f5b0 con 0x7fcc3806bc40 2024-04-05T22:25:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3806f890 con 0x7fcc3815abe0 2024-04-05T22:25:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcc3806bc40 0x7fcc38164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcc3806bc40 0x7fcc38164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36628/0 (socket says 172.21.15.73:36628) 2024-04-05T22:25:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.102+0000 7fcc3d200640 1 -- 172.21.15.73:0/3359869901 learned_addr learned my addr 172.21.15.73:0/3359869901 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.103+0000 7fcc37e00640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3815abe0 0x7fcc380712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.103+0000 7fcc3d200640 1 -- 172.21.15.73:0/3359869901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3815abe0 msgr2=0x7fcc380712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.103+0000 7fcc3d200640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3815abe0 0x7fcc380712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.103+0000 7fcc3d200640 1 -- 172.21.15.73:0/3359869901 >> v1:172.21.15.157:6789/0 conn(0x7fcc3806c9a0 legacy=0x7fcc3806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.103+0000 7fcc3d200640 1 -- 172.21.15.73:0/3359869901 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3806fb70 con 0x7fcc3806bc40 2024-04-05T22:25:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.104+0000 7fcc3d200640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3806bc40 0x7fcc38164740 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009e50 tx=0x7fcc3002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce71a939da17d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.104+0000 7fcc36a00640 1 -- 172.21.15.73:0/3359869901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3002c950 con 0x7fcc3806bc40 2024-04-05T22:25:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.104+0000 7fcc36a00640 1 -- 172.21.15.73:0/3359869901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3002cab0 con 0x7fcc3806bc40 2024-04-05T22:25:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.104+0000 7fcc36a00640 1 -- 172.21.15.73:0/3359869901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3002ce40 con 0x7fcc3806bc40 2024-04-05T22:25:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 -- 172.21.15.73:0/3359869901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3806bc40 msgr2=0x7fcc38164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3806bc40 0x7fcc38164740 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009e50 tx=0x7fcc3002bdb0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 -- 172.21.15.73:0/3359869901 shutdown_connections 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3806bc40 0x7fcc38164740 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 --2- 172.21.15.73:0/3359869901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3815abe0 0x7fcc380712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 -- 172.21.15.73:0/3359869901 >> 172.21.15.73:0/3359869901 conn(0x7fcc38069cb0 msgr2=0x7fcc3806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 -- 172.21.15.73:0/3359869901 shutdown_connections 2024-04-05T22:25:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.105+0000 7fcc3f200640 1 -- 172.21.15.73:0/3359869901 wait complete. 2024-04-05T22:25:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.107+0000 7fcc3f200640 1 Processor -- start 2024-04-05T22:25:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.107+0000 7fcc3f200640 1 -- start start 2024-04-05T22:25:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806bc40 0x7fcc380ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3806c9a0 0x7fcc38144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc381459b0 con 0x7fcc3815abe0 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380add00 con 0x7fcc3806c9a0 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380ae220 con 0x7fcc3806bc40 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36630/0 (socket says 172.21.15.73:36630) 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.108+0000 7fcc3d200640 1 -- 172.21.15.73:0/2479342944 learned_addr learned my addr 172.21.15.73:0/2479342944 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc37e00640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806bc40 0x7fcc380ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc37400640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3806c9a0 0x7fcc38144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 -- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806bc40 msgr2=0x7fcc380ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806bc40 0x7fcc380ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 -- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3806c9a0 msgr2=0x7fcc38144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3806c9a0 0x7fcc38144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 -- 172.21.15.73:0/2479342944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc30009b30 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3d200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc30036040 tx=0x7fcc30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30009510 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3003e450 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3003e7e0 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.109+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc381f2200 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.110+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3815b300 con 0x7fcc3815abe0 2024-04-05T22:25:40.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.111+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc38068160 con 0x7fcc3815abe0 2024-04-05T22:25:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.112+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fcc3003e940 con 0x7fcc3815abe0 2024-04-05T22:25:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.113+0000 7fcc2fe00640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 0x7fcc1806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.113+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc300ac2d0 con 0x7fcc3815abe0 2024-04-05T22:25:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.113+0000 7fcc37e00640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 0x7fcc1806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.114+0000 7fcc37e00640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 0x7fcc1806d270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc28006fd0 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:40.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.118+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcc30077220 con 0x7fcc3815abe0 2024-04-05T22:25:40.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.339+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol20041", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc38071820 con 0x7fcc1806adb0 2024-04-05T22:25:40.346 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:40.347+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:40.346 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:40.347+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:40.347 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:40.347+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:40.347 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:40.347+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:40.347 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:40.347+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:40.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.409+0000 7fcc2fe00640 1 -- 172.21.15.73:0/2479342944 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc2800b600 con 0x7fcc1806adb0 2024-04-05T22:25:40.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 msgr2=0x7fcc1806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 0x7fcc1806d270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc28006fd0 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 msgr2=0x7fcc380b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc30036040 tx=0x7fcc30004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 shutdown_connections 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcc1806adb0 0x7fcc1806d270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc3815abe0 0x7fcc380b2470 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3806c9a0 0x7fcc38144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 --2- 172.21.15.73:0/2479342944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806bc40 0x7fcc380ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 >> 172.21.15.73:0/2479342944 conn(0x7fcc38069cb0 msgr2=0x7fcc38070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 shutdown_connections 2024-04-05T22:25:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:40.412+0000 7fcc3f200640 1 -- 172.21.15.73:0/2479342944 wait complete. 2024-04-05T22:25:40.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:40.432 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol20041/.meta ./volumes/_nogroup/subvol43130) 2024-04-05T22:25:43.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol43130 2024-04-05T22:25:43.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.631+0000 7f38f9a00640 1 Processor -- start 2024-04-05T22:25:43.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.632+0000 7f38f9a00640 1 -- start start 2024-04-05T22:25:43.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.633+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.633+0000 7f38f9a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38f406ba70 0x7f38f4164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.633+0000 7f38f9a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f406bec0 con 0x7f38f415abe0 2024-04-05T22:25:43.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.633+0000 7f38f9a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f415afc0 con 0x7f38f406ba70 2024-04-05T22:25:43.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.633+0000 7f38f9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f406f5b0 con 0x7f38f406c9a0 2024-04-05T22:25:43.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.634+0000 7f38f3400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38f406ba70 0x7f38f4164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.634+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.634+0000 7f38f3400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38f406ba70 0x7f38f4164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:34674/0 (socket says 172.21.15.73:34674) 2024-04-05T22:25:43.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.634+0000 7f38f3400640 1 -- 172.21.15.73:0/3782402654 learned_addr learned my addr 172.21.15.73:0/3782402654 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:43.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.634+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:59810/0 (socket says 172.21.15.73:59810) 2024-04-05T22:25:43.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.635+0000 7f38f3400640 1 -- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 msgr2=0x7f38f406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:43.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.635+0000 7f38f3400640 1 --2- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.635+0000 7f38f3400640 1 -- 172.21.15.73:0/3782402654 >> v1:172.21.15.157:6789/0 conn(0x7f38f415abe0 legacy=0x7f38f40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.635+0000 7f38f3400640 1 -- 172.21.15.73:0/3782402654 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f406f890 con 0x7f38f406ba70 2024-04-05T22:25:43.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.636+0000 7f38f3400640 1 --2- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f4164740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009e50 tx=0x7f38ec02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32db8c5d0c7d248 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:43.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f2000640 1 -- 172.21.15.73:0/3782402654 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec02c9f0 con 0x7f38f406ba70 2024-04-05T22:25:43.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f2000640 1 -- 172.21.15.73:0/3782402654 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec02cb50 con 0x7f38f406ba70 2024-04-05T22:25:43.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f2000640 1 -- 172.21.15.73:0/3782402654 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0343f0 con 0x7f38f406ba70 2024-04-05T22:25:43.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 -- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 msgr2=0x7f38f4164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 --2- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f4164740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009e50 tx=0x7f38ec02be50 comp rx=0 tx=0).stop 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 -- 172.21.15.73:0/3782402654 shutdown_connections 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 --2- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f4164740 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 --2- 172.21.15.73:0/3782402654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.637+0000 7f38f9a00640 1 -- 172.21.15.73:0/3782402654 >> 172.21.15.73:0/3782402654 conn(0x7f38f4069cb0 msgr2=0x7f38f406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.638+0000 7f38f9a00640 1 -- 172.21.15.73:0/3782402654 shutdown_connections 2024-04-05T22:25:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.638+0000 7f38f9a00640 1 -- 172.21.15.73:0/3782402654 wait complete. 2024-04-05T22:25:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.639+0000 7f38f9a00640 1 Processor -- start 2024-04-05T22:25:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.639+0000 7f38f9a00640 1 -- start start 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f40ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f41f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f414a930 con 0x7f38f415abe0 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40ad2e0 con 0x7f38f406ba70 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.640+0000 7f38f9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4148b40 con 0x7f38f406c9a0 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36660/0 (socket says 172.21.15.73:36660) 2024-04-05T22:25:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 -- 172.21.15.73:0/1633222292 learned_addr learned my addr 172.21.15.73:0/1633222292 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:43.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f2a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f40ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38ebe00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f41f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 -- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 msgr2=0x7f38f41f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f41f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 -- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 msgr2=0x7f38f40ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f40ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.641+0000 7f38f3400640 1 -- 172.21.15.73:0/1633222292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec009b30 con 0x7f38f415abe0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38f3400640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009e50 tx=0x7f38ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec03e450 con 0x7f38f415abe0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec03e5f0 con 0x7f38f415abe0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec03e9a0 con 0x7f38f415abe0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4148e20 con 0x7f38f415abe0 2024-04-05T22:25:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.642+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f40adf90 con 0x7f38f415abe0 2024-04-05T22:25:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.643+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f4149010 con 0x7f38f415abe0 2024-04-05T22:25:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.644+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f38ec03eb00 con 0x7f38f415abe0 2024-04-05T22:25:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.645+0000 7f38eaa00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 0x7f38d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.645+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38ec0ac5f0 con 0x7f38f415abe0 2024-04-05T22:25:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.645+0000 7f38f2a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 0x7f38d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:43.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.646+0000 7f38f2a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 0x7f38d006d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38f414d720 tx=0x7f38e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.650+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f38ec077480 con 0x7f38f415abe0 2024-04-05T22:25:43.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.871+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol43130", "target": ["mon-mgr", ""]}) v1 -- 0x7f38f4070130 con 0x7f38d006ae40 2024-04-05T22:25:43.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.931+0000 7f38eaa00640 1 -- 172.21.15.73:0/1633222292 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f38e4001240 con 0x7f38d006ae40 2024-04-05T22:25:43.931 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol43130 2024-04-05T22:25:43.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 msgr2=0x7f38d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 0x7f38d006d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38f414d720 tx=0x7f38e4009040 comp rx=0 tx=0).stop 2024-04-05T22:25:43.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 msgr2=0x7f38f41f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009e50 tx=0x7f38ec004a40 comp rx=0 tx=0).stop 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 shutdown_connections 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f38d006ae40 0x7f38d006d300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38f415abe0 0x7f38f41f6790 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38f406c9a0 0x7f38f41f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 --2- 172.21.15.73:0/1633222292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f406ba70 0x7f38f40ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 >> 172.21.15.73:0/1633222292 conn(0x7f38f4069cb0 msgr2=0x7f38f40cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 shutdown_connections 2024-04-05T22:25:43.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:43.934+0000 7f38f9a00640 1 -- 172.21.15.73:0/1633222292 wait complete. 2024-04-05T22:25:43.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol43130 2024-04-05T22:25:44.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.163+0000 7f4283200640 1 Processor -- start 2024-04-05T22:25:44.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.163+0000 7f4283200640 1 -- start start 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.164+0000 7f4283200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 0x7f428415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.164+0000 7f4283200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428406bc50 con 0x7f428415e580 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.164+0000 7f4283200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428415ef70 con 0x7f4284070640 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4283200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428415f530 con 0x7f428415a850 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4281400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f428415e580 0x7f428406ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60168/0 (socket says 172.21.15.73:60168) 2024-04-05T22:25:44.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4281400640 1 -- 172.21.15.73:0/1502396528 learned_addr learned my addr 172.21.15.73:0/1502396528 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:44.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327774959 0 0) 0x7f428406bc50 con 0x7f428415e580 2024-04-05T22:25:44.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4281e00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 0x7f428415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.165+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4268003520 con 0x7f428415e580 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332661087 0 0) 0x7f4268003520 con 0x7f428415e580 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 msgr2=0x7f428415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 0x7f428415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 >> v1:172.21.15.157:6789/0 conn(0x7f4284070640 legacy=0x7f428406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428415f250 con 0x7f428415e580 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f426c001670 con 0x7f428415e580 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 >> v1:172.21.15.73:6789/0 conn(0x7f428415e580 legacy=0x7f428406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42680039a0 0x7f4268003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f4268004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 0x7f42680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428415f530 con 0x7f42680047c0 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426800bab0 con 0x7f42680039a0 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426800bc70 con 0x7f428415a850 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4281400640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f4268004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.166+0000 7f4281e00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42680039a0 0x7f4268003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281400640 1 -- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 msgr2=0x7f42680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281400640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f428415a850 0x7f42680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281400640 1 -- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42680039a0 msgr2=0x7f4268003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281400640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42680039a0 0x7f4268003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281400640 1 -- 172.21.15.73:0/1502396528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268003520 con 0x7f42680047c0 2024-04-05T22:25:44.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.167+0000 7f4281e00640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42680039a0 0x7f4268003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4281400640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f4268004c40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c240 tx=0x7f42680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282021018a2dfb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c005ad0 con 0x7f42680047c0 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c005c30 con 0x7f42680047c0 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4280a00640 1 -- 172.21.15.73:0/1502396528 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c02c8b0 con 0x7f42680047c0 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4283200640 1 -- 172.21.15.73:0/1502396528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 msgr2=0x7f4268004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4283200640 1 --2- 172.21.15.73:0/1502396528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f4268004c40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c240 tx=0x7f42680057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.168+0000 7f4282800640 1 -- 172.21.15.73:0/1502396528 reap_dead start 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.169+0000 7f4283200640 1 -- 172.21.15.73:0/1502396528 shutdown_connections 2024-04-05T22:25:44.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.169+0000 7f4283200640 1 -- 172.21.15.73:0/1502396528 >> 172.21.15.73:0/1502396528 conn(0x7f4284069cb0 msgr2=0x7f428406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.169+0000 7f4283200640 1 -- 172.21.15.73:0/1502396528 shutdown_connections 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.169+0000 7f4283200640 1 -- 172.21.15.73:0/1502396528 wait complete. 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.170+0000 7f4283200640 1 Processor -- start 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.170+0000 7f4283200640 1 -- start start 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4284151840 0x7f42840ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4284151cc0 0x7f42840ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4284150b50 con 0x7f42680047c0 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428413ac00 con 0x7f4284151cc0 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.171+0000 7f4283200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42841513b0 con 0x7f4284151840 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36672/0 (socket says 172.21.15.73:36672) 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 -- 172.21.15.73:0/2708950594 learned_addr learned my addr 172.21.15.73:0/2708950594 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 -- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4284151840 msgr2=0x7f42840ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4284151840 0x7f42840ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 -- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4284151cc0 msgr2=0x7f42840ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4284151cc0 0x7f42840ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 -- 172.21.15.73:0/2708950594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428413de80 con 0x7f42680047c0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f4281e00640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4274007f70 tx=0x7f427400f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.172+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274003990 con 0x7f42680047c0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.173+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274003af0 con 0x7f42680047c0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.173+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428413dba0 con 0x7f42680047c0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.173+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274003ea0 con 0x7f42680047c0 2024-04-05T22:25:44.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.173+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4284151040 con 0x7f42680047c0 2024-04-05T22:25:44.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.174+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428404e160 con 0x7f42680047c0 2024-04-05T22:25:44.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.175+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f4274004020 con 0x7f42680047c0 2024-04-05T22:25:44.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.176+0000 7f427a000640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.176+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f427408a2b0 con 0x7f42680047c0 2024-04-05T22:25:44.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.176+0000 7f4281400640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.177+0000 7f4281400640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 0x7f426006d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.181+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4274055020 con 0x7f42680047c0 2024-04-05T22:25:44.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.403+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol43130", "target": ["mon-mgr", ""]}) v1 -- 0x7f428406c1d0 con 0x7f426006ae40 2024-04-05T22:25:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.415+0000 7f427a000640 1 -- 172.21.15.73:0/2708950594 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f426c005930 con 0x7f426006ae40 2024-04-05T22:25:44.416 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol43130 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.417+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 msgr2=0x7f426006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.417+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 0x7f426006d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c007000 comp rx=0 tx=0).stop 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.417+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 msgr2=0x7f428414f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.417+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4274007f70 tx=0x7f427400f5e0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 shutdown_connections 2024-04-05T22:25:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4284151cc0 0x7f42840ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4284151840 0x7f42840ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 --2- 172.21.15.73:0/2708950594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42680047c0 0x7f428414f360 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 >> 172.21.15.73:0/2708950594 conn(0x7f4284069cb0 msgr2=0x7f42840b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 shutdown_connections 2024-04-05T22:25:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.418+0000 7f4283200640 1 -- 172.21.15.73:0/2708950594 wait complete. 2024-04-05T22:25:44.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:44.437 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/80b2387a1ec01b3bd7a84cd645aca02f.meta) 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol43130 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:25:44.532 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:44.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol43130 alice 2024-04-05T22:25:44.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.745+0000 7f54e5600640 1 Processor -- start 2024-04-05T22:25:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.745+0000 7f54e5600640 1 -- start start 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54e5600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54e015aa80 0x7f54e006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54e5600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54e00714b0 0x7f54e006b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54e5600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e00ccc50 con 0x7f54e006b9d0 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54e5600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e00cd5f0 con 0x7f54e015aa80 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54e5600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e00ccf30 con 0x7f54e00714b0 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54e015aa80 0x7f54e006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54e015aa80 0x7f54e006fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36682/0 (socket says 172.21.15.73:36682) 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.747+0000 7f54dfe00640 1 -- 172.21.15.73:0/4200207151 learned_addr learned my addr 172.21.15.73:0/4200207151 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.748+0000 7f54d7e00640 1 --2- 172.21.15.73:0/4200207151 >> v2:172.21.15.157:3300/0 conn(0x7f54e00714b0 0x7f54e006b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.748+0000 7f54dfe00640 1 -- 172.21.15.73:0/4200207151 >> v2:172.21.15.157:3300/0 conn(0x7f54e00714b0 msgr2=0x7f54e006b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.748+0000 7f54dfe00640 1 --2- 172.21.15.73:0/4200207151 >> v2:172.21.15.157:3300/0 conn(0x7f54e00714b0 0x7f54e006b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.748+0000 7f54dfe00640 1 -- 172.21.15.73:0/4200207151 >> v1:172.21.15.157:6789/0 conn(0x7f54e006b9d0 legacy=0x7f54e00cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.748+0000 7f54dfe00640 1 -- 172.21.15.73:0/4200207151 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e00cd210 con 0x7f54e015aa80 2024-04-05T22:25:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.749+0000 7f54dfe00640 1 --2- 172.21.15.73:0/4200207151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e006fea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54e0067280 tx=0x7f54d002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97496ce44e20e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.749+0000 7f54df400640 1 -- 172.21.15.73:0/4200207151 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0031750 con 0x7f54e015aa80 2024-04-05T22:25:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.749+0000 7f54df400640 1 -- 172.21.15.73:0/4200207151 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d00318b0 con 0x7f54e015aa80 2024-04-05T22:25:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.749+0000 7f54df400640 1 -- 172.21.15.73:0/4200207151 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0031c40 con 0x7f54e015aa80 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 -- 172.21.15.73:0/4200207151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 msgr2=0x7f54e006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 --2- 172.21.15.73:0/4200207151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e006fea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54e0067280 tx=0x7f54d002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 -- 172.21.15.73:0/4200207151 shutdown_connections 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 --2- 172.21.15.73:0/4200207151 >> v2:172.21.15.157:3300/0 conn(0x7f54e00714b0 0x7f54e006b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 --2- 172.21.15.73:0/4200207151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e006fea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 -- 172.21.15.73:0/4200207151 >> 172.21.15.73:0/4200207151 conn(0x7f54e00b0360 msgr2=0x7f54e00b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:44.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 -- 172.21.15.73:0/4200207151 shutdown_connections 2024-04-05T22:25:44.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.750+0000 7f54e5600640 1 -- 172.21.15.73:0/4200207151 wait complete. 2024-04-05T22:25:44.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.752+0000 7f54e5600640 1 Processor -- start 2024-04-05T22:25:44.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.752+0000 7f54e5600640 1 -- start start 2024-04-05T22:25:44.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e006b9d0 0x7f54e01517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 0x7f54e00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e013ffd0 con 0x7f54e015aa80 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0151320 con 0x7f54e00714b0 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0150a30 con 0x7f54e006b9d0 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36684/0 (socket says 172.21.15.73:36684) 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e4c00640 1 -- 172.21.15.73:0/1181059557 learned_addr learned my addr 172.21.15.73:0/1181059557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54d7e00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 0x7f54e00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54e4c00640 1 -- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e006b9d0 msgr2=0x7f54e01517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.753+0000 7f54dfe00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e006b9d0 0x7f54e01517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e4c00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e006b9d0 0x7f54e01517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e4c00640 1 -- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 msgr2=0x7f54e00ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e4c00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 0x7f54e00ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e4c00640 1 -- 172.21.15.73:0/1181059557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d00094d0 con 0x7f54e015aa80 2024-04-05T22:25:44.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54d7e00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 0x7f54e00ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e4c00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b9c0 tx=0x7f54d800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cba0 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d800cd00 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8014680 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e013fcf0 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.754+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e0150fb0 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.755+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e004e160 con 0x7f54e015aa80 2024-04-05T22:25:44.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.756+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f54d8007370 con 0x7f54e015aa80 2024-04-05T22:25:44.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.758+0000 7f54dcc00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 0x7f54bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:44.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.758+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54d8089780 con 0x7f54e015aa80 2024-04-05T22:25:44.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.758+0000 7f54dfe00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 0x7f54bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:44.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.759+0000 7f54dfe00640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 0x7f54bc06d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f54d0035000 tx=0x7f54d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:44.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.762+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f54d8054560 con 0x7f54e015aa80 2024-04-05T22:25:44.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:44.984+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol43130", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e00cc870 con 0x7f54bc06ae40 2024-04-05T22:25:44.998 INFO:tasks.ceph.mgr.y.smithi073.stderr:Creating meta for ID alice with tenant None 2024-04-05T22:25:45.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.069+0000 7f54dcc00640 1 -- 172.21.15.73:0/1181059557 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f54d00430d0 con 0x7f54bc06ae40 2024-04-05T22:25:45.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.072+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 msgr2=0x7f54bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.072+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 0x7f54bc06d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f54d0035000 tx=0x7f54d0008030 comp rx=0 tx=0).stop 2024-04-05T22:25:45.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.072+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 msgr2=0x7f54e00ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.072+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b9c0 tx=0x7f54d800be80 comp rx=0 tx=0).stop 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 shutdown_connections 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f54bc06ae40 0x7f54bc06d300 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54e015aa80 0x7f54e00ad990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e00714b0 0x7f54e00ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 --2- 172.21.15.73:0/1181059557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e006b9d0 0x7f54e01517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 >> 172.21.15.73:0/1181059557 conn(0x7f54e00b0360 msgr2=0x7f54e00a84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 shutdown_connections 2024-04-05T22:25:45.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.073+0000 7f54e5600640 1 -- 172.21.15.73:0/1181059557 wait complete. 2024-04-05T22:25:45.094 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-04-05T22:25:45.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.303+0000 7f553ea00640 1 Processor -- start 2024-04-05T22:25:45.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.303+0000 7f553ea00640 1 -- start start 2024-04-05T22:25:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.304+0000 7f553ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 0x7f55380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.304+0000 7f553ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f553ea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553806f5b0 con 0x7f553806d0c0 2024-04-05T22:25:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f553ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5538070410 con 0x7f55380ccd20 2024-04-05T22:25:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f553ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5538158bb0 con 0x7f55381587d0 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f552f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f552f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:34718/0 (socket says 172.21.15.73:34718) 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f552f400640 1 -- 172.21.15.73:0/3505676517 learned_addr learned my addr 172.21.15.73:0/3505676517 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.305+0000 7f5537e00640 1 --2- 172.21.15.73:0/3505676517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 0x7f55380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.306+0000 7f5537e00640 1 -- 172.21.15.73:0/3505676517 >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 msgr2=0x7f553806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.306+0000 7f5537e00640 1 --2- 172.21.15.73:0/3505676517 >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.306+0000 7f5537e00640 1 -- 172.21.15.73:0/3505676517 >> v1:172.21.15.157:6789/0 conn(0x7f553806d0c0 legacy=0x7f553815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.306+0000 7f5537e00640 1 -- 172.21.15.73:0/3505676517 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553806f800 con 0x7f55381587d0 2024-04-05T22:25:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.306+0000 7f552f400640 1 --2- 172.21.15.73:0/3505676517 >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:25:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.308+0000 7f5537e00640 1 --2- 172.21.15.73:0/3505676517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 0x7f55380cc7e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55280097a0 tx=0x7f552802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e9a3f14c6eb703 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.308+0000 7f5537400640 1 -- 172.21.15.73:0/3505676517 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552802d040 con 0x7f55381587d0 2024-04-05T22:25:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.308+0000 7f5537400640 1 -- 172.21.15.73:0/3505676517 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5528031810 con 0x7f55381587d0 2024-04-05T22:25:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.308+0000 7f5537400640 1 -- 172.21.15.73:0/3505676517 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528031ba0 con 0x7f55381587d0 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 -- 172.21.15.73:0/3505676517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 msgr2=0x7f55380cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 --2- 172.21.15.73:0/3505676517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 0x7f55380cc7e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55280097a0 tx=0x7f552802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 -- 172.21.15.73:0/3505676517 shutdown_connections 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 --2- 172.21.15.73:0/3505676517 >> v2:172.21.15.157:3300/0 conn(0x7f55380ccd20 0x7f553806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 --2- 172.21.15.73:0/3505676517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55381587d0 0x7f55380cc7e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 -- 172.21.15.73:0/3505676517 >> 172.21.15.73:0/3505676517 conn(0x7f55380ce8e0 msgr2=0x7f55380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 -- 172.21.15.73:0/3505676517 shutdown_connections 2024-04-05T22:25:45.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.309+0000 7f553ea00640 1 -- 172.21.15.73:0/3505676517 wait complete. 2024-04-05T22:25:45.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.311+0000 7f553ea00640 1 Processor -- start 2024-04-05T22:25:45.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.311+0000 7f553ea00640 1 -- start start 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 0x7f5538143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f553806a1f0 0x7f55381f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381f1740 con 0x7f55381587d0 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553806a6a0 con 0x7f553806a1f0 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55381470d0 con 0x7f553806d0c0 2024-04-05T22:25:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36700/0 (socket says 172.21.15.73:36700) 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 -- 172.21.15.73:0/154797092 learned_addr learned my addr 172.21.15.73:0/154797092 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f5537e00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 0x7f5538143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f553ca00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f553806a1f0 0x7f55381f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 -- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 msgr2=0x7f5538143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 0x7f5538143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 -- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f553806a1f0 msgr2=0x7f55381f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.312+0000 7f552f400640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f553806a1f0 0x7f55381f09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f552f400640 1 -- 172.21.15.73:0/154797092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200094d0 con 0x7f55381587d0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f5537e00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 0x7f5538143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f552f400640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f552000ea30 tx=0x7f552000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552000cac0 con 0x7f55381587d0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552000cc20 con 0x7f55381587d0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520013470 con 0x7f55381587d0 2024-04-05T22:25:45.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5528009400 con 0x7f55381587d0 2024-04-05T22:25:45.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.313+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55381465f0 con 0x7f55381587d0 2024-04-05T22:25:45.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.314+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538066af0 con 0x7f55381587d0 2024-04-05T22:25:45.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.315+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f55200135d0 con 0x7f55381587d0 2024-04-05T22:25:45.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.317+0000 7f5534c00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 0x7f551406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.317+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5520089b60 con 0x7f55381587d0 2024-04-05T22:25:45.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.317+0000 7f5537e00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 0x7f551406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.318+0000 7f5537e00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 0x7f551406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5528036000 tx=0x7f552803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.321+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5520054940 con 0x7f55381587d0 2024-04-05T22:25:45.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.584+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7f5538068160 con 0x7f55381587d0 2024-04-05T22:25:45.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.584+0000 7f5534c00640 1 -- 172.21.15.73:0/154797092 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v16) v1 ==== 107+0+280 (secure 0 0 0) 0x7f5520021130 con 0x7f55381587d0 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.alice", 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBpehBmmHh6ARAAy1xTWlOJ4b0Dr8dStWw8bQ==", 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol43130", 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:45.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:25:45.586 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 msgr2=0x7f551406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 0x7f551406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5528036000 tx=0x7f552803c040 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 msgr2=0x7f5538069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f552000ea30 tx=0x7f552000eef0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 shutdown_connections 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f551406ae40 0x7f551406d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f553806a1f0 0x7f55381f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55381587d0 0x7f5538069cb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 --2- 172.21.15.73:0/154797092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f553806d0c0 0x7f5538143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 >> 172.21.15.73:0/154797092 conn(0x7f55380ce8e0 msgr2=0x7f55380ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 shutdown_connections 2024-04-05T22:25:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.587+0000 7f553ea00640 1 -- 172.21.15.73:0/154797092 wait complete. 2024-04-05T22:25:45.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol43130 2024-04-05T22:25:45.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.814+0000 7f4ad4c00640 1 Processor -- start 2024-04-05T22:25:45.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.814+0000 7f4ad4c00640 1 -- start start 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 0x7f4ad006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4ad4c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ad006ba70 0x7f4ad0164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4ad4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad006bec0 con 0x7f4ad015abe0 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4ad4c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad015afc0 con 0x7f4ad006ba70 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad006f5b0 con 0x7f4ad006c9a0 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4acde00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4ad015abe0 0x7f4ad00712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60174/0 (socket says 172.21.15.73:60174) 2024-04-05T22:25:45.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.816+0000 7f4acde00640 1 -- 172.21.15.73:0/1156222003 learned_addr learned my addr 172.21.15.73:0/1156222003 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:45.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294576187 0 0) 0x7f4ac0001490 con 0x7f4ad015abe0 2024-04-05T22:25:45.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4ac5400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 0x7f4ad006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab4003520 con 0x7f4ad015abe0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4ace800640 1 --2- 172.21.15.73:0/1156222003 >> v2:172.21.15.157:3300/0 conn(0x7f4ad006ba70 0x7f4ad0164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 724265322 0 0) 0x7f4ab4003520 con 0x7f4ad015abe0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 msgr2=0x7f4ad006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 0x7f4ad006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 >> v2:172.21.15.157:3300/0 conn(0x7f4ad006ba70 msgr2=0x7f4ad0164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 --2- 172.21.15.73:0/1156222003 >> v2:172.21.15.157:3300/0 conn(0x7f4ad006ba70 0x7f4ad0164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad006f890 con 0x7f4ad015abe0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4ac0001b80 con 0x7f4ad015abe0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 >> v1:172.21.15.73:6789/0 conn(0x7f4ad015abe0 legacy=0x7f4ad00712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab4003970 0x7f4ab4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab40047c0 0x7f4ab4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 0x7f4ab4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad015afc0 con 0x7f4ab40047c0 2024-04-05T22:25:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad006f5b0 con 0x7f4ab4003970 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.817+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400bb90 con 0x7f4ad006c9a0 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab40047c0 0x7f4ab4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4acde00640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab4003970 0x7f4ab4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 -- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 msgr2=0x7f4ab4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad006c9a0 0x7f4ab4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 -- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab4003970 msgr2=0x7f4ab4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab4003970 0x7f4ab4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.818+0000 7f4ac5400640 1 -- 172.21.15.73:0/1156222003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4003520 con 0x7f4ab40047c0 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.819+0000 7f4ac5400640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab40047c0 0x7f4ab4004c40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800ba30 tx=0x7f4ab40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702ea45d4b9916d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.819+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8032680 con 0x7f4ab40047c0 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.819+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab80327e0 con 0x7f4ab40047c0 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.819+0000 7f4acd400640 1 -- 172.21.15.73:0/1156222003 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8032b70 con 0x7f4ab40047c0 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 -- 172.21.15.73:0/1156222003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab40047c0 msgr2=0x7f4ab4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/1156222003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab40047c0 0x7f4ab4004c40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800ba30 tx=0x7f4ab40057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ace800640 1 -- 172.21.15.73:0/1156222003 reap_dead start 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 -- 172.21.15.73:0/1156222003 shutdown_connections 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 -- 172.21.15.73:0/1156222003 >> 172.21.15.73:0/1156222003 conn(0x7f4ad0069cb0 msgr2=0x7f4ad006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 -- 172.21.15.73:0/1156222003 shutdown_connections 2024-04-05T22:25:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.820+0000 7f4ad4c00640 1 -- 172.21.15.73:0/1156222003 wait complete. 2024-04-05T22:25:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.822+0000 7f4ad4c00640 1 Processor -- start 2024-04-05T22:25:45.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.822+0000 7f4ad4c00640 1 -- start start 2024-04-05T22:25:45.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab40047c0 0x7f4ad015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 0x7f4ad01418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0144e80 con 0x7f4ad01f1a10 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0141e20 con 0x7f4ad0144a00 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad013faf0 con 0x7f4ab40047c0 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36714/0 (socket says 172.21.15.73:36714) 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 -- 172.21.15.73:0/2755076537 learned_addr learned my addr 172.21.15.73:0/2755076537 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ac5400640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 0x7f4ad01418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 -- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab40047c0 msgr2=0x7f4ad015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4acde00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab40047c0 0x7f4ad015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab40047c0 0x7f4ad015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 -- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 msgr2=0x7f4ad01418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 0x7f4ad01418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ace800640 1 -- 172.21.15.73:0/2755076537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad013eee0 con 0x7f4ad01f1a10 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.823+0000 7f4ac5400640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 0x7f4ad01418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ace800640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006d710 tx=0x7f4ac800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4ad01f1a10 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac8004020 con 0x7f4ad01f1a10 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80041b0 con 0x7f4ad01f1a10 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad013f1c0 con 0x7f4ad01f1a10 2024-04-05T22:25:45.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.824+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad013f780 con 0x7f4ad01f1a10 2024-04-05T22:25:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.825+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad013fc60 con 0x7f4ad01f1a10 2024-04-05T22:25:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.826+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f4ac8004310 con 0x7f4ad01f1a10 2024-04-05T22:25:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.827+0000 7f4ac6a00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 0x7f4aac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.827+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ac8088fe0 con 0x7f4ad01f1a10 2024-04-05T22:25:45.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.827+0000 7f4acde00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 0x7f4aac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:45.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.828+0000 7f4acde00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 0x7f4aac06d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007c40 tx=0x7f4ac0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:45.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:45.831+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4ac8053e70 con 0x7f4ad01f1a10 2024-04-05T22:25:46.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.052+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol43130", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad0070130 con 0x7f4aac06ae70 2024-04-05T22:25:46.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.091+0000 7f4ac6a00640 1 -- 172.21.15.73:0/2755076537 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4ac000d040 con 0x7f4aac06ae70 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 msgr2=0x7f4aac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 0x7f4aac06d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007c40 tx=0x7f4ac0009040 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 msgr2=0x7f4ad0150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006d710 tx=0x7f4ac800d820 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 shutdown_connections 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f4aac06ae70 0x7f4aac06d330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ad01f1a10 0x7f4ad0150320 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad0144a00 0x7f4ad01418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 --2- 172.21.15.73:0/2755076537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab40047c0 0x7f4ad015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.095+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 >> 172.21.15.73:0/2755076537 conn(0x7f4ad0069cb0 msgr2=0x7f4ad01536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:46.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.096+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 shutdown_connections 2024-04-05T22:25:46.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.096+0000 7f4ad4c00640 1 -- 172.21.15.73:0/2755076537 wait complete. 2024-04-05T22:25:46.097 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.097+0000 7f5500c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.097 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.097+0000 7f5500c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.097 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.097+0000 7f5500c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.098 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.097+0000 7f5500c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.098 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.097+0000 7f5500c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol20041 2024-04-05T22:25:46.123 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.123+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.123 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.123+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.123 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.123+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.123 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.123+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.123 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:25:46.123+0000 7f5501600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:25:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.293+0000 7f0841c00640 1 Processor -- start 2024-04-05T22:25:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.293+0000 7f0841c00640 1 -- start start 2024-04-05T22:25:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f0841c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f083c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f0841c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083c1612c0 con 0x7f083c06a090 2024-04-05T22:25:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f0841c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083c1615a0 con 0x7f083c06be40 2024-04-05T22:25:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f0841c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1610d0 con 0x7f083c069cb0 2024-04-05T22:25:46.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f083a000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f083c06a090 0x7f083c15df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51834/0 (socket says 172.21.15.73:51834) 2024-04-05T22:25:46.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.295+0000 7f083a000640 1 -- 172.21.15.73:0/2278602924 learned_addr learned my addr 172.21.15.73:0/2278602924 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f083aa00640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f083c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104635255 0 0) 0x7f083c1612c0 con 0x7f083c06a090 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081c003520 con 0x7f083c06a090 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144514242 0 0) 0x7f081c003520 con 0x7f083c06a090 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 msgr2=0x7f083c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f083c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 >> v1:172.21.15.157:6789/0 conn(0x7f083c06be40 legacy=0x7f083c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c161880 con 0x7f083c06a090 2024-04-05T22:25:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f08240019f0 con 0x7f083c06a090 2024-04-05T22:25:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 >> v1:172.21.15.73:6789/0 conn(0x7f083c06a090 legacy=0x7f083c15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c0039a0 0x7f081c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f081c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f081c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c00ba80 con 0x7f081c004d00 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c00bbf0 con 0x7f081c0039a0 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.296+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c00bdb0 con 0x7f083c069cb0 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f081c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083a000640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f081c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083aa00640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c0039a0 0x7f081c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 -- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 msgr2=0x7f081c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f081c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 -- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c0039a0 msgr2=0x7f081c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c0039a0 0x7f081c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083b400640 1 -- 172.21.15.73:0/2278602924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c003520 con 0x7f081c004d00 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083a000640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c069cb0 0x7f081c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.297+0000 7f083aa00640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c0039a0 0x7f081c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.298+0000 7f083b400640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f081c005180 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0834001750 tx=0x7f081c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93494c5d15460c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.298+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340052d0 con 0x7f081c004d00 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.298+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083402e050 con 0x7f081c004d00 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.298+0000 7f0839600640 1 -- 172.21.15.73:0/2278602924 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340035f0 con 0x7f081c004d00 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 -- 172.21.15.73:0/2278602924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 msgr2=0x7f081c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 --2- 172.21.15.73:0/2278602924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f081c005180 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0834001750 tx=0x7f081c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f083b400640 1 -- 172.21.15.73:0/2278602924 reap_dead start 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 -- 172.21.15.73:0/2278602924 shutdown_connections 2024-04-05T22:25:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 -- 172.21.15.73:0/2278602924 >> 172.21.15.73:0/2278602924 conn(0x7f083c0b0ce0 msgr2=0x7f083c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 -- 172.21.15.73:0/2278602924 shutdown_connections 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.299+0000 7f0841c00640 1 -- 172.21.15.73:0/2278602924 wait complete. 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.301+0000 7f0841c00640 1 Processor -- start 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.301+0000 7f0841c00640 1 -- start start 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c06d5b0 0x7f083c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f083c06da30 0x7f083c15f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1461c0 con 0x7f081c004d00 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c071ba0 con 0x7f083c06da30 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f0841c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083c072030 con 0x7f083c06d5b0 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55644/0 (socket says 172.21.15.73:55644) 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 -- 172.21.15.73:0/498014922 learned_addr learned my addr 172.21.15.73:0/498014922 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 -- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c06d5b0 msgr2=0x7f083c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c06d5b0 0x7f083c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 -- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f083c06da30 msgr2=0x7f083c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f083c06da30 0x7f083c15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 -- 172.21.15.73:0/498014922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c070f90 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.302+0000 7f083aa00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f082c0108b0 tx=0x7f082c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.303+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c00ea80 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.303+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082c00ebe0 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.303+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c071270 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.303+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c01c400 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.303+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c071830 con 0x7f081c004d00 2024-04-05T22:25:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.304+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c068160 con 0x7f081c004d00 2024-04-05T22:25:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.305+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f082c01c560 con 0x7f081c004d00 2024-04-05T22:25:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.306+0000 7f0832a00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 0x7f081806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.306+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f082c089d40 con 0x7f081c004d00 2024-04-05T22:25:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.306+0000 7f083a000640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 0x7f081806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.306+0000 7f083a000640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 0x7f081806d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0824007c40 tx=0x7f0824009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:46.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.310+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f082c054c50 con 0x7f081c004d00 2024-04-05T22:25:46.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.532+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol20041", "target": ["mon-mgr", ""]}) v1 -- 0x7f083c161b60 con 0x7f081806ae40 2024-04-05T22:25:46.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.566+0000 7f0832a00640 1 -- 172.21.15.73:0/498014922 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f08240070d0 con 0x7f081806ae40 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.568+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 msgr2=0x7f081806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.568+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 0x7f081806d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0824007c40 tx=0x7f0824009040 comp rx=0 tx=0).stop 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.568+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 msgr2=0x7f083c145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.568+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f082c0108b0 tx=0x7f082c010d70 comp rx=0 tx=0).stop 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 shutdown_connections 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f081806ae40 0x7f081806d300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f083c06da30 0x7f083c15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f083c06d5b0 0x7f083c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 --2- 172.21.15.73:0/498014922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081c004d00 0x7f083c145c80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:46.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 >> 172.21.15.73:0/498014922 conn(0x7f083c0b0ce0 msgr2=0x7f083c0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:46.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 shutdown_connections 2024-04-05T22:25:46.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:46.569+0000 7f0841c00640 1 -- 172.21.15.73:0/498014922 wait complete. 2024-04-05T22:25:46.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:46.592 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:25:46.648 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:25:46.649 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:25:46.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:46.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:25:46.879 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:25:46.879 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:46.879 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:46.937 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:46.937 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:46.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:47.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.202+0000 7f41fd600640 1 Processor -- start 2024-04-05T22:25:47.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.202+0000 7f41fd600640 1 -- start start 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.203+0000 7f41fd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41f8163e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.203+0000 7f41fd600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806ca50 con 0x7f41f813f3b0 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.203+0000 7f41fd600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806cd30 con 0x7f41f80b23e0 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.203+0000 7f41fd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f806f920 con 0x7f41f8070d50 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41f7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f41f813f3b0 0x7f41f80b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51840/0 (socket says 172.21.15.73:51840) 2024-04-05T22:25:47.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41f7e00640 1 -- 172.21.15.73:0/2565586331 learned_addr learned my addr 172.21.15.73:0/2565586331 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944722676 0 0) 0x7f41f806ca50 con 0x7f41f813f3b0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41fcc00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41f8163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41dc003520 con 0x7f41f813f3b0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.204+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905865905 0 0) 0x7f41dc003520 con 0x7f41f813f3b0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 msgr2=0x7f41f8163e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41f8163e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 >> v1:172.21.15.157:6789/0 conn(0x7f41f80b23e0 legacy=0x7f41f806bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f806d010 con 0x7f41f813f3b0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f41e8001670 con 0x7f41f813f3b0 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 >> v1:172.21.15.73:6789/0 conn(0x7f41f813f3b0 legacy=0x7f41f80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41dc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41dc004d00 0x7f41dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f806f920 con 0x7f41dc003fd0 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc00ffd0 con 0x7f41dc004d00 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc010190 con 0x7f41f8070d50 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41fcc00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41dc004d00 0x7f41dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7e00640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41dc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 -- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 msgr2=0x7f41dc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f8070d50 0x7f41dc0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 -- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41dc004d00 msgr2=0x7f41dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41dc004d00 0x7f41dc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.205+0000 7f41f7400640 1 -- 172.21.15.73:0/2565586331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc003520 con 0x7f41dc003fd0 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41f7400640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41dc0047c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f41e0001750 tx=0x7f41dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483b071295afd389 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00052d0 con 0x7f41dc003fd0 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e002e050 con 0x7f41dc003fd0 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41f6a00640 1 -- 172.21.15.73:0/2565586331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003830 con 0x7f41dc003fd0 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41fd600640 1 -- 172.21.15.73:0/2565586331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 msgr2=0x7f41dc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41fd600640 1 --2- 172.21.15.73:0/2565586331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41dc0047c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f41e0001750 tx=0x7f41dc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41fcc00640 1 -- 172.21.15.73:0/2565586331 reap_dead start 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41fd600640 1 -- 172.21.15.73:0/2565586331 shutdown_connections 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.207+0000 7f41fd600640 1 -- 172.21.15.73:0/2565586331 >> 172.21.15.73:0/2565586331 conn(0x7f41f80b06c0 msgr2=0x7f41f80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.208+0000 7f41fd600640 1 -- 172.21.15.73:0/2565586331 shutdown_connections 2024-04-05T22:25:47.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.208+0000 7f41fd600640 1 -- 172.21.15.73:0/2565586331 wait complete. 2024-04-05T22:25:47.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.209+0000 7f41fd600640 1 Processor -- start 2024-04-05T22:25:47.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.209+0000 7f41fd600640 1 -- start start 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 0x7f41f8155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f814e0c0 0x7f41f81562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f81504b0 con 0x7f41dc003fd0 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8150110 con 0x7f41f814dc40 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.210+0000 7f41fd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f814e650 con 0x7f41f814e0c0 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55670/0 (socket says 172.21.15.73:55670) 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 -- 172.21.15.73:0/4268364694 learned_addr learned my addr 172.21.15.73:0/4268364694 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:47.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7400640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 0x7f41f8155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41fcc00640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f814e0c0 0x7f41f81562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 -- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f814e0c0 msgr2=0x7f41f81562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f814e0c0 0x7f41f81562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 -- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 msgr2=0x7f41f8155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 0x7f41f8155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 -- 172.21.15.73:0/4268364694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f814e930 con 0x7f41dc003fd0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7400640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 0x7f41f8155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.211+0000 7f41f7e00640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d7e0 tx=0x7f41e800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.212+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80074e0 con 0x7f41dc003fd0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.212+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8007640 con 0x7f41dc003fd0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.212+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80049e0 con 0x7f41dc003fd0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.212+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f814b780 con 0x7f41dc003fd0 2024-04-05T22:25:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.212+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f813e790 con 0x7f41dc003fd0 2024-04-05T22:25:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.213+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f81505f0 con 0x7f41dc003fd0 2024-04-05T22:25:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.214+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f41e8004b40 con 0x7f41dc003fd0 2024-04-05T22:25:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.215+0000 7f41efe00640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 0x7f41d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.215+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41e80892d0 con 0x7f41dc003fd0 2024-04-05T22:25:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.215+0000 7f41f7400640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 0x7f41d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.216+0000 7f41f7400640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 0x7f41d406d450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41e0033000 tx=0x7f41e0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.219+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f41f81505f0 con 0x7f41dc003fd0 2024-04-05T22:25:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.421+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41f81505f0 con 0x7f41dc003fd0 2024-04-05T22:25:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.424+0000 7f41efe00640 1 -- 172.21.15.73:0/4268364694 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f41f81505f0 con 0x7f41dc003fd0 2024-04-05T22:25:47.425 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:47.425 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 msgr2=0x7f41d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 0x7f41d406d450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41e0033000 tx=0x7f41e0009030 comp rx=0 tx=0).stop 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 msgr2=0x7f41f814b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d7e0 tx=0x7f41e800dca0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 shutdown_connections 2024-04-05T22:25:47.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f41d406af90 0x7f41d406d450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f814e0c0 0x7f41f81562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f814dc40 0x7f41f8155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 --2- 172.21.15.73:0/4268364694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41dc003fd0 0x7f41f814b240 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 >> 172.21.15.73:0/4268364694 conn(0x7f41f80b06c0 msgr2=0x7f41f815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 shutdown_connections 2024-04-05T22:25:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.427+0000 7f41fd600640 1 -- 172.21.15.73:0/4268364694 wait complete. 2024-04-05T22:25:47.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:25:21.730760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:25:22.267497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:47.446 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:47.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.663+0000 7f9d20000640 1 Processor -- start 2024-04-05T22:25:47.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.663+0000 7f9d20000640 1 -- start start 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d20000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d1806c9a0 0x7f9d1806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d20000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d1806ba70 0x7f9d18164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d20000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1806bea0 con 0x7f9d1815abe0 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d20000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1815afc0 con 0x7f9d1806ba70 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d20000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1806f5b0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d1806ba70 0x7f9d18164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d1806ba70 0x7f9d18164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55676/0 (socket says 172.21.15.73:55676) 2024-04-05T22:25:47.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 -- 172.21.15.73:0/3979667662 learned_addr learned my addr 172.21.15.73:0/3979667662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:47.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1cc00640 1 --2- 172.21.15.73:0/3979667662 >> v2:172.21.15.157:3300/0 conn(0x7f9d1806c9a0 0x7f9d1806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 -- 172.21.15.73:0/3979667662 >> v2:172.21.15.157:3300/0 conn(0x7f9d1806c9a0 msgr2=0x7f9d1806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:47.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 --2- 172.21.15.73:0/3979667662 >> v2:172.21.15.157:3300/0 conn(0x7f9d1806c9a0 0x7f9d1806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 -- 172.21.15.73:0/3979667662 >> v1:172.21.15.157:6789/0 conn(0x7f9d1815abe0 legacy=0x7f9d180712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:47.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.665+0000 7f9d1e000640 1 -- 172.21.15.73:0/3979667662 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1806f890 con 0x7f9d1806ba70 2024-04-05T22:25:47.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d1e000640 1 --2- 172.21.15.73:0/3979667662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806ba70 0x7f9d18164740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9d1806c900 tx=0x7f9d1002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e7903a67bf012d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d17e00640 1 -- 172.21.15.73:0/3979667662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1002c950 con 0x7f9d1806ba70 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d17e00640 1 -- 172.21.15.73:0/3979667662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1002cab0 con 0x7f9d1806ba70 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d17e00640 1 -- 172.21.15.73:0/3979667662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1002ce40 con 0x7f9d1806ba70 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d20000640 1 -- 172.21.15.73:0/3979667662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806ba70 msgr2=0x7f9d18164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.667+0000 7f9d20000640 1 --2- 172.21.15.73:0/3979667662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806ba70 0x7f9d18164740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9d1806c900 tx=0x7f9d1002bdb0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 -- 172.21.15.73:0/3979667662 shutdown_connections 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 --2- 172.21.15.73:0/3979667662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806ba70 0x7f9d18164740 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 --2- 172.21.15.73:0/3979667662 >> v2:172.21.15.157:3300/0 conn(0x7f9d1806c9a0 0x7f9d1806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 -- 172.21.15.73:0/3979667662 >> 172.21.15.73:0/3979667662 conn(0x7f9d18069cb0 msgr2=0x7f9d1806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:47.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 -- 172.21.15.73:0/3979667662 shutdown_connections 2024-04-05T22:25:47.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.668+0000 7f9d20000640 1 -- 172.21.15.73:0/3979667662 wait complete. 2024-04-05T22:25:47.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.670+0000 7f9d20000640 1 Processor -- start 2024-04-05T22:25:47.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.670+0000 7f9d20000640 1 -- start start 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 0x7f9d1814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d1815abe0 0x7f9d1815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815e390 con 0x7f9d1806c9a0 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1814b750 con 0x7f9d1806ba70 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d20000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1814ba30 con 0x7f9d1815abe0 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d1cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d1cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55692/0 (socket says 172.21.15.73:55692) 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d1cc00640 1 -- 172.21.15.73:0/237893638 learned_addr learned my addr 172.21.15.73:0/237893638 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d1d600640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 0x7f9d1814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.671+0000 7f9d1cc00640 1 -- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d1815abe0 msgr2=0x7f9d1815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1cc00640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d1815abe0 0x7f9d1815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1cc00640 1 -- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 msgr2=0x7f9d1814b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1cc00640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 0x7f9d1814b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1cc00640 1 -- 172.21.15.73:0/237893638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009b30 con 0x7f9d1806c9a0 2024-04-05T22:25:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1d600640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 0x7f9d1814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d1cc00640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c7f0 tx=0x7f9d0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004220 con 0x7f9d1806c9a0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040043a0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04007860 con 0x7f9d1806c9a0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1814bd10 con 0x7f9d1806c9a0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.672+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1814c2d0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.673+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1814c4f0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.675+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f9d04007bd0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.676+0000 7f9d15600640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 0x7f9cf806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.676+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d040899a0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.676+0000 7f9d1d600640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 0x7f9cf806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:47.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.677+0000 7f9d1d600640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 0x7f9cf806d420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c007fd0 tx=0x7f9d0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:47.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.680+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9d040545d0 con 0x7f9d1806c9a0 2024-04-05T22:25:47.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.872+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d18068160 con 0x7f9d1806c9a0 2024-04-05T22:25:47.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.873+0000 7f9d15600640 1 -- 172.21.15.73:0/237893638 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d0401e070 con 0x7f9d1806c9a0 2024-04-05T22:25:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 msgr2=0x7f9cf806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 0x7f9cf806d420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c007fd0 tx=0x7f9d0c009040 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 msgr2=0x7f9d1814c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c7f0 tx=0x7f9d0400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 shutdown_connections 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f9cf806af60 0x7f9cf806d420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d1815abe0 0x7f9d1815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1806c9a0 0x7f9d1814c7f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 --2- 172.21.15.73:0/237893638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d1806ba70 0x7f9d1814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 >> 172.21.15.73:0/237893638 conn(0x7f9d18069cb0 msgr2=0x7f9d180af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 shutdown_connections 2024-04-05T22:25:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:47.876+0000 7f9d20000640 1 -- 172.21.15.73:0/237893638 wait complete. 2024-04-05T22:25:47.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:25:48.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.106+0000 7ff1f2400640 1 Processor -- start 2024-04-05T22:25:48.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.107+0000 7ff1f2400640 1 -- start start 2024-04-05T22:25:48.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1f2400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1ec15ce10 0x7ff1ec0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1f2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1ec0b10a0 0x7ff1ec15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1f2400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec15ddd0 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1f2400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15dbb0 con 0x7ff1ec15ce10 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1f2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15e0b0 con 0x7ff1ec0b10a0 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1eb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1ec15ce10 0x7ff1ec0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1eaa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1ec15d1f0 0x7ff1ec0b0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51852/0 (socket says 172.21.15.73:51852) 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.108+0000 7ff1eaa00640 1 -- 172.21.15.73:0/4038273483 learned_addr learned my addr 172.21.15.73:0/4038273483 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.109+0000 7ff1ebe00640 1 --2- 172.21.15.73:0/4038273483 >> v2:172.21.15.157:3300/0 conn(0x7ff1ec0b10a0 0x7ff1ec15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.109+0000 7ff1eb400640 1 -- 172.21.15.73:0/4038273483 >> v2:172.21.15.157:3300/0 conn(0x7ff1ec0b10a0 msgr2=0x7ff1ec15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.109+0000 7ff1eb400640 1 --2- 172.21.15.73:0/4038273483 >> v2:172.21.15.157:3300/0 conn(0x7ff1ec0b10a0 0x7ff1ec15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.109+0000 7ff1eb400640 1 -- 172.21.15.73:0/4038273483 >> v1:172.21.15.73:6789/0 conn(0x7ff1ec15d1f0 legacy=0x7ff1ec0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.109+0000 7ff1eb400640 1 -- 172.21.15.73:0/4038273483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0cd460 con 0x7ff1ec15ce10 2024-04-05T22:25:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.110+0000 7ff1eb400640 1 --2- 172.21.15.73:0/4038273483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15ce10 0x7ff1ec0cdc60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0715a0 tx=0x7ff1dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97277cbd66416dde server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.110+0000 7ff1ea000640 1 -- 172.21.15.73:0/4038273483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc031830 con 0x7ff1ec15ce10 2024-04-05T22:25:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1ea000640 1 -- 172.21.15.73:0/4038273483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc031990 con 0x7ff1ec15ce10 2024-04-05T22:25:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1ea000640 1 -- 172.21.15.73:0/4038273483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc031d20 con 0x7ff1ec15ce10 2024-04-05T22:25:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 -- 172.21.15.73:0/4038273483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15ce10 msgr2=0x7ff1ec0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4038273483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15ce10 0x7ff1ec0cdc60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0715a0 tx=0x7ff1dc02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 -- 172.21.15.73:0/4038273483 shutdown_connections 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4038273483 >> v2:172.21.15.157:3300/0 conn(0x7ff1ec0b10a0 0x7ff1ec15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4038273483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15ce10 0x7ff1ec0cdc60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 -- 172.21.15.73:0/4038273483 >> 172.21.15.73:0/4038273483 conn(0x7ff1ec099a60 msgr2=0x7ff1ec099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 -- 172.21.15.73:0/4038273483 shutdown_connections 2024-04-05T22:25:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.111+0000 7ff1f2400640 1 -- 172.21.15.73:0/4038273483 wait complete. 2024-04-05T22:25:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.113+0000 7ff1f2400640 1 Processor -- start 2024-04-05T22:25:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.113+0000 7ff1f2400640 1 -- start start 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec0b10a0 0x7ff1ec15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1ec15ce10 0x7ff1ec0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0b6330 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c45e0 con 0x7ff1ec0b10a0 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1f2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c48c0 con 0x7ff1ec15ce10 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55720/0 (socket says 172.21.15.73:55720) 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.114+0000 7ff1ebe00640 1 -- 172.21.15.73:0/4127926341 learned_addr learned my addr 172.21.15.73:0/4127926341 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1eb400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec0b10a0 0x7ff1ec15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1eaa00640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1ec15ce10 0x7ff1ec0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 -- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1ec15ce10 msgr2=0x7ff1ec0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1ec15ce10 0x7ff1ec0b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 -- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec0b10a0 msgr2=0x7ff1ec15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec0b10a0 0x7ff1ec15b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 -- 172.21.15.73:0/4127926341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009400 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1ebe00640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d800 tx=0x7ff1e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.115+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400b6c0 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.116+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400b840 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.116+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400bbf0 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.116+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0c4ba0 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.116+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0b4060 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.117+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec0c4440 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.118+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7ff1e400bd50 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.119+0000 7ff1e3400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 0x7ff1c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.119+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1e4089d50 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.119+0000 7ff1eb400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 0x7ff1c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.120+0000 7ff1eb400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 0x7ff1c806d450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc036000 tx=0x7ff1dc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.123+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff1ec0c4440 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.319+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1ec0c4440 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.319+0000 7ff1e3400640 1 -- 172.21.15.73:0/4127926341 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1ec0c4440 con 0x7ff1ec15d1f0 2024-04-05T22:25:48.320 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:25:48.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.322+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 msgr2=0x7ff1c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.322+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 0x7ff1c806d450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc036000 tx=0x7ff1dc03c040 comp rx=0 tx=0).stop 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.322+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 msgr2=0x7ff1ec14b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.322+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d800 tx=0x7ff1e400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 shutdown_connections 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff1c806af90 0x7ff1c806d450 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1ec15d1f0 0x7ff1ec14b200 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1ec15ce10 0x7ff1ec0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 --2- 172.21.15.73:0/4127926341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec0b10a0 0x7ff1ec15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 >> 172.21.15.73:0/4127926341 conn(0x7ff1ec099a60 msgr2=0x7ff1ec06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:48.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 shutdown_connections 2024-04-05T22:25:48.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.323+0000 7ff1f2400640 1 -- 172.21.15.73:0/4127926341 wait complete. 2024-04-05T22:25:48.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:25:48.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.546+0000 7f27edc00640 1 Processor -- start 2024-04-05T22:25:48.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.546+0000 7f27edc00640 1 -- start start 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.547+0000 7f27edc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27e8143340 0x7f27e815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.547+0000 7f27edc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80b2d00 con 0x7f27e806cc20 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.547+0000 7f27edc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80b1e20 con 0x7f27e80b19c0 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.547+0000 7f27edc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80b20f0 con 0x7f27e8143340 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27e8143340 0x7f27e815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f27e806cc20 0x7f27e80ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51862/0 (socket says 172.21.15.73:51862) 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27e8143340 0x7f27e815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55724/0 (socket says 172.21.15.73:55724) 2024-04-05T22:25:48.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e7400640 1 -- 172.21.15.73:0/2987530150 learned_addr learned my addr 172.21.15.73:0/2987530150 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:48.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e6a00640 1 -- 172.21.15.73:0/2987530150 >> v1:172.21.15.157:6789/0 conn(0x7f27e80b19c0 legacy=0x7f27e806c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e6a00640 1 -- 172.21.15.73:0/2987530150 >> v1:172.21.15.73:6789/0 conn(0x7f27e806cc20 legacy=0x7f27e80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.548+0000 7f27e6a00640 1 -- 172.21.15.73:0/2987530150 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80b23d0 con 0x7f27e8143340 2024-04-05T22:25:48.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.549+0000 7f27e6a00640 1 --2- 172.21.15.73:0/2987530150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e8143340 0x7f27e815e0d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097a0 tx=0x7f27d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29548a4435317159 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.549+0000 7f27e6000640 1 -- 172.21.15.73:0/2987530150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8031830 con 0x7f27e8143340 2024-04-05T22:25:48.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.549+0000 7f27e6000640 1 -- 172.21.15.73:0/2987530150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8031990 con 0x7f27e8143340 2024-04-05T22:25:48.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27e6000640 1 -- 172.21.15.73:0/2987530150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8031d20 con 0x7f27e8143340 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 -- 172.21.15.73:0/2987530150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e8143340 msgr2=0x7f27e815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 --2- 172.21.15.73:0/2987530150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e8143340 0x7f27e815e0d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097a0 tx=0x7f27d802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 -- 172.21.15.73:0/2987530150 shutdown_connections 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 --2- 172.21.15.73:0/2987530150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e8143340 0x7f27e815e0d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 -- 172.21.15.73:0/2987530150 >> 172.21.15.73:0/2987530150 conn(0x7f27e80b06c0 msgr2=0x7f27e80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 -- 172.21.15.73:0/2987530150 shutdown_connections 2024-04-05T22:25:48.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.550+0000 7f27edc00640 1 -- 172.21.15.73:0/2987530150 wait complete. 2024-04-05T22:25:48.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.552+0000 7f27edc00640 1 Processor -- start 2024-04-05T22:25:48.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.552+0000 7f27edc00640 1 -- start start 2024-04-05T22:25:48.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80b19c0 0x7f27e80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e8143340 0x7f27e80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80cb300 con 0x7f27e806cc20 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80c92a0 con 0x7f27e8143340 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27edc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8138410 con 0x7f27e80b19c0 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55740/0 (socket says 172.21.15.73:55740) 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.553+0000 7f27e6a00640 1 -- 172.21.15.73:0/577434080 learned_addr learned my addr 172.21.15.73:0/577434080 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 -- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80b19c0 msgr2=0x7f27e80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80b19c0 0x7f27e80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e7400640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e8143340 0x7f27e80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 -- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e8143340 msgr2=0x7f27e80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e8143340 0x7f27e80ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 -- 172.21.15.73:0/577434080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009400 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27e6a00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097a0 tx=0x7f27d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d803aa60 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d803abc0 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80443f0 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.554+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e80caa60 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.555+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e80cb020 con 0x7f27e806cc20 2024-04-05T22:25:48.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.556+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e8068160 con 0x7f27e806cc20 2024-04-05T22:25:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.557+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f27d804d450 con 0x7f27e806cc20 2024-04-05T22:25:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.558+0000 7f27dea00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 0x7f27c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.558+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27d80abc80 con 0x7f27e806cc20 2024-04-05T22:25:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.558+0000 7f27dfe00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 0x7f27c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.558+0000 7f27dfe00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 0x7f27c406d420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f27e813c070 tx=0x7f27d0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.562+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f27d80b2080 con 0x7f27e806cc20 2024-04-05T22:25:48.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.760+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27e8159a90 con 0x7f27e806cc20 2024-04-05T22:25:48.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.760+0000 7f27dea00640 1 -- 172.21.15.73:0/577434080 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27d80769d0 con 0x7f27e806cc20 2024-04-05T22:25:48.761 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 msgr2=0x7f27c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 0x7f27c406d420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f27e813c070 tx=0x7f27d0007990 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 msgr2=0x7f27e8137ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097a0 tx=0x7f27d80046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 shutdown_connections 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f27c406af60 0x7f27c406d420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e8143340 0x7f27e80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80b19c0 0x7f27e80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 --2- 172.21.15.73:0/577434080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e806cc20 0x7f27e8137ed0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 >> 172.21.15.73:0/577434080 conn(0x7f27e80b06c0 msgr2=0x7f27e815d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 shutdown_connections 2024-04-05T22:25:48.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.763+0000 7f27edc00640 1 -- 172.21.15.73:0/577434080 wait complete. 2024-04-05T22:25:48.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:25:48.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.984+0000 7f8f0ba00640 1 Processor -- start 2024-04-05T22:25:48.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.984+0000 7f8f0ba00640 1 -- start start 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f0ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8f040cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f0ba00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f0415e850 0x7f8f0415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f0ba00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0406f5b0 con 0x7f8f040ccd20 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f0ba00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0415f240 con 0x7f8f0415e850 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f0ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04070410 con 0x7f8f041587d0 2024-04-05T22:25:48.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f03e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f040ccd20 0x7f8f0406cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51864/0 (socket says 172.21.15.73:51864) 2024-04-05T22:25:48.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f03e00640 1 -- 172.21.15.73:0/3785856703 learned_addr learned my addr 172.21.15.73:0/3785856703 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:48.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.986+0000 7f8f09a00640 1 --2- 172.21.15.73:0/3785856703 >> v2:172.21.15.157:3300/0 conn(0x7f8f0415e850 0x7f8f0415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f09000640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8f040cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590789574 0 0) 0x7f8f0406f5b0 con 0x7f8f040ccd20 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eec003520 con 0x7f8f040ccd20 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955838988 0 0) 0x7f8eec003520 con 0x7f8f040ccd20 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 msgr2=0x7f8f040cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8f040cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 >> v2:172.21.15.157:3300/0 conn(0x7f8f0415e850 msgr2=0x7f8f0415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 --2- 172.21.15.73:0/3785856703 >> v2:172.21.15.157:3300/0 conn(0x7f8f0415e850 0x7f8f0415ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04158bb0 con 0x7f8f040ccd20 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8ef0001670 con 0x7f8f040ccd20 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 >> v1:172.21.15.73:6789/0 conn(0x7f8f040ccd20 legacy=0x7f8f0406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8eec003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eec003ef0 0x7f8eec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec004d00 0x7f8eec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0415f240 con 0x7f8eec003ef0 2024-04-05T22:25:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04070410 con 0x7f8eec004d00 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec00b9d0 con 0x7f8f041587d0 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.987+0000 7f8f03e00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eec003ef0 0x7f8eec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f03e00640 1 -- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 msgr2=0x7f8eec003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f09a00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec004d00 0x7f8eec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f09000640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8eec003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f03e00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f041587d0 0x7f8eec003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f03e00640 1 -- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec004d00 msgr2=0x7f8eec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f03e00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec004d00 0x7f8eec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.988+0000 7f8f03e00640 1 -- 172.21.15.73:0/3785856703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec003520 con 0x7f8eec003ef0 2024-04-05T22:25:48.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.989+0000 7f8f03e00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eec003ef0 0x7f8eec0047c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c210 tx=0x7f8eec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f56dd88429f17f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.989+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef002cae0 con 0x7f8eec003ef0 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.989+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef002cc40 con 0x7f8eec003ef0 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.989+0000 7f8f03400640 1 -- 172.21.15.73:0/3785856703 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0005af0 con 0x7f8eec003ef0 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3785856703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eec003ef0 msgr2=0x7f8eec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3785856703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eec003ef0 0x7f8eec0047c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c210 tx=0x7f8eec0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f09a00640 1 -- 172.21.15.73:0/3785856703 reap_dead start 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3785856703 shutdown_connections 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3785856703 >> 172.21.15.73:0/3785856703 conn(0x7f8f040ce8e0 msgr2=0x7f8f040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3785856703 shutdown_connections 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.990+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3785856703 wait complete. 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.991+0000 7f8f0ba00640 1 Processor -- start 2024-04-05T22:25:48.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 -- start start 2024-04-05T22:25:48.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec003ef0 0x7f8f0415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f040cb400 0x7f8f04141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040a84c0 con 0x7f8f0415d510 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040a87a0 con 0x7f8eec003ef0 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.992+0000 7f8f0ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040a8a80 con 0x7f8f040cb400 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55760/0 (socket says 172.21.15.73:55760) 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 -- 172.21.15.73:0/3409614021 learned_addr learned my addr 172.21.15.73:0/3409614021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 -- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f040cb400 msgr2=0x7f8f04141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f040cb400 0x7f8f04141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 -- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec003ef0 msgr2=0x7f8f0415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec003ef0 0x7f8f0415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 -- 172.21.15.73:0/3409614021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040a8d60 con 0x7f8f0415d510 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f09a00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8f0415ec30 tx=0x7f8efc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.993+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc016620 con 0x7f8f0415d510 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.994+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0167a0 con 0x7f8f0415d510 2024-04-05T22:25:48.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.994+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc016b50 con 0x7f8f0415d510 2024-04-05T22:25:48.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.994+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f040a9040 con 0x7f8f0415d510 2024-04-05T22:25:48.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.994+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0415cb40 con 0x7f8f0415d510 2024-04-05T22:25:48.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.995+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f04066af0 con 0x7f8f0415d510 2024-04-05T22:25:48.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.996+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f8efc016cb0 con 0x7f8f0415d510 2024-04-05T22:25:48.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.997+0000 7f8f00c00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 0x7f8ee406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:48.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.997+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8efc08a770 con 0x7f8f0415d510 2024-04-05T22:25:48.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.997+0000 7f8f09000640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 0x7f8ee406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:48.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:48.998+0000 7f8f09000640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 0x7f8ee406d450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006f00 tx=0x7f8ef8006e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:49.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.001+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8efc055410 con 0x7f8f0415d510 2024-04-05T22:25:49.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.205+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f040a9560 con 0x7f8f0415d510 2024-04-05T22:25:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.206+0000 7f8f00c00640 1 -- 172.21.15.73:0/3409614021 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8f040a9560 con 0x7f8f0415d510 2024-04-05T22:25:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:25:49.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 msgr2=0x7f8ee406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 0x7f8ee406d450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006f00 tx=0x7f8ef8006e10 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 msgr2=0x7f8f04142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8f0415ec30 tx=0x7f8efc00f520 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 shutdown_connections 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8ee406af90 0x7f8ee406d450 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0415d510 0x7f8f04142470 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f040cb400 0x7f8f04141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 --2- 172.21.15.73:0/3409614021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eec003ef0 0x7f8f0415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 >> 172.21.15.73:0/3409614021 conn(0x7f8f040ce8e0 msgr2=0x7f8f04158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 shutdown_connections 2024-04-05T22:25:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.209+0000 7f8f0ba00640 1 -- 172.21.15.73:0/3409614021 wait complete. 2024-04-05T22:25:49.227 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:25:49.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:49.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:25:49.358 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:25:49.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:49.358 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:25:49.407 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:49.407 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:49.447 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:49.447 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:49.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:49.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.736+0000 7fef14200640 1 Processor -- start 2024-04-05T22:25:49.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.736+0000 7fef14200640 1 -- start start 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef0c160620 0x7fef0c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c0b1870 0x7fef0c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef0c0ad6b0 0x7fef0c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0ce5d0 con 0x7fef0c160620 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0ce010 con 0x7fef0c0ad6b0 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef14200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0ce2f0 con 0x7fef0c0b1870 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef11800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef0c160620 0x7fef0c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:49.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef11800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef0c160620 0x7fef0c0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55768/0 (socket says 172.21.15.73:55768) 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.738+0000 7fef11800640 1 -- 172.21.15.73:0/1315734960 learned_addr learned my addr 172.21.15.73:0/1315734960 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.739+0000 7fef11800640 1 -- 172.21.15.73:0/1315734960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c0b1870 msgr2=0x7fef0c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.739+0000 7fef11800640 1 --2- 172.21.15.73:0/1315734960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c0b1870 0x7fef0c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.739+0000 7fef11800640 1 -- 172.21.15.73:0/1315734960 >> v2:172.21.15.157:3300/0 conn(0x7fef0c0ad6b0 msgr2=0x7fef0c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.739+0000 7fef11800640 1 --2- 172.21.15.73:0/1315734960 >> v2:172.21.15.157:3300/0 conn(0x7fef0c0ad6b0 0x7fef0c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.739+0000 7fef11800640 1 -- 172.21.15.73:0/1315734960 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0ac970 con 0x7fef0c160620 2024-04-05T22:25:49.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.740+0000 7fef11800640 1 --2- 172.21.15.73:0/1315734960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c160620 0x7fef0c0caa60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0cdec0 tx=0x7fef0002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe07986f51825b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:49.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.740+0000 7fef0be00640 1 -- 172.21.15.73:0/1315734960 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00031860 con 0x7fef0c160620 2024-04-05T22:25:49.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.740+0000 7fef0be00640 1 -- 172.21.15.73:0/1315734960 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef000319c0 con 0x7fef0c160620 2024-04-05T22:25:49.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.740+0000 7fef0be00640 1 -- 172.21.15.73:0/1315734960 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00031d50 con 0x7fef0c160620 2024-04-05T22:25:49.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 -- 172.21.15.73:0/1315734960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c160620 msgr2=0x7fef0c0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 --2- 172.21.15.73:0/1315734960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c160620 0x7fef0c0caa60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0cdec0 tx=0x7fef0002b900 comp rx=0 tx=0).stop 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 -- 172.21.15.73:0/1315734960 shutdown_connections 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 --2- 172.21.15.73:0/1315734960 >> v2:172.21.15.157:3300/0 conn(0x7fef0c0ad6b0 0x7fef0c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 --2- 172.21.15.73:0/1315734960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c0b1870 0x7fef0c0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 --2- 172.21.15.73:0/1315734960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c160620 0x7fef0c0caa60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 -- 172.21.15.73:0/1315734960 >> 172.21.15.73:0/1315734960 conn(0x7fef0c0ce8e0 msgr2=0x7fef0c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 -- 172.21.15.73:0/1315734960 shutdown_connections 2024-04-05T22:25:49.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.741+0000 7fef14200640 1 -- 172.21.15.73:0/1315734960 wait complete. 2024-04-05T22:25:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.742+0000 7fef14200640 1 Processor -- start 2024-04-05T22:25:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.743+0000 7fef14200640 1 -- start start 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.743+0000 7fef14200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0c0ad6b0 0x7fef0c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.743+0000 7fef14200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.743+0000 7fef14200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c160620 0x7fef0c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef14200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c15a8e0 con 0x7fef0c0b1870 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef14200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c159cd0 con 0x7fef0c0ad6b0 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef14200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c159fb0 con 0x7fef0c160620 2024-04-05T22:25:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55770/0 (socket says 172.21.15.73:55770) 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 -- 172.21.15.73:0/3856010143 learned_addr learned my addr 172.21.15.73:0/3856010143 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 -- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c160620 msgr2=0x7fef0c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c160620 0x7fef0c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 -- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0c0ad6b0 msgr2=0x7fef0c14a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0c0ad6b0 0x7fef0c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 -- 172.21.15.73:0/3856010143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000094d0 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.744+0000 7fef10e00640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b4a0 tx=0x7feefc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.745+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc010610 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.745+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc010770 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.745+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc010b20 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.745+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c15a290 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.745+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c14aa80 con 0x7fef0c0b1870 2024-04-05T22:25:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.746+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c066af0 con 0x7fef0c0b1870 2024-04-05T22:25:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.747+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7feefc010c80 con 0x7fef0c0b1870 2024-04-05T22:25:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.748+0000 7fef09600640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 0x7feef006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.748+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feefc089640 con 0x7fef0c0b1870 2024-04-05T22:25:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.748+0000 7fef11800640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 0x7feef006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:49.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.749+0000 7fef11800640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 0x7feef006d420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fef00036000 tx=0x7fef00030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:49.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.752+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feefc0542d0 con 0x7fef0c0b1870 2024-04-05T22:25:49.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.932+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef0c068160 con 0x7fef0c0b1870 2024-04-05T22:25:49.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.934+0000 7fef09600640 1 -- 172.21.15.73:0/3856010143 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7feefc0578f0 con 0x7fef0c0b1870 2024-04-05T22:25:49.935 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:49.935 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.936+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 msgr2=0x7feef006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.936+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 0x7feef006d420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fef00036000 tx=0x7fef00030000 comp rx=0 tx=0).stop 2024-04-05T22:25:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.936+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 msgr2=0x7fef0c0b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.936+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b4a0 tx=0x7feefc00b960 comp rx=0 tx=0).stop 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 shutdown_connections 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7feef006af60 0x7feef006d420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0c160620 0x7fef0c0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0c0b1870 0x7fef0c0b3200 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 --2- 172.21.15.73:0/3856010143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0c0ad6b0 0x7fef0c14a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 >> 172.21.15.73:0/3856010143 conn(0x7fef0c0ce8e0 msgr2=0x7fef0c06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 shutdown_connections 2024-04-05T22:25:49.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:49.937+0000 7fef14200640 1 -- 172.21.15.73:0/3856010143 wait complete. 2024-04-05T22:25:49.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:25:21.730760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:25:22.267497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:49.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:50.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.153+0000 7fdee4e00640 1 Processor -- start 2024-04-05T22:25:50.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.153+0000 7fdee4e00640 1 -- start start 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdee4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 0x7fdee0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdee4e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00b27f0 con 0x7fdee00add00 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdee4e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00b2ad0 con 0x7fdee006cd50 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdee4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b3890 con 0x7fdee01599f0 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdedd400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdee00add00 0x7fdee006c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51878/0 (socket says 172.21.15.73:51878) 2024-04-05T22:25:50.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdedd400640 1 -- 172.21.15.73:0/1147142333 learned_addr learned my addr 172.21.15.73:0/1147142333 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606597970 0 0) 0x7fdee00b27f0 con 0x7fdee00add00 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdedde00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 0x7fdee0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.155+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec4003520 con 0x7fdee00add00 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560372003 0 0) 0x7fdec4003520 con 0x7fdee00add00 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 msgr2=0x7fdee0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 0x7fdee0070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 >> v1:172.21.15.157:6789/0 conn(0x7fdee006cd50 legacy=0x7fdee0159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00b2db0 con 0x7fdee00add00 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fdecc001670 con 0x7fdee00add00 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 >> v1:172.21.15.73:6789/0 conn(0x7fdee00add00 legacy=0x7fdee006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdec4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40047c0 0x7fdec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 0x7fdec4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b3890 con 0x7fdec40039a0 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec400ba30 con 0x7fdec40047c0 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec400bbf0 con 0x7fdee01599f0 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.156+0000 7fdedde00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdec4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.157+0000 7fdedde00640 1 -- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 msgr2=0x7fdec4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.157+0000 7fdedde00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01599f0 0x7fdec4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.157+0000 7fdedde00640 1 -- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40047c0 msgr2=0x7fdec4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:50.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.157+0000 7fdedde00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40047c0 0x7fdec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.157+0000 7fdedde00640 1 -- 172.21.15.73:0/1147142333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4003520 con 0x7fdec40039a0 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdedde00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdec4003dc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a90 tx=0x7fdec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6b35e2a7911d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded002c6b0 con 0x7fdec40039a0 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded002c810 con 0x7fdec40039a0 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdedca00640 1 -- 172.21.15.73:0/1147142333 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded002cba0 con 0x7fdec40039a0 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 -- 172.21.15.73:0/1147142333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 msgr2=0x7fdec4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 --2- 172.21.15.73:0/1147142333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdec4003dc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a90 tx=0x7fdec40057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdede800640 1 -- 172.21.15.73:0/1147142333 reap_dead start 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 -- 172.21.15.73:0/1147142333 shutdown_connections 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 -- 172.21.15.73:0/1147142333 >> 172.21.15.73:0/1147142333 conn(0x7fdee0069cb0 msgr2=0x7fdee006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 -- 172.21.15.73:0/1147142333 shutdown_connections 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.158+0000 7fdee4e00640 1 -- 172.21.15.73:0/1147142333 wait complete. 2024-04-05T22:25:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.160+0000 7fdee4e00640 1 Processor -- start 2024-04-05T22:25:50.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.160+0000 7fdee4e00640 1 -- start start 2024-04-05T22:25:50.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00cd5a0 0x7fdee00cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee0159f90 0x7fdee015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ce6b0 con 0x7fdec40039a0 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0150910 con 0x7fdee0159f90 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdee4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee015a960 con 0x7fdee00cd5a0 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdedde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdedde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55778/0 (socket says 172.21.15.73:55778) 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.161+0000 7fdedde00640 1 -- 172.21.15.73:0/706369825 learned_addr learned my addr 172.21.15.73:0/706369825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 -- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00cd5a0 msgr2=0x7fdee00cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00cd5a0 0x7fdee00cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 -- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee0159f90 msgr2=0x7fdee015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee0159f90 0x7fdee015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 -- 172.21.15.73:0/706369825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00cd3e0 con 0x7fdec40039a0 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdedde00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fded0037040 tx=0x7fded0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0003810 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0003970 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0003ce0 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00cda30 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.162+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00cdff0 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.163+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00709f0 con 0x7fdec40039a0 2024-04-05T22:25:50.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.164+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fded0034440 con 0x7fdec40039a0 2024-04-05T22:25:50.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.166+0000 7fded6000640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.166+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fded00ab4f0 con 0x7fdec40039a0 2024-04-05T22:25:50.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.166+0000 7fdedd400640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.166+0000 7fdedd400640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 0x7fdec006d3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdecc000f60 tx=0x7fdecc001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.170+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fded0076520 con 0x7fdec40039a0 2024-04-05T22:25:50.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.364+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdee00cd3e0 con 0x7fdec40039a0 2024-04-05T22:25:50.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.365+0000 7fded6000640 1 -- 172.21.15.73:0/706369825 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdee00cd3e0 con 0x7fdec40039a0 2024-04-05T22:25:50.365 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 msgr2=0x7fdec006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 0x7fdec006d3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdecc000f60 tx=0x7fdecc001520 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 msgr2=0x7fdee01503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fded0037040 tx=0x7fded0004730 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 shutdown_connections 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee0159f90 0x7fdee015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00cd5a0 0x7fdee00cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 --2- 172.21.15.73:0/706369825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40039a0 0x7fdee01503d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.368+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 >> 172.21.15.73:0/706369825 conn(0x7fdee0069cb0 msgr2=0x7fdee0153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.369+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 shutdown_connections 2024-04-05T22:25:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.369+0000 7fdee4e00640 1 -- 172.21.15.73:0/706369825 wait complete. 2024-04-05T22:25:50.387 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:25:50.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.589+0000 7fdd66000640 1 Processor -- start 2024-04-05T22:25:50.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.589+0000 7fdd66000640 1 -- start start 2024-04-05T22:25:50.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.590+0000 7fdd66000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.590+0000 7fdd66000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd6015dd90 0x7fdd6006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd66000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd60071900 con 0x7fdd600714a0 2024-04-05T22:25:50.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd66000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015ae50 con 0x7fdd6015dd90 2024-04-05T22:25:50.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd66000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015e3b0 con 0x7fdd6015aa70 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58720/0 (socket says 172.21.15.73:58720) 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.591+0000 7fdd5f400640 1 -- 172.21.15.73:0/1872773910 learned_addr learned my addr 172.21.15.73:0/1872773910 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.592+0000 7fdd5fe00640 1 --2- 172.21.15.73:0/1872773910 >> v2:172.21.15.157:3300/0 conn(0x7fdd6015dd90 0x7fdd6006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.592+0000 7fdd5f400640 1 -- 172.21.15.73:0/1872773910 >> v2:172.21.15.157:3300/0 conn(0x7fdd6015dd90 msgr2=0x7fdd6006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.592+0000 7fdd5f400640 1 --2- 172.21.15.73:0/1872773910 >> v2:172.21.15.157:3300/0 conn(0x7fdd6015dd90 0x7fdd6006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.592+0000 7fdd5f400640 1 -- 172.21.15.73:0/1872773910 >> v1:172.21.15.157:6789/0 conn(0x7fdd600714a0 legacy=0x7fdd6015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.592+0000 7fdd5f400640 1 -- 172.21.15.73:0/1872773910 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600ac9e0 con 0x7fdd6015aa70 2024-04-05T22:25:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.593+0000 7fdd5f400640 1 --2- 172.21.15.73:0/1872773910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd5002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a12452e02d0cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.594+0000 7fdd5ea00640 1 -- 172.21.15.73:0/1872773910 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50030840 con 0x7fdd6015aa70 2024-04-05T22:25:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.594+0000 7fdd5ea00640 1 -- 172.21.15.73:0/1872773910 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd500309a0 con 0x7fdd6015aa70 2024-04-05T22:25:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.594+0000 7fdd5ea00640 1 -- 172.21.15.73:0/1872773910 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50030d30 con 0x7fdd6015aa70 2024-04-05T22:25:50.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 -- 172.21.15.73:0/1872773910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 msgr2=0x7fdd60070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 --2- 172.21.15.73:0/1872773910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd5002b980 comp rx=0 tx=0).stop 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 -- 172.21.15.73:0/1872773910 shutdown_connections 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 --2- 172.21.15.73:0/1872773910 >> v2:172.21.15.157:3300/0 conn(0x7fdd6015dd90 0x7fdd6006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 --2- 172.21.15.73:0/1872773910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015aa70 0x7fdd60070f60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 -- 172.21.15.73:0/1872773910 >> 172.21.15.73:0/1872773910 conn(0x7fdd60069cb0 msgr2=0x7fdd6006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 -- 172.21.15.73:0/1872773910 shutdown_connections 2024-04-05T22:25:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.595+0000 7fdd66000640 1 -- 172.21.15.73:0/1872773910 wait complete. 2024-04-05T22:25:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.597+0000 7fdd66000640 1 Processor -- start 2024-04-05T22:25:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.597+0000 7fdd66000640 1 -- start start 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6015aa70 0x7fdd600ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015dd90 0x7fdd6006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601f5ec0 con 0x7fdd600714a0 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601f61a0 con 0x7fdd6015aa70 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd66000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601f6480 con 0x7fdd6015dd90 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55786/0 (socket says 172.21.15.73:55786) 2024-04-05T22:25:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.598+0000 7fdd5f400640 1 -- 172.21.15.73:0/852950869 learned_addr learned my addr 172.21.15.73:0/852950869 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 -- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015dd90 msgr2=0x7fdd6006f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015dd90 0x7fdd6006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 -- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6015aa70 msgr2=0x7fdd600ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6015aa70 0x7fdd600ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 -- 172.21.15.73:0/852950869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500094d0 con 0x7fdd600714a0 2024-04-05T22:25:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd5f400640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50039c10 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50039d70 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601f6760 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.599+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50043640 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.600+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601f5ae0 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.601+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60158750 con 0x7fdd600714a0 2024-04-05T22:25:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.601+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fdd500437a0 con 0x7fdd600714a0 2024-04-05T22:25:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.602+0000 7fdd57400640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 0x7fdd3c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.602+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd5003e070 con 0x7fdd600714a0 2024-04-05T22:25:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.603+0000 7fdd57e00640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 0x7fdd3c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.603+0000 7fdd57e00640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 0x7fdd3c06d420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd480079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:50.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.607+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdd60158750 con 0x7fdd600714a0 2024-04-05T22:25:50.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.801+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd601f5c70 con 0x7fdd600714a0 2024-04-05T22:25:50.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.802+0000 7fdd57400640 1 -- 172.21.15.73:0/852950869 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd50034020 con 0x7fdd600714a0 2024-04-05T22:25:50.802 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:25:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.804+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 msgr2=0x7fdd3c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.804+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 0x7fdd3c06d420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd480079b0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.804+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 msgr2=0x7fdd601f1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:50.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.804+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd50004730 comp rx=0 tx=0).stop 2024-04-05T22:25:50.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 shutdown_connections 2024-04-05T22:25:50.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdd3c06af60 0x7fdd3c06d420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd6015dd90 0x7fdd6006f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd6015aa70 0x7fdd600ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 --2- 172.21.15.73:0/852950869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd600714a0 0x7fdd601f1e60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 >> 172.21.15.73:0/852950869 conn(0x7fdd60069cb0 msgr2=0x7fdd6006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 shutdown_connections 2024-04-05T22:25:50.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:50.805+0000 7fdd66000640 1 -- 172.21.15.73:0/852950869 wait complete. 2024-04-05T22:25:50.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:25:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.016+0000 7ffa08c00640 1 Processor -- start 2024-04-05T22:25:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.016+0000 7ffa08c00640 1 -- start start 2024-04-05T22:25:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.018+0000 7ffa08c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa0406c630 0x7ffa0406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.018+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.018+0000 7ffa08c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa040b1b30 con 0x7ffa040cbe00 2024-04-05T22:25:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.018+0000 7ffa08c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa040b1e10 con 0x7ffa0406c630 2024-04-05T22:25:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.018+0000 7ffa08c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040ce7b0 con 0x7ffa040b32b0 2024-04-05T22:25:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.019+0000 7ffa02800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.019+0000 7ff9fbe00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffa040cbe00 0x7ffa040ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:55576/0 (socket says 172.21.15.73:55576) 2024-04-05T22:25:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.019+0000 7ff9fbe00640 1 -- 172.21.15.73:0/3346930737 learned_addr learned my addr 172.21.15.73:0/3346930737 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.019+0000 7ffa02800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58738/0 (socket says 172.21.15.73:58738) 2024-04-05T22:25:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.019+0000 7ffa01e00640 1 --2- 172.21.15.73:0/3346930737 >> v2:172.21.15.157:3300/0 conn(0x7ffa0406c630 0x7ffa0406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.020+0000 7ffa02800640 1 -- 172.21.15.73:0/3346930737 >> v2:172.21.15.157:3300/0 conn(0x7ffa0406c630 msgr2=0x7ffa0406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.020+0000 7ffa02800640 1 --2- 172.21.15.73:0/3346930737 >> v2:172.21.15.157:3300/0 conn(0x7ffa0406c630 0x7ffa0406ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.020+0000 7ffa01400640 1 -- 172.21.15.73:0/3346930737 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079237166 0 0) 0x7ffa040b1b30 con 0x7ffa040cbe00 2024-04-05T22:25:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.020+0000 7ffa02800640 1 -- 172.21.15.73:0/3346930737 >> v1:172.21.15.157:6789/0 conn(0x7ffa040cbe00 legacy=0x7ffa040ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.020+0000 7ffa02800640 1 -- 172.21.15.73:0/3346930737 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0406cf50 con 0x7ffa040b32b0 2024-04-05T22:25:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.022+0000 7ffa02800640 1 --2- 172.21.15.73:0/3346930737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb50e733abb5906d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.022+0000 7ffa01400640 1 -- 172.21.15.73:0/3346930737 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc02cc60 con 0x7ffa040b32b0 2024-04-05T22:25:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.022+0000 7ffa01400640 1 -- 172.21.15.73:0/3346930737 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc02cdc0 con 0x7ffa040b32b0 2024-04-05T22:25:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.022+0000 7ffa01400640 1 -- 172.21.15.73:0/3346930737 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc034660 con 0x7ffa040b32b0 2024-04-05T22:25:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 -- 172.21.15.73:0/3346930737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 msgr2=0x7ffa041647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 --2- 172.21.15.73:0/3346930737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc02beb0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 -- 172.21.15.73:0/3346930737 shutdown_connections 2024-04-05T22:25:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 --2- 172.21.15.73:0/3346930737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040b32b0 0x7ffa041647c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 --2- 172.21.15.73:0/3346930737 >> v2:172.21.15.157:3300/0 conn(0x7ffa0406c630 0x7ffa0406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 -- 172.21.15.73:0/3346930737 >> 172.21.15.73:0/3346930737 conn(0x7ffa040b0830 msgr2=0x7ffa040b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 -- 172.21.15.73:0/3346930737 shutdown_connections 2024-04-05T22:25:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.023+0000 7ffa08c00640 1 -- 172.21.15.73:0/3346930737 wait complete. 2024-04-05T22:25:51.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.025+0000 7ffa08c00640 1 Processor -- start 2024-04-05T22:25:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.026+0000 7ffa08c00640 1 -- start start 2024-04-05T22:25:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0406c630 0x7ffa041f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040cbe00 0x7ffa0406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041f2bf0 con 0x7ffa040b32b0 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041f6590 con 0x7ffa0406c630 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ffa08c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041f6870 con 0x7ffa040cbe00 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55792/0 (socket says 172.21.15.73:55792) 2024-04-05T22:25:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.027+0000 7ff9fbe00640 1 -- 172.21.15.73:0/1730141389 learned_addr learned my addr 172.21.15.73:0/1730141389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ffa01e00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0406c630 0x7ffa041f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ffa02800640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040cbe00 0x7ffa0406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 -- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040cbe00 msgr2=0x7ffa0406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040cbe00 0x7ffa0406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 -- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0406c630 msgr2=0x7ffa041f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0406c630 0x7ffa041f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 -- 172.21.15.73:0/1730141389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc009a70 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fbe00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c430 tx=0x7ff9ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0074e0 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec007660 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.028+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec004a20 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.029+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa041f6b50 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.029+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041f61f0 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.030+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa041f2d30 con 0x7ffa040b32b0 2024-04-05T22:25:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.030+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7ff9ec004b80 con 0x7ffa040b32b0 2024-04-05T22:25:51.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.031+0000 7ff9fa000640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 0x7ff9e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.031+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ec0897c0 con 0x7ffa040b32b0 2024-04-05T22:25:51.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.032+0000 7ffa01e00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 0x7ff9e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.032+0000 7ffa01e00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 0x7ff9e006d420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ad0 tx=0x7ff9f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.036+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ffa041f2d30 con 0x7ffa040b32b0 2024-04-05T22:25:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.233+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa041f2d30 con 0x7ffa040b32b0 2024-04-05T22:25:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.234+0000 7ff9fa000640 1 -- 172.21.15.73:0/1730141389 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa041f2d30 con 0x7ffa040b32b0 2024-04-05T22:25:51.234 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:25:51.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 msgr2=0x7ff9e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 0x7ff9e006d420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ad0 tx=0x7ff9f4008040 comp rx=0 tx=0).stop 2024-04-05T22:25:51.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 msgr2=0x7ffa0415a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c430 tx=0x7ff9ec00c8f0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 shutdown_connections 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff9e006af60 0x7ff9e006d420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa040cbe00 0x7ffa0406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa040b32b0 0x7ffa0415a3f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 --2- 172.21.15.73:0/1730141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0406c630 0x7ffa041f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 >> 172.21.15.73:0/1730141389 conn(0x7ffa040b0830 msgr2=0x7ffa0415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 shutdown_connections 2024-04-05T22:25:51.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.237+0000 7ffa08c00640 1 -- 172.21.15.73:0/1730141389 wait complete. 2024-04-05T22:25:51.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:25:51.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.460+0000 7f8e6a600640 1 Processor -- start 2024-04-05T22:25:51.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.460+0000 7f8e6a600640 1 -- start start 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.461+0000 7f8e6a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8e64146490 0x7f8e6406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.461+0000 7f8e6a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 0x7f8e64070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e6a600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e640b3c40 con 0x7f8e640b2f30 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e6a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e640711e0 con 0x7f8e64146490 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e6a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6406c490 con 0x7f8e64070840 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e69200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8e64146490 0x7f8e6406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e69200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8e64146490 0x7f8e6406d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55800/0 (socket says 172.21.15.73:55800) 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e69200640 1 -- 172.21.15.73:0/2776605366 learned_addr learned my addr 172.21.15.73:0/2776605366 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.462+0000 7f8e63e00640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 0x7f8e64070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.463+0000 7f8e69200640 1 -- 172.21.15.73:0/2776605366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 msgr2=0x7f8e64070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.463+0000 7f8e69200640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 0x7f8e64070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.463+0000 7f8e69200640 1 -- 172.21.15.73:0/2776605366 >> v1:172.21.15.157:6789/0 conn(0x7f8e640b2f30 legacy=0x7f8e640b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.463+0000 7f8e69200640 1 -- 172.21.15.73:0/2776605366 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6406c6f0 con 0x7f8e64146490 2024-04-05T22:25:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.463+0000 7f8e63e00640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 0x7f8e64070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.464+0000 7f8e69200640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64146490 0x7f8e6406d4b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009870 tx=0x7f8e5802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14d38884443fa9c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.464+0000 7f8e63400640 1 -- 172.21.15.73:0/2776605366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58030840 con 0x7f8e64146490 2024-04-05T22:25:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e63400640 1 -- 172.21.15.73:0/2776605366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e580309a0 con 0x7f8e64146490 2024-04-05T22:25:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e63400640 1 -- 172.21.15.73:0/2776605366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58030d30 con 0x7f8e64146490 2024-04-05T22:25:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 -- 172.21.15.73:0/2776605366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64146490 msgr2=0x7f8e6406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64146490 0x7f8e6406d4b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009870 tx=0x7f8e5802b980 comp rx=0 tx=0).stop 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 -- 172.21.15.73:0/2776605366 shutdown_connections 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64070840 0x7f8e64070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 --2- 172.21.15.73:0/2776605366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64146490 0x7f8e6406d4b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 -- 172.21.15.73:0/2776605366 >> 172.21.15.73:0/2776605366 conn(0x7f8e640b0830 msgr2=0x7f8e640b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 -- 172.21.15.73:0/2776605366 shutdown_connections 2024-04-05T22:25:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.465+0000 7f8e6a600640 1 -- 172.21.15.73:0/2776605366 wait complete. 2024-04-05T22:25:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.467+0000 7f8e6a600640 1 Processor -- start 2024-04-05T22:25:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.467+0000 7f8e6a600640 1 -- start start 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e640b2f30 0x7f8e6415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64146490 0x7f8e64142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6415f5a0 con 0x7f8e64070840 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6415f880 con 0x7f8e640b2f30 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e6a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e64142450 con 0x7f8e64146490 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e69200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e69200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55810/0 (socket says 172.21.15.73:55810) 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.468+0000 7f8e69200640 1 -- 172.21.15.73:0/3594195104 learned_addr learned my addr 172.21.15.73:0/3594195104 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:51.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 -- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64146490 msgr2=0x7f8e64142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64146490 0x7f8e64142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 -- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e640b2f30 msgr2=0x7f8e6415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e63e00640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e640b2f30 0x7f8e6415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e640b2f30 0x7f8e6415ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 -- 172.21.15.73:0/3594195104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580094d0 con 0x7f8e64070840 2024-04-05T22:25:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.469+0000 7f8e69200640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8e58000c00 tx=0x7f8e58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.470+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58030810 con 0x7f8e64070840 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.470+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e58039d90 con 0x7f8e64070840 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.470+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58043670 con 0x7f8e64070840 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.470+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e64142730 con 0x7f8e64070840 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.470+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6415fb60 con 0x7f8e64070840 2024-04-05T22:25:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.471+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e64142940 con 0x7f8e64070840 2024-04-05T22:25:51.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.472+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f8e5802f390 con 0x7f8e64070840 2024-04-05T22:25:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.472+0000 7f8e60c00640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 0x7f8e4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.473+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e580abda0 con 0x7f8e64070840 2024-04-05T22:25:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.473+0000 7f8e63e00640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 0x7f8e4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.473+0000 7f8e63e00640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 0x7f8e4406d420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e640ad5f0 tx=0x7f8e50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:51.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.477+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8e58076b80 con 0x7f8e64070840 2024-04-05T22:25:51.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.685+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8e6404e160 con 0x7f8e64070840 2024-04-05T22:25:51.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.686+0000 7f8e60c00640 1 -- 172.21.15.73:0/3594195104 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8e58034080 con 0x7f8e64070840 2024-04-05T22:25:51.686 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.688+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 msgr2=0x7f8e4406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.688+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 0x7f8e4406d420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e640ad5f0 tx=0x7f8e50008040 comp rx=0 tx=0).stop 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.688+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 msgr2=0x7f8e641f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8e58000c00 tx=0x7f8e58004730 comp rx=0 tx=0).stop 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 shutdown_connections 2024-04-05T22:25:51.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f8e4406af60 0x7f8e4406d420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e64146490 0x7f8e64142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e640b2f30 0x7f8e6415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 --2- 172.21.15.73:0/3594195104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e64070840 0x7f8e641f2f80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 >> 172.21.15.73:0/3594195104 conn(0x7f8e640b0830 msgr2=0x7f8e6413aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 shutdown_connections 2024-04-05T22:25:51.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:51.689+0000 7f8e6a600640 1 -- 172.21.15.73:0/3594195104 wait complete. 2024-04-05T22:25:51.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:25:51.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:51.708 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:25:51.814 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:25:51.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:25:51.815 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:25:51.867 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:25:51.867 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:51.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:25:51.924 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:25:51.924 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:25:51.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:25:51.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:52.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.197+0000 7f564ba00640 1 Processor -- start 2024-04-05T22:25:52.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.197+0000 7f564ba00640 1 -- start start 2024-04-05T22:25:52.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.198+0000 7f564ba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56441541a0 0x7f56440b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.198+0000 7f564ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564406d710 0x7f56440cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.198+0000 7f564ba00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f564415aef0 0x7f56440cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.198+0000 7f564ba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564415aa60 con 0x7f56441541a0 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f564ba00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440b3b60 con 0x7f564415aef0 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f564ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440701c0 con 0x7f564406d710 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56441541a0 0x7f56440b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56441541a0 0x7f56440b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55824/0 (socket says 172.21.15.73:55824) 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 -- 172.21.15.73:0/1817577919 learned_addr learned my addr 172.21.15.73:0/1817577919 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 -- 172.21.15.73:0/1817577919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564406d710 msgr2=0x7f56440cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 --2- 172.21.15.73:0/1817577919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564406d710 0x7f56440cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 -- 172.21.15.73:0/1817577919 >> v2:172.21.15.157:3300/0 conn(0x7f564415aef0 msgr2=0x7f56440cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 --2- 172.21.15.73:0/1817577919 >> v2:172.21.15.157:3300/0 conn(0x7f564415aef0 0x7f56440cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.199+0000 7f5649000640 1 -- 172.21.15.73:0/1817577919 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440b3880 con 0x7f56441541a0 2024-04-05T22:25:52.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.200+0000 7f5649000640 1 --2- 172.21.15.73:0/1817577919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56441541a0 0x7f56440b3050 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5644071a40 tx=0x7f563802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b88a25c047e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f5643400640 1 -- 172.21.15.73:0/1817577919 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638031870 con 0x7f56441541a0 2024-04-05T22:25:52.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f5643400640 1 -- 172.21.15.73:0/1817577919 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56380319d0 con 0x7f56441541a0 2024-04-05T22:25:52.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f5643400640 1 -- 172.21.15.73:0/1817577919 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638031d60 con 0x7f56441541a0 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 -- 172.21.15.73:0/1817577919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56441541a0 msgr2=0x7f56440b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 --2- 172.21.15.73:0/1817577919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56441541a0 0x7f56440b3050 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5644071a40 tx=0x7f563802b910 comp rx=0 tx=0).stop 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 -- 172.21.15.73:0/1817577919 shutdown_connections 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 --2- 172.21.15.73:0/1817577919 >> v2:172.21.15.157:3300/0 conn(0x7f564415aef0 0x7f56440cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 --2- 172.21.15.73:0/1817577919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564406d710 0x7f56440cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 --2- 172.21.15.73:0/1817577919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56441541a0 0x7f56440b3050 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.201+0000 7f564ba00640 1 -- 172.21.15.73:0/1817577919 >> 172.21.15.73:0/1817577919 conn(0x7f56440ce8e0 msgr2=0x7f56440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.202+0000 7f564ba00640 1 -- 172.21.15.73:0/1817577919 shutdown_connections 2024-04-05T22:25:52.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.202+0000 7f564ba00640 1 -- 172.21.15.73:0/1817577919 wait complete. 2024-04-05T22:25:52.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.203+0000 7f564ba00640 1 Processor -- start 2024-04-05T22:25:52.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.203+0000 7f564ba00640 1 -- start start 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56441541a0 0x7f56441f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564415aef0 0x7f564415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f3710 con 0x7f564406d710 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f3150 con 0x7f56441541a0 2024-04-05T22:25:52.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f564ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f3430 con 0x7f564415aef0 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.204+0000 7f5649000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55840/0 (socket says 172.21.15.73:55840) 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 -- 172.21.15.73:0/3419298069 learned_addr learned my addr 172.21.15.73:0/3419298069 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 -- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564415aef0 msgr2=0x7f564415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564415aef0 0x7f564415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 -- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56441541a0 msgr2=0x7f56441f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56441541a0 0x7f56441f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 -- 172.21.15.73:0/3419298069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380094d0 con 0x7f564406d710 2024-04-05T22:25:52.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5649000640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f56380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.205+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563803ad10 con 0x7f564406d710 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.206+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563803ae70 con 0x7f564406d710 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.206+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638044610 con 0x7f564406d710 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.206+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56441f3a60 con 0x7f564406d710 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.206+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644151950 con 0x7f564406d710 2024-04-05T22:25:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.207+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5644066af0 con 0x7f564406d710 2024-04-05T22:25:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.208+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f563804d450 con 0x7f564406d710 2024-04-05T22:25:52.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.209+0000 7f5640c00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 0x7f562406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.209+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f563802d040 con 0x7f564406d710 2024-04-05T22:25:52.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.209+0000 7f5643e00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 0x7f562406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.210+0000 7f5643e00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 0x7f562406d420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f56440af830 tx=0x7f5630006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.213+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5638076df0 con 0x7f564406d710 2024-04-05T22:25:52.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.448+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56441f38c0 con 0x7f564406d710 2024-04-05T22:25:52.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.449+0000 7f5640c00640 1 -- 172.21.15.73:0/3419298069 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f56441f38c0 con 0x7f564406d710 2024-04-05T22:25:52.450 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:52.450 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:52.450 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 33 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 msgr2=0x7f562406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 0x7f562406d420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f56440af830 tx=0x7f5630006c90 comp rx=0 tx=0).stop 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 msgr2=0x7f5644151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f56380046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 shutdown_connections 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f562406af60 0x7f562406d420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564415aef0 0x7f564415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56441541a0 0x7f56441f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 --2- 172.21.15.73:0/3419298069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f564406d710 0x7f5644151410 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.452+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 >> 172.21.15.73:0/3419298069 conn(0x7f56440ce8e0 msgr2=0x7f56440b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:52.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.453+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 shutdown_connections 2024-04-05T22:25:52.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.453+0000 7f564ba00640 1 -- 172.21.15.73:0/3419298069 wait complete. 2024-04-05T22:25:52.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:52.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.674+0000 7f752e600640 1 Processor -- start 2024-04-05T22:25:52.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.675+0000 7f752e600640 1 -- start start 2024-04-05T22:25:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.676+0000 7f752e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281587d0 0x7f75280cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.676+0000 7f752e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f752815e850 0x7f752815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.676+0000 7f752e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752806f5b0 con 0x7f75280ccd20 2024-04-05T22:25:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.676+0000 7f752e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752815f240 con 0x7f752815e850 2024-04-05T22:25:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.676+0000 7f752e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7528070410 con 0x7f75281587d0 2024-04-05T22:25:52.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.677+0000 7f7526a00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75280ccd20 0x7f752806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:55586/0 (socket says 172.21.15.73:55586) 2024-04-05T22:25:52.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.677+0000 7f7526a00640 1 -- 172.21.15.73:0/1839943436 learned_addr learned my addr 172.21.15.73:0/1839943436 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:52.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.677+0000 7f7527400640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281587d0 0x7f75280cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.677+0000 7f7527e00640 1 --2- 172.21.15.73:0/1839943436 >> v2:172.21.15.157:3300/0 conn(0x7f752815e850 0x7f752815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.678+0000 7f7527e00640 1 -- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281587d0 msgr2=0x7f75280cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:52.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.678+0000 7f7527e00640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281587d0 0x7f75280cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.678+0000 7f7527e00640 1 -- 172.21.15.73:0/1839943436 >> v1:172.21.15.157:6789/0 conn(0x7f75280ccd20 legacy=0x7f752806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.678+0000 7f7527e00640 1 -- 172.21.15.73:0/1839943436 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7528158bb0 con 0x7f752815e850 2024-04-05T22:25:52.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.679+0000 7f7527e00640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752815e850 0x7f752815ed00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d90 tx=0x7f752002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a9310bd703b687 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.680+0000 7f7526000640 1 -- 172.21.15.73:0/1839943436 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752002cc50 con 0x7f752815e850 2024-04-05T22:25:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.680+0000 7f7526000640 1 -- 172.21.15.73:0/1839943436 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752002cdb0 con 0x7f752815e850 2024-04-05T22:25:52.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.680+0000 7f7526000640 1 -- 172.21.15.73:0/1839943436 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520034660 con 0x7f752815e850 2024-04-05T22:25:52.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 -- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752815e850 msgr2=0x7f752815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752815e850 0x7f752815ed00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d90 tx=0x7f752002bea0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 -- 172.21.15.73:0/1839943436 shutdown_connections 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752815e850 0x7f752815ed00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 --2- 172.21.15.73:0/1839943436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281587d0 0x7f75280cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 -- 172.21.15.73:0/1839943436 >> 172.21.15.73:0/1839943436 conn(0x7f75280ce8e0 msgr2=0x7f75280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 -- 172.21.15.73:0/1839943436 shutdown_connections 2024-04-05T22:25:52.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.681+0000 7f752e600640 1 -- 172.21.15.73:0/1839943436 wait complete. 2024-04-05T22:25:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.683+0000 7f752e600640 1 Processor -- start 2024-04-05T22:25:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.683+0000 7f752e600640 1 -- start start 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.684+0000 7f752e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75281587d0 0x7f75280a8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.684+0000 7f752e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f752e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281f1a30 0x7f752815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f752e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281f1fd0 con 0x7f752815e850 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f752e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281f22b0 con 0x7f75281587d0 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f752e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75281f2590 con 0x7f75281f1a30 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55854/0 (socket says 172.21.15.73:55854) 2024-04-05T22:25:52.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 -- 172.21.15.73:0/2723451335 learned_addr learned my addr 172.21.15.73:0/2723451335 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7527400640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75281587d0 0x7f75280a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 -- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281f1a30 msgr2=0x7f752815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281f1a30 0x7f752815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 -- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75281587d0 msgr2=0x7f75280a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75281587d0 0x7f75280a8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.685+0000 7f7526a00640 1 -- 172.21.15.73:0/2723451335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7520009a70 con 0x7f752815e850 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f7526a00640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f751000c4a0 tx=0x7f751000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510005ce0 con 0x7f752815e850 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7510005e40 con 0x7f752815e850 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510007270 con 0x7f752815e850 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75281f2870 con 0x7f752815e850 2024-04-05T22:25:52.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.686+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75281ee390 con 0x7f752815e850 2024-04-05T22:25:52.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.687+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528066af0 con 0x7f752815e850 2024-04-05T22:25:52.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.688+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f75100073d0 con 0x7f752815e850 2024-04-05T22:25:52.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.689+0000 7f751f400640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 0x7f750406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:52.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.689+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7510088d00 con 0x7f752815e850 2024-04-05T22:25:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.690+0000 7f7527400640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 0x7f750406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.690+0000 7f7527400640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 0x7f750406d450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7518009a10 tx=0x7f7518008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.694+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7510053ad0 con 0x7f752815e850 2024-04-05T22:25:52.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.874+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75281f2110 con 0x7f752815e850 2024-04-05T22:25:52.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.878+0000 7f751f400640 1 -- 172.21.15.73:0/2723451335 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f75281f2110 con 0x7f752815e850 2024-04-05T22:25:52.879 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:52.879 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:52.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 msgr2=0x7f750406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 0x7f750406d450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7518009a10 tx=0x7f7518008040 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 msgr2=0x7f75281ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f751000c4a0 tx=0x7f751000c960 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 shutdown_connections 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f750406af90 0x7f750406d450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75281f1a30 0x7f752815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752815e850 0x7f75281ede50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 --2- 172.21.15.73:0/2723451335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75281587d0 0x7f75280a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 >> 172.21.15.73:0/2723451335 conn(0x7f75280ce8e0 msgr2=0x7f752806c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 shutdown_connections 2024-04-05T22:25:52.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:52.881+0000 7f752e600640 1 -- 172.21.15.73:0/2723451335 wait complete. 2024-04-05T22:25:52.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:25:21.730760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:25:22.267497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:25:52.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:25:53.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.103+0000 7f6676800640 1 Processor -- start 2024-04-05T22:25:53.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.103+0000 7f6676800640 1 -- start start 2024-04-05T22:25:53.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.104+0000 7f6676800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.104+0000 7f6676800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f667015e580 0x7f667006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6676800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667006bc50 con 0x7f667015e9a0 2024-04-05T22:25:53.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6676800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667015f530 con 0x7f667015e580 2024-04-05T22:25:53.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6676800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667015ef70 con 0x7f667015a850 2024-04-05T22:25:53.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6675400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6675400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58814/0 (socket says 172.21.15.73:58814) 2024-04-05T22:25:53.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.105+0000 7f6675400640 1 -- 172.21.15.73:0/2957472022 learned_addr learned my addr 172.21.15.73:0/2957472022 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f6674a00640 1 --2- 172.21.15.73:0/2957472022 >> v2:172.21.15.157:3300/0 conn(0x7f667015e580 0x7f667006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f6675400640 1 -- 172.21.15.73:0/2957472022 >> v2:172.21.15.157:3300/0 conn(0x7f667015e580 msgr2=0x7f667006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f6675400640 1 --2- 172.21.15.73:0/2957472022 >> v2:172.21.15.157:3300/0 conn(0x7f667015e580 0x7f667006ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f6675400640 1 -- 172.21.15.73:0/2957472022 >> v1:172.21.15.157:6789/0 conn(0x7f667015e9a0 legacy=0x7f66700b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f6675400640 1 -- 172.21.15.73:0/2957472022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667015f250 con 0x7f667015a850 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.106+0000 7f666fe00640 1 -- 172.21.15.73:0/2957472022 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783408665 0 0) 0x7f667006bc50 con 0x7f667015e9a0 2024-04-05T22:25:53.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f6675400640 1 --2- 172.21.15.73:0/2957472022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6664009870 tx=0x7f666402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140b3e300c12e016 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f666fe00640 1 -- 172.21.15.73:0/2957472022 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664030840 con 0x7f667015a850 2024-04-05T22:25:53.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f666fe00640 1 -- 172.21.15.73:0/2957472022 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66640309a0 con 0x7f667015a850 2024-04-05T22:25:53.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f666fe00640 1 -- 172.21.15.73:0/2957472022 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664030d30 con 0x7f667015a850 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f6676800640 1 -- 172.21.15.73:0/2957472022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 msgr2=0x7f667015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.108+0000 7f6676800640 1 --2- 172.21.15.73:0/2957472022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6664009870 tx=0x7f666402b980 comp rx=0 tx=0).stop 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 -- 172.21.15.73:0/2957472022 shutdown_connections 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 --2- 172.21.15.73:0/2957472022 >> v2:172.21.15.157:3300/0 conn(0x7f667015e580 0x7f667006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 --2- 172.21.15.73:0/2957472022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015a850 0x7f667015ac30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 -- 172.21.15.73:0/2957472022 >> 172.21.15.73:0/2957472022 conn(0x7f6670069cb0 msgr2=0x7f667006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 -- 172.21.15.73:0/2957472022 shutdown_connections 2024-04-05T22:25:53.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.109+0000 7f6676800640 1 -- 172.21.15.73:0/2957472022 wait complete. 2024-04-05T22:25:53.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.110+0000 7f6676800640 1 Processor -- start 2024-04-05T22:25:53.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 -- start start 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 0x7f6670157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015e9a0 0x7f66700acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6670157170 con 0x7f667015e580 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6670157ad0 con 0x7f667015a850 2024-04-05T22:25:53.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.111+0000 7f6676800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66701564f0 con 0x7f667015e9a0 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55870/0 (socket says 172.21.15.73:55870) 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 -- 172.21.15.73:0/3719498738 learned_addr learned my addr 172.21.15.73:0/3719498738 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6675400640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 0x7f6670157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 -- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015e9a0 msgr2=0x7f66700acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015e9a0 0x7f66700acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 -- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 msgr2=0x7f6670157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 0x7f6670157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 -- 172.21.15.73:0/3719498738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640094d0 con 0x7f667015e580 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6675400640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 0x7f6670157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:53.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.112+0000 7f6674a00640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e9b0 tx=0x7f665c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.113+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c009670 con 0x7f667015e580 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.113+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00c9a0 con 0x7f667015e580 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.113+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cd50 con 0x7f667015e580 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.113+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66701567d0 con 0x7f667015e580 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.113+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6670156d90 con 0x7f667015e580 2024-04-05T22:25:53.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.114+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667004e160 con 0x7f667015e580 2024-04-05T22:25:53.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.115+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f665c013470 con 0x7f667015e580 2024-04-05T22:25:53.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.116+0000 7f666d600640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 0x7f665006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.116+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f665c089ae0 con 0x7f667015e580 2024-04-05T22:25:53.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.116+0000 7f6675400640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 0x7f665006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.117+0000 7f6675400640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 0x7f665006d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f666403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.120+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f665c054900 con 0x7f667015e580 2024-04-05T22:25:53.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.313+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66700b1f80 con 0x7f667015e580 2024-04-05T22:25:53.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.315+0000 7f666d600640 1 -- 172.21.15.73:0/3719498738 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f665c0170a0 con 0x7f667015e580 2024-04-05T22:25:53.315 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 msgr2=0x7f665006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 0x7f665006d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f666403b040 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 msgr2=0x7f6670140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e9b0 tx=0x7f665c00ee70 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 shutdown_connections 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f665006af60 0x7f665006d420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667015e9a0 0x7f66700acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f667015e580 0x7f6670140fb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 --2- 172.21.15.73:0/3719498738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667015a850 0x7f6670157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.317+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 >> 172.21.15.73:0/3719498738 conn(0x7f6670069cb0 msgr2=0x7f66700af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.318+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 shutdown_connections 2024-04-05T22:25:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.318+0000 7f6676800640 1 -- 172.21.15.73:0/3719498738 wait complete. 2024-04-05T22:25:53.336 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:25:53.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.543+0000 7fcd81800640 1 Processor -- start 2024-04-05T22:25:53.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.543+0000 7fcd81800640 1 -- start start 2024-04-05T22:25:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd81800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd7c06c9a0 0x7fcd7c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd81800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd7c06ba70 0x7fcd7c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd81800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c06bea0 con 0x7fcd7c15abe0 2024-04-05T22:25:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd81800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c15afc0 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd81800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c06f5b0 con 0x7fcd7c06ba70 2024-04-05T22:25:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd7c06c9a0 0x7fcd7c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd7c06c9a0 0x7fcd7c06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55874/0 (socket says 172.21.15.73:55874) 2024-04-05T22:25:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7a000640 1 -- 172.21.15.73:0/1651486523 learned_addr learned my addr 172.21.15.73:0/1651486523 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7aa00640 1 --1- 172.21.15.73:0/1651486523 >> v1:172.21.15.73:6789/0 conn(0x7fcd7c15abe0 0x7fcd7c0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51880/0 (socket says 172.21.15.73:51880) 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7a000640 1 -- 172.21.15.73:0/1651486523 >> v2:172.21.15.157:3300/0 conn(0x7fcd7c06ba70 msgr2=0x7fcd7c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.545+0000 7fcd7a000640 1 --2- 172.21.15.73:0/1651486523 >> v2:172.21.15.157:3300/0 conn(0x7fcd7c06ba70 0x7fcd7c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.546+0000 7fcd7a000640 1 -- 172.21.15.73:0/1651486523 >> v1:172.21.15.73:6789/0 conn(0x7fcd7c15abe0 legacy=0x7fcd7c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.546+0000 7fcd7a000640 1 -- 172.21.15.73:0/1651486523 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c06f890 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd7a000640 1 --2- 172.21.15.73:0/1651486523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c06b530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c06c900 tx=0x7fcd6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4337c28ef75980 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd79600640 1 -- 172.21.15.73:0/1651486523 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd680317e0 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd79600640 1 -- 172.21.15.73:0/1651486523 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68031940 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd79600640 1 -- 172.21.15.73:0/1651486523 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68031cd0 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 -- 172.21.15.73:0/1651486523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06c9a0 msgr2=0x7fcd7c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 --2- 172.21.15.73:0/1651486523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c06b530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c06c900 tx=0x7fcd6802b880 comp rx=0 tx=0).stop 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 -- 172.21.15.73:0/1651486523 shutdown_connections 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 --2- 172.21.15.73:0/1651486523 >> v2:172.21.15.157:3300/0 conn(0x7fcd7c06ba70 0x7fcd7c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 --2- 172.21.15.73:0/1651486523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c06b530 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.547+0000 7fcd81800640 1 -- 172.21.15.73:0/1651486523 >> 172.21.15.73:0/1651486523 conn(0x7fcd7c069cb0 msgr2=0x7fcd7c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.548+0000 7fcd81800640 1 -- 172.21.15.73:0/1651486523 shutdown_connections 2024-04-05T22:25:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.548+0000 7fcd81800640 1 -- 172.21.15.73:0/1651486523 wait complete. 2024-04-05T22:25:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.549+0000 7fcd81800640 1 Processor -- start 2024-04-05T22:25:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.549+0000 7fcd81800640 1 -- start start 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7c15abe0 0x7fcd7c152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c153d70 con 0x7fcd7c06ba70 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1537b0 con 0x7fcd7c06c9a0 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd81800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c153a90 con 0x7fcd7c15abe0 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55888/0 (socket says 172.21.15.73:55888) 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.550+0000 7fcd7aa00640 1 -- 172.21.15.73:0/2760666348 learned_addr learned my addr 172.21.15.73:0/2760666348 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7a000640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 -- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7c15abe0 msgr2=0x7fcd7c152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7b400640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7c15abe0 0x7fcd7c152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7c15abe0 0x7fcd7c152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 -- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7c06c9a0 msgr2=0x7fcd7c1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c1508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 -- 172.21.15.73:0/2760666348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680094d0 con 0x7fcd7c06ba70 2024-04-05T22:25:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd7aa00640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c3c0 tx=0x7fcd6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0074e0 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c007660 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.551+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c004a20 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.552+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c14e280 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.552+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c15bd10 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.553+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c14e7a0 con 0x7fcd7c06ba70 2024-04-05T22:25:53.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.554+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fcd6c004b80 con 0x7fcd7c06ba70 2024-04-05T22:25:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.555+0000 7fcd72a00640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 0x7fcd5c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.555+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd6c0891f0 con 0x7fcd7c06ba70 2024-04-05T22:25:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.555+0000 7fcd7a000640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 0x7fcd5c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.556+0000 7fcd7a000640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 0x7fcd5c06d420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcd68036000 tx=0x7fcd6803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.559+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcd6c054050 con 0x7fcd7c06ba70 2024-04-05T22:25:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.738+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd7c068160 con 0x7fcd7c06ba70 2024-04-05T22:25:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.739+0000 7fcd72a00640 1 -- 172.21.15.73:0/2760666348 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd6c057670 con 0x7fcd7c06ba70 2024-04-05T22:25:53.739 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:25:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 msgr2=0x7fcd5c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 0x7fcd5c06d420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcd68036000 tx=0x7fcd6803d040 comp rx=0 tx=0).stop 2024-04-05T22:25:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 msgr2=0x7fcd7c150380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c3c0 tx=0x7fcd6c00c880 comp rx=0 tx=0).stop 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 shutdown_connections 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcd5c06af60 0x7fcd5c06d420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7c15abe0 0x7fcd7c152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7c06c9a0 0x7fcd7c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 --2- 172.21.15.73:0/2760666348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd7c06ba70 0x7fcd7c150380 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 >> 172.21.15.73:0/2760666348 conn(0x7fcd7c069cb0 msgr2=0x7fcd7c0ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 shutdown_connections 2024-04-05T22:25:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.742+0000 7fcd81800640 1 -- 172.21.15.73:0/2760666348 wait complete. 2024-04-05T22:25:53.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:25:53.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.968+0000 7f2b33200640 1 Processor -- start 2024-04-05T22:25:53.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.968+0000 7f2b33200640 1 -- start start 2024-04-05T22:25:53.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.969+0000 7f2b33200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b2c15ad30 0x7f2b2c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.969+0000 7f2b33200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2b2c0cde30 0x7f2b2c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.969+0000 7f2b33200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0cf170 con 0x7f2b2c0b4440 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.969+0000 7f2b33200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0ce2c0 con 0x7f2b2c0cde30 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.969+0000 7f2b33200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0ce560 con 0x7f2b2c15ad30 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2b2c0cde30 0x7f2b2c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2b2c0cde30 0x7f2b2c15b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55890/0 (socket says 172.21.15.73:55890) 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 -- 172.21.15.73:0/1131687636 learned_addr learned my addr 172.21.15.73:0/1131687636 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 -- 172.21.15.73:0/1131687636 >> v2:172.21.15.157:3300/0 conn(0x7f2b2c15ad30 msgr2=0x7f2b2c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 --2- 172.21.15.73:0/1131687636 >> v2:172.21.15.157:3300/0 conn(0x7f2b2c15ad30 0x7f2b2c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 -- 172.21.15.73:0/1131687636 >> v1:172.21.15.157:6789/0 conn(0x7f2b2c0b4440 legacy=0x7f2b2c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:53.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.970+0000 7f2b2b400640 1 -- 172.21.15.73:0/1131687636 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0ce840 con 0x7f2b2c0cde30 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.971+0000 7f2b2b400640 1 --2- 172.21.15.73:0/1131687636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0cde30 0x7f2b2c15b110 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0725f0 tx=0x7f2b1c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eee50ac442db3df server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.971+0000 7f2b2aa00640 1 -- 172.21.15.73:0/1131687636 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c02c9f0 con 0x7f2b2c0cde30 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.971+0000 7f2b2aa00640 1 -- 172.21.15.73:0/1131687636 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02cb50 con 0x7f2b2c0cde30 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.971+0000 7f2b2aa00640 1 -- 172.21.15.73:0/1131687636 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c0343f0 con 0x7f2b2c0cde30 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 -- 172.21.15.73:0/1131687636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0cde30 msgr2=0x7f2b2c15b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 --2- 172.21.15.73:0/1131687636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0cde30 0x7f2b2c15b110 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0725f0 tx=0x7f2b1c02be50 comp rx=0 tx=0).stop 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 -- 172.21.15.73:0/1131687636 shutdown_connections 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 --2- 172.21.15.73:0/1131687636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0cde30 0x7f2b2c15b110 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 --2- 172.21.15.73:0/1131687636 >> v2:172.21.15.157:3300/0 conn(0x7f2b2c15ad30 0x7f2b2c0b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 -- 172.21.15.73:0/1131687636 >> 172.21.15.73:0/1131687636 conn(0x7f2b2c099a60 msgr2=0x7f2b2c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 -- 172.21.15.73:0/1131687636 shutdown_connections 2024-04-05T22:25:53.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.972+0000 7f2b33200640 1 -- 172.21.15.73:0/1131687636 wait complete. 2024-04-05T22:25:53.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.975+0000 7f2b33200640 1 Processor -- start 2024-04-05T22:25:53.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.975+0000 7f2b33200640 1 -- start start 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 0x7f2b2c1f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 0x7f2b2c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c09d1c0 con 0x7f2b2c0b4440 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c14aa10 con 0x7f2b2c15ad30 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b33200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c1f4830 con 0x7f2b2c0cde30 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55900/0 (socket says 172.21.15.73:55900) 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.976+0000 7f2b2be00640 1 -- 172.21.15.73:0/623379184 learned_addr learned my addr 172.21.15.73:0/623379184 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2b400640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 0x7f2b2c1f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b31200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 0x7f2b2c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 -- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 msgr2=0x7f2b2c1f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 0x7f2b2c1f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 -- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 msgr2=0x7f2b2c15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 0x7f2b2c15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 -- 172.21.15.73:0/623379184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009b30 con 0x7f2b2c0b4440 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2b400640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 0x7f2b2c1f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b31200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 0x7f2b2c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b2be00640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400b410 tx=0x7f2b1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011830 con 0x7f2b2c0b4440 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b140119b0 con 0x7f2b2c0b4440 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011d60 con 0x7f2b2c0b4440 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.977+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c1f4b10 con 0x7f2b2c0b4440 2024-04-05T22:25:53.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.978+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c09ca70 con 0x7f2b2c0b4440 2024-04-05T22:25:53.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.979+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0cf5e0 con 0x7f2b2c0b4440 2024-04-05T22:25:53.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.980+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f2b1401a400 con 0x7f2b2c0b4440 2024-04-05T22:25:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.981+0000 7f2b23e00640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 0x7f2b0806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.981+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b14089a80 con 0x7f2b2c0b4440 2024-04-05T22:25:53.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.981+0000 7f2b2b400640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 0x7f2b0806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:53.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.982+0000 7f2b2b400640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 0x7f2b0806d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c035000 tx=0x7f2b1c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:53.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:53.986+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2b2c0cf5e0 con 0x7f2b2c0b4440 2024-04-05T22:25:54.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.180+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b2c0cf5e0 con 0x7f2b2c0b4440 2024-04-05T22:25:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.181+0000 7f2b23e00640 1 -- 172.21.15.73:0/623379184 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b2c0cf5e0 con 0x7f2b2c0b4440 2024-04-05T22:25:54.181 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 msgr2=0x7f2b0806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 0x7f2b0806d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c035000 tx=0x7f2b1c030000 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 msgr2=0x7f2b2c1f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400b410 tx=0x7f2b1400b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 shutdown_connections 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2b0806af60 0x7f2b0806d420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c15ad30 0x7f2b2c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0cde30 0x7f2b2c1f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 --2- 172.21.15.73:0/623379184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c0b4440 0x7f2b2c1f3cc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.183+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 >> 172.21.15.73:0/623379184 conn(0x7f2b2c099a60 msgr2=0x7f2b2c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.184+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 shutdown_connections 2024-04-05T22:25:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.184+0000 7f2b33200640 1 -- 172.21.15.73:0/623379184 wait complete. 2024-04-05T22:25:54.203 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:54.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.410+0000 7fde10600640 1 Processor -- start 2024-04-05T22:25:54.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.410+0000 7fde10600640 1 -- start start 2024-04-05T22:25:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.411+0000 7fde10600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde0815ce20 0x7fde080c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.411+0000 7fde10600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fde080ca050 0x7fde080acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.411+0000 7fde10600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0806aa40 con 0x7fde0815d200 2024-04-05T22:25:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.411+0000 7fde10600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0809d110 con 0x7fde080ca050 2024-04-05T22:25:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.411+0000 7fde10600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0806ad20 con 0x7fde0815ce20 2024-04-05T22:25:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fde080ca050 0x7fde080acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fde080ca050 0x7fde080acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55906/0 (socket says 172.21.15.73:55906) 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 -- 172.21.15.73:0/1410580849 learned_addr learned my addr 172.21.15.73:0/1410580849 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0dc00640 1 --2- 172.21.15.73:0/1410580849 >> v2:172.21.15.157:3300/0 conn(0x7fde0815ce20 0x7fde080c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 -- 172.21.15.73:0/1410580849 >> v2:172.21.15.157:3300/0 conn(0x7fde0815ce20 msgr2=0x7fde080c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 --2- 172.21.15.73:0/1410580849 >> v2:172.21.15.157:3300/0 conn(0x7fde0815ce20 0x7fde080c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 -- 172.21.15.73:0/1410580849 >> v1:172.21.15.157:6789/0 conn(0x7fde0815d200 legacy=0x7fde080ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.412+0000 7fde0e600640 1 -- 172.21.15.73:0/1410580849 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080cde30 con 0x7fde080ca050 2024-04-05T22:25:54.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.414+0000 7fde0e600640 1 --2- 172.21.15.73:0/1410580849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde080acfd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fde080cf0f0 tx=0x7fde0002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b114302004790c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.414+0000 7fde07e00640 1 -- 172.21.15.73:0/1410580849 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0002cb20 con 0x7fde080ca050 2024-04-05T22:25:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.414+0000 7fde07e00640 1 -- 172.21.15.73:0/1410580849 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0002cc80 con 0x7fde080ca050 2024-04-05T22:25:54.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.414+0000 7fde07e00640 1 -- 172.21.15.73:0/1410580849 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000343f0 con 0x7fde080ca050 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 -- 172.21.15.73:0/1410580849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 msgr2=0x7fde080acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 --2- 172.21.15.73:0/1410580849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde080acfd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fde080cf0f0 tx=0x7fde0002bf80 comp rx=0 tx=0).stop 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 -- 172.21.15.73:0/1410580849 shutdown_connections 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 --2- 172.21.15.73:0/1410580849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde080acfd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 --2- 172.21.15.73:0/1410580849 >> v2:172.21.15.157:3300/0 conn(0x7fde0815ce20 0x7fde080c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 -- 172.21.15.73:0/1410580849 >> 172.21.15.73:0/1410580849 conn(0x7fde080cf190 msgr2=0x7fde080cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 -- 172.21.15.73:0/1410580849 shutdown_connections 2024-04-05T22:25:54.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.415+0000 7fde10600640 1 -- 172.21.15.73:0/1410580849 wait complete. 2024-04-05T22:25:54.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.417+0000 7fde10600640 1 Processor -- start 2024-04-05T22:25:54.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.417+0000 7fde10600640 1 -- start start 2024-04-05T22:25:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde0815ce20 0x7fde0815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0815d200 0x7fde0815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0815ed40 con 0x7fde080ca050 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0815f020 con 0x7fde0815d200 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde10600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0815f300 con 0x7fde0815ce20 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55910/0 (socket says 172.21.15.73:55910) 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 -- 172.21.15.73:0/815137272 learned_addr learned my addr 172.21.15.73:0/815137272 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 -- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde0815ce20 msgr2=0x7fde0815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde0815ce20 0x7fde0815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 -- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0815d200 msgr2=0x7fde0815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0815d200 0x7fde0815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.418+0000 7fde0dc00640 1 -- 172.21.15.73:0/815137272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00009b30 con 0x7fde080ca050 2024-04-05T22:25:54.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde0dc00640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d800 tx=0x7fddfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b6c0 con 0x7fde080ca050 2024-04-05T22:25:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00b820 con 0x7fde080ca050 2024-04-05T22:25:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00bbb0 con 0x7fde080ca050 2024-04-05T22:25:54.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde080ad690 con 0x7fde080ca050 2024-04-05T22:25:54.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.419+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0815f8c0 con 0x7fde080ca050 2024-04-05T22:25:54.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.420+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0815fa50 con 0x7fde080ca050 2024-04-05T22:25:54.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.421+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fddfc00bd10 con 0x7fde080ca050 2024-04-05T22:25:54.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.422+0000 7fde05600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 0x7fdde806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.422+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddfc089d70 con 0x7fde080ca050 2024-04-05T22:25:54.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.422+0000 7fde0d200640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 0x7fdde806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.423+0000 7fde0d200640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 0x7fdde806d450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fddf4000f60 tx=0x7fddf40079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.426+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fddfc08e070 con 0x7fde080ca050 2024-04-05T22:25:54.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.649+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde08068160 con 0x7fde080ca050 2024-04-05T22:25:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.650+0000 7fde05600640 1 -- 172.21.15.73:0/815137272 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fddfc054bd0 con 0x7fde080ca050 2024-04-05T22:25:54.650 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:54.650 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 33 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 msgr2=0x7fdde806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 0x7fdde806d450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fddf4000f60 tx=0x7fddf40079b0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 msgr2=0x7fde0815fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d800 tx=0x7fddfc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 shutdown_connections 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdde806af90 0x7fdde806d450 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde0815d200 0x7fde0815e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde0815ce20 0x7fde0815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 --2- 172.21.15.73:0/815137272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde080ca050 0x7fde0815fde0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 >> 172.21.15.73:0/815137272 conn(0x7fde080cf190 msgr2=0x7fde0806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:54.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 shutdown_connections 2024-04-05T22:25:54.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.652+0000 7fde10600640 1 -- 172.21.15.73:0/815137272 wait complete. 2024-04-05T22:25:54.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:54.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.871+0000 7fe24ca00640 1 Processor -- start 2024-04-05T22:25:54.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.871+0000 7fe24ca00640 1 -- start start 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe24ca00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe248159f40 0x7fe24806be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe24ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480cd7b0 con 0x7fe248065b10 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe24ca00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480cd230 con 0x7fe24806a090 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe24ca00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2480ad740 con 0x7fe248159f40 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe23d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe248065b10 0x7fe24806c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51884/0 (socket says 172.21.15.73:51884) 2024-04-05T22:25:54.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.873+0000 7fe23d400640 1 -- 172.21.15.73:0/3098234434 learned_addr learned my addr 172.21.15.73:0/3098234434 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695912604 0 0) 0x7fe2480cd7b0 con 0x7fe248065b10 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c003520 con 0x7fe248065b10 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245e00640 1 --2- 172.21.15.73:0/3098234434 >> v2:172.21.15.157:3300/0 conn(0x7fe248159f40 0x7fe24806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314763367 0 0) 0x7fe22c003520 con 0x7fe248065b10 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 >> v2:172.21.15.157:3300/0 conn(0x7fe248159f40 msgr2=0x7fe24806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 --2- 172.21.15.73:0/3098234434 >> v2:172.21.15.157:3300/0 conn(0x7fe248159f40 0x7fe24806be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 >> v1:172.21.15.157:6789/0 conn(0x7fe24806a090 legacy=0x7fe24806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24815a320 con 0x7fe248065b10 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe230001670 con 0x7fe248065b10 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 >> v1:172.21.15.73:6789/0 conn(0x7fe248065b10 legacy=0x7fe24806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480ad740 con 0x7fe22c005180 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00c0d0 con 0x7fe22c003970 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.874+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00c290 con 0x7fe22c0047c0 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe23d400640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 -- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22c0047c0 msgr2=0x7fe22c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe245e00640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 -- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c003970 msgr2=0x7fe22c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:54.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.875+0000 7fe246800640 1 -- 172.21.15.73:0/3098234434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c003520 con 0x7fe22c005180 2024-04-05T22:25:54.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.876+0000 7fe246800640 1 -- 172.21.15.73:0/3098234434 reap_dead start 2024-04-05T22:25:54.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.876+0000 7fe246800640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 0x7fe22c00b980 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe240001e50 tx=0x7fe22c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f589fec6974a1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.876+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240003b70 con 0x7fe22c005180 2024-04-05T22:25:54.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240003d10 con 0x7fe22c005180 2024-04-05T22:25:54.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe245400640 1 -- 172.21.15.73:0/3098234434 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240005310 con 0x7fe22c005180 2024-04-05T22:25:54.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 -- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 msgr2=0x7fe22c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 0x7fe22c00b980 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe240001e50 tx=0x7fe22c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 -- 172.21.15.73:0/3098234434 shutdown_connections 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3098234434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 -- 172.21.15.73:0/3098234434 >> 172.21.15.73:0/3098234434 conn(0x7fe2480cf190 msgr2=0x7fe2480cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 -- 172.21.15.73:0/3098234434 shutdown_connections 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.877+0000 7fe24ca00640 1 -- 172.21.15.73:0/3098234434 wait complete. 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.879+0000 7fe24ca00640 1 Processor -- start 2024-04-05T22:25:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.879+0000 7fe24ca00640 1 -- start start 2024-04-05T22:25:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c005180 0x7fe248153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe24815b7f0 0x7fe24815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480c8590 con 0x7fe2480c7bf0 2024-04-05T22:25:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24815a670 con 0x7fe22c005180 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe24ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24815a910 con 0x7fe24815b7f0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe246800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe246800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55926/0 (socket says 172.21.15.73:55926) 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.880+0000 7fe246800640 1 -- 172.21.15.73:0/3111873795 learned_addr learned my addr 172.21.15.73:0/3111873795 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe245e00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c005180 0x7fe248153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe23d400640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe24815b7f0 0x7fe24815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 -- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe24815b7f0 msgr2=0x7fe24815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe24815b7f0 0x7fe24815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 -- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c005180 msgr2=0x7fe248153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c005180 0x7fe248153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 -- 172.21.15.73:0/3111873795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24815abf0 con 0x7fe2480c7bf0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe246800640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe240001f70 tx=0x7fe2400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240006d50 con 0x7fe2480c7bf0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240007030 con 0x7fe2480c7bf0 2024-04-05T22:25:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.881+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24815aed0 con 0x7fe2480c7bf0 2024-04-05T22:25:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.882+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400073a0 con 0x7fe2480c7bf0 2024-04-05T22:25:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.882+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2480c8c20 con 0x7fe2480c7bf0 2024-04-05T22:25:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.883+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248068160 con 0x7fe2480c7bf0 2024-04-05T22:25:54.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.884+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe240007500 con 0x7fe2480c7bf0 2024-04-05T22:25:54.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.884+0000 7fe23ea00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:54.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.885+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2400ab5c0 con 0x7fe2480c7bf0 2024-04-05T22:25:54.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.885+0000 7fe245e00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:54.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.885+0000 7fe245e00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 0x7fe22406d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe238006fd0 tx=0x7fe238008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:54.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:54.889+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe240076140 con 0x7fe2480c7bf0 2024-04-05T22:25:55.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.080+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2480b5110 con 0x7fe2480c7bf0 2024-04-05T22:25:55.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.083+0000 7fe23ea00640 1 -- 172.21.15.73:0/3111873795 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7fe240079760 con 0x7fe2480c7bf0 2024-04-05T22:25:55.084 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:55.084 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.085+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 msgr2=0x7fe22406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 0x7fe22406d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe238006fd0 tx=0x7fe238008040 comp rx=0 tx=0).stop 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 msgr2=0x7fe2480c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe240001f70 tx=0x7fe2400014a0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 shutdown_connections 2024-04-05T22:25:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2480c7bf0 0x7fe2480c8050 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe24815b7f0 0x7fe24815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 --2- 172.21.15.73:0/3111873795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c005180 0x7fe248153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 >> 172.21.15.73:0/3111873795 conn(0x7fe2480cf190 msgr2=0x7fe2480abfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 shutdown_connections 2024-04-05T22:25:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.086+0000 7fe24ca00640 1 -- 172.21.15.73:0/3111873795 wait complete. 2024-04-05T22:25:55.104 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:25:55.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.308+0000 7fb0dfe00640 1 Processor -- start 2024-04-05T22:25:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.308+0000 7fb0dfe00640 1 -- start start 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0e00d1840 0x7fb0e00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0dfe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00b6070 con 0x7fb0e00d0740 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0dfe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00b5a60 con 0x7fb0e015cef0 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0dfe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00d0bd0 con 0x7fb0e00d1840 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0df400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0e00d1840 0x7fb0e00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0df400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0e00d1840 0x7fb0e00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55932/0 (socket says 172.21.15.73:55932) 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.309+0000 7fb0df400640 1 -- 172.21.15.73:0/205844746 learned_addr learned my addr 172.21.15.73:0/205844746 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.310+0000 7fb0df400640 1 -- 172.21.15.73:0/205844746 >> v1:172.21.15.157:6789/0 conn(0x7fb0e015cef0 legacy=0x7fb0e00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.310+0000 7fb0df400640 1 -- 172.21.15.73:0/205844746 >> v1:172.21.15.73:6789/0 conn(0x7fb0e00d0740 legacy=0x7fb0e00b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.310+0000 7fb0df400640 1 -- 172.21.15.73:0/205844746 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00b5d00 con 0x7fb0e00d1840 2024-04-05T22:25:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.311+0000 7fb0df400640 1 --2- 172.21.15.73:0/205844746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e00b6c10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009870 tx=0x7fb0d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca22fd52d5c1d12d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.311+0000 7fb0dd600640 1 -- 172.21.15.73:0/205844746 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8030860 con 0x7fb0e00d1840 2024-04-05T22:25:55.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.311+0000 7fb0dd600640 1 -- 172.21.15.73:0/205844746 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80309c0 con 0x7fb0e00d1840 2024-04-05T22:25:55.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.311+0000 7fb0dd600640 1 -- 172.21.15.73:0/205844746 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8030d50 con 0x7fb0e00d1840 2024-04-05T22:25:55.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 -- 172.21.15.73:0/205844746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 msgr2=0x7fb0e00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/205844746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e00b6c10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009870 tx=0x7fb0d802b9a0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 -- 172.21.15.73:0/205844746 shutdown_connections 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/205844746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e00b6c10 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 -- 172.21.15.73:0/205844746 >> 172.21.15.73:0/205844746 conn(0x7fb0e009be40 msgr2=0x7fb0e00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 -- 172.21.15.73:0/205844746 shutdown_connections 2024-04-05T22:25:55.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.312+0000 7fb0dfe00640 1 -- 172.21.15.73:0/205844746 wait complete. 2024-04-05T22:25:55.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.314+0000 7fb0dfe00640 1 Processor -- start 2024-04-05T22:25:55.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.314+0000 7fb0dfe00640 1 -- start start 2024-04-05T22:25:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e015cef0 0x7fb0e01f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e01f27d0 0x7fb0e00cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c6210 con 0x7fb0e00d1840 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00cac50 con 0x7fb0e015cef0 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c6420 con 0x7fb0e01f27d0 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55936/0 (socket says 172.21.15.73:55936) 2024-04-05T22:25:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.315+0000 7fb0dea00640 1 -- 172.21.15.73:0/1863756943 learned_addr learned my addr 172.21.15.73:0/1863756943 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 -- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e01f27d0 msgr2=0x7fb0e00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0df400640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e01f27d0 0x7fb0e00cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e01f27d0 0x7fb0e00cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 -- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e015cef0 msgr2=0x7fb0e01f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e015cef0 0x7fb0e01f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 -- 172.21.15.73:0/1863756943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80094d0 con 0x7fb0e00d1840 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.316+0000 7fb0dea00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ba80 tx=0x7fb0d000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.317+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000cc00 con 0x7fb0e00d1840 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.317+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d000cd60 con 0x7fb0e00d1840 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.317+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0003bb0 con 0x7fb0e00d1840 2024-04-05T22:25:55.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.317+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00c6700 con 0x7fb0e00d1840 2024-04-05T22:25:55.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.317+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00c6cc0 con 0x7fb0e00d1840 2024-04-05T22:25:55.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.318+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e004e160 con 0x7fb0e00d1840 2024-04-05T22:25:55.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.319+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fb0d0014440 con 0x7fb0e00d1840 2024-04-05T22:25:55.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.320+0000 7fb0d6a00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 0x7fb0bc06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.320+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0d0089620 con 0x7fb0e00d1840 2024-04-05T22:25:55.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.320+0000 7fb0de000640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 0x7fb0bc06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.321+0000 7fb0de000640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 0x7fb0bc06d470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000f60 tx=0x7fb0c80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.324+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb0d0054460 con 0x7fb0e00d1840 2024-04-05T22:25:55.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.556+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0e015fda0 con 0x7fb0e00d1840 2024-04-05T22:25:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.557+0000 7fb0d6a00640 1 -- 172.21.15.73:0/1863756943 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fb0d0057a80 con 0x7fb0e00d1840 2024-04-05T22:25:55.558 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:55.558 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 33 2024-04-05T22:25:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 msgr2=0x7fb0bc06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 0x7fb0bc06d470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000f60 tx=0x7fb0c80079b0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 msgr2=0x7fb0e0158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ba80 tx=0x7fb0d000bf40 comp rx=0 tx=0).stop 2024-04-05T22:25:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 shutdown_connections 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb0bc06afb0 0x7fb0bc06d470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e01f27d0 0x7fb0e00cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e015cef0 0x7fb0e01f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 --2- 172.21.15.73:0/1863756943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e00d1840 0x7fb0e0158fe0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 >> 172.21.15.73:0/1863756943 conn(0x7fb0e009be40 msgr2=0x7fb0e00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 shutdown_connections 2024-04-05T22:25:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.560+0000 7fb0dfe00640 1 -- 172.21.15.73:0/1863756943 wait complete. 2024-04-05T22:25:55.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:25:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.787+0000 7f2cece00640 1 Processor -- start 2024-04-05T22:25:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.787+0000 7f2cece00640 1 -- start start 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2cece00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ce813f3b0 0x7f2ce80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2cece00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce806ca50 con 0x7f2ce80b2210 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2cece00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce806cd30 con 0x7f2ce806c4a0 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2cece00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce806f920 con 0x7f2ce813f3b0 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2ce7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ce813f3b0 0x7f2ce80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2ce6a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ce80b2210 0x7f2ce806bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51902/0 (socket says 172.21.15.73:51902) 2024-04-05T22:25:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.789+0000 7f2ce6a00640 1 -- 172.21.15.73:0/295522777 learned_addr learned my addr 172.21.15.73:0/295522777 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.790+0000 7f2ce7400640 1 -- 172.21.15.73:0/295522777 >> v1:172.21.15.157:6789/0 conn(0x7f2ce806c4a0 legacy=0x7f2ce806fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.790+0000 7f2ce7400640 1 -- 172.21.15.73:0/295522777 >> v1:172.21.15.73:6789/0 conn(0x7f2ce80b2210 legacy=0x7f2ce806bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.790+0000 7f2ce7400640 1 -- 172.21.15.73:0/295522777 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce806d010 con 0x7f2ce813f3b0 2024-04-05T22:25:55.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.790+0000 7f2ce6000640 1 -- 172.21.15.73:0/295522777 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040724613 0 0) 0x7f2ce806ca50 con 0x7f2ce80b2210 2024-04-05T22:25:55.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2ce7400640 1 --2- 172.21.15.73:0/295522777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce80b1cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800a940 tx=0x7f2cd802cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd7c62b2ac90258 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2ce6000640 1 -- 172.21.15.73:0/295522777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802dab0 con 0x7f2ce813f3b0 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2ce6000640 1 -- 172.21.15.73:0/295522777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd802dc10 con 0x7f2ce813f3b0 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2ce6000640 1 -- 172.21.15.73:0/295522777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8035440 con 0x7f2ce813f3b0 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2cece00640 1 -- 172.21.15.73:0/295522777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 msgr2=0x7f2ce80b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2cece00640 1 --2- 172.21.15.73:0/295522777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce80b1cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800a940 tx=0x7f2cd802cd00 comp rx=0 tx=0).stop 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.791+0000 7f2cece00640 1 -- 172.21.15.73:0/295522777 shutdown_connections 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.792+0000 7f2cece00640 1 --2- 172.21.15.73:0/295522777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce80b1cd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.792+0000 7f2cece00640 1 -- 172.21.15.73:0/295522777 >> 172.21.15.73:0/295522777 conn(0x7f2ce80b06c0 msgr2=0x7f2ce80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.792+0000 7f2cece00640 1 -- 172.21.15.73:0/295522777 shutdown_connections 2024-04-05T22:25:55.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.792+0000 7f2cece00640 1 -- 172.21.15.73:0/295522777 wait complete. 2024-04-05T22:25:55.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.793+0000 7f2cece00640 1 Processor -- start 2024-04-05T22:25:55.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.793+0000 7f2cece00640 1 -- start start 2024-04-05T22:25:55.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce806c4a0 0x7f2ce814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce80b2210 0x7f2ce8158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8152d60 con 0x7f2ce813f3b0 2024-04-05T22:25:55.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce814fcf0 con 0x7f2ce806c4a0 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.794+0000 7f2cece00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81531f0 con 0x7f2ce80b2210 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55960/0 (socket says 172.21.15.73:55960) 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 -- 172.21.15.73:0/3334143278 learned_addr learned my addr 172.21.15.73:0/3334143278 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 -- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce80b2210 msgr2=0x7f2ce8158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce80b2210 0x7f2ce8158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 -- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce806c4a0 msgr2=0x7f2ce814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:55.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce806c4a0 0x7f2ce814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:55.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 -- 172.21.15.73:0/3334143278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd800a5a0 con 0x7f2ce813f3b0 2024-04-05T22:25:55.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.795+0000 7f2ce7e00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c780 tx=0x7f2ce000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.796+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004b60 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.796+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce000ce90 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.796+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8152150 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.796+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000a850 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.796+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce8152710 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.797+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce804e160 con 0x7f2ce813f3b0 2024-04-05T22:25:55.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.798+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f2ce000a9b0 con 0x7f2ce813f3b0 2024-04-05T22:25:55.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.799+0000 7f2cdf400640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 0x7f2cc806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:55.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.799+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ce0089b20 con 0x7f2ce813f3b0 2024-04-05T22:25:55.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.799+0000 7f2ce7400640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 0x7f2cc806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:55.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.800+0000 7f2ce7400640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 0x7f2cc806d450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8037000 tx=0x7f2cd802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:55.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:55.804+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2ce00547b0 con 0x7f2ce813f3b0 2024-04-05T22:25:56.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.035+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ce8152ea0 con 0x7f2ce813f3b0 2024-04-05T22:25:56.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.036+0000 7f2cdf400640 1 -- 172.21.15.73:0/3334143278 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f2ce0057dd0 con 0x7f2ce813f3b0 2024-04-05T22:25:56.037 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:56.037 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:25:23.270861+0000","modified":"2024-04-05T22:25:26.297974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2831205947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2831205947},{"type":"v1","addr":"172.21.15.73:6837","nonce":2831205947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-05T22:25:56.037 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 33 2024-04-05T22:25:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 msgr2=0x7f2cc806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 0x7f2cc806d450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8037000 tx=0x7f2cd802f000 comp rx=0 tx=0).stop 2024-04-05T22:25:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 msgr2=0x7f2ce8158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c780 tx=0x7f2ce000cc40 comp rx=0 tx=0).stop 2024-04-05T22:25:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 shutdown_connections 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f2cc806af90 0x7f2cc806d450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ce813f3b0 0x7f2ce8158770 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce80b2210 0x7f2ce8158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 --2- 172.21.15.73:0/3334143278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce806c4a0 0x7f2ce814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 >> 172.21.15.73:0/3334143278 conn(0x7f2ce80b06c0 msgr2=0x7f2ce8071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 shutdown_connections 2024-04-05T22:25:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.039+0000 7f2cece00640 1 -- 172.21.15.73:0/3334143278 wait complete. 2024-04-05T22:25:56.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:25:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.269+0000 7fc41d600640 1 Processor -- start 2024-04-05T22:25:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.269+0000 7fc41d600640 1 -- start start 2024-04-05T22:25:56.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc41d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4180b1870 0x7fc4180ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc41d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4180ad6b0 0x7fc4180cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc41d600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180ce5d0 con 0x7fc418160620 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc41d600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4180ce010 con 0x7fc4180b1870 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc41d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180ce2f0 con 0x7fc4180ad6b0 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4180b1870 0x7fc4180ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4180b1870 0x7fc4180ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56670/0 (socket says 172.21.15.73:56670) 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 -- 172.21.15.73:0/1429854361 learned_addr learned my addr 172.21.15.73:0/1429854361 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc417400640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4180ad6b0 0x7fc4180cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 -- 172.21.15.73:0/1429854361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4180ad6b0 msgr2=0x7fc4180cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4180ad6b0 0x7fc4180cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 -- 172.21.15.73:0/1429854361 >> v1:172.21.15.157:6789/0 conn(0x7fc418160620 legacy=0x7fc4180caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.271+0000 7fc40e000640 1 -- 172.21.15.73:0/1429854361 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4180ac970 con 0x7fc4180b1870 2024-04-05T22:25:56.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.273+0000 7fc40e000640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc4180ad170 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc41806af00 tx=0x7fc40002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518eff3fbff4e2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.273+0000 7fc416000640 1 -- 172.21.15.73:0/1429854361 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400031830 con 0x7fc4180b1870 2024-04-05T22:25:56.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.273+0000 7fc416000640 1 -- 172.21.15.73:0/1429854361 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400031990 con 0x7fc4180b1870 2024-04-05T22:25:56.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.273+0000 7fc416000640 1 -- 172.21.15.73:0/1429854361 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400031d20 con 0x7fc4180b1870 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 -- 172.21.15.73:0/1429854361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 msgr2=0x7fc4180ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc4180ad170 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc41806af00 tx=0x7fc40002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 -- 172.21.15.73:0/1429854361 shutdown_connections 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4180ad6b0 0x7fc4180cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 --2- 172.21.15.73:0/1429854361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc4180ad170 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 -- 172.21.15.73:0/1429854361 >> 172.21.15.73:0/1429854361 conn(0x7fc4180ce8e0 msgr2=0x7fc4180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 -- 172.21.15.73:0/1429854361 shutdown_connections 2024-04-05T22:25:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.274+0000 7fc41d600640 1 -- 172.21.15.73:0/1429854361 wait complete. 2024-04-05T22:25:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.276+0000 7fc41d600640 1 Processor -- start 2024-04-05T22:25:56.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.276+0000 7fc41d600640 1 -- start start 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 0x7fc418144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc418160620 0x7fc4181562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418144b80 con 0x7fc4180b1870 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181599c0 con 0x7fc4180ad6b0 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc41d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181448a0 con 0x7fc418160620 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56672/0 (socket says 172.21.15.73:56672) 2024-04-05T22:25:56.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 -- 172.21.15.73:0/499188499 learned_addr learned my addr 172.21.15.73:0/499188499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc417400640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc418160620 0x7fc4181562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 -- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc418160620 msgr2=0x7fc4181562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc418160620 0x7fc4181562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 -- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 msgr2=0x7fc418144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc416a00640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 0x7fc418144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 0x7fc418144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc40e000640 1 -- 172.21.15.73:0/499188499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4000094d0 con 0x7fc4180b1870 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.277+0000 7fc416a00640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 0x7fc418144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:56.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc40e000640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc40002b910 tx=0x7fc4000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000317d0 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40003aa60 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4181593b0 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40003ae10 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.278+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc418159650 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.279+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc418066af0 con 0x7fc4180b1870 2024-04-05T22:25:56.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.280+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fc4000443f0 con 0x7fc4180b1870 2024-04-05T22:25:56.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.281+0000 7fc40f400640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 0x7fc3f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.281+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4000abb00 con 0x7fc4180b1870 2024-04-05T22:25:56.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.281+0000 7fc416a00640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 0x7fc3f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.282+0000 7fc416a00640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 0x7fc3f406d420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc408003eb0 tx=0x7fc408009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.286+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc4000ade60 con 0x7fc4180b1870 2024-04-05T22:25:56.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.481+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc418159b70 con 0x7fc4180b1870 2024-04-05T22:25:56.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.483+0000 7fc40f400640 1 -- 172.21.15.73:0/499188499 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16097 (secure 0 0 0) 0x7fc400035090 con 0x7fc4180b1870 2024-04-05T22:25:56.484 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:25:56.484 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":62,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:24.275137+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:25:21.730760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-05T22:25:22.267497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 msgr2=0x7fc3f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 0x7fc3f406d420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc408003eb0 tx=0x7fc408009040 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 msgr2=0x7fc418155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc40002b910 tx=0x7fc4000046c0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 shutdown_connections 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc3f406af60 0x7fc3f406d420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc418160620 0x7fc4181562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4180b1870 0x7fc418155d90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 --2- 172.21.15.73:0/499188499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4180ad6b0 0x7fc418144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.486+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 >> 172.21.15.73:0/499188499 conn(0x7fc4180ce8e0 msgr2=0x7fc4180caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.487+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 shutdown_connections 2024-04-05T22:25:56.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.487+0000 7fc41d600640 1 -- 172.21.15.73:0/499188499 wait complete. 2024-04-05T22:25:56.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:25:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.713+0000 7fb09f000640 1 Processor -- start 2024-04-05T22:25:56.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.713+0000 7fb09f000640 1 -- start start 2024-04-05T22:25:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.714+0000 7fb09f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb09806d060 0x7fb0980b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098157560 0x7fb0980b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09f000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980b3ea0 con 0x7fb09815aaa0 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09815e2c0 con 0x7fb09806d060 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09815d500 con 0x7fb098157560 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb09806d060 0x7fb0980b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb09806d060 0x7fb0980b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56684/0 (socket says 172.21.15.73:56684) 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 -- 172.21.15.73:0/3143267172 learned_addr learned my addr 172.21.15.73:0/3143267172 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 -- 172.21.15.73:0/3143267172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098157560 msgr2=0x7fb0980b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09e600640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098157560 0x7fb0980b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098157560 0x7fb0980b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 -- 172.21.15.73:0/3143267172 >> v1:172.21.15.157:6789/0 conn(0x7fb09815aaa0 legacy=0x7fb098156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:56.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.715+0000 7fb09dc00640 1 -- 172.21.15.73:0/3143267172 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09815d7e0 con 0x7fb09806d060 2024-04-05T22:25:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.716+0000 7fb09dc00640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb09806d060 0x7fb0980b3530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00bb10 tx=0x7fb08c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904e4a5346ac9634 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb097e00640 1 -- 172.21.15.73:0/3143267172 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c02ee70 con 0x7fb09806d060 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb097e00640 1 -- 172.21.15.73:0/3143267172 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c036440 con 0x7fb09806d060 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb097e00640 1 -- 172.21.15.73:0/3143267172 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0367d0 con 0x7fb09806d060 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 -- 172.21.15.73:0/3143267172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb09806d060 msgr2=0x7fb0980b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb09806d060 0x7fb0980b3530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00bb10 tx=0x7fb08c02e0c0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 -- 172.21.15.73:0/3143267172 shutdown_connections 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb098157560 0x7fb0980b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 --2- 172.21.15.73:0/3143267172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb09806d060 0x7fb0980b3530 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 -- 172.21.15.73:0/3143267172 >> 172.21.15.73:0/3143267172 conn(0x7fb0980b0830 msgr2=0x7fb0980b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 -- 172.21.15.73:0/3143267172 shutdown_connections 2024-04-05T22:25:56.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.717+0000 7fb09f000640 1 -- 172.21.15.73:0/3143267172 wait complete. 2024-04-05T22:25:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.719+0000 7fb09f000640 1 Processor -- start 2024-04-05T22:25:56.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.719+0000 7fb09f000640 1 -- start start 2024-04-05T22:25:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09806d060 0x7fb0980ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb09815aaa0 0x7fb098154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981579e0 con 0x7fb098157560 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098157cc0 con 0x7fb09815aaa0 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098157fa0 con 0x7fb09806d060 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56686/0 (socket says 172.21.15.73:56686) 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.720+0000 7fb09d200640 1 -- 172.21.15.73:0/4101803488 learned_addr learned my addr 172.21.15.73:0/4101803488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 -- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09806d060 msgr2=0x7fb0980ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09806d060 0x7fb0980ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 -- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb09815aaa0 msgr2=0x7fb098154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb09815aaa0 0x7fb098154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 -- 172.21.15.73:0/4101803488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c00b770 con 0x7fb098157560 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09d200640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b3f0 tx=0x7fb08400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084011830 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb084011990 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb098158280 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084011d40 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.721+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0980b2ef0 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.722+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb098158470 con 0x7fb098157560 2024-04-05T22:25:56.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.723+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fb08401a400 con 0x7fb098157560 2024-04-05T22:25:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.725+0000 7fb095600640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.725+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0840896f0 con 0x7fb098157560 2024-04-05T22:25:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.725+0000 7fb09dc00640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:56.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.726+0000 7fb09dc00640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 0x7fb07806d420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb08c038000 tx=0x7fb08c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:56.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.729+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb084054500 con 0x7fb098157560 2024-04-05T22:25:56.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.945+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb09804e160 con 0x7fb098157560 2024-04-05T22:25:56.946 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:25:56.946+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:25:56.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.961+0000 7fb095600640 1 -- 172.21.15.73:0/4101803488 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb084057b20 con 0x7fb098157560 2024-04-05T22:25:56.962 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 msgr2=0x7fb07806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 0x7fb07806d420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb08c038000 tx=0x7fb08c030000 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 msgr2=0x7fb09814a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b3f0 tx=0x7fb08400b8b0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 shutdown_connections 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb09815aaa0 0x7fb098154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb098157560 0x7fb09814a110 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 --2- 172.21.15.73:0/4101803488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb09806d060 0x7fb0980ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.964+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 >> 172.21.15.73:0/4101803488 conn(0x7fb0980b0830 msgr2=0x7fb0981558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.965+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 shutdown_connections 2024-04-05T22:25:56.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:56.965+0000 7fb09f000640 1 -- 172.21.15.73:0/4101803488 wait complete. 2024-04-05T22:25:56.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:25:57.025 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:25:57.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.156+0000 7fc6d0c00640 1 Processor -- start 2024-04-05T22:25:57.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.156+0000 7fc6d0c00640 1 -- start start 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.157+0000 7fc6d0c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6cc1445e0 0x7fc6cc0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.157+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 0x7fc6cc0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.157+0000 7fc6d0c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0cc790 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.157+0000 7fc6d0c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc06f820 con 0x7fc6cc1445e0 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.157+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc06fb00 con 0x7fc6cc06cc20 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c9400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc6cc0cc2f0 0x7fc6cc06c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34292/0 (socket says 172.21.15.73:34292) 2024-04-05T22:25:57.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c9400640 1 -- 172.21.15.73:0/1920706330 learned_addr learned my addr 172.21.15.73:0/1920706330 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308896921 0 0) 0x7fc6cc0cc790 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c9e00640 1 --2- 172.21.15.73:0/1920706330 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc1445e0 0x7fc6cc0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6ca800640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 0x7fc6cc0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b0003520 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458125388 0 0) 0x7fc6b0003520 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 msgr2=0x7fc6cc0705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 0x7fc6cc0705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc1445e0 msgr2=0x7fc6cc0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.158+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1920706330 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc1445e0 0x7fc6cc0cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc06fde0 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc6b8001670 con 0x7fc6cc0cc2f0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 >> v1:172.21.15.73:6789/0 conn(0x7fc6cc0cc2f0 legacy=0x7fc6cc06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6b0003970 0x7fc6b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6b00047c0 0x7fc6b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 0x7fc6b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc06f820 con 0x7fc6b00047c0 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc06fb00 con 0x7fc6b0003970 2024-04-05T22:25:57.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b000b9b0 con 0x7fc6cc06cc20 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6b00047c0 0x7fc6b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 -- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 msgr2=0x7fc6b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc06cc20 0x7fc6b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 -- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6b0003970 msgr2=0x7fc6b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6b0003970 0x7fc6b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.159+0000 7fc6c9400640 1 -- 172.21.15.73:0/1920706330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b0003520 con 0x7fc6b00047c0 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.160+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6b00047c0 0x7fc6b0004c40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c240 tx=0x7fc6b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de0596ab09d634b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.160+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b802cb00 con 0x7fc6b00047c0 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.160+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b802cc60 con 0x7fc6b00047c0 2024-04-05T22:25:57.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.160+0000 7fc6c8a00640 1 -- 172.21.15.73:0/1920706330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b80058d0 con 0x7fc6b00047c0 2024-04-05T22:25:57.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1920706330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6b00047c0 msgr2=0x7fc6b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1920706330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6b00047c0 0x7fc6b0004c40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c240 tx=0x7fc6b00057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6ca800640 1 -- 172.21.15.73:0/1920706330 reap_dead start 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1920706330 shutdown_connections 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1920706330 >> 172.21.15.73:0/1920706330 conn(0x7fc6cc0b06c0 msgr2=0x7fc6cc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1920706330 shutdown_connections 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.161+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1920706330 wait complete. 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.163+0000 7fc6d0c00640 1 Processor -- start 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.163+0000 7fc6d0c00640 1 -- start start 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6b00047c0 0x7fc6cc15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc143600 0x7fc6cc1e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc143a90 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0acf80 con 0x7fc6cc143600 2024-04-05T22:25:57.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1e1150 con 0x7fc6b00047c0 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56732/0 (socket says 172.21.15.73:56732) 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6c9400640 1 -- 172.21.15.73:0/1537307378 learned_addr learned my addr 172.21.15.73:0/1537307378 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6ca800640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc143600 0x7fc6cc1e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.164+0000 7fc6c9e00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6b00047c0 0x7fc6cc15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 -- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6b00047c0 msgr2=0x7fc6cc15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6b00047c0 0x7fc6cc15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 -- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc143600 msgr2=0x7fc6cc1e03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc143600 0x7fc6cc1e03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 -- 172.21.15.73:0/1537307378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc1e1430 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc6b802c4a0 tx=0x7fc6b8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:57.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8039070 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8001630 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc0acc10 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b803c040 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.165+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc0ac830 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.166+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc0ad130 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.167+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fc6b8032070 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.168+0000 7fc6c2000640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 0x7fc6ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.168+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc6b80aaf20 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.169+0000 7fc6c9e00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 0x7fc6ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.169+0000 7fc6c9e00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 0x7fc6ac06d450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009ad0 tx=0x7fc6bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:57.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.172+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc6cc0ad130 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.384+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc6cc0aca50 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.964+0000 7fc6c2000640 1 -- 172.21.15.73:0/1537307378 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc6b80460a0 con 0x7fc6cc0a84c0 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 msgr2=0x7fc6ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 0x7fc6ac06d450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009ad0 tx=0x7fc6bc008040 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 msgr2=0x7fc6cc1430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc6b802c4a0 tx=0x7fc6b8001a00 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 shutdown_connections 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6ac06af90 0x7fc6ac06d450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc143600 0x7fc6cc1e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0a84c0 0x7fc6cc1430c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 --2- 172.21.15.73:0/1537307378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6b00047c0 0x7fc6cc15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 >> 172.21.15.73:0/1537307378 conn(0x7fc6cc0b06c0 msgr2=0x7fc6cc1e5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 shutdown_connections 2024-04-05T22:25:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:57.968+0000 7fc6d0c00640 1 -- 172.21.15.73:0/1537307378 wait complete. 2024-04-05T22:25:57.991 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:25:58.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.253+0000 7f03a5000640 1 Processor -- start 2024-04-05T22:25:58.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.253+0000 7f03a5000640 1 -- start start 2024-04-05T22:25:58.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f03a5000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03a00cf3e0 0x7f03a00b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f03a5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a00b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f03a5000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a015d900 con 0x7f03a01541a0 2024-04-05T22:25:58.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f03a5000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a015e1a0 con 0x7f03a00cf3e0 2024-04-05T22:25:58.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f03a5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00b3490 con 0x7f03a0154580 2024-04-05T22:25:58.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f039d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03a00cf3e0 0x7f03a00b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:58.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f039de00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f03a01541a0 0x7f03a00cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34300/0 (socket says 172.21.15.73:34300) 2024-04-05T22:25:58.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.254+0000 7f039de00640 1 -- 172.21.15.73:0/3106245248 learned_addr learned my addr 172.21.15.73:0/3106245248 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:58.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039e800640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a00b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:58.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039d400640 1 -- 172.21.15.73:0/3106245248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 msgr2=0x7f03a00b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039d400640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a00b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039d400640 1 -- 172.21.15.73:0/3106245248 >> v1:172.21.15.73:6789/0 conn(0x7f03a01541a0 legacy=0x7f03a00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039d400640 1 -- 172.21.15.73:0/3106245248 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a015db50 con 0x7f03a00cf3e0 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039e800640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a00b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.255+0000 7f039ca00640 1 -- 172.21.15.73:0/3106245248 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446901587 0 0) 0x7f03a015d900 con 0x7f03a01541a0 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.256+0000 7f039d400640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a00b2840 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097a0 tx=0x7f038c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d393ff8fe4ab1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:58.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.256+0000 7f039ca00640 1 -- 172.21.15.73:0/3106245248 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c02d040 con 0x7f03a00cf3e0 2024-04-05T22:25:58.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.256+0000 7f039ca00640 1 -- 172.21.15.73:0/3106245248 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c031810 con 0x7f03a00cf3e0 2024-04-05T22:25:58.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.256+0000 7f039ca00640 1 -- 172.21.15.73:0/3106245248 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c031ba0 con 0x7f03a00cf3e0 2024-04-05T22:25:58.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 -- 172.21.15.73:0/3106245248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 msgr2=0x7f03a00b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a00b2840 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097a0 tx=0x7f038c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 -- 172.21.15.73:0/3106245248 shutdown_connections 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a00b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 --2- 172.21.15.73:0/3106245248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a00b2840 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 -- 172.21.15.73:0/3106245248 >> 172.21.15.73:0/3106245248 conn(0x7f03a006a0d0 msgr2=0x7f03a006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 -- 172.21.15.73:0/3106245248 shutdown_connections 2024-04-05T22:25:58.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.257+0000 7f03a5000640 1 -- 172.21.15.73:0/3106245248 wait complete. 2024-04-05T22:25:58.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.259+0000 7f03a5000640 1 Processor -- start 2024-04-05T22:25:58.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.259+0000 7f03a5000640 1 -- start start 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a01541a0 0x7f03a0149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a01568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a014a010 con 0x7f03a00cf3e0 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0157740 con 0x7f03a01541a0 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f03a5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0157460 con 0x7f03a0154580 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56740/0 (socket says 172.21.15.73:56740) 2024-04-05T22:25:58.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 -- 172.21.15.73:0/2453701155 learned_addr learned my addr 172.21.15.73:0/2453701155 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:58.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 -- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 msgr2=0x7f03a01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a01568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 -- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a01541a0 msgr2=0x7f03a0149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a01541a0 0x7f03a0149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.260+0000 7f039de00640 1 -- 172.21.15.73:0/2453701155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c009400 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f039de00640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f039000c4d0 tx=0x7f039000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900074e0 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390007640 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390004a20 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a01f64b0 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.261+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a01f6a70 con 0x7f03a00cf3e0 2024-04-05T22:25:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.262+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a01f6c60 con 0x7f03a00cf3e0 2024-04-05T22:25:58.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.263+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f0390004b80 con 0x7f03a00cf3e0 2024-04-05T22:25:58.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.264+0000 7f0396000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 0x7f038006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:58.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.264+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0390088c00 con 0x7f03a00cf3e0 2024-04-05T22:25:58.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.264+0000 7f039d400640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 0x7f038006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:58.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.265+0000 7f039d400640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 0x7f038006d420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f038c036000 tx=0x7f038c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:58.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.269+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0390053870 con 0x7f03a00cf3e0 2024-04-05T22:25:58.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.448+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03a0068160 con 0x7f03a00cf3e0 2024-04-05T22:25:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.969+0000 7f0396000640 1 -- 172.21.15.73:0/2453701155 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0390018020 con 0x7f03a00cf3e0 2024-04-05T22:25:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:25:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.975+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 msgr2=0x7f038006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.975+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 0x7f038006d420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f038c036000 tx=0x7f038c030000 comp rx=0 tx=0).stop 2024-04-05T22:25:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.975+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 msgr2=0x7f03a01f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.975+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f039000c4d0 tx=0x7f039000c990 comp rx=0 tx=0).stop 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 shutdown_connections 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f038006af60 0x7f038006d420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03a0154580 0x7f03a01568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a01541a0 0x7f03a0149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 --2- 172.21.15.73:0/2453701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03a00cf3e0 0x7f03a01f6f90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.976+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 >> 172.21.15.73:0/2453701155 conn(0x7f03a006a0d0 msgr2=0x7f03a00b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.977+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 shutdown_connections 2024-04-05T22:25:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:58.977+0000 7f03a5000640 1 -- 172.21.15.73:0/2453701155 wait complete. 2024-04-05T22:25:58.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:25:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.193+0000 7fbac7200640 1 Processor -- start 2024-04-05T22:25:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.194+0000 7fbac7200640 1 -- start start 2024-04-05T22:25:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbac015ac00 0x7fbac006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbac00cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac7200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac015e570 con 0x7fbac00cf210 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac00cf6a0 con 0x7fbac015ac00 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015afe0 con 0x7fbac00b36e0 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac5200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbac00cf210 0x7fbac00ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34310/0 (socket says 172.21.15.73:34310) 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.195+0000 7fbac5200640 1 -- 172.21.15.73:0/2974580913 learned_addr learned my addr 172.21.15.73:0/2974580913 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabfe00640 1 --2- 172.21.15.73:0/2974580913 >> v2:172.21.15.157:3300/0 conn(0x7fbac015ac00 0x7fbac006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011561765 0 0) 0x7fbac015e570 con 0x7fbac00cf210 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa4003520 con 0x7fbac00cf210 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbab7400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbac00cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635891432 0 0) 0x7fbaa4003520 con 0x7fbac00cf210 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 msgr2=0x7fbac00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbac00cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 >> v2:172.21.15.157:3300/0 conn(0x7fbac015ac00 msgr2=0x7fbac006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 --2- 172.21.15.73:0/2974580913 >> v2:172.21.15.157:3300/0 conn(0x7fbac015ac00 0x7fbac006cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0161760 con 0x7fbac00cf210 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbab80019f0 con 0x7fbac00cf210 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabfe00640 1 --2- 172.21.15.73:0/2974580913 >> v2:172.21.15.157:3300/0 conn(0x7fbac015ac00 0x7fbac006cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 >> v1:172.21.15.73:6789/0 conn(0x7fbac00cf210 legacy=0x7fbac00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbaa4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4003fa0 0x7fbaa40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaa4004d00 0x7fbaa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00cf6a0 con 0x7fbaa4004d00 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015afe0 con 0x7fbaa4003fa0 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.196+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa400bb40 con 0x7fbac00b36e0 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaa4004d00 0x7fbaa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbabfe00640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbaa4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbab7400640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4003fa0 0x7fbaa40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 -- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 msgr2=0x7fbaa4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac00b36e0 0x7fbaa4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 -- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4003fa0 msgr2=0x7fbaa40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:25:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4003fa0 0x7fbaa40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.197+0000 7fbac5200640 1 -- 172.21.15.73:0/2974580913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4003520 con 0x7fbaa4004d00 2024-04-05T22:25:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.198+0000 7fbac5200640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaa4004d00 0x7fbaa4005180 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbab800a830 tx=0x7fbaa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b49339ab89a82c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.198+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8034630 con 0x7fbaa4004d00 2024-04-05T22:25:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.198+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab802f070 con 0x7fbaa4004d00 2024-04-05T22:25:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbabf400640 1 -- 172.21.15.73:0/2974580913 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8034a30 con 0x7fbaa4004d00 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 -- 172.21.15.73:0/2974580913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaa4004d00 msgr2=0x7fbaa4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 --2- 172.21.15.73:0/2974580913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaa4004d00 0x7fbaa4005180 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbab800a830 tx=0x7fbaa40057f0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac5200640 1 -- 172.21.15.73:0/2974580913 reap_dead start 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 -- 172.21.15.73:0/2974580913 shutdown_connections 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 -- 172.21.15.73:0/2974580913 >> 172.21.15.73:0/2974580913 conn(0x7fbac006a0d0 msgr2=0x7fbac006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 -- 172.21.15.73:0/2974580913 shutdown_connections 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.199+0000 7fbac7200640 1 -- 172.21.15.73:0/2974580913 wait complete. 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.201+0000 7fbac7200640 1 Processor -- start 2024-04-05T22:25:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.201+0000 7fbac7200640 1 -- start start 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4004d00 0x7fbac0141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac01552d0 0x7fbac0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01605d0 con 0x7fbac015f200 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac0155cc0 con 0x7fbaa4004d00 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbac7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015fd30 con 0x7fbac01552d0 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56752/0 (socket says 172.21.15.73:56752) 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 -- 172.21.15.73:0/1779149981 learned_addr learned my addr 172.21.15.73:0/1779149981 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:25:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 -- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac01552d0 msgr2=0x7fbac0155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac01552d0 0x7fbac0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 -- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4004d00 msgr2=0x7fbac0141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbabfe00640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4004d00 0x7fbac0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4004d00 0x7fbac0141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.202+0000 7fbab7400640 1 -- 172.21.15.73:0/1779149981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0160010 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbab7400640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e760 tx=0x7fbaa800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800c6d0 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa800c830 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800cbe0 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01602f0 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.203+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac0141970 con 0x7fbac015f200 2024-04-05T22:25:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.204+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac015b180 con 0x7fbac015f200 2024-04-05T22:25:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.205+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fbaa800cd40 con 0x7fbac015f200 2024-04-05T22:25:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.206+0000 7fbabcc00640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 0x7fba9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:25:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.206+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fbaa80895e0 con 0x7fbac015f200 2024-04-05T22:25:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.206+0000 7fbabfe00640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 0x7fba9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:25:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.207+0000 7fbabfe00640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 0x7fba9c06d390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbac00af0b0 tx=0x7fbab0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:25:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.210+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbac015b180 con 0x7fbac015f200 2024-04-05T22:25:59.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.400+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbac015b180 con 0x7fbac015f200 2024-04-05T22:25:59.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.987+0000 7fbabcc00640 1 -- 172.21.15.73:0/1779149981 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbac015b180 con 0x7fbac015f200 2024-04-05T22:25:59.992 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 msgr2=0x7fba9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 0x7fba9c06d390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbac00af0b0 tx=0x7fbab0008040 comp rx=0 tx=0).stop 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 msgr2=0x7fbac015f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e760 tx=0x7fbaa800ec20 comp rx=0 tx=0).stop 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 shutdown_connections 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fba9c06aed0 0x7fba9c06d390 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbac01552d0 0x7fbac0155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac015f200 0x7fbac015f680 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 --2- 172.21.15.73:0/1779149981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa4004d00 0x7fbac0141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:25:59.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.996+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 >> 172.21.15.73:0/1779149981 conn(0x7fbac006a0d0 msgr2=0x7fbac00ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:25:59.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.997+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 shutdown_connections 2024-04-05T22:25:59.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:25:59.997+0000 7fbac7200640 1 -- 172.21.15.73:0/1779149981 wait complete. 2024-04-05T22:26:00.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:26:00.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.212+0000 7f2562a00640 1 Processor -- start 2024-04-05T22:26:00.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.212+0000 7f2562a00640 1 -- start start 2024-04-05T22:26:00.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.213+0000 7f2562a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 0x7f255c0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2562a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f255c0b32b0 0x7f255c1647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2562a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c0b1b30 con 0x7f255c06c630 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2562a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0b1e10 con 0x7f255c0b32b0 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2562a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0cc300 con 0x7f255c0cbe00 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f255be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f255c06c630 0x7f255c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34312/0 (socket says 172.21.15.73:34312) 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f255be00640 1 -- 172.21.15.73:0/3674804829 learned_addr learned my addr 172.21.15.73:0/3674804829 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2560a00640 1 --2- 172.21.15.73:0/3674804829 >> v2:172.21.15.73:3300/0 conn(0x7f255c0b32b0 0x7f255c1647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f255b400640 1 -- 172.21.15.73:0/3674804829 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314591850 0 0) 0x7f255c0b1b30 con 0x7f255c06c630 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f255b400640 1 -- 172.21.15.73:0/3674804829 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2540003520 con 0x7f255c06c630 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2560a00640 1 -- 172.21.15.73:0/3674804829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 msgr2=0x7f255c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:00.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.214+0000 7f2553400640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 0x7f255c0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.215+0000 7f2560a00640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 0x7f255c0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.215+0000 7f2560a00640 1 -- 172.21.15.73:0/3674804829 >> v1:172.21.15.73:6789/0 conn(0x7f255c06c630 legacy=0x7f255c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.215+0000 7f2560a00640 1 -- 172.21.15.73:0/3674804829 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0cc5e0 con 0x7f255c0b32b0 2024-04-05T22:26:00.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.215+0000 7f2553400640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 0x7f255c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:00.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f2560a00640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0b32b0 0x7f255c1647c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929e5125654c0747 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f255b400640 1 -- 172.21.15.73:0/3674804829 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402cc60 con 0x7f255c0b32b0 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f255b400640 1 -- 172.21.15.73:0/3674804829 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255402cdc0 con 0x7f255c0b32b0 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f255b400640 1 -- 172.21.15.73:0/3674804829 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554034660 con 0x7f255c0b32b0 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f2562a00640 1 -- 172.21.15.73:0/3674804829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0b32b0 msgr2=0x7f255c1647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.216+0000 7f2562a00640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0b32b0 0x7f255c1647c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255402beb0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 -- 172.21.15.73:0/3674804829 shutdown_connections 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c0cbe00 0x7f255c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 --2- 172.21.15.73:0/3674804829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0b32b0 0x7f255c1647c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 -- 172.21.15.73:0/3674804829 >> 172.21.15.73:0/3674804829 conn(0x7f255c0b0830 msgr2=0x7f255c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 -- 172.21.15.73:0/3674804829 shutdown_connections 2024-04-05T22:26:00.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.217+0000 7f2562a00640 1 -- 172.21.15.73:0/3674804829 wait complete. 2024-04-05T22:26:00.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.218+0000 7f2562a00640 1 Processor -- start 2024-04-05T22:26:00.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 -- start start 2024-04-05T22:26:00.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c06c630 0x7f255c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c0b32b0 0x7f255c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1384b0 con 0x7f255c0cbe00 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c157410 con 0x7f255c0b32b0 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.219+0000 7f2562a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1576f0 con 0x7f255c06c630 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56770/0 (socket says 172.21.15.73:56770) 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 -- 172.21.15.73:0/3240016898 learned_addr learned my addr 172.21.15.73:0/3240016898 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f255be00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c06c630 0x7f255c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2553400640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c0b32b0 0x7f255c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 -- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c06c630 msgr2=0x7f255c0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c06c630 0x7f255c0a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 -- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c0b32b0 msgr2=0x7f255c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c0b32b0 0x7f255c154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.220+0000 7f2560a00640 1 -- 172.21.15.73:0/3240016898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554009a70 con 0x7f255c0cbe00 2024-04-05T22:26:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2560a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f2554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554040650 con 0x7f255c0cbe00 2024-04-05T22:26:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540407b0 con 0x7f255c0cbe00 2024-04-05T22:26:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c1579d0 con 0x7f255c0cbe00 2024-04-05T22:26:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554040b60 con 0x7f255c0cbe00 2024-04-05T22:26:00.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.221+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c157f90 con 0x7f255c0cbe00 2024-04-05T22:26:00.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.222+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c158180 con 0x7f255c0cbe00 2024-04-05T22:26:00.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.223+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f2554040cc0 con 0x7f255c0cbe00 2024-04-05T22:26:00.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.224+0000 7f2558c00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 0x7f253806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.224+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25540ac2b0 con 0x7f255c0cbe00 2024-04-05T22:26:00.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.224+0000 7f255be00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 0x7f253806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.225+0000 7f255be00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 0x7f253806d420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f254c0011e0 tx=0x7f254c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.228+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2554077410 con 0x7f255c0cbe00 2024-04-05T22:26:00.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.415+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f255c068160 con 0x7f255c0cbe00 2024-04-05T22:26:00.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.433+0000 7f2558c00640 1 -- 172.21.15.73:0/3240016898 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2554009450 con 0x7f255c0cbe00 2024-04-05T22:26:00.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 msgr2=0x7f253806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 0x7f253806d420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f254c0011e0 tx=0x7f254c009040 comp rx=0 tx=0).stop 2024-04-05T22:26:00.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 msgr2=0x7f255c1f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f2554004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 shutdown_connections 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f253806af60 0x7f253806d420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255c0cbe00 0x7f255c1f6080 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c0b32b0 0x7f255c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 --2- 172.21.15.73:0/3240016898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255c06c630 0x7f255c0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 >> 172.21.15.73:0/3240016898 conn(0x7f255c0b0830 msgr2=0x7f255c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 shutdown_connections 2024-04-05T22:26:00.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.436+0000 7f2562a00640 1 -- 172.21.15.73:0/3240016898 wait complete. 2024-04-05T22:26:00.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:26:00.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.646+0000 7fb2da400640 1 Processor -- start 2024-04-05T22:26:00.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.646+0000 7fb2da400640 1 -- start start 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2da400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2d41607a0 0x7fb2d40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2da400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d40d0d40 con 0x7fb2d40d17b0 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2da400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d40b2150 con 0x7fb2d41603c0 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2da400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d40b2430 con 0x7fb2d41607a0 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2cbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2d41607a0 0x7fb2d40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2cbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2d41607a0 0x7fb2d40cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56776/0 (socket says 172.21.15.73:56776) 2024-04-05T22:26:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.648+0000 7fb2cbe00640 1 -- 172.21.15.73:0/3808990745 learned_addr learned my addr 172.21.15.73:0/3808990745 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.649+0000 7fb2cbe00640 1 -- 172.21.15.73:0/3808990745 >> v1:172.21.15.157:6789/0 conn(0x7fb2d41603c0 legacy=0x7fb2d40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.649+0000 7fb2cbe00640 1 -- 172.21.15.73:0/3808990745 >> v1:172.21.15.73:6789/0 conn(0x7fb2d40d17b0 legacy=0x7fb2d40cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.649+0000 7fb2cbe00640 1 -- 172.21.15.73:0/3808990745 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d40b2710 con 0x7fb2d41607a0 2024-04-05T22:26:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.650+0000 7fb2cbe00640 1 --2- 172.21.15.73:0/3808990745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d41607a0 0x7fb2d40cd810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009870 tx=0x7fb2bc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35466fe471d6a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.650+0000 7fb2d3e00640 1 -- 172.21.15.73:0/3808990745 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc02f850 con 0x7fb2d41607a0 2024-04-05T22:26:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.650+0000 7fb2d3e00640 1 -- 172.21.15.73:0/3808990745 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc02f9b0 con 0x7fb2d41607a0 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.650+0000 7fb2d3e00640 1 -- 172.21.15.73:0/3808990745 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc02fd40 con 0x7fb2d41607a0 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 -- 172.21.15.73:0/3808990745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d41607a0 msgr2=0x7fb2d40cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 --2- 172.21.15.73:0/3808990745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d41607a0 0x7fb2d40cd810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009870 tx=0x7fb2bc02ba30 comp rx=0 tx=0).stop 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 -- 172.21.15.73:0/3808990745 shutdown_connections 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 --2- 172.21.15.73:0/3808990745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d41607a0 0x7fb2d40cd810 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 -- 172.21.15.73:0/3808990745 >> 172.21.15.73:0/3808990745 conn(0x7fb2d40d2b50 msgr2=0x7fb2d40d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 -- 172.21.15.73:0/3808990745 shutdown_connections 2024-04-05T22:26:00.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.651+0000 7fb2da400640 1 -- 172.21.15.73:0/3808990745 wait complete. 2024-04-05T22:26:00.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.652+0000 7fb2da400640 1 Processor -- start 2024-04-05T22:26:00.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.653+0000 7fb2da400640 1 -- start start 2024-04-05T22:26:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d41603c0 0x7fb2d406a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d41607a0 0x7fb2d406a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40b8a00 con 0x7fb2d40d17b0 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41f2590 con 0x7fb2d41607a0 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2da400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41f2df0 con 0x7fb2d41603c0 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56788/0 (socket says 172.21.15.73:56788) 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 -- 172.21.15.73:0/1674059751 learned_addr learned my addr 172.21.15.73:0/1674059751 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 -- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d41603c0 msgr2=0x7fb2d406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d41603c0 0x7fb2d406a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 -- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d41607a0 msgr2=0x7fb2d406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d41607a0 0x7fb2d406a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.654+0000 7fb2d9000640 1 -- 172.21.15.73:0/1674059751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0094d0 con 0x7fb2d40d17b0 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2d9000640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c7f0 tx=0x7fb2c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400a9f0 con 0x7fb2d40d17b0 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c400ab50 con 0x7fb2d40d17b0 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d40b8720 con 0x7fb2d40d17b0 2024-04-05T22:26:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012420 con 0x7fb2d40d17b0 2024-04-05T22:26:00.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.655+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41f2a80 con 0x7fb2d40d17b0 2024-04-05T22:26:00.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.656+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d4067250 con 0x7fb2d40d17b0 2024-04-05T22:26:00.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.657+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fb2c4012580 con 0x7fb2d40d17b0 2024-04-05T22:26:00.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.658+0000 7fb2d1600640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 0x7fb2b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:00.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.658+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2c4089410 con 0x7fb2d40d17b0 2024-04-05T22:26:00.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.658+0000 7fb2cbe00640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 0x7fb2b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:00.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.659+0000 7fb2cbe00640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 0x7fb2b006d420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc036000 tx=0x7fb2bc039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:00.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.663+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb2c40545b0 con 0x7fb2d40d17b0 2024-04-05T22:26:00.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.925+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fb2d404e160 con 0x7fb2d40d17b0 2024-04-05T22:26:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.989+0000 7fb2d1600640 1 -- 172.21.15.73:0/1674059751 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v249) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb2c4057bd0 con 0x7fb2d40d17b0 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 msgr2=0x7fb2b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 0x7fb2b006d420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc036000 tx=0x7fb2bc039040 comp rx=0 tx=0).stop 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 msgr2=0x7fb2d414a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c7f0 tx=0x7fb2c400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 shutdown_connections 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fb2b006af60 0x7fb2b006d420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d41607a0 0x7fb2d406a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d41603c0 0x7fb2d406a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 --2- 172.21.15.73:0/1674059751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d40d17b0 0x7fb2d414a2f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.992+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 >> 172.21.15.73:0/1674059751 conn(0x7fb2d40d2b50 msgr2=0x7fb2d415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.993+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 shutdown_connections 2024-04-05T22:26:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:00.993+0000 7fb2da400640 1 -- 172.21.15.73:0/1674059751 wait complete. 2024-04-05T22:26:01.012 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-04-05T22:26:01.012 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:26:01.012 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:01.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.218+0000 7f78ad600640 1 Processor -- start 2024-04-05T22:26:01.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.218+0000 7f78ad600640 1 -- start start 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f78ad600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f78a815eab0 0x7f78a806c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f78ad600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a806b500 con 0x7f78a815e620 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f78ad600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80b20d0 con 0x7f78a8158860 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f78ad600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80ccaf0 con 0x7f78a815eab0 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f789fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78a815e620 0x7f78a806bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34340/0 (socket says 172.21.15.73:34340) 2024-04-05T22:26:01.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f789fe00640 1 -- 172.21.15.73:0/3636029067 learned_addr learned my addr 172.21.15.73:0/3636029067 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:01.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.220+0000 7f78acc00640 1 --2- 172.21.15.73:0/3636029067 >> v2:172.21.15.157:3300/0 conn(0x7f78a815eab0 0x7f78a806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214413361 0 0) 0x7f78a806b500 con 0x7f78a815e620 2024-04-05T22:26:01.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f788c003520 con 0x7f78a815e620 2024-04-05T22:26:01.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605904729 0 0) 0x7f788c003520 con 0x7f78a815e620 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 >> v2:172.21.15.157:3300/0 conn(0x7f78a815eab0 msgr2=0x7f78a806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 --2- 172.21.15.73:0/3636029067 >> v2:172.21.15.157:3300/0 conn(0x7f78a815eab0 0x7f78a806c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 >> v1:172.21.15.157:6789/0 conn(0x7f78a8158860 legacy=0x7f78a80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80b23a0 con 0x7f78a815e620 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7890001670 con 0x7f78a815e620 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 >> v1:172.21.15.73:6789/0 conn(0x7f78a815e620 legacy=0x7f78a806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788c003970 0x7f788c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c0047c0 0x7f788c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 0x7f788c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80ccaf0 con 0x7f788c005180 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c00c120 con 0x7f788c0047c0 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c00c2e0 con 0x7f788c003970 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.221+0000 7f78acc00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 0x7f788c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78a7e00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788c003970 0x7f788c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f789fe00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c0047c0 0x7f788c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78acc00640 1 -- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788c003970 msgr2=0x7f788c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78acc00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788c003970 0x7f788c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78acc00640 1 -- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c0047c0 msgr2=0x7f788c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78acc00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c0047c0 0x7f788c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.222+0000 7f78acc00640 1 -- 172.21.15.73:0/3636029067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c003520 con 0x7f788c005180 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.223+0000 7f78acc00640 1 -- 172.21.15.73:0/3636029067 reap_dead start 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.223+0000 7f78acc00640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 0x7f788c00b980 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fc0 tx=0x7f788c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8f6b454f5cf5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.223+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a002cc60 con 0x7f788c005180 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.223+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a000f040 con 0x7f788c005180 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.223+0000 7f78a7400640 1 -- 172.21.15.73:0/3636029067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003990 con 0x7f788c005180 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 -- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 msgr2=0x7f788c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 0x7f788c00b980 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fc0 tx=0x7f788c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 -- 172.21.15.73:0/3636029067 shutdown_connections 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 --2- 172.21.15.73:0/3636029067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788c005180 0x7f788c00b980 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 -- 172.21.15.73:0/3636029067 >> 172.21.15.73:0/3636029067 conn(0x7f78a80b06c0 msgr2=0x7f78a80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 -- 172.21.15.73:0/3636029067 shutdown_connections 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.224+0000 7f78ad600640 1 -- 172.21.15.73:0/3636029067 wait complete. 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.226+0000 7f78ad600640 1 Processor -- start 2024-04-05T22:26:01.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.226+0000 7f78ad600640 1 -- start start 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 0x7f78a814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a8154620 0x7f78a8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80ac970 con 0x7f78a81535b0 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c3930 con 0x7f788c005180 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f78ad600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8071170 con 0x7f78a8154620 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f789fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f789fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56820/0 (socket says 172.21.15.73:56820) 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.227+0000 7f789fe00640 1 -- 172.21.15.73:0/1414375185 learned_addr learned my addr 172.21.15.73:0/1414375185 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78a7e00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 0x7f78a814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 -- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a8154620 msgr2=0x7f78a8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78acc00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a8154620 0x7f78a8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a8154620 0x7f78a8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 -- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 msgr2=0x7f78a814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 0x7f78a814b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 -- 172.21.15.73:0/1414375185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8071450 con 0x7f78a81535b0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78a7e00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 0x7f78a814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f789fe00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f789000e780 tx=0x7f789000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000ee90 con 0x7f78a81535b0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000c8d0 con 0x7f78a81535b0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000cc80 con 0x7f78a81535b0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80acc10 con 0x7f78a81535b0 2024-04-05T22:26:01.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.228+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a8153d20 con 0x7f78a81535b0 2024-04-05T22:26:01.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.230+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a80ace00 con 0x7f78a81535b0 2024-04-05T22:26:01.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.231+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f789001d460 con 0x7f78a81535b0 2024-04-05T22:26:01.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.231+0000 7f78a4c00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 0x7f788406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.232+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78900894a0 con 0x7f78a81535b0 2024-04-05T22:26:01.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.232+0000 7f78a7e00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 0x7f788406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.232+0000 7f78a7e00640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 0x7f788406d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a80b3a90 tx=0x7f7898009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.236+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f78900545e0 con 0x7f78a81535b0 2024-04-05T22:26:01.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.430+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78a804e160 con 0x7f78a81535b0 2024-04-05T22:26:01.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.434+0000 7f78a4c00640 1 -- 172.21.15.73:0/1414375185 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12518 (secure 0 0 0) 0x7f7890057c00 con 0x7f78a81535b0 2024-04-05T22:26:01.434 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:01.434 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":65,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:25:59.978832+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6837/2831205947":"2024-04-06T22:25:56.947182+0000","172.21.15.73:6835/2831205947":"2024-04-06T22:25:56.947182+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 msgr2=0x7f788406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 0x7f788406d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a80b3a90 tx=0x7f7898009040 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 msgr2=0x7f78a8157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f789000e780 tx=0x7f789000ec40 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 shutdown_connections 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f788406af60 0x7f788406d420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a8154620 0x7f78a8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a81535b0 0x7f78a8157300 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.436+0000 7f78ad600640 1 --2- 172.21.15.73:0/1414375185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f788c005180 0x7f78a814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.437+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 >> 172.21.15.73:0/1414375185 conn(0x7f78a80b06c0 msgr2=0x7f78a806bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.437+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 shutdown_connections 2024-04-05T22:26:01.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.437+0000 7f78ad600640 1 -- 172.21.15.73:0/1414375185 wait complete. 2024-04-05T22:26:01.454 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:01.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:01.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.665+0000 7fe8c7c00640 1 Processor -- start 2024-04-05T22:26:01.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.665+0000 7fe8c7c00640 1 -- start start 2024-04-05T22:26:01.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 0x7fe8c006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c7c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c006bec0 con 0x7fe8c006ba70 2024-04-05T22:26:01.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c7c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c015afc0 con 0x7fe8c015abe0 2024-04-05T22:26:01.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c006f5b0 con 0x7fe8c006c9a0 2024-04-05T22:26:01.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c5c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe8c006ba70 0x7fe8c0164740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34344/0 (socket says 172.21.15.73:34344) 2024-04-05T22:26:01.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.667+0000 7fe8c5c00640 1 -- 172.21.15.73:0/961050871 learned_addr learned my addr 172.21.15.73:0/961050871 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:01.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bfe00640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 0x7fe8c006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288881033 0 0) 0x7fe8b8001490 con 0x7fe8c006ba70 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a8003520 con 0x7fe8c006ba70 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848089413 0 0) 0x7fe8a8003520 con 0x7fe8c006ba70 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 msgr2=0x7fe8c006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 0x7fe8c006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 >> v1:172.21.15.157:6789/0 conn(0x7fe8c015abe0 legacy=0x7fe8c00712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c006f890 con 0x7fe8c006ba70 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe8b8001b80 con 0x7fe8c006ba70 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 >> v1:172.21.15.73:6789/0 conn(0x7fe8c006ba70 legacy=0x7fe8c0164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 0x7fe8a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a8004d00 0x7fe8a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c006f5b0 con 0x7fe8a80039a0 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a800bb70 con 0x7fe8a8004d00 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.668+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a800bd30 con 0x7fe8c006c9a0 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 -- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 msgr2=0x7fe8a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c006c9a0 0x7fe8a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5c00640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a8004d00 0x7fe8a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 -- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a8004d00 msgr2=0x7fe8a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a8004d00 0x7fe8a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5200640 1 -- 172.21.15.73:0/961050871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8003520 con 0x7fe8a80039a0 2024-04-05T22:26:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.669+0000 7fe8c5c00640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a8004d00 0x7fe8a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:01.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.670+0000 7fe8c5200640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8a8003dc0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4001750 tx=0x7fe8a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306cb4bc1fe92750 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40052d0 con 0x7fe8a80039a0 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b402e050 con 0x7fe8a80039a0 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8bf400640 1 -- 172.21.15.73:0/961050871 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40035f0 con 0x7fe8a80039a0 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 -- 172.21.15.73:0/961050871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 msgr2=0x7fe8a8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/961050871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8a8003dc0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4001750 tx=0x7fe8a80057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c5c00640 1 -- 172.21.15.73:0/961050871 reap_dead start 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 -- 172.21.15.73:0/961050871 shutdown_connections 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 -- 172.21.15.73:0/961050871 >> 172.21.15.73:0/961050871 conn(0x7fe8c0069cb0 msgr2=0x7fe8c006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 -- 172.21.15.73:0/961050871 shutdown_connections 2024-04-05T22:26:01.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.671+0000 7fe8c7c00640 1 -- 172.21.15.73:0/961050871 wait complete. 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.673+0000 7fe8c7c00640 1 Processor -- start 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.673+0000 7fe8c7c00640 1 -- start start 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c015b170 0x7fe8c013b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 0x7fe8c01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0159a20 con 0x7fe8a80039a0 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c013b740 con 0x7fe8c0159060 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c015b6e0 con 0x7fe8c015b170 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56838/0 (socket says 172.21.15.73:56838) 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.674+0000 7fe8c5200640 1 -- 172.21.15.73:0/3256791398 learned_addr learned my addr 172.21.15.73:0/3256791398 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 0x7fe8c01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8bfe00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c015b170 0x7fe8c013b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 -- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c015b170 msgr2=0x7fe8c013b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c015b170 0x7fe8c013b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 -- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 msgr2=0x7fe8c01594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 0x7fe8c01594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 -- 172.21.15.73:0/3256791398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c015b9c0 con 0x7fe8a80039a0 2024-04-05T22:26:01.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 0x7fe8c01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c5200640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4008fd0 tx=0x7fe8b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40076c0 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4007a30 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4006c00 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.675+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c015bca0 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.676+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c015c260 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.677+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c015c3f0 con 0x7fe8a80039a0 2024-04-05T22:26:01.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.678+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe8b40035f0 con 0x7fe8a80039a0 2024-04-05T22:26:01.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.678+0000 7fe8bcc00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 0x7fe8a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:01.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.679+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe8b40ab7b0 con 0x7fe8a80039a0 2024-04-05T22:26:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.679+0000 7fe8bfe00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 0x7fe8a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:01.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.679+0000 7fe8bfe00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 0x7fe8a006d420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8c015e170 tx=0x7fe8ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:01.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.683+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe8b407a1c0 con 0x7fe8a80039a0 2024-04-05T22:26:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.872+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8c0068160 con 0x7fe8a80039a0 2024-04-05T22:26:01.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.873+0000 7fe8bcc00640 1 -- 172.21.15.73:0/3256791398 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8b4079d50 con 0x7fe8a80039a0 2024-04-05T22:26:01.873 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:01.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 msgr2=0x7fe8a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 0x7fe8a006d420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8c015e170 tx=0x7fe8ac008040 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 msgr2=0x7fe8c0149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4008fd0 tx=0x7fe8b4001120 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 shutdown_connections 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe8a006af60 0x7fe8a006d420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c0159060 0x7fe8c01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8c015b170 0x7fe8c013b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 --2- 172.21.15.73:0/3256791398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a80039a0 0x7fe8c0149740 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 >> 172.21.15.73:0/3256791398 conn(0x7fe8c0069cb0 msgr2=0x7fe8c00acf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 shutdown_connections 2024-04-05T22:26:01.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:01.875+0000 7fe8c7c00640 1 -- 172.21.15.73:0/3256791398 wait complete. 2024-04-05T22:26:01.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-05T22:26:02.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.098+0000 7f7853c00640 1 Processor -- start 2024-04-05T22:26:02.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.098+0000 7f7853c00640 1 -- start start 2024-04-05T22:26:02.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7853c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f784c15cf50 0x7f784c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:02.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7853c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c0cb950 con 0x7f784c0cd7a0 2024-04-05T22:26:02.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7853c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c0cdce0 con 0x7f784c0d09b0 2024-04-05T22:26:02.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7853c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c0ce990 con 0x7f784c15cf50 2024-04-05T22:26:02.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7851200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f784c15cf50 0x7f784c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:02.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7851c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f784c0cd7a0 0x7f784c0cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34348/0 (socket says 172.21.15.73:34348) 2024-04-05T22:26:02.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.100+0000 7f7851c00640 1 -- 172.21.15.73:0/4001298779 learned_addr learned my addr 172.21.15.73:0/4001298779 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:02.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.101+0000 7f7851200640 1 -- 172.21.15.73:0/4001298779 >> v1:172.21.15.157:6789/0 conn(0x7f784c0d09b0 legacy=0x7f784c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:02.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.101+0000 7f7851200640 1 -- 172.21.15.73:0/4001298779 >> v1:172.21.15.73:6789/0 conn(0x7f784c0cd7a0 legacy=0x7f784c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:02.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.101+0000 7f7851200640 1 -- 172.21.15.73:0/4001298779 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0cdfc0 con 0x7f784c15cf50 2024-04-05T22:26:02.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.102+0000 7f7851200640 1 --2- 172.21.15.73:0/4001298779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c06ab60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f783c0097a0 tx=0x7f783c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255622dddecf44a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:02.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.102+0000 7f784be00640 1 -- 172.21.15.73:0/4001298779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c031830 con 0x7f784c15cf50 2024-04-05T22:26:02.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.102+0000 7f784be00640 1 -- 172.21.15.73:0/4001298779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c031990 con 0x7f784c15cf50 2024-04-05T22:26:02.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.102+0000 7f784be00640 1 -- 172.21.15.73:0/4001298779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c031d20 con 0x7f784c15cf50 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 -- 172.21.15.73:0/4001298779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 msgr2=0x7f784c06ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 --2- 172.21.15.73:0/4001298779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c06ab60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f783c0097a0 tx=0x7f783c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 -- 172.21.15.73:0/4001298779 shutdown_connections 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 --2- 172.21.15.73:0/4001298779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c06ab60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 -- 172.21.15.73:0/4001298779 >> 172.21.15.73:0/4001298779 conn(0x7f784c099a60 msgr2=0x7f784c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:02.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 -- 172.21.15.73:0/4001298779 shutdown_connections 2024-04-05T22:26:02.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.103+0000 7f7853c00640 1 -- 172.21.15.73:0/4001298779 wait complete. 2024-04-05T22:26:02.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.104+0000 7f7853c00640 1 Processor -- start 2024-04-05T22:26:02.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.104+0000 7f7853c00640 1 -- start start 2024-04-05T22:26:02.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f784c0cd7a0 0x7f784c148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f784c0d09b0 0x7f784c14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0c9160 con 0x7f784c15cf50 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0c9440 con 0x7f784c0cd7a0 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.105+0000 7f7853c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0c9720 con 0x7f784c0d09b0 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56864/0 (socket says 172.21.15.73:56864) 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 -- 172.21.15.73:0/1703447409 learned_addr learned my addr 172.21.15.73:0/1703447409 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 -- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f784c0d09b0 msgr2=0x7f784c14bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f784c0d09b0 0x7f784c14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 -- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f784c0cd7a0 msgr2=0x7f784c148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f784c0cd7a0 0x7f784c148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 -- 172.21.15.73:0/1703447409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c009400 con 0x7f784c15cf50 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7851c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f784400caf0 tx=0x7f784400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400aca0 con 0x7f784c15cf50 2024-04-05T22:26:02.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784400ae00 con 0x7f784c15cf50 2024-04-05T22:26:02.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.106+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844012600 con 0x7f784c15cf50 2024-04-05T22:26:02.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.107+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c0c9a00 con 0x7f784c15cf50 2024-04-05T22:26:02.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.107+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c14c2c0 con 0x7f784c15cf50 2024-04-05T22:26:02.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.108+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0c9f20 con 0x7f784c15cf50 2024-04-05T22:26:02.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.109+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f7844007070 con 0x7f784c15cf50 2024-04-05T22:26:02.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.109+0000 7f7849600640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 0x7f782806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:02.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.110+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78440894f0 con 0x7f784c15cf50 2024-04-05T22:26:02.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.110+0000 7f7851200640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 0x7f782806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:02.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.111+0000 7f7851200640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 0x7f782806d450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f783c036000 tx=0x7f783c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:02.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.114+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f784c0c9f20 con 0x7f784c15cf50 2024-04-05T22:26:02.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:02.376+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f784c0c9f20 con 0x7f784c15cf50 2024-04-05T22:26:03.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.001+0000 7f7849600640 1 -- 172.21.15.73:0/1703447409 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v251) v1 ==== 134+0+0 (secure 0 0 0) 0x7f784c0c9f20 con 0x7f784c15cf50 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 msgr2=0x7f782806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 0x7f782806d450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f783c036000 tx=0x7f783c030000 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 msgr2=0x7f784c0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f784400caf0 tx=0x7f784400a060 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 shutdown_connections 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f782806af90 0x7f782806d450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f784c15cf50 0x7f784c0b0420 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f784c0d09b0 0x7f784c14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 --2- 172.21.15.73:0/1703447409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f784c0cd7a0 0x7f784c148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.005+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 >> 172.21.15.73:0/1703447409 conn(0x7f784c099a60 msgr2=0x7f784c06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.006+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 shutdown_connections 2024-04-05T22:26:03.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.006+0000 7f7853c00640 1 -- 172.21.15.73:0/1703447409 wait complete. 2024-04-05T22:26:03.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:26:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.234+0000 7f3b15400640 1 Processor -- start 2024-04-05T22:26:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.234+0000 7f3b15400640 1 -- start start 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.235+0000 7f3b15400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 0x7f3b1015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.235+0000 7f3b15400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1006bc50 con 0x7f3b10070640 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.235+0000 7f3b15400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1015ef70 con 0x7f3b1015e580 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.235+0000 7f3b15400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1015f530 con 0x7f3b1015a850 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b14a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3b10070640 0x7f3b1006c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34360/0 (socket says 172.21.15.73:34360) 2024-04-05T22:26:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b14a00640 1 -- 172.21.15.73:0/1251012320 learned_addr learned my addr 172.21.15.73:0/1251012320 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:03.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899567972 0 0) 0x7f3b1006bc50 con 0x7f3b10070640 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0fe00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 0x7f3b1015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af4003520 con 0x7f3b10070640 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 84958336 0 0) 0x7f3af4003520 con 0x7f3b10070640 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 msgr2=0x7f3b1015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.236+0000 7f3b0f400640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 0x7f3b1015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 >> v1:172.21.15.157:6789/0 conn(0x7f3b1015e580 legacy=0x7f3b1006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1015f250 con 0x7f3b10070640 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3b08001670 con 0x7f3b10070640 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 >> v1:172.21.15.73:6789/0 conn(0x7f3b10070640 legacy=0x7f3b1006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af40039a0 0x7f3af4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 0x7f3af40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af4004d00 0x7f3af4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1015f530 con 0x7f3af4004d00 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af400bb10 con 0x7f3af40039a0 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af400bca0 con 0x7f3b1015a850 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af4004d00 0x7f3af4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b0fe00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af40039a0 0x7f3af4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 -- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 msgr2=0x7f3af40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b1015a850 0x7f3af40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 -- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af40039a0 msgr2=0x7f3af4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af40039a0 0x7f3af4003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.237+0000 7f3b14a00640 1 -- 172.21.15.73:0/1251012320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4003520 con 0x7f3af4004d00 2024-04-05T22:26:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.238+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af4004d00 0x7f3af4005180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c270 tx=0x7f3af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611c288cfae549f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08005ad0 con 0x7f3af4004d00 2024-04-05T22:26:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b08005c50 con 0x7f3af4004d00 2024-04-05T22:26:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b0f400640 1 -- 172.21.15.73:0/1251012320 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0802c8a0 con 0x7f3af4004d00 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 -- 172.21.15.73:0/1251012320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af4004d00 msgr2=0x7f3af4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 --2- 172.21.15.73:0/1251012320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af4004d00 0x7f3af4005180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c270 tx=0x7f3af40057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b14a00640 1 -- 172.21.15.73:0/1251012320 reap_dead start 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 -- 172.21.15.73:0/1251012320 shutdown_connections 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 -- 172.21.15.73:0/1251012320 >> 172.21.15.73:0/1251012320 conn(0x7f3b10069cb0 msgr2=0x7f3b1006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 -- 172.21.15.73:0/1251012320 shutdown_connections 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.239+0000 7f3b15400640 1 -- 172.21.15.73:0/1251012320 wait complete. 2024-04-05T22:26:03.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.241+0000 7f3b15400640 1 Processor -- start 2024-04-05T22:26:03.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.241+0000 7f3b15400640 1 -- start start 2024-04-05T22:26:03.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af4004d00 0x7f3b101541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b100c7a80 0x7f3b100cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c9150 con 0x7f3b100c7f00 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c84d0 con 0x7f3af4004d00 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b15400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c87b0 con 0x7f3b100c7a80 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b14a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b14a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56888/0 (socket says 172.21.15.73:56888) 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.242+0000 7f3b14a00640 1 -- 172.21.15.73:0/1351742482 learned_addr learned my addr 172.21.15.73:0/1351742482 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b0fe00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af4004d00 0x7f3b101541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 -- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b100c7a80 msgr2=0x7f3b100cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b100c7a80 0x7f3b100cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 -- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af4004d00 msgr2=0x7f3b101541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af4004d00 0x7f3b101541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 -- 172.21.15.73:0/1351742482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100c8a90 con 0x7f3b100c7f00 2024-04-05T22:26:03.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b14a00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3b08001930 tx=0x7f3b0802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0803c040 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0802ccb0 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08007290 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b100c8d70 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.243+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b10159960 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.244+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b100c9290 con 0x7f3b100c7f00 2024-04-05T22:26:03.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.245+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f3b08032070 con 0x7f3b100c7f00 2024-04-05T22:26:03.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.246+0000 7f3b0cc00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 0x7f3aec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:03.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.246+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3b080a9e10 con 0x7f3b100c7f00 2024-04-05T22:26:03.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.246+0000 7f3b0fe00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 0x7f3aec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:03.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.247+0000 7f3b0fe00640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 0x7f3aec06d450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009e10 tx=0x7f3b0000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:03.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.250+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3b100c9290 con 0x7f3b100c7f00 2024-04-05T22:26:03.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.430+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b100c9290 con 0x7f3b100c7f00 2024-04-05T22:26:03.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.439+0000 7f3b0cc00640 1 -- 172.21.15.73:0/1351742482 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b100c9290 con 0x7f3b100c7f00 2024-04-05T22:26:03.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 msgr2=0x7f3aec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 0x7f3aec06d450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009e10 tx=0x7f3b0000b040 comp rx=0 tx=0).stop 2024-04-05T22:26:03.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 msgr2=0x7f3b100cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3b08001930 tx=0x7f3b0802e070 comp rx=0 tx=0).stop 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 shutdown_connections 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f3aec06af90 0x7f3aec06d450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b100c7f00 0x7f3b100cb0f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b100c7a80 0x7f3b100cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 --2- 172.21.15.73:0/1351742482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af4004d00 0x7f3b101541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 >> 172.21.15.73:0/1351742482 conn(0x7f3b10069cb0 msgr2=0x7f3b10154c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 shutdown_connections 2024-04-05T22:26:03.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:03.442+0000 7f3b15400640 1 -- 172.21.15.73:0/1351742482 wait complete. 2024-04-05T22:26:03.459 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:26:03.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:03.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:03.676 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:26:03.676 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:03.676 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:03.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:03.728 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:03.728 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:03.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:03.785 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:03.786 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:03.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:03.841 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:03.841 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:03.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:03.899 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:26:03.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:04.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.115+0000 7f6855200640 1 Processor -- start 2024-04-05T22:26:04.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.115+0000 7f6855200640 1 -- start start 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.116+0000 7f6855200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f68501587d0 0x7f68500cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.116+0000 7f6855200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f68500ccd20 0x7f685006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.116+0000 7f6855200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685006f5b0 con 0x7f685006d0c0 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.116+0000 7f6855200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6850070410 con 0x7f68501587d0 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.116+0000 7f6855200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6850158bb0 con 0x7f68500ccd20 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f68501587d0 0x7f68500cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f68501587d0 0x7f68500cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56898/0 (socket says 172.21.15.73:56898) 2024-04-05T22:26:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 -- 172.21.15.73:0/3943842421 learned_addr learned my addr 172.21.15.73:0/3943842421 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 -- 172.21.15.73:0/3943842421 >> v2:172.21.15.157:3300/0 conn(0x7f68500ccd20 msgr2=0x7f685006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 --2- 172.21.15.73:0/3943842421 >> v2:172.21.15.157:3300/0 conn(0x7f68500ccd20 0x7f685006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 -- 172.21.15.73:0/3943842421 >> v1:172.21.15.157:6789/0 conn(0x7f685006d0c0 legacy=0x7f685015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.117+0000 7f684de00640 1 -- 172.21.15.73:0/3943842421 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685006f800 con 0x7f68501587d0 2024-04-05T22:26:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.119+0000 7f684de00640 1 --2- 172.21.15.73:0/3943842421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68501587d0 0x7f68500cc7e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f684002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d9de752fd3f80f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.119+0000 7f684d400640 1 -- 172.21.15.73:0/3943842421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840031870 con 0x7f68501587d0 2024-04-05T22:26:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.119+0000 7f684d400640 1 -- 172.21.15.73:0/3943842421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400319d0 con 0x7f68501587d0 2024-04-05T22:26:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.119+0000 7f684d400640 1 -- 172.21.15.73:0/3943842421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840031d60 con 0x7f68501587d0 2024-04-05T22:26:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 -- 172.21.15.73:0/3943842421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68501587d0 msgr2=0x7f68500cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 --2- 172.21.15.73:0/3943842421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68501587d0 0x7f68500cc7e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f684002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 -- 172.21.15.73:0/3943842421 shutdown_connections 2024-04-05T22:26:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 --2- 172.21.15.73:0/3943842421 >> v2:172.21.15.157:3300/0 conn(0x7f68500ccd20 0x7f685006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 --2- 172.21.15.73:0/3943842421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68501587d0 0x7f68500cc7e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 -- 172.21.15.73:0/3943842421 >> 172.21.15.73:0/3943842421 conn(0x7f68500ce8e0 msgr2=0x7f68500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 -- 172.21.15.73:0/3943842421 shutdown_connections 2024-04-05T22:26:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.120+0000 7f6855200640 1 -- 172.21.15.73:0/3943842421 wait complete. 2024-04-05T22:26:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.122+0000 7f6855200640 1 Processor -- start 2024-04-05T22:26:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.122+0000 7f6855200640 1 -- start start 2024-04-05T22:26:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68501587d0 0x7f68501e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685013c420 0x7f685013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685013aec0 con 0x7f685006d0c0 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685014e7b0 con 0x7f68501587d0 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f6855200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685013d1a0 con 0x7f685013c420 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56900/0 (socket says 172.21.15.73:56900) 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 -- 172.21.15.73:0/394543408 learned_addr learned my addr 172.21.15.73:0/394543408 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 -- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685013c420 msgr2=0x7f685013c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685013c420 0x7f685013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 -- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68501587d0 msgr2=0x7f68501e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68501587d0 0x7f68501e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.123+0000 7f684de00640 1 -- 172.21.15.73:0/394543408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400094d0 con 0x7f685006d0c0 2024-04-05T22:26:04.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f684de00640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f68400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684003ad30 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684003ae90 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840044610 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685013d480 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.124+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685013da40 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.125+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6850066af0 con 0x7f685006d0c0 2024-04-05T22:26:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.126+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f684004d450 con 0x7f685006d0c0 2024-04-05T22:26:04.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.127+0000 7f6846a00640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.127+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f684002d040 con 0x7f685006d0c0 2024-04-05T22:26:04.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.127+0000 7f6845400640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.128+0000 7f6845400640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 0x7f682c06d420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f685015d770 tx=0x7f6838009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.131+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6840076e80 con 0x7f685006d0c0 2024-04-05T22:26:04.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.361+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6850068160 con 0x7f685006d0c0 2024-04-05T22:26:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.362+0000 7f6846a00640 1 -- 172.21.15.73:0/394543408 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f684007a4a0 con 0x7f685006d0c0 2024-04-05T22:26:04.362 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:04.362 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2290039504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2290039504},{"type":"v1","addr":"172.21.15.157:6834","nonce":2290039504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2652316368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2652316368},{"type":"v1","addr":"172.21.15.157:6835","nonce":2652316368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2871192307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2871192307},{"type":"v1","addr":"172.21.15.73:6836","nonce":2871192307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6606,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/388225203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":388225203},{"type":"v1","addr":"172.21.15.73:6837","nonce":388225203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2024-04-05T22:26:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 35 2024-04-05T22:26:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 msgr2=0x7f682c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 0x7f682c06d420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f685015d770 tx=0x7f6838009250 comp rx=0 tx=0).stop 2024-04-05T22:26:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 msgr2=0x7f685015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f68400046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 shutdown_connections 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f685013c420 0x7f685013c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68501587d0 0x7f68501e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 --2- 172.21.15.73:0/394543408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f685006d0c0 0x7f685015cb10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.364+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 >> 172.21.15.73:0/394543408 conn(0x7f68500ce8e0 msgr2=0x7f6850159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.365+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 shutdown_connections 2024-04-05T22:26:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.365+0000 7f6855200640 1 -- 172.21.15.73:0/394543408 wait complete. 2024-04-05T22:26:04.382 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:26:04.382 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:26:04.382 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:26:04.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:04.383 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:26:04.383 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:26:04.383 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:26:04.383 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:04.383 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:26:04.383 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:26:04.383 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:26:04.383 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:04.384 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:26:04.384 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:26:04.384 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:26:04.384 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:04.403 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:26:04.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:26:04.406 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:26:04.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:26:04.408 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:26:04.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:26:04.409 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:26:04.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:26:04.455 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:26:04.455 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:04.456 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:26:04.456 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:04.461 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:26:04.461 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:04.462 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:26:04.462 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:04.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:26:04.613 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:26:04.613 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:26:04.619 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:26:04.621 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:26:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.657+0000 7ffa4b000640 1 Processor -- start 2024-04-05T22:26:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.657+0000 7ffa4b000640 1 -- start start 2024-04-05T22:26:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.658+0000 7ffa4b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 0x7ffa440d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.658+0000 7ffa4b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa440d1800 0x7ffa440b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.658+0000 7ffa4b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440af6d0 con 0x7ffa440d09d0 2024-04-05T22:26:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.658+0000 7ffa4b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa440b22b0 con 0x7ffa440d1800 2024-04-05T22:26:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.658+0000 7ffa4b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440b2590 con 0x7ffa441605d0 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa49200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ffa440d09d0 0x7ffa440d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34368/0 (socket says 172.21.15.73:34368) 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa49200640 1 -- 172.21.15.73:0/1349045646 learned_addr learned my addr 172.21.15.73:0/1349045646 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa4a600640 1 --2- 172.21.15.73:0/1349045646 >> v2:172.21.15.73:3300/0 conn(0x7ffa440d1800 0x7ffa440b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa49c00640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 0x7ffa440d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa43e00640 1 -- 172.21.15.73:0/1349045646 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197926858 0 0) 0x7ffa440af6d0 con 0x7ffa440d09d0 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa43e00640 1 -- 172.21.15.73:0/1349045646 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa2c003520 con 0x7ffa440d09d0 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa4a600640 1 -- 172.21.15.73:0/1349045646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 msgr2=0x7ffa440d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa4a600640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 0x7ffa440d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa4a600640 1 -- 172.21.15.73:0/1349045646 >> v1:172.21.15.73:6789/0 conn(0x7ffa440d09d0 legacy=0x7ffa440d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa4a600640 1 -- 172.21.15.73:0/1349045646 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa440b2870 con 0x7ffa440d1800 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.659+0000 7ffa49c00640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 0x7ffa440d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:26:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.660+0000 7ffa4a600640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d1800 0x7ffa440b3350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d0424aa220f6a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.660+0000 7ffa43e00640 1 -- 172.21.15.73:0/1349045646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c02cce0 con 0x7ffa440d1800 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.660+0000 7ffa43e00640 1 -- 172.21.15.73:0/1349045646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c02ce40 con 0x7ffa440d1800 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa43e00640 1 -- 172.21.15.73:0/1349045646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c034660 con 0x7ffa440d1800 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 -- 172.21.15.73:0/1349045646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d1800 msgr2=0x7ffa440b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d1800 0x7ffa440b3350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c02bf30 comp rx=0 tx=0).stop 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 -- 172.21.15.73:0/1349045646 shutdown_connections 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d1800 0x7ffa440b3350 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 --2- 172.21.15.73:0/1349045646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa441605d0 0x7ffa440d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 -- 172.21.15.73:0/1349045646 >> 172.21.15.73:0/1349045646 conn(0x7ffa440d2b10 msgr2=0x7ffa440d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 -- 172.21.15.73:0/1349045646 shutdown_connections 2024-04-05T22:26:04.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.661+0000 7ffa4b000640 1 -- 172.21.15.73:0/1349045646 wait complete. 2024-04-05T22:26:04.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.663+0000 7ffa4b000640 1 Processor -- start 2024-04-05T22:26:04.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.663+0000 7ffa4b000640 1 -- start start 2024-04-05T22:26:04.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa440d1800 0x7ffa440b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa441605d0 0x7ffa440b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415c1a0 con 0x7ffa440d09d0 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440b5c40 con 0x7ffa441605d0 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440b8860 con 0x7ffa440d1800 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa49c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa49c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56942/0 (socket says 172.21.15.73:56942) 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa49c00640 1 -- 172.21.15.73:0/3056217625 learned_addr learned my addr 172.21.15.73:0/3056217625 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa49200640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa440d1800 0x7ffa440b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.664+0000 7ffa4a600640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa441605d0 0x7ffa440b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 -- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa440d1800 msgr2=0x7ffa440b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa440d1800 0x7ffa440b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 -- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa441605d0 msgr2=0x7ffa440b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa441605d0 0x7ffa440b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 -- 172.21.15.73:0/3056217625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009a70 con 0x7ffa440d09d0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa49c00640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d7d0 tx=0x7ffa3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004030 con 0x7ffa440d09d0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa38004190 con 0x7ffa440d09d0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004520 con 0x7ffa440d09d0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa440c8270 con 0x7ffa440d09d0 2024-04-05T22:26:04.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.665+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa440c8830 con 0x7ffa440d09d0 2024-04-05T22:26:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.666+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4404e160 con 0x7ffa440d09d0 2024-04-05T22:26:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.667+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7ffa380096a0 con 0x7ffa440d09d0 2024-04-05T22:26:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.668+0000 7ffa41600640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 0x7ffa2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.668+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa38089580 con 0x7ffa440d09d0 2024-04-05T22:26:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.668+0000 7ffa49200640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 0x7ffa2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.669+0000 7ffa49200640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 0x7ffa2406d450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007ac0 tx=0x7ffa30007a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.673+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ffa38054700 con 0x7ffa440d09d0 2024-04-05T22:26:04.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:04.871+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffa440d0ed0 con 0x7ffa440d09d0 2024-04-05T22:26:05.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.651+0000 7ffa41600640 1 -- 172.21.15.73:0/3056217625 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffa38057d20 con 0x7ffa440d09d0 2024-04-05T22:26:05.651 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 msgr2=0x7ffa2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 0x7ffa2406d450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007ac0 tx=0x7ffa30007a30 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 msgr2=0x7ffa4415bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d7d0 tx=0x7ffa3800dc90 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 shutdown_connections 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ffa2406af90 0x7ffa2406d450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa441605d0 0x7ffa440b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa440d1800 0x7ffa440b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 --2- 172.21.15.73:0/3056217625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa440d09d0 0x7ffa4415bc60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 >> 172.21.15.73:0/3056217625 conn(0x7ffa440d2b10 msgr2=0x7ffa440b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 shutdown_connections 2024-04-05T22:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.654+0000 7ffa4b000640 1 -- 172.21.15.73:0/3056217625 wait complete. 2024-04-05T22:26:05.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:26:05.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.866+0000 7fbe7b000640 1 Processor -- start 2024-04-05T22:26:05.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.867+0000 7fbe7b000640 1 -- start start 2024-04-05T22:26:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe7b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe7406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe7b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7406f5b0 con 0x7fbe7406cb80 2024-04-05T22:26:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe7b000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe74070410 con 0x7fbe741587d0 2024-04-05T22:26:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe7b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7406d500 con 0x7fbe7415ef60 2024-04-05T22:26:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe6b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbe7406cb80 0x7fbe7415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59974/0 (socket says 172.21.15.73:59974) 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.868+0000 7fbe6b400640 1 -- 172.21.15.73:0/3126448370 learned_addr learned my addr 172.21.15.73:0/3126448370 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420303963 0 0) 0x7fbe7406f5b0 con 0x7fbe7406cb80 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe58003520 con 0x7fbe7406cb80 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe79000640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe7406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3938068261 0 0) 0x7fbe58003520 con 0x7fbe7406cb80 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 msgr2=0x7fbe7406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe7406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 >> v1:172.21.15.157:6789/0 conn(0x7fbe741587d0 legacy=0x7fbe740cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe740ccef0 con 0x7fbe7406cb80 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbe5c001670 con 0x7fbe7406cb80 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 >> v1:172.21.15.73:6789/0 conn(0x7fbe7406cb80 legacy=0x7fbe7415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe580039a0 0x7fbe58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.869+0000 7fbe73400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580047c0 0x7fbe58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7406d500 con 0x7fbe580039a0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5800bb10 con 0x7fbe580047c0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5800bcd0 con 0x7fbe7415ef60 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe580039a0 0x7fbe58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe6b400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580047c0 0x7fbe58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe79000640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 -- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 msgr2=0x7fbe580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe7415ef60 0x7fbe580051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 -- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580047c0 msgr2=0x7fbe58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580047c0 0x7fbe58004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe73e00640 1 -- 172.21.15.73:0/3126448370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58003520 con 0x7fbe580039a0 2024-04-05T22:26:05.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.870+0000 7fbe6b400640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580047c0 0x7fbe58004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.871+0000 7fbe73e00640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe580039a0 0x7fbe58003dc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbe64001750 tx=0x7fbe580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48d5e330814df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.871+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640052d0 con 0x7fbe580039a0 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.871+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6402e050 con 0x7fbe580039a0 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.871+0000 7fbe73400640 1 -- 172.21.15.73:0/3126448370 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640035f0 con 0x7fbe580039a0 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 -- 172.21.15.73:0/3126448370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe580039a0 msgr2=0x7fbe58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 --2- 172.21.15.73:0/3126448370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe580039a0 0x7fbe58003dc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbe64001750 tx=0x7fbe580057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe79000640 1 -- 172.21.15.73:0/3126448370 reap_dead start 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 -- 172.21.15.73:0/3126448370 shutdown_connections 2024-04-05T22:26:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 -- 172.21.15.73:0/3126448370 >> 172.21.15.73:0/3126448370 conn(0x7fbe740ce8e0 msgr2=0x7fbe740cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:05.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 -- 172.21.15.73:0/3126448370 shutdown_connections 2024-04-05T22:26:05.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.872+0000 7fbe7b000640 1 -- 172.21.15.73:0/3126448370 wait complete. 2024-04-05T22:26:05.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.874+0000 7fbe7b000640 1 Processor -- start 2024-04-05T22:26:05.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.874+0000 7fbe7b000640 1 -- start start 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580039a0 0x7fbe740b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe740ac8d0 0x7fbe740acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b39f0 con 0x7fbe740cd110 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740ad2a0 con 0x7fbe580039a0 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe7b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74153640 con 0x7fbe740ac8d0 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49614/0 (socket says 172.21.15.73:49614) 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 -- 172.21.15.73:0/2971973989 learned_addr learned my addr 172.21.15.73:0/2971973989 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 -- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe740ac8d0 msgr2=0x7fbe740acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe740ac8d0 0x7fbe740acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 -- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580039a0 msgr2=0x7fbe740b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580039a0 0x7fbe740b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.875+0000 7fbe6b400640 1 -- 172.21.15.73:0/2971973989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe741537d0 con 0x7fbe740cd110 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe6b400640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00d8b0 tx=0x7fbe5c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0074e0 con 0x7fbe740cd110 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c007640 con 0x7fbe740cd110 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0049e0 con 0x7fbe740cd110 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe740cd5a0 con 0x7fbe740cd110 2024-04-05T22:26:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.876+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe740cdb60 con 0x7fbe740cd110 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.877+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe74066af0 con 0x7fbe740cd110 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.878+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fbe5c004b40 con 0x7fbe740cd110 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.879+0000 7fbe70c00640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 0x7fbe5006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.879+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe5c089060 con 0x7fbe740cd110 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.879+0000 7fbe73e00640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 0x7fbe5006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:05.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.879+0000 7fbe73e00640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 0x7fbe5006d420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbe64033000 tx=0x7fbe6402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:05.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:05.883+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbe5c0bc650 con 0x7fbe740cd110 2024-04-05T22:26:06.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.137+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbe74153960 con 0x7fbe740cd110 2024-04-05T22:26:06.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.138+0000 7fbe70c00640 1 -- 172.21.15.73:0/2971973989 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5617 (secure 0 0 0) 0x7fbe5c019020 con 0x7fbe740cd110 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:26:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:26:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:26:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:26:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.alice", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBpehBmmHh6ARAAy1xTWlOJ4b0Dr8dStWw8bQ==", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol43130", 2024-04-05T22:26:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:26:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:06.152 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.140+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 msgr2=0x7fbe5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.140+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 0x7fbe5006d420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbe64033000 tx=0x7fbe6402f000 comp rx=0 tx=0).stop 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.140+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 msgr2=0x7fbe74151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.140+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00d8b0 tx=0x7fbe5c00dd70 comp rx=0 tx=0).stop 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 shutdown_connections 2024-04-05T22:26:06.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbe5006af60 0x7fbe5006d420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe740ac8d0 0x7fbe740acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe740cd110 0x7fbe74151680 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 --2- 172.21.15.73:0/2971973989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe580039a0 0x7fbe740b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 >> 172.21.15.73:0/2971973989 conn(0x7fbe740ce8e0 msgr2=0x7fbe74158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 shutdown_connections 2024-04-05T22:26:06.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.141+0000 7fbe7b000640 1 -- 172.21.15.73:0/2971973989 wait complete. 2024-04-05T22:26:06.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-04-05T22:26:06.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.357+0000 7fdd04000640 1 Processor -- start 2024-04-05T22:26:06.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.357+0000 7fdd04000640 1 -- start start 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd04000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdcfc1541a0 0x7fdcfc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd04000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 0x7fdcfc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd04000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc15aa60 con 0x7fdcfc06d710 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd04000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b3b60 con 0x7fdcfc1541a0 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd04000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0701c0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd01600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdcfc1541a0 0x7fdcfc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd00c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdcfc06d710 0x7fdcfc0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59982/0 (socket says 172.21.15.73:59982) 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd00c00640 1 -- 172.21.15.73:0/3460546630 learned_addr learned my addr 172.21.15.73:0/3460546630 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdd02000640 1 --2- 172.21.15.73:0/3460546630 >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 0x7fdcfc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.359+0000 7fdcfbe00640 1 -- 172.21.15.73:0/3460546630 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48612955 0 0) 0x7fdcfc15aa60 con 0x7fdcfc06d710 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdcfbe00640 1 -- 172.21.15.73:0/3460546630 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce4003520 con 0x7fdcfc06d710 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdd01600640 1 -- 172.21.15.73:0/3460546630 >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 msgr2=0x7fdcfc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdd01600640 1 --2- 172.21.15.73:0/3460546630 >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 0x7fdcfc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdd01600640 1 -- 172.21.15.73:0/3460546630 >> v1:172.21.15.73:6789/0 conn(0x7fdcfc06d710 legacy=0x7fdcfc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdd01600640 1 -- 172.21.15.73:0/3460546630 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0b3880 con 0x7fdcfc1541a0 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.360+0000 7fdd02000640 1 --2- 172.21.15.73:0/3460546630 >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 0x7fdcfc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:26:06.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.362+0000 7fdd01600640 1 --2- 172.21.15.73:0/3460546630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc1541a0 0x7fdcfc0b3050 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc15a9d0 tx=0x7fdcf002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bcadcf037ddce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.362+0000 7fdcfbe00640 1 -- 172.21.15.73:0/3460546630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf002e070 con 0x7fdcfc1541a0 2024-04-05T22:26:06.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.362+0000 7fdcfbe00640 1 -- 172.21.15.73:0/3460546630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf00315a0 con 0x7fdcfc1541a0 2024-04-05T22:26:06.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.362+0000 7fdcfbe00640 1 -- 172.21.15.73:0/3460546630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0031910 con 0x7fdcfc1541a0 2024-04-05T22:26:06.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 -- 172.21.15.73:0/3460546630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc1541a0 msgr2=0x7fdcfc0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 --2- 172.21.15.73:0/3460546630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc1541a0 0x7fdcfc0b3050 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc15a9d0 tx=0x7fdcf002b840 comp rx=0 tx=0).stop 2024-04-05T22:26:06.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 -- 172.21.15.73:0/3460546630 shutdown_connections 2024-04-05T22:26:06.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 --2- 172.21.15.73:0/3460546630 >> v2:172.21.15.157:3300/0 conn(0x7fdcfc0ce000 0x7fdcfc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 --2- 172.21.15.73:0/3460546630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc1541a0 0x7fdcfc0b3050 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 -- 172.21.15.73:0/3460546630 >> 172.21.15.73:0/3460546630 conn(0x7fdcfc0ce8e0 msgr2=0x7fdcfc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 -- 172.21.15.73:0/3460546630 shutdown_connections 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.363+0000 7fdd04000640 1 -- 172.21.15.73:0/3460546630 wait complete. 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.364+0000 7fdd04000640 1 Processor -- start 2024-04-05T22:26:06.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 -- start start 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc06d710 0x7fdcfc1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1541a0 0x7fdcfc06b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc14af90 con 0x7fdcfc0ce000 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1f6620 con 0x7fdcfc06d710 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.365+0000 7fdd04000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc14d1c0 con 0x7fdcfc1541a0 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49644/0 (socket says 172.21.15.73:49644) 2024-04-05T22:26:06.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 -- 172.21.15.73:0/2191352223 learned_addr learned my addr 172.21.15.73:0/2191352223 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd01600640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc06d710 0x7fdcfc1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 -- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1541a0 msgr2=0x7fdcfc06b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1541a0 0x7fdcfc06b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 -- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc06d710 msgr2=0x7fdcfc1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc06d710 0x7fdcfc1f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 -- 172.21.15.73:0/2191352223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0009400 con 0x7fdcfc0ce000 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.366+0000 7fdd00c00640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c460 tx=0x7fdce800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.367+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8005ce0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.367+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce8005e40 con 0x7fdcfc0ce000 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.367+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8007290 con 0x7fdcfc0ce000 2024-04-05T22:26:06.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.367+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc1f5d80 con 0x7fdcfc0ce000 2024-04-05T22:26:06.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.367+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc1f6340 con 0x7fdcfc0ce000 2024-04-05T22:26:06.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.368+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc066af0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.369+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fdce80073f0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.370+0000 7fdcf9600640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 0x7fdcdc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.370+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdce8088a90 con 0x7fdcfc0ce000 2024-04-05T22:26:06.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.370+0000 7fdd01600640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 0x7fdcdc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.371+0000 7fdd01600640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 0x7fdcdc06d450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.374+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdce8053c00 con 0x7fdcfc0ce000 2024-04-05T22:26:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.618+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7fdcfc14b0d0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.625+0000 7fdcf9600640 1 -- 172.21.15.73:0/2191352223 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v17) v1 ==== 82+0+0 (secure 0 0 0) 0x7fdcfc14b0d0 con 0x7fdcfc0ce000 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 msgr2=0x7fdcdc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 0x7fdcdc06d450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf003d040 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 msgr2=0x7fdcfc14cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c460 tx=0x7fdce800c920 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 shutdown_connections 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fdcdc06af90 0x7fdcdc06d450 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcfc1541a0 0x7fdcfc06b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc0ce000 0x7fdcfc14cc80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 --2- 172.21.15.73:0/2191352223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcfc06d710 0x7fdcfc1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 >> 172.21.15.73:0/2191352223 conn(0x7fdcfc0ce8e0 msgr2=0x7fdcfc153460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.628+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 shutdown_connections 2024-04-05T22:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.629+0000 7fdd04000640 1 -- 172.21.15.73:0/2191352223 wait complete. 2024-04-05T22:26:06.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:06.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.849+0000 7f188b200640 1 Processor -- start 2024-04-05T22:26:06.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.850+0000 7f188b200640 1 -- start start 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f188c0b20a0 0x7f188c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f188c0b1b50 0x7f188c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b2480 0x7f188c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c06d3e0 con 0x7f188c0b1b50 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c06d610 con 0x7f188c0b20a0 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f188b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c070230 con 0x7f188c0b2480 2024-04-05T22:26:06.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f1889400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f188c0b1b50 0x7f188c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f1889400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f188c0b1b50 0x7f188c06cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49656/0 (socket says 172.21.15.73:49656) 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.851+0000 7f1889400640 1 -- 172.21.15.73:0/4141976823 learned_addr learned my addr 172.21.15.73:0/4141976823 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889e00640 1 --2- 172.21.15.73:0/4141976823 >> v2:172.21.15.157:3300/0 conn(0x7f188c0b20a0 0x7f188c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f188a800640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b2480 0x7f188c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889400640 1 -- 172.21.15.73:0/4141976823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b2480 msgr2=0x7f188c0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889400640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b2480 0x7f188c0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889400640 1 -- 172.21.15.73:0/4141976823 >> v2:172.21.15.157:3300/0 conn(0x7f188c0b20a0 msgr2=0x7f188c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889400640 1 --2- 172.21.15.73:0/4141976823 >> v2:172.21.15.157:3300/0 conn(0x7f188c0b20a0 0x7f188c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.852+0000 7f1889400640 1 -- 172.21.15.73:0/4141976823 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c070510 con 0x7f188c0b1b50 2024-04-05T22:26:06.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.853+0000 7f1889400640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c06cea0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f188c0b40b0 tx=0x7f187802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0e1d7ba92bbccc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.853+0000 7f1888a00640 1 -- 172.21.15.73:0/4141976823 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878031750 con 0x7f188c0b1b50 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.853+0000 7f1888a00640 1 -- 172.21.15.73:0/4141976823 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18780318b0 con 0x7f188c0b1b50 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.853+0000 7f1888a00640 1 -- 172.21.15.73:0/4141976823 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878031c40 con 0x7f188c0b1b50 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 -- 172.21.15.73:0/4141976823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 msgr2=0x7f188c06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c06cea0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f188c0b40b0 tx=0x7f187802b7f0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 -- 172.21.15.73:0/4141976823 shutdown_connections 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c06cea0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 --2- 172.21.15.73:0/4141976823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b2480 0x7f188c0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 --2- 172.21.15.73:0/4141976823 >> v2:172.21.15.157:3300/0 conn(0x7f188c0b20a0 0x7f188c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 -- 172.21.15.73:0/4141976823 >> 172.21.15.73:0/4141976823 conn(0x7f188c0b06c0 msgr2=0x7f188c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 -- 172.21.15.73:0/4141976823 shutdown_connections 2024-04-05T22:26:06.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.854+0000 7f188b200640 1 -- 172.21.15.73:0/4141976823 wait complete. 2024-04-05T22:26:06.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.856+0000 7f188b200640 1 Processor -- start 2024-04-05T22:26:06.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.856+0000 7f188b200640 1 -- start start 2024-04-05T22:26:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b20a0 0x7f188c162760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f188c0b2480 0x7f188c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1497f0 con 0x7f188c0b1b50 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c149c80 con 0x7f188c0b2480 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f188b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1619a0 con 0x7f188c0b20a0 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f1889e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f1889e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49668/0 (socket says 172.21.15.73:49668) 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.857+0000 7f1889e00640 1 -- 172.21.15.73:0/3340661150 learned_addr learned my addr 172.21.15.73:0/3340661150 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889400640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b20a0 0x7f188c162760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 -- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b20a0 msgr2=0x7f188c162760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b20a0 0x7f188c162760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 -- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f188c0b2480 msgr2=0x7f188c0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f188c0b2480 0x7f188c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:06.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 -- 172.21.15.73:0/3340661150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780094d0 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1889e00640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d800 tx=0x7f187c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0096a0 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c009800 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c009b90 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c161c80 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.858+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c162240 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.859+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c162400 con 0x7f188c0b1b50 2024-04-05T22:26:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.860+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f187c009cf0 con 0x7f188c0b1b50 2024-04-05T22:26:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.861+0000 7f1882000640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 0x7f186c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.861+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f187c089910 con 0x7f188c0b1b50 2024-04-05T22:26:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.861+0000 7f1889400640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 0x7f186c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.862+0000 7f1889400640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 0x7f186c06d450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1878035000 tx=0x7f1878008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:06.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:06.865+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f187c054a80 con 0x7f188c0b1b50 2024-04-05T22:26:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.057+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f188c06fe90 con 0x7f188c0b1b50 2024-04-05T22:26:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.059+0000 7f1882000640 1 -- 172.21.15.73:0/3340661150 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f187c00b440 con 0x7f188c0b1b50 2024-04-05T22:26:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:07.059 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:05.642924+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 msgr2=0x7f186c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 0x7f186c06d450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1878035000 tx=0x7f1878008030 comp rx=0 tx=0).stop 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 msgr2=0x7f188c1492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d800 tx=0x7f187c00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 shutdown_connections 2024-04-05T22:26:07.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f186c06af90 0x7f186c06d450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f188c0b2480 0x7f188c0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f188c0b20a0 0x7f188c162760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 --2- 172.21.15.73:0/3340661150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f188c0b1b50 0x7f188c1492b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.061+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 >> 172.21.15.73:0/3340661150 conn(0x7f188c0b06c0 msgr2=0x7f188c06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.062+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 shutdown_connections 2024-04-05T22:26:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.062+0000 7f188b200640 1 -- 172.21.15.73:0/3340661150 wait complete. 2024-04-05T22:26:07.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:07.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:07.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.284+0000 7f5d43400640 1 Processor -- start 2024-04-05T22:26:07.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.284+0000 7f5d43400640 1 -- start start 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d43400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d3c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d43400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d3c165310 0x7f5d3c06b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d43400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c15f260 con 0x7f5d3c15ee80 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d43400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c1657c0 con 0x7f5d3c165310 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d43400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c070d20 con 0x7f5d3c0b1950 2024-04-05T22:26:07.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d40a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5d3c15ee80 0x7f5d3c0cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59988/0 (socket says 172.21.15.73:59988) 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d40a00640 1 -- 172.21.15.73:0/749276048 learned_addr learned my addr 172.21.15.73:0/749276048 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d3be00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d3c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d41400640 1 --2- 172.21.15.73:0/749276048 >> v2:172.21.15.157:3300/0 conn(0x7f5d3c165310 0x7f5d3c06b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34704821 0 0) 0x7f5d30001110 con 0x7f5d3c15ee80 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.286+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d24003520 con 0x7f5d3c15ee80 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162155315 0 0) 0x7f5d24003520 con 0x7f5d3c15ee80 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 msgr2=0x7f5d3c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d3c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 >> v2:172.21.15.157:3300/0 conn(0x7f5d3c165310 msgr2=0x7f5d3c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 --2- 172.21.15.73:0/749276048 >> v2:172.21.15.157:3300/0 conn(0x7f5d3c165310 0x7f5d3c06b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c06f5b0 con 0x7f5d3c15ee80 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5d30001800 con 0x7f5d3c15ee80 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 >> v1:172.21.15.73:6789/0 conn(0x7f5d3c15ee80 legacy=0x7f5d3c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d24003970 0x7f5d24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d24004d00 0x7f5d24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1657c0 con 0x7f5d24003970 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2400b980 con 0x7f5d24004d00 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2400bac0 con 0x7f5d3c0b1950 2024-04-05T22:26:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.287+0000 7f5d40a00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d24003970 0x7f5d24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d3be00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d41400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d24004d00 0x7f5d24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d40a00640 1 -- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 msgr2=0x7f5d240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d40a00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d3c0b1950 0x7f5d240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d40a00640 1 -- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d24004d00 msgr2=0x7f5d24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d40a00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d24004d00 0x7f5d24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d40a00640 1 -- 172.21.15.73:0/749276048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24003520 con 0x7f5d24003970 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.288+0000 7f5d41400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d24004d00 0x7f5d24005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.289+0000 7f5d40a00640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d24003970 0x7f5d24003d90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c240 tx=0x7f5d240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e0e5f1c0afaf22 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.289+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30005ad0 con 0x7f5d24003970 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.289+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30005c50 con 0x7f5d24003970 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.289+0000 7f5d3b400640 1 -- 172.21.15.73:0/749276048 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3002c8b0 con 0x7f5d24003970 2024-04-05T22:26:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 -- 172.21.15.73:0/749276048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d24003970 msgr2=0x7f5d24003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 --2- 172.21.15.73:0/749276048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d24003970 0x7f5d24003d90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c240 tx=0x7f5d240057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d41400640 1 -- 172.21.15.73:0/749276048 reap_dead start 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 -- 172.21.15.73:0/749276048 shutdown_connections 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 -- 172.21.15.73:0/749276048 >> 172.21.15.73:0/749276048 conn(0x7f5d3c069cb0 msgr2=0x7f5d3c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 -- 172.21.15.73:0/749276048 shutdown_connections 2024-04-05T22:26:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.290+0000 7f5d43400640 1 -- 172.21.15.73:0/749276048 wait complete. 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.291+0000 7f5d43400640 1 Processor -- start 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.292+0000 7f5d43400640 1 -- start start 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.292+0000 7f5d43400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 0x7f5d3c0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.292+0000 7f5d43400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d43400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c0b2450 0x7f5d3c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d43400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0aeb00 con 0x7f5d3c0b2010 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d43400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1494e0 con 0x7f5d3c0b2450 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d43400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1497c0 con 0x7f5d24003970 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49678/0 (socket says 172.21.15.73:49678) 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 -- 172.21.15.73:0/275799725 learned_addr learned my addr 172.21.15.73:0/275799725 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d41400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c0b2450 0x7f5d3c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d40a00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 0x7f5d3c0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 -- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 msgr2=0x7f5d3c0ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 0x7f5d3c0ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 -- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c0b2450 msgr2=0x7f5d3c148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c0b2450 0x7f5d3c148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d3be00640 1 -- 172.21.15.73:0/275799725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c149aa0 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.293+0000 7f5d40a00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 0x7f5d3c0ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d3be00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d28011590 tx=0x7f5d28011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004220 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28004380 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28015930 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c149d80 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.294+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c14b600 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.295+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c068160 con 0x7f5d3c0b2010 2024-04-05T22:26:07.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.296+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f5d28015a90 con 0x7f5d3c0b2010 2024-04-05T22:26:07.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.297+0000 7f5d38c00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 0x7f5d1c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.297+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d280896d0 con 0x7f5d3c0b2010 2024-04-05T22:26:07.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.297+0000 7f5d40a00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 0x7f5d1c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.298+0000 7f5d40a00640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 0x7f5d1c06d420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d30007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.302+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5d280547f0 con 0x7f5d3c0b2010 2024-04-05T22:26:07.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.497+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d3c06cbd0 con 0x7f5d3c0b2010 2024-04-05T22:26:07.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.498+0000 7f5d38c00640 1 -- 172.21.15.73:0/275799725 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d28057e10 con 0x7f5d3c0b2010 2024-04-05T22:26:07.498 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.500+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 msgr2=0x7f5d1c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.500+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 0x7f5d1c06d420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d30007000 comp rx=0 tx=0).stop 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.500+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 msgr2=0x7f5d3c0cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.500+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d28011590 tx=0x7f5d28011a50 comp rx=0 tx=0).stop 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 shutdown_connections 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5d1c06af60 0x7f5d1c06d420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d3c0b2450 0x7f5d3c148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d3c0b2010 0x7f5d3c0cbd60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 --2- 172.21.15.73:0/275799725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d24003970 0x7f5d3c0ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 >> 172.21.15.73:0/275799725 conn(0x7f5d3c069cb0 msgr2=0x7f5d3c1608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:07.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 shutdown_connections 2024-04-05T22:26:07.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.501+0000 7f5d43400640 1 -- 172.21.15.73:0/275799725 wait complete. 2024-04-05T22:26:07.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:26:07.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.728+0000 7fabfa600640 1 Processor -- start 2024-04-05T22:26:07.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.728+0000 7fabfa600640 1 -- start start 2024-04-05T22:26:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabfa600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabf406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabfa600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fabf415e850 0x7fabf415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabfa600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf406f5b0 con 0x7fabf41587d0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabfa600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf406d500 con 0x7fabf415e850 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabfa600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf415f220 con 0x7fabf406cb80 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabf3400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fabf41587d0 0x7fabf40cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59992/0 (socket says 172.21.15.73:59992) 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabf3400640 1 -- 172.21.15.73:0/247978292 learned_addr learned my addr 172.21.15.73:0/247978292 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabf2a00640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabf406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabf3e00640 1 --2- 172.21.15.73:0/247978292 >> v2:172.21.15.157:3300/0 conn(0x7fabf415e850 0x7fabf415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.730+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117978183 0 0) 0x7fabf406f5b0 con 0x7fabf41587d0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8003520 con 0x7fabf41587d0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574322020 0 0) 0x7fabd8003520 con 0x7fabf41587d0 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 msgr2=0x7fabf406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabf406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 >> v2:172.21.15.157:3300/0 conn(0x7fabf415e850 msgr2=0x7fabf415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 --2- 172.21.15.73:0/247978292 >> v2:172.21.15.157:3300/0 conn(0x7fabf415e850 0x7fabf415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4070410 con 0x7fabf41587d0 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fabe4001670 con 0x7fabf41587d0 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 >> v1:172.21.15.73:6789/0 conn(0x7fabf41587d0 legacy=0x7fabf40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabd80039a0 0x7fabd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf406d500 con 0x7fabd80039a0 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf415f220 con 0x7fabd8004d00 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd800b980 con 0x7fabf406cb80 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.731+0000 7fabf3400640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabd80039a0 0x7fabd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf2a00640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3e00640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3400640 1 -- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 msgr2=0x7fabd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3400640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabf406cb80 0x7fabd80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3400640 1 -- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd8004d00 msgr2=0x7fabd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3400640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3400640 1 -- 172.21.15.73:0/247978292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8003520 con 0x7fabd80039a0 2024-04-05T22:26:07.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.732+0000 7fabf3e00640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.733+0000 7fabf3400640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabd80039a0 0x7fabd8003dc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c240 tx=0x7fabd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e7c36e66017477 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.733+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4005ad0 con 0x7fabd80039a0 2024-04-05T22:26:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.733+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe4005c30 con 0x7fabd80039a0 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.733+0000 7fabf2000640 1 -- 172.21.15.73:0/247978292 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe402c870 con 0x7fabd80039a0 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 -- 172.21.15.73:0/247978292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabd80039a0 msgr2=0x7fabd8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 --2- 172.21.15.73:0/247978292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabd80039a0 0x7fabd8003dc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c240 tx=0x7fabd80057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabf3e00640 1 -- 172.21.15.73:0/247978292 reap_dead start 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 -- 172.21.15.73:0/247978292 shutdown_connections 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 -- 172.21.15.73:0/247978292 >> 172.21.15.73:0/247978292 conn(0x7fabf40ce8e0 msgr2=0x7fabf40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 -- 172.21.15.73:0/247978292 shutdown_connections 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.734+0000 7fabfa600640 1 -- 172.21.15.73:0/247978292 wait complete. 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.736+0000 7fabfa600640 1 Processor -- start 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.736+0000 7fabfa600640 1 -- start start 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd80039a0 0x7fabf41545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf41f1f50 0x7fabf4159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40ae960 con 0x7fabf40aacd0 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41f24d0 con 0x7fabf41f1f50 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabfa600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41f27b0 con 0x7fabd80039a0 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49704/0 (socket says 172.21.15.73:49704) 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 -- 172.21.15.73:0/1658121704 learned_addr learned my addr 172.21.15.73:0/1658121704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf3e00640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf41f1f50 0x7fabf4159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 -- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd80039a0 msgr2=0x7fabf41545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd80039a0 0x7fabf41545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 -- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf41f1f50 msgr2=0x7fabf4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf41f1f50 0x7fabf4159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.737+0000 7fabf2a00640 1 -- 172.21.15.73:0/1658121704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40aeb70 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabf2a00640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fabe0010860 tx=0x7fabe0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0009610 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe000eb60 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf40aee50 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe001c400 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.738+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4154b30 con 0x7fabf40aacd0 2024-04-05T22:26:07.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.739+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4066af0 con 0x7fabf40aacd0 2024-04-05T22:26:07.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.740+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fabe001c560 con 0x7fabf40aacd0 2024-04-05T22:26:07.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.742+0000 7fabeb400640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 0x7fabd406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:07.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.742+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabe0089b20 con 0x7fabf40aacd0 2024-04-05T22:26:07.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.742+0000 7fabf3400640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 0x7fabd406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:07.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.743+0000 7fabf3400640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 0x7fabd406d410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabe4034000 tx=0x7fabe402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:07.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.746+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fabe0054cb0 con 0x7fabf40aacd0 2024-04-05T22:26:07.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.947+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fabf4068160 con 0x7fabf40aacd0 2024-04-05T22:26:07.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.947+0000 7fabeb400640 1 -- 172.21.15.73:0/1658121704 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7fabe00582d0 con 0x7fabf40aacd0 2024-04-05T22:26:07.948 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:07.948 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:26:07.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:07.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:26:07.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:26:07.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:07.949 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:26:07.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 msgr2=0x7fabd406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 0x7fabd406d410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabe4034000 tx=0x7fabe402e000 comp rx=0 tx=0).stop 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 msgr2=0x7fabf41f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fabe0010860 tx=0x7fabe0010d20 comp rx=0 tx=0).stop 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 shutdown_connections 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fabd406af50 0x7fabd406d410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf41f1f50 0x7fabf4159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf40aacd0 0x7fabf41f1a10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 --2- 172.21.15.73:0/1658121704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd80039a0 0x7fabf41545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:07.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.950+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 >> 172.21.15.73:0/1658121704 conn(0x7fabf40ce8e0 msgr2=0x7fabf41574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:07.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.951+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 shutdown_connections 2024-04-05T22:26:07.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:07.951+0000 7fabfa600640 1 -- 172.21.15.73:0/1658121704 wait complete. 2024-04-05T22:26:07.970 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:26:07.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:26:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.180+0000 7fbb63c00640 1 Processor -- start 2024-04-05T22:26:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.181+0000 7fbb63c00640 1 -- start start 2024-04-05T22:26:08.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.182+0000 7fbb63c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb5c15ce50 0x7fbb5c1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.182+0000 7fbb63c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.182+0000 7fbb63c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c15e3d0 con 0x7fbb5c15d230 2024-04-05T22:26:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.182+0000 7fbb63c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c1619f0 con 0x7fbb5c15ce50 2024-04-05T22:26:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.182+0000 7fbb63c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c15d7c0 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.183+0000 7fbb61c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.183+0000 7fbb61200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb5c15ce50 0x7fbb5c1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.183+0000 7fbb61c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:35604/0 (socket says 172.21.15.73:35604) 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.183+0000 7fbb61c00640 1 -- 172.21.15.73:0/2308310505 learned_addr learned my addr 172.21.15.73:0/2308310505 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.183+0000 7fbb5be00640 1 --1- 172.21.15.73:0/2308310505 >> v1:172.21.15.157:6789/0 conn(0x7fbb5c15d230 0x7fbb5c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:39044/0 (socket says 172.21.15.73:39044) 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.184+0000 7fbb61200640 1 -- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 msgr2=0x7fbb5c06a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.184+0000 7fbb61200640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.184+0000 7fbb61200640 1 -- 172.21.15.73:0/2308310505 >> v1:172.21.15.157:6789/0 conn(0x7fbb5c15d230 legacy=0x7fbb5c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.184+0000 7fbb61200640 1 -- 172.21.15.73:0/2308310505 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c15daa0 con 0x7fbb5c15ce50 2024-04-05T22:26:08.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.184+0000 7fbb61c00640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:26:08.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.185+0000 7fbb61200640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c1614b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097a0 tx=0x7fbb4402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c961632c0be98dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:08.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.186+0000 7fbb5b400640 1 -- 172.21.15.73:0/2308310505 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4402d040 con 0x7fbb5c15ce50 2024-04-05T22:26:08.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.186+0000 7fbb5b400640 1 -- 172.21.15.73:0/2308310505 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44031810 con 0x7fbb5c15ce50 2024-04-05T22:26:08.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.186+0000 7fbb5b400640 1 -- 172.21.15.73:0/2308310505 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44031ba0 con 0x7fbb5c15ce50 2024-04-05T22:26:08.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 -- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 msgr2=0x7fbb5c1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:08.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c1614b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097a0 tx=0x7fbb4402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 -- 172.21.15.73:0/2308310505 shutdown_connections 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c0b07f0 0x7fbb5c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 --2- 172.21.15.73:0/2308310505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c1614b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 -- 172.21.15.73:0/2308310505 >> 172.21.15.73:0/2308310505 conn(0x7fbb5c0b3ed0 msgr2=0x7fbb5c0b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 -- 172.21.15.73:0/2308310505 shutdown_connections 2024-04-05T22:26:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.187+0000 7fbb63c00640 1 -- 172.21.15.73:0/2308310505 wait complete. 2024-04-05T22:26:08.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.189+0000 7fbb63c00640 1 Processor -- start 2024-04-05T22:26:08.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.189+0000 7fbb63c00640 1 -- start start 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c06a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 0x7fbb5c06a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0cfa40 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c14a470 con 0x7fbb5c15ce50 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb63c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c147590 con 0x7fbb5c15d230 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb61200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb61200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49708/0 (socket says 172.21.15.73:49708) 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb61200640 1 -- 172.21.15.73:0/3807069228 learned_addr learned my addr 172.21.15.73:0/3807069228 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:08.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.190+0000 7fbb5be00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c06a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 0x7fbb5c06a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 -- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 msgr2=0x7fbb5c06a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 0x7fbb5c06a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 -- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 msgr2=0x7fbb5c06a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c06a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 -- 172.21.15.73:0/3807069228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540094d0 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 0x7fbb5c06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:08.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb61200640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb440098c0 tx=0x7fbb440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:08.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4403ace0 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4403ae60 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44044640 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.191+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb44009400 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.192+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c147b50 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.192+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5c068160 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.194+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fbb4404d440 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.194+0000 7fbb58c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 0x7fbb3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:08.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.195+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb440ab960 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.195+0000 7fbb5be00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 0x7fbb3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:08.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.195+0000 7fbb5be00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 0x7fbb3806d420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c15b880 tx=0x7fbb4c005b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:08.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.199+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbb44076970 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:08.387+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb5c0ae9c0 con 0x7fbb5c0b07f0 2024-04-05T22:26:08.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:26:08.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:26:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.048+0000 7fbb58c00640 1 -- 172.21.15.73:0/3807069228 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb440350b0 con 0x7fbb5c0b07f0 2024-04-05T22:26:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:26:09.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 msgr2=0x7fbb3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 0x7fbb3806d420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c15b880 tx=0x7fbb4c005b70 comp rx=0 tx=0).stop 2024-04-05T22:26:09.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 msgr2=0x7fbb5c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb440098c0 tx=0x7fbb440046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 shutdown_connections 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fbb3806af60 0x7fbb3806d420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb5c15d230 0x7fbb5c06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5c15ce50 0x7fbb5c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.055+0000 7fbb63c00640 1 --2- 172.21.15.73:0/3807069228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb5c0b07f0 0x7fbb5c0cf500 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.056+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 >> 172.21.15.73:0/3807069228 conn(0x7fbb5c0b3ed0 msgr2=0x7fbb5c0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.056+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 shutdown_connections 2024-04-05T22:26:09.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.056+0000 7fbb63c00640 1 -- 172.21.15.73:0/3807069228 wait complete. 2024-04-05T22:26:09.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:26:09.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.279+0000 7f1839800640 1 Processor -- start 2024-04-05T22:26:09.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.280+0000 7f1839800640 1 -- start start 2024-04-05T22:26:09.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1839800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18340d35a0 0x7f18340ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1839800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18340b6630 0x7f18340b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1839800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340d3a10 con 0x7f18340cf500 2024-04-05T22:26:09.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1839800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340b6210 con 0x7f18340d35a0 2024-04-05T22:26:09.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1839800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340ce650 con 0x7f18340b6630 2024-04-05T22:26:09.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1833e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f18340cf500 0x7f18340cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60002/0 (socket says 172.21.15.73:60002) 2024-04-05T22:26:09.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1833e00640 1 -- 172.21.15.73:0/3071644586 learned_addr learned my addr 172.21.15.73:0/3071644586 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:09.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.281+0000 7f1833400640 1 --2- 172.21.15.73:0/3071644586 >> v2:172.21.15.73:3300/0 conn(0x7f18340d35a0 0x7f18340ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1838e00640 1 --2- 172.21.15.73:0/3071644586 >> v2:172.21.15.157:3300/0 conn(0x7f18340b6630 0x7f18340b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1832a00640 1 -- 172.21.15.73:0/3071644586 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652525126 0 0) 0x7f18340d3a10 con 0x7f18340cf500 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1832a00640 1 -- 172.21.15.73:0/3071644586 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1818003520 con 0x7f18340cf500 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1833400640 1 -- 172.21.15.73:0/3071644586 >> v2:172.21.15.157:3300/0 conn(0x7f18340b6630 msgr2=0x7f18340b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1833400640 1 --2- 172.21.15.73:0/3071644586 >> v2:172.21.15.157:3300/0 conn(0x7f18340b6630 0x7f18340b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1833400640 1 -- 172.21.15.73:0/3071644586 >> v1:172.21.15.73:6789/0 conn(0x7f18340cf500 legacy=0x7f18340cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.282+0000 7f1833400640 1 -- 172.21.15.73:0/3071644586 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340ce840 con 0x7f18340d35a0 2024-04-05T22:26:09.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.283+0000 7f1833400640 1 --2- 172.21.15.73:0/3071644586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340ce110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f18340d0b00 tx=0x7f181c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e20f80af4af574 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:09.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.283+0000 7f1832a00640 1 -- 172.21.15.73:0/3071644586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c02e070 con 0x7f18340d35a0 2024-04-05T22:26:09.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.283+0000 7f1832a00640 1 -- 172.21.15.73:0/3071644586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0315a0 con 0x7f18340d35a0 2024-04-05T22:26:09.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.283+0000 7f1832a00640 1 -- 172.21.15.73:0/3071644586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031910 con 0x7f18340d35a0 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 -- 172.21.15.73:0/3071644586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 msgr2=0x7f18340ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 --2- 172.21.15.73:0/3071644586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340ce110 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f18340d0b00 tx=0x7f181c02b840 comp rx=0 tx=0).stop 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 -- 172.21.15.73:0/3071644586 shutdown_connections 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 --2- 172.21.15.73:0/3071644586 >> v2:172.21.15.157:3300/0 conn(0x7f18340b6630 0x7f18340b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 --2- 172.21.15.73:0/3071644586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340ce110 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 -- 172.21.15.73:0/3071644586 >> 172.21.15.73:0/3071644586 conn(0x7f183409be30 msgr2=0x7f18340d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 -- 172.21.15.73:0/3071644586 shutdown_connections 2024-04-05T22:26:09.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.284+0000 7f1839800640 1 -- 172.21.15.73:0/3071644586 wait complete. 2024-04-05T22:26:09.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.286+0000 7f1839800640 1 Processor -- start 2024-04-05T22:26:09.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.286+0000 7f1839800640 1 -- start start 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18340b6630 0x7f18341f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18340cf500 0x7f18340c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340cd1b0 con 0x7f18340d35a0 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183415e990 con 0x7f18340cf500 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1839800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183415ec70 con 0x7f18340b6630 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1838e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1838e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49722/0 (socket says 172.21.15.73:49722) 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1838e00640 1 -- 172.21.15.73:0/3782794397 learned_addr learned my addr 172.21.15.73:0/3782794397 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1833400640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18340cf500 0x7f18340c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.287+0000 7f1833e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18340b6630 0x7f18341f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 -- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18340b6630 msgr2=0x7f18341f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18340b6630 0x7f18341f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 -- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18340cf500 msgr2=0x7f18340c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:09.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18340cf500 0x7f18340c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:09.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 -- 172.21.15.73:0/3782794397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c009400 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1838e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f182c00efc0 tx=0x7f182c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c009ba0 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00f040 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183415ef50 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.288+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0146c0 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.289+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340c7790 con 0x7f18340d35a0 2024-04-05T22:26:09.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.289+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183415f470 con 0x7f18340d35a0 2024-04-05T22:26:09.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.291+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f182c014820 con 0x7f18340d35a0 2024-04-05T22:26:09.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.292+0000 7f182be00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 0x7f181006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:09.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.292+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f182c08a730 con 0x7f18340d35a0 2024-04-05T22:26:09.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.292+0000 7f1833e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 0x7f181006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:09.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.293+0000 7f1833e00640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 0x7f181006d450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1824005880 tx=0x7f1824009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:09.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.296+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f182c0556c0 con 0x7f18340d35a0 2024-04-05T22:26:09.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:09.489+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f183415f470 con 0x7f18340d35a0 2024-04-05T22:26:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.061+0000 7f182be00640 1 -- 172.21.15.73:0/3782794397 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f183415f470 con 0x7f18340d35a0 2024-04-05T22:26:10.064 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:26:10.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.067+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 msgr2=0x7f181006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.067+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 0x7f181006d450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1824005880 tx=0x7f1824009040 comp rx=0 tx=0).stop 2024-04-05T22:26:10.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.067+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 msgr2=0x7f18340cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.067+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f182c00efc0 tx=0x7f182c00c490 comp rx=0 tx=0).stop 2024-04-05T22:26:10.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 shutdown_connections 2024-04-05T22:26:10.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f181006af90 0x7f181006d450 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18340d35a0 0x7f18340cc8a0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18340cf500 0x7f18340c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 --2- 172.21.15.73:0/3782794397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18340b6630 0x7f18341f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.070+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 >> 172.21.15.73:0/3782794397 conn(0x7f183409be30 msgr2=0x7f18340b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:10.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.071+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 shutdown_connections 2024-04-05T22:26:10.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.071+0000 7f1839800640 1 -- 172.21.15.73:0/3782794397 wait complete. 2024-04-05T22:26:10.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:26:10.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.285+0000 7fc6cb400640 1 Processor -- start 2024-04-05T22:26:10.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.285+0000 7fc6cb400640 1 -- start start 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.286+0000 7fc6cb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6c406be40 0x7fc6c40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.286+0000 7fc6cb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c4161010 con 0x7fc6c4069cb0 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6cb400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c41612c0 con 0x7fc6c4160b90 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6cb400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40c85d0 con 0x7fc6c406be40 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c8a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc6c4069cb0 0x7fc6c406a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60014/0 (socket says 172.21.15.73:60014) 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c8a00640 1 -- 172.21.15.73:0/2988756700 learned_addr learned my addr 172.21.15.73:0/2988756700 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c9400640 1 --2- 172.21.15.73:0/2988756700 >> v2:172.21.15.73:3300/0 conn(0x7fc6c406be40 0x7fc6c40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c9400640 1 -- 172.21.15.73:0/2988756700 >> v1:172.21.15.157:6789/0 conn(0x7fc6c4160b90 legacy=0x7fc6c415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:10.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c9400640 1 -- 172.21.15.73:0/2988756700 >> v1:172.21.15.73:6789/0 conn(0x7fc6c4069cb0 legacy=0x7fc6c406a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.287+0000 7fc6c9400640 1 -- 172.21.15.73:0/2988756700 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c41615a0 con 0x7fc6c406be40 2024-04-05T22:26:10.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.288+0000 7fc6c9400640 1 --2- 172.21.15.73:0/2988756700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c40c8a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40097a0 tx=0x7fc6b402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79e480bf2a2b730 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.289+0000 7fc6c3400640 1 -- 172.21.15.73:0/2988756700 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b402e070 con 0x7fc6c406be40 2024-04-05T22:26:10.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.289+0000 7fc6c3400640 1 -- 172.21.15.73:0/2988756700 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40315a0 con 0x7fc6c406be40 2024-04-05T22:26:10.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.289+0000 7fc6c3400640 1 -- 172.21.15.73:0/2988756700 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4031930 con 0x7fc6c406be40 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.289+0000 7fc6cb400640 1 -- 172.21.15.73:0/2988756700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 msgr2=0x7fc6c40c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.289+0000 7fc6cb400640 1 --2- 172.21.15.73:0/2988756700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c40c8a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40097a0 tx=0x7fc6b402b840 comp rx=0 tx=0).stop 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.290+0000 7fc6cb400640 1 -- 172.21.15.73:0/2988756700 shutdown_connections 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.290+0000 7fc6cb400640 1 --2- 172.21.15.73:0/2988756700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c40c8a20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.290+0000 7fc6cb400640 1 -- 172.21.15.73:0/2988756700 >> 172.21.15.73:0/2988756700 conn(0x7fc6c40b0ce0 msgr2=0x7fc6c40b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.290+0000 7fc6cb400640 1 -- 172.21.15.73:0/2988756700 shutdown_connections 2024-04-05T22:26:10.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.290+0000 7fc6cb400640 1 -- 172.21.15.73:0/2988756700 wait complete. 2024-04-05T22:26:10.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.291+0000 7fc6cb400640 1 Processor -- start 2024-04-05T22:26:10.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.291+0000 7fc6cb400640 1 -- start start 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c4069cb0 0x7fc6c40cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c4160b90 0x7fc6c415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41f3e40 con 0x7fc6c406be40 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41f3880 con 0x7fc6c4160b90 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.292+0000 7fc6cb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41f3b60 con 0x7fc6c4069cb0 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49746/0 (socket says 172.21.15.73:49746) 2024-04-05T22:26:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 -- 172.21.15.73:0/1524737105 learned_addr learned my addr 172.21.15.73:0/1524737105 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c4069cb0 0x7fc6c40cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 -- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c4069cb0 msgr2=0x7fc6c40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c9400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c4160b90 0x7fc6c415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c4069cb0 0x7fc6c40cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 -- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c4160b90 msgr2=0x7fc6c415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c4160b90 0x7fc6c415c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 -- 172.21.15.73:0/1524737105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4009400 con 0x7fc6c406be40 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.293+0000 7fc6c3e00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3f0 tx=0x7fc6ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.294+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011830 con 0x7fc6c406be40 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.294+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac011990 con 0x7fc6c406be40 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.294+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c41f4470 con 0x7fc6c406be40 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.294+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011d40 con 0x7fc6c406be40 2024-04-05T22:26:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.294+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c409a8e0 con 0x7fc6c406be40 2024-04-05T22:26:10.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.295+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c41f4990 con 0x7fc6c406be40 2024-04-05T22:26:10.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.296+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fc6ac01a400 con 0x7fc6c406be40 2024-04-05T22:26:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.297+0000 7fc6c0c00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 0x7fc6a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.297+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc6ac0896a0 con 0x7fc6c406be40 2024-04-05T22:26:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.297+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 0x7fc6a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.298+0000 7fc6c8a00640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 0x7fc6a006d420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005c00 tx=0x7fc6bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.302+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc6c41f4990 con 0x7fc6c406be40 2024-04-05T22:26:10.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.534+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc6c41f4990 con 0x7fc6c406be40 2024-04-05T22:26:10.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.552+0000 7fc6c0c00640 1 -- 172.21.15.73:0/1524737105 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc6c41f4990 con 0x7fc6c406be40 2024-04-05T22:26:10.554 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 10 and data pool 11 2024-04-05T22:26:10.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 msgr2=0x7fc6a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 0x7fc6a006d420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005c00 tx=0x7fc6bc009040 comp rx=0 tx=0).stop 2024-04-05T22:26:10.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 msgr2=0x7fc6c41f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3f0 tx=0x7fc6ac00b8b0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 shutdown_connections 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fc6a006af60 0x7fc6a006d420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c4160b90 0x7fc6c415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c406be40 0x7fc6c41f2e00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 --2- 172.21.15.73:0/1524737105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c4069cb0 0x7fc6c40cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.557+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 >> 172.21.15.73:0/1524737105 conn(0x7fc6c40b0ce0 msgr2=0x7fc6c415ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:10.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.561+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 shutdown_connections 2024-04-05T22:26:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.562+0000 7fc6cb400640 1 -- 172.21.15.73:0/1524737105 wait complete. 2024-04-05T22:26:10.592 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:10.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.765+0000 7f0058600640 1 Processor -- start 2024-04-05T22:26:10.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.765+0000 7f0058600640 1 -- start start 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.766+0000 7f0058600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f005006d710 0x7f00500cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.766+0000 7f0058600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005015aa60 con 0x7f005015aef0 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.766+0000 7f0058600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00500b3b60 con 0x7f00501541a0 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.766+0000 7f0058600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00500701c0 con 0x7f005006d710 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0056600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f005015aef0 0x7f005015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60030/0 (socket says 172.21.15.73:60030) 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0055200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f005006d710 0x7f00500cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0056600640 1 -- 172.21.15.73:0/1183341974 learned_addr learned my addr 172.21.15.73:0/1183341974 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:10.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0055200640 1 -- 172.21.15.73:0/1183341974 >> v1:172.21.15.157:6789/0 conn(0x7f00501541a0 legacy=0x7f00500b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0055200640 1 -- 172.21.15.73:0/1183341974 >> v1:172.21.15.73:6789/0 conn(0x7f005015aef0 legacy=0x7f005015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f004fe00640 1 -- 172.21.15.73:0/1183341974 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057544030 0 0) 0x7f005015aa60 con 0x7f005015aef0 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.767+0000 7f0055200640 1 -- 172.21.15.73:0/1183341974 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500b3880 con 0x7f005006d710 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.768+0000 7f0055200640 1 --2- 172.21.15.73:0/1183341974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500cd8f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f00440097a0 tx=0x7f004402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0a299eb83ee200 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.768+0000 7f004fe00640 1 -- 172.21.15.73:0/1183341974 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044031830 con 0x7f005006d710 2024-04-05T22:26:10.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.768+0000 7f004fe00640 1 -- 172.21.15.73:0/1183341974 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044031990 con 0x7f005006d710 2024-04-05T22:26:10.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.768+0000 7f004fe00640 1 -- 172.21.15.73:0/1183341974 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044031d20 con 0x7f005006d710 2024-04-05T22:26:10.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 -- 172.21.15.73:0/1183341974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 msgr2=0x7f00500cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 --2- 172.21.15.73:0/1183341974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500cd8f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f00440097a0 tx=0x7f004402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 -- 172.21.15.73:0/1183341974 shutdown_connections 2024-04-05T22:26:10.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 --2- 172.21.15.73:0/1183341974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500cd8f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 -- 172.21.15.73:0/1183341974 >> 172.21.15.73:0/1183341974 conn(0x7f00500ce8e0 msgr2=0x7f00500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:10.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 -- 172.21.15.73:0/1183341974 shutdown_connections 2024-04-05T22:26:10.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.769+0000 7f0058600640 1 -- 172.21.15.73:0/1183341974 wait complete. 2024-04-05T22:26:10.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.771+0000 7f0058600640 1 Processor -- start 2024-04-05T22:26:10.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.771+0000 7f0058600640 1 -- start start 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 0x7f00501512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f005015aef0 0x7f00501f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00500aa6d0 con 0x7f005006d710 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00500aa9b0 con 0x7f00501541a0 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0058600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00500aac90 con 0x7f005015aef0 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0055c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0055c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49756/0 (socket says 172.21.15.73:49756) 2024-04-05T22:26:10.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0055c00640 1 -- 172.21.15.73:0/2142210942 learned_addr learned my addr 172.21.15.73:0/2142210942 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0055200640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 0x7f00501512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0056600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f005015aef0 0x7f00501f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.772+0000 7f0055c00640 1 -- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f005015aef0 msgr2=0x7f00501f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055c00640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f005015aef0 0x7f00501f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055c00640 1 -- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 msgr2=0x7f00501512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055c00640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 0x7f00501512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055c00640 1 -- 172.21.15.73:0/2142210942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0044009400 con 0x7f005006d710 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055200640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 0x7f00501512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0055c00640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f004000c430 tx=0x7f004000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040011790 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00400118f0 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00500aaf70 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040011ce0 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.773+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00500af400 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.774+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0050066af0 con 0x7f005006d710 2024-04-05T22:26:10.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.776+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f004001c460 con 0x7f005006d710 2024-04-05T22:26:10.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.777+0000 7f004d600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 0x7f003406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:10.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.779+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0040089780 con 0x7f005006d710 2024-04-05T22:26:10.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.779+0000 7f0055200640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 0x7f003406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:10.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.779+0000 7f0055200640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 0x7f003406d420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0044036000 tx=0x7f004403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:10.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.783+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00400545c0 con 0x7f005006d710 2024-04-05T22:26:10.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.970+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00500ab490 con 0x7f005006d710 2024-04-05T22:26:10.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.972+0000 7f004d600640 1 -- 172.21.15.73:0/2142210942 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16113 (secure 0 0 0) 0x7f00500ab490 con 0x7f005006d710 2024-04-05T22:26:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:10.972 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":73,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:10.535986+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:10.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 msgr2=0x7f003406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 0x7f003406d420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0044036000 tx=0x7f004403c040 comp rx=0 tx=0).stop 2024-04-05T22:26:10.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 msgr2=0x7f00500aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f004000c430 tx=0x7f004000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 shutdown_connections 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f003406af60 0x7f003406d420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f005015aef0 0x7f00501f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00501541a0 0x7f00501512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 --2- 172.21.15.73:0/2142210942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f005006d710 0x7f00500aeec0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.974+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 >> 172.21.15.73:0/2142210942 conn(0x7f00500ce8e0 msgr2=0x7f005006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.976+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 shutdown_connections 2024-04-05T22:26:10.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:10.976+0000 7f0058600640 1 -- 172.21.15.73:0/2142210942 wait complete. 2024-04-05T22:26:10.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:11.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.188+0000 7fae99400640 1 Processor -- start 2024-04-05T22:26:11.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.188+0000 7fae99400640 1 -- start start 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae99400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae940d1600 0x7fae9415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae99400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae940d35e0 0x7fae940d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae99400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940b2420 con 0x7fae9415ccd0 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae99400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940b2770 con 0x7fae940d1600 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae99400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940ce380 con 0x7fae940d35e0 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.190+0000 7fae93400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae940d1600 0x7fae9415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fae9415ccd0 0x7fae940d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60040/0 (socket says 172.21.15.73:60040) 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93e00640 1 -- 172.21.15.73:0/1886360972 learned_addr learned my addr 172.21.15.73:0/1886360972 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae98a00640 1 --2- 172.21.15.73:0/1886360972 >> v2:172.21.15.157:3300/0 conn(0x7fae940d35e0 0x7fae940d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93400640 1 -- 172.21.15.73:0/1886360972 >> v2:172.21.15.157:3300/0 conn(0x7fae940d35e0 msgr2=0x7fae940d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93400640 1 --2- 172.21.15.73:0/1886360972 >> v2:172.21.15.157:3300/0 conn(0x7fae940d35e0 0x7fae940d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93400640 1 -- 172.21.15.73:0/1886360972 >> v1:172.21.15.73:6789/0 conn(0x7fae9415ccd0 legacy=0x7fae940d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.191+0000 7fae93400640 1 -- 172.21.15.73:0/1886360972 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940ce660 con 0x7fae940d1600 2024-04-05T22:26:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.192+0000 7fae93400640 1 --2- 172.21.15.73:0/1886360972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d1600 0x7fae9415d0b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fae9408cdb0 tx=0x7fae8002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d8feb019183b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.192+0000 7fae92a00640 1 -- 172.21.15.73:0/1886360972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80031520 con 0x7fae940d1600 2024-04-05T22:26:11.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae92a00640 1 -- 172.21.15.73:0/1886360972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80031680 con 0x7fae940d1600 2024-04-05T22:26:11.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae92a00640 1 -- 172.21.15.73:0/1886360972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80031a10 con 0x7fae940d1600 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 -- 172.21.15.73:0/1886360972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d1600 msgr2=0x7fae9415d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 --2- 172.21.15.73:0/1886360972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d1600 0x7fae9415d0b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fae9408cdb0 tx=0x7fae8002b7c0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 -- 172.21.15.73:0/1886360972 shutdown_connections 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 --2- 172.21.15.73:0/1886360972 >> v2:172.21.15.157:3300/0 conn(0x7fae940d35e0 0x7fae940d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 --2- 172.21.15.73:0/1886360972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d1600 0x7fae9415d0b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 -- 172.21.15.73:0/1886360972 >> 172.21.15.73:0/1886360972 conn(0x7fae940d2ac0 msgr2=0x7fae940d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 -- 172.21.15.73:0/1886360972 shutdown_connections 2024-04-05T22:26:11.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.193+0000 7fae99400640 1 -- 172.21.15.73:0/1886360972 wait complete. 2024-04-05T22:26:11.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.195+0000 7fae99400640 1 Processor -- start 2024-04-05T22:26:11.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.195+0000 7fae99400640 1 -- start start 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 0x7fae9409dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 0x7fae9415bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9409cca0 con 0x7fae940d35e0 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9409cf80 con 0x7fae9415ccd0 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae99400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9409d260 con 0x7fae940d1600 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae93400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae93400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49772/0 (socket says 172.21.15.73:49772) 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.196+0000 7fae93400640 1 -- 172.21.15.73:0/480495501 learned_addr learned my addr 172.21.15.73:0/480495501 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 -- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 msgr2=0x7fae9409dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93e00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 0x7fae9409dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae98a00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 0x7fae9415bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 0x7fae9409dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 -- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 msgr2=0x7fae9415bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 0x7fae9415bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 -- 172.21.15.73:0/480495501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009400 con 0x7fae940d35e0 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae98a00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 0x7fae9415bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93e00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 0x7fae9409dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae93400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fae80000c00 tx=0x7fae800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80031520 con 0x7fae940d35e0 2024-04-05T22:26:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80033c50 con 0x7fae940d35e0 2024-04-05T22:26:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8003d410 con 0x7fae940d35e0 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.197+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9409d540 con 0x7fae940d35e0 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.198+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9409e280 con 0x7fae940d35e0 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.198+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae94067250 con 0x7fae940d35e0 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.199+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fae8003a050 con 0x7fae940d35e0 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.200+0000 7fae8be00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.201+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae800ab810 con 0x7fae940d35e0 2024-04-05T22:26:11.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.201+0000 7fae93e00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.201+0000 7fae93e00640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 0x7fae7406d420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae84000f60 tx=0x7fae84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.205+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae800764f0 con 0x7fae940d35e0 2024-04-05T22:26:11.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.382+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae9409da00 con 0x7fae940d35e0 2024-04-05T22:26:11.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.388+0000 7fae8be00640 1 -- 172.21.15.73:0/480495501 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16113 (secure 0 0 0) 0x7fae80079b10 con 0x7fae940d35e0 2024-04-05T22:26:11.388 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:11.388 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":73,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:10.535986+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:11.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 msgr2=0x7fae7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 0x7fae7406d420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae84000f60 tx=0x7fae84009040 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 msgr2=0x7fae940cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fae80000c00 tx=0x7fae800046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 shutdown_connections 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae9415ccd0 0x7fae9415bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae940d35e0 0x7fae940cc520 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 --2- 172.21.15.73:0/480495501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae940d1600 0x7fae9409dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 >> 172.21.15.73:0/480495501 conn(0x7fae940d2ac0 msgr2=0x7fae940d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 shutdown_connections 2024-04-05T22:26:11.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.391+0000 7fae99400640 1 -- 172.21.15.73:0/480495501 wait complete. 2024-04-05T22:26:11.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:26:11.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.621+0000 7f91da800640 1 Processor -- start 2024-04-05T22:26:11.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.621+0000 7f91da800640 1 -- start start 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91da800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 0x7f91d40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91da800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d406b530 0x7f91d40adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91da800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d40b2ff0 con 0x7f91d40ae0b0 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91da800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d41597b0 con 0x7f91d406b530 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91da800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b2ab0 con 0x7f91d40b2190 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d406b530 0x7f91d40adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d406b530 0x7f91d40adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49778/0 (socket says 172.21.15.73:49778) 2024-04-05T22:26:11.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d2a00640 1 -- 172.21.15.73:0/2926214779 learned_addr learned my addr 172.21.15.73:0/2926214779 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:11.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d3e00640 1 --1- 172.21.15.73:0/2926214779 >> v1:172.21.15.73:6789/0 conn(0x7f91d40ae0b0 0x7f91d406f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60044/0 (socket says 172.21.15.73:60044) 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d3400640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 0x7f91d40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.623+0000 7f91d2a00640 1 -- 172.21.15.73:0/2926214779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 msgr2=0x7f91d40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.624+0000 7f91d2a00640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 0x7f91d40b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.624+0000 7f91d2a00640 1 -- 172.21.15.73:0/2926214779 >> v1:172.21.15.73:6789/0 conn(0x7f91d40ae0b0 legacy=0x7f91d406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.624+0000 7f91d2a00640 1 -- 172.21.15.73:0/2926214779 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d41599a0 con 0x7f91d406b530 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.624+0000 7f91d2000640 1 -- 172.21.15.73:0/2926214779 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645753269 0 0) 0x7f91d40b2ff0 con 0x7f91d40ae0b0 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.624+0000 7f91d3400640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 0x7f91d40b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91d2a00640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d406b530 0x7f91d40adb70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009e50 tx=0x7f91bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72f05526354809f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91d2000640 1 -- 172.21.15.73:0/2926214779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc02ca80 con 0x7f91d406b530 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91d2000640 1 -- 172.21.15.73:0/2926214779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc02cbe0 con 0x7f91d406b530 2024-04-05T22:26:11.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91d2000640 1 -- 172.21.15.73:0/2926214779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0343f0 con 0x7f91d406b530 2024-04-05T22:26:11.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 -- 172.21.15.73:0/2926214779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d406b530 msgr2=0x7f91d40adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d406b530 0x7f91d40adb70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009e50 tx=0x7f91bc02bee0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 -- 172.21.15.73:0/2926214779 shutdown_connections 2024-04-05T22:26:11.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d406b530 0x7f91d40adb70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 --2- 172.21.15.73:0/2926214779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40b2190 0x7f91d40b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.625+0000 7f91da800640 1 -- 172.21.15.73:0/2926214779 >> 172.21.15.73:0/2926214779 conn(0x7f91d4069cb0 msgr2=0x7f91d406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.626+0000 7f91da800640 1 -- 172.21.15.73:0/2926214779 shutdown_connections 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.626+0000 7f91da800640 1 -- 172.21.15.73:0/2926214779 wait complete. 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.627+0000 7f91da800640 1 Processor -- start 2024-04-05T22:26:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.627+0000 7f91da800640 1 -- start start 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d406b530 0x7f91d406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 0x7f91d41f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41461f0 con 0x7f91d40b2190 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41464d0 con 0x7f91d406b530 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91da800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41467b0 con 0x7f91d40ae0b0 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49782/0 (socket says 172.21.15.73:49782) 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.628+0000 7f91d3e00640 1 -- 172.21.15.73:0/2882158696 learned_addr learned my addr 172.21.15.73:0/2882158696 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d2a00640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 0x7f91d41f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3400640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d406b530 0x7f91d406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 -- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 msgr2=0x7f91d41f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 0x7f91d41f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 -- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d406b530 msgr2=0x7f91d406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d406b530 0x7f91d406f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 -- 172.21.15.73:0/2882158696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc009b30 con 0x7f91d40b2190 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d2a00640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 0x7f91d41f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91d3e00640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c450 tx=0x7f91cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0074e0 con 0x7f91d40b2190 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc007660 con 0x7f91d40b2190 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d4146a90 con 0x7f91d40b2190 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.629+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc004a20 con 0x7f91d40b2190 2024-04-05T22:26:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.630+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d406fd60 con 0x7f91d40b2190 2024-04-05T22:26:11.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.631+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d406ff50 con 0x7f91d40b2190 2024-04-05T22:26:11.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.632+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f91cc004b80 con 0x7f91d40b2190 2024-04-05T22:26:11.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.632+0000 7f91cb400640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 0x7f91b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:11.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.632+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91cc088ea0 con 0x7f91d40b2190 2024-04-05T22:26:11.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.633+0000 7f91d3400640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 0x7f91b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:11.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.633+0000 7f91d3400640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 0x7f91b006d420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b840 tx=0x7f91c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:11.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.637+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f91cc053bf0 con 0x7f91d40b2190 2024-04-05T22:26:11.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:11.850+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f91d4146fb0 con 0x7f91d40b2190 2024-04-05T22:26:12.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.558+0000 7f91cb400640 1 -- 172.21.15.73:0/2882158696 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f91cc018020 con 0x7f91d40b2190 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 msgr2=0x7f91b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 0x7f91b006d420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b840 tx=0x7f91c4008040 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 msgr2=0x7f91d41f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c450 tx=0x7f91cc00c910 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 shutdown_connections 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f91b006af60 0x7f91b006d420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d40b2190 0x7f91d41f44b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d40ae0b0 0x7f91d41f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 --2- 172.21.15.73:0/2882158696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d406b530 0x7f91d406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 >> 172.21.15.73:0/2882158696 conn(0x7f91d4069cb0 msgr2=0x7f91d406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 shutdown_connections 2024-04-05T22:26:12.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.562+0000 7f91da800640 1 -- 172.21.15.73:0/2882158696 wait complete. 2024-04-05T22:26:12.583 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:26:12.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:26:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.792+0000 7fe4ef000640 1 Processor -- start 2024-04-05T22:26:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.792+0000 7fe4ef000640 1 -- start start 2024-04-05T22:26:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.793+0000 7fe4ef000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe4e81599f0 0x7fe4e8070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4ef000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4ef000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e80ae190 con 0x7fe4e806cb80 2024-04-05T22:26:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4ef000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e806d010 con 0x7fe4e81599f0 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4ef000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80b3890 con 0x7fe4e80add00 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe4e81599f0 0x7fe4e8070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe4e81599f0 0x7fe4e8070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49784/0 (socket says 172.21.15.73:49784) 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 -- 172.21.15.73:0/2940086050 learned_addr learned my addr 172.21.15.73:0/2940086050 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 -- 172.21.15.73:0/2940086050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 msgr2=0x7fe4e806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 --2- 172.21.15.73:0/2940086050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 -- 172.21.15.73:0/2940086050 >> v1:172.21.15.157:6789/0 conn(0x7fe4e806cb80 legacy=0x7fe4e815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.794+0000 7fe4e7e00640 1 -- 172.21.15.73:0/2940086050 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80b27f0 con 0x7fe4e81599f0 2024-04-05T22:26:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.795+0000 7fe4e7e00640 1 --2- 172.21.15.73:0/2940086050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e81599f0 0x7fe4e8070a40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d802db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8703c8ff76f52e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4e7400640 1 -- 172.21.15.73:0/2940086050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d802f8c0 con 0x7fe4e81599f0 2024-04-05T22:26:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4e7400640 1 -- 172.21.15.73:0/2940086050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d802fa20 con 0x7fe4e81599f0 2024-04-05T22:26:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4e7400640 1 -- 172.21.15.73:0/2940086050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d802fdb0 con 0x7fe4e81599f0 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 -- 172.21.15.73:0/2940086050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e81599f0 msgr2=0x7fe4e8070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 --2- 172.21.15.73:0/2940086050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e81599f0 0x7fe4e8070a40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d802db00 comp rx=0 tx=0).stop 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 -- 172.21.15.73:0/2940086050 shutdown_connections 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 --2- 172.21.15.73:0/2940086050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e806c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 --2- 172.21.15.73:0/2940086050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e81599f0 0x7fe4e8070a40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.796+0000 7fe4ef000640 1 -- 172.21.15.73:0/2940086050 >> 172.21.15.73:0/2940086050 conn(0x7fe4e8069cb0 msgr2=0x7fe4e806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.797+0000 7fe4ef000640 1 -- 172.21.15.73:0/2940086050 shutdown_connections 2024-04-05T22:26:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.797+0000 7fe4ef000640 1 -- 172.21.15.73:0/2940086050 wait complete. 2024-04-05T22:26:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.798+0000 7fe4ef000640 1 Processor -- start 2024-04-05T22:26:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.798+0000 7fe4ef000640 1 -- start start 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e81599f0 0x7fe4e81f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80ce3c0 con 0x7fe4e806cb80 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80cc7d0 con 0x7fe4e81599f0 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4ef000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80ccab0 con 0x7fe4e80add00 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49800/0 (socket says 172.21.15.73:49800) 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.799+0000 7fe4e7e00640 1 -- 172.21.15.73:0/3918205474 learned_addr learned my addr 172.21.15.73:0/3918205474 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 -- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 msgr2=0x7fe4e80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4dfe00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e80cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 -- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e81599f0 msgr2=0x7fe4e81f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e81599f0 0x7fe4e81f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 -- 172.21.15.73:0/3918205474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80094d0 con 0x7fe4e806cb80 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4dfe00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e80cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e7e00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8037930 con 0x7fe4e806cb80 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8037ab0 con 0x7fe4e806cb80 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e80ccd90 con 0x7fe4e806cb80 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.800+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8037e60 con 0x7fe4e806cb80 2024-04-05T22:26:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.801+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e80cd350 con 0x7fe4e806cb80 2024-04-05T22:26:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.801+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e8068160 con 0x7fe4e806cb80 2024-04-05T22:26:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.802+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe4d8042470 con 0x7fe4e806cb80 2024-04-05T22:26:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.803+0000 7fe4e4c00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 0x7fe4c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.803+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4d8033070 con 0x7fe4e806cb80 2024-04-05T22:26:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.804+0000 7fe4dfe00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 0x7fe4c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.804+0000 7fe4dfe00640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 0x7fe4c406d390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0006fd0 tx=0x7fe4d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:12.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:12.808+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe4d8077420 con 0x7fe4e806cb80 2024-04-05T22:26:13.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.024+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe4e80ccbf0 con 0x7fe4e806cb80 2024-04-05T22:26:13.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.571+0000 7fe4e4c00640 1 -- 172.21.15.73:0/3918205474 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4e80ccbf0 con 0x7fe4e806cb80 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 msgr2=0x7fe4c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 0x7fe4c406d390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0006fd0 tx=0x7fe4d0008040 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 msgr2=0x7fe4e8151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d80046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 shutdown_connections 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fe4c406aed0 0x7fe4c406d390 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e81599f0 0x7fe4e81f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e80add00 0x7fe4e80cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 --2- 172.21.15.73:0/3918205474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e806cb80 0x7fe4e8151820 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.574+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 >> 172.21.15.73:0/3918205474 conn(0x7fe4e8069cb0 msgr2=0x7fe4e8156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:13.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.575+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 shutdown_connections 2024-04-05T22:26:13.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.575+0000 7fe4ef000640 1 -- 172.21.15.73:0/3918205474 wait complete. 2024-04-05T22:26:13.596 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:26:13.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:13.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.801+0000 7f1e6ea00640 1 Processor -- start 2024-04-05T22:26:13.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.802+0000 7f1e6ea00640 1 -- start start 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.803+0000 7f1e6ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e681587d0 0x7f1e680cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.803+0000 7f1e6ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e680ccd20 0x7f1e6806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.803+0000 7f1e6ea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6806f5b0 con 0x7f1e6806d0c0 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.803+0000 7f1e6ea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e68070410 con 0x7f1e681587d0 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.803+0000 7f1e6ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e68158bb0 con 0x7f1e680ccd20 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e681587d0 0x7f1e680cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e6ca00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1e6806d0c0 0x7f1e6815e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60052/0 (socket says 172.21.15.73:60052) 2024-04-05T22:26:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e6ca00640 1 -- 172.21.15.73:0/3821998014 learned_addr learned my addr 172.21.15.73:0/3821998014 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67400640 1 --2- 172.21.15.73:0/3821998014 >> v2:172.21.15.157:3300/0 conn(0x7f1e680ccd20 0x7f1e6806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67e00640 1 -- 172.21.15.73:0/3821998014 >> v2:172.21.15.157:3300/0 conn(0x7f1e680ccd20 msgr2=0x7f1e6806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67e00640 1 --2- 172.21.15.73:0/3821998014 >> v2:172.21.15.157:3300/0 conn(0x7f1e680ccd20 0x7f1e6806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67e00640 1 -- 172.21.15.73:0/3821998014 >> v1:172.21.15.73:6789/0 conn(0x7f1e6806d0c0 legacy=0x7f1e6815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e67e00640 1 -- 172.21.15.73:0/3821998014 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6806f800 con 0x7f1e681587d0 2024-04-05T22:26:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.804+0000 7f1e66a00640 1 -- 172.21.15.73:0/3821998014 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160937877 0 0) 0x7f1e6806f5b0 con 0x7f1e6806d0c0 2024-04-05T22:26:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e67e00640 1 --2- 172.21.15.73:0/3821998014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680cc7e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097a0 tx=0x7f1e5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21a45eb6d63830c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:13.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e66a00640 1 -- 172.21.15.73:0/3821998014 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58031830 con 0x7f1e681587d0 2024-04-05T22:26:13.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e66a00640 1 -- 172.21.15.73:0/3821998014 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58031990 con 0x7f1e681587d0 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e66a00640 1 -- 172.21.15.73:0/3821998014 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58031d20 con 0x7f1e681587d0 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3821998014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 msgr2=0x7f1e680cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.806+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3821998014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680cc7e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097a0 tx=0x7f1e5802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3821998014 shutdown_connections 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3821998014 >> v2:172.21.15.157:3300/0 conn(0x7f1e680ccd20 0x7f1e6806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3821998014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680cc7e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3821998014 >> 172.21.15.73:0/3821998014 conn(0x7f1e680ce8e0 msgr2=0x7f1e680cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3821998014 shutdown_connections 2024-04-05T22:26:13.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.807+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3821998014 wait complete. 2024-04-05T22:26:13.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.808+0000 7f1e6ea00640 1 Processor -- start 2024-04-05T22:26:13.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.809+0000 7f1e6ea00640 1 -- start start 2024-04-05T22:26:13.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.809+0000 7f1e6ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e6806d0c0 0x7f1e680c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.809+0000 7f1e6ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e6ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e6814b6f0 0x7f1e680b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e6ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680c4a40 con 0x7f1e681587d0 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e6ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680c3dc0 con 0x7f1e6814b6f0 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e6ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680c40a0 con 0x7f1e6806d0c0 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49822/0 (socket says 172.21.15.73:49822) 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 -- 172.21.15.73:0/3763260099 learned_addr learned my addr 172.21.15.73:0/3763260099 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 -- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e6806d0c0 msgr2=0x7f1e680c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e6806d0c0 0x7f1e680c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 -- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e6814b6f0 msgr2=0x7f1e680b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:13.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e6ca00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e6814b6f0 0x7f1e680b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:13.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e6814b6f0 0x7f1e680b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:13.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.810+0000 7f1e67400640 1 -- 172.21.15.73:0/3763260099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009400 con 0x7f1e681587d0 2024-04-05T22:26:13.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e67400640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000d800 tx=0x7f1e5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000b6c0 con 0x7f1e681587d0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5000b820 con 0x7f1e681587d0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000bbb0 con 0x7f1e681587d0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e680c4380 con 0x7f1e681587d0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.811+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e680aedd0 con 0x7f1e681587d0 2024-04-05T22:26:13.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.812+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e68066af0 con 0x7f1e681587d0 2024-04-05T22:26:13.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.813+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f1e5000bd10 con 0x7f1e681587d0 2024-04-05T22:26:13.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.814+0000 7f1e5fe00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 0x7f1e4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:13.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.814+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e50089db0 con 0x7f1e681587d0 2024-04-05T22:26:13.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.814+0000 7f1e67e00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 0x7f1e4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:13.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.815+0000 7f1e67e00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 0x7f1e4406d420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e58036000 tx=0x7f1e58030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:13.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:13.819+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1e50054c10 con 0x7f1e681587d0 2024-04-05T22:26:14.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.054+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e68068160 con 0x7f1e681587d0 2024-04-05T22:26:14.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.056+0000 7f1e5fe00640 1 -- 172.21.15.73:0/3763260099 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f1e50058230 con 0x7f1e681587d0 2024-04-05T22:26:14.055 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:14.055 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:14.055 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 41 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 msgr2=0x7f1e4406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 0x7f1e4406d420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e58036000 tx=0x7f1e58030000 comp rx=0 tx=0).stop 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 msgr2=0x7f1e680ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000d800 tx=0x7f1e5000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 shutdown_connections 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1e4406af60 0x7f1e4406d420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e6814b6f0 0x7f1e680b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e681587d0 0x7f1e680ae890 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 --2- 172.21.15.73:0/3763260099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e6806d0c0 0x7f1e680c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.058+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 >> 172.21.15.73:0/3763260099 conn(0x7f1e680ce8e0 msgr2=0x7f1e680cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:14.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.059+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 shutdown_connections 2024-04-05T22:26:14.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.059+0000 7f1e6ea00640 1 -- 172.21.15.73:0/3763260099 wait complete. 2024-04-05T22:26:14.075 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.287+0000 7f601cc00640 1 Processor -- start 2024-04-05T22:26:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.287+0000 7f601cc00640 1 -- start start 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f601cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f601815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f601cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601815efc0 con 0x7f6018158860 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f601cc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601806b500 con 0x7f601806bc70 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f601cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ccaf0 con 0x7f601815e620 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f6017400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6018158860 0x7f60180b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60062/0 (socket says 172.21.15.73:60062) 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f6017400640 1 -- 172.21.15.73:0/2477241051 learned_addr learned my addr 172.21.15.73:0/2477241051 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f6016a00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f601815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101324081 0 0) 0x7f601815efc0 con 0x7f6018158860 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.289+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ffc003520 con 0x7f6018158860 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293274766 0 0) 0x7f5ffc003520 con 0x7f6018158860 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 msgr2=0x7f601815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f601815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 >> v1:172.21.15.157:6789/0 conn(0x7f601806bc70 legacy=0x7f601806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180b20d0 con 0x7f6018158860 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6008001670 con 0x7f6018158860 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 >> v1:172.21.15.73:6789/0 conn(0x7f6018158860 legacy=0x7f60180b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f5ffc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ffc003f20 0x7f5ffc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc004d00 0x7f5ffc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ccaf0 con 0x7f5ffc003f20 2024-04-05T22:26:14.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc00bb10 con 0x7f5ffc004d00 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc00bcd0 con 0x7f601815e620 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.290+0000 7f6016a00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ffc003f20 0x7f5ffc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6017e00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc004d00 0x7f5ffc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6017400640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f5ffc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6016a00640 1 -- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 msgr2=0x7f5ffc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6016a00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f5ffc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6016a00640 1 -- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc004d00 msgr2=0x7f5ffc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6016a00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc004d00 0x7f5ffc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6016a00640 1 -- 172.21.15.73:0/2477241051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc003520 con 0x7f5ffc003f20 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6017e00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc004d00 0x7f5ffc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:14.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.291+0000 7f6017400640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f601815e620 0x7f5ffc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.292+0000 7f6016a00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ffc003f20 0x7f5ffc0047c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6004009ac0 tx=0x7f5ffc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bde0c0b2e67537 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.292+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600402c6b0 con 0x7f5ffc003f20 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.292+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600402c810 con 0x7f5ffc003f20 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.292+0000 7f6016000640 1 -- 172.21.15.73:0/2477241051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600402cba0 con 0x7f5ffc003f20 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 -- 172.21.15.73:0/2477241051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ffc003f20 msgr2=0x7f5ffc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 --2- 172.21.15.73:0/2477241051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ffc003f20 0x7f5ffc0047c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6004009ac0 tx=0x7f5ffc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f6017e00640 1 -- 172.21.15.73:0/2477241051 reap_dead start 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 -- 172.21.15.73:0/2477241051 shutdown_connections 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 -- 172.21.15.73:0/2477241051 >> 172.21.15.73:0/2477241051 conn(0x7f60180b06c0 msgr2=0x7f60180b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 -- 172.21.15.73:0/2477241051 shutdown_connections 2024-04-05T22:26:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.293+0000 7f601cc00640 1 -- 172.21.15.73:0/2477241051 wait complete. 2024-04-05T22:26:14.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.295+0000 7f601cc00640 1 Processor -- start 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.295+0000 7f601cc00640 1 -- start start 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc003f20 0x7f60180ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 0x7f601814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601814bb90 con 0x7f601814aad0 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ce2e0 con 0x7f601814b280 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f601cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180cecb0 con 0x7f5ffc003f20 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f6017e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f6017e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49840/0 (socket says 172.21.15.73:49840) 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f6017e00640 1 -- 172.21.15.73:0/2542533016 learned_addr learned my addr 172.21.15.73:0/2542533016 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f6017400640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc003f20 0x7f60180ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.296+0000 7f6016a00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 0x7f601814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 -- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc003f20 msgr2=0x7f60180ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc003f20 0x7f60180ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 -- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 msgr2=0x7f601814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 0x7f601814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 -- 172.21.15.73:0/2542533016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180cd710 con 0x7f601814aad0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6016a00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 0x7f601814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f6017e00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f60180b45d0 tx=0x7f601000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010006d50 con 0x7f601814aad0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010003970 con 0x7f601814aad0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601814cf30 con 0x7f601814aad0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010003d60 con 0x7f601814aad0 2024-04-05T22:26:14.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.297+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60180cdc90 con 0x7f601814aad0 2024-04-05T22:26:14.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.298+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601804e160 con 0x7f601814aad0 2024-04-05T22:26:14.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.299+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f6010005070 con 0x7f601814aad0 2024-04-05T22:26:14.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.300+0000 7f600f400640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 0x7f5ff806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.300+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6010089630 con 0x7f601814aad0 2024-04-05T22:26:14.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.300+0000 7f6017400640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 0x7f5ff806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.301+0000 7f6017400640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 0x7f5ff806d420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6008001eb0 tx=0x7f6008001510 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.304+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6010054340 con 0x7f601814aad0 2024-04-05T22:26:14.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.499+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60180ce420 con 0x7f601814aad0 2024-04-05T22:26:14.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.501+0000 7f600f400640 1 -- 172.21.15.73:0/2542533016 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f6010057960 con 0x7f601814aad0 2024-04-05T22:26:14.501 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:14.501 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:11.541344+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:14.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 msgr2=0x7f5ff806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 0x7f5ff806d420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6008001eb0 tx=0x7f6008001510 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 msgr2=0x7f60180aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f60180b45d0 tx=0x7f601000dc00 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 shutdown_connections 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f5ff806af60 0x7f5ff806d420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f601814aad0 0x7f60180aede0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601814b280 0x7f601814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 --2- 172.21.15.73:0/2542533016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc003f20 0x7f60180ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 >> 172.21.15.73:0/2542533016 conn(0x7f60180b06c0 msgr2=0x7f60180b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 shutdown_connections 2024-04-05T22:26:14.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.504+0000 7f601cc00640 1 -- 172.21.15.73:0/2542533016 wait complete. 2024-04-05T22:26:14.522 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:26:14.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:26:14.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.736+0000 7fcbff000640 1 Processor -- start 2024-04-05T22:26:14.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.736+0000 7fcbff000640 1 -- start start 2024-04-05T22:26:14.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbff000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbf80b20a0 0x7fcbf8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbff000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 0x7fcbf806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbff000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf806d3e0 con 0x7fcbf80b2480 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbff000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf806d610 con 0x7fcbf80b20a0 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbff000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf8070230 con 0x7fcbf80b1b50 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbfdc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbf80b20a0 0x7fcbf8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbfdc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbf80b20a0 0x7fcbf8070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49850/0 (socket says 172.21.15.73:49850) 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbfdc00640 1 -- 172.21.15.73:0/2156731020 learned_addr learned my addr 172.21.15.73:0/2156731020 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:14.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.738+0000 7fcbfe600640 1 --1- 172.21.15.73:0/2156731020 >> v1:172.21.15.73:6789/0 conn(0x7fcbf80b2480 0x7fcbf80b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60070/0 (socket says 172.21.15.73:60070) 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfd200640 1 --2- 172.21.15.73:0/2156731020 >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 0x7fcbf806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfdc00640 1 -- 172.21.15.73:0/2156731020 >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 msgr2=0x7fcbf806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfdc00640 1 --2- 172.21.15.73:0/2156731020 >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 0x7fcbf806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfdc00640 1 -- 172.21.15.73:0/2156731020 >> v1:172.21.15.73:6789/0 conn(0x7fcbf80b2480 legacy=0x7fcbf80b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfdc00640 1 -- 172.21.15.73:0/2156731020 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8070510 con 0x7fcbf80b20a0 2024-04-05T22:26:14.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.739+0000 7fcbfd200640 1 --2- 172.21.15.73:0/2156731020 >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 0x7fcbf806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.740+0000 7fcbfdc00640 1 --2- 172.21.15.73:0/2156731020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8070ff0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbf806f380 tx=0x7fcbec02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04e5e88f10ae52c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.740+0000 7fcbf7e00640 1 -- 172.21.15.73:0/2156731020 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0317f0 con 0x7fcbf80b20a0 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.740+0000 7fcbf7e00640 1 -- 172.21.15.73:0/2156731020 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec031950 con 0x7fcbf80b20a0 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.740+0000 7fcbf7e00640 1 -- 172.21.15.73:0/2156731020 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec031ce0 con 0x7fcbf80b20a0 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 -- 172.21.15.73:0/2156731020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b20a0 msgr2=0x7fcbf8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 --2- 172.21.15.73:0/2156731020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8070ff0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbf806f380 tx=0x7fcbec02b890 comp rx=0 tx=0).stop 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 -- 172.21.15.73:0/2156731020 shutdown_connections 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 --2- 172.21.15.73:0/2156731020 >> v2:172.21.15.157:3300/0 conn(0x7fcbf80b1b50 0x7fcbf806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 --2- 172.21.15.73:0/2156731020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8070ff0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 -- 172.21.15.73:0/2156731020 >> 172.21.15.73:0/2156731020 conn(0x7fcbf80b06c0 msgr2=0x7fcbf80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:14.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 -- 172.21.15.73:0/2156731020 shutdown_connections 2024-04-05T22:26:14.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.741+0000 7fcbff000640 1 -- 172.21.15.73:0/2156731020 wait complete. 2024-04-05T22:26:14.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.743+0000 7fcbff000640 1 Processor -- start 2024-04-05T22:26:14.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.743+0000 7fcbff000640 1 -- start start 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbf80b1b50 0x7fcbf8149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8152d00 con 0x7fcbf80b2480 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80acc00 con 0x7fcbf80b20a0 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbff000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf81520c0 con 0x7fcbf80b1b50 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbfe600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbfe600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49866/0 (socket says 172.21.15.73:49866) 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbfe600640 1 -- 172.21.15.73:0/1597538875 learned_addr learned my addr 172.21.15.73:0/1597538875 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbfd200640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.744+0000 7fcbfdc00640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbf80b1b50 0x7fcbf8149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 -- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbf80b1b50 msgr2=0x7fcbf8149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbf80b1b50 0x7fcbf8149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 -- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 msgr2=0x7fcbf8156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 -- 172.21.15.73:0/1597538875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec0094d0 con 0x7fcbf80b2480 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfd200640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbfe600640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c7b0 tx=0x7fcbf000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000a640 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf000a7a0 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000ab10 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf8152360 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.745+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf8152920 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.746+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf8152e40 con 0x7fcbf80b2480 2024-04-05T22:26:14.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.747+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fcbf000ac70 con 0x7fcbf80b2480 2024-04-05T22:26:14.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.748+0000 7fcbf5600640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 0x7fcbd806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:14.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.748+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbf0089130 con 0x7fcbf80b2480 2024-04-05T22:26:14.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.748+0000 7fcbfdc00640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 0x7fcbd806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:14.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.749+0000 7fcbfdc00640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 0x7fcbd806d420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcbec036000 tx=0x7fcbec03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:14.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.753+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcbf8152e40 con 0x7fcbf80b2480 2024-04-05T22:26:14.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.941+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbf8070ad0 con 0x7fcbd806af60 2024-04-05T22:26:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.965+0000 7fcbf5600640 1 -- 172.21.15.73:0/1597538875 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286807 (secure 0 0 0) 0x7fcbec031c40 con 0x7fcbd806af60 2024-04-05T22:26:14.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:14.971 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:26:14.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 msgr2=0x7fcbd806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 0x7fcbd806d420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcbec036000 tx=0x7fcbec03d040 comp rx=0 tx=0).stop 2024-04-05T22:26:14.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 msgr2=0x7fcbf80ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:14.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c7b0 tx=0x7fcbf000cc70 comp rx=0 tx=0).stop 2024-04-05T22:26:14.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 shutdown_connections 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcbd806af60 0x7fcbd806d420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf80b2480 0x7fcbf80ccad0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf80b20a0 0x7fcbf8156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 --2- 172.21.15.73:0/1597538875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbf80b1b50 0x7fcbf8149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.972+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 >> 172.21.15.73:0/1597538875 conn(0x7fcbf80b06c0 msgr2=0x7fcbf80b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.973+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 shutdown_connections 2024-04-05T22:26:14.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:14.973+0000 7fcbff000640 1 -- 172.21.15.73:0/1597538875 wait complete. 2024-04-05T22:26:14.990 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-05T22:26:14.590696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257184,"kb_used_data":8208,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754717536,"statfs":{"total":773094113280,"available":772830756864,"internally_reserved":0,"allocated":8404992,"data_stored":4194476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.548632"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243873+0000","last_change":"2024-04-05T22:26:10.584470+0000","last_active":"2024-04-05T22:26:12.243873+0000","last_peered":"2024-04-05T22:26:12.243873+0000","last_clean":"2024-04-05T22:26:12.243873+0000","last_became_active":"2024-04-05T22:26:10.583641+0000","last_became_peered":"2024-04-05T22:26:10.583641+0000","last_unstale":"2024-04-05T22:26:12.243873+0000","last_undegraded":"2024-04-05T22:26:12.243873+0000","last_fullsized":"2024-04-05T22:26:12.243873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:22.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246148+0000","last_change":"2024-04-05T22:26:10.589009+0000","last_active":"2024-04-05T22:26:12.246148+0000","last_peered":"2024-04-05T22:26:12.246148+0000","last_clean":"2024-04-05T22:26:12.246148+0000","last_became_active":"2024-04-05T22:26:10.588729+0000","last_became_peered":"2024-04-05T22:26:10.588729+0000","last_unstale":"2024-04-05T22:26:12.246148+0000","last_undegraded":"2024-04-05T22:26:12.246148+0000","last_fullsized":"2024-04-05T22:26:12.246148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:53:52.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242155+0000","last_change":"2024-04-05T22:26:10.569517+0000","last_active":"2024-04-05T22:26:12.242155+0000","last_peered":"2024-04-05T22:26:12.242155+0000","last_clean":"2024-04-05T22:26:12.242155+0000","last_became_active":"2024-04-05T22:26:10.568811+0000","last_became_peered":"2024-04-05T22:26:10.568811+0000","last_unstale":"2024-04-05T22:26:12.242155+0000","last_undegraded":"2024-04-05T22:26:12.242155+0000","last_fullsized":"2024-04-05T22:26:12.242155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:12:34.064555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557883+0000","last_change":"2024-04-05T22:26:10.578648+0000","last_active":"2024-04-05T22:26:11.557883+0000","last_peered":"2024-04-05T22:26:11.557883+0000","last_clean":"2024-04-05T22:26:11.557883+0000","last_became_active":"2024-04-05T22:26:10.578342+0000","last_became_peered":"2024-04-05T22:26:10.578342+0000","last_unstale":"2024-04-05T22:26:11.557883+0000","last_undegraded":"2024-04-05T22:26:11.557883+0000","last_fullsized":"2024-04-05T22:26:11.557883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:18:37.871348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560829+0000","last_change":"2024-04-05T22:26:10.567910+0000","last_active":"2024-04-05T22:26:11.560829+0000","last_peered":"2024-04-05T22:26:11.560829+0000","last_clean":"2024-04-05T22:26:11.560829+0000","last_became_active":"2024-04-05T22:26:10.567621+0000","last_became_peered":"2024-04-05T22:26:10.567621+0000","last_unstale":"2024-04-05T22:26:11.560829+0000","last_undegraded":"2024-04-05T22:26:11.560829+0000","last_fullsized":"2024-04-05T22:26:11.560829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:11:26.707227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244079+0000","last_change":"2024-04-05T22:26:10.565467+0000","last_active":"2024-04-05T22:26:12.244079+0000","last_peered":"2024-04-05T22:26:12.244079+0000","last_clean":"2024-04-05T22:26:12.244079+0000","last_became_active":"2024-04-05T22:26:10.565069+0000","last_became_peered":"2024-04-05T22:26:10.565069+0000","last_unstale":"2024-04-05T22:26:12.244079+0000","last_undegraded":"2024-04-05T22:26:12.244079+0000","last_fullsized":"2024-04-05T22:26:12.244079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:48.544527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558399+0000","last_change":"2024-04-05T22:26:10.581377+0000","last_active":"2024-04-05T22:26:11.558399+0000","last_peered":"2024-04-05T22:26:11.558399+0000","last_clean":"2024-04-05T22:26:11.558399+0000","last_became_active":"2024-04-05T22:26:10.579510+0000","last_became_peered":"2024-04-05T22:26:10.579510+0000","last_unstale":"2024-04-05T22:26:11.558399+0000","last_undegraded":"2024-04-05T22:26:11.558399+0000","last_fullsized":"2024-04-05T22:26:11.558399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244404+0000","last_change":"2024-04-05T22:26:10.587800+0000","last_active":"2024-04-05T22:26:12.244404+0000","last_peered":"2024-04-05T22:26:12.244404+0000","last_clean":"2024-04-05T22:26:12.244404+0000","last_became_active":"2024-04-05T22:26:10.587311+0000","last_became_peered":"2024-04-05T22:26:10.587311+0000","last_unstale":"2024-04-05T22:26:12.244404+0000","last_undegraded":"2024-04-05T22:26:12.244404+0000","last_fullsized":"2024-04-05T22:26:12.244404+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:43.465833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560725+0000","last_change":"2024-04-05T22:26:10.575075+0000","last_active":"2024-04-05T22:26:11.560725+0000","last_peered":"2024-04-05T22:26:11.560725+0000","last_clean":"2024-04-05T22:26:11.560725+0000","last_became_active":"2024-04-05T22:26:10.574500+0000","last_became_peered":"2024-04-05T22:26:10.574500+0000","last_unstale":"2024-04-05T22:26:11.560725+0000","last_undegraded":"2024-04-05T22:26:11.560725+0000","last_fullsized":"2024-04-05T22:26:11.560725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:07.391519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245474+0000","last_change":"2024-04-05T22:26:10.580244+0000","last_active":"2024-04-05T22:26:12.245474+0000","last_peered":"2024-04-05T22:26:12.245474+0000","last_clean":"2024-04-05T22:26:12.245474+0000","last_became_active":"2024-04-05T22:26:10.579954+0000","last_became_peered":"2024-04-05T22:26:10.579954+0000","last_unstale":"2024-04-05T22:26:12.245474+0000","last_undegraded":"2024-04-05T22:26:12.245474+0000","last_fullsized":"2024-04-05T22:26:12.245474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:44:33.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242725+0000","last_change":"2024-04-05T22:26:10.587090+0000","last_active":"2024-04-05T22:26:12.242725+0000","last_peered":"2024-04-05T22:26:12.242725+0000","last_clean":"2024-04-05T22:26:12.242725+0000","last_became_active":"2024-04-05T22:26:10.586794+0000","last_became_peered":"2024-04-05T22:26:10.586794+0000","last_unstale":"2024-04-05T22:26:12.242725+0000","last_undegraded":"2024-04-05T22:26:12.242725+0000","last_fullsized":"2024-04-05T22:26:12.242725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:34:30.583583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557985+0000","last_change":"2024-04-05T22:26:10.556444+0000","last_active":"2024-04-05T22:26:11.557985+0000","last_peered":"2024-04-05T22:26:11.557985+0000","last_clean":"2024-04-05T22:26:11.557985+0000","last_became_active":"2024-04-05T22:26:10.556154+0000","last_became_peered":"2024-04-05T22:26:10.556154+0000","last_unstale":"2024-04-05T22:26:11.557985+0000","last_undegraded":"2024-04-05T22:26:11.557985+0000","last_fullsized":"2024-04-05T22:26:11.557985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:36.699342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241801+0000","last_change":"2024-04-05T22:26:10.567200+0000","last_active":"2024-04-05T22:26:12.241801+0000","last_peered":"2024-04-05T22:26:12.241801+0000","last_clean":"2024-04-05T22:26:12.241801+0000","last_became_active":"2024-04-05T22:26:10.566952+0000","last_became_peered":"2024-04-05T22:26:10.566952+0000","last_unstale":"2024-04-05T22:26:12.241801+0000","last_undegraded":"2024-04-05T22:26:12.241801+0000","last_fullsized":"2024-04-05T22:26:12.241801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:33:56.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560524+0000","last_change":"2024-04-05T22:26:10.566979+0000","last_active":"2024-04-05T22:26:11.560524+0000","last_peered":"2024-04-05T22:26:11.560524+0000","last_clean":"2024-04-05T22:26:11.560524+0000","last_became_active":"2024-04-05T22:26:10.566697+0000","last_became_peered":"2024-04-05T22:26:10.566697+0000","last_unstale":"2024-04-05T22:26:11.560524+0000","last_undegraded":"2024-04-05T22:26:11.560524+0000","last_fullsized":"2024-04-05T22:26:11.560524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:46.949572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242659+0000","last_change":"2024-04-05T22:26:10.561447+0000","last_active":"2024-04-05T22:26:12.242659+0000","last_peered":"2024-04-05T22:26:12.242659+0000","last_clean":"2024-04-05T22:26:12.242659+0000","last_became_active":"2024-04-05T22:26:10.561064+0000","last_became_peered":"2024-04-05T22:26:10.561064+0000","last_unstale":"2024-04-05T22:26:12.242659+0000","last_undegraded":"2024-04-05T22:26:12.242659+0000","last_fullsized":"2024-04-05T22:26:12.242659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:45:19.947550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242946+0000","last_change":"2024-04-05T22:26:10.579209+0000","last_active":"2024-04-05T22:26:12.242946+0000","last_peered":"2024-04-05T22:26:12.242946+0000","last_clean":"2024-04-05T22:26:12.242946+0000","last_became_active":"2024-04-05T22:26:10.570235+0000","last_became_peered":"2024-04-05T22:26:10.570235+0000","last_unstale":"2024-04-05T22:26:12.242946+0000","last_undegraded":"2024-04-05T22:26:12.242946+0000","last_fullsized":"2024-04-05T22:26:12.242946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:21:07.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560831+0000","last_change":"2024-04-05T22:26:10.574663+0000","last_active":"2024-04-05T22:26:11.560831+0000","last_peered":"2024-04-05T22:26:11.560831+0000","last_clean":"2024-04-05T22:26:11.560831+0000","last_became_active":"2024-04-05T22:26:10.574220+0000","last_became_peered":"2024-04-05T22:26:10.574220+0000","last_unstale":"2024-04-05T22:26:11.560831+0000","last_undegraded":"2024-04-05T22:26:11.560831+0000","last_fullsized":"2024-04-05T22:26:11.560831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:56.605089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558421+0000","last_change":"2024-04-05T22:26:10.569850+0000","last_active":"2024-04-05T22:26:11.558421+0000","last_peered":"2024-04-05T22:26:11.558421+0000","last_clean":"2024-04-05T22:26:11.558421+0000","last_became_active":"2024-04-05T22:26:10.569562+0000","last_became_peered":"2024-04-05T22:26:10.569562+0000","last_unstale":"2024-04-05T22:26:11.558421+0000","last_undegraded":"2024-04-05T22:26:11.558421+0000","last_fullsized":"2024-04-05T22:26:11.558421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:30:08.688197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245029+0000","last_change":"2024-04-05T22:26:10.586906+0000","last_active":"2024-04-05T22:26:12.245029+0000","last_peered":"2024-04-05T22:26:12.245029+0000","last_clean":"2024-04-05T22:26:12.245029+0000","last_became_active":"2024-04-05T22:26:10.578421+0000","last_became_peered":"2024-04-05T22:26:10.578421+0000","last_unstale":"2024-04-05T22:26:12.245029+0000","last_undegraded":"2024-04-05T22:26:12.245029+0000","last_fullsized":"2024-04-05T22:26:12.245029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:52.827753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243011+0000","last_change":"2024-04-05T22:26:10.566736+0000","last_active":"2024-04-05T22:26:12.243011+0000","last_peered":"2024-04-05T22:26:12.243011+0000","last_clean":"2024-04-05T22:26:12.243011+0000","last_became_active":"2024-04-05T22:26:10.566445+0000","last_became_peered":"2024-04-05T22:26:10.566445+0000","last_unstale":"2024-04-05T22:26:12.243011+0000","last_undegraded":"2024-04-05T22:26:12.243011+0000","last_fullsized":"2024-04-05T22:26:12.243011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:57.069630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243627+0000","last_change":"2024-04-05T22:26:10.564326+0000","last_active":"2024-04-05T22:26:12.243627+0000","last_peered":"2024-04-05T22:26:12.243627+0000","last_clean":"2024-04-05T22:26:12.243627+0000","last_became_active":"2024-04-05T22:26:10.564107+0000","last_became_peered":"2024-04-05T22:26:10.564107+0000","last_unstale":"2024-04-05T22:26:12.243627+0000","last_undegraded":"2024-04-05T22:26:12.243627+0000","last_fullsized":"2024-04-05T22:26:12.243627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:16.485563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243503+0000","last_change":"2024-04-05T22:26:10.569091+0000","last_active":"2024-04-05T22:26:12.243503+0000","last_peered":"2024-04-05T22:26:12.243503+0000","last_clean":"2024-04-05T22:26:12.243503+0000","last_became_active":"2024-04-05T22:26:10.568796+0000","last_became_peered":"2024-04-05T22:26:10.568796+0000","last_unstale":"2024-04-05T22:26:12.243503+0000","last_undegraded":"2024-04-05T22:26:12.243503+0000","last_fullsized":"2024-04-05T22:26:12.243503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:28:42.459472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242041+0000","last_change":"2024-04-05T22:26:10.572005+0000","last_active":"2024-04-05T22:26:12.242041+0000","last_peered":"2024-04-05T22:26:12.242041+0000","last_clean":"2024-04-05T22:26:12.242041+0000","last_became_active":"2024-04-05T22:26:10.571689+0000","last_became_peered":"2024-04-05T22:26:10.571689+0000","last_unstale":"2024-04-05T22:26:12.242041+0000","last_undegraded":"2024-04-05T22:26:12.242041+0000","last_fullsized":"2024-04-05T22:26:12.242041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:12.493196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241838+0000","last_change":"2024-04-05T22:26:10.590873+0000","last_active":"2024-04-05T22:26:12.241838+0000","last_peered":"2024-04-05T22:26:12.241838+0000","last_clean":"2024-04-05T22:26:12.241838+0000","last_became_active":"2024-04-05T22:26:10.590587+0000","last_became_peered":"2024-04-05T22:26:10.590587+0000","last_unstale":"2024-04-05T22:26:12.241838+0000","last_undegraded":"2024-04-05T22:26:12.241838+0000","last_fullsized":"2024-04-05T22:26:12.241838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245978+0000","last_change":"2024-04-05T22:26:10.572471+0000","last_active":"2024-04-05T22:26:12.245978+0000","last_peered":"2024-04-05T22:26:12.245978+0000","last_clean":"2024-04-05T22:26:12.245978+0000","last_became_active":"2024-04-05T22:26:10.572216+0000","last_became_peered":"2024-04-05T22:26:10.572216+0000","last_unstale":"2024-04-05T22:26:12.245978+0000","last_undegraded":"2024-04-05T22:26:12.245978+0000","last_fullsized":"2024-04-05T22:26:12.245978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:19.667713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558879+0000","last_change":"2024-04-05T22:26:10.574476+0000","last_active":"2024-04-05T22:26:11.558879+0000","last_peered":"2024-04-05T22:26:11.558879+0000","last_clean":"2024-04-05T22:26:11.558879+0000","last_became_active":"2024-04-05T22:26:10.574173+0000","last_became_peered":"2024-04-05T22:26:10.574173+0000","last_unstale":"2024-04-05T22:26:11.558879+0000","last_undegraded":"2024-04-05T22:26:11.558879+0000","last_fullsized":"2024-04-05T22:26:11.558879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:59.523605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242403+0000","last_change":"2024-04-05T22:26:10.581955+0000","last_active":"2024-04-05T22:26:12.242403+0000","last_peered":"2024-04-05T22:26:12.242403+0000","last_clean":"2024-04-05T22:26:12.242403+0000","last_became_active":"2024-04-05T22:26:10.581680+0000","last_became_peered":"2024-04-05T22:26:10.581680+0000","last_unstale":"2024-04-05T22:26:12.242403+0000","last_undegraded":"2024-04-05T22:26:12.242403+0000","last_fullsized":"2024-04-05T22:26:12.242403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:11:18.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245200+0000","last_change":"2024-04-05T22:26:10.571157+0000","last_active":"2024-04-05T22:26:12.245200+0000","last_peered":"2024-04-05T22:26:12.245200+0000","last_clean":"2024-04-05T22:26:12.245200+0000","last_became_active":"2024-04-05T22:26:10.570896+0000","last_became_peered":"2024-04-05T22:26:10.570896+0000","last_unstale":"2024-04-05T22:26:12.245200+0000","last_undegraded":"2024-04-05T22:26:12.245200+0000","last_fullsized":"2024-04-05T22:26:12.245200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:37:24.002924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242034+0000","last_change":"2024-04-05T22:26:10.571475+0000","last_active":"2024-04-05T22:26:12.242034+0000","last_peered":"2024-04-05T22:26:12.242034+0000","last_clean":"2024-04-05T22:26:12.242034+0000","last_became_active":"2024-04-05T22:26:10.571206+0000","last_became_peered":"2024-04-05T22:26:10.571206+0000","last_unstale":"2024-04-05T22:26:12.242034+0000","last_undegraded":"2024-04-05T22:26:12.242034+0000","last_fullsized":"2024-04-05T22:26:12.242034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:01:45.884036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558657+0000","last_change":"2024-04-05T22:26:10.590577+0000","last_active":"2024-04-05T22:26:11.558657+0000","last_peered":"2024-04-05T22:26:11.558657+0000","last_clean":"2024-04-05T22:26:11.558657+0000","last_became_active":"2024-04-05T22:26:10.590237+0000","last_became_peered":"2024-04-05T22:26:10.590237+0000","last_unstale":"2024-04-05T22:26:11.558657+0000","last_undegraded":"2024-04-05T22:26:11.558657+0000","last_fullsized":"2024-04-05T22:26:11.558657+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:06:50.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558672+0000","last_change":"2024-04-05T22:26:10.574307+0000","last_active":"2024-04-05T22:26:11.558672+0000","last_peered":"2024-04-05T22:26:11.558672+0000","last_clean":"2024-04-05T22:26:11.558672+0000","last_became_active":"2024-04-05T22:26:10.573972+0000","last_became_peered":"2024-04-05T22:26:10.573972+0000","last_unstale":"2024-04-05T22:26:11.558672+0000","last_undegraded":"2024-04-05T22:26:11.558672+0000","last_fullsized":"2024-04-05T22:26:11.558672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:22.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244471+0000","last_change":"2024-04-05T22:26:10.578259+0000","last_active":"2024-04-05T22:26:12.244471+0000","last_peered":"2024-04-05T22:26:12.244471+0000","last_clean":"2024-04-05T22:26:12.244471+0000","last_became_active":"2024-04-05T22:26:10.577941+0000","last_became_peered":"2024-04-05T22:26:10.577941+0000","last_unstale":"2024-04-05T22:26:12.244471+0000","last_undegraded":"2024-04-05T22:26:12.244471+0000","last_fullsized":"2024-04-05T22:26:12.244471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:29:13.296087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243602+0000","last_change":"2024-04-05T22:26:10.592487+0000","last_active":"2024-04-05T22:26:12.243602+0000","last_peered":"2024-04-05T22:26:12.243602+0000","last_clean":"2024-04-05T22:26:12.243602+0000","last_became_active":"2024-04-05T22:26:10.592201+0000","last_became_peered":"2024-04-05T22:26:10.592201+0000","last_unstale":"2024-04-05T22:26:12.243602+0000","last_undegraded":"2024-04-05T22:26:12.243602+0000","last_fullsized":"2024-04-05T22:26:12.243602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244588+0000","last_change":"2024-04-05T22:26:10.087306+0000","last_active":"2024-04-05T22:26:12.244588+0000","last_peered":"2024-04-05T22:26:12.244588+0000","last_clean":"2024-04-05T22:26:12.244588+0000","last_became_active":"2024-04-05T22:26:10.086991+0000","last_became_peered":"2024-04-05T22:26:10.086991+0000","last_unstale":"2024-04-05T22:26:12.244588+0000","last_undegraded":"2024-04-05T22:26:12.244588+0000","last_fullsized":"2024-04-05T22:26:12.244588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:12.517227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560942+0000","last_change":"2024-04-05T22:26:10.574491+0000","last_active":"2024-04-05T22:26:11.560942+0000","last_peered":"2024-04-05T22:26:11.560942+0000","last_clean":"2024-04-05T22:26:11.560942+0000","last_became_active":"2024-04-05T22:26:10.574094+0000","last_became_peered":"2024-04-05T22:26:10.574094+0000","last_unstale":"2024-04-05T22:26:11.560942+0000","last_undegraded":"2024-04-05T22:26:11.560942+0000","last_fullsized":"2024-04-05T22:26:11.560942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:49:15.370190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243194+0000","last_change":"2024-04-05T22:26:10.087418+0000","last_active":"2024-04-05T22:26:12.243194+0000","last_peered":"2024-04-05T22:26:12.243194+0000","last_clean":"2024-04-05T22:26:12.243194+0000","last_became_active":"2024-04-05T22:26:10.087115+0000","last_became_peered":"2024-04-05T22:26:10.087115+0000","last_unstale":"2024-04-05T22:26:12.243194+0000","last_undegraded":"2024-04-05T22:26:12.243194+0000","last_fullsized":"2024-04-05T22:26:12.243194+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:15.473436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244129+0000","last_change":"2024-04-05T22:26:10.578151+0000","last_active":"2024-04-05T22:26:12.244129+0000","last_peered":"2024-04-05T22:26:12.244129+0000","last_clean":"2024-04-05T22:26:12.244129+0000","last_became_active":"2024-04-05T22:26:10.573361+0000","last_became_peered":"2024-04-05T22:26:10.573361+0000","last_unstale":"2024-04-05T22:26:12.244129+0000","last_undegraded":"2024-04-05T22:26:12.244129+0000","last_fullsized":"2024-04-05T22:26:12.244129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:40:30.650718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245421+0000","last_change":"2024-04-05T22:26:10.086105+0000","last_active":"2024-04-05T22:26:12.245421+0000","last_peered":"2024-04-05T22:26:12.245421+0000","last_clean":"2024-04-05T22:26:12.245421+0000","last_became_active":"2024-04-05T22:26:10.085852+0000","last_became_peered":"2024-04-05T22:26:10.085852+0000","last_unstale":"2024-04-05T22:26:12.245421+0000","last_undegraded":"2024-04-05T22:26:12.245421+0000","last_fullsized":"2024-04-05T22:26:12.245421+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:48.275500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242812+0000","last_change":"2024-04-05T22:26:10.584232+0000","last_active":"2024-04-05T22:26:12.242812+0000","last_peered":"2024-04-05T22:26:12.242812+0000","last_clean":"2024-04-05T22:26:12.242812+0000","last_became_active":"2024-04-05T22:26:10.583918+0000","last_became_peered":"2024-04-05T22:26:10.583918+0000","last_unstale":"2024-04-05T22:26:12.242812+0000","last_undegraded":"2024-04-05T22:26:12.242812+0000","last_fullsized":"2024-04-05T22:26:12.242812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:05.948210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244425+0000","last_change":"2024-04-05T22:26:10.090172+0000","last_active":"2024-04-05T22:26:12.244425+0000","last_peered":"2024-04-05T22:26:12.244425+0000","last_clean":"2024-04-05T22:26:12.244425+0000","last_became_active":"2024-04-05T22:26:10.089492+0000","last_became_peered":"2024-04-05T22:26:10.089492+0000","last_unstale":"2024-04-05T22:26:12.244425+0000","last_undegraded":"2024-04-05T22:26:12.244425+0000","last_fullsized":"2024-04-05T22:26:12.244425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:40.596358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558661+0000","last_change":"2024-04-05T22:26:10.576222+0000","last_active":"2024-04-05T22:26:11.558661+0000","last_peered":"2024-04-05T22:26:11.558661+0000","last_clean":"2024-04-05T22:26:11.558661+0000","last_became_active":"2024-04-05T22:26:10.574204+0000","last_became_peered":"2024-04-05T22:26:10.574204+0000","last_unstale":"2024-04-05T22:26:11.558661+0000","last_undegraded":"2024-04-05T22:26:11.558661+0000","last_fullsized":"2024-04-05T22:26:11.558661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:16:17.822694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242508+0000","last_change":"2024-04-05T22:26:10.084099+0000","last_active":"2024-04-05T22:26:12.242508+0000","last_peered":"2024-04-05T22:26:12.242508+0000","last_clean":"2024-04-05T22:26:12.242508+0000","last_became_active":"2024-04-05T22:26:10.083850+0000","last_became_peered":"2024-04-05T22:26:10.083850+0000","last_unstale":"2024-04-05T22:26:12.242508+0000","last_undegraded":"2024-04-05T22:26:12.242508+0000","last_fullsized":"2024-04-05T22:26:12.242508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:16:03.136925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243923+0000","last_change":"2024-04-05T22:26:10.573301+0000","last_active":"2024-04-05T22:26:12.243923+0000","last_peered":"2024-04-05T22:26:12.243923+0000","last_clean":"2024-04-05T22:26:12.243923+0000","last_became_active":"2024-04-05T22:26:10.573035+0000","last_became_peered":"2024-04-05T22:26:10.573035+0000","last_unstale":"2024-04-05T22:26:12.243923+0000","last_undegraded":"2024-04-05T22:26:12.243923+0000","last_fullsized":"2024-04-05T22:26:12.243923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:42.938950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244339+0000","last_change":"2024-04-05T22:26:10.087963+0000","last_active":"2024-04-05T22:26:12.244339+0000","last_peered":"2024-04-05T22:26:12.244339+0000","last_clean":"2024-04-05T22:26:12.244339+0000","last_became_active":"2024-04-05T22:26:10.087426+0000","last_became_peered":"2024-04-05T22:26:10.087426+0000","last_unstale":"2024-04-05T22:26:12.244339+0000","last_undegraded":"2024-04-05T22:26:12.244339+0000","last_fullsized":"2024-04-05T22:26:12.244339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:28:20.626791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560335+0000","last_change":"2024-04-05T22:26:10.558000+0000","last_active":"2024-04-05T22:26:11.560335+0000","last_peered":"2024-04-05T22:26:11.560335+0000","last_clean":"2024-04-05T22:26:11.560335+0000","last_became_active":"2024-04-05T22:26:10.557638+0000","last_became_peered":"2024-04-05T22:26:10.557638+0000","last_unstale":"2024-04-05T22:26:11.560335+0000","last_undegraded":"2024-04-05T22:26:11.560335+0000","last_fullsized":"2024-04-05T22:26:11.560335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:34.229086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557380+0000","last_change":"2024-04-05T22:26:10.065593+0000","last_active":"2024-04-05T22:26:11.557380+0000","last_peered":"2024-04-05T22:26:11.557380+0000","last_clean":"2024-04-05T22:26:11.557380+0000","last_became_active":"2024-04-05T22:26:10.064772+0000","last_became_peered":"2024-04-05T22:26:10.064772+0000","last_unstale":"2024-04-05T22:26:11.557380+0000","last_undegraded":"2024-04-05T22:26:11.557380+0000","last_fullsized":"2024-04-05T22:26:11.557380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:33.216959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242920+0000","last_change":"2024-04-05T22:26:10.567810+0000","last_active":"2024-04-05T22:26:12.242920+0000","last_peered":"2024-04-05T22:26:12.242920+0000","last_clean":"2024-04-05T22:26:12.242920+0000","last_became_active":"2024-04-05T22:26:10.567518+0000","last_became_peered":"2024-04-05T22:26:10.567518+0000","last_unstale":"2024-04-05T22:26:12.242920+0000","last_undegraded":"2024-04-05T22:26:12.242920+0000","last_fullsized":"2024-04-05T22:26:12.242920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560182+0000","last_change":"2024-04-05T22:26:10.079847+0000","last_active":"2024-04-05T22:26:11.560182+0000","last_peered":"2024-04-05T22:26:11.560182+0000","last_clean":"2024-04-05T22:26:11.560182+0000","last_became_active":"2024-04-05T22:26:10.079573+0000","last_became_peered":"2024-04-05T22:26:10.079573+0000","last_unstale":"2024-04-05T22:26:11.560182+0000","last_undegraded":"2024-04-05T22:26:11.560182+0000","last_fullsized":"2024-04-05T22:26:11.560182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:26.349225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.559025+0000","last_change":"2024-04-05T22:26:10.577127+0000","last_active":"2024-04-05T22:26:11.559025+0000","last_peered":"2024-04-05T22:26:11.559025+0000","last_clean":"2024-04-05T22:26:11.559025+0000","last_became_active":"2024-04-05T22:26:10.576830+0000","last_became_peered":"2024-04-05T22:26:10.576830+0000","last_unstale":"2024-04-05T22:26:11.559025+0000","last_undegraded":"2024-04-05T22:26:11.559025+0000","last_fullsized":"2024-04-05T22:26:11.559025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:34:30.583583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246147+0000","last_change":"2024-04-05T22:26:10.082259+0000","last_active":"2024-04-05T22:26:12.246147+0000","last_peered":"2024-04-05T22:26:12.246147+0000","last_clean":"2024-04-05T22:26:12.246147+0000","last_became_active":"2024-04-05T22:26:10.079891+0000","last_became_peered":"2024-04-05T22:26:10.079891+0000","last_unstale":"2024-04-05T22:26:12.246147+0000","last_undegraded":"2024-04-05T22:26:12.246147+0000","last_fullsized":"2024-04-05T22:26:12.246147+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:37:32.101662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244115+0000","last_change":"2024-04-05T22:26:10.592599+0000","last_active":"2024-04-05T22:26:12.244115+0000","last_peered":"2024-04-05T22:26:12.244115+0000","last_clean":"2024-04-05T22:26:12.244115+0000","last_became_active":"2024-04-05T22:26:10.592300+0000","last_became_peered":"2024-04-05T22:26:10.592300+0000","last_unstale":"2024-04-05T22:26:12.244115+0000","last_undegraded":"2024-04-05T22:26:12.244115+0000","last_fullsized":"2024-04-05T22:26:12.244115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:59.523605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246362+0000","last_change":"2024-04-05T22:26:10.089251+0000","last_active":"2024-04-05T22:26:12.246362+0000","last_peered":"2024-04-05T22:26:12.246362+0000","last_clean":"2024-04-05T22:26:12.246362+0000","last_became_active":"2024-04-05T22:26:10.088797+0000","last_became_peered":"2024-04-05T22:26:10.088797+0000","last_unstale":"2024-04-05T22:26:12.246362+0000","last_undegraded":"2024-04-05T22:26:12.246362+0000","last_fullsized":"2024-04-05T22:26:12.246362+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:27.831999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244285+0000","last_change":"2024-04-05T22:26:10.567965+0000","last_active":"2024-04-05T22:26:12.244285+0000","last_peered":"2024-04-05T22:26:12.244285+0000","last_clean":"2024-04-05T22:26:12.244285+0000","last_became_active":"2024-04-05T22:26:10.567649+0000","last_became_peered":"2024-04-05T22:26:10.567649+0000","last_unstale":"2024-04-05T22:26:12.244285+0000","last_undegraded":"2024-04-05T22:26:12.244285+0000","last_fullsized":"2024-04-05T22:26:12.244285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:55:10.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242636+0000","last_change":"2024-04-05T22:26:10.088879+0000","last_active":"2024-04-05T22:26:12.242636+0000","last_peered":"2024-04-05T22:26:12.242636+0000","last_clean":"2024-04-05T22:26:12.242636+0000","last_became_active":"2024-04-05T22:26:10.088619+0000","last_became_peered":"2024-04-05T22:26:10.088619+0000","last_unstale":"2024-04-05T22:26:12.242636+0000","last_undegraded":"2024-04-05T22:26:12.242636+0000","last_fullsized":"2024-04-05T22:26:12.242636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:55:53.649114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560630+0000","last_change":"2024-04-05T22:26:10.560370+0000","last_active":"2024-04-05T22:26:11.560630+0000","last_peered":"2024-04-05T22:26:11.560630+0000","last_clean":"2024-04-05T22:26:11.560630+0000","last_became_active":"2024-04-05T22:26:10.560077+0000","last_became_peered":"2024-04-05T22:26:10.560077+0000","last_unstale":"2024-04-05T22:26:11.560630+0000","last_undegraded":"2024-04-05T22:26:11.560630+0000","last_fullsized":"2024-04-05T22:26:11.560630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:44:51.950325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558184+0000","last_change":"2024-04-05T22:26:10.065821+0000","last_active":"2024-04-05T22:26:11.558184+0000","last_peered":"2024-04-05T22:26:11.558184+0000","last_clean":"2024-04-05T22:26:11.558184+0000","last_became_active":"2024-04-05T22:26:10.065074+0000","last_became_peered":"2024-04-05T22:26:10.065074+0000","last_unstale":"2024-04-05T22:26:11.558184+0000","last_undegraded":"2024-04-05T22:26:11.558184+0000","last_fullsized":"2024-04-05T22:26:11.558184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:49:14.358062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241998+0000","last_change":"2024-04-05T22:26:10.576694+0000","last_active":"2024-04-05T22:26:12.241998+0000","last_peered":"2024-04-05T22:26:12.241998+0000","last_clean":"2024-04-05T22:26:12.241998+0000","last_became_active":"2024-04-05T22:26:10.576510+0000","last_became_peered":"2024-04-05T22:26:10.576510+0000","last_unstale":"2024-04-05T22:26:12.241998+0000","last_undegraded":"2024-04-05T22:26:12.241998+0000","last_fullsized":"2024-04-05T22:26:12.241998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:24:51.667812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557940+0000","last_change":"2024-04-05T22:26:10.079530+0000","last_active":"2024-04-05T22:26:11.557940+0000","last_peered":"2024-04-05T22:26:11.557940+0000","last_clean":"2024-04-05T22:26:11.557940+0000","last_became_active":"2024-04-05T22:26:10.079156+0000","last_became_peered":"2024-04-05T22:26:10.079156+0000","last_unstale":"2024-04-05T22:26:11.557940+0000","last_undegraded":"2024-04-05T22:26:11.557940+0000","last_fullsized":"2024-04-05T22:26:11.557940+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:45.937444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558454+0000","last_change":"2024-04-05T22:26:10.583349+0000","last_active":"2024-04-05T22:26:11.558454+0000","last_peered":"2024-04-05T22:26:11.558454+0000","last_clean":"2024-04-05T22:26:11.558454+0000","last_became_active":"2024-04-05T22:26:10.583084+0000","last_became_peered":"2024-04-05T22:26:10.583084+0000","last_unstale":"2024-04-05T22:26:11.558454+0000","last_undegraded":"2024-04-05T22:26:11.558454+0000","last_fullsized":"2024-04-05T22:26:11.558454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:12.493196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244570+0000","last_change":"2024-04-05T22:26:10.089479+0000","last_active":"2024-04-05T22:26:12.244570+0000","last_peered":"2024-04-05T22:26:12.244570+0000","last_clean":"2024-04-05T22:26:12.244570+0000","last_became_active":"2024-04-05T22:26:10.089154+0000","last_became_peered":"2024-04-05T22:26:10.089154+0000","last_unstale":"2024-04-05T22:26:12.244570+0000","last_undegraded":"2024-04-05T22:26:12.244570+0000","last_fullsized":"2024-04-05T22:26:12.244570+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:58:14.277312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241973+0000","last_change":"2024-04-05T22:26:10.572234+0000","last_active":"2024-04-05T22:26:12.241973+0000","last_peered":"2024-04-05T22:26:12.241973+0000","last_clean":"2024-04-05T22:26:12.241973+0000","last_became_active":"2024-04-05T22:26:10.570149+0000","last_became_peered":"2024-04-05T22:26:10.570149+0000","last_unstale":"2024-04-05T22:26:12.241973+0000","last_undegraded":"2024-04-05T22:26:12.241973+0000","last_fullsized":"2024-04-05T22:26:12.241973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558469+0000","last_change":"2024-04-05T22:26:10.086162+0000","last_active":"2024-04-05T22:26:11.558469+0000","last_peered":"2024-04-05T22:26:11.558469+0000","last_clean":"2024-04-05T22:26:11.558469+0000","last_became_active":"2024-04-05T22:26:10.085908+0000","last_became_peered":"2024-04-05T22:26:10.085908+0000","last_unstale":"2024-04-05T22:26:11.558469+0000","last_undegraded":"2024-04-05T22:26:11.558469+0000","last_fullsized":"2024-04-05T22:26:11.558469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:28:41.447345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560720+0000","last_change":"2024-04-05T22:26:10.573501+0000","last_active":"2024-04-05T22:26:11.560720+0000","last_peered":"2024-04-05T22:26:11.560720+0000","last_clean":"2024-04-05T22:26:11.560720+0000","last_became_active":"2024-04-05T22:26:10.573207+0000","last_became_peered":"2024-04-05T22:26:10.573207+0000","last_unstale":"2024-04-05T22:26:11.560720+0000","last_undegraded":"2024-04-05T22:26:11.560720+0000","last_fullsized":"2024-04-05T22:26:11.560720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:25:01.784393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243150+0000","last_change":"2024-04-05T22:26:10.081963+0000","last_active":"2024-04-05T22:26:12.243150+0000","last_peered":"2024-04-05T22:26:12.243150+0000","last_clean":"2024-04-05T22:26:12.243150+0000","last_became_active":"2024-04-05T22:26:10.080820+0000","last_became_peered":"2024-04-05T22:26:10.080820+0000","last_unstale":"2024-04-05T22:26:12.243150+0000","last_undegraded":"2024-04-05T22:26:12.243150+0000","last_fullsized":"2024-04-05T22:26:12.243150+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:06.379391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245879+0000","last_change":"2024-04-05T22:26:10.562895+0000","last_active":"2024-04-05T22:26:12.245879+0000","last_peered":"2024-04-05T22:26:12.245879+0000","last_clean":"2024-04-05T22:26:12.245879+0000","last_became_active":"2024-04-05T22:26:10.562539+0000","last_became_peered":"2024-04-05T22:26:10.562539+0000","last_unstale":"2024-04-05T22:26:12.245879+0000","last_undegraded":"2024-04-05T22:26:12.245879+0000","last_fullsized":"2024-04-05T22:26:12.245879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:51.195268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244692+0000","last_change":"2024-04-05T22:26:10.088325+0000","last_active":"2024-04-05T22:26:12.244692+0000","last_peered":"2024-04-05T22:26:12.244692+0000","last_clean":"2024-04-05T22:26:12.244692+0000","last_became_active":"2024-04-05T22:26:10.087891+0000","last_became_peered":"2024-04-05T22:26:10.087891+0000","last_unstale":"2024-04-05T22:26:12.244692+0000","last_undegraded":"2024-04-05T22:26:12.244692+0000","last_fullsized":"2024-04-05T22:26:12.244692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:37.248834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244355+0000","last_change":"2024-04-05T22:26:10.588302+0000","last_active":"2024-04-05T22:26:12.244355+0000","last_peered":"2024-04-05T22:26:12.244355+0000","last_clean":"2024-04-05T22:26:12.244355+0000","last_became_active":"2024-04-05T22:26:10.588062+0000","last_became_peered":"2024-04-05T22:26:10.588062+0000","last_unstale":"2024-04-05T22:26:12.244355+0000","last_undegraded":"2024-04-05T22:26:12.244355+0000","last_fullsized":"2024-04-05T22:26:12.244355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:19.667713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245864+0000","last_change":"2024-04-05T22:26:10.086292+0000","last_active":"2024-04-05T22:26:12.245864+0000","last_peered":"2024-04-05T22:26:12.245864+0000","last_clean":"2024-04-05T22:26:12.245864+0000","last_became_active":"2024-04-05T22:26:10.085911+0000","last_became_peered":"2024-04-05T22:26:10.085911+0000","last_unstale":"2024-04-05T22:26:12.245864+0000","last_undegraded":"2024-04-05T22:26:12.245864+0000","last_fullsized":"2024-04-05T22:26:12.245864+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:13.129030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558085+0000","last_change":"2024-04-05T22:26:10.556562+0000","last_active":"2024-04-05T22:26:11.558085+0000","last_peered":"2024-04-05T22:26:11.558085+0000","last_clean":"2024-04-05T22:26:11.558085+0000","last_became_active":"2024-04-05T22:26:10.555745+0000","last_became_peered":"2024-04-05T22:26:10.555745+0000","last_unstale":"2024-04-05T22:26:11.558085+0000","last_undegraded":"2024-04-05T22:26:11.558085+0000","last_fullsized":"2024-04-05T22:26:11.558085+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:13.529355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245433+0000","last_change":"2024-04-05T22:26:10.092146+0000","last_active":"2024-04-05T22:26:12.245433+0000","last_peered":"2024-04-05T22:26:12.245433+0000","last_clean":"2024-04-05T22:26:12.245433+0000","last_became_active":"2024-04-05T22:26:10.091743+0000","last_became_peered":"2024-04-05T22:26:10.091743+0000","last_unstale":"2024-04-05T22:26:12.245433+0000","last_undegraded":"2024-04-05T22:26:12.245433+0000","last_fullsized":"2024-04-05T22:26:12.245433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:51.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560876+0000","last_change":"2024-04-05T22:26:10.557716+0000","last_active":"2024-04-05T22:26:11.560876+0000","last_peered":"2024-04-05T22:26:11.560876+0000","last_clean":"2024-04-05T22:26:11.560876+0000","last_became_active":"2024-04-05T22:26:10.557338+0000","last_became_peered":"2024-04-05T22:26:10.557338+0000","last_unstale":"2024-04-05T22:26:11.560876+0000","last_undegraded":"2024-04-05T22:26:11.560876+0000","last_fullsized":"2024-04-05T22:26:11.560876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:01.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243113+0000","last_change":"2024-04-05T22:26:10.082040+0000","last_active":"2024-04-05T22:26:12.243113+0000","last_peered":"2024-04-05T22:26:12.243113+0000","last_clean":"2024-04-05T22:26:12.243113+0000","last_became_active":"2024-04-05T22:26:10.081668+0000","last_became_peered":"2024-04-05T22:26:10.081668+0000","last_unstale":"2024-04-05T22:26:12.243113+0000","last_undegraded":"2024-04-05T22:26:12.243113+0000","last_fullsized":"2024-04-05T22:26:12.243113+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:35.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242310+0000","last_change":"2024-04-05T22:26:10.571263+0000","last_active":"2024-04-05T22:26:12.242310+0000","last_peered":"2024-04-05T22:26:12.242310+0000","last_clean":"2024-04-05T22:26:12.242310+0000","last_became_active":"2024-04-05T22:26:10.570857+0000","last_became_peered":"2024-04-05T22:26:10.570857+0000","last_unstale":"2024-04-05T22:26:12.242310+0000","last_undegraded":"2024-04-05T22:26:12.242310+0000","last_fullsized":"2024-04-05T22:26:12.242310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:49.772016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244813+0000","last_change":"2024-04-05T22:26:10.085450+0000","last_active":"2024-04-05T22:26:12.244813+0000","last_peered":"2024-04-05T22:26:12.244813+0000","last_clean":"2024-04-05T22:26:12.244813+0000","last_became_active":"2024-04-05T22:26:10.084927+0000","last_became_peered":"2024-04-05T22:26:10.084927+0000","last_unstale":"2024-04-05T22:26:12.244813+0000","last_undegraded":"2024-04-05T22:26:12.244813+0000","last_fullsized":"2024-04-05T22:26:12.244813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:28:29.386651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241323+0000","last_change":"2024-04-05T22:26:10.589210+0000","last_active":"2024-04-05T22:26:12.241323+0000","last_peered":"2024-04-05T22:26:12.241323+0000","last_clean":"2024-04-05T22:26:12.241323+0000","last_became_active":"2024-04-05T22:26:10.588942+0000","last_became_peered":"2024-04-05T22:26:10.588942+0000","last_unstale":"2024-04-05T22:26:12.241323+0000","last_undegraded":"2024-04-05T22:26:12.241323+0000","last_fullsized":"2024-04-05T22:26:12.241323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:41.846577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558644+0000","last_change":"2024-04-05T22:26:10.089430+0000","last_active":"2024-04-05T22:26:11.558644+0000","last_peered":"2024-04-05T22:26:11.558644+0000","last_clean":"2024-04-05T22:26:11.558644+0000","last_became_active":"2024-04-05T22:26:10.089068+0000","last_became_peered":"2024-04-05T22:26:10.089068+0000","last_unstale":"2024-04-05T22:26:11.558644+0000","last_undegraded":"2024-04-05T22:26:11.558644+0000","last_fullsized":"2024-04-05T22:26:11.558644+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:06:43.278506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241619+0000","last_change":"2024-04-05T22:26:10.589877+0000","last_active":"2024-04-05T22:26:12.241619+0000","last_peered":"2024-04-05T22:26:12.241619+0000","last_clean":"2024-04-05T22:26:12.241619+0000","last_became_active":"2024-04-05T22:26:10.589551+0000","last_became_peered":"2024-04-05T22:26:10.589551+0000","last_unstale":"2024-04-05T22:26:12.241619+0000","last_undegraded":"2024-04-05T22:26:12.241619+0000","last_fullsized":"2024-04-05T22:26:12.241619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:29:54.545828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241708+0000","last_change":"2024-04-05T22:26:10.087563+0000","last_active":"2024-04-05T22:26:12.241708+0000","last_peered":"2024-04-05T22:26:12.241708+0000","last_clean":"2024-04-05T22:26:12.241708+0000","last_became_active":"2024-04-05T22:26:10.087242+0000","last_became_peered":"2024-04-05T22:26:10.087242+0000","last_unstale":"2024-04-05T22:26:12.241708+0000","last_undegraded":"2024-04-05T22:26:12.241708+0000","last_fullsized":"2024-04-05T22:26:12.241708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:18.655586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243221+0000","last_change":"2024-04-05T22:26:10.565840+0000","last_active":"2024-04-05T22:26:12.243221+0000","last_peered":"2024-04-05T22:26:12.243221+0000","last_clean":"2024-04-05T22:26:12.243221+0000","last_became_active":"2024-04-05T22:26:10.565553+0000","last_became_peered":"2024-04-05T22:26:10.565553+0000","last_unstale":"2024-04-05T22:26:12.243221+0000","last_undegraded":"2024-04-05T22:26:12.243221+0000","last_fullsized":"2024-04-05T22:26:12.243221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:32:05.479366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244897+0000","last_change":"2024-04-05T22:26:10.088150+0000","last_active":"2024-04-05T22:26:12.244897+0000","last_peered":"2024-04-05T22:26:12.244897+0000","last_clean":"2024-04-05T22:26:12.244897+0000","last_became_active":"2024-04-05T22:26:10.087504+0000","last_became_peered":"2024-04-05T22:26:10.087504+0000","last_unstale":"2024-04-05T22:26:12.244897+0000","last_undegraded":"2024-04-05T22:26:12.244897+0000","last_fullsized":"2024-04-05T22:26:12.244897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:22.356009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558854+0000","last_change":"2024-04-05T22:26:10.582030+0000","last_active":"2024-04-05T22:26:11.558854+0000","last_peered":"2024-04-05T22:26:11.558854+0000","last_clean":"2024-04-05T22:26:11.558854+0000","last_became_active":"2024-04-05T22:26:10.580582+0000","last_became_peered":"2024-04-05T22:26:10.580582+0000","last_unstale":"2024-04-05T22:26:11.558854+0000","last_undegraded":"2024-04-05T22:26:11.558854+0000","last_fullsized":"2024-04-05T22:26:11.558854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.559036+0000","last_change":"2024-04-05T22:26:10.091683+0000","last_active":"2024-04-05T22:26:11.559036+0000","last_peered":"2024-04-05T22:26:11.559036+0000","last_clean":"2024-04-05T22:26:11.559036+0000","last_became_active":"2024-04-05T22:26:10.091417+0000","last_became_peered":"2024-04-05T22:26:10.091417+0000","last_unstale":"2024-04-05T22:26:11.559036+0000","last_undegraded":"2024-04-05T22:26:11.559036+0000","last_fullsized":"2024-04-05T22:26:11.559036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:00.451532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242905+0000","last_change":"2024-04-05T22:26:10.592440+0000","last_active":"2024-04-05T22:26:12.242905+0000","last_peered":"2024-04-05T22:26:12.242905+0000","last_clean":"2024-04-05T22:26:12.242905+0000","last_became_active":"2024-04-05T22:26:10.592173+0000","last_became_peered":"2024-04-05T22:26:10.592173+0000","last_unstale":"2024-04-05T22:26:12.242905+0000","last_undegraded":"2024-04-05T22:26:12.242905+0000","last_fullsized":"2024-04-05T22:26:12.242905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:16:17.822694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560972+0000","last_change":"2024-04-05T22:26:10.083690+0000","last_active":"2024-04-05T22:26:11.560972+0000","last_peered":"2024-04-05T22:26:11.560972+0000","last_clean":"2024-04-05T22:26:11.560972+0000","last_became_active":"2024-04-05T22:26:10.083397+0000","last_became_peered":"2024-04-05T22:26:10.083397+0000","last_unstale":"2024-04-05T22:26:11.560972+0000","last_undegraded":"2024-04-05T22:26:11.560972+0000","last_fullsized":"2024-04-05T22:26:11.560972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:42:04.187427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244876+0000","last_change":"2024-04-05T22:26:10.592052+0000","last_active":"2024-04-05T22:26:12.244876+0000","last_peered":"2024-04-05T22:26:12.244876+0000","last_clean":"2024-04-05T22:26:12.244876+0000","last_became_active":"2024-04-05T22:26:10.591524+0000","last_became_peered":"2024-04-05T22:26:10.591524+0000","last_unstale":"2024-04-05T22:26:12.244876+0000","last_undegraded":"2024-04-05T22:26:12.244876+0000","last_fullsized":"2024-04-05T22:26:12.244876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:06:50.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243106+0000","last_change":"2024-04-05T22:26:10.083510+0000","last_active":"2024-04-05T22:26:12.243106+0000","last_peered":"2024-04-05T22:26:12.243106+0000","last_clean":"2024-04-05T22:26:12.243106+0000","last_became_active":"2024-04-05T22:26:10.083158+0000","last_became_peered":"2024-04-05T22:26:10.083158+0000","last_unstale":"2024-04-05T22:26:12.243106+0000","last_undegraded":"2024-04-05T22:26:12.243106+0000","last_fullsized":"2024-04-05T22:26:12.243106+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:41.926822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560932+0000","last_change":"2024-04-05T22:26:10.571917+0000","last_active":"2024-04-05T22:26:11.560932+0000","last_peered":"2024-04-05T22:26:11.560932+0000","last_clean":"2024-04-05T22:26:11.560932+0000","last_became_active":"2024-04-05T22:26:10.571673+0000","last_became_peered":"2024-04-05T22:26:10.571673+0000","last_unstale":"2024-04-05T22:26:11.560932+0000","last_undegraded":"2024-04-05T22:26:11.560932+0000","last_fullsized":"2024-04-05T22:26:11.560932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:40:59.772321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.561049+0000","last_change":"2024-04-05T22:26:10.091113+0000","last_active":"2024-04-05T22:26:11.561049+0000","last_peered":"2024-04-05T22:26:11.561049+0000","last_clean":"2024-04-05T22:26:11.561049+0000","last_became_active":"2024-04-05T22:26:10.089154+0000","last_became_peered":"2024-04-05T22:26:10.089154+0000","last_unstale":"2024-04-05T22:26:11.561049+0000","last_undegraded":"2024-04-05T22:26:11.561049+0000","last_fullsized":"2024-04-05T22:26:11.561049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:41:20.095091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245445+0000","last_change":"2024-04-05T22:26:10.578205+0000","last_active":"2024-04-05T22:26:12.245445+0000","last_peered":"2024-04-05T22:26:12.245445+0000","last_clean":"2024-04-05T22:26:12.245445+0000","last_became_active":"2024-04-05T22:26:10.577883+0000","last_became_peered":"2024-04-05T22:26:10.577883+0000","last_unstale":"2024-04-05T22:26:12.245445+0000","last_undegraded":"2024-04-05T22:26:12.245445+0000","last_fullsized":"2024-04-05T22:26:12.245445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:48.025720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558235+0000","last_change":"2024-04-05T22:26:10.065941+0000","last_active":"2024-04-05T22:26:11.558235+0000","last_peered":"2024-04-05T22:26:11.558235+0000","last_clean":"2024-04-05T22:26:11.558235+0000","last_became_active":"2024-04-05T22:26:10.065253+0000","last_became_peered":"2024-04-05T22:26:10.065253+0000","last_unstale":"2024-04-05T22:26:11.558235+0000","last_undegraded":"2024-04-05T22:26:11.558235+0000","last_fullsized":"2024-04-05T22:26:11.558235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:44:50.938197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557435+0000","last_change":"2024-04-05T22:26:10.573880+0000","last_active":"2024-04-05T22:26:11.557435+0000","last_peered":"2024-04-05T22:26:11.557435+0000","last_clean":"2024-04-05T22:26:11.557435+0000","last_became_active":"2024-04-05T22:26:10.572917+0000","last_became_peered":"2024-04-05T22:26:10.572917+0000","last_unstale":"2024-04-05T22:26:11.557435+0000","last_undegraded":"2024-04-05T22:26:11.557435+0000","last_fullsized":"2024-04-05T22:26:11.557435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:53.929778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243140+0000","last_change":"2024-04-05T22:26:10.080919+0000","last_active":"2024-04-05T22:26:12.243140+0000","last_peered":"2024-04-05T22:26:12.243140+0000","last_clean":"2024-04-05T22:26:12.243140+0000","last_became_active":"2024-04-05T22:26:10.079625+0000","last_became_peered":"2024-04-05T22:26:10.079625+0000","last_unstale":"2024-04-05T22:26:12.243140+0000","last_undegraded":"2024-04-05T22:26:12.243140+0000","last_fullsized":"2024-04-05T22:26:12.243140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:55.592962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244950+0000","last_change":"2024-04-05T22:26:10.578503+0000","last_active":"2024-04-05T22:26:12.244950+0000","last_peered":"2024-04-05T22:26:12.244950+0000","last_clean":"2024-04-05T22:26:12.244950+0000","last_became_active":"2024-04-05T22:26:10.578174+0000","last_became_peered":"2024-04-05T22:26:10.578174+0000","last_unstale":"2024-04-05T22:26:12.244950+0000","last_undegraded":"2024-04-05T22:26:12.244950+0000","last_fullsized":"2024-04-05T22:26:12.244950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:09.578237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244654+0000","last_change":"2024-04-05T22:26:10.083814+0000","last_active":"2024-04-05T22:26:12.244654+0000","last_peered":"2024-04-05T22:26:12.244654+0000","last_clean":"2024-04-05T22:26:12.244654+0000","last_became_active":"2024-04-05T22:26:10.082357+0000","last_became_peered":"2024-04-05T22:26:10.082357+0000","last_unstale":"2024-04-05T22:26:12.244654+0000","last_undegraded":"2024-04-05T22:26:12.244654+0000","last_fullsized":"2024-04-05T22:26:12.244654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:39.001670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241874+0000","last_change":"2024-04-05T22:26:10.591072+0000","last_active":"2024-04-05T22:26:12.241874+0000","last_peered":"2024-04-05T22:26:12.241874+0000","last_clean":"2024-04-05T22:26:12.241874+0000","last_became_active":"2024-04-05T22:26:10.590676+0000","last_became_peered":"2024-04-05T22:26:10.590676+0000","last_unstale":"2024-04-05T22:26:12.241874+0000","last_undegraded":"2024-04-05T22:26:12.241874+0000","last_fullsized":"2024-04-05T22:26:12.241874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:07.646105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245661+0000","last_change":"2024-04-05T22:26:10.090017+0000","last_active":"2024-04-05T22:26:12.245661+0000","last_peered":"2024-04-05T22:26:12.245661+0000","last_clean":"2024-04-05T22:26:12.245661+0000","last_became_active":"2024-04-05T22:26:10.089687+0000","last_became_peered":"2024-04-05T22:26:10.089687+0000","last_unstale":"2024-04-05T22:26:12.245661+0000","last_undegraded":"2024-04-05T22:26:12.245661+0000","last_fullsized":"2024-04-05T22:26:12.245661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:56:34.420409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-05T22:26:10.567576+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:26:10.567576+0000","last_peered":"2024-04-05T22:26:10.567576+0000","last_clean":"2024-04-05T22:26:10.567576+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:26:10.567576+0000","last_undegraded":"2024-04-05T22:26:10.567576+0000","last_fullsized":"2024-04-05T22:26:10.567576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":968686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.345}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.214}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.284},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.999}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.3660000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.7350000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.615},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5620000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.877},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.659}]}]},{"osd":6,"up_from":14,"seq":60129542209,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":905216,"data_stored":377794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":1.1699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.01}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.226}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.073},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.6120000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.387},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.272}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.538},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.494}]}]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":901120,"data_stored":377792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.50900000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":901120,"data_stored":377772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0509999999999999,"15min":0.999},"min":{"1min":0.84699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":1.673,"15min":1.673},"last":0.877},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0980000000000001,"15min":1.0800000000000001},"min":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.6459999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1439999999999999,"15min":1.1299999999999999},"min":{"1min":0.875,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.607},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1539999999999999,"15min":1.1699999999999999},"min":{"1min":0.83399999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4450000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.341}]},{"osd":2,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.95099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.482,"5min":1.974,"15min":1.974},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.89000000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.379},"min":{"1min":0.95999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.843,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.45},{"interface":"front","average":{"1min":1.385,"5min":1.369,"15min":1.407},"min":{"1min":1.107,"5min":0.874,"15min":0.874},"max":{"1min":1.754,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.111}]},{"osd":5,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.492},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3440000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.625,"15min":0.625},"max":{"1min":1.786,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.77}]},{"osd":6,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4610000000000001},"min":{"1min":0.95599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6379999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4410000000000001,"15min":1.4950000000000001},"min":{"1min":1.0149999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.655,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.734}]},{"osd":7,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5029999999999999,"15min":1.536},"min":{"1min":1.1639999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.895,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.8400000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5549999999999999,"15min":1.5760000000000001},"min":{"1min":1.05,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":2.54,"15min":2.54},"last":1.964}]}]},{"osd":3,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":892928,"data_stored":371514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.395}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.157}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.093},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.048},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2669999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.175}]}]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":909312,"data_stored":379266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.115}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.161}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.2549999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":917504,"data_stored":379882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.9119999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.258}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.167},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.216}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.5489999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":2.1120000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.302}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":2.0259999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.607}]}]},{"osd":0,"up_from":14,"seq":60129542209,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1478656,"data_stored":961770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.747},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.004}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.254},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.462}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.288},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.127}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.516}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.339}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:26:14.996 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:26:15.027 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-04-05T22:26:15.027+0000 7f041f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:26:15.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.237+0000 7f2954c00640 1 Processor -- start 2024-04-05T22:26:15.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.237+0000 7f2954c00640 1 -- start start 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f2954c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2950162370 0x7f29500b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f2954c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29500b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f2954c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29501627f0 con 0x7f2950161f90 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f2954c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2950158790 con 0x7f2950162370 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f2954c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b3550 con 0x7f29500b2ba0 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2950162370 0x7f29500b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2950162370 0x7f29500b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49876/0 (socket says 172.21.15.73:49876) 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 -- 172.21.15.73:0/3135968356 learned_addr learned my addr 172.21.15.73:0/3135968356 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:15.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 -- 172.21.15.73:0/3135968356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 msgr2=0x7f29500b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:15.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 --2- 172.21.15.73:0/3135968356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29500b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 -- 172.21.15.73:0/3135968356 >> v1:172.21.15.157:6789/0 conn(0x7f2950161f90 legacy=0x7f2950164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:15.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.239+0000 7f294d400640 1 -- 172.21.15.73:0/3135968356 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950165cf0 con 0x7f2950162370 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.241+0000 7f294d400640 1 --2- 172.21.15.73:0/3135968356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500b2110 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f293c0098a0 tx=0x7f293c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1834b08cf5938a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.241+0000 7f294ca00640 1 -- 172.21.15.73:0/3135968356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c031640 con 0x7f2950162370 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.241+0000 7f294ca00640 1 -- 172.21.15.73:0/3135968356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c0317a0 con 0x7f2950162370 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.241+0000 7f294ca00640 1 -- 172.21.15.73:0/3135968356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c031b30 con 0x7f2950162370 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 -- 172.21.15.73:0/3135968356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 msgr2=0x7f29500b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 --2- 172.21.15.73:0/3135968356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500b2110 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f293c0098a0 tx=0x7f293c02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 -- 172.21.15.73:0/3135968356 shutdown_connections 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 --2- 172.21.15.73:0/3135968356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29500b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 --2- 172.21.15.73:0/3135968356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500b2110 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 -- 172.21.15.73:0/3135968356 >> 172.21.15.73:0/3135968356 conn(0x7f29500ce8e0 msgr2=0x7f29500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 -- 172.21.15.73:0/3135968356 shutdown_connections 2024-04-05T22:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.242+0000 7f2954c00640 1 -- 172.21.15.73:0/3135968356 wait complete. 2024-04-05T22:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.244+0000 7f2954c00640 1 Processor -- start 2024-04-05T22:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.244+0000 7f2954c00640 1 -- start start 2024-04-05T22:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29501f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2950161f90 0x7f29500c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2950148600 con 0x7f2950162370 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2950155ed0 con 0x7f2950161f90 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f2954c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2950155bf0 con 0x7f29500b2ba0 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.245+0000 7f294e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49878/0 (socket says 172.21.15.73:49878) 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 -- 172.21.15.73:0/85470479 learned_addr learned my addr 172.21.15.73:0/85470479 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294d400640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2950161f90 0x7f29500c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294de00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29501f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 -- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 msgr2=0x7f29501f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29501f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 -- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2950161f90 msgr2=0x7f29500c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2950161f90 0x7f29500c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 -- 172.21.15.73:0/85470479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094d0 con 0x7f2950162370 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294de00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29501f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.246+0000 7f294e800640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f294800b970 tx=0x7f294800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.247+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800cb40 con 0x7f2950162370 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.247+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800ccc0 con 0x7f2950162370 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.247+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948014680 con 0x7f2950162370 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.247+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500c4350 con 0x7f2950162370 2024-04-05T22:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.247+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29500c4910 con 0x7f2950162370 2024-04-05T22:26:15.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.248+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2950066af0 con 0x7f2950162370 2024-04-05T22:26:15.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.249+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f294801d3b0 con 0x7f2950162370 2024-04-05T22:26:15.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.250+0000 7f2946000640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.250+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2948089700 con 0x7f2950162370 2024-04-05T22:26:15.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.250+0000 7f294de00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.251+0000 7f294de00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 0x7f293006d420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2940007fd0 tx=0x7f2940009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.254+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f29480545e0 con 0x7f2950162370 2024-04-05T22:26:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.443+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29500b1910 con 0x7f293006af60 2024-04-05T22:26:15.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.465+0000 7f2946000640 1 -- 172.21.15.73:0/85470479 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286807 (secure 0 0 0) 0x7f2940001370 con 0x7f293006af60 2024-04-05T22:26:15.468 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 msgr2=0x7f293006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 0x7f293006d420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2940007fd0 tx=0x7f2940009040 comp rx=0 tx=0).stop 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 msgr2=0x7f29500c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f294800b970 tx=0x7f294800be30 comp rx=0 tx=0).stop 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 shutdown_connections 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950162370 0x7f29500c70c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2950161f90 0x7f29500c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 --2- 172.21.15.73:0/85470479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b2ba0 0x7f29501f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 >> 172.21.15.73:0/85470479 conn(0x7f29500ce8e0 msgr2=0x7f295006ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:15.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 shutdown_connections 2024-04-05T22:26:15.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.471+0000 7f2954c00640 1 -- 172.21.15.73:0/85470479 wait complete. 2024-04-05T22:26:15.488 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-05T22:26:14.590696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257184,"kb_used_data":8208,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754717536,"statfs":{"total":773094113280,"available":772830756864,"internally_reserved":0,"allocated":8404992,"data_stored":4194476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.548632"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243873+0000","last_change":"2024-04-05T22:26:10.584470+0000","last_active":"2024-04-05T22:26:12.243873+0000","last_peered":"2024-04-05T22:26:12.243873+0000","last_clean":"2024-04-05T22:26:12.243873+0000","last_became_active":"2024-04-05T22:26:10.583641+0000","last_became_peered":"2024-04-05T22:26:10.583641+0000","last_unstale":"2024-04-05T22:26:12.243873+0000","last_undegraded":"2024-04-05T22:26:12.243873+0000","last_fullsized":"2024-04-05T22:26:12.243873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:22.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246148+0000","last_change":"2024-04-05T22:26:10.589009+0000","last_active":"2024-04-05T22:26:12.246148+0000","last_peered":"2024-04-05T22:26:12.246148+0000","last_clean":"2024-04-05T22:26:12.246148+0000","last_became_active":"2024-04-05T22:26:10.588729+0000","last_became_peered":"2024-04-05T22:26:10.588729+0000","last_unstale":"2024-04-05T22:26:12.246148+0000","last_undegraded":"2024-04-05T22:26:12.246148+0000","last_fullsized":"2024-04-05T22:26:12.246148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:53:52.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242155+0000","last_change":"2024-04-05T22:26:10.569517+0000","last_active":"2024-04-05T22:26:12.242155+0000","last_peered":"2024-04-05T22:26:12.242155+0000","last_clean":"2024-04-05T22:26:12.242155+0000","last_became_active":"2024-04-05T22:26:10.568811+0000","last_became_peered":"2024-04-05T22:26:10.568811+0000","last_unstale":"2024-04-05T22:26:12.242155+0000","last_undegraded":"2024-04-05T22:26:12.242155+0000","last_fullsized":"2024-04-05T22:26:12.242155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:12:34.064555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557883+0000","last_change":"2024-04-05T22:26:10.578648+0000","last_active":"2024-04-05T22:26:11.557883+0000","last_peered":"2024-04-05T22:26:11.557883+0000","last_clean":"2024-04-05T22:26:11.557883+0000","last_became_active":"2024-04-05T22:26:10.578342+0000","last_became_peered":"2024-04-05T22:26:10.578342+0000","last_unstale":"2024-04-05T22:26:11.557883+0000","last_undegraded":"2024-04-05T22:26:11.557883+0000","last_fullsized":"2024-04-05T22:26:11.557883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:18:37.871348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560829+0000","last_change":"2024-04-05T22:26:10.567910+0000","last_active":"2024-04-05T22:26:11.560829+0000","last_peered":"2024-04-05T22:26:11.560829+0000","last_clean":"2024-04-05T22:26:11.560829+0000","last_became_active":"2024-04-05T22:26:10.567621+0000","last_became_peered":"2024-04-05T22:26:10.567621+0000","last_unstale":"2024-04-05T22:26:11.560829+0000","last_undegraded":"2024-04-05T22:26:11.560829+0000","last_fullsized":"2024-04-05T22:26:11.560829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:11:26.707227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244079+0000","last_change":"2024-04-05T22:26:10.565467+0000","last_active":"2024-04-05T22:26:12.244079+0000","last_peered":"2024-04-05T22:26:12.244079+0000","last_clean":"2024-04-05T22:26:12.244079+0000","last_became_active":"2024-04-05T22:26:10.565069+0000","last_became_peered":"2024-04-05T22:26:10.565069+0000","last_unstale":"2024-04-05T22:26:12.244079+0000","last_undegraded":"2024-04-05T22:26:12.244079+0000","last_fullsized":"2024-04-05T22:26:12.244079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:48.544527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558399+0000","last_change":"2024-04-05T22:26:10.581377+0000","last_active":"2024-04-05T22:26:11.558399+0000","last_peered":"2024-04-05T22:26:11.558399+0000","last_clean":"2024-04-05T22:26:11.558399+0000","last_became_active":"2024-04-05T22:26:10.579510+0000","last_became_peered":"2024-04-05T22:26:10.579510+0000","last_unstale":"2024-04-05T22:26:11.558399+0000","last_undegraded":"2024-04-05T22:26:11.558399+0000","last_fullsized":"2024-04-05T22:26:11.558399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244404+0000","last_change":"2024-04-05T22:26:10.587800+0000","last_active":"2024-04-05T22:26:12.244404+0000","last_peered":"2024-04-05T22:26:12.244404+0000","last_clean":"2024-04-05T22:26:12.244404+0000","last_became_active":"2024-04-05T22:26:10.587311+0000","last_became_peered":"2024-04-05T22:26:10.587311+0000","last_unstale":"2024-04-05T22:26:12.244404+0000","last_undegraded":"2024-04-05T22:26:12.244404+0000","last_fullsized":"2024-04-05T22:26:12.244404+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:43.465833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560725+0000","last_change":"2024-04-05T22:26:10.575075+0000","last_active":"2024-04-05T22:26:11.560725+0000","last_peered":"2024-04-05T22:26:11.560725+0000","last_clean":"2024-04-05T22:26:11.560725+0000","last_became_active":"2024-04-05T22:26:10.574500+0000","last_became_peered":"2024-04-05T22:26:10.574500+0000","last_unstale":"2024-04-05T22:26:11.560725+0000","last_undegraded":"2024-04-05T22:26:11.560725+0000","last_fullsized":"2024-04-05T22:26:11.560725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:07.391519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245474+0000","last_change":"2024-04-05T22:26:10.580244+0000","last_active":"2024-04-05T22:26:12.245474+0000","last_peered":"2024-04-05T22:26:12.245474+0000","last_clean":"2024-04-05T22:26:12.245474+0000","last_became_active":"2024-04-05T22:26:10.579954+0000","last_became_peered":"2024-04-05T22:26:10.579954+0000","last_unstale":"2024-04-05T22:26:12.245474+0000","last_undegraded":"2024-04-05T22:26:12.245474+0000","last_fullsized":"2024-04-05T22:26:12.245474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:44:33.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242725+0000","last_change":"2024-04-05T22:26:10.587090+0000","last_active":"2024-04-05T22:26:12.242725+0000","last_peered":"2024-04-05T22:26:12.242725+0000","last_clean":"2024-04-05T22:26:12.242725+0000","last_became_active":"2024-04-05T22:26:10.586794+0000","last_became_peered":"2024-04-05T22:26:10.586794+0000","last_unstale":"2024-04-05T22:26:12.242725+0000","last_undegraded":"2024-04-05T22:26:12.242725+0000","last_fullsized":"2024-04-05T22:26:12.242725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:34:30.583583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557985+0000","last_change":"2024-04-05T22:26:10.556444+0000","last_active":"2024-04-05T22:26:11.557985+0000","last_peered":"2024-04-05T22:26:11.557985+0000","last_clean":"2024-04-05T22:26:11.557985+0000","last_became_active":"2024-04-05T22:26:10.556154+0000","last_became_peered":"2024-04-05T22:26:10.556154+0000","last_unstale":"2024-04-05T22:26:11.557985+0000","last_undegraded":"2024-04-05T22:26:11.557985+0000","last_fullsized":"2024-04-05T22:26:11.557985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:36.699342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241801+0000","last_change":"2024-04-05T22:26:10.567200+0000","last_active":"2024-04-05T22:26:12.241801+0000","last_peered":"2024-04-05T22:26:12.241801+0000","last_clean":"2024-04-05T22:26:12.241801+0000","last_became_active":"2024-04-05T22:26:10.566952+0000","last_became_peered":"2024-04-05T22:26:10.566952+0000","last_unstale":"2024-04-05T22:26:12.241801+0000","last_undegraded":"2024-04-05T22:26:12.241801+0000","last_fullsized":"2024-04-05T22:26:12.241801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:33:56.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560524+0000","last_change":"2024-04-05T22:26:10.566979+0000","last_active":"2024-04-05T22:26:11.560524+0000","last_peered":"2024-04-05T22:26:11.560524+0000","last_clean":"2024-04-05T22:26:11.560524+0000","last_became_active":"2024-04-05T22:26:10.566697+0000","last_became_peered":"2024-04-05T22:26:10.566697+0000","last_unstale":"2024-04-05T22:26:11.560524+0000","last_undegraded":"2024-04-05T22:26:11.560524+0000","last_fullsized":"2024-04-05T22:26:11.560524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:46.949572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242659+0000","last_change":"2024-04-05T22:26:10.561447+0000","last_active":"2024-04-05T22:26:12.242659+0000","last_peered":"2024-04-05T22:26:12.242659+0000","last_clean":"2024-04-05T22:26:12.242659+0000","last_became_active":"2024-04-05T22:26:10.561064+0000","last_became_peered":"2024-04-05T22:26:10.561064+0000","last_unstale":"2024-04-05T22:26:12.242659+0000","last_undegraded":"2024-04-05T22:26:12.242659+0000","last_fullsized":"2024-04-05T22:26:12.242659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:45:19.947550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242946+0000","last_change":"2024-04-05T22:26:10.579209+0000","last_active":"2024-04-05T22:26:12.242946+0000","last_peered":"2024-04-05T22:26:12.242946+0000","last_clean":"2024-04-05T22:26:12.242946+0000","last_became_active":"2024-04-05T22:26:10.570235+0000","last_became_peered":"2024-04-05T22:26:10.570235+0000","last_unstale":"2024-04-05T22:26:12.242946+0000","last_undegraded":"2024-04-05T22:26:12.242946+0000","last_fullsized":"2024-04-05T22:26:12.242946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:21:07.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560831+0000","last_change":"2024-04-05T22:26:10.574663+0000","last_active":"2024-04-05T22:26:11.560831+0000","last_peered":"2024-04-05T22:26:11.560831+0000","last_clean":"2024-04-05T22:26:11.560831+0000","last_became_active":"2024-04-05T22:26:10.574220+0000","last_became_peered":"2024-04-05T22:26:10.574220+0000","last_unstale":"2024-04-05T22:26:11.560831+0000","last_undegraded":"2024-04-05T22:26:11.560831+0000","last_fullsized":"2024-04-05T22:26:11.560831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:56.605089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558421+0000","last_change":"2024-04-05T22:26:10.569850+0000","last_active":"2024-04-05T22:26:11.558421+0000","last_peered":"2024-04-05T22:26:11.558421+0000","last_clean":"2024-04-05T22:26:11.558421+0000","last_became_active":"2024-04-05T22:26:10.569562+0000","last_became_peered":"2024-04-05T22:26:10.569562+0000","last_unstale":"2024-04-05T22:26:11.558421+0000","last_undegraded":"2024-04-05T22:26:11.558421+0000","last_fullsized":"2024-04-05T22:26:11.558421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:30:08.688197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245029+0000","last_change":"2024-04-05T22:26:10.586906+0000","last_active":"2024-04-05T22:26:12.245029+0000","last_peered":"2024-04-05T22:26:12.245029+0000","last_clean":"2024-04-05T22:26:12.245029+0000","last_became_active":"2024-04-05T22:26:10.578421+0000","last_became_peered":"2024-04-05T22:26:10.578421+0000","last_unstale":"2024-04-05T22:26:12.245029+0000","last_undegraded":"2024-04-05T22:26:12.245029+0000","last_fullsized":"2024-04-05T22:26:12.245029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:52.827753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243011+0000","last_change":"2024-04-05T22:26:10.566736+0000","last_active":"2024-04-05T22:26:12.243011+0000","last_peered":"2024-04-05T22:26:12.243011+0000","last_clean":"2024-04-05T22:26:12.243011+0000","last_became_active":"2024-04-05T22:26:10.566445+0000","last_became_peered":"2024-04-05T22:26:10.566445+0000","last_unstale":"2024-04-05T22:26:12.243011+0000","last_undegraded":"2024-04-05T22:26:12.243011+0000","last_fullsized":"2024-04-05T22:26:12.243011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:57.069630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243627+0000","last_change":"2024-04-05T22:26:10.564326+0000","last_active":"2024-04-05T22:26:12.243627+0000","last_peered":"2024-04-05T22:26:12.243627+0000","last_clean":"2024-04-05T22:26:12.243627+0000","last_became_active":"2024-04-05T22:26:10.564107+0000","last_became_peered":"2024-04-05T22:26:10.564107+0000","last_unstale":"2024-04-05T22:26:12.243627+0000","last_undegraded":"2024-04-05T22:26:12.243627+0000","last_fullsized":"2024-04-05T22:26:12.243627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:16.485563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243503+0000","last_change":"2024-04-05T22:26:10.569091+0000","last_active":"2024-04-05T22:26:12.243503+0000","last_peered":"2024-04-05T22:26:12.243503+0000","last_clean":"2024-04-05T22:26:12.243503+0000","last_became_active":"2024-04-05T22:26:10.568796+0000","last_became_peered":"2024-04-05T22:26:10.568796+0000","last_unstale":"2024-04-05T22:26:12.243503+0000","last_undegraded":"2024-04-05T22:26:12.243503+0000","last_fullsized":"2024-04-05T22:26:12.243503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:28:42.459472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242041+0000","last_change":"2024-04-05T22:26:10.572005+0000","last_active":"2024-04-05T22:26:12.242041+0000","last_peered":"2024-04-05T22:26:12.242041+0000","last_clean":"2024-04-05T22:26:12.242041+0000","last_became_active":"2024-04-05T22:26:10.571689+0000","last_became_peered":"2024-04-05T22:26:10.571689+0000","last_unstale":"2024-04-05T22:26:12.242041+0000","last_undegraded":"2024-04-05T22:26:12.242041+0000","last_fullsized":"2024-04-05T22:26:12.242041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:12.493196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241838+0000","last_change":"2024-04-05T22:26:10.590873+0000","last_active":"2024-04-05T22:26:12.241838+0000","last_peered":"2024-04-05T22:26:12.241838+0000","last_clean":"2024-04-05T22:26:12.241838+0000","last_became_active":"2024-04-05T22:26:10.590587+0000","last_became_peered":"2024-04-05T22:26:10.590587+0000","last_unstale":"2024-04-05T22:26:12.241838+0000","last_undegraded":"2024-04-05T22:26:12.241838+0000","last_fullsized":"2024-04-05T22:26:12.241838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245978+0000","last_change":"2024-04-05T22:26:10.572471+0000","last_active":"2024-04-05T22:26:12.245978+0000","last_peered":"2024-04-05T22:26:12.245978+0000","last_clean":"2024-04-05T22:26:12.245978+0000","last_became_active":"2024-04-05T22:26:10.572216+0000","last_became_peered":"2024-04-05T22:26:10.572216+0000","last_unstale":"2024-04-05T22:26:12.245978+0000","last_undegraded":"2024-04-05T22:26:12.245978+0000","last_fullsized":"2024-04-05T22:26:12.245978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:19.667713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558879+0000","last_change":"2024-04-05T22:26:10.574476+0000","last_active":"2024-04-05T22:26:11.558879+0000","last_peered":"2024-04-05T22:26:11.558879+0000","last_clean":"2024-04-05T22:26:11.558879+0000","last_became_active":"2024-04-05T22:26:10.574173+0000","last_became_peered":"2024-04-05T22:26:10.574173+0000","last_unstale":"2024-04-05T22:26:11.558879+0000","last_undegraded":"2024-04-05T22:26:11.558879+0000","last_fullsized":"2024-04-05T22:26:11.558879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:59.523605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242403+0000","last_change":"2024-04-05T22:26:10.581955+0000","last_active":"2024-04-05T22:26:12.242403+0000","last_peered":"2024-04-05T22:26:12.242403+0000","last_clean":"2024-04-05T22:26:12.242403+0000","last_became_active":"2024-04-05T22:26:10.581680+0000","last_became_peered":"2024-04-05T22:26:10.581680+0000","last_unstale":"2024-04-05T22:26:12.242403+0000","last_undegraded":"2024-04-05T22:26:12.242403+0000","last_fullsized":"2024-04-05T22:26:12.242403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:11:18.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245200+0000","last_change":"2024-04-05T22:26:10.571157+0000","last_active":"2024-04-05T22:26:12.245200+0000","last_peered":"2024-04-05T22:26:12.245200+0000","last_clean":"2024-04-05T22:26:12.245200+0000","last_became_active":"2024-04-05T22:26:10.570896+0000","last_became_peered":"2024-04-05T22:26:10.570896+0000","last_unstale":"2024-04-05T22:26:12.245200+0000","last_undegraded":"2024-04-05T22:26:12.245200+0000","last_fullsized":"2024-04-05T22:26:12.245200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:37:24.002924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242034+0000","last_change":"2024-04-05T22:26:10.571475+0000","last_active":"2024-04-05T22:26:12.242034+0000","last_peered":"2024-04-05T22:26:12.242034+0000","last_clean":"2024-04-05T22:26:12.242034+0000","last_became_active":"2024-04-05T22:26:10.571206+0000","last_became_peered":"2024-04-05T22:26:10.571206+0000","last_unstale":"2024-04-05T22:26:12.242034+0000","last_undegraded":"2024-04-05T22:26:12.242034+0000","last_fullsized":"2024-04-05T22:26:12.242034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:01:45.884036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558657+0000","last_change":"2024-04-05T22:26:10.590577+0000","last_active":"2024-04-05T22:26:11.558657+0000","last_peered":"2024-04-05T22:26:11.558657+0000","last_clean":"2024-04-05T22:26:11.558657+0000","last_became_active":"2024-04-05T22:26:10.590237+0000","last_became_peered":"2024-04-05T22:26:10.590237+0000","last_unstale":"2024-04-05T22:26:11.558657+0000","last_undegraded":"2024-04-05T22:26:11.558657+0000","last_fullsized":"2024-04-05T22:26:11.558657+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:06:50.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558672+0000","last_change":"2024-04-05T22:26:10.574307+0000","last_active":"2024-04-05T22:26:11.558672+0000","last_peered":"2024-04-05T22:26:11.558672+0000","last_clean":"2024-04-05T22:26:11.558672+0000","last_became_active":"2024-04-05T22:26:10.573972+0000","last_became_peered":"2024-04-05T22:26:10.573972+0000","last_unstale":"2024-04-05T22:26:11.558672+0000","last_undegraded":"2024-04-05T22:26:11.558672+0000","last_fullsized":"2024-04-05T22:26:11.558672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:22.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244471+0000","last_change":"2024-04-05T22:26:10.578259+0000","last_active":"2024-04-05T22:26:12.244471+0000","last_peered":"2024-04-05T22:26:12.244471+0000","last_clean":"2024-04-05T22:26:12.244471+0000","last_became_active":"2024-04-05T22:26:10.577941+0000","last_became_peered":"2024-04-05T22:26:10.577941+0000","last_unstale":"2024-04-05T22:26:12.244471+0000","last_undegraded":"2024-04-05T22:26:12.244471+0000","last_fullsized":"2024-04-05T22:26:12.244471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:29:13.296087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243602+0000","last_change":"2024-04-05T22:26:10.592487+0000","last_active":"2024-04-05T22:26:12.243602+0000","last_peered":"2024-04-05T22:26:12.243602+0000","last_clean":"2024-04-05T22:26:12.243602+0000","last_became_active":"2024-04-05T22:26:10.592201+0000","last_became_peered":"2024-04-05T22:26:10.592201+0000","last_unstale":"2024-04-05T22:26:12.243602+0000","last_undegraded":"2024-04-05T22:26:12.243602+0000","last_fullsized":"2024-04-05T22:26:12.243602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244588+0000","last_change":"2024-04-05T22:26:10.087306+0000","last_active":"2024-04-05T22:26:12.244588+0000","last_peered":"2024-04-05T22:26:12.244588+0000","last_clean":"2024-04-05T22:26:12.244588+0000","last_became_active":"2024-04-05T22:26:10.086991+0000","last_became_peered":"2024-04-05T22:26:10.086991+0000","last_unstale":"2024-04-05T22:26:12.244588+0000","last_undegraded":"2024-04-05T22:26:12.244588+0000","last_fullsized":"2024-04-05T22:26:12.244588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:12.517227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560942+0000","last_change":"2024-04-05T22:26:10.574491+0000","last_active":"2024-04-05T22:26:11.560942+0000","last_peered":"2024-04-05T22:26:11.560942+0000","last_clean":"2024-04-05T22:26:11.560942+0000","last_became_active":"2024-04-05T22:26:10.574094+0000","last_became_peered":"2024-04-05T22:26:10.574094+0000","last_unstale":"2024-04-05T22:26:11.560942+0000","last_undegraded":"2024-04-05T22:26:11.560942+0000","last_fullsized":"2024-04-05T22:26:11.560942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:49:15.370190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243194+0000","last_change":"2024-04-05T22:26:10.087418+0000","last_active":"2024-04-05T22:26:12.243194+0000","last_peered":"2024-04-05T22:26:12.243194+0000","last_clean":"2024-04-05T22:26:12.243194+0000","last_became_active":"2024-04-05T22:26:10.087115+0000","last_became_peered":"2024-04-05T22:26:10.087115+0000","last_unstale":"2024-04-05T22:26:12.243194+0000","last_undegraded":"2024-04-05T22:26:12.243194+0000","last_fullsized":"2024-04-05T22:26:12.243194+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:12:15.473436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244129+0000","last_change":"2024-04-05T22:26:10.578151+0000","last_active":"2024-04-05T22:26:12.244129+0000","last_peered":"2024-04-05T22:26:12.244129+0000","last_clean":"2024-04-05T22:26:12.244129+0000","last_became_active":"2024-04-05T22:26:10.573361+0000","last_became_peered":"2024-04-05T22:26:10.573361+0000","last_unstale":"2024-04-05T22:26:12.244129+0000","last_undegraded":"2024-04-05T22:26:12.244129+0000","last_fullsized":"2024-04-05T22:26:12.244129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:40:30.650718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245421+0000","last_change":"2024-04-05T22:26:10.086105+0000","last_active":"2024-04-05T22:26:12.245421+0000","last_peered":"2024-04-05T22:26:12.245421+0000","last_clean":"2024-04-05T22:26:12.245421+0000","last_became_active":"2024-04-05T22:26:10.085852+0000","last_became_peered":"2024-04-05T22:26:10.085852+0000","last_unstale":"2024-04-05T22:26:12.245421+0000","last_undegraded":"2024-04-05T22:26:12.245421+0000","last_fullsized":"2024-04-05T22:26:12.245421+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:48.275500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242812+0000","last_change":"2024-04-05T22:26:10.584232+0000","last_active":"2024-04-05T22:26:12.242812+0000","last_peered":"2024-04-05T22:26:12.242812+0000","last_clean":"2024-04-05T22:26:12.242812+0000","last_became_active":"2024-04-05T22:26:10.583918+0000","last_became_peered":"2024-04-05T22:26:10.583918+0000","last_unstale":"2024-04-05T22:26:12.242812+0000","last_undegraded":"2024-04-05T22:26:12.242812+0000","last_fullsized":"2024-04-05T22:26:12.242812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:05.948210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244425+0000","last_change":"2024-04-05T22:26:10.090172+0000","last_active":"2024-04-05T22:26:12.244425+0000","last_peered":"2024-04-05T22:26:12.244425+0000","last_clean":"2024-04-05T22:26:12.244425+0000","last_became_active":"2024-04-05T22:26:10.089492+0000","last_became_peered":"2024-04-05T22:26:10.089492+0000","last_unstale":"2024-04-05T22:26:12.244425+0000","last_undegraded":"2024-04-05T22:26:12.244425+0000","last_fullsized":"2024-04-05T22:26:12.244425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:40.596358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558661+0000","last_change":"2024-04-05T22:26:10.576222+0000","last_active":"2024-04-05T22:26:11.558661+0000","last_peered":"2024-04-05T22:26:11.558661+0000","last_clean":"2024-04-05T22:26:11.558661+0000","last_became_active":"2024-04-05T22:26:10.574204+0000","last_became_peered":"2024-04-05T22:26:10.574204+0000","last_unstale":"2024-04-05T22:26:11.558661+0000","last_undegraded":"2024-04-05T22:26:11.558661+0000","last_fullsized":"2024-04-05T22:26:11.558661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:16:17.822694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242508+0000","last_change":"2024-04-05T22:26:10.084099+0000","last_active":"2024-04-05T22:26:12.242508+0000","last_peered":"2024-04-05T22:26:12.242508+0000","last_clean":"2024-04-05T22:26:12.242508+0000","last_became_active":"2024-04-05T22:26:10.083850+0000","last_became_peered":"2024-04-05T22:26:10.083850+0000","last_unstale":"2024-04-05T22:26:12.242508+0000","last_undegraded":"2024-04-05T22:26:12.242508+0000","last_fullsized":"2024-04-05T22:26:12.242508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:16:03.136925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243923+0000","last_change":"2024-04-05T22:26:10.573301+0000","last_active":"2024-04-05T22:26:12.243923+0000","last_peered":"2024-04-05T22:26:12.243923+0000","last_clean":"2024-04-05T22:26:12.243923+0000","last_became_active":"2024-04-05T22:26:10.573035+0000","last_became_peered":"2024-04-05T22:26:10.573035+0000","last_unstale":"2024-04-05T22:26:12.243923+0000","last_undegraded":"2024-04-05T22:26:12.243923+0000","last_fullsized":"2024-04-05T22:26:12.243923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:42.938950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244339+0000","last_change":"2024-04-05T22:26:10.087963+0000","last_active":"2024-04-05T22:26:12.244339+0000","last_peered":"2024-04-05T22:26:12.244339+0000","last_clean":"2024-04-05T22:26:12.244339+0000","last_became_active":"2024-04-05T22:26:10.087426+0000","last_became_peered":"2024-04-05T22:26:10.087426+0000","last_unstale":"2024-04-05T22:26:12.244339+0000","last_undegraded":"2024-04-05T22:26:12.244339+0000","last_fullsized":"2024-04-05T22:26:12.244339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:28:20.626791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560335+0000","last_change":"2024-04-05T22:26:10.558000+0000","last_active":"2024-04-05T22:26:11.560335+0000","last_peered":"2024-04-05T22:26:11.560335+0000","last_clean":"2024-04-05T22:26:11.560335+0000","last_became_active":"2024-04-05T22:26:10.557638+0000","last_became_peered":"2024-04-05T22:26:10.557638+0000","last_unstale":"2024-04-05T22:26:11.560335+0000","last_undegraded":"2024-04-05T22:26:11.560335+0000","last_fullsized":"2024-04-05T22:26:11.560335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:34.229086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557380+0000","last_change":"2024-04-05T22:26:10.065593+0000","last_active":"2024-04-05T22:26:11.557380+0000","last_peered":"2024-04-05T22:26:11.557380+0000","last_clean":"2024-04-05T22:26:11.557380+0000","last_became_active":"2024-04-05T22:26:10.064772+0000","last_became_peered":"2024-04-05T22:26:10.064772+0000","last_unstale":"2024-04-05T22:26:11.557380+0000","last_undegraded":"2024-04-05T22:26:11.557380+0000","last_fullsized":"2024-04-05T22:26:11.557380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:33.216959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242920+0000","last_change":"2024-04-05T22:26:10.567810+0000","last_active":"2024-04-05T22:26:12.242920+0000","last_peered":"2024-04-05T22:26:12.242920+0000","last_clean":"2024-04-05T22:26:12.242920+0000","last_became_active":"2024-04-05T22:26:10.567518+0000","last_became_peered":"2024-04-05T22:26:10.567518+0000","last_unstale":"2024-04-05T22:26:12.242920+0000","last_undegraded":"2024-04-05T22:26:12.242920+0000","last_fullsized":"2024-04-05T22:26:12.242920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:05:14.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560182+0000","last_change":"2024-04-05T22:26:10.079847+0000","last_active":"2024-04-05T22:26:11.560182+0000","last_peered":"2024-04-05T22:26:11.560182+0000","last_clean":"2024-04-05T22:26:11.560182+0000","last_became_active":"2024-04-05T22:26:10.079573+0000","last_became_peered":"2024-04-05T22:26:10.079573+0000","last_unstale":"2024-04-05T22:26:11.560182+0000","last_undegraded":"2024-04-05T22:26:11.560182+0000","last_fullsized":"2024-04-05T22:26:11.560182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:26.349225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.559025+0000","last_change":"2024-04-05T22:26:10.577127+0000","last_active":"2024-04-05T22:26:11.559025+0000","last_peered":"2024-04-05T22:26:11.559025+0000","last_clean":"2024-04-05T22:26:11.559025+0000","last_became_active":"2024-04-05T22:26:10.576830+0000","last_became_peered":"2024-04-05T22:26:10.576830+0000","last_unstale":"2024-04-05T22:26:11.559025+0000","last_undegraded":"2024-04-05T22:26:11.559025+0000","last_fullsized":"2024-04-05T22:26:11.559025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:34:30.583583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246147+0000","last_change":"2024-04-05T22:26:10.082259+0000","last_active":"2024-04-05T22:26:12.246147+0000","last_peered":"2024-04-05T22:26:12.246147+0000","last_clean":"2024-04-05T22:26:12.246147+0000","last_became_active":"2024-04-05T22:26:10.079891+0000","last_became_peered":"2024-04-05T22:26:10.079891+0000","last_unstale":"2024-04-05T22:26:12.246147+0000","last_undegraded":"2024-04-05T22:26:12.246147+0000","last_fullsized":"2024-04-05T22:26:12.246147+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:37:32.101662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244115+0000","last_change":"2024-04-05T22:26:10.592599+0000","last_active":"2024-04-05T22:26:12.244115+0000","last_peered":"2024-04-05T22:26:12.244115+0000","last_clean":"2024-04-05T22:26:12.244115+0000","last_became_active":"2024-04-05T22:26:10.592300+0000","last_became_peered":"2024-04-05T22:26:10.592300+0000","last_unstale":"2024-04-05T22:26:12.244115+0000","last_undegraded":"2024-04-05T22:26:12.244115+0000","last_fullsized":"2024-04-05T22:26:12.244115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:59.523605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.246362+0000","last_change":"2024-04-05T22:26:10.089251+0000","last_active":"2024-04-05T22:26:12.246362+0000","last_peered":"2024-04-05T22:26:12.246362+0000","last_clean":"2024-04-05T22:26:12.246362+0000","last_became_active":"2024-04-05T22:26:10.088797+0000","last_became_peered":"2024-04-05T22:26:10.088797+0000","last_unstale":"2024-04-05T22:26:12.246362+0000","last_undegraded":"2024-04-05T22:26:12.246362+0000","last_fullsized":"2024-04-05T22:26:12.246362+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:27.831999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244285+0000","last_change":"2024-04-05T22:26:10.567965+0000","last_active":"2024-04-05T22:26:12.244285+0000","last_peered":"2024-04-05T22:26:12.244285+0000","last_clean":"2024-04-05T22:26:12.244285+0000","last_became_active":"2024-04-05T22:26:10.567649+0000","last_became_peered":"2024-04-05T22:26:10.567649+0000","last_unstale":"2024-04-05T22:26:12.244285+0000","last_undegraded":"2024-04-05T22:26:12.244285+0000","last_fullsized":"2024-04-05T22:26:12.244285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:55:10.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242636+0000","last_change":"2024-04-05T22:26:10.088879+0000","last_active":"2024-04-05T22:26:12.242636+0000","last_peered":"2024-04-05T22:26:12.242636+0000","last_clean":"2024-04-05T22:26:12.242636+0000","last_became_active":"2024-04-05T22:26:10.088619+0000","last_became_peered":"2024-04-05T22:26:10.088619+0000","last_unstale":"2024-04-05T22:26:12.242636+0000","last_undegraded":"2024-04-05T22:26:12.242636+0000","last_fullsized":"2024-04-05T22:26:12.242636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:55:53.649114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560630+0000","last_change":"2024-04-05T22:26:10.560370+0000","last_active":"2024-04-05T22:26:11.560630+0000","last_peered":"2024-04-05T22:26:11.560630+0000","last_clean":"2024-04-05T22:26:11.560630+0000","last_became_active":"2024-04-05T22:26:10.560077+0000","last_became_peered":"2024-04-05T22:26:10.560077+0000","last_unstale":"2024-04-05T22:26:11.560630+0000","last_undegraded":"2024-04-05T22:26:11.560630+0000","last_fullsized":"2024-04-05T22:26:11.560630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:44:51.950325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558184+0000","last_change":"2024-04-05T22:26:10.065821+0000","last_active":"2024-04-05T22:26:11.558184+0000","last_peered":"2024-04-05T22:26:11.558184+0000","last_clean":"2024-04-05T22:26:11.558184+0000","last_became_active":"2024-04-05T22:26:10.065074+0000","last_became_peered":"2024-04-05T22:26:10.065074+0000","last_unstale":"2024-04-05T22:26:11.558184+0000","last_undegraded":"2024-04-05T22:26:11.558184+0000","last_fullsized":"2024-04-05T22:26:11.558184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:49:14.358062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241998+0000","last_change":"2024-04-05T22:26:10.576694+0000","last_active":"2024-04-05T22:26:12.241998+0000","last_peered":"2024-04-05T22:26:12.241998+0000","last_clean":"2024-04-05T22:26:12.241998+0000","last_became_active":"2024-04-05T22:26:10.576510+0000","last_became_peered":"2024-04-05T22:26:10.576510+0000","last_unstale":"2024-04-05T22:26:12.241998+0000","last_undegraded":"2024-04-05T22:26:12.241998+0000","last_fullsized":"2024-04-05T22:26:12.241998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:24:51.667812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557940+0000","last_change":"2024-04-05T22:26:10.079530+0000","last_active":"2024-04-05T22:26:11.557940+0000","last_peered":"2024-04-05T22:26:11.557940+0000","last_clean":"2024-04-05T22:26:11.557940+0000","last_became_active":"2024-04-05T22:26:10.079156+0000","last_became_peered":"2024-04-05T22:26:10.079156+0000","last_unstale":"2024-04-05T22:26:11.557940+0000","last_undegraded":"2024-04-05T22:26:11.557940+0000","last_fullsized":"2024-04-05T22:26:11.557940+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:22:45.937444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558454+0000","last_change":"2024-04-05T22:26:10.583349+0000","last_active":"2024-04-05T22:26:11.558454+0000","last_peered":"2024-04-05T22:26:11.558454+0000","last_clean":"2024-04-05T22:26:11.558454+0000","last_became_active":"2024-04-05T22:26:10.583084+0000","last_became_peered":"2024-04-05T22:26:10.583084+0000","last_unstale":"2024-04-05T22:26:11.558454+0000","last_undegraded":"2024-04-05T22:26:11.558454+0000","last_fullsized":"2024-04-05T22:26:11.558454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:12.493196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244570+0000","last_change":"2024-04-05T22:26:10.089479+0000","last_active":"2024-04-05T22:26:12.244570+0000","last_peered":"2024-04-05T22:26:12.244570+0000","last_clean":"2024-04-05T22:26:12.244570+0000","last_became_active":"2024-04-05T22:26:10.089154+0000","last_became_peered":"2024-04-05T22:26:10.089154+0000","last_unstale":"2024-04-05T22:26:12.244570+0000","last_undegraded":"2024-04-05T22:26:12.244570+0000","last_fullsized":"2024-04-05T22:26:12.244570+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:58:14.277312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241973+0000","last_change":"2024-04-05T22:26:10.572234+0000","last_active":"2024-04-05T22:26:12.241973+0000","last_peered":"2024-04-05T22:26:12.241973+0000","last_clean":"2024-04-05T22:26:12.241973+0000","last_became_active":"2024-04-05T22:26:10.570149+0000","last_became_peered":"2024-04-05T22:26:10.570149+0000","last_unstale":"2024-04-05T22:26:12.241973+0000","last_undegraded":"2024-04-05T22:26:12.241973+0000","last_fullsized":"2024-04-05T22:26:12.241973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558469+0000","last_change":"2024-04-05T22:26:10.086162+0000","last_active":"2024-04-05T22:26:11.558469+0000","last_peered":"2024-04-05T22:26:11.558469+0000","last_clean":"2024-04-05T22:26:11.558469+0000","last_became_active":"2024-04-05T22:26:10.085908+0000","last_became_peered":"2024-04-05T22:26:10.085908+0000","last_unstale":"2024-04-05T22:26:11.558469+0000","last_undegraded":"2024-04-05T22:26:11.558469+0000","last_fullsized":"2024-04-05T22:26:11.558469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:28:41.447345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560720+0000","last_change":"2024-04-05T22:26:10.573501+0000","last_active":"2024-04-05T22:26:11.560720+0000","last_peered":"2024-04-05T22:26:11.560720+0000","last_clean":"2024-04-05T22:26:11.560720+0000","last_became_active":"2024-04-05T22:26:10.573207+0000","last_became_peered":"2024-04-05T22:26:10.573207+0000","last_unstale":"2024-04-05T22:26:11.560720+0000","last_undegraded":"2024-04-05T22:26:11.560720+0000","last_fullsized":"2024-04-05T22:26:11.560720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:25:01.784393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243150+0000","last_change":"2024-04-05T22:26:10.081963+0000","last_active":"2024-04-05T22:26:12.243150+0000","last_peered":"2024-04-05T22:26:12.243150+0000","last_clean":"2024-04-05T22:26:12.243150+0000","last_became_active":"2024-04-05T22:26:10.080820+0000","last_became_peered":"2024-04-05T22:26:10.080820+0000","last_unstale":"2024-04-05T22:26:12.243150+0000","last_undegraded":"2024-04-05T22:26:12.243150+0000","last_fullsized":"2024-04-05T22:26:12.243150+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:12:06.379391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245879+0000","last_change":"2024-04-05T22:26:10.562895+0000","last_active":"2024-04-05T22:26:12.245879+0000","last_peered":"2024-04-05T22:26:12.245879+0000","last_clean":"2024-04-05T22:26:12.245879+0000","last_became_active":"2024-04-05T22:26:10.562539+0000","last_became_peered":"2024-04-05T22:26:10.562539+0000","last_unstale":"2024-04-05T22:26:12.245879+0000","last_undegraded":"2024-04-05T22:26:12.245879+0000","last_fullsized":"2024-04-05T22:26:12.245879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:51.195268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244692+0000","last_change":"2024-04-05T22:26:10.088325+0000","last_active":"2024-04-05T22:26:12.244692+0000","last_peered":"2024-04-05T22:26:12.244692+0000","last_clean":"2024-04-05T22:26:12.244692+0000","last_became_active":"2024-04-05T22:26:10.087891+0000","last_became_peered":"2024-04-05T22:26:10.087891+0000","last_unstale":"2024-04-05T22:26:12.244692+0000","last_undegraded":"2024-04-05T22:26:12.244692+0000","last_fullsized":"2024-04-05T22:26:12.244692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:37.248834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244355+0000","last_change":"2024-04-05T22:26:10.588302+0000","last_active":"2024-04-05T22:26:12.244355+0000","last_peered":"2024-04-05T22:26:12.244355+0000","last_clean":"2024-04-05T22:26:12.244355+0000","last_became_active":"2024-04-05T22:26:10.588062+0000","last_became_peered":"2024-04-05T22:26:10.588062+0000","last_unstale":"2024-04-05T22:26:12.244355+0000","last_undegraded":"2024-04-05T22:26:12.244355+0000","last_fullsized":"2024-04-05T22:26:12.244355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:19.667713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245864+0000","last_change":"2024-04-05T22:26:10.086292+0000","last_active":"2024-04-05T22:26:12.245864+0000","last_peered":"2024-04-05T22:26:12.245864+0000","last_clean":"2024-04-05T22:26:12.245864+0000","last_became_active":"2024-04-05T22:26:10.085911+0000","last_became_peered":"2024-04-05T22:26:10.085911+0000","last_unstale":"2024-04-05T22:26:12.245864+0000","last_undegraded":"2024-04-05T22:26:12.245864+0000","last_fullsized":"2024-04-05T22:26:12.245864+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:13.129030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558085+0000","last_change":"2024-04-05T22:26:10.556562+0000","last_active":"2024-04-05T22:26:11.558085+0000","last_peered":"2024-04-05T22:26:11.558085+0000","last_clean":"2024-04-05T22:26:11.558085+0000","last_became_active":"2024-04-05T22:26:10.555745+0000","last_became_peered":"2024-04-05T22:26:10.555745+0000","last_unstale":"2024-04-05T22:26:11.558085+0000","last_undegraded":"2024-04-05T22:26:11.558085+0000","last_fullsized":"2024-04-05T22:26:11.558085+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:13.529355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245433+0000","last_change":"2024-04-05T22:26:10.092146+0000","last_active":"2024-04-05T22:26:12.245433+0000","last_peered":"2024-04-05T22:26:12.245433+0000","last_clean":"2024-04-05T22:26:12.245433+0000","last_became_active":"2024-04-05T22:26:10.091743+0000","last_became_peered":"2024-04-05T22:26:10.091743+0000","last_unstale":"2024-04-05T22:26:12.245433+0000","last_undegraded":"2024-04-05T22:26:12.245433+0000","last_fullsized":"2024-04-05T22:26:12.245433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:51.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560876+0000","last_change":"2024-04-05T22:26:10.557716+0000","last_active":"2024-04-05T22:26:11.560876+0000","last_peered":"2024-04-05T22:26:11.560876+0000","last_clean":"2024-04-05T22:26:11.560876+0000","last_became_active":"2024-04-05T22:26:10.557338+0000","last_became_peered":"2024-04-05T22:26:10.557338+0000","last_unstale":"2024-04-05T22:26:11.560876+0000","last_undegraded":"2024-04-05T22:26:11.560876+0000","last_fullsized":"2024-04-05T22:26:11.560876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:01.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243113+0000","last_change":"2024-04-05T22:26:10.082040+0000","last_active":"2024-04-05T22:26:12.243113+0000","last_peered":"2024-04-05T22:26:12.243113+0000","last_clean":"2024-04-05T22:26:12.243113+0000","last_became_active":"2024-04-05T22:26:10.081668+0000","last_became_peered":"2024-04-05T22:26:10.081668+0000","last_unstale":"2024-04-05T22:26:12.243113+0000","last_undegraded":"2024-04-05T22:26:12.243113+0000","last_fullsized":"2024-04-05T22:26:12.243113+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:35.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242310+0000","last_change":"2024-04-05T22:26:10.571263+0000","last_active":"2024-04-05T22:26:12.242310+0000","last_peered":"2024-04-05T22:26:12.242310+0000","last_clean":"2024-04-05T22:26:12.242310+0000","last_became_active":"2024-04-05T22:26:10.570857+0000","last_became_peered":"2024-04-05T22:26:10.570857+0000","last_unstale":"2024-04-05T22:26:12.242310+0000","last_undegraded":"2024-04-05T22:26:12.242310+0000","last_fullsized":"2024-04-05T22:26:12.242310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:49.772016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244813+0000","last_change":"2024-04-05T22:26:10.085450+0000","last_active":"2024-04-05T22:26:12.244813+0000","last_peered":"2024-04-05T22:26:12.244813+0000","last_clean":"2024-04-05T22:26:12.244813+0000","last_became_active":"2024-04-05T22:26:10.084927+0000","last_became_peered":"2024-04-05T22:26:10.084927+0000","last_unstale":"2024-04-05T22:26:12.244813+0000","last_undegraded":"2024-04-05T22:26:12.244813+0000","last_fullsized":"2024-04-05T22:26:12.244813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:28:29.386651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241323+0000","last_change":"2024-04-05T22:26:10.589210+0000","last_active":"2024-04-05T22:26:12.241323+0000","last_peered":"2024-04-05T22:26:12.241323+0000","last_clean":"2024-04-05T22:26:12.241323+0000","last_became_active":"2024-04-05T22:26:10.588942+0000","last_became_peered":"2024-04-05T22:26:10.588942+0000","last_unstale":"2024-04-05T22:26:12.241323+0000","last_undegraded":"2024-04-05T22:26:12.241323+0000","last_fullsized":"2024-04-05T22:26:12.241323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:41.846577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558644+0000","last_change":"2024-04-05T22:26:10.089430+0000","last_active":"2024-04-05T22:26:11.558644+0000","last_peered":"2024-04-05T22:26:11.558644+0000","last_clean":"2024-04-05T22:26:11.558644+0000","last_became_active":"2024-04-05T22:26:10.089068+0000","last_became_peered":"2024-04-05T22:26:10.089068+0000","last_unstale":"2024-04-05T22:26:11.558644+0000","last_undegraded":"2024-04-05T22:26:11.558644+0000","last_fullsized":"2024-04-05T22:26:11.558644+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:06:43.278506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241619+0000","last_change":"2024-04-05T22:26:10.589877+0000","last_active":"2024-04-05T22:26:12.241619+0000","last_peered":"2024-04-05T22:26:12.241619+0000","last_clean":"2024-04-05T22:26:12.241619+0000","last_became_active":"2024-04-05T22:26:10.589551+0000","last_became_peered":"2024-04-05T22:26:10.589551+0000","last_unstale":"2024-04-05T22:26:12.241619+0000","last_undegraded":"2024-04-05T22:26:12.241619+0000","last_fullsized":"2024-04-05T22:26:12.241619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:29:54.545828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241708+0000","last_change":"2024-04-05T22:26:10.087563+0000","last_active":"2024-04-05T22:26:12.241708+0000","last_peered":"2024-04-05T22:26:12.241708+0000","last_clean":"2024-04-05T22:26:12.241708+0000","last_became_active":"2024-04-05T22:26:10.087242+0000","last_became_peered":"2024-04-05T22:26:10.087242+0000","last_unstale":"2024-04-05T22:26:12.241708+0000","last_undegraded":"2024-04-05T22:26:12.241708+0000","last_fullsized":"2024-04-05T22:26:12.241708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:18.655586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243221+0000","last_change":"2024-04-05T22:26:10.565840+0000","last_active":"2024-04-05T22:26:12.243221+0000","last_peered":"2024-04-05T22:26:12.243221+0000","last_clean":"2024-04-05T22:26:12.243221+0000","last_became_active":"2024-04-05T22:26:10.565553+0000","last_became_peered":"2024-04-05T22:26:10.565553+0000","last_unstale":"2024-04-05T22:26:12.243221+0000","last_undegraded":"2024-04-05T22:26:12.243221+0000","last_fullsized":"2024-04-05T22:26:12.243221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:32:05.479366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244897+0000","last_change":"2024-04-05T22:26:10.088150+0000","last_active":"2024-04-05T22:26:12.244897+0000","last_peered":"2024-04-05T22:26:12.244897+0000","last_clean":"2024-04-05T22:26:12.244897+0000","last_became_active":"2024-04-05T22:26:10.087504+0000","last_became_peered":"2024-04-05T22:26:10.087504+0000","last_unstale":"2024-04-05T22:26:12.244897+0000","last_undegraded":"2024-04-05T22:26:12.244897+0000","last_fullsized":"2024-04-05T22:26:12.244897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:41:22.356009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558854+0000","last_change":"2024-04-05T22:26:10.582030+0000","last_active":"2024-04-05T22:26:11.558854+0000","last_peered":"2024-04-05T22:26:11.558854+0000","last_clean":"2024-04-05T22:26:11.558854+0000","last_became_active":"2024-04-05T22:26:10.580582+0000","last_became_peered":"2024-04-05T22:26:10.580582+0000","last_unstale":"2024-04-05T22:26:11.558854+0000","last_undegraded":"2024-04-05T22:26:11.558854+0000","last_fullsized":"2024-04-05T22:26:11.558854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:58:01.378901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.559036+0000","last_change":"2024-04-05T22:26:10.091683+0000","last_active":"2024-04-05T22:26:11.559036+0000","last_peered":"2024-04-05T22:26:11.559036+0000","last_clean":"2024-04-05T22:26:11.559036+0000","last_became_active":"2024-04-05T22:26:10.091417+0000","last_became_peered":"2024-04-05T22:26:10.091417+0000","last_unstale":"2024-04-05T22:26:11.559036+0000","last_undegraded":"2024-04-05T22:26:11.559036+0000","last_fullsized":"2024-04-05T22:26:11.559036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:28:00.451532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.242905+0000","last_change":"2024-04-05T22:26:10.592440+0000","last_active":"2024-04-05T22:26:12.242905+0000","last_peered":"2024-04-05T22:26:12.242905+0000","last_clean":"2024-04-05T22:26:12.242905+0000","last_became_active":"2024-04-05T22:26:10.592173+0000","last_became_peered":"2024-04-05T22:26:10.592173+0000","last_unstale":"2024-04-05T22:26:12.242905+0000","last_undegraded":"2024-04-05T22:26:12.242905+0000","last_fullsized":"2024-04-05T22:26:12.242905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:16:17.822694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560972+0000","last_change":"2024-04-05T22:26:10.083690+0000","last_active":"2024-04-05T22:26:11.560972+0000","last_peered":"2024-04-05T22:26:11.560972+0000","last_clean":"2024-04-05T22:26:11.560972+0000","last_became_active":"2024-04-05T22:26:10.083397+0000","last_became_peered":"2024-04-05T22:26:10.083397+0000","last_unstale":"2024-04-05T22:26:11.560972+0000","last_undegraded":"2024-04-05T22:26:11.560972+0000","last_fullsized":"2024-04-05T22:26:11.560972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:42:04.187427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244876+0000","last_change":"2024-04-05T22:26:10.592052+0000","last_active":"2024-04-05T22:26:12.244876+0000","last_peered":"2024-04-05T22:26:12.244876+0000","last_clean":"2024-04-05T22:26:12.244876+0000","last_became_active":"2024-04-05T22:26:10.591524+0000","last_became_peered":"2024-04-05T22:26:10.591524+0000","last_unstale":"2024-04-05T22:26:12.244876+0000","last_undegraded":"2024-04-05T22:26:12.244876+0000","last_fullsized":"2024-04-05T22:26:12.244876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:06:50.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243106+0000","last_change":"2024-04-05T22:26:10.083510+0000","last_active":"2024-04-05T22:26:12.243106+0000","last_peered":"2024-04-05T22:26:12.243106+0000","last_clean":"2024-04-05T22:26:12.243106+0000","last_became_active":"2024-04-05T22:26:10.083158+0000","last_became_peered":"2024-04-05T22:26:10.083158+0000","last_unstale":"2024-04-05T22:26:12.243106+0000","last_undegraded":"2024-04-05T22:26:12.243106+0000","last_fullsized":"2024-04-05T22:26:12.243106+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:41:41.926822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.560932+0000","last_change":"2024-04-05T22:26:10.571917+0000","last_active":"2024-04-05T22:26:11.560932+0000","last_peered":"2024-04-05T22:26:11.560932+0000","last_clean":"2024-04-05T22:26:11.560932+0000","last_became_active":"2024-04-05T22:26:10.571673+0000","last_became_peered":"2024-04-05T22:26:10.571673+0000","last_unstale":"2024-04-05T22:26:11.560932+0000","last_undegraded":"2024-04-05T22:26:11.560932+0000","last_fullsized":"2024-04-05T22:26:11.560932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:40:59.772321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.561049+0000","last_change":"2024-04-05T22:26:10.091113+0000","last_active":"2024-04-05T22:26:11.561049+0000","last_peered":"2024-04-05T22:26:11.561049+0000","last_clean":"2024-04-05T22:26:11.561049+0000","last_became_active":"2024-04-05T22:26:10.089154+0000","last_became_peered":"2024-04-05T22:26:10.089154+0000","last_unstale":"2024-04-05T22:26:11.561049+0000","last_undegraded":"2024-04-05T22:26:11.561049+0000","last_fullsized":"2024-04-05T22:26:11.561049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:41:20.095091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245445+0000","last_change":"2024-04-05T22:26:10.578205+0000","last_active":"2024-04-05T22:26:12.245445+0000","last_peered":"2024-04-05T22:26:12.245445+0000","last_clean":"2024-04-05T22:26:12.245445+0000","last_became_active":"2024-04-05T22:26:10.577883+0000","last_became_peered":"2024-04-05T22:26:10.577883+0000","last_unstale":"2024-04-05T22:26:12.245445+0000","last_undegraded":"2024-04-05T22:26:12.245445+0000","last_fullsized":"2024-04-05T22:26:12.245445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:26:48.025720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.558235+0000","last_change":"2024-04-05T22:26:10.065941+0000","last_active":"2024-04-05T22:26:11.558235+0000","last_peered":"2024-04-05T22:26:11.558235+0000","last_clean":"2024-04-05T22:26:11.558235+0000","last_became_active":"2024-04-05T22:26:10.065253+0000","last_became_peered":"2024-04-05T22:26:10.065253+0000","last_unstale":"2024-04-05T22:26:11.558235+0000","last_undegraded":"2024-04-05T22:26:11.558235+0000","last_fullsized":"2024-04-05T22:26:11.558235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:44:50.938197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:11.557435+0000","last_change":"2024-04-05T22:26:10.573880+0000","last_active":"2024-04-05T22:26:11.557435+0000","last_peered":"2024-04-05T22:26:11.557435+0000","last_clean":"2024-04-05T22:26:11.557435+0000","last_became_active":"2024-04-05T22:26:10.572917+0000","last_became_peered":"2024-04-05T22:26:10.572917+0000","last_unstale":"2024-04-05T22:26:11.557435+0000","last_undegraded":"2024-04-05T22:26:11.557435+0000","last_fullsized":"2024-04-05T22:26:11.557435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:53.929778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.243140+0000","last_change":"2024-04-05T22:26:10.080919+0000","last_active":"2024-04-05T22:26:12.243140+0000","last_peered":"2024-04-05T22:26:12.243140+0000","last_clean":"2024-04-05T22:26:12.243140+0000","last_became_active":"2024-04-05T22:26:10.079625+0000","last_became_peered":"2024-04-05T22:26:10.079625+0000","last_unstale":"2024-04-05T22:26:12.243140+0000","last_undegraded":"2024-04-05T22:26:12.243140+0000","last_fullsized":"2024-04-05T22:26:12.243140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:55.592962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244950+0000","last_change":"2024-04-05T22:26:10.578503+0000","last_active":"2024-04-05T22:26:12.244950+0000","last_peered":"2024-04-05T22:26:12.244950+0000","last_clean":"2024-04-05T22:26:12.244950+0000","last_became_active":"2024-04-05T22:26:10.578174+0000","last_became_peered":"2024-04-05T22:26:10.578174+0000","last_unstale":"2024-04-05T22:26:12.244950+0000","last_undegraded":"2024-04-05T22:26:12.244950+0000","last_fullsized":"2024-04-05T22:26:12.244950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:09.578237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.244654+0000","last_change":"2024-04-05T22:26:10.083814+0000","last_active":"2024-04-05T22:26:12.244654+0000","last_peered":"2024-04-05T22:26:12.244654+0000","last_clean":"2024-04-05T22:26:12.244654+0000","last_became_active":"2024-04-05T22:26:10.082357+0000","last_became_peered":"2024-04-05T22:26:10.082357+0000","last_unstale":"2024-04-05T22:26:12.244654+0000","last_undegraded":"2024-04-05T22:26:12.244654+0000","last_fullsized":"2024-04-05T22:26:12.244654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:54:39.001670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.241874+0000","last_change":"2024-04-05T22:26:10.591072+0000","last_active":"2024-04-05T22:26:12.241874+0000","last_peered":"2024-04-05T22:26:12.241874+0000","last_clean":"2024-04-05T22:26:12.241874+0000","last_became_active":"2024-04-05T22:26:10.590676+0000","last_became_peered":"2024-04-05T22:26:10.590676+0000","last_unstale":"2024-04-05T22:26:12.241874+0000","last_undegraded":"2024-04-05T22:26:12.241874+0000","last_fullsized":"2024-04-05T22:26:12.241874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:10.040692+0000","last_clean_scrub_stamp":"2024-04-05T22:26:10.040692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:07.646105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-05T22:26:12.245661+0000","last_change":"2024-04-05T22:26:10.090017+0000","last_active":"2024-04-05T22:26:12.245661+0000","last_peered":"2024-04-05T22:26:12.245661+0000","last_clean":"2024-04-05T22:26:12.245661+0000","last_became_active":"2024-04-05T22:26:10.089687+0000","last_became_peered":"2024-04-05T22:26:10.089687+0000","last_unstale":"2024-04-05T22:26:12.245661+0000","last_undegraded":"2024-04-05T22:26:12.245661+0000","last_fullsized":"2024-04-05T22:26:12.245661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:09.028564+0000","last_clean_scrub_stamp":"2024-04-05T22:26:09.028564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:56:34.420409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-05T22:26:10.567576+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:26:10.567576+0000","last_peered":"2024-04-05T22:26:10.567576+0000","last_clean":"2024-04-05T22:26:10.567576+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:26:10.567576+0000","last_undegraded":"2024-04-05T22:26:10.567576+0000","last_fullsized":"2024-04-05T22:26:10.567576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":968686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0589999999999999,"15min":1.0309999999999999},"min":{"1min":0.84999999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3009999999999999,"5min":1.796,"15min":1.796},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0489999999999999,"15min":1.0069999999999999},"min":{"1min":0.752,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.345}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.115,"15min":1.0429999999999999},"min":{"1min":0.75600000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0720000000000001,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4750000000000001,"5min":1.516,"15min":1.516},"last":1.214}]},{"osd":2,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1910000000000001,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.415,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.284},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.25,"15min":1.214},"min":{"1min":0.89600000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.999}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1759999999999999,"15min":1.1279999999999999},"min":{"1min":0.81799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6579999999999999,"5min":1.712,"15min":1.712},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.22},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5529999999999999,"5min":1.643,"15min":1.643},"last":1.3660000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2430000000000001,"15min":1.222},"min":{"1min":0.77300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.623,"5min":1.865,"15min":1.865},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.58399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.772,"5min":1.788,"15min":1.788},"last":1.7350000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.28,"15min":1.2310000000000001},"min":{"1min":0.67600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5620000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.615},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.365,"15min":1.345},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.891,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5620000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.488,"15min":1.454},"min":{"1min":0.90600000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9450000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.877},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.516,"15min":1.494},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9199999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.659}]}]},{"osd":6,"up_from":14,"seq":60129542209,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":905216,"data_stored":377794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0669999999999999},"min":{"1min":0.86599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.097},"min":{"1min":0.79500000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1579999999999999,"15min":1.177},"min":{"1min":0.78800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.575,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.72299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3879999999999999,"5min":1.722,"15min":1.722},"last":1.1699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1879999999999999,"15min":1.2030000000000001},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5349999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.76100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.706,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.01}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.29,"15min":1.339},"min":{"1min":0.82699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.482,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4179999999999999},"min":{"1min":1.0029999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6830000000000001,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.226}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.3360000000000001},"min":{"1min":0.77000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.556,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.073},{"interface":"front","average":{"1min":1.415,"5min":1.48,"15min":1.5840000000000001},"min":{"1min":0.91700000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8029999999999999,"5min":3.27,"15min":3.27},"last":1.6120000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.363,"15min":1.4670000000000001},"min":{"1min":0.96099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.752,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.387},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.492,"15min":1.5669999999999999},"min":{"1min":1.1279999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.625,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.272}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4450000000000001,"15min":1.5620000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.903,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.538},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.476,"15min":1.5780000000000001},"min":{"1min":1.0269999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.661,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.494}]}]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":901120,"data_stored":377792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.052,"15min":1.006},"min":{"1min":0.78200000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.177,"5min":1.107,"15min":1.089},"min":{"1min":0.85499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.167,"15min":1.167},"min":{"1min":0.90800000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1799999999999999,"15min":1.1579999999999999},"min":{"1min":0.82699999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.508,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1759999999999999,"15min":1.1539999999999999},"min":{"1min":0.73099999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":2.544,"15min":2.544},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.23,"15min":1.2529999999999999},"min":{"1min":0.82199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7210000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2,"15min":1.1799999999999999},"min":{"1min":0.93999999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.448,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.306,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.597,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.84199999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.512,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.79700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.655,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.524,"5min":1.972,"15min":1.972},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.474,"15min":1.4890000000000001},"min":{"1min":1.052,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7609999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.403,"15min":1.4510000000000001},"min":{"1min":0.69999999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.615,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.526,"15min":1.6240000000000001},"min":{"1min":0.84399999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6739999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.50900000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":901120,"data_stored":377772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0509999999999999,"15min":0.999},"min":{"1min":0.84699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":1.673,"15min":1.673},"last":0.877},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0980000000000001,"15min":1.0800000000000001},"min":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.6459999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1439999999999999,"15min":1.1299999999999999},"min":{"1min":0.875,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.607},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1539999999999999,"15min":1.1699999999999999},"min":{"1min":0.83399999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4450000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.341}]},{"osd":2,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.95099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.482,"5min":1.974,"15min":1.974},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.89000000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.649,"5min":1.673,"15min":1.673},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.379},"min":{"1min":0.95999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.843,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.45},{"interface":"front","average":{"1min":1.385,"5min":1.369,"15min":1.407},"min":{"1min":1.107,"5min":0.874,"15min":0.874},"max":{"1min":1.754,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.111}]},{"osd":5,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.492},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3440000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.625,"15min":0.625},"max":{"1min":1.786,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.77}]},{"osd":6,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4610000000000001},"min":{"1min":0.95599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6379999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4410000000000001,"15min":1.4950000000000001},"min":{"1min":1.0149999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.655,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.734}]},{"osd":7,"last update":"Fri Apr 5 22:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5029999999999999,"15min":1.536},"min":{"1min":1.1639999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.895,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.8400000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5549999999999999,"15min":1.5760000000000001},"min":{"1min":1.05,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":2.54,"15min":2.54},"last":1.964}]}]},{"osd":3,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":892928,"data_stored":371514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4910000000000001,"5min":1.744,"15min":1.744},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.90000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3300000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.026},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.468,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1,"15min":1.026},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.45,"5min":1.827,"15min":1.827},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.085,"15min":1.0860000000000001},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.675,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.395}]},{"osd":4,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.202,"15min":1.246},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5209999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.242,"15min":1.3129999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6719999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.157}]},{"osd":5,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3129999999999999,"15min":1.375},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7390000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.093},{"interface":"front","average":{"1min":1.335,"5min":1.286,"15min":1.2949999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.704,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.327,"15min":1.3680000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6930000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.048},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8180000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2669999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.393,"15min":1.446},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9410000000000001,"5min":2.468,"15min":2.468},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4059999999999999,"15min":1.4350000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.865,"5min":2.798,"15min":2.798},"last":1.175}]}]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":909312,"data_stored":379266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3700000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.91700000000000004,"15min":0.91400000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.198,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0660000000000001,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.605,"15min":1.605},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.143,"15min":1.129},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5629999999999999,"5min":2.694,"15min":2.694},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.262,"15min":1.222},"min":{"1min":0.83299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.542,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.238,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4810000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.115}]},{"osd":5,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5640000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6040000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.161}]},{"osd":6,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2669999999999999,"15min":1.268},"min":{"1min":0.85399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5169999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.319,"5min":1.294,"15min":1.2749999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.639,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:25:15 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.84499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5169999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.391,"15min":1.393},"min":{"1min":0.88,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6990000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.2549999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":917504,"data_stored":379882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.87,"15min":0.84699999999999998},"min":{"1min":0.66900000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.9119999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.84299999999999997,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2849999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0169999999999999,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.258}]},{"osd":3,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.75800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.504,"5min":1.75,"15min":1.75},"last":1.167},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.96899999999999997,"15min":0.96699999999999997},"min":{"1min":0.73699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3879999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.216}]},{"osd":4,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.131},"min":{"1min":0.89300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.482,"5min":1.855,"15min":1.855},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.806,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.5489999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":1.0840000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.833,"5min":1.887,"15min":1.887},"last":2.1120000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.23,"15min":1.2370000000000001},"min":{"1min":1.0169999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.8069999999999999,"5min":2.173,"15min":2.173},"last":1.302}]},{"osd":6,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.88200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9059999999999999,"5min":1.917,"15min":1.917},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3360000000000001,"15min":1.3660000000000001},"min":{"1min":1.1120000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":2.0259999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:25:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3600000000000001,"15min":1.3640000000000001},"min":{"1min":1.1279999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.117,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.3460000000000001,"15min":1.353},"min":{"1min":0.96399999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.879,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.607}]}]},{"osd":0,"up_from":14,"seq":60129542209,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1478656,"data_stored":961770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80200000000000005,"15min":0.79600000000000004},"min":{"1min":0.63100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3400000000000001,"5min":1.367,"15min":1.367},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90300000000000002,"15min":0.875},"min":{"1min":0.65900000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4159999999999999,"5min":1.669,"15min":1.669},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4119999999999999,"5min":1.514,"15min":1.514},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.40400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.4570000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.93400000000000005,"15min":0.91800000000000004},"min":{"1min":0.65700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5860000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.747},{"interface":"front","average":{"1min":1.127,"5min":1.1040000000000001,"15min":1.1080000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.474,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.004}]},{"osd":4,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1890000000000001,"15min":1.165},"min":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.448,"5min":1.929,"15min":1.929},"last":1.254},{"interface":"front","average":{"1min":1.385,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":1.0920000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.462}]},{"osd":5,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.27},"min":{"1min":0.88600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.659,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.288},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.198},"min":{"1min":0.97699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7110000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.127}]},{"osd":6,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3839999999999999,"15min":1.3939999999999999},"min":{"1min":1.042,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.385,"15min":1.387},"min":{"1min":1.0509999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.891,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.516}]},{"osd":7,"last update":"Fri Apr 5 22:25:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.361,"15min":1.347},"min":{"1min":1.0009999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3200000000000001,"15min":1.3340000000000001},"min":{"1min":1.0760000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.595,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.339}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:26:15.529 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:26:15.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:26:15.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.752+0000 7f1a86e00640 1 Processor -- start 2024-04-05T22:26:15.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.752+0000 7f1a86e00640 1 -- start start 2024-04-05T22:26:15.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a86e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a800b0720 0x7f1a800b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a86e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b1750 0x7f1a80166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a86e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8006adc0 con 0x7f1a800b1bf0 2024-04-05T22:26:15.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a86e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a800b2460 con 0x7f1a800b0720 2024-04-05T22:26:15.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a86e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8015d4f0 con 0x7f1a800b1750 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a7fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a800b0720 0x7f1a800b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a7f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1a800b1bf0 0x7f1a800b1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60074/0 (socket says 172.21.15.73:60074) 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a7f400640 1 -- 172.21.15.73:0/3407656791 learned_addr learned my addr 172.21.15.73:0/3407656791 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.754+0000 7f1a7fe00640 1 -- 172.21.15.73:0/3407656791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b1750 msgr2=0x7f1a80166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.755+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/3407656791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b1750 0x7f1a80166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.755+0000 7f1a7fe00640 1 -- 172.21.15.73:0/3407656791 >> v1:172.21.15.73:6789/0 conn(0x7f1a800b1bf0 legacy=0x7f1a800b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.755+0000 7f1a7fe00640 1 -- 172.21.15.73:0/3407656791 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800b51c0 con 0x7f1a800b0720 2024-04-05T22:26:15.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.755+0000 7f1a7ea00640 1 -- 172.21.15.73:0/3407656791 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425157708 0 0) 0x7f1a8006adc0 con 0x7f1a800b1bf0 2024-04-05T22:26:15.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.756+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/3407656791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b0720 0x7f1a800b0b00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198d60fc3b7f9a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.756+0000 7f1a7ea00640 1 -- 172.21.15.73:0/3407656791 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7002e070 con 0x7f1a800b0720 2024-04-05T22:26:15.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.756+0000 7f1a7ea00640 1 -- 172.21.15.73:0/3407656791 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a700315a0 con 0x7f1a800b0720 2024-04-05T22:26:15.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.756+0000 7f1a7ea00640 1 -- 172.21.15.73:0/3407656791 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70031930 con 0x7f1a800b0720 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 -- 172.21.15.73:0/3407656791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b0720 msgr2=0x7f1a800b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 --2- 172.21.15.73:0/3407656791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b0720 0x7f1a800b0b00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002b840 comp rx=0 tx=0).stop 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 -- 172.21.15.73:0/3407656791 shutdown_connections 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 --2- 172.21.15.73:0/3407656791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b1750 0x7f1a80166c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 --2- 172.21.15.73:0/3407656791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b0720 0x7f1a800b0b00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 -- 172.21.15.73:0/3407656791 >> 172.21.15.73:0/3407656791 conn(0x7f1a80099a60 msgr2=0x7f1a80099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 -- 172.21.15.73:0/3407656791 shutdown_connections 2024-04-05T22:26:15.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.757+0000 7f1a86e00640 1 -- 172.21.15.73:0/3407656791 wait complete. 2024-04-05T22:26:15.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.758+0000 7f1a86e00640 1 Processor -- start 2024-04-05T22:26:15.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.759+0000 7f1a86e00640 1 -- start start 2024-04-05T22:26:15.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 0x7f1a80142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a800b1bf0 0x7f1a8015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8015b110 con 0x7f1a800b1750 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8015a070 con 0x7f1a800b1bf0 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a86e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8015a350 con 0x7f1a800b0720 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49888/0 (socket says 172.21.15.73:49888) 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 -- 172.21.15.73:0/2490153690 learned_addr learned my addr 172.21.15.73:0/2490153690 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 0x7f1a80142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a84e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a800b1bf0 0x7f1a8015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 -- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 msgr2=0x7f1a80142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 0x7f1a80142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 -- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a800b1bf0 msgr2=0x7f1a8015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:15.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a800b1bf0 0x7f1a8015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:15.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7f400640 1 -- 172.21.15.73:0/2490153690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009400 con 0x7f1a800b1750 2024-04-05T22:26:15.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.760+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 0x7f1a80142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:15.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a7f400640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c430 tx=0x7f1a6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0074e0 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c007660 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8015a630 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c004a20 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.761+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8015abf0 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.762+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a80067290 con 0x7f1a800b1750 2024-04-05T22:26:15.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.763+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f1a6c004b80 con 0x7f1a800b1750 2024-04-05T22:26:15.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.764+0000 7f1a77e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 0x7f1a6006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:15.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.764+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a6c089860 con 0x7f1a800b1750 2024-04-05T22:26:15.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.764+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 0x7f1a6006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:15.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.765+0000 7f1a7fe00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 0x7f1a6006d420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1a70036000 tx=0x7f1a7003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:15.769+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1a6c0544f0 con 0x7f1a800b1750 2024-04-05T22:26:16.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.031+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1a80068160 con 0x7f1a800b1750 2024-04-05T22:26:16.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.040+0000 7f1a77e00640 1 -- 172.21.15.73:0/2490153690 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1a6c089020 con 0x7f1a800b1750 2024-04-05T22:26:16.039 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:26:16.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 msgr2=0x7f1a6006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 0x7f1a6006d420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1a70036000 tx=0x7f1a7003d040 comp rx=0 tx=0).stop 2024-04-05T22:26:16.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 msgr2=0x7f1a8015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c430 tx=0x7f1a6c00c8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 shutdown_connections 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1a6006af60 0x7f1a6006d420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a800b1bf0 0x7f1a8015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a800b1750 0x7f1a8015be20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 --2- 172.21.15.73:0/2490153690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a800b0720 0x7f1a80142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 >> 172.21.15.73:0/2490153690 conn(0x7f1a80099a60 msgr2=0x7f1a800aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 shutdown_connections 2024-04-05T22:26:16.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.043+0000 7f1a86e00640 1 -- 172.21.15.73:0/2490153690 wait complete. 2024-04-05T22:26:16.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:16.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.265+0000 7f580fa00640 1 Processor -- start 2024-04-05T22:26:16.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.266+0000 7f580fa00640 1 -- start start 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f580806cb80 0x7f580806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580fa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 0x7f580815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580fa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580806f5b0 con 0x7f58081587d0 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580806d500 con 0x7f580806cb80 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580fa00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580815f220 con 0x7f580815e850 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f5807e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f580806cb80 0x7f580806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580d000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f58081587d0 0x7f58080cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58090/0 (socket says 172.21.15.73:58090) 2024-04-05T22:26:16.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580d000640 1 -- 172.21.15.73:0/1989405603 learned_addr learned my addr 172.21.15.73:0/1989405603 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580da00640 1 --2- 172.21.15.73:0/1989405603 >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 0x7f580815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f5807e00640 1 -- 172.21.15.73:0/1989405603 >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 msgr2=0x7f580815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f5807e00640 1 --2- 172.21.15.73:0/1989405603 >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 0x7f580815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f5807e00640 1 -- 172.21.15.73:0/1989405603 >> v1:172.21.15.73:6789/0 conn(0x7f58081587d0 legacy=0x7f58080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f5807e00640 1 -- 172.21.15.73:0/1989405603 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808070410 con 0x7f580806cb80 2024-04-05T22:26:16.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.268+0000 7f580da00640 1 --2- 172.21.15.73:0/1989405603 >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 0x7f580815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:16.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f5807e00640 1 --2- 172.21.15.73:0/1989405603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f580806cb80 0x7f580806cfc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f580815c270 tx=0x7f57f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d0bf6aa84c2699 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:16.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f5807400640 1 -- 172.21.15.73:0/1989405603 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0031830 con 0x7f580806cb80 2024-04-05T22:26:16.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f5807400640 1 -- 172.21.15.73:0/1989405603 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0031990 con 0x7f580806cb80 2024-04-05T22:26:16.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f5807400640 1 -- 172.21.15.73:0/1989405603 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0031d20 con 0x7f580806cb80 2024-04-05T22:26:16.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f580fa00640 1 -- 172.21.15.73:0/1989405603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f580806cb80 msgr2=0x7f580806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.270+0000 7f580fa00640 1 --2- 172.21.15.73:0/1989405603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f580806cb80 0x7f580806cfc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f580815c270 tx=0x7f57f002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 -- 172.21.15.73:0/1989405603 shutdown_connections 2024-04-05T22:26:16.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 --2- 172.21.15.73:0/1989405603 >> v2:172.21.15.157:3300/0 conn(0x7f580815e850 0x7f580815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 --2- 172.21.15.73:0/1989405603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f580806cb80 0x7f580806cfc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 -- 172.21.15.73:0/1989405603 >> 172.21.15.73:0/1989405603 conn(0x7f58080ce8e0 msgr2=0x7f58080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:16.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 -- 172.21.15.73:0/1989405603 shutdown_connections 2024-04-05T22:26:16.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.271+0000 7f580fa00640 1 -- 172.21.15.73:0/1989405603 wait complete. 2024-04-05T22:26:16.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.272+0000 7f580fa00640 1 Processor -- start 2024-04-05T22:26:16.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 -- start start 2024-04-05T22:26:16.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580806cb80 0x7f58081459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580815e850 0x7f580814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580814d3a0 con 0x7f58081587d0 2024-04-05T22:26:16.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808145f00 con 0x7f580806cb80 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.273+0000 7f580fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580814c790 con 0x7f580815e850 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55864/0 (socket says 172.21.15.73:55864) 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 -- 172.21.15.73:0/1736348287 learned_addr learned my addr 172.21.15.73:0/1736348287 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 -- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580815e850 msgr2=0x7f580814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f580da00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580815e850 0x7f580814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f580d000640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580806cb80 0x7f58081459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580815e850 0x7f580814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 -- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580806cb80 msgr2=0x7f58081459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580806cb80 0x7f58081459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.274+0000 7f5807e00640 1 -- 172.21.15.73:0/1736348287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009400 con 0x7f58081587d0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f5807e00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009890 tx=0x7f57f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f003ba70 con 0x7f58081587d0 2024-04-05T22:26:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f003bbf0 con 0x7f58081587d0 2024-04-05T22:26:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0045400 con 0x7f58081587d0 2024-04-05T22:26:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580814ca70 con 0x7f58081587d0 2024-04-05T22:26:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.275+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580814d030 con 0x7f58081587d0 2024-04-05T22:26:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.276+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580814d1f0 con 0x7f58081587d0 2024-04-05T22:26:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.277+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f57f004e440 con 0x7f58081587d0 2024-04-05T22:26:16.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.278+0000 7f5804c00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 0x7f57e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:16.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.279+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f00abbc0 con 0x7f58081587d0 2024-04-05T22:26:16.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.279+0000 7f580d000640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 0x7f57e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:16.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.279+0000 7f580d000640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 0x7f57e406d450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005880 tx=0x7f57f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:16.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.283+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f57f00768d0 con 0x7f58081587d0 2024-04-05T22:26:16.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.513+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5808066af0 con 0x7f58081587d0 2024-04-05T22:26:16.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.514+0000 7f5804c00640 1 -- 172.21.15.73:0/1736348287 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f57f0035090 con 0x7f58081587d0 2024-04-05T22:26:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:16.514 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:16.514 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 41 2024-04-05T22:26:16.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 msgr2=0x7f57e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 0x7f57e406d450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005880 tx=0x7f57f8009040 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 msgr2=0x7f580814d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009890 tx=0x7f57f00046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 shutdown_connections 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f57e406af90 0x7f57e406d450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580815e850 0x7f580814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081587d0 0x7f580814d9b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 --2- 172.21.15.73:0/1736348287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580806cb80 0x7f58081459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 >> 172.21.15.73:0/1736348287 conn(0x7f58080ce8e0 msgr2=0x7f58080708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.517+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 shutdown_connections 2024-04-05T22:26:16.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:16.518+0000 7f580fa00640 1 -- 172.21.15.73:0/1736348287 wait complete. 2024-04-05T22:26:16.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:26:10.536474+0000', 'modified': '2024-04-05T22:26:13.555487+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'd', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/911489923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 911489923}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 911489923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:26:16.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:26:16.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-05T22:26:17.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.156+0000 7f25faa00640 1 Processor -- start 2024-04-05T22:26:17.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.157+0000 7f25faa00640 1 -- start start 2024-04-05T22:26:17.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.158+0000 7f25faa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25f41587d0 0x7f25f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.158+0000 7f25faa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f40ccd20 0x7f25f406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.158+0000 7f25faa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f406f5b0 con 0x7f25f406d0c0 2024-04-05T22:26:17.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.158+0000 7f25faa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f4070410 con 0x7f25f41587d0 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25faa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4158bb0 con 0x7f25f40ccd20 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25f41587d0 0x7f25f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25f41587d0 0x7f25f40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55866/0 (socket says 172.21.15.73:55866) 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 -- 172.21.15.73:0/707121089 learned_addr learned my addr 172.21.15.73:0/707121089 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3400640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f40ccd20 0x7f25f406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 -- 172.21.15.73:0/707121089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f40ccd20 msgr2=0x7f25f406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f40ccd20 0x7f25f406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 -- 172.21.15.73:0/707121089 >> v1:172.21.15.157:6789/0 conn(0x7f25f406d0c0 legacy=0x7f25f415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.159+0000 7f25f3e00640 1 -- 172.21.15.73:0/707121089 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f406f800 con 0x7f25f41587d0 2024-04-05T22:26:17.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.160+0000 7f25f3e00640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f40cc7e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009870 tx=0x7f25e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06f0d2913d92f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.161+0000 7f25f2a00640 1 -- 172.21.15.73:0/707121089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4031640 con 0x7f25f41587d0 2024-04-05T22:26:17.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.161+0000 7f25f2a00640 1 -- 172.21.15.73:0/707121089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e40317a0 con 0x7f25f41587d0 2024-04-05T22:26:17.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.161+0000 7f25f2a00640 1 -- 172.21.15.73:0/707121089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4031b30 con 0x7f25f41587d0 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 -- 172.21.15.73:0/707121089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 msgr2=0x7f25f40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f40cc7e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009870 tx=0x7f25e402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 -- 172.21.15.73:0/707121089 shutdown_connections 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f40ccd20 0x7f25f406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 --2- 172.21.15.73:0/707121089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f40cc7e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 -- 172.21.15.73:0/707121089 >> 172.21.15.73:0/707121089 conn(0x7f25f40ce8e0 msgr2=0x7f25f40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 -- 172.21.15.73:0/707121089 shutdown_connections 2024-04-05T22:26:17.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.162+0000 7f25faa00640 1 -- 172.21.15.73:0/707121089 wait complete. 2024-04-05T22:26:17.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.164+0000 7f25faa00640 1 Processor -- start 2024-04-05T22:26:17.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.164+0000 7f25faa00640 1 -- start start 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f406d0c0 0x7f25f4149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 0x7f25f415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40b1bb0 con 0x7f25f41587d0 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41546a0 con 0x7f25f4154020 2024-04-05T22:26:17.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.165+0000 7f25faa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4154980 con 0x7f25f406d0c0 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55880/0 (socket says 172.21.15.73:55880) 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 -- 172.21.15.73:0/3808641973 learned_addr learned my addr 172.21.15.73:0/3808641973 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3e00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f406d0c0 0x7f25f4149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f8a00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 0x7f25f415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 -- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f406d0c0 msgr2=0x7f25f4149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f406d0c0 0x7f25f4149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 -- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 msgr2=0x7f25f415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 0x7f25f415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 -- 172.21.15.73:0/3808641973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40094d0 con 0x7f25f41587d0 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f8a00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 0x7f25f415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:17.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.166+0000 7f25f3400640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f25e000ce10 tx=0x7f25e000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.167+0000 7f25ebe00640 1 -- 172.21.15.73:0/3808641973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0003b90 con 0x7f25f41587d0 2024-04-05T22:26:17.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.167+0000 7f25ebe00640 1 -- 172.21.15.73:0/3808641973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e0003cf0 con 0x7f25f41587d0 2024-04-05T22:26:17.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.167+0000 7f25ebe00640 1 -- 172.21.15.73:0/3808641973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0003ea0 con 0x7f25f41587d0 2024-04-05T22:26:17.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.167+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f4154c60 con 0x7f25f41587d0 2024-04-05T22:26:17.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.167+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f4149880 con 0x7f25f41587d0 2024-04-05T22:26:17.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.169+0000 7f25ebe00640 1 -- 172.21.15.73:0/3808641973 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f25e0004240 con 0x7f25f41587d0 2024-04-05T22:26:17.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.170+0000 7f25ebe00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 0x7f25d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.170+0000 7f25ebe00640 1 -- 172.21.15.73:0/3808641973 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25e0012070 con 0x7f25f41587d0 2024-04-05T22:26:17.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.170+0000 7f25f3e00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 0x7f25d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.171+0000 7f25fd7ed740 1 Processor -- start 2024-04-05T22:26:17.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.171+0000 7f25f3e00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 0x7f25d406d420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25e4036000 tx=0x7f25e403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.171+0000 7f25fd7ed740 1 -- start start 2024-04-05T22:26:17.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f20f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f17f8c0 con 0x55bc5f20f6d0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f17d050 con 0x55bc5f23c1f0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f178940 con 0x55bc5f23c5d0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55892/0 (socket says 172.21.15.73:55892) 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.172+0000 7f25f8a00640 1 -- 172.21.15.73:0/837737624 learned_addr learned my addr 172.21.15.73:0/837737624 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f3e00640 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f20f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f3400640 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 -- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 msgr2=0x55bc5f23ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 -- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 msgr2=0x55bc5f20f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f20f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 -- 172.21.15.73:0/837737624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5f1439e0 con 0x55bc5f20f6d0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25f8a00640 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f25ec003ed0 tx=0x7f25ec004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa7e0b58e857f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25e8c00640 1 -- 172.21.15.73:0/837737624 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec005230 con 0x55bc5f20f6d0 2024-04-05T22:26:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25e8c00640 1 -- 172.21.15.73:0/837737624 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec0053b0 con 0x55bc5f20f6d0 2024-04-05T22:26:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.173+0000 7f25e8c00640 1 -- 172.21.15.73:0/837737624 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00db80 con 0x55bc5f20f6d0 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 -- 172.21.15.73:0/837737624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 msgr2=0x55bc5f23f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f25ec003ed0 tx=0x7f25ec004390 comp rx=0 tx=0).stop 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 -- 172.21.15.73:0/837737624 shutdown_connections 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f20f6d0 0x55bc5f23f4c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/837737624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f20f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 -- 172.21.15.73:0/837737624 >> 172.21.15.73:0/837737624 conn(0x55bc5f211400 msgr2=0x55bc5f259f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 -- 172.21.15.73:0/837737624 shutdown_connections 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.174+0000 7f25fd7ed740 1 -- 172.21.15.73:0/837737624 wait complete. 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.175+0000 7f25fd7ed740 1 -- start start 2024-04-05T22:26:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.175+0000 7f25fd7ed740 1 Processor -- start 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f242570 0x55bc5f269dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f11a070 con 0x55bc5f23c5d0 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f119d10 con 0x55bc5f23c1f0 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc5eedff20 con 0x55bc5f242570 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25fd7ed740 10 client.0 _ll_register_callbacks cb 0x55bc5f171fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55894/0 (socket says 172.21.15.73:55894) 2024-04-05T22:26:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.176+0000 7f25f3400640 1 -- 172.21.15.73:0/2213208756 learned_addr learned my addr 172.21.15.73:0/2213208756 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 -- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f242570 msgr2=0x55bc5f269dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f242570 0x55bc5f269dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 -- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 msgr2=0x55bc5f22c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 -- 172.21.15.73:0/2213208756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec003ad0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25f3400640 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000f80 tx=0x7f25e000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918289bcdc39e86f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25dd600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e008a2b0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25e0058d50 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.177+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc5f1a2ae0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e0088100 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 10 client.6855 fetch_fsmap learned FSMap version 41 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55bc5f176fc0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc5f1a2ae0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25e001d060 con 0x55bc5f23c5d0 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 10 client.6855 fetch_fsmap finished waiting for FSMap version 41 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 10 client.6855 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 20 client.6855 populate_metadata read hostname 'smithi073' 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f275040 0x55bc5f277430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 4 client.6855 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.178+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 --> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5ee6c3c0 con 0x55bc5f275040 2024-04-05T22:26:17.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.179+0000 7f25f3e00640 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f275040 0x55bc5f277430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.180+0000 7f25f3e00640 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f275040 0x55bc5f277430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945d761836751050 server_cookie=6f3b226469b318e4 in_seq=0 out_seq=0 2024-04-05T22:26:17.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.180+0000 7f25dd600640 10 client.6855 ms_handle_connect on v2:172.21.15.157:6834/911489923 2024-04-05T22:26:17.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.181+0000 7f25dd600640 1 -- 172.21.15.73:0/2213208756 <== mds.0 v2:172.21.15.157:6834/911489923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc5ee6c3c0 con 0x55bc5f275040 2024-04-05T22:26:17.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.181+0000 7f25dd600640 10 client.6855 handle_command_reply: tid=0 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 client.6855 shutdown 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 msgr2=0x55bc5f242030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000f80 tx=0x7f25e000af70 comp rx=0 tx=0).stop 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 shutdown_connections 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f275040 0x55bc5f277430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f242570 0x55bc5f269dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f23c5d0 0x55bc5f242030 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2213208756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25dd600640 0 client.6855 ms_handle_reset on v2:172.21.15.157:6834/911489923 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 >> 172.21.15.73:0/2213208756 conn(0x55bc5f20f6d0 msgr2=0x55bc5f212fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 shutdown_connections 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2213208756 wait complete. 2024-04-05T22:26:17.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.182+0000 7f25fd7ed740 20 client.6855 trim_cache size 0 max 16384 2024-04-05T22:26:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 Processor -- start 2024-04-05T22:26:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 -- start start 2024-04-05T22:26:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f2150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f25cb40 con 0x55bc5f242570 2024-04-05T22:26:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f242ef0 con 0x55bc5f23c1f0 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.199+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f20fab0 con 0x55bc5f23c5d0 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55902/0 (socket says 172.21.15.73:55902) 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 -- 172.21.15.73:0/3974112065 learned_addr learned my addr 172.21.15.73:0/3974112065 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f3e00640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f2150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f3400640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 -- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 msgr2=0x55bc5f23d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 -- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 msgr2=0x55bc5f2150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f2150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f8a00640 1 -- 172.21.15.73:0/3974112065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5f1abf80 con 0x55bc5f242570 2024-04-05T22:26:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.200+0000 7f25f3e00640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f2150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25f8a00640 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005000 tx=0x7f25ec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ff635ae1aab0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25e9600640 1 -- 172.21.15.73:0/3974112065 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec007cc0 con 0x55bc5f242570 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25e9600640 1 -- 172.21.15.73:0/3974112065 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec021070 con 0x55bc5f242570 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25e9600640 1 -- 172.21.15.73:0/3974112065 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec005250 con 0x55bc5f242570 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 -- 172.21.15.73:0/3974112065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 msgr2=0x55bc5f23ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005000 tx=0x7f25ec010000 comp rx=0 tx=0).stop 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 -- 172.21.15.73:0/3974112065 shutdown_connections 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f23ab40 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f23d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/3974112065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f2150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.201+0000 7f25fd7ed740 1 -- 172.21.15.73:0/3974112065 >> 172.21.15.73:0/3974112065 conn(0x55bc5f20f6d0 msgr2=0x55bc5f22c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.202+0000 7f25fd7ed740 1 -- 172.21.15.73:0/3974112065 shutdown_connections 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.202+0000 7f25fd7ed740 1 -- 172.21.15.73:0/3974112065 wait complete. 2024-04-05T22:26:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.202+0000 7f25fd7ed740 1 -- start start 2024-04-05T22:26:17.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.202+0000 7f25fd7ed740 1 Processor -- start 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f2afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f22b430 con 0x55bc5f242570 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f23b980 con 0x55bc5f23c1f0 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc5f23ab40 con 0x55bc5f23c5d0 2024-04-05T22:26:17.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.203+0000 7f25fd7ed740 10 client.0 _ll_register_callbacks cb 0x7f25e0007a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55914/0 (socket says 172.21.15.73:55914) 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 -- 172.21.15.73:0/2103275559 learned_addr learned my addr 172.21.15.73:0/2103275559 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f3400640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f2afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f3e00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 -- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 msgr2=0x55bc5f2afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f2afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 -- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 msgr2=0x55bc5f22aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 -- 172.21.15.73:0/2103275559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5f1439e0 con 0x55bc5f242570 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25f8a00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0043d0 tx=0x7f25ec0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d566f6a76007680 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.204+0000 7f25dd600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec02b070 con 0x55bc5f242570 2024-04-05T22:26:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25ec00dc50 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25ec0053b0 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec007cc0 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 10 client.6861 fetch_fsmap learned FSMap version 41 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55bc5f119280 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec0053b0 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25ec007040 con 0x55bc5f242570 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 10 client.6861 fetch_fsmap finished waiting for FSMap version 41 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 10 client.6861 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.205+0000 7f25fd7ed740 20 client.6861 populate_metadata read hostname 'smithi073' 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.206+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f25a2f0 0x55bc5f25a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.206+0000 7f25fd7ed740 4 client.6861 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.206+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 --> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc5ee7d6c0 con 0x55bc5f25a2f0 2024-04-05T22:26:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.206+0000 7f25f3e00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f25a2f0 0x55bc5f25a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.207+0000 7f25f3e00640 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f25a2f0 0x55bc5f25a6d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb2ec427aa217a7 server_cookie=8fea53d782a5e1f5 in_seq=0 out_seq=0 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.207+0000 7f25dd600640 10 client.6861 ms_handle_connect on v2:172.21.15.157:6834/911489923 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25dd600640 1 -- 172.21.15.73:0/2103275559 <== mds.0 v2:172.21.15.157:6834/911489923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bc5ee7d6c0 con 0x55bc5f25a2f0 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25dd600640 10 client.6861 handle_command_reply: tid=0 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 client.6861 shutdown 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 msgr2=0x55bc5f213e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0043d0 tx=0x7f25ec0036f0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 shutdown_connections 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc5f242570 0x55bc5f213e90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:6834/911489923,v1:172.21.15.157:6835/911489923] conn(0x55bc5f25a2f0 0x55bc5f25a6d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bc5f23c5d0 0x55bc5f2afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 --2- 172.21.15.73:0/2103275559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5f23c1f0 0x55bc5f22aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25dd600640 0 client.6861 ms_handle_reset on v2:172.21.15.157:6834/911489923 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 >> 172.21.15.73:0/2103275559 conn(0x55bc5f20f6d0 msgr2=0x55bc5f20a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 shutdown_connections 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.208+0000 7f25fd7ed740 1 -- 172.21.15.73:0/2103275559 wait complete. 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25fd7ed740 20 client.6861 trim_cache size 0 max 16384 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 6699, 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.6658962080000004, 2024-04-05T22:26:17.210 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 41, 2024-04-05T22:26:17.211 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 74, 2024-04-05T22:26:17.211 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 74, 2024-04-05T22:26:17.211 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 12.579151792999999 2024-04-05T22:26:17.211 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:26:17.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 msgr2=0x7f25d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 0x7f25d406d420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25e4036000 tx=0x7f25e403c040 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 msgr2=0x7f25f4153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f25e000ce10 tx=0x7f25e000a660 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 shutdown_connections 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f25d406af60 0x7f25d406d420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f4154020 0x7f25f415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f41587d0 0x7f25f4153ae0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 --2- 172.21.15.73:0/3808641973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f406d0c0 0x7f25f4149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 >> 172.21.15.73:0/3808641973 conn(0x7f25f40ce8e0 msgr2=0x7f25f40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 shutdown_connections 2024-04-05T22:26:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.209+0000 7f25faa00640 1 -- 172.21.15.73:0/3808641973 wait complete. 2024-04-05T22:26:17.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.426+0000 7f7d29800640 1 Processor -- start 2024-04-05T22:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.426+0000 7f7d29800640 1 -- start start 2024-04-05T22:26:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.427+0000 7f7d29800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d241587d0 0x7f7d240cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.427+0000 7f7d29800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d240ccd20 0x7f7d2406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.427+0000 7f7d29800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2406f5b0 con 0x7f7d2406d0c0 2024-04-05T22:26:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.427+0000 7f7d29800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d24070410 con 0x7f7d241587d0 2024-04-05T22:26:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.427+0000 7f7d29800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24158bb0 con 0x7f7d240ccd20 2024-04-05T22:26:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.428+0000 7f7d22a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d241587d0 0x7f7d240cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.428+0000 7f7d22000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d240ccd20 0x7f7d2406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.428+0000 7f7d22a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d241587d0 0x7f7d240cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57954/0 (socket says 172.21.15.73:57954) 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.428+0000 7f7d22a00640 1 -- 172.21.15.73:0/3461023038 learned_addr learned my addr 172.21.15.73:0/3461023038 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.428+0000 7f7d23400640 1 --1- 172.21.15.73:0/3461023038 >> v1:172.21.15.157:6789/0 conn(0x7f7d2406d0c0 0x7f7d2415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60828/0 (socket says 172.21.15.73:60828) 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.429+0000 7f7d22a00640 1 -- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d240ccd20 msgr2=0x7f7d2406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.429+0000 7f7d22a00640 1 --2- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d240ccd20 0x7f7d2406cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.429+0000 7f7d22a00640 1 -- 172.21.15.73:0/3461023038 >> v1:172.21.15.157:6789/0 conn(0x7f7d2406d0c0 legacy=0x7f7d2415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.429+0000 7f7d22a00640 1 -- 172.21.15.73:0/3461023038 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2406f800 con 0x7f7d241587d0 2024-04-05T22:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.431+0000 7f7d22a00640 1 --2- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d241587d0 0x7f7d240cc7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009870 tx=0x7f7d1402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b86808dc8bb5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d21600640 1 -- 172.21.15.73:0/3461023038 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14030840 con 0x7f7d241587d0 2024-04-05T22:26:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d21600640 1 -- 172.21.15.73:0/3461023038 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d140309a0 con 0x7f7d241587d0 2024-04-05T22:26:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d21600640 1 -- 172.21.15.73:0/3461023038 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14030d30 con 0x7f7d241587d0 2024-04-05T22:26:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 -- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d241587d0 msgr2=0x7f7d240cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 --2- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d241587d0 0x7f7d240cc7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009870 tx=0x7f7d1402b980 comp rx=0 tx=0).stop 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 -- 172.21.15.73:0/3461023038 shutdown_connections 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 --2- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d240ccd20 0x7f7d2406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 --2- 172.21.15.73:0/3461023038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d241587d0 0x7f7d240cc7e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.432+0000 7f7d29800640 1 -- 172.21.15.73:0/3461023038 >> 172.21.15.73:0/3461023038 conn(0x7f7d240ce8e0 msgr2=0x7f7d240cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.433+0000 7f7d29800640 1 -- 172.21.15.73:0/3461023038 shutdown_connections 2024-04-05T22:26:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.433+0000 7f7d29800640 1 -- 172.21.15.73:0/3461023038 wait complete. 2024-04-05T22:26:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.435+0000 7f7d29800640 1 Processor -- start 2024-04-05T22:26:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.435+0000 7f7d29800640 1 -- start start 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2406d0c0 0x7f7d2415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d241587d0 0x7f7d2415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415ba50 con 0x7f7d2415b5a0 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24155370 con 0x7f7d2406d0c0 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d29800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415bc40 con 0x7f7d241587d0 2024-04-05T22:26:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55918/0 (socket says 172.21.15.73:55918) 2024-04-05T22:26:17.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.436+0000 7f7d23400640 1 -- 172.21.15.73:0/3142166883 learned_addr learned my addr 172.21.15.73:0/3142166883 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d22a00640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2406d0c0 0x7f7d2415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d22000640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d241587d0 0x7f7d2415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 -- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d241587d0 msgr2=0x7f7d2415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d241587d0 0x7f7d2415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 -- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2406d0c0 msgr2=0x7f7d2415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2406d0c0 0x7f7d2415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 -- 172.21.15.73:0/3142166883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140094d0 con 0x7f7d2415b5a0 2024-04-05T22:26:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d23400640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c780 tx=0x7f7d1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ce90 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00a640 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00a9b0 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.437+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2415bf20 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.438+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2415c4e0 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.439+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24066af0 con 0x7f7d2415b5a0 2024-04-05T22:26:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.439+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f7d1c00ab10 con 0x7f7d2415b5a0 2024-04-05T22:26:17.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.440+0000 7f7d1aa00640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 0x7f7d0406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:17.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.440+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d1c089130 con 0x7f7d2415b5a0 2024-04-05T22:26:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.441+0000 7f7d22a00640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 0x7f7d0406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.441+0000 7f7d22a00640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 0x7f7d0406d420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d14036000 tx=0x7f7d1403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:17.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.445+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7d1c057990 con 0x7f7d2415b5a0 2024-04-05T22:26:17.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.679+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d24068160 con 0x7f7d2415b5a0 2024-04-05T22:26:17.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.679+0000 7f7d1aa00640 1 -- 172.21.15.73:0/3142166883 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d1c057520 con 0x7f7d2415b5a0 2024-04-05T22:26:17.679 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:26:17.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 msgr2=0x7f7d0406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 0x7f7d0406d420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d14036000 tx=0x7f7d1403b040 comp rx=0 tx=0).stop 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 msgr2=0x7f7d2414b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c780 tx=0x7f7d1c00cc40 comp rx=0 tx=0).stop 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 shutdown_connections 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f7d0406af60 0x7f7d0406d420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2415b5a0 0x7f7d2414b160 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d241587d0 0x7f7d2415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 --2- 172.21.15.73:0/3142166883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2406d0c0 0x7f7d2415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.682+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 >> 172.21.15.73:0/3142166883 conn(0x7f7d240ce8e0 msgr2=0x7f7d240add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.683+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 shutdown_connections 2024-04-05T22:26:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:17.683+0000 7f7d29800640 1 -- 172.21.15.73:0/3142166883 wait complete. 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:26:17.699 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:17.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:17.700 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:26:17.750 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:26:17.750 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:26:17.751 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42285sec preferred_lft 42285sec 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:26:17.752 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.753 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:26:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:26:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:17.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:17.754 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:26:17.841 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:26:17.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:17.845 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:26:17.897 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:26:17.897 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:26:17.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:26:17.899 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:26:18.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:18.078 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:26:18.129 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:26:18.129 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:26:18.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:18.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:18.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:26:18.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:18.219 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:18.271 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:18.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:18.271 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:18.318 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:26:18.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:18.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:18.502 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:18.502 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:18.536 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:18.536 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:18.537 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:26:18.537 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:26:18.537 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:26:18.538 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:26:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:26:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:26:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:26:18.539 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:26:18.544 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:26:18.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:18.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:18.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:18.628 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:26:18.702 INFO:teuthology.orchestra.run.smithi073.stdout:client5006 2024-04-05T22:26:18.703 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:18.703 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:18.753 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:26:19.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.010+0000 7f9622000640 1 Processor -- start 2024-04-05T22:26:19.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.011+0000 7f9622000640 1 -- start start 2024-04-05T22:26:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.012+0000 7f9622000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f961c15d1b0 0x7f961c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.012+0000 7f9622000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c0b4d40 con 0x7f961c0b48f0 2024-04-05T22:26:19.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.012+0000 7f9622000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c15d750 con 0x7f961c15af40 2024-04-05T22:26:19.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.012+0000 7f9622000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c15da30 con 0x7f961c15d1b0 2024-04-05T22:26:19.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f9621600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f961c0b48f0 0x7f961c09a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58104/0 (socket says 172.21.15.73:58104) 2024-04-05T22:26:19.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f9621600640 1 -- 172.21.15.73:0/3973776132 learned_addr learned my addr 172.21.15.73:0/3973776132 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:19.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961be00640 1 --2- 172.21.15.73:0/3973776132 >> v2:172.21.15.157:3300/0 conn(0x7f961c15d1b0 0x7f961c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:19.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506952546 0 0) 0x7f9614001110 con 0x7f961c0b48f0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9600003520 con 0x7f961c0b48f0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183856800 0 0) 0x7f9600003520 con 0x7f961c0b48f0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 >> v2:172.21.15.157:3300/0 conn(0x7f961c15d1b0 msgr2=0x7f961c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 --2- 172.21.15.73:0/3973776132 >> v2:172.21.15.157:3300/0 conn(0x7f961c15d1b0 0x7f961c0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 >> v1:172.21.15.157:6789/0 conn(0x7f961c15af40 legacy=0x7f961c0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c15dd10 con 0x7f961c0b48f0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9614001800 con 0x7f961c0b48f0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 >> v1:172.21.15.73:6789/0 conn(0x7f961c0b48f0 legacy=0x7f961c09a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.013+0000 7f961b400640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f961b400640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f961b400640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9600005180 0x7f960000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c15da30 con 0x7f96000039a0 2024-04-05T22:26:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960000c030 con 0x7f96000047c0 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960000c1f0 con 0x7f9600005180 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 -- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9600005180 msgr2=0x7f960000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9600005180 0x7f960000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 -- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000047c0 msgr2=0x7f9600004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9620c00640 1 -- 172.21.15.73:0/3973776132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600003520 con 0x7f96000039a0 2024-04-05T22:26:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.014+0000 7f9621600640 1 -- 172.21.15.73:0/3973776132 reap_dead start 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.015+0000 7f9620c00640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9604001750 tx=0x7f96000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd002d23392a0c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040052d0 con 0x7f96000039a0 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960402e050 con 0x7f96000039a0 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f961b400640 1 -- 172.21.15.73:0/3973776132 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604003830 con 0x7f96000039a0 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 -- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 msgr2=0x7f9600003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9604001750 tx=0x7f96000057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 -- 172.21.15.73:0/3973776132 shutdown_connections 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 --2- 172.21.15.73:0/3973776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 -- 172.21.15.73:0/3973776132 >> 172.21.15.73:0/3973776132 conn(0x7f961c0b5d50 msgr2=0x7f961c0b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 -- 172.21.15.73:0/3973776132 shutdown_connections 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.016+0000 7f9622000640 1 -- 172.21.15.73:0/3973776132 wait complete. 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.018+0000 7f9622000640 1 Processor -- start 2024-04-05T22:26:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.018+0000 7f9622000640 1 -- start start 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000039a0 0x7f961c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f961c06adc0 0x7f961c06b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1519d0 con 0x7f961c14f530 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c14fef0 con 0x7f96000039a0 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f9622000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c06c5e0 con 0x7f961c06adc0 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.019+0000 7f961be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55928/0 (socket says 172.21.15.73:55928) 2024-04-05T22:26:19.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 -- 172.21.15.73:0/2184336517 learned_addr learned my addr 172.21.15.73:0/2184336517 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 -- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f961c06adc0 msgr2=0x7f961c06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f961c06adc0 0x7f961c06b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 -- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000039a0 msgr2=0x7f961c152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000039a0 0x7f961c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 -- 172.21.15.73:0/2184336517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c06b780 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f961be00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f960c00ef40 tx=0x7f960c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.020+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c009e80 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.021+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c00f040 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.021+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0146a0 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.021+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c06b9d0 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.021+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c06bf90 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.022+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c04e160 con 0x7f961c14f530 2024-04-05T22:26:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.023+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f960c014800 con 0x7f961c14f530 2024-04-05T22:26:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.023+0000 7f9618c00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 0x7f95f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.024+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f960c08a5e0 con 0x7f961c14f530 2024-04-05T22:26:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.024+0000 7f9620c00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 0x7f95f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.024+0000 7f9620c00640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 0x7f95f806d450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9604033000 tx=0x7f960402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:19.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.028+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f960c055260 con 0x7f961c14f530 2024-04-05T22:26:19.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.260+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f961c0d1760 con 0x7f961c14f530 2024-04-05T22:26:19.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.261+0000 7f9618c00640 1 -- 172.21.15.73:0/2184336517 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f960c058880 con 0x7f961c14f530 2024-04-05T22:26:19.260 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 msgr2=0x7f95f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 0x7f95f806d450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9604033000 tx=0x7f960402f000 comp rx=0 tx=0).stop 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 msgr2=0x7f961c14f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f960c00ef40 tx=0x7f960c00c490 comp rx=0 tx=0).stop 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 shutdown_connections 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f95f806af90 0x7f95f806d450 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f961c06adc0 0x7f961c06b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f961c14f530 0x7f961c14f9b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 --2- 172.21.15.73:0/2184336517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000039a0 0x7f961c152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 >> 172.21.15.73:0/2184336517 conn(0x7f961c0b5d50 msgr2=0x7f961c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 shutdown_connections 2024-04-05T22:26:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:19.264+0000 7f9622000640 1 -- 172.21.15.73:0/2184336517 wait complete. 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:26:19.283 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:19.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.284 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:19.344 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.345 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:26:19.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:19.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.347 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:26:19.438 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:26:19.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.443 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:26:19.489 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:26:19.490 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:26:19.490 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:26:19.490 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:26:19.490 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:26:19.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:26:19.491 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:26:19.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.672 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:26:19.721 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:26:19.721 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:26:19.722 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:19.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:26:19.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:19.813 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:19.862 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:19.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:19.862 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:19.948 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:26:19.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:19.949 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:20.119 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:20.119 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:20.152 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:20.152 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:20.152 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:26:20.152 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:26:20.152 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:26:20.154 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:26:20.154 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:26:20.154 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:26:20.154 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:26:20.154 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:26:20.159 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:26:20.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:20.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:20.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:20.232 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:26:20.282 INFO:teuthology.orchestra.run.smithi157.stdout:client5009 2024-04-05T22:26:20.283 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:20.283 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:20.370 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:26:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.594+0000 7f6373800640 1 Processor -- start 2024-04-05T22:26:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.595+0000 7f6373800640 1 -- start start 2024-04-05T22:26:20.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.596+0000 7f6373800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.596+0000 7f6373800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f636c0b2f50 0x7f636c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.596+0000 7f6373800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c15d900 con 0x7f636c154580 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.596+0000 7f6373800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636c15e1a0 con 0x7f636c0b2f50 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.596+0000 7f6373800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0b3490 con 0x7f636c1541a0 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.597+0000 7f636be00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f636c154580 0x7f636c0b2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:60846/0 (socket says 172.21.15.73:60846) 2024-04-05T22:26:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.597+0000 7f636be00640 1 -- 172.21.15.73:0/3452977631 learned_addr learned my addr 172.21.15.73:0/3452977631 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.597+0000 7f6370e00640 1 --2- 172.21.15.73:0/3452977631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.597+0000 7f6371800640 1 --2- 172.21.15.73:0/3452977631 >> v2:172.21.15.157:3300/0 conn(0x7f636c0b2f50 0x7f636c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.598+0000 7f6370e00640 1 -- 172.21.15.73:0/3452977631 >> v2:172.21.15.157:3300/0 conn(0x7f636c0b2f50 msgr2=0x7f636c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.598+0000 7f6370e00640 1 --2- 172.21.15.73:0/3452977631 >> v2:172.21.15.157:3300/0 conn(0x7f636c0b2f50 0x7f636c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.598+0000 7f6370e00640 1 -- 172.21.15.73:0/3452977631 >> v1:172.21.15.157:6789/0 conn(0x7f636c154580 legacy=0x7f636c0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.598+0000 7f6370e00640 1 -- 172.21.15.73:0/3452977631 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c15db50 con 0x7f636c1541a0 2024-04-05T22:26:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.599+0000 7f6370e00640 1 --2- 172.21.15.73:0/3452977631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c0cecd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f635c009d90 tx=0x7f635c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8354797430ca68f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.600+0000 7f636b400640 1 -- 172.21.15.73:0/3452977631 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c02cc50 con 0x7f636c1541a0 2024-04-05T22:26:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.600+0000 7f636b400640 1 -- 172.21.15.73:0/3452977631 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c02cdb0 con 0x7f636c1541a0 2024-04-05T22:26:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.600+0000 7f636b400640 1 -- 172.21.15.73:0/3452977631 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c034660 con 0x7f636c1541a0 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 -- 172.21.15.73:0/3452977631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 msgr2=0x7f636c0cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 --2- 172.21.15.73:0/3452977631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c0cecd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f635c009d90 tx=0x7f635c02bea0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 -- 172.21.15.73:0/3452977631 shutdown_connections 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 --2- 172.21.15.73:0/3452977631 >> v2:172.21.15.157:3300/0 conn(0x7f636c0b2f50 0x7f636c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 --2- 172.21.15.73:0/3452977631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c0cecd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 -- 172.21.15.73:0/3452977631 >> 172.21.15.73:0/3452977631 conn(0x7f636c06a0d0 msgr2=0x7f636c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 -- 172.21.15.73:0/3452977631 shutdown_connections 2024-04-05T22:26:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.601+0000 7f6373800640 1 -- 172.21.15.73:0/3452977631 wait complete. 2024-04-05T22:26:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.603+0000 7f6373800640 1 Processor -- start 2024-04-05T22:26:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.603+0000 7f6373800640 1 -- start start 2024-04-05T22:26:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f636c154580 0x7f636c149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c14b430 con 0x7f636c0b2f50 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c14a820 con 0x7f636c154580 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.604+0000 7f6373800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c14ab00 con 0x7f636c1541a0 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55938/0 (socket says 172.21.15.73:55938) 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 -- 172.21.15.73:0/4265362684 learned_addr learned my addr 172.21.15.73:0/4265362684 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6371800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f636c154580 0x7f636c149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f636be00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 -- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 msgr2=0x7f636c14b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c14b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 -- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f636c154580 msgr2=0x7f636c149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f636c154580 0x7f636c149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f6370e00640 1 -- 172.21.15.73:0/4265362684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c009a70 con 0x7f636c0b2f50 2024-04-05T22:26:20.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.605+0000 7f636be00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:20.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6370e00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f635c009d90 tx=0x7f635c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:20.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c040650 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c0407b0 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c040b60 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c14ade0 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.606+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c149b80 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.607+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c0b3b50 con 0x7f636c0b2f50 2024-04-05T22:26:20.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.608+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f635c040cc0 con 0x7f636c0b2f50 2024-04-05T22:26:20.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.609+0000 7f6368c00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 0x7f634806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:20.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.609+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f635c0ac750 con 0x7f636c0b2f50 2024-04-05T22:26:20.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.609+0000 7f636be00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 0x7f634806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:20.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.610+0000 7f636be00640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 0x7f634806d420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f636c15b110 tx=0x7f6354009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:20.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.614+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f635c0776b0 con 0x7f636c0b2f50 2024-04-05T22:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:20.851+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f636c068160 con 0x7f636c0b2f50 2024-04-05T22:26:21.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.658+0000 7f6368c00640 1 -- 172.21.15.73:0/4265362684 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 136+0+0 (secure 0 0 0) 0x7f635c07acd0 con 0x7f636c0b2f50 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 msgr2=0x7f634806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 0x7f634806d420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f636c15b110 tx=0x7f6354009040 comp rx=0 tx=0).stop 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 msgr2=0x7f636c149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f635c009d90 tx=0x7f635c004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 shutdown_connections 2024-04-05T22:26:21.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f634806af60 0x7f634806d420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f636c154580 0x7f636c149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f636c1541a0 0x7f636c14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 --2- 172.21.15.73:0/4265362684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f636c0b2f50 0x7f636c149640 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 >> 172.21.15.73:0/4265362684 conn(0x7f636c06a0d0 msgr2=0x7f636c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 shutdown_connections 2024-04-05T22:26:21.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.663+0000 7f6373800640 1 -- 172.21.15.73:0/4265362684 wait complete. 2024-04-05T22:26:21.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:26:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.890+0000 7f1ff9e00640 1 Processor -- start 2024-04-05T22:26:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.890+0000 7f1ff9e00640 1 -- start start 2024-04-05T22:26:21.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff41587d0 0x7f1ff40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ff415e850 0x7f1ff415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff9e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff406f5b0 con 0x7f1ff40ccd20 2024-04-05T22:26:21.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff9e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff415f240 con 0x7f1ff415e850 2024-04-05T22:26:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4070410 con 0x7f1ff41587d0 2024-04-05T22:26:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff2000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ff40ccd20 0x7f1ff406cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58106/0 (socket says 172.21.15.73:58106) 2024-04-05T22:26:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.892+0000 7f1ff2000640 1 -- 172.21.15.73:0/2878116522 learned_addr learned my addr 172.21.15.73:0/2878116522 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff3400640 1 --2- 172.21.15.73:0/2878116522 >> v2:172.21.15.73:3300/0 conn(0x7f1ff415e850 0x7f1ff415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff2a00640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff41587d0 0x7f1ff40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff1600640 1 -- 172.21.15.73:0/2878116522 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462450560 0 0) 0x7f1ff406f5b0 con 0x7f1ff40ccd20 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff1600640 1 -- 172.21.15.73:0/2878116522 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd8003520 con 0x7f1ff40ccd20 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff3400640 1 -- 172.21.15.73:0/2878116522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff41587d0 msgr2=0x7f1ff40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff3400640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff41587d0 0x7f1ff40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff3400640 1 -- 172.21.15.73:0/2878116522 >> v1:172.21.15.73:6789/0 conn(0x7f1ff40ccd20 legacy=0x7f1ff406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.893+0000 7f1ff3400640 1 -- 172.21.15.73:0/2878116522 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4158bb0 con 0x7f1ff415e850 2024-04-05T22:26:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.895+0000 7f1ff3400640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff415e850 0x7f1ff415ed00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009d90 tx=0x7f1fec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcbdf764449007e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:21.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.895+0000 7f1ff1600640 1 -- 172.21.15.73:0/2878116522 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec02cc50 con 0x7f1ff415e850 2024-04-05T22:26:21.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.895+0000 7f1ff1600640 1 -- 172.21.15.73:0/2878116522 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec02cdb0 con 0x7f1ff415e850 2024-04-05T22:26:21.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.895+0000 7f1ff1600640 1 -- 172.21.15.73:0/2878116522 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec034640 con 0x7f1ff415e850 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 -- 172.21.15.73:0/2878116522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff415e850 msgr2=0x7f1ff415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff415e850 0x7f1ff415ed00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009d90 tx=0x7f1fec02bea0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 -- 172.21.15.73:0/2878116522 shutdown_connections 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff415e850 0x7f1ff415ed00 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/2878116522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff41587d0 0x7f1ff40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 -- 172.21.15.73:0/2878116522 >> 172.21.15.73:0/2878116522 conn(0x7f1ff40ce8e0 msgr2=0x7f1ff40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 -- 172.21.15.73:0/2878116522 shutdown_connections 2024-04-05T22:26:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.896+0000 7f1ff9e00640 1 -- 172.21.15.73:0/2878116522 wait complete. 2024-04-05T22:26:21.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.898+0000 7f1ff9e00640 1 Processor -- start 2024-04-05T22:26:21.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.898+0000 7f1ff9e00640 1 -- start start 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ff41587d0 0x7f1ff40b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 0x7f1ff40c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40b1f30 con 0x7f1ff40b1970 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40b2210 con 0x7f1ff41587d0 2024-04-05T22:26:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40b24f0 con 0x7f1ff415e850 2024-04-05T22:26:21.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55954/0 (socket says 172.21.15.73:55954) 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff3400640 1 -- 172.21.15.73:0/1928949602 learned_addr learned my addr 172.21.15.73:0/1928949602 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.899+0000 7f1ff2a00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ff41587d0 0x7f1ff40b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff2000640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 0x7f1ff40c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 -- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 msgr2=0x7f1ff40c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 0x7f1ff40c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 -- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ff41587d0 msgr2=0x7f1ff40b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ff41587d0 0x7f1ff40b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 -- 172.21.15.73:0/1928949602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009a70 con 0x7f1ff40b1970 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff2000640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 0x7f1ff40c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff3400640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009eb0 tx=0x7f1fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03e610 con 0x7f1ff40b1970 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec03e770 con 0x7f1ff40b1970 2024-04-05T22:26:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03eb60 con 0x7f1ff40b1970 2024-04-05T22:26:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff40b27d0 con 0x7f1ff40b1970 2024-04-05T22:26:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.900+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4159820 con 0x7f1ff40b1970 2024-04-05T22:26:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.902+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff4066af0 con 0x7f1ff40b1970 2024-04-05T22:26:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.903+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f1fec03b050 con 0x7f1ff40b1970 2024-04-05T22:26:21.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.904+0000 7f1feb400640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 0x7f1fd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:21.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.904+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fec0abaf0 con 0x7f1ff40b1970 2024-04-05T22:26:21.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.904+0000 7f1ff2a00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 0x7f1fd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:21.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.905+0000 7f1ff2a00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 0x7f1fd406d420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4157ad0 tx=0x7f1fe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:21.909+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1fec076970 con 0x7f1ff40b1970 2024-04-05T22:26:22.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.130+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ff40b2380 con 0x7f1fd406af60 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.132+0000 7f1feb400640 1 -- 172.21.15.73:0/1928949602 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1ff40b2380 con 0x7f1fd406af60 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:22.132 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:26:22.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 msgr2=0x7f1fd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 0x7f1fd406d420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4157ad0 tx=0x7f1fe4008040 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 msgr2=0x7f1ff415b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009eb0 tx=0x7f1fec004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 shutdown_connections 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f1fd406af60 0x7f1fd406d420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ff40b1970 0x7f1ff415b210 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ff415e850 0x7f1ff40c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 --2- 172.21.15.73:0/1928949602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ff41587d0 0x7f1ff40b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 >> 172.21.15.73:0/1928949602 conn(0x7f1ff40ce8e0 msgr2=0x7f1ff40cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.135+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 shutdown_connections 2024-04-05T22:26:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.136+0000 7f1ff9e00640 1 -- 172.21.15.73:0/1928949602 wait complete. 2024-04-05T22:26:22.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.361+0000 7fa2f0a00640 1 Processor -- start 2024-04-05T22:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.361+0000 7fa2f0a00640 1 -- start start 2024-04-05T22:26:22.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa2ec15aa70 0x7fa2ec070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec0714a0 0x7fa2ec15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2ec15dbc0 0x7fa2ec1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec071900 con 0x7fa2ec15aa70 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec15ae50 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2f0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15e3b0 con 0x7fa2ec0714a0 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.363+0000 7fa2e9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa2ec15aa70 0x7fa2ec070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa2ec15aa70 0x7fa2ec070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55958/0 (socket says 172.21.15.73:55958) 2024-04-05T22:26:22.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 -- 172.21.15.73:0/1444289498 learned_addr learned my addr 172.21.15.73:0/1444289498 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 -- 172.21.15.73:0/1444289498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec0714a0 msgr2=0x7fa2ec15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/1444289498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec0714a0 0x7fa2ec15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 -- 172.21.15.73:0/1444289498 >> v2:172.21.15.157:3300/0 conn(0x7fa2ec15dbc0 msgr2=0x7fa2ec1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/1444289498 >> v2:172.21.15.157:3300/0 conn(0x7fa2ec15dbc0 0x7fa2ec1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.364+0000 7fa2e9e00640 1 -- 172.21.15.73:0/1444289498 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0ac9e0 con 0x7fa2ec15aa70 2024-04-05T22:26:22.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.365+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/1444289498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15aa70 0x7fa2ec070f60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec06d880 tx=0x7fa2dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd615b442ffbe09 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.366+0000 7fa2e8a00640 1 -- 172.21.15.73:0/1444289498 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc02ca80 con 0x7fa2ec15aa70 2024-04-05T22:26:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.366+0000 7fa2e8a00640 1 -- 172.21.15.73:0/1444289498 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc02cbe0 con 0x7fa2ec15aa70 2024-04-05T22:26:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.366+0000 7fa2e8a00640 1 -- 172.21.15.73:0/1444289498 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0343f0 con 0x7fa2ec15aa70 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.366+0000 7fa2f0a00640 1 -- 172.21.15.73:0/1444289498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15aa70 msgr2=0x7fa2ec070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.366+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/1444289498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15aa70 0x7fa2ec070f60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec06d880 tx=0x7fa2dc02bee0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 -- 172.21.15.73:0/1444289498 shutdown_connections 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/1444289498 >> v2:172.21.15.157:3300/0 conn(0x7fa2ec15dbc0 0x7fa2ec1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/1444289498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec0714a0 0x7fa2ec15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/1444289498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15aa70 0x7fa2ec070f60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 -- 172.21.15.73:0/1444289498 >> 172.21.15.73:0/1444289498 conn(0x7fa2ec069cb0 msgr2=0x7fa2ec06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 -- 172.21.15.73:0/1444289498 shutdown_connections 2024-04-05T22:26:22.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.367+0000 7fa2f0a00640 1 -- 172.21.15.73:0/1444289498 wait complete. 2024-04-05T22:26:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.368+0000 7fa2f0a00640 1 Processor -- start 2024-04-05T22:26:22.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.369+0000 7fa2f0a00640 1 -- start start 2024-04-05T22:26:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.369+0000 7fa2f0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2ec0714a0 0x7fa2ec1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.369+0000 7fa2f0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 0x7fa2ec1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.369+0000 7fa2f0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.369+0000 7fa2f0a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0af500 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2f0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0af710 con 0x7fa2ec0714a0 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2f0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0af9f0 con 0x7fa2ec15aa70 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55968/0 (socket says 172.21.15.73:55968) 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 -- 172.21.15.73:0/4129910797 learned_addr learned my addr 172.21.15.73:0/4129910797 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2e9400640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 0x7fa2ec1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2ec0714a0 0x7fa2ec1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 -- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 msgr2=0x7fa2ec1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 0x7fa2ec1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 -- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2ec0714a0 msgr2=0x7fa2ec1f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2ec0714a0 0x7fa2ec1f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2ea800640 1 -- 172.21.15.73:0/4129910797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc009b30 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.370+0000 7fa2e9400640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 0x7fa2ec1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2ea800640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009fd0 tx=0x7fa2e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4011830 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4011990 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4011d20 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec0afcd0 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.371+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec0b0290 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.372+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec068160 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.373+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fa2e401b460 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.374+0000 7fa2e2000640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 0x7fa2c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.374+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2e4089640 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.374+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 0x7fa2c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.375+0000 7fa2e9e00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 0x7fa2c806d420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc035000 tx=0x7fa2dc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.378+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa2e4057ef0 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.560+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2ec0ae1c0 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.561+0000 7fa2e2000640 1 -- 172.21.15.73:0/4129910797 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2e4057a80 con 0x7fa2ec15dbc0 2024-04-05T22:26:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.563+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 msgr2=0x7fa2c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.563+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 0x7fa2c806d420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc035000 tx=0x7fa2dc030000 comp rx=0 tx=0).stop 2024-04-05T22:26:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.563+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 msgr2=0x7fa2ec06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.563+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009fd0 tx=0x7fa2e400b8e0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 shutdown_connections 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa2c806af60 0x7fa2c806d420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2ec15dbc0 0x7fa2ec06ba40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2ec15aa70 0x7fa2ec1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 --2- 172.21.15.73:0/4129910797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2ec0714a0 0x7fa2ec1f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 >> 172.21.15.73:0/4129910797 conn(0x7fa2ec069cb0 msgr2=0x7fa2ec070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 shutdown_connections 2024-04-05T22:26:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.564+0000 7fa2f0a00640 1 -- 172.21.15.73:0/4129910797 wait complete. 2024-04-05T22:26:22.581 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:22.581 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:22.635 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:22.635 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:22.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.931+0000 7fa053200640 1 Processor -- start 2024-04-05T22:26:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.931+0000 7fa053200640 1 -- start start 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa053200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa04c1587d0 0x7fa04c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa053200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa04c15e850 0x7fa04c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa053200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04c06f5b0 con 0x7fa04c0ccd20 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa053200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c15f240 con 0x7fa04c1587d0 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa053200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c070410 con 0x7fa04c15e850 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa04be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa04c1587d0 0x7fa04c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa04be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa04c1587d0 0x7fa04c0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55978/0 (socket says 172.21.15.73:55978) 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.933+0000 7fa04be00640 1 -- 172.21.15.73:0/1981468372 learned_addr learned my addr 172.21.15.73:0/1981468372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.934+0000 7fa051200640 1 --2- 172.21.15.73:0/1981468372 >> v2:172.21.15.157:3300/0 conn(0x7fa04c15e850 0x7fa04c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.934+0000 7fa04be00640 1 -- 172.21.15.73:0/1981468372 >> v2:172.21.15.157:3300/0 conn(0x7fa04c15e850 msgr2=0x7fa04c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.934+0000 7fa04be00640 1 --2- 172.21.15.73:0/1981468372 >> v2:172.21.15.157:3300/0 conn(0x7fa04c15e850 0x7fa04c15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.934+0000 7fa04be00640 1 -- 172.21.15.73:0/1981468372 >> v1:172.21.15.157:6789/0 conn(0x7fa04c0ccd20 legacy=0x7fa04c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.934+0000 7fa04be00640 1 -- 172.21.15.73:0/1981468372 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c158bb0 con 0x7fa04c1587d0 2024-04-05T22:26:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.935+0000 7fa04be00640 1 --2- 172.21.15.73:0/1981468372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c1587d0 0x7fa04c0cc7e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009870 tx=0x7fa03c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8f294e599f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa04aa00640 1 -- 172.21.15.73:0/1981468372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c031870 con 0x7fa04c1587d0 2024-04-05T22:26:22.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa04aa00640 1 -- 172.21.15.73:0/1981468372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c0319d0 con 0x7fa04c1587d0 2024-04-05T22:26:22.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa04aa00640 1 -- 172.21.15.73:0/1981468372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c031d60 con 0x7fa04c1587d0 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa053200640 1 -- 172.21.15.73:0/1981468372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c1587d0 msgr2=0x7fa04c0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa053200640 1 --2- 172.21.15.73:0/1981468372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c1587d0 0x7fa04c0cc7e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009870 tx=0x7fa03c02b910 comp rx=0 tx=0).stop 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa053200640 1 -- 172.21.15.73:0/1981468372 shutdown_connections 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa053200640 1 --2- 172.21.15.73:0/1981468372 >> v2:172.21.15.157:3300/0 conn(0x7fa04c15e850 0x7fa04c15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.936+0000 7fa053200640 1 --2- 172.21.15.73:0/1981468372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c1587d0 0x7fa04c0cc7e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.937+0000 7fa053200640 1 -- 172.21.15.73:0/1981468372 >> 172.21.15.73:0/1981468372 conn(0x7fa04c0ce8e0 msgr2=0x7fa04c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.937+0000 7fa053200640 1 -- 172.21.15.73:0/1981468372 shutdown_connections 2024-04-05T22:26:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.937+0000 7fa053200640 1 -- 172.21.15.73:0/1981468372 wait complete. 2024-04-05T22:26:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.938+0000 7fa053200640 1 Processor -- start 2024-04-05T22:26:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.938+0000 7fa053200640 1 -- start start 2024-04-05T22:26:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa04c1587d0 0x7fa04c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04c15e850 0x7fa04c0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0cb700 con 0x7fa04c0aff40 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0c3880 con 0x7fa04c15e850 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa053200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0c3b60 con 0x7fa04c1587d0 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.939+0000 7fa051200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55984/0 (socket says 172.21.15.73:55984) 2024-04-05T22:26:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 -- 172.21.15.73:0/1083392544 learned_addr learned my addr 172.21.15.73:0/1083392544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 -- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa04c1587d0 msgr2=0x7fa04c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa04c1587d0 0x7fa04c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 -- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04c15e850 msgr2=0x7fa04c0c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04c15e850 0x7fa04c0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 -- 172.21.15.73:0/1083392544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c0094d0 con 0x7fa04c0aff40 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa051200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa044007e60 tx=0x7fa04400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044014420 con 0x7fa04c0aff40 2024-04-05T22:26:22.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440145a0 con 0x7fa04c0aff40 2024-04-05T22:26:22.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044014950 con 0x7fa04c0aff40 2024-04-05T22:26:22.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.940+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c0c3e40 con 0x7fa04c0aff40 2024-04-05T22:26:22.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.941+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c0c4400 con 0x7fa04c0aff40 2024-04-05T22:26:22.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.942+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c066af0 con 0x7fa04c0aff40 2024-04-05T22:26:22.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.943+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fa044014ab0 con 0x7fa04c0aff40 2024-04-05T22:26:22.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.944+0000 7fa043e00640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 0x7fa02c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:22.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.944+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa04408a1e0 con 0x7fa04c0aff40 2024-04-05T22:26:22.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.944+0000 7fa04be00640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 0x7fa02c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:22.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.945+0000 7fa04be00640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 0x7fa02c06d420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa03c036000 tx=0x7fa03c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:22.948+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa044054f70 con 0x7fa04c0aff40 2024-04-05T22:26:23.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.128+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa04c0c4590 con 0x7fa04c0aff40 2024-04-05T22:26:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.129+0000 7fa043e00640 1 -- 172.21.15.73:0/1083392544 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16100 (secure 0 0 0) 0x7fa044058590 con 0x7fa04c0aff40 2024-04-05T22:26:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:23.130 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:11.541344+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 msgr2=0x7fa02c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 0x7fa02c06d420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa03c036000 tx=0x7fa03c030000 comp rx=0 tx=0).stop 2024-04-05T22:26:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 msgr2=0x7fa04c0c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa044007e60 tx=0x7fa04400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 shutdown_connections 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fa02c06af60 0x7fa02c06d420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0aff40 0x7fa04c0c54f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04c15e850 0x7fa04c0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 --2- 172.21.15.73:0/1083392544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa04c1587d0 0x7fa04c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 >> 172.21.15.73:0/1083392544 conn(0x7fa04c0ce8e0 msgr2=0x7fa04c0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 shutdown_connections 2024-04-05T22:26:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.132+0000 7fa053200640 1 -- 172.21.15.73:0/1083392544 wait complete. 2024-04-05T22:26:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:08.439249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:09.543608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:23.152 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.361+0000 7f4343200640 1 Processor -- start 2024-04-05T22:26:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.361+0000 7f4343200640 1 -- start start 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f433c15a850 0x7f433c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 0x7f433c06ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c15f530 con 0x7f433c15a850 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c06bc50 con 0x7f433c15e580 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4343200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15ef70 con 0x7f433c15e9d0 2024-04-05T22:26:23.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4341e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f433c15a850 0x7f433c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4341e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f433c15a850 0x7f433c15ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55996/0 (socket says 172.21.15.73:55996) 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.363+0000 7f4341e00640 1 -- 172.21.15.73:0/3077376268 learned_addr learned my addr 172.21.15.73:0/3077376268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341e00640 1 -- 172.21.15.73:0/3077376268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 msgr2=0x7f433c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4342800640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341400640 1 --2- 172.21.15.73:0/3077376268 >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 0x7f433c06ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341e00640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341e00640 1 -- 172.21.15.73:0/3077376268 >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 msgr2=0x7f433c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341e00640 1 --2- 172.21.15.73:0/3077376268 >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 0x7f433c06ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341e00640 1 -- 172.21.15.73:0/3077376268 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c15f250 con 0x7f433c15a850 2024-04-05T22:26:23.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.364+0000 7f4341400640 1 --2- 172.21.15.73:0/3077376268 >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 0x7f433c06ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.365+0000 7f4341e00640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c15ac30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f433c06be20 tx=0x7f433002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24a86b80c7ed371 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.365+0000 7f4340a00640 1 -- 172.21.15.73:0/3077376268 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300317c0 con 0x7f433c15a850 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.365+0000 7f4340a00640 1 -- 172.21.15.73:0/3077376268 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330031920 con 0x7f433c15a850 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.365+0000 7f4340a00640 1 -- 172.21.15.73:0/3077376268 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330031cb0 con 0x7f433c15a850 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 -- 172.21.15.73:0/3077376268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 msgr2=0x7f433c15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c15ac30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f433c06be20 tx=0x7f433002b860 comp rx=0 tx=0).stop 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 -- 172.21.15.73:0/3077376268 shutdown_connections 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 --2- 172.21.15.73:0/3077376268 >> v2:172.21.15.157:3300/0 conn(0x7f433c15e580 0x7f433c06ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 --2- 172.21.15.73:0/3077376268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c15ac30 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 -- 172.21.15.73:0/3077376268 >> 172.21.15.73:0/3077376268 conn(0x7f433c069cb0 msgr2=0x7f433c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 -- 172.21.15.73:0/3077376268 shutdown_connections 2024-04-05T22:26:23.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.366+0000 7f4343200640 1 -- 172.21.15.73:0/3077376268 wait complete. 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.367+0000 7f4343200640 1 Processor -- start 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.368+0000 7f4343200640 1 -- start start 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c15e580 0x7f433c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c14f4b0 con 0x7f433c15a850 2024-04-05T22:26:23.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c156bf0 con 0x7f433c15e580 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4343200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c14eef0 con 0x7f433c15e9d0 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55998/0 (socket says 172.21.15.73:55998) 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 -- 172.21.15.73:0/2642850052 learned_addr learned my addr 172.21.15.73:0/2642850052 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 -- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 msgr2=0x7f433c14e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341400640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c15e580 0x7f433c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 -- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c15e580 msgr2=0x7f433c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c15e580 0x7f433c14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.369+0000 7f4341e00640 1 -- 172.21.15.73:0/2642850052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300094d0 con 0x7f433c15a850 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4341e00640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c00 tx=0x7f43300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300328e0 con 0x7f433c15a850 2024-04-05T22:26:23.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330032a80 con 0x7f433c15a850 2024-04-05T22:26:23.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c14f1d0 con 0x7f433c15a850 2024-04-05T22:26:23.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330032e30 con 0x7f433c15a850 2024-04-05T22:26:23.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.370+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c0ae6b0 con 0x7f433c15a850 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.371+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c1566e0 con 0x7f433c15a850 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.372+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f433003b3f0 con 0x7f433c15a850 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.373+0000 7f4336000640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 0x7f432006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.373+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43300abe00 con 0x7f433c15a850 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.373+0000 7f4341400640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 0x7f432006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.374+0000 7f4341400640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 0x7f432006d420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f433c14ba90 tx=0x7f432c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.377+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f433c1566e0 con 0x7f433c15a850 2024-04-05T22:26:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.566+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f433c1566e0 con 0x7f433c15a850 2024-04-05T22:26:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.567+0000 7f4336000640 1 -- 172.21.15.73:0/2642850052 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f433c1566e0 con 0x7f433c15a850 2024-04-05T22:26:23.568 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 msgr2=0x7f432006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 0x7f432006d420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f433c14ba90 tx=0x7f432c008040 comp rx=0 tx=0).stop 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 msgr2=0x7f433c14c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c00 tx=0x7f43300046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 shutdown_connections 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f432006af60 0x7f432006d420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c15e9d0 0x7f433c14e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c15e580 0x7f433c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 --2- 172.21.15.73:0/2642850052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c15a850 0x7f433c14c760 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 >> 172.21.15.73:0/2642850052 conn(0x7f433c069cb0 msgr2=0x7f433c0add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 shutdown_connections 2024-04-05T22:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.570+0000 7f4343200640 1 -- 172.21.15.73:0/2642850052 wait complete. 2024-04-05T22:26:23.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:26:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.796+0000 7f254f200640 1 Processor -- start 2024-04-05T22:26:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.796+0000 7f254f200640 1 -- start start 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f254f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f255015e0d0 0x7f25500cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f254f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 0x7f25500cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f254f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500715a0 con 0x7f2550154090 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f254f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2550071880 con 0x7f255015e0d0 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f254f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255015aea0 con 0x7f2550071010 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f2545400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f255015e0d0 0x7f25500cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f2545400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f255015e0d0 0x7f25500cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56014/0 (socket says 172.21.15.73:56014) 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.798+0000 7f2545400640 1 -- 172.21.15.73:0/2774285524 learned_addr learned my addr 172.21.15.73:0/2774285524 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f254e800640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 0x7f25500cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f2545400640 1 -- 172.21.15.73:0/2774285524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 msgr2=0x7f25500cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f2545400640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 0x7f25500cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f2545400640 1 -- 172.21.15.73:0/2774285524 >> v1:172.21.15.157:6789/0 conn(0x7f2550154090 legacy=0x7f255006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f2545400640 1 -- 172.21.15.73:0/2774285524 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255015d5f0 con 0x7f255015e0d0 2024-04-05T22:26:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.799+0000 7f254e800640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 0x7f25500cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.800+0000 7f2545400640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255015e0d0 0x7f25500cb0e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f25500cd730 tx=0x7f253802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d089dafcd496e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.800+0000 7f254d400640 1 -- 172.21.15.73:0/2774285524 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253802e070 con 0x7f255015e0d0 2024-04-05T22:26:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.800+0000 7f254d400640 1 -- 172.21.15.73:0/2774285524 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25380315a0 con 0x7f255015e0d0 2024-04-05T22:26:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.800+0000 7f254d400640 1 -- 172.21.15.73:0/2774285524 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538031930 con 0x7f255015e0d0 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 -- 172.21.15.73:0/2774285524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255015e0d0 msgr2=0x7f25500cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255015e0d0 0x7f25500cb0e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f25500cd730 tx=0x7f253802b840 comp rx=0 tx=0).stop 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 -- 172.21.15.73:0/2774285524 shutdown_connections 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550071010 0x7f25500cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 --2- 172.21.15.73:0/2774285524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f255015e0d0 0x7f25500cb0e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 -- 172.21.15.73:0/2774285524 >> 172.21.15.73:0/2774285524 conn(0x7f2550069cb0 msgr2=0x7f255006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 -- 172.21.15.73:0/2774285524 shutdown_connections 2024-04-05T22:26:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.801+0000 7f254f200640 1 -- 172.21.15.73:0/2774285524 wait complete. 2024-04-05T22:26:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.802+0000 7f254f200640 1 Processor -- start 2024-04-05T22:26:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.803+0000 7f254f200640 1 -- start start 2024-04-05T22:26:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.803+0000 7f254f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.803+0000 7f254f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550154090 0x7f255006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255015e0d0 0x7f25500aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255006bc30 con 0x7f2550071010 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255006bf10 con 0x7f255015e0d0 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255006c1f0 con 0x7f2550154090 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56020/0 (socket says 172.21.15.73:56020) 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 -- 172.21.15.73:0/3627968874 learned_addr learned my addr 172.21.15.73:0/3627968874 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 -- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550154090 msgr2=0x7f255006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550154090 0x7f255006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 -- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255015e0d0 msgr2=0x7f25500aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255015e0d0 0x7f25500aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 -- 172.21.15.73:0/3627968874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380094d0 con 0x7f2550071010 2024-04-05T22:26:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.804+0000 7f254de00640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f254000c430 tx=0x7f254000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.805+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011790 con 0x7f2550071010 2024-04-05T22:26:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.805+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400118f0 con 0x7f2550071010 2024-04-05T22:26:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.805+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255006c4d0 con 0x7f2550071010 2024-04-05T22:26:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.805+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011ce0 con 0x7f2550071010 2024-04-05T22:26:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.805+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255006b8e0 con 0x7f2550071010 2024-04-05T22:26:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.806+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255004e160 con 0x7f2550071010 2024-04-05T22:26:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.807+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f254001c460 con 0x7f2550071010 2024-04-05T22:26:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.807+0000 7f2546a00640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 0x7f252c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.808+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2540089900 con 0x7f2550071010 2024-04-05T22:26:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.808+0000 7f2545400640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 0x7f252c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.808+0000 7f2545400640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 0x7f252c06d420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2538036000 tx=0x7f2538030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:23.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.812+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2540054600 con 0x7f2550071010 2024-04-05T22:26:23.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.988+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25500ae050 con 0x7f2550071010 2024-04-05T22:26:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.989+0000 7f2546a00640 1 -- 172.21.15.73:0/3627968874 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2540057c20 con 0x7f2550071010 2024-04-05T22:26:23.989 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.991+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 msgr2=0x7f252c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.991+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 0x7f252c06d420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2538036000 tx=0x7f2538030000 comp rx=0 tx=0).stop 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.991+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 msgr2=0x7f255014d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.991+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f254000c430 tx=0x7f254000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 shutdown_connections 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f252c06af60 0x7f252c06d420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255015e0d0 0x7f25500aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2550154090 0x7f255006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 --2- 172.21.15.73:0/3627968874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2550071010 0x7f255014d460 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 >> 172.21.15.73:0/3627968874 conn(0x7f2550069cb0 msgr2=0x7f255015c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 shutdown_connections 2024-04-05T22:26:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:23.992+0000 7f254f200640 1 -- 172.21.15.73:0/3627968874 wait complete. 2024-04-05T22:26:24.010 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:26:24.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.213+0000 7ff3c2a00640 1 Processor -- start 2024-04-05T22:26:24.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.213+0000 7ff3c2a00640 1 -- start start 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.214+0000 7ff3c2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3bc0add00 0x7ff3bc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.214+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 0x7ff3bc06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.214+0000 7ff3c2a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc15d380 con 0x7ff3bc06c220 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.214+0000 7ff3c2a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc06c6b0 con 0x7ff3bc0add00 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.214+0000 7ff3c2a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b1950 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c1600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3bc0add00 0x7ff3bc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c2000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff3bc06c220 0x7ff3bc13e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58114/0 (socket says 172.21.15.73:58114) 2024-04-05T22:26:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c2000640 1 -- 172.21.15.73:0/1055737986 learned_addr learned my addr 172.21.15.73:0/1055737986 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3bbe00640 1 -- 172.21.15.73:0/1055737986 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588772754 0 0) 0x7ff3bc15d380 con 0x7ff3bc06c220 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3bbe00640 1 -- 172.21.15.73:0/1055737986 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a0003440 con 0x7ff3bc06c220 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c0c00640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 0x7ff3bc06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c1600640 1 -- 172.21.15.73:0/1055737986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 msgr2=0x7ff3bc06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c1600640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 0x7ff3bc06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c1600640 1 -- 172.21.15.73:0/1055737986 >> v1:172.21.15.73:6789/0 conn(0x7ff3bc06c220 legacy=0x7ff3bc13e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c1600640 1 -- 172.21.15.73:0/1055737986 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc15d660 con 0x7ff3bc0add00 2024-04-05T22:26:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.215+0000 7ff3c0c00640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 0x7ff3bc06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.217+0000 7ff3c1600640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc0add00 0x7ff3bc15a170 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0097a0 tx=0x7ff3ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c89e19e2e5066c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.217+0000 7ff3bbe00640 1 -- 172.21.15.73:0/1055737986 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac031830 con 0x7ff3bc0add00 2024-04-05T22:26:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.217+0000 7ff3bbe00640 1 -- 172.21.15.73:0/1055737986 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac031990 con 0x7ff3bc0add00 2024-04-05T22:26:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.217+0000 7ff3bbe00640 1 -- 172.21.15.73:0/1055737986 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac031d20 con 0x7ff3bc0add00 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 -- 172.21.15.73:0/1055737986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc0add00 msgr2=0x7ff3bc15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc0add00 0x7ff3bc15a170 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0097a0 tx=0x7ff3ac02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 -- 172.21.15.73:0/1055737986 shutdown_connections 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc15cbc0 0x7ff3bc06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/1055737986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc0add00 0x7ff3bc15a170 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 -- 172.21.15.73:0/1055737986 >> 172.21.15.73:0/1055737986 conn(0x7ff3bc0b0830 msgr2=0x7ff3bc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 -- 172.21.15.73:0/1055737986 shutdown_connections 2024-04-05T22:26:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.218+0000 7ff3c2a00640 1 -- 172.21.15.73:0/1055737986 wait complete. 2024-04-05T22:26:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.219+0000 7ff3c2a00640 1 Processor -- start 2024-04-05T22:26:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 -- start start 2024-04-05T22:26:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc06c220 0x7ff3bc13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3bc0add00 0x7ff3bc1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1e6ea0 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1e6580 con 0x7ff3bc0add00 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.220+0000 7ff3c2a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1e6800 con 0x7ff3bc06c220 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56050/0 (socket says 172.21.15.73:56050) 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 -- 172.21.15.73:0/3717263375 learned_addr learned my addr 172.21.15.73:0/3717263375 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 -- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc06c220 msgr2=0x7ff3bc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc06c220 0x7ff3bc13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 -- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3bc0add00 msgr2=0x7ff3bc1e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c0c00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3bc0add00 0x7ff3bc1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3bc0add00 0x7ff3bc1e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 -- 172.21.15.73:0/3717263375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac009400 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.221+0000 7ff3c2000640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c430 tx=0x7ff3b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.222+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4005ce0 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.222+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4005e40 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.222+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc13fd20 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.222+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4007310 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.222+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc1e6ac0 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.223+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc04e160 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.224+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7ff3b4007470 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.225+0000 7ff3b9600640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 0x7ff39c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.225+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3b4088f30 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.225+0000 7ff3c1600640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 0x7ff39c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.226+0000 7ff3c1600640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 0x7ff39c06d420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac036000 tx=0x7ff3ac030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.230+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff3b4053cc0 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.422+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3bc0ad380 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.423+0000 7ff3b9600640 1 -- 172.21.15.73:0/3717263375 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3b40170a0 con 0x7ff3bc15cbc0 2024-04-05T22:26:24.423 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 msgr2=0x7ff39c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 0x7ff39c06d420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac036000 tx=0x7ff3ac030000 comp rx=0 tx=0).stop 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 msgr2=0x7ff3bc0af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c430 tx=0x7ff3b400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 shutdown_connections 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7ff39c06af60 0x7ff39c06d420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3bc15cbc0 0x7ff3bc0af740 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3bc0add00 0x7ff3bc1e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 --2- 172.21.15.73:0/3717263375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3bc06c220 0x7ff3bc13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 >> 172.21.15.73:0/3717263375 conn(0x7ff3bc0b0830 msgr2=0x7ff3bc1419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:24.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 shutdown_connections 2024-04-05T22:26:24.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.426+0000 7ff3c2a00640 1 -- 172.21.15.73:0/3717263375 wait complete. 2024-04-05T22:26:24.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:26:24.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.636+0000 7fce09200640 1 Processor -- start 2024-04-05T22:26:24.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.636+0000 7fce09200640 1 -- start start 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce09200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce04160620 0x7fce040caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce09200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 0x7fce040cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce09200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce040ad880 con 0x7fce040b1870 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce09200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040cbcc0 con 0x7fce04160620 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce09200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce040ce5d0 con 0x7fce040cb2f0 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce01400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fce040b1870 0x7fce040ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58118/0 (socket says 172.21.15.73:58118) 2024-04-05T22:26:24.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.638+0000 7fce01400640 1 -- 172.21.15.73:0/3440833496 learned_addr learned my addr 172.21.15.73:0/3440833496 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce01e00640 1 --2- 172.21.15.73:0/3440833496 >> v2:172.21.15.157:3300/0 conn(0x7fce04160620 0x7fce040caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce02800640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 0x7fce040cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447772916 0 0) 0x7fce040ad880 con 0x7fce040b1870 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde8003520 con 0x7fce040b1870 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327546122 0 0) 0x7fcde8003520 con 0x7fce040b1870 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 msgr2=0x7fce040cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 0x7fce040cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 >> v2:172.21.15.157:3300/0 conn(0x7fce04160620 msgr2=0x7fce040caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 --2- 172.21.15.73:0/3440833496 >> v2:172.21.15.157:3300/0 conn(0x7fce04160620 0x7fce040caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040ce010 con 0x7fce040b1870 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fcdec001690 con 0x7fce040b1870 2024-04-05T22:26:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 >> v1:172.21.15.73:6789/0 conn(0x7fce040b1870 legacy=0x7fce040ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde8003970 0x7fcde8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcde80047c0 0x7fcde8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 0x7fcde8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040cbcc0 con 0x7fcde80047c0 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040ce5d0 con 0x7fcde8003970 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde800b980 con 0x7fce040cb2f0 2024-04-05T22:26:24.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.639+0000 7fce01400640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcde80047c0 0x7fcde8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01e00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde8003970 0x7fcde8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01400640 1 -- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 msgr2=0x7fcde8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01400640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce040cb2f0 0x7fcde8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01400640 1 -- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde8003970 msgr2=0x7fcde8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01400640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde8003970 0x7fcde8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01400640 1 -- 172.21.15.73:0/3440833496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8003520 con 0x7fcde80047c0 2024-04-05T22:26:24.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.640+0000 7fce01e00640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde8003970 0x7fcde8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:24.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.641+0000 7fce01400640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcde80047c0 0x7fcde8004c40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c210 tx=0x7fcde80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f386e04e69ecb4f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.641+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec005ad0 con 0x7fcde80047c0 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.641+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec005c50 con 0x7fcde80047c0 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.641+0000 7fce00a00640 1 -- 172.21.15.73:0/3440833496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec02c8b0 con 0x7fcde80047c0 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 -- 172.21.15.73:0/3440833496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcde80047c0 msgr2=0x7fcde8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 --2- 172.21.15.73:0/3440833496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcde80047c0 0x7fcde8004c40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c210 tx=0x7fcde80057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce02800640 1 -- 172.21.15.73:0/3440833496 reap_dead start 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 -- 172.21.15.73:0/3440833496 shutdown_connections 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 -- 172.21.15.73:0/3440833496 >> 172.21.15.73:0/3440833496 conn(0x7fce040ce8e0 msgr2=0x7fce040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 -- 172.21.15.73:0/3440833496 shutdown_connections 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.642+0000 7fce09200640 1 -- 172.21.15.73:0/3440833496 wait complete. 2024-04-05T22:26:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.643+0000 7fce09200640 1 Processor -- start 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.644+0000 7fce09200640 1 -- start start 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.644+0000 7fce09200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80047c0 0x7fce04142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.644+0000 7fce09200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce04153ee0 0x7fce04144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce09200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce09200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415d7a0 con 0x7fce0415cde0 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce09200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce04142aa0 con 0x7fce04153ee0 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce09200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce04145720 con 0x7fcde80047c0 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56064/0 (socket says 172.21.15.73:56064) 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 -- 172.21.15.73:0/1832516372 learned_addr learned my addr 172.21.15.73:0/1832516372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce01e00640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80047c0 0x7fce04142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce01400640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce04153ee0 0x7fce04144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 -- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80047c0 msgr2=0x7fce04142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80047c0 0x7fce04142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 -- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce04153ee0 msgr2=0x7fce04144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce04153ee0 0x7fce04144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 -- 172.21.15.73:0/1832516372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04145a00 con 0x7fce0415cde0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.645+0000 7fce02800640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fce04071ee0 tx=0x7fcdfc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.646+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc010040 con 0x7fce0415cde0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.646+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc00bae0 con 0x7fce0415cde0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.646+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce04145ce0 con 0x7fce0415cde0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.646+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00be70 con 0x7fce0415cde0 2024-04-05T22:26:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.646+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce04143110 con 0x7fce0415cde0 2024-04-05T22:26:24.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.647+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04066af0 con 0x7fce0415cde0 2024-04-05T22:26:24.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.648+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fcdfc01e410 con 0x7fce0415cde0 2024-04-05T22:26:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.649+0000 7fcdfa000640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 0x7fcde006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.649+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdfc089c90 con 0x7fce0415cde0 2024-04-05T22:26:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.649+0000 7fce01e00640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 0x7fcde006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.650+0000 7fce01e00640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 0x7fcde006d420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4006fd0 tx=0x7fcdf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.653+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcdfc054880 con 0x7fce0415cde0 2024-04-05T22:26:24.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.858+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce040ac930 con 0x7fce0415cde0 2024-04-05T22:26:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.859+0000 7fcdfa000640 1 -- 172.21.15.73:0/1832516372 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcdfc057ea0 con 0x7fce0415cde0 2024-04-05T22:26:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:26:24.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.861+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 msgr2=0x7fcde006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.861+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 0x7fcde006d420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4006fd0 tx=0x7fcdf4008040 comp rx=0 tx=0).stop 2024-04-05T22:26:24.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.861+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 msgr2=0x7fce0415d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:24.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.861+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fce04071ee0 tx=0x7fcdfc00b480 comp rx=0 tx=0).stop 2024-04-05T22:26:24.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 shutdown_connections 2024-04-05T22:26:24.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcde006af60 0x7fcde006d420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0415cde0 0x7fce0415d260 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce04153ee0 0x7fce04144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 --2- 172.21.15.73:0/1832516372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde80047c0 0x7fce04142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 >> 172.21.15.73:0/1832516372 conn(0x7fce040ce8e0 msgr2=0x7fce040ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 shutdown_connections 2024-04-05T22:26:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:24.862+0000 7fce09200640 1 -- 172.21.15.73:0/1832516372 wait complete. 2024-04-05T22:26:24.880 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:26:24.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:24.880 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:24.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:26:24.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:24.988 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:26:25.037 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:25.038 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:25.095 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:25.096 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:25.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:25.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.371+0000 7fccce000640 1 Processor -- start 2024-04-05T22:26:25.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.372+0000 7fccce000640 1 -- start start 2024-04-05T22:26:25.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccce000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccce000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccc815e850 0x7fccc815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccce000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc806f5b0 con 0x7fccc81587d0 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccce000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc806d500 con 0x7fccc815e850 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccce000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815f220 con 0x7fccc806cb80 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccc7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccc815e850 0x7fccc815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccc7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccc815e850 0x7fccc815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56066/0 (socket says 172.21.15.73:56066) 2024-04-05T22:26:25.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.373+0000 7fccc7e00640 1 -- 172.21.15.73:0/3753244621 learned_addr learned my addr 172.21.15.73:0/3753244621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.374+0000 7fccbfe00640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.374+0000 7fccc7e00640 1 -- 172.21.15.73:0/3753244621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 msgr2=0x7fccc806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.374+0000 7fccc7e00640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.374+0000 7fccc7e00640 1 -- 172.21.15.73:0/3753244621 >> v1:172.21.15.157:6789/0 conn(0x7fccc81587d0 legacy=0x7fccc80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.374+0000 7fccc7e00640 1 -- 172.21.15.73:0/3753244621 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8070410 con 0x7fccc815e850 2024-04-05T22:26:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.375+0000 7fccc7e00640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc815ece0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009e80 tx=0x7fccc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406ec69dcfa94ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.375+0000 7fccc6a00640 1 -- 172.21.15.73:0/3753244621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc002ca50 con 0x7fccc815e850 2024-04-05T22:26:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.375+0000 7fccc6a00640 1 -- 172.21.15.73:0/3753244621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc002cbb0 con 0x7fccc815e850 2024-04-05T22:26:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.375+0000 7fccc6a00640 1 -- 172.21.15.73:0/3753244621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00343f0 con 0x7fccc815e850 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 -- 172.21.15.73:0/3753244621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 msgr2=0x7fccc815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc815ece0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009e80 tx=0x7fccc002beb0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 -- 172.21.15.73:0/3753244621 shutdown_connections 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc815ece0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 --2- 172.21.15.73:0/3753244621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 -- 172.21.15.73:0/3753244621 >> 172.21.15.73:0/3753244621 conn(0x7fccc80ce8e0 msgr2=0x7fccc80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 -- 172.21.15.73:0/3753244621 shutdown_connections 2024-04-05T22:26:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.376+0000 7fccce000640 1 -- 172.21.15.73:0/3753244621 wait complete. 2024-04-05T22:26:25.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.378+0000 7fccce000640 1 Processor -- start 2024-04-05T22:26:25.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.378+0000 7fccce000640 1 -- start start 2024-04-05T22:26:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc8154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 0x7fccc80aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815cac0 con 0x7fccc815e850 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815cda0 con 0x7fccc81587d0 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccce000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815d080 con 0x7fccc806cb80 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56070/0 (socket says 172.21.15.73:56070) 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.379+0000 7fccc7e00640 1 -- 172.21.15.73:0/1848262748 learned_addr learned my addr 172.21.15.73:0/1848262748 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccbfe00640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 0x7fccc80aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7400640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc8154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 -- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 msgr2=0x7fccc8154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc8154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 -- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 msgr2=0x7fccc80aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 0x7fccc80aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 -- 172.21.15.73:0/1848262748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009b30 con 0x7fccc815e850 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccbfe00640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 0x7fccc80aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7400640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc8154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccc7e00640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fccc0036040 tx=0x7fccc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0040670 con 0x7fccc815e850 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc00407f0 con 0x7fccc815e850 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc815d360 con 0x7fccc815e850 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.380+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0040ba0 con 0x7fccc815e850 2024-04-05T22:26:25.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.381+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc815d920 con 0x7fccc815e850 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.382+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8066af0 con 0x7fccc815e850 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.383+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fccc0040d00 con 0x7fccc815e850 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.383+0000 7fccbf400640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 0x7fcca406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.383+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccc00ac9e0 con 0x7fccc815e850 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.384+0000 7fccc7400640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 0x7fcca406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.384+0000 7fccc7400640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 0x7fcca406d420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccc8154ea0 tx=0x7fccb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.388+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fccc0077890 con 0x7fccc815e850 2024-04-05T22:26:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.582+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccc80a84c0 con 0x7fccc815e850 2024-04-05T22:26:25.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.584+0000 7fccbf400640 1 -- 172.21.15.73:0/1848262748 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16100 (secure 0 0 0) 0x7fccc80a84c0 con 0x7fccc815e850 2024-04-05T22:26:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:25.585 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:11.541344+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 msgr2=0x7fcca406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 0x7fcca406d420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccc8154ea0 tx=0x7fccb8009040 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 msgr2=0x7fccc80aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fccc0036040 tx=0x7fccc0004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 shutdown_connections 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7fcca406af60 0x7fcca406d420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc815e850 0x7fccc80aac20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc81587d0 0x7fccc80aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 --2- 172.21.15.73:0/1848262748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc806cb80 0x7fccc8154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 >> 172.21.15.73:0/1848262748 conn(0x7fccc80ce8e0 msgr2=0x7fccc80708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.587+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 shutdown_connections 2024-04-05T22:26:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.588+0000 7fccce000640 1 -- 172.21.15.73:0/1848262748 wait complete. 2024-04-05T22:26:25.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:08.439249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:09.543608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:25.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.819+0000 7f0642c00640 1 Processor -- start 2024-04-05T22:26:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.819+0000 7f0642c00640 1 -- start start 2024-04-05T22:26:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.821+0000 7f0642c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.821+0000 7f0642c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f063c070650 0x7f063c070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.821+0000 7f0642c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c15e200 con 0x7f063c0ae770 2024-04-05T22:26:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.821+0000 7f0642c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063c15d6d0 con 0x7f063c070650 2024-04-05T22:26:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.821+0000 7f0642c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063c071040 con 0x7f063c0adea0 2024-04-05T22:26:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.822+0000 7f063be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.822+0000 7f0640c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f063c070650 0x7f063c070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.822+0000 7f063be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:57480/0 (socket says 172.21.15.73:57480) 2024-04-05T22:26:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.822+0000 7f063be00640 1 -- 172.21.15.73:0/151430756 learned_addr learned my addr 172.21.15.73:0/151430756 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.823+0000 7f063be00640 1 -- 172.21.15.73:0/151430756 >> v2:172.21.15.157:3300/0 conn(0x7f063c070650 msgr2=0x7f063c070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.823+0000 7f063be00640 1 --2- 172.21.15.73:0/151430756 >> v2:172.21.15.157:3300/0 conn(0x7f063c070650 0x7f063c070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.823+0000 7f063be00640 1 -- 172.21.15.73:0/151430756 >> v1:172.21.15.157:6789/0 conn(0x7f063c0ae770 legacy=0x7f063c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.823+0000 7f063be00640 1 -- 172.21.15.73:0/151430756 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c0aecc0 con 0x7f063c0adea0 2024-04-05T22:26:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.824+0000 7f063be00640 1 --2- 172.21.15.73:0/151430756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f062c009870 tx=0x7f062c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b7a653b735954 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.824+0000 7f063aa00640 1 -- 172.21.15.73:0/151430756 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c031640 con 0x7f063c0adea0 2024-04-05T22:26:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.824+0000 7f063aa00640 1 -- 172.21.15.73:0/151430756 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0317a0 con 0x7f063c0adea0 2024-04-05T22:26:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.824+0000 7f063aa00640 1 -- 172.21.15.73:0/151430756 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c031b30 con 0x7f063c0adea0 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 -- 172.21.15.73:0/151430756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 msgr2=0x7f063c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 --2- 172.21.15.73:0/151430756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f062c009870 tx=0x7f062c02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 -- 172.21.15.73:0/151430756 shutdown_connections 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 --2- 172.21.15.73:0/151430756 >> v2:172.21.15.157:3300/0 conn(0x7f063c070650 0x7f063c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 --2- 172.21.15.73:0/151430756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c0adea0 0x7f063c0b47b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 -- 172.21.15.73:0/151430756 >> 172.21.15.73:0/151430756 conn(0x7f063c0b18b0 msgr2=0x7f063c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 -- 172.21.15.73:0/151430756 shutdown_connections 2024-04-05T22:26:25.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.825+0000 7f0642c00640 1 -- 172.21.15.73:0/151430756 wait complete. 2024-04-05T22:26:25.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.827+0000 7f0642c00640 1 Processor -- start 2024-04-05T22:26:25.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.827+0000 7f0642c00640 1 -- start start 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 0x7f063c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f063c0adea0 0x7f063c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c14a490 con 0x7f063c0ae770 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c14a6e0 con 0x7f063c0adea0 2024-04-05T22:26:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0642c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063c14a9c0 con 0x7f063c070650 2024-04-05T22:26:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0640c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0640c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50936/0 (socket says 172.21.15.73:50936) 2024-04-05T22:26:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.828+0000 7f0640c00640 1 -- 172.21.15.73:0/3733295392 learned_addr learned my addr 172.21.15.73:0/3733295392 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f063be00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 0x7f063c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f063b400640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f063c0adea0 0x7f063c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 -- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 msgr2=0x7f063c15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 0x7f063c15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 -- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f063c0adea0 msgr2=0x7f063c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f063c0adea0 0x7f063c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 -- 172.21.15.73:0/3733295392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0094d0 con 0x7f063c0ae770 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f063be00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 0x7f063c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.829+0000 7f0640c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f063400efc0 tx=0x7f063400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.830+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634009ba0 con 0x7f063c0ae770 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.830+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063400f040 con 0x7f063c0ae770 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.830+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c14aca0 con 0x7f063c0ae770 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.830+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340146c0 con 0x7f063c0ae770 2024-04-05T22:26:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.830+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c0c7d10 con 0x7f063c0ae770 2024-04-05T22:26:25.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.831+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063c14b1c0 con 0x7f063c0ae770 2024-04-05T22:26:25.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.832+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f0634014820 con 0x7f063c0ae770 2024-04-05T22:26:25.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.833+0000 7f0633e00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 0x7f061c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:25.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.833+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f063408a6f0 con 0x7f063c0ae770 2024-04-05T22:26:25.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.833+0000 7f063be00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 0x7f061c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:25.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.834+0000 7f063be00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 0x7f061c06d3c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f062c036000 tx=0x7f062c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:25.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:25.838+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f063c14b1c0 con 0x7f063c0ae770 2024-04-05T22:26:26.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.032+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f063c14b1c0 con 0x7f063c0ae770 2024-04-05T22:26:26.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.033+0000 7f0633e00640 1 -- 172.21.15.73:0/3733295392 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f063c14b1c0 con 0x7f063c0ae770 2024-04-05T22:26:26.033 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 msgr2=0x7f061c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 0x7f061c06d3c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f062c036000 tx=0x7f062c03c040 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 msgr2=0x7f063c0c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f063400efc0 tx=0x7f063400c490 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 shutdown_connections 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f061c06af00 0x7f061c06d3c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063c0ae770 0x7f063c0c77d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f063c0adea0 0x7f063c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 --2- 172.21.15.73:0/3733295392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063c070650 0x7f063c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.035+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 >> 172.21.15.73:0/3733295392 conn(0x7f063c0b18b0 msgr2=0x7f063c06a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.036+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 shutdown_connections 2024-04-05T22:26:26.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.036+0000 7f0642c00640 1 -- 172.21.15.73:0/3733295392 wait complete. 2024-04-05T22:26:26.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:26:26.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.267+0000 7f6b23800640 1 Processor -- start 2024-04-05T22:26:26.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.267+0000 7f6b23800640 1 -- start start 2024-04-05T22:26:26.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b23800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 0x7f6b1c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b23800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b23800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c1612c0 con 0x7f6b1c06be40 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b23800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c1610d0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b23800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c85d0 con 0x7f6b1c06a090 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b20e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 0x7f6b1c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36078/0 (socket says 172.21.15.73:36078) 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b1be00640 1 -- 172.21.15.73:0/726798360 learned_addr learned my addr 172.21.15.73:0/726798360 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.269+0000 7f6b20e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 0x7f6b1c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:38056/0 (socket says 172.21.15.73:38056) 2024-04-05T22:26:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.270+0000 7f6b1be00640 1 -- 172.21.15.73:0/726798360 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 msgr2=0x7f6b1c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.270+0000 7f6b1be00640 1 --2- 172.21.15.73:0/726798360 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 0x7f6b1c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.270+0000 7f6b1be00640 1 -- 172.21.15.73:0/726798360 >> v1:172.21.15.157:6789/0 conn(0x7f6b1c06be40 legacy=0x7f6b1c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.270+0000 7f6b1be00640 1 -- 172.21.15.73:0/726798360 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c1615a0 con 0x7f6b1c06a090 2024-04-05T22:26:26.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.272+0000 7f6b1be00640 1 --2- 172.21.15.73:0/726798360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009870 tx=0x7f6b0802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1227c2d19b0616 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.272+0000 7f6b1b400640 1 -- 172.21.15.73:0/726798360 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031640 con 0x7f6b1c06a090 2024-04-05T22:26:26.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.272+0000 7f6b1b400640 1 -- 172.21.15.73:0/726798360 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b080317a0 con 0x7f6b1c06a090 2024-04-05T22:26:26.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.272+0000 7f6b1b400640 1 -- 172.21.15.73:0/726798360 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031b30 con 0x7f6b1c06a090 2024-04-05T22:26:26.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 -- 172.21.15.73:0/726798360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 msgr2=0x7f6b1c15df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 --2- 172.21.15.73:0/726798360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009870 tx=0x7f6b0802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 -- 172.21.15.73:0/726798360 shutdown_connections 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 --2- 172.21.15.73:0/726798360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c15df40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 --2- 172.21.15.73:0/726798360 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c069cb0 0x7f6b1c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 -- 172.21.15.73:0/726798360 >> 172.21.15.73:0/726798360 conn(0x7f6b1c0b0ce0 msgr2=0x7f6b1c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 -- 172.21.15.73:0/726798360 shutdown_connections 2024-04-05T22:26:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.273+0000 7f6b23800640 1 -- 172.21.15.73:0/726798360 wait complete. 2024-04-05T22:26:26.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.274+0000 7f6b23800640 1 Processor -- start 2024-04-05T22:26:26.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.275+0000 7f6b23800640 1 -- start start 2024-04-05T22:26:26.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.275+0000 7f6b23800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b23800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c09c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b23800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c06be40 0x7f6b1c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b23800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c15f390 con 0x7f6b1c069cb0 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b23800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0cfc90 con 0x7f6b1c06be40 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b23800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c15e710 con 0x7f6b1c06a090 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50942/0 (socket says 172.21.15.73:50942) 2024-04-05T22:26:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 -- 172.21.15.73:0/2481959727 learned_addr learned my addr 172.21.15.73:0/2481959727 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b21800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c06be40 0x7f6b1c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 -- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 msgr2=0x7f6b1c09c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c09c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 -- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c06be40 msgr2=0x7f6b1c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c06be40 0x7f6b1c1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.276+0000 7f6b20e00640 1 -- 172.21.15.73:0/2481959727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094d0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b20e00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000e8a0 tx=0x7f6b1000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100094f0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000c940 con 0x7f6b1c069cb0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000ccf0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c15e9f0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.277+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c15efb0 con 0x7f6b1c069cb0 2024-04-05T22:26:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.278+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c15e480 con 0x7f6b1c069cb0 2024-04-05T22:26:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.279+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f6b10013470 con 0x7f6b1c069cb0 2024-04-05T22:26:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.280+0000 7f6b18c00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 0x7f6afc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.280+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b10089920 con 0x7f6b1c069cb0 2024-04-05T22:26:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.280+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 0x7f6afc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.281+0000 7f6b1be00640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 0x7f6afc06d450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b08030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.284+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6b10054740 con 0x7f6b1c069cb0 2024-04-05T22:26:26.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.475+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b1c068160 con 0x7f6b1c069cb0 2024-04-05T22:26:26.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.476+0000 7f6b18c00640 1 -- 172.21.15.73:0/2481959727 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b10017020 con 0x7f6b1c069cb0 2024-04-05T22:26:26.476 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:26:26.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.478+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 msgr2=0x7f6afc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 0x7f6afc06d450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b08030000 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 msgr2=0x7f6b1c0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000e8a0 tx=0x7f6b1000ed60 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 shutdown_connections 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f6afc06af90 0x7f6afc06d450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c06be40 0x7f6b1c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c06a090 0x7f6b1c09c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 --2- 172.21.15.73:0/2481959727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1c069cb0 0x7f6b1c0cf750 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 >> 172.21.15.73:0/2481959727 conn(0x7f6b1c0b0ce0 msgr2=0x7f6b1c15b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 shutdown_connections 2024-04-05T22:26:26.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.479+0000 7f6b23800640 1 -- 172.21.15.73:0/2481959727 wait complete. 2024-04-05T22:26:26.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:26:26.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.713+0000 7f9355600640 1 Processor -- start 2024-04-05T22:26:26.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.713+0000 7f9355600640 1 -- start start 2024-04-05T22:26:26.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f9355600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9350160490 0x7f93500cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f9355600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93500cfd70 0x7f93500d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f9355600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500ce460 con 0x7f93500b3c70 2024-04-05T22:26:26.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f9355600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500ce740 con 0x7f9350160490 2024-04-05T22:26:26.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f9355600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500cea20 con 0x7f93500cfd70 2024-04-05T22:26:26.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f934ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9350160490 0x7f93500cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f934ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9350160490 0x7f93500cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50956/0 (socket says 172.21.15.73:50956) 2024-04-05T22:26:26.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.715+0000 7f934ea00640 1 -- 172.21.15.73:0/2741876308 learned_addr learned my addr 172.21.15.73:0/2741876308 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:26.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.716+0000 7f934ea00640 1 -- 172.21.15.73:0/2741876308 >> v2:172.21.15.157:3300/0 conn(0x7f93500cfd70 msgr2=0x7f93500d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.716+0000 7f934f400640 1 --2- 172.21.15.73:0/2741876308 >> v2:172.21.15.157:3300/0 conn(0x7f93500cfd70 0x7f93500d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.716+0000 7f934ea00640 1 --2- 172.21.15.73:0/2741876308 >> v2:172.21.15.157:3300/0 conn(0x7f93500cfd70 0x7f93500d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.716+0000 7f934ea00640 1 -- 172.21.15.73:0/2741876308 >> v1:172.21.15.157:6789/0 conn(0x7f93500b3c70 legacy=0x7f93500cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.716+0000 7f934ea00640 1 -- 172.21.15.73:0/2741876308 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500ced00 con 0x7f9350160490 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.717+0000 7f934ea00640 1 --2- 172.21.15.73:0/2741876308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9350160490 0x7f93500cd9a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f93500d03c0 tx=0x7f934002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221356243eabf8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.717+0000 7f934e000640 1 -- 172.21.15.73:0/2741876308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031870 con 0x7f9350160490 2024-04-05T22:26:26.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.717+0000 7f934e000640 1 -- 172.21.15.73:0/2741876308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400319d0 con 0x7f9350160490 2024-04-05T22:26:26.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.717+0000 7f934e000640 1 -- 172.21.15.73:0/2741876308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031d60 con 0x7f9350160490 2024-04-05T22:26:26.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 -- 172.21.15.73:0/2741876308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9350160490 msgr2=0x7f93500cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 --2- 172.21.15.73:0/2741876308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9350160490 0x7f93500cd9a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f93500d03c0 tx=0x7f934002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 -- 172.21.15.73:0/2741876308 shutdown_connections 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 --2- 172.21.15.73:0/2741876308 >> v2:172.21.15.157:3300/0 conn(0x7f93500cfd70 0x7f93500d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 --2- 172.21.15.73:0/2741876308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9350160490 0x7f93500cd9a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 -- 172.21.15.73:0/2741876308 >> 172.21.15.73:0/2741876308 conn(0x7f935009c020 msgr2=0x7f93500d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 -- 172.21.15.73:0/2741876308 shutdown_connections 2024-04-05T22:26:26.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.718+0000 7f9355600640 1 -- 172.21.15.73:0/2741876308 wait complete. 2024-04-05T22:26:26.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.720+0000 7f9355600640 1 Processor -- start 2024-04-05T22:26:26.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.720+0000 7f9355600640 1 -- start start 2024-04-05T22:26:26.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93500cfd70 0x7f935015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 0x7f935015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cc6f0 con 0x7f93500b3c70 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b19e0 con 0x7f9350160490 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f9355600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b0530 con 0x7f93500cfd70 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f934ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f934ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50966/0 (socket says 172.21.15.73:50966) 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.721+0000 7f934ea00640 1 -- 172.21.15.73:0/1747213362 learned_addr learned my addr 172.21.15.73:0/1747213362 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f9347e00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93500cfd70 0x7f935015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934f400640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 0x7f935015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 -- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93500cfd70 msgr2=0x7f935015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93500cfd70 0x7f935015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 -- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 msgr2=0x7f935015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 0x7f935015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 -- 172.21.15.73:0/1747213362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400094d0 con 0x7f93500b3c70 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934f400640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 0x7f935015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f934ea00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.722+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031840 con 0x7f93500b3c70 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.723+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400443f0 con 0x7f93500b3c70 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.723+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400447a0 con 0x7f93500b3c70 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.723+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500af9b0 con 0x7f93500b3c70 2024-04-05T22:26:26.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.723+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500aff70 con 0x7f93500b3c70 2024-04-05T22:26:26.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.724+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9350068160 con 0x7f93500b3c70 2024-04-05T22:26:26.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.725+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f934004d450 con 0x7f93500b3c70 2024-04-05T22:26:26.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.726+0000 7f9346a00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 0x7f932c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:26.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.726+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f934002d040 con 0x7f93500b3c70 2024-04-05T22:26:26.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.726+0000 7f9347e00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 0x7f932c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:26.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.727+0000 7f9347e00640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 0x7f932c06d420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93500b8d80 tx=0x7f9338009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:26.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.730+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9340076e20 con 0x7f93500b3c70 2024-04-05T22:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.925+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93500d38c0 con 0x7f93500b3c70 2024-04-05T22:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.926+0000 7f9346a00640 1 -- 172.21.15.73:0/1747213362 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f934007a440 con 0x7f93500b3c70 2024-04-05T22:26:26.927 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:26:26.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 msgr2=0x7f932c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 0x7f932c06d420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93500b8d80 tx=0x7f9338009040 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 msgr2=0x7f935015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 shutdown_connections 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f932c06af60 0x7f932c06d420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9350160490 0x7f935015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93500cfd70 0x7f935015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 --2- 172.21.15.73:0/1747213362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500b3c70 0x7f935015d4a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 >> 172.21.15.73:0/1747213362 conn(0x7f935009c020 msgr2=0x7f9350161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 shutdown_connections 2024-04-05T22:26:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:26.929+0000 7f9355600640 1 -- 172.21.15.73:0/1747213362 wait complete. 2024-04-05T22:26:26.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:26:27.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.157+0000 7f513ac00640 1 Processor -- start 2024-04-05T22:26:27.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.157+0000 7f513ac00640 1 -- start start 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.158+0000 7f513ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 0x7f51340b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f513ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340b3ea0 con 0x7f513415aaa0 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f513ac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513415e2c0 con 0x7f5134157560 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f513ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513415d500 con 0x7f513406d060 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f5138e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f513415aaa0 0x7f5134156e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50674/0 (socket says 172.21.15.73:50674) 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f5138e00640 1 -- 172.21.15.73:0/4047445440 learned_addr learned my addr 172.21.15.73:0/4047445440 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:27.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f5139800640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 0x7f51340b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629942354 0 0) 0x7f51340b3ea0 con 0x7f513415aaa0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.159+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c003520 con 0x7f513415aaa0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500248135 0 0) 0x7f511c003520 con 0x7f513415aaa0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 msgr2=0x7f51340b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 0x7f51340b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 >> v1:172.21.15.157:6789/0 conn(0x7f5134157560 legacy=0x7f51340b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513415d7e0 con 0x7f513415aaa0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5120001670 con 0x7f513415aaa0 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 >> v1:172.21.15.73:6789/0 conn(0x7f513415aaa0 legacy=0x7f5134156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 0x7f511c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f511c003f20 0x7f511c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c004d00 0x7f511c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513415d500 con 0x7f511c003f20 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c00ba60 con 0x7f511c004d00 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c00bc20 con 0x7f513406d060 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.160+0000 7f5138e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f511c003f20 0x7f511c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.161+0000 7f5138e00640 1 -- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 msgr2=0x7f511c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.161+0000 7f5138e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513406d060 0x7f511c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.161+0000 7f5138e00640 1 -- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c004d00 msgr2=0x7f511c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.161+0000 7f5138e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c004d00 0x7f511c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.161+0000 7f5138e00640 1 -- 172.21.15.73:0/4047445440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c003520 con 0x7f511c003f20 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f5138e00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f511c003f20 0x7f511c0047c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f512000c210 tx=0x7f511c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879ed275799d4d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120005ad0 con 0x7f511c003f20 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120005c50 con 0x7f511c003f20 2024-04-05T22:26:27.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f5133e00640 1 -- 172.21.15.73:0/4047445440 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512002c890 con 0x7f511c003f20 2024-04-05T22:26:27.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f513ac00640 1 -- 172.21.15.73:0/4047445440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f511c003f20 msgr2=0x7f511c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f513ac00640 1 --2- 172.21.15.73:0/4047445440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f511c003f20 0x7f511c0047c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f512000c210 tx=0x7f511c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.162+0000 7f513a200640 1 -- 172.21.15.73:0/4047445440 reap_dead start 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.163+0000 7f513ac00640 1 -- 172.21.15.73:0/4047445440 shutdown_connections 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.163+0000 7f513ac00640 1 -- 172.21.15.73:0/4047445440 >> 172.21.15.73:0/4047445440 conn(0x7f51340b0830 msgr2=0x7f51340b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.163+0000 7f513ac00640 1 -- 172.21.15.73:0/4047445440 shutdown_connections 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.163+0000 7f513ac00640 1 -- 172.21.15.73:0/4047445440 wait complete. 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.164+0000 7f513ac00640 1 Processor -- start 2024-04-05T22:26:27.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.164+0000 7f513ac00640 1 -- start start 2024-04-05T22:26:27.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 0x7f5134145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 0x7f513414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513414b850 con 0x7f5134145eb0 2024-04-05T22:26:27.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513414baa0 con 0x7f511c003f20 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.165+0000 7f513ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513414bd80 con 0x7f513414c860 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50984/0 (socket says 172.21.15.73:50984) 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 -- 172.21.15.73:0/3727165622 learned_addr learned my addr 172.21.15.73:0/3727165622 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5139800640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 0x7f5134145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f513a200640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 0x7f513414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 -- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 msgr2=0x7f513414f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 0x7f513414f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 -- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 msgr2=0x7f5134145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 0x7f5134145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5138e00640 1 -- 172.21.15.73:0/3727165622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513414c060 con 0x7f5134145eb0 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f5139800640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 0x7f5134145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.166+0000 7f513a200640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 0x7f513414f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f5138e00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f512002c430 tx=0x7f5120001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120039070 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512002dd30 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513414c340 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512003c040 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.167+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513414def0 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.168+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513404e160 con 0x7f5134145eb0 2024-04-05T22:26:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.169+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f5120032070 con 0x7f5134145eb0 2024-04-05T22:26:27.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.170+0000 7f5131600640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 0x7f511406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.171+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51200aab20 con 0x7f5134145eb0 2024-04-05T22:26:27.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.171+0000 7f5139800640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 0x7f511406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.171+0000 7f5139800640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 0x7f511406d450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5134159830 tx=0x7f5128008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.175+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f51200758b0 con 0x7f5134145eb0 2024-04-05T22:26:27.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.382+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f513414c4d0 con 0x7f5134145eb0 2024-04-05T22:26:27.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.383+0000 7f5131600640 1 -- 172.21.15.73:0/3727165622 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5120046070 con 0x7f5134145eb0 2024-04-05T22:26:27.383 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:26:27.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.385+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 msgr2=0x7f511406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.385+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 0x7f511406d450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5134159830 tx=0x7f5128008040 comp rx=0 tx=0).stop 2024-04-05T22:26:27.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.385+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 msgr2=0x7f513414f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f512002c430 tx=0x7f5120001a00 comp rx=0 tx=0).stop 2024-04-05T22:26:27.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 shutdown_connections 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f511406af90 0x7f511406d450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513414c860 0x7f513414f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134145eb0 0x7f513414f2a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 --2- 172.21.15.73:0/3727165622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c003f20 0x7f5134145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 >> 172.21.15.73:0/3727165622 conn(0x7f51340b0830 msgr2=0x7f51340cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 shutdown_connections 2024-04-05T22:26:27.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.386+0000 7f513ac00640 1 -- 172.21.15.73:0/3727165622 wait complete. 2024-04-05T22:26:27.404 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:26:27.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:27.404 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:27.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:26:27.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:27.505 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:26:27.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.775+0000 7f028ec00640 1 Processor -- start 2024-04-05T22:26:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.775+0000 7f028ec00640 1 -- start start 2024-04-05T22:26:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.776+0000 7f028ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0288159440 0x7f02880b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.776+0000 7f028ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f028806d3a0 0x7f0288070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f028ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028815a830 con 0x7f0288071340 2024-04-05T22:26:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f028ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880b2280 con 0x7f0288159440 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f028ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880b3a20 con 0x7f028806d3a0 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0288159440 0x7f02880b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0288159440 0x7f02880b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50998/0 (socket says 172.21.15.73:50998) 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 -- 172.21.15.73:0/3095391051 learned_addr learned my addr 172.21.15.73:0/3095391051 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f028cc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0288071340 0x7f02880ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50684/0 (socket says 172.21.15.73:50684) 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 -- 172.21.15.73:0/3095391051 >> v2:172.21.15.157:3300/0 conn(0x7f028806d3a0 msgr2=0x7f0288070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 --2- 172.21.15.73:0/3095391051 >> v2:172.21.15.157:3300/0 conn(0x7f028806d3a0 0x7f0288070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 -- 172.21.15.73:0/3095391051 >> v1:172.21.15.73:6789/0 conn(0x7f0288071340 legacy=0x7f02880ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287e00640 1 -- 172.21.15.73:0/3095391051 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880b2520 con 0x7f0288159440 2024-04-05T22:26:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.777+0000 7f0287400640 1 -- 172.21.15.73:0/3095391051 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873385215 0 0) 0x7f0280001490 con 0x7f0288071340 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.778+0000 7f0287e00640 1 --2- 172.21.15.73:0/3095391051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f02880b3440 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0288071fa0 tx=0x7f027802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab1232dbc36cd75 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f0287400640 1 -- 172.21.15.73:0/3095391051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278030860 con 0x7f0288159440 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f0287400640 1 -- 172.21.15.73:0/3095391051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02780309c0 con 0x7f0288159440 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f0287400640 1 -- 172.21.15.73:0/3095391051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278030d50 con 0x7f0288159440 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 -- 172.21.15.73:0/3095391051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 msgr2=0x7f02880b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 --2- 172.21.15.73:0/3095391051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f02880b3440 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0288071fa0 tx=0x7f027802b9a0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 -- 172.21.15.73:0/3095391051 shutdown_connections 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 --2- 172.21.15.73:0/3095391051 >> v2:172.21.15.157:3300/0 conn(0x7f028806d3a0 0x7f0288070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 --2- 172.21.15.73:0/3095391051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f02880b3440 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.779+0000 7f028ec00640 1 -- 172.21.15.73:0/3095391051 >> 172.21.15.73:0/3095391051 conn(0x7f02880ce8e0 msgr2=0x7f02880cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.780+0000 7f028ec00640 1 -- 172.21.15.73:0/3095391051 shutdown_connections 2024-04-05T22:26:27.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.780+0000 7f028ec00640 1 -- 172.21.15.73:0/3095391051 wait complete. 2024-04-05T22:26:27.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.781+0000 7f028ec00640 1 Processor -- start 2024-04-05T22:26:27.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.781+0000 7f028ec00640 1 -- start start 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.782+0000 7f028ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806d3a0 0x7f02881f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.782+0000 7f028ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 0x7f028814bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881f5d90 con 0x7f0288159440 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881f6150 con 0x7f0288071340 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02881f6430 con 0x7f028806d3a0 2024-04-05T22:26:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51000/0 (socket says 172.21.15.73:51000) 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 -- 172.21.15.73:0/212212157 learned_addr learned my addr 172.21.15.73:0/212212157 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f027f400640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 0x7f028814bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 -- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806d3a0 msgr2=0x7f02881f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f0287e00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806d3a0 0x7f02881f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806d3a0 0x7f02881f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 -- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 msgr2=0x7f028814bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 0x7f028814bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f028cc00640 1 -- 172.21.15.73:0/212212157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780094d0 con 0x7f0288159440 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.783+0000 7f027f400640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 0x7f028814bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f028cc00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f028000c7d0 tx=0x7f028000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000a640 con 0x7f0288159440 2024-04-05T22:26:27.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000a7c0 con 0x7f0288159440 2024-04-05T22:26:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000abb0 con 0x7f0288159440 2024-04-05T22:26:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02881f6710 con 0x7f0288159440 2024-04-05T22:26:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.784+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02881f6cd0 con 0x7f0288159440 2024-04-05T22:26:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.785+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0288066af0 con 0x7f0288159440 2024-04-05T22:26:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.786+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f028000ad10 con 0x7f0288159440 2024-04-05T22:26:27.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.787+0000 7f0284c00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 0x7f026406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:27.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.787+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0280088ce0 con 0x7f0288159440 2024-04-05T22:26:27.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.787+0000 7f0287e00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 0x7f026406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.788+0000 7f0287e00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 0x7f026406d420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0278036000 tx=0x7f027803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:27.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:27.792+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0280053a70 con 0x7f0288159440 2024-04-05T22:26:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.021+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f028815a830 con 0x7f0288159440 2024-04-05T22:26:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.022+0000 7f0284c00640 1 -- 172.21.15.73:0/212212157 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f028815a830 con 0x7f0288159440 2024-04-05T22:26:28.023 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:28.023 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:28.023 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 msgr2=0x7f026406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 0x7f026406d420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0278036000 tx=0x7f027803b040 comp rx=0 tx=0).stop 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 msgr2=0x7f028815b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f028000c7d0 tx=0x7f028000cc90 comp rx=0 tx=0).stop 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 shutdown_connections 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f026406af60 0x7f026406d420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0288159440 0x7f028815b530 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0288071340 0x7f028814bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 --2- 172.21.15.73:0/212212157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806d3a0 0x7f02881f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 >> 172.21.15.73:0/212212157 conn(0x7f02880ce8e0 msgr2=0x7f0288157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:28.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 shutdown_connections 2024-04-05T22:26:28.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.025+0000 7f028ec00640 1 -- 172.21.15.73:0/212212157 wait complete. 2024-04-05T22:26:28.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:26:28.280 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2024-04-05T22:26:28.280 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-05T22:26:28.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol8470 2024-04-05T22:26:28.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.500+0000 7f516ae00640 1 Processor -- start 2024-04-05T22:26:28.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.500+0000 7f516ae00640 1 -- start start 2024-04-05T22:26:28.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f516ae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5164157030 0x7f51640ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f516ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 0x7f5164160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f516ae00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516406d800 con 0x7f51640cb0b0 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f516ae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51640b3090 con 0x7f5164157030 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f516ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51640b3370 con 0x7f5164157410 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f5169a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5164157030 0x7f51640ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f5169a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5164157030 0x7f51640ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51010/0 (socket says 172.21.15.73:51010) 2024-04-05T22:26:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.502+0000 7f5169a00640 1 -- 172.21.15.73:0/1595547436 learned_addr learned my addr 172.21.15.73:0/1595547436 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169000640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 0x7f5164160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169a00640 1 -- 172.21.15.73:0/1595547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 msgr2=0x7f5164160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169a00640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 0x7f5164160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169a00640 1 -- 172.21.15.73:0/1595547436 >> v1:172.21.15.157:6789/0 conn(0x7f51640cb0b0 legacy=0x7f516415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169a00640 1 -- 172.21.15.73:0/1595547436 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51640b3650 con 0x7f5164157030 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.503+0000 7f5169000640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 0x7f5164160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.504+0000 7f5169a00640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157030 0x7f51640ad950 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5158009870 tx=0x7f515802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cf9c5d35091fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.504+0000 7f5163e00640 1 -- 172.21.15.73:0/1595547436 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158031640 con 0x7f5164157030 2024-04-05T22:26:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.504+0000 7f5163e00640 1 -- 172.21.15.73:0/1595547436 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51580317a0 con 0x7f5164157030 2024-04-05T22:26:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.504+0000 7f5163e00640 1 -- 172.21.15.73:0/1595547436 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158031b30 con 0x7f5164157030 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 -- 172.21.15.73:0/1595547436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157030 msgr2=0x7f51640ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157030 0x7f51640ad950 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5158009870 tx=0x7f515802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 -- 172.21.15.73:0/1595547436 shutdown_connections 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5164157410 0x7f5164160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 --2- 172.21.15.73:0/1595547436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157030 0x7f51640ad950 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 -- 172.21.15.73:0/1595547436 >> 172.21.15.73:0/1595547436 conn(0x7f5164069cb0 msgr2=0x7f516406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 -- 172.21.15.73:0/1595547436 shutdown_connections 2024-04-05T22:26:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.505+0000 7f516ae00640 1 -- 172.21.15.73:0/1595547436 wait complete. 2024-04-05T22:26:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.507+0000 7f516ae00640 1 Processor -- start 2024-04-05T22:26:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.507+0000 7f516ae00640 1 -- start start 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640cb0b0 0x7f5164150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164157030 0x7f5164155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5164155630 con 0x7f5164157410 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51641549b0 con 0x7f5164157030 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.508+0000 7f516ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5164154c90 con 0x7f51640cb0b0 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51026/0 (socket says 172.21.15.73:51026) 2024-04-05T22:26:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 -- 172.21.15.73:0/4032468879 learned_addr learned my addr 172.21.15.73:0/4032468879 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f5169000640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164157030 0x7f5164155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f5169a00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640cb0b0 0x7f5164150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 -- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640cb0b0 msgr2=0x7f5164150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640cb0b0 0x7f5164150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 -- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164157030 msgr2=0x7f5164155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164157030 0x7f5164155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.509+0000 7f516a400640 1 -- 172.21.15.73:0/4032468879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580094d0 con 0x7f5164157410 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f516a400640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c780 tx=0x7f515c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c004b60 con 0x7f5164157410 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c00ce90 con 0x7f5164157410 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00a850 con 0x7f5164157410 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5164154f70 con 0x7f5164157410 2024-04-05T22:26:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.510+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51641545d0 con 0x7f5164157410 2024-04-05T22:26:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.511+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5164155770 con 0x7f5164157410 2024-04-05T22:26:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.512+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f515c00a9b0 con 0x7f5164157410 2024-04-05T22:26:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.513+0000 7f5161600640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 0x7f514406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.513+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f515c089bd0 con 0x7f5164157410 2024-04-05T22:26:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.513+0000 7f5169a00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 0x7f514406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.514+0000 7f5169a00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 0x7f514406d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5158036000 tx=0x7f515803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:28.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.518+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f515c054860 con 0x7f5164157410 2024-04-05T22:26:28.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.738+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 --> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol8470", "target": ["mon-mgr", ""]}) v1 -- 0x7f51640b3930 con 0x7f514406af60 2024-04-05T22:26:28.747 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:28.747+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:28.747 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:28.747+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:28.747 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:28.747+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:28.747 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:28.747+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:28.748 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:28.747+0000 7f54fee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:28.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.813+0000 7f5161600640 1 -- 172.21.15.73:0/4032468879 <== mgr.4107 v2:172.21.15.73:6832/37063 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5158031a10 con 0x7f514406af60 2024-04-05T22:26:28.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 msgr2=0x7f514406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 0x7f514406d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5158036000 tx=0x7f515803c040 comp rx=0 tx=0).stop 2024-04-05T22:26:28.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 msgr2=0x7f5164151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:28.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c780 tx=0x7f515c00cc40 comp rx=0 tx=0).stop 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 shutdown_connections 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f514406af60 0x7f514406d420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5164157410 0x7f5164151900 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5164157030 0x7f5164155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 --2- 172.21.15.73:0/4032468879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51640cb0b0 0x7f5164150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 >> 172.21.15.73:0/4032468879 conn(0x7f5164069cb0 msgr2=0x7f51640cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 shutdown_connections 2024-04-05T22:26:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:28.816+0000 7f516ae00640 1 -- 172.21.15.73:0/4032468879 wait complete. 2024-04-05T22:26:28.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.029+0000 7f727f400640 1 Processor -- start 2024-04-05T22:26:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.029+0000 7f727f400640 1 -- start start 2024-04-05T22:26:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.030+0000 7f727f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72781541a0 0x7f72780b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.030+0000 7f727f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727815aa60 con 0x7f72780ce000 2024-04-05T22:26:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.030+0000 7f727f400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72780b3b60 con 0x7f727806d710 2024-04-05T22:26:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.030+0000 7f727f400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72780701c0 con 0x7f72781541a0 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f727d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f72780ce000 0x7f727815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50690/0 (socket says 172.21.15.73:50690) 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f727d400640 1 -- 172.21.15.73:0/2515464983 learned_addr learned my addr 172.21.15.73:0/2515464983 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f727ca00640 1 --2- 172.21.15.73:0/2515464983 >> v2:172.21.15.157:3300/0 conn(0x7f72781541a0 0x7f72780b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536500305 0 0) 0x7f727815aa60 con 0x7f72780ce000 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c003520 con 0x7f72780ce000 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271815916 0 0) 0x7f725c003520 con 0x7f72780ce000 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 >> v2:172.21.15.157:3300/0 conn(0x7f72781541a0 msgr2=0x7f72780b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 --2- 172.21.15.73:0/2515464983 >> v2:172.21.15.157:3300/0 conn(0x7f72781541a0 0x7f72780b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.031+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 >> v1:172.21.15.157:6789/0 conn(0x7f727806d710 legacy=0x7f72780cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72780b3880 con 0x7f72780ce000 2024-04-05T22:26:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f72700019f0 con 0x7f72780ce000 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 >> v1:172.21.15.73:6789/0 conn(0x7f72780ce000 legacy=0x7f727815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f725c003970 0x7f725c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725c0047c0 0x7f725c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f725c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780701c0 con 0x7f725c005180 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c00c120 con 0x7f725c003970 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c00c2e0 con 0x7f725c0047c0 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727d400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f725c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727ca00640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f725c003970 0x7f725c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f726fe00640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725c0047c0 0x7f725c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727d400640 1 -- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725c0047c0 msgr2=0x7f725c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727d400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f725c0047c0 0x7f725c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727d400640 1 -- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f725c003970 msgr2=0x7f725c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.032+0000 7f727d400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f725c003970 0x7f725c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.033+0000 7f727d400640 1 -- 172.21.15.73:0/2515464983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c003520 con 0x7f725c005180 2024-04-05T22:26:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.033+0000 7f727d400640 1 -- 172.21.15.73:0/2515464983 reap_dead start 2024-04-05T22:26:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727d400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f725c00b980 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f727000a820 tx=0x7f725c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14725e0a6f69e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700072d0 con 0x7f725c005180 2024-04-05T22:26:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727002f070 con 0x7f725c005180 2024-04-05T22:26:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f7277e00640 1 -- 172.21.15.73:0/2515464983 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700076b0 con 0x7f725c005180 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727f400640 1 -- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 msgr2=0x7f725c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727f400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f725c00b980 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f727000a820 tx=0x7f725c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727f400640 1 -- 172.21.15.73:0/2515464983 shutdown_connections 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727f400640 1 --2- 172.21.15.73:0/2515464983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f725c00b980 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.034+0000 7f727f400640 1 -- 172.21.15.73:0/2515464983 >> 172.21.15.73:0/2515464983 conn(0x7f72780ce8e0 msgr2=0x7f72780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.035+0000 7f727f400640 1 -- 172.21.15.73:0/2515464983 shutdown_connections 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.035+0000 7f727f400640 1 -- 172.21.15.73:0/2515464983 wait complete. 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.036+0000 7f727f400640 1 Processor -- start 2024-04-05T22:26:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.036+0000 7f727f400640 1 -- start start 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727814d200 0x7f727814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72780cc7e0 0x7f72780ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278152b90 con 0x7f725c005180 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780cd1d0 con 0x7f72780cc7e0 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.037+0000 7f727f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727814ef90 con 0x7f727814d200 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51056/0 (socket says 172.21.15.73:51056) 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 -- 172.21.15.73:0/3041710555 learned_addr learned my addr 172.21.15.73:0/3041710555 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f726fe00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727814d200 0x7f727814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727d400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72780cc7e0 0x7f72780ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 -- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727814d200 msgr2=0x7f727814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727814d200 0x7f727814f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 -- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72780cc7e0 msgr2=0x7f72780ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72780cc7e0 0x7f72780ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 -- 172.21.15.73:0/3041710555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727814f820 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.038+0000 7f727ca00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7268011540 tx=0x7f7268011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.039+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268015930 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.039+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268015a90 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.039+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268015e60 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.039+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727814d6e0 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.039+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727814dca0 con 0x7f725c005180 2024-04-05T22:26:29.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.040+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7278066af0 con 0x7f725c005180 2024-04-05T22:26:29.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.041+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f726800f460 con 0x7f725c005180 2024-04-05T22:26:29.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.042+0000 7f7275600640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 0x7f725406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.042+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7268089fb0 con 0x7f725c005180 2024-04-05T22:26:29.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.042+0000 7f726fe00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 0x7f725406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.043+0000 7f726fe00640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 0x7f725406d450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7278150730 tx=0x7f7260009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.047+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7268054cb0 con 0x7f725c005180 2024-04-05T22:26:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.279+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f727814f820 con 0x7f725c005180 2024-04-05T22:26:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.280+0000 7f7275600640 1 -- 172.21.15.73:0/3041710555 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f727814f820 con 0x7f725c005180 2024-04-05T22:26:29.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:29.281 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:29.281 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 msgr2=0x7f725406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 0x7f725406d450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7278150730 tx=0x7f7260009040 comp rx=0 tx=0).stop 2024-04-05T22:26:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 msgr2=0x7f7278152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7268011540 tx=0x7f7268011a00 comp rx=0 tx=0).stop 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 shutdown_connections 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f725406af90 0x7f725406d450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72780cc7e0 0x7f72780ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727814d200 0x7f727814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 --2- 172.21.15.73:0/3041710555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f725c005180 0x7f7278152650 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 >> 172.21.15.73:0/3041710555 conn(0x7f72780ce8e0 msgr2=0x7f72780b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 shutdown_connections 2024-04-05T22:26:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.283+0000 7f727f400640 1 -- 172.21.15.73:0/3041710555 wait complete. 2024-04-05T22:26:29.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:26:29.539 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":6966,"entity":{"name":{"type":"client","num":6966},"addr":{"type":"any","addr":"172.21.15.73:0","nonce":265479964}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.77198599199999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961892172279921,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6966 172.21.15.73:0/265479964","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"y","hostname":"smithi073","pid":"37063","root":"/"}}] 2024-04-05T22:26:29.539 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi073", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37063", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.73:0", "nonce": 265479964, "type": "any" }, "name": { "num": 6966, "type": "client" } }, "id": 6966, "inst": "client.6966 172.21.15.73:0/265479964", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.96189217227992 }, "state": "open", "uptime": 0.771985992 } ] 2024-04-05T22:26:29.540 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-05T22:26:29.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.756+0000 7f58b5e00640 1 Processor -- start 2024-04-05T22:26:29.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.756+0000 7f58b5e00640 1 -- start start 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58b5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58b015cf50 0x7f58b006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58b5e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58b00d09b0 0x7f58b00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58b5e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00cb950 con 0x7f58b00cd090 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58b5e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b00ce990 con 0x7f58b015cf50 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58b5e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b00d1350 con 0x7f58b00d09b0 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58af400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f58b00cd090 0x7f58b00cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50694/0 (socket says 172.21.15.73:50694) 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58af400640 1 -- 172.21.15.73:0/3978579285 learned_addr learned my addr 172.21.15.73:0/3978579285 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58aea00640 1 --2- 172.21.15.73:0/3978579285 >> v2:172.21.15.73:3300/0 conn(0x7f58b015cf50 0x7f58b006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.758+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928533134 0 0) 0x7f58b00cb950 con 0x7f58b00cd090 2024-04-05T22:26:29.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5894003520 con 0x7f58b00cd090 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58a6000640 1 --2- 172.21.15.73:0/3978579285 >> v2:172.21.15.157:3300/0 conn(0x7f58b00d09b0 0x7f58b00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3601142491 0 0) 0x7f5894003520 con 0x7f58b00cd090 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58aea00640 1 -- 172.21.15.73:0/3978579285 >> v2:172.21.15.157:3300/0 conn(0x7f58b00d09b0 msgr2=0x7f58b00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58aea00640 1 --2- 172.21.15.73:0/3978579285 >> v2:172.21.15.157:3300/0 conn(0x7f58b00d09b0 0x7f58b00d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58aea00640 1 -- 172.21.15.73:0/3978579285 >> v1:172.21.15.73:6789/0 conn(0x7f58b00cd090 legacy=0x7f58b00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.759+0000 7f58aea00640 1 -- 172.21.15.73:0/3978579285 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00cdce0 con 0x7f58b015cf50 2024-04-05T22:26:29.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58aea00640 1 --2- 172.21.15.73:0/3978579285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b015cf50 0x7f58b006ab60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f58b0071db0 tx=0x7f58a002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91db34125504e601 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80019f0 con 0x7f58b015cf50 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a0033440 con 0x7f58b015cf50 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58ae000640 1 -- 172.21.15.73:0/3978579285 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00337b0 con 0x7f58b015cf50 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58b5e00640 1 -- 172.21.15.73:0/3978579285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b015cf50 msgr2=0x7f58b006ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.761+0000 7f58b5e00640 1 --2- 172.21.15.73:0/3978579285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b015cf50 0x7f58b006ab60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f58b0071db0 tx=0x7f58a002b7a0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 -- 172.21.15.73:0/3978579285 shutdown_connections 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 --2- 172.21.15.73:0/3978579285 >> v2:172.21.15.157:3300/0 conn(0x7f58b00d09b0 0x7f58b00d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 --2- 172.21.15.73:0/3978579285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b015cf50 0x7f58b006ab60 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 -- 172.21.15.73:0/3978579285 >> 172.21.15.73:0/3978579285 conn(0x7f58b0099a60 msgr2=0x7f58b0099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 -- 172.21.15.73:0/3978579285 shutdown_connections 2024-04-05T22:26:29.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.762+0000 7f58b5e00640 1 -- 172.21.15.73:0/3978579285 wait complete. 2024-04-05T22:26:29.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.763+0000 7f58b5e00640 1 Processor -- start 2024-04-05T22:26:29.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.764+0000 7f58b5e00640 1 -- start start 2024-04-05T22:26:29.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.764+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b00cd090 0x7f58b01575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.764+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.764+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b015cf50 0x7f58b0158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.764+0000 7f58b5e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b00b1a40 con 0x7f58b00d09b0 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58b5e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0157af0 con 0x7f58b015cf50 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58b5e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b01f64e0 con 0x7f58b00cd090 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51076/0 (socket says 172.21.15.73:51076) 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 -- 172.21.15.73:0/1022346863 learned_addr learned my addr 172.21.15.73:0/1022346863 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58aea00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b00cd090 0x7f58b01575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 -- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b00cd090 msgr2=0x7f58b01575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b00cd090 0x7f58b01575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58af400640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b015cf50 0x7f58b0158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 -- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b015cf50 msgr2=0x7f58b0158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b015cf50 0x7f58b0158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.765+0000 7f58a6000640 1 -- 172.21.15.73:0/1022346863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a0009400 con 0x7f58b00d09b0 2024-04-05T22:26:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58a6000640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f589800d800 tx=0x7f589800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800b6c0 con 0x7f58b00d09b0 2024-04-05T22:26:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589800b820 con 0x7f58b00d09b0 2024-04-05T22:26:29.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800bbb0 con 0x7f58b00d09b0 2024-04-05T22:26:29.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b01f67c0 con 0x7f58b00d09b0 2024-04-05T22:26:29.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.766+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b00b1f30 con 0x7f58b00d09b0 2024-04-05T22:26:29.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.767+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b00b20f0 con 0x7f58b00d09b0 2024-04-05T22:26:29.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.768+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f589800bd10 con 0x7f58b00d09b0 2024-04-05T22:26:29.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.769+0000 7f58a7e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:29.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.769+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5898089d90 con 0x7f58b00d09b0 2024-04-05T22:26:29.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.769+0000 7f58aea00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:29.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.770+0000 7f58aea00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 0x7f588c06d420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f58a0035000 tx=0x7f58a002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:29.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:29.774+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5898054bb0 con 0x7f58b00d09b0 2024-04-05T22:26:30.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.022+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f58b0068160 con 0x7f58b00d09b0 2024-04-05T22:26:30.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.041+0000 7f58a7e00640 1 -- 172.21.15.73:0/1022346863 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) v1 ==== 81+0+467314 (secure 0 0 0) 0x7f58980581d0 con 0x7f58b00d09b0 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 11, 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "active_gid": 4107, 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "active_name": "y", 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "active_addrs": { 2024-04-05T22:26:30.050 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6832", 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 37063 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6833", 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 37063 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.051 INFO:teuthology.orchestra.run.smithi073.stdout: "active_addr": "172.21.15.73:6833/37063", 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "active_change": "2024-04-05T22:21:00.469624+0000", 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "active_mgr_features": 4540138322906710015, 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "available": true, 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "standbys": [ 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 4104, 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "x", 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_features": 4540138322906710015, 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "available_modules": [ 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "alerts", 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.052 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How frequently to reexamine health status", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.054 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.055 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.056 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.057 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_destination": { 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_destination", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email address to send alerts to", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.059 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_from_name": { 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_from_name", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "Ceph", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email From: name", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.060 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_host": { 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_host", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP server", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_password": { 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_password", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Password to authenticate with", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.062 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_port": { 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_port", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "465", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP port", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_sender": { 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_sender", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP envelope sender", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.064 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_ssl": { 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_ssl", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.065 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_user": { 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_user", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "User to authenticate as", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.066 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "balancer", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "active": { 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "active", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.067 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically balance PGs across cluster", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_time": { 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_time", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0000", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.068 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "beginning time of day to automatically balance", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_weekday": { 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_weekday", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-04-05T22:26:30.069 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_max_iterations": { 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_max_iterations", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "25", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "250", 2024-04-05T22:26:30.070 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_metrics": { 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_metrics", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "pgs,objects,bytes", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.071 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_step": { 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_step", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.5", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0.001", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "0.999", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "aggressiveness of optimization", 2024-04-05T22:26:30.072 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "end_time": { 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_time", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2359", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ending time of day to automatically balance", 2024-04-05T22:26:30.073 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "end_weekday": { 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_weekday", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-05T22:26:30.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.075 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.076 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.077 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.078 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.079 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "min_score": { 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_score", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "upmap", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-compat", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "none", 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap" 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.081 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Balancer mode", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_ids": { 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_ids", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.082 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-05T22:26:30.083 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_deviation": { 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_deviation", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-05T22:26:30.084 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_optimizations": { 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_optimizations", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm", 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_down_multiplier": { 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_down_multiplier", 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_refresh_rate": { 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_refresh_rate", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.087 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "20", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_starting_port": { 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_starting_port", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.088 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4721", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_ptrace": { 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_ptrace", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.089 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_interval": { 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_interval", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.090 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_memory_target_ratio": { 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_memory_target_ratio", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.091 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.7", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "cgroups_split": { 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cgroups_split", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.092 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "config_checks_enabled": { 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_checks_enabled", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "config_dashboard": { 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_dashboard", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.094 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_alertmanager": { 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_alertmanager", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.095 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_base": { 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_base", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-05T22:26:30.096 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Container image name, without the tag", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_elasticsearch": { 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_elasticsearch", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.097 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "elasticsearch container image", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_grafana": { 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_grafana", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.098 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_haproxy": { 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_haproxy", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.099 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "HAproxy container image", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_agent": { 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_agent", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger agent container image", 2024-04-05T22:26:30.100 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_collector": { 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_collector", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.101 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger collector container image", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_query": { 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_query", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger query container image", 2024-04-05T22:26:30.102 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_keepalived": { 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_keepalived", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Keepalived container image", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.103 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_loki": { 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_loki", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Loki container image", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_node_exporter": { 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_node_exporter", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.105 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_nvmeof": { 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_nvmeof", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Nvme-of container image", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_prometheus": { 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_prometheus", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_promtail": { 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_promtail", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Promtail container image", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.108 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_snmp_gateway": { 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_snmp_gateway", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SNMP Gateway container image", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.109 INFO:teuthology.orchestra.run.smithi073.stdout: "container_init": { 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_init", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Run podman/docker with `--init`", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "daemon_cache_timeout": { 2024-04-05T22:26:30.110 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "daemon_cache_timeout", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "default_cephadm_command_timeout": { 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_cephadm_command_timeout", 2024-04-05T22:26:30.111 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "900", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "default_registry": { 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_registry", 2024-04-05T22:26:30.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "device_cache_timeout": { 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_cache_timeout", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.113 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1800", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache device inventory", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "device_enhanced_scan": { 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_enhanced_scan", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.114 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "facts_cache_timeout": { 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "facts_cache_timeout", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.115 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache host facts data", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "host_check_interval": { 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "host_check_interval", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.116 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to perform a host check", 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "hw_monitoring": { 2024-04-05T22:26:30.117 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hw_monitoring", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "inventory_list_all": { 2024-04-05T22:26:30.118 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "inventory_list_all", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.120 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "log_refresh_metadata": { 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_refresh_metadata", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.121 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-05T22:26:30.122 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.123 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.124 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.125 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.126 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "*", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "max_count_per_host": { 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_count_per_host", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.127 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of daemons per service per host", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd_draining_count": { 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_osd_draining_count", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.128 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "migration_current": { 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "migration_current", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.129 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "internal - do not modify", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "root", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.130 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm-package", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "root" 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "mode for remote execution of cephadm", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "oob_default_addr": { 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "oob_default_addr", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "169.254.1.1", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.131 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "prometheus_alerts_path": { 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus_alerts_path", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.132 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_insecure": { 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_insecure", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.133 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_password": { 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_password", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-05T22:26:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_url": { 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_url", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_username": { 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_username", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-05T22:26:30.136 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "secure_monitoring_stack": { 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "secure_monitoring_stack", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.137 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "service_discovery_port": { 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "service_discovery_port", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8765", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "cephadm service discovery port", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.138 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "ssh_config_file": { 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssh_config_file", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "use_agent": { 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_agent", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.140 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "use_repo_digest": { 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_repo_digest", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.141 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_failed_host_check": { 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_failed_host_check", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if the host check fails", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.142 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_daemons": { 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_daemons", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.143 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_hosts": { 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_hosts", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crash", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.145 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.146 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.147 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.148 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.149 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "retain_interval": { 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retain_interval", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "31536000", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.150 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_recent_interval": { 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_recent_interval", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1209600", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.151 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dashboard", 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.152 INFO:teuthology.orchestra.run.smithi073.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_HOST": { 2024-04-05T22:26:30.153 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-05T22:26:30.154 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_ENABLED": { 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_ENABLED", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.155 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.156 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "ENABLE_BROWSABLE_API": { 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-05T22:26:30.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-05T22:26:30.158 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.159 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.160 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.161 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.165 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_PASSWORD": { 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.166 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.167 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_URL": { 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_URL", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.168 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_USERNAME": { 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.169 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.170 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.171 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.172 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.173 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_HOST": { 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.174 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.175 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.176 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.177 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.178 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.179 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.180 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.182 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_ENABLED": { 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.183 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.186 INFO:teuthology.orchestra.run.smithi073.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "45", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ACCESS_KEY": { 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.188 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SECRET_KEY": { 2024-04-05T22:26:30.189 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SSL_VERIFY": { 2024-04-05T22:26:30.190 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "UNSAFE_TLS_v1_2": { 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-05T22:26:30.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-05T22:26:30.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.193 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.194 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "cross_origin_url": { 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cross_origin_url", 2024-04-05T22:26:30.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "crt_file": { 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crt_file", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.196 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "debug": { 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "debug", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.197 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable/disable debug options", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "jwt_token_ttl": { 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "jwt_token_ttl", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.198 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "28800", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.200 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.202 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.204 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "motd": { 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "motd", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.205 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "The message of the day", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect_resolve_ip_addr": { 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "redirect_resolve_ip_addr", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.206 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-04-05T22:26:30.207 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8080", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.208 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.209 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl_server_port": { 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl_server_port", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8443", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.210 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "redirect", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.211 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect" 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-04-05T22:26:30.212 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "url_prefix": { 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url_prefix", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.213 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_monitoring": { 2024-04-05T22:26:30.214 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_monitoring", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "monitor device health metrics", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.215 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.216 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.217 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.218 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "mark_out_threshold": { 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mark_out_threshold", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.220 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2419200", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": { 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_name", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.221 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "device_health_metrics", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "retention_period": { 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retention_period", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.222 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15552000", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain device health metrics", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_frequency": { 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_frequency", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "self_heal": { 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "self_heal", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.224 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.225 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and check device health", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_threshold": { 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_threshold", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.226 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7257600", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "diskprediction_local", 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.227 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.228 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.229 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.230 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.232 INFO:teuthology.orchestra.run.smithi073.stdout: "predict_interval": { 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predict_interval", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "predictor_model": { 2024-04-05T22:26:30.233 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predictor_model", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "prophetstor", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.234 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "influx", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": false, 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "influxdb python module not found", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "batch_size": { 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "batch_size", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5000", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.236 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "database": { 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "database", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "ceph", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.237 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": { 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hostname", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.238 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server hostname", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "30", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "5", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.239 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.240 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.243 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.244 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "password": { 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "password", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "password of InfluxDB server user", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.245 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "port": { 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "port", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8086", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server port", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.246 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "false", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "threads": { 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threads", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "32", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "username": { 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "username", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "username of InfluxDB server user", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.249 INFO:teuthology.orchestra.run.smithi073.stdout: "verify_ssl": { 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "verify_ssl", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "true", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "insights", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.251 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.252 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.253 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.254 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.255 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "iostat", 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.257 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.258 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.259 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.261 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "localpool", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "failure_domain": { 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "failure_domain", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "host", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.262 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "failure domain for any created local pool", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.263 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.264 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.266 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.267 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.267 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.269 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.269 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.269 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.269 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": { 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_size", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.270 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default min_size for any created local pool", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "num_rep": { 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "num_rep", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default replica count for any created local pool", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": { 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_num", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.272 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "128", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default pg_num for any created local pool", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "prefix": { 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prefix", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.273 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name prefix for any created local pool", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "subtree": { 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subtree", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "rack", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mds_autoscaler", 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.275 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.276 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.277 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.278 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.279 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirroring", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.281 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.283 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.284 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.285 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "nfs", 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.287 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.289 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.290 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "fail_fs": { 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fail_fs", 2024-04-05T22:26:30.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.292 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.293 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.294 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.295 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator": { 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-04-05T22:26:30.297 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "rook", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "test_orchestrator" 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Orchestrator backend", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.298 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_perf_query", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.299 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.302 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.303 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_support", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.304 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.306 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.308 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_autoscaler", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.310 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.311 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.312 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.314 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.315 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "threshold": { 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threshold", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1.0", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "scaling threshold", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-05T22:26:30.316 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "progress", 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_pg_recovery_event": { 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_pg_recovery_event", 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.317 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.318 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.319 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.320 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.321 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.322 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.323 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "max_completed_events": { 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_completed_events", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "50", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "number of past completed events to remember", 2024-04-05T22:26:30.324 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long the module is going to sleep", 2024-04-05T22:26:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus", 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "cache": { 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cache", 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.326 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "exclude_perf_counters": { 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "exclude_perf_counters", 2024-04-05T22:26:30.327 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.328 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.329 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.330 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.331 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.332 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools": { 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.333 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.334 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "300", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_interval": { 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_interval", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.335 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15.0", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.336 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.337 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "9283", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "stale_cache_strategy": { 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stale_cache_strategy", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.338 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "log", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.339 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "default", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "default", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "error" 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-04-05T22:26:30.340 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.342 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.343 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.344 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_snap_create": { 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_snap_create", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.347 INFO:teuthology.orchestra.run.smithi073.stdout: "mirror_snapshot_schedule": { 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirror_snapshot_schedule", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.348 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "trash_purge_schedule": { 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "trash_purge_schedule", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.349 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "restful", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_auth": { 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_auth", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.350 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.351 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.353 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.354 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.355 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.357 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.358 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rgw", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.359 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.360 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.362 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.363 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.364 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rook", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.365 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.366 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.367 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.368 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.370 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "storage_class": { 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "storage_class", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "local", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "selftest", 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.372 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.373 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.374 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.375 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.376 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.377 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "roption1": { 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption1", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.378 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "roption2": { 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption2", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "xyz", 2024-04-05T22:26:30.379 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption1": { 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption1", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.380 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption2": { 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption2", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.381 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption3": { 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption3", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.382 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption4": { 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption4", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.383 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption5": { 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption5", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.384 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption6": { 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption6", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.385 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption7": { 2024-04-05T22:26:30.386 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption7", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "42", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.387 INFO:teuthology.orchestra.run.smithi073.stdout: "testkey": { 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testkey", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "testlkey": { 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testlkey", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "testnewline": { 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testnewline", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.390 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_m_granularity": { 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_m_granularity", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.391 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow minute scheduled snapshots", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "dump_on_update": { 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dump_on_update", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.392 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "dump database to debug log on update", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.395 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.396 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.397 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.398 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stats", 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.399 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.401 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.403 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.404 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "status", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.406 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.407 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.408 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.409 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.410 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.411 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telegraf", 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "address": { 2024-04-05T22:26:30.412 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "address", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.413 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.414 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.415 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.418 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.419 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telemetry", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_basic": { 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_basic", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.421 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_crash": { 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_crash", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_device": { 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_device", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.423 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_ident": { 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_ident", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.424 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_perf": { 2024-04-05T22:26:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_perf", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "contact": { 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "contact", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.427 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "description": { 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "description", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.428 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "device_url": { 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_url", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.429 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.430 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.431 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "24", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "8", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "last_opt_revision": { 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "last_opt_revision", 2024-04-05T22:26:30.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard": { 2024-04-05T22:26:30.433 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard_description": { 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard_description", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.436 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.438 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.439 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.440 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "organization": { 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "organization", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "proxy": { 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "proxy", 2024-04-05T22:26:30.442 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "url": { 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url", 2024-04-05T22:26:30.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "test_orchestrator", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.445 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.446 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.448 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.449 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.450 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.451 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.452 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.453 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.454 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.455 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_clones": { 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_clones", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.456 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_delay": { 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_delay", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.457 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_no_wait": { 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_no_wait", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.458 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix", 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "discovery_interval": { 2024-04-05T22:26:30.459 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "discovery_interval", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "100", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.460 INFO:teuthology.orchestra.run.smithi073.stdout: "identifier": { 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "identifier", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.463 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.465 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.466 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.467 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_host": { 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_host", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.468 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_port": { 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_port", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.469 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10051", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_sender": { 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_sender", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.470 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "modules": [ 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "iostat", 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "nfs", 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "restful" 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "available_modules": [ 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "alerts", 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.472 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How frequently to reexamine health status", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.473 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.474 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.475 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.476 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.477 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.478 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_destination": { 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_destination", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.479 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email address to send alerts to", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_from_name": { 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_from_name", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.480 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "Ceph", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email From: name", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_host": { 2024-04-05T22:26:30.481 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_host", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP server", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.482 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_password": { 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_password", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Password to authenticate with", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_port": { 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_port", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "465", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP port", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.484 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_sender": { 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_sender", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP envelope sender", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.485 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_ssl": { 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_ssl", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.486 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_user": { 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_user", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.487 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "User to authenticate as", 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "balancer", 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.488 INFO:teuthology.orchestra.run.smithi073.stdout: "active": { 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "active", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically balance PGs across cluster", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.489 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_time": { 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_time", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0000", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "beginning time of day to automatically balance", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.490 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_weekday": { 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_weekday", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-05T22:26:30.491 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_max_iterations": { 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_max_iterations", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "25", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "250", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-05T22:26:30.492 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_metrics": { 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_metrics", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "pgs,objects,bytes", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.493 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_step": { 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_step", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.5", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0.001", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "0.999", 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.494 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "aggressiveness of optimization", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "end_time": { 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_time", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2359", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.495 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ending time of day to automatically balance", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "end_weekday": { 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_weekday", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-05T22:26:30.496 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.497 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.498 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.499 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.500 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.501 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "min_score": { 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_score", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.502 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "upmap", 2024-04-05T22:26:30.503 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-compat", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "none", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap" 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Balancer mode", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_ids": { 2024-04-05T22:26:30.504 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_ids", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_deviation": { 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_deviation", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-05T22:26:30.507 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_optimizations": { 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_optimizations", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.508 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm", 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_down_multiplier": { 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_down_multiplier", 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_refresh_rate": { 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_refresh_rate", 2024-04-05T22:26:30.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "20", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_starting_port": { 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_starting_port", 2024-04-05T22:26:30.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4721", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_ptrace": { 2024-04-05T22:26:30.512 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_ptrace", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.513 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_interval": { 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_interval", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-05T22:26:30.514 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_memory_target_ratio": { 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_memory_target_ratio", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.7", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.515 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "cgroups_split": { 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cgroups_split", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.516 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "config_checks_enabled": { 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_checks_enabled", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.517 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "config_dashboard": { 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_dashboard", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_alertmanager": { 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_alertmanager", 2024-04-05T22:26:30.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_base": { 2024-04-05T22:26:30.520 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_base", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Container image name, without the tag", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.521 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_elasticsearch": { 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_elasticsearch", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "elasticsearch container image", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.522 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_grafana": { 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_grafana", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_haproxy": { 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_haproxy", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.524 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "HAproxy container image", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_agent": { 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_agent", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.525 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger agent container image", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_collector": { 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_collector", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.526 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger collector container image", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_query": { 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_query", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.527 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger query container image", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_keepalived": { 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_keepalived", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.528 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Keepalived container image", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_loki": { 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_loki", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.529 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Loki container image", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_node_exporter": { 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_node_exporter", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-05T22:26:30.530 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_nvmeof": { 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_nvmeof", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-05T22:26:30.531 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Nvme-of container image", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_prometheus": { 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_prometheus", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_promtail": { 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_promtail", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-05T22:26:30.533 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Promtail container image", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_snmp_gateway": { 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_snmp_gateway", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.534 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SNMP Gateway container image", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "container_init": { 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_init", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.535 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Run podman/docker with `--init`", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "daemon_cache_timeout": { 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "daemon_cache_timeout", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.536 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "default_cephadm_command_timeout": { 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_cephadm_command_timeout", 2024-04-05T22:26:30.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "900", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "default_registry": { 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_registry", 2024-04-05T22:26:30.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "device_cache_timeout": { 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_cache_timeout", 2024-04-05T22:26:30.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1800", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache device inventory", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "device_enhanced_scan": { 2024-04-05T22:26:30.540 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_enhanced_scan", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "facts_cache_timeout": { 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "facts_cache_timeout", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache host facts data", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.542 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "host_check_interval": { 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "host_check_interval", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to perform a host check", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "hw_monitoring": { 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hw_monitoring", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.544 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "inventory_list_all": { 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "inventory_list_all", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.545 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.546 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "log_refresh_metadata": { 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_refresh_metadata", 2024-04-05T22:26:30.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.548 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.549 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.550 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.551 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.552 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.553 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "*", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "max_count_per_host": { 2024-04-05T22:26:30.554 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_count_per_host", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of daemons per service per host", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd_draining_count": { 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_osd_draining_count", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-05T22:26:30.556 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "migration_current": { 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "migration_current", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "internal - do not modify", 2024-04-05T22:26:30.557 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "root", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.558 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm-package", 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "root" 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "mode for remote execution of cephadm", 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "oob_default_addr": { 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "oob_default_addr", 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.559 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "169.254.1.1", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "prometheus_alerts_path": { 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus_alerts_path", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_insecure": { 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_insecure", 2024-04-05T22:26:30.561 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.562 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_password": { 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_password", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_url": { 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_url", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-05T22:26:30.564 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_username": { 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_username", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.565 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "secure_monitoring_stack": { 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "secure_monitoring_stack", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.566 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "service_discovery_port": { 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "service_discovery_port", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.567 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8765", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "cephadm service discovery port", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "ssh_config_file": { 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssh_config_file", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.568 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "use_agent": { 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_agent", 2024-04-05T22:26:30.569 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.570 INFO:teuthology.orchestra.run.smithi073.stdout: "use_repo_digest": { 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_repo_digest", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_failed_host_check": { 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_failed_host_check", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if the host check fails", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.572 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_daemons": { 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_daemons", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.573 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_hosts": { 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_hosts", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.574 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crash", 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.575 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.576 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.577 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.579 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.581 INFO:teuthology.orchestra.run.smithi073.stdout: "retain_interval": { 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retain_interval", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "31536000", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_recent_interval": { 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_recent_interval", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1209600", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.583 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dashboard", 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-05T22:26:30.584 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.585 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_HOST": { 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.586 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.587 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_ENABLED": { 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_ENABLED", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.588 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.589 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "ENABLE_BROWSABLE_API": { 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-05T22:26:30.590 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-05T22:26:30.591 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.594 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.595 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.596 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.597 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-05T22:26:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-05T22:26:30.599 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.600 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_PASSWORD": { 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_URL": { 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_URL", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.603 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_USERNAME": { 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.604 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.605 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-05T22:26:30.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.607 INFO:teuthology.orchestra.run.smithi073.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.608 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.609 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_HOST": { 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.610 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.611 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.612 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.613 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-05T22:26:30.614 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-05T22:26:30.615 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.616 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.618 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.619 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_ENABLED": { 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-05T22:26:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "45", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ACCESS_KEY": { 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-04-05T22:26:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SECRET_KEY": { 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SSL_VERIFY": { 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-05T22:26:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "UNSAFE_TLS_v1_2": { 2024-04-05T22:26:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "cross_origin_url": { 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cross_origin_url", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "crt_file": { 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crt_file", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "debug": { 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "debug", 2024-04-05T22:26:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable/disable debug options", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "jwt_token_ttl": { 2024-04-05T22:26:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "jwt_token_ttl", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "28800", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "motd": { 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "motd", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "The message of the day", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect_resolve_ip_addr": { 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "redirect_resolve_ip_addr", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8080", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl_server_port": { 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl_server_port", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8443", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "redirect", 2024-04-05T22:26:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect" 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-04-05T22:26:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "url_prefix": { 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url_prefix", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_monitoring": { 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_monitoring", 2024-04-05T22:26:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "monitor device health metrics", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mark_out_threshold": { 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mark_out_threshold", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2419200", 2024-04-05T22:26:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": { 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_name", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "device_health_metrics", 2024-04-05T22:26:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "retention_period": { 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retention_period", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15552000", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain device health metrics", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_frequency": { 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_frequency", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "self_heal": { 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "self_heal", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and check device health", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_threshold": { 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_threshold", 2024-04-05T22:26:30.670 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7257600", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "diskprediction_local", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.672 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.673 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.674 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.677 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.678 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.679 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "predict_interval": { 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predict_interval", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.680 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "predictor_model": { 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predictor_model", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.681 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "prophetstor", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "influx", 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": false, 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "influxdb python module not found", 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "batch_size": { 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "batch_size", 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.684 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5000", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "database": { 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "database", 2024-04-05T22:26:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "ceph", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.686 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": { 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hostname", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server hostname", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "30", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "5", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.688 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.689 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.690 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.692 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.693 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "password": { 2024-04-05T22:26:30.694 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "password", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "password of InfluxDB server user", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "port": { 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "port", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8086", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server port", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "false", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "threads": { 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threads", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "32", 2024-04-05T22:26:30.698 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "username": { 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "username", 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.699 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "username of InfluxDB server user", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "verify_ssl": { 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "verify_ssl", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.700 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "true", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "insights", 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.702 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.703 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.704 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.705 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.706 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.707 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "iostat", 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.708 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.709 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.710 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.712 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.713 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "localpool", 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.714 INFO:teuthology.orchestra.run.smithi073.stdout: "failure_domain": { 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "failure_domain", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "host", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "failure domain for any created local pool", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.715 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.717 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.718 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.719 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.720 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": { 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_size", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.721 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default min_size for any created local pool", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "num_rep": { 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "num_rep", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.722 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default replica count for any created local pool", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": { 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_num", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "128", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.723 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default pg_num for any created local pool", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "prefix": { 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prefix", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.724 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name prefix for any created local pool", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "subtree": { 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subtree", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "rack", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.725 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mds_autoscaler", 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.726 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.727 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirroring", 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.732 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.733 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.734 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.735 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.736 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "nfs", 2024-04-05T22:26:30.738 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.739 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.740 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.741 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.742 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.744 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "fail_fs": { 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fail_fs", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.746 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.747 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.748 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.749 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.750 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator": { 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.751 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rook", 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "test_orchestrator" 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Orchestrator backend", 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_perf_query", 2024-04-05T22:26:30.752 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.753 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.754 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.755 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.756 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.757 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_support", 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.758 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.759 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.760 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.761 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.762 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.763 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_autoscaler", 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.765 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.766 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.767 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.768 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.770 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "threshold": { 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threshold", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1.0", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.771 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "scaling threshold", 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "progress", 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_pg_recovery_event": { 2024-04-05T22:26:30.772 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_pg_recovery_event", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.773 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.774 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.775 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.777 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.779 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "max_completed_events": { 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_completed_events", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "50", 2024-04-05T22:26:30.780 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "number of past completed events to remember", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.781 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long the module is going to sleep", 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.782 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "cache": { 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cache", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.783 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "exclude_perf_counters": { 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "exclude_perf_counters", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.784 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.785 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.786 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.787 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.788 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.789 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.790 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools": { 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.791 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.792 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "300", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_interval": { 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_interval", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.793 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15.0", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.794 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.795 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "9283", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.796 INFO:teuthology.orchestra.run.smithi073.stdout: "stale_cache_strategy": { 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stale_cache_strategy", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "log", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "default", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "default", 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: "error" 2024-04-05T22:26:30.798 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-04-05T22:26:30.799 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.800 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.801 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.802 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.803 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.804 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.805 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_snap_create": { 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_snap_create", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.806 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "mirror_snapshot_schedule": { 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirror_snapshot_schedule", 2024-04-05T22:26:30.807 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.808 INFO:teuthology.orchestra.run.smithi073.stdout: "trash_purge_schedule": { 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "trash_purge_schedule", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.809 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "restful", 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_auth": { 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_auth", 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.810 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-04-05T22:26:30.811 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.814 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.815 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.816 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.817 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.818 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.819 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rgw", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.821 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.822 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.823 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.824 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.825 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rook", 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.827 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.828 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.829 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.830 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.831 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.832 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "storage_class": { 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "storage_class", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "local", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.833 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "selftest", 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.834 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.835 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.836 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.838 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.839 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.840 INFO:teuthology.orchestra.run.smithi073.stdout: "roption1": { 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption1", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.841 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "roption2": { 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption2", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "xyz", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.842 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption1": { 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption1", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.843 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption2": { 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption2", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.844 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption3": { 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption3", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.845 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption4": { 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption4", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.846 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption5": { 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption5", 2024-04-05T22:26:30.847 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.848 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption6": { 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption6", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.849 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption7": { 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption7", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "42", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.850 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "testkey": { 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testkey", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.851 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "testlkey": { 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testlkey", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.852 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "testnewline": { 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testnewline", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.853 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.854 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_m_granularity": { 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_m_granularity", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow minute scheduled snapshots", 2024-04-05T22:26:30.855 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "dump_on_update": { 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dump_on_update", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.856 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "dump database to debug log on update", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.857 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.859 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.860 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.861 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stats", 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.863 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.864 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.865 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.866 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.867 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.868 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "status", 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.870 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.871 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.872 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.873 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.874 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telegraf", 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "address": { 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "address", 2024-04-05T22:26:30.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.876 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.878 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.881 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.881 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.881 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.882 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.883 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.884 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.885 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telemetry", 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_basic": { 2024-04-05T22:26:30.886 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_basic", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.887 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_crash": { 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_crash", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_device": { 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_device", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.889 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_ident": { 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_ident", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.890 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_perf": { 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_perf", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.891 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "contact": { 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "contact", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.892 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "description": { 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "description", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.893 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "device_url": { 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_url", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.894 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.895 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.896 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "24", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "8", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: "last_opt_revision": { 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "last_opt_revision", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.898 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard": { 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.899 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard_description": { 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard_description", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.900 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.901 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.904 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.905 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "organization": { 2024-04-05T22:26:30.906 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "organization", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "proxy": { 2024-04-05T22:26:30.907 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "proxy", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.908 INFO:teuthology.orchestra.run.smithi073.stdout: "url": { 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.909 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "test_orchestrator", 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.910 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.911 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.912 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.913 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.914 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.915 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.916 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.917 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.918 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.919 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.921 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.922 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_clones": { 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_clones", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.923 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_delay": { 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_delay", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.924 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_no_wait": { 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_no_wait", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix", 2024-04-05T22:26:30.926 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "discovery_interval": { 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "discovery_interval", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "100", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.927 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "identifier": { 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "identifier", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.928 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.929 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.930 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.931 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.932 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.933 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.934 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.935 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_host": { 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_host", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.936 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_port": { 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_port", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10051", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.937 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_sender": { 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_sender", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-05T22:26:30.938 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "services": {}, 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "always_on_modules": { 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "octopus": [ 2024-04-05T22:26:30.939 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "pacific": [ 2024-04-05T22:26:30.940 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "quincy": [ 2024-04-05T22:26:30.941 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "reef": [ 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-04-05T22:26:30.942 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_failure_osd_epoch": 0, 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "active_clients": [ 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.943 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3883631467 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "libcephsqlite", 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2340596025 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 745661956 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 206467021 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.946 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3668905466 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.947 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1002310496 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1452282649 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4239333780 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.949 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1767413521 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-04-05T22:26:30.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 307490620 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:30.951 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:26:30.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 msgr2=0x7f588c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:30.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 0x7f588c06d420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f58a0035000 tx=0x7f58a002f000 comp rx=0 tx=0).stop 2024-04-05T22:26:30.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 msgr2=0x7f58b01f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f589800d800 tx=0x7f589800dcc0 comp rx=0 tx=0).stop 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 shutdown_connections 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b015cf50 0x7f58b0158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b00d09b0 0x7f58b01f5a20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 --2- 172.21.15.73:0/1022346863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b00cd090 0x7f58b01575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 >> 172.21.15.73:0/1022346863 conn(0x7f58b0099a60 msgr2=0x7f58b00cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 shutdown_connections 2024-04-05T22:26:30.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:30.051+0000 7f58b5e00640 1 -- 172.21.15.73:0/1022346863 wait complete. 2024-04-05T22:26:30.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2024-04-05T22:26:31.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.175+0000 7f594be00640 1 Processor -- start 2024-04-05T22:26:31.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.175+0000 7f594be00640 1 -- start start 2024-04-05T22:26:31.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f594be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5944160620 0x7f59440caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f594be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f59440ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f594be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440ce5d0 con 0x7f59440ad6b0 2024-04-05T22:26:31.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f594be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440ce010 con 0x7f5944160620 2024-04-05T22:26:31.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f594be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440ce2f0 con 0x7f59440b1870 2024-04-05T22:26:31.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f5949e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f59440ad6b0 0x7f59440cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50700/0 (socket says 172.21.15.73:50700) 2024-04-05T22:26:31.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.177+0000 7f5949e00640 1 -- 172.21.15.73:0/4260984678 learned_addr learned my addr 172.21.15.73:0/4260984678 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:31.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5948a00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f59440ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5949400640 1 --2- 172.21.15.73:0/4260984678 >> v2:172.21.15.157:3300/0 conn(0x7f5944160620 0x7f59440caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947113962 0 0) 0x7f59440ce5d0 con 0x7f59440ad6b0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f592c003520 con 0x7f59440ad6b0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384589658 0 0) 0x7f592c003520 con 0x7f59440ad6b0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 msgr2=0x7f59440ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f59440ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 >> v2:172.21.15.157:3300/0 conn(0x7f5944160620 msgr2=0x7f59440caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 --2- 172.21.15.73:0/4260984678 >> v2:172.21.15.157:3300/0 conn(0x7f5944160620 0x7f59440caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440ac970 con 0x7f59440ad6b0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f593c0019f0 con 0x7f59440ad6b0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 >> v1:172.21.15.73:6789/0 conn(0x7f59440ad6b0 legacy=0x7f59440cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f592c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c003ef0 0x7f592c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c004d00 0x7f592c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440ce010 con 0x7f592c003ef0 2024-04-05T22:26:31.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440ce2f0 con 0x7f592c004d00 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.178+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c00b980 con 0x7f59440b1870 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c003ef0 0x7f592c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5949400640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f592c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5949e00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c004d00 0x7f592c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 -- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 msgr2=0x7f592c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440b1870 0x7f592c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 -- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c004d00 msgr2=0x7f592c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c004d00 0x7f592c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.179+0000 7f5948a00640 1 -- 172.21.15.73:0/4260984678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c003520 con 0x7f592c003ef0 2024-04-05T22:26:31.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.180+0000 7f5948a00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c003ef0 0x7f592c0047c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ac0 tx=0x7f592c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24eaf619e7dad909 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:31.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.180+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593002c6b0 con 0x7f592c003ef0 2024-04-05T22:26:31.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.180+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593002c810 con 0x7f592c003ef0 2024-04-05T22:26:31.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.180+0000 7f5943e00640 1 -- 172.21.15.73:0/4260984678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593002cba0 con 0x7f592c003ef0 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 -- 172.21.15.73:0/4260984678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c003ef0 msgr2=0x7f592c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 --2- 172.21.15.73:0/4260984678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c003ef0 0x7f592c0047c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ac0 tx=0x7f592c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f5949e00640 1 -- 172.21.15.73:0/4260984678 reap_dead start 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 -- 172.21.15.73:0/4260984678 shutdown_connections 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 -- 172.21.15.73:0/4260984678 >> 172.21.15.73:0/4260984678 conn(0x7f59440ce8e0 msgr2=0x7f59440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 -- 172.21.15.73:0/4260984678 shutdown_connections 2024-04-05T22:26:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.181+0000 7f594be00640 1 -- 172.21.15.73:0/4260984678 wait complete. 2024-04-05T22:26:31.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.183+0000 7f594be00640 1 Processor -- start 2024-04-05T22:26:31.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.183+0000 7f594be00640 1 -- start start 2024-04-05T22:26:31.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c003ef0 0x7f5944158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f594415d130 0x7f594415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b0430 con 0x7f594414aae0 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440aec40 con 0x7f594415d130 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f594be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594415a4b0 con 0x7f592c003ef0 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f5948a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f5948a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51092/0 (socket says 172.21.15.73:51092) 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.184+0000 7f5948a00640 1 -- 172.21.15.73:0/2478647277 learned_addr learned my addr 172.21.15.73:0/2478647277 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5949e00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f594415d130 0x7f594415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5949400640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c003ef0 0x7f5944158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 -- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c003ef0 msgr2=0x7f5944158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c003ef0 0x7f5944158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 -- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f594415d130 msgr2=0x7f594415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f594415d130 0x7f594415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 -- 172.21.15.73:0/2478647277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440b0150 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5948a00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5930037040 tx=0x7f5930004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930003810 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930003970 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930003ce0 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440ae580 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.185+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59440afe40 con 0x7f594414aae0 2024-04-05T22:26:31.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.186+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944066af0 con 0x7f594414aae0 2024-04-05T22:26:31.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.188+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 12) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f5930034440 con 0x7f594414aae0 2024-04-05T22:26:31.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.189+0000 7f5941600640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 0x7f592406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.189+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59300ab530 con 0x7f594414aae0 2024-04-05T22:26:31.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.189+0000 7f5949400640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 0x7f592406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.190+0000 7f5949400640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 0x7f592406d4b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5938009990 tx=0x7f5938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:31.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.193+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5930076180 con 0x7f594414aae0 2024-04-05T22:26:31.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.435+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) v1 -- 0x7f5944068160 con 0x7f594414aae0 2024-04-05T22:26:31.445 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:31.445+0000 7f5581e00640 -1 mgr handle_mgr_map I was active but no longer am 2024-04-05T22:26:31.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.449+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 13) v1 ==== 44528+0+0 (secure 0 0 0) 0x7f5930076550 con 0x7f594414aae0 2024-04-05T22:26:31.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.449+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 msgr2=0x7f592406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.449+0000 7f5941600640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 0x7f592406d4b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5938009990 tx=0x7f5938008040 comp rx=0 tx=0).stop 2024-04-05T22:26:31.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.451+0000 7f5941600640 1 -- 172.21.15.73:0/2478647277 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v13) v1 ==== 68+0+0 (secure 0 0 0) 0x7f593007a540 con 0x7f594414aae0 2024-04-05T22:26:31.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.455+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 msgr2=0x7f59440af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.455+0000 7f594be00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5930037040 tx=0x7f5930004730 comp rx=0 tx=0).stop 2024-04-05T22:26:31.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 shutdown_connections 2024-04-05T22:26:31.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:6832/37063,v1:172.21.15.73:6833/37063] conn(0x7f592406aff0 0x7f592406d4b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f594415d130 0x7f594415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f594414aae0 0x7f59440af900 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 --2- 172.21.15.73:0/2478647277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c003ef0 0x7f5944158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.456+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 >> 172.21.15.73:0/2478647277 conn(0x7f59440ce8e0 msgr2=0x7f59440cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:31.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.458+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 shutdown_connections 2024-04-05T22:26:31.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.460+0000 7f594be00640 1 -- 172.21.15.73:0/2478647277 wait complete. 2024-04-05T22:26:31.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:26:31.565+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:26:31.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.679+0000 7f794ba00640 1 Processor -- start 2024-04-05T22:26:31.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.679+0000 7f794ba00640 1 -- start start 2024-04-05T22:26:31.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f794415cf40 0x7f79440cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f79440d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79440b0930 0x7f79440b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79440d11f0 con 0x7f79440b0930 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794406bd40 con 0x7f794415cf40 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.680+0000 7f794ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794415d320 con 0x7f79440d0850 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79440b0930 0x7f79440b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79440b0930 0x7f79440b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51134/0 (socket says 172.21.15.73:51134) 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 -- 172.21.15.73:0/2919391692 learned_addr learned my addr 172.21.15.73:0/2919391692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7943e00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f79440d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949000640 1 --2- 172.21.15.73:0/2919391692 >> v2:172.21.15.157:3300/0 conn(0x7f794415cf40 0x7f79440cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 -- 172.21.15.73:0/2919391692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 msgr2=0x7f79440d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f79440d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 -- 172.21.15.73:0/2919391692 >> v2:172.21.15.157:3300/0 conn(0x7f794415cf40 msgr2=0x7f79440cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:31.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 --2- 172.21.15.73:0/2919391692 >> v2:172.21.15.157:3300/0 conn(0x7f794415cf40 0x7f79440cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7949a00640 1 -- 172.21.15.73:0/2919391692 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440b2340 con 0x7f79440b0930 2024-04-05T22:26:31.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.681+0000 7f7943e00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f79440d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:31.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.682+0000 7f7949a00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f79440b0d90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7944070240 tx=0x7f793c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d07bdb25a50931f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f7943400640 1 -- 172.21.15.73:0/2919391692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c02ca70 con 0x7f79440b0930 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f7943400640 1 -- 172.21.15.73:0/2919391692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c02cbd0 con 0x7f79440b0930 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f7943400640 1 -- 172.21.15.73:0/2919391692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0343f0 con 0x7f79440b0930 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 -- 172.21.15.73:0/2919391692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 msgr2=0x7f79440b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f79440b0d90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7944070240 tx=0x7f793c02bed0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 -- 172.21.15.73:0/2919391692 shutdown_connections 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f79440b0d90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 --2- 172.21.15.73:0/2919391692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f79440d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 --2- 172.21.15.73:0/2919391692 >> v2:172.21.15.157:3300/0 conn(0x7f794415cf40 0x7f79440cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.683+0000 7f794ba00640 1 -- 172.21.15.73:0/2919391692 >> 172.21.15.73:0/2919391692 conn(0x7f7944099a60 msgr2=0x7f7944099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.684+0000 7f794ba00640 1 -- 172.21.15.73:0/2919391692 shutdown_connections 2024-04-05T22:26:31.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.684+0000 7f794ba00640 1 -- 172.21.15.73:0/2919391692 wait complete. 2024-04-05T22:26:31.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.685+0000 7f794ba00640 1 Processor -- start 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.685+0000 7f794ba00640 1 -- start start 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f7944160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f794415cf40 0x7f794415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441f3200 con 0x7f79440b0930 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794415f880 con 0x7f794415cf40 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f794ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794415fb60 con 0x7f79440d0850 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f7949000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f7949000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51144/0 (socket says 172.21.15.73:51144) 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.686+0000 7f7949000640 1 -- 172.21.15.73:0/456143811 learned_addr learned my addr 172.21.15.73:0/456143811 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 -- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 msgr2=0x7f7944160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f7944160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 -- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f794415cf40 msgr2=0x7f794415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f794415cf40 0x7f794415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 -- 172.21.15.73:0/456143811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c009b30 con 0x7f79440b0930 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7949000640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f793800c7f0 tx=0x7f793800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004220 con 0x7f79440b0930 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380043a0 con 0x7f79440b0930 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938007860 con 0x7f79440b0930 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.687+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794415fe40 con 0x7f79440b0930 2024-04-05T22:26:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.688+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794415a800 con 0x7f79440b0930 2024-04-05T22:26:31.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.688+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) v1 ==== 44528+0+0 (secure 0 0 0) 0x7f7938007bd0 con 0x7f79440b0930 2024-04-05T22:26:31.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.689+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7944160080 con 0x7f79440b0930 2024-04-05T22:26:31.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.689+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f793804a660 con 0x7f79440b0930 2024-04-05T22:26:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.694+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7944160080 con 0x7f79440b0930 2024-04-05T22:26:31.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.921+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7944160080 con 0x7f79440b0930 2024-04-05T22:26:31.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.922+0000 7f7940c00640 1 -- 172.21.15.73:0/456143811 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f7944160080 con 0x7f79440b0930 2024-04-05T22:26:31.922 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:31.923 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:31.923 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:31.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 msgr2=0x7f794409e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:31.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f793800c7f0 tx=0x7f793800ccb0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 shutdown_connections 2024-04-05T22:26:31.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f794415cf40 0x7f794415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79440d0850 0x7f7944160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 --2- 172.21.15.73:0/456143811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79440b0930 0x7f794409e210 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:31.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 >> 172.21.15.73:0/456143811 conn(0x7f7944099a60 msgr2=0x7f794406e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:31.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 shutdown_connections 2024-04-05T22:26:31.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:31.925+0000 7f794ba00640 1 -- 172.21.15.73:0/456143811 wait complete. 2024-04-05T22:26:31.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-05T22:26:32.005 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.005+0000 7f7984450040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-05T22:26:32.058 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.058+0000 7f7984450040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-05T22:26:32.179 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":5416,"entity":{"name":{"type":"client","num":5416},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4183978402}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.59398922200000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9972962937554031,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5416 172.21.15.157:0/4183978402","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"x","hostname":"smithi157","pid":"37023","root":"/"}},{"id":7011,"entity":{"name":{"type":"client","num":7011},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":2226463802}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.59398922200000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9972986010810283,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7011 172.21.15.157:0/2226463802","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8919605b0a5dc1f1650c8aec3051c77c0e3e240e","ceph_version":"ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)","entity_id":"x","hostname":"smithi157","pid":"37023","root":"/"}}] 2024-04-05T22:26:32.180 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37023", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4183978402, "type": "any" }, "name": { "num": 5416, "type": "client" } }, "id": 5416, "inst": "client.5416 172.21.15.157:0/4183978402", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.997296293755403 }, "state": "open", "uptime": 0.593989222 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8919605b0a5dc1f1650c8aec3051c77c0e3e240e", "ceph_version": "ceph version 18.2.2-1173-g8919605b (8919605b0a5dc1f1650c8aec3051c77c0e3e240e) reef (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37023", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 2226463802, "type": "any" }, "name": { "num": 7011, "type": "client" } }, "id": 7011, "inst": "client.7011 172.21.15.157:0/2226463802", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9972986010810283 }, "state": "open", "uptime": 0.593989222 } ] 2024-04-05T22:26:32.180 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-05T22:26:32.180 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:26:32.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:32.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:32.408 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:26:32.408 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:32.408 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:32.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:32.462 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:32.462 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:32.506 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.506+0000 7f7984450040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-05T22:26:32.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:32.515 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:32.515 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:32.560 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.560+0000 7f7984450040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-05T22:26:32.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:32.565 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:32.565 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:32.612 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.612+0000 7f7984450040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-05T22:26:32.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:32.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:32.675 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.675+0000 7f7984450040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-05T22:26:32.727 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.727+0000 7f7984450040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-05T22:26:32.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.829+0000 7f71f5800640 1 Processor -- start 2024-04-05T22:26:32.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.829+0000 7f71f5800640 1 -- start start 2024-04-05T22:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71f006a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f5800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00aed30 con 0x7f71f00af190 2024-04-05T22:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f5800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00b3cf0 con 0x7f71f006db60 2024-04-05T22:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00b3ae0 con 0x7f71f015cee0 2024-04-05T22:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f4e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f71f00af190 0x7f71f00ad070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50702/0 (socket says 172.21.15.73:50702) 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.831+0000 7f71f4e00640 1 -- 172.21.15.73:0/1406482059 learned_addr learned my addr 172.21.15.73:0/1406482059 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71e7400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71f006a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975119669 0 0) 0x7f71f00aed30 con 0x7f71f00af190 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d4003520 con 0x7f71f00af190 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481176303 0 0) 0x7f71d4003520 con 0x7f71f00af190 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 msgr2=0x7f71f006a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71f006a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 >> v1:172.21.15.157:6789/0 conn(0x7f71f006db60 legacy=0x7f71f006df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00b3fd0 con 0x7f71f00af190 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.832+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f71e8001670 con 0x7f71f00af190 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 >> v1:172.21.15.73:6789/0 conn(0x7f71f00af190 legacy=0x7f71f00ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71d40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d4004d00 0x7f71d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00b3ae0 con 0x7f71d4003f20 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d400ba60 con 0x7f71d4004d00 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d400bc20 con 0x7f71f015cee0 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71efe00640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71d40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71f4e00640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d4004d00 0x7f71d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 -- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 msgr2=0x7f71d40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f015cee0 0x7f71d40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 -- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d4004d00 msgr2=0x7f71d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d4004d00 0x7f71d4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.833+0000 7f71e7400640 1 -- 172.21.15.73:0/1406482059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4003520 con 0x7f71d4003f20 2024-04-05T22:26:32.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.834+0000 7f71e7400640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71d40047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009a90 tx=0x7f71d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79813c5ac13fc82f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:32.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d802c680 con 0x7f71d4003f20 2024-04-05T22:26:32.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d802c7e0 con 0x7f71d4003f20 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71ef400640 1 -- 172.21.15.73:0/1406482059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d802cb70 con 0x7f71d4003f20 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f5800640 1 -- 172.21.15.73:0/1406482059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 msgr2=0x7f71d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f5800640 1 --2- 172.21.15.73:0/1406482059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71d40047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009a90 tx=0x7f71d40057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f4e00640 1 -- 172.21.15.73:0/1406482059 reap_dead start 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f5800640 1 -- 172.21.15.73:0/1406482059 shutdown_connections 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f5800640 1 -- 172.21.15.73:0/1406482059 >> 172.21.15.73:0/1406482059 conn(0x7f71f00706c0 msgr2=0x7f71f0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.835+0000 7f71f5800640 1 -- 172.21.15.73:0/1406482059 shutdown_connections 2024-04-05T22:26:32.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.836+0000 7f71f5800640 1 -- 172.21.15.73:0/1406482059 wait complete. 2024-04-05T22:26:32.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.838+0000 7f71f5800640 1 Processor -- start 2024-04-05T22:26:32.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.838+0000 7f71f5800640 1 -- start start 2024-04-05T22:26:32.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 0x7f71f01f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f00ceec0 0x7f71f01f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0143af0 con 0x7f71d4003f20 2024-04-05T22:26:32.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01562e0 con 0x7f71f00ceec0 2024-04-05T22:26:32.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71f5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01565c0 con 0x7f71f00cea40 2024-04-05T22:26:32.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51156/0 (socket says 172.21.15.73:51156) 2024-04-05T22:26:32.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71efe00640 1 -- 172.21.15.73:0/1255136073 learned_addr learned my addr 172.21.15.73:0/1255136073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.840+0000 7f71e7400640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 0x7f71f01f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71f4e00640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f00ceec0 0x7f71f01f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 -- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 msgr2=0x7f71f01f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 0x7f71f01f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 -- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f00ceec0 msgr2=0x7f71f01f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f00ceec0 0x7f71f01f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 -- 172.21.15.73:0/1255136073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f01568a0 con 0x7f71d4003f20 2024-04-05T22:26:32.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71e7400640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 0x7f71f01f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71efe00640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e8c0 tx=0x7f71e000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003b80 con 0x7f71d4003f20 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0003ce0 con 0x7f71d4003f20 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00052f0 con 0x7f71d4003f20 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f0156b80 con 0x7f71d4003f20 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.841+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f01f6430 con 0x7f71d4003f20 2024-04-05T22:26:32.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.843+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 44655+0+0 (secure 0 0 0) 0x7f71e0005450 con 0x7f71d4003f20 2024-04-05T22:26:32.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.843+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f004e160 con 0x7f71d4003f20 2024-04-05T22:26:32.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.843+0000 7f71ecc00640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 0x7f71cc039b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:32.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.843+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f71e004a930 con 0x7f71d4003f20 2024-04-05T22:26:32.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.844+0000 7f71e7400640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 0x7f71cc039b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:32.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.845+0000 7f71e7400640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 0x7f71cc039b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71d8036000 tx=0x7f71d8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:32.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:32.849+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f71e001b360 con 0x7f71d4003f20 2024-04-05T22:26:32.864 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.864+0000 7f7984450040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-05T22:26:32.928 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.928+0000 7f7984450040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-05T22:26:32.979 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:32.979+0000 7f7984450040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-05T22:26:33.037 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.037+0000 7f7984450040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-05T22:26:33.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.079+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f00b4ae0 con 0x7f71d4003f20 2024-04-05T22:26:33.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.080+0000 7f71ecc00640 1 -- 172.21.15.73:0/1255136073 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f71e0016120 con 0x7f71d4003f20 2024-04-05T22:26:33.080 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:33.080 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:33.080 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:33.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 msgr2=0x7f71cc039b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 0x7f71cc039b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71d8036000 tx=0x7f71d8030000 comp rx=0 tx=0).stop 2024-04-05T22:26:33.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 msgr2=0x7f71f01f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e8c0 tx=0x7f71e000ed80 comp rx=0 tx=0).stop 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 shutdown_connections 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71cc037690 0x7f71cc039b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f00ceec0 0x7f71f01f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f00cea40 0x7f71f01f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 --2- 172.21.15.73:0/1255136073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d4003f20 0x7f71f01f1930 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 >> 172.21.15.73:0/1255136073 conn(0x7f71f00706c0 msgr2=0x7f71f015c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 shutdown_connections 2024-04-05T22:26:33.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.082+0000 7f71f5800640 1 -- 172.21.15.73:0/1255136073 wait complete. 2024-04-05T22:26:33.089 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.089+0000 7f7984450040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-05T22:26:33.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:33.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.302+0000 7fdb68a00640 1 Processor -- start 2024-04-05T22:26:33.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.303+0000 7fdb68a00640 1 -- start start 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb68a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb6415eab0 0x7fdb6406c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb68a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6406b500 con 0x7fdb64158860 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb68a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb640b20d0 con 0x7fdb6415e620 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb68a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb640ccaf0 con 0x7fdb6415eab0 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb63400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdb64158860 0x7fdb640b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50712/0 (socket says 172.21.15.73:50712) 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb63400640 1 -- 172.21.15.73:0/1319719275 learned_addr learned my addr 172.21.15.73:0/1319719275 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.304+0000 7fdb63e00640 1 --2- 172.21.15.73:0/1319719275 >> v2:172.21.15.73:3300/0 conn(0x7fdb6415eab0 0x7fdb6406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.305+0000 7fdb62a00640 1 -- 172.21.15.73:0/1319719275 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204338887 0 0) 0x7fdb6406b500 con 0x7fdb64158860 2024-04-05T22:26:33.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.305+0000 7fdb62a00640 1 -- 172.21.15.73:0/1319719275 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003520 con 0x7fdb64158860 2024-04-05T22:26:33.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.305+0000 7fdb63e00640 1 -- 172.21.15.73:0/1319719275 >> v1:172.21.15.157:6789/0 conn(0x7fdb6415e620 legacy=0x7fdb6406bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:33.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.305+0000 7fdb63e00640 1 -- 172.21.15.73:0/1319719275 >> v1:172.21.15.73:6789/0 conn(0x7fdb64158860 legacy=0x7fdb640b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.305+0000 7fdb63e00640 1 -- 172.21.15.73:0/1319719275 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb640b23a0 con 0x7fdb6415eab0 2024-04-05T22:26:33.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.306+0000 7fdb63e00640 1 --2- 172.21.15.73:0/1319719275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415eab0 0x7fdb6406c380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d90 tx=0x7fdb5c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9445d7ae1990c5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.306+0000 7fdb62a00640 1 -- 172.21.15.73:0/1319719275 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c02cbd0 con 0x7fdb6415eab0 2024-04-05T22:26:33.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.306+0000 7fdb62a00640 1 -- 172.21.15.73:0/1319719275 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c02cd30 con 0x7fdb6415eab0 2024-04-05T22:26:33.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.306+0000 7fdb62a00640 1 -- 172.21.15.73:0/1319719275 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c02cea0 con 0x7fdb6415eab0 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 -- 172.21.15.73:0/1319719275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415eab0 msgr2=0x7fdb6406c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 --2- 172.21.15.73:0/1319719275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415eab0 0x7fdb6406c380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d90 tx=0x7fdb5c02be20 comp rx=0 tx=0).stop 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 -- 172.21.15.73:0/1319719275 shutdown_connections 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 --2- 172.21.15.73:0/1319719275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415eab0 0x7fdb6406c380 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 -- 172.21.15.73:0/1319719275 >> 172.21.15.73:0/1319719275 conn(0x7fdb640b06c0 msgr2=0x7fdb640b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 -- 172.21.15.73:0/1319719275 shutdown_connections 2024-04-05T22:26:33.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.307+0000 7fdb68a00640 1 -- 172.21.15.73:0/1319719275 wait complete. 2024-04-05T22:26:33.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.309+0000 7fdb68a00640 1 Processor -- start 2024-04-05T22:26:33.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.309+0000 7fdb68a00640 1 -- start start 2024-04-05T22:26:33.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb6415eab0 0x7fdb640adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb6414bfc0 0x7fdb64158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6406f780 con 0x7fdb6415e620 2024-04-05T22:26:33.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640ae430 con 0x7fdb6415eab0 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb68a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb64156550 con 0x7fdb6414bfc0 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51174/0 (socket says 172.21.15.73:51174) 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 -- 172.21.15.73:0/3516514467 learned_addr learned my addr 172.21.15.73:0/3516514467 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 -- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb6414bfc0 msgr2=0x7fdb64158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb6414bfc0 0x7fdb64158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 -- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb6415eab0 msgr2=0x7fdb640adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb6415eab0 0x7fdb640adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.310+0000 7fdb63400640 1 -- 172.21.15.73:0/3516514467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c009a70 con 0x7fdb6415e620 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb63400640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c500 tx=0x7fdb5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54005ce0 con 0x7fdb6415e620 2024-04-05T22:26:33.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb54005e40 con 0x7fdb6415e620 2024-04-05T22:26:33.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb64156830 con 0x7fdb6415e620 2024-04-05T22:26:33.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54007330 con 0x7fdb6415e620 2024-04-05T22:26:33.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.311+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6406fc70 con 0x7fdb6415e620 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.312+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 44655+0+0 (secure 0 0 0) 0x7fdb54007550 con 0x7fdb6415e620 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.312+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6406fe90 con 0x7fdb6415e620 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.313+0000 7fdb5b400640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 0x7fdb40039b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.313+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fdb54049ef0 con 0x7fdb6415e620 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.313+0000 7fdb5be00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 0x7fdb40039b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.314+0000 7fdb5be00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 0x7fdb40039b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb6406d3a0 tx=0x7fdb4c0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.318+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdb5400e280 con 0x7fdb6415e620 2024-04-05T22:26:33.488 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.488+0000 7f7984450040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-05T22:26:33.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.512+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb6404e160 con 0x7fdb6415e620 2024-04-05T22:26:33.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.515+0000 7fdb5b400640 1 -- 172.21.15.73:0/3516514467 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fdb54017020 con 0x7fdb6415e620 2024-04-05T22:26:33.516 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:33.516 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":75,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:31.437790+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/4239333780":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3883631467":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1002310496":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/206467021":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/307490620":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3668905466":"2024-04-06T22:26:31.437732+0000","172.21.15.73:6833/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/2340596025":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/265479964":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1452282649":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1767413521":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/745661956":"2024-04-06T22:26:31.437732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:33.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 msgr2=0x7fdb40039b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 0x7fdb40039b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb6406d3a0 tx=0x7fdb4c0079b0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 msgr2=0x7fdb64155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c500 tx=0x7fdb5400c9c0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 shutdown_connections 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb40037660 0x7fdb40039b20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb6414bfc0 0x7fdb64158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb6415eab0 0x7fdb640adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 --2- 172.21.15.73:0/3516514467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb6415e620 0x7fdb64155e50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 >> 172.21.15.73:0/3516514467 conn(0x7fdb640b06c0 msgr2=0x7fdb64072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 shutdown_connections 2024-04-05T22:26:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.518+0000 7fdb68a00640 1 -- 172.21.15.73:0/3516514467 wait complete. 2024-04-05T22:26:33.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:08.439249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:09.543608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:33.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:33.658 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.658+0000 7f7984450040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-05T22:26:33.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.733+0000 7f25bbe00640 1 Processor -- start 2024-04-05T22:26:33.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.734+0000 7f25bbe00640 1 -- start start 2024-04-05T22:26:33.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 0x7f25bc0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25bbe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25bc0cf900 0x7f25bc0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25bbe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc0d1bf0 con 0x7f25bc15d370 2024-04-05T22:26:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25bbe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0cd6d0 con 0x7f25bc0cf900 2024-04-05T22:26:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25bbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0b3a70 con 0x7f25bc15cf90 2024-04-05T22:26:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25b3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25bc15d370 0x7f25bc0cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50728/0 (socket says 172.21.15.73:50728) 2024-04-05T22:26:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.735+0000 7f25b3e00640 1 -- 172.21.15.73:0/2591508693 learned_addr learned my addr 172.21.15.73:0/2591508693 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25bb400640 1 --2- 172.21.15.73:0/2591508693 >> v2:172.21.15.157:3300/0 conn(0x7f25bc0cf900 0x7f25bc0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557174997 0 0) 0x7f25bc0d1bf0 con 0x7f25bc15d370 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a0003520 con 0x7f25bc15d370 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25baa00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 0x7f25bc0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815274871 0 0) 0x7f25a0003520 con 0x7f25bc15d370 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 msgr2=0x7f25bc0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 0x7f25bc0b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 >> v2:172.21.15.157:3300/0 conn(0x7f25bc0cf900 msgr2=0x7f25bc0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 --2- 172.21.15.73:0/2591508693 >> v2:172.21.15.157:3300/0 conn(0x7f25bc0cf900 0x7f25bc0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0d1350 con 0x7f25bc15d370 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f25a4001670 con 0x7f25bc15d370 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 >> v1:172.21.15.73:6789/0 conn(0x7f25bc15d370 legacy=0x7f25bc0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00047c0 0x7f25a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 0x7f25a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0cd6d0 con 0x7f25a0003970 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0b3a70 con 0x7f25a00047c0 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.736+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a000bac0 con 0x7f25bc15cf90 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 -- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 msgr2=0x7f25a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25bc15cf90 0x7f25a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 -- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00047c0 msgr2=0x7f25a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:33.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00047c0 0x7f25a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.737+0000 7f25baa00640 1 -- 172.21.15.73:0/2591508693 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0003520 con 0x7f25a0003970 2024-04-05T22:26:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.738+0000 7f25baa00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00ba00 tx=0x7f25a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8684ca5ed76891 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.741+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac032630 con 0x7f25a0003970 2024-04-05T22:26:33.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.741+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac032790 con 0x7f25a0003970 2024-04-05T22:26:33.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.741+0000 7f25ba000640 1 -- 172.21.15.73:0/2591508693 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac032b20 con 0x7f25a0003970 2024-04-05T22:26:33.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 -- 172.21.15.73:0/2591508693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a0003970 msgr2=0x7f25a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 --2- 172.21.15.73:0/2591508693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a0003970 0x7f25a0003d90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00ba00 tx=0x7f25a00057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bb400640 1 -- 172.21.15.73:0/2591508693 reap_dead start 2024-04-05T22:26:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 -- 172.21.15.73:0/2591508693 shutdown_connections 2024-04-05T22:26:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 -- 172.21.15.73:0/2591508693 >> 172.21.15.73:0/2591508693 conn(0x7f25bc0d2ac0 msgr2=0x7f25bc0d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 -- 172.21.15.73:0/2591508693 shutdown_connections 2024-04-05T22:26:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.742+0000 7f25bbe00640 1 -- 172.21.15.73:0/2591508693 wait complete. 2024-04-05T22:26:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.744+0000 7f25bbe00640 1 Processor -- start 2024-04-05T22:26:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.744+0000 7f25bbe00640 1 -- start start 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a0003970 0x7f25bc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc0cdfe0 0x7f25bc15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15e180 con 0x7f25bc159b80 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc160700 con 0x7f25bc0cdfe0 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25bbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0cdb50 con 0x7f25a0003970 2024-04-05T22:26:33.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51190/0 (socket says 172.21.15.73:51190) 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.745+0000 7f25b3e00640 1 -- 172.21.15.73:0/884468029 learned_addr learned my addr 172.21.15.73:0/884468029 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 -- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a0003970 msgr2=0x7f25bc15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a0003970 0x7f25bc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 -- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc0cdfe0 msgr2=0x7f25bc15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc0cdfe0 0x7f25bc15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 -- 172.21.15.73:0/884468029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc15a000 con 0x7f25bc159b80 2024-04-05T22:26:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b3e00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c7d0 tx=0x7f25a400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.746+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40074e0 con 0x7f25bc159b80 2024-04-05T22:26:33.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.747+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a4007640 con 0x7f25bc159b80 2024-04-05T22:26:33.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.747+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc15e390 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.747+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4004a20 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.747+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc15e950 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.748+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 44863+0+0 (secure 0 0 0) 0x7f25a4004b80 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.748+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc067250 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.748+0000 7f25b2a00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 0x7f2598039d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.748+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f25a404a350 con 0x7f25bc159b80 2024-04-05T22:26:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.749+0000 7f25baa00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 0x7f2598039d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.750+0000 7f25baa00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 0x7f2598039d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25a00057a0 tx=0x7f25ac032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:33.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.753+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f25a401a2c0 con 0x7f25bc159b80 2024-04-05T22:26:33.772 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.772+0000 7f7984450040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-05T22:26:33.823 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.823+0000 7f7984450040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-05T22:26:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.946+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25bc04e160 con 0x7f25bc159b80 2024-04-05T22:26:33.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.946+0000 7f25b2a00640 1 -- 172.21.15.73:0/884468029 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25a40180a0 con 0x7f25bc159b80 2024-04-05T22:26:33.947 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 msgr2=0x7f2598039d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 0x7f2598039d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25a00057a0 tx=0x7f25ac032000 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 msgr2=0x7f25bc1601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c7d0 tx=0x7f25a400cc90 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 shutdown_connections 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25980378d0 0x7f2598039d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc0cdfe0 0x7f25bc15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25bc159b80 0x7f25bc1601c0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 --2- 172.21.15.73:0/884468029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a0003970 0x7f25bc15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 >> 172.21.15.73:0/884468029 conn(0x7f25bc0d2ac0 msgr2=0x7f25bc0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 shutdown_connections 2024-04-05T22:26:33.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:33.950+0000 7f25bbe00640 1 -- 172.21.15.73:0/884468029 wait complete. 2024-04-05T22:26:33.951 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:33.949+0000 7f7984450040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:26:33.966 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:26:34.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:34.062+0000 7f7984450040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-05T22:26:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.157+0000 7f8f78e00640 1 Processor -- start 2024-04-05T22:26:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.157+0000 7f8f78e00640 1 -- start start 2024-04-05T22:26:34.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.158+0000 7f8f78e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f740d1910 0x7f8f740d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f78e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 0x7f8f740b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f78e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f740ceec0 con 0x7f8f7415cf50 2024-04-05T22:26:34.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f78e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740ce590 con 0x7f8f740d1910 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f78e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740ce870 con 0x7f8f740d0bb0 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f740d1910 0x7f8f740d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f740d1910 0x7f8f740d38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51206/0 (socket says 172.21.15.73:51206) 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 -- 172.21.15.73:0/239960095 learned_addr learned my addr 172.21.15.73:0/239960095 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f7415cf50 0x7f8f740d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50734/0 (socket says 172.21.15.73:50734) 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f72800640 1 --2- 172.21.15.73:0/239960095 >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 0x7f8f740b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 -- 172.21.15.73:0/239960095 >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 msgr2=0x7f8f740b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 --2- 172.21.15.73:0/239960095 >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 0x7f8f740b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 -- 172.21.15.73:0/239960095 >> v1:172.21.15.73:6789/0 conn(0x7f8f7415cf50 legacy=0x7f8f740d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f71400640 1 -- 172.21.15.73:0/239960095 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740ceb50 con 0x7f8f740d1910 2024-04-05T22:26:34.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.159+0000 7f8f72800640 1 --2- 172.21.15.73:0/239960095 >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 0x7f8f740b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:34.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.160+0000 7f8f71400640 1 --2- 172.21.15.73:0/239960095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f740d1910 0x7f8f740d38f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8f740cee30 tx=0x7f8f6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff79941e8ff48627 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f70a00640 1 -- 172.21.15.73:0/239960095 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60031870 con 0x7f8f740d1910 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f70a00640 1 -- 172.21.15.73:0/239960095 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f600319d0 con 0x7f8f740d1910 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f70a00640 1 -- 172.21.15.73:0/239960095 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60031d60 con 0x7f8f740d1910 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 -- 172.21.15.73:0/239960095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f740d1910 msgr2=0x7f8f740d38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 --2- 172.21.15.73:0/239960095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f740d1910 0x7f8f740d38f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8f740cee30 tx=0x7f8f6002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 -- 172.21.15.73:0/239960095 shutdown_connections 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 --2- 172.21.15.73:0/239960095 >> v2:172.21.15.157:3300/0 conn(0x7f8f740d0bb0 0x7f8f740b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 --2- 172.21.15.73:0/239960095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f740d1910 0x7f8f740d38f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 -- 172.21.15.73:0/239960095 >> 172.21.15.73:0/239960095 conn(0x7f8f740d2dd0 msgr2=0x7f8f740d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 -- 172.21.15.73:0/239960095 shutdown_connections 2024-04-05T22:26:34.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.161+0000 7f8f78e00640 1 -- 172.21.15.73:0/239960095 wait complete. 2024-04-05T22:26:34.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.163+0000 7f8f78e00640 1 Processor -- start 2024-04-05T22:26:34.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.163+0000 7f8f78e00640 1 -- start start 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f740d0bb0 0x7f8f74155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f740d1910 0x7f8f740cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74155e70 con 0x7f8f7415cf50 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740b73c0 con 0x7f8f740d1910 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f78e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740c8d90 con 0x7f8f740d0bb0 2024-04-05T22:26:34.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.164+0000 7f8f72800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51212/0 (socket says 172.21.15.73:51212) 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 -- 172.21.15.73:0/2378137450 learned_addr learned my addr 172.21.15.73:0/2378137450 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f71400640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f740d1910 0x7f8f740cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f71e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f740d0bb0 0x7f8f74155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 -- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f740d0bb0 msgr2=0x7f8f74155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f740d0bb0 0x7f8f74155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 -- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f740d1910 msgr2=0x7f8f740cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f740d1910 0x7f8f740cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 -- 172.21.15.73:0/2378137450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600094d0 con 0x7f8f7415cf50 2024-04-05T22:26:34.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f72800640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00ce80 tx=0x7f8f6c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.165+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00d840 con 0x7f8f7415cf50 2024-04-05T22:26:34.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.166+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00d9e0 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.166+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00dd90 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.166+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740c9070 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.166+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f740b7660 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.167+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 44863+0+0 (secure 0 0 0) 0x7f8f6c015440 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.167+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74068160 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.167+0000 7f8f6a000640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 0x7f8f54039d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.167+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8f6c04b640 con 0x7f8f7415cf50 2024-04-05T22:26:34.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.168+0000 7f8f71e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 0x7f8f54039d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.169+0000 7f8f71e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 0x7f8f54039d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007fd0 tx=0x7f8f64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.172+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8f6c008320 con 0x7f8f7415cf50 2024-04-05T22:26:34.248 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:34.248+0000 7f7984450040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-05T22:26:34.298 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:34.298+0000 7f7984450040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-05T22:26:34.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.364+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f740b4610 con 0x7f8f7415cf50 2024-04-05T22:26:34.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.364+0000 7f8f6a000640 1 -- 172.21.15.73:0/2378137450 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f6c01a0a0 con 0x7f8f7415cf50 2024-04-05T22:26:34.365 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 msgr2=0x7f8f54039d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 0x7f8f54039d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007fd0 tx=0x7f8f64009040 comp rx=0 tx=0).stop 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 msgr2=0x7f8f740cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00ce80 tx=0x7f8f6c00a710 comp rx=0 tx=0).stop 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 shutdown_connections 2024-04-05T22:26:34.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f540378a0 0x7f8f54039d60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f7415cf50 0x7f8f740cc3b0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f740d1910 0x7f8f740cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 --2- 172.21.15.73:0/2378137450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f740d0bb0 0x7f8f74155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 >> 172.21.15.73:0/2378137450 conn(0x7f8f740d2dd0 msgr2=0x7f8f740cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 shutdown_connections 2024-04-05T22:26:34.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.367+0000 7f8f78e00640 1 -- 172.21.15.73:0/2378137450 wait complete. 2024-04-05T22:26:34.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:26:34.446 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:34.446+0000 7f7984450040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-05T22:26:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.595+0000 7fc282000640 1 Processor -- start 2024-04-05T22:26:34.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.595+0000 7fc282000640 1 -- start start 2024-04-05T22:26:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc282000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27c06b7b0 0x7fc27c06bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc282000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc282000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c0b0500 con 0x7fc27c0b4430 2024-04-05T22:26:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc282000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0b3f30 con 0x7fc27c06b7b0 2024-04-05T22:26:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc282000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b4fd0 con 0x7fc27c0b0990 2024-04-05T22:26:34.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc27be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.597+0000 7fc27b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27c06b7b0 0x7fc27c06bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36366/0 (socket says 172.21.15.73:36366) 2024-04-05T22:26:34.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27aa00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc27c0b4430 0x7fc27c0b48c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:49658/0 (socket says 172.21.15.73:49658) 2024-04-05T22:26:34.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27be00640 1 -- 172.21.15.73:0/2682996222 learned_addr learned my addr 172.21.15.73:0/2682996222 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27b400640 1 -- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 msgr2=0x7fc27c0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27b400640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27b400640 1 -- 172.21.15.73:0/2682996222 >> v1:172.21.15.157:6789/0 conn(0x7fc27c0b4430 legacy=0x7fc27c0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27b400640 1 -- 172.21.15.73:0/2682996222 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c06c0d0 con 0x7fc27c06b7b0 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.598+0000 7fc27be00640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:26:34.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.599+0000 7fc27b400640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c06b7b0 0x7fc27c06bb90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0097a0 tx=0x7fc26c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4c51203b947a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.600+0000 7fc27a000640 1 -- 172.21.15.73:0/2682996222 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c02d040 con 0x7fc27c06b7b0 2024-04-05T22:26:34.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.600+0000 7fc27a000640 1 -- 172.21.15.73:0/2682996222 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c031810 con 0x7fc27c06b7b0 2024-04-05T22:26:34.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.600+0000 7fc27a000640 1 -- 172.21.15.73:0/2682996222 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c031ba0 con 0x7fc27c06b7b0 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 -- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c06b7b0 msgr2=0x7fc27c06bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c06b7b0 0x7fc27c06bb90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0097a0 tx=0x7fc26c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 -- 172.21.15.73:0/2682996222 shutdown_connections 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c0b0990 0x7fc27c0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 --2- 172.21.15.73:0/2682996222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c06b7b0 0x7fc27c06bb90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 -- 172.21.15.73:0/2682996222 >> 172.21.15.73:0/2682996222 conn(0x7fc27c099a60 msgr2=0x7fc27c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 -- 172.21.15.73:0/2682996222 shutdown_connections 2024-04-05T22:26:34.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.601+0000 7fc282000640 1 -- 172.21.15.73:0/2682996222 wait complete. 2024-04-05T22:26:34.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.603+0000 7fc282000640 1 Processor -- start 2024-04-05T22:26:34.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.603+0000 7fc282000640 1 -- start start 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c06b7b0 0x7fc27c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 0x7fc27c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c159b80 con 0x7fc27c0b0990 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0cb170 con 0x7fc27c0b4430 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc282000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0cb450 con 0x7fc27c06b7b0 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc27aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc27aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51218/0 (socket says 172.21.15.73:51218) 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.604+0000 7fc27aa00640 1 -- 172.21.15.73:0/2660163077 learned_addr learned my addr 172.21.15.73:0/2660163077 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27b400640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c06b7b0 0x7fc27c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27be00640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 0x7fc27c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 -- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c06b7b0 msgr2=0x7fc27c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c06b7b0 0x7fc27c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 -- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 msgr2=0x7fc27c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 0x7fc27c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 -- 172.21.15.73:0/2660163077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009b30 con 0x7fc27c0b0990 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27be00640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 0x7fc27c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:34.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc27aa00640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c3c0 tx=0x7fc26800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680074e0 con 0x7fc27c0b0990 2024-04-05T22:26:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268007640 con 0x7fc27c0b0990 2024-04-05T22:26:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c009400 con 0x7fc27c0b0990 2024-04-05T22:26:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268004a00 con 0x7fc27c0b0990 2024-04-05T22:26:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.605+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c0cba10 con 0x7fc27c0b0990 2024-04-05T22:26:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.607+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c068160 con 0x7fc27c0b0990 2024-04-05T22:26:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.607+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 44863+0+0 (secure 0 0 0) 0x7fc268004b60 con 0x7fc27c0b0990 2024-04-05T22:26:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.607+0000 7fc273400640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 0x7fc25c039d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.607+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fc26804a610 con 0x7fc27c0b0990 2024-04-05T22:26:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.608+0000 7fc27b400640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 0x7fc25c039d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.609+0000 7fc27b400640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 0x7fc25c039d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036000 tx=0x7fc26c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.612+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc2680192c0 con 0x7fc27c0b0990 2024-04-05T22:26:34.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.808+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc27c0afcb0 con 0x7fc27c0b0990 2024-04-05T22:26:34.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.809+0000 7fc273400640 1 -- 172.21.15.73:0/2660163077 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc26801aab0 con 0x7fc27c0b0990 2024-04-05T22:26:34.809 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 msgr2=0x7fc25c039d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 0x7fc25c039d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036000 tx=0x7fc26c03c040 comp rx=0 tx=0).stop 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 msgr2=0x7fc27c0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c3c0 tx=0x7fc26800c880 comp rx=0 tx=0).stop 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 shutdown_connections 2024-04-05T22:26:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc25c0378a0 0x7fc25c039d60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27c0b4430 0x7fc27c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc27c0b0990 0x7fc27c0cd850 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 --2- 172.21.15.73:0/2660163077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27c06b7b0 0x7fc27c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 >> 172.21.15.73:0/2660163077 conn(0x7fc27c099a60 msgr2=0x7fc27c15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 shutdown_connections 2024-04-05T22:26:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:34.812+0000 7fc282000640 1 -- 172.21.15.73:0/2660163077 wait complete. 2024-04-05T22:26:34.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:35.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.027+0000 7f079f800640 1 Processor -- start 2024-04-05T22:26:35.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.027+0000 7f079f800640 1 -- start start 2024-04-05T22:26:35.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.028+0000 7f079f800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07981587d0 0x7f07980cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.028+0000 7f079f800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079806f5b0 con 0x7f079815e850 2024-04-05T22:26:35.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.028+0000 7f079f800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0798070410 con 0x7f07980ccd20 2024-04-05T22:26:35.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.028+0000 7f079f800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0798158bb0 con 0x7f07981587d0 2024-04-05T22:26:35.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f079d800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f079815e850 0x7f07980708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50742/0 (socket says 172.21.15.73:50742) 2024-04-05T22:26:35.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f079d800640 1 -- 172.21.15.73:0/761336127 learned_addr learned my addr 172.21.15.73:0/761336127 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:35.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335222915 0 0) 0x7f079806f5b0 con 0x7f079815e850 2024-04-05T22:26:35.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0780003520 con 0x7f079815e850 2024-04-05T22:26:35.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f079ce00640 1 --2- 172.21.15.73:0/761336127 >> v2:172.21.15.157:3300/0 conn(0x7f07981587d0 0x7f07980cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1944749247 0 0) 0x7f0780003520 con 0x7f079815e850 2024-04-05T22:26:35.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 >> v2:172.21.15.157:3300/0 conn(0x7f07981587d0 msgr2=0x7f07980cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 --2- 172.21.15.73:0/761336127 >> v2:172.21.15.157:3300/0 conn(0x7f07981587d0 0x7f07980cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 >> v1:172.21.15.157:6789/0 conn(0x7f07980ccd20 legacy=0x7f079806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079806f800 con 0x7f079815e850 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.029+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f07900019f0 con 0x7f079815e850 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 >> v1:172.21.15.73:6789/0 conn(0x7f079815e850 legacy=0x7f07980708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0780003970 0x7f0780003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07800047c0 0x7f0780004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 0x7f078000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798158bb0 con 0x7f0780005180 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078000c0d0 con 0x7f07800047c0 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078000c290 con 0x7f0780003970 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 0x7f078000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797e00640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07800047c0 0x7f0780004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079ce00640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0780003970 0x7f0780003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 -- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0780003970 msgr2=0x7f0780003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0780003970 0x7f0780003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 -- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07800047c0 msgr2=0x7f0780004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07800047c0 0x7f0780004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f079d800640 1 -- 172.21.15.73:0/761336127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780003520 con 0x7f0780005180 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.030+0000 7f0797e00640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07800047c0 0x7f0780004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.031+0000 7f079d800640 1 -- 172.21.15.73:0/761336127 reap_dead start 2024-04-05T22:26:35.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.031+0000 7f079d800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 0x7f078000b980 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f079000a820 tx=0x7f07800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437d7d5b396d1921 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07900072d0 con 0x7f0780005180 2024-04-05T22:26:35.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079002f070 con 0x7f0780005180 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f0797400640 1 -- 172.21.15.73:0/761336127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07900076b0 con 0x7f0780005180 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f079f800640 1 -- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 msgr2=0x7f078000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f079f800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 0x7f078000b980 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f079000a820 tx=0x7f07800057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f079f800640 1 -- 172.21.15.73:0/761336127 shutdown_connections 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f079f800640 1 --2- 172.21.15.73:0/761336127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0780005180 0x7f078000b980 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.032+0000 7f079f800640 1 -- 172.21.15.73:0/761336127 >> 172.21.15.73:0/761336127 conn(0x7f07980ce8e0 msgr2=0x7f07980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.033+0000 7f079f800640 1 -- 172.21.15.73:0/761336127 shutdown_connections 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.033+0000 7f079f800640 1 -- 172.21.15.73:0/761336127 wait complete. 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.034+0000 7f079f800640 1 Processor -- start 2024-04-05T22:26:35.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.034+0000 7f079f800640 1 -- start start 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 0x7f07981f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f079813a070 0x7f07980b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798159c00 con 0x7f079813a4f0 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798159260 con 0x7f0780005180 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.035+0000 7f079f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0798159540 con 0x7f079813a070 2024-04-05T22:26:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51234/0 (socket says 172.21.15.73:51234) 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 -- 172.21.15.73:0/2853931221 learned_addr learned my addr 172.21.15.73:0/2853931221 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f0797e00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f079813a070 0x7f07980b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079ce00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 0x7f07981f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 -- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f079813a070 msgr2=0x7f07980b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f079813a070 0x7f07980b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 -- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 msgr2=0x7f07981f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 0x7f07981f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 -- 172.21.15.73:0/2853931221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798159820 con 0x7f079813a4f0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079ce00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 0x7f07981f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f079d800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0790000f80 tx=0x7f079806b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.036+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790001930 con 0x7f079813a4f0 2024-04-05T22:26:35.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.037+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790004b60 con 0x7f079813a4f0 2024-04-05T22:26:35.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.037+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790007400 con 0x7f079813a4f0 2024-04-05T22:26:35.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.037+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07981f2d70 con 0x7f079813a4f0 2024-04-05T22:26:35.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.037+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07980b2a60 con 0x7f079813a4f0 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.038+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 44863+0+0 (secure 0 0 0) 0x7f0790031030 con 0x7f079813a4f0 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.038+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0798066af0 con 0x7f079813a4f0 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.038+0000 7f0794c00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 0x7f0778039d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.038+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f079006ca60 con 0x7f079813a4f0 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.039+0000 7f079ce00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 0x7f0778039d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.040+0000 7f079ce00640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 0x7f0778039d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f078c006fd0 tx=0x7f078c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.043+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f079000a450 con 0x7f079813a4f0 2024-04-05T22:26:35.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.260+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0798066af0 con 0x7f079813a4f0 2024-04-05T22:26:35.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.261+0000 7f0794c00640 1 -- 172.21.15.73:0/2853931221 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f079003e410 con 0x7f079813a4f0 2024-04-05T22:26:35.262 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:35.262 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 msgr2=0x7f0778039d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 0x7f0778039d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f078c006fd0 tx=0x7f078c008040 comp rx=0 tx=0).stop 2024-04-05T22:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 msgr2=0x7f07980c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0790000f80 tx=0x7f079806b640 comp rx=0 tx=0).stop 2024-04-05T22:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 shutdown_connections 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07780378a0 0x7f0778039d60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f079813a4f0 0x7f07980c3ff0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f079813a070 0x7f07980b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 --2- 172.21.15.73:0/2853931221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0780005180 0x7f07981f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 >> 172.21.15.73:0/2853931221 conn(0x7f07980ce8e0 msgr2=0x7f079806c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 shutdown_connections 2024-04-05T22:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.264+0000 7f079f800640 1 -- 172.21.15.73:0/2853931221 wait complete. 2024-04-05T22:26:35.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:35.287 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:35.287+0000 7f7984450040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-05T22:26:35.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.472+0000 7f2c84e00640 1 Processor -- start 2024-04-05T22:26:35.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.472+0000 7f2c84e00640 1 -- start start 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c84e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c801541a0 0x7f2c800b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c84e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c8006d710 0x7f2c800cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c84e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8015aa60 con 0x7f2c8015aef0 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c84e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800b3b60 con 0x7f2c801541a0 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c84e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800701c0 con 0x7f2c8006d710 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c7de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c801541a0 0x7f2c800b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c7de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c801541a0 0x7f2c800b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51238/0 (socket says 172.21.15.73:51238) 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.473+0000 7f2c7de00640 1 -- 172.21.15.73:0/161257207 learned_addr learned my addr 172.21.15.73:0/161257207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.474+0000 7f2c7de00640 1 -- 172.21.15.73:0/161257207 >> v2:172.21.15.157:3300/0 conn(0x7f2c8006d710 msgr2=0x7f2c800cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.474+0000 7f2c7de00640 1 --2- 172.21.15.73:0/161257207 >> v2:172.21.15.157:3300/0 conn(0x7f2c8006d710 0x7f2c800cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.474+0000 7f2c7de00640 1 -- 172.21.15.73:0/161257207 >> v1:172.21.15.157:6789/0 conn(0x7f2c8015aef0 legacy=0x7f2c8015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.474+0000 7f2c7de00640 1 -- 172.21.15.73:0/161257207 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800b3880 con 0x7f2c801541a0 2024-04-05T22:26:35.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.475+0000 7f2c7de00640 1 --2- 172.21.15.73:0/161257207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c801541a0 0x7f2c800b3050 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2c8015e980 tx=0x7f2c7002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b509c1ca5631b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.475+0000 7f2c7d400640 1 -- 172.21.15.73:0/161257207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7002eea0 con 0x7f2c801541a0 2024-04-05T22:26:35.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.475+0000 7f2c7d400640 1 -- 172.21.15.73:0/161257207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70036440 con 0x7f2c801541a0 2024-04-05T22:26:35.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.475+0000 7f2c7d400640 1 -- 172.21.15.73:0/161257207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700367d0 con 0x7f2c801541a0 2024-04-05T22:26:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 -- 172.21.15.73:0/161257207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c801541a0 msgr2=0x7f2c800b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 --2- 172.21.15.73:0/161257207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c801541a0 0x7f2c800b3050 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2c8015e980 tx=0x7f2c7002e0f0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 -- 172.21.15.73:0/161257207 shutdown_connections 2024-04-05T22:26:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 --2- 172.21.15.73:0/161257207 >> v2:172.21.15.157:3300/0 conn(0x7f2c8006d710 0x7f2c800cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 --2- 172.21.15.73:0/161257207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c801541a0 0x7f2c800b3050 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 -- 172.21.15.73:0/161257207 >> 172.21.15.73:0/161257207 conn(0x7f2c800ce8e0 msgr2=0x7f2c800cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:35.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 -- 172.21.15.73:0/161257207 shutdown_connections 2024-04-05T22:26:35.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.476+0000 7f2c84e00640 1 -- 172.21.15.73:0/161257207 wait complete. 2024-04-05T22:26:35.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.478+0000 7f2c84e00640 1 Processor -- start 2024-04-05T22:26:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.478+0000 7f2c84e00640 1 -- start start 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c801541a0 0x7f2c801522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c8015aef0 0x7f2c80152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80151250 con 0x7f2c8006d710 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80151530 con 0x7f2c801541a0 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c84e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80151810 con 0x7f2c8015aef0 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51240/0 (socket says 172.21.15.73:51240) 2024-04-05T22:26:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 -- 172.21.15.73:0/730779082 learned_addr learned my addr 172.21.15.73:0/730779082 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 -- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c8015aef0 msgr2=0x7f2c80152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c8015aef0 0x7f2c80152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 -- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c801541a0 msgr2=0x7f2c801522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c801541a0 0x7f2c801522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.479+0000 7f2c7de00640 1 -- 172.21.15.73:0/730779082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7000b770 con 0x7f2c8006d710 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c7de00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2c7002e130 tx=0x7f2c70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70034950 con 0x7f2c8006d710 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70034ab0 con 0x7f2c8006d710 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70034e60 con 0x7f2c8006d710 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80151af0 con 0x7f2c8006d710 2024-04-05T22:26:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.480+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80146820 con 0x7f2c8006d710 2024-04-05T22:26:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.481+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 44863+0+0 (secure 0 0 0) 0x7f2c70041480 con 0x7f2c8006d710 2024-04-05T22:26:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.481+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80066af0 con 0x7f2c8006d710 2024-04-05T22:26:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.481+0000 7f2c76000640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 0x7f2c5c039d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.482+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f2c7006def0 con 0x7f2c8006d710 2024-04-05T22:26:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.482+0000 7f2c77e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 0x7f2c5c039d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.483+0000 7f2c77e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 0x7f2c5c039d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c80153bb0 tx=0x7f2c68006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.487+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2c7003f930 con 0x7f2c8006d710 2024-04-05T22:26:35.608 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:35.608+0000 7f7984450040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-05T22:26:35.669 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:35.668+0000 7f7984450040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-05T22:26:35.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.672+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c80066af0 con 0x7f2c8006d710 2024-04-05T22:26:35.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.675+0000 7f2c76000640 1 -- 172.21.15.73:0/730779082 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f2c7003b030 con 0x7f2c8006d710 2024-04-05T22:26:35.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:35.676 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":75,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:31.437790+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/4239333780":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3883631467":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1002310496":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/206467021":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/307490620":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3668905466":"2024-04-06T22:26:31.437732+0000","172.21.15.73:6833/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/2340596025":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/265479964":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1452282649":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1767413521":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/745661956":"2024-04-06T22:26:31.437732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.677+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 msgr2=0x7f2c5c039d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.677+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 0x7f2c5c039d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c80153bb0 tx=0x7f2c68006c90 comp rx=0 tx=0).stop 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.677+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 msgr2=0x7f2c801e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.677+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2c7002e130 tx=0x7f2c70004730 comp rx=0 tx=0).stop 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 shutdown_connections 2024-04-05T22:26:35.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2c5c0378a0 0x7f2c5c039d60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c8015aef0 0x7f2c80152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c801541a0 0x7f2c801522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 --2- 172.21.15.73:0/730779082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c8006d710 0x7f2c801e2af0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 >> 172.21.15.73:0/730779082 conn(0x7f2c800ce8e0 msgr2=0x7f2c800ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 shutdown_connections 2024-04-05T22:26:35.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.678+0000 7f2c84e00640 1 -- 172.21.15.73:0/730779082 wait complete. 2024-04-05T22:26:35.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:26:35.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:35.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-05T22:26:35.809+0000 7f7984450040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-05T22:26:35.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.930+0000 7f73bdc00640 1 Processor -- start 2024-04-05T22:26:35.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.930+0000 7f73bdc00640 1 -- start start 2024-04-05T22:26:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73bdc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73b80ce000 0x7f73b815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73bdc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b815aa60 con 0x7f73b806d710 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73bdc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b80b3b60 con 0x7f73b80ce000 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73bdc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80701c0 con 0x7f73b81541a0 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.932+0000 7f73b6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:55370/0 (socket says 172.21.15.73:55370) 2024-04-05T22:26:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 -- 172.21.15.73:0/2296094042 learned_addr learned my addr 172.21.15.73:0/2296094042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b7400640 1 --2- 172.21.15.73:0/2296094042 >> v2:172.21.15.157:3300/0 conn(0x7f73b80ce000 0x7f73b815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 -- 172.21.15.73:0/2296094042 >> v2:172.21.15.157:3300/0 conn(0x7f73b80ce000 msgr2=0x7f73b815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 --2- 172.21.15.73:0/2296094042 >> v2:172.21.15.157:3300/0 conn(0x7f73b80ce000 0x7f73b815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 -- 172.21.15.73:0/2296094042 >> v1:172.21.15.157:6789/0 conn(0x7f73b806d710 legacy=0x7f73b80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.933+0000 7f73b6a00640 1 -- 172.21.15.73:0/2296094042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80b3880 con 0x7f73b81541a0 2024-04-05T22:26:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.934+0000 7f73b6a00640 1 --2- 172.21.15.73:0/2296094042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009e50 tx=0x7f73a802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda375edb69c54a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.935+0000 7f73b5600640 1 -- 172.21.15.73:0/2296094042 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a802cb00 con 0x7f73b81541a0 2024-04-05T22:26:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.935+0000 7f73b5600640 1 -- 172.21.15.73:0/2296094042 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a802cc60 con 0x7f73b81541a0 2024-04-05T22:26:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.935+0000 7f73b5600640 1 -- 172.21.15.73:0/2296094042 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80343f0 con 0x7f73b81541a0 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.935+0000 7f73bdc00640 1 -- 172.21.15.73:0/2296094042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 msgr2=0x7f73b80b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.935+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2296094042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009e50 tx=0x7f73a802bf60 comp rx=0 tx=0).stop 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 -- 172.21.15.73:0/2296094042 shutdown_connections 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2296094042 >> v2:172.21.15.157:3300/0 conn(0x7f73b80ce000 0x7f73b815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2296094042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b81541a0 0x7f73b80b3050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 -- 172.21.15.73:0/2296094042 >> 172.21.15.73:0/2296094042 conn(0x7f73b80ce8e0 msgr2=0x7f73b80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:35.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 -- 172.21.15.73:0/2296094042 shutdown_connections 2024-04-05T22:26:35.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.936+0000 7f73bdc00640 1 -- 172.21.15.73:0/2296094042 wait complete. 2024-04-05T22:26:35.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.937+0000 7f73bdc00640 1 Processor -- start 2024-04-05T22:26:35.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 -- start start 2024-04-05T22:26:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b806d710 0x7f73b8150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 0x7f73b80a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80aa430 con 0x7f73b81541a0 2024-04-05T22:26:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80a9390 con 0x7f73b806d710 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.938+0000 7f73bdc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80a9670 con 0x7f73b80ce000 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59624/0 (socket says 172.21.15.73:59624) 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 -- 172.21.15.73:0/2375694377 learned_addr learned my addr 172.21.15.73:0/2375694377 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b6000640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 0x7f73b80a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b6a00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b806d710 0x7f73b8150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 -- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 msgr2=0x7f73b80a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 0x7f73b80a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 -- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b806d710 msgr2=0x7f73b8150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b806d710 0x7f73b8150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 -- 172.21.15.73:0/2375694377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8009b30 con 0x7f73b81541a0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b6000640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 0x7f73b80a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.939+0000 7f73b7400640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d800 tx=0x7f73b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.940+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000b6c0 con 0x7f73b81541a0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.940+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000b840 con 0x7f73b81541a0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.940+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000bbf0 con 0x7f73b81541a0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.940+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b80a9950 con 0x7f73b81541a0 2024-04-05T22:26:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.940+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b80a9f10 con 0x7f73b81541a0 2024-04-05T22:26:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.941+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 44919+0+0 (secure 0 0 0) 0x7f73b000bd50 con 0x7f73b81541a0 2024-04-05T22:26:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.941+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b8066af0 con 0x7f73b81541a0 2024-04-05T22:26:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.942+0000 7f73aea00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 0x7f7398039df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.942+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f73b004b3a0 con 0x7f73b81541a0 2024-04-05T22:26:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.942+0000 7f73b6a00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 0x7f7398039df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.943+0000 7f73b6a00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 0x7f7398039df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73a8035000 tx=0x7f73a8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:35.946+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f73b0014430 con 0x7f73b81541a0 2024-04-05T22:26:36.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.161+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73b8068160 con 0x7f73b81541a0 2024-04-05T22:26:36.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.161+0000 7f73aea00640 1 -- 172.21.15.73:0/2375694377 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f73b001d3c0 con 0x7f73b81541a0 2024-04-05T22:26:36.162 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:36.162 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:36.162 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 msgr2=0x7f7398039df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 0x7f7398039df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73a8035000 tx=0x7f73a8030000 comp rx=0 tx=0).stop 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 msgr2=0x7f73b81e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d800 tx=0x7f73b000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 shutdown_connections 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7398037930 0x7f7398039df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73b81541a0 0x7f73b81e2370 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73b80ce000 0x7f73b80a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 --2- 172.21.15.73:0/2375694377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b806d710 0x7f73b8150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.164+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 >> 172.21.15.73:0/2375694377 conn(0x7f73b80ce8e0 msgr2=0x7f73b80b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:36.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.165+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 shutdown_connections 2024-04-05T22:26:36.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.165+0000 7f73bdc00640 1 -- 172.21.15.73:0/2375694377 wait complete. 2024-04-05T22:26:36.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:36.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.381+0000 7ff386200640 1 Processor -- start 2024-04-05T22:26:36.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.381+0000 7ff386200640 1 -- start start 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff386200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff38006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff386200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38006f5b0 con 0x7ff38006cb80 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff386200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff380070410 con 0x7ff3801587d0 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff386200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38006d500 con 0x7ff38015ef60 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff37ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff38006cb80 0x7ff38015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38426/0 (socket says 172.21.15.73:38426) 2024-04-05T22:26:36.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff37ea00640 1 -- 172.21.15.73:0/1561036112 learned_addr learned my addr 172.21.15.73:0/1561036112 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:36.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.383+0000 7ff37fe00640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff38006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749197001 0 0) 0x7ff38006f5b0 con 0x7ff38006cb80 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff364003520 con 0x7ff38006cb80 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888674065 0 0) 0x7ff364003520 con 0x7ff38006cb80 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 msgr2=0x7ff38006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff38006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 >> v1:172.21.15.157:6789/0 conn(0x7ff3801587d0 legacy=0x7ff3800cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800ccef0 con 0x7ff38006cb80 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff36c001670 con 0x7ff38006cb80 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 >> v1:172.21.15.73:6789/0 conn(0x7ff38006cb80 legacy=0x7ff38015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff3640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff364003f20 0x7ff3640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364004d00 0x7ff364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38006d500 con 0x7ff364003f20 2024-04-05T22:26:36.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36400ba60 con 0x7ff364004d00 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36400bc20 con 0x7ff38015ef60 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.384+0000 7ff37ea00640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff364003f20 0x7ff3640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37f400640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff3640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37ea00640 1 -- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 msgr2=0x7ff3640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37ea00640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff3640039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37ea00640 1 -- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364004d00 msgr2=0x7ff364005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37ea00640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364004d00 0x7ff364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37ea00640 1 -- 172.21.15.73:0/1561036112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff364003520 con 0x7ff364003f20 2024-04-05T22:26:36.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.385+0000 7ff37f400640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38015ef60 0x7ff3640039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:36.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.386+0000 7ff37ea00640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff364003f20 0x7ff3640047c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff3640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3655f4f3b1600b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.386+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c005ad0 con 0x7ff364003f20 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.386+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c005c30 con 0x7ff364003f20 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.386+0000 7ff37e000640 1 -- 172.21.15.73:0/1561036112 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c02cae0 con 0x7ff364003f20 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 -- 172.21.15.73:0/1561036112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff364003f20 msgr2=0x7ff3640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 --2- 172.21.15.73:0/1561036112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff364003f20 0x7ff3640047c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff3640057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff37fe00640 1 -- 172.21.15.73:0/1561036112 reap_dead start 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 -- 172.21.15.73:0/1561036112 shutdown_connections 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 -- 172.21.15.73:0/1561036112 >> 172.21.15.73:0/1561036112 conn(0x7ff3800ce8e0 msgr2=0x7ff3800cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 -- 172.21.15.73:0/1561036112 shutdown_connections 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.387+0000 7ff386200640 1 -- 172.21.15.73:0/1561036112 wait complete. 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.389+0000 7ff386200640 1 Processor -- start 2024-04-05T22:26:36.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.389+0000 7ff386200640 1 -- start start 2024-04-05T22:26:36.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.389+0000 7ff386200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff364003f20 0x7ff380140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.389+0000 7ff386200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff386200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff380143a10 0x7ff380143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff386200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3800ce570 con 0x7ff3800cdbd0 2024-04-05T22:26:36.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff386200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff380144780 con 0x7ff380143a10 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff386200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3801418d0 con 0x7ff364003f20 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59640/0 (socket says 172.21.15.73:59640) 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 -- 172.21.15.73:0/164507396 learned_addr learned my addr 172.21.15.73:0/164507396 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 -- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff364003f20 msgr2=0x7ff380140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff364003f20 0x7ff380140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37fe00640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff380143a10 0x7ff380143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 -- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff380143a10 msgr2=0x7ff380143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff380143a10 0x7ff380143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 -- 172.21.15.73:0/164507396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380141bb0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.390+0000 7ff37ea00640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff38015c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.391+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c039070 con 0x7ff3800cdbd0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.391+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c02ccb0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.391+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff380141e90 con 0x7ff3800cdbd0 2024-04-05T22:26:36.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.391+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c03c040 con 0x7ff3800cdbd0 2024-04-05T22:26:36.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.391+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3801443e0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.392+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 44919+0+0 (secure 0 0 0) 0x7ff36c032070 con 0x7ff3800cdbd0 2024-04-05T22:26:36.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.392+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff380066af0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.392+0000 7ff377400640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 0x7ff360039df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.392+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7ff36c06bff0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.393+0000 7ff37f400640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 0x7ff360039df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.394+0000 7ff37f400640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 0x7ff360039df0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff370001670 tx=0x7ff370009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.397+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff36c007ce0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.632+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff380068160 con 0x7ff3800cdbd0 2024-04-05T22:26:36.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.633+0000 7ff377400640 1 -- 172.21.15.73:0/164507396 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4739 (secure 0 0 0) 0x7ff36c0403b0 con 0x7ff3800cdbd0 2024-04-05T22:26:36.633 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:36.633 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:10.536474+0000","modified":"2024-04-05T22:26:13.555487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/911489923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":911489923},{"type":"v1","addr":"172.21.15.157:6835","nonce":911489923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-05T22:26:36.634 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.635+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 msgr2=0x7ff360039df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.635+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 0x7ff360039df0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff370001670 tx=0x7ff370009040 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.635+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 msgr2=0x7ff3800ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.635+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff38015c270 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 shutdown_connections 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff360037930 0x7ff360039df0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff380143a10 0x7ff380143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3800cdbd0 0x7ff3800ce030 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 --2- 172.21.15.73:0/164507396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff364003f20 0x7ff380140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 >> 172.21.15.73:0/164507396 conn(0x7ff3800ce8e0 msgr2=0x7ff380159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 shutdown_connections 2024-04-05T22:26:36.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.636+0000 7ff386200640 1 -- 172.21.15.73:0/164507396 wait complete. 2024-04-05T22:26:36.654 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:36.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.852+0000 7fd380000640 1 Processor -- start 2024-04-05T22:26:36.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.852+0000 7fd380000640 1 -- start start 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd380000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 0x7fd3780b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd380000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37815aa60 con 0x7fd37806d710 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd380000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780b3b60 con 0x7fd3780ce000 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd380000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780701c0 con 0x7fd3781541a0 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd37cc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd37806d710 0x7fd3780cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38428/0 (socket says 172.21.15.73:38428) 2024-04-05T22:26:36.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd37cc00640 1 -- 172.21.15.73:0/2081178529 learned_addr learned my addr 172.21.15.73:0/2081178529 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:36.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd37d600640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 0x7fd3780b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203061196 0 0) 0x7fd37815aa60 con 0x7fd37806d710 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.854+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003520 con 0x7fd37806d710 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817094446 0 0) 0x7fd35c003520 con 0x7fd37806d710 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 msgr2=0x7fd3780b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 0x7fd3780b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 >> v1:172.21.15.157:6789/0 conn(0x7fd3780ce000 legacy=0x7fd37815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780b3880 con 0x7fd37806d710 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fd3640019f0 con 0x7fd37806d710 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 >> v1:172.21.15.73:6789/0 conn(0x7fd37806d710 legacy=0x7fd3780cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 0x7fd35c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780701c0 con 0x7fd35c0039a0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00ba30 con 0x7fd35c0047c0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00de00 con 0x7fd3781541a0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd37d600640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd37d600640 1 -- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 msgr2=0x7fd35c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd37d600640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3781541a0 0x7fd35c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd37d600640 1 -- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35c0047c0 msgr2=0x7fd35c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.855+0000 7fd37d600640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.856+0000 7fd37d600640 1 -- 172.21.15.73:0/2081178529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c003520 con 0x7fd35c0039a0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.856+0000 7fd37d600640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd368009a90 tx=0x7fd35c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb1f89cfdaae587 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36802c6b0 con 0x7fd35c0039a0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36802c810 con 0x7fd35c0039a0 2024-04-05T22:26:36.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd377e00640 1 -- 172.21.15.73:0/2081178529 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36802cba0 con 0x7fd35c0039a0 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 -- 172.21.15.73:0/2081178529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35c0039a0 msgr2=0x7fd35c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 --2- 172.21.15.73:0/2081178529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd368009a90 tx=0x7fd35c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd37e000640 1 -- 172.21.15.73:0/2081178529 reap_dead start 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 -- 172.21.15.73:0/2081178529 shutdown_connections 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 -- 172.21.15.73:0/2081178529 >> 172.21.15.73:0/2081178529 conn(0x7fd3780ce8e0 msgr2=0x7fd3780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:36.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 -- 172.21.15.73:0/2081178529 shutdown_connections 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.857+0000 7fd380000640 1 -- 172.21.15.73:0/2081178529 wait complete. 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.859+0000 7fd380000640 1 Processor -- start 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.859+0000 7fd380000640 1 -- start start 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35c0039a0 0x7fd378141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd37813e7c0 0x7fd378150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780abf60 con 0x7fd3780ab5a0 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd378151250 con 0x7fd37813e7c0 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd380000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781514f0 con 0x7fd35c0039a0 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd37e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd37e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59660/0 (socket says 172.21.15.73:59660) 2024-04-05T22:26:36.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.860+0000 7fd37e000640 1 -- 172.21.15.73:0/1419901270 learned_addr learned my addr 172.21.15.73:0/1419901270 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37cc00640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd37813e7c0 0x7fd378150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37d600640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35c0039a0 0x7fd378141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 -- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35c0039a0 msgr2=0x7fd378141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35c0039a0 0x7fd378141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 -- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd37813e7c0 msgr2=0x7fd378150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd37813e7c0 0x7fd378150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 -- 172.21.15.73:0/1419901270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3781517d0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd37e000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd378070c30 tx=0x7fd37000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700074e0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd370007640 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378151ab0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370004260 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.861+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378152070 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.863+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378066af0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.863+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fd3700057c0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.864+0000 7fd375600640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 0x7fd35806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.864+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fd3700887f0 con 0x7fd3780ab5a0 2024-04-05T22:26:36.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.865+0000 7fd37d600640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 0x7fd35806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:36.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.866+0000 7fd37d600640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 0x7fd35806d210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd368036000 tx=0x7fd368030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:36.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:36.868+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd370053690 con 0x7fd3780ab5a0 2024-04-05T22:26:37.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.059+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd378068160 con 0x7fd3780ab5a0 2024-04-05T22:26:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.061+0000 7fd375600640 1 -- 172.21.15.73:0/1419901270 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fd370056cb0 con 0x7fd3780ab5a0 2024-04-05T22:26:37.062 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:37.062 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":75,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:31.437790+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:08.439249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:09.543608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/4239333780":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3883631467":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1002310496":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/206467021":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/307490620":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3668905466":"2024-04-06T22:26:31.437732+0000","172.21.15.73:6833/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/2340596025":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/265479964":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1452282649":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1767413521":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/745661956":"2024-04-06T22:26:31.437732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.063+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 msgr2=0x7fd35806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.063+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 0x7fd35806d210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd368036000 tx=0x7fd368030000 comp rx=0 tx=0).stop 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 msgr2=0x7fd3780aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd378070c30 tx=0x7fd37000c820 comp rx=0 tx=0).stop 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 shutdown_connections 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd35806ad50 0x7fd35806d210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3780ab5a0 0x7fd3780aba20 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd37813e7c0 0x7fd378150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 --2- 172.21.15.73:0/1419901270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35c0039a0 0x7fd378141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 >> 172.21.15.73:0/1419901270 conn(0x7fd3780ce8e0 msgr2=0x7fd3780cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:37.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 shutdown_connections 2024-04-05T22:26:37.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.064+0000 7fd380000640 1 -- 172.21.15.73:0/1419901270 wait complete. 2024-04-05T22:26:37.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:26:37.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.287+0000 7f62d2e00640 1 Processor -- start 2024-04-05T22:26:37.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.287+0000 7f62d2e00640 1 -- start start 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.288+0000 7f62d2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62cc06d060 0x7f62cc0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62cc15aaa0 0x7f62cc156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc0b3ea0 con 0x7f62cc06d060 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc15e2c0 con 0x7f62cc15aaa0 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15d500 con 0x7f62cc06d440 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62cc06d060 0x7f62cc0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62cc06d060 0x7f62cc0b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59664/0 (socket says 172.21.15.73:59664) 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 -- 172.21.15.73:0/2622797479 learned_addr learned my addr 172.21.15.73:0/2622797479 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1000640 1 --2- 172.21.15.73:0/2622797479 >> v2:172.21.15.157:3300/0 conn(0x7f62cc15aaa0 0x7f62cc156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d2400640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 -- 172.21.15.73:0/2622797479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 msgr2=0x7f62cc157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 -- 172.21.15.73:0/2622797479 >> v2:172.21.15.157:3300/0 conn(0x7f62cc15aaa0 msgr2=0x7f62cc156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 --2- 172.21.15.73:0/2622797479 >> v2:172.21.15.157:3300/0 conn(0x7f62cc15aaa0 0x7f62cc156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.289+0000 7f62d1a00640 1 -- 172.21.15.73:0/2622797479 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc15d7e0 con 0x7f62cc06d060 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.290+0000 7f62d1a00640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc06d060 0x7f62cc0b3530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f62cc071590 tx=0x7f62c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a13b4637f1f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.291+0000 7f62cbe00640 1 -- 172.21.15.73:0/2622797479 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0031870 con 0x7f62cc06d060 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.291+0000 7f62cbe00640 1 -- 172.21.15.73:0/2622797479 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c00319d0 con 0x7f62cc06d060 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.291+0000 7f62cbe00640 1 -- 172.21.15.73:0/2622797479 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0031d60 con 0x7f62cc06d060 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 -- 172.21.15.73:0/2622797479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc06d060 msgr2=0x7f62cc0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc06d060 0x7f62cc0b3530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f62cc071590 tx=0x7f62c002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:37.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 -- 172.21.15.73:0/2622797479 shutdown_connections 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 --2- 172.21.15.73:0/2622797479 >> v2:172.21.15.157:3300/0 conn(0x7f62cc15aaa0 0x7f62cc156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 --2- 172.21.15.73:0/2622797479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc06d060 0x7f62cc0b3530 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 -- 172.21.15.73:0/2622797479 >> 172.21.15.73:0/2622797479 conn(0x7f62cc0b0830 msgr2=0x7f62cc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 -- 172.21.15.73:0/2622797479 shutdown_connections 2024-04-05T22:26:37.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.292+0000 7f62d2e00640 1 -- 172.21.15.73:0/2622797479 wait complete. 2024-04-05T22:26:37.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.294+0000 7f62d2e00640 1 Processor -- start 2024-04-05T22:26:37.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.294+0000 7f62d2e00640 1 -- start start 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc06d060 0x7f62cc144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc144e40 con 0x7f62cc15aaa0 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc146380 con 0x7f62cc06d060 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15c450 con 0x7f62cc06d440 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.295+0000 7f62d2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59678/0 (socket says 172.21.15.73:59678) 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 -- 172.21.15.73:0/1799668745 learned_addr learned my addr 172.21.15.73:0/1799668745 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d1a00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc06d060 0x7f62cc144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d1000640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 -- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 msgr2=0x7f62cc15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc15bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 -- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc06d060 msgr2=0x7f62cc144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc06d060 0x7f62cc144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 -- 172.21.15.73:0/1799668745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c00094d0 con 0x7f62cc15aaa0 2024-04-05T22:26:37.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62d2400640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cd60 tx=0x7f62c400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.296+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003b90 con 0x7f62cc15aaa0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.297+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4003d10 con 0x7f62cc15aaa0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.297+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004260 con 0x7f62cc15aaa0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.297+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc145b20 con 0x7f62cc15aaa0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.297+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc15c1b0 con 0x7f62cc15aaa0 2024-04-05T22:26:37.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.298+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc155600 con 0x7f62cc15aaa0 2024-04-05T22:26:37.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.299+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f62c40043c0 con 0x7f62cc15aaa0 2024-04-05T22:26:37.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.300+0000 7f62c9600640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 0x7f62ac06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.300+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f62c4012070 con 0x7f62cc15aaa0 2024-04-05T22:26:37.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.300+0000 7f62d1a00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 0x7f62ac06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.301+0000 7f62d1a00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 0x7f62ac06d1e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f62c0036000 tx=0x7f62c003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.304+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f62c4054f50 con 0x7f62cc15aaa0 2024-04-05T22:26:37.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.536+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f62cc04e160 con 0x7f62cc15aaa0 2024-04-05T22:26:37.538 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:26:37.538+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:26:37.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.554+0000 7f62c9600640 1 -- 172.21.15.73:0/1799668745 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f62c4003e80 con 0x7f62cc15aaa0 2024-04-05T22:26:37.555 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:26:37.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 msgr2=0x7f62ac06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 0x7f62ac06d1e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f62c0036000 tx=0x7f62c003c040 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 msgr2=0x7f62cc152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cd60 tx=0x7f62c400a660 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 shutdown_connections 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62ac06ad20 0x7f62ac06d1e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62cc15aaa0 0x7f62cc152540 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc06d440 0x7f62cc15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 --2- 172.21.15.73:0/1799668745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc06d060 0x7f62cc144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 >> 172.21.15.73:0/1799668745 conn(0x7f62cc0b0830 msgr2=0x7f62cc0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 shutdown_connections 2024-04-05T22:26:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.557+0000 7f62d2e00640 1 -- 172.21.15.73:0/1799668745 wait complete. 2024-04-05T22:26:37.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:26:37.589 INFO:tasks.ceph.mds.d.smithi157.stderr: -7675> 2024-04-05T22:26:15.027+0000 7f041f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:26:37.678 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:26:37.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.786+0000 7f6eb4800640 1 Processor -- start 2024-04-05T22:26:37.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.786+0000 7f6eb4800640 1 -- start start 2024-04-05T22:26:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb4800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6eac06c630 0x7f6eac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb4800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0b1b30 con 0x7f6eac0cbe00 2024-04-05T22:26:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb4800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0b1e10 con 0x7f6eac0b32b0 2024-04-05T22:26:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb4800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eac0ce7b0 con 0x7f6eac06c630 2024-04-05T22:26:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb1400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6eac0cbe00 0x7f6eac0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38430/0 (socket says 172.21.15.73:38430) 2024-04-05T22:26:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb1400640 1 -- 172.21.15.73:0/1156805819 learned_addr learned my addr 172.21.15.73:0/1156805819 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.788+0000 7f6eb1e00640 1 --2- 172.21.15.73:0/1156805819 >> v2:172.21.15.73:3300/0 conn(0x7f6eac06c630 0x7f6eac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.789+0000 7f6eb1e00640 1 -- 172.21.15.73:0/1156805819 >> v1:172.21.15.157:6789/0 conn(0x7f6eac0b32b0 legacy=0x7f6eac1647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.789+0000 7f6eb1e00640 1 -- 172.21.15.73:0/1156805819 >> v1:172.21.15.73:6789/0 conn(0x7f6eac0cbe00 legacy=0x7f6eac0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.789+0000 7f6eb1e00640 1 -- 172.21.15.73:0/1156805819 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac06cf50 con 0x7f6eac06c630 2024-04-05T22:26:37.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.790+0000 7f6eb1e00640 1 --2- 172.21.15.73:0/1156805819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac06c630 0x7f6eac06ca10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00097a0 tx=0x7f6ea002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cf1a33b2479f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.790+0000 7f6eb0a00640 1 -- 172.21.15.73:0/1156805819 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0031830 con 0x7f6eac06c630 2024-04-05T22:26:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.790+0000 7f6eb0a00640 1 -- 172.21.15.73:0/1156805819 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea0031990 con 0x7f6eac06c630 2024-04-05T22:26:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.790+0000 7f6eb0a00640 1 -- 172.21.15.73:0/1156805819 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0031d20 con 0x7f6eac06c630 2024-04-05T22:26:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 -- 172.21.15.73:0/1156805819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac06c630 msgr2=0x7f6eac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1156805819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac06c630 0x7f6eac06ca10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00097a0 tx=0x7f6ea002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 -- 172.21.15.73:0/1156805819 shutdown_connections 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1156805819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac06c630 0x7f6eac06ca10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 -- 172.21.15.73:0/1156805819 >> 172.21.15.73:0/1156805819 conn(0x7f6eac0b0830 msgr2=0x7f6eac0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 -- 172.21.15.73:0/1156805819 shutdown_connections 2024-04-05T22:26:37.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.791+0000 7f6eb4800640 1 -- 172.21.15.73:0/1156805819 wait complete. 2024-04-05T22:26:37.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.793+0000 7f6eb4800640 1 Processor -- start 2024-04-05T22:26:37.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.793+0000 7f6eb4800640 1 -- start start 2024-04-05T22:26:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6eac0b32b0 0x7f6eac142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac13f2e0 0x7f6eac1e4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1416b0 con 0x7f6eac0cbe00 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0c3cd0 con 0x7f6eac13f2e0 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac140e10 con 0x7f6eac0b32b0 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59696/0 (socket says 172.21.15.73:59696) 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.794+0000 7f6eb1400640 1 -- 172.21.15.73:0/1679625121 learned_addr learned my addr 172.21.15.73:0/1679625121 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1e00640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6eac0b32b0 0x7f6eac142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 -- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6eac0b32b0 msgr2=0x7f6eac142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6eac0b32b0 0x7f6eac142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 -- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac13f2e0 msgr2=0x7f6eac1e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac13f2e0 0x7f6eac1e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 -- 172.21.15.73:0/1679625121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0009400 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb1400640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c4a0 tx=0x7f6e9c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005ce0 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c005e40 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007270 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.795+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac1410f0 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.796+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac140b30 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.797+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac068160 con 0x7f6eac0cbe00 2024-04-05T22:26:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.797+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f6e9c007490 con 0x7f6eac0cbe00 2024-04-05T22:26:37.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.798+0000 7f6ea6000640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 0x7f6e9006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:37.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.798+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f6e9c088bb0 con 0x7f6eac0cbe00 2024-04-05T22:26:37.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.799+0000 7f6eb1e00640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 0x7f6e9006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:37.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.800+0000 7f6eb1e00640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 0x7f6e9006d1e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0036000 tx=0x7f6ea0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:37.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:37.803+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6e9c053a90 con 0x7f6eac0cbe00 2024-04-05T22:26:38.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.037+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6eac0c3e10 con 0x7f6eac0cbe00 2024-04-05T22:26:38.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.558+0000 7f6ea6000640 1 -- 172.21.15.73:0/1679625121 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6e9c0170a0 con 0x7f6eac0cbe00 2024-04-05T22:26:38.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 msgr2=0x7f6e9006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 0x7f6e9006d1e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0036000 tx=0x7f6ea0008030 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 msgr2=0x7f6eac1405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c4a0 tx=0x7f6e9c00c960 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 shutdown_connections 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6e9006ad20 0x7f6e9006d1e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac13f2e0 0x7f6eac1e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eac0cbe00 0x7f6eac1405f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 --2- 172.21.15.73:0/1679625121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6eac0b32b0 0x7f6eac142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 >> 172.21.15.73:0/1679625121 conn(0x7f6eac0b0830 msgr2=0x7f6eac0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 shutdown_connections 2024-04-05T22:26:38.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.562+0000 7f6eb4800640 1 -- 172.21.15.73:0/1679625121 wait complete. 2024-04-05T22:26:38.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:26:38.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.795+0000 7f3f6e600640 1 Processor -- start 2024-04-05T22:26:38.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.795+0000 7f3f6e600640 1 -- start start 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f6e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f6815ae50 0x7f3f680b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f6e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f680b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f6e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6815b230 con 0x7f3f680cce90 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f6e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f680cd320 con 0x7f3f6815ae50 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f6e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680ce2f0 con 0x7f3f680cf3e0 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f6815ae50 0x7f3f680b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f6815ae50 0x7f3f680b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59698/0 (socket says 172.21.15.73:59698) 2024-04-05T22:26:38.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 -- 172.21.15.73:0/4186821384 learned_addr learned my addr 172.21.15.73:0/4186821384 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:38.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 -- 172.21.15.73:0/4186821384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 msgr2=0x7f3f680b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:38.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 --2- 172.21.15.73:0/4186821384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f680b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 -- 172.21.15.73:0/4186821384 >> v1:172.21.15.73:6789/0 conn(0x7f3f680cce90 legacy=0x7f3f680cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f67400640 1 -- 172.21.15.73:0/4186821384 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6815f4a0 con 0x7f3f6815ae50 2024-04-05T22:26:38.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.797+0000 7f3f66000640 1 -- 172.21.15.73:0/4186821384 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703247724 0 0) 0x7f3f6815b230 con 0x7f3f680cce90 2024-04-05T22:26:38.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.799+0000 7f3f67400640 1 --2- 172.21.15.73:0/4186821384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f6815ae50 0x7f3f680b3aa0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3f680ce430 tx=0x7f3f5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6966ac46026292 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:38.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.799+0000 7f3f66000640 1 -- 172.21.15.73:0/4186821384 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58031640 con 0x7f3f6815ae50 2024-04-05T22:26:38.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.799+0000 7f3f66000640 1 -- 172.21.15.73:0/4186821384 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580317a0 con 0x7f3f6815ae50 2024-04-05T22:26:38.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.799+0000 7f3f66000640 1 -- 172.21.15.73:0/4186821384 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58031b30 con 0x7f3f6815ae50 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 -- 172.21.15.73:0/4186821384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f6815ae50 msgr2=0x7f3f680b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 --2- 172.21.15.73:0/4186821384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f6815ae50 0x7f3f680b3aa0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3f680ce430 tx=0x7f3f5802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 -- 172.21.15.73:0/4186821384 shutdown_connections 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 --2- 172.21.15.73:0/4186821384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f680b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 --2- 172.21.15.73:0/4186821384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f6815ae50 0x7f3f680b3aa0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 -- 172.21.15.73:0/4186821384 >> 172.21.15.73:0/4186821384 conn(0x7f3f6806a0d0 msgr2=0x7f3f6806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 -- 172.21.15.73:0/4186821384 shutdown_connections 2024-04-05T22:26:38.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.800+0000 7f3f6e600640 1 -- 172.21.15.73:0/4186821384 wait complete. 2024-04-05T22:26:38.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.801+0000 7f3f6e600640 1 Processor -- start 2024-04-05T22:26:38.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.802+0000 7f3f6e600640 1 -- start start 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.802+0000 7f3f6e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f6e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f6815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f6e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6815ae50 0x7f3f681583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f6e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c7fe0 con 0x7f3f680cce90 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f6e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c5330 con 0x7f3f6815ae50 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f6e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c47b0 con 0x7f3f680cf3e0 2024-04-05T22:26:38.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59706/0 (socket says 172.21.15.73:59706) 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 -- 172.21.15.73:0/1065785621 learned_addr learned my addr 172.21.15.73:0/1065785621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 -- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 msgr2=0x7f3f6815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f6815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 -- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6815ae50 msgr2=0x7f3f681583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6815ae50 0x7f3f681583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:38.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.803+0000 7f3f67400640 1 -- 172.21.15.73:0/1065785621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580094d0 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f67400640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3f580090b0 tx=0x7f3f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004230 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f58004390 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5803aa90 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f680c4a90 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.804+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680c5050 con 0x7f3f680cce90 2024-04-05T22:26:38.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.805+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f680c8150 con 0x7f3f680cce90 2024-04-05T22:26:38.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.806+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f3f58042070 con 0x7f3f680cce90 2024-04-05T22:26:38.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.807+0000 7f3f5f400640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 0x7f3f4406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:38.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.807+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f3f580ab660 con 0x7f3f680cce90 2024-04-05T22:26:38.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.808+0000 7f3f66a00640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 0x7f3f4406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:38.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.809+0000 7f3f66a00640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 0x7f3f4406d1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f6806c820 tx=0x7f3f50007910 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:38.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:38.811+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3f580762f0 con 0x7f3f680cce90 2024-04-05T22:26:38.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:26:38.919 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:26:39.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.008+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f68068160 con 0x7f3f680cce90 2024-04-05T22:26:39.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.804+0000 7f3f5f400640 1 -- 172.21.15.73:0/1065785621 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3f58031e60 con 0x7f3f680cce90 2024-04-05T22:26:39.806 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.810+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 msgr2=0x7f3f4406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.810+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 0x7f3f4406d1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f6806c820 tx=0x7f3f50007910 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.810+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 msgr2=0x7f3f680c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.810+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3f580090b0 tx=0x7f3f580046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 shutdown_connections 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f4406ad20 0x7f3f4406d1e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6815ae50 0x7f3f681583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f680cf3e0 0x7f3f6815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 --2- 172.21.15.73:0/1065785621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f680cce90 0x7f3f680c61c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 >> 172.21.15.73:0/1065785621 conn(0x7f3f6806a0d0 msgr2=0x7f3f680ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 shutdown_connections 2024-04-05T22:26:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:39.811+0000 7f3f6e600640 1 -- 172.21.15.73:0/1065785621 wait complete. 2024-04-05T22:26:39.836 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:26:40.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.024+0000 7f3084000640 1 Processor -- start 2024-04-05T22:26:40.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.025+0000 7f3084000640 1 -- start start 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3084000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f307c0b3670 0x7f307c0d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3084000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 0x7f307c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3084000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0cffd0 con 0x7f307c0b3ca0 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3084000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0d1ed0 con 0x7f307c0b3670 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3084000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0cd9a0 con 0x7f307c0cf640 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3081600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f307c0b3670 0x7f307c0d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3081600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f307c0b3670 0x7f307c0d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59708/0 (socket says 172.21.15.73:59708) 2024-04-05T22:26:40.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.026+0000 7f3081600640 1 -- 172.21.15.73:0/2551188580 learned_addr learned my addr 172.21.15.73:0/2551188580 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3081600640 1 -- 172.21.15.73:0/2551188580 >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 msgr2=0x7f307c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3082000640 1 --2- 172.21.15.73:0/2551188580 >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 0x7f307c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3081600640 1 --2- 172.21.15.73:0/2551188580 >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 0x7f307c0cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3081600640 1 -- 172.21.15.73:0/2551188580 >> v1:172.21.15.157:6789/0 conn(0x7f307c0b3ca0 legacy=0x7f307c0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3081600640 1 -- 172.21.15.73:0/2551188580 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0b3a50 con 0x7f307c0b3670 2024-04-05T22:26:40.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.027+0000 7f3082000640 1 --2- 172.21.15.73:0/2551188580 >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 0x7f307c0cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.028+0000 7f3081600640 1 --2- 172.21.15.73:0/2551188580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0d3800 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3070009870 tx=0x7f307002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934436f079ead2f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.028+0000 7f307be00640 1 -- 172.21.15.73:0/2551188580 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031870 con 0x7f307c0b3670 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.028+0000 7f307be00640 1 -- 172.21.15.73:0/2551188580 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700319d0 con 0x7f307c0b3670 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.028+0000 7f307be00640 1 -- 172.21.15.73:0/2551188580 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031d60 con 0x7f307c0b3670 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 -- 172.21.15.73:0/2551188580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 msgr2=0x7f307c0d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:40.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 --2- 172.21.15.73:0/2551188580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0d3800 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3070009870 tx=0x7f307002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 -- 172.21.15.73:0/2551188580 shutdown_connections 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 --2- 172.21.15.73:0/2551188580 >> v2:172.21.15.157:3300/0 conn(0x7f307c0cf640 0x7f307c0cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 --2- 172.21.15.73:0/2551188580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0d3800 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 -- 172.21.15.73:0/2551188580 >> 172.21.15.73:0/2551188580 conn(0x7f307c0d2ce0 msgr2=0x7f307c0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 -- 172.21.15.73:0/2551188580 shutdown_connections 2024-04-05T22:26:40.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.029+0000 7f3084000640 1 -- 172.21.15.73:0/2551188580 wait complete. 2024-04-05T22:26:40.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.031+0000 7f3084000640 1 Processor -- start 2024-04-05T22:26:40.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.031+0000 7f3084000640 1 -- start start 2024-04-05T22:26:40.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 0x7f307c0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307c0cf640 0x7f307c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0a05b0 con 0x7f307c0b3670 2024-04-05T22:26:40.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0b08a0 con 0x7f307c0b3ca0 2024-04-05T22:26:40.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3084000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c156980 con 0x7f307c0cf640 2024-04-05T22:26:40.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3081600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3081600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59722/0 (socket says 172.21.15.73:59722) 2024-04-05T22:26:40.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.032+0000 7f3081600640 1 -- 172.21.15.73:0/1512630229 learned_addr learned my addr 172.21.15.73:0/1512630229 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:40.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3080c00640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 0x7f307c0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3082000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307c0cf640 0x7f307c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 -- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307c0cf640 msgr2=0x7f307c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:40.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307c0cf640 0x7f307c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 -- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 msgr2=0x7f307c0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 0x7f307c0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 -- 172.21.15.73:0/1512630229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700094d0 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3080c00640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 0x7f307c0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3081600640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3070000c00 tx=0x7f30700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031820 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700443f0 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700447a0 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.033+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0b0ad0 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.034+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c1f32f0 con 0x7f307c0b3670 2024-04-05T22:26:40.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.035+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c068160 con 0x7f307c0b3670 2024-04-05T22:26:40.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.036+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f307004d450 con 0x7f307c0b3670 2024-04-05T22:26:40.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.037+0000 7f3079600640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 0x7f306006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:40.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.038+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5916+0+0 (secure 0 0 0) 0x7f307002d040 con 0x7f307c0b3670 2024-04-05T22:26:40.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.038+0000 7f3080c00640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 0x7f306006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:40.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.039+0000 7f3080c00640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 0x7f306006d1e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f307c0b8a80 tx=0x7f306c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:40.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.042+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3070076bb0 con 0x7f307c0b3670 2024-04-05T22:26:40.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.242+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f307c0cffd0 con 0x7f307c0b3670 2024-04-05T22:26:40.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.811+0000 7f3079600640 1 -- 172.21.15.73:0/1512630229 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f307c0cffd0 con 0x7f307c0b3670 2024-04-05T22:26:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:26:40.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 msgr2=0x7f306006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:40.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 0x7f306006d1e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f307c0b8a80 tx=0x7f306c009040 comp rx=0 tx=0).stop 2024-04-05T22:26:40.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 msgr2=0x7f307c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:40.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3070000c00 tx=0x7f30700046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 shutdown_connections 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f306006ad20 0x7f306006d1e0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f307c0b8a80 tx=0x7f306c009040 comp rx=0 tx=0).stop 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307c0cf640 0x7f307c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307c0b3ca0 0x7f307c0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.820+0000 7f3084000640 1 --2- 172.21.15.73:0/1512630229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307c0b3670 0x7f307c0b2b90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:40.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.821+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 >> 172.21.15.73:0/1512630229 conn(0x7f307c0d2ce0 msgr2=0x7f307c15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:40.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.822+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 shutdown_connections 2024-04-05T22:26:40.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:40.822+0000 7f3084000640 1 -- 172.21.15.73:0/1512630229 wait complete. 2024-04-05T22:26:40.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:26:41.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.039+0000 7fa6de800640 1 Processor -- start 2024-04-05T22:26:41.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.039+0000 7fa6de800640 1 -- start start 2024-04-05T22:26:41.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6d815cdd0 0x7fa6d80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 0x7fa6d815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6d81597e0 0x7fa6d8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80ce760 con 0x7fa6d815cdd0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80cd090 con 0x7fa6d81597e0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.040+0000 7fa6de800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b1bd0 con 0x7fa6d815d970 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6d815cdd0 0x7fa6d80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6d815cdd0 0x7fa6d80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59730/0 (socket says 172.21.15.73:59730) 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 -- 172.21.15.73:0/1279917078 learned_addr learned my addr 172.21.15.73:0/1279917078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6cfe00640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 0x7fa6d815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 -- 172.21.15.73:0/1279917078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 msgr2=0x7fa6d815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 0x7fa6d815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 -- 172.21.15.73:0/1279917078 >> v2:172.21.15.157:3300/0 conn(0x7fa6d81597e0 msgr2=0x7fa6d8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7e00640 1 --2- 172.21.15.73:0/1279917078 >> v2:172.21.15.157:3300/0 conn(0x7fa6d81597e0 0x7fa6d8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 --2- 172.21.15.73:0/1279917078 >> v2:172.21.15.157:3300/0 conn(0x7fa6d81597e0 0x7fa6d8159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6d7400640 1 -- 172.21.15.73:0/1279917078 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d815d6f0 con 0x7fa6d815cdd0 2024-04-05T22:26:41.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.041+0000 7fa6cfe00640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 0x7fa6d815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.042+0000 7fa6d7400640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815cdd0 0x7fa6d80ce220 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80666b0 tx=0x7fa6c802e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cfc8a815bf48d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.043+0000 7fa6d6a00640 1 -- 172.21.15.73:0/1279917078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c802ee80 con 0x7fa6d815cdd0 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.043+0000 7fa6d6a00640 1 -- 172.21.15.73:0/1279917078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8036440 con 0x7fa6d815cdd0 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.043+0000 7fa6d6a00640 1 -- 172.21.15.73:0/1279917078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80367b0 con 0x7fa6d815cdd0 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.043+0000 7fa6de800640 1 -- 172.21.15.73:0/1279917078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815cdd0 msgr2=0x7fa6d80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.043+0000 7fa6de800640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815cdd0 0x7fa6d80ce220 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80666b0 tx=0x7fa6c802e0d0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 -- 172.21.15.73:0/1279917078 shutdown_connections 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 --2- 172.21.15.73:0/1279917078 >> v2:172.21.15.157:3300/0 conn(0x7fa6d81597e0 0x7fa6d8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815d970 0x7fa6d815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 --2- 172.21.15.73:0/1279917078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815cdd0 0x7fa6d80ce220 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 -- 172.21.15.73:0/1279917078 >> 172.21.15.73:0/1279917078 conn(0x7fa6d80b0830 msgr2=0x7fa6d80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 -- 172.21.15.73:0/1279917078 shutdown_connections 2024-04-05T22:26:41.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.044+0000 7fa6de800640 1 -- 172.21.15.73:0/1279917078 wait complete. 2024-04-05T22:26:41.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.045+0000 7fa6de800640 1 Processor -- start 2024-04-05T22:26:41.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.046+0000 7fa6de800640 1 -- start start 2024-04-05T22:26:41.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.046+0000 7fa6de800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 0x7fa6d81528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6de800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815cdd0 0x7fa6d81421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6de800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6de800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8152410 con 0x7fa6d815d970 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6de800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8151800 con 0x7fa6d81597e0 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6de800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8151ae0 con 0x7fa6d815cdd0 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59736/0 (socket says 172.21.15.73:59736) 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 -- 172.21.15.73:0/3803270089 learned_addr learned my addr 172.21.15.73:0/3803270089 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6cfe00640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815cdd0 0x7fa6d81421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7400640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 0x7fa6d81528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 -- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815cdd0 msgr2=0x7fa6d81421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815cdd0 0x7fa6d81421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 -- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 msgr2=0x7fa6d81528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 0x7fa6d81528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7e00640 1 -- 172.21.15.73:0/3803270089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c800b770 con 0x7fa6d815d970 2024-04-05T22:26:41.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.047+0000 7fa6d7400640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 0x7fa6d81528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:41.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6d7e00640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000cb80 tx=0x7fa6d000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000ce70 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d000abd0 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8151dc0 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0011410 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.048+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d8142720 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.049+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d81525c0 con 0x7fa6d815d970 2024-04-05T22:26:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.050+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fa6d0011570 con 0x7fa6d815d970 2024-04-05T22:26:41.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.051+0000 7fa6cf400640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 0x7fa6b406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.051+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fa6d0089270 con 0x7fa6d815d970 2024-04-05T22:26:41.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.052+0000 7fa6d7400640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 0x7fa6b406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.053+0000 7fa6d7400640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 0x7fa6b406d1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8038000 tx=0x7fa6c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.056+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa6d81525c0 con 0x7fa6d815d970 2024-04-05T22:26:41.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.254+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa6d8068160 con 0x7fa6d815d970 2024-04-05T22:26:41.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.260+0000 7fa6cf400640 1 -- 172.21.15.73:0/3803270089 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa6d0054390 con 0x7fa6d815d970 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 msgr2=0x7fa6b406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 0x7fa6b406d1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8038000 tx=0x7fa6c8008040 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 msgr2=0x7fa6d8150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000cb80 tx=0x7fa6d000a4c0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 shutdown_connections 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa6b406ad20 0x7fa6b406d1e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d815d970 0x7fa6d8150f00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d815cdd0 0x7fa6d81421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 --2- 172.21.15.73:0/3803270089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81597e0 0x7fa6d81528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.263+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 >> 172.21.15.73:0/3803270089 conn(0x7fa6d80b0830 msgr2=0x7fa6d80b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.264+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 shutdown_connections 2024-04-05T22:26:41.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.264+0000 7fa6de800640 1 -- 172.21.15.73:0/3803270089 wait complete. 2024-04-05T22:26:41.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-05T22:26:41.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.487+0000 7fb337400640 1 Processor -- start 2024-04-05T22:26:41.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.487+0000 7fb337400640 1 -- start start 2024-04-05T22:26:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb337400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3300cb2f0 0x7fb3300cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb337400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300cbc70 con 0x7fb330160620 2024-04-05T22:26:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb337400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300ad880 con 0x7fb3300b1870 2024-04-05T22:26:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb337400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300ce5d0 con 0x7fb3300cb2f0 2024-04-05T22:26:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb334a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb330160620 0x7fb3300caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38444/0 (socket says 172.21.15.73:38444) 2024-04-05T22:26:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.489+0000 7fb334a00640 1 -- 172.21.15.73:0/999044885 learned_addr learned my addr 172.21.15.73:0/999044885 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693210054 0 0) 0x7fb3300cbc70 con 0x7fb330160620 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb314003520 con 0x7fb330160620 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb335400640 1 --2- 172.21.15.73:0/999044885 >> v2:172.21.15.157:3300/0 conn(0x7fb3300cb2f0 0x7fb3300cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3764470214 0 0) 0x7fb314003520 con 0x7fb330160620 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 >> v2:172.21.15.157:3300/0 conn(0x7fb3300cb2f0 msgr2=0x7fb3300cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 --2- 172.21.15.73:0/999044885 >> v2:172.21.15.157:3300/0 conn(0x7fb3300cb2f0 0x7fb3300cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 >> v1:172.21.15.157:6789/0 conn(0x7fb3300b1870 legacy=0x7fb3300ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300ce010 con 0x7fb330160620 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb3200019f0 con 0x7fb330160620 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 >> v1:172.21.15.73:6789/0 conn(0x7fb330160620 legacy=0x7fb3300caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.490+0000 7fb32f400640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb314003970 0x7fb314003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32f400640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 0x7fb314004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32f400640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb314005180 0x7fb31400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300ce5d0 con 0x7fb3140047c0 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31400c0b0 con 0x7fb314003970 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31400c270 con 0x7fb314005180 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 0x7fb314004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb334a00640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb314003970 0x7fb314003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 -- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb314005180 msgr2=0x7fb31400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb314005180 0x7fb31400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 -- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb314003970 msgr2=0x7fb314003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb314003970 0x7fb314003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb32fe00640 1 -- 172.21.15.73:0/999044885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314003520 con 0x7fb3140047c0 2024-04-05T22:26:41.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.491+0000 7fb335400640 1 -- 172.21.15.73:0/999044885 reap_dead start 2024-04-05T22:26:41.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.492+0000 7fb32fe00640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 0x7fb314004c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb31c003d80 tx=0x7fb3140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b336e82366ac09c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c032600 con 0x7fb3140047c0 2024-04-05T22:26:41.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c02e050 con 0x7fb3140047c0 2024-04-05T22:26:41.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb32f400640 1 -- 172.21.15.73:0/999044885 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0329e0 con 0x7fb3140047c0 2024-04-05T22:26:41.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 -- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 msgr2=0x7fb314004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 0x7fb314004c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb31c003d80 tx=0x7fb3140057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 -- 172.21.15.73:0/999044885 shutdown_connections 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 --2- 172.21.15.73:0/999044885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3140047c0 0x7fb314004c40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 -- 172.21.15.73:0/999044885 >> 172.21.15.73:0/999044885 conn(0x7fb3300ce8e0 msgr2=0x7fb3300cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 -- 172.21.15.73:0/999044885 shutdown_connections 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.493+0000 7fb337400640 1 -- 172.21.15.73:0/999044885 wait complete. 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.495+0000 7fb337400640 1 Processor -- start 2024-04-05T22:26:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.495+0000 7fb337400640 1 -- start start 2024-04-05T22:26:41.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3140047c0 0x7fb33015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 0x7fb3301f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300cd2b0 con 0x7fb3300cc8c0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3301f2a30 con 0x7fb3140047c0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.496+0000 7fb337400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3301f2d10 con 0x7fb33014e620 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59756/0 (socket says 172.21.15.73:59756) 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 -- 172.21.15.73:0/3664711499 learned_addr learned my addr 172.21.15.73:0/3664711499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb32fe00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 0x7fb3301f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb334a00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3140047c0 0x7fb33015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 -- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 msgr2=0x7fb3301f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 0x7fb3301f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 -- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3140047c0 msgr2=0x7fb33015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3140047c0 0x7fb33015c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb335400640 1 -- 172.21.15.73:0/3664711499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3301f2ff0 con 0x7fb3300cc8c0 2024-04-05T22:26:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.497+0000 7fb32fe00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 0x7fb3301f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb335400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb330071ee0 tx=0x7fb32800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328017070 con 0x7fb3300cc8c0 2024-04-05T22:26:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328004020 con 0x7fb3300cc8c0 2024-04-05T22:26:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300c9150 con 0x7fb3300cc8c0 2024-04-05T22:26:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280041f0 con 0x7fb3300cc8c0 2024-04-05T22:26:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.498+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3300c8e40 con 0x7fb3300cc8c0 2024-04-05T22:26:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.499+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330066af0 con 0x7fb3300cc8c0 2024-04-05T22:26:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.500+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb328004350 con 0x7fb3300cc8c0 2024-04-05T22:26:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.501+0000 7fb32cc00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 0x7fb31006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.501+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fb328088d90 con 0x7fb3300cc8c0 2024-04-05T22:26:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.501+0000 7fb334a00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 0x7fb31006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.503+0000 7fb334a00640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 0x7fb31006d1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb320002230 tx=0x7fb320001390 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:41.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.505+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb328053e90 con 0x7fb3300cc8c0 2024-04-05T22:26:41.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.766+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7fb3300c9390 con 0x7fb3300cc8c0 2024-04-05T22:26:41.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.826+0000 7fb32cc00640 1 -- 172.21.15.73:0/3664711499 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v288) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb3300c9390 con 0x7fb3300cc8c0 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 msgr2=0x7fb31006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 0x7fb31006d1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb320002230 tx=0x7fb320001390 comp rx=0 tx=0).stop 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 msgr2=0x7fb3300ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb330071ee0 tx=0x7fb32800d820 comp rx=0 tx=0).stop 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 shutdown_connections 2024-04-05T22:26:41.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb31006ad20 0x7fb31006d1e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3300cc8c0 0x7fb3300ccd70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb33014e620 0x7fb3301f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 --2- 172.21.15.73:0/3664711499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3140047c0 0x7fb33015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 >> 172.21.15.73:0/3664711499 conn(0x7fb3300ce8e0 msgr2=0x7fb3300caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 shutdown_connections 2024-04-05T22:26:41.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:41.829+0000 7fb337400640 1 -- 172.21.15.73:0/3664711499 wait complete. 2024-04-05T22:26:41.846 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-05T22:26:41.846 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-04-05T22:26:41.846 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:26:41.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.047+0000 7fc1ab200640 1 Processor -- start 2024-04-05T22:26:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.047+0000 7fc1ab200640 1 -- start start 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.048+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1ac142ba0 0x7fc1ac06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.048+0000 7fc1ab200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ac143330 con 0x7fc1ac13fa60 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.048+0000 7fc1ab200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ac143050 con 0x7fc1ac13fe40 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.048+0000 7fc1ab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac06d3a0 con 0x7fc1ac142ba0 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc1ac13fa60 0x7fc1ac142490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38458/0 (socket says 172.21.15.73:38458) 2024-04-05T22:26:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9e00640 1 -- 172.21.15.73:0/105520815 learned_addr learned my addr 172.21.15.73:0/105520815 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813227249 0 0) 0x7fc1ac143330 con 0x7fc1ac13fa60 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc190003520 con 0x7fc1ac13fa60 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1ac142ba0 0x7fc1ac06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 260590475 0 0) 0x7fc190003520 con 0x7fc1ac13fa60 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1ac142ba0 msgr2=0x7fc1ac06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1ac142ba0 0x7fc1ac06f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.049+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 >> v1:172.21.15.157:6789/0 conn(0x7fc1ac13fe40 legacy=0x7fc1ac15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0ac970 con 0x7fc1ac13fa60 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc19c001670 con 0x7fc1ac13fa60 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 >> v1:172.21.15.73:6789/0 conn(0x7fc1ac13fa60 legacy=0x7fc1ac142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900039a0 0x7fc190003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1900047c0 0x7fc190004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc190005180 0x7fc19000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac06d3a0 con 0x7fc1900047c0 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19000c020 con 0x7fc1900039a0 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19000c1e0 con 0x7fc190005180 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1ac142ba0 0x7fc1ac06f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1900047c0 0x7fc190004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1aa800640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc190005180 0x7fc19000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a9e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900039a0 0x7fc190003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 -- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc190005180 msgr2=0x7fc19000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc190005180 0x7fc19000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 -- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900039a0 msgr2=0x7fc190003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900039a0 0x7fc190003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.050+0000 7fc1a3e00640 1 -- 172.21.15.73:0/105520815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc190003520 con 0x7fc1900047c0 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.051+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1900047c0 0x7fc190004c40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc1900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b25b6af457df84 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194032600 con 0x7fc1900047c0 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19402e050 con 0x7fc1900047c0 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1a9400640 1 -- 172.21.15.73:0/105520815 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194032980 con 0x7fc1900047c0 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 -- 172.21.15.73:0/105520815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1900047c0 msgr2=0x7fc190004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 --2- 172.21.15.73:0/105520815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1900047c0 0x7fc190004c40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc1900057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1aa800640 1 -- 172.21.15.73:0/105520815 reap_dead start 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 -- 172.21.15.73:0/105520815 shutdown_connections 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 -- 172.21.15.73:0/105520815 >> 172.21.15.73:0/105520815 conn(0x7fc1ac0b06c0 msgr2=0x7fc1ac0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 -- 172.21.15.73:0/105520815 shutdown_connections 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.052+0000 7fc1ab200640 1 -- 172.21.15.73:0/105520815 wait complete. 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.054+0000 7fc1ab200640 1 Processor -- start 2024-04-05T22:26:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.054+0000 7fc1ab200640 1 -- start start 2024-04-05T22:26:42.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1900047c0 0x7fc1ac15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac13e240 0x7fc1ac0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac15c700 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac15c140 con 0x7fc1ac13e240 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1ab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac15c420 con 0x7fc1900047c0 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1aa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1aa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59768/0 (socket says 172.21.15.73:59768) 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.055+0000 7fc1aa800640 1 -- 172.21.15.73:0/3809353888 learned_addr learned my addr 172.21.15.73:0/3809353888 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1a3e00640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac13e240 0x7fc1ac0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 -- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1900047c0 msgr2=0x7fc1ac15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1900047c0 0x7fc1ac15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 -- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac13e240 msgr2=0x7fc1ac0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac13e240 0x7fc1ac0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 -- 172.21.15.73:0/3809353888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac13da40 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1aa800640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac140220 tx=0x7fc1a400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4003970 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4003ad0 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4003e80 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.056+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac13dd20 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.057+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac13c340 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.058+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac15c870 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.059+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc1a40050d0 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.060+0000 7fc1a2000640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 0x7fc18806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.060+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fc1a40892b0 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.060+0000 7fc1a9e00640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 0x7fc18806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.061+0000 7fc1a9e00640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 0x7fc18806d210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.064+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc1a40543b0 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.256+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1ac13df10 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.257+0000 7fc1a2000640 1 -- 172.21.15.73:0/3809353888 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13317 (secure 0 0 0) 0x7fc1a40579d0 con 0x7fc1ac0cdf20 2024-04-05T22:26:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:42.258 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":78,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:40.800510+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/911489923":"2024-04-06T22:26:37.538434+0000","172.21.15.73:0/745661956":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1767413521":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1452282649":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/265479964":"2024-04-06T22:26:31.437732+0000","172.21.15.73:6833/37063":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3668905466":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/1002310496":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/2340596025":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/307490620":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/206467021":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/3883631467":"2024-04-06T22:26:31.437732+0000","172.21.15.73:0/4239333780":"2024-04-06T22:26:31.437732+0000","172.21.15.157:6834/911489923":"2024-04-06T22:26:37.538434+0000","172.21.15.73:6832/37063":"2024-04-06T22:26:31.437732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.260+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 msgr2=0x7fc18806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.260+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 0x7fc18806d210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009040 comp rx=0 tx=0).stop 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.260+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 msgr2=0x7fc1ac0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.260+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac140220 tx=0x7fc1a400ec10 comp rx=0 tx=0).stop 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 shutdown_connections 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc18806ad50 0x7fc18806d210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1ac0cdf20 0x7fc1ac0cb9a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1ac13e240 0x7fc1ac0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 --2- 172.21.15.73:0/3809353888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1900047c0 0x7fc1ac15b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 >> 172.21.15.73:0/3809353888 conn(0x7fc1ac0b06c0 msgr2=0x7fc1ac141fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:42.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 shutdown_connections 2024-04-05T22:26:42.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.261+0000 7fc1ab200640 1 -- 172.21.15.73:0/3809353888 wait complete. 2024-04-05T22:26:42.280 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:42.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.486+0000 7f6d90e00640 1 Processor -- start 2024-04-05T22:26:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.486+0000 7f6d90e00640 1 -- start start 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.487+0000 7f6d90e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 0x7f6d8c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d90e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c15ddd0 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d90e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c15e0b0 con 0x7f6d8c0b10a0 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d90e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0cd460 con 0x7f6d8c15ce10 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d83e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d8c15d1f0 0x7f6d8c0b0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38472/0 (socket says 172.21.15.73:38472) 2024-04-05T22:26:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d83e00640 1 -- 172.21.15.73:0/3849628707 learned_addr learned my addr 172.21.15.73:0/3849628707 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d89e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 0x7f6d8c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544684836 0 0) 0x7f6d8c15ddd0 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.488+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d70003520 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528254324 0 0) 0x7f6d70003520 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 msgr2=0x7f6d8c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 0x7f6d8c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 >> v1:172.21.15.157:6789/0 conn(0x7f6d8c0b10a0 legacy=0x7f6d8c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0cd740 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6d74001670 con 0x7f6d8c15d1f0 2024-04-05T22:26:42.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 >> v1:172.21.15.73:6789/0 conn(0x7f6d8c15d1f0 legacy=0x7f6d8c0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 0x7f6d700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d70003f20 0x7f6d700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d70004d00 0x7f6d70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0cd460 con 0x7f6d70003f20 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7000ba60 con 0x7f6d70004d00 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7000bc20 con 0x7f6d8c15ce10 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.489+0000 7f6d83e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d70003f20 0x7f6d700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.490+0000 7f6d83e00640 1 -- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 msgr2=0x7f6d700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.490+0000 7f6d83e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d8c15ce10 0x7f6d700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.490+0000 7f6d83e00640 1 -- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d70004d00 msgr2=0x7f6d70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.490+0000 7f6d83e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d70004d00 0x7f6d70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.490+0000 7f6d83e00640 1 -- 172.21.15.73:0/3849628707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70003520 con 0x7f6d70003f20 2024-04-05T22:26:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d83e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d70003f20 0x7f6d700047c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400c240 tx=0x7f6d700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a222635c76d6c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74005ad0 con 0x7f6d70003f20 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74005c30 con 0x7f6d70003f20 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d89400640 1 -- 172.21.15.73:0/3849628707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7402caf0 con 0x7f6d70003f20 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 -- 172.21.15.73:0/3849628707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d70003f20 msgr2=0x7f6d700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 --2- 172.21.15.73:0/3849628707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d70003f20 0x7f6d700047c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400c240 tx=0x7f6d700057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d8a800640 1 -- 172.21.15.73:0/3849628707 reap_dead start 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 -- 172.21.15.73:0/3849628707 shutdown_connections 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 -- 172.21.15.73:0/3849628707 >> 172.21.15.73:0/3849628707 conn(0x7f6d8c099a60 msgr2=0x7f6d8c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 -- 172.21.15.73:0/3849628707 shutdown_connections 2024-04-05T22:26:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.491+0000 7f6d90e00640 1 -- 172.21.15.73:0/3849628707 wait complete. 2024-04-05T22:26:42.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.493+0000 7f6d90e00640 1 Processor -- start 2024-04-05T22:26:42.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.493+0000 7f6d90e00640 1 -- start start 2024-04-05T22:26:42.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 0x7f6d8c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c160700 0x7f6d8c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1610c0 con 0x7f6d8c1556e0 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0c4a30 con 0x7f6d8c160700 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d90e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15c130 con 0x7f6d70003f20 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59784/0 (socket says 172.21.15.73:59784) 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.494+0000 7f6d83e00640 1 -- 172.21.15.73:0/1927933893 learned_addr learned my addr 172.21.15.73:0/1927933893 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d89e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 0x7f6d8c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 -- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 msgr2=0x7f6d8c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d8a800640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c160700 0x7f6d8c160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 0x7f6d8c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 -- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c160700 msgr2=0x7f6d8c160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c160700 0x7f6d8c160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 -- 172.21.15.73:0/1927933893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0b36f0 con 0x7f6d8c1556e0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d89e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 0x7f6d8c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d83e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6d74001930 tx=0x7f6d74001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74039070 con 0x7f6d8c1556e0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7402ccc0 con 0x7f6d8c1556e0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7403c040 con 0x7f6d8c1556e0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c0b39d0 con 0x7f6d8c1556e0 2024-04-05T22:26:42.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.495+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0b3f90 con 0x7f6d8c1556e0 2024-04-05T22:26:42.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.497+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c0b4120 con 0x7f6d8c1556e0 2024-04-05T22:26:42.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.497+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f6d74032070 con 0x7f6d8c1556e0 2024-04-05T22:26:42.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.499+0000 7f6d82000640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 0x7f6d6806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.499+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f6d740aa850 con 0x7f6d8c1556e0 2024-04-05T22:26:42.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.499+0000 7f6d89e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 0x7f6d6806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.501+0000 7f6d89e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 0x7f6d6806d210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009a40 tx=0x7f6d7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.503+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d74075950 con 0x7f6d8c1556e0 2024-04-05T22:26:42.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.765+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d8c067480 con 0x7f6d8c1556e0 2024-04-05T22:26:42.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.766+0000 7f6d82000640 1 -- 172.21.15.73:0/1927933893 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d74078f70 con 0x7f6d8c1556e0 2024-04-05T22:26:42.766 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:42.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 msgr2=0x7f6d6806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 0x7f6d6806d210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009a40 tx=0x7f6d7c008040 comp rx=0 tx=0).stop 2024-04-05T22:26:42.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 msgr2=0x7f6d8c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6d74001930 tx=0x7f6d74001a00 comp rx=0 tx=0).stop 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 shutdown_connections 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6806ad50 0x7f6d6806d210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d8c160700 0x7f6d8c160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d8c1556e0 0x7f6d8c154f00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 --2- 172.21.15.73:0/1927933893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d70003f20 0x7f6d8c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 >> 172.21.15.73:0/1927933893 conn(0x7f6d8c099a60 msgr2=0x7f6d8c0cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 shutdown_connections 2024-04-05T22:26:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.769+0000 7f6d90e00640 1 -- 172.21.15.73:0/1927933893 wait complete. 2024-04-05T22:26:42.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-05T22:26:42.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.986+0000 7f189ae00640 1 Processor -- start 2024-04-05T22:26:42.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.986+0000 7f189ae00640 1 -- start start 2024-04-05T22:26:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.987+0000 7f189ae00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18941541a0 0x7f18940b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.987+0000 7f189ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18940cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.987+0000 7f189ae00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189415aa60 con 0x7f189415aef0 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f189ae00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940b3b60 con 0x7f18941541a0 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f189ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940701c0 con 0x7f189406d710 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f188b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18940cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f1893e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18941541a0 0x7f18940b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f1898e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f189415aef0 0x7f189415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:43214/0 (socket says 172.21.15.73:43214) 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.988+0000 7f1898e00640 1 -- 172.21.15.73:0/1787417731 learned_addr learned my addr 172.21.15.73:0/1787417731 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.989+0000 7f188b400640 1 -- 172.21.15.73:0/1787417731 >> v2:172.21.15.157:3300/0 conn(0x7f18941541a0 msgr2=0x7f18940b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.989+0000 7f188b400640 1 --2- 172.21.15.73:0/1787417731 >> v2:172.21.15.157:3300/0 conn(0x7f18941541a0 0x7f18940b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.989+0000 7f188b400640 1 -- 172.21.15.73:0/1787417731 >> v1:172.21.15.157:6789/0 conn(0x7f189415aef0 legacy=0x7f189415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.989+0000 7f188b400640 1 -- 172.21.15.73:0/1787417731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940b3880 con 0x7f189406d710 2024-04-05T22:26:42.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.991+0000 7f188b400640 1 --2- 172.21.15.73:0/1787417731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18940cd8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f187c009d90 tx=0x7f187c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33139fc24e6d6d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.991+0000 7f1893400640 1 -- 172.21.15.73:0/1787417731 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c02cc50 con 0x7f189406d710 2024-04-05T22:26:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.991+0000 7f1893400640 1 -- 172.21.15.73:0/1787417731 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c02cdb0 con 0x7f189406d710 2024-04-05T22:26:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.991+0000 7f1893400640 1 -- 172.21.15.73:0/1787417731 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c034660 con 0x7f189406d710 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 -- 172.21.15.73:0/1787417731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 msgr2=0x7f18940cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 --2- 172.21.15.73:0/1787417731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18940cd8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f187c009d90 tx=0x7f187c02bea0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 -- 172.21.15.73:0/1787417731 shutdown_connections 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 --2- 172.21.15.73:0/1787417731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18940cd8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 --2- 172.21.15.73:0/1787417731 >> v2:172.21.15.157:3300/0 conn(0x7f18941541a0 0x7f18940b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 -- 172.21.15.73:0/1787417731 >> 172.21.15.73:0/1787417731 conn(0x7f18940ce8e0 msgr2=0x7f18940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:42.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 -- 172.21.15.73:0/1787417731 shutdown_connections 2024-04-05T22:26:42.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.992+0000 7f189ae00640 1 -- 172.21.15.73:0/1787417731 wait complete. 2024-04-05T22:26:42.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.994+0000 7f189ae00640 1 Processor -- start 2024-04-05T22:26:42.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.994+0000 7f189ae00640 1 -- start start 2024-04-05T22:26:42.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18941e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 0x7f18941e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941468c0 con 0x7f189415aef0 2024-04-05T22:26:42.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894141120 con 0x7f18941541a0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f189ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894141400 con 0x7f189406d710 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f1898e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f1898e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59786/0 (socket says 172.21.15.73:59786) 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f1898e00640 1 -- 172.21.15.73:0/2831941346 learned_addr learned my addr 172.21.15.73:0/2831941346 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f1893e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18941e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.995+0000 7f188b400640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 0x7f18941e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 -- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 msgr2=0x7f18941e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18941e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 -- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 msgr2=0x7f18941e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 0x7f18941e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 -- 172.21.15.73:0/2831941346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c009a70 con 0x7f189415aef0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f188b400640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 0x7f18941e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1898e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c370 tx=0x7f188c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:42.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c005ce0 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c005e40 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0072d0 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18941416e0 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.996+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894141ca0 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.997+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1894066af0 con 0x7f189415aef0 2024-04-05T22:26:42.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.998+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f188c0074f0 con 0x7f189415aef0 2024-04-05T22:26:42.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.999+0000 7f1890c00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:42.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:42.999+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f188c088830 con 0x7f189415aef0 2024-04-05T22:26:43.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.000+0000 7f1893e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:43.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.001+0000 7f1893e00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1884009ad0 tx=0x7f1884008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:43.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.003+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f188c053a80 con 0x7f189415aef0 2024-04-05T22:26:43.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.254+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f1894141e90 con 0x7f189415aef0 2024-04-05T22:26:43.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.838+0000 7f1890c00640 1 -- 172.21.15.73:0/2831941346 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v290) v1 ==== 149+0+0 (secure 0 0 0) 0x7f188c0170d0 con 0x7f189415aef0 2024-04-05T22:26:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.841+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 msgr2=0x7f187006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.841+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1884009ad0 tx=0x7f1884008040 comp rx=0 tx=0).stop 2024-04-05T22:26:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 msgr2=0x7f1894140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c370 tx=0x7f188c00c830 comp rx=0 tx=0).stop 2024-04-05T22:26:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 shutdown_connections 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189415aef0 0x7f1894140460 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941541a0 0x7f18941e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 --2- 172.21.15.73:0/2831941346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189406d710 0x7f18941e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 >> 172.21.15.73:0/2831941346 conn(0x7f18940ce8e0 msgr2=0x7f18940b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 shutdown_connections 2024-04-05T22:26:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:43.842+0000 7f189ae00640 1 -- 172.21.15.73:0/2831941346 wait complete. 2024-04-05T22:26:43.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:26:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.071+0000 7f40ef800640 1 Processor -- start 2024-04-05T22:26:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.071+0000 7f40ef800640 1 -- start start 2024-04-05T22:26:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ef800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40e806cd50 0x7f40e8070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ef800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e80b27f0 con 0x7f40e81599f0 2024-04-05T22:26:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ef800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e80b2ad0 con 0x7f40e80add00 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ef800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80b3890 con 0x7f40e806cd50 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40e806cd50 0x7f40e8070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ece00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f40e81599f0 0x7f40e806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38480/0 (socket says 172.21.15.73:38480) 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40ece00640 1 -- 172.21.15.73:0/2095387127 learned_addr learned my addr 172.21.15.73:0/2095387127 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40e7e00640 1 -- 172.21.15.73:0/2095387127 >> v1:172.21.15.157:6789/0 conn(0x7f40e80add00 legacy=0x7f40e8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40e7e00640 1 -- 172.21.15.73:0/2095387127 >> v1:172.21.15.73:6789/0 conn(0x7f40e81599f0 legacy=0x7f40e806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40e7e00640 1 -- 172.21.15.73:0/2095387127 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80b2db0 con 0x7f40e806cd50 2024-04-05T22:26:44.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.073+0000 7f40e7400640 1 -- 172.21.15.73:0/2095387127 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232422145 0 0) 0x7f40e80b27f0 con 0x7f40e81599f0 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40e7e00640 1 --2- 172.21.15.73:0/2095387127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e806cd50 0x7f40e8070a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f40d800bfa0 tx=0x7f40d802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2295728b78d12599 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40e7400640 1 -- 172.21.15.73:0/2095387127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d802ea60 con 0x7f40e806cd50 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40e7400640 1 -- 172.21.15.73:0/2095387127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d802ebc0 con 0x7f40e806cd50 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40e7400640 1 -- 172.21.15.73:0/2095387127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8009540 con 0x7f40e806cd50 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40ef800640 1 -- 172.21.15.73:0/2095387127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e806cd50 msgr2=0x7f40e8070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40ef800640 1 --2- 172.21.15.73:0/2095387127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e806cd50 0x7f40e8070a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f40d800bfa0 tx=0x7f40d802e0c0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40ef800640 1 -- 172.21.15.73:0/2095387127 shutdown_connections 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40ef800640 1 --2- 172.21.15.73:0/2095387127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e806cd50 0x7f40e8070a40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.075+0000 7f40ef800640 1 -- 172.21.15.73:0/2095387127 >> 172.21.15.73:0/2095387127 conn(0x7f40e8069cb0 msgr2=0x7f40e806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.076+0000 7f40ef800640 1 -- 172.21.15.73:0/2095387127 shutdown_connections 2024-04-05T22:26:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.076+0000 7f40ef800640 1 -- 172.21.15.73:0/2095387127 wait complete. 2024-04-05T22:26:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.077+0000 7f40ef800640 1 Processor -- start 2024-04-05T22:26:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.077+0000 7f40ef800640 1 -- start start 2024-04-05T22:26:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e806cd50 0x7f40e80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40e81599f0 0x7f40e81476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80ce5d0 con 0x7f40e80add00 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e814a650 con 0x7f40e806cd50 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40ef800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80ccff0 con 0x7f40e81599f0 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.078+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59808/0 (socket says 172.21.15.73:59808) 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 -- 172.21.15.73:0/949474246 learned_addr learned my addr 172.21.15.73:0/949474246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 -- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40e81599f0 msgr2=0x7f40e81476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40e81599f0 0x7f40e81476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 -- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e806cd50 msgr2=0x7f40e80ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e806cd50 0x7f40e80ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 -- 172.21.15.73:0/949474246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d800bc80 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e7e00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f80 tx=0x7f40d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8036800 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d8036980 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8036d30 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.079+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e80cd2d0 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.080+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e80cd890 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.081+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e8068160 con 0x7f40e80add00 2024-04-05T22:26:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.082+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f40d804a460 con 0x7f40e80add00 2024-04-05T22:26:44.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.082+0000 7f40e4c00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 0x7f40c406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.083+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f40d80ac7d0 con 0x7f40e80add00 2024-04-05T22:26:44.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.083+0000 7f40ece00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 0x7f40c406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:44.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.084+0000 7f40ece00640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 0x7f40c406d1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40d0007fd0 tx=0x7f40d0007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:44.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.087+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f40d8077b20 con 0x7f40e80add00 2024-04-05T22:26:44.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.278+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f40e8068160 con 0x7f40e80add00 2024-04-05T22:26:44.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.286+0000 7f40e4c00640 1 -- 172.21.15.73:0/949474246 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f40d807b140 con 0x7f40e80add00 2024-04-05T22:26:44.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.288+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 msgr2=0x7f40c406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.288+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 0x7f40c406d1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40d0007fd0 tx=0x7f40d0007960 comp rx=0 tx=0).stop 2024-04-05T22:26:44.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 msgr2=0x7f40e814a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f80 tx=0x7f40d8004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:44.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 shutdown_connections 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40c406ad20 0x7f40c406d1e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40e81599f0 0x7f40e81476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e80add00 0x7f40e814a110 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 --2- 172.21.15.73:0/949474246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e806cd50 0x7f40e80ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 >> 172.21.15.73:0/949474246 conn(0x7f40e8069cb0 msgr2=0x7f40e80ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 shutdown_connections 2024-04-05T22:26:44.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.289+0000 7f40ef800640 1 -- 172.21.15.73:0/949474246 wait complete. 2024-04-05T22:26:44.307 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:26:44.307 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:44.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:26:44.540 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:26:44.541 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:44.541 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:44.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:44.598 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:44.598 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:44.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:44.656 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:44.657 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:44.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:26:44.713 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:26:44.713 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:44.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:26:44.770 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:26:44.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.989+0000 7fc3b3200640 1 Processor -- start 2024-04-05T22:26:44.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.989+0000 7fc3b3200640 1 -- start start 2024-04-05T22:26:44.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.990+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.990+0000 7fc3b3200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 0x7fc3ac0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3b3200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ac15aa60 con 0x7fc3ac1541a0 2024-04-05T22:26:44.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3b3200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ac0b3b60 con 0x7fc3ac15aef0 2024-04-05T22:26:44.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3b3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac0701c0 con 0x7fc3ac06d710 2024-04-05T22:26:44.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:44.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:55606/0 (socket says 172.21.15.73:55606) 2024-04-05T22:26:44.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.991+0000 7fc3ab400640 1 -- 172.21.15.73:0/762534962 learned_addr learned my addr 172.21.15.73:0/762534962 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:44.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3b1200640 1 --2- 172.21.15.73:0/762534962 >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 0x7fc3ac0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3ab400640 1 -- 172.21.15.73:0/762534962 >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 msgr2=0x7fc3ac0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3ab400640 1 --2- 172.21.15.73:0/762534962 >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 0x7fc3ac0cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3ab400640 1 -- 172.21.15.73:0/762534962 >> v1:172.21.15.157:6789/0 conn(0x7fc3ac1541a0 legacy=0x7fc3ac0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3ab400640 1 -- 172.21.15.73:0/762534962 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0b3880 con 0x7fc3ac06d710 2024-04-05T22:26:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.992+0000 7fc3b1200640 1 --2- 172.21.15.73:0/762534962 >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 0x7fc3ac0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:26:44.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.995+0000 7fc3ab400640 1 --2- 172.21.15.73:0/762534962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b242fc67cf3d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.995+0000 7fc3aaa00640 1 -- 172.21.15.73:0/762534962 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398030840 con 0x7fc3ac06d710 2024-04-05T22:26:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.995+0000 7fc3aaa00640 1 -- 172.21.15.73:0/762534962 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980309a0 con 0x7fc3ac06d710 2024-04-05T22:26:44.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.995+0000 7fc3aaa00640 1 -- 172.21.15.73:0/762534962 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398030d30 con 0x7fc3ac06d710 2024-04-05T22:26:44.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 -- 172.21.15.73:0/762534962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 msgr2=0x7fc3ac0cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:44.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 --2- 172.21.15.73:0/762534962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802b980 comp rx=0 tx=0).stop 2024-04-05T22:26:44.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 -- 172.21.15.73:0/762534962 shutdown_connections 2024-04-05T22:26:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 --2- 172.21.15.73:0/762534962 >> v2:172.21.15.157:3300/0 conn(0x7fc3ac15aef0 0x7fc3ac0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 --2- 172.21.15.73:0/762534962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac0cd8f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 -- 172.21.15.73:0/762534962 >> 172.21.15.73:0/762534962 conn(0x7fc3ac0ce8e0 msgr2=0x7fc3ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 -- 172.21.15.73:0/762534962 shutdown_connections 2024-04-05T22:26:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.996+0000 7fc3b3200640 1 -- 172.21.15.73:0/762534962 wait complete. 2024-04-05T22:26:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.998+0000 7fc3b3200640 1 Processor -- start 2024-04-05T22:26:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.998+0000 7fc3b3200640 1 -- start start 2024-04-05T22:26:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 0x7fc3ac152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac14b5c0 con 0x7fc3ac15aef0 2024-04-05T22:26:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1538b0 con 0x7fc3ac1541a0 2024-04-05T22:26:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac06b330 con 0x7fc3ac06d710 2024-04-05T22:26:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59822/0 (socket says 172.21.15.73:59822) 2024-04-05T22:26:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:44.999+0000 7fc3b1200640 1 -- 172.21.15.73:0/2439062051 learned_addr learned my addr 172.21.15.73:0/2439062051 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3ab400640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 0x7fc3ac152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3abe00640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 -- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 msgr2=0x7fc3ac14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac14aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 -- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 msgr2=0x7fc3ac152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 0x7fc3ac152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 -- 172.21.15.73:0/2439062051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009a70 con 0x7fc3ac15aef0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3ab400640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 0x7fc3ac152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b1200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400efc0 tx=0x7fc3a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4009cd0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a400f040 con 0x7fc3ac15aef0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4003990 con 0x7fc3ac15aef0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.000+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3980094d0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.001+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ac06b8f0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.002+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ac066af0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.003+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc3a4014480 con 0x7fc3ac15aef0 2024-04-05T22:26:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.003+0000 7fc3a3e00640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 0x7fc38c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.004+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fc3a408a160 con 0x7fc3ac15aef0 2024-04-05T22:26:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.004+0000 7fc3abe00640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 0x7fc38c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.006+0000 7fc3abe00640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 0x7fc38c06d1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac151260 tx=0x7fc39c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:45.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.008+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc3a40551f0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.236+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3ac068160 con 0x7fc3ac15aef0 2024-04-05T22:26:45.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.238+0000 7fc3a3e00640 1 -- 172.21.15.73:0/2439062051 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fc3a4018de0 con 0x7fc3ac15aef0 2024-04-05T22:26:45.238 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:45.238 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1400680628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1400680628},{"type":"v1","addr":"172.21.15.157:6835","nonce":1400680628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2206119680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2206119680},{"type":"v1","addr":"172.21.15.157:6833","nonce":2206119680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3465759356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3465759356},{"type":"v1","addr":"172.21.15.73:6836","nonce":3465759356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2098315012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2098315012},{"type":"v1","addr":"172.21.15.73:6837","nonce":2098315012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2024-04-05T22:26:45.238 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 44 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 msgr2=0x7fc38c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 0x7fc38c06d1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac151260 tx=0x7fc39c009040 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 msgr2=0x7fc3ac153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400efc0 tx=0x7fc3a400c490 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 shutdown_connections 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc38c06ad20 0x7fc38c06d1e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3ac15aef0 0x7fc3ac153280 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac1541a0 0x7fc3ac152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 --2- 172.21.15.73:0/2439062051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ac06d710 0x7fc3ac14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.240+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 >> 172.21.15.73:0/2439062051 conn(0x7fc3ac0ce8e0 msgr2=0x7fc3ac159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.241+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 shutdown_connections 2024-04-05T22:26:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.241+0000 7fc3b3200640 1 -- 172.21.15.73:0/2439062051 wait complete. 2024-04-05T22:26:45.258 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:26:45.259 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:26:45.259 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:26:45.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:45.259 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:26:45.259 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:26:45.259 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:26:45.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:45.260 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:26:45.260 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:26:45.260 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:26:45.260 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:45.260 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:26:45.260 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:26:45.260 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:26:45.260 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:26:45.278 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:26:45.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:26:45.284 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:26:45.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:26:45.287 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:26:45.288 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:26:45.288 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:26:45.289 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:26:45.330 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:26:45.331 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:45.336 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:26:45.336 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:45.337 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:26:45.337 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:45.341 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:26:45.341 DEBUG:teuthology.parallel:result is None 2024-04-05T22:26:45.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:26:45.485 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:26:45.488 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:26:45.497 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:26:45.511 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:26:45.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.531+0000 7f0496800640 1 Processor -- start 2024-04-05T22:26:45.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.531+0000 7f0496800640 1 -- start start 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0496800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f049006cb80 0x7f049006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0496800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049006f5b0 con 0x7f049015e850 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0496800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0490070410 con 0x7f04901587d0 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0496800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049006d500 con 0x7f049006cb80 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0486a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f049006cb80 0x7f049006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f048fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f049015e850 0x7f04900708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38502/0 (socket says 172.21.15.73:38502) 2024-04-05T22:26:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f048fe00640 1 -- 172.21.15.73:0/1232156758 learned_addr learned my addr 172.21.15.73:0/1232156758 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:45.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0486a00640 1 -- 172.21.15.73:0/1232156758 >> v1:172.21.15.157:6789/0 conn(0x7f04901587d0 legacy=0x7f04900cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0486a00640 1 -- 172.21.15.73:0/1232156758 >> v1:172.21.15.73:6789/0 conn(0x7f049015e850 legacy=0x7f04900708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f0486a00640 1 -- 172.21.15.73:0/1232156758 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04900ccef0 con 0x7f049006cb80 2024-04-05T22:26:45.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.533+0000 7f048ea00640 1 -- 172.21.15.73:0/1232156758 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816685297 0 0) 0x7f049006f5b0 con 0x7f049015e850 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.534+0000 7f0486a00640 1 --2- 172.21.15.73:0/1232156758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049006cb80 0x7f049006cfc0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f04780097a0 tx=0x7f047802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be013a0e0d1303c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f048ea00640 1 -- 172.21.15.73:0/1232156758 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478030610 con 0x7f049006cb80 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f048ea00640 1 -- 172.21.15.73:0/1232156758 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478030770 con 0x7f049006cb80 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f048ea00640 1 -- 172.21.15.73:0/1232156758 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478030b00 con 0x7f049006cb80 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f0496800640 1 -- 172.21.15.73:0/1232156758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049006cb80 msgr2=0x7f049006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f0496800640 1 --2- 172.21.15.73:0/1232156758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049006cb80 0x7f049006cfc0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f04780097a0 tx=0x7f047802b960 comp rx=0 tx=0).stop 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f0496800640 1 -- 172.21.15.73:0/1232156758 shutdown_connections 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f0496800640 1 --2- 172.21.15.73:0/1232156758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049006cb80 0x7f049006cfc0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.535+0000 7f0496800640 1 -- 172.21.15.73:0/1232156758 >> 172.21.15.73:0/1232156758 conn(0x7f04900ce8e0 msgr2=0x7f04900cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:45.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.536+0000 7f0496800640 1 -- 172.21.15.73:0/1232156758 shutdown_connections 2024-04-05T22:26:45.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.536+0000 7f0496800640 1 -- 172.21.15.73:0/1232156758 wait complete. 2024-04-05T22:26:45.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.537+0000 7f0496800640 1 Processor -- start 2024-04-05T22:26:45.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.537+0000 7f0496800640 1 -- start start 2024-04-05T22:26:45.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049006cb80 0x7f04900ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04901587d0 0x7f04900aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900af690 con 0x7f049015e850 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0490157b60 con 0x7f049006cb80 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.538+0000 7f0496800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04900ad3d0 con 0x7f04901587d0 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f0486a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04901587d0 0x7f04900aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59852/0 (socket says 172.21.15.73:59852) 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 -- 172.21.15.73:0/1218319802 learned_addr learned my addr 172.21.15.73:0/1218319802 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048f400640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049006cb80 0x7f04900ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 -- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04901587d0 msgr2=0x7f04900aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04901587d0 0x7f04900aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 -- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049006cb80 msgr2=0x7f04900ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049006cb80 0x7f04900ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 -- 172.21.15.73:0/1218319802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478009400 con 0x7f049015e850 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f048fe00640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f048800c7b0 tx=0x7f048800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:45.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.539+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800a640 con 0x7f049015e850 2024-04-05T22:26:45.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.540+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800a7c0 con 0x7f049015e850 2024-04-05T22:26:45.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.540+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800ab70 con 0x7f049015e850 2024-04-05T22:26:45.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.540+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04900aea10 con 0x7f049015e850 2024-04-05T22:26:45.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.540+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04900aefd0 con 0x7f049015e850 2024-04-05T22:26:45.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.541+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0490066af0 con 0x7f049015e850 2024-04-05T22:26:45.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.542+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f048800acd0 con 0x7f049015e850 2024-04-05T22:26:45.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.542+0000 7f0487e00640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 0x7f046c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:45.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.543+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f04880887b0 con 0x7f049015e850 2024-04-05T22:26:45.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.543+0000 7f048f400640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 0x7f046c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:45.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.544+0000 7f048f400640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 0x7f046c06d1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0480000f60 tx=0x7f0480009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:45.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.548+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f04880539f0 con 0x7f049015e850 2024-04-05T22:26:45.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:45.752+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0490068160 con 0x7f049015e850 2024-04-05T22:26:46.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.535+0000 7f0487e00640 1 -- 172.21.15.73:0/1218319802 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0488018080 con 0x7f049015e850 2024-04-05T22:26:46.535 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:26:46.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 msgr2=0x7f046c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 0x7f046c06d1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0480000f60 tx=0x7f0480009040 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 msgr2=0x7f04900ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f048800c7b0 tx=0x7f048800cc70 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 shutdown_connections 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f046c06ad20 0x7f046c06d1e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f049015e850 0x7f04900ab3e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04901587d0 0x7f04900aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 --2- 172.21.15.73:0/1218319802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049006cb80 0x7f04900ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 >> 172.21.15.73:0/1218319802 conn(0x7f04900ce8e0 msgr2=0x7f0490158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.538+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 shutdown_connections 2024-04-05T22:26:46.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.539+0000 7f0496800640 1 -- 172.21.15.73:0/1218319802 wait complete. 2024-04-05T22:26:46.559 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:26:46.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.770+0000 7f813ce00640 1 Processor -- start 2024-04-05T22:26:46.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.770+0000 7f813ce00640 1 -- start start 2024-04-05T22:26:46.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f813ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81381587d0 0x7f81380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f813ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81380ccd20 0x7f813806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f813ce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813806f5b0 con 0x7f813806d0c0 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f813ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8138070410 con 0x7f81381587d0 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f813ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8138158bb0 con 0x7f81380ccd20 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81381587d0 0x7f81380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81381587d0 0x7f81380cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42782/0 (socket says 172.21.15.73:42782) 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 -- 172.21.15.73:0/1112390998 learned_addr learned my addr 172.21.15.73:0/1112390998 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 -- 172.21.15.73:0/1112390998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81380ccd20 msgr2=0x7f813806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 --2- 172.21.15.73:0/1112390998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81380ccd20 0x7f813806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 -- 172.21.15.73:0/1112390998 >> v1:172.21.15.157:6789/0 conn(0x7f813806d0c0 legacy=0x7f813815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:46.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.772+0000 7f8135e00640 1 -- 172.21.15.73:0/1112390998 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813806f800 con 0x7f81381587d0 2024-04-05T22:26:46.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f8135e00640 1 --2- 172.21.15.73:0/1112390998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381587d0 0x7f81380cc7e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8128009870 tx=0x7f812802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974f05eeace36ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:46.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f8134a00640 1 -- 172.21.15.73:0/1112390998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128031640 con 0x7f81381587d0 2024-04-05T22:26:46.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f8134a00640 1 -- 172.21.15.73:0/1112390998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81280317a0 con 0x7f81381587d0 2024-04-05T22:26:46.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f8134a00640 1 -- 172.21.15.73:0/1112390998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128031b30 con 0x7f81381587d0 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f813ce00640 1 -- 172.21.15.73:0/1112390998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381587d0 msgr2=0x7f81380cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.774+0000 7f813ce00640 1 --2- 172.21.15.73:0/1112390998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381587d0 0x7f81380cc7e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8128009870 tx=0x7f812802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 -- 172.21.15.73:0/1112390998 shutdown_connections 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 --2- 172.21.15.73:0/1112390998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81380ccd20 0x7f813806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 --2- 172.21.15.73:0/1112390998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381587d0 0x7f81380cc7e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 -- 172.21.15.73:0/1112390998 >> 172.21.15.73:0/1112390998 conn(0x7f81380ce8e0 msgr2=0x7f81380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 -- 172.21.15.73:0/1112390998 shutdown_connections 2024-04-05T22:26:46.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.775+0000 7f813ce00640 1 -- 172.21.15.73:0/1112390998 wait complete. 2024-04-05T22:26:46.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.776+0000 7f813ce00640 1 Processor -- start 2024-04-05T22:26:46.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 -- start start 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813806d0c0 0x7f8138142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 0x7f813815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81381427d0 con 0x7f81381e9460 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813815d380 con 0x7f81381587d0 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.777+0000 7f813ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813815c770 con 0x7f813806d0c0 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42790/0 (socket says 172.21.15.73:42790) 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 -- 172.21.15.73:0/1348137363 learned_addr learned my addr 172.21.15.73:0/1348137363 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8135e00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813806d0c0 0x7f8138142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8135400640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 0x7f813815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 -- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813806d0c0 msgr2=0x7f8138142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813806d0c0 0x7f8138142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 -- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 msgr2=0x7f813815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 0x7f813815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8136800640 1 -- 172.21.15.73:0/1348137363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280094d0 con 0x7f81381e9460 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.778+0000 7f8135400640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 0x7f813815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f8136800640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f813000c780 tx=0x7f813000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130004b60 con 0x7f81381e9460 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000ce90 con 0x7f81381e9460 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000a850 con 0x7f81381e9460 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813815ca50 con 0x7f81381e9460 2024-04-05T22:26:46.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.779+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813815d010 con 0x7f81381e9460 2024-04-05T22:26:46.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.780+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8138066af0 con 0x7f81381e9460 2024-04-05T22:26:46.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.781+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f813000a9b0 con 0x7f81381e9460 2024-04-05T22:26:46.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.782+0000 7f812e000640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 0x7f811806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:46.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.782+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8130089320 con 0x7f81381e9460 2024-04-05T22:26:46.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.783+0000 7f8135e00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 0x7f811806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:46.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.784+0000 7f8135e00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 0x7f811806d1e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8128036000 tx=0x7f812803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:46.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:46.786+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8130054730 con 0x7f81381e9460 2024-04-05T22:26:47.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.050+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8138158df0 con 0x7f81381e9460 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.051+0000 7f812e000640 1 -- 172.21.15.73:0/1348137363 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8138158df0 con 0x7f81381e9460 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:26:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:26:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:26:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:26:47.060 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:26:47.062 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:26:47.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.053+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 msgr2=0x7f811806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.053+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 0x7f811806d1e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8128036000 tx=0x7f812803c040 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.053+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 msgr2=0x7f813815da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.053+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f813000c780 tx=0x7f813000cc40 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 shutdown_connections 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811806ad20 0x7f811806d1e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381e9460 0x7f813815da70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81381587d0 0x7f813815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 --2- 172.21.15.73:0/1348137363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813806d0c0 0x7f8138142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 >> 172.21.15.73:0/1348137363 conn(0x7f81380ce8e0 msgr2=0x7f81380cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 shutdown_connections 2024-04-05T22:26:47.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.054+0000 7f813ce00640 1 -- 172.21.15.73:0/1348137363 wait complete. 2024-04-05T22:26:47.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:47.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.279+0000 7fa5c8600640 1 Processor -- start 2024-04-05T22:26:47.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.280+0000 7fa5c8600640 1 -- start start 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5c015ad90 0x7fa5c00ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c00adc20 0x7fa5c00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5c00cf390 0x7fa5c00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00b1d00 con 0x7fa5c00cf390 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00aef40 con 0x7fa5c015ad90 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c015b170 con 0x7fa5c00adc20 2024-04-05T22:26:47.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5c00cf390 0x7fa5c00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5c00cf390 0x7fa5c00b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42792/0 (socket says 172.21.15.73:42792) 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.281+0000 7fa5c6600640 1 -- 172.21.15.73:0/1141182571 learned_addr learned my addr 172.21.15.73:0/1141182571 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c5200640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c00adc20 0x7fa5c00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c5c00640 1 --2- 172.21.15.73:0/1141182571 >> v2:172.21.15.157:3300/0 conn(0x7fa5c015ad90 0x7fa5c00ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c6600640 1 -- 172.21.15.73:0/1141182571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c00adc20 msgr2=0x7fa5c00cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c6600640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c00adc20 0x7fa5c00cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c6600640 1 -- 172.21.15.73:0/1141182571 >> v2:172.21.15.157:3300/0 conn(0x7fa5c015ad90 msgr2=0x7fa5c00ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c6600640 1 --2- 172.21.15.73:0/1141182571 >> v2:172.21.15.157:3300/0 conn(0x7fa5c015ad90 0x7fa5c00ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.282+0000 7fa5c6600640 1 -- 172.21.15.73:0/1141182571 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00ae080 con 0x7fa5c00cf390 2024-04-05T22:26:47.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.283+0000 7fa5c6600640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00b09c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa5c015e7d0 tx=0x7fa5b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd9653a8ba5f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.283+0000 7fa5bfe00640 1 -- 172.21.15.73:0/1141182571 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80317e0 con 0x7fa5c00cf390 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5bfe00640 1 -- 172.21.15.73:0/1141182571 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8031940 con 0x7fa5c00cf390 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5bfe00640 1 -- 172.21.15.73:0/1141182571 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8031cd0 con 0x7fa5c00cf390 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 -- 172.21.15.73:0/1141182571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 msgr2=0x7fa5c00b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00b09c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa5c015e7d0 tx=0x7fa5b802b880 comp rx=0 tx=0).stop 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 -- 172.21.15.73:0/1141182571 shutdown_connections 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00b09c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 --2- 172.21.15.73:0/1141182571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c00adc20 0x7fa5c00cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 --2- 172.21.15.73:0/1141182571 >> v2:172.21.15.157:3300/0 conn(0x7fa5c015ad90 0x7fa5c00ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 -- 172.21.15.73:0/1141182571 >> 172.21.15.73:0/1141182571 conn(0x7fa5c0099a60 msgr2=0x7fa5c0099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 -- 172.21.15.73:0/1141182571 shutdown_connections 2024-04-05T22:26:47.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.284+0000 7fa5c8600640 1 -- 172.21.15.73:0/1141182571 wait complete. 2024-04-05T22:26:47.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.287+0000 7fa5c8600640 1 Processor -- start 2024-04-05T22:26:47.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.287+0000 7fa5c8600640 1 -- start start 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c00adc20 0x7fa5c01491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c015ad90 0x7fa5c00c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01508b0 con 0x7fa5c00cf390 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c006adc0 con 0x7fa5c00adc20 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00ca960 con 0x7fa5c015ad90 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42804/0 (socket says 172.21.15.73:42804) 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.288+0000 7fa5c5200640 1 -- 172.21.15.73:0/3576283708 learned_addr learned my addr 172.21.15.73:0/3576283708 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5c00640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c00adc20 0x7fa5c01491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c6600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c015ad90 0x7fa5c00c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 -- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c015ad90 msgr2=0x7fa5c00c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c015ad90 0x7fa5c00c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 -- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c00adc20 msgr2=0x7fa5c01491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c00adc20 0x7fa5c01491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 -- 172.21.15.73:0/3576283708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80094d0 con 0x7fa5c00cf390 2024-04-05T22:26:47.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c5200640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac007e60 tx=0x7fa5ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac014620 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00ee90 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0149a0 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c00c9090 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.289+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c00c9650 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.291+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c0068160 con 0x7fa5c00cf390 2024-04-05T22:26:47.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.291+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fa5ac014b00 con 0x7fa5c00cf390 2024-04-05T22:26:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.292+0000 7fa5bd600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 0x7fa5a006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.292+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5ac089ae0 con 0x7fa5c00cf390 2024-04-05T22:26:47.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.293+0000 7fa5c5c00640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 0x7fa5a006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.294+0000 7fa5c5c00640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 0x7fa5a006d1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000a970 tx=0x7fa5b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.297+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa5ac054ef0 con 0x7fa5c00cf390 2024-04-05T22:26:47.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.493+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c00b2300 con 0x7fa5c00cf390 2024-04-05T22:26:47.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.494+0000 7fa5bd600640 1 -- 172.21.15.73:0/3576283708 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12390 (secure 0 0 0) 0x7fa5ac009980 con 0x7fa5c00cf390 2024-04-05T22:26:47.494 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:47.494 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:46.528205+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 msgr2=0x7fa5a006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 0x7fa5a006d1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000a970 tx=0x7fa5b0008040 comp rx=0 tx=0).stop 2024-04-05T22:26:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 msgr2=0x7fa5c00ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac007e60 tx=0x7fa5ac00e4b0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 shutdown_connections 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa5a006ad20 0x7fa5a006d1e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c015ad90 0x7fa5c00c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c00cf390 0x7fa5c00ca420 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 --2- 172.21.15.73:0/3576283708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c00adc20 0x7fa5c01491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 >> 172.21.15.73:0/3576283708 conn(0x7fa5c0099a60 msgr2=0x7fa5c006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 shutdown_connections 2024-04-05T22:26:47.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.497+0000 7fa5c8600640 1 -- 172.21.15.73:0/3576283708 wait complete. 2024-04-05T22:26:47.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:26:47.516 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:26:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.719+0000 7f6ccea00640 1 Processor -- start 2024-04-05T22:26:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.719+0000 7f6ccea00640 1 -- start start 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6ccea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cc815aa70 0x7f6cc8070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6ccea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cc815dd90 0x7f6cc806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6ccea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc815ae50 con 0x7f6cc80714a0 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6ccea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc8071930 con 0x7f6cc815aa70 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6ccea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc815e1a0 con 0x7f6cc815dd90 2024-04-05T22:26:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cc815aa70 0x7f6cc8070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cc815aa70 0x7f6cc8070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42806/0 (socket says 172.21.15.73:42806) 2024-04-05T22:26:47.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.721+0000 7f6cc7e00640 1 -- 172.21.15.73:0/2198016347 learned_addr learned my addr 172.21.15.73:0/2198016347 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:47.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.722+0000 7f6cc7e00640 1 -- 172.21.15.73:0/2198016347 >> v2:172.21.15.157:3300/0 conn(0x7f6cc815dd90 msgr2=0x7f6cc806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.722+0000 7f6cc7e00640 1 --2- 172.21.15.73:0/2198016347 >> v2:172.21.15.157:3300/0 conn(0x7f6cc815dd90 0x7f6cc806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.722+0000 7f6cc7e00640 1 -- 172.21.15.73:0/2198016347 >> v1:172.21.15.157:6789/0 conn(0x7f6cc80714a0 legacy=0x7f6cc815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.722+0000 7f6cc7e00640 1 -- 172.21.15.73:0/2198016347 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc815e3b0 con 0x7f6cc815aa70 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.723+0000 7f6cc7e00640 1 --2- 172.21.15.73:0/2198016347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc8070f60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6cc806cf00 tx=0x7f6cb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7782a41a65bec1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.723+0000 7f6cc7400640 1 -- 172.21.15.73:0/2198016347 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8031870 con 0x7f6cc815aa70 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.723+0000 7f6cc7400640 1 -- 172.21.15.73:0/2198016347 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb80319d0 con 0x7f6cc815aa70 2024-04-05T22:26:47.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.723+0000 7f6cc7400640 1 -- 172.21.15.73:0/2198016347 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8031d60 con 0x7f6cc815aa70 2024-04-05T22:26:47.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 -- 172.21.15.73:0/2198016347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 msgr2=0x7f6cc8070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 --2- 172.21.15.73:0/2198016347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc8070f60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6cc806cf00 tx=0x7f6cb802b910 comp rx=0 tx=0).stop 2024-04-05T22:26:47.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 -- 172.21.15.73:0/2198016347 shutdown_connections 2024-04-05T22:26:47.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 --2- 172.21.15.73:0/2198016347 >> v2:172.21.15.157:3300/0 conn(0x7f6cc815dd90 0x7f6cc806ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 --2- 172.21.15.73:0/2198016347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc8070f60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 -- 172.21.15.73:0/2198016347 >> 172.21.15.73:0/2198016347 conn(0x7f6cc8069cb0 msgr2=0x7f6cc806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:47.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 -- 172.21.15.73:0/2198016347 shutdown_connections 2024-04-05T22:26:47.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.724+0000 7f6ccea00640 1 -- 172.21.15.73:0/2198016347 wait complete. 2024-04-05T22:26:47.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.726+0000 7f6ccea00640 1 Processor -- start 2024-04-05T22:26:47.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.726+0000 7f6ccea00640 1 -- start start 2024-04-05T22:26:47.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cc80714a0 0x7f6cc815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cc815dd90 0x7f6cc806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81584d0 con 0x7f6cc815aa70 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81578c0 con 0x7f6cc80714a0 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.727+0000 7f6ccea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8157ba0 con 0x7f6cc815dd90 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42816/0 (socket says 172.21.15.73:42816) 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 -- 172.21.15.73:0/1959457399 learned_addr learned my addr 172.21.15.73:0/1959457399 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 -- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cc815dd90 msgr2=0x7f6cc806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cc815dd90 0x7f6cc806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cc7e00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cc80714a0 0x7f6cc815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 -- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cc80714a0 msgr2=0x7f6cc815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cc80714a0 0x7f6cc815cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 -- 172.21.15.73:0/1959457399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80094d0 con 0x7f6cc815aa70 2024-04-05T22:26:47.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.728+0000 7f6cbf400640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c410 tx=0x7f6cb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.729+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb00035f0 con 0x7f6cc815aa70 2024-04-05T22:26:47.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.729+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0003750 con 0x7f6cc815aa70 2024-04-05T22:26:47.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.729+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0003b40 con 0x7f6cc815aa70 2024-04-05T22:26:47.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.729+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc8157e80 con 0x7f6cc815aa70 2024-04-05T22:26:47.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.729+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc815f9e0 con 0x7f6cc815aa70 2024-04-05T22:26:47.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.730+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8158680 con 0x7f6cc815aa70 2024-04-05T22:26:47.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.731+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f6cb0003ca0 con 0x7f6cc815aa70 2024-04-05T22:26:47.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.732+0000 7f6cc4c00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 0x7f6ca406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:47.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.732+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6cb0088330 con 0x7f6cc815aa70 2024-04-05T22:26:47.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.733+0000 7f6cc7e00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 0x7f6ca406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:47.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.734+0000 7f6cc7e00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 0x7f6ca406d210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8036000 tx=0x7f6cb803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:47.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.736+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6cc8158680 con 0x7f6cc815aa70 2024-04-05T22:26:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.932+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cc8068160 con 0x7f6cc815aa70 2024-04-05T22:26:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.933+0000 7f6cc4c00640 1 -- 172.21.15.73:0/1959457399 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6cb0053750 con 0x7f6cc815aa70 2024-04-05T22:26:47.933 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 msgr2=0x7f6ca406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 0x7f6ca406d210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8036000 tx=0x7f6cb803c040 comp rx=0 tx=0).stop 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 msgr2=0x7f6cc806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c410 tx=0x7f6cb000c8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 shutdown_connections 2024-04-05T22:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6ca406ad50 0x7f6ca406d210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cc815dd90 0x7f6cc806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cc815aa70 0x7f6cc806ba40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 --2- 172.21.15.73:0/1959457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cc80714a0 0x7f6cc815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 >> 172.21.15.73:0/1959457399 conn(0x7f6cc8069cb0 msgr2=0x7f6cc815d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 shutdown_connections 2024-04-05T22:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:47.936+0000 7f6ccea00640 1 -- 172.21.15.73:0/1959457399 wait complete. 2024-04-05T22:26:47.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:26:48.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.168+0000 7f532b800640 1 Processor -- start 2024-04-05T22:26:48.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.168+0000 7f532b800640 1 -- start start 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f53241541a0 0x7f53240b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f532406d710 0x7f53240cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f532415aef0 0x7f53240cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532415aa60 con 0x7f532406d710 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53240b3b60 con 0x7f53241541a0 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f532b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53240701c0 con 0x7f532415aef0 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f531be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f532406d710 0x7f53240cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f531be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f532406d710 0x7f53240cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42828/0 (socket says 172.21.15.73:42828) 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.170+0000 7f531be00640 1 -- 172.21.15.73:0/1918205754 learned_addr learned my addr 172.21.15.73:0/1918205754 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f5329800640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f532415aef0 0x7f53240cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f5328e00640 1 --2- 172.21.15.73:0/1918205754 >> v2:172.21.15.157:3300/0 conn(0x7f53241541a0 0x7f53240b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f531be00640 1 -- 172.21.15.73:0/1918205754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f532415aef0 msgr2=0x7f53240cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f531be00640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f532415aef0 0x7f53240cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f531be00640 1 -- 172.21.15.73:0/1918205754 >> v2:172.21.15.157:3300/0 conn(0x7f53241541a0 msgr2=0x7f53240b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f531be00640 1 --2- 172.21.15.73:0/1918205754 >> v2:172.21.15.157:3300/0 conn(0x7f53241541a0 0x7f53240b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.171+0000 7f531be00640 1 -- 172.21.15.73:0/1918205754 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240b3880 con 0x7f532406d710 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.172+0000 7f531be00640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532406d710 0x7f53240cd8f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5324066850 tx=0x7f530c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7763f6baaa2a0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.172+0000 7f5323e00640 1 -- 172.21.15.73:0/1918205754 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c031870 con 0x7f532406d710 2024-04-05T22:26:48.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.172+0000 7f5323e00640 1 -- 172.21.15.73:0/1918205754 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c0319d0 con 0x7f532406d710 2024-04-05T22:26:48.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.172+0000 7f5323e00640 1 -- 172.21.15.73:0/1918205754 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c031d60 con 0x7f532406d710 2024-04-05T22:26:48.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 -- 172.21.15.73:0/1918205754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532406d710 msgr2=0x7f53240cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532406d710 0x7f53240cd8f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5324066850 tx=0x7f530c02b910 comp rx=0 tx=0).stop 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 -- 172.21.15.73:0/1918205754 shutdown_connections 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532406d710 0x7f53240cd8f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 --2- 172.21.15.73:0/1918205754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f532415aef0 0x7f53240cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 --2- 172.21.15.73:0/1918205754 >> v2:172.21.15.157:3300/0 conn(0x7f53241541a0 0x7f53240b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 -- 172.21.15.73:0/1918205754 >> 172.21.15.73:0/1918205754 conn(0x7f53240ce8e0 msgr2=0x7f53240cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 -- 172.21.15.73:0/1918205754 shutdown_connections 2024-04-05T22:26:48.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.173+0000 7f532b800640 1 -- 172.21.15.73:0/1918205754 wait complete. 2024-04-05T22:26:48.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.175+0000 7f532b800640 1 Processor -- start 2024-04-05T22:26:48.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.175+0000 7f532b800640 1 -- start start 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 0x7f5324156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 0x7f5324139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5324151420 con 0x7f532415aef0 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53241516c0 con 0x7f532406d710 2024-04-05T22:26:48.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f532b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53241519a0 con 0x7f53241541a0 2024-04-05T22:26:48.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f5329800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f5329800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42842/0 (socket says 172.21.15.73:42842) 2024-04-05T22:26:48.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f5329800640 1 -- 172.21.15.73:0/2910266643 learned_addr learned my addr 172.21.15.73:0/2910266643 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.176+0000 7f5328e00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 0x7f5324156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f531be00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 0x7f5324139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 -- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 msgr2=0x7f5324139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 0x7f5324139180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 -- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 msgr2=0x7f5324156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 0x7f5324156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 -- 172.21.15.73:0/2910266643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0094d0 con 0x7f532415aef0 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5328e00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 0x7f5324156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f531be00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 0x7f5324139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:48.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5329800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f531c007e40 tx=0x7f531c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c014620 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c00ee90 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0149e0 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5324151c80 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.177+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5324152240 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.178+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5324066af0 con 0x7f532415aef0 2024-04-05T22:26:48.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.179+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f531c014b40 con 0x7f532415aef0 2024-04-05T22:26:48.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.180+0000 7f5321600640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 0x7f530006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.180+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f531c089bf0 con 0x7f532415aef0 2024-04-05T22:26:48.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.181+0000 7f5328e00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 0x7f530006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.184+0000 7f5328e00640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 0x7f530006d1e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5314006fd0 tx=0x7f5314008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.184+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f531c054f70 con 0x7f532415aef0 2024-04-05T22:26:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.378+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5324152430 con 0x7f532415aef0 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.379+0000 7f5321600640 1 -- 172.21.15.73:0/2910266643 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f531c058590 con 0x7f532415aef0 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:26:48.379 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:26:48.380 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.381+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 msgr2=0x7f530006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.381+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 0x7f530006d1e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5314006fd0 tx=0x7f5314008040 comp rx=0 tx=0).stop 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.381+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 msgr2=0x7f53241548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.381+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f531c007e40 tx=0x7f531c00e4b0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 shutdown_connections 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f530006ad20 0x7f530006d1e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f532415aef0 0x7f53241548b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53241541a0 0x7f5324139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 --2- 172.21.15.73:0/2910266643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f532406d710 0x7f5324156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 >> 172.21.15.73:0/2910266643 conn(0x7f53240ce8e0 msgr2=0x7f53240cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 shutdown_connections 2024-04-05T22:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.382+0000 7f532b800640 1 -- 172.21.15.73:0/2910266643 wait complete. 2024-04-05T22:26:48.401 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:26:48.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:26:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.616+0000 7f95fea00640 1 Processor -- start 2024-04-05T22:26:48.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.617+0000 7f95fea00640 1 -- start start 2024-04-05T22:26:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f80cf3e0 0x7f95f80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95f80cce90 0x7f95f80b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f80cd2e0 con 0x7f95f815ae50 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80ce2f0 con 0x7f95f80cce90 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815b230 con 0x7f95f80cf3e0 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95fca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95f80cce90 0x7f95f80b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.618+0000 7f95f7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f95f815ae50 0x7f95f80cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37822/0 (socket says 172.21.15.73:37822) 2024-04-05T22:26:48.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95f7e00640 1 -- 172.21.15.73:0/1652366543 learned_addr learned my addr 172.21.15.73:0/1652366543 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95ef400640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f80cf3e0 0x7f95f80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95fca00640 1 -- 172.21.15.73:0/1652366543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f80cf3e0 msgr2=0x7f95f80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95fca00640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f80cf3e0 0x7f95f80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95fca00640 1 -- 172.21.15.73:0/1652366543 >> v1:172.21.15.73:6789/0 conn(0x7f95f815ae50 legacy=0x7f95f80cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95fca00640 1 -- 172.21.15.73:0/1652366543 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f815f4a0 con 0x7f95f80cce90 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.619+0000 7f95f7400640 1 -- 172.21.15.73:0/1652366543 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691381449 0 0) 0x7f95f80cd2e0 con 0x7f95f815ae50 2024-04-05T22:26:48.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.620+0000 7f95fca00640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f80b2e00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd7969f3265eb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.620+0000 7f95f7400640 1 -- 172.21.15.73:0/1652366543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f002cbc0 con 0x7f95f80cce90 2024-04-05T22:26:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.620+0000 7f95f7400640 1 -- 172.21.15.73:0/1652366543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f002cd20 con 0x7f95f80cce90 2024-04-05T22:26:48.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.620+0000 7f95f7400640 1 -- 172.21.15.73:0/1652366543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0034660 con 0x7f95f80cce90 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 -- 172.21.15.73:0/1652366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 msgr2=0x7f95f80b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f80b2e00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f002be10 comp rx=0 tx=0).stop 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 -- 172.21.15.73:0/1652366543 shutdown_connections 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f80cf3e0 0x7f95f80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 --2- 172.21.15.73:0/1652366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f80b2e00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 -- 172.21.15.73:0/1652366543 >> 172.21.15.73:0/1652366543 conn(0x7f95f806a0d0 msgr2=0x7f95f806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 -- 172.21.15.73:0/1652366543 shutdown_connections 2024-04-05T22:26:48.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.621+0000 7f95fea00640 1 -- 172.21.15.73:0/1652366543 wait complete. 2024-04-05T22:26:48.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.623+0000 7f95fea00640 1 Processor -- start 2024-04-05T22:26:48.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.623+0000 7f95fea00640 1 -- start start 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95f80cf3e0 0x7f95f80c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f815ae50 0x7f95f80c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815d7b0 con 0x7f95f80cce90 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815c900 con 0x7f95f80cf3e0 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95fea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815cba0 con 0x7f95f815ae50 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42866/0 (socket says 172.21.15.73:42866) 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.624+0000 7f95f7e00640 1 -- 172.21.15.73:0/517268357 learned_addr learned my addr 172.21.15.73:0/517268357 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95fca00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f815ae50 0x7f95f80c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 -- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f815ae50 msgr2=0x7f95f80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95ef400640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95f80cf3e0 0x7f95f80c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f815ae50 0x7f95f80c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 -- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95f80cf3e0 msgr2=0x7f95f80c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95f80cf3e0 0x7f95f80c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 -- 172.21.15.73:0/517268357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009a70 con 0x7f95f80cce90 2024-04-05T22:26:48.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f7e00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c400 tx=0x7f95e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005ce0 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e8005e40 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007310 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f815ce80 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.625+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f815d440 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.626+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f815d5d0 con 0x7f95f80cce90 2024-04-05T22:26:48.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.627+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f95e8007530 con 0x7f95f80cce90 2024-04-05T22:26:48.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.628+0000 7f95f4c00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 0x7f95d406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:48.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.628+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95e80887a0 con 0x7f95f80cce90 2024-04-05T22:26:48.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.629+0000 7f95ef400640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 0x7f95d406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:48.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.630+0000 7f95ef400640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 0x7f95d406d1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95f80c5950 tx=0x7f95e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:48.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.633+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f95e8053bb0 con 0x7f95f80cce90 2024-04-05T22:26:48.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.826+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f95f8068160 con 0x7f95f80cce90 2024-04-05T22:26:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.939+0000 7f95f4c00640 1 -- 172.21.15.73:0/517268357 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f95e80571d0 con 0x7f95f80cce90 2024-04-05T22:26:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:26:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.944+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 msgr2=0x7f95d406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 0x7f95d406d1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95f80c5950 tx=0x7f95e0008040 comp rx=0 tx=0).stop 2024-04-05T22:26:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 msgr2=0x7f95f815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c400 tx=0x7f95e800c8c0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 shutdown_connections 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95d406ad20 0x7f95d406d1e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95f815ae50 0x7f95f80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95f80cf3e0 0x7f95f80c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 --2- 172.21.15.73:0/517268357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f80cce90 0x7f95f815c3c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 >> 172.21.15.73:0/517268357 conn(0x7f95f806a0d0 msgr2=0x7f95f80ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.945+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 shutdown_connections 2024-04-05T22:26:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:48.946+0000 7f95fea00640 1 -- 172.21.15.73:0/517268357 wait complete. 2024-04-05T22:26:48.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:26:49.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.167+0000 7f49d0200640 1 Processor -- start 2024-04-05T22:26:49.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.167+0000 7f49d0200640 1 -- start start 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49d0200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49c815abe0 0x7f49c80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49d0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c8164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49d0200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c815afc0 con 0x7f49c806c9a0 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49d0200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c806f5b0 con 0x7f49c815abe0 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49d0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c806f890 con 0x7f49c806bc40 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49cd800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49c815abe0 0x7f49c80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49cd800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49c815abe0 0x7f49c80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42872/0 (socket says 172.21.15.73:42872) 2024-04-05T22:26:49.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49cd800640 1 -- 172.21.15.73:0/1318728042 learned_addr learned my addr 172.21.15.73:0/1318728042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:49.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49ce200640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c8164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49cd800640 1 -- 172.21.15.73:0/1318728042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 msgr2=0x7f49c8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:49.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.169+0000 7f49cd800640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c8164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.170+0000 7f49cd800640 1 -- 172.21.15.73:0/1318728042 >> v1:172.21.15.157:6789/0 conn(0x7f49c806c9a0 legacy=0x7f49c806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:49.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.170+0000 7f49cd800640 1 -- 172.21.15.73:0/1318728042 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c806fb70 con 0x7f49c815abe0 2024-04-05T22:26:49.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.170+0000 7f49ce200640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c8164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:49.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.171+0000 7f49cd800640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c815abe0 0x7f49c80712e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009870 tx=0x7f49bc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37371d272a0639c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:49.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.171+0000 7f49c7e00640 1 -- 172.21.15.73:0/1318728042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc031740 con 0x7f49c815abe0 2024-04-05T22:26:49.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.171+0000 7f49c7e00640 1 -- 172.21.15.73:0/1318728042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc0318a0 con 0x7f49c815abe0 2024-04-05T22:26:49.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.171+0000 7f49c7e00640 1 -- 172.21.15.73:0/1318728042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc031c30 con 0x7f49c815abe0 2024-04-05T22:26:49.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 -- 172.21.15.73:0/1318728042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c815abe0 msgr2=0x7f49c80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c815abe0 0x7f49c80712e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009870 tx=0x7f49bc02b7e0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 -- 172.21.15.73:0/1318728042 shutdown_connections 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c8164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 --2- 172.21.15.73:0/1318728042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c815abe0 0x7f49c80712e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 -- 172.21.15.73:0/1318728042 >> 172.21.15.73:0/1318728042 conn(0x7f49c8069cb0 msgr2=0x7f49c806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 -- 172.21.15.73:0/1318728042 shutdown_connections 2024-04-05T22:26:49.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.172+0000 7f49d0200640 1 -- 172.21.15.73:0/1318728042 wait complete. 2024-04-05T22:26:49.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.174+0000 7f49d0200640 1 Processor -- start 2024-04-05T22:26:49.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.174+0000 7f49d0200640 1 -- start start 2024-04-05T22:26:49.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c813a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815abe0 0x7f49c8159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813ae90 con 0x7f49c806c9a0 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813b150 con 0x7f49c815abe0 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49d0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813b430 con 0x7f49c806bc40 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49cce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49cce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42886/0 (socket says 172.21.15.73:42886) 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49cce00640 1 -- 172.21.15.73:0/1136885108 learned_addr learned my addr 172.21.15.73:0/1136885108 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49ce200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815abe0 0x7f49c8159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.175+0000 7f49cd800640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c813a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 -- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 msgr2=0x7f49c813a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c813a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 -- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815abe0 msgr2=0x7f49c8159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:49.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815abe0 0x7f49c8159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 -- 172.21.15.73:0/1136885108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0094d0 con 0x7f49c806c9a0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49cce00640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c430 tx=0x7f49b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011790 con 0x7f49c806c9a0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b40118f0 con 0x7f49c806c9a0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011ce0 con 0x7f49c806c9a0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c813b710 con 0x7f49c806c9a0 2024-04-05T22:26:49.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.176+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c815a110 con 0x7f49c806c9a0 2024-04-05T22:26:49.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.177+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c813bc30 con 0x7f49c806c9a0 2024-04-05T22:26:49.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.179+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f49b401c460 con 0x7f49c806c9a0 2024-04-05T22:26:49.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.180+0000 7f49c5600640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 0x7f49a806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:49.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.180+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f49b4089af0 con 0x7f49c806c9a0 2024-04-05T22:26:49.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.180+0000 7f49cd800640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 0x7f49a806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:49.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.181+0000 7f49cd800640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 0x7f49a806d1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49bc035000 tx=0x7f49bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.184+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f49b4054e80 con 0x7f49c806c9a0 2024-04-05T22:26:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.375+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f49c813bc30 con 0x7f49c806c9a0 2024-04-05T22:26:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.958+0000 7f49c5600640 1 -- 172.21.15.73:0/1136885108 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f49c813bc30 con 0x7f49c806c9a0 2024-04-05T22:26:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 msgr2=0x7f49a806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 0x7f49a806d1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49bc035000 tx=0x7f49bc008030 comp rx=0 tx=0).stop 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 msgr2=0x7f49c8159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c430 tx=0x7f49b400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 shutdown_connections 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f49a806ad20 0x7f49a806d1e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815abe0 0x7f49c8159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49c806c9a0 0x7f49c8159690 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 --2- 172.21.15.73:0/1136885108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c806bc40 0x7f49c813a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.966+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 >> 172.21.15.73:0/1136885108 conn(0x7f49c8069cb0 msgr2=0x7f49c815b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.971+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 shutdown_connections 2024-04-05T22:26:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:49.971+0000 7f49d0200640 1 -- 172.21.15.73:0/1136885108 wait complete. 2024-04-05T22:26:49.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:26:50.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.181+0000 7f93e9400640 1 Processor -- start 2024-04-05T22:26:50.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.181+0000 7f93e9400640 1 -- start start 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e9400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93e4070a40 0x7f93e40add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e9400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e4070e20 con 0x7f93e41599f0 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e9400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40b27f0 con 0x7f93e406cd50 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e9400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40ae240 con 0x7f93e4070a40 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e2800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f93e41599f0 0x7f93e406c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37836/0 (socket says 172.21.15.73:37836) 2024-04-05T22:26:50.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.183+0000 7f93e2800640 1 -- 172.21.15.73:0/2440039280 learned_addr learned my addr 172.21.15.73:0/2440039280 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e3200640 1 --2- 172.21.15.73:0/2440039280 >> v2:172.21.15.157:3300/0 conn(0x7f93e4070a40 0x7f93e40add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328699345 0 0) 0x7f93e4070e20 con 0x7f93e41599f0 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c8003520 con 0x7f93e41599f0 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862326634 0 0) 0x7f93c8003520 con 0x7f93e41599f0 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 >> v2:172.21.15.157:3300/0 conn(0x7f93e4070a40 msgr2=0x7f93e40add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:50.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 --2- 172.21.15.73:0/2440039280 >> v2:172.21.15.157:3300/0 conn(0x7f93e4070a40 0x7f93e40add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 >> v1:172.21.15.157:6789/0 conn(0x7f93e406cd50 legacy=0x7f93e4159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40b2ad0 con 0x7f93e41599f0 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f93d4001670 con 0x7f93e41599f0 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 >> v1:172.21.15.73:6789/0 conn(0x7f93e41599f0 legacy=0x7f93e406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 0x7f93c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c80047c0 0x7f93c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93c8005180 0x7f93c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40ae240 con 0x7f93c8003970 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800c020 con 0x7f93c80047c0 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.184+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800c1e0 con 0x7f93c8005180 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 0x7f93c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 -- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93c8005180 msgr2=0x7f93c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:50.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93c8005180 0x7f93c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 -- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c80047c0 msgr2=0x7f93c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:50.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93d9e00640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c80047c0 0x7f93c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c80047c0 0x7f93c8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e3200640 1 -- 172.21.15.73:0/2440039280 reap_dead start 2024-04-05T22:26:50.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.185+0000 7f93e2800640 1 -- 172.21.15.73:0/2440039280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8003520 con 0x7f93c8003970 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.186+0000 7f93e2800640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 0x7f93c8003d90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c190 tx=0x7f93c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6172a18cd3cf4450 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.186+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4005ad0 con 0x7f93c8003970 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.186+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d4005c30 con 0x7f93c8003970 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.186+0000 7f93e1e00640 1 -- 172.21.15.73:0/2440039280 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d402cab0 con 0x7f93c8003970 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 -- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 msgr2=0x7f93c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 0x7f93c8003d90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c190 tx=0x7f93c80057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 -- 172.21.15.73:0/2440039280 shutdown_connections 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 --2- 172.21.15.73:0/2440039280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c8003970 0x7f93c8003d90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 -- 172.21.15.73:0/2440039280 >> 172.21.15.73:0/2440039280 conn(0x7f93e4069cb0 msgr2=0x7f93e406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:50.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 -- 172.21.15.73:0/2440039280 shutdown_connections 2024-04-05T22:26:50.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.187+0000 7f93e9400640 1 -- 172.21.15.73:0/2440039280 wait complete. 2024-04-05T22:26:50.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.189+0000 7f93e9400640 1 Processor -- start 2024-04-05T22:26:50.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.189+0000 7f93e9400640 1 -- start start 2024-04-05T22:26:50.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c8003970 0x7f93e40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93e413ffe0 0x7f93e4140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4151490 con 0x7f93e41501e0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e414b2e0 con 0x7f93c8003970 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41409a0 con 0x7f93e413ffe0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42916/0 (socket says 172.21.15.73:42916) 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.190+0000 7f93e3200640 1 -- 172.21.15.73:0/3863977992 learned_addr learned my addr 172.21.15.73:0/3863977992 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93d9e00640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93e413ffe0 0x7f93e4140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e2800640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c8003970 0x7f93e40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 -- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93e413ffe0 msgr2=0x7f93e4140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93e413ffe0 0x7f93e4140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 -- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c8003970 msgr2=0x7f93e40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c8003970 0x7f93e40af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 -- 172.21.15.73:0/3863977992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e4150ed0 con 0x7f93e41501e0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e3200640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f93e40ae550 tx=0x7f93dc00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc010040 con 0x7f93e41501e0 2024-04-05T22:26:50.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc00bba0 con 0x7f93e41501e0 2024-04-05T22:26:50.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc014400 con 0x7f93e41501e0 2024-04-05T22:26:50.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41511b0 con 0x7f93e41501e0 2024-04-05T22:26:50.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.191+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4150ba0 con 0x7f93e41501e0 2024-04-05T22:26:50.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.193+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e414ad60 con 0x7f93e41501e0 2024-04-05T22:26:50.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.194+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f93dc014560 con 0x7f93e41501e0 2024-04-05T22:26:50.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.195+0000 7f93db400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 0x7f93c006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.195+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f93dc089ac0 con 0x7f93e41501e0 2024-04-05T22:26:50.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.196+0000 7f93e2800640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 0x7f93c006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.197+0000 7f93e2800640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 0x7f93c006d210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93d4034000 tx=0x7f93d4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.200+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f93e414ad60 con 0x7f93e41501e0 2024-04-05T22:26:50.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.422+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f93e414b420 con 0x7f93e41501e0 2024-04-05T22:26:50.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.433+0000 7f93db400640 1 -- 172.21.15.73:0/3863977992 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f93e414b420 con 0x7f93e41501e0 2024-04-05T22:26:50.434 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 12 and data pool 13 2024-04-05T22:26:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.438+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 msgr2=0x7f93c006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.438+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 0x7f93c006d210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93d4034000 tx=0x7f93d4009030 comp rx=0 tx=0).stop 2024-04-05T22:26:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.438+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 msgr2=0x7f93e4150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.438+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f93e40ae550 tx=0x7f93dc00ef30 comp rx=0 tx=0).stop 2024-04-05T22:26:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 shutdown_connections 2024-04-05T22:26:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93c006ad50 0x7f93c006d210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e41501e0 0x7f93e4150660 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93e413ffe0 0x7f93e4140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 --2- 172.21.15.73:0/3863977992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c8003970 0x7f93e40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.442+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 >> 172.21.15.73:0/3863977992 conn(0x7f93e4069cb0 msgr2=0x7f93e406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.443+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 shutdown_connections 2024-04-05T22:26:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.443+0000 7f93e9400640 1 -- 172.21.15.73:0/3863977992 wait complete. 2024-04-05T22:26:50.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:50.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.649+0000 7feaf3e00640 1 Processor -- start 2024-04-05T22:26:50.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.649+0000 7feaf3e00640 1 -- start start 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaf3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feaf4158860 0x7feaf40b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaf3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feaf415ed30 0x7feaf406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaf3e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf406b500 con 0x7feaf40b1f00 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaf3e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40ccaf0 con 0x7feaf415ed30 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaf3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40b23a0 con 0x7feaf4158860 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaea000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feaf40b1f00 0x7feaf415e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37844/0 (socket says 172.21.15.73:37844) 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.650+0000 7feaea000640 1 -- 172.21.15.73:0/4242794472 learned_addr learned my addr 172.21.15.73:0/4242794472 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf3400640 1 --2- 172.21.15.73:0/4242794472 >> v2:172.21.15.73:3300/0 conn(0x7feaf415ed30 0x7feaf406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf2a00640 1 --2- 172.21.15.73:0/4242794472 >> v2:172.21.15.157:3300/0 conn(0x7feaf4158860 0x7feaf40b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf2000640 1 -- 172.21.15.73:0/4242794472 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339061468 0 0) 0x7feaf406b500 con 0x7feaf40b1f00 2024-04-05T22:26:50.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf2000640 1 -- 172.21.15.73:0/4242794472 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003520 con 0x7feaf40b1f00 2024-04-05T22:26:50.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf3400640 1 -- 172.21.15.73:0/4242794472 >> v2:172.21.15.157:3300/0 conn(0x7feaf4158860 msgr2=0x7feaf40b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf3400640 1 --2- 172.21.15.73:0/4242794472 >> v2:172.21.15.157:3300/0 conn(0x7feaf4158860 0x7feaf40b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf3400640 1 -- 172.21.15.73:0/4242794472 >> v1:172.21.15.73:6789/0 conn(0x7feaf40b1f00 legacy=0x7feaf415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.651+0000 7feaf3400640 1 -- 172.21.15.73:0/4242794472 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40b2680 con 0x7feaf415ed30 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.652+0000 7feaf3400640 1 --2- 172.21.15.73:0/4242794472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf415ed30 0x7feaf406bc70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f380 tx=0x7feaec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea5d8b599e7877 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.652+0000 7feaf2000640 1 -- 172.21.15.73:0/4242794472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec02cbf0 con 0x7feaf415ed30 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf2000640 1 -- 172.21.15.73:0/4242794472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec02cd50 con 0x7feaf415ed30 2024-04-05T22:26:50.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf2000640 1 -- 172.21.15.73:0/4242794472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec034660 con 0x7feaf415ed30 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 -- 172.21.15.73:0/4242794472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf415ed30 msgr2=0x7feaf406bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 --2- 172.21.15.73:0/4242794472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf415ed30 0x7feaf406bc70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f380 tx=0x7feaec02be40 comp rx=0 tx=0).stop 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 -- 172.21.15.73:0/4242794472 shutdown_connections 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 --2- 172.21.15.73:0/4242794472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf415ed30 0x7feaf406bc70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 --2- 172.21.15.73:0/4242794472 >> v2:172.21.15.157:3300/0 conn(0x7feaf4158860 0x7feaf40b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 -- 172.21.15.73:0/4242794472 >> 172.21.15.73:0/4242794472 conn(0x7feaf40b06c0 msgr2=0x7feaf40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 -- 172.21.15.73:0/4242794472 shutdown_connections 2024-04-05T22:26:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.653+0000 7feaf3e00640 1 -- 172.21.15.73:0/4242794472 wait complete. 2024-04-05T22:26:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.655+0000 7feaf3e00640 1 Processor -- start 2024-04-05T22:26:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.655+0000 7feaf3e00640 1 -- start start 2024-04-05T22:26:50.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf415ed30 0x7feaf4157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaf415cb50 0x7feaf40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415d040 con 0x7feaf40b1f00 2024-04-05T22:26:50.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415d320 con 0x7feaf415ed30 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415d600 con 0x7feaf415cb50 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42932/0 (socket says 172.21.15.73:42932) 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.656+0000 7feaf2a00640 1 -- 172.21.15.73:0/2100289060 learned_addr learned my addr 172.21.15.73:0/2100289060 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 -- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaf415cb50 msgr2=0x7feaf40ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaf415cb50 0x7feaf40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 -- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf415ed30 msgr2=0x7feaf4157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf415ed30 0x7feaf4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 -- 172.21.15.73:0/2100289060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009a70 con 0x7feaf40b1f00 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf2a00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feae400d800 tx=0x7feae400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400b6c0 con 0x7feaf40b1f00 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae400b820 con 0x7feaf40b1f00 2024-04-05T22:26:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf415d8e0 con 0x7feaf40b1f00 2024-04-05T22:26:50.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400bbb0 con 0x7feaf40b1f00 2024-04-05T22:26:50.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.657+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4145e20 con 0x7feaf40b1f00 2024-04-05T22:26:50.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.658+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf404e160 con 0x7feaf40b1f00 2024-04-05T22:26:50.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.659+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7feae400bd10 con 0x7feaf40b1f00 2024-04-05T22:26:50.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.660+0000 7feaebe00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 0x7fead006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:50.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.660+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae4089af0 con 0x7feaf40b1f00 2024-04-05T22:26:50.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.661+0000 7feaea000640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 0x7fead006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:50.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.662+0000 7feaea000640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 0x7fead006d1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feaf415a900 tx=0x7feadc007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:50.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.665+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feae4054b20 con 0x7feaf40b1f00 2024-04-05T22:26:50.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.844+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf40b13f0 con 0x7feaf40b1f00 2024-04-05T22:26:50.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.846+0000 7feaebe00640 1 -- 172.21.15.73:0/2100289060 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16105 (secure 0 0 0) 0x7feae4058140 con 0x7feaf40b1f00 2024-04-05T22:26:50.846 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:50.847 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":86,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:50.424855+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 msgr2=0x7fead006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 0x7fead006d1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feaf415a900 tx=0x7feadc007930 comp rx=0 tx=0).stop 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 msgr2=0x7feaf41458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feae400d800 tx=0x7feae400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 shutdown_connections 2024-04-05T22:26:50.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fead006ad20 0x7fead006d1e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaf415cb50 0x7feaf40ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf415ed30 0x7feaf4157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 --2- 172.21.15.73:0/2100289060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf40b1f00 0x7feaf41458e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 >> 172.21.15.73:0/2100289060 conn(0x7feaf40b06c0 msgr2=0x7feaf40ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 shutdown_connections 2024-04-05T22:26:50.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:50.849+0000 7feaf3e00640 1 -- 172.21.15.73:0/2100289060 wait complete. 2024-04-05T22:26:50.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:51.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.053+0000 7fc81e000640 1 Processor -- start 2024-04-05T22:26:51.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.054+0000 7fc81e000640 1 -- start start 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8181587d0 0x7fc8180cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8180ccd20 0x7fc81806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81806d0c0 0x7fc81815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806f5b0 con 0x7fc8180ccd20 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc818070410 con 0x7fc8181587d0 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc81e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818158bb0 con 0x7fc81806d0c0 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc80ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8180ccd20 0x7fc81806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.055+0000 7fc80ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8180ccd20 0x7fc81806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42936/0 (socket says 172.21.15.73:42936) 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 -- 172.21.15.73:0/1537120677 learned_addr learned my addr 172.21.15.73:0/1537120677 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc817e00640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81806d0c0 0x7fc81815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc817400640 1 --2- 172.21.15.73:0/1537120677 >> v2:172.21.15.157:3300/0 conn(0x7fc8181587d0 0x7fc8180cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 -- 172.21.15.73:0/1537120677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81806d0c0 msgr2=0x7fc81815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81806d0c0 0x7fc81815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 -- 172.21.15.73:0/1537120677 >> v2:172.21.15.157:3300/0 conn(0x7fc8181587d0 msgr2=0x7fc8180cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 --2- 172.21.15.73:0/1537120677 >> v2:172.21.15.157:3300/0 conn(0x7fc8181587d0 0x7fc8180cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.056+0000 7fc80ea00640 1 -- 172.21.15.73:0/1537120677 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81806f800 con 0x7fc8180ccd20 2024-04-05T22:26:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.057+0000 7fc80ea00640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8180ccd20 0x7fc81806cb80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc8180667b0 tx=0x7fc80002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d37a8c88d796aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.057+0000 7fc816a00640 1 -- 172.21.15.73:0/1537120677 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800031870 con 0x7fc8180ccd20 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.057+0000 7fc816a00640 1 -- 172.21.15.73:0/1537120677 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000319d0 con 0x7fc8180ccd20 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.057+0000 7fc816a00640 1 -- 172.21.15.73:0/1537120677 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800031d60 con 0x7fc8180ccd20 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 -- 172.21.15.73:0/1537120677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8180ccd20 msgr2=0x7fc81806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8180ccd20 0x7fc81806cb80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc8180667b0 tx=0x7fc80002b910 comp rx=0 tx=0).stop 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 -- 172.21.15.73:0/1537120677 shutdown_connections 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8180ccd20 0x7fc81806cb80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 --2- 172.21.15.73:0/1537120677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81806d0c0 0x7fc81815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 --2- 172.21.15.73:0/1537120677 >> v2:172.21.15.157:3300/0 conn(0x7fc8181587d0 0x7fc8180cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 -- 172.21.15.73:0/1537120677 >> 172.21.15.73:0/1537120677 conn(0x7fc8180ce8e0 msgr2=0x7fc8180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 -- 172.21.15.73:0/1537120677 shutdown_connections 2024-04-05T22:26:51.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.058+0000 7fc81e000640 1 -- 172.21.15.73:0/1537120677 wait complete. 2024-04-05T22:26:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.060+0000 7fc81e000640 1 Processor -- start 2024-04-05T22:26:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.060+0000 7fc81e000640 1 -- start start 2024-04-05T22:26:51.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81806d0c0 0x7fc81814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8180b2b90 0x7fc81814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c40f0 con 0x7fc8181587d0 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c37e0 con 0x7fc81806d0c0 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc81e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180b30d0 con 0x7fc8180b2b90 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42948/0 (socket says 172.21.15.73:42948) 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 -- 172.21.15.73:0/2427551241 learned_addr learned my addr 172.21.15.73:0/2427551241 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc817e00640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8180b2b90 0x7fc81814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc817400640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81806d0c0 0x7fc81814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 -- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8180b2b90 msgr2=0x7fc81814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8180b2b90 0x7fc81814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 -- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81806d0c0 msgr2=0x7fc81814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81806d0c0 0x7fc81814b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.062+0000 7fc80ea00640 1 -- 172.21.15.73:0/2427551241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000094d0 con 0x7fc8181587d0 2024-04-05T22:26:51.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc80ea00640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc80002b950 tx=0x7fc8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80003ad30 con 0x7fc8181587d0 2024-04-05T22:26:51.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000443f0 con 0x7fc8181587d0 2024-04-05T22:26:51.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180c3a30 con 0x7fc8181587d0 2024-04-05T22:26:51.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000447a0 con 0x7fc8181587d0 2024-04-05T22:26:51.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.063+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818157210 con 0x7fc8181587d0 2024-04-05T22:26:51.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.064+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818066af0 con 0x7fc8181587d0 2024-04-05T22:26:51.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.065+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc800044900 con 0x7fc8181587d0 2024-04-05T22:26:51.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.066+0000 7fc80fe00640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 0x7fc7f406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.066+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc80002d040 con 0x7fc8181587d0 2024-04-05T22:26:51.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.067+0000 7fc817400640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 0x7fc7f406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.068+0000 7fc817400640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 0x7fc7f406d1e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc808006fd0 tx=0x7fc808008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.071+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc800076c00 con 0x7fc8181587d0 2024-04-05T22:26:51.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.260+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8180c4230 con 0x7fc8181587d0 2024-04-05T22:26:51.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.262+0000 7fc80fe00640 1 -- 172.21.15.73:0/2427551241 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16105 (secure 0 0 0) 0x7fc8180c4230 con 0x7fc8181587d0 2024-04-05T22:26:51.262 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:51.262 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":86,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:50.424855+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 msgr2=0x7fc7f406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 0x7fc7f406d1e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc808006fd0 tx=0x7fc808008040 comp rx=0 tx=0).stop 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 msgr2=0x7fc8180b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc80002b950 tx=0x7fc8000046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 shutdown_connections 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc7f406ad20 0x7fc7f406d1e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8180b2b90 0x7fc81814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181587d0 0x7fc8180b2650 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 --2- 172.21.15.73:0/2427551241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81806d0c0 0x7fc81814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 >> 172.21.15.73:0/2427551241 conn(0x7fc8180ce8e0 msgr2=0x7fc81815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:51.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 shutdown_connections 2024-04-05T22:26:51.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.265+0000 7fc81e000640 1 -- 172.21.15.73:0/2427551241 wait complete. 2024-04-05T22:26:51.283 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:26:51.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.488+0000 7f2bc2200640 1 Processor -- start 2024-04-05T22:26:51.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.488+0000 7f2bc2200640 1 -- start start 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.489+0000 7f2bc2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2bbc15a850 0x7f2bbc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.489+0000 7f2bc2200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc06bc50 con 0x7f2bbc15e580 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.489+0000 7f2bc2200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc15ef70 con 0x7f2bbc070640 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.489+0000 7f2bc2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc15f530 con 0x7f2bbc15a850 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bbbe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2bbc15e580 0x7f2bbc06ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37850/0 (socket says 172.21.15.73:37850) 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bc0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2bbc15a850 0x7f2bbc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bbbe00640 1 -- 172.21.15.73:0/625080430 learned_addr learned my addr 172.21.15.73:0/625080430 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:51.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bbb400640 1 -- 172.21.15.73:0/625080430 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548503081 0 0) 0x7f2bbc06bc50 con 0x7f2bbc15e580 2024-04-05T22:26:51.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bc0e00640 1 -- 172.21.15.73:0/625080430 >> v1:172.21.15.157:6789/0 conn(0x7f2bbc070640 legacy=0x7f2bbc06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:51.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bc0e00640 1 -- 172.21.15.73:0/625080430 >> v1:172.21.15.73:6789/0 conn(0x7f2bbc15e580 legacy=0x7f2bbc06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.490+0000 7f2bc0e00640 1 -- 172.21.15.73:0/625080430 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc15f250 con 0x7f2bbc15a850 2024-04-05T22:26:51.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.491+0000 7f2bc0e00640 1 --2- 172.21.15.73:0/625080430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc15a850 0x7f2bbc15ac30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00097a0 tx=0x7f2bb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a79d30be0ead5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.491+0000 7f2bbb400640 1 -- 172.21.15.73:0/625080430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb002e070 con 0x7f2bbc15a850 2024-04-05T22:26:51.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.491+0000 7f2bbb400640 1 -- 172.21.15.73:0/625080430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb00315a0 con 0x7f2bbc15a850 2024-04-05T22:26:51.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.491+0000 7f2bbb400640 1 -- 172.21.15.73:0/625080430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0031930 con 0x7f2bbc15a850 2024-04-05T22:26:51.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 -- 172.21.15.73:0/625080430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc15a850 msgr2=0x7f2bbc15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 --2- 172.21.15.73:0/625080430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc15a850 0x7f2bbc15ac30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00097a0 tx=0x7f2bb002b840 comp rx=0 tx=0).stop 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 -- 172.21.15.73:0/625080430 shutdown_connections 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 --2- 172.21.15.73:0/625080430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc15a850 0x7f2bbc15ac30 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 -- 172.21.15.73:0/625080430 >> 172.21.15.73:0/625080430 conn(0x7f2bbc069cb0 msgr2=0x7f2bbc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 -- 172.21.15.73:0/625080430 shutdown_connections 2024-04-05T22:26:51.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.492+0000 7f2bc2200640 1 -- 172.21.15.73:0/625080430 wait complete. 2024-04-05T22:26:51.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.494+0000 7f2bc2200640 1 Processor -- start 2024-04-05T22:26:51.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.494+0000 7f2bc2200640 1 -- start start 2024-04-05T22:26:51.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bbc15a850 0x7f2bbc15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc15e580 0x7f2bbc15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc15c0e0 con 0x7f2bbc070640 2024-04-05T22:26:51.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc15c310 con 0x7f2bbc15e580 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc14c6a0 con 0x7f2bbc15a850 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42960/0 (socket says 172.21.15.73:42960) 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 -- 172.21.15.73:0/2831980293 learned_addr learned my addr 172.21.15.73:0/2831980293 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 -- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bbc15a850 msgr2=0x7f2bbc15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bbc15a850 0x7f2bbc15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 -- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc15e580 msgr2=0x7f2bbc15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.495+0000 7f2bc0e00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc15e580 0x7f2bbc15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bc0e00640 1 -- 172.21.15.73:0/2831980293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0009400 con 0x7f2bbc070640 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bc0e00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0000c00 tx=0x7f2bb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0032900 con 0x7f2bbc070640 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0032a80 con 0x7f2bbc070640 2024-04-05T22:26:51.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0032e30 con 0x7f2bbc070640 2024-04-05T22:26:51.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc14c980 con 0x7f2bbc070640 2024-04-05T22:26:51.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.496+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc14cf40 con 0x7f2bbc070640 2024-04-05T22:26:51.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.497+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc04e160 con 0x7f2bbc070640 2024-04-05T22:26:51.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.498+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f2bb003b3f0 con 0x7f2bbc070640 2024-04-05T22:26:51.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.499+0000 7f2bb8c00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 0x7f2b9c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:51.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.500+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bb00ab710 con 0x7f2bbc070640 2024-04-05T22:26:51.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.500+0000 7f2bbbe00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 0x7f2b9c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:51.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.501+0000 7f2bbbe00640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 0x7f2b9c06d1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8007fd0 tx=0x7f2ba8007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:51.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.504+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2bb00766c0 con 0x7f2bbc070640 2024-04-05T22:26:51.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:51.738+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2bbc0b1f80 con 0x7f2bbc070640 2024-04-05T22:26:52.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.455+0000 7f2bb8c00640 1 -- 172.21.15.73:0/2831980293 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2bb0079ce0 con 0x7f2bbc070640 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 msgr2=0x7f2b9c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 0x7f2b9c06d1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8007fd0 tx=0x7f2ba8007960 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 msgr2=0x7f2bbc14d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0000c00 tx=0x7f2bb00046c0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 shutdown_connections 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b9c06ad20 0x7f2b9c06d1e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bbc15e580 0x7f2bbc15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2bbc15a850 0x7f2bbc15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 --2- 172.21.15.73:0/2831980293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bbc070640 0x7f2bbc14d460 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 >> 172.21.15.73:0/2831980293 conn(0x7f2bbc069cb0 msgr2=0x7f2bbc06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 shutdown_connections 2024-04-05T22:26:52.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.457+0000 7f2bc2200640 1 -- 172.21.15.73:0/2831980293 wait complete. 2024-04-05T22:26:52.479 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:26:52.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:26:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.692+0000 7ff8c1e00640 1 Processor -- start 2024-04-05T22:26:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.692+0000 7ff8c1e00640 1 -- start start 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8c1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8bc15abe0 0x7ff8bc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06bc40 0x7ff8bc164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8c1e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc15afc0 con 0x7ff8bc06c9a0 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8c1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc06f5b0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8c1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc06f890 con 0x7ff8bc06bc40 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8b2000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff8bc06c9a0 0x7ff8bc06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37866/0 (socket says 172.21.15.73:37866) 2024-04-05T22:26:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8b2000640 1 -- 172.21.15.73:0/552599633 learned_addr learned my addr 172.21.15.73:0/552599633 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.694+0000 7ff8baa00640 1 --2- 172.21.15.73:0/552599633 >> v2:172.21.15.73:3300/0 conn(0x7ff8bc15abe0 0x7ff8bc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8bb400640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06bc40 0x7ff8bc164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176154416 0 0) 0x7ff8bc15afc0 con 0x7ff8bc06c9a0 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a0003520 con 0x7ff8bc06c9a0 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8baa00640 1 -- 172.21.15.73:0/552599633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06bc40 msgr2=0x7ff8bc164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8baa00640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06bc40 0x7ff8bc164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8baa00640 1 -- 172.21.15.73:0/552599633 >> v1:172.21.15.73:6789/0 conn(0x7ff8bc06c9a0 legacy=0x7ff8bc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3806489224 0 0) 0x7ff8a0003520 con 0x7ff8bc06c9a0 2024-04-05T22:26:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.695+0000 7ff8baa00640 1 -- 172.21.15.73:0/552599633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc06fb70 con 0x7ff8bc15abe0 2024-04-05T22:26:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.697+0000 7ff8baa00640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc0712e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0097a0 tx=0x7ff8ac02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e9a7b95e60d9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.697+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4001670 con 0x7ff8bc15abe0 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.697+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac02e050 con 0x7ff8bc15abe0 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.697+0000 7ff8ba000640 1 -- 172.21.15.73:0/552599633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac030730 con 0x7ff8bc15abe0 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 -- 172.21.15.73:0/552599633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 msgr2=0x7ff8bc0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc0712e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0097a0 tx=0x7ff8ac02b660 comp rx=0 tx=0).stop 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 -- 172.21.15.73:0/552599633 shutdown_connections 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06bc40 0x7ff8bc164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/552599633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc0712e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 -- 172.21.15.73:0/552599633 >> 172.21.15.73:0/552599633 conn(0x7ff8bc069cb0 msgr2=0x7ff8bc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 -- 172.21.15.73:0/552599633 shutdown_connections 2024-04-05T22:26:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.698+0000 7ff8c1e00640 1 -- 172.21.15.73:0/552599633 wait complete. 2024-04-05T22:26:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.700+0000 7ff8c1e00640 1 Processor -- start 2024-04-05T22:26:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.700+0000 7ff8c1e00640 1 -- start start 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 0x7ff8bc13b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06c9a0 0x7ff8bc13a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc13c310 con 0x7ff8bc15abe0 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc14aa50 con 0x7ff8bc06bc40 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8c1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc13b700 con 0x7ff8bc06c9a0 2024-04-05T22:26:52.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42984/0 (socket says 172.21.15.73:42984) 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 -- 172.21.15.73:0/2061145165 learned_addr learned my addr 172.21.15.73:0/2061145165 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8b2000640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06c9a0 0x7ff8bc13a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8baa00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 0x7ff8bc13b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 -- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06c9a0 msgr2=0x7ff8bc13a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06c9a0 0x7ff8bc13a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 -- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 msgr2=0x7ff8bc13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 0x7ff8bc13b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8bb400640 1 -- 172.21.15.73:0/2061145165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac009400 con 0x7ff8bc15abe0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.701+0000 7ff8baa00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 0x7ff8bc13b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8bb400640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007e40 tx=0x7ff8b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4014620 con 0x7ff8bc15abe0 2024-04-05T22:26:52.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b400ee90 con 0x7ff8bc15abe0 2024-04-05T22:26:52.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40149e0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc13b9e0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.702+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc13bfa0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.703+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0700f0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.704+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7ff8b4014b40 con 0x7ff8bc15abe0 2024-04-05T22:26:52.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.705+0000 7ff8b3e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 0x7ff89806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:52.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.705+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b4089de0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.706+0000 7ff8baa00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 0x7ff89806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.707+0000 7ff8baa00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 0x7ff89806d1e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac034000 tx=0x7ff8ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:52.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.709+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff8b4054eb0 con 0x7ff8bc15abe0 2024-04-05T22:26:52.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:52.923+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff8bc068160 con 0x7ff8bc15abe0 2024-04-05T22:26:53.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.458+0000 7ff8b3e00640 1 -- 172.21.15.73:0/2061145165 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8b40584d0 con 0x7ff8bc15abe0 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 msgr2=0x7ff89806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 0x7ff89806d1e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac034000 tx=0x7ff8ac008030 comp rx=0 tx=0).stop 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 msgr2=0x7ff8bc13c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007e40 tx=0x7ff8b400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 shutdown_connections 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff89806ad20 0x7ff89806d1e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8bc15abe0 0x7ff8bc13c7a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8bc06c9a0 0x7ff8bc13a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 --2- 172.21.15.73:0/2061145165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8bc06bc40 0x7ff8bc13b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.462+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 >> 172.21.15.73:0/2061145165 conn(0x7ff8bc069cb0 msgr2=0x7ff8bc0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:53.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.463+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 shutdown_connections 2024-04-05T22:26:53.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.463+0000 7ff8c1e00640 1 -- 172.21.15.73:0/2061145165 wait complete. 2024-04-05T22:26:53.482 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:26:53.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:53.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.695+0000 7fddf8600640 1 Processor -- start 2024-04-05T22:26:53.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.695+0000 7fddf8600640 1 -- start start 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddf0160620 0x7fddf00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddf00b1870 0x7fddf00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf00ce5d0 con 0x7fddf00b1870 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf00ce010 con 0x7fddf0160620 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.696+0000 7fddf8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00ce2f0 con 0x7fddf00ad6b0 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddf00b1870 0x7fddf00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddf00b1870 0x7fddf00ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42994/0 (socket says 172.21.15.73:42994) 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 -- 172.21.15.73:0/3045708179 learned_addr learned my addr 172.21.15.73:0/3045708179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 -- 172.21.15.73:0/3045708179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 msgr2=0x7fddf00cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:53.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 --2- 172.21.15.73:0/3045708179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 -- 172.21.15.73:0/3045708179 >> v2:172.21.15.157:3300/0 conn(0x7fddf0160620 msgr2=0x7fddf00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:53.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 --2- 172.21.15.73:0/3045708179 >> v2:172.21.15.157:3300/0 conn(0x7fddf0160620 0x7fddf00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.697+0000 7fddf5200640 1 -- 172.21.15.73:0/3045708179 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf00ac970 con 0x7fddf00b1870 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddf5200640 1 --2- 172.21.15.73:0/3045708179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf00b1870 0x7fddf00ad170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fddf00667b0 tx=0x7fdddc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73672988261457ee server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddefe00640 1 -- 172.21.15.73:0/3045708179 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc031860 con 0x7fddf00b1870 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddefe00640 1 -- 172.21.15.73:0/3045708179 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc0319c0 con 0x7fddf00b1870 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddefe00640 1 -- 172.21.15.73:0/3045708179 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc031d50 con 0x7fddf00b1870 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddf8600640 1 -- 172.21.15.73:0/3045708179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf00b1870 msgr2=0x7fddf00ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.699+0000 7fddf8600640 1 --2- 172.21.15.73:0/3045708179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf00b1870 0x7fddf00ad170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fddf00667b0 tx=0x7fdddc02b900 comp rx=0 tx=0).stop 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 -- 172.21.15.73:0/3045708179 shutdown_connections 2024-04-05T22:26:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 --2- 172.21.15.73:0/3045708179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf00b1870 0x7fddf00ad170 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 --2- 172.21.15.73:0/3045708179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf00cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 --2- 172.21.15.73:0/3045708179 >> v2:172.21.15.157:3300/0 conn(0x7fddf0160620 0x7fddf00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 -- 172.21.15.73:0/3045708179 >> 172.21.15.73:0/3045708179 conn(0x7fddf00ce8e0 msgr2=0x7fddf00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 -- 172.21.15.73:0/3045708179 shutdown_connections 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.700+0000 7fddf8600640 1 -- 172.21.15.73:0/3045708179 wait complete. 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.701+0000 7fddf8600640 1 Processor -- start 2024-04-05T22:26:53.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 -- start start 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf0070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf00b1870 0x7fddf015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf015ac90 con 0x7fddf0160620 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c3d50 con 0x7fddf00b1870 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.702+0000 7fddf8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c4030 con 0x7fddf00ad6b0 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:53.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43008/0 (socket says 172.21.15.73:43008) 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 -- 172.21.15.73:0/2160410398 learned_addr learned my addr 172.21.15.73:0/2160410398 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 -- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 msgr2=0x7fddf0070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf0070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 -- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf00b1870 msgr2=0x7fddf015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf00b1870 0x7fddf015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 -- 172.21.15.73:0/2160410398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0094d0 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.703+0000 7fddf6600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdde800b970 tx=0x7fdde800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.704+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800cb40 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.704+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800ccc0 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.704+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8014680 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.704+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf00c4310 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.704+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf00c48d0 con 0x7fddf0160620 2024-04-05T22:26:53.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.705+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf0066af0 con 0x7fddf0160620 2024-04-05T22:26:53.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.706+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fdde801d3b0 con 0x7fddf0160620 2024-04-05T22:26:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.707+0000 7fdded600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 0x7fddd006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.707+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdde8089510 con 0x7fddf0160620 2024-04-05T22:26:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.707+0000 7fddf5c00640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 0x7fddd006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.709+0000 7fddf5c00640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 0x7fddd006d1e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009670 tx=0x7fdde4006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.711+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdde8057fd0 con 0x7fddf0160620 2024-04-05T22:26:53.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.945+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddf00c4ac0 con 0x7fddf0160620 2024-04-05T22:26:53.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.946+0000 7fdded600640 1 -- 172.21.15.73:0/2160410398 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fdde8057b60 con 0x7fddf0160620 2024-04-05T22:26:53.946 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:53.946 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2227587240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2227587240},{"type":"v1","addr":"172.21.15.157:6835","nonce":2227587240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:26:53.946 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 50 2024-04-05T22:26:53.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 msgr2=0x7fddd006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 0x7fddd006d1e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009670 tx=0x7fdde4006bf0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 msgr2=0x7fddf00c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdde800b970 tx=0x7fdde800be30 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 shutdown_connections 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddd006ad20 0x7fddd006d1e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddf0160620 0x7fddf00c7a40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf00b1870 0x7fddf015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 --2- 172.21.15.73:0/2160410398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf00ad6b0 0x7fddf0070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 >> 172.21.15.73:0/2160410398 conn(0x7fddf00ce8e0 msgr2=0x7fddf00cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 shutdown_connections 2024-04-05T22:26:53.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:53.949+0000 7fddf8600640 1 -- 172.21.15.73:0/2160410398 wait complete. 2024-04-05T22:26:53.966 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:26:54.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.172+0000 7f036f200640 1 Processor -- start 2024-04-05T22:26:54.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.172+0000 7f036f200640 1 -- start start 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03681587d0 0x7f03680cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f036815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036806f5b0 con 0x7f03680ccd20 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036815f240 con 0x7f03681587d0 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0368070410 con 0x7f036815e850 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f035fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f03680ccd20 0x7f036806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37872/0 (socket says 172.21.15.73:37872) 2024-04-05T22:26:54.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f035fe00640 1 -- 172.21.15.73:0/715216550 learned_addr learned my addr 172.21.15.73:0/715216550 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237195635 0 0) 0x7f036806f5b0 con 0x7f03680ccd20 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f0367e00640 1 --2- 172.21.15.73:0/715216550 >> v2:172.21.15.157:3300/0 conn(0x7f03681587d0 0x7f03680cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0348003520 con 0x7f03680ccd20 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.174+0000 7f036d200640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f036815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181826586 0 0) 0x7f0348003520 con 0x7f03680ccd20 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 msgr2=0x7f036815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f036815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 >> v2:172.21.15.157:3300/0 conn(0x7f03681587d0 msgr2=0x7f03680cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 --2- 172.21.15.73:0/715216550 >> v2:172.21.15.157:3300/0 conn(0x7f03681587d0 0x7f03680cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368158bb0 con 0x7f03680ccd20 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0350001670 con 0x7f03680ccd20 2024-04-05T22:26:54.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 >> v1:172.21.15.73:6789/0 conn(0x7f03680ccd20 legacy=0x7f036806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f0348003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0348003fa0 0x7f03480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036815f240 con 0x7f0348003fa0 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368070410 con 0x7f0348004d00 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034800ba80 con 0x7f036815e850 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f035fe00640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0348003fa0 0x7f03480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.175+0000 7f036d200640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f035fe00640 1 -- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 msgr2=0x7f0348003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f0367e00640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f0348003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f035fe00640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f036815e850 0x7f0348003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f035fe00640 1 -- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0348004d00 msgr2=0x7f0348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f035fe00640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f035fe00640 1 -- 172.21.15.73:0/715216550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348003520 con 0x7f0348003fa0 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.176+0000 7f036d200640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f035fe00640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0348003fa0 0x7f03480047c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f035000c210 tx=0x7f03480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b8b698ead7dde5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035002cae0 con 0x7f0348003fa0 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035002cc40 con 0x7f0348003fa0 2024-04-05T22:26:54.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f0367400640 1 -- 172.21.15.73:0/715216550 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350005af0 con 0x7f0348003fa0 2024-04-05T22:26:54.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f036f200640 1 -- 172.21.15.73:0/715216550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0348003fa0 msgr2=0x7f03480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f036f200640 1 --2- 172.21.15.73:0/715216550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0348003fa0 0x7f03480047c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f035000c210 tx=0x7f03480057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.177+0000 7f036d200640 1 -- 172.21.15.73:0/715216550 reap_dead start 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.178+0000 7f036f200640 1 -- 172.21.15.73:0/715216550 shutdown_connections 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.178+0000 7f036f200640 1 -- 172.21.15.73:0/715216550 >> 172.21.15.73:0/715216550 conn(0x7f03680ce8e0 msgr2=0x7f03680cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.178+0000 7f036f200640 1 -- 172.21.15.73:0/715216550 shutdown_connections 2024-04-05T22:26:54.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.178+0000 7f036f200640 1 -- 172.21.15.73:0/715216550 wait complete. 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.179+0000 7f036f200640 1 Processor -- start 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 -- start start 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 0x7f03680b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03680af620 0x7f036813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681401c0 con 0x7f0368142dc0 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368143790 con 0x7f03680af620 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.180+0000 7f036f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680ae910 con 0x7f0348003fa0 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43024/0 (socket says 172.21.15.73:43024) 2024-04-05T22:26:54.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 -- 172.21.15.73:0/4131637179 learned_addr learned my addr 172.21.15.73:0/4131637179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f035fe00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03680af620 0x7f036813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f0367e00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 0x7f03680b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 -- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 msgr2=0x7f03680b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 0x7f03680b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 -- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03680af620 msgr2=0x7f036813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03680af620 0x7f036813fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f036d200640 1 -- 172.21.15.73:0/4131637179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680afb00 con 0x7f0368142dc0 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.181+0000 7f0367e00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 0x7f03680b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f036d200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0368070550 tx=0x7f0360009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360015070 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360010280 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360010b60 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680adfe0 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.182+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680ae5a0 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.183+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0368066af0 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.184+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f0360010cc0 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.185+0000 7f0364c00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 0x7f034406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.185+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f036008cc90 con 0x7f0368142dc0 2024-04-05T22:26:54.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.186+0000 7f0367e00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 0x7f034406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.187+0000 7f0367e00640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 0x7f034406d210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f03680a9660 tx=0x7f0358008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.189+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0360057c50 con 0x7f0368142dc0 2024-04-05T22:26:54.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.382+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0368068160 con 0x7f0368142dc0 2024-04-05T22:26:54.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.385+0000 7f0364c00640 1 -- 172.21.15.73:0/4131637179 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16092 (secure 0 0 0) 0x7f036005b270 con 0x7f0368142dc0 2024-04-05T22:26:54.388 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:54.388 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":87,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:26:51.430688+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 msgr2=0x7f034406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 0x7f034406d210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f03680a9660 tx=0x7f0358008040 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 msgr2=0x7f0368143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0368070550 tx=0x7f0360009d40 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 shutdown_connections 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f034406ad50 0x7f034406d210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368142dc0 0x7f0368143250 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03680af620 0x7f036813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 --2- 172.21.15.73:0/4131637179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0348003fa0 0x7f03680b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 >> 172.21.15.73:0/4131637179 conn(0x7f03680ce8e0 msgr2=0x7f0368141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.388+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 shutdown_connections 2024-04-05T22:26:54.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.389+0000 7f036f200640 1 -- 172.21.15.73:0/4131637179 wait complete. 2024-04-05T22:26:54.408 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:26:54.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:26:54.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.622+0000 7f052e000640 1 Processor -- start 2024-04-05T22:26:54.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.622+0000 7f052e000640 1 -- start start 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f052e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 0x7f05280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f052e000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f052815aef0 0x7f05280cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f052e000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052815aa60 con 0x7f05281541a0 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f052e000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05280b3b60 con 0x7f052815aef0 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f052e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05280701c0 con 0x7f052806d710 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0527400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f05281541a0 0x7f05280b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37886/0 (socket says 172.21.15.73:37886) 2024-04-05T22:26:54.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0527400640 1 -- 172.21.15.73:0/2368512298 learned_addr learned my addr 172.21.15.73:0/2368512298 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:54.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0527e00640 1 --2- 172.21.15.73:0/2368512298 >> v2:172.21.15.157:3300/0 conn(0x7f052815aef0 0x7f05280cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0526a00640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 0x7f05280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303172207 0 0) 0x7f052815aa60 con 0x7f05281541a0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.624+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f050c003520 con 0x7f05281541a0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2132602345 0 0) 0x7f050c003520 con 0x7f05281541a0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 msgr2=0x7f05280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 0x7f05280cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 >> v2:172.21.15.157:3300/0 conn(0x7f052815aef0 msgr2=0x7f05280cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 --2- 172.21.15.73:0/2368512298 >> v2:172.21.15.157:3300/0 conn(0x7f052815aef0 0x7f05280cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280b3880 con 0x7f05281541a0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0518001670 con 0x7f05281541a0 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 >> v1:172.21.15.73:6789/0 conn(0x7f05281541a0 legacy=0x7f05280b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f050c003970 0x7f050c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 0x7f050c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c004d00 0x7f050c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280b3b60 con 0x7f050c003970 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280701c0 con 0x7f050c004d00 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00db90 con 0x7f052806d710 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.625+0000 7f0527400640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f050c003970 0x7f050c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.626+0000 7f0527400640 1 -- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 msgr2=0x7f050c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.626+0000 7f0527400640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052806d710 0x7f050c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.626+0000 7f0527400640 1 -- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c004d00 msgr2=0x7f050c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.626+0000 7f0527400640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c004d00 0x7f050c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.626+0000 7f0527400640 1 -- 172.21.15.73:0/2368512298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c003520 con 0x7f050c003970 2024-04-05T22:26:54.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f0527400640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f050c003970 0x7f050c003d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f051800c240 tx=0x7f050c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3857d8734101e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051802cb00 con 0x7f050c003970 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051802cc60 con 0x7f050c003970 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f0526000640 1 -- 172.21.15.73:0/2368512298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518005af0 con 0x7f050c003970 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f052e000640 1 -- 172.21.15.73:0/2368512298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f050c003970 msgr2=0x7f050c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.627+0000 7f052e000640 1 --2- 172.21.15.73:0/2368512298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f050c003970 0x7f050c003d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f051800c240 tx=0x7f050c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.628+0000 7f0527e00640 1 -- 172.21.15.73:0/2368512298 reap_dead start 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.628+0000 7f052e000640 1 -- 172.21.15.73:0/2368512298 shutdown_connections 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.628+0000 7f052e000640 1 -- 172.21.15.73:0/2368512298 >> 172.21.15.73:0/2368512298 conn(0x7f05280ce8e0 msgr2=0x7f05280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.628+0000 7f052e000640 1 -- 172.21.15.73:0/2368512298 shutdown_connections 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.628+0000 7f052e000640 1 -- 172.21.15.73:0/2368512298 wait complete. 2024-04-05T22:26:54.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.629+0000 7f052e000640 1 Processor -- start 2024-04-05T22:26:54.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 -- start start 2024-04-05T22:26:54.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f050c003970 0x7f052814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 0x7f0528069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052814ef10 con 0x7f052806a1f0 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052806a670 con 0x7f052814da40 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.630+0000 7f052e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052814e290 con 0x7f050c003970 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43050/0 (socket says 172.21.15.73:43050) 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 -- 172.21.15.73:0/3166924297 learned_addr learned my addr 172.21.15.73:0/3166924297 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527400640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f050c003970 0x7f052814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0526a00640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 0x7f0528069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 -- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f050c003970 msgr2=0x7f052814d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f050c003970 0x7f052814d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 -- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 msgr2=0x7f0528069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 0x7f0528069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 -- 172.21.15.73:0/3166924297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052814e570 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0526a00640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 0x7f0528069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.631+0000 7f0527e00640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f052815b2d0 tx=0x7f052000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.632+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520010040 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.632+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052000bae0 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.632+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000be70 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.632+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052814e850 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.632+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052814dec0 con 0x7f052806a1f0 2024-04-05T22:26:54.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.633+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0528066af0 con 0x7f052806a1f0 2024-04-05T22:26:54.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.634+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f052001e410 con 0x7f052806a1f0 2024-04-05T22:26:54.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.635+0000 7f051f400640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 0x7f050806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:54.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.635+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0520089840 con 0x7f052806a1f0 2024-04-05T22:26:54.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.635+0000 7f0527400640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 0x7f050806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:54.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.637+0000 7f0527400640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 0x7f050806d210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0518034000 tx=0x7f0518007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:54.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.639+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f05200581f0 con 0x7f052806a1f0 2024-04-05T22:26:54.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.828+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f052815a6f0 con 0x7f050806ad50 2024-04-05T22:26:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.856+0000 7f051f400640 1 -- 172.21.15.73:0/3166924297 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287451 (secure 0 0 0) 0x7f051802c980 con 0x7f050806ad50 2024-04-05T22:26:54.860 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 msgr2=0x7f050806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 0x7f050806d210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0518034000 tx=0x7f0518007000 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 msgr2=0x7f05281ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f052815b2d0 tx=0x7f052000b480 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 shutdown_connections 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050806ad50 0x7f050806d210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052806a1f0 0x7f05281ede50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052814da40 0x7f0528069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 --2- 172.21.15.73:0/3166924297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f050c003970 0x7f052814d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 >> 172.21.15.73:0/3166924297 conn(0x7f05280ce8e0 msgr2=0x7f05280b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:54.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 shutdown_connections 2024-04-05T22:26:54.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:54.863+0000 7f052e000640 1 -- 172.21.15.73:0/3166924297 wait complete. 2024-04-05T22:26:54.880 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-05T22:26:53.505215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258524,"kb_used_data":9532,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754716196,"statfs":{"total":773094113280,"available":772829384704,"internally_reserved":0,"allocated":9760768,"data_stored":4840056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.567176"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449170+0000","last_change":"2024-04-05T22:26:50.476629+0000","last_active":"2024-04-05T22:26:51.449170+0000","last_peered":"2024-04-05T22:26:51.449170+0000","last_clean":"2024-04-05T22:26:51.449170+0000","last_became_active":"2024-04-05T22:26:50.474717+0000","last_became_peered":"2024-04-05T22:26:50.474717+0000","last_unstale":"2024-04-05T22:26:51.449170+0000","last_undegraded":"2024-04-05T22:26:51.449170+0000","last_fullsized":"2024-04-05T22:26:51.449170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:45:43.915282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448564+0000","last_change":"2024-04-05T22:26:50.469282+0000","last_active":"2024-04-05T22:26:51.448564+0000","last_peered":"2024-04-05T22:26:51.448564+0000","last_clean":"2024-04-05T22:26:51.448564+0000","last_became_active":"2024-04-05T22:26:50.469040+0000","last_became_peered":"2024-04-05T22:26:50.469040+0000","last_unstale":"2024-04-05T22:26:51.448564+0000","last_undegraded":"2024-04-05T22:26:51.448564+0000","last_fullsized":"2024-04-05T22:26:51.448564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:37.376824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670392+0000","last_change":"2024-04-05T22:26:50.468244+0000","last_active":"2024-04-05T22:26:51.670392+0000","last_peered":"2024-04-05T22:26:51.670392+0000","last_clean":"2024-04-05T22:26:51.670392+0000","last_became_active":"2024-04-05T22:26:50.467992+0000","last_became_peered":"2024-04-05T22:26:50.467992+0000","last_unstale":"2024-04-05T22:26:51.670392+0000","last_undegraded":"2024-04-05T22:26:51.670392+0000","last_fullsized":"2024-04-05T22:26:51.670392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445295+0000","last_change":"2024-04-05T22:26:50.445496+0000","last_active":"2024-04-05T22:26:51.445295+0000","last_peered":"2024-04-05T22:26:51.445295+0000","last_clean":"2024-04-05T22:26:51.445295+0000","last_became_active":"2024-04-05T22:26:50.441637+0000","last_became_peered":"2024-04-05T22:26:50.441637+0000","last_unstale":"2024-04-05T22:26:51.445295+0000","last_undegraded":"2024-04-05T22:26:51.445295+0000","last_fullsized":"2024-04-05T22:26:51.445295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:09.720274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669833+0000","last_change":"2024-04-05T22:26:50.446938+0000","last_active":"2024-04-05T22:26:51.669833+0000","last_peered":"2024-04-05T22:26:51.669833+0000","last_clean":"2024-04-05T22:26:51.669833+0000","last_became_active":"2024-04-05T22:26:50.443504+0000","last_became_peered":"2024-04-05T22:26:50.443504+0000","last_unstale":"2024-04-05T22:26:51.669833+0000","last_undegraded":"2024-04-05T22:26:51.669833+0000","last_fullsized":"2024-04-05T22:26:51.669833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:52.078681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542519+0000","last_change":"2024-04-05T22:26:50.470078+0000","last_active":"2024-04-05T22:26:51.542519+0000","last_peered":"2024-04-05T22:26:51.542519+0000","last_clean":"2024-04-05T22:26:51.542519+0000","last_became_active":"2024-04-05T22:26:50.467067+0000","last_became_peered":"2024-04-05T22:26:50.467067+0000","last_unstale":"2024-04-05T22:26:51.542519+0000","last_undegraded":"2024-04-05T22:26:51.542519+0000","last_fullsized":"2024-04-05T22:26:51.542519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444775+0000","last_change":"2024-04-05T22:26:50.465526+0000","last_active":"2024-04-05T22:26:51.444775+0000","last_peered":"2024-04-05T22:26:51.444775+0000","last_clean":"2024-04-05T22:26:51.444775+0000","last_became_active":"2024-04-05T22:26:50.465185+0000","last_became_peered":"2024-04-05T22:26:50.465185+0000","last_unstale":"2024-04-05T22:26:51.444775+0000","last_undegraded":"2024-04-05T22:26:51.444775+0000","last_fullsized":"2024-04-05T22:26:51.444775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:37.781661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542166+0000","last_change":"2024-04-05T22:26:50.473175+0000","last_active":"2024-04-05T22:26:51.542166+0000","last_peered":"2024-04-05T22:26:51.542166+0000","last_clean":"2024-04-05T22:26:51.542166+0000","last_became_active":"2024-04-05T22:26:50.471020+0000","last_became_peered":"2024-04-05T22:26:50.471020+0000","last_unstale":"2024-04-05T22:26:51.542166+0000","last_undegraded":"2024-04-05T22:26:51.542166+0000","last_fullsized":"2024-04-05T22:26:51.542166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:14:01.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542648+0000","last_change":"2024-04-05T22:26:50.464379+0000","last_active":"2024-04-05T22:26:51.542648+0000","last_peered":"2024-04-05T22:26:51.542648+0000","last_clean":"2024-04-05T22:26:51.542648+0000","last_became_active":"2024-04-05T22:26:50.464052+0000","last_became_peered":"2024-04-05T22:26:50.464052+0000","last_unstale":"2024-04-05T22:26:51.542648+0000","last_undegraded":"2024-04-05T22:26:51.542648+0000","last_fullsized":"2024-04-05T22:26:51.542648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:03:13.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449017+0000","last_change":"2024-04-05T22:26:50.468565+0000","last_active":"2024-04-05T22:26:51.449017+0000","last_peered":"2024-04-05T22:26:51.449017+0000","last_clean":"2024-04-05T22:26:51.449017+0000","last_became_active":"2024-04-05T22:26:50.468164+0000","last_became_peered":"2024-04-05T22:26:50.468164+0000","last_unstale":"2024-04-05T22:26:51.449017+0000","last_undegraded":"2024-04-05T22:26:51.449017+0000","last_fullsized":"2024-04-05T22:26:51.449017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:47:31.128655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447860+0000","last_change":"2024-04-05T22:26:50.464116+0000","last_active":"2024-04-05T22:26:51.447860+0000","last_peered":"2024-04-05T22:26:51.447860+0000","last_clean":"2024-04-05T22:26:51.447860+0000","last_became_active":"2024-04-05T22:26:50.463823+0000","last_became_peered":"2024-04-05T22:26:50.463823+0000","last_unstale":"2024-04-05T22:26:51.447860+0000","last_undegraded":"2024-04-05T22:26:51.447860+0000","last_fullsized":"2024-04-05T22:26:51.447860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:59:38.707931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447993+0000","last_change":"2024-04-05T22:26:50.467702+0000","last_active":"2024-04-05T22:26:51.447993+0000","last_peered":"2024-04-05T22:26:51.447993+0000","last_clean":"2024-04-05T22:26:51.447993+0000","last_became_active":"2024-04-05T22:26:50.466998+0000","last_became_peered":"2024-04-05T22:26:50.466998+0000","last_unstale":"2024-04-05T22:26:51.447993+0000","last_undegraded":"2024-04-05T22:26:51.447993+0000","last_fullsized":"2024-04-05T22:26:51.447993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:10.669899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669140+0000","last_change":"2024-04-05T22:26:50.468702+0000","last_active":"2024-04-05T22:26:51.669140+0000","last_peered":"2024-04-05T22:26:51.669140+0000","last_clean":"2024-04-05T22:26:51.669140+0000","last_became_active":"2024-04-05T22:26:50.465428+0000","last_became_peered":"2024-04-05T22:26:50.465428+0000","last_unstale":"2024-04-05T22:26:51.669140+0000","last_undegraded":"2024-04-05T22:26:51.669140+0000","last_fullsized":"2024-04-05T22:26:51.669140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:52.078681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444886+0000","last_change":"2024-04-05T22:26:50.444087+0000","last_active":"2024-04-05T22:26:51.444886+0000","last_peered":"2024-04-05T22:26:51.444886+0000","last_clean":"2024-04-05T22:26:51.444886+0000","last_became_active":"2024-04-05T22:26:50.443460+0000","last_became_peered":"2024-04-05T22:26:50.443460+0000","last_unstale":"2024-04-05T22:26:51.444886+0000","last_undegraded":"2024-04-05T22:26:51.444886+0000","last_fullsized":"2024-04-05T22:26:51.444886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:29.267191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668172+0000","last_change":"2024-04-05T22:26:50.471110+0000","last_active":"2024-04-05T22:26:51.668172+0000","last_peered":"2024-04-05T22:26:51.668172+0000","last_clean":"2024-04-05T22:26:51.668172+0000","last_became_active":"2024-04-05T22:26:50.470787+0000","last_became_peered":"2024-04-05T22:26:50.470787+0000","last_unstale":"2024-04-05T22:26:51.668172+0000","last_undegraded":"2024-04-05T22:26:51.668172+0000","last_fullsized":"2024-04-05T22:26:51.668172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:47.570328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669331+0000","last_change":"2024-04-05T22:26:50.475371+0000","last_active":"2024-04-05T22:26:51.669331+0000","last_peered":"2024-04-05T22:26:51.669331+0000","last_clean":"2024-04-05T22:26:51.669331+0000","last_became_active":"2024-04-05T22:26:50.474817+0000","last_became_peered":"2024-04-05T22:26:50.474817+0000","last_unstale":"2024-04-05T22:26:51.669331+0000","last_undegraded":"2024-04-05T22:26:51.669331+0000","last_fullsized":"2024-04-05T22:26:51.669331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:38.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670065+0000","last_change":"2024-04-05T22:26:50.452125+0000","last_active":"2024-04-05T22:26:51.670065+0000","last_peered":"2024-04-05T22:26:51.670065+0000","last_clean":"2024-04-05T22:26:51.670065+0000","last_became_active":"2024-04-05T22:26:50.445701+0000","last_became_peered":"2024-04-05T22:26:50.445701+0000","last_unstale":"2024-04-05T22:26:51.670065+0000","last_undegraded":"2024-04-05T22:26:51.670065+0000","last_fullsized":"2024-04-05T22:26:51.670065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:33.206553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445589+0000","last_change":"2024-04-05T22:26:50.469089+0000","last_active":"2024-04-05T22:26:51.445589+0000","last_peered":"2024-04-05T22:26:51.445589+0000","last_clean":"2024-04-05T22:26:51.445589+0000","last_became_active":"2024-04-05T22:26:50.468813+0000","last_became_peered":"2024-04-05T22:26:50.468813+0000","last_unstale":"2024-04-05T22:26:51.445589+0000","last_undegraded":"2024-04-05T22:26:51.445589+0000","last_fullsized":"2024-04-05T22:26:51.445589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:49.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671016+0000","last_change":"2024-04-05T22:26:50.449995+0000","last_active":"2024-04-05T22:26:51.671016+0000","last_peered":"2024-04-05T22:26:51.671016+0000","last_clean":"2024-04-05T22:26:51.671016+0000","last_became_active":"2024-04-05T22:26:50.449116+0000","last_became_peered":"2024-04-05T22:26:50.449116+0000","last_unstale":"2024-04-05T22:26:51.671016+0000","last_undegraded":"2024-04-05T22:26:51.671016+0000","last_fullsized":"2024-04-05T22:26:51.671016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:15.920151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445194+0000","last_change":"2024-04-05T22:26:50.446837+0000","last_active":"2024-04-05T22:26:51.445194+0000","last_peered":"2024-04-05T22:26:51.445194+0000","last_clean":"2024-04-05T22:26:51.445194+0000","last_became_active":"2024-04-05T22:26:50.446588+0000","last_became_peered":"2024-04-05T22:26:50.446588+0000","last_unstale":"2024-04-05T22:26:51.445194+0000","last_undegraded":"2024-04-05T22:26:51.445194+0000","last_fullsized":"2024-04-05T22:26:51.445194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:32:13.186227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445313+0000","last_change":"2024-04-05T22:26:50.451460+0000","last_active":"2024-04-05T22:26:51.445313+0000","last_peered":"2024-04-05T22:26:51.445313+0000","last_clean":"2024-04-05T22:26:51.445313+0000","last_became_active":"2024-04-05T22:26:50.451208+0000","last_became_peered":"2024-04-05T22:26:50.451208+0000","last_unstale":"2024-04-05T22:26:51.445313+0000","last_undegraded":"2024-04-05T22:26:51.445313+0000","last_fullsized":"2024-04-05T22:26:51.445313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:07.310789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447727+0000","last_change":"2024-04-05T22:26:50.450227+0000","last_active":"2024-04-05T22:26:51.447727+0000","last_peered":"2024-04-05T22:26:51.447727+0000","last_clean":"2024-04-05T22:26:51.447727+0000","last_became_active":"2024-04-05T22:26:50.449958+0000","last_became_peered":"2024-04-05T22:26:50.449958+0000","last_unstale":"2024-04-05T22:26:51.447727+0000","last_undegraded":"2024-04-05T22:26:51.447727+0000","last_fullsized":"2024-04-05T22:26:51.447727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:14:01.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448117+0000","last_change":"2024-04-05T22:26:50.469917+0000","last_active":"2024-04-05T22:26:51.448117+0000","last_peered":"2024-04-05T22:26:51.448117+0000","last_clean":"2024-04-05T22:26:51.448117+0000","last_became_active":"2024-04-05T22:26:50.469437+0000","last_became_peered":"2024-04-05T22:26:50.469437+0000","last_unstale":"2024-04-05T22:26:51.448117+0000","last_undegraded":"2024-04-05T22:26:51.448117+0000","last_fullsized":"2024-04-05T22:26:51.448117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:04.878802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444439+0000","last_change":"2024-04-05T22:26:50.444209+0000","last_active":"2024-04-05T22:26:51.444439+0000","last_peered":"2024-04-05T22:26:51.444439+0000","last_clean":"2024-04-05T22:26:51.444439+0000","last_became_active":"2024-04-05T22:26:50.443958+0000","last_became_peered":"2024-04-05T22:26:50.443958+0000","last_unstale":"2024-04-05T22:26:51.444439+0000","last_undegraded":"2024-04-05T22:26:51.444439+0000","last_fullsized":"2024-04-05T22:26:51.444439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:12:23.375572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447971+0000","last_change":"2024-04-05T22:26:50.468403+0000","last_active":"2024-04-05T22:26:51.447971+0000","last_peered":"2024-04-05T22:26:51.447971+0000","last_clean":"2024-04-05T22:26:51.447971+0000","last_became_active":"2024-04-05T22:26:50.468129+0000","last_became_peered":"2024-04-05T22:26:50.468129+0000","last_unstale":"2024-04-05T22:26:51.447971+0000","last_undegraded":"2024-04-05T22:26:51.447971+0000","last_fullsized":"2024-04-05T22:26:51.447971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:55.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670198+0000","last_change":"2024-04-05T22:26:50.464439+0000","last_active":"2024-04-05T22:26:51.670198+0000","last_peered":"2024-04-05T22:26:51.670198+0000","last_clean":"2024-04-05T22:26:51.670198+0000","last_became_active":"2024-04-05T22:26:50.464139+0000","last_became_peered":"2024-04-05T22:26:50.464139+0000","last_unstale":"2024-04-05T22:26:51.670198+0000","last_undegraded":"2024-04-05T22:26:51.670198+0000","last_fullsized":"2024-04-05T22:26:51.670198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:38.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670043+0000","last_change":"2024-04-05T22:26:50.461152+0000","last_active":"2024-04-05T22:26:51.670043+0000","last_peered":"2024-04-05T22:26:51.670043+0000","last_clean":"2024-04-05T22:26:51.670043+0000","last_became_active":"2024-04-05T22:26:50.460942+0000","last_became_peered":"2024-04-05T22:26:50.460942+0000","last_unstale":"2024-04-05T22:26:51.670043+0000","last_undegraded":"2024-04-05T22:26:51.670043+0000","last_fullsized":"2024-04-05T22:26:51.670043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448448+0000","last_change":"2024-04-05T22:26:50.474498+0000","last_active":"2024-04-05T22:26:51.448448+0000","last_peered":"2024-04-05T22:26:51.448448+0000","last_clean":"2024-04-05T22:26:51.448448+0000","last_became_active":"2024-04-05T22:26:50.474219+0000","last_became_peered":"2024-04-05T22:26:50.474219+0000","last_unstale":"2024-04-05T22:26:51.448448+0000","last_undegraded":"2024-04-05T22:26:51.448448+0000","last_fullsized":"2024-04-05T22:26:51.448448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:44:18.909071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448107+0000","last_change":"2024-04-05T22:26:50.449009+0000","last_active":"2024-04-05T22:26:51.448107+0000","last_peered":"2024-04-05T22:26:51.448107+0000","last_clean":"2024-04-05T22:26:51.448107+0000","last_became_active":"2024-04-05T22:26:50.448734+0000","last_became_peered":"2024-04-05T22:26:50.448734+0000","last_unstale":"2024-04-05T22:26:51.448107+0000","last_undegraded":"2024-04-05T22:26:51.448107+0000","last_fullsized":"2024-04-05T22:26:51.448107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:55:54.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447849+0000","last_change":"2024-04-05T22:26:50.447939+0000","last_active":"2024-04-05T22:26:51.447849+0000","last_peered":"2024-04-05T22:26:51.447849+0000","last_clean":"2024-04-05T22:26:51.447849+0000","last_became_active":"2024-04-05T22:26:50.447664+0000","last_became_peered":"2024-04-05T22:26:50.447664+0000","last_unstale":"2024-04-05T22:26:51.447849+0000","last_undegraded":"2024-04-05T22:26:51.447849+0000","last_fullsized":"2024-04-05T22:26:51.447849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:03.818840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669804+0000","last_change":"2024-04-05T22:26:50.445584+0000","last_active":"2024-04-05T22:26:51.669804+0000","last_peered":"2024-04-05T22:26:51.669804+0000","last_clean":"2024-04-05T22:26:51.669804+0000","last_became_active":"2024-04-05T22:26:50.443977+0000","last_became_peered":"2024-04-05T22:26:50.443977+0000","last_unstale":"2024-04-05T22:26:51.669804+0000","last_undegraded":"2024-04-05T22:26:51.669804+0000","last_fullsized":"2024-04-05T22:26:51.669804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447901+0000","last_change":"2024-04-05T22:26:50.463355+0000","last_active":"2024-04-05T22:26:51.447901+0000","last_peered":"2024-04-05T22:26:51.447901+0000","last_clean":"2024-04-05T22:26:51.447901+0000","last_became_active":"2024-04-05T22:26:50.463093+0000","last_became_peered":"2024-04-05T22:26:50.463093+0000","last_unstale":"2024-04-05T22:26:51.447901+0000","last_undegraded":"2024-04-05T22:26:51.447901+0000","last_fullsized":"2024-04-05T22:26:51.447901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:25:45.237590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669797+0000","last_change":"2024-04-05T22:26:50.471056+0000","last_active":"2024-04-05T22:26:51.669797+0000","last_peered":"2024-04-05T22:26:51.669797+0000","last_clean":"2024-04-05T22:26:51.669797+0000","last_became_active":"2024-04-05T22:26:50.468922+0000","last_became_peered":"2024-04-05T22:26:50.468922+0000","last_unstale":"2024-04-05T22:26:51.669797+0000","last_undegraded":"2024-04-05T22:26:51.669797+0000","last_fullsized":"2024-04-05T22:26:51.669797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:07.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449390+0000","last_change":"2024-04-05T22:26:49.974129+0000","last_active":"2024-04-05T22:26:51.449390+0000","last_peered":"2024-04-05T22:26:51.449390+0000","last_clean":"2024-04-05T22:26:51.449390+0000","last_became_active":"2024-04-05T22:26:49.970785+0000","last_became_peered":"2024-04-05T22:26:49.970785+0000","last_unstale":"2024-04-05T22:26:51.449390+0000","last_undegraded":"2024-04-05T22:26:51.449390+0000","last_fullsized":"2024-04-05T22:26:51.449390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:23:59.162101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449252+0000","last_change":"2024-04-05T22:26:50.471535+0000","last_active":"2024-04-05T22:26:51.449252+0000","last_peered":"2024-04-05T22:26:51.449252+0000","last_clean":"2024-04-05T22:26:51.449252+0000","last_became_active":"2024-04-05T22:26:50.468202+0000","last_became_peered":"2024-04-05T22:26:50.468202+0000","last_unstale":"2024-04-05T22:26:51.449252+0000","last_undegraded":"2024-04-05T22:26:51.449252+0000","last_fullsized":"2024-04-05T22:26:51.449252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:45.948664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670689+0000","last_change":"2024-04-05T22:26:49.990804+0000","last_active":"2024-04-05T22:26:51.670689+0000","last_peered":"2024-04-05T22:26:51.670689+0000","last_clean":"2024-04-05T22:26:51.670689+0000","last_became_active":"2024-04-05T22:26:49.989935+0000","last_became_peered":"2024-04-05T22:26:49.989935+0000","last_unstale":"2024-04-05T22:26:51.670689+0000","last_undegraded":"2024-04-05T22:26:51.670689+0000","last_fullsized":"2024-04-05T22:26:51.670689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.801248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448567+0000","last_change":"2024-04-05T22:26:50.467508+0000","last_active":"2024-04-05T22:26:51.448567+0000","last_peered":"2024-04-05T22:26:51.448567+0000","last_clean":"2024-04-05T22:26:51.448567+0000","last_became_active":"2024-04-05T22:26:50.466631+0000","last_became_peered":"2024-04-05T22:26:50.466631+0000","last_unstale":"2024-04-05T22:26:51.448567+0000","last_undegraded":"2024-04-05T22:26:51.448567+0000","last_fullsized":"2024-04-05T22:26:51.448567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:55.136198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671012+0000","last_change":"2024-04-05T22:26:49.978114+0000","last_active":"2024-04-05T22:26:51.671012+0000","last_peered":"2024-04-05T22:26:51.671012+0000","last_clean":"2024-04-05T22:26:51.671012+0000","last_became_active":"2024-04-05T22:26:49.977857+0000","last_became_peered":"2024-04-05T22:26:49.977857+0000","last_unstale":"2024-04-05T22:26:51.671012+0000","last_undegraded":"2024-04-05T22:26:51.671012+0000","last_fullsized":"2024-04-05T22:26:51.671012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:03.227249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670631+0000","last_change":"2024-04-05T22:26:50.467444+0000","last_active":"2024-04-05T22:26:51.670631+0000","last_peered":"2024-04-05T22:26:51.670631+0000","last_clean":"2024-04-05T22:26:51.670631+0000","last_became_active":"2024-04-05T22:26:50.465607+0000","last_became_peered":"2024-04-05T22:26:50.465607+0000","last_unstale":"2024-04-05T22:26:51.670631+0000","last_undegraded":"2024-04-05T22:26:51.670631+0000","last_fullsized":"2024-04-05T22:26:51.670631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:03:13.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670724+0000","last_change":"2024-04-05T22:26:49.979137+0000","last_active":"2024-04-05T22:26:51.670724+0000","last_peered":"2024-04-05T22:26:51.670724+0000","last_clean":"2024-04-05T22:26:51.670724+0000","last_became_active":"2024-04-05T22:26:49.978895+0000","last_became_peered":"2024-04-05T22:26:49.978895+0000","last_unstale":"2024-04-05T22:26:51.670724+0000","last_undegraded":"2024-04-05T22:26:51.670724+0000","last_fullsized":"2024-04-05T22:26:51.670724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:37.302152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448125+0000","last_change":"2024-04-05T22:26:50.447907+0000","last_active":"2024-04-05T22:26:51.448125+0000","last_peered":"2024-04-05T22:26:51.448125+0000","last_clean":"2024-04-05T22:26:51.448125+0000","last_became_active":"2024-04-05T22:26:50.447519+0000","last_became_peered":"2024-04-05T22:26:50.447519+0000","last_unstale":"2024-04-05T22:26:51.448125+0000","last_undegraded":"2024-04-05T22:26:51.448125+0000","last_fullsized":"2024-04-05T22:26:51.448125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:00.147854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671202+0000","last_change":"2024-04-05T22:26:49.992180+0000","last_active":"2024-04-05T22:26:51.671202+0000","last_peered":"2024-04-05T22:26:51.671202+0000","last_clean":"2024-04-05T22:26:51.671202+0000","last_became_active":"2024-04-05T22:26:49.991456+0000","last_became_peered":"2024-04-05T22:26:49.991456+0000","last_unstale":"2024-04-05T22:26:51.671202+0000","last_undegraded":"2024-04-05T22:26:51.671202+0000","last_fullsized":"2024-04-05T22:26:51.671202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:57:13.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671060+0000","last_change":"2024-04-05T22:26:50.450153+0000","last_active":"2024-04-05T22:26:51.671060+0000","last_peered":"2024-04-05T22:26:51.671060+0000","last_clean":"2024-04-05T22:26:51.671060+0000","last_became_active":"2024-04-05T22:26:50.449403+0000","last_became_peered":"2024-04-05T22:26:50.449403+0000","last_unstale":"2024-04-05T22:26:51.671060+0000","last_undegraded":"2024-04-05T22:26:51.671060+0000","last_fullsized":"2024-04-05T22:26:51.671060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669608+0000","last_change":"2024-04-05T22:26:49.989374+0000","last_active":"2024-04-05T22:26:51.669608+0000","last_peered":"2024-04-05T22:26:51.669608+0000","last_clean":"2024-04-05T22:26:51.669608+0000","last_became_active":"2024-04-05T22:26:49.989105+0000","last_became_peered":"2024-04-05T22:26:49.989105+0000","last_unstale":"2024-04-05T22:26:51.669608+0000","last_undegraded":"2024-04-05T22:26:51.669608+0000","last_fullsized":"2024-04-05T22:26:51.669608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:02:59.501337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668866+0000","last_change":"2024-04-05T22:26:50.463960+0000","last_active":"2024-04-05T22:26:51.668866+0000","last_peered":"2024-04-05T22:26:51.668866+0000","last_clean":"2024-04-05T22:26:51.668866+0000","last_became_active":"2024-04-05T22:26:50.463648+0000","last_became_peered":"2024-04-05T22:26:50.463648+0000","last_unstale":"2024-04-05T22:26:51.668866+0000","last_undegraded":"2024-04-05T22:26:51.668866+0000","last_fullsized":"2024-04-05T22:26:51.668866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670068+0000","last_change":"2024-04-05T22:26:49.988141+0000","last_active":"2024-04-05T22:26:51.670068+0000","last_peered":"2024-04-05T22:26:51.670068+0000","last_clean":"2024-04-05T22:26:51.670068+0000","last_became_active":"2024-04-05T22:26:49.987827+0000","last_became_peered":"2024-04-05T22:26:49.987827+0000","last_unstale":"2024-04-05T22:26:51.670068+0000","last_undegraded":"2024-04-05T22:26:51.670068+0000","last_fullsized":"2024-04-05T22:26:51.670068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:06.729896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668219+0000","last_change":"2024-04-05T22:26:50.465723+0000","last_active":"2024-04-05T22:26:51.668219+0000","last_peered":"2024-04-05T22:26:51.668219+0000","last_clean":"2024-04-05T22:26:51.668219+0000","last_became_active":"2024-04-05T22:26:50.465451+0000","last_became_peered":"2024-04-05T22:26:50.465451+0000","last_unstale":"2024-04-05T22:26:51.668219+0000","last_undegraded":"2024-04-05T22:26:51.668219+0000","last_fullsized":"2024-04-05T22:26:51.668219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:42.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447197+0000","last_change":"2024-04-05T22:26:49.991508+0000","last_active":"2024-04-05T22:26:51.447197+0000","last_peered":"2024-04-05T22:26:51.447197+0000","last_clean":"2024-04-05T22:26:51.447197+0000","last_became_active":"2024-04-05T22:26:49.991230+0000","last_became_peered":"2024-04-05T22:26:49.991230+0000","last_unstale":"2024-04-05T22:26:51.447197+0000","last_undegraded":"2024-04-05T22:26:51.447197+0000","last_fullsized":"2024-04-05T22:26:51.447197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:42.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669525+0000","last_change":"2024-04-05T22:26:50.446398+0000","last_active":"2024-04-05T22:26:51.669525+0000","last_peered":"2024-04-05T22:26:51.669525+0000","last_clean":"2024-04-05T22:26:51.669525+0000","last_became_active":"2024-04-05T22:26:50.446108+0000","last_became_peered":"2024-04-05T22:26:50.446108+0000","last_unstale":"2024-04-05T22:26:51.669525+0000","last_undegraded":"2024-04-05T22:26:51.669525+0000","last_fullsized":"2024-04-05T22:26:51.669525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:26.072253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449420+0000","last_change":"2024-04-05T22:26:49.982818+0000","last_active":"2024-04-05T22:26:51.449420+0000","last_peered":"2024-04-05T22:26:51.449420+0000","last_clean":"2024-04-05T22:26:51.449420+0000","last_became_active":"2024-04-05T22:26:49.978059+0000","last_became_peered":"2024-04-05T22:26:49.978059+0000","last_unstale":"2024-04-05T22:26:51.449420+0000","last_undegraded":"2024-04-05T22:26:51.449420+0000","last_fullsized":"2024-04-05T22:26:51.449420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:40.758748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542535+0000","last_change":"2024-04-05T22:26:50.472098+0000","last_active":"2024-04-05T22:26:51.542535+0000","last_peered":"2024-04-05T22:26:51.542535+0000","last_clean":"2024-04-05T22:26:51.542535+0000","last_became_active":"2024-04-05T22:26:50.471796+0000","last_became_peered":"2024-04-05T22:26:50.471796+0000","last_unstale":"2024-04-05T22:26:51.542535+0000","last_undegraded":"2024-04-05T22:26:51.542535+0000","last_fullsized":"2024-04-05T22:26:51.542535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449079+0000","last_change":"2024-04-05T22:26:49.990638+0000","last_active":"2024-04-05T22:26:51.449079+0000","last_peered":"2024-04-05T22:26:51.449079+0000","last_clean":"2024-04-05T22:26:51.449079+0000","last_became_active":"2024-04-05T22:26:49.990152+0000","last_became_peered":"2024-04-05T22:26:49.990152+0000","last_unstale":"2024-04-05T22:26:51.449079+0000","last_undegraded":"2024-04-05T22:26:51.449079+0000","last_fullsized":"2024-04-05T22:26:51.449079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:23.965031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670462+0000","last_change":"2024-04-05T22:26:50.472069+0000","last_active":"2024-04-05T22:26:51.670462+0000","last_peered":"2024-04-05T22:26:51.670462+0000","last_clean":"2024-04-05T22:26:51.670462+0000","last_became_active":"2024-04-05T22:26:50.471829+0000","last_became_peered":"2024-04-05T22:26:50.471829+0000","last_unstale":"2024-04-05T22:26:51.670462+0000","last_undegraded":"2024-04-05T22:26:51.670462+0000","last_fullsized":"2024-04-05T22:26:51.670462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:02:20.024821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448676+0000","last_change":"2024-04-05T22:26:49.979113+0000","last_active":"2024-04-05T22:26:51.448676+0000","last_peered":"2024-04-05T22:26:51.448676+0000","last_clean":"2024-04-05T22:26:51.448676+0000","last_became_active":"2024-04-05T22:26:49.978597+0000","last_became_peered":"2024-04-05T22:26:49.978597+0000","last_unstale":"2024-04-05T22:26:51.448676+0000","last_undegraded":"2024-04-05T22:26:51.448676+0000","last_fullsized":"2024-04-05T22:26:51.448676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:52.064458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670296+0000","last_change":"2024-04-05T22:26:50.457478+0000","last_active":"2024-04-05T22:26:51.670296+0000","last_peered":"2024-04-05T22:26:51.670296+0000","last_clean":"2024-04-05T22:26:51.670296+0000","last_became_active":"2024-04-05T22:26:50.457188+0000","last_became_peered":"2024-04-05T22:26:50.457188+0000","last_unstale":"2024-04-05T22:26:51.670296+0000","last_undegraded":"2024-04-05T22:26:51.670296+0000","last_fullsized":"2024-04-05T22:26:51.670296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:09:52.041376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542784+0000","last_change":"2024-04-05T22:26:49.990358+0000","last_active":"2024-04-05T22:26:51.542784+0000","last_peered":"2024-04-05T22:26:51.542784+0000","last_clean":"2024-04-05T22:26:51.542784+0000","last_became_active":"2024-04-05T22:26:49.988603+0000","last_became_peered":"2024-04-05T22:26:49.988603+0000","last_unstale":"2024-04-05T22:26:51.542784+0000","last_undegraded":"2024-04-05T22:26:51.542784+0000","last_fullsized":"2024-04-05T22:26:51.542784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:54.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670167+0000","last_change":"2024-04-05T22:26:50.458580+0000","last_active":"2024-04-05T22:26:51.670167+0000","last_peered":"2024-04-05T22:26:51.670167+0000","last_clean":"2024-04-05T22:26:51.670167+0000","last_became_active":"2024-04-05T22:26:50.458266+0000","last_became_peered":"2024-04-05T22:26:50.458266+0000","last_unstale":"2024-04-05T22:26:51.670167+0000","last_undegraded":"2024-04-05T22:26:51.670167+0000","last_fullsized":"2024-04-05T22:26:51.670167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:04.246740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448221+0000","last_change":"2024-04-05T22:26:49.979872+0000","last_active":"2024-04-05T22:26:51.448221+0000","last_peered":"2024-04-05T22:26:51.448221+0000","last_clean":"2024-04-05T22:26:51.448221+0000","last_became_active":"2024-04-05T22:26:49.979599+0000","last_became_peered":"2024-04-05T22:26:49.979599+0000","last_unstale":"2024-04-05T22:26:51.448221+0000","last_undegraded":"2024-04-05T22:26:51.448221+0000","last_fullsized":"2024-04-05T22:26:51.448221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:39.665785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669260+0000","last_change":"2024-04-05T22:26:50.469359+0000","last_active":"2024-04-05T22:26:51.669260+0000","last_peered":"2024-04-05T22:26:51.669260+0000","last_clean":"2024-04-05T22:26:51.669260+0000","last_became_active":"2024-04-05T22:26:50.469115+0000","last_became_peered":"2024-04-05T22:26:50.469115+0000","last_unstale":"2024-04-05T22:26:51.669260+0000","last_undegraded":"2024-04-05T22:26:51.669260+0000","last_fullsized":"2024-04-05T22:26:51.669260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668900+0000","last_change":"2024-04-05T22:26:49.978360+0000","last_active":"2024-04-05T22:26:51.668900+0000","last_peered":"2024-04-05T22:26:51.668900+0000","last_clean":"2024-04-05T22:26:51.668900+0000","last_became_active":"2024-04-05T22:26:49.978079+0000","last_became_peered":"2024-04-05T22:26:49.978079+0000","last_unstale":"2024-04-05T22:26:51.668900+0000","last_undegraded":"2024-04-05T22:26:51.668900+0000","last_fullsized":"2024-04-05T22:26:51.668900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:28.225378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447115+0000","last_change":"2024-04-05T22:26:50.445207+0000","last_active":"2024-04-05T22:26:51.447115+0000","last_peered":"2024-04-05T22:26:51.447115+0000","last_clean":"2024-04-05T22:26:51.447115+0000","last_became_active":"2024-04-05T22:26:50.444925+0000","last_became_peered":"2024-04-05T22:26:50.444925+0000","last_unstale":"2024-04-05T22:26:51.447115+0000","last_undegraded":"2024-04-05T22:26:51.447115+0000","last_fullsized":"2024-04-05T22:26:51.447115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:08:15.805465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542315+0000","last_change":"2024-04-05T22:26:49.991225+0000","last_active":"2024-04-05T22:26:51.542315+0000","last_peered":"2024-04-05T22:26:51.542315+0000","last_clean":"2024-04-05T22:26:51.542315+0000","last_became_active":"2024-04-05T22:26:49.990894+0000","last_became_peered":"2024-04-05T22:26:49.990894+0000","last_unstale":"2024-04-05T22:26:51.542315+0000","last_undegraded":"2024-04-05T22:26:51.542315+0000","last_fullsized":"2024-04-05T22:26:51.542315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:13.465943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447042+0000","last_change":"2024-04-05T22:26:50.477395+0000","last_active":"2024-04-05T22:26:51.447042+0000","last_peered":"2024-04-05T22:26:51.447042+0000","last_clean":"2024-04-05T22:26:51.447042+0000","last_became_active":"2024-04-05T22:26:50.476951+0000","last_became_peered":"2024-04-05T22:26:50.476951+0000","last_unstale":"2024-04-05T22:26:51.447042+0000","last_undegraded":"2024-04-05T22:26:51.447042+0000","last_fullsized":"2024-04-05T22:26:51.447042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:14.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669695+0000","last_change":"2024-04-05T22:26:49.984930+0000","last_active":"2024-04-05T22:26:51.669695+0000","last_peered":"2024-04-05T22:26:51.669695+0000","last_clean":"2024-04-05T22:26:51.669695+0000","last_became_active":"2024-04-05T22:26:49.984491+0000","last_became_peered":"2024-04-05T22:26:49.984491+0000","last_unstale":"2024-04-05T22:26:51.669695+0000","last_undegraded":"2024-04-05T22:26:51.669695+0000","last_fullsized":"2024-04-05T22:26:51.669695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:44.827747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669489+0000","last_change":"2024-04-05T22:26:50.466432+0000","last_active":"2024-04-05T22:26:51.669489+0000","last_peered":"2024-04-05T22:26:51.669489+0000","last_clean":"2024-04-05T22:26:51.669489+0000","last_became_active":"2024-04-05T22:26:50.466106+0000","last_became_peered":"2024-04-05T22:26:50.466106+0000","last_unstale":"2024-04-05T22:26:51.669489+0000","last_undegraded":"2024-04-05T22:26:51.669489+0000","last_fullsized":"2024-04-05T22:26:51.669489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:07.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449553+0000","last_change":"2024-04-05T22:26:49.987839+0000","last_active":"2024-04-05T22:26:51.449553+0000","last_peered":"2024-04-05T22:26:51.449553+0000","last_clean":"2024-04-05T22:26:51.449553+0000","last_became_active":"2024-04-05T22:26:49.987573+0000","last_became_peered":"2024-04-05T22:26:49.987573+0000","last_unstale":"2024-04-05T22:26:51.449553+0000","last_undegraded":"2024-04-05T22:26:51.449553+0000","last_fullsized":"2024-04-05T22:26:51.449553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:09.178326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449305+0000","last_change":"2024-04-05T22:26:50.471349+0000","last_active":"2024-04-05T22:26:51.449305+0000","last_peered":"2024-04-05T22:26:51.449305+0000","last_clean":"2024-04-05T22:26:51.449305+0000","last_became_active":"2024-04-05T22:26:50.467942+0000","last_became_peered":"2024-04-05T22:26:50.467942+0000","last_unstale":"2024-04-05T22:26:51.449305+0000","last_undegraded":"2024-04-05T22:26:51.449305+0000","last_fullsized":"2024-04-05T22:26:51.449305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:21:58.606106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670859+0000","last_change":"2024-04-05T22:26:49.986587+0000","last_active":"2024-04-05T22:26:51.670859+0000","last_peered":"2024-04-05T22:26:51.670859+0000","last_clean":"2024-04-05T22:26:51.670859+0000","last_became_active":"2024-04-05T22:26:49.986247+0000","last_became_peered":"2024-04-05T22:26:49.986247+0000","last_unstale":"2024-04-05T22:26:51.670859+0000","last_undegraded":"2024-04-05T22:26:51.670859+0000","last_fullsized":"2024-04-05T22:26:51.670859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:38.116846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670545+0000","last_change":"2024-04-05T22:26:50.450125+0000","last_active":"2024-04-05T22:26:51.670545+0000","last_peered":"2024-04-05T22:26:51.670545+0000","last_clean":"2024-04-05T22:26:51.670545+0000","last_became_active":"2024-04-05T22:26:50.447366+0000","last_became_peered":"2024-04-05T22:26:50.447366+0000","last_unstale":"2024-04-05T22:26:51.670545+0000","last_undegraded":"2024-04-05T22:26:51.670545+0000","last_fullsized":"2024-04-05T22:26:51.670545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670744+0000","last_change":"2024-04-05T22:26:49.987077+0000","last_active":"2024-04-05T22:26:51.670744+0000","last_peered":"2024-04-05T22:26:51.670744+0000","last_clean":"2024-04-05T22:26:51.670744+0000","last_became_active":"2024-04-05T22:26:49.986796+0000","last_became_peered":"2024-04-05T22:26:49.986796+0000","last_unstale":"2024-04-05T22:26:51.670744+0000","last_undegraded":"2024-04-05T22:26:51.670744+0000","last_fullsized":"2024-04-05T22:26:51.670744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:51.073153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444979+0000","last_change":"2024-04-05T22:26:50.444320+0000","last_active":"2024-04-05T22:26:51.444979+0000","last_peered":"2024-04-05T22:26:51.444979+0000","last_clean":"2024-04-05T22:26:51.444979+0000","last_became_active":"2024-04-05T22:26:50.443799+0000","last_became_peered":"2024-04-05T22:26:50.443799+0000","last_unstale":"2024-04-05T22:26:51.444979+0000","last_undegraded":"2024-04-05T22:26:51.444979+0000","last_fullsized":"2024-04-05T22:26:51.444979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:40.671313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668775+0000","last_change":"2024-04-05T22:26:49.976556+0000","last_active":"2024-04-05T22:26:51.668775+0000","last_peered":"2024-04-05T22:26:51.668775+0000","last_clean":"2024-04-05T22:26:51.668775+0000","last_became_active":"2024-04-05T22:26:49.976246+0000","last_became_peered":"2024-04-05T22:26:49.976246+0000","last_unstale":"2024-04-05T22:26:51.668775+0000","last_undegraded":"2024-04-05T22:26:51.668775+0000","last_fullsized":"2024-04-05T22:26:51.668775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:15:48.875016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542206+0000","last_change":"2024-04-05T22:26:50.465488+0000","last_active":"2024-04-05T22:26:51.542206+0000","last_peered":"2024-04-05T22:26:51.542206+0000","last_clean":"2024-04-05T22:26:51.542206+0000","last_became_active":"2024-04-05T22:26:50.465245+0000","last_became_peered":"2024-04-05T22:26:50.465245+0000","last_unstale":"2024-04-05T22:26:51.542206+0000","last_undegraded":"2024-04-05T22:26:51.542206+0000","last_fullsized":"2024-04-05T22:26:51.542206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444976+0000","last_change":"2024-04-05T22:26:49.980432+0000","last_active":"2024-04-05T22:26:51.444976+0000","last_peered":"2024-04-05T22:26:51.444976+0000","last_clean":"2024-04-05T22:26:51.444976+0000","last_became_active":"2024-04-05T22:26:49.980084+0000","last_became_peered":"2024-04-05T22:26:49.980084+0000","last_unstale":"2024-04-05T22:26:51.444976+0000","last_undegraded":"2024-04-05T22:26:51.444976+0000","last_fullsized":"2024-04-05T22:26:51.444976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:13:12.954092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444537+0000","last_change":"2024-04-05T22:26:50.471761+0000","last_active":"2024-04-05T22:26:51.444537+0000","last_peered":"2024-04-05T22:26:51.444537+0000","last_clean":"2024-04-05T22:26:51.444537+0000","last_became_active":"2024-04-05T22:26:50.470938+0000","last_became_peered":"2024-04-05T22:26:50.470938+0000","last_unstale":"2024-04-05T22:26:51.444537+0000","last_undegraded":"2024-04-05T22:26:51.444537+0000","last_fullsized":"2024-04-05T22:26:51.444537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:57.806776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444865+0000","last_change":"2024-04-05T22:26:49.982431+0000","last_active":"2024-04-05T22:26:51.444865+0000","last_peered":"2024-04-05T22:26:51.444865+0000","last_clean":"2024-04-05T22:26:51.444865+0000","last_became_active":"2024-04-05T22:26:49.982096+0000","last_became_peered":"2024-04-05T22:26:49.982096+0000","last_unstale":"2024-04-05T22:26:51.444865+0000","last_undegraded":"2024-04-05T22:26:51.444865+0000","last_fullsized":"2024-04-05T22:26:51.444865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:30:33.435365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542151+0000","last_change":"2024-04-05T22:26:50.463449+0000","last_active":"2024-04-05T22:26:51.542151+0000","last_peered":"2024-04-05T22:26:51.542151+0000","last_clean":"2024-04-05T22:26:51.542151+0000","last_became_active":"2024-04-05T22:26:50.463178+0000","last_became_peered":"2024-04-05T22:26:50.463178+0000","last_unstale":"2024-04-05T22:26:51.542151+0000","last_undegraded":"2024-04-05T22:26:51.542151+0000","last_fullsized":"2024-04-05T22:26:51.542151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668184+0000","last_change":"2024-04-05T22:26:49.976365+0000","last_active":"2024-04-05T22:26:51.668184+0000","last_peered":"2024-04-05T22:26:51.668184+0000","last_clean":"2024-04-05T22:26:51.668184+0000","last_became_active":"2024-04-05T22:26:49.975241+0000","last_became_peered":"2024-04-05T22:26:49.975241+0000","last_unstale":"2024-04-05T22:26:51.668184+0000","last_undegraded":"2024-04-05T22:26:51.668184+0000","last_fullsized":"2024-04-05T22:26:51.668184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:32.043965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670074+0000","last_change":"2024-04-05T22:26:50.471326+0000","last_active":"2024-04-05T22:26:51.670074+0000","last_peered":"2024-04-05T22:26:51.670074+0000","last_clean":"2024-04-05T22:26:51.670074+0000","last_became_active":"2024-04-05T22:26:50.471086+0000","last_became_peered":"2024-04-05T22:26:50.471086+0000","last_unstale":"2024-04-05T22:26:51.670074+0000","last_undegraded":"2024-04-05T22:26:51.670074+0000","last_fullsized":"2024-04-05T22:26:51.670074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:04.232777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448084+0000","last_change":"2024-04-05T22:26:49.984673+0000","last_active":"2024-04-05T22:26:51.448084+0000","last_peered":"2024-04-05T22:26:51.448084+0000","last_clean":"2024-04-05T22:26:51.448084+0000","last_became_active":"2024-04-05T22:26:49.984418+0000","last_became_peered":"2024-04-05T22:26:49.984418+0000","last_unstale":"2024-04-05T22:26:51.448084+0000","last_undegraded":"2024-04-05T22:26:51.448084+0000","last_fullsized":"2024-04-05T22:26:51.448084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:20:10.386093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671196+0000","last_change":"2024-04-05T22:26:50.475387+0000","last_active":"2024-04-05T22:26:51.671196+0000","last_peered":"2024-04-05T22:26:51.671196+0000","last_clean":"2024-04-05T22:26:51.671196+0000","last_became_active":"2024-04-05T22:26:50.475128+0000","last_became_peered":"2024-04-05T22:26:50.475128+0000","last_unstale":"2024-04-05T22:26:51.671196+0000","last_undegraded":"2024-04-05T22:26:51.671196+0000","last_fullsized":"2024-04-05T22:26:51.671196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448499+0000","last_change":"2024-04-05T22:26:49.978251+0000","last_active":"2024-04-05T22:26:51.448499+0000","last_peered":"2024-04-05T22:26:51.448499+0000","last_clean":"2024-04-05T22:26:51.448499+0000","last_became_active":"2024-04-05T22:26:49.977947+0000","last_became_peered":"2024-04-05T22:26:49.977947+0000","last_unstale":"2024-04-05T22:26:51.448499+0000","last_undegraded":"2024-04-05T22:26:51.448499+0000","last_fullsized":"2024-04-05T22:26:51.448499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:07:56.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449055+0000","last_change":"2024-04-05T22:26:50.467334+0000","last_active":"2024-04-05T22:26:51.449055+0000","last_peered":"2024-04-05T22:26:51.449055+0000","last_clean":"2024-04-05T22:26:51.449055+0000","last_became_active":"2024-04-05T22:26:50.466362+0000","last_became_peered":"2024-04-05T22:26:50.466362+0000","last_unstale":"2024-04-05T22:26:51.449055+0000","last_undegraded":"2024-04-05T22:26:51.449055+0000","last_fullsized":"2024-04-05T22:26:51.449055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:05.166659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448933+0000","last_change":"2024-04-05T22:26:49.972712+0000","last_active":"2024-04-05T22:26:51.448933+0000","last_peered":"2024-04-05T22:26:51.448933+0000","last_clean":"2024-04-05T22:26:51.448933+0000","last_became_active":"2024-04-05T22:26:49.970681+0000","last_became_peered":"2024-04-05T22:26:49.970681+0000","last_unstale":"2024-04-05T22:26:51.448933+0000","last_undegraded":"2024-04-05T22:26:51.448933+0000","last_fullsized":"2024-04-05T22:26:51.448933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:11:34.505583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670813+0000","last_change":"2024-04-05T22:26:50.453314+0000","last_active":"2024-04-05T22:26:51.670813+0000","last_peered":"2024-04-05T22:26:51.670813+0000","last_clean":"2024-04-05T22:26:51.670813+0000","last_became_active":"2024-04-05T22:26:50.451034+0000","last_became_peered":"2024-04-05T22:26:50.451034+0000","last_unstale":"2024-04-05T22:26:51.670813+0000","last_undegraded":"2024-04-05T22:26:51.670813+0000","last_fullsized":"2024-04-05T22:26:51.670813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:45.863406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448847+0000","last_change":"2024-04-05T22:26:49.977248+0000","last_active":"2024-04-05T22:26:51.448847+0000","last_peered":"2024-04-05T22:26:51.448847+0000","last_clean":"2024-04-05T22:26:51.448847+0000","last_became_active":"2024-04-05T22:26:49.976563+0000","last_became_peered":"2024-04-05T22:26:49.976563+0000","last_unstale":"2024-04-05T22:26:51.448847+0000","last_undegraded":"2024-04-05T22:26:51.448847+0000","last_fullsized":"2024-04-05T22:26:51.448847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:52.607533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542212+0000","last_change":"2024-04-05T22:26:50.472474+0000","last_active":"2024-04-05T22:26:51.542212+0000","last_peered":"2024-04-05T22:26:51.542212+0000","last_clean":"2024-04-05T22:26:51.542212+0000","last_became_active":"2024-04-05T22:26:50.471793+0000","last_became_peered":"2024-04-05T22:26:50.471793+0000","last_unstale":"2024-04-05T22:26:51.542212+0000","last_undegraded":"2024-04-05T22:26:51.542212+0000","last_fullsized":"2024-04-05T22:26:51.542212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:09:52.041376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670386+0000","last_change":"2024-04-05T22:26:49.987189+0000","last_active":"2024-04-05T22:26:51.670386+0000","last_peered":"2024-04-05T22:26:51.670386+0000","last_clean":"2024-04-05T22:26:51.670386+0000","last_became_active":"2024-04-05T22:26:49.986926+0000","last_became_peered":"2024-04-05T22:26:49.986926+0000","last_unstale":"2024-04-05T22:26:51.670386+0000","last_undegraded":"2024-04-05T22:26:51.670386+0000","last_fullsized":"2024-04-05T22:26:51.670386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:46.945379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670290+0000","last_change":"2024-04-05T22:26:50.463207+0000","last_active":"2024-04-05T22:26:51.670290+0000","last_peered":"2024-04-05T22:26:51.670290+0000","last_clean":"2024-04-05T22:26:51.670290+0000","last_became_active":"2024-04-05T22:26:50.462935+0000","last_became_peered":"2024-04-05T22:26:50.462935+0000","last_unstale":"2024-04-05T22:26:51.670290+0000","last_undegraded":"2024-04-05T22:26:51.670290+0000","last_fullsized":"2024-04-05T22:26:51.670290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:18:30.236918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669679+0000","last_change":"2024-04-05T22:26:49.985572+0000","last_active":"2024-04-05T22:26:51.669679+0000","last_peered":"2024-04-05T22:26:51.669679+0000","last_clean":"2024-04-05T22:26:51.669679+0000","last_became_active":"2024-04-05T22:26:49.985295+0000","last_became_peered":"2024-04-05T22:26:49.985295+0000","last_unstale":"2024-04-05T22:26:51.669679+0000","last_undegraded":"2024-04-05T22:26:51.669679+0000","last_fullsized":"2024-04-05T22:26:51.669679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:12:22.370044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669039+0000","last_change":"2024-04-05T22:26:50.467165+0000","last_active":"2024-04-05T22:26:51.669039+0000","last_peered":"2024-04-05T22:26:51.669039+0000","last_clean":"2024-04-05T22:26:51.669039+0000","last_became_active":"2024-04-05T22:26:50.466895+0000","last_became_peered":"2024-04-05T22:26:50.466895+0000","last_unstale":"2024-04-05T22:26:51.669039+0000","last_undegraded":"2024-04-05T22:26:51.669039+0000","last_fullsized":"2024-04-05T22:26:51.669039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:04.232777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448299+0000","last_change":"2024-04-05T22:26:49.975710+0000","last_active":"2024-04-05T22:26:51.448299+0000","last_peered":"2024-04-05T22:26:51.448299+0000","last_clean":"2024-04-05T22:26:51.448299+0000","last_became_active":"2024-04-05T22:26:49.974070+0000","last_became_peered":"2024-04-05T22:26:49.974070+0000","last_unstale":"2024-04-05T22:26:51.448299+0000","last_undegraded":"2024-04-05T22:26:51.448299+0000","last_fullsized":"2024-04-05T22:26:51.448299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:00.737675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448115+0000","last_change":"2024-04-05T22:26:50.469692+0000","last_active":"2024-04-05T22:26:51.448115+0000","last_peered":"2024-04-05T22:26:51.448115+0000","last_clean":"2024-04-05T22:26:51.448115+0000","last_became_active":"2024-04-05T22:26:50.469163+0000","last_became_peered":"2024-04-05T22:26:50.469163+0000","last_unstale":"2024-04-05T22:26:51.448115+0000","last_undegraded":"2024-04-05T22:26:51.448115+0000","last_fullsized":"2024-04-05T22:26:51.448115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:31:01.941910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669650+0000","last_change":"2024-04-05T22:26:49.991763+0000","last_active":"2024-04-05T22:26:51.669650+0000","last_peered":"2024-04-05T22:26:51.669650+0000","last_clean":"2024-04-05T22:26:51.669650+0000","last_became_active":"2024-04-05T22:26:49.991472+0000","last_became_peered":"2024-04-05T22:26:49.991472+0000","last_unstale":"2024-04-05T22:26:51.669650+0000","last_undegraded":"2024-04-05T22:26:51.669650+0000","last_fullsized":"2024-04-05T22:26:51.669650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:14.320056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447767+0000","last_change":"2024-04-05T22:26:50.472266+0000","last_active":"2024-04-05T22:26:51.447767+0000","last_peered":"2024-04-05T22:26:51.447767+0000","last_clean":"2024-04-05T22:26:51.447767+0000","last_became_active":"2024-04-05T22:26:50.471033+0000","last_became_peered":"2024-04-05T22:26:50.471033+0000","last_unstale":"2024-04-05T22:26:51.447767+0000","last_undegraded":"2024-04-05T22:26:51.447767+0000","last_fullsized":"2024-04-05T22:26:51.447767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:01.424253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670900+0000","last_change":"2024-04-05T22:26:49.985636+0000","last_active":"2024-04-05T22:26:51.670900+0000","last_peered":"2024-04-05T22:26:51.670900+0000","last_clean":"2024-04-05T22:26:51.670900+0000","last_became_active":"2024-04-05T22:26:49.985358+0000","last_became_peered":"2024-04-05T22:26:49.985358+0000","last_unstale":"2024-04-05T22:26:51.670900+0000","last_undegraded":"2024-04-05T22:26:51.670900+0000","last_fullsized":"2024-04-05T22:26:51.670900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:13.465943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":257,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-05T22:26:50.449786+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:26:50.449786+0000","last_peered":"2024-04-05T22:26:50.449786+0000","last_clean":"2024-04-05T22:26:50.449786+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:26:50.449786+0000","last_undegraded":"2024-04-05T22:26:50.449786+0000","last_fullsized":"2024-04-05T22:26:50.449786+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1662976,"data_stored":1048050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0580000000000001,"15min":1.03},"min":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.54,"5min":1.796,"15min":1.796},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0640000000000001,"15min":1.012},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5129999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1419999999999999,"15min":1.052},"min":{"1min":0.748,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.48,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.0860000000000001,"15min":1.0629999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1819999999999999,"15min":1.1739999999999999},"min":{"1min":0.71699999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.262,"15min":1.218},"min":{"1min":0.84899999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1839999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.204,"15min":1.1379999999999999},"min":{"1min":0.997,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.667,"5min":1.712,"15min":1.712},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2310000000000001,"15min":1.22},"min":{"1min":0.876,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.093}]},{"osd":4,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.244,"15min":1.222},"min":{"1min":0.68700000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.77,"5min":1.865,"15min":1.865},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.276,"5min":1.3049999999999999,"15min":1.3160000000000001},"min":{"1min":0.86099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2969999999999999,"15min":1.2370000000000001},"min":{"1min":0.92800000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.716,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.3839999999999999,"15min":1.351},"min":{"1min":1.1459999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.048}]},{"osd":6,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5109999999999999,"15min":1.4610000000000001},"min":{"1min":1.206,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.5049999999999999,"15min":1.49},"min":{"1min":0.73699999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1480000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.4350000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542217,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1077248,"data_stored":458306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0349999999999999,"15min":1.0569999999999999},"min":{"1min":0.65700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5029999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.093,"15min":1.097},"min":{"1min":0.66800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.6559999999999999,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.161},"min":{"1min":0.73999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3129999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.1479999999999999},"min":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.585,"5min":1.722,"15min":1.722},"last":1.4139999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1990000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.464,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.48},{"interface":"front","average":{"1min":1.038,"5min":1.177,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4390000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.347}]},{"osd":3,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.2230000000000001,"15min":1.3169999999999999},"min":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.25,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.3400000000000001,"15min":1.4039999999999999},"min":{"1min":0.77500000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5469999999999999,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.095}]},{"osd":4,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2470000000000001,"15min":1.323},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.595,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4219999999999999,"15min":1.5640000000000001},"min":{"1min":0.66200000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.728,"5min":3.27,"15min":3.27},"last":1.6870000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2749999999999999,"15min":1.4379999999999999},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4430000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.855},{"interface":"front","average":{"1min":1.355,"5min":1.4419999999999999,"15min":1.55},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8149999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.667}]},{"osd":7,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.399,"15min":1.5469999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.796,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.407,"15min":1.5549999999999999},"min":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.718,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.599}]}]},{"osd":5,"up_from":14,"seq":60129542216,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1081344,"data_stored":459198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0740000000000001,"15min":1.0129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.111,"15min":1.0900000000000001},"min":{"1min":0.65200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.268}]},{"osd":1,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.179,"15min":1.171},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.093},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.1990000000000001,"15min":1.165},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8540000000000001,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.147}]},{"osd":2,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6639999999999999,"5min":2.544,"15min":2.544},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.24,"15min":1.256},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0939999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.518}]},{"osd":3,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2150000000000001,"15min":1.1850000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.306,"5min":1.286,"15min":1.363},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7729999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4730000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.75700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.069},{"interface":"front","average":{"1min":1.359,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.8300000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.296,"15min":1.325},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.871,"5min":1.972,"15min":1.972},"last":1.381},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.49,"15min":1.494},"min":{"1min":0.72999999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0960000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.603}]},{"osd":7,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.70199999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.964,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.04},{"interface":"front","average":{"1min":1.556,"5min":1.502,"15min":1.6160000000000001},"min":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0289999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.5449999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":1052672,"data_stored":451312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":0.998},"min":{"1min":0.68200000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.55,"5min":1.673,"15min":1.673},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.347}]},{"osd":1,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.141,"15min":1.129},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.637,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1399999999999999,"15min":1.1659999999999999},"min":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3740000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0860000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6140000000000001,"5min":1.974,"15min":1.974},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.196,"15min":1.1859999999999999},"min":{"1min":0.76200000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.2430000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.339,"15min":1.3720000000000001},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.7549999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3580000000000001,"15min":1.4039999999999999},"min":{"1min":1.026,"5min":0.874,"15min":0.874},"max":{"1min":1.5840000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1339999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2430000000000001,"15min":1.2949999999999999},"min":{"1min":0.89400000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7729999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.425,"5min":1.345,"15min":1.393},"min":{"1min":0.90300000000000002,"5min":0.625,"15min":0.625},"max":{"1min":2.0350000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.363,"15min":1.4550000000000001},"min":{"1min":0.64900000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.706,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.181}]},{"osd":7,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.534},"min":{"1min":0.96899999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9530000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.5600000000000001,"15min":1.5780000000000001},"min":{"1min":0.94699999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.1699999999999999,"5min":2.54,"15min":2.54},"last":1.3799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":459176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.85899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2509999999999999,"5min":1.744,"15min":1.744},"last":1.486},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.552}]},{"osd":1,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.53700000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.407,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":1.0169999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4870000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97899999999999998,"15min":1.0189999999999999},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.296,"5min":1.827,"15min":1.827},"last":1.276},{"interface":"front","average":{"1min":1.026,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4379999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.228}]},{"osd":4,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.181,"15min":1.2390000000000001},"min":{"1min":0.76600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.222,"15min":1.306},"min":{"1min":0.85799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.609,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.286,"15min":1.3660000000000001},"min":{"1min":0.85199999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6819999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.276,"15min":1.292},"min":{"1min":0.72599999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.502,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3009999999999999,"15min":1.359},"min":{"1min":0.89300000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5349999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3009999999999999,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7370000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.6160000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3600000000000001,"15min":1.4350000000000001},"min":{"1min":0.92000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6080000000000001,"5min":2.468,"15min":2.468},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.371,"15min":1.423},"min":{"1min":0.79400000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5900000000000001,"5min":2.798,"15min":2.798},"last":1.6739999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1069056,"data_stored":458194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84199999999999997,"15min":0.83499999999999996},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92700000000000005,"15min":0.91700000000000004},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81899999999999995,"15min":0.84299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.2849999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.43099999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5269999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.0600000000000001,"15min":1.04},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5,"5min":1.605,"15min":1.605},"last":1.224},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1419999999999999,"15min":1.1279999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.901,"5min":2.694,"15min":2.694},"last":0.92400000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.278,"15min":1.2270000000000001},"min":{"1min":0.81399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.823,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.228,"15min":1.234},"min":{"1min":0.70199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8320000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2090000000000001,"15min":1.2529999999999999},"min":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.3180000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.242,"15min":1.272},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9530000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.79400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0230000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.167},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.298,"15min":1.276},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.964,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3260000000000001,"15min":1.3560000000000001},"min":{"1min":0.749,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9830000000000001,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.391,"15min":1.393},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.867,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1100000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1077248,"data_stored":457806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1499999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88800000000000001,"15min":0.85299999999999998},"min":{"1min":0.56000000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2729999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.004,"15min":1.002},"min":{"1min":0.52100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4970000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.2509999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.59099999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5189999999999999,"5min":1.75,"15min":1.75},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.96099999999999997,"15min":0.96399999999999997},"min":{"1min":0.55500000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.216,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.143,"15min":1.133},"min":{"1min":0.77300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5720000000000001,"5min":1.855,"15min":1.855},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.282,"15min":1.2849999999999999},"min":{"1min":0.81999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6200000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.121}]},{"osd":5,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.262,"15min":1.2689999999999999},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.234,"15min":1.238},"min":{"1min":0.77400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.7270000000000001,"5min":2.173,"15min":2.173},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.92300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.617,"5min":1.917,"15min":1.917},"last":1.202},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3220000000000001,"15min":1.361},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0259999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.0620000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.345,"15min":1.359},"min":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3320000000000001,"15min":1.3480000000000001},"min":{"1min":0.86699999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.6579999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.4630000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":1048014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82199999999999995,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93500000000000005,"15min":0.88600000000000001},"min":{"1min":0.44,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.4870000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.008,"15min":0.97399999999999998},"min":{"1min":0.66400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.105,"15min":1.109},"min":{"1min":0.55100000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1950000000000001,"15min":1.167},"min":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6879999999999999,"5min":1.929,"15min":1.929},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3089999999999999},"min":{"1min":0.76400000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9950000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.012}]},{"osd":5,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.278,"15min":1.27},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.105,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2310000000000001,"15min":1.2010000000000001},"min":{"1min":0.85999999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6180000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2849999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3759999999999999,"15min":1.391},"min":{"1min":0.91600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0430000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.0630000000000002,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.51}]},{"osd":7,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3620000000000001,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.1400000000000001,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.3069999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8180000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.2230000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:26:54.885 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:26:55.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.130+0000 7fc924c00640 1 Processor -- start 2024-04-05T22:26:55.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.130+0000 7fc924c00640 1 -- start start 2024-04-05T22:26:55.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc924c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 0x7fc9200b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc924c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc924c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92015aa60 con 0x7fc92015aef0 2024-04-05T22:26:55.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc924c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9200b3b60 con 0x7fc9201541a0 2024-04-05T22:26:55.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc924c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200701c0 con 0x7fc92006d710 2024-04-05T22:26:55.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc91d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.132+0000 7fc91de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 0x7fc9200b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:46510/0 (socket says 172.21.15.73:46510) 2024-04-05T22:26:55.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 -- 172.21.15.73:0/4276094812 learned_addr learned my addr 172.21.15.73:0/4276094812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:55.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91e800640 1 --1- 172.21.15.73:0/4276094812 >> v1:172.21.15.157:6789/0 conn(0x7fc92015aef0 0x7fc92015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:36232/0 (socket says 172.21.15.73:36232) 2024-04-05T22:26:55.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 -- 172.21.15.73:0/4276094812 >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 msgr2=0x7fc9200b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:55.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 --2- 172.21.15.73:0/4276094812 >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 0x7fc9200b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 -- 172.21.15.73:0/4276094812 >> v1:172.21.15.157:6789/0 conn(0x7fc92015aef0 legacy=0x7fc92015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91d400640 1 -- 172.21.15.73:0/4276094812 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200b3880 con 0x7fc92006d710 2024-04-05T22:26:55.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.133+0000 7fc91de00640 1 --2- 172.21.15.73:0/4276094812 >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 0x7fc9200b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:26:55.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.135+0000 7fc91d400640 1 --2- 172.21.15.73:0/4276094812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd94e6067ef7f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.135+0000 7fc91ca00640 1 -- 172.21.15.73:0/4276094812 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c030840 con 0x7fc92006d710 2024-04-05T22:26:55.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.135+0000 7fc91ca00640 1 -- 172.21.15.73:0/4276094812 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c0309a0 con 0x7fc92006d710 2024-04-05T22:26:55.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.135+0000 7fc91ca00640 1 -- 172.21.15.73:0/4276094812 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c030d30 con 0x7fc92006d710 2024-04-05T22:26:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 -- 172.21.15.73:0/4276094812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 msgr2=0x7fc9200cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 --2- 172.21.15.73:0/4276094812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02b980 comp rx=0 tx=0).stop 2024-04-05T22:26:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 -- 172.21.15.73:0/4276094812 shutdown_connections 2024-04-05T22:26:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 --2- 172.21.15.73:0/4276094812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92006d710 0x7fc9200cd8f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 --2- 172.21.15.73:0/4276094812 >> v2:172.21.15.157:3300/0 conn(0x7fc9201541a0 0x7fc9200b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 -- 172.21.15.73:0/4276094812 >> 172.21.15.73:0/4276094812 conn(0x7fc9200ce8e0 msgr2=0x7fc9200cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:55.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 -- 172.21.15.73:0/4276094812 shutdown_connections 2024-04-05T22:26:55.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.136+0000 7fc924c00640 1 -- 172.21.15.73:0/4276094812 wait complete. 2024-04-05T22:26:55.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.138+0000 7fc924c00640 1 Processor -- start 2024-04-05T22:26:55.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.139+0000 7fc924c00640 1 -- start start 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 0x7fc92014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92015aef0 0x7fc9200af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92014fbf0 con 0x7fc92006d710 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920146dd0 con 0x7fc9201541a0 2024-04-05T22:26:55.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc924c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9201472c0 con 0x7fc92015aef0 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43058/0 (socket says 172.21.15.73:43058) 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91de00640 1 -- 172.21.15.73:0/242509877 learned_addr learned my addr 172.21.15.73:0/242509877 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91d400640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 0x7fc92014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91e800640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92015aef0 0x7fc9200af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91de00640 1 -- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92015aef0 msgr2=0x7fc9200af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.140+0000 7fc91de00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92015aef0 0x7fc9200af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc91de00640 1 -- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 msgr2=0x7fc92014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc91de00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 0x7fc92014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc91de00640 1 -- 172.21.15.73:0/242509877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009400 con 0x7fc92006d710 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc91d400640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 0x7fc92014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc91de00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc91000ebe0 tx=0x7fc91000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910009760 con 0x7fc92006d710 2024-04-05T22:26:55.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91000cc30 con 0x7fc92006d710 2024-04-05T22:26:55.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91000eee0 con 0x7fc92006d710 2024-04-05T22:26:55.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c0094d0 con 0x7fc92006d710 2024-04-05T22:26:55.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.141+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9201476b0 con 0x7fc92006d710 2024-04-05T22:26:55.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.142+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920066af0 con 0x7fc92006d710 2024-04-05T22:26:55.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.143+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc910013470 con 0x7fc92006d710 2024-04-05T22:26:55.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.145+0000 7fc916000640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 0x7fc90006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.145+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc910089860 con 0x7fc92006d710 2024-04-05T22:26:55.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.145+0000 7fc91d400640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 0x7fc90006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.146+0000 7fc91d400640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 0x7fc90006d210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.149+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc910054890 con 0x7fc92006d710 2024-04-05T22:26:55.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.338+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9200b3590 con 0x7fc90006ad50 2024-04-05T22:26:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.361+0000 7fc916000640 1 -- 172.21.15.73:0/242509877 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287451 (secure 0 0 0) 0x7fc90c030be0 con 0x7fc90006ad50 2024-04-05T22:26:55.364 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:55.365 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:26:55.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.366+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 msgr2=0x7fc90006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.366+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 0x7fc90006d210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c03b040 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.366+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 msgr2=0x7fc920167810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.366+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc91000ebe0 tx=0x7fc91000c520 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 shutdown_connections 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc90006ad50 0x7fc90006d210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc92015aef0 0x7fc9200af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9201541a0 0x7fc92014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 --2- 172.21.15.73:0/242509877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc92006d710 0x7fc920167810 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 >> 172.21.15.73:0/242509877 conn(0x7fc9200ce8e0 msgr2=0x7fc92006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 shutdown_connections 2024-04-05T22:26:55.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.367+0000 7fc924c00640 1 -- 172.21.15.73:0/242509877 wait complete. 2024-04-05T22:26:55.386 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-05T22:26:53.505215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258524,"kb_used_data":9532,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754716196,"statfs":{"total":773094113280,"available":772829384704,"internally_reserved":0,"allocated":9760768,"data_stored":4840056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.567176"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449170+0000","last_change":"2024-04-05T22:26:50.476629+0000","last_active":"2024-04-05T22:26:51.449170+0000","last_peered":"2024-04-05T22:26:51.449170+0000","last_clean":"2024-04-05T22:26:51.449170+0000","last_became_active":"2024-04-05T22:26:50.474717+0000","last_became_peered":"2024-04-05T22:26:50.474717+0000","last_unstale":"2024-04-05T22:26:51.449170+0000","last_undegraded":"2024-04-05T22:26:51.449170+0000","last_fullsized":"2024-04-05T22:26:51.449170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:45:43.915282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448564+0000","last_change":"2024-04-05T22:26:50.469282+0000","last_active":"2024-04-05T22:26:51.448564+0000","last_peered":"2024-04-05T22:26:51.448564+0000","last_clean":"2024-04-05T22:26:51.448564+0000","last_became_active":"2024-04-05T22:26:50.469040+0000","last_became_peered":"2024-04-05T22:26:50.469040+0000","last_unstale":"2024-04-05T22:26:51.448564+0000","last_undegraded":"2024-04-05T22:26:51.448564+0000","last_fullsized":"2024-04-05T22:26:51.448564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:37.376824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670392+0000","last_change":"2024-04-05T22:26:50.468244+0000","last_active":"2024-04-05T22:26:51.670392+0000","last_peered":"2024-04-05T22:26:51.670392+0000","last_clean":"2024-04-05T22:26:51.670392+0000","last_became_active":"2024-04-05T22:26:50.467992+0000","last_became_peered":"2024-04-05T22:26:50.467992+0000","last_unstale":"2024-04-05T22:26:51.670392+0000","last_undegraded":"2024-04-05T22:26:51.670392+0000","last_fullsized":"2024-04-05T22:26:51.670392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445295+0000","last_change":"2024-04-05T22:26:50.445496+0000","last_active":"2024-04-05T22:26:51.445295+0000","last_peered":"2024-04-05T22:26:51.445295+0000","last_clean":"2024-04-05T22:26:51.445295+0000","last_became_active":"2024-04-05T22:26:50.441637+0000","last_became_peered":"2024-04-05T22:26:50.441637+0000","last_unstale":"2024-04-05T22:26:51.445295+0000","last_undegraded":"2024-04-05T22:26:51.445295+0000","last_fullsized":"2024-04-05T22:26:51.445295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:09.720274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669833+0000","last_change":"2024-04-05T22:26:50.446938+0000","last_active":"2024-04-05T22:26:51.669833+0000","last_peered":"2024-04-05T22:26:51.669833+0000","last_clean":"2024-04-05T22:26:51.669833+0000","last_became_active":"2024-04-05T22:26:50.443504+0000","last_became_peered":"2024-04-05T22:26:50.443504+0000","last_unstale":"2024-04-05T22:26:51.669833+0000","last_undegraded":"2024-04-05T22:26:51.669833+0000","last_fullsized":"2024-04-05T22:26:51.669833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:52.078681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542519+0000","last_change":"2024-04-05T22:26:50.470078+0000","last_active":"2024-04-05T22:26:51.542519+0000","last_peered":"2024-04-05T22:26:51.542519+0000","last_clean":"2024-04-05T22:26:51.542519+0000","last_became_active":"2024-04-05T22:26:50.467067+0000","last_became_peered":"2024-04-05T22:26:50.467067+0000","last_unstale":"2024-04-05T22:26:51.542519+0000","last_undegraded":"2024-04-05T22:26:51.542519+0000","last_fullsized":"2024-04-05T22:26:51.542519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444775+0000","last_change":"2024-04-05T22:26:50.465526+0000","last_active":"2024-04-05T22:26:51.444775+0000","last_peered":"2024-04-05T22:26:51.444775+0000","last_clean":"2024-04-05T22:26:51.444775+0000","last_became_active":"2024-04-05T22:26:50.465185+0000","last_became_peered":"2024-04-05T22:26:50.465185+0000","last_unstale":"2024-04-05T22:26:51.444775+0000","last_undegraded":"2024-04-05T22:26:51.444775+0000","last_fullsized":"2024-04-05T22:26:51.444775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:37.781661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542166+0000","last_change":"2024-04-05T22:26:50.473175+0000","last_active":"2024-04-05T22:26:51.542166+0000","last_peered":"2024-04-05T22:26:51.542166+0000","last_clean":"2024-04-05T22:26:51.542166+0000","last_became_active":"2024-04-05T22:26:50.471020+0000","last_became_peered":"2024-04-05T22:26:50.471020+0000","last_unstale":"2024-04-05T22:26:51.542166+0000","last_undegraded":"2024-04-05T22:26:51.542166+0000","last_fullsized":"2024-04-05T22:26:51.542166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:14:01.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542648+0000","last_change":"2024-04-05T22:26:50.464379+0000","last_active":"2024-04-05T22:26:51.542648+0000","last_peered":"2024-04-05T22:26:51.542648+0000","last_clean":"2024-04-05T22:26:51.542648+0000","last_became_active":"2024-04-05T22:26:50.464052+0000","last_became_peered":"2024-04-05T22:26:50.464052+0000","last_unstale":"2024-04-05T22:26:51.542648+0000","last_undegraded":"2024-04-05T22:26:51.542648+0000","last_fullsized":"2024-04-05T22:26:51.542648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:03:13.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449017+0000","last_change":"2024-04-05T22:26:50.468565+0000","last_active":"2024-04-05T22:26:51.449017+0000","last_peered":"2024-04-05T22:26:51.449017+0000","last_clean":"2024-04-05T22:26:51.449017+0000","last_became_active":"2024-04-05T22:26:50.468164+0000","last_became_peered":"2024-04-05T22:26:50.468164+0000","last_unstale":"2024-04-05T22:26:51.449017+0000","last_undegraded":"2024-04-05T22:26:51.449017+0000","last_fullsized":"2024-04-05T22:26:51.449017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:47:31.128655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447860+0000","last_change":"2024-04-05T22:26:50.464116+0000","last_active":"2024-04-05T22:26:51.447860+0000","last_peered":"2024-04-05T22:26:51.447860+0000","last_clean":"2024-04-05T22:26:51.447860+0000","last_became_active":"2024-04-05T22:26:50.463823+0000","last_became_peered":"2024-04-05T22:26:50.463823+0000","last_unstale":"2024-04-05T22:26:51.447860+0000","last_undegraded":"2024-04-05T22:26:51.447860+0000","last_fullsized":"2024-04-05T22:26:51.447860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:59:38.707931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447993+0000","last_change":"2024-04-05T22:26:50.467702+0000","last_active":"2024-04-05T22:26:51.447993+0000","last_peered":"2024-04-05T22:26:51.447993+0000","last_clean":"2024-04-05T22:26:51.447993+0000","last_became_active":"2024-04-05T22:26:50.466998+0000","last_became_peered":"2024-04-05T22:26:50.466998+0000","last_unstale":"2024-04-05T22:26:51.447993+0000","last_undegraded":"2024-04-05T22:26:51.447993+0000","last_fullsized":"2024-04-05T22:26:51.447993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:54:10.669899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669140+0000","last_change":"2024-04-05T22:26:50.468702+0000","last_active":"2024-04-05T22:26:51.669140+0000","last_peered":"2024-04-05T22:26:51.669140+0000","last_clean":"2024-04-05T22:26:51.669140+0000","last_became_active":"2024-04-05T22:26:50.465428+0000","last_became_peered":"2024-04-05T22:26:50.465428+0000","last_unstale":"2024-04-05T22:26:51.669140+0000","last_undegraded":"2024-04-05T22:26:51.669140+0000","last_fullsized":"2024-04-05T22:26:51.669140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:52.078681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444886+0000","last_change":"2024-04-05T22:26:50.444087+0000","last_active":"2024-04-05T22:26:51.444886+0000","last_peered":"2024-04-05T22:26:51.444886+0000","last_clean":"2024-04-05T22:26:51.444886+0000","last_became_active":"2024-04-05T22:26:50.443460+0000","last_became_peered":"2024-04-05T22:26:50.443460+0000","last_unstale":"2024-04-05T22:26:51.444886+0000","last_undegraded":"2024-04-05T22:26:51.444886+0000","last_fullsized":"2024-04-05T22:26:51.444886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:29.267191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668172+0000","last_change":"2024-04-05T22:26:50.471110+0000","last_active":"2024-04-05T22:26:51.668172+0000","last_peered":"2024-04-05T22:26:51.668172+0000","last_clean":"2024-04-05T22:26:51.668172+0000","last_became_active":"2024-04-05T22:26:50.470787+0000","last_became_peered":"2024-04-05T22:26:50.470787+0000","last_unstale":"2024-04-05T22:26:51.668172+0000","last_undegraded":"2024-04-05T22:26:51.668172+0000","last_fullsized":"2024-04-05T22:26:51.668172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:47.570328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669331+0000","last_change":"2024-04-05T22:26:50.475371+0000","last_active":"2024-04-05T22:26:51.669331+0000","last_peered":"2024-04-05T22:26:51.669331+0000","last_clean":"2024-04-05T22:26:51.669331+0000","last_became_active":"2024-04-05T22:26:50.474817+0000","last_became_peered":"2024-04-05T22:26:50.474817+0000","last_unstale":"2024-04-05T22:26:51.669331+0000","last_undegraded":"2024-04-05T22:26:51.669331+0000","last_fullsized":"2024-04-05T22:26:51.669331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:38.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670065+0000","last_change":"2024-04-05T22:26:50.452125+0000","last_active":"2024-04-05T22:26:51.670065+0000","last_peered":"2024-04-05T22:26:51.670065+0000","last_clean":"2024-04-05T22:26:51.670065+0000","last_became_active":"2024-04-05T22:26:50.445701+0000","last_became_peered":"2024-04-05T22:26:50.445701+0000","last_unstale":"2024-04-05T22:26:51.670065+0000","last_undegraded":"2024-04-05T22:26:51.670065+0000","last_fullsized":"2024-04-05T22:26:51.670065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:33.206553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445589+0000","last_change":"2024-04-05T22:26:50.469089+0000","last_active":"2024-04-05T22:26:51.445589+0000","last_peered":"2024-04-05T22:26:51.445589+0000","last_clean":"2024-04-05T22:26:51.445589+0000","last_became_active":"2024-04-05T22:26:50.468813+0000","last_became_peered":"2024-04-05T22:26:50.468813+0000","last_unstale":"2024-04-05T22:26:51.445589+0000","last_undegraded":"2024-04-05T22:26:51.445589+0000","last_fullsized":"2024-04-05T22:26:51.445589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:49.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671016+0000","last_change":"2024-04-05T22:26:50.449995+0000","last_active":"2024-04-05T22:26:51.671016+0000","last_peered":"2024-04-05T22:26:51.671016+0000","last_clean":"2024-04-05T22:26:51.671016+0000","last_became_active":"2024-04-05T22:26:50.449116+0000","last_became_peered":"2024-04-05T22:26:50.449116+0000","last_unstale":"2024-04-05T22:26:51.671016+0000","last_undegraded":"2024-04-05T22:26:51.671016+0000","last_fullsized":"2024-04-05T22:26:51.671016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:04:15.920151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445194+0000","last_change":"2024-04-05T22:26:50.446837+0000","last_active":"2024-04-05T22:26:51.445194+0000","last_peered":"2024-04-05T22:26:51.445194+0000","last_clean":"2024-04-05T22:26:51.445194+0000","last_became_active":"2024-04-05T22:26:50.446588+0000","last_became_peered":"2024-04-05T22:26:50.446588+0000","last_unstale":"2024-04-05T22:26:51.445194+0000","last_undegraded":"2024-04-05T22:26:51.445194+0000","last_fullsized":"2024-04-05T22:26:51.445194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:32:13.186227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.445313+0000","last_change":"2024-04-05T22:26:50.451460+0000","last_active":"2024-04-05T22:26:51.445313+0000","last_peered":"2024-04-05T22:26:51.445313+0000","last_clean":"2024-04-05T22:26:51.445313+0000","last_became_active":"2024-04-05T22:26:50.451208+0000","last_became_peered":"2024-04-05T22:26:50.451208+0000","last_unstale":"2024-04-05T22:26:51.445313+0000","last_undegraded":"2024-04-05T22:26:51.445313+0000","last_fullsized":"2024-04-05T22:26:51.445313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:07.310789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447727+0000","last_change":"2024-04-05T22:26:50.450227+0000","last_active":"2024-04-05T22:26:51.447727+0000","last_peered":"2024-04-05T22:26:51.447727+0000","last_clean":"2024-04-05T22:26:51.447727+0000","last_became_active":"2024-04-05T22:26:50.449958+0000","last_became_peered":"2024-04-05T22:26:50.449958+0000","last_unstale":"2024-04-05T22:26:51.447727+0000","last_undegraded":"2024-04-05T22:26:51.447727+0000","last_fullsized":"2024-04-05T22:26:51.447727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:14:01.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448117+0000","last_change":"2024-04-05T22:26:50.469917+0000","last_active":"2024-04-05T22:26:51.448117+0000","last_peered":"2024-04-05T22:26:51.448117+0000","last_clean":"2024-04-05T22:26:51.448117+0000","last_became_active":"2024-04-05T22:26:50.469437+0000","last_became_peered":"2024-04-05T22:26:50.469437+0000","last_unstale":"2024-04-05T22:26:51.448117+0000","last_undegraded":"2024-04-05T22:26:51.448117+0000","last_fullsized":"2024-04-05T22:26:51.448117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:04.878802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444439+0000","last_change":"2024-04-05T22:26:50.444209+0000","last_active":"2024-04-05T22:26:51.444439+0000","last_peered":"2024-04-05T22:26:51.444439+0000","last_clean":"2024-04-05T22:26:51.444439+0000","last_became_active":"2024-04-05T22:26:50.443958+0000","last_became_peered":"2024-04-05T22:26:50.443958+0000","last_unstale":"2024-04-05T22:26:51.444439+0000","last_undegraded":"2024-04-05T22:26:51.444439+0000","last_fullsized":"2024-04-05T22:26:51.444439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:12:23.375572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447971+0000","last_change":"2024-04-05T22:26:50.468403+0000","last_active":"2024-04-05T22:26:51.447971+0000","last_peered":"2024-04-05T22:26:51.447971+0000","last_clean":"2024-04-05T22:26:51.447971+0000","last_became_active":"2024-04-05T22:26:50.468129+0000","last_became_peered":"2024-04-05T22:26:50.468129+0000","last_unstale":"2024-04-05T22:26:51.447971+0000","last_undegraded":"2024-04-05T22:26:51.447971+0000","last_fullsized":"2024-04-05T22:26:51.447971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:55.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670198+0000","last_change":"2024-04-05T22:26:50.464439+0000","last_active":"2024-04-05T22:26:51.670198+0000","last_peered":"2024-04-05T22:26:51.670198+0000","last_clean":"2024-04-05T22:26:51.670198+0000","last_became_active":"2024-04-05T22:26:50.464139+0000","last_became_peered":"2024-04-05T22:26:50.464139+0000","last_unstale":"2024-04-05T22:26:51.670198+0000","last_undegraded":"2024-04-05T22:26:51.670198+0000","last_fullsized":"2024-04-05T22:26:51.670198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:38.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670043+0000","last_change":"2024-04-05T22:26:50.461152+0000","last_active":"2024-04-05T22:26:51.670043+0000","last_peered":"2024-04-05T22:26:51.670043+0000","last_clean":"2024-04-05T22:26:51.670043+0000","last_became_active":"2024-04-05T22:26:50.460942+0000","last_became_peered":"2024-04-05T22:26:50.460942+0000","last_unstale":"2024-04-05T22:26:51.670043+0000","last_undegraded":"2024-04-05T22:26:51.670043+0000","last_fullsized":"2024-04-05T22:26:51.670043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448448+0000","last_change":"2024-04-05T22:26:50.474498+0000","last_active":"2024-04-05T22:26:51.448448+0000","last_peered":"2024-04-05T22:26:51.448448+0000","last_clean":"2024-04-05T22:26:51.448448+0000","last_became_active":"2024-04-05T22:26:50.474219+0000","last_became_peered":"2024-04-05T22:26:50.474219+0000","last_unstale":"2024-04-05T22:26:51.448448+0000","last_undegraded":"2024-04-05T22:26:51.448448+0000","last_fullsized":"2024-04-05T22:26:51.448448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:44:18.909071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448107+0000","last_change":"2024-04-05T22:26:50.449009+0000","last_active":"2024-04-05T22:26:51.448107+0000","last_peered":"2024-04-05T22:26:51.448107+0000","last_clean":"2024-04-05T22:26:51.448107+0000","last_became_active":"2024-04-05T22:26:50.448734+0000","last_became_peered":"2024-04-05T22:26:50.448734+0000","last_unstale":"2024-04-05T22:26:51.448107+0000","last_undegraded":"2024-04-05T22:26:51.448107+0000","last_fullsized":"2024-04-05T22:26:51.448107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:55:54.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447849+0000","last_change":"2024-04-05T22:26:50.447939+0000","last_active":"2024-04-05T22:26:51.447849+0000","last_peered":"2024-04-05T22:26:51.447849+0000","last_clean":"2024-04-05T22:26:51.447849+0000","last_became_active":"2024-04-05T22:26:50.447664+0000","last_became_peered":"2024-04-05T22:26:50.447664+0000","last_unstale":"2024-04-05T22:26:51.447849+0000","last_undegraded":"2024-04-05T22:26:51.447849+0000","last_fullsized":"2024-04-05T22:26:51.447849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:03.818840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669804+0000","last_change":"2024-04-05T22:26:50.445584+0000","last_active":"2024-04-05T22:26:51.669804+0000","last_peered":"2024-04-05T22:26:51.669804+0000","last_clean":"2024-04-05T22:26:51.669804+0000","last_became_active":"2024-04-05T22:26:50.443977+0000","last_became_peered":"2024-04-05T22:26:50.443977+0000","last_unstale":"2024-04-05T22:26:51.669804+0000","last_undegraded":"2024-04-05T22:26:51.669804+0000","last_fullsized":"2024-04-05T22:26:51.669804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447901+0000","last_change":"2024-04-05T22:26:50.463355+0000","last_active":"2024-04-05T22:26:51.447901+0000","last_peered":"2024-04-05T22:26:51.447901+0000","last_clean":"2024-04-05T22:26:51.447901+0000","last_became_active":"2024-04-05T22:26:50.463093+0000","last_became_peered":"2024-04-05T22:26:50.463093+0000","last_unstale":"2024-04-05T22:26:51.447901+0000","last_undegraded":"2024-04-05T22:26:51.447901+0000","last_fullsized":"2024-04-05T22:26:51.447901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:25:45.237590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669797+0000","last_change":"2024-04-05T22:26:50.471056+0000","last_active":"2024-04-05T22:26:51.669797+0000","last_peered":"2024-04-05T22:26:51.669797+0000","last_clean":"2024-04-05T22:26:51.669797+0000","last_became_active":"2024-04-05T22:26:50.468922+0000","last_became_peered":"2024-04-05T22:26:50.468922+0000","last_unstale":"2024-04-05T22:26:51.669797+0000","last_undegraded":"2024-04-05T22:26:51.669797+0000","last_fullsized":"2024-04-05T22:26:51.669797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:07.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449390+0000","last_change":"2024-04-05T22:26:49.974129+0000","last_active":"2024-04-05T22:26:51.449390+0000","last_peered":"2024-04-05T22:26:51.449390+0000","last_clean":"2024-04-05T22:26:51.449390+0000","last_became_active":"2024-04-05T22:26:49.970785+0000","last_became_peered":"2024-04-05T22:26:49.970785+0000","last_unstale":"2024-04-05T22:26:51.449390+0000","last_undegraded":"2024-04-05T22:26:51.449390+0000","last_fullsized":"2024-04-05T22:26:51.449390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:23:59.162101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449252+0000","last_change":"2024-04-05T22:26:50.471535+0000","last_active":"2024-04-05T22:26:51.449252+0000","last_peered":"2024-04-05T22:26:51.449252+0000","last_clean":"2024-04-05T22:26:51.449252+0000","last_became_active":"2024-04-05T22:26:50.468202+0000","last_became_peered":"2024-04-05T22:26:50.468202+0000","last_unstale":"2024-04-05T22:26:51.449252+0000","last_undegraded":"2024-04-05T22:26:51.449252+0000","last_fullsized":"2024-04-05T22:26:51.449252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:45.948664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670689+0000","last_change":"2024-04-05T22:26:49.990804+0000","last_active":"2024-04-05T22:26:51.670689+0000","last_peered":"2024-04-05T22:26:51.670689+0000","last_clean":"2024-04-05T22:26:51.670689+0000","last_became_active":"2024-04-05T22:26:49.989935+0000","last_became_peered":"2024-04-05T22:26:49.989935+0000","last_unstale":"2024-04-05T22:26:51.670689+0000","last_undegraded":"2024-04-05T22:26:51.670689+0000","last_fullsized":"2024-04-05T22:26:51.670689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:56.801248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448567+0000","last_change":"2024-04-05T22:26:50.467508+0000","last_active":"2024-04-05T22:26:51.448567+0000","last_peered":"2024-04-05T22:26:51.448567+0000","last_clean":"2024-04-05T22:26:51.448567+0000","last_became_active":"2024-04-05T22:26:50.466631+0000","last_became_peered":"2024-04-05T22:26:50.466631+0000","last_unstale":"2024-04-05T22:26:51.448567+0000","last_undegraded":"2024-04-05T22:26:51.448567+0000","last_fullsized":"2024-04-05T22:26:51.448567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:41:55.136198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671012+0000","last_change":"2024-04-05T22:26:49.978114+0000","last_active":"2024-04-05T22:26:51.671012+0000","last_peered":"2024-04-05T22:26:51.671012+0000","last_clean":"2024-04-05T22:26:51.671012+0000","last_became_active":"2024-04-05T22:26:49.977857+0000","last_became_peered":"2024-04-05T22:26:49.977857+0000","last_unstale":"2024-04-05T22:26:51.671012+0000","last_undegraded":"2024-04-05T22:26:51.671012+0000","last_fullsized":"2024-04-05T22:26:51.671012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:03.227249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670631+0000","last_change":"2024-04-05T22:26:50.467444+0000","last_active":"2024-04-05T22:26:51.670631+0000","last_peered":"2024-04-05T22:26:51.670631+0000","last_clean":"2024-04-05T22:26:51.670631+0000","last_became_active":"2024-04-05T22:26:50.465607+0000","last_became_peered":"2024-04-05T22:26:50.465607+0000","last_unstale":"2024-04-05T22:26:51.670631+0000","last_undegraded":"2024-04-05T22:26:51.670631+0000","last_fullsized":"2024-04-05T22:26:51.670631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:03:13.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670724+0000","last_change":"2024-04-05T22:26:49.979137+0000","last_active":"2024-04-05T22:26:51.670724+0000","last_peered":"2024-04-05T22:26:51.670724+0000","last_clean":"2024-04-05T22:26:51.670724+0000","last_became_active":"2024-04-05T22:26:49.978895+0000","last_became_peered":"2024-04-05T22:26:49.978895+0000","last_unstale":"2024-04-05T22:26:51.670724+0000","last_undegraded":"2024-04-05T22:26:51.670724+0000","last_fullsized":"2024-04-05T22:26:51.670724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:37.302152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448125+0000","last_change":"2024-04-05T22:26:50.447907+0000","last_active":"2024-04-05T22:26:51.448125+0000","last_peered":"2024-04-05T22:26:51.448125+0000","last_clean":"2024-04-05T22:26:51.448125+0000","last_became_active":"2024-04-05T22:26:50.447519+0000","last_became_peered":"2024-04-05T22:26:50.447519+0000","last_unstale":"2024-04-05T22:26:51.448125+0000","last_undegraded":"2024-04-05T22:26:51.448125+0000","last_fullsized":"2024-04-05T22:26:51.448125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:00.147854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671202+0000","last_change":"2024-04-05T22:26:49.992180+0000","last_active":"2024-04-05T22:26:51.671202+0000","last_peered":"2024-04-05T22:26:51.671202+0000","last_clean":"2024-04-05T22:26:51.671202+0000","last_became_active":"2024-04-05T22:26:49.991456+0000","last_became_peered":"2024-04-05T22:26:49.991456+0000","last_unstale":"2024-04-05T22:26:51.671202+0000","last_undegraded":"2024-04-05T22:26:51.671202+0000","last_fullsized":"2024-04-05T22:26:51.671202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:57:13.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671060+0000","last_change":"2024-04-05T22:26:50.450153+0000","last_active":"2024-04-05T22:26:51.671060+0000","last_peered":"2024-04-05T22:26:51.671060+0000","last_clean":"2024-04-05T22:26:51.671060+0000","last_became_active":"2024-04-05T22:26:50.449403+0000","last_became_peered":"2024-04-05T22:26:50.449403+0000","last_unstale":"2024-04-05T22:26:51.671060+0000","last_undegraded":"2024-04-05T22:26:51.671060+0000","last_fullsized":"2024-04-05T22:26:51.671060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669608+0000","last_change":"2024-04-05T22:26:49.989374+0000","last_active":"2024-04-05T22:26:51.669608+0000","last_peered":"2024-04-05T22:26:51.669608+0000","last_clean":"2024-04-05T22:26:51.669608+0000","last_became_active":"2024-04-05T22:26:49.989105+0000","last_became_peered":"2024-04-05T22:26:49.989105+0000","last_unstale":"2024-04-05T22:26:51.669608+0000","last_undegraded":"2024-04-05T22:26:51.669608+0000","last_fullsized":"2024-04-05T22:26:51.669608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:02:59.501337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668866+0000","last_change":"2024-04-05T22:26:50.463960+0000","last_active":"2024-04-05T22:26:51.668866+0000","last_peered":"2024-04-05T22:26:51.668866+0000","last_clean":"2024-04-05T22:26:51.668866+0000","last_became_active":"2024-04-05T22:26:50.463648+0000","last_became_peered":"2024-04-05T22:26:50.463648+0000","last_unstale":"2024-04-05T22:26:51.668866+0000","last_undegraded":"2024-04-05T22:26:51.668866+0000","last_fullsized":"2024-04-05T22:26:51.668866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670068+0000","last_change":"2024-04-05T22:26:49.988141+0000","last_active":"2024-04-05T22:26:51.670068+0000","last_peered":"2024-04-05T22:26:51.670068+0000","last_clean":"2024-04-05T22:26:51.670068+0000","last_became_active":"2024-04-05T22:26:49.987827+0000","last_became_peered":"2024-04-05T22:26:49.987827+0000","last_unstale":"2024-04-05T22:26:51.670068+0000","last_undegraded":"2024-04-05T22:26:51.670068+0000","last_fullsized":"2024-04-05T22:26:51.670068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:06.729896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668219+0000","last_change":"2024-04-05T22:26:50.465723+0000","last_active":"2024-04-05T22:26:51.668219+0000","last_peered":"2024-04-05T22:26:51.668219+0000","last_clean":"2024-04-05T22:26:51.668219+0000","last_became_active":"2024-04-05T22:26:50.465451+0000","last_became_peered":"2024-04-05T22:26:50.465451+0000","last_unstale":"2024-04-05T22:26:51.668219+0000","last_undegraded":"2024-04-05T22:26:51.668219+0000","last_fullsized":"2024-04-05T22:26:51.668219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:42.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447197+0000","last_change":"2024-04-05T22:26:49.991508+0000","last_active":"2024-04-05T22:26:51.447197+0000","last_peered":"2024-04-05T22:26:51.447197+0000","last_clean":"2024-04-05T22:26:51.447197+0000","last_became_active":"2024-04-05T22:26:49.991230+0000","last_became_peered":"2024-04-05T22:26:49.991230+0000","last_unstale":"2024-04-05T22:26:51.447197+0000","last_undegraded":"2024-04-05T22:26:51.447197+0000","last_fullsized":"2024-04-05T22:26:51.447197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:42.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669525+0000","last_change":"2024-04-05T22:26:50.446398+0000","last_active":"2024-04-05T22:26:51.669525+0000","last_peered":"2024-04-05T22:26:51.669525+0000","last_clean":"2024-04-05T22:26:51.669525+0000","last_became_active":"2024-04-05T22:26:50.446108+0000","last_became_peered":"2024-04-05T22:26:50.446108+0000","last_unstale":"2024-04-05T22:26:51.669525+0000","last_undegraded":"2024-04-05T22:26:51.669525+0000","last_fullsized":"2024-04-05T22:26:51.669525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:40:26.072253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449420+0000","last_change":"2024-04-05T22:26:49.982818+0000","last_active":"2024-04-05T22:26:51.449420+0000","last_peered":"2024-04-05T22:26:51.449420+0000","last_clean":"2024-04-05T22:26:51.449420+0000","last_became_active":"2024-04-05T22:26:49.978059+0000","last_became_peered":"2024-04-05T22:26:49.978059+0000","last_unstale":"2024-04-05T22:26:51.449420+0000","last_undegraded":"2024-04-05T22:26:51.449420+0000","last_fullsized":"2024-04-05T22:26:51.449420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:40.758748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542535+0000","last_change":"2024-04-05T22:26:50.472098+0000","last_active":"2024-04-05T22:26:51.542535+0000","last_peered":"2024-04-05T22:26:51.542535+0000","last_clean":"2024-04-05T22:26:51.542535+0000","last_became_active":"2024-04-05T22:26:50.471796+0000","last_became_peered":"2024-04-05T22:26:50.471796+0000","last_unstale":"2024-04-05T22:26:51.542535+0000","last_undegraded":"2024-04-05T22:26:51.542535+0000","last_fullsized":"2024-04-05T22:26:51.542535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449079+0000","last_change":"2024-04-05T22:26:49.990638+0000","last_active":"2024-04-05T22:26:51.449079+0000","last_peered":"2024-04-05T22:26:51.449079+0000","last_clean":"2024-04-05T22:26:51.449079+0000","last_became_active":"2024-04-05T22:26:49.990152+0000","last_became_peered":"2024-04-05T22:26:49.990152+0000","last_unstale":"2024-04-05T22:26:51.449079+0000","last_undegraded":"2024-04-05T22:26:51.449079+0000","last_fullsized":"2024-04-05T22:26:51.449079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:23.965031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670462+0000","last_change":"2024-04-05T22:26:50.472069+0000","last_active":"2024-04-05T22:26:51.670462+0000","last_peered":"2024-04-05T22:26:51.670462+0000","last_clean":"2024-04-05T22:26:51.670462+0000","last_became_active":"2024-04-05T22:26:50.471829+0000","last_became_peered":"2024-04-05T22:26:50.471829+0000","last_unstale":"2024-04-05T22:26:51.670462+0000","last_undegraded":"2024-04-05T22:26:51.670462+0000","last_fullsized":"2024-04-05T22:26:51.670462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:02:20.024821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448676+0000","last_change":"2024-04-05T22:26:49.979113+0000","last_active":"2024-04-05T22:26:51.448676+0000","last_peered":"2024-04-05T22:26:51.448676+0000","last_clean":"2024-04-05T22:26:51.448676+0000","last_became_active":"2024-04-05T22:26:49.978597+0000","last_became_peered":"2024-04-05T22:26:49.978597+0000","last_unstale":"2024-04-05T22:26:51.448676+0000","last_undegraded":"2024-04-05T22:26:51.448676+0000","last_fullsized":"2024-04-05T22:26:51.448676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:52.064458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670296+0000","last_change":"2024-04-05T22:26:50.457478+0000","last_active":"2024-04-05T22:26:51.670296+0000","last_peered":"2024-04-05T22:26:51.670296+0000","last_clean":"2024-04-05T22:26:51.670296+0000","last_became_active":"2024-04-05T22:26:50.457188+0000","last_became_peered":"2024-04-05T22:26:50.457188+0000","last_unstale":"2024-04-05T22:26:51.670296+0000","last_undegraded":"2024-04-05T22:26:51.670296+0000","last_fullsized":"2024-04-05T22:26:51.670296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:09:52.041376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542784+0000","last_change":"2024-04-05T22:26:49.990358+0000","last_active":"2024-04-05T22:26:51.542784+0000","last_peered":"2024-04-05T22:26:51.542784+0000","last_clean":"2024-04-05T22:26:51.542784+0000","last_became_active":"2024-04-05T22:26:49.988603+0000","last_became_peered":"2024-04-05T22:26:49.988603+0000","last_unstale":"2024-04-05T22:26:51.542784+0000","last_undegraded":"2024-04-05T22:26:51.542784+0000","last_fullsized":"2024-04-05T22:26:51.542784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:02:54.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670167+0000","last_change":"2024-04-05T22:26:50.458580+0000","last_active":"2024-04-05T22:26:51.670167+0000","last_peered":"2024-04-05T22:26:51.670167+0000","last_clean":"2024-04-05T22:26:51.670167+0000","last_became_active":"2024-04-05T22:26:50.458266+0000","last_became_peered":"2024-04-05T22:26:50.458266+0000","last_unstale":"2024-04-05T22:26:51.670167+0000","last_undegraded":"2024-04-05T22:26:51.670167+0000","last_fullsized":"2024-04-05T22:26:51.670167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:04.246740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448221+0000","last_change":"2024-04-05T22:26:49.979872+0000","last_active":"2024-04-05T22:26:51.448221+0000","last_peered":"2024-04-05T22:26:51.448221+0000","last_clean":"2024-04-05T22:26:51.448221+0000","last_became_active":"2024-04-05T22:26:49.979599+0000","last_became_peered":"2024-04-05T22:26:49.979599+0000","last_unstale":"2024-04-05T22:26:51.448221+0000","last_undegraded":"2024-04-05T22:26:51.448221+0000","last_fullsized":"2024-04-05T22:26:51.448221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:39.665785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669260+0000","last_change":"2024-04-05T22:26:50.469359+0000","last_active":"2024-04-05T22:26:51.669260+0000","last_peered":"2024-04-05T22:26:51.669260+0000","last_clean":"2024-04-05T22:26:51.669260+0000","last_became_active":"2024-04-05T22:26:50.469115+0000","last_became_peered":"2024-04-05T22:26:50.469115+0000","last_unstale":"2024-04-05T22:26:51.669260+0000","last_undegraded":"2024-04-05T22:26:51.669260+0000","last_fullsized":"2024-04-05T22:26:51.669260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:47.950907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668900+0000","last_change":"2024-04-05T22:26:49.978360+0000","last_active":"2024-04-05T22:26:51.668900+0000","last_peered":"2024-04-05T22:26:51.668900+0000","last_clean":"2024-04-05T22:26:51.668900+0000","last_became_active":"2024-04-05T22:26:49.978079+0000","last_became_peered":"2024-04-05T22:26:49.978079+0000","last_unstale":"2024-04-05T22:26:51.668900+0000","last_undegraded":"2024-04-05T22:26:51.668900+0000","last_fullsized":"2024-04-05T22:26:51.668900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:46:28.225378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447115+0000","last_change":"2024-04-05T22:26:50.445207+0000","last_active":"2024-04-05T22:26:51.447115+0000","last_peered":"2024-04-05T22:26:51.447115+0000","last_clean":"2024-04-05T22:26:51.447115+0000","last_became_active":"2024-04-05T22:26:50.444925+0000","last_became_peered":"2024-04-05T22:26:50.444925+0000","last_unstale":"2024-04-05T22:26:51.447115+0000","last_undegraded":"2024-04-05T22:26:51.447115+0000","last_fullsized":"2024-04-05T22:26:51.447115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:08:15.805465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542315+0000","last_change":"2024-04-05T22:26:49.991225+0000","last_active":"2024-04-05T22:26:51.542315+0000","last_peered":"2024-04-05T22:26:51.542315+0000","last_clean":"2024-04-05T22:26:51.542315+0000","last_became_active":"2024-04-05T22:26:49.990894+0000","last_became_peered":"2024-04-05T22:26:49.990894+0000","last_unstale":"2024-04-05T22:26:51.542315+0000","last_undegraded":"2024-04-05T22:26:51.542315+0000","last_fullsized":"2024-04-05T22:26:51.542315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:13.465943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447042+0000","last_change":"2024-04-05T22:26:50.477395+0000","last_active":"2024-04-05T22:26:51.447042+0000","last_peered":"2024-04-05T22:26:51.447042+0000","last_clean":"2024-04-05T22:26:51.447042+0000","last_became_active":"2024-04-05T22:26:50.476951+0000","last_became_peered":"2024-04-05T22:26:50.476951+0000","last_unstale":"2024-04-05T22:26:51.447042+0000","last_undegraded":"2024-04-05T22:26:51.447042+0000","last_fullsized":"2024-04-05T22:26:51.447042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:14.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669695+0000","last_change":"2024-04-05T22:26:49.984930+0000","last_active":"2024-04-05T22:26:51.669695+0000","last_peered":"2024-04-05T22:26:51.669695+0000","last_clean":"2024-04-05T22:26:51.669695+0000","last_became_active":"2024-04-05T22:26:49.984491+0000","last_became_peered":"2024-04-05T22:26:49.984491+0000","last_unstale":"2024-04-05T22:26:51.669695+0000","last_undegraded":"2024-04-05T22:26:51.669695+0000","last_fullsized":"2024-04-05T22:26:51.669695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:44.827747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669489+0000","last_change":"2024-04-05T22:26:50.466432+0000","last_active":"2024-04-05T22:26:51.669489+0000","last_peered":"2024-04-05T22:26:51.669489+0000","last_clean":"2024-04-05T22:26:51.669489+0000","last_became_active":"2024-04-05T22:26:50.466106+0000","last_became_peered":"2024-04-05T22:26:50.466106+0000","last_unstale":"2024-04-05T22:26:51.669489+0000","last_undegraded":"2024-04-05T22:26:51.669489+0000","last_fullsized":"2024-04-05T22:26:51.669489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:07.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449553+0000","last_change":"2024-04-05T22:26:49.987839+0000","last_active":"2024-04-05T22:26:51.449553+0000","last_peered":"2024-04-05T22:26:51.449553+0000","last_clean":"2024-04-05T22:26:51.449553+0000","last_became_active":"2024-04-05T22:26:49.987573+0000","last_became_peered":"2024-04-05T22:26:49.987573+0000","last_unstale":"2024-04-05T22:26:51.449553+0000","last_undegraded":"2024-04-05T22:26:51.449553+0000","last_fullsized":"2024-04-05T22:26:51.449553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:09.178326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449305+0000","last_change":"2024-04-05T22:26:50.471349+0000","last_active":"2024-04-05T22:26:51.449305+0000","last_peered":"2024-04-05T22:26:51.449305+0000","last_clean":"2024-04-05T22:26:51.449305+0000","last_became_active":"2024-04-05T22:26:50.467942+0000","last_became_peered":"2024-04-05T22:26:50.467942+0000","last_unstale":"2024-04-05T22:26:51.449305+0000","last_undegraded":"2024-04-05T22:26:51.449305+0000","last_fullsized":"2024-04-05T22:26:51.449305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:21:58.606106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670859+0000","last_change":"2024-04-05T22:26:49.986587+0000","last_active":"2024-04-05T22:26:51.670859+0000","last_peered":"2024-04-05T22:26:51.670859+0000","last_clean":"2024-04-05T22:26:51.670859+0000","last_became_active":"2024-04-05T22:26:49.986247+0000","last_became_peered":"2024-04-05T22:26:49.986247+0000","last_unstale":"2024-04-05T22:26:51.670859+0000","last_undegraded":"2024-04-05T22:26:51.670859+0000","last_fullsized":"2024-04-05T22:26:51.670859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:38.116846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670545+0000","last_change":"2024-04-05T22:26:50.450125+0000","last_active":"2024-04-05T22:26:51.670545+0000","last_peered":"2024-04-05T22:26:51.670545+0000","last_clean":"2024-04-05T22:26:51.670545+0000","last_became_active":"2024-04-05T22:26:50.447366+0000","last_became_peered":"2024-04-05T22:26:50.447366+0000","last_unstale":"2024-04-05T22:26:51.670545+0000","last_undegraded":"2024-04-05T22:26:51.670545+0000","last_fullsized":"2024-04-05T22:26:51.670545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670744+0000","last_change":"2024-04-05T22:26:49.987077+0000","last_active":"2024-04-05T22:26:51.670744+0000","last_peered":"2024-04-05T22:26:51.670744+0000","last_clean":"2024-04-05T22:26:51.670744+0000","last_became_active":"2024-04-05T22:26:49.986796+0000","last_became_peered":"2024-04-05T22:26:49.986796+0000","last_unstale":"2024-04-05T22:26:51.670744+0000","last_undegraded":"2024-04-05T22:26:51.670744+0000","last_fullsized":"2024-04-05T22:26:51.670744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:43:51.073153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444979+0000","last_change":"2024-04-05T22:26:50.444320+0000","last_active":"2024-04-05T22:26:51.444979+0000","last_peered":"2024-04-05T22:26:51.444979+0000","last_clean":"2024-04-05T22:26:51.444979+0000","last_became_active":"2024-04-05T22:26:50.443799+0000","last_became_peered":"2024-04-05T22:26:50.443799+0000","last_unstale":"2024-04-05T22:26:51.444979+0000","last_undegraded":"2024-04-05T22:26:51.444979+0000","last_fullsized":"2024-04-05T22:26:51.444979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:40.671313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668775+0000","last_change":"2024-04-05T22:26:49.976556+0000","last_active":"2024-04-05T22:26:51.668775+0000","last_peered":"2024-04-05T22:26:51.668775+0000","last_clean":"2024-04-05T22:26:51.668775+0000","last_became_active":"2024-04-05T22:26:49.976246+0000","last_became_peered":"2024-04-05T22:26:49.976246+0000","last_unstale":"2024-04-05T22:26:51.668775+0000","last_undegraded":"2024-04-05T22:26:51.668775+0000","last_fullsized":"2024-04-05T22:26:51.668775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:15:48.875016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542206+0000","last_change":"2024-04-05T22:26:50.465488+0000","last_active":"2024-04-05T22:26:51.542206+0000","last_peered":"2024-04-05T22:26:51.542206+0000","last_clean":"2024-04-05T22:26:51.542206+0000","last_became_active":"2024-04-05T22:26:50.465245+0000","last_became_peered":"2024-04-05T22:26:50.465245+0000","last_unstale":"2024-04-05T22:26:51.542206+0000","last_undegraded":"2024-04-05T22:26:51.542206+0000","last_fullsized":"2024-04-05T22:26:51.542206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:39.122374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444976+0000","last_change":"2024-04-05T22:26:49.980432+0000","last_active":"2024-04-05T22:26:51.444976+0000","last_peered":"2024-04-05T22:26:51.444976+0000","last_clean":"2024-04-05T22:26:51.444976+0000","last_became_active":"2024-04-05T22:26:49.980084+0000","last_became_peered":"2024-04-05T22:26:49.980084+0000","last_unstale":"2024-04-05T22:26:51.444976+0000","last_undegraded":"2024-04-05T22:26:51.444976+0000","last_fullsized":"2024-04-05T22:26:51.444976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:13:12.954092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444537+0000","last_change":"2024-04-05T22:26:50.471761+0000","last_active":"2024-04-05T22:26:51.444537+0000","last_peered":"2024-04-05T22:26:51.444537+0000","last_clean":"2024-04-05T22:26:51.444537+0000","last_became_active":"2024-04-05T22:26:50.470938+0000","last_became_peered":"2024-04-05T22:26:50.470938+0000","last_unstale":"2024-04-05T22:26:51.444537+0000","last_undegraded":"2024-04-05T22:26:51.444537+0000","last_fullsized":"2024-04-05T22:26:51.444537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:01:57.806776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.444865+0000","last_change":"2024-04-05T22:26:49.982431+0000","last_active":"2024-04-05T22:26:51.444865+0000","last_peered":"2024-04-05T22:26:51.444865+0000","last_clean":"2024-04-05T22:26:51.444865+0000","last_became_active":"2024-04-05T22:26:49.982096+0000","last_became_peered":"2024-04-05T22:26:49.982096+0000","last_unstale":"2024-04-05T22:26:51.444865+0000","last_undegraded":"2024-04-05T22:26:51.444865+0000","last_fullsized":"2024-04-05T22:26:51.444865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:30:33.435365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542151+0000","last_change":"2024-04-05T22:26:50.463449+0000","last_active":"2024-04-05T22:26:51.542151+0000","last_peered":"2024-04-05T22:26:51.542151+0000","last_clean":"2024-04-05T22:26:51.542151+0000","last_became_active":"2024-04-05T22:26:50.463178+0000","last_became_peered":"2024-04-05T22:26:50.463178+0000","last_unstale":"2024-04-05T22:26:51.542151+0000","last_undegraded":"2024-04-05T22:26:51.542151+0000","last_fullsized":"2024-04-05T22:26:51.542151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:43:40.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.668184+0000","last_change":"2024-04-05T22:26:49.976365+0000","last_active":"2024-04-05T22:26:51.668184+0000","last_peered":"2024-04-05T22:26:51.668184+0000","last_clean":"2024-04-05T22:26:51.668184+0000","last_became_active":"2024-04-05T22:26:49.975241+0000","last_became_peered":"2024-04-05T22:26:49.975241+0000","last_unstale":"2024-04-05T22:26:51.668184+0000","last_undegraded":"2024-04-05T22:26:51.668184+0000","last_fullsized":"2024-04-05T22:26:51.668184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:51:32.043965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670074+0000","last_change":"2024-04-05T22:26:50.471326+0000","last_active":"2024-04-05T22:26:51.670074+0000","last_peered":"2024-04-05T22:26:51.670074+0000","last_clean":"2024-04-05T22:26:51.670074+0000","last_became_active":"2024-04-05T22:26:50.471086+0000","last_became_peered":"2024-04-05T22:26:50.471086+0000","last_unstale":"2024-04-05T22:26:51.670074+0000","last_undegraded":"2024-04-05T22:26:51.670074+0000","last_fullsized":"2024-04-05T22:26:51.670074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:04.232777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448084+0000","last_change":"2024-04-05T22:26:49.984673+0000","last_active":"2024-04-05T22:26:51.448084+0000","last_peered":"2024-04-05T22:26:51.448084+0000","last_clean":"2024-04-05T22:26:51.448084+0000","last_became_active":"2024-04-05T22:26:49.984418+0000","last_became_peered":"2024-04-05T22:26:49.984418+0000","last_unstale":"2024-04-05T22:26:51.448084+0000","last_undegraded":"2024-04-05T22:26:51.448084+0000","last_fullsized":"2024-04-05T22:26:51.448084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:20:10.386093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.671196+0000","last_change":"2024-04-05T22:26:50.475387+0000","last_active":"2024-04-05T22:26:51.671196+0000","last_peered":"2024-04-05T22:26:51.671196+0000","last_clean":"2024-04-05T22:26:51.671196+0000","last_became_active":"2024-04-05T22:26:50.475128+0000","last_became_peered":"2024-04-05T22:26:50.475128+0000","last_unstale":"2024-04-05T22:26:51.671196+0000","last_undegraded":"2024-04-05T22:26:51.671196+0000","last_fullsized":"2024-04-05T22:26:51.671196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:45.833275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448499+0000","last_change":"2024-04-05T22:26:49.978251+0000","last_active":"2024-04-05T22:26:51.448499+0000","last_peered":"2024-04-05T22:26:51.448499+0000","last_clean":"2024-04-05T22:26:51.448499+0000","last_became_active":"2024-04-05T22:26:49.977947+0000","last_became_peered":"2024-04-05T22:26:49.977947+0000","last_unstale":"2024-04-05T22:26:51.448499+0000","last_undegraded":"2024-04-05T22:26:51.448499+0000","last_fullsized":"2024-04-05T22:26:51.448499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:07:56.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.449055+0000","last_change":"2024-04-05T22:26:50.467334+0000","last_active":"2024-04-05T22:26:51.449055+0000","last_peered":"2024-04-05T22:26:51.449055+0000","last_clean":"2024-04-05T22:26:51.449055+0000","last_became_active":"2024-04-05T22:26:50.466362+0000","last_became_peered":"2024-04-05T22:26:50.466362+0000","last_unstale":"2024-04-05T22:26:51.449055+0000","last_undegraded":"2024-04-05T22:26:51.449055+0000","last_fullsized":"2024-04-05T22:26:51.449055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:37:05.166659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448933+0000","last_change":"2024-04-05T22:26:49.972712+0000","last_active":"2024-04-05T22:26:51.448933+0000","last_peered":"2024-04-05T22:26:51.448933+0000","last_clean":"2024-04-05T22:26:51.448933+0000","last_became_active":"2024-04-05T22:26:49.970681+0000","last_became_peered":"2024-04-05T22:26:49.970681+0000","last_unstale":"2024-04-05T22:26:51.448933+0000","last_undegraded":"2024-04-05T22:26:51.448933+0000","last_fullsized":"2024-04-05T22:26:51.448933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:11:34.505583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670813+0000","last_change":"2024-04-05T22:26:50.453314+0000","last_active":"2024-04-05T22:26:51.670813+0000","last_peered":"2024-04-05T22:26:51.670813+0000","last_clean":"2024-04-05T22:26:51.670813+0000","last_became_active":"2024-04-05T22:26:50.451034+0000","last_became_peered":"2024-04-05T22:26:50.451034+0000","last_unstale":"2024-04-05T22:26:51.670813+0000","last_undegraded":"2024-04-05T22:26:51.670813+0000","last_fullsized":"2024-04-05T22:26:51.670813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:45.863406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448847+0000","last_change":"2024-04-05T22:26:49.977248+0000","last_active":"2024-04-05T22:26:51.448847+0000","last_peered":"2024-04-05T22:26:51.448847+0000","last_clean":"2024-04-05T22:26:51.448847+0000","last_became_active":"2024-04-05T22:26:49.976563+0000","last_became_peered":"2024-04-05T22:26:49.976563+0000","last_unstale":"2024-04-05T22:26:51.448847+0000","last_undegraded":"2024-04-05T22:26:51.448847+0000","last_fullsized":"2024-04-05T22:26:51.448847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:39:52.607533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.542212+0000","last_change":"2024-04-05T22:26:50.472474+0000","last_active":"2024-04-05T22:26:51.542212+0000","last_peered":"2024-04-05T22:26:51.542212+0000","last_clean":"2024-04-05T22:26:51.542212+0000","last_became_active":"2024-04-05T22:26:50.471793+0000","last_became_peered":"2024-04-05T22:26:50.471793+0000","last_unstale":"2024-04-05T22:26:51.542212+0000","last_undegraded":"2024-04-05T22:26:51.542212+0000","last_fullsized":"2024-04-05T22:26:51.542212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:09:52.041376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670386+0000","last_change":"2024-04-05T22:26:49.987189+0000","last_active":"2024-04-05T22:26:51.670386+0000","last_peered":"2024-04-05T22:26:51.670386+0000","last_clean":"2024-04-05T22:26:51.670386+0000","last_became_active":"2024-04-05T22:26:49.986926+0000","last_became_peered":"2024-04-05T22:26:49.986926+0000","last_unstale":"2024-04-05T22:26:51.670386+0000","last_undegraded":"2024-04-05T22:26:51.670386+0000","last_fullsized":"2024-04-05T22:26:51.670386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:24:46.945379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670290+0000","last_change":"2024-04-05T22:26:50.463207+0000","last_active":"2024-04-05T22:26:51.670290+0000","last_peered":"2024-04-05T22:26:51.670290+0000","last_clean":"2024-04-05T22:26:51.670290+0000","last_became_active":"2024-04-05T22:26:50.462935+0000","last_became_peered":"2024-04-05T22:26:50.462935+0000","last_unstale":"2024-04-05T22:26:51.670290+0000","last_undegraded":"2024-04-05T22:26:51.670290+0000","last_fullsized":"2024-04-05T22:26:51.670290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:18:30.236918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669679+0000","last_change":"2024-04-05T22:26:49.985572+0000","last_active":"2024-04-05T22:26:51.669679+0000","last_peered":"2024-04-05T22:26:51.669679+0000","last_clean":"2024-04-05T22:26:51.669679+0000","last_became_active":"2024-04-05T22:26:49.985295+0000","last_became_peered":"2024-04-05T22:26:49.985295+0000","last_unstale":"2024-04-05T22:26:51.669679+0000","last_undegraded":"2024-04-05T22:26:51.669679+0000","last_fullsized":"2024-04-05T22:26:51.669679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:12:22.370044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669039+0000","last_change":"2024-04-05T22:26:50.467165+0000","last_active":"2024-04-05T22:26:51.669039+0000","last_peered":"2024-04-05T22:26:51.669039+0000","last_clean":"2024-04-05T22:26:51.669039+0000","last_became_active":"2024-04-05T22:26:50.466895+0000","last_became_peered":"2024-04-05T22:26:50.466895+0000","last_unstale":"2024-04-05T22:26:51.669039+0000","last_undegraded":"2024-04-05T22:26:51.669039+0000","last_fullsized":"2024-04-05T22:26:51.669039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:37:04.232777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448299+0000","last_change":"2024-04-05T22:26:49.975710+0000","last_active":"2024-04-05T22:26:51.448299+0000","last_peered":"2024-04-05T22:26:51.448299+0000","last_clean":"2024-04-05T22:26:51.448299+0000","last_became_active":"2024-04-05T22:26:49.974070+0000","last_became_peered":"2024-04-05T22:26:49.974070+0000","last_unstale":"2024-04-05T22:26:51.448299+0000","last_undegraded":"2024-04-05T22:26:51.448299+0000","last_fullsized":"2024-04-05T22:26:51.448299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:16:00.737675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.448115+0000","last_change":"2024-04-05T22:26:50.469692+0000","last_active":"2024-04-05T22:26:51.448115+0000","last_peered":"2024-04-05T22:26:51.448115+0000","last_clean":"2024-04-05T22:26:51.448115+0000","last_became_active":"2024-04-05T22:26:50.469163+0000","last_became_peered":"2024-04-05T22:26:50.469163+0000","last_unstale":"2024-04-05T22:26:51.448115+0000","last_undegraded":"2024-04-05T22:26:51.448115+0000","last_fullsized":"2024-04-05T22:26:51.448115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:31:01.941910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.669650+0000","last_change":"2024-04-05T22:26:49.991763+0000","last_active":"2024-04-05T22:26:51.669650+0000","last_peered":"2024-04-05T22:26:51.669650+0000","last_clean":"2024-04-05T22:26:51.669650+0000","last_became_active":"2024-04-05T22:26:49.991472+0000","last_became_peered":"2024-04-05T22:26:49.991472+0000","last_unstale":"2024-04-05T22:26:51.669650+0000","last_undegraded":"2024-04-05T22:26:51.669650+0000","last_fullsized":"2024-04-05T22:26:51.669650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:14.320056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.447767+0000","last_change":"2024-04-05T22:26:50.472266+0000","last_active":"2024-04-05T22:26:51.447767+0000","last_peered":"2024-04-05T22:26:51.447767+0000","last_clean":"2024-04-05T22:26:51.447767+0000","last_became_active":"2024-04-05T22:26:50.471033+0000","last_became_peered":"2024-04-05T22:26:50.471033+0000","last_unstale":"2024-04-05T22:26:51.447767+0000","last_undegraded":"2024-04-05T22:26:51.447767+0000","last_fullsized":"2024-04-05T22:26:51.447767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:49.935757+0000","last_clean_scrub_stamp":"2024-04-05T22:26:49.935757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:55:01.424253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-05T22:26:51.670900+0000","last_change":"2024-04-05T22:26:49.985636+0000","last_active":"2024-04-05T22:26:51.670900+0000","last_peered":"2024-04-05T22:26:51.670900+0000","last_clean":"2024-04-05T22:26:51.670900+0000","last_became_active":"2024-04-05T22:26:49.985358+0000","last_became_peered":"2024-04-05T22:26:49.985358+0000","last_unstale":"2024-04-05T22:26:51.670900+0000","last_undegraded":"2024-04-05T22:26:51.670900+0000","last_fullsized":"2024-04-05T22:26:51.670900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:26:48.930229+0000","last_clean_scrub_stamp":"2024-04-05T22:26:48.930229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:31:13.465943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":257,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-05T22:26:50.449786+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:26:50.449786+0000","last_peered":"2024-04-05T22:26:50.449786+0000","last_clean":"2024-04-05T22:26:50.449786+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:26:50.449786+0000","last_undegraded":"2024-04-05T22:26:50.449786+0000","last_fullsized":"2024-04-05T22:26:50.449786+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1662976,"data_stored":1048050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0580000000000001,"15min":1.03},"min":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.54,"5min":1.796,"15min":1.796},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0640000000000001,"15min":1.012},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5129999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1419999999999999,"15min":1.052},"min":{"1min":0.748,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.48,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.0860000000000001,"15min":1.0629999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1819999999999999,"15min":1.1739999999999999},"min":{"1min":0.71699999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.262,"15min":1.218},"min":{"1min":0.84899999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1839999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.204,"15min":1.1379999999999999},"min":{"1min":0.997,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.667,"5min":1.712,"15min":1.712},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2310000000000001,"15min":1.22},"min":{"1min":0.876,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.093}]},{"osd":4,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.244,"15min":1.222},"min":{"1min":0.68700000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.77,"5min":1.865,"15min":1.865},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.276,"5min":1.3049999999999999,"15min":1.3160000000000001},"min":{"1min":0.86099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2969999999999999,"15min":1.2370000000000001},"min":{"1min":0.92800000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.716,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.3839999999999999,"15min":1.351},"min":{"1min":1.1459999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.048}]},{"osd":6,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5109999999999999,"15min":1.4610000000000001},"min":{"1min":1.206,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.5049999999999999,"15min":1.49},"min":{"1min":0.73699999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1480000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.4350000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542217,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1077248,"data_stored":458306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0349999999999999,"15min":1.0569999999999999},"min":{"1min":0.65700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5029999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.093,"15min":1.097},"min":{"1min":0.66800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.6559999999999999,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.161},"min":{"1min":0.73999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3129999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.1479999999999999},"min":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.585,"5min":1.722,"15min":1.722},"last":1.4139999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1990000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.464,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.48},{"interface":"front","average":{"1min":1.038,"5min":1.177,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4390000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.347}]},{"osd":3,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.2230000000000001,"15min":1.3169999999999999},"min":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.25,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.3400000000000001,"15min":1.4039999999999999},"min":{"1min":0.77500000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5469999999999999,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":1.095}]},{"osd":4,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2470000000000001,"15min":1.323},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.595,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4219999999999999,"15min":1.5640000000000001},"min":{"1min":0.66200000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.728,"5min":3.27,"15min":3.27},"last":1.6870000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2749999999999999,"15min":1.4379999999999999},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4430000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.855},{"interface":"front","average":{"1min":1.355,"5min":1.4419999999999999,"15min":1.55},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8149999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.667}]},{"osd":7,"last update":"Fri Apr 5 22:26:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.399,"15min":1.5469999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.796,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.407,"15min":1.5549999999999999},"min":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.718,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":1.599}]}]},{"osd":5,"up_from":14,"seq":60129542216,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1081344,"data_stored":459198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0740000000000001,"15min":1.0129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.111,"15min":1.0900000000000001},"min":{"1min":0.65200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.268}]},{"osd":1,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.179,"15min":1.171},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.093},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.1990000000000001,"15min":1.165},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8540000000000001,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.147}]},{"osd":2,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6639999999999999,"5min":2.544,"15min":2.544},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.24,"15min":1.256},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0939999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.518}]},{"osd":3,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2150000000000001,"15min":1.1850000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.306,"5min":1.286,"15min":1.363},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7729999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4730000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.75700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.069},{"interface":"front","average":{"1min":1.359,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.8300000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.296,"15min":1.325},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.871,"5min":1.972,"15min":1.972},"last":1.381},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.49,"15min":1.494},"min":{"1min":0.72999999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0960000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.603}]},{"osd":7,"last update":"Fri Apr 5 22:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.70199999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.964,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.04},{"interface":"front","average":{"1min":1.556,"5min":1.502,"15min":1.6160000000000001},"min":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0289999999999999,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.5449999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":1052672,"data_stored":451312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":0.998},"min":{"1min":0.68200000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.55,"5min":1.673,"15min":1.673},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.347}]},{"osd":1,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.141,"15min":1.129},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.637,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1399999999999999,"15min":1.1659999999999999},"min":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3740000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0860000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6140000000000001,"5min":1.974,"15min":1.974},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.196,"15min":1.1859999999999999},"min":{"1min":0.76200000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.2430000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.339,"15min":1.3720000000000001},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.7549999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3580000000000001,"15min":1.4039999999999999},"min":{"1min":1.026,"5min":0.874,"15min":0.874},"max":{"1min":1.5840000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1339999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2430000000000001,"15min":1.2949999999999999},"min":{"1min":0.89400000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7729999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.425,"5min":1.345,"15min":1.393},"min":{"1min":0.90300000000000002,"5min":0.625,"15min":0.625},"max":{"1min":2.0350000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.363,"15min":1.4550000000000001},"min":{"1min":0.64900000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.706,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.181}]},{"osd":7,"last update":"Fri Apr 5 22:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.534},"min":{"1min":0.96899999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9530000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.5600000000000001,"15min":1.5780000000000001},"min":{"1min":0.94699999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.1699999999999999,"5min":2.54,"15min":2.54},"last":1.3799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":459176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.85899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2509999999999999,"5min":1.744,"15min":1.744},"last":1.486},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.552}]},{"osd":1,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.53700000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.407,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":1.0169999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4870000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97899999999999998,"15min":1.0189999999999999},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.296,"5min":1.827,"15min":1.827},"last":1.276},{"interface":"front","average":{"1min":1.026,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4379999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.228}]},{"osd":4,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.181,"15min":1.2390000000000001},"min":{"1min":0.76600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.222,"15min":1.306},"min":{"1min":0.85799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.609,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.286,"15min":1.3660000000000001},"min":{"1min":0.85199999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6819999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.276,"15min":1.292},"min":{"1min":0.72599999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.502,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3009999999999999,"15min":1.359},"min":{"1min":0.89300000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5349999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3009999999999999,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7370000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.6160000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3600000000000001,"15min":1.4350000000000001},"min":{"1min":0.92000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6080000000000001,"5min":2.468,"15min":2.468},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.371,"15min":1.423},"min":{"1min":0.79400000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5900000000000001,"5min":2.798,"15min":2.798},"last":1.6739999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1069056,"data_stored":458194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84199999999999997,"15min":0.83499999999999996},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92700000000000005,"15min":0.91700000000000004},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81899999999999995,"15min":0.84299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.2849999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.43099999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5269999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.0600000000000001,"15min":1.04},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5,"5min":1.605,"15min":1.605},"last":1.224},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1419999999999999,"15min":1.1279999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.901,"5min":2.694,"15min":2.694},"last":0.92400000000000004}]},{"osd":4,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.278,"15min":1.2270000000000001},"min":{"1min":0.81399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.823,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.228,"15min":1.234},"min":{"1min":0.70199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8320000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2090000000000001,"15min":1.2529999999999999},"min":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.3180000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.242,"15min":1.272},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9530000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.79400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0230000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.167},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.298,"15min":1.276},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.964,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3260000000000001,"15min":1.3560000000000001},"min":{"1min":0.749,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9830000000000001,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.391,"15min":1.393},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.867,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1100000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1077248,"data_stored":457806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1499999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88800000000000001,"15min":0.85299999999999998},"min":{"1min":0.56000000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2729999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.004,"15min":1.002},"min":{"1min":0.52100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4970000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.2509999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.59099999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5189999999999999,"5min":1.75,"15min":1.75},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.96099999999999997,"15min":0.96399999999999997},"min":{"1min":0.55500000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.216,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.143,"15min":1.133},"min":{"1min":0.77300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5720000000000001,"5min":1.855,"15min":1.855},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.282,"15min":1.2849999999999999},"min":{"1min":0.81999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6200000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.121}]},{"osd":5,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.262,"15min":1.2689999999999999},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.234,"15min":1.238},"min":{"1min":0.77400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.7270000000000001,"5min":2.173,"15min":2.173},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.92300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.617,"5min":1.917,"15min":1.917},"last":1.202},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3220000000000001,"15min":1.361},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0259999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.0620000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:26:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.345,"15min":1.359},"min":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3320000000000001,"15min":1.3480000000000001},"min":{"1min":0.86699999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.6579999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.4630000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":1048014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82199999999999995,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93500000000000005,"15min":0.88600000000000001},"min":{"1min":0.44,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.4870000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.008,"15min":0.97399999999999998},"min":{"1min":0.66400000000000003,"5min":0.318,"15min":0.318},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.105,"15min":1.109},"min":{"1min":0.55100000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1950000000000001,"15min":1.167},"min":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6879999999999999,"5min":1.929,"15min":1.929},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3089999999999999},"min":{"1min":0.76400000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9950000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.012}]},{"osd":5,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.278,"15min":1.27},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.105,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2310000000000001,"15min":1.2010000000000001},"min":{"1min":0.85999999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6180000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2849999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3759999999999999,"15min":1.391},"min":{"1min":0.91600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0430000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.0630000000000002,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.51}]},{"osd":7,"last update":"Fri Apr 5 22:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3620000000000001,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.1400000000000001,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.3069999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8180000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.2230000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:26:55.390 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:26:55.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:26:55.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.651+0000 7f450ec00640 1 Processor -- start 2024-04-05T22:26:55.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.652+0000 7f450ec00640 1 -- start start 2024-04-05T22:26:55.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.653+0000 7f450ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f450806c630 0x7f450806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.653+0000 7f450ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f45080ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.653+0000 7f450ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45080b1b30 con 0x7f45080b22a0 2024-04-05T22:26:55.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.653+0000 7f450ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45080b1e10 con 0x7f450806c630 2024-04-05T22:26:55.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.653+0000 7f450ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450806cf50 con 0x7f45080cbe00 2024-04-05T22:26:55.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f450cc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f45080b22a0 0x7f45080b2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37888/0 (socket says 172.21.15.73:37888) 2024-04-05T22:26:55.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f450806c630 0x7f450806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f450cc00640 1 -- 172.21.15.73:0/2057542373 learned_addr learned my addr 172.21.15.73:0/2057542373 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507400640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f45080ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507e00640 1 -- 172.21.15.73:0/2057542373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 msgr2=0x7f45080ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507e00640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f45080ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507e00640 1 -- 172.21.15.73:0/2057542373 >> v1:172.21.15.73:6789/0 conn(0x7f45080b22a0 legacy=0x7f45080b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4507e00640 1 -- 172.21.15.73:0/2057542373 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080cc300 con 0x7f450806c630 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.654+0000 7f4506a00640 1 -- 172.21.15.73:0/2057542373 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759286150 0 0) 0x7f45080b1b30 con 0x7f45080b22a0 2024-04-05T22:26:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.655+0000 7f4507e00640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450806c630 0x7f450806ca10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963ca7e177735155 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f4506a00640 1 -- 172.21.15.73:0/2057542373 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8031830 con 0x7f450806c630 2024-04-05T22:26:55.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f4506a00640 1 -- 172.21.15.73:0/2057542373 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8031990 con 0x7f450806c630 2024-04-05T22:26:55.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f4506a00640 1 -- 172.21.15.73:0/2057542373 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8031d20 con 0x7f450806c630 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 -- 172.21.15.73:0/2057542373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450806c630 msgr2=0x7f450806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450806c630 0x7f450806ca10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 -- 172.21.15.73:0/2057542373 shutdown_connections 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f45080ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 --2- 172.21.15.73:0/2057542373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450806c630 0x7f450806ca10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 -- 172.21.15.73:0/2057542373 >> 172.21.15.73:0/2057542373 conn(0x7f45080b0830 msgr2=0x7f45080b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.656+0000 7f450ec00640 1 -- 172.21.15.73:0/2057542373 shutdown_connections 2024-04-05T22:26:55.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.657+0000 7f450ec00640 1 -- 172.21.15.73:0/2057542373 wait complete. 2024-04-05T22:26:55.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.658+0000 7f450ec00640 1 Processor -- start 2024-04-05T22:26:55.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.658+0000 7f450ec00640 1 -- start start 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f450806c630 0x7f450813c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f450815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450813c980 con 0x7f45080b22a0 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080ae510 con 0x7f450806c630 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f450ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4508141090 con 0x7f45080cbe00 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.659+0000 7f4507400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43072/0 (socket says 172.21.15.73:43072) 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 -- 172.21.15.73:0/888590660 learned_addr learned my addr 172.21.15.73:0/888590660 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 -- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 msgr2=0x7f450815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f450815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 -- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f450806c630 msgr2=0x7f450813c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f450806c630 0x7f450813c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 -- 172.21.15.73:0/888590660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8009400 con 0x7f45080b22a0 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f4507400640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d5f0 tx=0x7f44f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40125f0 con 0x7f45080b22a0 2024-04-05T22:26:55.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4012750 con 0x7f45080b22a0 2024-04-05T22:26:55.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.660+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508141370 con 0x7f45080b22a0 2024-04-05T22:26:55.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.661+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4012b00 con 0x7f45080b22a0 2024-04-05T22:26:55.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.661+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4508141930 con 0x7f45080b22a0 2024-04-05T22:26:55.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.662+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4508068160 con 0x7f45080b22a0 2024-04-05T22:26:55.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.663+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f44f4012c60 con 0x7f45080b22a0 2024-04-05T22:26:55.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.663+0000 7f44ffe00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 0x7f44e806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:55.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.663+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f4089a20 con 0x7f45080b22a0 2024-04-05T22:26:55.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.664+0000 7f4507e00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 0x7f44e806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:55.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.665+0000 7f4507e00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 0x7f44e806d1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44f8036000 tx=0x7f44f8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:55.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.668+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f44f4054af0 con 0x7f45080b22a0 2024-04-05T22:26:55.863 INFO:tasks.ceph.mds.a.smithi073.stderr:2024-04-05T22:26:55.864+0000 7fb46a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:26:55.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.930+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f45080b27b0 con 0x7f45080b22a0 2024-04-05T22:26:55.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.938+0000 7f44ffe00640 1 -- 172.21.15.73:0/888590660 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f44f4058110 con 0x7f45080b22a0 2024-04-05T22:26:55.937 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:26:55.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 msgr2=0x7f44e806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 0x7f44e806d1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44f8036000 tx=0x7f44f8030000 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 msgr2=0x7f450815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d5f0 tx=0x7f44f400dab0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 shutdown_connections 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44e806ad20 0x7f44e806d1e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080cbe00 0x7f450815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45080b22a0 0x7f450815b230 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 --2- 172.21.15.73:0/888590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f450806c630 0x7f450813c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 >> 172.21.15.73:0/888590660 conn(0x7f45080b0830 msgr2=0x7f45080aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 shutdown_connections 2024-04-05T22:26:55.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:55.941+0000 7f450ec00640 1 -- 172.21.15.73:0/888590660 wait complete. 2024-04-05T22:26:55.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:26:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.164+0000 7fe9f2200640 1 Processor -- start 2024-04-05T22:26:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.164+0000 7fe9f2200640 1 -- start start 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.165+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec1587d0 0x7fe9ec0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.165+0000 7fe9f2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.165+0000 7fe9f2200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9ec06d0c0 0x7fe9ec15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9f2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec06f5b0 con 0x7fe9ec0ccd20 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9f2200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec070410 con 0x7fe9ec06d0c0 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9f2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec158bb0 con 0x7fe9ec1587d0 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60840/0 (socket says 172.21.15.73:60840) 2024-04-05T22:26:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 -- 172.21.15.73:0/1202088368 learned_addr learned my addr 172.21.15.73:0/1202088368 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eb400640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec1587d0 0x7fe9ec0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 -- 172.21.15.73:0/1202088368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec1587d0 msgr2=0x7fe9ec0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9ebe00640 1 --2- 172.21.15.73:0/1202088368 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec06d0c0 0x7fe9ec15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec1587d0 0x7fe9ec0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 -- 172.21.15.73:0/1202088368 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec06d0c0 msgr2=0x7fe9ec15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:56.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 --2- 172.21.15.73:0/1202088368 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec06d0c0 0x7fe9ec15e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.166+0000 7fe9eaa00640 1 -- 172.21.15.73:0/1202088368 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec06f800 con 0x7fe9ec0ccd20 2024-04-05T22:26:56.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.167+0000 7fe9eaa00640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0ae4b0 tx=0x7fe9d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c52b645034678a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9ea000640 1 -- 172.21.15.73:0/1202088368 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d402ca70 con 0x7fe9ec0ccd20 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9ea000640 1 -- 172.21.15.73:0/1202088368 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d402cbd0 con 0x7fe9ec0ccd20 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9ea000640 1 -- 172.21.15.73:0/1202088368 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40343f0 con 0x7fe9ec0ccd20 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 -- 172.21.15.73:0/1202088368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec0ccd20 msgr2=0x7fe9ec06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0ae4b0 tx=0x7fe9d402bed0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 -- 172.21.15.73:0/1202088368 shutdown_connections 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec0ccd20 0x7fe9ec06cb80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1202088368 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec06d0c0 0x7fe9ec15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1202088368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec1587d0 0x7fe9ec0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.168+0000 7fe9f2200640 1 -- 172.21.15.73:0/1202088368 >> 172.21.15.73:0/1202088368 conn(0x7fe9ec0ce8e0 msgr2=0x7fe9ec0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.169+0000 7fe9f2200640 1 -- 172.21.15.73:0/1202088368 shutdown_connections 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.169+0000 7fe9f2200640 1 -- 172.21.15.73:0/1202088368 wait complete. 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.170+0000 7fe9f2200640 1 Processor -- start 2024-04-05T22:26:56.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.170+0000 7fe9f2200640 1 -- start start 2024-04-05T22:26:56.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec06d0c0 0x7fe9ec1494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1587d0 0x7fe9ec0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b25e0 con 0x7fe9ec154820 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b1960 con 0x7fe9ec1587d0 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9f2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b1c40 con 0x7fe9ec06d0c0 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.171+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60844/0 (socket says 172.21.15.73:60844) 2024-04-05T22:26:56.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 -- 172.21.15.73:0/1785664998 learned_addr learned my addr 172.21.15.73:0/1785664998 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9eaa00640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1587d0 0x7fe9ec0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 -- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec06d0c0 msgr2=0x7fe9ec1494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec06d0c0 0x7fe9ec1494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 -- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1587d0 msgr2=0x7fe9ec0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1587d0 0x7fe9ec0b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 -- 172.21.15.73:0/1785664998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4009b30 con 0x7fe9ec154820 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9ebe00640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c570 tx=0x7fe9e400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4004220 con 0x7fe9ec154820 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4004380 con 0x7fe9ec154820 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4003830 con 0x7fe9ec154820 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.172+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec0b1f20 con 0x7fe9ec154820 2024-04-05T22:26:56.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.173+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec0b2f40 con 0x7fe9ec154820 2024-04-05T22:26:56.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.174+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec066af0 con 0x7fe9ec154820 2024-04-05T22:26:56.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.175+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe9e4003990 con 0x7fe9ec154820 2024-04-05T22:26:56.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.175+0000 7fe9e3400640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 0x7fe9c806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:56.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.176+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9e4089390 con 0x7fe9ec154820 2024-04-05T22:26:56.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.176+0000 7fe9eb400640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 0x7fe9c806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:56.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.177+0000 7fe9eb400640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 0x7fe9c806d1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00a810 tx=0x7fe9dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:56.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.180+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe9e40542b0 con 0x7fe9ec154820 2024-04-05T22:26:56.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.416+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9ec0b2720 con 0x7fe9ec154820 2024-04-05T22:26:56.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.417+0000 7fe9e3400640 1 -- 172.21.15.73:0/1785664998 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fe9ec0b2720 con 0x7fe9ec154820 2024-04-05T22:26:56.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:26:56.416 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2227587240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2227587240},{"type":"v1","addr":"172.21.15.157:6835","nonce":2227587240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:26:56.416 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 50 2024-04-05T22:26:56.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.419+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 msgr2=0x7fe9c806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:56.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.419+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 0x7fe9c806d1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00a810 tx=0x7fe9dc008040 comp rx=0 tx=0).stop 2024-04-05T22:26:56.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.419+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 msgr2=0x7fe9ec0c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:56.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.419+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c570 tx=0x7fe9e400ca30 comp rx=0 tx=0).stop 2024-04-05T22:26:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 shutdown_connections 2024-04-05T22:26:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe9c806ad20 0x7fe9c806d1e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9ec154820 0x7fe9ec0c37e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1587d0 0x7fe9ec0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 --2- 172.21.15.73:0/1785664998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec06d0c0 0x7fe9ec1494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 >> 172.21.15.73:0/1785664998 conn(0x7fe9ec0ce8e0 msgr2=0x7fe9ec15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 shutdown_connections 2024-04-05T22:26:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:56.420+0000 7fe9f2200640 1 -- 172.21.15.73:0/1785664998 wait complete. 2024-04-05T22:26:56.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:26:50.425705+0000', 'modified': '2024-04-05T22:26:53.451569+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7176}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7176': {'gid': 7176, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6833/3038019968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 3038019968}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 3038019968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:26:56.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:26:56.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-05T22:26:57.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.061+0000 7fe6efe00640 1 Processor -- start 2024-04-05T22:26:57.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.061+0000 7fe6efe00640 1 -- start start 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 0x7fe6f006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6efe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f00b3640 0x7fe6f0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6efe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f00b3c40 con 0x7fe6f0070840 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6efe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00711e0 con 0x7fe6f00b3640 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6efe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f006c490 con 0x7fe6f0146490 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f00b3640 0x7fe6f0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f00b3640 0x7fe6f0070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60854/0 (socket says 172.21.15.73:60854) 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 -- 172.21.15.73:0/2511685247 learned_addr learned my addr 172.21.15.73:0/2511685247 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 -- 172.21.15.73:0/2511685247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 msgr2=0x7fe6f006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6eea00640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 0x7fe6f006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 0x7fe6f006d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 -- 172.21.15.73:0/2511685247 >> v1:172.21.15.157:6789/0 conn(0x7fe6f0070840 legacy=0x7fe6f00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.063+0000 7fe6ef400640 1 -- 172.21.15.73:0/2511685247 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f006c6f0 con 0x7fe6f00b3640 2024-04-05T22:26:57.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.064+0000 7fe6eea00640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 0x7fe6f006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.065+0000 7fe6ef400640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f00b3640 0x7fe6f0070ca0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009e50 tx=0x7fe6e802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7357c430c0c587b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.065+0000 7fe6ee000640 1 -- 172.21.15.73:0/2511685247 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e802ca80 con 0x7fe6f00b3640 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.065+0000 7fe6ee000640 1 -- 172.21.15.73:0/2511685247 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e802cbe0 con 0x7fe6f00b3640 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.065+0000 7fe6ee000640 1 -- 172.21.15.73:0/2511685247 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80343f0 con 0x7fe6f00b3640 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 -- 172.21.15.73:0/2511685247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f00b3640 msgr2=0x7fe6f0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f00b3640 0x7fe6f0070ca0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009e50 tx=0x7fe6e802bee0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 -- 172.21.15.73:0/2511685247 shutdown_connections 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f00b3640 0x7fe6f0070ca0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2511685247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f0146490 0x7fe6f006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 -- 172.21.15.73:0/2511685247 >> 172.21.15.73:0/2511685247 conn(0x7fe6f00b0830 msgr2=0x7fe6f00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 -- 172.21.15.73:0/2511685247 shutdown_connections 2024-04-05T22:26:57.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.066+0000 7fe6efe00640 1 -- 172.21.15.73:0/2511685247 wait complete. 2024-04-05T22:26:57.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.068+0000 7fe6efe00640 1 Processor -- start 2024-04-05T22:26:57.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.068+0000 7fe6efe00640 1 -- start start 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f0070840 0x7fe6f01f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 0x7fe6f012b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0157d80 con 0x7fe6f0146490 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f01420e0 con 0x7fe6f0070840 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6efe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f01423c0 con 0x7fe6f00b3640 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60860/0 (socket says 172.21.15.73:60860) 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6ef400640 1 -- 172.21.15.73:0/2624786505 learned_addr learned my addr 172.21.15.73:0/2624786505 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.069+0000 7fe6eea00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f0070840 0x7fe6f01f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 0x7fe6f012b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 -- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 msgr2=0x7fe6f012b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 0x7fe6f012b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 -- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f0070840 msgr2=0x7fe6f01f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f0070840 0x7fe6f01f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 -- 172.21.15.73:0/2624786505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009b30 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 0x7fe6f012b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6ef400640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8036040 tx=0x7fe6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6e6a00640 1 -- 172.21.15.73:0/2624786505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e803e450 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6e6a00640 1 -- 172.21.15.73:0/2624786505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e803e5d0 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6e6a00640 1 -- 172.21.15.73:0/2624786505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e803e980 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.070+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f01426a0 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.071+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f0142c60 con 0x7fe6f0146490 2024-04-05T22:26:57.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.073+0000 7fe6e6a00640 1 -- 172.21.15.73:0/2624786505 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe6e802cd50 con 0x7fe6f0146490 2024-04-05T22:26:57.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.073+0000 7fe6e6a00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 0x7fe6cc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.073+0000 7fe6e6a00640 1 -- 172.21.15.73:0/2624786505 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6e80abee0 con 0x7fe6f0146490 2024-04-05T22:26:57.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.074+0000 7fe6f85dd740 1 Processor -- start 2024-04-05T22:26:57.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.074+0000 7fe6eea00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 0x7fe6cc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.074+0000 7fe6f85dd740 1 -- start start 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382360 0x563254355300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254355840 0x563254385630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632542c58c0 con 0x563254382740 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632542c3050 con 0x563254382360 2024-04-05T22:26:57.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632542be940 con 0x563254355840 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6eea00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 0x7fe6cc06d1e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000a950 tx=0x7fe6e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60868/0 (socket says 172.21.15.73:60868) 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.075+0000 7fe6e7e00640 1 -- 172.21.15.73:0/852071824 learned_addr learned my addr 172.21.15.73:0/852071824 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 -- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254355840 msgr2=0x563254385630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6ef400640 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254355840 0x563254385630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6eea00640 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382360 0x563254355300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254355840 0x563254385630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 -- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382360 msgr2=0x563254355300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382360 0x563254355300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 -- 172.21.15.73:0/852071824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632542899e0 con 0x563254382740 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8003ef0 tx=0x7fe6d80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340b35294772532e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6df400640 1 -- 172.21.15.73:0/852071824 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800db40 con 0x563254382740 2024-04-05T22:26:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6df400640 1 -- 172.21.15.73:0/852071824 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d800dcc0 con 0x563254382740 2024-04-05T22:26:57.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.076+0000 7fe6df400640 1 -- 172.21.15.73:0/852071824 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8007270 con 0x563254382740 2024-04-05T22:26:57.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 -- 172.21.15.73:0/852071824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 msgr2=0x5632543850f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8003ef0 tx=0x7fe6d80043b0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 -- 172.21.15.73:0/852071824 shutdown_connections 2024-04-05T22:26:57.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254355840 0x563254385630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563254382740 0x5632543850f0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/852071824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382360 0x563254355300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 -- 172.21.15.73:0/852071824 >> 172.21.15.73:0/852071824 conn(0x563254357570 msgr2=0x5632543a00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 -- 172.21.15.73:0/852071824 shutdown_connections 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.077+0000 7fe6f85dd740 1 -- 172.21.15.73:0/852071824 wait complete. 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.078+0000 7fe6f85dd740 1 -- start start 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.078+0000 7fe6f85dd740 1 Processor -- start 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x5632543764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254388020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563254260070 con 0x5632543aff50 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325425fd10 con 0x563254382740 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563254025f20 con 0x563254382360 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6f85dd740 10 client.0 _ll_register_callbacks cb 0x5632542b7fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:26:57.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60884/0 (socket says 172.21.15.73:60884) 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 -- 172.21.15.73:0/464846102 learned_addr learned my addr 172.21.15.73:0/464846102 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6eea00640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x5632543764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254388020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 -- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 msgr2=0x5632543764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x5632543764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 -- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 msgr2=0x563254388020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254388020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.079+0000 7fe6ef400640 1 -- 172.21.15.73:0/464846102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d8003af0 con 0x5632543aff50 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6ef400640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe6e807aa20 tx=0x7fe6e807ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53a68fda69ec873 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e807a410 con 0x5632543aff50 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632542e8ae0 con 0x5632543aff50 2024-04-05T22:26:57.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6e8082bd0 con 0x5632543aff50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e8082d90 con 0x5632543aff50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6f85dd740 10 client.7326 fetch_fsmap learned FSMap version 50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5632542bcfc0 con 0x5632543aff50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632542e8ae0 con 0x5632543aff50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe6e804f020 con 0x5632543aff50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6f85dd740 10 client.7326 fetch_fsmap finished waiting for FSMap version 50 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.080+0000 7fe6f85dd740 10 client.7326 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6f85dd740 20 client.7326 populate_metadata read hostname 'smithi073' 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x5632543981f0 0x5632543bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6f85dd740 4 client.7326 mds_command: new command op to 7176 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 --> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563253fb23c0 con 0x5632543981f0 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6eea00640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x5632543981f0 0x5632543bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6eea00640 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x5632543981f0 0x5632543bd800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb04f7777a16fc3 server_cookie=d083324c68ad15fc in_seq=0 out_seq=0 2024-04-05T22:26:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.081+0000 7fe6d3e00640 10 client.7326 ms_handle_connect on v2:172.21.15.73:6832/3038019968 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6d3e00640 1 -- 172.21.15.73:0/464846102 <== mds.0 v2:172.21.15.73:6832/3038019968 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563253fb23c0 con 0x5632543981f0 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6d3e00640 10 client.7326 handle_command_reply: tid=0 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 client.7326 shutdown 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 msgr2=0x5632543998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe6e807aa20 tx=0x7fe6e807ad20 comp rx=0 tx=0).stop 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 shutdown_connections 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x5632543981f0 0x5632543bd800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x5632543998f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254388020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/464846102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x5632543764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6d3e00640 0 client.7326 ms_handle_reset on v2:172.21.15.73:6832/3038019968 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.082+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 >> 172.21.15.73:0/464846102 conn(0x563254355840 msgr2=0x5632543a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.083+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 shutdown_connections 2024-04-05T22:26:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.083+0000 7fe6f85dd740 1 -- 172.21.15.73:0/464846102 wait complete. 2024-04-05T22:26:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.083+0000 7fe6f85dd740 20 client.7326 trim_cache size 0 max 16384 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 Processor -- start 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 -- start start 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x563254354400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254354940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632543afa80 con 0x5632543aff50 2024-04-05T22:26:57.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325435b210 con 0x563254382740 2024-04-05T22:26:57.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.099+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563254382b20 con 0x563254382360 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60892/0 (socket says 172.21.15.73:60892) 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 -- 172.21.15.73:0/1873394684 learned_addr learned my addr 172.21.15.73:0/1873394684 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 -- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 msgr2=0x563254354400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x563254354400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 -- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 msgr2=0x563254354940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254354940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 -- 172.21.15.73:0/1873394684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632542f1f80 con 0x5632543aff50 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6ef400640 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ac1c0 tx=0x7fe6e808f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1ba174779d025d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6dfe00640 1 -- 172.21.15.73:0/1873394684 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e807a410 con 0x5632543aff50 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6dfe00640 1 -- 172.21.15.73:0/1873394684 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8082bd0 con 0x5632543aff50 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.100+0000 7fe6dfe00640 1 -- 172.21.15.73:0/1873394684 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8076b10 con 0x5632543aff50 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 -- 172.21.15.73:0/1873394684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 msgr2=0x563254398f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ac1c0 tx=0x7fe6e808f000 comp rx=0 tx=0).stop 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 -- 172.21.15.73:0/1873394684 shutdown_connections 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x563254354940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/1873394684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x563254354400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 -- 172.21.15.73:0/1873394684 >> 172.21.15.73:0/1873394684 conn(0x563254355840 msgr2=0x563254350b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 -- 172.21.15.73:0/1873394684 shutdown_connections 2024-04-05T22:26:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.101+0000 7fe6f85dd740 1 -- 172.21.15.73:0/1873394684 wait complete. 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.102+0000 7fe6f85dd740 1 -- start start 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.102+0000 7fe6f85dd740 1 Processor -- start 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x56325439fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x56325440fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563254381240 con 0x5632543aff50 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563254414070 con 0x563254382740 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563254395e90 con 0x563254382360 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6f85dd740 10 client.0 _ll_register_callbacks cb 0x7fe6e80095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60894/0 (socket says 172.21.15.73:60894) 2024-04-05T22:26:57.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6ef400640 1 -- 172.21.15.73:0/625071967 learned_addr learned my addr 172.21.15.73:0/625071967 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.103+0000 7fe6eea00640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x56325439fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 -- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 msgr2=0x56325439fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x56325439fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6e7e00640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x56325440fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 -- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 msgr2=0x56325440fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x56325440fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 -- 172.21.15.73:0/625071967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632542bcfc0 con 0x5632543aff50 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6eea00640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x56325439fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6ef400640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009e50 tx=0x7fe6e8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fb6bc19ad4f4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e808eab0 con 0x5632543aff50 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6d800dcc0 con 0x5632543aff50 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6e808ec10 con 0x5632543aff50 2024-04-05T22:26:57.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e808edd0 con 0x5632543aff50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6f85dd740 10 client.7332 fetch_fsmap learned FSMap version 50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.104+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x56325425f280 con 0x5632543aff50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d800dcc0 con 0x5632543aff50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe6e8084070 con 0x5632543aff50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 10 client.7332 fetch_fsmap finished waiting for FSMap version 50 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 10 client.7332 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 20 client.7332 populate_metadata read hostname 'smithi073' 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x563254385680 0x563254354400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 4 client.7332 mds_command: new command op to 7176 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 --> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563253fc36c0 con 0x563254385680 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6eea00640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x563254385680 0x563254354400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6d3e00640 10 client.7332 ms_handle_connect on v2:172.21.15.73:6832/3038019968 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.105+0000 7fe6eea00640 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x563254385680 0x563254354400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2281b24a593a34 server_cookie=ad483f6e0f5d729 in_seq=0 out_seq=0 2024-04-05T22:26:57.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6d3e00640 1 -- 172.21.15.73:0/625071967 <== mds.0 v2:172.21.15.73:6832/3038019968 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563253fc36c0 con 0x563254385680 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6d3e00640 10 client.7332 handle_command_reply: tid=0 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 client.7332 shutdown 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 msgr2=0x563254398f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009e50 tx=0x7fe6e8009040 comp rx=0 tx=0).stop 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 shutdown_connections 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:6832/3038019968,v1:172.21.15.73:6833/3038019968] conn(0x563254385680 0x563254354400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632543aff50 0x563254398f50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563254382740 0x56325440fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6d3e00640 0 client.7332 ms_handle_reset on v2:172.21.15.73:6832/3038019968 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 --2- 172.21.15.73:0/625071967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563254382360 0x56325439fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 >> 172.21.15.73:0/625071967 conn(0x563254355840 msgr2=0x563254396ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 shutdown_connections 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 1 -- 172.21.15.73:0/625071967 wait complete. 2024-04-05T22:26:57.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.106+0000 7fe6f85dd740 20 client.7332 trim_cache size 0 max 16384 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 7176, 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.6754337240000003, 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 50, 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 87, 2024-04-05T22:26:57.110 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 87, 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 11.615172185 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 msgr2=0x7fe6cc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 0x7fe6cc06d1e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000a950 tx=0x7fe6e0008040 comp rx=0 tx=0).stop 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 msgr2=0x7fe6f0143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8036040 tx=0x7fe6e8004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 shutdown_connections 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6cc06ad20 0x7fe6cc06d1e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f0146490 0x7fe6f0143180 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6f00b3640 0x7fe6f012b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 --2- 172.21.15.73:0/2624786505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6f0070840 0x7fe6f01f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 >> 172.21.15.73:0/2624786505 conn(0x7fe6f00b0830 msgr2=0x7fe6f015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 shutdown_connections 2024-04-05T22:26:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.107+0000 7fe6efe00640 1 -- 172.21.15.73:0/2624786505 wait complete. 2024-04-05T22:26:57.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:26:57.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.305+0000 7fecc7000640 1 Processor -- start 2024-04-05T22:26:57.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.305+0000 7fecc7000640 1 -- start start 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.306+0000 7fecc7000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecc01587d0 0x7fecc00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.306+0000 7fecc7000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecc00ccd20 0x7fecc006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecc7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc006d0c0 0x7fecc015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecc7000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc006f5b0 con 0x7fecc01587d0 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecc7000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc0070410 con 0x7fecc00ccd20 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecc7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0158bb0 con 0x7fecc006d0c0 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecc01587d0 0x7fecc00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecc01587d0 0x7fecc00cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60898/0 (socket says 172.21.15.73:60898) 2024-04-05T22:26:57.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 -- 172.21.15.73:0/3136230709 learned_addr learned my addr 172.21.15.73:0/3136230709 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 -- 172.21.15.73:0/3136230709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc006d0c0 msgr2=0x7fecc015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 --2- 172.21.15.73:0/3136230709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc006d0c0 0x7fecc015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 -- 172.21.15.73:0/3136230709 >> v2:172.21.15.157:3300/0 conn(0x7fecc00ccd20 msgr2=0x7fecc006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 --2- 172.21.15.73:0/3136230709 >> v2:172.21.15.157:3300/0 conn(0x7fecc00ccd20 0x7fecc006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.307+0000 7fecbfe00640 1 -- 172.21.15.73:0/3136230709 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc006f800 con 0x7fecc01587d0 2024-04-05T22:26:57.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.308+0000 7fecbfe00640 1 --2- 172.21.15.73:0/3136230709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00cc7e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fecc006b560 tx=0x7fecb002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3530f0e0eb697de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecbea00640 1 -- 172.21.15.73:0/3136230709 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0031860 con 0x7fecc01587d0 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecbea00640 1 -- 172.21.15.73:0/3136230709 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00319c0 con 0x7fecc01587d0 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecbea00640 1 -- 172.21.15.73:0/3136230709 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0031d50 con 0x7fecc01587d0 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 -- 172.21.15.73:0/3136230709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 msgr2=0x7fecc00cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 --2- 172.21.15.73:0/3136230709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00cc7e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fecc006b560 tx=0x7fecb002b900 comp rx=0 tx=0).stop 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 -- 172.21.15.73:0/3136230709 shutdown_connections 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 --2- 172.21.15.73:0/3136230709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc006d0c0 0x7fecc015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 --2- 172.21.15.73:0/3136230709 >> v2:172.21.15.157:3300/0 conn(0x7fecc00ccd20 0x7fecc006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 --2- 172.21.15.73:0/3136230709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00cc7e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 -- 172.21.15.73:0/3136230709 >> 172.21.15.73:0/3136230709 conn(0x7fecc00ce8e0 msgr2=0x7fecc00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 -- 172.21.15.73:0/3136230709 shutdown_connections 2024-04-05T22:26:57.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.309+0000 7fecc7000640 1 -- 172.21.15.73:0/3136230709 wait complete. 2024-04-05T22:26:57.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.311+0000 7fecc7000640 1 Processor -- start 2024-04-05T22:26:57.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.311+0000 7fecc7000640 1 -- start start 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc006d0c0 0x7fecc01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc0157b90 0x7fecc00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00ad340 con 0x7fecc01587d0 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00a9290 con 0x7fecc006d0c0 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecc7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00a9570 con 0x7fecc0157b90 2024-04-05T22:26:57.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.312+0000 7fecbf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60900/0 (socket says 172.21.15.73:60900) 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 -- 172.21.15.73:0/38273883 learned_addr learned my addr 172.21.15.73:0/38273883 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 -- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc0157b90 msgr2=0x7fecc00ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc0157b90 0x7fecc00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 -- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc006d0c0 msgr2=0x7fecc01549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbfe00640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc006d0c0 0x7fecc01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc006d0c0 0x7fecc01549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 -- 172.21.15.73:0/38273883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00094d0 con 0x7fecc01587d0 2024-04-05T22:26:57.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.313+0000 7fecbf400640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b4a0 tx=0x7fecac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.314+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac010610 con 0x7fecc01587d0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.314+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac010770 con 0x7fecc01587d0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.314+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc00a9850 con 0x7fecc01587d0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.314+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac010b20 con 0x7fecc01587d0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.314+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc00a9e10 con 0x7fecc01587d0 2024-04-05T22:26:57.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.315+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc0066af0 con 0x7fecc01587d0 2024-04-05T22:26:57.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.316+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fecac010c80 con 0x7fecc01587d0 2024-04-05T22:26:57.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.317+0000 7fecb7e00640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 0x7feca006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:57.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.317+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecac089050 con 0x7fecc01587d0 2024-04-05T22:26:57.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.318+0000 7fecbfe00640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 0x7feca006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:57.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.319+0000 7fecbfe00640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 0x7feca006d1e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecb0036000 tx=0x7fecb003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.322+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fecac0541b0 con 0x7fecc01587d0 2024-04-05T22:26:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.553+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fecc0068160 con 0x7fecc01587d0 2024-04-05T22:26:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.554+0000 7fecb7e00640 1 -- 172.21.15.73:0/38273883 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7fecac014060 con 0x7fecc01587d0 2024-04-05T22:26:57.554 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:26:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 msgr2=0x7feca006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 0x7feca006d1e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecb0036000 tx=0x7fecb003c040 comp rx=0 tx=0).stop 2024-04-05T22:26:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 msgr2=0x7fecc00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b4a0 tx=0x7fecac00b960 comp rx=0 tx=0).stop 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 shutdown_connections 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feca006ad20 0x7feca006d1e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecc0157b90 0x7fecc00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecc01587d0 0x7fecc00ac8c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 --2- 172.21.15.73:0/38273883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc006d0c0 0x7fecc01549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 >> 172.21.15.73:0/38273883 conn(0x7fecc00ce8e0 msgr2=0x7fecc015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 shutdown_connections 2024-04-05T22:26:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:57.557+0000 7fecc7000640 1 -- 172.21.15.73:0/38273883 wait complete. 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:26:57.574 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:57.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.575 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:26:57.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.628 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:26:57.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:57.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.630 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:26:57.713 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:26:57.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.717 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:26:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:26:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:26:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:26:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:26:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:26:57.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:26:57.768 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:26:57.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.952 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:26:57.997 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:26:57.997 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:26:57.998 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:57.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:57.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:26:58.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:58.137 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:58.190 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:26:58.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:58.190 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:58.241 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:26:58.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:58.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:58.371 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:58.371 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:58.410 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:58.410 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:58.410 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:26:58.410 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:26:58.410 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:26:58.415 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:26:58.415 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:26:58.415 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:26:58.415 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:26:58.415 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:26:58.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:26:58.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:58.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:26:58.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:58.506 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:26:58.568 INFO:teuthology.orchestra.run.smithi073.stdout:client5183 2024-04-05T22:26:58.569 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:26:58.569 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:26:58.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:26:58.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.871+0000 7f4604600640 1 Processor -- start 2024-04-05T22:26:58.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.871+0000 7f4604600640 1 -- start start 2024-04-05T22:26:58.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.872+0000 7f4604600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45fc15ef60 0x7f45fc06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:58.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.872+0000 7f4604600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc06f5b0 con 0x7f45fc06cb80 2024-04-05T22:26:58.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.872+0000 7f4604600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc070410 con 0x7f45fc1587d0 2024-04-05T22:26:58.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.872+0000 7f4604600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc06d500 con 0x7f45fc15ef60 2024-04-05T22:26:58.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.873+0000 7f4601c00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f45fc1587d0 0x7f45fc0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:56860/0 (socket says 172.21.15.73:56860) 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.873+0000 7f4601c00640 1 -- 172.21.15.73:0/3374652099 learned_addr learned my addr 172.21.15.73:0/3374652099 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.873+0000 7f4601200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f45fc06cb80 0x7f45fc15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37022/0 (socket says 172.21.15.73:37022) 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.873+0000 7f4602600640 1 --2- 172.21.15.73:0/3374652099 >> v2:172.21.15.73:3300/0 conn(0x7f45fc15ef60 0x7f45fc06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.873+0000 7f45fbe00640 1 -- 172.21.15.73:0/3374652099 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157534918 0 0) 0x7f45fc06f5b0 con 0x7f45fc06cb80 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.874+0000 7f4602600640 1 -- 172.21.15.73:0/3374652099 >> v1:172.21.15.157:6789/0 conn(0x7f45fc1587d0 legacy=0x7f45fc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.874+0000 7f4602600640 1 -- 172.21.15.73:0/3374652099 >> v1:172.21.15.73:6789/0 conn(0x7f45fc06cb80 legacy=0x7f45fc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.874+0000 7f4602600640 1 -- 172.21.15.73:0/3374652099 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0ccef0 con 0x7f45fc15ef60 2024-04-05T22:26:58.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f4602600640 1 --2- 172.21.15.73:0/3374652099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc06cfc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009d90 tx=0x7f45f402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5da7c52bef09e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:58.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f45fbe00640 1 -- 172.21.15.73:0/3374652099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f402ccf0 con 0x7f45fc15ef60 2024-04-05T22:26:58.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f45fbe00640 1 -- 172.21.15.73:0/3374652099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f402ce50 con 0x7f45fc15ef60 2024-04-05T22:26:58.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f45fbe00640 1 -- 172.21.15.73:0/3374652099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4034660 con 0x7f45fc15ef60 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f4604600640 1 -- 172.21.15.73:0/3374652099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 msgr2=0x7f45fc06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.875+0000 7f4604600640 1 --2- 172.21.15.73:0/3374652099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc06cfc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009d90 tx=0x7f45f402bf40 comp rx=0 tx=0).stop 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.876+0000 7f4604600640 1 -- 172.21.15.73:0/3374652099 shutdown_connections 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.876+0000 7f4604600640 1 --2- 172.21.15.73:0/3374652099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc06cfc0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.876+0000 7f4604600640 1 -- 172.21.15.73:0/3374652099 >> 172.21.15.73:0/3374652099 conn(0x7f45fc0ce8e0 msgr2=0x7f45fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.876+0000 7f4604600640 1 -- 172.21.15.73:0/3374652099 shutdown_connections 2024-04-05T22:26:58.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.876+0000 7f4604600640 1 -- 172.21.15.73:0/3374652099 wait complete. 2024-04-05T22:26:58.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.877+0000 7f4604600640 1 Processor -- start 2024-04-05T22:26:58.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 -- start start 2024-04-05T22:26:58.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45fc06cb80 0x7f45fc1f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:58.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45fc1587d0 0x7f45fc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:58.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:58.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc146e60 con 0x7f45fc15ef60 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc0b2680 con 0x7f45fc06cb80 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.878+0000 7f4604600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc0b2960 con 0x7f45fc1587d0 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60916/0 (socket says 172.21.15.73:60916) 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 -- 172.21.15.73:0/42734137 learned_addr learned my addr 172.21.15.73:0/42734137 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 -- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45fc1587d0 msgr2=0x7f45fc0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45fc1587d0 0x7f45fc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 -- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45fc06cb80 msgr2=0x7f45fc1f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45fc06cb80 0x7f45fc1f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.879+0000 7f4602600640 1 -- 172.21.15.73:0/42734137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4009a70 con 0x7f45fc15ef60 2024-04-05T22:26:58.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f4602600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f45f402dfd0 tx=0x7f45f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f403d600 con 0x7f45fc15ef60 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f403d760 con 0x7f45fc15ef60 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f403daf0 con 0x7f45fc15ef60 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc0b2c40 con 0x7f45fc15ef60 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.880+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc0b3200 con 0x7f45fc15ef60 2024-04-05T22:26:58.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.881+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45fc066af0 con 0x7f45fc15ef60 2024-04-05T22:26:58.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.882+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f45f403b040 con 0x7f45fc15ef60 2024-04-05T22:26:58.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.883+0000 7f45f9600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 0x7f45dc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:26:58.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.883+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f40ab5b0 con 0x7f45fc15ef60 2024-04-05T22:26:58.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.884+0000 7f4601c00640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 0x7f45dc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:26:58.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.885+0000 7f4601c00640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 0x7f45dc06d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45f00015f0 tx=0x7f45f0007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:26:58.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:58.888+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f45f40765e0 con 0x7f45fc15ef60 2024-04-05T22:26:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.126+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f45fc0b33f0 con 0x7f45fc15ef60 2024-04-05T22:26:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.127+0000 7f45f9600640 1 -- 172.21.15.73:0/42734137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f45f40a9e60 con 0x7f45fc15ef60 2024-04-05T22:26:59.126 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:26:59.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 msgr2=0x7f45dc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 0x7f45dc06d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45f00015f0 tx=0x7f45f0007a40 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 msgr2=0x7f45fc154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f45f402dfd0 tx=0x7f45f4004a40 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 shutdown_connections 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45dc06ad20 0x7f45dc06d1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45fc15ef60 0x7f45fc154620 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45fc1587d0 0x7f45fc0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 --2- 172.21.15.73:0/42734137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45fc06cb80 0x7f45fc1f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 >> 172.21.15.73:0/42734137 conn(0x7f45fc0ce8e0 msgr2=0x7f45fc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 shutdown_connections 2024-04-05T22:26:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:26:59.129+0000 7f4604600640 1 -- 172.21.15.73:0/42734137 wait complete. 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:26:59.148 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:59.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.148 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.207 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42268sec preferred_lft 42268sec 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.208 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:26:59.209 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:26:59.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.210 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:26:59.297 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:26:59.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.301 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:26:59.353 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:26:59.353 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:26:59.353 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:26:59.353 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:26:59.353 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:26:59.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.354 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:26:59.354 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:26:59.354 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:26:59.354 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:26:59.354 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:26:59.355 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:26:59.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.531 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:26:59.580 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:26:59.580 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:26:59.580 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:59.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:26:59.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:59.662 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:59.713 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:26:59.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:26:59.714 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:26:59.763 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:26:59.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:59.900 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:59.900 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:59.936 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:26:59.936 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:26:59.936 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:26:59.936 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:26:59.936 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:26:59.939 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:26:59.939 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:26:59.939 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:26:59.939 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:26:59.939 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:26:59.945 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:26:59.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:26:59.945 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:00.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:00.027 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:27:00.085 INFO:teuthology.orchestra.run.smithi157.stdout:client5569 2024-04-05T22:27:00.085 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:00.085 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:00.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:27:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.362+0000 7f31d7400640 1 Processor -- start 2024-04-05T22:27:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.363+0000 7f31d7400640 1 -- start start 2024-04-05T22:27:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.364+0000 7f31d7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31d00b3ab0 0x7f31d0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.364+0000 7f31d7400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31d00ad720 0x7f31d0164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.364+0000 7f31d7400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d01620e0 con 0x7f31d00cecd0 2024-04-05T22:27:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.364+0000 7f31d7400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d00ae370 con 0x7f31d00b3ab0 2024-04-05T22:27:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.364+0000 7f31d7400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d00ae090 con 0x7f31d00ad720 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31d00b3ab0 0x7f31d0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31d5400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31d00ad720 0x7f31d0164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31d00b3ab0 0x7f31d0162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60924/0 (socket says 172.21.15.73:60924) 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 -- 172.21.15.73:0/2284842704 learned_addr learned my addr 172.21.15.73:0/2284842704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 -- 172.21.15.73:0/2284842704 >> v2:172.21.15.157:3300/0 conn(0x7f31d00ad720 msgr2=0x7f31d0164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 --2- 172.21.15.73:0/2284842704 >> v2:172.21.15.157:3300/0 conn(0x7f31d00ad720 0x7f31d0164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 -- 172.21.15.73:0/2284842704 >> v1:172.21.15.157:6789/0 conn(0x7f31d00cecd0 legacy=0x7f31d00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.365+0000 7f31c7e00640 1 -- 172.21.15.73:0/2284842704 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0161a90 con 0x7f31d00b3ab0 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31c7e00640 1 --2- 172.21.15.73:0/2284842704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d0162570 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f31d00b05e0 tx=0x7f31c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9f449b4625c676 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31cfe00640 1 -- 172.21.15.73:0/2284842704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0030860 con 0x7f31d00b3ab0 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31cfe00640 1 -- 172.21.15.73:0/2284842704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c00309c0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31cfe00640 1 -- 172.21.15.73:0/2284842704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0030d50 con 0x7f31d00b3ab0 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31d7400640 1 -- 172.21.15.73:0/2284842704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 msgr2=0x7f31d0162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.367+0000 7f31d7400640 1 --2- 172.21.15.73:0/2284842704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d0162570 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f31d00b05e0 tx=0x7f31c002b9a0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 -- 172.21.15.73:0/2284842704 shutdown_connections 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 --2- 172.21.15.73:0/2284842704 >> v2:172.21.15.157:3300/0 conn(0x7f31d00ad720 0x7f31d0164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 --2- 172.21.15.73:0/2284842704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d0162570 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 -- 172.21.15.73:0/2284842704 >> 172.21.15.73:0/2284842704 conn(0x7f31d006a0d0 msgr2=0x7f31d006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 -- 172.21.15.73:0/2284842704 shutdown_connections 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.368+0000 7f31d7400640 1 -- 172.21.15.73:0/2284842704 wait complete. 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.369+0000 7f31d7400640 1 Processor -- start 2024-04-05T22:27:00.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.370+0000 7f31d7400640 1 -- start start 2024-04-05T22:27:00.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.370+0000 7f31d7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31d00ad720 0x7f31d0149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.370+0000 7f31d7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31d7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d00cecd0 0x7f31d0147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31d7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0148f60 con 0x7f31d00b3ab0 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31d7400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d00ad580 con 0x7f31d00cecd0 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31d7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0147680 con 0x7f31d00ad720 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60926/0 (socket says 172.21.15.73:60926) 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 -- 172.21.15.73:0/449323137 learned_addr learned my addr 172.21.15.73:0/449323137 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 -- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31d00ad720 msgr2=0x7f31d0149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31d00ad720 0x7f31d0149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 -- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d00cecd0 msgr2=0x7f31d0147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31d5400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d00cecd0 0x7f31d0147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d00cecd0 0x7f31d0147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:00.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.371+0000 7f31c7e00640 1 -- 172.21.15.73:0/449323137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00094d0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31c7e00640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f31c002b9e0 tx=0x7f31c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0039d60 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0043400 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c00437b0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d01482e0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.372+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01488a0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.373+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d01490a0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.374+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f31c0043910 con 0x7f31d00b3ab0 2024-04-05T22:27:00.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.375+0000 7f31cd600640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 0x7f31ac06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:00.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.375+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31c003e070 con 0x7f31d00b3ab0 2024-04-05T22:27:00.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.376+0000 7f31d4a00640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 0x7f31ac06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:00.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.377+0000 7f31d4a00640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 0x7f31ac06d210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000f60 tx=0x7f31b80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:00.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.380+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f31d01490a0 con 0x7f31d00b3ab0 2024-04-05T22:27:00.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:00.617+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31d01490a0 con 0x7f31d00b3ab0 2024-04-05T22:27:01.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.607+0000 7f31cd600640 1 -- 172.21.15.73:0/449323137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 136+0+0 (secure 0 0 0) 0x7f31d01490a0 con 0x7f31d00b3ab0 2024-04-05T22:27:01.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 msgr2=0x7f31ac06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 0x7f31ac06d210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000f60 tx=0x7f31b80079b0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 msgr2=0x7f31d00ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f31c002b9e0 tx=0x7f31c0004730 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 shutdown_connections 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f31ac06ad50 0x7f31ac06d210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d00cecd0 0x7f31d0147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d00b3ab0 0x7f31d00ad040 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 --2- 172.21.15.73:0/449323137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31d00ad720 0x7f31d0149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 >> 172.21.15.73:0/449323137 conn(0x7f31d006a0d0 msgr2=0x7f31d0164d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 shutdown_connections 2024-04-05T22:27:01.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.611+0000 7f31d7400640 1 -- 172.21.15.73:0/449323137 wait complete. 2024-04-05T22:27:01.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:27:01.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.836+0000 7fc1c9200640 1 Processor -- start 2024-04-05T22:27:01.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.836+0000 7fc1c9200640 1 -- start start 2024-04-05T22:27:01.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.837+0000 7fc1c9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1c40caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.837+0000 7fc1c9200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40cb870 con 0x7fc1c40cb400 2024-04-05T22:27:01.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.837+0000 7fc1c9200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40cfdb0 con 0x7fc1c415cba0 2024-04-05T22:27:01.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.837+0000 7fc1c9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40cfad0 con 0x7fc1c40cf4e0 2024-04-05T22:27:01.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc1c40cb400 0x7fc1c4160850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37028/0 (socket says 172.21.15.73:37028) 2024-04-05T22:27:01.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c3e00640 1 -- 172.21.15.73:0/2449290504 learned_addr learned my addr 172.21.15.73:0/2449290504 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768737505 0 0) 0x7fc1c40cb870 con 0x7fc1c40cb400 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2a00640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1c40caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a8003520 con 0x7fc1c40cb400 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1616974683 0 0) 0x7fc1a8003520 con 0x7fc1c40cb400 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 msgr2=0x7fc1c40caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1c40caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 >> v1:172.21.15.157:6789/0 conn(0x7fc1c415cba0 legacy=0x7fc1c406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406d870 con 0x7fc1c40cb400 2024-04-05T22:27:01.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc1bc001670 con 0x7fc1c40cb400 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.838+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 >> v1:172.21.15.73:6789/0 conn(0x7fc1c40cb400 legacy=0x7fc1c4160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a80047c0 0x7fc1a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40cfad0 con 0x7fc1a80039a0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a800bab0 con 0x7fc1a80047c0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a800bc70 con 0x7fc1c40cf4e0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c2a00640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a80047c0 0x7fc1a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3e00640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 -- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 msgr2=0x7fc1a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 -- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a80047c0 msgr2=0x7fc1a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a80047c0 0x7fc1a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3400640 1 -- 172.21.15.73:0/2449290504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8003520 con 0x7fc1a80039a0 2024-04-05T22:27:01.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.839+0000 7fc1c3e00640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40cf4e0 0x7fc1a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:01.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1a8003dc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4001750 tx=0x7fc1a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a83a01e4534496 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:01.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40052d0 con 0x7fc1a80039a0 2024-04-05T22:27:01.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b402e050 con 0x7fc1a80039a0 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c2000640 1 -- 172.21.15.73:0/2449290504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40035f0 con 0x7fc1a80039a0 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c9200640 1 -- 172.21.15.73:0/2449290504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 msgr2=0x7fc1a8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2449290504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1a8003dc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4001750 tx=0x7fc1a80057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c3e00640 1 -- 172.21.15.73:0/2449290504 reap_dead start 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c9200640 1 -- 172.21.15.73:0/2449290504 shutdown_connections 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.841+0000 7fc1c9200640 1 -- 172.21.15.73:0/2449290504 >> 172.21.15.73:0/2449290504 conn(0x7fc1c40b23c0 msgr2=0x7fc1c40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.842+0000 7fc1c9200640 1 -- 172.21.15.73:0/2449290504 shutdown_connections 2024-04-05T22:27:01.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.842+0000 7fc1c9200640 1 -- 172.21.15.73:0/2449290504 wait complete. 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.843+0000 7fc1c9200640 1 Processor -- start 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.843+0000 7fc1c9200640 1 -- start start 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.844+0000 7fc1c9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.844+0000 7fc1c9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c415d870 0x7fc1c40ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c40adbe0 0x7fc1c40ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415b270 con 0x7fc1a80039a0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40b4160 con 0x7fc1c40adbe0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40b4a00 con 0x7fc1c415d870 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60932/0 (socket says 172.21.15.73:60932) 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 -- 172.21.15.73:0/2437217986 learned_addr learned my addr 172.21.15.73:0/2437217986 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c2a00640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c415d870 0x7fc1c40ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 -- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c415d870 msgr2=0x7fc1c40ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c415d870 0x7fc1c40ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 -- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c40adbe0 msgr2=0x7fc1c40ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3e00640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c40adbe0 0x7fc1c40ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c40adbe0 0x7fc1c40ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.845+0000 7fc1c3400640 1 -- 172.21.15.73:0/2437217986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c41521d0 con 0x7fc1a80039a0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1c3400640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4008fd0 tx=0x7fc1b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:01.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40076c0 con 0x7fc1a80039a0 2024-04-05T22:27:01.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4007a30 con 0x7fc1a80039a0 2024-04-05T22:27:01.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4006c00 con 0x7fc1a80039a0 2024-04-05T22:27:01.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c415ddc0 con 0x7fc1a80039a0 2024-04-05T22:27:01.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.846+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40b4690 con 0x7fc1a80039a0 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.847+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40b4820 con 0x7fc1a80039a0 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.848+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc1b40035f0 con 0x7fc1a80039a0 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.849+0000 7fc1bb400640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 0x7fc1a006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.849+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1b40ab710 con 0x7fc1a80039a0 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.849+0000 7fc1c2a00640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 0x7fc1a006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:01.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.851+0000 7fc1c2a00640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 0x7fc1a006d1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40d18f0 tx=0x7fc1ac00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:01.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:01.853+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc1b407a0d0 con 0x7fc1a80039a0 2024-04-05T22:27:02.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.073+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1c40b31a0 con 0x7fc1a006ad20 2024-04-05T22:27:02.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.077+0000 7fc1bb400640 1 -- 172.21.15.73:0/2437217986 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc1ac003ab0 con 0x7fc1a006ad20 2024-04-05T22:27:02.077 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:27:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:27:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:02.077 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 msgr2=0x7fc1a006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 0x7fc1a006d1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40d18f0 tx=0x7fc1ac00b040 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 msgr2=0x7fc1c4151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4008fd0 tx=0x7fc1b4001120 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 shutdown_connections 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc1a006ad20 0x7fc1a006d1e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c40adbe0 0x7fc1c40ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c415d870 0x7fc1c40ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 --2- 172.21.15.73:0/2437217986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a80039a0 0x7fc1c4151c90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 >> 172.21.15.73:0/2437217986 conn(0x7fc1c40b23c0 msgr2=0x7fc1c40b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 shutdown_connections 2024-04-05T22:27:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.080+0000 7fc1c9200640 1 -- 172.21.15.73:0/2437217986 wait complete. 2024-04-05T22:27:02.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:27:02.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.310+0000 7fcecc000640 1 Processor -- start 2024-04-05T22:27:02.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.311+0000 7fcecc000640 1 -- start start 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec41599f0 0x7fcec4070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcec40add00 0x7fcec406c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 0x7fcec415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec406d010 con 0x7fcec40add00 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec40b3890 con 0x7fcec406cb80 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcecc000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40b27f0 con 0x7fcec41599f0 2024-04-05T22:27:02.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.312+0000 7fcec8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcec40add00 0x7fcec406c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcec40add00 0x7fcec406c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60948/0 (socket says 172.21.15.73:60948) 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 -- 172.21.15.73:0/2483343678 learned_addr learned my addr 172.21.15.73:0/2483343678 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec9600640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec41599f0 0x7fcec4070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fceca000640 1 --2- 172.21.15.73:0/2483343678 >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 0x7fcec415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 -- 172.21.15.73:0/2483343678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec41599f0 msgr2=0x7fcec4070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec41599f0 0x7fcec4070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 -- 172.21.15.73:0/2483343678 >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 msgr2=0x7fcec415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 --2- 172.21.15.73:0/2483343678 >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 0x7fcec415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fcec8c00640 1 -- 172.21.15.73:0/2483343678 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec40b2ad0 con 0x7fcec40add00 2024-04-05T22:27:02.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.313+0000 7fceca000640 1 --2- 172.21.15.73:0/2483343678 >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 0x7fcec415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:02.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.314+0000 7fcec8c00640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec40add00 0x7fcec406c640 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcec40ae450 tx=0x7fceb002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0096b49b4a8cac server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.314+0000 7fcec3e00640 1 -- 172.21.15.73:0/2483343678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0031870 con 0x7fcec40add00 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.314+0000 7fcec3e00640 1 -- 172.21.15.73:0/2483343678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb00319d0 con 0x7fcec40add00 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.314+0000 7fcec3e00640 1 -- 172.21.15.73:0/2483343678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0031d60 con 0x7fcec40add00 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 -- 172.21.15.73:0/2483343678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec40add00 msgr2=0x7fcec406c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec40add00 0x7fcec406c640 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcec40ae450 tx=0x7fceb002b910 comp rx=0 tx=0).stop 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 -- 172.21.15.73:0/2483343678 shutdown_connections 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec40add00 0x7fcec406c640 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 --2- 172.21.15.73:0/2483343678 >> v2:172.21.15.157:3300/0 conn(0x7fcec406cb80 0x7fcec415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 --2- 172.21.15.73:0/2483343678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec41599f0 0x7fcec4070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 -- 172.21.15.73:0/2483343678 >> 172.21.15.73:0/2483343678 conn(0x7fcec4069cb0 msgr2=0x7fcec406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 -- 172.21.15.73:0/2483343678 shutdown_connections 2024-04-05T22:27:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.315+0000 7fcecc000640 1 -- 172.21.15.73:0/2483343678 wait complete. 2024-04-05T22:27:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.317+0000 7fcecc000640 1 Processor -- start 2024-04-05T22:27:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.317+0000 7fcecc000640 1 -- start start 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 0x7fcec414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 0x7fcec41f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4150430 con 0x7fcec406cb80 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec414fb90 con 0x7fcec41599f0 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcecc000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec414fe70 con 0x7fcec40add00 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcec9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcec9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60956/0 (socket says 172.21.15.73:60956) 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.318+0000 7fcec9600640 1 -- 172.21.15.73:0/768185700 learned_addr learned my addr 172.21.15.73:0/768185700 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fceca000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 0x7fcec41f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec8c00640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 0x7fcec414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 -- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 msgr2=0x7fcec414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 0x7fcec414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 -- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 msgr2=0x7fcec41f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 0x7fcec41f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 -- 172.21.15.73:0/768185700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00094d0 con 0x7fcec406cb80 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fceca000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 0x7fcec41f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec8c00640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 0x7fcec414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:02.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec9600640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d800 tx=0x7fceb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400b6c0 con 0x7fcec406cb80 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.319+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb400b820 con 0x7fcec406cb80 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.320+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec4150150 con 0x7fcec406cb80 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.320+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400bbb0 con 0x7fcec406cb80 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.320+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec414f810 con 0x7fcec406cb80 2024-04-05T22:27:02.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.321+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec4068160 con 0x7fcec406cb80 2024-04-05T22:27:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.322+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fceb400bd10 con 0x7fcec406cb80 2024-04-05T22:27:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.323+0000 7fcec1600640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 0x7fcea406d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.323+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fceb40899a0 con 0x7fcec406cb80 2024-04-05T22:27:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.323+0000 7fcec8c00640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 0x7fcea406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.324+0000 7fcec8c00640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 0x7fcea406d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fceb0036000 tx=0x7fceb003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.327+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fceb4054a60 con 0x7fcec406cb80 2024-04-05T22:27:02.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.522+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcec4156140 con 0x7fcec406cb80 2024-04-05T22:27:02.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.523+0000 7fcec1600640 1 -- 172.21.15.73:0/768185700 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fceb4058080 con 0x7fcec406cb80 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 msgr2=0x7fcea406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 0x7fcea406d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fceb0036000 tx=0x7fceb003c040 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 msgr2=0x7fcec414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d800 tx=0x7fceb400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 shutdown_connections 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcea406ad80 0x7fcea406d240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec41599f0 0x7fcec41f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcec40add00 0x7fcec414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 --2- 172.21.15.73:0/768185700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec406cb80 0x7fcec414da30 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 >> 172.21.15.73:0/768185700 conn(0x7fcec4069cb0 msgr2=0x7fcec4070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:02.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 shutdown_connections 2024-04-05T22:27:02.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.526+0000 7fcecc000640 1 -- 172.21.15.73:0/768185700 wait complete. 2024-04-05T22:27:02.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-04-05T22:27:02.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.752+0000 7f1e58e00640 1 Processor -- start 2024-04-05T22:27:02.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.753+0000 7f1e58e00640 1 -- start start 2024-04-05T22:27:02.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e58e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e540d18c0 0x7f1e540d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e58e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e540b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e58e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540d1d50 con 0x7f1e5415ce70 2024-04-05T22:27:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e58e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540d03a0 con 0x7f1e540d18c0 2024-04-05T22:27:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e58e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ce400 con 0x7f1e540b26a0 2024-04-05T22:27:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e53400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1e5415ce70 0x7f1e540d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37030/0 (socket says 172.21.15.73:37030) 2024-04-05T22:27:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.754+0000 7f1e53400640 1 -- 172.21.15.73:0/31132422 learned_addr learned my addr 172.21.15.73:0/31132422 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126243821 0 0) 0x7f1e540d1d50 con 0x7f1e5415ce70 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e38003520 con 0x7f1e5415ce70 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52a00640 1 --2- 172.21.15.73:0/31132422 >> v2:172.21.15.157:3300/0 conn(0x7f1e540d18c0 0x7f1e540d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e53e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e540b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922500531 0 0) 0x7f1e38003520 con 0x7f1e5415ce70 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 msgr2=0x7f1e540b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e540b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 >> v2:172.21.15.157:3300/0 conn(0x7f1e540d18c0 msgr2=0x7f1e540d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 --2- 172.21.15.73:0/31132422 >> v2:172.21.15.157:3300/0 conn(0x7f1e540d18c0 0x7f1e540d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540ce6e0 con 0x7f1e5415ce70 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f1e44001670 con 0x7f1e5415ce70 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 >> v1:172.21.15.73:6789/0 conn(0x7f1e5415ce70 legacy=0x7f1e540d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.755+0000 7f1e52000640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e38003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52000640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38003fa0 0x7f1e380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52000640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e38004d00 0x7f1e38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540d03a0 con 0x7f1e38004d00 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ce400 con 0x7f1e38003fa0 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3800ba80 con 0x7f1e540b26a0 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e38004d00 0x7f1e38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e52a00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38003fa0 0x7f1e380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53400640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e38003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 -- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 msgr2=0x7f1e38003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e540b26a0 0x7f1e38003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 -- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38003fa0 msgr2=0x7f1e380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:02.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38003fa0 0x7f1e380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.756+0000 7f1e53e00640 1 -- 172.21.15.73:0/31132422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38003520 con 0x7f1e38004d00 2024-04-05T22:27:02.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.757+0000 7f1e53e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e38004d00 0x7f1e38005180 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c007360 tx=0x7f1e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d1b7723f800da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c02ccd0 con 0x7f1e38004d00 2024-04-05T22:27:02.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c02ce30 con 0x7f1e38004d00 2024-04-05T22:27:02.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e52000640 1 -- 172.21.15.73:0/31132422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c034660 con 0x7f1e38004d00 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 -- 172.21.15.73:0/31132422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e38004d00 msgr2=0x7f1e38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 --2- 172.21.15.73:0/31132422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e38004d00 0x7f1e38005180 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c007360 tx=0x7f1e380057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e53e00640 1 -- 172.21.15.73:0/31132422 reap_dead start 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 -- 172.21.15.73:0/31132422 shutdown_connections 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 -- 172.21.15.73:0/31132422 >> 172.21.15.73:0/31132422 conn(0x7f1e540d2a60 msgr2=0x7f1e540d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 -- 172.21.15.73:0/31132422 shutdown_connections 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.758+0000 7f1e58e00640 1 -- 172.21.15.73:0/31132422 wait complete. 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.760+0000 7f1e58e00640 1 Processor -- start 2024-04-05T22:27:02.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.760+0000 7f1e58e00640 1 -- start start 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38004d00 0x7f1e541f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5414c620 0x7f1e540c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e540b8650 0x7f1e5414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b81c0 con 0x7f1e540b8650 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e541f3670 con 0x7f1e38004d00 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e58e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b75b0 con 0x7f1e5414c620 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e540b8650 0x7f1e5414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e540b8650 0x7f1e5414b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60984/0 (socket says 172.21.15.73:60984) 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.761+0000 7f1e53e00640 1 -- 172.21.15.73:0/4247712203 learned_addr learned my addr 172.21.15.73:0/4247712203 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53400640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38004d00 0x7f1e541f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 -- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5414c620 msgr2=0x7f1e540c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e5414c620 0x7f1e540c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 -- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38004d00 msgr2=0x7f1e541f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e38004d00 0x7f1e541f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 -- 172.21.15.73:0/4247712203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540b7890 con 0x7f1e540b8650 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e53e00640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e540b8650 0x7f1e5414b9d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c002230 tx=0x7f1e4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c03e600 con 0x7f1e540b8650 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c03e760 con 0x7f1e540b8650 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c03ead0 con 0x7f1e540b8650 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.762+0000 7f1e58e00640 1 -- 172.21.15.73:0/4247712203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540b7b70 con 0x7f1e540b8650 2024-04-05T22:27:02.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.763+0000 7f1e58e00640 1 -- 172.21.15.73:0/4247712203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e540b7260 con 0x7f1e540b8650 2024-04-05T22:27:02.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.764+0000 7f1e58e00640 1 -- 172.21.15.73:0/4247712203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e540b8370 con 0x7f1e540b8650 2024-04-05T22:27:02.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.765+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f1e4c03ec30 con 0x7f1e540b8650 2024-04-05T22:27:02.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.765+0000 7f1e4b400640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e3006ad20 0x7f1e3006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:02.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.765+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c0aba90 con 0x7f1e540b8650 2024-04-05T22:27:02.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.766+0000 7f1e53400640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e3006ad20 0x7f1e3006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:02.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.767+0000 7f1e53400640 1 --2- 172.21.15.73:0/4247712203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e3006ad20 0x7f1e3006d1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e44001eb0 tx=0x7f1e44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:02.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:02.770+0000 7f1e4b400640 1 -- 172.21.15.73:0/4247712203 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1e540b8370 con 0x7f1e540b8650 2024-04-05T22:27:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-05T22:27:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:fs volume create [] : Create a CephFS volume 2024-04-05T22:27:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-05T22:27:02.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-05T22:27:02.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-04-05T22:27:03.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.191+0000 7f13e7e00640 1 Processor -- start 2024-04-05T22:27:03.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.192+0000 7f13e7e00640 1 -- start start 2024-04-05T22:27:03.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.193+0000 7f13e7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13e00ad720 0x7f13e006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.193+0000 7f13e7e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 0x7f13e00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.193+0000 7f13e7e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e0070060 con 0x7f13e015ab60 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.193+0000 7f13e7e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00adb80 con 0x7f13e00ad720 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.193+0000 7f13e7e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e0071220 con 0x7f13e00ccbc0 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13e00ad720 0x7f13e006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13e00ad720 0x7f13e006bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60996/0 (socket says 172.21.15.73:60996) 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 -- 172.21.15.73:0/1333053483 learned_addr learned my addr 172.21.15.73:0/1333053483 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e5e00640 1 --2- 172.21.15.73:0/1333053483 >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 0x7f13e00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 -- 172.21.15.73:0/1333053483 >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 msgr2=0x7f13e00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 --2- 172.21.15.73:0/1333053483 >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 0x7f13e00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 -- 172.21.15.73:0/1333053483 >> v1:172.21.15.157:6789/0 conn(0x7f13e015ab60 legacy=0x7f13e006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e4a00640 1 -- 172.21.15.73:0/1333053483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0071500 con 0x7f13e00ad720 2024-04-05T22:27:03.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.194+0000 7f13e5e00640 1 --2- 172.21.15.73:0/1333053483 >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 0x7f13e00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.195+0000 7f13e4a00640 1 --2- 172.21.15.73:0/1333053483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ad720 0x7f13e006bb60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f13e0071110 tx=0x7f13cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a8e988c8922e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.195+0000 7f13dfe00640 1 -- 172.21.15.73:0/1333053483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031870 con 0x7f13e00ad720 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13dfe00640 1 -- 172.21.15.73:0/1333053483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0319d0 con 0x7f13e00ad720 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13dfe00640 1 -- 172.21.15.73:0/1333053483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031d60 con 0x7f13e00ad720 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 -- 172.21.15.73:0/1333053483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ad720 msgr2=0x7f13e006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 --2- 172.21.15.73:0/1333053483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ad720 0x7f13e006bb60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f13e0071110 tx=0x7f13cc02b910 comp rx=0 tx=0).stop 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 -- 172.21.15.73:0/1333053483 shutdown_connections 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 --2- 172.21.15.73:0/1333053483 >> v2:172.21.15.157:3300/0 conn(0x7f13e00ccbc0 0x7f13e00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 --2- 172.21.15.73:0/1333053483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ad720 0x7f13e006bb60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 -- 172.21.15.73:0/1333053483 >> 172.21.15.73:0/1333053483 conn(0x7f13e0069e90 msgr2=0x7f13e006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:03.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 -- 172.21.15.73:0/1333053483 shutdown_connections 2024-04-05T22:27:03.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.196+0000 7f13e7e00640 1 -- 172.21.15.73:0/1333053483 wait complete. 2024-04-05T22:27:03.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.198+0000 7f13e7e00640 1 Processor -- start 2024-04-05T22:27:03.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.198+0000 7f13e7e00640 1 -- start start 2024-04-05T22:27:03.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e00ad720 0x7f13e01484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e015ab60 0x7f13e0158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0148a30 con 0x7f13e00ccbc0 2024-04-05T22:27:03.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e015b0c0 con 0x7f13e015ab60 2024-04-05T22:27:03.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.199+0000 7f13e7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e015b310 con 0x7f13e00ad720 2024-04-05T22:27:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32772/0 (socket says 172.21.15.73:32772) 2024-04-05T22:27:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 -- 172.21.15.73:0/4206972251 learned_addr learned my addr 172.21.15.73:0/4206972251 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e5e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e015ab60 0x7f13e0158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e5400640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e00ad720 0x7f13e01484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 -- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e00ad720 msgr2=0x7f13e01484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e00ad720 0x7f13e01484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 -- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e015ab60 msgr2=0x7f13e0158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e015ab60 0x7f13e0158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.200+0000 7f13e4a00640 1 -- 172.21.15.73:0/4206972251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0094d0 con 0x7f13e00ccbc0 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13e4a00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f13cc02b950 tx=0x7f13cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031840 con 0x7f13e00ccbc0 2024-04-05T22:27:03.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0443f0 con 0x7f13e00ccbc0 2024-04-05T22:27:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0447a0 con 0x7f13e00ccbc0 2024-04-05T22:27:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e015b5f0 con 0x7f13e00ccbc0 2024-04-05T22:27:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.201+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e015bbb0 con 0x7f13e00ccbc0 2024-04-05T22:27:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.202+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e0067690 con 0x7f13e00ccbc0 2024-04-05T22:27:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.203+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f13cc044900 con 0x7f13e00ccbc0 2024-04-05T22:27:03.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.204+0000 7f13dd600640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 0x7f13c006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:03.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.204+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13cc02d040 con 0x7f13e00ccbc0 2024-04-05T22:27:03.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.205+0000 7f13e5400640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 0x7f13c006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:03.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.206+0000 7f13e5400640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 0x7f13c006d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13d4006fd0 tx=0x7f13d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:03.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.208+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f13cc076e10 con 0x7f13e00ccbc0 2024-04-05T22:27:03.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:03.425+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e00717e0 con 0x7f13c006ad20 2024-04-05T22:27:04.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.677+0000 7f13dd600640 1 -- 172.21.15.73:0/4206972251 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f13d400bd80 con 0x7f13c006ad20 2024-04-05T22:27:04.678 INFO:teuthology.orchestra.run.smithi073.stderr:Volume created successfully (no MDS daemons created) 2024-04-05T22:27:04.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.686+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 msgr2=0x7f13c006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.686+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 0x7f13c006d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13d4006fd0 tx=0x7f13d4008040 comp rx=0 tx=0).stop 2024-04-05T22:27:04.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.687+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 msgr2=0x7f13e01589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.687+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f13cc02b950 tx=0x7f13cc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 shutdown_connections 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f13c006ad20 0x7f13c006d1e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e015ab60 0x7f13e0158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13e00ccbc0 0x7f13e01589a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 --2- 172.21.15.73:0/4206972251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e00ad720 0x7f13e01484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.688+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 >> 172.21.15.73:0/4206972251 conn(0x7f13e0069e90 msgr2=0x7f13e0155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.689+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 shutdown_connections 2024-04-05T22:27:04.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.689+0000 7f13e7e00640 1 -- 172.21.15.73:0/4206972251 wait complete. 2024-04-05T22:27:04.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-04-05T22:27:04.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.897+0000 7f1f07200640 1 Processor -- start 2024-04-05T22:27:04.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.897+0000 7f1f07200640 1 -- start start 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.898+0000 7f1f07200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1f0815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f07200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f08070640 0x7f1f0806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f07200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0806bc50 con 0x7f1f0815e580 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f07200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0815f530 con 0x7f1f08070640 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f07200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0815ef70 con 0x7f1f0815a850 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f05400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1f0815e580 0x7f1f0806ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37038/0 (socket says 172.21.15.73:37038) 2024-04-05T22:27:04.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f05400640 1 -- 172.21.15.73:0/3558932837 learned_addr learned my addr 172.21.15.73:0/3558932837 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f05e00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1f0815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> v2:172.21.15.157:3300/0 conn(0x7f1f08070640 0x7f1f0806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.899+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033419830 0 0) 0x7f1f0806bc50 con 0x7f1f0815e580 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eec003520 con 0x7f1f0815e580 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951913198 0 0) 0x7f1eec003520 con 0x7f1f0815e580 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 msgr2=0x7f1f0815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1f0815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 >> v2:172.21.15.157:3300/0 conn(0x7f1f08070640 msgr2=0x7f1f0806c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 --2- 172.21.15.73:0/3558932837 >> v2:172.21.15.157:3300/0 conn(0x7f1f08070640 0x7f1f0806c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0815f250 con 0x7f1f0815e580 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f1ef4001670 con 0x7f1f0815e580 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> v2:172.21.15.157:3300/0 conn(0x7f1f08070640 0x7f1f0806c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 >> v1:172.21.15.73:6789/0 conn(0x7f1f0815e580 legacy=0x7f1f0806ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eec003970 0x7f1eec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1eec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec004d00 0x7f1eec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0815f530 con 0x7f1eec004d00 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0815ef70 con 0x7f1eec003970 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec00b980 con 0x7f1f0815a850 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.900+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec004d00 0x7f1eec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f05e00640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eec003970 0x7f1eec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f05400640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1eec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f06800640 1 -- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 msgr2=0x7f1eec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0815a850 0x7f1eec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f06800640 1 -- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eec003970 msgr2=0x7f1eec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eec003970 0x7f1eec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.901+0000 7f1f06800640 1 -- 172.21.15.73:0/3558932837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec003520 con 0x7f1eec004d00 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.902+0000 7f1f06800640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec004d00 0x7f1eec005180 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f000098d0 tx=0x7f1eec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb7ea1bfff9618c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.902+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004230 con 0x7f1eec004d00 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.902+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00004390 con 0x7f1eec004d00 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.902+0000 7f1f04a00640 1 -- 172.21.15.73:0/3558932837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00003830 con 0x7f1eec004d00 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 -- 172.21.15.73:0/3558932837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec004d00 msgr2=0x7f1eec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 --2- 172.21.15.73:0/3558932837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec004d00 0x7f1eec005180 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f000098d0 tx=0x7f1eec0057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f06800640 1 -- 172.21.15.73:0/3558932837 reap_dead start 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 -- 172.21.15.73:0/3558932837 shutdown_connections 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 -- 172.21.15.73:0/3558932837 >> 172.21.15.73:0/3558932837 conn(0x7f1f08069cb0 msgr2=0x7f1f0806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 -- 172.21.15.73:0/3558932837 shutdown_connections 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.903+0000 7f1f07200640 1 -- 172.21.15.73:0/3558932837 wait complete. 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.905+0000 7f1f07200640 1 Processor -- start 2024-04-05T22:27:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.905+0000 7f1f07200640 1 -- start start 2024-04-05T22:27:04.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f081efcb0 0x7f1f081f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0806c5f0 0x7f1f0806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081f0090 0x7f1f081f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0806cfb0 con 0x7f1f0806c5f0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f3970 con 0x7f1f081f0090 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f07200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f0e40 con 0x7f1f081efcb0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0806c5f0 0x7f1f0806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0806c5f0 0x7f1f0806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32780/0 (socket says 172.21.15.73:32780) 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f05400640 1 -- 172.21.15.73:0/892956883 learned_addr learned my addr 172.21.15.73:0/892956883 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.906+0000 7f1f05e00640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f081efcb0 0x7f1f081f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f06800640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081f0090 0x7f1f081f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 -- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f081efcb0 msgr2=0x7f1f081f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f081efcb0 0x7f1f081f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 -- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081f0090 msgr2=0x7f1f081f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081f0090 0x7f1f081f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 -- 172.21.15.73:0/892956883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f081f1120 con 0x7f1f0806c5f0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f05400640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0806c5f0 0x7f1f0806ca70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400eaf0 tx=0x7f1ef4005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400c930 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef400ca90 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400ce40 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.907+0000 7f1f07200640 1 -- 172.21.15.73:0/892956883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f081f3bc0 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.908+0000 7f1f07200640 1 -- 172.21.15.73:0/892956883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081588a0 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.909+0000 7f1f07200640 1 -- 172.21.15.73:0/892956883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f081f1310 con 0x7f1f0806c5f0 2024-04-05T22:27:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.910+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f1ef401d460 con 0x7f1f0806c5f0 2024-04-05T22:27:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.910+0000 7f1efe000640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ee806ad50 0x7f1ee806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.911+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f1ef40899a0 con 0x7f1f0806c5f0 2024-04-05T22:27:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.911+0000 7f1f05e00640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ee806ad50 0x7f1ee806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.912+0000 7f1f05e00640 1 --2- 172.21.15.73:0/892956883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ee806ad50 0x7f1ee806d210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009fd0 tx=0x7f1ef800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:04.916+0000 7f1efe000640 1 -- 172.21.15.73:0/892956883 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1f081f1310 con 0x7f1f0806c5f0 2024-04-05T22:27:05.137 INFO:teuthology.orchestra.run.smithi073.stderr:Invalid command: invalid chars #@ in abcd@# 2024-04-05T22:27:05.137 INFO:teuthology.orchestra.run.smithi073.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-04-05T22:27:05.137 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-05T22:27:05.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-05T22:27:05.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-04-05T22:27:05.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.365+0000 7f88d7400640 1 Processor -- start 2024-04-05T22:27:05.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.365+0000 7f88d7400640 1 -- start start 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f88d00add00 0x7f88d0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d7400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d00ae190 con 0x7f88d01599f0 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d7400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d00b3890 con 0x7f88d00add00 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b27f0 con 0x7f88d006cd50 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d4a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f88d01599f0 0x7f88d006c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37052/0 (socket says 172.21.15.73:37052) 2024-04-05T22:27:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d4a00640 1 -- 172.21.15.73:0/767535117 learned_addr learned my addr 172.21.15.73:0/767535117 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88d5400640 1 --2- 172.21.15.73:0/767535117 >> v2:172.21.15.73:3300/0 conn(0x7f88d00add00 0x7f88d0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.367+0000 7f88cfe00640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88cf400640 1 -- 172.21.15.73:0/767535117 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442640228 0 0) 0x7f88d00ae190 con 0x7f88d01599f0 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88cf400640 1 -- 172.21.15.73:0/767535117 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b8003520 con 0x7f88d01599f0 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88d5400640 1 -- 172.21.15.73:0/767535117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 msgr2=0x7f88d0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88d5400640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d0070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88d5400640 1 -- 172.21.15.73:0/767535117 >> v1:172.21.15.73:6789/0 conn(0x7f88d01599f0 legacy=0x7f88d006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.368+0000 7f88d5400640 1 -- 172.21.15.73:0/767535117 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00b2ad0 con 0x7f88d00add00 2024-04-05T22:27:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.369+0000 7f88d5400640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d00add00 0x7f88d0159dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65084f17f4e1d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.369+0000 7f88cf400640 1 -- 172.21.15.73:0/767535117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c802cc50 con 0x7f88d00add00 2024-04-05T22:27:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.369+0000 7f88cf400640 1 -- 172.21.15.73:0/767535117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c802cdb0 con 0x7f88d00add00 2024-04-05T22:27:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.369+0000 7f88cf400640 1 -- 172.21.15.73:0/767535117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8034660 con 0x7f88d00add00 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 -- 172.21.15.73:0/767535117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d00add00 msgr2=0x7f88d0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d00add00 0x7f88d0159dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c802bea0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 -- 172.21.15.73:0/767535117 shutdown_connections 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d0070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 --2- 172.21.15.73:0/767535117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d00add00 0x7f88d0159dd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 -- 172.21.15.73:0/767535117 >> 172.21.15.73:0/767535117 conn(0x7f88d0069cb0 msgr2=0x7f88d006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 -- 172.21.15.73:0/767535117 shutdown_connections 2024-04-05T22:27:05.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.370+0000 7f88d7400640 1 -- 172.21.15.73:0/767535117 wait complete. 2024-04-05T22:27:05.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.372+0000 7f88d7400640 1 Processor -- start 2024-04-05T22:27:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.372+0000 7f88d7400640 1 -- start start 2024-04-05T22:27:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88d00add00 0x7f88d015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d01599f0 0x7f88d00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01574b0 con 0x7f88d01599f0 2024-04-05T22:27:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01f2dd0 con 0x7f88d00add00 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00ceb50 con 0x7f88d006cd50 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d01599f0 0x7f88d00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d01599f0 0x7f88d00cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32788/0 (socket says 172.21.15.73:32788) 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 -- 172.21.15.73:0/1714364144 learned_addr learned my addr 172.21.15.73:0/1714364144 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d4a00640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 -- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 msgr2=0x7f88d015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88d006cd50 0x7f88d015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88cfe00640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88d00add00 0x7f88d015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 -- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88d00add00 msgr2=0x7f88d015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88d00add00 0x7f88d015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.373+0000 7f88d5400640 1 -- 172.21.15.73:0/1714364144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8009a70 con 0x7f88d01599f0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88d5400640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d01599f0 0x7f88d00cdd60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8040650 con 0x7f88d01599f0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80407b0 con 0x7f88d01599f0 2024-04-05T22:27:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8040b60 con 0x7f88d01599f0 2024-04-05T22:27:05.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88d7400640 1 -- 172.21.15.73:0/1714364144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d01f3260 con 0x7f88d01599f0 2024-04-05T22:27:05.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.374+0000 7f88d7400640 1 -- 172.21.15.73:0/1714364144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d01f2780 con 0x7f88d01599f0 2024-04-05T22:27:05.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.375+0000 7f88d7400640 1 -- 172.21.15.73:0/1714364144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d01f2f80 con 0x7f88d01599f0 2024-04-05T22:27:05.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.376+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f88c8040cc0 con 0x7f88d01599f0 2024-04-05T22:27:05.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.378+0000 7f88ccc00640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88b406ad20 0x7f88b406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.378+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f88c80ac520 con 0x7f88d01599f0 2024-04-05T22:27:05.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.378+0000 7f88d4a00640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88b406ad20 0x7f88b406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.379+0000 7f88d4a00640 1 --2- 172.21.15.73:0/1714364144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88b406ad20 0x7f88b406d1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88c00011e0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.382+0000 7f88ccc00640 1 -- 172.21.15.73:0/1714364144 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f88d01f2f80 con 0x7f88d01599f0 2024-04-05T22:27:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-05T22:27:05.597 INFO:teuthology.orchestra.run.smithi073.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-04-05T22:27:05.597 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-05T22:27:05.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-05T22:27:05.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-04-05T22:27:05.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.824+0000 7f6d95400640 1 Processor -- start 2024-04-05T22:27:05.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.824+0000 7f6d95400640 1 -- start start 2024-04-05T22:27:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d95400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 0x7f6d900712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d95400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d9006c9a0 0x7f6d9006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d95400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9006bec0 con 0x7f6d9006ba70 2024-04-05T22:27:05.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d95400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9015afc0 con 0x7f6d9006c9a0 2024-04-05T22:27:05.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d95400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9006f5b0 con 0x7f6d9015abe0 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d8e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 0x7f6d900712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d87e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d9006c9a0 0x7f6d9006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d8e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 0x7f6d900712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:53618/0 (socket says 172.21.15.73:53618) 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d87e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d9006c9a0 0x7f6d9006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32790/0 (socket says 172.21.15.73:32790) 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.826+0000 7f6d8e800640 1 -- 172.21.15.73:0/2959997388 learned_addr learned my addr 172.21.15.73:0/2959997388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.827+0000 7f6d87e00640 1 -- 172.21.15.73:0/2959997388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 msgr2=0x7f6d900712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.827+0000 7f6d87e00640 1 --2- 172.21.15.73:0/2959997388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 0x7f6d900712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.827+0000 7f6d87e00640 1 -- 172.21.15.73:0/2959997388 >> v1:172.21.15.157:6789/0 conn(0x7f6d9006ba70 legacy=0x7f6d900cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.827+0000 7f6d87e00640 1 -- 172.21.15.73:0/2959997388 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9006f890 con 0x7f6d9006c9a0 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.828+0000 7f6d87e00640 1 --2- 172.21.15.73:0/2959997388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006c9a0 0x7f6d9006b530 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6d780097a0 tx=0x7f6d7802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05bbfd66db2f998 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.828+0000 7f6d8de00640 1 -- 172.21.15.73:0/2959997388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7802f050 con 0x7f6d9006c9a0 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.828+0000 7f6d8de00640 1 -- 172.21.15.73:0/2959997388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78031700 con 0x7f6d9006c9a0 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.828+0000 7f6d8de00640 1 -- 172.21.15.73:0/2959997388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78031a90 con 0x7f6d9006c9a0 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 -- 172.21.15.73:0/2959997388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006c9a0 msgr2=0x7f6d9006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 --2- 172.21.15.73:0/2959997388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006c9a0 0x7f6d9006b530 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6d780097a0 tx=0x7f6d7802b7a0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 -- 172.21.15.73:0/2959997388 shutdown_connections 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 --2- 172.21.15.73:0/2959997388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006c9a0 0x7f6d9006b530 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 --2- 172.21.15.73:0/2959997388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9015abe0 0x7f6d900712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 -- 172.21.15.73:0/2959997388 >> 172.21.15.73:0/2959997388 conn(0x7f6d90069cb0 msgr2=0x7f6d9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 -- 172.21.15.73:0/2959997388 shutdown_connections 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.829+0000 7f6d95400640 1 -- 172.21.15.73:0/2959997388 wait complete. 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.831+0000 7f6d95400640 1 Processor -- start 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.831+0000 7f6d95400640 1 -- start start 2024-04-05T22:27:05.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9006c9a0 0x7f6d901516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015abe0 0x7f6d90151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900b29c0 con 0x7f6d9006ba70 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ae0f0 con 0x7f6d9015abe0 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d95400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900b2c90 con 0x7f6d9006c9a0 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d8e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d8e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36168/0 (socket says 172.21.15.73:36168) 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.832+0000 7f6d8e800640 1 -- 172.21.15.73:0/3317832466 learned_addr learned my addr 172.21.15.73:0/3317832466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d87e00640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9006c9a0 0x7f6d901516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8f200640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015abe0 0x7f6d90151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 -- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9006c9a0 msgr2=0x7f6d901516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9006c9a0 0x7f6d901516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 -- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015abe0 msgr2=0x7f6d90151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:05.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015abe0 0x7f6d90151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:05.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 -- 172.21.15.73:0/3317832466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009400 con 0x7f6d9006ba70 2024-04-05T22:27:05.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d8e800640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e900 tx=0x7f6d8000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000ca00 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000cb60 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80013470 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900b2f70 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.833+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d901417a0 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.834+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90156ea0 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.835+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f6d800135d0 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.836+0000 7f6d86a00640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 0x7f6d6c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.836+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f6d80089b80 con 0x7f6d9006ba70 2024-04-05T22:27:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.837+0000 7f6d87e00640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 0x7f6d6c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:05.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.838+0000 7f6d87e00640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 0x7f6d6c06d1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d78035000 tx=0x7f6d78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:05.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:05.841+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d80054870 con 0x7f6d9006ba70 2024-04-05T22:27:05.865 INFO:tasks.ceph.mds.b.smithi157.stderr:2024-04-05T22:27:05.866+0000 7f677e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:27:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:06.061+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d90070130 con 0x7f6d6c06ad20 2024-04-05T22:27:06.074 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:27:06.074+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:27:06.103 INFO:tasks.ceph.mds.b.smithi157.stderr: -18> 2024-04-05T22:27:05.866+0000 7f677e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:27:06.149 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:27:09.053 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:27:09.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:27:09.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.127+0000 7f6d86a00640 1 -- 172.21.15.73:0/3317832466 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7f6d780319f0 con 0x7f6d6c06ad20 2024-04-05T22:27:09.127 INFO:teuthology.orchestra.run.smithi073.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 msgr2=0x7f6d6c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 0x7f6d6c06d1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d78035000 tx=0x7f6d78008030 comp rx=0 tx=0).stop 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 msgr2=0x7f6d90141260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e900 tx=0x7f6d8000edc0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 shutdown_connections 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06ad20 0x7f6d6c06d1e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015abe0 0x7f6d90151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d9006c9a0 0x7f6d901516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 --2- 172.21.15.73:0/3317832466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9006ba70 0x7f6d90141260 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.130+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 >> 172.21.15.73:0/3317832466 conn(0x7f6d90069cb0 msgr2=0x7f6d90158260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:09.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.131+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 shutdown_connections 2024-04-05T22:27:09.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.131+0000 7f6d95400640 1 -- 172.21.15.73:0/3317832466 wait complete. 2024-04-05T22:27:09.149 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:27:09.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:27:09.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:27:09.360 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:27:09.361 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:09.361 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:09.417 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:09.417 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:09.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:09.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.684+0000 7efc42000640 1 Processor -- start 2024-04-05T22:27:09.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.684+0000 7efc42000640 1 -- start start 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc42000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc42000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc3c0ccd20 0x7efc3c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc42000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c06f5b0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc42000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c070410 con 0x7efc3c0ccd20 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc42000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c158bb0 con 0x7efc3c1587d0 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc32a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc3c0ccd20 0x7efc3c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc32a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc3c0ccd20 0x7efc3c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36174/0 (socket says 172.21.15.73:36174) 2024-04-05T22:27:09.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc32a00640 1 -- 172.21.15.73:0/750406633 learned_addr learned my addr 172.21.15.73:0/750406633 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc32a00640 1 -- 172.21.15.73:0/750406633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 msgr2=0x7efc3c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc3b400640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.686+0000 7efc32a00640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.687+0000 7efc32a00640 1 -- 172.21.15.73:0/750406633 >> v1:172.21.15.157:6789/0 conn(0x7efc3c06d0c0 legacy=0x7efc3c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.687+0000 7efc32a00640 1 -- 172.21.15.73:0/750406633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c06f800 con 0x7efc3c0ccd20 2024-04-05T22:27:09.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.687+0000 7efc3b400640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:09.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc32a00640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c0ccd20 0x7efc3c06cb80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efc24009870 tx=0x7efc2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdcf62de9db1fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:09.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc3aa00640 1 -- 172.21.15.73:0/750406633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24031640 con 0x7efc3c0ccd20 2024-04-05T22:27:09.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc3aa00640 1 -- 172.21.15.73:0/750406633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc240317a0 con 0x7efc3c0ccd20 2024-04-05T22:27:09.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc3aa00640 1 -- 172.21.15.73:0/750406633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24031b30 con 0x7efc3c0ccd20 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc42000640 1 -- 172.21.15.73:0/750406633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c0ccd20 msgr2=0x7efc3c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.688+0000 7efc42000640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c0ccd20 0x7efc3c06cb80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efc24009870 tx=0x7efc2402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 -- 172.21.15.73:0/750406633 shutdown_connections 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c0ccd20 0x7efc3c06cb80 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 --2- 172.21.15.73:0/750406633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 -- 172.21.15.73:0/750406633 >> 172.21.15.73:0/750406633 conn(0x7efc3c0ce8e0 msgr2=0x7efc3c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 -- 172.21.15.73:0/750406633 shutdown_connections 2024-04-05T22:27:09.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.689+0000 7efc42000640 1 -- 172.21.15.73:0/750406633 wait complete. 2024-04-05T22:27:09.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.690+0000 7efc42000640 1 Processor -- start 2024-04-05T22:27:09.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 -- start start 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc3c153810 0x7efc3c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f5f10 con 0x7efc3c06d0c0 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f60f0 con 0x7efc3c153810 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.691+0000 7efc42000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f63d0 con 0x7efc3c1587d0 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36186/0 (socket says 172.21.15.73:36186) 2024-04-05T22:27:09.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 -- 172.21.15.73:0/416707205 learned_addr learned my addr 172.21.15.73:0/416707205 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:09.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 -- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 msgr2=0x7efc3c156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 -- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc3c153810 msgr2=0x7efc3c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc3c153810 0x7efc3c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 -- 172.21.15.73:0/416707205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240094d0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.692+0000 7efc3b400640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ce00 tx=0x7efc2c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.693+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00f890 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.693+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c00f9f0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.693+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00fdc0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.693+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c1f66b0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.693+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c1f6c70 con 0x7efc3c06d0c0 2024-04-05T22:27:09.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.694+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c066af0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.695+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7efc2c00b040 con 0x7efc3c06d0c0 2024-04-05T22:27:09.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.696+0000 7efc33e00640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 0x7efc1806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:09.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.696+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efc2c08a0b0 con 0x7efc3c06d0c0 2024-04-05T22:27:09.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.696+0000 7efc32a00640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 0x7efc1806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:09.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.698+0000 7efc32a00640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 0x7efc1806d1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc24036000 tx=0x7efc24030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:09.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.700+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7efc2c054f70 con 0x7efc3c06d0c0 2024-04-05T22:27:09.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.893+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3c074210 con 0x7efc3c06d0c0 2024-04-05T22:27:09.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.896+0000 7efc33e00640 1 -- 172.21.15.73:0/416707205 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16223 (secure 0 0 0) 0x7efc2c01d020 con 0x7efc3c06d0c0 2024-04-05T22:27:09.896 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:09.896 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:09.109725+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.898+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 msgr2=0x7efc1806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.898+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 0x7efc1806d1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc24036000 tx=0x7efc24030000 comp rx=0 tx=0).stop 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.898+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 msgr2=0x7efc3c14c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.898+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ce00 tx=0x7efc2c00a650 comp rx=0 tx=0).stop 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 shutdown_connections 2024-04-05T22:27:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc1806ad20 0x7efc1806d1e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc3c153810 0x7efc3c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc3c1587d0 0x7efc3c156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 --2- 172.21.15.73:0/416707205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc3c06d0c0 0x7efc3c14c350 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 >> 172.21.15.73:0/416707205 conn(0x7efc3c0ce8e0 msgr2=0x7efc3c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 shutdown_connections 2024-04-05T22:27:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:09.899+0000 7efc42000640 1 -- 172.21.15.73:0/416707205 wait complete. 2024-04-05T22:27:09.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:48.879992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:49.430091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:27:09.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:27:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.123+0000 7f2715e00640 1 Processor -- start 2024-04-05T22:27:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.123+0000 7f2715e00640 1 -- start start 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271015cc20 0x7f27100b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27100cdb80 0x7f27100cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27100cc7c0 0x7f27101606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100ad940 con 0x7f27100cdb80 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100ccc10 con 0x7f27100cc7c0 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100b39e0 con 0x7f271015cc20 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f270fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27100cdb80 0x7f27100cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f270fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27100cdb80 0x7f27100cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36196/0 (socket says 172.21.15.73:36196) 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f270fe00640 1 -- 172.21.15.73:0/2088138812 learned_addr learned my addr 172.21.15.73:0/2088138812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2714a00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271015cc20 0x7f27100b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.125+0000 7f2715400640 1 --2- 172.21.15.73:0/2088138812 >> v2:172.21.15.157:3300/0 conn(0x7f27100cc7c0 0x7f27101606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.126+0000 7f270fe00640 1 -- 172.21.15.73:0/2088138812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271015cc20 msgr2=0x7f27100b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.126+0000 7f270fe00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271015cc20 0x7f27100b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.126+0000 7f270fe00640 1 -- 172.21.15.73:0/2088138812 >> v2:172.21.15.157:3300/0 conn(0x7f27100cc7c0 msgr2=0x7f27101606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.126+0000 7f270fe00640 1 --2- 172.21.15.73:0/2088138812 >> v2:172.21.15.157:3300/0 conn(0x7f27100cc7c0 0x7f27101606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.126+0000 7f270fe00640 1 -- 172.21.15.73:0/2088138812 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100b1dc0 con 0x7f27100cdb80 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.127+0000 7f270fe00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27100cdb80 0x7f27100cc280 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2710161750 tx=0x7f26fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8f5566c62637ec server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.127+0000 7f270f400640 1 -- 172.21.15.73:0/2088138812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc031870 con 0x7f27100cdb80 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.127+0000 7f270f400640 1 -- 172.21.15.73:0/2088138812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc0319d0 con 0x7f27100cdb80 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.127+0000 7f270f400640 1 -- 172.21.15.73:0/2088138812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc031d60 con 0x7f27100cdb80 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 -- 172.21.15.73:0/2088138812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27100cdb80 msgr2=0x7f27100cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27100cdb80 0x7f27100cc280 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2710161750 tx=0x7f26fc02b910 comp rx=0 tx=0).stop 2024-04-05T22:27:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 -- 172.21.15.73:0/2088138812 shutdown_connections 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27100cdb80 0x7f27100cc280 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 --2- 172.21.15.73:0/2088138812 >> v2:172.21.15.157:3300/0 conn(0x7f27100cc7c0 0x7f27101606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 --2- 172.21.15.73:0/2088138812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271015cc20 0x7f27100b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 -- 172.21.15.73:0/2088138812 >> 172.21.15.73:0/2088138812 conn(0x7f27100b0400 msgr2=0x7f27100b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 -- 172.21.15.73:0/2088138812 shutdown_connections 2024-04-05T22:27:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.128+0000 7f2715e00640 1 -- 172.21.15.73:0/2088138812 wait complete. 2024-04-05T22:27:10.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.130+0000 7f2715e00640 1 Processor -- start 2024-04-05T22:27:10.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.130+0000 7f2715e00640 1 -- start start 2024-04-05T22:27:10.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27100cc7c0 0x7f27100c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27100cdb80 0x7f271015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271015dcf0 con 0x7f271015cc20 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271015dfd0 con 0x7f27100cdb80 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271015e2b0 con 0x7f27100cc7c0 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36212/0 (socket says 172.21.15.73:36212) 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2715400640 1 -- 172.21.15.73:0/2734091776 learned_addr learned my addr 172.21.15.73:0/2734091776 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f270fe00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27100cdb80 0x7f271015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.131+0000 7f2714a00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27100cc7c0 0x7f27100c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 -- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27100cc7c0 msgr2=0x7f27100c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27100cc7c0 0x7f27100c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 -- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27100cdb80 msgr2=0x7f271015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27100cdb80 0x7f271015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 -- 172.21.15.73:0/2734091776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0094d0 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715400640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2708007e60 tx=0x7f270800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708014420 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27080145a0 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708014950 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271015e590 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.132+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100ca400 con 0x7f271015cc20 2024-04-05T22:27:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.134+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710156c90 con 0x7f271015cc20 2024-04-05T22:27:10.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.135+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f2708014ab0 con 0x7f271015cc20 2024-04-05T22:27:10.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.136+0000 7f270cc00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 0x7f26f006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.136+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2708089d00 con 0x7f271015cc20 2024-04-05T22:27:10.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.136+0000 7f2714a00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 0x7f26f006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.137+0000 7f2714a00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 0x7f26f006d1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2704007920 tx=0x7f2704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.140+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2710156c90 con 0x7f271015cc20 2024-04-05T22:27:10.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.351+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2710156c90 con 0x7f271015cc20 2024-04-05T22:27:10.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.351+0000 7f270cc00640 1 -- 172.21.15.73:0/2734091776 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2710156c90 con 0x7f271015cc20 2024-04-05T22:27:10.352 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:27:10.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 msgr2=0x7f26f006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 0x7f26f006d1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2704007920 tx=0x7f2704008040 comp rx=0 tx=0).stop 2024-04-05T22:27:10.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 msgr2=0x7f271015b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2708007e60 tx=0x7f270800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 shutdown_connections 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f26f006ad20 0x7f26f006d1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f271015cc20 0x7f271015b530 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27100cdb80 0x7f271015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 --2- 172.21.15.73:0/2734091776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27100cc7c0 0x7f27100c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 >> 172.21.15.73:0/2734091776 conn(0x7f27100b0400 msgr2=0x7f27100b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 shutdown_connections 2024-04-05T22:27:10.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.354+0000 7f2715e00640 1 -- 172.21.15.73:0/2734091776 wait complete. 2024-04-05T22:27:10.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:27:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.583+0000 7f4506600640 1 Processor -- start 2024-04-05T22:27:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.583+0000 7f4506600640 1 -- start start 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.584+0000 7f4506600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 0x7f45000712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.584+0000 7f4506600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f450006bc40 0x7f4500164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.584+0000 7f4506600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450015afc0 con 0x7f450006c9a0 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.584+0000 7f4506600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450006f5b0 con 0x7f450006bc40 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.584+0000 7f4506600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450006f890 con 0x7f450015abe0 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44f6a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f450006c9a0 0x7f450006b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51472/0 (socket says 172.21.15.73:51472) 2024-04-05T22:27:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44f6a00640 1 -- 172.21.15.73:0/128251817 learned_addr learned my addr 172.21.15.73:0/128251817 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44ffe00640 1 --2- 172.21.15.73:0/128251817 >> v2:172.21.15.157:3300/0 conn(0x7f450006bc40 0x7f4500164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44ff400640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 0x7f45000712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997029846 0 0) 0x7f450015afc0 con 0x7f450006c9a0 2024-04-05T22:27:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.585+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e4003520 con 0x7f450006c9a0 2024-04-05T22:27:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3832964557 0 0) 0x7f44e4003520 con 0x7f450006c9a0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 msgr2=0x7f45000712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 0x7f45000712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 >> v2:172.21.15.157:3300/0 conn(0x7f450006bc40 msgr2=0x7f4500164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 --2- 172.21.15.73:0/128251817 >> v2:172.21.15.157:3300/0 conn(0x7f450006bc40 0x7f4500164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450006fb70 con 0x7f450006c9a0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f44e8001670 con 0x7f450006c9a0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 >> v1:172.21.15.73:6789/0 conn(0x7f450006c9a0 legacy=0x7f450006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e4003970 0x7f44e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44e40047c0 0x7f44e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 0x7f44e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450006f5b0 con 0x7f44e40047c0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450006f890 con 0x7f44e4003970 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e400ba00 con 0x7f450015abe0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.586+0000 7f44f6a00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44e40047c0 0x7f44e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44ff400640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e4003970 0x7f44e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44f6a00640 1 -- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 msgr2=0x7f44e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44f6a00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450015abe0 0x7f44e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44f6a00640 1 -- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e4003970 msgr2=0x7f44e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44f6a00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e4003970 0x7f44e4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.587+0000 7f44f6a00640 1 -- 172.21.15.73:0/128251817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4003520 con 0x7f44e40047c0 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f44f6a00640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44e40047c0 0x7f44e4004c40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c210 tx=0x7f44e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2061863345697a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e802cae0 con 0x7f44e40047c0 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e802cc40 con 0x7f44e40047c0 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f44fea00640 1 -- 172.21.15.73:0/128251817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e80058d0 con 0x7f44e40047c0 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f4506600640 1 -- 172.21.15.73:0/128251817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44e40047c0 msgr2=0x7f44e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f4506600640 1 --2- 172.21.15.73:0/128251817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44e40047c0 0x7f44e4004c40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c210 tx=0x7f44e40057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.588+0000 7f44ffe00640 1 -- 172.21.15.73:0/128251817 reap_dead start 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.589+0000 7f4506600640 1 -- 172.21.15.73:0/128251817 shutdown_connections 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.589+0000 7f4506600640 1 -- 172.21.15.73:0/128251817 >> 172.21.15.73:0/128251817 conn(0x7f4500069cb0 msgr2=0x7f450006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.589+0000 7f4506600640 1 -- 172.21.15.73:0/128251817 shutdown_connections 2024-04-05T22:27:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.589+0000 7f4506600640 1 -- 172.21.15.73:0/128251817 wait complete. 2024-04-05T22:27:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.590+0000 7f4506600640 1 Processor -- start 2024-04-05T22:27:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 -- start start 2024-04-05T22:27:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e40047c0 0x7f4500156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45001efcb0 0x7f4500158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450013dc60 con 0x7f450013e0f0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45001f0130 con 0x7f44e40047c0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.591+0000 7f4506600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450013d050 con 0x7f45001efcb0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36224/0 (socket says 172.21.15.73:36224) 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 -- 172.21.15.73:0/1564134382 learned_addr learned my addr 172.21.15.73:0/1564134382 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ff400640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e40047c0 0x7f4500156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44f6a00640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45001efcb0 0x7f4500158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 -- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45001efcb0 msgr2=0x7f4500158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45001efcb0 0x7f4500158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 -- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e40047c0 msgr2=0x7f4500156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e40047c0 0x7f4500156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.592+0000 7f44ffe00640 1 -- 172.21.15.73:0/1564134382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450013d330 con 0x7f450013e0f0 2024-04-05T22:27:10.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f44ffe00640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f450006d300 tx=0x7f44f800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f800c680 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f800c7e0 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f800cb90 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450013d610 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.593+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4500158680 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.594+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4500068160 con 0x7f450013e0f0 2024-04-05T22:27:10.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.595+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f44f800ccf0 con 0x7f450013e0f0 2024-04-05T22:27:10.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.596+0000 7f44f7e00640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 0x7f44dc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:10.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.596+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44f8088e90 con 0x7f450013e0f0 2024-04-05T22:27:10.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.597+0000 7f44ff400640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 0x7f44dc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:10.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.598+0000 7f44ff400640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 0x7f44dc06d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009fd0 tx=0x7f44f000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.601+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f44f8053e60 con 0x7f450013e0f0 2024-04-05T22:27:10.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.792+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f450015c500 con 0x7f450013e0f0 2024-04-05T22:27:10.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.793+0000 7f44f7e00640 1 -- 172.21.15.73:0/1564134382 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44f8057480 con 0x7f450013e0f0 2024-04-05T22:27:10.794 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:27:10.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 msgr2=0x7f44dc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 0x7f44dc06d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009fd0 tx=0x7f44f000d040 comp rx=0 tx=0).stop 2024-04-05T22:27:10.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 msgr2=0x7f450013e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f450006d300 tx=0x7f44f800ec00 comp rx=0 tx=0).stop 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 shutdown_connections 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f44dc06ad20 0x7f44dc06d1e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f450013e0f0 0x7f450013e570 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45001efcb0 0x7f4500158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 --2- 172.21.15.73:0/1564134382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44e40047c0 0x7f4500156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 >> 172.21.15.73:0/1564134382 conn(0x7f4500069cb0 msgr2=0x7f450015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 shutdown_connections 2024-04-05T22:27:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:10.796+0000 7f4506600640 1 -- 172.21.15.73:0/1564134382 wait complete. 2024-04-05T22:27:10.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:27:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.025+0000 7f11fb200640 1 Processor -- start 2024-04-05T22:27:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.025+0000 7f11fb200640 1 -- start start 2024-04-05T22:27:11.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.026+0000 7f11fb200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11fc144260 0x7f11fc1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.026+0000 7f11fb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11fc0cdbb0 0x7f11fc0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.026+0000 7f11fb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc06b7a0 con 0x7f11fc1435f0 2024-04-05T22:27:11.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.026+0000 7f11fb200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc06ba80 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.026+0000 7f11fb200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc06bd60 con 0x7f11fc144260 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11f9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f11fc1435f0 0x7f11fc06c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51484/0 (socket says 172.21.15.73:51484) 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11f9e00640 1 -- 172.21.15.73:0/1571546933 learned_addr learned my addr 172.21.15.73:0/1571546933 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11fa800640 1 --2- 172.21.15.73:0/1571546933 >> v2:172.21.15.73:3300/0 conn(0x7f11fc0cdbb0 0x7f11fc0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11f1400640 1 --2- 172.21.15.73:0/1571546933 >> v2:172.21.15.157:3300/0 conn(0x7f11fc144260 0x7f11fc1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11fa800640 1 -- 172.21.15.73:0/1571546933 >> v2:172.21.15.157:3300/0 conn(0x7f11fc144260 msgr2=0x7f11fc1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11fa800640 1 --2- 172.21.15.73:0/1571546933 >> v2:172.21.15.157:3300/0 conn(0x7f11fc144260 0x7f11fc1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11fa800640 1 -- 172.21.15.73:0/1571546933 >> v1:172.21.15.73:6789/0 conn(0x7f11fc1435f0 legacy=0x7f11fc06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.027+0000 7f11f9400640 1 -- 172.21.15.73:0/1571546933 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320359139 0 0) 0x7f11fc06b7a0 con 0x7f11fc1435f0 2024-04-05T22:27:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.028+0000 7f11fa800640 1 -- 172.21.15.73:0/1571546933 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc06c040 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.029+0000 7f11fa800640 1 --2- 172.21.15.73:0/1571546933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc0cdbb0 0x7f11fc0cdfc0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f11fc06da80 tx=0x7f11f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311527848356ef72 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.029+0000 7f11f9400640 1 -- 172.21.15.73:0/1571546933 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f402cc80 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.029+0000 7f11f9400640 1 -- 172.21.15.73:0/1571546933 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f402cde0 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.029+0000 7f11f9400640 1 -- 172.21.15.73:0/1571546933 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4034660 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 -- 172.21.15.73:0/1571546933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc0cdbb0 msgr2=0x7f11fc0cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 --2- 172.21.15.73:0/1571546933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc0cdbb0 0x7f11fc0cdfc0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f11fc06da80 tx=0x7f11f402bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 -- 172.21.15.73:0/1571546933 shutdown_connections 2024-04-05T22:27:11.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 --2- 172.21.15.73:0/1571546933 >> v2:172.21.15.157:3300/0 conn(0x7f11fc144260 0x7f11fc1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 --2- 172.21.15.73:0/1571546933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc0cdbb0 0x7f11fc0cdfc0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 -- 172.21.15.73:0/1571546933 >> 172.21.15.73:0/1571546933 conn(0x7f11fc0b0830 msgr2=0x7f11fc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 -- 172.21.15.73:0/1571546933 shutdown_connections 2024-04-05T22:27:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.030+0000 7f11fb200640 1 -- 172.21.15.73:0/1571546933 wait complete. 2024-04-05T22:27:11.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.032+0000 7f11fb200640 1 Processor -- start 2024-04-05T22:27:11.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.032+0000 7f11fb200640 1 -- start start 2024-04-05T22:27:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11fc0cdbb0 0x7f11fc1f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc144260 0x7f11fc15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15afb0 con 0x7f11fc1435f0 2024-04-05T22:27:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15bac0 con 0x7f11fc144260 2024-04-05T22:27:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11fb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15b190 con 0x7f11fc0cdbb0 2024-04-05T22:27:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36246/0 (socket says 172.21.15.73:36246) 2024-04-05T22:27:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.033+0000 7f11f1400640 1 -- 172.21.15.73:0/3501053379 learned_addr learned my addr 172.21.15.73:0/3501053379 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 -- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11fc0cdbb0 msgr2=0x7f11fc1f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11fc0cdbb0 0x7f11fc1f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 -- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc144260 msgr2=0x7f11fc15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc144260 0x7f11fc15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 -- 172.21.15.73:0/3501053379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4009a70 con 0x7f11fc1435f0 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f1400640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d5f0 tx=0x7f11e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40125f0 con 0x7f11fc1435f0 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4012750 con 0x7f11fc1435f0 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4012b00 con 0x7f11fc1435f0 2024-04-05T22:27:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc15b470 con 0x7f11fc1435f0 2024-04-05T22:27:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.034+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1f32b0 con 0x7f11fc1435f0 2024-04-05T22:27:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.035+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc04e160 con 0x7f11fc1435f0 2024-04-05T22:27:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.036+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f11e4012c60 con 0x7f11fc1435f0 2024-04-05T22:27:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.037+0000 7f11f2a00640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 0x7f11d806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.037+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11e4089ad0 con 0x7f11fc1435f0 2024-04-05T22:27:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.038+0000 7f11f9e00640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 0x7f11d806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.039+0000 7f11f9e00640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 0x7f11d806d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11ec007fd0 tx=0x7f11ec007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.042+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f11e4054aa0 con 0x7f11fc1435f0 2024-04-05T22:27:11.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.234+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11fc15bc00 con 0x7f11fc1435f0 2024-04-05T22:27:11.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.235+0000 7f11f2a00640 1 -- 172.21.15.73:0/3501053379 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11e40580c0 con 0x7f11fc1435f0 2024-04-05T22:27:11.236 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:27:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 msgr2=0x7f11d806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 0x7f11d806d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11ec007fd0 tx=0x7f11ec007920 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 msgr2=0x7f11fc1f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d5f0 tx=0x7f11e400dab0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 shutdown_connections 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f11d806ad20 0x7f11d806d1e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc144260 0x7f11fc15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11fc1435f0 0x7f11fc1f3600 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 --2- 172.21.15.73:0/3501053379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11fc0cdbb0 0x7f11fc1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 >> 172.21.15.73:0/3501053379 conn(0x7f11fc0b0830 msgr2=0x7f11fc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 shutdown_connections 2024-04-05T22:27:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.238+0000 7f11fb200640 1 -- 172.21.15.73:0/3501053379 wait complete. 2024-04-05T22:27:11.257 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:27:11.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.469+0000 7f346b000640 1 Processor -- start 2024-04-05T22:27:11.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.469+0000 7f346b000640 1 -- start start 2024-04-05T22:27:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34641541a0 0x7f34640b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f346406d710 0x7f34640cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 0x7f34640cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346415aa60 con 0x7f346406d710 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640b3b60 con 0x7f346415aef0 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.470+0000 7f346b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34640701c0 con 0x7f34641541a0 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f346406d710 0x7f34640cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f346406d710 0x7f34640cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36262/0 (socket says 172.21.15.73:36262) 2024-04-05T22:27:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 -- 172.21.15.73:0/2000528423 learned_addr learned my addr 172.21.15.73:0/2000528423 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463e00640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34641541a0 0x7f34640b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 -- 172.21.15.73:0/2000528423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34641541a0 msgr2=0x7f34640b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34641541a0 0x7f34640b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 -- 172.21.15.73:0/2000528423 >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 msgr2=0x7f34640cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3469000640 1 --2- 172.21.15.73:0/2000528423 >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 0x7f34640cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 --2- 172.21.15.73:0/2000528423 >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 0x7f34640cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3463400640 1 -- 172.21.15.73:0/2000528423 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640b3880 con 0x7f346406d710 2024-04-05T22:27:11.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.471+0000 7f3469000640 1 --2- 172.21.15.73:0/2000528423 >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 0x7f34640cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:11.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.472+0000 7f3463400640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f346406d710 0x7f34640cd8f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3464071a40 tx=0x7f344c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9636638d4206c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f3462a00640 1 -- 172.21.15.73:0/2000528423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c02ca70 con 0x7f346406d710 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f3462a00640 1 -- 172.21.15.73:0/2000528423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c02cbd0 con 0x7f346406d710 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f3462a00640 1 -- 172.21.15.73:0/2000528423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c0343f0 con 0x7f346406d710 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 -- 172.21.15.73:0/2000528423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f346406d710 msgr2=0x7f34640cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f346406d710 0x7f34640cd8f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3464071a40 tx=0x7f344c02bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 -- 172.21.15.73:0/2000528423 shutdown_connections 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f346406d710 0x7f34640cd8f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 --2- 172.21.15.73:0/2000528423 >> v2:172.21.15.157:3300/0 conn(0x7f346415aef0 0x7f34640cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 --2- 172.21.15.73:0/2000528423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34641541a0 0x7f34640b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 -- 172.21.15.73:0/2000528423 >> 172.21.15.73:0/2000528423 conn(0x7f34640ce8e0 msgr2=0x7f34640cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 -- 172.21.15.73:0/2000528423 shutdown_connections 2024-04-05T22:27:11.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.473+0000 7f346b000640 1 -- 172.21.15.73:0/2000528423 wait complete. 2024-04-05T22:27:11.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.475+0000 7f346b000640 1 Processor -- start 2024-04-05T22:27:11.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.475+0000 7f346b000640 1 -- start start 2024-04-05T22:27:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346406d710 0x7f34641415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f346415aef0 0x7f3464152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464141b00 con 0x7f34641541a0 2024-04-05T22:27:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464159140 con 0x7f346406d710 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.476+0000 7f346b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34641599e0 con 0x7f346415aef0 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36268/0 (socket says 172.21.15.73:36268) 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 -- 172.21.15.73:0/2860447083 learned_addr learned my addr 172.21.15.73:0/2860447083 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3469000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f346415aef0 0x7f3464152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463e00640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346406d710 0x7f34641415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 -- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f346415aef0 msgr2=0x7f3464152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f346415aef0 0x7f3464152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 -- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346406d710 msgr2=0x7f34641415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346406d710 0x7f34641415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.477+0000 7f3463400640 1 -- 172.21.15.73:0/2860447083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c009b30 con 0x7f34641541a0 2024-04-05T22:27:11.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f3463400640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f344c036040 tx=0x7f344c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c03e450 con 0x7f34641541a0 2024-04-05T22:27:11.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c03e5d0 con 0x7f34641541a0 2024-04-05T22:27:11.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c03e980 con 0x7f34641541a0 2024-04-05T22:27:11.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641f66f0 con 0x7f34641541a0 2024-04-05T22:27:11.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.478+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3464159670 con 0x7f34641541a0 2024-04-05T22:27:11.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.479+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464066af0 con 0x7f34641541a0 2024-04-05T22:27:11.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.480+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f344c03eae0 con 0x7f34641541a0 2024-04-05T22:27:11.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.481+0000 7f345be00640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 0x7f344006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:11.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.482+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f344c0ac7c0 con 0x7f34641541a0 2024-04-05T22:27:11.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.482+0000 7f3463e00640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 0x7f344006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:11.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.483+0000 7f3463e00640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 0x7f344006d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f345400a990 tx=0x7f3454008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:11.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.486+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f344c077790 con 0x7f34641541a0 2024-04-05T22:27:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.692+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3464159800 con 0x7f34641541a0 2024-04-05T22:27:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.693+0000 7f345be00640 1 -- 172.21.15.73:0/2860447083 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) v1 ==== 80+0+124 (secure 0 0 0) 0x7f344c033e80 con 0x7f34641541a0 2024-04-05T22:27:11.693 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.157:6835/2227587240 2024-04-06T22:27:06.074507+0000 2024-04-05T22:27:11.693 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.157:6834/2227587240 2024-04-06T22:27:06.074507+0000 2024-04-05T22:27:11.693 INFO:teuthology.orchestra.run.smithi073.stderr:listed 2 entries 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 msgr2=0x7f344006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 0x7f344006d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f345400a990 tx=0x7f3454008040 comp rx=0 tx=0).stop 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 msgr2=0x7f3464151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f344c036040 tx=0x7f344c004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 shutdown_connections 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f344006ad20 0x7f344006d1e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f346415aef0 0x7f3464152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34641541a0 0x7f3464151d30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 --2- 172.21.15.73:0/2860447083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f346406d710 0x7f34641415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.695+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 >> 172.21.15.73:0/2860447083 conn(0x7f34640ce8e0 msgr2=0x7f346415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.696+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 shutdown_connections 2024-04-05T22:27:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:11.696+0000 7f346b000640 1 -- 172.21.15.73:0/2860447083 wait complete. 2024-04-05T22:27:11.716 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:27:11.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:11.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:27:11.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:27:11.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:11.821 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:27:11.870 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:11.870 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:11.928 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:11.928 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:11.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:12.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.198+0000 7f61cf200640 1 Processor -- start 2024-04-05T22:27:12.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.199+0000 7f61cf200640 1 -- start start 2024-04-05T22:27:12.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d015e620 0x7f61d015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cf200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61d006bc70 0x7f61d006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cf200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d015efc0 con 0x7f61d0158860 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cf200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00ccaf0 con 0x7f61d006bc70 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d006b500 con 0x7f61d015e620 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cde00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f61d0158860 0x7f61d00b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51498/0 (socket says 172.21.15.73:51498) 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61cde00640 1 -- 172.21.15.73:0/3980004318 learned_addr learned my addr 172.21.15.73:0/3980004318 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:12.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.200+0000 7f61ce800640 1 --2- 172.21.15.73:0/3980004318 >> v2:172.21.15.73:3300/0 conn(0x7f61d006bc70 0x7f61d006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61cd400640 1 -- 172.21.15.73:0/3980004318 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280270794 0 0) 0x7f61d015efc0 con 0x7f61d0158860 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61c7e00640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d015e620 0x7f61d015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61cd400640 1 -- 172.21.15.73:0/3980004318 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b4003520 con 0x7f61d0158860 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61ce800640 1 -- 172.21.15.73:0/3980004318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d015e620 msgr2=0x7f61d015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61ce800640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d015e620 0x7f61d015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61ce800640 1 -- 172.21.15.73:0/3980004318 >> v1:172.21.15.73:6789/0 conn(0x7f61d0158860 legacy=0x7f61d00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.201+0000 7f61ce800640 1 -- 172.21.15.73:0/3980004318 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00b20d0 con 0x7f61d006bc70 2024-04-05T22:27:12.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.202+0000 7f61ce800640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d006bc70 0x7f61d006c100 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ccc30 tx=0x7f61c802be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87f8f2e5b279d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.202+0000 7f61cd400640 1 -- 172.21.15.73:0/3980004318 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c802cbb0 con 0x7f61d006bc70 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.202+0000 7f61cd400640 1 -- 172.21.15.73:0/3980004318 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c802cd10 con 0x7f61d006bc70 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.202+0000 7f61cd400640 1 -- 172.21.15.73:0/3980004318 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8034660 con 0x7f61d006bc70 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 -- 172.21.15.73:0/3980004318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d006bc70 msgr2=0x7f61d006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d006bc70 0x7f61d006c100 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ccc30 tx=0x7f61c802be00 comp rx=0 tx=0).stop 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 -- 172.21.15.73:0/3980004318 shutdown_connections 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d015e620 0x7f61d015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 --2- 172.21.15.73:0/3980004318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d006bc70 0x7f61d006c100 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 -- 172.21.15.73:0/3980004318 >> 172.21.15.73:0/3980004318 conn(0x7f61d00b06c0 msgr2=0x7f61d00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 -- 172.21.15.73:0/3980004318 shutdown_connections 2024-04-05T22:27:12.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.203+0000 7f61cf200640 1 -- 172.21.15.73:0/3980004318 wait complete. 2024-04-05T22:27:12.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.205+0000 7f61cf200640 1 Processor -- start 2024-04-05T22:27:12.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.205+0000 7f61cf200640 1 -- start start 2024-04-05T22:27:12.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 0x7f61d015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d015e620 0x7f61d015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0157890 con 0x7f61d0156ea0 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00c8390 con 0x7f61d015e620 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61cf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d015d5f0 con 0x7f61d006bc70 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.206+0000 7f61ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36286/0 (socket says 172.21.15.73:36286) 2024-04-05T22:27:12.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 -- 172.21.15.73:0/1564130321 learned_addr learned my addr 172.21.15.73:0/1564130321 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:12.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61cde00640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 0x7f61d015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61c7e00640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d015e620 0x7f61d015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 -- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 msgr2=0x7f61d015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 0x7f61d015c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 -- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d015e620 msgr2=0x7f61d015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d015e620 0x7f61d015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 -- 172.21.15.73:0/1564130321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009a70 con 0x7f61d0156ea0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61cde00640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 0x7f61d015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.207+0000 7f61ce800640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009e80 tx=0x7f61c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.208+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8040650 con 0x7f61d0156ea0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.208+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80407b0 con 0x7f61d0156ea0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.208+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d015ccc0 con 0x7f61d0156ea0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.208+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8040b60 con 0x7f61d0156ea0 2024-04-05T22:27:12.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.208+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00c7d40 con 0x7f61d0156ea0 2024-04-05T22:27:12.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.209+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d004e160 con 0x7f61d0156ea0 2024-04-05T22:27:12.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.210+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f61c8040cc0 con 0x7f61d0156ea0 2024-04-05T22:27:12.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.211+0000 7f61c6000640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 0x7f61ac06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.211+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f61c80ac360 con 0x7f61d0156ea0 2024-04-05T22:27:12.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.211+0000 7f61cde00640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 0x7f61ac06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.212+0000 7f61cde00640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 0x7f61ac06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61c00011e0 tx=0x7f61c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.215+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f61c8077450 con 0x7f61d0156ea0 2024-04-05T22:27:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.412+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61d015da60 con 0x7f61d0156ea0 2024-04-05T22:27:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.415+0000 7f61c6000640 1 -- 172.21.15.73:0/1564130321 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f61c807fe60 con 0x7f61d0156ea0 2024-04-05T22:27:12.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:12.416 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:09.109725+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 msgr2=0x7f61ac06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 0x7f61ac06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61c00011e0 tx=0x7f61c0009040 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 msgr2=0x7f61d0157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009e80 tx=0x7f61c8004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 shutdown_connections 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61ac06ad20 0x7f61ac06d1e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d0156ea0 0x7f61d0157350 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d015e620 0x7f61d015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 --2- 172.21.15.73:0/1564130321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61d006bc70 0x7f61d015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 >> 172.21.15.73:0/1564130321 conn(0x7f61d00b06c0 msgr2=0x7f61d0154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 shutdown_connections 2024-04-05T22:27:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.418+0000 7f61cf200640 1 -- 172.21.15.73:0/1564130321 wait complete. 2024-04-05T22:27:12.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:48.879992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:49.430091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:27:12.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:27:12.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.648+0000 7ff711a00640 1 Processor -- start 2024-04-05T22:27:12.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.648+0000 7ff711a00640 1 -- start start 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff70c160310 0x7ff70c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c06c2c0 0x7ff70c06acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0cd070 con 0x7ff70c0cb170 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c0cd330 con 0x7ff70c160310 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0ca150 con 0x7ff70c06c2c0 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff70be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff70c160310 0x7ff70c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff703e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff70c0cb170 0x7ff70c06bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51504/0 (socket says 172.21.15.73:51504) 2024-04-05T22:27:12.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff703e00640 1 -- 172.21.15.73:0/2059335477 learned_addr learned my addr 172.21.15.73:0/2059335477 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.650+0000 7ff711000640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c06c2c0 0x7ff70c06acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.651+0000 7ff70be00640 1 -- 172.21.15.73:0/2059335477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c06c2c0 msgr2=0x7ff70c06acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.651+0000 7ff70be00640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c06c2c0 0x7ff70c06acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.651+0000 7ff70be00640 1 -- 172.21.15.73:0/2059335477 >> v1:172.21.15.73:6789/0 conn(0x7ff70c0cb170 legacy=0x7ff70c06bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.651+0000 7ff70be00640 1 -- 172.21.15.73:0/2059335477 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0ca430 con 0x7ff70c160310 2024-04-05T22:27:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.651+0000 7ff70b400640 1 -- 172.21.15.73:0/2059335477 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523600831 0 0) 0x7ff70c0cd070 con 0x7ff70c0cb170 2024-04-05T22:27:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.652+0000 7ff70be00640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c160310 0x7ff70c0cac30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0097a0 tx=0x7ff6fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26c51780102b8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.652+0000 7ff70b400640 1 -- 172.21.15.73:0/2059335477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc031830 con 0x7ff70c160310 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff70b400640 1 -- 172.21.15.73:0/2059335477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc031990 con 0x7ff70c160310 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff70b400640 1 -- 172.21.15.73:0/2059335477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc031d20 con 0x7ff70c160310 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 -- 172.21.15.73:0/2059335477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c160310 msgr2=0x7ff70c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c160310 0x7ff70c0cac30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0097a0 tx=0x7ff6fc02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 -- 172.21.15.73:0/2059335477 shutdown_connections 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c06c2c0 0x7ff70c06acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 --2- 172.21.15.73:0/2059335477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c160310 0x7ff70c0cac30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 -- 172.21.15.73:0/2059335477 >> 172.21.15.73:0/2059335477 conn(0x7ff70c0cba40 msgr2=0x7ff70c0cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 -- 172.21.15.73:0/2059335477 shutdown_connections 2024-04-05T22:27:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.653+0000 7ff711a00640 1 -- 172.21.15.73:0/2059335477 wait complete. 2024-04-05T22:27:12.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.655+0000 7ff711a00640 1 Processor -- start 2024-04-05T22:27:12.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.655+0000 7ff711a00640 1 -- start start 2024-04-05T22:27:12.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c160310 0x7ff70c09a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 0x7ff70c0c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0c58c0 con 0x7ff70c15ba60 2024-04-05T22:27:12.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c09a9b0 con 0x7ff70c0c4f00 2024-04-05T22:27:12.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.656+0000 7ff711a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c09b0f0 con 0x7ff70c160310 2024-04-05T22:27:12.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36302/0 (socket says 172.21.15.73:36302) 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 -- 172.21.15.73:0/584475494 learned_addr learned my addr 172.21.15.73:0/584475494 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff70be00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c160310 0x7ff70c09a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff711000640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 0x7ff70c0c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 -- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c160310 msgr2=0x7ff70c09a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c160310 0x7ff70c09a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 -- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 msgr2=0x7ff70c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 0x7ff70c0c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 -- 172.21.15.73:0/584475494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc009400 con 0x7ff70c15ba60 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff711000640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 0x7ff70c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.657+0000 7ff703e00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c4d0 tx=0x7ff6f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.658+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40074e0 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.658+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f4007660 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.658+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4004a20 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.658+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c09b3d0 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.658+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c09b990 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.659+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c06f8a0 con 0x7ff70c15ba60 2024-04-05T22:27:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.660+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7ff6f4004b80 con 0x7ff70c15ba60 2024-04-05T22:27:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.661+0000 7ff708c00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 0x7ff6e806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.661+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff6f4088720 con 0x7ff70c15ba60 2024-04-05T22:27:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.661+0000 7ff70be00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 0x7ff6e806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.663+0000 7ff70be00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 0x7ff6e806d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc036000 tx=0x7ff6fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:12.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.666+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff6f4053780 con 0x7ff70c15ba60 2024-04-05T22:27:12.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.858+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff70c067480 con 0x7ff70c15ba60 2024-04-05T22:27:12.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.859+0000 7ff708c00640 1 -- 172.21.15.73:0/584475494 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6f4056da0 con 0x7ff70c15ba60 2024-04-05T22:27:12.859 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:27:12.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.861+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 msgr2=0x7ff6e806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.861+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 0x7ff6e806d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc036000 tx=0x7ff6fc008030 comp rx=0 tx=0).stop 2024-04-05T22:27:12.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.861+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 msgr2=0x7ff70c09c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:12.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.861+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c4d0 tx=0x7ff6f400c990 comp rx=0 tx=0).stop 2024-04-05T22:27:12.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 shutdown_connections 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6e806ad20 0x7ff6e806d1e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70c0c4f00 0x7ff70c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70c15ba60 0x7ff70c09c190 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 --2- 172.21.15.73:0/584475494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70c160310 0x7ff70c09a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 >> 172.21.15.73:0/584475494 conn(0x7ff70c0cba40 msgr2=0x7ff70c0b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 shutdown_connections 2024-04-05T22:27:12.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:12.862+0000 7ff711a00640 1 -- 172.21.15.73:0/584475494 wait complete. 2024-04-05T22:27:12.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:27:13.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.090+0000 7fb63ea00640 1 Processor -- start 2024-04-05T22:27:13.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.091+0000 7fb63ea00640 1 -- start start 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63815abe0 0x7fb6380712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb63806c9a0 0x7fb63806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb63806ba70 0x7fb6380cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63806bec0 con 0x7fb63806ba70 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63815afc0 con 0x7fb63806c9a0 2024-04-05T22:27:13.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63806f5b0 con 0x7fb63815abe0 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb63806ba70 0x7fb6380cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb63806ba70 0x7fb6380cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36312/0 (socket says 172.21.15.73:36312) 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.093+0000 7fb63ca00640 1 -- 172.21.15.73:0/1850397897 learned_addr learned my addr 172.21.15.73:0/1850397897 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb637e00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63815abe0 0x7fb6380712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb637400640 1 --2- 172.21.15.73:0/1850397897 >> v2:172.21.15.157:3300/0 conn(0x7fb63806c9a0 0x7fb63806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb63ca00640 1 -- 172.21.15.73:0/1850397897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63815abe0 msgr2=0x7fb6380712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb63ca00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63815abe0 0x7fb6380712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb63ca00640 1 -- 172.21.15.73:0/1850397897 >> v2:172.21.15.157:3300/0 conn(0x7fb63806c9a0 msgr2=0x7fb63806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb63ca00640 1 --2- 172.21.15.73:0/1850397897 >> v2:172.21.15.157:3300/0 conn(0x7fb63806c9a0 0x7fb63806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.094+0000 7fb63ca00640 1 -- 172.21.15.73:0/1850397897 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63806f890 con 0x7fb63806ba70 2024-04-05T22:27:13.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.095+0000 7fb63ca00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63806ba70 0x7fb6380cccb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb6380b3cb0 tx=0x7fb63002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461827a09f43e464 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.095+0000 7fb636a00640 1 -- 172.21.15.73:0/1850397897 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63002c950 con 0x7fb63806ba70 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.095+0000 7fb636a00640 1 -- 172.21.15.73:0/1850397897 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63002cab0 con 0x7fb63806ba70 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.095+0000 7fb636a00640 1 -- 172.21.15.73:0/1850397897 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63002ce40 con 0x7fb63806ba70 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 -- 172.21.15.73:0/1850397897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63806ba70 msgr2=0x7fb6380cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63806ba70 0x7fb6380cccb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb6380b3cb0 tx=0x7fb63002bdb0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 -- 172.21.15.73:0/1850397897 shutdown_connections 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63806ba70 0x7fb6380cccb0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 --2- 172.21.15.73:0/1850397897 >> v2:172.21.15.157:3300/0 conn(0x7fb63806c9a0 0x7fb63806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 --2- 172.21.15.73:0/1850397897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63815abe0 0x7fb6380712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 -- 172.21.15.73:0/1850397897 >> 172.21.15.73:0/1850397897 conn(0x7fb638069cb0 msgr2=0x7fb63806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 -- 172.21.15.73:0/1850397897 shutdown_connections 2024-04-05T22:27:13.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.096+0000 7fb63ea00640 1 -- 172.21.15.73:0/1850397897 wait complete. 2024-04-05T22:27:13.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.098+0000 7fb63ea00640 1 Processor -- start 2024-04-05T22:27:13.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.098+0000 7fb63ea00640 1 -- start start 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63806ba70 0x7fb638148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63806c9a0 0x7fb638070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6381f0650 con 0x7fb63815abe0 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638148a40 con 0x7fb63806ba70 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6381f08f0 con 0x7fb63806c9a0 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36316/0 (socket says 172.21.15.73:36316) 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.099+0000 7fb63ca00640 1 -- 172.21.15.73:0/3999601662 learned_addr learned my addr 172.21.15.73:0/3999601662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:13.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb637400640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63806c9a0 0x7fb638070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb637e00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63806ba70 0x7fb638148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 -- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63806c9a0 msgr2=0x7fb638070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63806c9a0 0x7fb638070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 -- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63806ba70 msgr2=0x7fb638148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63806ba70 0x7fb638148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 -- 172.21.15.73:0/3999601662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630009b30 con 0x7fb63815abe0 2024-04-05T22:27:13.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb63ca00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb63002bdf0 tx=0x7fb630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63003e450 con 0x7fb63815abe0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63003e5b0 con 0x7fb63815abe0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.100+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63003e960 con 0x7fb63815abe0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.101+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6381f0bd0 con 0x7fb63815abe0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.101+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb638148690 con 0x7fb63815abe0 2024-04-05T22:27:13.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.102+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6381f0480 con 0x7fb63815abe0 2024-04-05T22:27:13.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.103+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb63003eac0 con 0x7fb63815abe0 2024-04-05T22:27:13.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.104+0000 7fb62fe00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 0x7fb61806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.104+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6300ac3c0 con 0x7fb63815abe0 2024-04-05T22:27:13.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.105+0000 7fb637e00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 0x7fb61806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.106+0000 7fb637e00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 0x7fb61806d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb628006fd0 tx=0x7fb628008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.109+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb630077540 con 0x7fb63815abe0 2024-04-05T22:27:13.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.300+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb638068160 con 0x7fb63815abe0 2024-04-05T22:27:13.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.301+0000 7fb62fe00640 1 -- 172.21.15.73:0/3999601662 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb63007ab60 con 0x7fb63815abe0 2024-04-05T22:27:13.302 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:27:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 msgr2=0x7fb61806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 0x7fb61806d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb628006fd0 tx=0x7fb628008040 comp rx=0 tx=0).stop 2024-04-05T22:27:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 msgr2=0x7fb638146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb63002bdf0 tx=0x7fb630004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 shutdown_connections 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb61806ad20 0x7fb61806d1e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63815abe0 0x7fb638146590 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63806c9a0 0x7fb638070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 --2- 172.21.15.73:0/3999601662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63806ba70 0x7fb638148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 >> 172.21.15.73:0/3999601662 conn(0x7fb638069cb0 msgr2=0x7fb638154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 shutdown_connections 2024-04-05T22:27:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.304+0000 7fb63ea00640 1 -- 172.21.15.73:0/3999601662 wait complete. 2024-04-05T22:27:13.324 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:27:13.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.533+0000 7f111e200640 1 Processor -- start 2024-04-05T22:27:13.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.533+0000 7f111e200640 1 -- start start 2024-04-05T22:27:13.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f111e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11180cbe00 0x7f11180ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f111e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11180b32b0 0x7f11181647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f111e200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180b1b30 con 0x7f111806c630 2024-04-05T22:27:13.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f111e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11180b1e10 con 0x7f11180b32b0 2024-04-05T22:27:13.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f111e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180cc300 con 0x7f11180cbe00 2024-04-05T22:27:13.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f1117e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11180b32b0 0x7f11181647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f1117e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11180b32b0 0x7f11181647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36332/0 (socket says 172.21.15.73:36332) 2024-04-05T22:27:13.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.535+0000 7f1117e00640 1 -- 172.21.15.73:0/1628494434 learned_addr learned my addr 172.21.15.73:0/1628494434 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.536+0000 7f110fe00640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11180cbe00 0x7f11180ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.536+0000 7f1117e00640 1 -- 172.21.15.73:0/1628494434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11180cbe00 msgr2=0x7f11180ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.536+0000 7f1117e00640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11180cbe00 0x7f11180ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.536+0000 7f1117e00640 1 -- 172.21.15.73:0/1628494434 >> v1:172.21.15.157:6789/0 conn(0x7f111806c630 legacy=0x7f111806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.536+0000 7f1117e00640 1 -- 172.21.15.73:0/1628494434 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11180cc5e0 con 0x7f11180b32b0 2024-04-05T22:27:13.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.537+0000 7f1117e00640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11181647c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1110009e50 tx=0x7f111002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4228fafc188656f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.537+0000 7f1116a00640 1 -- 172.21.15.73:0/1628494434 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111002ca70 con 0x7f11180b32b0 2024-04-05T22:27:13.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.537+0000 7f1116a00640 1 -- 172.21.15.73:0/1628494434 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111002cbd0 con 0x7f11180b32b0 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.537+0000 7f1116a00640 1 -- 172.21.15.73:0/1628494434 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100343f0 con 0x7f11180b32b0 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 -- 172.21.15.73:0/1628494434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 msgr2=0x7f11181647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11181647c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1110009e50 tx=0x7f111002bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 -- 172.21.15.73:0/1628494434 shutdown_connections 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11181647c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 --2- 172.21.15.73:0/1628494434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11180cbe00 0x7f11180ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 -- 172.21.15.73:0/1628494434 >> 172.21.15.73:0/1628494434 conn(0x7f11180b0830 msgr2=0x7f11180b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 -- 172.21.15.73:0/1628494434 shutdown_connections 2024-04-05T22:27:13.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.538+0000 7f111e200640 1 -- 172.21.15.73:0/1628494434 wait complete. 2024-04-05T22:27:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.540+0000 7f111e200640 1 Processor -- start 2024-04-05T22:27:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.541+0000 7f111e200640 1 -- start start 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111806c630 0x7f11181f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11180cbe00 0x7f1118135280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c3aa0 con 0x7f11180b32b0 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181f3650 con 0x7f11180cbe00 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f111e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c3cb0 con 0x7f111806c630 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f110fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f110fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36338/0 (socket says 172.21.15.73:36338) 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f110fe00640 1 -- 172.21.15.73:0/4212839662 learned_addr learned my addr 172.21.15.73:0/4212839662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f1117400640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111806c630 0x7f11181f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.542+0000 7f1117e00640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11180cbe00 0x7f1118135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 -- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111806c630 msgr2=0x7f11181f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111806c630 0x7f11181f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 -- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11180cbe00 msgr2=0x7f1118135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11180cbe00 0x7f1118135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 -- 172.21.15.73:0/4212839662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009b30 con 0x7f11180b32b0 2024-04-05T22:27:13.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110fe00640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f110000d5f0 tx=0x7f110000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11000125f0 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100012750 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100012b00 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11180c3f90 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.543+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11180c4550 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.544+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1118068160 con 0x7f11180b32b0 2024-04-05T22:27:13.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.545+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f1100012c60 con 0x7f11180b32b0 2024-04-05T22:27:13.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.546+0000 7f110f400640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 0x7f10f406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.547+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11000899b0 con 0x7f11180b32b0 2024-04-05T22:27:13.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.547+0000 7f1117400640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 0x7f10f406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.548+0000 7f1117400640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 0x7f10f406d1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11181f5c00 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.551+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1100054a20 con 0x7f11180b32b0 2024-04-05T22:27:13.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.745+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11180ad800 con 0x7f11180b32b0 2024-04-05T22:27:13.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.746+0000 7f110f400640 1 -- 172.21.15.73:0/4212839662 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1100058040 con 0x7f11180b32b0 2024-04-05T22:27:13.747 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 msgr2=0x7f10f406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 0x7f10f406d1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11181f5c00 tx=0x7f1108009040 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 msgr2=0x7f11180caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f110000d5f0 tx=0x7f110000dab0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 shutdown_connections 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10f406ad20 0x7f10f406d1e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11180cbe00 0x7f1118135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11180b32b0 0x7f11180caa30 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 --2- 172.21.15.73:0/4212839662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f111806c630 0x7f11181f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 >> 172.21.15.73:0/4212839662 conn(0x7f11180b0830 msgr2=0x7f11180ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.749+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 shutdown_connections 2024-04-05T22:27:13.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.750+0000 7f111e200640 1 -- 172.21.15.73:0/4212839662 wait complete. 2024-04-05T22:27:13.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:27:13.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.976+0000 7f3a44000640 1 Processor -- start 2024-04-05T22:27:13.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.976+0000 7f3a44000640 1 -- start start 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a44000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a3c0d0bd0 0x7f3a3c0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a44000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0ceee0 con 0x7f3a3c0d1930 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a44000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0ce5b0 con 0x7f3a3c15cf70 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a44000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c0ce890 con 0x7f3a3c0d0bd0 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a40c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3a3c0d1930 0x7f3a3c0b2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51516/0 (socket says 172.21.15.73:51516) 2024-04-05T22:27:13.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a40c00640 1 -- 172.21.15.73:0/1106980739 learned_addr learned my addr 172.21.15.73:0/1106980739 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a42000640 1 --2- 172.21.15.73:0/1106980739 >> v2:172.21.15.157:3300/0 conn(0x7f3a3c0d0bd0 0x7f3a3c0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.978+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724343891 0 0) 0x7f3a3c0ceee0 con 0x7f3a3c0d1930 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a24003520 con 0x7f3a3c0d1930 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375277376 0 0) 0x7f3a24003520 con 0x7f3a3c0d1930 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 >> v2:172.21.15.157:3300/0 conn(0x7f3a3c0d0bd0 msgr2=0x7f3a3c0d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 --2- 172.21.15.73:0/1106980739 >> v2:172.21.15.157:3300/0 conn(0x7f3a3c0d0bd0 0x7f3a3c0d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 >> v1:172.21.15.157:6789/0 conn(0x7f3a3c15cf70 legacy=0x7f3a3c0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0ceb70 con 0x7f3a3c0d1930 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3a28001670 con 0x7f3a3c0d1930 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 >> v1:172.21.15.73:6789/0 conn(0x7f3a3c0d1930 legacy=0x7f3a3c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24003970 0x7f3a24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a240047c0 0x7f3a24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 0x7f3a2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0ce890 con 0x7f3a24005180 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2400c120 con 0x7f3a240047c0 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.979+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2400c2e0 con 0x7f3a24003970 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 0x7f3a2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a40c00640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a240047c0 0x7f3a24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 -- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24003970 msgr2=0x7f3a24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a41600640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24003970 0x7f3a24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24003970 0x7f3a24003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 -- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a240047c0 msgr2=0x7f3a24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a240047c0 0x7f3a24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.980+0000 7f3a42000640 1 -- 172.21.15.73:0/1106980739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24003520 con 0x7f3a24005180 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.981+0000 7f3a42000640 1 -- 172.21.15.73:0/1106980739 reap_dead start 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.981+0000 7f3a42000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 0x7f3a2400b980 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009f90 tx=0x7f3a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38107531a0bb3343 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.981+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3402cc20 con 0x7f3a24005180 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.981+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3400f040 con 0x7f3a24005180 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.981+0000 7f3a3be00640 1 -- 172.21.15.73:0/1106980739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34003990 con 0x7f3a24005180 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 -- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 msgr2=0x7f3a2400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 0x7f3a2400b980 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009f90 tx=0x7f3a240057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 -- 172.21.15.73:0/1106980739 shutdown_connections 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 --2- 172.21.15.73:0/1106980739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a24005180 0x7f3a2400b980 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 -- 172.21.15.73:0/1106980739 >> 172.21.15.73:0/1106980739 conn(0x7f3a3c0d2df0 msgr2=0x7f3a3c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 -- 172.21.15.73:0/1106980739 shutdown_connections 2024-04-05T22:27:13.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.982+0000 7f3a44000640 1 -- 172.21.15.73:0/1106980739 wait complete. 2024-04-05T22:27:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.984+0000 7f3a44000640 1 Processor -- start 2024-04-05T22:27:13.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.984+0000 7f3a44000640 1 -- start start 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24005180 0x7f3a3c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 0x7f3a3c14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0c6f60 con 0x7f3a3c14fef0 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0cbaa0 con 0x7f3a3c0b8680 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a44000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0cbd60 con 0x7f3a24005180 2024-04-05T22:27:13.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a40c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a40c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36362/0 (socket says 172.21.15.73:36362) 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.985+0000 7f3a40c00640 1 -- 172.21.15.73:0/2529229738 learned_addr learned my addr 172.21.15.73:0/2529229738 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a42000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 0x7f3a3c14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 -- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24005180 msgr2=0x7f3a3c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24005180 0x7f3a3c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 -- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 msgr2=0x7f3a3c14f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 0x7f3a3c14f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 -- 172.21.15.73:0/2529229738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0cc040 con 0x7f3a3c14fef0 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a42000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 0x7f3a3c14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:13.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a40c00640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800e700 tx=0x7f3a2800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.986+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800c6c0 con 0x7f3a3c14fef0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.987+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2800c840 con 0x7f3a3c14fef0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.987+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800cbf0 con 0x7f3a3c14fef0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.987+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0cc320 con 0x7f3a3c14fef0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.987+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0cc840 con 0x7f3a3c14fef0 2024-04-05T22:27:13.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.988+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c0cc510 con 0x7f3a3c14fef0 2024-04-05T22:27:13.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.989+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f3a2800cd50 con 0x7f3a3c14fef0 2024-04-05T22:27:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.990+0000 7f3a39600640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 0x7f3a1c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.990+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3a28089430 con 0x7f3a3c14fef0 2024-04-05T22:27:13.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.991+0000 7f3a41600640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 0x7f3a1c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:13.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.992+0000 7f3a41600640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 0x7f3a1c06d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000f60 tx=0x7f3a30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:13.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:13.995+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3a28054490 con 0x7f3a3c14fef0 2024-04-05T22:27:14.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.205+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a3c068160 con 0x7f3a3c14fef0 2024-04-05T22:27:14.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.206+0000 7f3a39600640 1 -- 172.21.15.73:0/2529229738 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) v1 ==== 80+0+124 (secure 0 0 0) 0x7f3a28057ab0 con 0x7f3a3c14fef0 2024-04-05T22:27:14.206 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.157:6835/2227587240 2024-04-06T22:27:06.074507+0000 2024-04-05T22:27:14.206 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.157:6834/2227587240 2024-04-06T22:27:06.074507+0000 2024-04-05T22:27:14.207 INFO:teuthology.orchestra.run.smithi073.stderr:listed 2 entries 2024-04-05T22:27:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 msgr2=0x7f3a1c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 0x7f3a1c06d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000f60 tx=0x7f3a30009040 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 msgr2=0x7f3a3c0b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800e700 tx=0x7f3a2800ebc0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 shutdown_connections 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a1c06ad20 0x7f3a1c06d1e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a3c0b8680 0x7f3a3c14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c14fef0 0x7f3a3c0b8140 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 --2- 172.21.15.73:0/2529229738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a24005180 0x7f3a3c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 >> 172.21.15.73:0/2529229738 conn(0x7f3a3c0d2df0 msgr2=0x7f3a3c0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 shutdown_connections 2024-04-05T22:27:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.209+0000 7f3a44000640 1 -- 172.21.15.73:0/2529229738 wait complete. 2024-04-05T22:27:14.228 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:27:14.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:14.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:14.338 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:27:14.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:14.338 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:27:14.387 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:14.387 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:14.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:27:14.445 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:14.445 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:14.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:27:14.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:14.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.710+0000 7f7941200640 1 Processor -- start 2024-04-05T22:27:14.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.710+0000 7f7941200640 1 -- start start 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7941200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 0x7f793c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7941200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f793c15e850 0x7f793c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7941200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c06f5b0 con 0x7f793c1587d0 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7941200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c06d500 con 0x7f793c15e850 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7941200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c15f220 con 0x7f793c06cb80 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7939e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f793c1587d0 0x7f793c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51532/0 (socket says 172.21.15.73:51532) 2024-04-05T22:27:14.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7939e00640 1 -- 172.21.15.73:0/1393515726 learned_addr learned my addr 172.21.15.73:0/1393515726 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:14.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f793a800640 1 --2- 172.21.15.73:0/1393515726 >> v2:172.21.15.157:3300/0 conn(0x7f793c15e850 0x7f793c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.712+0000 7f7939400640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 0x7f793c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611023439 0 0) 0x7f793c06f5b0 con 0x7f793c1587d0 2024-04-05T22:27:14.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7920003520 con 0x7f793c1587d0 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974481947 0 0) 0x7f7920003520 con 0x7f793c1587d0 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 msgr2=0x7f793c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 0x7f793c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 >> v2:172.21.15.157:3300/0 conn(0x7f793c15e850 msgr2=0x7f793c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 --2- 172.21.15.73:0/1393515726 >> v2:172.21.15.157:3300/0 conn(0x7f793c15e850 0x7f793c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c070410 con 0x7f793c1587d0 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f792c001670 con 0x7f793c1587d0 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 >> v1:172.21.15.73:6789/0 conn(0x7f793c1587d0 legacy=0x7f793c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 0x7f79200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7920003f20 0x7f79200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f7920005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06d500 con 0x7f7920004d00 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c15f220 con 0x7f7920003f20 2024-04-05T22:27:14.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792000b980 con 0x7f793c06cb80 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.713+0000 7f793a800640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f7920005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f7939400640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7920003f20 0x7f79200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f793a800640 1 -- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 msgr2=0x7f79200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f793a800640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c06cb80 0x7f79200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f793a800640 1 -- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7920003f20 msgr2=0x7f79200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f793a800640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7920003f20 0x7f79200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f793a800640 1 -- 172.21.15.73:0/1393515726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920003520 con 0x7f7920004d00 2024-04-05T22:27:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.714+0000 7f7939400640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7920003f20 0x7f79200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:14.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.715+0000 7f793a800640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f7920005180 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7934009fc0 tx=0x7f79200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfe8d28cdd21f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.715+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793402cc60 con 0x7f7920004d00 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.715+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793400f040 con 0x7f7920004d00 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.715+0000 7f7938a00640 1 -- 172.21.15.73:0/1393515726 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934034440 con 0x7f7920004d00 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 -- 172.21.15.73:0/1393515726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 msgr2=0x7f7920005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 --2- 172.21.15.73:0/1393515726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f7920005180 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7934009fc0 tx=0x7f79200057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f793a800640 1 -- 172.21.15.73:0/1393515726 reap_dead start 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 -- 172.21.15.73:0/1393515726 shutdown_connections 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 -- 172.21.15.73:0/1393515726 >> 172.21.15.73:0/1393515726 conn(0x7f793c0ce8e0 msgr2=0x7f793c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 -- 172.21.15.73:0/1393515726 shutdown_connections 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.716+0000 7f7941200640 1 -- 172.21.15.73:0/1393515726 wait complete. 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.718+0000 7f7941200640 1 Processor -- start 2024-04-05T22:27:14.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.718+0000 7f7941200640 1 -- start start 2024-04-05T22:27:14.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c0a87b0 0x7f793c1eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c153f50 0x7f793c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c154910 con 0x7f7920004d00 2024-04-05T22:27:14.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0a8e90 con 0x7f793c153f50 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7941200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0a90e0 con 0x7f793c0a87b0 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36390/0 (socket says 172.21.15.73:36390) 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 -- 172.21.15.73:0/3375892445 learned_addr learned my addr 172.21.15.73:0/3375892445 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f793a800640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c153f50 0x7f793c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 -- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c0a87b0 msgr2=0x7f793c1eec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c0a87b0 0x7f793c1eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 -- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c153f50 msgr2=0x7f793c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c153f50 0x7f793c1543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.719+0000 7f7939e00640 1 -- 172.21.15.73:0/3375892445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0a93c0 con 0x7f7920004d00 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7939e00640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f792c001eb0 tx=0x7f792c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0074e0 con 0x7f7920004d00 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c007640 con 0x7f7920004d00 2024-04-05T22:27:14.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c004a00 con 0x7f7920004d00 2024-04-05T22:27:14.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c1ef620 con 0x7f7920004d00 2024-04-05T22:27:14.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.720+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c06a460 con 0x7f7920004d00 2024-04-05T22:27:14.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.721+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c066af0 con 0x7f7920004d00 2024-04-05T22:27:14.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.722+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f792c004b60 con 0x7f7920004d00 2024-04-05T22:27:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.724+0000 7f7932000640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 0x7f791c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.724+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f792c088b10 con 0x7f7920004d00 2024-04-05T22:27:14.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.724+0000 7f7939400640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 0x7f791c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:14.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.725+0000 7f7939400640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 0x7f791c06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f793c15a2c0 tx=0x7f7928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:14.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.728+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f792c053b80 con 0x7f7920004d00 2024-04-05T22:27:14.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.959+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793c1efb40 con 0x7f7920004d00 2024-04-05T22:27:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.960+0000 7f7932000640 1 -- 172.21.15.73:0/3375892445 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f793c1efb40 con 0x7f7920004d00 2024-04-05T22:27:14.960 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:14.960 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/602096336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":602096336},{"type":"v1","addr":"172.21.15.157:6835","nonce":602096336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:27:14.961 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 55 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 msgr2=0x7f791c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 0x7f791c06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f793c15a2c0 tx=0x7f7928008040 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 msgr2=0x7f793c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f792c001eb0 tx=0x7f792c00ccd0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 shutdown_connections 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f791c06ad20 0x7f791c06d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c153f50 0x7f793c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c0a87b0 0x7f793c1eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.962+0000 7f7941200640 1 --2- 172.21.15.73:0/3375892445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7920004d00 0x7f793c069f20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.963+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 >> 172.21.15.73:0/3375892445 conn(0x7f793c0ce8e0 msgr2=0x7f793c0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.963+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 shutdown_connections 2024-04-05T22:27:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:14.963+0000 7f7941200640 1 -- 172.21.15.73:0/3375892445 wait complete. 2024-04-05T22:27:14.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:15.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.185+0000 7fe990e00640 1 Processor -- start 2024-04-05T22:27:15.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.185+0000 7fe990e00640 1 -- start start 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.186+0000 7fe990e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 0x7fe98c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.186+0000 7fe990e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c06f5b0 con 0x7fe98c1587d0 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.186+0000 7fe990e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c070410 con 0x7fe98c15e850 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe990e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06d500 con 0x7fe98c06cb80 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe989e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe98c1587d0 0x7fe98c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51540/0 (socket says 172.21.15.73:51540) 2024-04-05T22:27:15.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe989e00640 1 -- 172.21.15.73:0/2593310532 learned_addr learned my addr 172.21.15.73:0/2593310532 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe981400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 0x7fe98c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639062350 0 0) 0x7fe98c06f5b0 con 0x7fe98c1587d0 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.187+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe970003520 con 0x7fe98c1587d0 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1894136447 0 0) 0x7fe970003520 con 0x7fe98c1587d0 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 msgr2=0x7fe98c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 0x7fe98c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 >> v1:172.21.15.157:6789/0 conn(0x7fe98c15e850 legacy=0x7fe98c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0ccef0 con 0x7fe98c1587d0 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe97c0019f0 con 0x7fe98c1587d0 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 >> v1:172.21.15.73:6789/0 conn(0x7fe98c1587d0 legacy=0x7fe98c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 0x7fe9700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe970003fd0 0x7fe9700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe970004d00 0x7fe970005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06d500 con 0x7fe970003fd0 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97000bb40 con 0x7fe970004d00 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97000bd00 con 0x7fe98c06cb80 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe970003fd0 0x7fe9700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe98a800640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe970004d00 0x7fe970005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 -- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 msgr2=0x7fe9700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98c06cb80 0x7fe9700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 -- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe970004d00 msgr2=0x7fe970005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe970004d00 0x7fe970005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe981400640 1 -- 172.21.15.73:0/2593310532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970003520 con 0x7fe970003fd0 2024-04-05T22:27:15.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.188+0000 7fe98a800640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe970004d00 0x7fe970005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.189+0000 7fe981400640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe970003fd0 0x7fe9700047c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe97400ba30 tx=0x7fe9700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c8a72db990ab8c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974032680 con 0x7fe970003fd0 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9740327e0 con 0x7fe970003fd0 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe989400640 1 -- 172.21.15.73:0/2593310532 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974032b70 con 0x7fe970003fd0 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe990e00640 1 -- 172.21.15.73:0/2593310532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe970003fd0 msgr2=0x7fe9700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe990e00640 1 --2- 172.21.15.73:0/2593310532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe970003fd0 0x7fe9700047c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe97400ba30 tx=0x7fe9700057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe98a800640 1 -- 172.21.15.73:0/2593310532 reap_dead start 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe990e00640 1 -- 172.21.15.73:0/2593310532 shutdown_connections 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.190+0000 7fe990e00640 1 -- 172.21.15.73:0/2593310532 >> 172.21.15.73:0/2593310532 conn(0x7fe98c0ce8e0 msgr2=0x7fe98c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.191+0000 7fe990e00640 1 -- 172.21.15.73:0/2593310532 shutdown_connections 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.191+0000 7fe990e00640 1 -- 172.21.15.73:0/2593310532 wait complete. 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.192+0000 7fe990e00640 1 Processor -- start 2024-04-05T22:27:15.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.192+0000 7fe990e00640 1 -- start start 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe970003fd0 0x7fe98c0a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98c0af1d0 0x7fe98c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0afb90 con 0x7fe98c13f1c0 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c15cef0 con 0x7fe98c0af1d0 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.193+0000 7fe990e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c15d1d0 con 0x7fe970003fd0 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36408/0 (socket says 172.21.15.73:36408) 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 -- 172.21.15.73:0/916506387 learned_addr learned my addr 172.21.15.73:0/916506387 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 -- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe970003fd0 msgr2=0x7fe98c0a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe970003fd0 0x7fe98c0a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe98a800640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98c0af1d0 0x7fe98c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 -- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98c0af1d0 msgr2=0x7fe98c0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98c0af1d0 0x7fe98c0af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 -- 172.21.15.73:0/916506387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c15d4b0 con 0x7fe98c13f1c0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.194+0000 7fe981400640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe97402bf00 tx=0x7fe974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.195+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974030aa0 con 0x7fe98c13f1c0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.195+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974030c20 con 0x7fe98c13f1c0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.195+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c15d790 con 0x7fe98c13f1c0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.195+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97403b420 con 0x7fe98c13f1c0 2024-04-05T22:27:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.195+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c15caa0 con 0x7fe98c13f1c0 2024-04-05T22:27:15.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.196+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c066af0 con 0x7fe98c13f1c0 2024-04-05T22:27:15.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.197+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe97403b580 con 0x7fe98c13f1c0 2024-04-05T22:27:15.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.198+0000 7fe982a00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 0x7fe96806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.198+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe9740ac150 con 0x7fe98c13f1c0 2024-04-05T22:27:15.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.198+0000 7fe989e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 0x7fe96806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.200+0000 7fe989e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 0x7fe96806d210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe97c001f60 tx=0x7fe97c0018a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.202+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe97407abc0 con 0x7fe98c13f1c0 2024-04-05T22:27:15.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.397+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe98c068160 con 0x7fe98c13f1c0 2024-04-05T22:27:15.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.399+0000 7fe982a00640 1 -- 172.21.15.73:0/916506387 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16223 (secure 0 0 0) 0x7fe97407a750 con 0x7fe98c13f1c0 2024-04-05T22:27:15.400 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:15.400 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:09.109725+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 msgr2=0x7fe96806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 0x7fe96806d210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe97c001f60 tx=0x7fe97c0018a0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 msgr2=0x7fe98c15c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe97402bf00 tx=0x7fe974004730 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 shutdown_connections 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe96806ad50 0x7fe96806d210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98c0af1d0 0x7fe98c0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c13f1c0 0x7fe98c15c560 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 --2- 172.21.15.73:0/916506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe970003fd0 0x7fe98c0a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 >> 172.21.15.73:0/916506387 conn(0x7fe98c0ce8e0 msgr2=0x7fe98c069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 shutdown_connections 2024-04-05T22:27:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.402+0000 7fe990e00640 1 -- 172.21.15.73:0/916506387 wait complete. 2024-04-05T22:27:15.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:26:48.879992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:26:49.430091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:27:15.422 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:27:15.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.627+0000 7fe65b400640 1 Processor -- start 2024-04-05T22:27:15.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.627+0000 7fe65b400640 1 -- start start 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.628+0000 7fe65b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe65415abe0 0x7fe6540712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe65b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe65406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe65b400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65406bec0 con 0x7fe65406ba70 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe65b400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65415afc0 con 0x7fe65415abe0 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe65b400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65406f5b0 con 0x7fe65406c9a0 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe659400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe65406ba70 0x7fe6540cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51550/0 (socket says 172.21.15.73:51550) 2024-04-05T22:27:15.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe659400640 1 -- 172.21.15.73:0/423562450 learned_addr learned my addr 172.21.15.73:0/423562450 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe658a00640 1 --2- 172.21.15.73:0/423562450 >> v2:172.21.15.157:3300/0 conn(0x7fe65415abe0 0x7fe6540712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.629+0000 7fe653e00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe65406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064243856 0 0) 0x7fe64c001490 con 0x7fe65406ba70 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe63c003520 con 0x7fe65406ba70 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291170389 0 0) 0x7fe63c003520 con 0x7fe65406ba70 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 msgr2=0x7fe65406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe65406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 >> v2:172.21.15.157:3300/0 conn(0x7fe65415abe0 msgr2=0x7fe6540712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 --2- 172.21.15.73:0/423562450 >> v2:172.21.15.157:3300/0 conn(0x7fe65415abe0 0x7fe6540712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65406f890 con 0x7fe65406ba70 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe64c001b80 con 0x7fe65406ba70 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 >> v1:172.21.15.73:6789/0 conn(0x7fe65406ba70 legacy=0x7fe6540cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe63c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c003ef0 0x7fe63c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c004d00 0x7fe63c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415afc0 con 0x7fe63c003ef0 2024-04-05T22:27:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65406f5b0 con 0x7fe63c004d00 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.630+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c00ba60 con 0x7fe65406c9a0 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c003ef0 0x7fe63c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe658a00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe63c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 -- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 msgr2=0x7fe63c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65406c9a0 0x7fe63c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 -- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c004d00 msgr2=0x7fe63c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c004d00 0x7fe63c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.631+0000 7fe653e00640 1 -- 172.21.15.73:0/423562450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c003520 con 0x7fe63c003ef0 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.632+0000 7fe653e00640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c003ef0 0x7fe63c0047c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe640009ac0 tx=0x7fe63c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2907ec50ea386fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.632+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64002c6b0 con 0x7fe63c003ef0 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.632+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64002c810 con 0x7fe63c003ef0 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.632+0000 7fe653400640 1 -- 172.21.15.73:0/423562450 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64002cba0 con 0x7fe63c003ef0 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 -- 172.21.15.73:0/423562450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c003ef0 msgr2=0x7fe63c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 --2- 172.21.15.73:0/423562450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c003ef0 0x7fe63c0047c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe640009ac0 tx=0x7fe63c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe659400640 1 -- 172.21.15.73:0/423562450 reap_dead start 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 -- 172.21.15.73:0/423562450 shutdown_connections 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 -- 172.21.15.73:0/423562450 >> 172.21.15.73:0/423562450 conn(0x7fe654069cb0 msgr2=0x7fe65406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 -- 172.21.15.73:0/423562450 shutdown_connections 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.633+0000 7fe65b400640 1 -- 172.21.15.73:0/423562450 wait complete. 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.635+0000 7fe65b400640 1 Processor -- start 2024-04-05T22:27:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.635+0000 7fe65b400640 1 -- start start 2024-04-05T22:27:15.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c003ef0 0x7fe65414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 0x7fe6541501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415a470 con 0x7fe6541f68f0 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540acc00 con 0x7fe65414fd40 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe65b400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe654152140 con 0x7fe63c003ef0 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe653e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe653e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36428/0 (socket says 172.21.15.73:36428) 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.636+0000 7fe653e00640 1 -- 172.21.15.73:0/4275118134 learned_addr learned my addr 172.21.15.73:0/4275118134 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe659400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 0x7fe6541501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe658a00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c003ef0 0x7fe65414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 -- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c003ef0 msgr2=0x7fe65414c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c003ef0 0x7fe65414c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 -- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 msgr2=0x7fe6541501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 0x7fe6541501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 -- 172.21.15.73:0/4275118134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540ac8d0 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe659400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 0x7fe6541501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe653e00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe64002c250 tx=0x7fe640004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640003810 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe640003970 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640003ce0 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.637+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6540b3380 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.638+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6540b3940 con 0x7fe6541f68f0 2024-04-05T22:27:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.639+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6540acdb0 con 0x7fe6541f68f0 2024-04-05T22:27:15.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.640+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe640047450 con 0x7fe6541f68f0 2024-04-05T22:27:15.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.641+0000 7fe650c00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 0x7fe63406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:15.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.641+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6400ab700 con 0x7fe6541f68f0 2024-04-05T22:27:15.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.641+0000 7fe658a00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 0x7fe63406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:15.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.643+0000 7fe658a00640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 0x7fe63406d210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6480099d0 tx=0x7fe648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:15.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.645+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe6540acdb0 con 0x7fe6541f68f0 2024-04-05T22:27:15.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.841+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe65415a470 con 0x7fe6541f68f0 2024-04-05T22:27:15.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.842+0000 7fe650c00640 1 -- 172.21.15.73:0/4275118134 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe640076930 con 0x7fe6541f68f0 2024-04-05T22:27:15.842 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 msgr2=0x7fe63406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 0x7fe63406d210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6480099d0 tx=0x7fe648008040 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 msgr2=0x7fe65414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe64002c250 tx=0x7fe640004730 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 shutdown_connections 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe63406ad50 0x7fe63406d210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe65414fd40 0x7fe6541501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6541f68f0 0x7fe65414f800 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 --2- 172.21.15.73:0/4275118134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c003ef0 0x7fe65414c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.844+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 >> 172.21.15.73:0/4275118134 conn(0x7fe654069cb0 msgr2=0x7fe6540ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:15.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.845+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 shutdown_connections 2024-04-05T22:27:15.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:15.845+0000 7fe65b400640 1 -- 172.21.15.73:0/4275118134 wait complete. 2024-04-05T22:27:15.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:27:16.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.075+0000 7ffa9ac00640 1 Processor -- start 2024-04-05T22:27:16.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.075+0000 7ffa9ac00640 1 -- start start 2024-04-05T22:27:16.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.077+0000 7ffa9ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa94070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.077+0000 7ffa9ac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa9415dd90 0x7ffa9406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.077+0000 7ffa9ac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa94071900 con 0x7ffa940714a0 2024-04-05T22:27:16.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.077+0000 7ffa9ac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9415ae50 con 0x7ffa9415dd90 2024-04-05T22:27:16.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.077+0000 7ffa9ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9415e3b0 con 0x7ffa9415aa70 2024-04-05T22:27:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.078+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa94070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.078+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa94070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:46786/0 (socket says 172.21.15.73:46786) 2024-04-05T22:27:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.078+0000 7ffa93e00640 1 -- 172.21.15.73:0/295127628 learned_addr learned my addr 172.21.15.73:0/295127628 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.078+0000 7ffa98c00640 1 --2- 172.21.15.73:0/295127628 >> v2:172.21.15.157:3300/0 conn(0x7ffa9415dd90 0x7ffa9406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.079+0000 7ffa98c00640 1 -- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 msgr2=0x7ffa94070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.079+0000 7ffa98c00640 1 --2- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa94070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.079+0000 7ffa98c00640 1 -- 172.21.15.73:0/295127628 >> v1:172.21.15.157:6789/0 conn(0x7ffa940714a0 legacy=0x7ffa9415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.079+0000 7ffa98c00640 1 -- 172.21.15.73:0/295127628 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa940ac9e0 con 0x7ffa9415dd90 2024-04-05T22:27:16.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.080+0000 7ffa98c00640 1 --2- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9406ba40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009d90 tx=0x7ffa8c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4502cdf96237425c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.081+0000 7ffa93400640 1 -- 172.21.15.73:0/295127628 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c02cc60 con 0x7ffa9415dd90 2024-04-05T22:27:16.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.081+0000 7ffa93400640 1 -- 172.21.15.73:0/295127628 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c02cdc0 con 0x7ffa9415dd90 2024-04-05T22:27:16.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.081+0000 7ffa93400640 1 -- 172.21.15.73:0/295127628 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c034660 con 0x7ffa9415dd90 2024-04-05T22:27:16.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 -- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 msgr2=0x7ffa9406ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9406ba40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009d90 tx=0x7ffa8c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 -- 172.21.15.73:0/295127628 shutdown_connections 2024-04-05T22:27:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9406ba40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/295127628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa94070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 -- 172.21.15.73:0/295127628 >> 172.21.15.73:0/295127628 conn(0x7ffa94069cb0 msgr2=0x7ffa9406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 -- 172.21.15.73:0/295127628 shutdown_connections 2024-04-05T22:27:16.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.082+0000 7ffa9ac00640 1 -- 172.21.15.73:0/295127628 wait complete. 2024-04-05T22:27:16.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.084+0000 7ffa9ac00640 1 Processor -- start 2024-04-05T22:27:16.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.085+0000 7ffa9ac00640 1 -- start start 2024-04-05T22:27:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa941516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941539a0 con 0x7ffa940714a0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941530b0 con 0x7ffa9415dd90 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa9ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94153350 con 0x7ffa9415aa70 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49334/0 (socket says 172.21.15.73:49334) 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa93e00640 1 -- 172.21.15.73:0/3480484137 learned_addr learned my addr 172.21.15.73:0/3480484137 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa98c00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.086+0000 7ffa8be00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa941516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 -- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 msgr2=0x7ffa941516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa941516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 -- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 msgr2=0x7ffa9415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9415f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 -- 172.21.15.73:0/3480484137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c009a70 con 0x7ffa940714a0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa93e00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009fd0 tx=0x7ffa8400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400c940 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400caa0 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400ce30 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa94153630 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.087+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa940ae5e0 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.089+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa94153b50 con 0x7ffa940714a0 2024-04-05T22:27:16.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.089+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7ffa84013470 con 0x7ffa940714a0 2024-04-05T22:27:16.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.091+0000 7ffa90c00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 0x7ffa7006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.091+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffa84089750 con 0x7ffa940714a0 2024-04-05T22:27:16.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.091+0000 7ffa8be00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 0x7ffa7006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.093+0000 7ffa8be00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 0x7ffa7006d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa940cb2a0 tx=0x7ffa7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.096+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ffa84054720 con 0x7ffa940714a0 2024-04-05T22:27:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.290+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa94068160 con 0x7ffa940714a0 2024-04-05T22:27:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.290+0000 7ffa90c00640 1 -- 172.21.15.73:0/3480484137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa84057d40 con 0x7ffa940714a0 2024-04-05T22:27:16.291 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:27:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 msgr2=0x7ffa7006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 0x7ffa7006d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa940cb2a0 tx=0x7ffa7c009040 comp rx=0 tx=0).stop 2024-04-05T22:27:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 msgr2=0x7ffa9415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009fd0 tx=0x7ffa8400ed40 comp rx=0 tx=0).stop 2024-04-05T22:27:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 shutdown_connections 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa7006ad20 0x7ffa7006d1e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa9415dd90 0x7ffa9415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9415aa70 0x7ffa941516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 --2- 172.21.15.73:0/3480484137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa940714a0 0x7ffa9415c850 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 >> 172.21.15.73:0/3480484137 conn(0x7ffa94069cb0 msgr2=0x7ffa9415d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 shutdown_connections 2024-04-05T22:27:16.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.293+0000 7ffa9ac00640 1 -- 172.21.15.73:0/3480484137 wait complete. 2024-04-05T22:27:16.312 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:27:16.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.527+0000 7f4bd2800640 1 Processor -- start 2024-04-05T22:27:16.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.527+0000 7f4bd2800640 1 -- start start 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bd2800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bcc0cbe00 0x7f4bcc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bd2800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc0b1b30 con 0x7f4bcc06c630 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bd2800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc0b1e10 con 0x7f4bcc0b32b0 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bd2800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc0cc300 con 0x7f4bcc0cbe00 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bcb400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4bcc06c630 0x7f4bcc06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49300/0 (socket says 172.21.15.73:49300) 2024-04-05T22:27:16.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.529+0000 7f4bcb400640 1 -- 172.21.15.73:0/3721108271 learned_addr learned my addr 172.21.15.73:0/3721108271 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:16.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bcaa00640 1 --2- 172.21.15.73:0/3721108271 >> v2:172.21.15.157:3300/0 conn(0x7f4bcc0cbe00 0x7f4bcc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623937853 0 0) 0x7f4bcc0b1b30 con 0x7f4bcc06c630 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb0003520 con 0x7f4bcc06c630 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953527229 0 0) 0x7f4bb0003520 con 0x7f4bcc06c630 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 >> v2:172.21.15.157:3300/0 conn(0x7f4bcc0cbe00 msgr2=0x7f4bcc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 --2- 172.21.15.73:0/3721108271 >> v2:172.21.15.157:3300/0 conn(0x7f4bcc0cbe00 0x7f4bcc0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 >> v1:172.21.15.157:6789/0 conn(0x7f4bcc0b32b0 legacy=0x7f4bcc1647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0cc5e0 con 0x7f4bcc06c630 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4bbc001670 con 0x7f4bcc06c630 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 >> v1:172.21.15.73:6789/0 conn(0x7f4bcc06c630 legacy=0x7f4bcc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.530+0000 7f4bca000640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb00039c0 0x7f4bb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bca000640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00047c0 0x7f4bb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bca000640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 0x7f4bb000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0cc300 con 0x7f4bb0005180 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb000c080 con 0x7f4bb00047c0 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb000c240 con 0x7f4bb00039c0 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 0x7f4bb000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcb400640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb00039c0 0x7f4bb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcaa00640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00047c0 0x7f4bb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 -- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb00039c0 msgr2=0x7f4bb0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb00039c0 0x7f4bb0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 -- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00047c0 msgr2=0x7f4bb0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00047c0 0x7f4bb0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.531+0000 7f4bcbe00640 1 -- 172.21.15.73:0/3721108271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0003520 con 0x7f4bb0005180 2024-04-05T22:27:16.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.532+0000 7f4bcbe00640 1 -- 172.21.15.73:0/3721108271 reap_dead start 2024-04-05T22:27:16.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.532+0000 7f4bcbe00640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 0x7f4bb000b980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4001e20 tx=0x7f4bb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28916b3f23eaea1f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4003b70 con 0x7f4bb0005180 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc4003cd0 con 0x7f4bb0005180 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bca000640 1 -- 172.21.15.73:0/3721108271 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4005310 con 0x7f4bb0005180 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bd2800640 1 -- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 msgr2=0x7f4bb000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bd2800640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 0x7f4bb000b980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4001e20 tx=0x7f4bb00057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bd2800640 1 -- 172.21.15.73:0/3721108271 shutdown_connections 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bd2800640 1 --2- 172.21.15.73:0/3721108271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bb0005180 0x7f4bb000b980 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.533+0000 7f4bd2800640 1 -- 172.21.15.73:0/3721108271 >> 172.21.15.73:0/3721108271 conn(0x7f4bcc0b0830 msgr2=0x7f4bcc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.534+0000 7f4bd2800640 1 -- 172.21.15.73:0/3721108271 shutdown_connections 2024-04-05T22:27:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.534+0000 7f4bd2800640 1 -- 172.21.15.73:0/3721108271 wait complete. 2024-04-05T22:27:16.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.535+0000 7f4bd2800640 1 Processor -- start 2024-04-05T22:27:16.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.535+0000 7f4bd2800640 1 -- start start 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb0005180 0x7f4bcc1f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bcc0b22a0 0x7f4bcc0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0b2c90 con 0x7f4bcc1f3540 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0a84c0 con 0x7f4bb0005180 2024-04-05T22:27:16.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.536+0000 7f4bd2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1f3ce0 con 0x7f4bcc0b22a0 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49366/0 (socket says 172.21.15.73:49366) 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 -- 172.21.15.73:0/4170140219 learned_addr learned my addr 172.21.15.73:0/4170140219 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcbe00640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bcc0b22a0 0x7f4bcc0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcb400640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb0005180 0x7f4bcc1f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 -- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bcc0b22a0 msgr2=0x7f4bcc0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bcc0b22a0 0x7f4bcc0b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 -- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb0005180 msgr2=0x7f4bcc1f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb0005180 0x7f4bcc1f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 -- 172.21.15.73:0/4170140219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc1f3fc0 con 0x7f4bcc1f3540 2024-04-05T22:27:16.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.537+0000 7f4bcaa00640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8011540 tx=0x7f4bb8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.538+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8015930 con 0x7f4bcc1f3540 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.538+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8015a90 con 0x7f4bcc1f3540 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.538+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc1f42a0 con 0x7f4bcc1f3540 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.538+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8015e60 con 0x7f4bcc1f3540 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.538+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc0aa3e0 con 0x7f4bcc1f3540 2024-04-05T22:27:16.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.539+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc068160 con 0x7f4bcc1f3540 2024-04-05T22:27:16.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.540+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f4bb801d400 con 0x7f4bcc1f3540 2024-04-05T22:27:16.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.541+0000 7f4bc3400640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 0x7f4bac06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.541+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4bb8089cc0 con 0x7f4bcc1f3540 2024-04-05T22:27:16.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.542+0000 7f4bcb400640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 0x7f4bac06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.543+0000 7f4bcb400640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 0x7f4bac06d210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc1425a0 tx=0x7f4bbc000f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.545+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4bb8054d20 con 0x7f4bcc1f3540 2024-04-05T22:27:16.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.740+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bcc0ccd50 con 0x7f4bcc1f3540 2024-04-05T22:27:16.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.741+0000 7f4bc3400640 1 -- 172.21.15.73:0/4170140219 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4bb8058340 con 0x7f4bcc1f3540 2024-04-05T22:27:16.742 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 msgr2=0x7f4bac06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 0x7f4bac06d210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc1425a0 tx=0x7f4bbc000f50 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 msgr2=0x7f4bcc0a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8011540 tx=0x7f4bb8011a00 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 shutdown_connections 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bac06ad50 0x7f4bac06d210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bcc0b22a0 0x7f4bcc0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1f3540 0x7f4bcc0a9ea0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 --2- 172.21.15.73:0/4170140219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb0005180 0x7f4bcc1f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 >> 172.21.15.73:0/4170140219 conn(0x7f4bcc0b0830 msgr2=0x7f4bcc15b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 shutdown_connections 2024-04-05T22:27:16.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.744+0000 7f4bd2800640 1 -- 172.21.15.73:0/4170140219 wait complete. 2024-04-05T22:27:16.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.970+0000 7f36eac00640 1 Processor -- start 2024-04-05T22:27:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.970+0000 7f36eac00640 1 -- start start 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36eac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36e4160620 0x7f36e40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36eac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 0x7f36e40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36eac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e40cbc70 con 0x7f36e40b1870 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36eac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40ad880 con 0x7f36e4160620 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36eac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40ce5d0 con 0x7f36e40cb2f0 2024-04-05T22:27:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36db400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f36e40b1870 0x7f36e40ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49310/0 (socket says 172.21.15.73:49310) 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36db400640 1 -- 172.21.15.73:0/2057032199 learned_addr learned my addr 172.21.15.73:0/2057032199 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.972+0000 7f36e3e00640 1 --2- 172.21.15.73:0/2057032199 >> v2:172.21.15.73:3300/0 conn(0x7f36e4160620 0x7f36e40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e8c00640 1 --2- 172.21.15.73:0/2057032199 >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 0x7f36e40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3400640 1 -- 172.21.15.73:0/2057032199 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830963552 0 0) 0x7f36e40cbc70 con 0x7f36e40b1870 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3400640 1 -- 172.21.15.73:0/2057032199 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c8003520 con 0x7f36e40b1870 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3e00640 1 -- 172.21.15.73:0/2057032199 >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 msgr2=0x7f36e40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3e00640 1 --2- 172.21.15.73:0/2057032199 >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 0x7f36e40cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3e00640 1 -- 172.21.15.73:0/2057032199 >> v1:172.21.15.73:6789/0 conn(0x7f36e40b1870 legacy=0x7f36e40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e3e00640 1 -- 172.21.15.73:0/2057032199 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40ce010 con 0x7f36e4160620 2024-04-05T22:27:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.973+0000 7f36e8c00640 1 --2- 172.21.15.73:0/2057032199 >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 0x7f36e40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.974+0000 7f36e3e00640 1 --2- 172.21.15.73:0/2057032199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e4160620 0x7f36e40caa60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f36e406f1b0 tx=0x7f36d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b580efb571f1f255 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.974+0000 7f36e3400640 1 -- 172.21.15.73:0/2057032199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4031830 con 0x7f36e4160620 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.974+0000 7f36e3400640 1 -- 172.21.15.73:0/2057032199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4031990 con 0x7f36e4160620 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.974+0000 7f36e3400640 1 -- 172.21.15.73:0/2057032199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4031d20 con 0x7f36e4160620 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 -- 172.21.15.73:0/2057032199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e4160620 msgr2=0x7f36e40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 --2- 172.21.15.73:0/2057032199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e4160620 0x7f36e40caa60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f36e406f1b0 tx=0x7f36d402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 -- 172.21.15.73:0/2057032199 shutdown_connections 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 --2- 172.21.15.73:0/2057032199 >> v2:172.21.15.157:3300/0 conn(0x7f36e40cb2f0 0x7f36e40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 --2- 172.21.15.73:0/2057032199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e4160620 0x7f36e40caa60 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 -- 172.21.15.73:0/2057032199 >> 172.21.15.73:0/2057032199 conn(0x7f36e40ce8e0 msgr2=0x7f36e40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 -- 172.21.15.73:0/2057032199 shutdown_connections 2024-04-05T22:27:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.975+0000 7f36eac00640 1 -- 172.21.15.73:0/2057032199 wait complete. 2024-04-05T22:27:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.977+0000 7f36eac00640 1 Processor -- start 2024-04-05T22:27:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.977+0000 7f36eac00640 1 -- start start 2024-04-05T22:27:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36e40b1870 0x7f36e4158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e4160620 0x7f36e4154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4154c50 con 0x7f36e40cb2f0 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4157d80 con 0x7f36e4160620 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36eac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e415c160 con 0x7f36e40b1870 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36db400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36db400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49396/0 (socket says 172.21.15.73:49396) 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.978+0000 7f36db400640 1 -- 172.21.15.73:0/385031290 learned_addr learned my addr 172.21.15.73:0/385031290 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36e3e00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36e40b1870 0x7f36e4158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36e8c00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e4160620 0x7f36e4154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 -- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36e40b1870 msgr2=0x7f36e4158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36e40b1870 0x7f36e4158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 -- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e4160620 msgr2=0x7f36e4154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e4160620 0x7f36e4154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 -- 172.21.15.73:0/385031290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4009400 con 0x7f36e40cb2f0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36db400640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00c500 tx=0x7f36cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc005ce0 con 0x7f36e40cb2f0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc005e40 con 0x7f36e40cb2f0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc007330 con 0x7f36e40cb2f0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.979+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e415c440 con 0x7f36e40cb2f0 2024-04-05T22:27:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.980+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e415ca00 con 0x7f36e40cb2f0 2024-04-05T22:27:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.981+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4066af0 con 0x7f36e40cb2f0 2024-04-05T22:27:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.981+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f36cc007490 con 0x7f36e40cb2f0 2024-04-05T22:27:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.982+0000 7f36e0c00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 0x7f36c006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.982+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f36cc088bc0 con 0x7f36e40cb2f0 2024-04-05T22:27:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.983+0000 7f36e3e00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 0x7f36c006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.984+0000 7f36e3e00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 0x7f36c006d210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36d4036000 tx=0x7f36d403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:16.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:16.987+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f36cc053ce0 con 0x7f36e40cb2f0 2024-04-05T22:27:17.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.212+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36e4068160 con 0x7f36e40cb2f0 2024-04-05T22:27:17.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.213+0000 7f36e0c00640 1 -- 172.21.15.73:0/385031290 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f36cc017020 con 0x7f36e40cb2f0 2024-04-05T22:27:17.213 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:17.213 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/602096336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":602096336},{"type":"v1","addr":"172.21.15.157:6835","nonce":602096336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:27:17.214 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 55 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 msgr2=0x7f36c006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 0x7f36c006d210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36d4036000 tx=0x7f36d403c040 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 msgr2=0x7f36e415bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00c500 tx=0x7f36cc00c9c0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 shutdown_connections 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f36c006ad50 0x7f36c006d210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.215+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36e4160620 0x7f36e4154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.216+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36e40cb2f0 0x7f36e415bad0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.216+0000 7f36eac00640 1 --2- 172.21.15.73:0/385031290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36e40b1870 0x7f36e4158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.216+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 >> 172.21.15.73:0/385031290 conn(0x7f36e40ce8e0 msgr2=0x7f36e406ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:17.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.216+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 shutdown_connections 2024-04-05T22:27:17.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.216+0000 7f36eac00640 1 -- 172.21.15.73:0/385031290 wait complete. 2024-04-05T22:27:17.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.429+0000 7f2379c00640 1 Processor -- start 2024-04-05T22:27:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.429+0000 7f2379c00640 1 -- start start 2024-04-05T22:27:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2379c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23740cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2379c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f237415aef0 0x7f23740cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2379c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237415aa60 con 0x7f23741541a0 2024-04-05T22:27:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2379c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740b3b60 con 0x7f237415aef0 2024-04-05T22:27:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2379c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740701c0 con 0x7f237406d710 2024-04-05T22:27:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2372a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f23741541a0 0x7f23740b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49322/0 (socket says 172.21.15.73:49322) 2024-04-05T22:27:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.430+0000 7f2372a00640 1 -- 172.21.15.73:0/1014568890 learned_addr learned my addr 172.21.15.73:0/1014568890 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f236be00640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23740cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870457554 0 0) 0x7f237415aa60 con 0x7f23741541a0 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2373400640 1 --2- 172.21.15.73:0/1014568890 >> v2:172.21.15.157:3300/0 conn(0x7f237415aef0 0x7f23740cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2358003520 con 0x7f23741541a0 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073131986 0 0) 0x7f2358003520 con 0x7f23741541a0 2024-04-05T22:27:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 msgr2=0x7f23740cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23740cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 >> v2:172.21.15.157:3300/0 conn(0x7f237415aef0 msgr2=0x7f23740cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 --2- 172.21.15.73:0/1014568890 >> v2:172.21.15.157:3300/0 conn(0x7f237415aef0 0x7f23740cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740b3880 con 0x7f23741541a0 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2364001670 con 0x7f23741541a0 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 >> v1:172.21.15.73:6789/0 conn(0x7f23741541a0 legacy=0x7f23740b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2358003970 0x7f2358003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f2358005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b3b60 con 0x7f2358004d00 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740701c0 con 0x7f2358003970 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.431+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235800ba30 con 0x7f237406d710 2024-04-05T22:27:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f2358005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f236be00640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2372a00640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2358003970 0x7f2358003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 -- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 msgr2=0x7f23580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 -- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2358003970 msgr2=0x7f2358003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2358003970 0x7f2358003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f2373400640 1 -- 172.21.15.73:0/1014568890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358003520 con 0x7f2358004d00 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.432+0000 7f236be00640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406d710 0x7f23580047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.433+0000 7f2373400640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f2358005180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f236c007390 tx=0x7f23580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6af92ef06f37e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.433+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c02ccf0 con 0x7f2358004d00 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.433+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c02ce50 con 0x7f2358004d00 2024-04-05T22:27:17.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.433+0000 7f2372000640 1 -- 172.21.15.73:0/1014568890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c034660 con 0x7f2358004d00 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 -- 172.21.15.73:0/1014568890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 msgr2=0x7f2358005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 --2- 172.21.15.73:0/1014568890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f2358005180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f236c007390 tx=0x7f23580057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2373400640 1 -- 172.21.15.73:0/1014568890 reap_dead start 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 -- 172.21.15.73:0/1014568890 shutdown_connections 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 -- 172.21.15.73:0/1014568890 >> 172.21.15.73:0/1014568890 conn(0x7f23740ce8e0 msgr2=0x7f23740cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 -- 172.21.15.73:0/1014568890 shutdown_connections 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.434+0000 7f2379c00640 1 -- 172.21.15.73:0/1014568890 wait complete. 2024-04-05T22:27:17.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.436+0000 7f2379c00640 1 Processor -- start 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.436+0000 7f2379c00640 1 -- start start 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2374142250 0x7f2374141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406c470 0x7f237406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741420c0 con 0x7f2358004d00 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237415bd50 con 0x7f2374142250 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2379c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237415c030 con 0x7f237406c470 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49426/0 (socket says 172.21.15.73:49426) 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 -- 172.21.15.73:0/1612097203 learned_addr learned my addr 172.21.15.73:0/1612097203 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 -- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406c470 msgr2=0x7f237406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406c470 0x7f237406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 -- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2374142250 msgr2=0x7f2374141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2374142250 0x7f2374141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 -- 172.21.15.73:0/1612097203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740ae480 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.437+0000 7f2372a00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f236400d810 tx=0x7f236400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.438+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640074e0 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.438+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364007640 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.438+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740ae760 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.438+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364004a20 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.438+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237406ce30 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.439+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374066af0 con 0x7f2358004d00 2024-04-05T22:27:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.440+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f2364004b80 con 0x7f2358004d00 2024-04-05T22:27:17.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.440+0000 7f236aa00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 0x7f235006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.441+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2364089270 con 0x7f2358004d00 2024-04-05T22:27:17.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.441+0000 7f236be00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 0x7f235006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.442+0000 7f236be00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 0x7f235006d1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2374167b60 tx=0x7f235c009350 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.445+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2364054260 con 0x7f2358004d00 2024-04-05T22:27:17.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.635+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2374068160 con 0x7f2358004d00 2024-04-05T22:27:17.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.637+0000 7f236aa00640 1 -- 172.21.15.73:0/1612097203 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f2364057880 con 0x7f2358004d00 2024-04-05T22:27:17.638 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:17.638 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:09.109725+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 msgr2=0x7f235006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 0x7f235006d1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2374167b60 tx=0x7f235c009350 comp rx=0 tx=0).stop 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 msgr2=0x7f237415ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f236400d810 tx=0x7f236400dcd0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 shutdown_connections 2024-04-05T22:27:17.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f235006ad20 0x7f235006d1e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237406c470 0x7f237406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2374142250 0x7f2374141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 --2- 172.21.15.73:0/1612097203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2358004d00 0x7f237415ce80 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 >> 172.21.15.73:0/1612097203 conn(0x7f23740ce8e0 msgr2=0x7f23740cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 shutdown_connections 2024-04-05T22:27:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.640+0000 7f2379c00640 1 -- 172.21.15.73:0/1612097203 wait complete. 2024-04-05T22:27:17.659 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:27:17.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.863+0000 7fc07d000640 1 Processor -- start 2024-04-05T22:27:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.863+0000 7fc07d000640 1 -- start start 2024-04-05T22:27:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.865+0000 7fc07d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.865+0000 7fc07d000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0780b2ba0 0x7fc0780b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.865+0000 7fc07d000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc078165cf0 con 0x7fc078164e50 2024-04-05T22:27:17.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.865+0000 7fc07d000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc078158790 con 0x7fc0780b2ba0 2024-04-05T22:27:17.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.865+0000 7fc07d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780b2820 con 0x7fc078161f90 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.866+0000 7fc075e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.866+0000 7fc075400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc078164e50 0x7fc0780b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:42834/0 (socket says 172.21.15.73:42834) 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.866+0000 7fc075400640 1 -- 172.21.15.73:0/1868202035 learned_addr learned my addr 172.21.15.73:0/1868202035 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.866+0000 7fc076800640 1 --2- 172.21.15.73:0/1868202035 >> v2:172.21.15.157:3300/0 conn(0x7fc0780b2ba0 0x7fc0780b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.867+0000 7fc075e00640 1 -- 172.21.15.73:0/1868202035 >> v2:172.21.15.157:3300/0 conn(0x7fc0780b2ba0 msgr2=0x7fc0780b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.867+0000 7fc075e00640 1 --2- 172.21.15.73:0/1868202035 >> v2:172.21.15.157:3300/0 conn(0x7fc0780b2ba0 0x7fc0780b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.867+0000 7fc074a00640 1 -- 172.21.15.73:0/1868202035 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226511948 0 0) 0x7fc078165cf0 con 0x7fc078164e50 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.867+0000 7fc075e00640 1 -- 172.21.15.73:0/1868202035 >> v1:172.21.15.157:6789/0 conn(0x7fc078164e50 legacy=0x7fc0780b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.867+0000 7fc075e00640 1 -- 172.21.15.73:0/1868202035 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078165730 con 0x7fc078161f90 2024-04-05T22:27:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.868+0000 7fc075e00640 1 --2- 172.21.15.73:0/1868202035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078162370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097a0 tx=0x7fc06002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc6b47c55cfa579 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc074a00640 1 -- 172.21.15.73:0/1868202035 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060031830 con 0x7fc078161f90 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc074a00640 1 -- 172.21.15.73:0/1868202035 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060031990 con 0x7fc078161f90 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc074a00640 1 -- 172.21.15.73:0/1868202035 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060031d20 con 0x7fc078161f90 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 -- 172.21.15.73:0/1868202035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 msgr2=0x7fc078162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 --2- 172.21.15.73:0/1868202035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078162370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097a0 tx=0x7fc06002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 -- 172.21.15.73:0/1868202035 shutdown_connections 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 --2- 172.21.15.73:0/1868202035 >> v2:172.21.15.157:3300/0 conn(0x7fc0780b2ba0 0x7fc0780b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 --2- 172.21.15.73:0/1868202035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078162370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 -- 172.21.15.73:0/1868202035 >> 172.21.15.73:0/1868202035 conn(0x7fc0780ce8e0 msgr2=0x7fc0780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:17.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 -- 172.21.15.73:0/1868202035 shutdown_connections 2024-04-05T22:27:17.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.869+0000 7fc07d000640 1 -- 172.21.15.73:0/1868202035 wait complete. 2024-04-05T22:27:17.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.871+0000 7fc07d000640 1 Processor -- start 2024-04-05T22:27:17.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.871+0000 7fc07d000640 1 -- start start 2024-04-05T22:27:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 0x7fc0780a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0781449a0 con 0x7fc078164e50 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078147950 con 0x7fc0780b2ba0 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.872+0000 7fc07d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780a9c90 con 0x7fc078161f90 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49436/0 (socket says 172.21.15.73:49436) 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 -- 172.21.15.73:0/6694205 learned_addr learned my addr 172.21.15.73:0/6694205 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc075e00640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 0x7fc0780a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc075400640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 -- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 msgr2=0x7fc078147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 -- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 msgr2=0x7fc0780a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 0x7fc0780a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc076800640 1 -- 172.21.15.73:0/6694205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009400 con 0x7fc078164e50 2024-04-05T22:27:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc075e00640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 0x7fc0780a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.873+0000 7fc075400640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc076800640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e930 tx=0x7fc07000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070009690 con 0x7fc078164e50 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07000c980 con 0x7fc078164e50 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0780a9f70 con 0x7fc078164e50 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07000cd30 con 0x7fc078164e50 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.874+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0780aa530 con 0x7fc078164e50 2024-04-05T22:27:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.875+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078066af0 con 0x7fc078164e50 2024-04-05T22:27:17.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.876+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc070013470 con 0x7fc078164e50 2024-04-05T22:27:17.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.877+0000 7fc06e000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 0x7fc05406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:17.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.877+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0700897c0 con 0x7fc078164e50 2024-04-05T22:27:17.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.878+0000 7fc075e00640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 0x7fc05406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:17.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.879+0000 7fc075e00640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 0x7fc05406d1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc060036000 tx=0x7fc060008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:17.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:17.881+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc070054850 con 0x7fc078164e50 2024-04-05T22:27:18.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.111+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc078144b50 con 0x7fc078164e50 2024-04-05T22:27:18.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.113+0000 7fc06e000640 1 -- 172.21.15.73:0/6694205 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fc078144b50 con 0x7fc078164e50 2024-04-05T22:27:18.113 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:18.113 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/602096336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":602096336},{"type":"v1","addr":"172.21.15.157:6835","nonce":602096336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:27:18.113 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 55 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 msgr2=0x7fc05406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 0x7fc05406d1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc060036000 tx=0x7fc060008030 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 msgr2=0x7fc078144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e930 tx=0x7fc07000edf0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 shutdown_connections 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc05406ad20 0x7fc05406d1e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc078164e50 0x7fc078144110 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc078161f90 0x7fc078147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 --2- 172.21.15.73:0/6694205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0780b2ba0 0x7fc0780a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.115+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 >> 172.21.15.73:0/6694205 conn(0x7fc0780ce8e0 msgr2=0x7fc0781630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:18.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.116+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 shutdown_connections 2024-04-05T22:27:18.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.116+0000 7fc07d000640 1 -- 172.21.15.73:0/6694205 wait complete. 2024-04-05T22:27:18.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:18.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.340+0000 7fa8b3e00640 1 Processor -- start 2024-04-05T22:27:18.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.340+0000 7fa8b3e00640 1 -- start start 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8ac1587d0 0x7fa8ac0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8ac15e850 0x7fa8ac15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b3e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac06f5b0 con 0x7fa8ac0ccd20 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b3e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac15f240 con 0x7fa8ac1587d0 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac070410 con 0x7fa8ac15e850 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8ac1587d0 0x7fa8ac0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b0a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa8ac0ccd20 0x7fa8ac06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49326/0 (socket says 172.21.15.73:49326) 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.342+0000 7fa8b0a00640 1 -- 172.21.15.73:0/2986314347 learned_addr learned my addr 172.21.15.73:0/2986314347 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8b1e00640 1 --2- 172.21.15.73:0/2986314347 >> v2:172.21.15.157:3300/0 conn(0x7fa8ac15e850 0x7fa8ac15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8b1400640 1 -- 172.21.15.73:0/2986314347 >> v2:172.21.15.157:3300/0 conn(0x7fa8ac15e850 msgr2=0x7fa8ac15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:18.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8b1400640 1 --2- 172.21.15.73:0/2986314347 >> v2:172.21.15.157:3300/0 conn(0x7fa8ac15e850 0x7fa8ac15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8b1400640 1 -- 172.21.15.73:0/2986314347 >> v1:172.21.15.73:6789/0 conn(0x7fa8ac0ccd20 legacy=0x7fa8ac06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8abe00640 1 -- 172.21.15.73:0/2986314347 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327427553 0 0) 0x7fa8ac06f5b0 con 0x7fa8ac0ccd20 2024-04-05T22:27:18.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.343+0000 7fa8b1400640 1 -- 172.21.15.73:0/2986314347 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac158bb0 con 0x7fa8ac1587d0 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8b1400640 1 --2- 172.21.15.73:0/2986314347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac1587d0 0x7fa8ac0cc7e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0097a0 tx=0x7fa89c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3674919b67877e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8abe00640 1 -- 172.21.15.73:0/2986314347 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c031830 con 0x7fa8ac1587d0 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8abe00640 1 -- 172.21.15.73:0/2986314347 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c031990 con 0x7fa8ac1587d0 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8abe00640 1 -- 172.21.15.73:0/2986314347 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c031d20 con 0x7fa8ac1587d0 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2986314347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac1587d0 msgr2=0x7fa8ac0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.344+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2986314347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac1587d0 0x7fa8ac0cc7e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0097a0 tx=0x7fa89c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2986314347 shutdown_connections 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2986314347 >> v2:172.21.15.157:3300/0 conn(0x7fa8ac15e850 0x7fa8ac15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2986314347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac1587d0 0x7fa8ac0cc7e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2986314347 >> 172.21.15.73:0/2986314347 conn(0x7fa8ac0ce8e0 msgr2=0x7fa8ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2986314347 shutdown_connections 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.345+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2986314347 wait complete. 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.346+0000 7fa8b3e00640 1 Processor -- start 2024-04-05T22:27:18.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.346+0000 7fa8b3e00640 1 -- start start 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac1587d0 0x7fa8ac0ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac144150 0x7fa8ac1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0ab950 con 0x7fa8ac15e850 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac143ec0 con 0x7fa8ac144150 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.347+0000 7fa8b3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac146310 con 0x7fa8ac1587d0 2024-04-05T22:27:18.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49454/0 (socket says 172.21.15.73:49454) 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 -- 172.21.15.73:0/2686287520 learned_addr learned my addr 172.21.15.73:0/2686287520 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 -- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac1587d0 msgr2=0x7fa8ac0ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac1587d0 0x7fa8ac0ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 -- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac144150 msgr2=0x7fa8ac1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac144150 0x7fa8ac1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 -- 172.21.15.73:0/2686287520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009400 con 0x7fa8ac15e850 2024-04-05T22:27:18.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.348+0000 7fa8b0a00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa89800c4d0 tx=0x7fa89800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.349+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980074e0 con 0x7fa8ac15e850 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.349+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa898007660 con 0x7fa8ac15e850 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.349+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac1465f0 con 0x7fa8ac15e850 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.349+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898004a20 con 0x7fa8ac15e850 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.349+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac146bb0 con 0x7fa8ac15e850 2024-04-05T22:27:18.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.350+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac066af0 con 0x7fa8ac15e850 2024-04-05T22:27:18.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.351+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fa898004b80 con 0x7fa8ac15e850 2024-04-05T22:27:18.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.352+0000 7fa8a9600640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 0x7fa88c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.352+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa898088690 con 0x7fa8ac15e850 2024-04-05T22:27:18.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.353+0000 7fa8b1400640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 0x7fa88c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.354+0000 7fa8b1400640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 0x7fa88c06d210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036000 tx=0x7fa89c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.357+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa898053700 con 0x7fa8ac15e850 2024-04-05T22:27:18.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.589+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8ac068160 con 0x7fa8ac15e850 2024-04-05T22:27:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.590+0000 7fa8a9600640 1 -- 172.21.15.73:0/2686287520 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fa898056d20 con 0x7fa8ac15e850 2024-04-05T22:27:18.591 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:18.591 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/602096336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":602096336},{"type":"v1","addr":"172.21.15.157:6835","nonce":602096336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:26:50.425705+0000","modified":"2024-04-05T22:26:53.451569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6833/3038019968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3038019968},{"type":"v1","addr":"172.21.15.73:6833","nonce":3038019968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-05T22:27:18.591 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 55 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 msgr2=0x7fa88c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 0x7fa88c06d210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036000 tx=0x7fa89c030000 comp rx=0 tx=0).stop 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 msgr2=0x7fa8ac158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa89800c4d0 tx=0x7fa89800c990 comp rx=0 tx=0).stop 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 shutdown_connections 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa88c06ad50 0x7fa88c06d210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac144150 0x7fa8ac1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8ac15e850 0x7fa8ac158e60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 --2- 172.21.15.73:0/2686287520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac1587d0 0x7fa8ac0ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 >> 172.21.15.73:0/2686287520 conn(0x7fa8ac0ce8e0 msgr2=0x7fa8ac0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.593+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 shutdown_connections 2024-04-05T22:27:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.594+0000 7fa8b3e00640 1 -- 172.21.15.73:0/2686287520 wait complete. 2024-04-05T22:27:18.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:18.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.821+0000 7f904c400640 1 Processor -- start 2024-04-05T22:27:18.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.821+0000 7f904c400640 1 -- start start 2024-04-05T22:27:18.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.822+0000 7f904c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f90440cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.822+0000 7f904c400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90440ccd20 0x7f904406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f904c400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904406f5b0 con 0x7f904406d0c0 2024-04-05T22:27:18.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f904c400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9044070410 con 0x7f90440ccd20 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f904c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9044158bb0 con 0x7f90441587d0 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f9049000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90440ccd20 0x7f904406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f9049000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90440ccd20 0x7f904406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49458/0 (socket says 172.21.15.73:49458) 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f9049000640 1 -- 172.21.15.73:0/2502071094 learned_addr learned my addr 172.21.15.73:0/2502071094 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.823+0000 7f9049a00640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f90440cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.824+0000 7f9049000640 1 -- 172.21.15.73:0/2502071094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 msgr2=0x7f90440cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:18.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.824+0000 7f9049000640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f90440cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.824+0000 7f9049000640 1 -- 172.21.15.73:0/2502071094 >> v1:172.21.15.157:6789/0 conn(0x7f904406d0c0 legacy=0x7f904415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.824+0000 7f9049000640 1 -- 172.21.15.73:0/2502071094 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904406f800 con 0x7f90440ccd20 2024-04-05T22:27:18.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.825+0000 7f9049000640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90440ccd20 0x7f904406cb80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9034009870 tx=0x7f903402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e95017c16d791c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.825+0000 7f9043e00640 1 -- 172.21.15.73:0/2502071094 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034030840 con 0x7f90440ccd20 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.825+0000 7f9043e00640 1 -- 172.21.15.73:0/2502071094 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90340309a0 con 0x7f90440ccd20 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.825+0000 7f9043e00640 1 -- 172.21.15.73:0/2502071094 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034030d30 con 0x7f90440ccd20 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 -- 172.21.15.73:0/2502071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90440ccd20 msgr2=0x7f904406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90440ccd20 0x7f904406cb80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9034009870 tx=0x7f903402b980 comp rx=0 tx=0).stop 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 -- 172.21.15.73:0/2502071094 shutdown_connections 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90440ccd20 0x7f904406cb80 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 --2- 172.21.15.73:0/2502071094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f90440cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 -- 172.21.15.73:0/2502071094 >> 172.21.15.73:0/2502071094 conn(0x7f90440ce8e0 msgr2=0x7f90440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 -- 172.21.15.73:0/2502071094 shutdown_connections 2024-04-05T22:27:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.826+0000 7f904c400640 1 -- 172.21.15.73:0/2502071094 wait complete. 2024-04-05T22:27:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.827+0000 7f904c400640 1 Processor -- start 2024-04-05T22:27:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.828+0000 7f904c400640 1 -- start start 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f904406d0c0 0x7f90440aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f9044145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044146ba0 con 0x7f9044146430 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044146e80 con 0x7f904406d0c0 2024-04-05T22:27:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9044147160 con 0x7f90441587d0 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49462/0 (socket says 172.21.15.73:49462) 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 -- 172.21.15.73:0/215784842 learned_addr learned my addr 172.21.15.73:0/215784842 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f9049000640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f9044145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 -- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 msgr2=0x7f9044145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f9044145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f9049a00640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f904406d0c0 0x7f90440aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 -- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f904406d0c0 msgr2=0x7f90440aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f904406d0c0 0x7f90440aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.829+0000 7f904a400640 1 -- 172.21.15.73:0/215784842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340094d0 con 0x7f9044146430 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f904a400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c7a0 tx=0x7f903c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c003b70 con 0x7f9044146430 2024-04-05T22:27:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c003cf0 con 0x7f9044146430 2024-04-05T22:27:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c003ea0 con 0x7f9044146430 2024-04-05T22:27:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9044147440 con 0x7f9044146430 2024-04-05T22:27:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.830+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90441468c0 con 0x7f9044146430 2024-04-05T22:27:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.831+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9044066af0 con 0x7f9044146430 2024-04-05T22:27:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.832+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f903c00aa90 con 0x7f9044146430 2024-04-05T22:27:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.833+0000 7f9041600640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 0x7f902806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.833+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f903c089430 con 0x7f9044146430 2024-04-05T22:27:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.834+0000 7f9049a00640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 0x7f902806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.835+0000 7f9049a00640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 0x7f902806d1e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9038009970 tx=0x7f9038008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:18.837+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f903c054400 con 0x7f9044146430 2024-04-05T22:27:19.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.031+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9044147960 con 0x7f9044146430 2024-04-05T22:27:19.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.033+0000 7f9041600640 1 -- 172.21.15.73:0/215784842 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f9044147960 con 0x7f9044146430 2024-04-05T22:27:19.033 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:19.033 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:09.109725+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:26:48.879992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-05T22:26:49.430091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.035+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 msgr2=0x7f902806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.035+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 0x7f902806d1e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9038009970 tx=0x7f9038008040 comp rx=0 tx=0).stop 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.035+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 msgr2=0x7f9044147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.035+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c7a0 tx=0x7f903c00cc60 comp rx=0 tx=0).stop 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 shutdown_connections 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f902806ad20 0x7f902806d1e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9044146430 0x7f9044147c40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90441587d0 0x7f9044145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 --2- 172.21.15.73:0/215784842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f904406d0c0 0x7f90440aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 >> 172.21.15.73:0/215784842 conn(0x7f90440ce8e0 msgr2=0x7f90440ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:19.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 shutdown_connections 2024-04-05T22:27:19.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.036+0000 7f904c400640 1 -- 172.21.15.73:0/215784842 wait complete. 2024-04-05T22:27:19.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:27:19.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.268+0000 7fbc96e00640 1 Processor -- start 2024-04-05T22:27:19.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.268+0000 7fbc96e00640 1 -- start start 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc96e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbc900ce000 0x7fbc9015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc96e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9015aa60 con 0x7fbc901541a0 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc96e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900b3b60 con 0x7fbc9006d710 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc96e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc900701c0 con 0x7fbc900ce000 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc8fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbc901541a0 0x7fbc900b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49338/0 (socket says 172.21.15.73:49338) 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc8fe00640 1 -- 172.21.15.73:0/3890420576 learned_addr learned my addr 172.21.15.73:0/3890420576 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc94e00640 1 --2- 172.21.15.73:0/3890420576 >> v2:172.21.15.73:3300/0 conn(0x7fbc900ce000 0x7fbc9015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc8ea00640 1 -- 172.21.15.73:0/3890420576 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579748991 0 0) 0x7fbc9015aa60 con 0x7fbc901541a0 2024-04-05T22:27:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.270+0000 7fbc8ea00640 1 -- 172.21.15.73:0/3890420576 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc74005730 con 0x7fbc901541a0 2024-04-05T22:27:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.271+0000 7fbc94e00640 1 -- 172.21.15.73:0/3890420576 >> v1:172.21.15.157:6789/0 conn(0x7fbc9006d710 legacy=0x7fbc900cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.271+0000 7fbc94e00640 1 -- 172.21.15.73:0/3890420576 >> v1:172.21.15.73:6789/0 conn(0x7fbc901541a0 legacy=0x7fbc900b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.271+0000 7fbc94e00640 1 -- 172.21.15.73:0/3890420576 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900b3880 con 0x7fbc900ce000 2024-04-05T22:27:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.272+0000 7fbc94e00640 1 --2- 172.21.15.73:0/3890420576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc900ce000 0x7fbc9015d860 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009d90 tx=0x7fbc8802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05fc1791bb22dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.272+0000 7fbc8ea00640 1 -- 172.21.15.73:0/3890420576 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8802cd00 con 0x7fbc900ce000 2024-04-05T22:27:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.272+0000 7fbc8ea00640 1 -- 172.21.15.73:0/3890420576 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8802ce60 con 0x7fbc900ce000 2024-04-05T22:27:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.272+0000 7fbc8ea00640 1 -- 172.21.15.73:0/3890420576 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88034660 con 0x7fbc900ce000 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 -- 172.21.15.73:0/3890420576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc900ce000 msgr2=0x7fbc9015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3890420576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc900ce000 0x7fbc9015d860 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009d90 tx=0x7fbc8802bf50 comp rx=0 tx=0).stop 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 -- 172.21.15.73:0/3890420576 shutdown_connections 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3890420576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc900ce000 0x7fbc9015d860 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 -- 172.21.15.73:0/3890420576 >> 172.21.15.73:0/3890420576 conn(0x7fbc900ce8e0 msgr2=0x7fbc900cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 -- 172.21.15.73:0/3890420576 shutdown_connections 2024-04-05T22:27:19.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.273+0000 7fbc96e00640 1 -- 172.21.15.73:0/3890420576 wait complete. 2024-04-05T22:27:19.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.274+0000 7fbc96e00640 1 Processor -- start 2024-04-05T22:27:19.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.275+0000 7fbc96e00640 1 -- start start 2024-04-05T22:27:19.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 0x7fbc900ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc900ce000 0x7fbc900af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901556d0 con 0x7fbc901541a0 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901559b0 con 0x7fbc900ce000 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc96e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901410c0 con 0x7fbc9006d710 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49480/0 (socket says 172.21.15.73:49480) 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 -- 172.21.15.73:0/3201915751 learned_addr learned my addr 172.21.15.73:0/3201915751 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc8fe00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 0x7fbc900ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 -- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 msgr2=0x7fbc900ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 0x7fbc900ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc8f400640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc900ce000 0x7fbc900af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 -- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc900ce000 msgr2=0x7fbc900af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc900ce000 0x7fbc900af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc94e00640 1 -- 172.21.15.73:0/3201915751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88009a70 con 0x7fbc901541a0 2024-04-05T22:27:19.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.276+0000 7fbc8fe00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 0x7fbc900ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:19.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc94e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009eb0 tx=0x7fbc88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8803d600 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8803d780 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8803db30 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc901413a0 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.277+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc900ad580 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.278+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc90066af0 con 0x7fbc901541a0 2024-04-05T22:27:19.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.279+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fbc8803b040 con 0x7fbc901541a0 2024-04-05T22:27:19.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.280+0000 7fbc87e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 0x7fbc6c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.280+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbc880abab0 con 0x7fbc901541a0 2024-04-05T22:27:19.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.281+0000 7fbc8fe00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 0x7fbc6c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.282+0000 7fbc8fe00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 0x7fbc6c06d1e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc800011e0 tx=0x7fbc80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.285+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbc88076b20 con 0x7fbc901541a0 2024-04-05T22:27:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.518+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc900adc00 con 0x7fbc901541a0 2024-04-05T22:27:19.520 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:27:19.520+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:27:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.535+0000 7fbc87e00640 1 -- 172.21.15.73:0/3201915751 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc88039020 con 0x7fbc901541a0 2024-04-05T22:27:19.535 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:27:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 msgr2=0x7fbc6c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 0x7fbc6c06d1e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc800011e0 tx=0x7fbc80009040 comp rx=0 tx=0).stop 2024-04-05T22:27:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 msgr2=0x7fbc900af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009eb0 tx=0x7fbc88004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 shutdown_connections 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbc6c06ad20 0x7fbc6c06d1e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc901541a0 0x7fbc900af9f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc900ce000 0x7fbc900af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 --2- 172.21.15.73:0/3201915751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc9006d710 0x7fbc900ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 >> 172.21.15.73:0/3201915751 conn(0x7fbc900ce8e0 msgr2=0x7fbc9006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 shutdown_connections 2024-04-05T22:27:19.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.538+0000 7fbc96e00640 1 -- 172.21.15.73:0/3201915751 wait complete. 2024-04-05T22:27:19.553 INFO:tasks.ceph.mds.a.smithi073.stderr: -2524> 2024-04-05T22:26:55.864+0000 7fb46a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:27:19.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:27:19.615 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:27:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.746+0000 7fe0c7e00640 1 Processor -- start 2024-04-05T22:27:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.746+0000 7fe0c7e00640 1 -- start start 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.747+0000 7fe0c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0c8158a30 0x7fe0c80b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b30a0 0x7fe0c80b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 0x7fe0c80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c80b1b10 con 0x7fe0c8158a30 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c80b35b0 con 0x7fe0c80b2470 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80b3890 con 0x7fe0c80b30a0 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0c8158a30 0x7fe0c80b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0c8158a30 0x7fe0c80b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49500/0 (socket says 172.21.15.73:49500) 2024-04-05T22:27:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 -- 172.21.15.73:0/2601152610 learned_addr learned my addr 172.21.15.73:0/2601152610 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6000640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b30a0 0x7fe0c80b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7400640 1 --2- 172.21.15.73:0/2601152610 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 0x7fe0c80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 -- 172.21.15.73:0/2601152610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b30a0 msgr2=0x7fe0c80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b30a0 0x7fe0c80b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 -- 172.21.15.73:0/2601152610 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 msgr2=0x7fe0c80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/2601152610 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 0x7fe0c80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c6a00640 1 -- 172.21.15.73:0/2601152610 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80b3b70 con 0x7fe0c8158a30 2024-04-05T22:27:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.748+0000 7fe0c7400640 1 --2- 172.21.15.73:0/2601152610 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 0x7fe0c80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.750+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c8158a30 0x7fe0c80b2b60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe0c806f320 tx=0x7fe0b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c520380477fdd97e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.750+0000 7fe0c5600640 1 -- 172.21.15.73:0/2601152610 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8031870 con 0x7fe0c8158a30 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.750+0000 7fe0c5600640 1 -- 172.21.15.73:0/2601152610 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b80319d0 con 0x7fe0c8158a30 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.750+0000 7fe0c5600640 1 -- 172.21.15.73:0/2601152610 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8031d60 con 0x7fe0c8158a30 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 -- 172.21.15.73:0/2601152610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c8158a30 msgr2=0x7fe0c80b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c8158a30 0x7fe0c80b2b60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe0c806f320 tx=0x7fe0b802b910 comp rx=0 tx=0).stop 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 -- 172.21.15.73:0/2601152610 shutdown_connections 2024-04-05T22:27:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/2601152610 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80b2470 0x7fe0c80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b30a0 0x7fe0c80b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/2601152610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c8158a30 0x7fe0c80b2b60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 -- 172.21.15.73:0/2601152610 >> 172.21.15.73:0/2601152610 conn(0x7fe0c8069e90 msgr2=0x7fe0c806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:19.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 -- 172.21.15.73:0/2601152610 shutdown_connections 2024-04-05T22:27:19.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.751+0000 7fe0c7e00640 1 -- 172.21.15.73:0/2601152610 wait complete. 2024-04-05T22:27:19.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.753+0000 7fe0c7e00640 1 Processor -- start 2024-04-05T22:27:19.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.753+0000 7fe0c7e00640 1 -- start start 2024-04-05T22:27:19.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b2470 0x7fe0c8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 0x7fe0c815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c806c2a0 con 0x7fe0c80b30a0 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c806b690 con 0x7fe0c8158a30 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c806b970 con 0x7fe0c80b2470 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49508/0 (socket says 172.21.15.73:49508) 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 -- 172.21.15.73:0/1435086783 learned_addr learned my addr 172.21.15.73:0/1435086783 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b2470 0x7fe0c8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 -- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b2470 msgr2=0x7fe0c8148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b2470 0x7fe0c8148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c6000640 1 -- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 msgr2=0x7fe0c815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.754+0000 7fe0c7400640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 0x7fe0c815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c6000640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 0x7fe0c815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:19.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c6000640 1 -- 172.21.15.73:0/1435086783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80094d0 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c7400640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 0x7fe0c815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c6000640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c7f0 tx=0x7fe0b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4004220 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4004380 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4007840 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c806bc50 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.755+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c81495e0 con 0x7fe0c80b30a0 2024-04-05T22:27:19.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.756+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c804e160 con 0x7fe0c80b30a0 2024-04-05T22:27:19.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.757+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe0b4007bb0 con 0x7fe0c80b30a0 2024-04-05T22:27:19.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.758+0000 7fe0bea00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 0x7fe0a806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:19.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.758+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe0b4089610 con 0x7fe0c80b30a0 2024-04-05T22:27:19.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.759+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 0x7fe0a806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:19.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.760+0000 7fe0c6a00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 0x7fe0a806d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8036000 tx=0x7fe0b803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:19.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.762+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe0b4054510 con 0x7fe0c80b30a0 2024-04-05T22:27:19.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:19.985+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe0c8157fa0 con 0x7fe0c80b30a0 2024-04-05T22:27:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.538+0000 7fe0bea00640 1 -- 172.21.15.73:0/1435086783 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0b4007e60 con 0x7fe0c80b30a0 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 msgr2=0x7fe0a806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 0x7fe0a806d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8036000 tx=0x7fe0b803c040 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 msgr2=0x7fe0c806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c7f0 tx=0x7fe0b400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 shutdown_connections 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe0a806ad20 0x7fe0a806d1e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c8158a30 0x7fe0c815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0c80b30a0 0x7fe0c806cda0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 --2- 172.21.15.73:0/1435086783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c80b2470 0x7fe0c8148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 >> 172.21.15.73:0/1435086783 conn(0x7fe0c8069e90 msgr2=0x7fe0c80ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:20.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.542+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 shutdown_connections 2024-04-05T22:27:20.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.543+0000 7fe0c7e00640 1 -- 172.21.15.73:0/1435086783 wait complete. 2024-04-05T22:27:20.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:27:20.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.768+0000 7f18d1600640 1 Processor -- start 2024-04-05T22:27:20.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.768+0000 7f18d1600640 1 -- start start 2024-04-05T22:27:20.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18d1600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18cc0b3aa0 0x7f18cc0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:20.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18d1600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15b230 con 0x7f18cc15ae50 2024-04-05T22:27:20.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18d1600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15f4a0 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18d1600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18cc0b3f30 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18caa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f18cc15ae50 0x7f18cc0cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49354/0 (socket says 172.21.15.73:49354) 2024-04-05T22:27:20.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18caa00640 1 -- 172.21.15.73:0/1916009423 learned_addr learned my addr 172.21.15.73:0/1916009423 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:20.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.770+0000 7f18cb400640 1 --2- 172.21.15.73:0/1916009423 >> v2:172.21.15.73:3300/0 conn(0x7f18cc0b3aa0 0x7f18cc0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.771+0000 7f18c9600640 1 -- 172.21.15.73:0/1916009423 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743270013 0 0) 0x7f18cc15b230 con 0x7f18cc15ae50 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.771+0000 7f18c9600640 1 -- 172.21.15.73:0/1916009423 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b0003520 con 0x7f18cc15ae50 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.771+0000 7f18cb400640 1 -- 172.21.15.73:0/1916009423 >> v1:172.21.15.157:6789/0 conn(0x7f18cc0cf3e0 legacy=0x7f18cc0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.771+0000 7f18cb400640 1 -- 172.21.15.73:0/1916009423 >> v1:172.21.15.73:6789/0 conn(0x7f18cc15ae50 legacy=0x7f18cc0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.771+0000 7f18cb400640 1 -- 172.21.15.73:0/1916009423 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc15f780 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.772+0000 7f18cb400640 1 --2- 172.21.15.73:0/1916009423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc0cce90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009d90 tx=0x7f18c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9696487e73e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:20.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.772+0000 7f18c9600640 1 -- 172.21.15.73:0/1916009423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c402cc60 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18c9600640 1 -- 172.21.15.73:0/1916009423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c402cdc0 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18c9600640 1 -- 172.21.15.73:0/1916009423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c4034660 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 -- 172.21.15.73:0/1916009423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0b3aa0 msgr2=0x7f18cc0cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 --2- 172.21.15.73:0/1916009423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc0cce90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009d90 tx=0x7f18c402beb0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 -- 172.21.15.73:0/1916009423 shutdown_connections 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 --2- 172.21.15.73:0/1916009423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc0cce90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 -- 172.21.15.73:0/1916009423 >> 172.21.15.73:0/1916009423 conn(0x7f18cc06a0d0 msgr2=0x7f18cc06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 -- 172.21.15.73:0/1916009423 shutdown_connections 2024-04-05T22:27:20.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.773+0000 7f18d1600640 1 -- 172.21.15.73:0/1916009423 wait complete. 2024-04-05T22:27:20.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.775+0000 7f18d1600640 1 Processor -- start 2024-04-05T22:27:20.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.775+0000 7f18d1600640 1 -- start start 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18cc15ae50 0x7f18cc06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0c8ba0 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0c7cd0 con 0x7f18cc0b3aa0 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18d1600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc1572a0 con 0x7f18cc15ae50 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18ca000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18ca000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49538/0 (socket says 172.21.15.73:49538) 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.776+0000 7f18ca000640 1 -- 172.21.15.73:0/1800840036 learned_addr learned my addr 172.21.15.73:0/1800840036 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18caa00640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 -- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18cc15ae50 msgr2=0x7f18cc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18cc15ae50 0x7f18cc06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 -- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 msgr2=0x7f18cc1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 -- 172.21.15.73:0/1800840036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4009a70 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18caa00640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:20.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18ca000640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c400 tx=0x7f18b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80052d0 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8005430 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0c7f90 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8003830 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.777+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc0c8550 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.778+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc068160 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.779+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f18b8003990 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.780+0000 7f18c2a00640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 0x7f18ac06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:20.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.780+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f18b8088b40 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.781+0000 7f18caa00640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 0x7f18ac06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:20.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.782+0000 7f18caa00640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 0x7f18ac06d210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0011e0 tx=0x7f18bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:20.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.785+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f18b8053ac0 con 0x7f18cc0cf3e0 2024-04-05T22:27:20.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:20.977+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18cc0ad720 con 0x7f18cc0cf3e0 2024-04-05T22:27:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.542+0000 7f18c2a00640 1 -- 172.21.15.73:0/1800840036 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f18b80570e0 con 0x7f18cc0cf3e0 2024-04-05T22:27:21.546 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:27:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 msgr2=0x7f18ac06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 0x7f18ac06d210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0011e0 tx=0x7f18bc009040 comp rx=0 tx=0).stop 2024-04-05T22:27:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 msgr2=0x7f18cc156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c400 tx=0x7f18b800c8c0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 shutdown_connections 2024-04-05T22:27:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f18ac06ad50 0x7f18ac06d210 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0011e0 tx=0x7f18bc009040 comp rx=0 tx=0).stop 2024-04-05T22:27:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18cc15ae50 0x7f18cc06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18cc0cf3e0 0x7f18cc156d60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 --2- 172.21.15.73:0/1800840036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18cc0b3aa0 0x7f18cc1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.550+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 >> 172.21.15.73:0/1800840036 conn(0x7f18cc06a0d0 msgr2=0x7f18cc15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.552+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 shutdown_connections 2024-04-05T22:27:21.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.552+0000 7f18d1600640 1 -- 172.21.15.73:0/1800840036 wait complete. 2024-04-05T22:27:21.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:27:21.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.770+0000 7f236ae00640 1 Processor -- start 2024-04-05T22:27:21.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.770+0000 7f236ae00640 1 -- start start 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f236ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23640b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f236ae00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236415b230 con 0x7f23640cf3e0 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f236ae00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236415f4a0 con 0x7f23640cce90 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f236ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23640ce2f0 con 0x7f236415ae50 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f2368e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f23640cf3e0 0x7f23640b1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49370/0 (socket says 172.21.15.73:49370) 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.772+0000 7f2368e00640 1 -- 172.21.15.73:0/3039994947 learned_addr learned my addr 172.21.15.73:0/3039994947 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2363e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23640b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205788617 0 0) 0x7f236415b230 con 0x7f23640cf3e0 2024-04-05T22:27:21.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2344003520 con 0x7f23640cf3e0 2024-04-05T22:27:21.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439991914 0 0) 0x7f2344003520 con 0x7f23640cf3e0 2024-04-05T22:27:21.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 msgr2=0x7f23640b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:21.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23640b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 >> v1:172.21.15.157:6789/0 conn(0x7f23640cce90 legacy=0x7f23640cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236415f780 con 0x7f23640cf3e0 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f235c001670 con 0x7f23640cf3e0 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 >> v1:172.21.15.73:6789/0 conn(0x7f23640cf3e0 legacy=0x7f23640b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344003f20 0x7f23440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2344004d00 0x7f2344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640ce2f0 con 0x7f2344004d00 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234400bb10 con 0x7f2344003f20 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.773+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234400bcd0 con 0x7f236415ae50 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2344004d00 0x7f2344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2363e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 -- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 msgr2=0x7f23440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236415ae50 0x7f23440039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 -- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344003f20 msgr2=0x7f23440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344003f20 0x7f23440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.774+0000 7f2368e00640 1 -- 172.21.15.73:0/3039994947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344003520 con 0x7f2344004d00 2024-04-05T22:27:21.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.775+0000 7f2368e00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2344004d00 0x7f2344005180 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c240 tx=0x7f23440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d68109340a984e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:21.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.775+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c005ad0 con 0x7f2344004d00 2024-04-05T22:27:21.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.775+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c005c30 con 0x7f2344004d00 2024-04-05T22:27:21.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.775+0000 7f2362a00640 1 -- 172.21.15.73:0/3039994947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c02c8b0 con 0x7f2344004d00 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 -- 172.21.15.73:0/3039994947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2344004d00 msgr2=0x7f2344005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 --2- 172.21.15.73:0/3039994947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2344004d00 0x7f2344005180 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c240 tx=0x7f23440057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f2368e00640 1 -- 172.21.15.73:0/3039994947 reap_dead start 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 -- 172.21.15.73:0/3039994947 shutdown_connections 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 -- 172.21.15.73:0/3039994947 >> 172.21.15.73:0/3039994947 conn(0x7f236406a0d0 msgr2=0x7f236406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 -- 172.21.15.73:0/3039994947 shutdown_connections 2024-04-05T22:27:21.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.776+0000 7f236ae00640 1 -- 172.21.15.73:0/3039994947 wait complete. 2024-04-05T22:27:21.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.778+0000 7f236ae00640 1 Processor -- start 2024-04-05T22:27:21.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.778+0000 7f236ae00640 1 -- start start 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344004d00 0x7f23640700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236414f2e0 0x7f23640ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640ae2e0 con 0x7f23640ade60 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640705f0 con 0x7f2344004d00 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f236ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23640c3ca0 con 0x7f236414f2e0 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49564/0 (socket says 172.21.15.73:49564) 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 -- 172.21.15.73:0/713976923 learned_addr learned my addr 172.21.15.73:0/713976923 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2363400640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236414f2e0 0x7f23640ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 -- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236414f2e0 msgr2=0x7f23640ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236414f2e0 0x7f23640ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 -- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344004d00 msgr2=0x7f23640700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2363e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344004d00 0x7f23640700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344004d00 0x7f23640700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.779+0000 7f2368e00640 1 -- 172.21.15.73:0/713976923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640c3f80 con 0x7f23640ade60 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f2368e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c330 tx=0x7f235c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c03c040 con 0x7f23640ade60 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c02ccc0 con 0x7f23640ade60 2024-04-05T22:27:21.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c007290 con 0x7f23640ade60 2024-04-05T22:27:21.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23640c4260 con 0x7f23640ade60 2024-04-05T22:27:21.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.780+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23640c4820 con 0x7f23640ade60 2024-04-05T22:27:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.781+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2364068160 con 0x7f23640ade60 2024-04-05T22:27:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.782+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f235c032070 con 0x7f23640ade60 2024-04-05T22:27:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.783+0000 7f235be00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 0x7f234006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.783+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f235c0a9fe0 con 0x7f23640ade60 2024-04-05T22:27:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.784+0000 7f2363e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 0x7f234006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.785+0000 7f2363e00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 0x7f234006d1e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2354009950 tx=0x7f2354008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.788+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f235c075170 con 0x7f23640ade60 2024-04-05T22:27:21.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:21.984+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f236415a7a0 con 0x7f23640ade60 2024-04-05T22:27:22.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.556+0000 7f235be00640 1 -- 172.21.15.73:0/713976923 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f235c078790 con 0x7f23640ade60 2024-04-05T22:27:22.556 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:27:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.563+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 msgr2=0x7f234006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.563+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 0x7f234006d1e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2354009950 tx=0x7f2354008040 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.563+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 msgr2=0x7f23640707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.563+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c330 tx=0x7f235c02e070 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 shutdown_connections 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f234006ad20 0x7f234006d1e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23640ade60 0x7f23640707d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f236414f2e0 0x7f23640ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 --2- 172.21.15.73:0/713976923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2344004d00 0x7f23640700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 >> 172.21.15.73:0/713976923 conn(0x7f236406a0d0 msgr2=0x7f23640ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.564+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 shutdown_connections 2024-04-05T22:27:22.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.565+0000 7f236ae00640 1 -- 172.21.15.73:0/713976923 wait complete. 2024-04-05T22:27:22.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:27:22.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.789+0000 7ff725c00640 1 Processor -- start 2024-04-05T22:27:22.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.789+0000 7ff725c00640 1 -- start start 2024-04-05T22:27:22.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 0x7ff7200b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7200b30a0 0x7ff7200b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 0x7ff7200ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7200b1b10 con 0x7ff7200b30a0 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7200b35b0 con 0x7ff7200b2470 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.790+0000 7ff725c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200b3890 con 0x7ff720158a30 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7200b30a0 0x7ff7200b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7200b30a0 0x7ff7200b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49570/0 (socket says 172.21.15.73:49570) 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 -- 172.21.15.73:0/3848765190 learned_addr learned my addr 172.21.15.73:0/3848765190 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 -- 172.21.15.73:0/3848765190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 msgr2=0x7ff7200b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff71fe00640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 0x7ff7200b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff725200640 1 --2- 172.21.15.73:0/3848765190 >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 0x7ff7200ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 0x7ff7200b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 -- 172.21.15.73:0/3848765190 >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 msgr2=0x7ff7200ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 --2- 172.21.15.73:0/3848765190 >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 0x7ff7200ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff717400640 1 -- 172.21.15.73:0/3848765190 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200b3b70 con 0x7ff7200b30a0 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff725200640 1 --2- 172.21.15.73:0/3848765190 >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 0x7ff7200ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.791+0000 7ff71fe00640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 0x7ff7200b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:22.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.793+0000 7ff717400640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff7200b1f30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff72006f320 tx=0x7ff70802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c248d7d3ab4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:22.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.793+0000 7ff71f400640 1 -- 172.21.15.73:0/3848765190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708031870 con 0x7ff7200b30a0 2024-04-05T22:27:22.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.793+0000 7ff71f400640 1 -- 172.21.15.73:0/3848765190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080319d0 con 0x7ff7200b30a0 2024-04-05T22:27:22.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.793+0000 7ff71f400640 1 -- 172.21.15.73:0/3848765190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708031d60 con 0x7ff7200b30a0 2024-04-05T22:27:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 -- 172.21.15.73:0/3848765190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 msgr2=0x7ff7200b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff7200b1f30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff72006f320 tx=0x7ff70802b910 comp rx=0 tx=0).stop 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 -- 172.21.15.73:0/3848765190 shutdown_connections 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff7200b1f30 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 --2- 172.21.15.73:0/3848765190 >> v2:172.21.15.157:3300/0 conn(0x7ff7200b2470 0x7ff7200ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 --2- 172.21.15.73:0/3848765190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720158a30 0x7ff7200b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 -- 172.21.15.73:0/3848765190 >> 172.21.15.73:0/3848765190 conn(0x7ff720069e90 msgr2=0x7ff72006a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 -- 172.21.15.73:0/3848765190 shutdown_connections 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.794+0000 7ff725c00640 1 -- 172.21.15.73:0/3848765190 wait complete. 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.796+0000 7ff725c00640 1 Processor -- start 2024-04-05T22:27:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.796+0000 7ff725c00640 1 -- start start 2024-04-05T22:27:22.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7200b2470 0x7ff720142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720158a30 0x7ff72015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff720154950 con 0x7ff7200b30a0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff720155140 con 0x7ff720158a30 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff725c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201429a0 con 0x7ff7200b2470 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49578/0 (socket says 172.21.15.73:49578) 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 -- 172.21.15.73:0/3280966652 learned_addr learned my addr 172.21.15.73:0/3280966652 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 -- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7200b2470 msgr2=0x7ff720142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7200b2470 0x7ff720142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 -- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720158a30 msgr2=0x7ff72015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720158a30 0x7ff72015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.797+0000 7ff717400640 1 -- 172.21.15.73:0/3280966652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080094d0 con 0x7ff7200b30a0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff717400640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708031840 con 0x7ff7200b30a0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080443f0 con 0x7ff7200b30a0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080447a0 con 0x7ff7200b30a0 2024-04-05T22:27:22.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7200cda50 con 0x7ff7200b30a0 2024-04-05T22:27:22.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.798+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720155690 con 0x7ff7200b30a0 2024-04-05T22:27:22.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.799+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff720155880 con 0x7ff7200b30a0 2024-04-05T22:27:22.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.800+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7ff70804d450 con 0x7ff7200b30a0 2024-04-05T22:27:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.801+0000 7ff71cc00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 0x7ff6fc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.801+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff70802d040 con 0x7ff7200b30a0 2024-04-05T22:27:22.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.802+0000 7ff71fe00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 0x7ff6fc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:22.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.803+0000 7ff71fe00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 0x7ff6fc06d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff710006fd0 tx=0x7ff710008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:22.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:22.806+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff708076db0 con 0x7ff7200b30a0 2024-04-05T22:27:23.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.002+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff72004e160 con 0x7ff7200b30a0 2024-04-05T22:27:23.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.011+0000 7ff71cc00640 1 -- 172.21.15.73:0/3280966652 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff70807a3d0 con 0x7ff7200b30a0 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 msgr2=0x7ff6fc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 0x7ff6fc06d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff710006fd0 tx=0x7ff710008040 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 msgr2=0x7ff720151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff7080046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 shutdown_connections 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6fc06ad20 0x7ff6fc06d1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720158a30 0x7ff72015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7200b30a0 0x7ff720151b20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 --2- 172.21.15.73:0/3280966652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7200b2470 0x7ff720142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 >> 172.21.15.73:0/3280966652 conn(0x7ff720069e90 msgr2=0x7ff72015a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 shutdown_connections 2024-04-05T22:27:23.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.013+0000 7ff725c00640 1 -- 172.21.15.73:0/3280966652 wait complete. 2024-04-05T22:27:23.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-05T22:27:23.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.239+0000 7f2215200640 1 Processor -- start 2024-04-05T22:27:23.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.239+0000 7f2215200640 1 -- start start 2024-04-05T22:27:23.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.240+0000 7f2215200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f221006d710 0x7f22100cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f2215200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221015aa60 con 0x7f221015aef0 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f2215200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100b3b60 con 0x7f22101541a0 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f2215200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100701c0 con 0x7f221006d710 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f220e800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f221015aef0 0x7f221015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49374/0 (socket says 172.21.15.73:49374) 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f220e800640 1 -- 172.21.15.73:0/663856278 learned_addr learned my addr 172.21.15.73:0/663856278 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:23.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f220d400640 1 --2- 172.21.15.73:0/663856278 >> v2:172.21.15.157:3300/0 conn(0x7f221006d710 0x7f22100cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.241+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181571262 0 0) 0x7f221015aa60 con 0x7f221015aef0 2024-04-05T22:27:23.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f4005730 con 0x7f221015aef0 2024-04-05T22:27:23.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2203090947 0 0) 0x7f21f4005730 con 0x7f221015aef0 2024-04-05T22:27:23.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 >> v2:172.21.15.157:3300/0 conn(0x7f221006d710 msgr2=0x7f22100cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 --2- 172.21.15.73:0/663856278 >> v2:172.21.15.157:3300/0 conn(0x7f221006d710 0x7f22100cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 >> v1:172.21.15.157:6789/0 conn(0x7f22101541a0 legacy=0x7f22100b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100b3880 con 0x7f221015aef0 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f22080019f0 con 0x7f221015aef0 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 >> v1:172.21.15.73:6789/0 conn(0x7f221015aef0 legacy=0x7f221015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 0x7f21f4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f40069d0 0x7f21f4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21f4007390 0x7f21f400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100701c0 con 0x7f21f4005b80 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f400e230 con 0x7f21f40069d0 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f400e3f0 con 0x7f21f4007390 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.242+0000 7f220de00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 0x7f21f4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220de00640 1 -- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21f4007390 msgr2=0x7f21f400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220de00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21f4007390 0x7f21f400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220de00640 1 -- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f40069d0 msgr2=0x7f21f4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220de00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f40069d0 0x7f21f4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220de00640 1 -- 172.21.15.73:0/663856278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4005730 con 0x7f21f4005b80 2024-04-05T22:27:23.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.243+0000 7f220e800640 1 -- 172.21.15.73:0/663856278 reap_dead start 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.244+0000 7f220de00640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 0x7f21f4005fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2200001750 tx=0x7f21f4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b39a91ba05d7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.244+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000052d0 con 0x7f21f4005b80 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.244+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220002e050 con 0x7f21f4005b80 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.244+0000 7f220ca00640 1 -- 172.21.15.73:0/663856278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200003830 con 0x7f21f4005b80 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 -- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 msgr2=0x7f21f4005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 0x7f21f4005fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2200001750 tx=0x7f21f4007a00 comp rx=0 tx=0).stop 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 -- 172.21.15.73:0/663856278 shutdown_connections 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 --2- 172.21.15.73:0/663856278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f4005b80 0x7f21f4005fa0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 -- 172.21.15.73:0/663856278 >> 172.21.15.73:0/663856278 conn(0x7f22100ce8e0 msgr2=0x7f22100cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 -- 172.21.15.73:0/663856278 shutdown_connections 2024-04-05T22:27:23.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.245+0000 7f2215200640 1 -- 172.21.15.73:0/663856278 wait complete. 2024-04-05T22:27:23.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.247+0000 7f2215200640 1 Processor -- start 2024-04-05T22:27:23.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.247+0000 7f2215200640 1 -- start start 2024-04-05T22:27:23.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f4005b80 0x7f2210142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100ad050 0x7f2210142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22101401d0 con 0x7f221006bcf0 2024-04-05T22:27:23.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221006c520 con 0x7f21f4005b80 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f2215200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221006c800 con 0x7f22100ad050 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49598/0 (socket says 172.21.15.73:49598) 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 -- 172.21.15.73:0/595446643 learned_addr learned my addr 172.21.15.73:0/595446643 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 -- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100ad050 msgr2=0x7f2210142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100ad050 0x7f2210142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 -- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f4005b80 msgr2=0x7f2210142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f4005b80 0x7f2210142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.248+0000 7f220d400640 1 -- 172.21.15.73:0/595446643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221006cae0 con 0x7f221006bcf0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f220d400640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00e770 tx=0x7f21fc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00c730 con 0x7f221006bcf0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc00c890 con 0x7f221006bcf0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00cc40 con 0x7f221006bcf0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22101403e0 con 0x7f221006bcf0 2024-04-05T22:27:23.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.249+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2210142740 con 0x7f221006bcf0 2024-04-05T22:27:23.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.250+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2210066af0 con 0x7f221006bcf0 2024-04-05T22:27:23.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.251+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f21fc013470 con 0x7f221006bcf0 2024-04-05T22:27:23.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.252+0000 7f2206000640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 0x7f21f006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.252+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f21fc0893b0 con 0x7f221006bcf0 2024-04-05T22:27:23.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.253+0000 7f220de00640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 0x7f21f006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.254+0000 7f220de00640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 0x7f21f006d210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2200033000 tx=0x7f220002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.257+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f21fc054720 con 0x7f221006bcf0 2024-04-05T22:27:23.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.524+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f2210068160 con 0x7f221006bcf0 2024-04-05T22:27:23.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.560+0000 7f2206000640 1 -- 172.21.15.73:0/595446643 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v327) v1 ==== 146+0+0 (secure 0 0 0) 0x7f21fc057d40 con 0x7f221006bcf0 2024-04-05T22:27:23.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 msgr2=0x7f21f006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 0x7f21f006d210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2200033000 tx=0x7f220002f000 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 msgr2=0x7f22100acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00e770 tx=0x7f21fc00ec30 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 shutdown_connections 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f21f006ad50 0x7f21f006d210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100ad050 0x7f2210142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221006bcf0 0x7f22100acb10 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 --2- 172.21.15.73:0/595446643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21f4005b80 0x7f2210142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 >> 172.21.15.73:0/595446643 conn(0x7f22100ce8e0 msgr2=0x7f221015b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 shutdown_connections 2024-04-05T22:27:23.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.563+0000 7f2215200640 1 -- 172.21.15.73:0/595446643 wait complete. 2024-04-05T22:27:23.583 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-04-05T22:27:23.583 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-05T22:27:23.583 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:27:23.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:23.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.790+0000 7f05eec00640 1 Processor -- start 2024-04-05T22:27:23.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.791+0000 7f05eec00640 1 -- start start 2024-04-05T22:27:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05eec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05e80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05eec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05e806d710 0x7f05e80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05eec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e815aa60 con 0x7f05e815aef0 2024-04-05T22:27:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05eec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80b3b60 con 0x7f05e806d710 2024-04-05T22:27:23.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05eec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80701c0 con 0x7f05e81541a0 2024-04-05T22:27:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05ecc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f05e815aef0 0x7f05e815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49380/0 (socket says 172.21.15.73:49380) 2024-04-05T22:27:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.792+0000 7f05ecc00640 1 -- 172.21.15.73:0/3650799142 learned_addr learned my addr 172.21.15.73:0/3650799142 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e7e00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05e80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913176853 0 0) 0x7f05e815aa60 con 0x7f05e815aef0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05cc005730 con 0x7f05e815aef0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e7400640 1 --2- 172.21.15.73:0/3650799142 >> v2:172.21.15.157:3300/0 conn(0x7f05e806d710 0x7f05e80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704859391 0 0) 0x7f05cc005730 con 0x7f05e815aef0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 msgr2=0x7f05e80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05e80b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 >> v2:172.21.15.157:3300/0 conn(0x7f05e806d710 msgr2=0x7f05e80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 --2- 172.21.15.73:0/3650799142 >> v2:172.21.15.157:3300/0 conn(0x7f05e806d710 0x7f05e80cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80b3880 con 0x7f05e815aef0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f05e00019f0 con 0x7f05e815aef0 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 >> v1:172.21.15.73:6789/0 conn(0x7f05e815aef0 legacy=0x7f05e815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.793+0000 7f05e6a00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05cc005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e6a00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc006100 0x7f05cc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e6a00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05cc006f10 0x7f05cc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b3b60 con 0x7f05cc006f10 2024-04-05T22:27:23.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80701c0 con 0x7f05cc006100 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc00dbe0 con 0x7f05e81541a0 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05cc006f10 0x7f05cc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05e7e00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05cc005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 -- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 msgr2=0x7f05cc005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e81541a0 0x7f05cc005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 -- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc006100 msgr2=0x7f05cc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc006100 0x7f05cc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.794+0000 7f05ecc00640 1 -- 172.21.15.73:0/3650799142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc005730 con 0x7f05cc006f10 2024-04-05T22:27:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.795+0000 7f05ecc00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05cc006f10 0x7f05cc007390 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f05e000a8d0 tx=0x7f05cc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d533c0111c58343 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.795+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0034630 con 0x7f05cc006f10 2024-04-05T22:27:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.795+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e002f070 con 0x7f05cc006f10 2024-04-05T22:27:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.795+0000 7f05e6a00640 1 -- 172.21.15.73:0/3650799142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00349d0 con 0x7f05cc006f10 2024-04-05T22:27:23.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 -- 172.21.15.73:0/3650799142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05cc006f10 msgr2=0x7f05cc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 --2- 172.21.15.73:0/3650799142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05cc006f10 0x7f05cc007390 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f05e000a8d0 tx=0x7f05cc007a00 comp rx=0 tx=0).stop 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05ecc00640 1 -- 172.21.15.73:0/3650799142 reap_dead start 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 -- 172.21.15.73:0/3650799142 shutdown_connections 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 -- 172.21.15.73:0/3650799142 >> 172.21.15.73:0/3650799142 conn(0x7f05e80ce8e0 msgr2=0x7f05e80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 -- 172.21.15.73:0/3650799142 shutdown_connections 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.796+0000 7f05eec00640 1 -- 172.21.15.73:0/3650799142 wait complete. 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.798+0000 7f05eec00640 1 Processor -- start 2024-04-05T22:27:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.798+0000 7f05eec00640 1 -- start start 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05cc006f10 0x7f05e8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e813fa30 0x7f05e813feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81403f0 con 0x7f05e8143360 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8140680 con 0x7f05e813fa30 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05eec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81f1a00 con 0x7f05cc006f10 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05e7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05e7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49628/0 (socket says 172.21.15.73:49628) 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.799+0000 7f05e7400640 1 -- 172.21.15.73:0/3588495878 learned_addr learned my addr 172.21.15.73:0/3588495878 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7e00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05cc006f10 0x7f05e8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 -- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05cc006f10 msgr2=0x7f05e8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05cc006f10 0x7f05e8158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 -- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e813fa30 msgr2=0x7f05e813feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05ecc00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e813fa30 0x7f05e813feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e813fa30 0x7f05e813feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 -- 172.21.15.73:0/3588495878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e81f1ce0 con 0x7f05e8143360 2024-04-05T22:27:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05e7400640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f05d000d7d0 tx=0x7f05d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000b6c0 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000b860 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.800+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e81f1fc0 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.801+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000bc10 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.801+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e81f2580 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.802+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e8066af0 con 0x7f05e8143360 2024-04-05T22:27:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.803+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f05d001e460 con 0x7f05e8143360 2024-04-05T22:27:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.803+0000 7f05dfe00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 0x7f05c406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.804+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f05d00897a0 con 0x7f05e8143360 2024-04-05T22:27:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.804+0000 7f05e7e00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 0x7f05c406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.805+0000 7f05e7e00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 0x7f05c406d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009a10 tx=0x7f05d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:23.808+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f05d0054ba0 con 0x7f05e8143360 2024-04-05T22:27:24.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.001+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05e8068160 con 0x7f05e8143360 2024-04-05T22:27:24.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.003+0000 7f05dfe00640 1 -- 172.21.15.73:0/3588495878 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+12651 (secure 0 0 0) 0x7f05d00581c0 con 0x7f05e8143360 2024-04-05T22:27:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:24.004 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":98,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:22.539665+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/3038019968":"2024-04-06T22:27:19.520637+0000","172.21.15.73:6832/3038019968":"2024-04-06T22:27:19.520637+0000","172.21.15.157:6835/2227587240":"2024-04-06T22:27:06.074507+0000","172.21.15.157:6834/2227587240":"2024-04-06T22:27:06.074507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.006+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 msgr2=0x7f05c406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.006+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 0x7f05c406d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009a10 tx=0x7f05d8008040 comp rx=0 tx=0).stop 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.006+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 msgr2=0x7f05e80af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.006+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f05d000d7d0 tx=0x7f05d000dc90 comp rx=0 tx=0).stop 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 shutdown_connections 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c406ad20 0x7f05c406d1e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e813fa30 0x7f05e813feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e8143360 0x7f05e80af010 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 --2- 172.21.15.73:0/3588495878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05cc006f10 0x7f05e8158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 >> 172.21.15.73:0/3588495878 conn(0x7f05e80ce8e0 msgr2=0x7f05e8151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 shutdown_connections 2024-04-05T22:27:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.007+0000 7f05eec00640 1 -- 172.21.15.73:0/3588495878 wait complete. 2024-04-05T22:27:24.026 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:27:24.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:27:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.238+0000 7f4556200640 1 Processor -- start 2024-04-05T22:27:24.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.238+0000 7f4556200640 1 -- start start 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.239+0000 7f4556200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 0x7f45500712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f4556200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f455006bc40 0x7f4550164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f4556200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455015afc0 con 0x7f455006c9a0 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f4556200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455006f5b0 con 0x7f455006bc40 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f4556200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455006f890 con 0x7f455015abe0 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f455006c9a0 0x7f455006b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49384/0 (socket says 172.21.15.73:49384) 2024-04-05T22:27:24.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454ea00640 1 -- 172.21.15.73:0/3595339508 learned_addr learned my addr 172.21.15.73:0/3595339508 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:24.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454f400640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 0x7f45500712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454fe00640 1 --2- 172.21.15.73:0/3595339508 >> v2:172.21.15.157:3300/0 conn(0x7f455006bc40 0x7f4550164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993038589 0 0) 0x7f455015afc0 con 0x7f455006c9a0 2024-04-05T22:27:24.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.240+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4534003520 con 0x7f455006c9a0 2024-04-05T22:27:24.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046184507 0 0) 0x7f4534003520 con 0x7f455006c9a0 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 msgr2=0x7f45500712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 0x7f45500712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 >> v2:172.21.15.157:3300/0 conn(0x7f455006bc40 msgr2=0x7f4550164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 --2- 172.21.15.73:0/3595339508 >> v2:172.21.15.157:3300/0 conn(0x7f455006bc40 0x7f4550164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455006fb70 con 0x7f455006c9a0 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f453c001670 con 0x7f455006c9a0 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 >> v1:172.21.15.73:6789/0 conn(0x7f455006c9a0 legacy=0x7f455006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4534003970 0x7f4534003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45340047c0 0x7f4534004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 0x7f4534005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455006f5b0 con 0x7f4534003970 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455006f890 con 0x7f45340047c0 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453400bab0 con 0x7f455015abe0 2024-04-05T22:27:24.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.241+0000 7f454f400640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4534003970 0x7f4534003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454ea00640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45340047c0 0x7f4534004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454f400640 1 -- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 msgr2=0x7f4534005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454f400640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455015abe0 0x7f4534005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454f400640 1 -- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45340047c0 msgr2=0x7f4534004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454f400640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45340047c0 0x7f4534004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454f400640 1 -- 172.21.15.73:0/3595339508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534003520 con 0x7f4534003970 2024-04-05T22:27:24.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.242+0000 7f454ea00640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45340047c0 0x7f4534004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.243+0000 7f454f400640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4534003970 0x7f4534003d90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4540009ac0 tx=0x7f45340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3770557702355590 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.243+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454002c710 con 0x7f4534003970 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.243+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454002c870 con 0x7f4534003970 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.243+0000 7f454e000640 1 -- 172.21.15.73:0/3595339508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454002cc00 con 0x7f4534003970 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 -- 172.21.15.73:0/3595339508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4534003970 msgr2=0x7f4534003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 --2- 172.21.15.73:0/3595339508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4534003970 0x7f4534003d90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4540009ac0 tx=0x7f45340057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f454fe00640 1 -- 172.21.15.73:0/3595339508 reap_dead start 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 -- 172.21.15.73:0/3595339508 shutdown_connections 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 -- 172.21.15.73:0/3595339508 >> 172.21.15.73:0/3595339508 conn(0x7f4550069cb0 msgr2=0x7f455006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 -- 172.21.15.73:0/3595339508 shutdown_connections 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.244+0000 7f4556200640 1 -- 172.21.15.73:0/3595339508 wait complete. 2024-04-05T22:27:24.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.245+0000 7f4556200640 1 Processor -- start 2024-04-05T22:27:24.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.246+0000 7f4556200640 1 -- start start 2024-04-05T22:27:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.246+0000 7f4556200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 0x7f455014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.246+0000 7f4556200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.246+0000 7f4556200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 0x7f45500af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f4556200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455014f020 con 0x7f4550149830 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f4556200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4550151040 con 0x7f4550150ba0 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f4556200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455014e680 con 0x7f4534003970 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49642/0 (socket says 172.21.15.73:49642) 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 -- 172.21.15.73:0/1809858059 learned_addr learned my addr 172.21.15.73:0/1809858059 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454f400640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 0x7f455014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454fe00640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 0x7f45500af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 -- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 msgr2=0x7f455014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 0x7f455014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 -- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 msgr2=0x7f45500af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 0x7f45500af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454ea00640 1 -- 172.21.15.73:0/1809858059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455014e960 con 0x7f4550149830 2024-04-05T22:27:24.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454f400640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 0x7f455014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.247+0000 7f454fe00640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 0x7f45500af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f454ea00640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f453c001120 tx=0x7f453c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c011dc0 con 0x7f4550149830 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c0070d0 con 0x7f4550149830 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455014ec40 con 0x7f4550149830 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c007440 con 0x7f4550149830 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.248+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455014a6d0 con 0x7f4550149830 2024-04-05T22:27:24.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.249+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4550068160 con 0x7f4550149830 2024-04-05T22:27:24.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.250+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f453c0075a0 con 0x7f4550149830 2024-04-05T22:27:24.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.251+0000 7f4547400640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 0x7f453006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.251+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f453c08d910 con 0x7f4550149830 2024-04-05T22:27:24.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.252+0000 7f454f400640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 0x7f453006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.253+0000 7f454f400640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 0x7f453006d1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4540036000 tx=0x7f4540030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.256+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f453c058be0 con 0x7f4550149830 2024-04-05T22:27:24.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.446+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4550155870 con 0x7f4550149830 2024-04-05T22:27:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.447+0000 7f4547400640 1 -- 172.21.15.73:0/1809858059 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f453c05c200 con 0x7f4550149830 2024-04-05T22:27:24.447 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:27:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 msgr2=0x7f453006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 0x7f453006d1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4540036000 tx=0x7f4540030000 comp rx=0 tx=0).stop 2024-04-05T22:27:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 msgr2=0x7f4550150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f453c001120 tx=0x7f453c0114b0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 shutdown_connections 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f453006ad20 0x7f453006d1e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550150ba0 0x7f45500af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4550149830 0x7f4550150660 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 --2- 172.21.15.73:0/1809858059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4534003970 0x7f455014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 >> 172.21.15.73:0/1809858059 conn(0x7f4550069cb0 msgr2=0x7f4550155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 shutdown_connections 2024-04-05T22:27:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.449+0000 7f4556200640 1 -- 172.21.15.73:0/1809858059 wait complete. 2024-04-05T22:27:24.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-05T22:27:24.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.676+0000 7fb184600640 1 Processor -- start 2024-04-05T22:27:24.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.676+0000 7fb184600640 1 -- start start 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb184600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb17c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb184600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c06f5b0 con 0x7fb17c15e850 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb184600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c070410 con 0x7fb17c1587d0 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb184600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c06d500 con 0x7fb17c06cb80 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb182600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb17c15e850 0x7fb17c0708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49386/0 (socket says 172.21.15.73:49386) 2024-04-05T22:27:24.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb182600640 1 -- 172.21.15.73:0/4027577192 learned_addr learned my addr 172.21.15.73:0/4027577192 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:24.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb181200640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb17c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.678+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020129843 0 0) 0x7fb17c06f5b0 con 0x7fb17c15e850 2024-04-05T22:27:24.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb164003520 con 0x7fb17c15e850 2024-04-05T22:27:24.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2336362916 0 0) 0x7fb164003520 con 0x7fb17c15e850 2024-04-05T22:27:24.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 msgr2=0x7fb17c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb17c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 >> v1:172.21.15.157:6789/0 conn(0x7fb17c1587d0 legacy=0x7fb17c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0ccef0 con 0x7fb17c15e850 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb1740019f0 con 0x7fb17c15e850 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 >> v1:172.21.15.73:6789/0 conn(0x7fb17c15e850 legacy=0x7fb17c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640039a0 0x7fb164003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1640047c0 0x7fb164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb164005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c06d500 con 0x7fb1640047c0 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16400ba30 con 0x7fb1640039a0 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16400bbf0 con 0x7fb17c06cb80 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.679+0000 7fb181200640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1640047c0 0x7fb164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181c00640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640039a0 0x7fb164003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb182600640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb164005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181200640 1 -- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 msgr2=0x7fb164005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181200640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb164005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181200640 1 -- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640039a0 msgr2=0x7fb164003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181200640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640039a0 0x7fb164003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb181200640 1 -- 172.21.15.73:0/4027577192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164003520 con 0x7fb1640047c0 2024-04-05T22:27:24.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.680+0000 7fb182600640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c06cb80 0x7fb164005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:24.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.681+0000 7fb181200640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1640047c0 0x7fb164004c40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb168009a90 tx=0x7fb1640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da1153d3e706563 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16802c660 con 0x7fb1640047c0 2024-04-05T22:27:24.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16802c7c0 con 0x7fb1640047c0 2024-04-05T22:27:24.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb17be00640 1 -- 172.21.15.73:0/4027577192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16802cb50 con 0x7fb1640047c0 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 -- 172.21.15.73:0/4027577192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1640047c0 msgr2=0x7fb164004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 --2- 172.21.15.73:0/4027577192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1640047c0 0x7fb164004c40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb168009a90 tx=0x7fb1640057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb182600640 1 -- 172.21.15.73:0/4027577192 reap_dead start 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 -- 172.21.15.73:0/4027577192 shutdown_connections 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 -- 172.21.15.73:0/4027577192 >> 172.21.15.73:0/4027577192 conn(0x7fb17c0ce8e0 msgr2=0x7fb17c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 -- 172.21.15.73:0/4027577192 shutdown_connections 2024-04-05T22:27:24.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.682+0000 7fb184600640 1 -- 172.21.15.73:0/4027577192 wait complete. 2024-04-05T22:27:24.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.685+0000 7fb184600640 1 Processor -- start 2024-04-05T22:27:24.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 -- start start 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640047c0 0x7fb17c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c0ac8d0 0x7fb17c0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0b34e0 con 0x7fb17c0b1b30 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0ad290 con 0x7fb1640047c0 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.686+0000 7fb184600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0b2860 con 0x7fb17c0ac8d0 2024-04-05T22:27:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49668/0 (socket says 172.21.15.73:49668) 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 -- 172.21.15.73:0/2827658710 learned_addr learned my addr 172.21.15.73:0/2827658710 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 -- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c0ac8d0 msgr2=0x7fb17c0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c0ac8d0 0x7fb17c0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 -- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640047c0 msgr2=0x7fb17c0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640047c0 0x7fb17c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 -- 172.21.15.73:0/2827658710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0b2b40 con 0x7fb17c0b1b30 2024-04-05T22:27:24.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.687+0000 7fb182600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb1740012e0 tx=0x7fb17400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.688+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400bb80 con 0x7fb17c0b1b30 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.688+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17400bd00 con 0x7fb17c0b1b30 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.688+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174007350 con 0x7fb17c0b1b30 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.688+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c0b2e20 con 0x7fb17c0b1b30 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.688+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0b24f0 con 0x7fb17c0b1b30 2024-04-05T22:27:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.689+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c066af0 con 0x7fb17c0b1b30 2024-04-05T22:27:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.690+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb1740074b0 con 0x7fb17c0b1b30 2024-04-05T22:27:24.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.691+0000 7fb179600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 0x7fb15c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:24.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.691+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb174089910 con 0x7fb17c0b1b30 2024-04-05T22:27:24.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.691+0000 7fb181c00640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 0x7fb15c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:24.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.693+0000 7fb181c00640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 0x7fb15c06d1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb170007fd0 tx=0x7fb170009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:24.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.695+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb174054b60 con 0x7fb17c0b1b30 2024-04-05T22:27:24.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:24.956+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7fb17c068160 con 0x7fb17c0b1b30 2024-04-05T22:27:25.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.572+0000 7fb179600640 1 -- 172.21.15.73:0/2827658710 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v329) v1 ==== 152+0+0 (secure 0 0 0) 0x7fb174058180 con 0x7fb17c0b1b30 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 msgr2=0x7fb15c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 0x7fb15c06d1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb170007fd0 tx=0x7fb170009040 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 msgr2=0x7fb17c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb1740012e0 tx=0x7fb17400b4a0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 shutdown_connections 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb15c06ad20 0x7fb15c06d1e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb17c0b1b30 0x7fb17c0b1fb0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb17c0ac8d0 0x7fb17c0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 --2- 172.21.15.73:0/2827658710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1640047c0 0x7fb17c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 >> 172.21.15.73:0/2827658710 conn(0x7fb17c0ce8e0 msgr2=0x7fb17c15d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 shutdown_connections 2024-04-05T22:27:25.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.575+0000 7fb184600640 1 -- 172.21.15.73:0/2827658710 wait complete. 2024-04-05T22:27:25.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:27:25.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.805+0000 7fe86da00640 1 Processor -- start 2024-04-05T22:27:25.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.805+0000 7fe86da00640 1 -- start start 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe86da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe86815cdd0 0x7fe8680ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe86da00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe8680d37d0 0x7fe8680d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe86da00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680cd800 con 0x7fe8680cf4b0 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe86da00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680cdae0 con 0x7fe8680d37d0 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe86da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680cddc0 con 0x7fe86815cdd0 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe867400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe8680d37d0 0x7fe8680d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe867400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe8680d37d0 0x7fe8680d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49674/0 (socket says 172.21.15.73:49674) 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.807+0000 7fe867400640 1 -- 172.21.15.73:0/572927426 learned_addr learned my addr 172.21.15.73:0/572927426 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.808+0000 7fe867e00640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe86815cdd0 0x7fe8680ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.808+0000 7fe867400640 1 -- 172.21.15.73:0/572927426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe86815cdd0 msgr2=0x7fe8680ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.808+0000 7fe867400640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe86815cdd0 0x7fe8680ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.808+0000 7fe867400640 1 -- 172.21.15.73:0/572927426 >> v1:172.21.15.157:6789/0 conn(0x7fe8680cf4b0 legacy=0x7fe8680b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:25.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.808+0000 7fe867400640 1 -- 172.21.15.73:0/572927426 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680ce0a0 con 0x7fe8680d37d0 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.809+0000 7fe867400640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8680d37d0 0x7fe8680d1130 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe854009870 tx=0x7fe85402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ecc66761fb1524 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.809+0000 7fe866a00640 1 -- 172.21.15.73:0/572927426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540317c0 con 0x7fe8680d37d0 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.809+0000 7fe866a00640 1 -- 172.21.15.73:0/572927426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854031920 con 0x7fe8680d37d0 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.809+0000 7fe866a00640 1 -- 172.21.15.73:0/572927426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854031cb0 con 0x7fe8680d37d0 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 -- 172.21.15.73:0/572927426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8680d37d0 msgr2=0x7fe8680d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8680d37d0 0x7fe8680d1130 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe854009870 tx=0x7fe85402b860 comp rx=0 tx=0).stop 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 -- 172.21.15.73:0/572927426 shutdown_connections 2024-04-05T22:27:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8680d37d0 0x7fe8680d1130 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 --2- 172.21.15.73:0/572927426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe86815cdd0 0x7fe8680ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 -- 172.21.15.73:0/572927426 >> 172.21.15.73:0/572927426 conn(0x7fe86809be20 msgr2=0x7fe8680d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:25.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 -- 172.21.15.73:0/572927426 shutdown_connections 2024-04-05T22:27:25.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.810+0000 7fe86da00640 1 -- 172.21.15.73:0/572927426 wait complete. 2024-04-05T22:27:25.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.812+0000 7fe86da00640 1 Processor -- start 2024-04-05T22:27:25.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.812+0000 7fe86da00640 1 -- start start 2024-04-05T22:27:25.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 0x7fe8680b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 0x7fe8680b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b8960 con 0x7fe86815cdd0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868151510 con 0x7fe8680cf4b0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680c8520 con 0x7fe8680d37d0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49684/0 (socket says 172.21.15.73:49684) 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 -- 172.21.15.73:0/3829970794 learned_addr learned my addr 172.21.15.73:0/3829970794 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe867e00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 0x7fe8680b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe867400640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 0x7fe8680b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 -- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 msgr2=0x7fe8680b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 0x7fe8680b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 -- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 msgr2=0x7fe8680b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 0x7fe8680b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe86d000640 1 -- 172.21.15.73:0/3829970794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540094d0 con 0x7fe86815cdd0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe867e00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 0x7fe8680b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.813+0000 7fe867400640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 0x7fe8680b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe86d000640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c7b0 tx=0x7fe86000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860004b60 con 0x7fe86815cdd0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860004cc0 con 0x7fe86815cdd0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86815d330 con 0x7fe86815cdd0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86000aa90 con 0x7fe86815cdd0 2024-04-05T22:27:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.814+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8680c7e90 con 0x7fe86815cdd0 2024-04-05T22:27:25.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.815+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8680c86f0 con 0x7fe86815cdd0 2024-04-05T22:27:25.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.816+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe86000abf0 con 0x7fe86815cdd0 2024-04-05T22:27:25.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.817+0000 7fe85fe00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 0x7fe84806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:25.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.817+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe860089530 con 0x7fe86815cdd0 2024-04-05T22:27:25.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.818+0000 7fe867e00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 0x7fe84806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.819+0000 7fe867e00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 0x7fe84806d1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe858006fd0 tx=0x7fe858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:25.822+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe860054770 con 0x7fe86815cdd0 2024-04-05T22:27:26.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.000+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe86804e160 con 0x7fe86815cdd0 2024-04-05T22:27:26.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.009+0000 7fe85fe00640 1 -- 172.21.15.73:0/3829970794 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe860057d90 con 0x7fe86815cdd0 2024-04-05T22:27:26.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 msgr2=0x7fe84806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 0x7fe84806d1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe858006fd0 tx=0x7fe858008040 comp rx=0 tx=0).stop 2024-04-05T22:27:26.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 msgr2=0x7fe868150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c7b0 tx=0x7fe86000cc70 comp rx=0 tx=0).stop 2024-04-05T22:27:26.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 shutdown_connections 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe84806ad20 0x7fe84806d1e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe86815cdd0 0x7fe868150fd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680d37d0 0x7fe8680b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 --2- 172.21.15.73:0/3829970794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680cf4b0 0x7fe8680b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 >> 172.21.15.73:0/3829970794 conn(0x7fe86809be20 msgr2=0x7fe86806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 shutdown_connections 2024-04-05T22:27:26.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.012+0000 7fe86da00640 1 -- 172.21.15.73:0/3829970794 wait complete. 2024-04-05T22:27:26.029 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:27:26.029 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:27:26.126 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:27:26.241 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:27:26.242 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:26.242 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:26.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:27:26.296 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:26.296 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:26.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:27:26.351 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:26.351 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:26.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:27:26.408 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:26.408 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:26.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:27:26.464 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:27:26.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:26.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.680+0000 7f47b1c00640 1 Processor -- start 2024-04-05T22:27:26.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.680+0000 7f47b1c00640 1 -- start start 2024-04-05T22:27:26.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.681+0000 7f47b1c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47ac1541a0 0x7f47ac0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47b1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 0x7f47ac15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47b1c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac15aa60 con 0x7f47ac06d710 2024-04-05T22:27:26.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47b1c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0b3b60 con 0x7f47ac1541a0 2024-04-05T22:27:26.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47b1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0701c0 con 0x7f47ac0ce000 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47ab400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 0x7f47ac15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47aaa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47ac1541a0 0x7f47ac0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47aa000640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47ac06d710 0x7f47ac0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:48844/0 (socket says 172.21.15.73:48844) 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.682+0000 7f47aa000640 1 -- 172.21.15.73:0/2402435359 learned_addr learned my addr 172.21.15.73:0/2402435359 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.683+0000 7f47ab400640 1 -- 172.21.15.73:0/2402435359 >> v2:172.21.15.157:3300/0 conn(0x7f47ac1541a0 msgr2=0x7f47ac0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:26.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.683+0000 7f47ab400640 1 --2- 172.21.15.73:0/2402435359 >> v2:172.21.15.157:3300/0 conn(0x7f47ac1541a0 0x7f47ac0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.683+0000 7f47ab400640 1 -- 172.21.15.73:0/2402435359 >> v1:172.21.15.157:6789/0 conn(0x7f47ac06d710 legacy=0x7f47ac0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.683+0000 7f47ab400640 1 -- 172.21.15.73:0/2402435359 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0b3880 con 0x7f47ac0ce000 2024-04-05T22:27:26.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.683+0000 7f47a9600640 1 -- 172.21.15.73:0/2402435359 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456359206 0 0) 0x7f47ac15aa60 con 0x7f47ac06d710 2024-04-05T22:27:26.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.685+0000 7f47ab400640 1 --2- 172.21.15.73:0/2402435359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 0x7f47ac15d860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d90 tx=0x7f47a402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae99bd47245783e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:26.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.685+0000 7f47a9600640 1 -- 172.21.15.73:0/2402435359 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a402cc50 con 0x7f47ac0ce000 2024-04-05T22:27:26.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.685+0000 7f47a9600640 1 -- 172.21.15.73:0/2402435359 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a402cdb0 con 0x7f47ac0ce000 2024-04-05T22:27:26.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.685+0000 7f47a9600640 1 -- 172.21.15.73:0/2402435359 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4034660 con 0x7f47ac0ce000 2024-04-05T22:27:26.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 -- 172.21.15.73:0/2402435359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 msgr2=0x7f47ac15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 --2- 172.21.15.73:0/2402435359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 0x7f47ac15d860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d90 tx=0x7f47a402bea0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 -- 172.21.15.73:0/2402435359 shutdown_connections 2024-04-05T22:27:26.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 --2- 172.21.15.73:0/2402435359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac0ce000 0x7f47ac15d860 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 --2- 172.21.15.73:0/2402435359 >> v2:172.21.15.157:3300/0 conn(0x7f47ac1541a0 0x7f47ac0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 -- 172.21.15.73:0/2402435359 >> 172.21.15.73:0/2402435359 conn(0x7f47ac0ce8e0 msgr2=0x7f47ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:26.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 -- 172.21.15.73:0/2402435359 shutdown_connections 2024-04-05T22:27:26.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.686+0000 7f47b1c00640 1 -- 172.21.15.73:0/2402435359 wait complete. 2024-04-05T22:27:26.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.688+0000 7f47b1c00640 1 Processor -- start 2024-04-05T22:27:26.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.688+0000 7f47b1c00640 1 -- start start 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ac0ce000 0x7f47ac14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac1541a0 0x7f47ac151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac154950 con 0x7f47ac06d710 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac14bba0 con 0x7f47ac0ce000 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47b1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1514e0 con 0x7f47ac1541a0 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:26.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38910/0 (socket says 172.21.15.73:38910) 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 -- 172.21.15.73:0/1957347196 learned_addr learned my addr 172.21.15.73:0/1957347196 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 -- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac1541a0 msgr2=0x7f47ac151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac1541a0 0x7f47ac151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 -- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ac0ce000 msgr2=0x7f47ac14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ac0ce000 0x7f47ac14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.689+0000 7f47aaa00640 1 -- 172.21.15.73:0/1957347196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4009a70 con 0x7f47ac06d710 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47aaa00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e870 tx=0x7f479c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0094f0 con 0x7f47ac06d710 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c00c940 con 0x7f47ac06d710 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00ccf0 con 0x7f47ac06d710 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac1517c0 con 0x7f47ac06d710 2024-04-05T22:27:26.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.690+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac15bb80 con 0x7f47ac06d710 2024-04-05T22:27:26.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.691+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac066af0 con 0x7f47ac06d710 2024-04-05T22:27:26.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.692+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f479c013470 con 0x7f47ac06d710 2024-04-05T22:27:26.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.693+0000 7f47a2a00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 0x7f478c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:26.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.693+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f479c089800 con 0x7f47ac06d710 2024-04-05T22:27:26.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.693+0000 7f47aa000640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 0x7f478c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:26.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.695+0000 7f47aa000640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 0x7f478c06d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4798007fd0 tx=0x7f4798007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:26.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.697+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f479c054a40 con 0x7f47ac06d710 2024-04-05T22:27:26.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.928+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47ac068160 con 0x7f47ac06d710 2024-04-05T22:27:26.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.930+0000 7f47a2a00640 1 -- 172.21.15.73:0/1957347196 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f479c021de0 con 0x7f47ac06d710 2024-04-05T22:27:26.930 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:26.930 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3752807160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3752807160},{"type":"v1","addr":"172.21.15.73:6835","nonce":3752807160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5135,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2868134067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2868134067},{"type":"v1","addr":"172.21.15.157:6833","nonce":2868134067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/602096336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":602096336},{"type":"v1","addr":"172.21.15.157:6835","nonce":602096336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7551,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1406340232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1406340232},{"type":"v1","addr":"172.21.15.73:6833","nonce":1406340232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[]} 2024-04-05T22:27:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 57 2024-04-05T22:27:26.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 msgr2=0x7f478c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 0x7f478c06d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4798007fd0 tx=0x7f4798007920 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 msgr2=0x7f47ac15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e870 tx=0x7f479c00ed30 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 shutdown_connections 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478c06ad20 0x7f478c06d1e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ac1541a0 0x7f47ac151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ac0ce000 0x7f47ac14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 --2- 172.21.15.73:0/1957347196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47ac06d710 0x7f47ac15c380 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 >> 172.21.15.73:0/1957347196 conn(0x7f47ac0ce8e0 msgr2=0x7f47ac0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 shutdown_connections 2024-04-05T22:27:26.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:26.932+0000 7f47b1c00640 1 -- 172.21.15.73:0/1957347196 wait complete. 2024-04-05T22:27:26.951 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:27:26.952 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:27:26.952 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:27:26.952 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:27:26.952 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:27:26.952 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:27:26.952 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:27:26.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:27:26.953 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:27:26.953 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:27:26.953 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:27:26.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:27:26.953 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:27:26.953 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:27:26.954 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:27:26.954 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:27:26.969 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:27:26.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:27:26.970 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:27:26.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:27:26.982 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:27:26.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:27:26.985 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:27:26.985 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:27:27.017 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:27:27.017 DEBUG:teuthology.parallel:result is None 2024-04-05T22:27:27.022 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:27:27.023 DEBUG:teuthology.parallel:result is None 2024-04-05T22:27:27.031 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:27:27.032 DEBUG:teuthology.parallel:result is None 2024-04-05T22:27:27.038 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:27:27.038 DEBUG:teuthology.parallel:result is None 2024-04-05T22:27:27.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:27:27.170 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:27:27.171 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:27:27.199 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:27:27.213 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:27:27.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.227+0000 7f6789e00640 1 Processor -- start 2024-04-05T22:27:27.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.227+0000 7f6789e00640 1 -- start start 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6789e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f678406cb80 0x7f678406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6789e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678406f5b0 con 0x7f678415e850 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6789e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6784070410 con 0x7f67841587d0 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6789e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678406d500 con 0x7f678406cb80 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6782000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f678406cb80 0x7f678406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6783400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f678415e850 0x7f67840708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53082/0 (socket says 172.21.15.73:53082) 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6783400640 1 -- 172.21.15.73:0/1046457949 learned_addr learned my addr 172.21.15.73:0/1046457949 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:27.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.229+0000 7f6782a00640 1 --1- 172.21.15.73:0/1046457949 >> v1:172.21.15.157:6789/0 conn(0x7f67841587d0 0x7f67840cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:48862/0 (socket says 172.21.15.73:48862) 2024-04-05T22:27:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.230+0000 7f6782000640 1 -- 172.21.15.73:0/1046457949 >> v1:172.21.15.157:6789/0 conn(0x7f67841587d0 legacy=0x7f67840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.230+0000 7f6782000640 1 -- 172.21.15.73:0/1046457949 >> v1:172.21.15.73:6789/0 conn(0x7f678415e850 legacy=0x7f67840708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:27.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.230+0000 7f6782000640 1 -- 172.21.15.73:0/1046457949 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840ccef0 con 0x7f678406cb80 2024-04-05T22:27:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.231+0000 7f6782000640 1 --2- 172.21.15.73:0/1046457949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678406cb80 0x7f678406cfc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f67700097a0 tx=0x7f677002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee79e7b850d42b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.231+0000 7f6781600640 1 -- 172.21.15.73:0/1046457949 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770030610 con 0x7f678406cb80 2024-04-05T22:27:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.231+0000 7f6781600640 1 -- 172.21.15.73:0/1046457949 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6770030770 con 0x7f678406cb80 2024-04-05T22:27:27.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.231+0000 7f6781600640 1 -- 172.21.15.73:0/1046457949 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770030b00 con 0x7f678406cb80 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 -- 172.21.15.73:0/1046457949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678406cb80 msgr2=0x7f678406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 --2- 172.21.15.73:0/1046457949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678406cb80 0x7f678406cfc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f67700097a0 tx=0x7f677002b960 comp rx=0 tx=0).stop 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 -- 172.21.15.73:0/1046457949 shutdown_connections 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 --2- 172.21.15.73:0/1046457949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678406cb80 0x7f678406cfc0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 -- 172.21.15.73:0/1046457949 >> 172.21.15.73:0/1046457949 conn(0x7f67840ce8e0 msgr2=0x7f67840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:27.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 -- 172.21.15.73:0/1046457949 shutdown_connections 2024-04-05T22:27:27.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.232+0000 7f6789e00640 1 -- 172.21.15.73:0/1046457949 wait complete. 2024-04-05T22:27:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.234+0000 7f6789e00640 1 Processor -- start 2024-04-05T22:27:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.234+0000 7f6789e00640 1 -- start start 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406cb80 0x7f67841462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67841587d0 0x7f67841e5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67841e4670 con 0x7f678415e850 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67841e48c0 con 0x7f67841587d0 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6789e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67841e4ba0 con 0x7f678406cb80 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6783400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6783400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38976/0 (socket says 172.21.15.73:38976) 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6783400640 1 -- 172.21.15.73:0/2590913264 learned_addr learned my addr 172.21.15.73:0/2590913264 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6782000640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67841587d0 0x7f67841e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.235+0000 7f6782a00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406cb80 0x7f67841462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 -- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406cb80 msgr2=0x7f67841462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406cb80 0x7f67841462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 -- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67841587d0 msgr2=0x7f67841e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67841587d0 0x7f67841e5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 -- 172.21.15.73:0/2590913264 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770009400 con 0x7f678415e850 2024-04-05T22:27:27.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6783400640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f677c00ca80 tx=0x7f677c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c00ac30 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c00ad90 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67841e4e80 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c012600 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.236+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67841e5440 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.238+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6784066af0 con 0x7f678415e850 2024-04-05T22:27:27.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.239+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f677c007070 con 0x7f678415e850 2024-04-05T22:27:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.239+0000 7f677b400640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 0x7f676406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.240+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f677c088ff0 con 0x7f678415e850 2024-04-05T22:27:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.240+0000 7f6782a00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 0x7f676406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.241+0000 7f6782a00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 0x7f676406d1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67840ad5c0 tx=0x7f6774001050 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.244+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f677c054410 con 0x7f678415e850 2024-04-05T22:27:27.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:27.451+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67841e5630 con 0x7f678415e850 2024-04-05T22:27:28.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.246+0000 7f677b400640 1 -- 172.21.15.73:0/2590913264 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f677c057a30 con 0x7f678415e850 2024-04-05T22:27:28.247 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:27:28.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 msgr2=0x7f676406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 0x7f676406d1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67840ad5c0 tx=0x7f6774001050 comp rx=0 tx=0).stop 2024-04-05T22:27:28.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 msgr2=0x7f67840af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f677c00ca80 tx=0x7f677c00cf40 comp rx=0 tx=0).stop 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 shutdown_connections 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676406ad20 0x7f676406d1e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678415e850 0x7f67840af4f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67841587d0 0x7f67841e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 --2- 172.21.15.73:0/2590913264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678406cb80 0x7f67841462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 >> 172.21.15.73:0/2590913264 conn(0x7f67840ce8e0 msgr2=0x7f678415ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.251+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 shutdown_connections 2024-04-05T22:27:28.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.252+0000 7f6789e00640 1 -- 172.21.15.73:0/2590913264 wait complete. 2024-04-05T22:27:28.273 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:27:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.482+0000 7fc9e6000640 1 Processor -- start 2024-04-05T22:27:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.482+0000 7fc9e6000640 1 -- start start 2024-04-05T22:27:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9e6000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 0x7fc9e006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9e6000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9e015e850 0x7fc9e015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9e6000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e006f5b0 con 0x7fc9e01587d0 2024-04-05T22:27:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9e6000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e006d500 con 0x7fc9e015e850 2024-04-05T22:27:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9e6000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e015f220 con 0x7fc9e006cb80 2024-04-05T22:27:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9e015e850 0x7fc9e015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9e015e850 0x7fc9e015ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38984/0 (socket says 172.21.15.73:38984) 2024-04-05T22:27:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.484+0000 7fc9dfe00640 1 -- 172.21.15.73:0/451705936 learned_addr learned my addr 172.21.15.73:0/451705936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dea00640 1 --2- 172.21.15.73:0/451705936 >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 0x7fc9e006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dfe00640 1 -- 172.21.15.73:0/451705936 >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 msgr2=0x7fc9e006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dfe00640 1 --2- 172.21.15.73:0/451705936 >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 0x7fc9e006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dfe00640 1 -- 172.21.15.73:0/451705936 >> v1:172.21.15.73:6789/0 conn(0x7fc9e01587d0 legacy=0x7fc9e00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dfe00640 1 -- 172.21.15.73:0/451705936 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0070410 con 0x7fc9e015e850 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.485+0000 7fc9dea00640 1 --2- 172.21.15.73:0/451705936 >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 0x7fc9e006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.486+0000 7fc9dfe00640 1 --2- 172.21.15.73:0/451705936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e015e850 0x7fc9e015ece0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc9e006b560 tx=0x7fc9d802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f544af070b885df server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.486+0000 7fc9de000640 1 -- 172.21.15.73:0/451705936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d802cb10 con 0x7fc9e015e850 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.486+0000 7fc9de000640 1 -- 172.21.15.73:0/451705936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d802cc70 con 0x7fc9e015e850 2024-04-05T22:27:28.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.486+0000 7fc9de000640 1 -- 172.21.15.73:0/451705936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80343f0 con 0x7fc9e015e850 2024-04-05T22:27:28.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 -- 172.21.15.73:0/451705936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e015e850 msgr2=0x7fc9e015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 --2- 172.21.15.73:0/451705936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e015e850 0x7fc9e015ece0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc9e006b560 tx=0x7fc9d802bf70 comp rx=0 tx=0).stop 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 -- 172.21.15.73:0/451705936 shutdown_connections 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 --2- 172.21.15.73:0/451705936 >> v2:172.21.15.157:3300/0 conn(0x7fc9e006cb80 0x7fc9e006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 --2- 172.21.15.73:0/451705936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e015e850 0x7fc9e015ece0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 -- 172.21.15.73:0/451705936 >> 172.21.15.73:0/451705936 conn(0x7fc9e00ce8e0 msgr2=0x7fc9e00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 -- 172.21.15.73:0/451705936 shutdown_connections 2024-04-05T22:27:28.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.487+0000 7fc9e6000640 1 -- 172.21.15.73:0/451705936 wait complete. 2024-04-05T22:27:28.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.489+0000 7fc9e6000640 1 Processor -- start 2024-04-05T22:27:28.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.489+0000 7fc9e6000640 1 -- start start 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 0x7fc9e0157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9e015e850 0x7fc9e00c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01f03f0 con 0x7fc9e006cb80 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e015b5a0 con 0x7fc9e01587d0 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.490+0000 7fc9e6000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01f62d0 con 0x7fc9e015e850 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38990/0 (socket says 172.21.15.73:38990) 2024-04-05T22:27:28.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 -- 172.21.15.73:0/3238923227 learned_addr learned my addr 172.21.15.73:0/3238923227 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9dea00640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 0x7fc9e0157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 -- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9e015e850 msgr2=0x7fc9e00c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9dfe00640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9e015e850 0x7fc9e00c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9e015e850 0x7fc9e00c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 -- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 msgr2=0x7fc9e0157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 0x7fc9e0157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9df400640 1 -- 172.21.15.73:0/3238923227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009b30 con 0x7fc9e006cb80 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.491+0000 7fc9dea00640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 0x7fc9e0157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9df400640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000c450 tx=0x7fc9d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00052d0 con 0x7fc9e006cb80 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d0005430 con 0x7fc9e006cb80 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0012870 con 0x7fc9e006cb80 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e01f65b0 con 0x7fc9e006cb80 2024-04-05T22:27:28.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.492+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e01f6b70 con 0x7fc9e006cb80 2024-04-05T22:27:28.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.493+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e0066af0 con 0x7fc9e006cb80 2024-04-05T22:27:28.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.494+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc9d00129d0 con 0x7fc9e006cb80 2024-04-05T22:27:28.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.495+0000 7fc9d7400640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 0x7fc9c006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.495+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9d0088c00 con 0x7fc9e006cb80 2024-04-05T22:27:28.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.496+0000 7fc9dea00640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 0x7fc9c006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.497+0000 7fc9dea00640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 0x7fc9c006d1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc006fd0 tx=0x7fc9cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:28.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.499+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc9d008f080 con 0x7fc9e006cb80 2024-04-05T22:27:28.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.765+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc9e0068160 con 0x7fc9e006cb80 2024-04-05T22:27:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.767+0000 7fc9d7400640 1 -- 172.21.15.73:0/3238923227 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc9d00540b0 con 0x7fc9e006cb80 2024-04-05T22:27:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:28.767 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:27:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:27:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:27:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:27:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:27:28.780 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.769+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 msgr2=0x7fc9c006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.769+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 0x7fc9c006d1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc006fd0 tx=0x7fc9cc008040 comp rx=0 tx=0).stop 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.769+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 msgr2=0x7fc9e015b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.769+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000c450 tx=0x7fc9d000c910 comp rx=0 tx=0).stop 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 shutdown_connections 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9c006ad20 0x7fc9c006d1e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9e015e850 0x7fc9e00c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01587d0 0x7fc9e0157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 --2- 172.21.15.73:0/3238923227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9e006cb80 0x7fc9e015b060 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 >> 172.21.15.73:0/3238923227 conn(0x7fc9e00ce8e0 msgr2=0x7fc9e0159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:28.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 shutdown_connections 2024-04-05T22:27:28.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.770+0000 7fc9e6000640 1 -- 172.21.15.73:0/3238923227 wait complete. 2024-04-05T22:27:28.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:28.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.992+0000 7f408ec00640 1 Processor -- start 2024-04-05T22:27:28.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.992+0000 7f408ec00640 1 -- start start 2024-04-05T22:27:28.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.994+0000 7f408ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40881541a0 0x7f40880b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.994+0000 7f408ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:28.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.994+0000 7f408ec00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408815aa60 con 0x7f408806d710 2024-04-05T22:27:28.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.994+0000 7f408ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40880b3b60 con 0x7f40881541a0 2024-04-05T22:27:28.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.994+0000 7f408ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40880701c0 con 0x7f40880ce000 2024-04-05T22:27:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.995+0000 7f408cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.995+0000 7f408cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:44156/0 (socket says 172.21.15.73:44156) 2024-04-05T22:27:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.995+0000 7f408cc00640 1 -- 172.21.15.73:0/1335134246 learned_addr learned my addr 172.21.15.73:0/1335134246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.995+0000 7f4087e00640 1 --2- 172.21.15.73:0/1335134246 >> v2:172.21.15.157:3300/0 conn(0x7f40881541a0 0x7f40880b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.996+0000 7f408cc00640 1 -- 172.21.15.73:0/1335134246 >> v2:172.21.15.157:3300/0 conn(0x7f40881541a0 msgr2=0x7f40880b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.996+0000 7f408cc00640 1 --2- 172.21.15.73:0/1335134246 >> v2:172.21.15.157:3300/0 conn(0x7f40881541a0 0x7f40880b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.996+0000 7f408cc00640 1 -- 172.21.15.73:0/1335134246 >> v1:172.21.15.157:6789/0 conn(0x7f408806d710 legacy=0x7f40880cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.996+0000 7f408cc00640 1 -- 172.21.15.73:0/1335134246 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40880b3880 con 0x7f40880ce000 2024-04-05T22:27:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.997+0000 7f408cc00640 1 --2- 172.21.15.73:0/1335134246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4080009870 tx=0x7f408002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2903f218e03bc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.997+0000 7f4086a00640 1 -- 172.21.15.73:0/1335134246 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080030840 con 0x7f40880ce000 2024-04-05T22:27:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.997+0000 7f4086a00640 1 -- 172.21.15.73:0/1335134246 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800309a0 con 0x7f40880ce000 2024-04-05T22:27:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.997+0000 7f4086a00640 1 -- 172.21.15.73:0/1335134246 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080030d30 con 0x7f40880ce000 2024-04-05T22:27:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 -- 172.21.15.73:0/1335134246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 msgr2=0x7f408815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 --2- 172.21.15.73:0/1335134246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4080009870 tx=0x7f408002b980 comp rx=0 tx=0).stop 2024-04-05T22:27:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 -- 172.21.15.73:0/1335134246 shutdown_connections 2024-04-05T22:27:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 --2- 172.21.15.73:0/1335134246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40880ce000 0x7f408815d860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 --2- 172.21.15.73:0/1335134246 >> v2:172.21.15.157:3300/0 conn(0x7f40881541a0 0x7f40880b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 -- 172.21.15.73:0/1335134246 >> 172.21.15.73:0/1335134246 conn(0x7f40880ce8e0 msgr2=0x7f40880cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 -- 172.21.15.73:0/1335134246 shutdown_connections 2024-04-05T22:27:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:28.998+0000 7f408ec00640 1 -- 172.21.15.73:0/1335134246 wait complete. 2024-04-05T22:27:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.000+0000 7f408ec00640 1 Processor -- start 2024-04-05T22:27:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.000+0000 7f408ec00640 1 -- start start 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 0x7f40880aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 0x7f4088148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4088144930 con 0x7f408806d710 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408815b7a0 con 0x7f40880ce000 2024-04-05T22:27:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408815ba80 con 0x7f40881541a0 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f4087e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f4087e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38992/0 (socket says 172.21.15.73:38992) 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f4087e00640 1 -- 172.21.15.73:0/1837466008 learned_addr learned my addr 172.21.15.73:0/1837466008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f408cc00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 0x7f4088148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.001+0000 7f4087400640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 0x7f40880aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 -- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 msgr2=0x7f4088148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 0x7f4088148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 -- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 msgr2=0x7f40880aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 0x7f40880aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 -- 172.21.15.73:0/1837466008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800094d0 con 0x7f408806d710 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f408cc00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 0x7f4088148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087400640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 0x7f40880aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f4087e00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f407800efc0 tx=0x7f407800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078009b80 con 0x7f408806d710 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407800f040 con 0x7f408806d710 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408815bd60 con 0x7f408806d710 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780146a0 con 0x7f408806d710 2024-04-05T22:27:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.002+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408815c320 con 0x7f408806d710 2024-04-05T22:27:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.003+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088066af0 con 0x7f408806d710 2024-04-05T22:27:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.005+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f4078014800 con 0x7f408806d710 2024-04-05T22:27:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.005+0000 7f407fe00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 0x7f406406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.006+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4078089e70 con 0x7f408806d710 2024-04-05T22:27:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.006+0000 7f4087400640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 0x7f406406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.007+0000 7f4087400640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 0x7f406406d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f40881f5a10 tx=0x7f4070009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.010+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4078055180 con 0x7f408806d710 2024-04-05T22:27:29.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.202+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40880b3540 con 0x7f408806d710 2024-04-05T22:27:29.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.203+0000 7f407fe00640 1 -- 172.21.15.73:0/1837466008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+12391 (secure 0 0 0) 0x7f40780587a0 con 0x7f408806d710 2024-04-05T22:27:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:29.204 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":103,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:28.235370+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 msgr2=0x7f406406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 0x7f406406d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f40881f5a10 tx=0x7f4070009040 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 msgr2=0x7f40881443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f407800efc0 tx=0x7f407800c490 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 shutdown_connections 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f406406ad20 0x7f406406d1e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40881541a0 0x7f4088148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880ce000 0x7f40880aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 --2- 172.21.15.73:0/1837466008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f408806d710 0x7f40881443f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 >> 172.21.15.73:0/1837466008 conn(0x7f40880ce8e0 msgr2=0x7f408806b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 shutdown_connections 2024-04-05T22:27:29.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.206+0000 7f408ec00640 1 -- 172.21.15.73:0/1837466008 wait complete. 2024-04-05T22:27:29.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:27:29.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:27:29.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.432+0000 7f7218200640 1 Processor -- start 2024-04-05T22:27:29.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.433+0000 7f7218200640 1 -- start start 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7218200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f721006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7218200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f721006ba70 0x7f7210164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7218200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721006bec0 con 0x7f721015abe0 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7218200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721015afc0 con 0x7f721006ba70 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7218200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721006f5b0 con 0x7f721006c9a0 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7215800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f721015abe0 0x7f72100712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53102/0 (socket says 172.21.15.73:53102) 2024-04-05T22:27:29.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.434+0000 7f7215800640 1 -- 172.21.15.73:0/883172597 learned_addr learned my addr 172.21.15.73:0/883172597 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f7214e00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f721006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939051872 0 0) 0x7f7204001110 con 0x7f721015abe0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f7216200640 1 --2- 172.21.15.73:0/883172597 >> v2:172.21.15.157:3300/0 conn(0x7f721006ba70 0x7f7210164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f8003520 con 0x7f721015abe0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812217467 0 0) 0x7f71f8003520 con 0x7f721015abe0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 msgr2=0x7f721006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f721006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 >> v2:172.21.15.157:3300/0 conn(0x7f721006ba70 msgr2=0x7f7210164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 --2- 172.21.15.73:0/883172597 >> v2:172.21.15.157:3300/0 conn(0x7f721006ba70 0x7f7210164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721006f890 con 0x7f721015abe0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7204001800 con 0x7f721015abe0 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 >> v1:172.21.15.73:6789/0 conn(0x7f721015abe0 legacy=0x7f72100712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.435+0000 7f720fe00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8003970 0x7f71f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f720fe00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f720fe00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f71f8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721015afc0 con 0x7f71f8003970 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721006f5b0 con 0x7f71f80047c0 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f800bb90 con 0x7f721006c9a0 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8003970 0x7f71f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7216200640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f71f8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 -- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 msgr2=0x7f71f8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721006c9a0 0x7f71f8005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7214e00640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 -- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f80047c0 msgr2=0x7f71f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.436+0000 7f7215800640 1 -- 172.21.15.73:0/883172597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8003520 con 0x7f71f8003970 2024-04-05T22:27:29.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.437+0000 7f7215800640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8003970 0x7f71f8003d90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f720400c210 tx=0x7f71f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e7611831935abe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720402cae0 con 0x7f71f8003970 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720402cc60 con 0x7f71f8003970 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f720fe00640 1 -- 172.21.15.73:0/883172597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72040058d0 con 0x7f71f8003970 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 -- 172.21.15.73:0/883172597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8003970 msgr2=0x7f71f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 --2- 172.21.15.73:0/883172597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8003970 0x7f71f8003d90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f720400c210 tx=0x7f71f80057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7216200640 1 -- 172.21.15.73:0/883172597 reap_dead start 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 -- 172.21.15.73:0/883172597 shutdown_connections 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 -- 172.21.15.73:0/883172597 >> 172.21.15.73:0/883172597 conn(0x7f7210069cb0 msgr2=0x7f721006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 -- 172.21.15.73:0/883172597 shutdown_connections 2024-04-05T22:27:29.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.438+0000 7f7218200640 1 -- 172.21.15.73:0/883172597 wait complete. 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.440+0000 7f7218200640 1 Processor -- start 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.440+0000 7f7218200640 1 -- start start 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 0x7f721014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721015ccc0 0x7f721015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101373e0 con 0x7f721013a160 2024-04-05T22:27:29.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721015d680 con 0x7f71f8003970 2024-04-05T22:27:29.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7218200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721013a5e0 con 0x7f721015ccc0 2024-04-05T22:27:29.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7214e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7214e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39016/0 (socket says 172.21.15.73:39016) 2024-04-05T22:27:29.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.441+0000 7f7214e00640 1 -- 172.21.15.73:0/951750311 learned_addr learned my addr 172.21.15.73:0/951750311 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7215800640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 0x7f721014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7216200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721015ccc0 0x7f721015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 -- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721015ccc0 msgr2=0x7f721015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721015ccc0 0x7f721015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 -- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 msgr2=0x7f721014e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 0x7f721014e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 -- 172.21.15.73:0/951750311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721014e810 con 0x7f721013a160 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7215800640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 0x7f721014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7214e00640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00e7e0 tx=0x7f71fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00c670 con 0x7f721013a160 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc00c7d0 con 0x7f721013a160 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721014eaf0 con 0x7f721013a160 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.442+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00cb40 con 0x7f721013a160 2024-04-05T22:27:29.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.443+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721014f0b0 con 0x7f721013a160 2024-04-05T22:27:29.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.444+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210068160 con 0x7f721013a160 2024-04-05T22:27:29.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.445+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f71fc00cca0 con 0x7f721013a160 2024-04-05T22:27:29.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.445+0000 7f720d600640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 0x7f71f006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.446+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f71fc088fc0 con 0x7f721013a160 2024-04-05T22:27:29.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.446+0000 7f7215800640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 0x7f71f006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.447+0000 7f7215800640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 0x7f71f006d1e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7204034000 tx=0x7f7204007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.450+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f71fc054360 con 0x7f721013a160 2024-04-05T22:27:29.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.643+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72100b1300 con 0x7f721013a160 2024-04-05T22:27:29.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.644+0000 7f720d600640 1 -- 172.21.15.73:0/951750311 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71fc057980 con 0x7f721013a160 2024-04-05T22:27:29.644 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.646+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 msgr2=0x7f71f006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.646+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 0x7f71f006d1e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7204034000 tx=0x7f7204007000 comp rx=0 tx=0).stop 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.646+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 msgr2=0x7f7210136ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.646+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00e7e0 tx=0x7f71fc00eca0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 shutdown_connections 2024-04-05T22:27:29.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f71f006ad20 0x7f71f006d1e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f721015ccc0 0x7f721015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721013a160 0x7f7210136ea0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 --2- 172.21.15.73:0/951750311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71f8003970 0x7f721014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 >> 172.21.15.73:0/951750311 conn(0x7f7210069cb0 msgr2=0x7f72100b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 shutdown_connections 2024-04-05T22:27:29.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.647+0000 7f7218200640 1 -- 172.21.15.73:0/951750311 wait complete. 2024-04-05T22:27:29.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:27:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.871+0000 7fd29e600640 1 Processor -- start 2024-04-05T22:27:29.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.871+0000 7fd29e600640 1 -- start start 2024-04-05T22:27:29.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd29e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2981587d0 0x7fd2980cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd29e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd29815e850 0x7fd29815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd29e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29806f5b0 con 0x7fd2980ccd20 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd29e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29815f240 con 0x7fd29815e850 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd29e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd298070410 con 0x7fd2981587d0 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd29815e850 0x7fd29815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd29815e850 0x7fd29815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39032/0 (socket says 172.21.15.73:39032) 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 -- 172.21.15.73:0/3794842610 learned_addr learned my addr 172.21.15.73:0/3794842610 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 -- 172.21.15.73:0/3794842610 >> v2:172.21.15.157:3300/0 conn(0x7fd2981587d0 msgr2=0x7fd2980cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297400640 1 --2- 172.21.15.73:0/3794842610 >> v2:172.21.15.157:3300/0 conn(0x7fd2981587d0 0x7fd2980cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 --2- 172.21.15.73:0/3794842610 >> v2:172.21.15.157:3300/0 conn(0x7fd2981587d0 0x7fd2980cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 -- 172.21.15.73:0/3794842610 >> v1:172.21.15.157:6789/0 conn(0x7fd2980ccd20 legacy=0x7fd29806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:29.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.873+0000 7fd297e00640 1 -- 172.21.15.73:0/3794842610 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd298158bb0 con 0x7fd29815e850 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.874+0000 7fd297e00640 1 --2- 172.21.15.73:0/3794842610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd29815ed00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd290009e50 tx=0x7fd29002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e76b537ed21204 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.875+0000 7fd296a00640 1 -- 172.21.15.73:0/3794842610 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29002ca80 con 0x7fd29815e850 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.875+0000 7fd296a00640 1 -- 172.21.15.73:0/3794842610 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29002cbe0 con 0x7fd29815e850 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.875+0000 7fd296a00640 1 -- 172.21.15.73:0/3794842610 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900343f0 con 0x7fd29815e850 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.875+0000 7fd29e600640 1 -- 172.21.15.73:0/3794842610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 msgr2=0x7fd29815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.875+0000 7fd29e600640 1 --2- 172.21.15.73:0/3794842610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd29815ed00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd290009e50 tx=0x7fd29002bee0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 -- 172.21.15.73:0/3794842610 shutdown_connections 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 --2- 172.21.15.73:0/3794842610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd29815ed00 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 --2- 172.21.15.73:0/3794842610 >> v2:172.21.15.157:3300/0 conn(0x7fd2981587d0 0x7fd2980cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 -- 172.21.15.73:0/3794842610 >> 172.21.15.73:0/3794842610 conn(0x7fd2980ce8e0 msgr2=0x7fd2980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 -- 172.21.15.73:0/3794842610 shutdown_connections 2024-04-05T22:27:29.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.876+0000 7fd29e600640 1 -- 172.21.15.73:0/3794842610 wait complete. 2024-04-05T22:27:29.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.877+0000 7fd29e600640 1 Processor -- start 2024-04-05T22:27:29.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.878+0000 7fd29e600640 1 -- start start 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2981587d0 0x7fd2981f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298155d90 0x7fd2980ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298144050 con 0x7fd29815e850 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981f0550 con 0x7fd2981587d0 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd29e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981562a0 con 0x7fd298155d90 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39036/0 (socket says 172.21.15.73:39036) 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 -- 172.21.15.73:0/3692068920 learned_addr learned my addr 172.21.15.73:0/3692068920 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 -- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298155d90 msgr2=0x7fd2980ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298155d90 0x7fd2980ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 -- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2981587d0 msgr2=0x7fd2981f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2981587d0 0x7fd2981f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 -- 172.21.15.73:0/3692068920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290009b30 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.879+0000 7fd28ea00640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c410 tx=0x7fd28000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.880+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800035f0 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.880+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280003750 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.880+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280003b40 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.880+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298156580 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.880+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981442a0 con 0x7fd29815e850 2024-04-05T22:27:29.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.881+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298066af0 con 0x7fd29815e850 2024-04-05T22:27:29.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.882+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fd280003ca0 con 0x7fd29815e850 2024-04-05T22:27:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.883+0000 7fd28fe00640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 0x7fd27406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.883+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd280088310 con 0x7fd29815e850 2024-04-05T22:27:29.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.883+0000 7fd297400640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 0x7fd27406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:29.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.885+0000 7fd297400640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 0x7fd27406d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd288006fd0 tx=0x7fd288008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:29.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:29.887+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd2800537c0 con 0x7fd29815e850 2024-04-05T22:27:30.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.084+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd298068160 con 0x7fd29815e850 2024-04-05T22:27:30.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.085+0000 7fd28fe00640 1 -- 172.21.15.73:0/3692068920 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd280056de0 con 0x7fd29815e850 2024-04-05T22:27:30.085 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:27:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:27:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:27:30.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:27:30.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 msgr2=0x7fd27406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 0x7fd27406d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd288006fd0 tx=0x7fd288008040 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 msgr2=0x7fd298155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c410 tx=0x7fd28000c8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 shutdown_connections 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27406ad20 0x7fd27406d1e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298155d90 0x7fd2980ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29815e850 0x7fd298155850 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 --2- 172.21.15.73:0/3692068920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2981587d0 0x7fd2981f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 >> 172.21.15.73:0/3692068920 conn(0x7fd2980ce8e0 msgr2=0x7fd298159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 shutdown_connections 2024-04-05T22:27:30.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.088+0000 7fd29e600640 1 -- 172.21.15.73:0/3692068920 wait complete. 2024-04-05T22:27:30.108 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:27:30.108 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:27:30.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.325+0000 7fc2b4800640 1 Processor -- start 2024-04-05T22:27:30.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.325+0000 7fc2b4800640 1 -- start start 2024-04-05T22:27:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.327+0000 7fc2b4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac15ae50 0x7fc2ac0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.327+0000 7fc2b4800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.327+0000 7fc2b4800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ac15b230 con 0x7fc2ac0cce90 2024-04-05T22:27:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.327+0000 7fc2b4800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ac0ce2f0 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.327+0000 7fc2b4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0cd2f0 con 0x7fc2ac15ae50 2024-04-05T22:27:30.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39038/0 (socket says 172.21.15.73:39038) 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 -- 172.21.15.73:0/1200639997 learned_addr learned my addr 172.21.15.73:0/1200639997 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b1e00640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac15ae50 0x7fc2ac0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 -- 172.21.15.73:0/1200639997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac15ae50 msgr2=0x7fc2ac0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac15ae50 0x7fc2ac0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 -- 172.21.15.73:0/1200639997 >> v1:172.21.15.157:6789/0 conn(0x7fc2ac0cce90 legacy=0x7fc2ac0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.328+0000 7fc2b2800640 1 -- 172.21.15.73:0/1200639997 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac15f4a0 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.329+0000 7fc2b2800640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ce430 tx=0x7fc2a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ca51b6164c6798 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b0a00640 1 -- 172.21.15.73:0/1200639997 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8031640 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b0a00640 1 -- 172.21.15.73:0/1200639997 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a80317a0 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b0a00640 1 -- 172.21.15.73:0/1200639997 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8031b30 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 -- 172.21.15.73:0/1200639997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cf3e0 msgr2=0x7fc2ac0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ce430 tx=0x7fc2a802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 -- 172.21.15.73:0/1200639997 shutdown_connections 2024-04-05T22:27:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cf3e0 0x7fc2ac0b2e00 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1200639997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac15ae50 0x7fc2ac0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 -- 172.21.15.73:0/1200639997 >> 172.21.15.73:0/1200639997 conn(0x7fc2ac06a0d0 msgr2=0x7fc2ac06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 -- 172.21.15.73:0/1200639997 shutdown_connections 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.330+0000 7fc2b4800640 1 -- 172.21.15.73:0/1200639997 wait complete. 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.332+0000 7fc2b4800640 1 Processor -- start 2024-04-05T22:27:30.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.332+0000 7fc2b4800640 1 -- start start 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac0cf3e0 0x7fc2ac13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac15ae50 0x7fc2ac151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac148930 con 0x7fc2ac0cce90 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac145b60 con 0x7fc2ac15ae50 2024-04-05T22:27:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.333+0000 7fc2b4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac145e00 con 0x7fc2ac0cf3e0 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39040/0 (socket says 172.21.15.73:39040) 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 -- 172.21.15.73:0/1426807374 learned_addr learned my addr 172.21.15.73:0/1426807374 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 -- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac0cf3e0 msgr2=0x7fc2ac13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac0cf3e0 0x7fc2ac13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 -- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac15ae50 msgr2=0x7fc2ac151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac15ae50 0x7fc2ac151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 -- 172.21.15.73:0/1426807374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80094d0 con 0x7fc2ac0cce90 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.334+0000 7fc2b1e00640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007e70 tx=0x7fc2a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.335+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4014620 con 0x7fc2ac0cce90 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.335+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a4014780 con 0x7fc2ac0cce90 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.335+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4014b30 con 0x7fc2ac0cce90 2024-04-05T22:27:30.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.335+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac13fbe0 con 0x7fc2ac0cce90 2024-04-05T22:27:30.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.335+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac146050 con 0x7fc2ac0cce90 2024-04-05T22:27:30.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.336+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac15e4c0 con 0x7fc2ac0cce90 2024-04-05T22:27:30.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.337+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fc2a4014c90 con 0x7fc2ac0cce90 2024-04-05T22:27:30.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.338+0000 7fc2a2000640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 0x7fc29006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.338+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2a4089aa0 con 0x7fc2ac0cce90 2024-04-05T22:27:30.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.338+0000 7fc2b1400640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 0x7fc29006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.339+0000 7fc2b1400640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 0x7fc29006d1e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000f60 tx=0x7fc29c0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.342+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc2a4054f50 con 0x7fc2ac0cce90 2024-04-05T22:27:30.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.536+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc2ac068160 con 0x7fc2ac0cce90 2024-04-05T22:27:30.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.606+0000 7fc2a2000640 1 -- 172.21.15.73:0/1426807374 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc2a4058570 con 0x7fc2ac0cce90 2024-04-05T22:27:30.607 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:27:30.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.611+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 msgr2=0x7fc29006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.611+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 0x7fc29006d1e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000f60 tx=0x7fc29c0079b0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.611+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 msgr2=0x7fc2ac153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.611+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007e70 tx=0x7fc2a400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 shutdown_connections 2024-04-05T22:27:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc29006ad20 0x7fc29006d1e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac15ae50 0x7fc2ac151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac0cf3e0 0x7fc2ac13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 --2- 172.21.15.73:0/1426807374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac0cce90 0x7fc2ac153aa0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 >> 172.21.15.73:0/1426807374 conn(0x7fc2ac06a0d0 msgr2=0x7fc2ac0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 shutdown_connections 2024-04-05T22:27:30.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.612+0000 7fc2b4800640 1 -- 172.21.15.73:0/1426807374 wait complete. 2024-04-05T22:27:30.636 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:27:30.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.840+0000 7f4533a00640 1 Processor -- start 2024-04-05T22:27:30.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.840+0000 7f4533a00640 1 -- start start 2024-04-05T22:27:30.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4533a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f452c15ee80 0x7f452c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4533a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f452c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4533a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c15f260 con 0x7f452c165310 2024-04-05T22:27:30.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4533a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c1657c0 con 0x7f452c15ee80 2024-04-05T22:27:30.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4533a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c070d20 con 0x7f452c0b1950 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.842+0000 7f4531a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f452c165310 0x7f452c161850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53116/0 (socket says 172.21.15.73:53116) 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f4531a00640 1 -- 172.21.15.73:0/1128836445 learned_addr learned my addr 172.21.15.73:0/1128836445 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f4523e00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f452c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> v2:172.21.15.157:3300/0 conn(0x7f452c15ee80 0x7f452c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024316357 0 0) 0x7f4524001490 con 0x7f452c165310 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4510003520 con 0x7f452c165310 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3020444850 0 0) 0x7f4510003520 con 0x7f452c165310 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 msgr2=0x7f452c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f452c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 >> v2:172.21.15.157:3300/0 conn(0x7f452c15ee80 msgr2=0x7f452c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:30.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 --2- 172.21.15.73:0/1128836445 >> v2:172.21.15.157:3300/0 conn(0x7f452c15ee80 0x7f452c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c06f5b0 con 0x7f452c165310 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4524001b80 con 0x7f452c165310 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.843+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 >> v1:172.21.15.73:6789/0 conn(0x7f452c165310 legacy=0x7f452c161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f4510003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f45100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4510004d00 0x7f4510005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1657c0 con 0x7f4510003970 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451000b980 con 0x7f4510004d00 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451000bac0 con 0x7f452c0b1950 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> v2:172.21.15.157:3300/0 conn(0x7f452c15ee80 0x7f452c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f4510003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4523e00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f45100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 -- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 msgr2=0x7f45100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:30.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c0b1950 0x7f45100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 -- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4510004d00 msgr2=0x7f4510005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:30.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4510004d00 0x7f4510005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.844+0000 7f4531000640 1 -- 172.21.15.73:0/1128836445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510003520 con 0x7f4510003970 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4531000640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f4510003d90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098d0 tx=0x7f45100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea067696f4b4f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c030680 con 0x7f4510003970 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c0307e0 con 0x7f4510003970 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f452be00640 1 -- 172.21.15.73:0/1128836445 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c030b50 con 0x7f4510003970 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4533a00640 1 -- 172.21.15.73:0/1128836445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 msgr2=0x7f4510003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4533a00640 1 --2- 172.21.15.73:0/1128836445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f4510003d90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098d0 tx=0x7f45100057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4531a00640 1 -- 172.21.15.73:0/1128836445 reap_dead start 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4533a00640 1 -- 172.21.15.73:0/1128836445 shutdown_connections 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4533a00640 1 -- 172.21.15.73:0/1128836445 >> 172.21.15.73:0/1128836445 conn(0x7f452c069cb0 msgr2=0x7f452c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.846+0000 7f4533a00640 1 -- 172.21.15.73:0/1128836445 shutdown_connections 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.847+0000 7f4533a00640 1 -- 172.21.15.73:0/1128836445 wait complete. 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.848+0000 7f4533a00640 1 Processor -- start 2024-04-05T22:27:30.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.848+0000 7f4533a00640 1 -- start start 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452c153310 0x7f452c1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c1f72e0 0x7f452c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0cb890 con 0x7f4510003970 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0cbb30 con 0x7f452c153310 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.849+0000 7f4533a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0cbe10 con 0x7f452c1f72e0 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39058/0 (socket says 172.21.15.73:39058) 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 -- 172.21.15.73:0/4143082237 learned_addr learned my addr 172.21.15.73:0/4143082237 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c1f72e0 0x7f452c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4523e00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452c153310 0x7f452c1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 -- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c1f72e0 msgr2=0x7f452c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c1f72e0 0x7f452c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 -- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452c153310 msgr2=0x7f452c1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452c153310 0x7f452c1f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 -- 172.21.15.73:0/4143082237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0cc0f0 con 0x7f4510003970 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.850+0000 7f4531000640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f451c02bf10 tx=0x7f451c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.851+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c039a00 con 0x7f4510003970 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.851+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c03f040 con 0x7f4510003970 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.851+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c039de0 con 0x7f4510003970 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.851+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0cc3d0 con 0x7f4510003970 2024-04-05T22:27:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.851+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0ae5c0 con 0x7f4510003970 2024-04-05T22:27:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.852+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0cb1c0 con 0x7f4510003970 2024-04-05T22:27:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.853+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f451c056430 con 0x7f4510003970 2024-04-05T22:27:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.854+0000 7f4529600640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 0x7f450806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.854+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f451c0ac350 con 0x7f4510003970 2024-04-05T22:27:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.855+0000 7f4523e00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 0x7f450806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.856+0000 7f4523e00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 0x7f450806d210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f452c1602c0 tx=0x7f4514008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:30.859+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f451c077710 con 0x7f4510003970 2024-04-05T22:27:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.052+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f452c0cb890 con 0x7f4510003970 2024-04-05T22:27:31.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.629+0000 7f4529600640 1 -- 172.21.15.73:0/4143082237 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f451c034020 con 0x7f4510003970 2024-04-05T22:27:31.634 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:27:31.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.637+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 msgr2=0x7f450806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:31.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.637+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 0x7f450806d210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f452c1602c0 tx=0x7f4514008040 comp rx=0 tx=0).stop 2024-04-05T22:27:31.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.637+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 msgr2=0x7f452c0cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:31.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.637+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f451c02bf10 tx=0x7f451c004730 comp rx=0 tx=0).stop 2024-04-05T22:27:31.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 shutdown_connections 2024-04-05T22:27:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f450806ad50 0x7f450806d210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f452c1f72e0 0x7f452c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452c153310 0x7f452c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 --2- 172.21.15.73:0/4143082237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510003970 0x7f452c0cb350 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.638+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 >> 172.21.15.73:0/4143082237 conn(0x7f452c069cb0 msgr2=0x7f452c0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.640+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 shutdown_connections 2024-04-05T22:27:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.641+0000 7f4533a00640 1 -- 172.21.15.73:0/4143082237 wait complete. 2024-04-05T22:27:31.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:27:31.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.854+0000 7f12c9a00640 1 Processor -- start 2024-04-05T22:27:31.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.854+0000 7f12c9a00640 1 -- start start 2024-04-05T22:27:31.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c9a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12c406c9a0 0x7f12c406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c9a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12c406ba70 0x7f12c4164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c9a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c406bea0 con 0x7f12c415abe0 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c9a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c415afc0 con 0x7f12c406ba70 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c9a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c406f5b0 con 0x7f12c406c9a0 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12c406ba70 0x7f12c4164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12c406ba70 0x7f12c4164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39062/0 (socket says 172.21.15.73:39062) 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.856+0000 7f12c3400640 1 -- 172.21.15.73:0/943089633 learned_addr learned my addr 172.21.15.73:0/943089633 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.857+0000 7f12c3400640 1 -- 172.21.15.73:0/943089633 >> v2:172.21.15.157:3300/0 conn(0x7f12c406c9a0 msgr2=0x7f12c406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.857+0000 7f12c3400640 1 --2- 172.21.15.73:0/943089633 >> v2:172.21.15.157:3300/0 conn(0x7f12c406c9a0 0x7f12c406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.857+0000 7f12c3400640 1 -- 172.21.15.73:0/943089633 >> v1:172.21.15.157:6789/0 conn(0x7f12c415abe0 legacy=0x7f12c40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:31.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.857+0000 7f12c3400640 1 -- 172.21.15.73:0/943089633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c406f890 con 0x7f12c406ba70 2024-04-05T22:27:31.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.858+0000 7f12c3400640 1 --2- 172.21.15.73:0/943089633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c4164740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f12c406c900 tx=0x7f12bc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710e68670dc2ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:31.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.858+0000 7f12c1600640 1 -- 172.21.15.73:0/943089633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc02c960 con 0x7f12c406ba70 2024-04-05T22:27:31.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.858+0000 7f12c1600640 1 -- 172.21.15.73:0/943089633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc02cac0 con 0x7f12c406ba70 2024-04-05T22:27:31.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.858+0000 7f12c1600640 1 -- 172.21.15.73:0/943089633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc02ce50 con 0x7f12c406ba70 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 -- 172.21.15.73:0/943089633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 msgr2=0x7f12c4164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 --2- 172.21.15.73:0/943089633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c4164740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f12c406c900 tx=0x7f12bc02bdc0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 -- 172.21.15.73:0/943089633 shutdown_connections 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 --2- 172.21.15.73:0/943089633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c4164740 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 --2- 172.21.15.73:0/943089633 >> v2:172.21.15.157:3300/0 conn(0x7f12c406c9a0 0x7f12c406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 -- 172.21.15.73:0/943089633 >> 172.21.15.73:0/943089633 conn(0x7f12c4069cb0 msgr2=0x7f12c406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:31.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 -- 172.21.15.73:0/943089633 shutdown_connections 2024-04-05T22:27:31.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.859+0000 7f12c9a00640 1 -- 172.21.15.73:0/943089633 wait complete. 2024-04-05T22:27:31.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.861+0000 7f12c9a00640 1 Processor -- start 2024-04-05T22:27:31.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.861+0000 7f12c9a00640 1 -- start start 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c406c9a0 0x7f12c4143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 0x7f12c4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4153ca0 con 0x7f12c406ba70 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4153f00 con 0x7f12c415abe0 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4150550 con 0x7f12c406c9a0 2024-04-05T22:27:31.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39064/0 (socket says 172.21.15.73:39064) 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.862+0000 7f12c2a00640 1 -- 172.21.15.73:0/3152782402 learned_addr learned my addr 172.21.15.73:0/3152782402 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c3400640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 0x7f12c4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2000640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c406c9a0 0x7f12c4143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 -- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c406c9a0 msgr2=0x7f12c4143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c406c9a0 0x7f12c4143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 -- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 msgr2=0x7f12c4153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 0x7f12c4153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 -- 172.21.15.73:0/3152782402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc009b30 con 0x7f12c406ba70 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c3400640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 0x7f12c4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:31.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.863+0000 7f12c2a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f12b400b3e0 tx=0x7f12b400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.864+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4011830 con 0x7f12c406ba70 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.864+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b40119b0 con 0x7f12c406ba70 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.864+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4011d60 con 0x7f12c406ba70 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.864+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c4150830 con 0x7f12c406ba70 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.864+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c4150df0 con 0x7f12c406ba70 2024-04-05T22:27:31.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.865+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4151010 con 0x7f12c406ba70 2024-04-05T22:27:31.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.866+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f12b401a400 con 0x7f12c406ba70 2024-04-05T22:27:31.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.867+0000 7f12baa00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 0x7f12a406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:31.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.867+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f12b4089b60 con 0x7f12c406ba70 2024-04-05T22:27:31.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.868+0000 7f12c2000640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 0x7f12a406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:31.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.869+0000 7f12c2000640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 0x7f12a406d1e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12c4147a10 tx=0x7f12b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:31.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:31.871+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f12b4054e20 con 0x7f12c406ba70 2024-04-05T22:27:32.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.102+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f12c4068160 con 0x7f12c406ba70 2024-04-05T22:27:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.116+0000 7f12baa00640 1 -- 172.21.15.73:0/3152782402 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f12b40150c0 con 0x7f12c406ba70 2024-04-05T22:27:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 16 and data pool 17 2024-04-05T22:27:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.122+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 msgr2=0x7f12a406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.122+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 0x7f12a406d1e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12c4147a10 tx=0x7f12b0008040 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.122+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 msgr2=0x7f12c41f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.122+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f12b400b3e0 tx=0x7f12b400b8a0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 shutdown_connections 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f12a406ad20 0x7f12a406d1e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c415abe0 0x7f12c4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c406c9a0 0x7f12c4143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 --2- 172.21.15.73:0/3152782402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c406ba70 0x7f12c41f6a90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 >> 172.21.15.73:0/3152782402 conn(0x7f12c4069cb0 msgr2=0x7f12c40af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 shutdown_connections 2024-04-05T22:27:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.123+0000 7f12c9a00640 1 -- 172.21.15.73:0/3152782402 wait complete. 2024-04-05T22:27:32.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:32.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.326+0000 7f133fe00640 1 Processor -- start 2024-04-05T22:27:32.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.326+0000 7f133fe00640 1 -- start start 2024-04-05T22:27:32.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f133815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133815aa60 con 0x7f13381541a0 2024-04-05T22:27:32.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133fe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380b3b60 con 0x7f133806d710 2024-04-05T22:27:32.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380701c0 con 0x7f13380ce000 2024-04-05T22:27:32.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f13381541a0 0x7f13380b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53124/0 (socket says 172.21.15.73:53124) 2024-04-05T22:27:32.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.327+0000 7f133d400640 1 -- 172.21.15.73:0/1544779016 learned_addr learned my addr 172.21.15.73:0/1544779016 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944098724 0 0) 0x7f133815aa60 con 0x7f13381541a0 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320003520 con 0x7f13381541a0 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f133de00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f133815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4236090329 0 0) 0x7f1320003520 con 0x7f13381541a0 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 msgr2=0x7f133815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f133815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 >> v1:172.21.15.157:6789/0 conn(0x7f133806d710 legacy=0x7f13380cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380b3880 con 0x7f13381541a0 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f132c001670 con 0x7f13381541a0 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 >> v1:172.21.15.73:6789/0 conn(0x7f13381541a0 legacy=0x7f13380b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13200039a0 0x7f1320003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.328+0000 7f1337e00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f13200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f1337e00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1320004d00 0x7f1320005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380701c0 con 0x7f13200039a0 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132000bab0 con 0x7f1320004d00 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132000bc70 con 0x7f13380ce000 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13200039a0 0x7f1320003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133ca00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f13200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 -- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 msgr2=0x7f13200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13380ce000 0x7f13200047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133de00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1320004d00 0x7f1320005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 -- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1320004d00 msgr2=0x7f1320005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1320004d00 0x7f1320005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.329+0000 7f133d400640 1 -- 172.21.15.73:0/1544779016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320003520 con 0x7f13200039a0 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.330+0000 7f133d400640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13200039a0 0x7f1320003da0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c210 tx=0x7f13200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c80445fa012abb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c005ad0 con 0x7f13200039a0 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c005c50 con 0x7f13200039a0 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f1337e00640 1 -- 172.21.15.73:0/1544779016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c02c830 con 0x7f13200039a0 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 -- 172.21.15.73:0/1544779016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13200039a0 msgr2=0x7f1320003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 --2- 172.21.15.73:0/1544779016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13200039a0 0x7f1320003da0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c210 tx=0x7f13200057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133de00640 1 -- 172.21.15.73:0/1544779016 reap_dead start 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 -- 172.21.15.73:0/1544779016 shutdown_connections 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 -- 172.21.15.73:0/1544779016 >> 172.21.15.73:0/1544779016 conn(0x7f13380ce8e0 msgr2=0x7f13380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:32.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 -- 172.21.15.73:0/1544779016 shutdown_connections 2024-04-05T22:27:32.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.331+0000 7f133fe00640 1 -- 172.21.15.73:0/1544779016 wait complete. 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.333+0000 7f133fe00640 1 Processor -- start 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.333+0000 7f133fe00640 1 -- start start 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13200039a0 0x7f1338155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338146f60 0x7f13381473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338147920 con 0x7f133815b9e0 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13381eeb40 con 0x7f1338146f60 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133815c270 con 0x7f13200039a0 2024-04-05T22:27:32.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39086/0 (socket says 172.21.15.73:39086) 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.334+0000 7f133ca00640 1 -- 172.21.15.73:0/1655122677 learned_addr learned my addr 172.21.15.73:0/1655122677 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 -- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13200039a0 msgr2=0x7f1338155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13200039a0 0x7f1338155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 -- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338146f60 msgr2=0x7f13381473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338146f60 0x7f13381473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 -- 172.21.15.73:0/1655122677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133815c550 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133ca00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f132400d4e0 tx=0x7f132400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324012800 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324012960 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324012d50 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133815c830 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.335+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13380afdf0 con 0x7f133815b9e0 2024-04-05T22:27:32.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.337+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338066af0 con 0x7f133815b9e0 2024-04-05T22:27:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.337+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f1324019360 con 0x7f133815b9e0 2024-04-05T22:27:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.339+0000 7f1335600640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 0x7f131806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.339+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1324089770 con 0x7f133815b9e0 2024-04-05T22:27:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.339+0000 7f133d400640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 0x7f131806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.340+0000 7f133d400640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 0x7f131806d1e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f132c034000 tx=0x7f132c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.343+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f13240546a0 con 0x7f133815b9e0 2024-04-05T22:27:32.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.532+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1338068160 con 0x7f133815b9e0 2024-04-05T22:27:32.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.534+0000 7f1335600640 1 -- 172.21.15.73:0/1655122677 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16072 (secure 0 0 0) 0x7f1324020090 con 0x7f133815b9e0 2024-04-05T22:27:32.534 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:32.535 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":106,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:32.104121+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:32.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.536+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 msgr2=0x7f131806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 0x7f131806d1e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f132c034000 tx=0x7f132c007000 comp rx=0 tx=0).stop 2024-04-05T22:27:32.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 msgr2=0x7f13380af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f132400d4e0 tx=0x7f132400d9a0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 shutdown_connections 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f131806ad20 0x7f131806d1e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338146f60 0x7f13381473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133815b9e0 0x7f13380af8b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 --2- 172.21.15.73:0/1655122677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13200039a0 0x7f1338155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 >> 172.21.15.73:0/1655122677 conn(0x7f13380ce8e0 msgr2=0x7f1338159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 shutdown_connections 2024-04-05T22:27:32.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.537+0000 7f133fe00640 1 -- 172.21.15.73:0/1655122677 wait complete. 2024-04-05T22:27:32.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:32.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.757+0000 7f4ea4200640 1 Processor -- start 2024-04-05T22:27:32.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.757+0000 7f4ea4200640 1 -- start start 2024-04-05T22:27:32.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c1587d0 0x7f4e9c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e9c0ccd20 0x7f4e9c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e9c06d0c0 0x7f4e9c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c06f5b0 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c070410 con 0x7f4e9c0ccd20 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c158bb0 con 0x7f4e9c1587d0 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e9c06d0c0 0x7f4e9c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e9c06d0c0 0x7f4e9c15e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39098/0 (socket says 172.21.15.73:39098) 2024-04-05T22:27:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.759+0000 7f4ea2200640 1 -- 172.21.15.73:0/3645399630 learned_addr learned my addr 172.21.15.73:0/3645399630 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea1800640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c1587d0 0x7f4e9c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea2200640 1 -- 172.21.15.73:0/3645399630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c1587d0 msgr2=0x7f4e9c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea0e00640 1 --2- 172.21.15.73:0/3645399630 >> v2:172.21.15.157:3300/0 conn(0x7f4e9c0ccd20 0x7f4e9c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea2200640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c1587d0 0x7f4e9c0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea2200640 1 -- 172.21.15.73:0/3645399630 >> v2:172.21.15.157:3300/0 conn(0x7f4e9c0ccd20 msgr2=0x7f4e9c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea2200640 1 --2- 172.21.15.73:0/3645399630 >> v2:172.21.15.157:3300/0 conn(0x7f4e9c0ccd20 0x7f4e9c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.760+0000 7f4ea2200640 1 -- 172.21.15.73:0/3645399630 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c06f800 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.761+0000 7f4ea2200640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c06d0c0 0x7f4e9c15e850 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c066850 tx=0x7f4e9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e0f0a89c6728c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.761+0000 7f4e9be00640 1 -- 172.21.15.73:0/3645399630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9402ca70 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.761+0000 7f4e9be00640 1 -- 172.21.15.73:0/3645399630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9402cbd0 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.761+0000 7f4e9be00640 1 -- 172.21.15.73:0/3645399630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940343f0 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 -- 172.21.15.73:0/3645399630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c06d0c0 msgr2=0x7f4e9c15e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c06d0c0 0x7f4e9c15e850 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c066850 tx=0x7f4e9402bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 -- 172.21.15.73:0/3645399630 shutdown_connections 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c06d0c0 0x7f4e9c15e850 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 --2- 172.21.15.73:0/3645399630 >> v2:172.21.15.157:3300/0 conn(0x7f4e9c0ccd20 0x7f4e9c06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 --2- 172.21.15.73:0/3645399630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c1587d0 0x7f4e9c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 -- 172.21.15.73:0/3645399630 >> 172.21.15.73:0/3645399630 conn(0x7f4e9c0ce8e0 msgr2=0x7f4e9c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 -- 172.21.15.73:0/3645399630 shutdown_connections 2024-04-05T22:27:32.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.762+0000 7f4ea4200640 1 -- 172.21.15.73:0/3645399630 wait complete. 2024-04-05T22:27:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.764+0000 7f4ea4200640 1 Processor -- start 2024-04-05T22:27:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.764+0000 7f4ea4200640 1 -- start start 2024-04-05T22:27:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c06d0c0 0x7f4e9c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9c1587d0 0x7f4e9c14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b26d0 con 0x7f4e9c0b3880 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c14cdb0 con 0x7f4e9c1587d0 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c14d5f0 con 0x7f4e9c06d0c0 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39104/0 (socket says 172.21.15.73:39104) 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea2200640 1 -- 172.21.15.73:0/315541101 learned_addr learned my addr 172.21.15.73:0/315541101 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.765+0000 7f4ea1800640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c06d0c0 0x7f4e9c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 -- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c06d0c0 msgr2=0x7f4e9c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c06d0c0 0x7f4e9c14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 -- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9c1587d0 msgr2=0x7f4e9c14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9c1587d0 0x7f4e9c14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 -- 172.21.15.73:0/315541101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009b30 con 0x7f4e9c0b3880 2024-04-05T22:27:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea2200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036040 tx=0x7f4e94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009510 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9403e450 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9403e800 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0b23f0 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.766+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c14d280 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.768+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c066af0 con 0x7f4e9c0b3880 2024-04-05T22:27:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.768+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f4e9403e960 con 0x7f4e9c0b3880 2024-04-05T22:27:32.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.769+0000 7f4e99600640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 0x7f4e7c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:32.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.769+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e940ac640 con 0x7f4e9c0b3880 2024-04-05T22:27:32.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.770+0000 7f4ea1800640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 0x7f4e7c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:32.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.771+0000 7f4ea1800640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 0x7f4e7c06d1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:32.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.774+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4e940777c0 con 0x7f4e9c0b3880 2024-04-05T22:27:32.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.968+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e9c14d760 con 0x7f4e9c0b3880 2024-04-05T22:27:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.971+0000 7f4e99600640 1 -- 172.21.15.73:0/315541101 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16072 (secure 0 0 0) 0x7f4e9407ade0 con 0x7f4e9c0b3880 2024-04-05T22:27:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:32.971 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":106,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:32.104121+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:32.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 msgr2=0x7f4e7c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 0x7f4e7c06d1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90008040 comp rx=0 tx=0).stop 2024-04-05T22:27:32.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 msgr2=0x7f4e9c15c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:32.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036040 tx=0x7f4e94004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 shutdown_connections 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e7c06ad20 0x7f4e7c06d1e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e9c0b3880 0x7f4e9c15c3a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9c1587d0 0x7f4e9c14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 --2- 172.21.15.73:0/315541101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e9c06d0c0 0x7f4e9c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 >> 172.21.15.73:0/315541101 conn(0x7f4e9c0ce8e0 msgr2=0x7f4e9c069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 shutdown_connections 2024-04-05T22:27:32.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:32.974+0000 7f4ea4200640 1 -- 172.21.15.73:0/315541101 wait complete. 2024-04-05T22:27:32.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:27:33.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.198+0000 7fb261400640 1 Processor -- start 2024-04-05T22:27:33.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.198+0000 7fb261400640 1 -- start start 2024-04-05T22:27:33.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb25c144420 0x7fb25c06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb25c070120 0x7fb25c0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c15e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c0706a0 con 0x7fb25c070120 2024-04-05T22:27:33.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c070980 con 0x7fb25c144420 2024-04-05T22:27:33.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb261400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c070c60 con 0x7fb25c0ce080 2024-04-05T22:27:33.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.199+0000 7fb253e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb25c070120 0x7fb25c0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:33.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb25c070120 0x7fb25c0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39118/0 (socket says 172.21.15.73:39118) 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 -- 172.21.15.73:0/266554553 learned_addr learned my addr 172.21.15.73:0/266554553 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb25be00640 1 --2- 172.21.15.73:0/266554553 >> v2:172.21.15.157:3300/0 conn(0x7fb25c144420 0x7fb25c06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb260a00640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c15e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 -- 172.21.15.73:0/266554553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 msgr2=0x7fb25c15e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c15e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 -- 172.21.15.73:0/266554553 >> v2:172.21.15.157:3300/0 conn(0x7fb25c144420 msgr2=0x7fb25c06fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 --2- 172.21.15.73:0/266554553 >> v2:172.21.15.157:3300/0 conn(0x7fb25c144420 0x7fb25c06fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.200+0000 7fb253e00640 1 -- 172.21.15.73:0/266554553 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c06b5a0 con 0x7fb25c070120 2024-04-05T22:27:33.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.202+0000 7fb253e00640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c0cdb40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0b3db0 tx=0x7fb24402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd89c0ee47c3b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:33.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.202+0000 7fb25b400640 1 -- 172.21.15.73:0/266554553 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2440317e0 con 0x7fb25c070120 2024-04-05T22:27:33.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.202+0000 7fb25b400640 1 -- 172.21.15.73:0/266554553 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244031940 con 0x7fb25c070120 2024-04-05T22:27:33.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.202+0000 7fb25b400640 1 -- 172.21.15.73:0/266554553 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244031cd0 con 0x7fb25c070120 2024-04-05T22:27:33.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 -- 172.21.15.73:0/266554553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 msgr2=0x7fb25c0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:33.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c0cdb40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0b3db0 tx=0x7fb24402b880 comp rx=0 tx=0).stop 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 -- 172.21.15.73:0/266554553 shutdown_connections 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c0cdb40 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 --2- 172.21.15.73:0/266554553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c15e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 --2- 172.21.15.73:0/266554553 >> v2:172.21.15.157:3300/0 conn(0x7fb25c144420 0x7fb25c06fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 -- 172.21.15.73:0/266554553 >> 172.21.15.73:0/266554553 conn(0x7fb25c0b07c0 msgr2=0x7fb25c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 -- 172.21.15.73:0/266554553 shutdown_connections 2024-04-05T22:27:33.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.203+0000 7fb261400640 1 -- 172.21.15.73:0/266554553 wait complete. 2024-04-05T22:27:33.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.205+0000 7fb261400640 1 Processor -- start 2024-04-05T22:27:33.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.205+0000 7fb261400640 1 -- start start 2024-04-05T22:27:33.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c144420 0x7fb25c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2720 con 0x7fb25c070120 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c1f2200 con 0x7fb25c144420 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb261400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b1b10 con 0x7fb25c0ce080 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb25be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb25be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39132/0 (socket says 172.21.15.73:39132) 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.206+0000 7fb25be00640 1 -- 172.21.15.73:0/619932161 learned_addr learned my addr 172.21.15.73:0/619932161 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 -- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 msgr2=0x7fb25c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 -- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c144420 msgr2=0x7fb25c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c144420 0x7fb25c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:33.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 -- 172.21.15.73:0/619932161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440094d0 con 0x7fb25c070120 2024-04-05T22:27:33.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb25be00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d620 tx=0x7fb24c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:33.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00b620 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c00b780 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c0b1df0 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00bb30 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.207+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c0b23b0 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.208+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0b2540 con 0x7fb25c070120 2024-04-05T22:27:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.209+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb24c00bc90 con 0x7fb25c070120 2024-04-05T22:27:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.210+0000 7fb258c00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 0x7fb23806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.210+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb24c089920 con 0x7fb25c070120 2024-04-05T22:27:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.211+0000 7fb253e00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 0x7fb23806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.212+0000 7fb253e00640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 0x7fb23806d1e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb244036000 tx=0x7fb244030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:33.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.215+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb24c054a80 con 0x7fb25c070120 2024-04-05T22:27:33.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:33.441+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb25c071430 con 0x7fb25c070120 2024-04-05T22:27:34.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.136+0000 7fb258c00640 1 -- 172.21.15.73:0/619932161 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb24c0580a0 con 0x7fb25c070120 2024-04-05T22:27:34.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 msgr2=0x7fb23806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 0x7fb23806d1e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb244036000 tx=0x7fb244030000 comp rx=0 tx=0).stop 2024-04-05T22:27:34.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 msgr2=0x7fb25c1f1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d620 tx=0x7fb24c00dae0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 shutdown_connections 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb23806ad20 0x7fb23806d1e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c144420 0x7fb25c06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.139+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c0ce080 0x7fb25c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.140+0000 7fb261400640 1 --2- 172.21.15.73:0/619932161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c070120 0x7fb25c1f1cc0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.140+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 >> 172.21.15.73:0/619932161 conn(0x7fb25c0b07c0 msgr2=0x7fb25c0ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.140+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 shutdown_connections 2024-04-05T22:27:34.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.140+0000 7fb261400640 1 -- 172.21.15.73:0/619932161 wait complete. 2024-04-05T22:27:34.158 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:27:34.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:27:34.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.375+0000 7f51f9800640 1 Processor -- start 2024-04-05T22:27:34.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.375+0000 7f51f9800640 1 -- start start 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f9800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51f415cf90 0x7f51f40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f9800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f40d1bf0 con 0x7f51f40cf900 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f9800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f40d1350 con 0x7f51f415d370 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f9800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f40cd6d0 con 0x7f51f415cf90 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51f415cf90 0x7f51f40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f8e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f51f40cf900 0x7f51f40cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53132/0 (socket says 172.21.15.73:53132) 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51f415cf90 0x7f51f40b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39146/0 (socket says 172.21.15.73:39146) 2024-04-05T22:27:34.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f8e00640 1 -- 172.21.15.73:0/3544970700 learned_addr learned my addr 172.21.15.73:0/3544970700 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:34.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f3e00640 1 -- 172.21.15.73:0/3544970700 >> v1:172.21.15.157:6789/0 conn(0x7f51f415d370 legacy=0x7f51f40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:34.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.377+0000 7f51f3e00640 1 -- 172.21.15.73:0/3544970700 >> v1:172.21.15.73:6789/0 conn(0x7f51f40cf900 legacy=0x7f51f40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.378+0000 7f51f3e00640 1 -- 172.21.15.73:0/3544970700 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40d1630 con 0x7f51f415cf90 2024-04-05T22:27:34.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.378+0000 7f51f2a00640 1 -- 172.21.15.73:0/3544970700 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187563788 0 0) 0x7f51f40d1bf0 con 0x7f51f40cf900 2024-04-05T22:27:34.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f3e00640 1 --2- 172.21.15.73:0/3544970700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f415cf90 0x7f51f40b3c10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f51e40097a0 tx=0x7f51e402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c55d5e27d451677 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:34.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f2a00640 1 -- 172.21.15.73:0/3544970700 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4030610 con 0x7f51f415cf90 2024-04-05T22:27:34.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f2a00640 1 -- 172.21.15.73:0/3544970700 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4030770 con 0x7f51f415cf90 2024-04-05T22:27:34.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f2a00640 1 -- 172.21.15.73:0/3544970700 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4030b00 con 0x7f51f415cf90 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f9800640 1 -- 172.21.15.73:0/3544970700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f415cf90 msgr2=0x7f51f40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.379+0000 7f51f9800640 1 --2- 172.21.15.73:0/3544970700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f415cf90 0x7f51f40b3c10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f51e40097a0 tx=0x7f51e402b960 comp rx=0 tx=0).stop 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.380+0000 7f51f9800640 1 -- 172.21.15.73:0/3544970700 shutdown_connections 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.380+0000 7f51f9800640 1 --2- 172.21.15.73:0/3544970700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f415cf90 0x7f51f40b3c10 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.380+0000 7f51f9800640 1 -- 172.21.15.73:0/3544970700 >> 172.21.15.73:0/3544970700 conn(0x7f51f40d2ac0 msgr2=0x7f51f40d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.380+0000 7f51f9800640 1 -- 172.21.15.73:0/3544970700 shutdown_connections 2024-04-05T22:27:34.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.380+0000 7f51f9800640 1 -- 172.21.15.73:0/3544970700 wait complete. 2024-04-05T22:27:34.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.381+0000 7f51f9800640 1 Processor -- start 2024-04-05T22:27:34.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.381+0000 7f51f9800640 1 -- start start 2024-04-05T22:27:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51f415cf90 0x7f51f40cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51f415d370 0x7f51f406ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4152590 con 0x7f51f40cf900 2024-04-05T22:27:34.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4153e00 con 0x7f51f415d370 2024-04-05T22:27:34.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f9800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41540e0 con 0x7f51f415cf90 2024-04-05T22:27:34.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:34.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39150/0 (socket says 172.21.15.73:39150) 2024-04-05T22:27:34.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.382+0000 7f51f3e00640 1 -- 172.21.15.73:0/3901361574 learned_addr learned my addr 172.21.15.73:0/3901361574 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 -- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51f415cf90 msgr2=0x7f51f40cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51f415cf90 0x7f51f40cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 -- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51f415d370 msgr2=0x7f51f406ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51f415d370 0x7f51f406ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 -- 172.21.15.73:0/3901361574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009400 con 0x7f51f40cf900 2024-04-05T22:27:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f3e00640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098c0 tx=0x7f51e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e403e040 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4039c90 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e40443f0 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.383+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f41543c0 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.384+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f4154980 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.385+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4067250 con 0x7f51f40cf900 2024-04-05T22:27:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.385+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f51e4043390 con 0x7f51f40cf900 2024-04-05T22:27:34.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.386+0000 7f51ebe00640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 0x7f51d006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:34.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.386+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51e40ab9c0 con 0x7f51f40cf900 2024-04-05T22:27:34.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.387+0000 7f51f3400640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 0x7f51d006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:34.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.388+0000 7f51f3400640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 0x7f51d006d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f51f406a1e0 tx=0x7f51dc009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.390+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f51e4076960 con 0x7f51f40cf900 2024-04-05T22:27:34.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:34.603+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51f404e160 con 0x7f51f40cf900 2024-04-05T22:27:35.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.140+0000 7f51ebe00640 1 -- 172.21.15.73:0/3901361574 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51e40340c0 con 0x7f51f40cf900 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.143+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 msgr2=0x7f51d006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.143+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 0x7f51d006d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f51f406a1e0 tx=0x7f51dc009230 comp rx=0 tx=0).stop 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.143+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 msgr2=0x7f51f4152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.143+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098c0 tx=0x7f51e4004730 comp rx=0 tx=0).stop 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 shutdown_connections 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f51d006ad20 0x7f51d006d1e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51f415d370 0x7f51f406ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51f415cf90 0x7f51f40cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 --2- 172.21.15.73:0/3901361574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51f40cf900 0x7f51f4152050 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 >> 172.21.15.73:0/3901361574 conn(0x7f51f40d2ac0 msgr2=0x7f51f40ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:35.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 shutdown_connections 2024-04-05T22:27:35.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.144+0000 7f51f9800640 1 -- 172.21.15.73:0/3901361574 wait complete. 2024-04-05T22:27:35.165 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:27:35.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:35.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.372+0000 7f3f8ec00640 1 Processor -- start 2024-04-05T22:27:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.373+0000 7f3f8ec00640 1 -- start start 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.374+0000 7f3f8ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f8813fa60 0x7f3f88142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.374+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 0x7f3f881429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.374+0000 7f3f8ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f88143330 con 0x7f3f8813fe40 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.374+0000 7f3f8ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8806d3a0 con 0x7f3f8813fa60 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.374+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88143050 con 0x7f3f8806fe90 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f8ce00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3f8813fe40 0x7f3f8806f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53146/0 (socket says 172.21.15.73:53146) 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f8ce00640 1 -- 172.21.15.73:0/3783037987 learned_addr learned my addr 172.21.15.73:0/3783037987 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f8e200640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 0x7f3f881429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527249117 0 0) 0x7f3f88143330 con 0x7f3f8813fe40 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f8d800640 1 --2- 172.21.15.73:0/3783037987 >> v2:172.21.15.157:3300/0 conn(0x7f3f8813fa60 0x7f3f88142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f70003520 con 0x7f3f8813fe40 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2002993086 0 0) 0x7f3f70003520 con 0x7f3f8813fe40 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 msgr2=0x7f3f881429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 0x7f3f881429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 >> v2:172.21.15.157:3300/0 conn(0x7f3f8813fa60 msgr2=0x7f3f88142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 --2- 172.21.15.73:0/3783037987 >> v2:172.21.15.157:3300/0 conn(0x7f3f8813fa60 0x7f3f88142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880ac970 con 0x7f3f8813fe40 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3f74001670 con 0x7f3f8813fe40 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 >> v1:172.21.15.73:6789/0 conn(0x7f3f8813fe40 legacy=0x7f3f8806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f70003970 0x7f3f70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.375+0000 7f3f87e00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f87e00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 0x7f3f70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8806d3a0 con 0x7f3f700047c0 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88143050 con 0x7f3f70003970 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7000ba60 con 0x7f3f8806fe90 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 -- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 msgr2=0x7f3f70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8806fe90 0x7f3f70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 -- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f70003970 msgr2=0x7f3f70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:35.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f70003970 0x7f3f70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.376+0000 7f3f8ce00640 1 -- 172.21.15.73:0/3783037987 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70003520 con 0x7f3f700047c0 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.377+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f70004c40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3f881402d0 tx=0x7f3f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32860688b0167414 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.377+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005ad0 con 0x7f3f700047c0 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.377+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74005c30 con 0x7f3f700047c0 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.377+0000 7f3f87e00640 1 -- 172.21.15.73:0/3783037987 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7402c870 con 0x7f3f700047c0 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3783037987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 msgr2=0x7f3f70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3783037987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f70004c40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3f881402d0 tx=0x7f3f700057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8e200640 1 -- 172.21.15.73:0/3783037987 reap_dead start 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3783037987 shutdown_connections 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3783037987 >> 172.21.15.73:0/3783037987 conn(0x7f3f880b06c0 msgr2=0x7f3f880b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3783037987 shutdown_connections 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.378+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3783037987 wait complete. 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.380+0000 7f3f8ec00640 1 Processor -- start 2024-04-05T22:27:35.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.380+0000 7f3f8ec00640 1 -- start start 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f881571e0 0x7f3f8815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8815f610 0x7f3f88159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8812f1f0 con 0x7f3f700047c0 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815e4c0 con 0x7f3f881571e0 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8812f400 con 0x7f3f8815f610 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39162/0 (socket says 172.21.15.73:39162) 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 -- 172.21.15.73:0/3814723291 learned_addr learned my addr 172.21.15.73:0/3814723291 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 -- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8815f610 msgr2=0x7f3f88159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8815f610 0x7f3f88159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 -- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f881571e0 msgr2=0x7f3f8815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f881571e0 0x7f3f8815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.381+0000 7f3f8d800640 1 -- 172.21.15.73:0/3814723291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8812f6e0 con 0x7f3f700047c0 2024-04-05T22:27:35.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f8d800640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d800 tx=0x7f3f7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00b6c0 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00b820 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8812f9c0 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00bbd0 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.382+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8813f5b0 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.383+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8804e160 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.384+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f3f7c00bd30 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.385+0000 7f3f85600640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 0x7f3f6806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.385+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f7c089ad0 con 0x7f3f700047c0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.386+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 0x7f3f6806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.387+0000 7f3f8ce00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 0x7f3f6806d210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f74034000 tx=0x7f3f7402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.389+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3f7c054b00 con 0x7f3f700047c0 2024-04-05T22:27:35.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.618+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f881605b0 con 0x7f3f700047c0 2024-04-05T22:27:35.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.619+0000 7f3f85600640 1 -- 172.21.15.73:0/3814723291 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f3f7c058120 con 0x7f3f700047c0 2024-04-05T22:27:35.619 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:35.619 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:27:35.619 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 63 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 msgr2=0x7f3f6806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 0x7f3f6806d210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f74034000 tx=0x7f3f7402e000 comp rx=0 tx=0).stop 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 msgr2=0x7f3f8813f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d800 tx=0x7f3f7c00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 shutdown_connections 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3f6806ad50 0x7f3f6806d210 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f8815f610 0x7f3f88159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f881571e0 0x7f3f8815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 --2- 172.21.15.73:0/3814723291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f700047c0 0x7f3f8813f070 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 >> 172.21.15.73:0/3814723291 conn(0x7f3f880b06c0 msgr2=0x7f3f880b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:35.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 shutdown_connections 2024-04-05T22:27:35.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.622+0000 7f3f8ec00640 1 -- 172.21.15.73:0/3814723291 wait complete. 2024-04-05T22:27:35.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:27:35.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.851+0000 7f65b3800640 1 Processor -- start 2024-04-05T22:27:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.851+0000 7f65b3800640 1 -- start start 2024-04-05T22:27:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b3800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65ac0cc170 0x7f65ac0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b3800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65ac070c00 0x7f65ac15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b3800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac06cf30 con 0x7f65ac15de40 2024-04-05T22:27:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b3800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac0710b0 con 0x7f65ac070c00 2024-04-05T22:27:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b3800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac0cc550 con 0x7f65ac0cc170 2024-04-05T22:27:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b1800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f65ac15de40 0x7f65ac0b3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49610/0 (socket says 172.21.15.73:49610) 2024-04-05T22:27:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65b1800640 1 -- 172.21.15.73:0/2428057224 learned_addr learned my addr 172.21.15.73:0/2428057224 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.853+0000 7f65a3e00640 1 --2- 172.21.15.73:0/2428057224 >> v2:172.21.15.73:3300/0 conn(0x7f65ac070c00 0x7f65ac15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65b0e00640 1 --2- 172.21.15.73:0/2428057224 >> v2:172.21.15.157:3300/0 conn(0x7f65ac0cc170 0x7f65ac0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65abe00640 1 -- 172.21.15.73:0/2428057224 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137057295 0 0) 0x7f65ac06cf30 con 0x7f65ac15de40 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65abe00640 1 -- 172.21.15.73:0/2428057224 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6590005730 con 0x7f65ac15de40 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65a3e00640 1 -- 172.21.15.73:0/2428057224 >> v2:172.21.15.157:3300/0 conn(0x7f65ac0cc170 msgr2=0x7f65ac0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65a3e00640 1 --2- 172.21.15.73:0/2428057224 >> v2:172.21.15.157:3300/0 conn(0x7f65ac0cc170 0x7f65ac0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65a3e00640 1 -- 172.21.15.73:0/2428057224 >> v1:172.21.15.73:6789/0 conn(0x7f65ac15de40 legacy=0x7f65ac0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.854+0000 7f65a3e00640 1 -- 172.21.15.73:0/2428057224 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac06c600 con 0x7f65ac070c00 2024-04-05T22:27:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.855+0000 7f65a3e00640 1 --2- 172.21.15.73:0/2428057224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac15d900 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0667b0 tx=0x7f659402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a68659c0e2d966d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.855+0000 7f65abe00640 1 -- 172.21.15.73:0/2428057224 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594031830 con 0x7f65ac070c00 2024-04-05T22:27:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.855+0000 7f65abe00640 1 -- 172.21.15.73:0/2428057224 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594031990 con 0x7f65ac070c00 2024-04-05T22:27:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.855+0000 7f65abe00640 1 -- 172.21.15.73:0/2428057224 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594031d20 con 0x7f65ac070c00 2024-04-05T22:27:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 -- 172.21.15.73:0/2428057224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 msgr2=0x7f65ac15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 --2- 172.21.15.73:0/2428057224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac15d900 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0667b0 tx=0x7f659402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 -- 172.21.15.73:0/2428057224 shutdown_connections 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 --2- 172.21.15.73:0/2428057224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac15d900 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 --2- 172.21.15.73:0/2428057224 >> v2:172.21.15.157:3300/0 conn(0x7f65ac0cc170 0x7f65ac0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 -- 172.21.15.73:0/2428057224 >> 172.21.15.73:0/2428057224 conn(0x7f65ac0b16d0 msgr2=0x7f65ac0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 -- 172.21.15.73:0/2428057224 shutdown_connections 2024-04-05T22:27:35.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.856+0000 7f65b3800640 1 -- 172.21.15.73:0/2428057224 wait complete. 2024-04-05T22:27:35.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.858+0000 7f65b3800640 1 Processor -- start 2024-04-05T22:27:35.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.858+0000 7f65b3800640 1 -- start start 2024-04-05T22:27:35.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ac0cc170 0x7f65ac0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 0x7f65ac1464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0ce810 con 0x7f65ac070c00 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0cdd40 con 0x7f65ac0cc170 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac1469e0 con 0x7f65ac15de40 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52036/0 (socket says 172.21.15.73:52036) 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65b0e00640 1 -- 172.21.15.73:0/1057517041 learned_addr learned my addr 172.21.15.73:0/1057517041 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.859+0000 7f65a3e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ac0cc170 0x7f65ac0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b1800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 0x7f65ac1464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 -- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 msgr2=0x7f65ac1464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 0x7f65ac1464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 -- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ac0cc170 msgr2=0x7f65ac0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ac0cc170 0x7f65ac0cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 -- 172.21.15.73:0/1057517041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594009400 con 0x7f65ac070c00 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b1800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 0x7f65ac1464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:35.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b0e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d830 tx=0x7f659c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00b6c0 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00b820 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00bbb0 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac0cdf70 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.860+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac0ce530 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.861+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac068160 con 0x7f65ac070c00 2024-04-05T22:27:35.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.862+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f659c01e460 con 0x7f65ac070c00 2024-04-05T22:27:35.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.863+0000 7f65a9600640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 0x7f658806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:35.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.863+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f659c089d00 con 0x7f65ac070c00 2024-04-05T22:27:35.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.864+0000 7f65a3e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 0x7f658806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:35.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.865+0000 7f65a3e00640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 0x7f658806d1e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6594036000 tx=0x7f659403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:35.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:35.868+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f659c054d30 con 0x7f65ac070c00 2024-04-05T22:27:36.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.046+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65ac0af6e0 con 0x7f65ac070c00 2024-04-05T22:27:36.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.049+0000 7f65a9600640 1 -- 172.21.15.73:0/1057517041 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f659c058350 con 0x7f65ac070c00 2024-04-05T22:27:36.049 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:36.049 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":107,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:27:33.109858+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 msgr2=0x7f658806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 0x7f658806d1e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6594036000 tx=0x7f659403c040 comp rx=0 tx=0).stop 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 msgr2=0x7f65ac151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d830 tx=0x7f659c00dcf0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 shutdown_connections 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f658806ad20 0x7f658806d1e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ac15de40 0x7f65ac1464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ac0cc170 0x7f65ac0cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 --2- 172.21.15.73:0/1057517041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac070c00 0x7f65ac151bd0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 >> 172.21.15.73:0/1057517041 conn(0x7f65ac0b16d0 msgr2=0x7f65ac0afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:36.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 shutdown_connections 2024-04-05T22:27:36.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.052+0000 7f65b3800640 1 -- 172.21.15.73:0/1057517041 wait complete. 2024-04-05T22:27:36.070 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:27:36.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:27:36.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.277+0000 7f6d62c00640 1 Processor -- start 2024-04-05T22:27:36.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.277+0000 7f6d62c00640 1 -- start start 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.278+0000 7f6d62c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 0x7f6d5c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.278+0000 7f6d62c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6d5c15e850 0x7f6d5c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d62c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c06f5b0 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d62c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c15f240 con 0x7f6d5c15e850 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d62c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c070410 con 0x7f6d5c1587d0 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d5b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d5c0ccd20 0x7f6d5c06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49618/0 (socket says 172.21.15.73:49618) 2024-04-05T22:27:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d5b400640 1 -- 172.21.15.73:0/268155455 learned_addr learned my addr 172.21.15.73:0/268155455 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d5be00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 0x7f6d5c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d60c00640 1 --2- 172.21.15.73:0/268155455 >> v2:172.21.15.157:3300/0 conn(0x7f6d5c15e850 0x7f6d5c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385738769 0 0) 0x7f6d5c06f5b0 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.279+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d40003520 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832493873 0 0) 0x7f6d40003520 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 msgr2=0x7f6d5c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 0x7f6d5c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 >> v2:172.21.15.157:3300/0 conn(0x7f6d5c15e850 msgr2=0x7f6d5c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 --2- 172.21.15.73:0/268155455 >> v2:172.21.15.157:3300/0 conn(0x7f6d5c15e850 0x7f6d5c15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c158bb0 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6d44001670 con 0x7f6d5c0ccd20 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 >> v1:172.21.15.73:6789/0 conn(0x7f6d5c0ccd20 legacy=0x7f6d5c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 0x7f6d40003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d40003ef0 0x7f6d400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d40004d00 0x7f6d40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c15f240 con 0x7f6d40003ef0 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c070410 con 0x7f6d40004d00 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4000fd90 con 0x7f6d5c1587d0 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5b400640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d40003ef0 0x7f6d400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5b400640 1 -- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 msgr2=0x7f6d40003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.280+0000 7f6d5b400640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d5c1587d0 0x7f6d40003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.281+0000 7f6d5b400640 1 -- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d40004d00 msgr2=0x7f6d40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.281+0000 7f6d5b400640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d40004d00 0x7f6d40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.281+0000 7f6d5b400640 1 -- 172.21.15.73:0/268155455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40003520 con 0x7f6d40003ef0 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d5b400640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d40003ef0 0x7f6d400047c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c210 tx=0x7f6d400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb96c3ebbfd9e85d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44005ad0 con 0x7f6d40003ef0 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d44005c30 con 0x7f6d40003ef0 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d5aa00640 1 -- 172.21.15.73:0/268155455 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4402caf0 con 0x7f6d40003ef0 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d62c00640 1 -- 172.21.15.73:0/268155455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d40003ef0 msgr2=0x7f6d400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d62c00640 1 --2- 172.21.15.73:0/268155455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d40003ef0 0x7f6d400047c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c210 tx=0x7f6d400057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.282+0000 7f6d60c00640 1 -- 172.21.15.73:0/268155455 reap_dead start 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.283+0000 7f6d62c00640 1 -- 172.21.15.73:0/268155455 shutdown_connections 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.283+0000 7f6d62c00640 1 -- 172.21.15.73:0/268155455 >> 172.21.15.73:0/268155455 conn(0x7f6d5c0ce8e0 msgr2=0x7f6d5c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.283+0000 7f6d62c00640 1 -- 172.21.15.73:0/268155455 shutdown_connections 2024-04-05T22:27:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.283+0000 7f6d62c00640 1 -- 172.21.15.73:0/268155455 wait complete. 2024-04-05T22:27:36.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.284+0000 7f6d62c00640 1 Processor -- start 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.284+0000 7f6d62c00640 1 -- start start 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d40003ef0 0x7f6d5c0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d5c1eedb0 0x7f6d5c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1ef4f0 con 0x7f6d5c15c390 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c15cd50 con 0x7f6d5c1eedb0 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.285+0000 7f6d62c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1f1720 con 0x7f6d40003ef0 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52056/0 (socket says 172.21.15.73:52056) 2024-04-05T22:27:36.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 -- 172.21.15.73:0/1352229407 learned_addr learned my addr 172.21.15.73:0/1352229407 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:36.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 -- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d40003ef0 msgr2=0x7f6d5c0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d40003ef0 0x7f6d5c0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d5b400640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d5c1eedb0 0x7f6d5c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 -- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d5c1eedb0 msgr2=0x7f6d5c152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d5c1eedb0 0x7f6d5c152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 -- 172.21.15.73:0/1352229407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c1ef230 con 0x7f6d5c15c390 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d60c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c15ec30 tx=0x7f6d5400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.286+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54010040 con 0x7f6d5c15c390 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.287+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400ba80 con 0x7f6d5c15c390 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.287+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400be10 con 0x7f6d5c15c390 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.287+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c1531d0 con 0x7f6d5c15c390 2024-04-05T22:27:36.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.287+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c0af140 con 0x7f6d5c15c390 2024-04-05T22:27:36.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.288+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c066af0 con 0x7f6d5c15c390 2024-04-05T22:27:36.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.289+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f6d5401d400 con 0x7f6d5c15c390 2024-04-05T22:27:36.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.290+0000 7f6d53e00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 0x7f6d3806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.290+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d54089ca0 con 0x7f6d5c15c390 2024-04-05T22:27:36.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.290+0000 7f6d5be00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 0x7f6d3806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.292+0000 7f6d5be00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 0x7f6d3806d210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009970 tx=0x7f6d4c009d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.294+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d540586e0 con 0x7f6d5c15c390 2024-04-05T22:27:36.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.486+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d5c06fdc0 con 0x7f6d3806ad50 2024-04-05T22:27:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.514+0000 7f6d53e00640 1 -- 172.21.15.73:0/1352229407 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287348 (secure 0 0 0) 0x7f6d4c00e6d0 con 0x7f6d3806ad50 2024-04-05T22:27:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:36.519 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.520+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 msgr2=0x7f6d3806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.520+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 0x7f6d3806d210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009970 tx=0x7f6d4c009d70 comp rx=0 tx=0).stop 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.520+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 msgr2=0x7f6d5c15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.520+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c15ec30 tx=0x7f6d5400b480 comp rx=0 tx=0).stop 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 shutdown_connections 2024-04-05T22:27:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d3806ad50 0x7f6d3806d210 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d5c15c390 0x7f6d5c15c810 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d5c1eedb0 0x7f6d5c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 --2- 172.21.15.73:0/1352229407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d40003ef0 0x7f6d5c0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 >> 172.21.15.73:0/1352229407 conn(0x7f6d5c0ce8e0 msgr2=0x7f6d5c15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 shutdown_connections 2024-04-05T22:27:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.521+0000 7f6d62c00640 1 -- 172.21.15.73:0/1352229407 wait complete. 2024-04-05T22:27:36.540 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-05T22:27:35.521324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260304,"kb_used_data":11356,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714416,"statfs":{"total":773094113280,"available":772827561984,"internally_reserved":0,"allocated":11628544,"data_stored":5744667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.913060"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158217+0000","last_change":"2024-04-05T22:27:32.157635+0000","last_active":"2024-04-05T22:27:32.158217+0000","last_peered":"2024-04-05T22:27:32.158217+0000","last_clean":"2024-04-05T22:27:32.158217+0000","last_became_active":"2024-04-05T22:27:32.157328+0000","last_became_peered":"2024-04-05T22:27:32.157328+0000","last_unstale":"2024-04-05T22:27:32.158217+0000","last_undegraded":"2024-04-05T22:27:32.158217+0000","last_fullsized":"2024-04-05T22:27:32.158217+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:28:52.775014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.135700+0000","last_change":"2024-04-05T22:27:32.128480+0000","last_active":"2024-04-05T22:27:32.135700+0000","last_peered":"2024-04-05T22:27:32.135700+0000","last_clean":"2024-04-05T22:27:32.135700+0000","last_became_active":"2024-04-05T22:27:32.127744+0000","last_became_peered":"2024-04-05T22:27:32.127744+0000","last_unstale":"2024-04-05T22:27:32.135700+0000","last_undegraded":"2024-04-05T22:27:32.135700+0000","last_fullsized":"2024-04-05T22:27:32.135700+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:21.246110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159011+0000","last_change":"2024-04-05T22:27:32.154608+0000","last_active":"2024-04-05T22:27:32.159011+0000","last_peered":"2024-04-05T22:27:32.159011+0000","last_clean":"2024-04-05T22:27:32.159011+0000","last_became_active":"2024-04-05T22:27:32.152098+0000","last_became_peered":"2024-04-05T22:27:32.152098+0000","last_unstale":"2024-04-05T22:27:32.159011+0000","last_undegraded":"2024-04-05T22:27:32.159011+0000","last_fullsized":"2024-04-05T22:27:32.159011+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:36.855438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146212+0000","last_change":"2024-04-05T22:27:32.140555+0000","last_active":"2024-04-05T22:27:32.146212+0000","last_peered":"2024-04-05T22:27:32.146212+0000","last_clean":"2024-04-05T22:27:32.146212+0000","last_became_active":"2024-04-05T22:27:32.139526+0000","last_became_peered":"2024-04-05T22:27:32.139526+0000","last_unstale":"2024-04-05T22:27:32.146212+0000","last_undegraded":"2024-04-05T22:27:32.146212+0000","last_fullsized":"2024-04-05T22:27:32.146212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:13.650988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144201+0000","last_change":"2024-04-05T22:27:32.136309+0000","last_active":"2024-04-05T22:27:32.144201+0000","last_peered":"2024-04-05T22:27:32.144201+0000","last_clean":"2024-04-05T22:27:32.144201+0000","last_became_active":"2024-04-05T22:27:32.135855+0000","last_became_peered":"2024-04-05T22:27:32.135855+0000","last_unstale":"2024-04-05T22:27:32.144201+0000","last_undegraded":"2024-04-05T22:27:32.144201+0000","last_fullsized":"2024-04-05T22:27:32.144201+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:44.881691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140088+0000","last_change":"2024-04-05T22:27:32.132707+0000","last_active":"2024-04-05T22:27:32.140088+0000","last_peered":"2024-04-05T22:27:32.140088+0000","last_clean":"2024-04-05T22:27:32.140088+0000","last_became_active":"2024-04-05T22:27:32.132464+0000","last_became_peered":"2024-04-05T22:27:32.132464+0000","last_unstale":"2024-04-05T22:27:32.140088+0000","last_undegraded":"2024-04-05T22:27:32.140088+0000","last_fullsized":"2024-04-05T22:27:32.140088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:20:01.110274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.152865+0000","last_change":"2024-04-05T22:27:32.149061+0000","last_active":"2024-04-05T22:27:32.152865+0000","last_peered":"2024-04-05T22:27:32.152865+0000","last_clean":"2024-04-05T22:27:32.152865+0000","last_became_active":"2024-04-05T22:27:32.148768+0000","last_became_peered":"2024-04-05T22:27:32.148768+0000","last_unstale":"2024-04-05T22:27:32.152865+0000","last_undegraded":"2024-04-05T22:27:32.152865+0000","last_fullsized":"2024-04-05T22:27:32.152865+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:57.461648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157998+0000","last_change":"2024-04-05T22:27:32.152096+0000","last_active":"2024-04-05T22:27:32.157998+0000","last_peered":"2024-04-05T22:27:32.157998+0000","last_clean":"2024-04-05T22:27:32.157998+0000","last_became_active":"2024-04-05T22:27:32.151133+0000","last_became_peered":"2024-04-05T22:27:32.151133+0000","last_unstale":"2024-04-05T22:27:32.157998+0000","last_undegraded":"2024-04-05T22:27:32.157998+0000","last_fullsized":"2024-04-05T22:27:32.157998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:35:08.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138637+0000","last_change":"2024-04-05T22:27:32.133509+0000","last_active":"2024-04-05T22:27:32.138637+0000","last_peered":"2024-04-05T22:27:32.138637+0000","last_clean":"2024-04-05T22:27:32.138637+0000","last_became_active":"2024-04-05T22:27:32.131818+0000","last_became_peered":"2024-04-05T22:27:32.131818+0000","last_unstale":"2024-04-05T22:27:32.138637+0000","last_undegraded":"2024-04-05T22:27:32.138637+0000","last_fullsized":"2024-04-05T22:27:32.138637+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:59:43.241703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.136324+0000","last_change":"2024-04-05T22:27:32.128788+0000","last_active":"2024-04-05T22:27:32.136324+0000","last_peered":"2024-04-05T22:27:32.136324+0000","last_clean":"2024-04-05T22:27:32.136324+0000","last_became_active":"2024-04-05T22:27:32.128100+0000","last_became_peered":"2024-04-05T22:27:32.128100+0000","last_unstale":"2024-04-05T22:27:32.136324+0000","last_undegraded":"2024-04-05T22:27:32.136324+0000","last_fullsized":"2024-04-05T22:27:32.136324+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:37:45.504548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158348+0000","last_change":"2024-04-05T22:27:32.150306+0000","last_active":"2024-04-05T22:27:32.158348+0000","last_peered":"2024-04-05T22:27:32.158348+0000","last_clean":"2024-04-05T22:27:32.158348+0000","last_became_active":"2024-04-05T22:27:32.150037+0000","last_became_peered":"2024-04-05T22:27:32.150037+0000","last_unstale":"2024-04-05T22:27:32.158348+0000","last_undegraded":"2024-04-05T22:27:32.158348+0000","last_fullsized":"2024-04-05T22:27:32.158348+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:26.515897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.141800+0000","last_change":"2024-04-05T22:27:32.130409+0000","last_active":"2024-04-05T22:27:32.141800+0000","last_peered":"2024-04-05T22:27:32.141800+0000","last_clean":"2024-04-05T22:27:32.141800+0000","last_became_active":"2024-04-05T22:27:32.129153+0000","last_became_peered":"2024-04-05T22:27:32.129153+0000","last_unstale":"2024-04-05T22:27:32.141800+0000","last_undegraded":"2024-04-05T22:27:32.141800+0000","last_fullsized":"2024-04-05T22:27:32.141800+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:16:42.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157102+0000","last_change":"2024-04-05T22:27:32.150741+0000","last_active":"2024-04-05T22:27:32.157102+0000","last_peered":"2024-04-05T22:27:32.157102+0000","last_clean":"2024-04-05T22:27:32.157102+0000","last_became_active":"2024-04-05T22:27:32.150463+0000","last_became_peered":"2024-04-05T22:27:32.150463+0000","last_unstale":"2024-04-05T22:27:32.157102+0000","last_undegraded":"2024-04-05T22:27:32.157102+0000","last_fullsized":"2024-04-05T22:27:32.157102+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:37:04.887872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157867+0000","last_change":"2024-04-05T22:27:32.150532+0000","last_active":"2024-04-05T22:27:32.157867+0000","last_peered":"2024-04-05T22:27:32.157867+0000","last_clean":"2024-04-05T22:27:32.157867+0000","last_became_active":"2024-04-05T22:27:32.150193+0000","last_became_peered":"2024-04-05T22:27:32.150193+0000","last_unstale":"2024-04-05T22:27:32.157867+0000","last_undegraded":"2024-04-05T22:27:32.157867+0000","last_fullsized":"2024-04-05T22:27:32.157867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140447+0000","last_change":"2024-04-05T22:27:32.134048+0000","last_active":"2024-04-05T22:27:32.140447+0000","last_peered":"2024-04-05T22:27:32.140447+0000","last_clean":"2024-04-05T22:27:32.140447+0000","last_became_active":"2024-04-05T22:27:32.133797+0000","last_became_peered":"2024-04-05T22:27:32.133797+0000","last_unstale":"2024-04-05T22:27:32.140447+0000","last_undegraded":"2024-04-05T22:27:32.140447+0000","last_fullsized":"2024-04-05T22:27:32.140447+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:57.461648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157944+0000","last_change":"2024-04-05T22:27:32.153109+0000","last_active":"2024-04-05T22:27:32.157944+0000","last_peered":"2024-04-05T22:27:32.157944+0000","last_clean":"2024-04-05T22:27:32.157944+0000","last_became_active":"2024-04-05T22:27:32.152412+0000","last_became_peered":"2024-04-05T22:27:32.152412+0000","last_unstale":"2024-04-05T22:27:32.157944+0000","last_undegraded":"2024-04-05T22:27:32.157944+0000","last_fullsized":"2024-04-05T22:27:32.157944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:25.167803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159705+0000","last_change":"2024-04-05T22:27:32.158134+0000","last_active":"2024-04-05T22:27:32.159705+0000","last_peered":"2024-04-05T22:27:32.159705+0000","last_clean":"2024-04-05T22:27:32.159705+0000","last_became_active":"2024-04-05T22:27:32.157852+0000","last_became_peered":"2024-04-05T22:27:32.157852+0000","last_unstale":"2024-04-05T22:27:32.159705+0000","last_undegraded":"2024-04-05T22:27:32.159705+0000","last_fullsized":"2024-04-05T22:27:32.159705+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:25.013402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.156419+0000","last_change":"2024-04-05T22:27:32.149242+0000","last_active":"2024-04-05T22:27:32.156419+0000","last_peered":"2024-04-05T22:27:32.156419+0000","last_clean":"2024-04-05T22:27:32.156419+0000","last_became_active":"2024-04-05T22:27:32.148981+0000","last_became_peered":"2024-04-05T22:27:32.148981+0000","last_unstale":"2024-04-05T22:27:32.156419+0000","last_undegraded":"2024-04-05T22:27:32.156419+0000","last_fullsized":"2024-04-05T22:27:32.156419+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:47:36.810811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159053+0000","last_change":"2024-04-05T22:27:32.155369+0000","last_active":"2024-04-05T22:27:32.159053+0000","last_peered":"2024-04-05T22:27:32.159053+0000","last_clean":"2024-04-05T22:27:32.159053+0000","last_became_active":"2024-04-05T22:27:32.154861+0000","last_became_peered":"2024-04-05T22:27:32.154861+0000","last_unstale":"2024-04-05T22:27:32.159053+0000","last_undegraded":"2024-04-05T22:27:32.159053+0000","last_fullsized":"2024-04-05T22:27:32.159053+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:17:43.310001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159061+0000","last_change":"2024-04-05T22:27:32.153912+0000","last_active":"2024-04-05T22:27:32.159061+0000","last_peered":"2024-04-05T22:27:32.159061+0000","last_clean":"2024-04-05T22:27:32.159061+0000","last_became_active":"2024-04-05T22:27:32.153633+0000","last_became_peered":"2024-04-05T22:27:32.153633+0000","last_unstale":"2024-04-05T22:27:32.159061+0000","last_undegraded":"2024-04-05T22:27:32.159061+0000","last_fullsized":"2024-04-05T22:27:32.159061+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:35:08.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144125+0000","last_change":"2024-04-05T22:27:32.134200+0000","last_active":"2024-04-05T22:27:32.144125+0000","last_peered":"2024-04-05T22:27:32.144125+0000","last_clean":"2024-04-05T22:27:32.144125+0000","last_became_active":"2024-04-05T22:27:32.132856+0000","last_became_peered":"2024-04-05T22:27:32.132856+0000","last_unstale":"2024-04-05T22:27:32.144125+0000","last_undegraded":"2024-04-05T22:27:32.144125+0000","last_fullsized":"2024-04-05T22:27:32.144125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:35:35.331299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.153023+0000","last_change":"2024-04-05T22:27:32.148948+0000","last_active":"2024-04-05T22:27:32.153023+0000","last_peered":"2024-04-05T22:27:32.153023+0000","last_clean":"2024-04-05T22:27:32.153023+0000","last_became_active":"2024-04-05T22:27:32.148720+0000","last_became_peered":"2024-04-05T22:27:32.148720+0000","last_unstale":"2024-04-05T22:27:32.153023+0000","last_undegraded":"2024-04-05T22:27:32.153023+0000","last_fullsized":"2024-04-05T22:27:32.153023+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:22:04.714703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159026+0000","last_change":"2024-04-05T22:27:32.153105+0000","last_active":"2024-04-05T22:27:32.159026+0000","last_peered":"2024-04-05T22:27:32.159026+0000","last_clean":"2024-04-05T22:27:32.159026+0000","last_became_active":"2024-04-05T22:27:32.152834+0000","last_became_peered":"2024-04-05T22:27:32.152834+0000","last_unstale":"2024-04-05T22:27:32.159026+0000","last_undegraded":"2024-04-05T22:27:32.159026+0000","last_fullsized":"2024-04-05T22:27:32.159026+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:38:58.315925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157359+0000","last_change":"2024-04-05T22:27:32.151986+0000","last_active":"2024-04-05T22:27:32.157359+0000","last_peered":"2024-04-05T22:27:32.157359+0000","last_clean":"2024-04-05T22:27:32.157359+0000","last_became_active":"2024-04-05T22:27:32.151725+0000","last_became_peered":"2024-04-05T22:27:32.151725+0000","last_unstale":"2024-04-05T22:27:32.157359+0000","last_undegraded":"2024-04-05T22:27:32.157359+0000","last_fullsized":"2024-04-05T22:27:32.157359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:44.804179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158218+0000","last_change":"2024-04-05T22:27:32.149943+0000","last_active":"2024-04-05T22:27:32.158218+0000","last_peered":"2024-04-05T22:27:32.158218+0000","last_clean":"2024-04-05T22:27:32.158218+0000","last_became_active":"2024-04-05T22:27:32.149602+0000","last_became_peered":"2024-04-05T22:27:32.149602+0000","last_unstale":"2024-04-05T22:27:32.158218+0000","last_undegraded":"2024-04-05T22:27:32.158218+0000","last_fullsized":"2024-04-05T22:27:32.158218+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140119+0000","last_change":"2024-04-05T22:27:32.137693+0000","last_active":"2024-04-05T22:27:32.140119+0000","last_peered":"2024-04-05T22:27:32.140119+0000","last_clean":"2024-04-05T22:27:32.140119+0000","last_became_active":"2024-04-05T22:27:32.137422+0000","last_became_peered":"2024-04-05T22:27:32.137422+0000","last_unstale":"2024-04-05T22:27:32.140119+0000","last_undegraded":"2024-04-05T22:27:32.140119+0000","last_fullsized":"2024-04-05T22:27:32.140119+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:50:49.869635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138442+0000","last_change":"2024-04-05T22:27:32.128032+0000","last_active":"2024-04-05T22:27:32.138442+0000","last_peered":"2024-04-05T22:27:32.138442+0000","last_clean":"2024-04-05T22:27:32.138442+0000","last_became_active":"2024-04-05T22:27:32.127767+0000","last_became_peered":"2024-04-05T22:27:32.127767+0000","last_unstale":"2024-04-05T22:27:32.138442+0000","last_undegraded":"2024-04-05T22:27:32.138442+0000","last_fullsized":"2024-04-05T22:27:32.138442+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:01.524104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143729+0000","last_change":"2024-04-05T22:27:32.134496+0000","last_active":"2024-04-05T22:27:32.143729+0000","last_peered":"2024-04-05T22:27:32.143729+0000","last_clean":"2024-04-05T22:27:32.143729+0000","last_became_active":"2024-04-05T22:27:32.134230+0000","last_became_peered":"2024-04-05T22:27:32.134230+0000","last_unstale":"2024-04-05T22:27:32.143729+0000","last_undegraded":"2024-04-05T22:27:32.143729+0000","last_fullsized":"2024-04-05T22:27:32.143729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:29:35.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154077+0000","last_change":"2024-04-05T22:27:32.148558+0000","last_active":"2024-04-05T22:27:32.154077+0000","last_peered":"2024-04-05T22:27:32.154077+0000","last_clean":"2024-04-05T22:27:32.154077+0000","last_became_active":"2024-04-05T22:27:32.148017+0000","last_became_peered":"2024-04-05T22:27:32.148017+0000","last_unstale":"2024-04-05T22:27:32.154077+0000","last_undegraded":"2024-04-05T22:27:32.154077+0000","last_fullsized":"2024-04-05T22:27:32.154077+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:17:43.310001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140967+0000","last_change":"2024-04-05T22:27:32.137556+0000","last_active":"2024-04-05T22:27:32.140967+0000","last_peered":"2024-04-05T22:27:32.140967+0000","last_clean":"2024-04-05T22:27:32.140967+0000","last_became_active":"2024-04-05T22:27:32.137060+0000","last_became_peered":"2024-04-05T22:27:32.137060+0000","last_unstale":"2024-04-05T22:27:32.140967+0000","last_undegraded":"2024-04-05T22:27:32.140967+0000","last_fullsized":"2024-04-05T22:27:32.140967+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:29.910437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.141350+0000","last_change":"2024-04-05T22:27:32.136505+0000","last_active":"2024-04-05T22:27:32.141350+0000","last_peered":"2024-04-05T22:27:32.141350+0000","last_clean":"2024-04-05T22:27:32.141350+0000","last_became_active":"2024-04-05T22:27:32.136266+0000","last_became_peered":"2024-04-05T22:27:32.136266+0000","last_unstale":"2024-04-05T22:27:32.141350+0000","last_undegraded":"2024-04-05T22:27:32.141350+0000","last_fullsized":"2024-04-05T22:27:32.141350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:48.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140150+0000","last_change":"2024-04-05T22:27:32.133312+0000","last_active":"2024-04-05T22:27:32.140150+0000","last_peered":"2024-04-05T22:27:32.140150+0000","last_clean":"2024-04-05T22:27:32.140150+0000","last_became_active":"2024-04-05T22:27:32.132649+0000","last_became_peered":"2024-04-05T22:27:32.132649+0000","last_unstale":"2024-04-05T22:27:32.140150+0000","last_undegraded":"2024-04-05T22:27:32.140150+0000","last_fullsized":"2024-04-05T22:27:32.140150+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.139752+0000","last_change":"2024-04-05T22:27:32.129501+0000","last_active":"2024-04-05T22:27:32.139752+0000","last_peered":"2024-04-05T22:27:32.139752+0000","last_clean":"2024-04-05T22:27:32.139752+0000","last_became_active":"2024-04-05T22:27:32.129197+0000","last_became_peered":"2024-04-05T22:27:32.129197+0000","last_unstale":"2024-04-05T22:27:32.139752+0000","last_undegraded":"2024-04-05T22:27:32.139752+0000","last_fullsized":"2024-04-05T22:27:32.139752+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:36:41.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154787+0000","last_change":"2024-04-05T22:27:31.660851+0000","last_active":"2024-04-05T22:27:32.154787+0000","last_peered":"2024-04-05T22:27:32.154787+0000","last_clean":"2024-04-05T22:27:32.154787+0000","last_became_active":"2024-04-05T22:27:31.660534+0000","last_became_peered":"2024-04-05T22:27:31.660534+0000","last_unstale":"2024-04-05T22:27:32.154787+0000","last_undegraded":"2024-04-05T22:27:32.154787+0000","last_fullsized":"2024-04-05T22:27:32.154787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:32.947166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158936+0000","last_change":"2024-04-05T22:27:32.157085+0000","last_active":"2024-04-05T22:27:32.158936+0000","last_peered":"2024-04-05T22:27:32.158936+0000","last_clean":"2024-04-05T22:27:32.158936+0000","last_became_active":"2024-04-05T22:27:32.156503+0000","last_became_peered":"2024-04-05T22:27:32.156503+0000","last_unstale":"2024-04-05T22:27:32.158936+0000","last_undegraded":"2024-04-05T22:27:32.158936+0000","last_fullsized":"2024-04-05T22:27:32.158936+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:20:15.117257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.151117+0000","last_change":"2024-04-05T22:27:31.663795+0000","last_active":"2024-04-05T22:27:32.151117+0000","last_peered":"2024-04-05T22:27:32.151117+0000","last_clean":"2024-04-05T22:27:32.151117+0000","last_became_active":"2024-04-05T22:27:31.663319+0000","last_became_peered":"2024-04-05T22:27:31.663319+0000","last_unstale":"2024-04-05T22:27:32.151117+0000","last_undegraded":"2024-04-05T22:27:32.151117+0000","last_fullsized":"2024-04-05T22:27:32.151117+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:06.096111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143180+0000","last_change":"2024-04-05T22:27:32.140848+0000","last_active":"2024-04-05T22:27:32.143180+0000","last_peered":"2024-04-05T22:27:32.143180+0000","last_clean":"2024-04-05T22:27:32.143180+0000","last_became_active":"2024-04-05T22:27:32.139745+0000","last_became_peered":"2024-04-05T22:27:32.139745+0000","last_unstale":"2024-04-05T22:27:32.143180+0000","last_undegraded":"2024-04-05T22:27:32.143180+0000","last_fullsized":"2024-04-05T22:27:32.143180+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:29.910437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144536+0000","last_change":"2024-04-05T22:27:31.653855+0000","last_active":"2024-04-05T22:27:32.144536+0000","last_peered":"2024-04-05T22:27:32.144536+0000","last_clean":"2024-04-05T22:27:32.144536+0000","last_became_active":"2024-04-05T22:27:31.653563+0000","last_became_peered":"2024-04-05T22:27:31.653563+0000","last_unstale":"2024-04-05T22:27:32.144536+0000","last_undegraded":"2024-04-05T22:27:32.144536+0000","last_fullsized":"2024-04-05T22:27:32.144536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:37:20.277508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159033+0000","last_change":"2024-04-05T22:27:32.155118+0000","last_active":"2024-04-05T22:27:32.159033+0000","last_peered":"2024-04-05T22:27:32.159033+0000","last_clean":"2024-04-05T22:27:32.159033+0000","last_became_active":"2024-04-05T22:27:32.154327+0000","last_became_peered":"2024-04-05T22:27:32.154327+0000","last_unstale":"2024-04-05T22:27:32.159033+0000","last_undegraded":"2024-04-05T22:27:32.159033+0000","last_fullsized":"2024-04-05T22:27:32.159033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:38:58.315925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.145764+0000","last_change":"2024-04-05T22:27:31.649498+0000","last_active":"2024-04-05T22:27:32.145764+0000","last_peered":"2024-04-05T22:27:32.145764+0000","last_clean":"2024-04-05T22:27:32.145764+0000","last_became_active":"2024-04-05T22:27:31.647928+0000","last_became_peered":"2024-04-05T22:27:31.647928+0000","last_unstale":"2024-04-05T22:27:32.145764+0000","last_undegraded":"2024-04-05T22:27:32.145764+0000","last_fullsized":"2024-04-05T22:27:32.145764+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.139259+0000","last_change":"2024-04-05T22:27:32.133816+0000","last_active":"2024-04-05T22:27:32.139259+0000","last_peered":"2024-04-05T22:27:32.139259+0000","last_clean":"2024-04-05T22:27:32.139259+0000","last_became_active":"2024-04-05T22:27:32.133542+0000","last_became_peered":"2024-04-05T22:27:32.133542+0000","last_unstale":"2024-04-05T22:27:32.139259+0000","last_undegraded":"2024-04-05T22:27:32.139259+0000","last_fullsized":"2024-04-05T22:27:32.139259+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:46.460747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146398+0000","last_change":"2024-04-05T22:27:31.651770+0000","last_active":"2024-04-05T22:27:32.146398+0000","last_peered":"2024-04-05T22:27:32.146398+0000","last_clean":"2024-04-05T22:27:32.146398+0000","last_became_active":"2024-04-05T22:27:31.648415+0000","last_became_peered":"2024-04-05T22:27:31.648415+0000","last_unstale":"2024-04-05T22:27:32.146398+0000","last_undegraded":"2024-04-05T22:27:32.146398+0000","last_fullsized":"2024-04-05T22:27:32.146398+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144142+0000","last_change":"2024-04-05T22:27:32.134502+0000","last_active":"2024-04-05T22:27:32.144142+0000","last_peered":"2024-04-05T22:27:32.144142+0000","last_clean":"2024-04-05T22:27:32.144142+0000","last_became_active":"2024-04-05T22:27:32.133666+0000","last_became_peered":"2024-04-05T22:27:32.133666+0000","last_unstale":"2024-04-05T22:27:32.144142+0000","last_undegraded":"2024-04-05T22:27:32.144142+0000","last_fullsized":"2024-04-05T22:27:32.144142+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:52.724372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146490+0000","last_change":"2024-04-05T22:27:31.654819+0000","last_active":"2024-04-05T22:27:32.146490+0000","last_peered":"2024-04-05T22:27:32.146490+0000","last_clean":"2024-04-05T22:27:32.146490+0000","last_became_active":"2024-04-05T22:27:31.654476+0000","last_became_peered":"2024-04-05T22:27:31.654476+0000","last_unstale":"2024-04-05T22:27:32.146490+0000","last_undegraded":"2024-04-05T22:27:32.146490+0000","last_fullsized":"2024-04-05T22:27:32.146490+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:34.557930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.131616+0000","last_change":"2024-04-05T22:27:32.125645+0000","last_active":"2024-04-05T22:27:32.131616+0000","last_peered":"2024-04-05T22:27:32.131616+0000","last_clean":"2024-04-05T22:27:32.131616+0000","last_became_active":"2024-04-05T22:27:32.124429+0000","last_became_peered":"2024-04-05T22:27:32.124429+0000","last_unstale":"2024-04-05T22:27:32.131616+0000","last_undegraded":"2024-04-05T22:27:32.131616+0000","last_fullsized":"2024-04-05T22:27:32.131616+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:22:04.714703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122685+0000","last_change":"2024-04-05T22:27:31.656501+0000","last_active":"2024-04-05T22:27:32.122685+0000","last_peered":"2024-04-05T22:27:32.122685+0000","last_clean":"2024-04-05T22:27:32.122685+0000","last_became_active":"2024-04-05T22:27:31.656225+0000","last_became_peered":"2024-04-05T22:27:31.656225+0000","last_unstale":"2024-04-05T22:27:32.122685+0000","last_undegraded":"2024-04-05T22:27:32.122685+0000","last_fullsized":"2024-04-05T22:27:32.122685+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:00.512191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.152884+0000","last_change":"2024-04-05T22:27:32.149318+0000","last_active":"2024-04-05T22:27:32.152884+0000","last_peered":"2024-04-05T22:27:32.152884+0000","last_clean":"2024-04-05T22:27:32.152884+0000","last_became_active":"2024-04-05T22:27:32.148943+0000","last_became_peered":"2024-04-05T22:27:32.148943+0000","last_unstale":"2024-04-05T22:27:32.152884+0000","last_undegraded":"2024-04-05T22:27:32.152884+0000","last_fullsized":"2024-04-05T22:27:32.152884+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:20:01.110274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113566+0000","last_change":"2024-04-05T22:27:31.654830+0000","last_active":"2024-04-05T22:27:32.113566+0000","last_peered":"2024-04-05T22:27:32.113566+0000","last_clean":"2024-04-05T22:27:32.113566+0000","last_became_active":"2024-04-05T22:27:31.654260+0000","last_became_peered":"2024-04-05T22:27:31.654260+0000","last_unstale":"2024-04-05T22:27:32.113566+0000","last_undegraded":"2024-04-05T22:27:32.113566+0000","last_fullsized":"2024-04-05T22:27:32.113566+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:25.503984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.150564+0000","last_change":"2024-04-05T22:27:32.144118+0000","last_active":"2024-04-05T22:27:32.150564+0000","last_peered":"2024-04-05T22:27:32.150564+0000","last_clean":"2024-04-05T22:27:32.150564+0000","last_became_active":"2024-04-05T22:27:32.143842+0000","last_became_peered":"2024-04-05T22:27:32.143842+0000","last_unstale":"2024-04-05T22:27:32.150564+0000","last_undegraded":"2024-04-05T22:27:32.150564+0000","last_fullsized":"2024-04-05T22:27:32.150564+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.125436+0000","last_change":"2024-04-05T22:27:31.649844+0000","last_active":"2024-04-05T22:27:32.125436+0000","last_peered":"2024-04-05T22:27:32.125436+0000","last_clean":"2024-04-05T22:27:32.125436+0000","last_became_active":"2024-04-05T22:27:31.649587+0000","last_became_peered":"2024-04-05T22:27:31.649587+0000","last_unstale":"2024-04-05T22:27:32.125436+0000","last_undegraded":"2024-04-05T22:27:32.125436+0000","last_fullsized":"2024-04-05T22:27:32.125436+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.675407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157577+0000","last_change":"2024-04-05T22:27:32.152471+0000","last_active":"2024-04-05T22:27:32.157577+0000","last_peered":"2024-04-05T22:27:32.157577+0000","last_clean":"2024-04-05T22:27:32.157577+0000","last_became_active":"2024-04-05T22:27:32.152179+0000","last_became_peered":"2024-04-05T22:27:32.152179+0000","last_unstale":"2024-04-05T22:27:32.157577+0000","last_undegraded":"2024-04-05T22:27:32.157577+0000","last_fullsized":"2024-04-05T22:27:32.157577+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:36.175015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.124261+0000","last_change":"2024-04-05T22:27:31.661594+0000","last_active":"2024-04-05T22:27:32.124261+0000","last_peered":"2024-04-05T22:27:32.124261+0000","last_clean":"2024-04-05T22:27:32.124261+0000","last_became_active":"2024-04-05T22:27:31.661326+0000","last_became_peered":"2024-04-05T22:27:31.661326+0000","last_unstale":"2024-04-05T22:27:32.124261+0000","last_undegraded":"2024-04-05T22:27:32.124261+0000","last_fullsized":"2024-04-05T22:27:32.124261+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:24.001489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143873+0000","last_change":"2024-04-05T22:27:32.136804+0000","last_active":"2024-04-05T22:27:32.143873+0000","last_peered":"2024-04-05T22:27:32.143873+0000","last_clean":"2024-04-05T22:27:32.143873+0000","last_became_active":"2024-04-05T22:27:32.136577+0000","last_became_peered":"2024-04-05T22:27:32.136577+0000","last_unstale":"2024-04-05T22:27:32.143873+0000","last_undegraded":"2024-04-05T22:27:32.143873+0000","last_fullsized":"2024-04-05T22:27:32.143873+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:57:46.503662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143478+0000","last_change":"2024-04-05T22:27:31.650080+0000","last_active":"2024-04-05T22:27:32.143478+0000","last_peered":"2024-04-05T22:27:32.143478+0000","last_clean":"2024-04-05T22:27:32.143478+0000","last_became_active":"2024-04-05T22:27:31.649371+0000","last_became_peered":"2024-04-05T22:27:31.649371+0000","last_unstale":"2024-04-05T22:27:32.143478+0000","last_undegraded":"2024-04-05T22:27:32.143478+0000","last_fullsized":"2024-04-05T22:27:32.143478+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:34.557930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.155092+0000","last_change":"2024-04-05T22:27:32.153957+0000","last_active":"2024-04-05T22:27:32.155092+0000","last_peered":"2024-04-05T22:27:32.155092+0000","last_clean":"2024-04-05T22:27:32.155092+0000","last_became_active":"2024-04-05T22:27:32.153675+0000","last_became_peered":"2024-04-05T22:27:32.153675+0000","last_unstale":"2024-04-05T22:27:32.155092+0000","last_undegraded":"2024-04-05T22:27:32.155092+0000","last_fullsized":"2024-04-05T22:27:32.155092+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:44:44.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.150118+0000","last_change":"2024-04-05T22:27:31.652577+0000","last_active":"2024-04-05T22:27:32.150118+0000","last_peered":"2024-04-05T22:27:32.150118+0000","last_clean":"2024-04-05T22:27:32.150118+0000","last_became_active":"2024-04-05T22:27:31.652248+0000","last_became_peered":"2024-04-05T22:27:31.652248+0000","last_unstale":"2024-04-05T22:27:32.150118+0000","last_undegraded":"2024-04-05T22:27:32.150118+0000","last_fullsized":"2024-04-05T22:27:32.150118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:36.909692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144083+0000","last_change":"2024-04-05T22:27:32.138144+0000","last_active":"2024-04-05T22:27:32.144083+0000","last_peered":"2024-04-05T22:27:32.144083+0000","last_clean":"2024-04-05T22:27:32.144083+0000","last_became_active":"2024-04-05T22:27:32.137808+0000","last_became_peered":"2024-04-05T22:27:32.137808+0000","last_unstale":"2024-04-05T22:27:32.144083+0000","last_undegraded":"2024-04-05T22:27:32.144083+0000","last_fullsized":"2024-04-05T22:27:32.144083+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:35.096886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.153369+0000","last_change":"2024-04-05T22:27:31.662465+0000","last_active":"2024-04-05T22:27:32.153369+0000","last_peered":"2024-04-05T22:27:32.153369+0000","last_clean":"2024-04-05T22:27:32.153369+0000","last_became_active":"2024-04-05T22:27:31.661824+0000","last_became_peered":"2024-04-05T22:27:31.661824+0000","last_unstale":"2024-04-05T22:27:32.153369+0000","last_undegraded":"2024-04-05T22:27:32.153369+0000","last_fullsized":"2024-04-05T22:27:32.153369+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157724+0000","last_change":"2024-04-05T22:27:32.155119+0000","last_active":"2024-04-05T22:27:32.157724+0000","last_peered":"2024-04-05T22:27:32.157724+0000","last_clean":"2024-04-05T22:27:32.157724+0000","last_became_active":"2024-04-05T22:27:32.154632+0000","last_became_peered":"2024-04-05T22:27:32.154632+0000","last_unstale":"2024-04-05T22:27:32.157724+0000","last_undegraded":"2024-04-05T22:27:32.157724+0000","last_fullsized":"2024-04-05T22:27:32.157724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:56:14.562747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143983+0000","last_change":"2024-04-05T22:27:31.658087+0000","last_active":"2024-04-05T22:27:32.143983+0000","last_peered":"2024-04-05T22:27:32.143983+0000","last_clean":"2024-04-05T22:27:32.143983+0000","last_became_active":"2024-04-05T22:27:31.657595+0000","last_became_peered":"2024-04-05T22:27:31.657595+0000","last_unstale":"2024-04-05T22:27:32.143983+0000","last_undegraded":"2024-04-05T22:27:32.143983+0000","last_fullsized":"2024-04-05T22:27:32.143983+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:13.300136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144822+0000","last_change":"2024-04-05T22:27:32.137673+0000","last_active":"2024-04-05T22:27:32.144822+0000","last_peered":"2024-04-05T22:27:32.144822+0000","last_clean":"2024-04-05T22:27:32.144822+0000","last_became_active":"2024-04-05T22:27:32.136510+0000","last_became_peered":"2024-04-05T22:27:32.136510+0000","last_unstale":"2024-04-05T22:27:32.144822+0000","last_undegraded":"2024-04-05T22:27:32.144822+0000","last_fullsized":"2024-04-05T22:27:32.144822+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:04:06.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.147391+0000","last_change":"2024-04-05T22:27:31.658771+0000","last_active":"2024-04-05T22:27:32.147391+0000","last_peered":"2024-04-05T22:27:32.147391+0000","last_clean":"2024-04-05T22:27:32.147391+0000","last_became_active":"2024-04-05T22:27:31.657722+0000","last_became_peered":"2024-04-05T22:27:31.657722+0000","last_unstale":"2024-04-05T22:27:32.147391+0000","last_undegraded":"2024-04-05T22:27:32.147391+0000","last_fullsized":"2024-04-05T22:27:32.147391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:23:41.153422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.131968+0000","last_change":"2024-04-05T22:27:32.128073+0000","last_active":"2024-04-05T22:27:32.131968+0000","last_peered":"2024-04-05T22:27:32.131968+0000","last_clean":"2024-04-05T22:27:32.131968+0000","last_became_active":"2024-04-05T22:27:32.127693+0000","last_became_peered":"2024-04-05T22:27:32.127693+0000","last_unstale":"2024-04-05T22:27:32.131968+0000","last_undegraded":"2024-04-05T22:27:32.131968+0000","last_fullsized":"2024-04-05T22:27:32.131968+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:18:04.839710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146927+0000","last_change":"2024-04-05T22:27:31.657566+0000","last_active":"2024-04-05T22:27:32.146927+0000","last_peered":"2024-04-05T22:27:32.146927+0000","last_clean":"2024-04-05T22:27:32.146927+0000","last_became_active":"2024-04-05T22:27:31.657338+0000","last_became_peered":"2024-04-05T22:27:31.657338+0000","last_unstale":"2024-04-05T22:27:32.146927+0000","last_undegraded":"2024-04-05T22:27:32.146927+0000","last_fullsized":"2024-04-05T22:27:32.146927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:39:25.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138690+0000","last_change":"2024-04-05T22:27:32.130139+0000","last_active":"2024-04-05T22:27:32.138690+0000","last_peered":"2024-04-05T22:27:32.138690+0000","last_clean":"2024-04-05T22:27:32.138690+0000","last_became_active":"2024-04-05T22:27:32.129813+0000","last_became_peered":"2024-04-05T22:27:32.129813+0000","last_unstale":"2024-04-05T22:27:32.138690+0000","last_undegraded":"2024-04-05T22:27:32.138690+0000","last_fullsized":"2024-04-05T22:27:32.138690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:25.750070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146985+0000","last_change":"2024-04-05T22:27:31.656663+0000","last_active":"2024-04-05T22:27:32.146985+0000","last_peered":"2024-04-05T22:27:32.146985+0000","last_clean":"2024-04-05T22:27:32.146985+0000","last_became_active":"2024-04-05T22:27:31.656380+0000","last_became_peered":"2024-04-05T22:27:31.656380+0000","last_unstale":"2024-04-05T22:27:32.146985+0000","last_undegraded":"2024-04-05T22:27:32.146985+0000","last_fullsized":"2024-04-05T22:27:32.146985+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:22.616723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157350+0000","last_change":"2024-04-05T22:27:32.149910+0000","last_active":"2024-04-05T22:27:32.157350+0000","last_peered":"2024-04-05T22:27:32.157350+0000","last_clean":"2024-04-05T22:27:32.157350+0000","last_became_active":"2024-04-05T22:27:32.149606+0000","last_became_peered":"2024-04-05T22:27:32.149606+0000","last_unstale":"2024-04-05T22:27:32.157350+0000","last_undegraded":"2024-04-05T22:27:32.157350+0000","last_fullsized":"2024-04-05T22:27:32.157350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:47:40.767763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154737+0000","last_change":"2024-04-05T22:27:31.653732+0000","last_active":"2024-04-05T22:27:32.154737+0000","last_peered":"2024-04-05T22:27:32.154737+0000","last_clean":"2024-04-05T22:27:32.154737+0000","last_became_active":"2024-04-05T22:27:31.653196+0000","last_became_peered":"2024-04-05T22:27:31.653196+0000","last_unstale":"2024-04-05T22:27:32.154737+0000","last_undegraded":"2024-04-05T22:27:32.154737+0000","last_fullsized":"2024-04-05T22:27:32.154737+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:38:33.507385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159072+0000","last_change":"2024-04-05T22:27:32.155550+0000","last_active":"2024-04-05T22:27:32.159072+0000","last_peered":"2024-04-05T22:27:32.159072+0000","last_clean":"2024-04-05T22:27:32.159072+0000","last_became_active":"2024-04-05T22:27:32.154935+0000","last_became_peered":"2024-04-05T22:27:32.154935+0000","last_unstale":"2024-04-05T22:27:32.159072+0000","last_undegraded":"2024-04-05T22:27:32.159072+0000","last_fullsized":"2024-04-05T22:27:32.159072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:52.724372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.128625+0000","last_change":"2024-04-05T22:27:31.658814+0000","last_active":"2024-04-05T22:27:32.128625+0000","last_peered":"2024-04-05T22:27:32.128625+0000","last_clean":"2024-04-05T22:27:32.128625+0000","last_became_active":"2024-04-05T22:27:31.658509+0000","last_became_peered":"2024-04-05T22:27:31.658509+0000","last_unstale":"2024-04-05T22:27:32.128625+0000","last_undegraded":"2024-04-05T22:27:32.128625+0000","last_fullsized":"2024-04-05T22:27:32.128625+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.675407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157885+0000","last_change":"2024-04-05T22:27:32.151800+0000","last_active":"2024-04-05T22:27:32.157885+0000","last_peered":"2024-04-05T22:27:32.157885+0000","last_clean":"2024-04-05T22:27:32.157885+0000","last_became_active":"2024-04-05T22:27:32.151494+0000","last_became_peered":"2024-04-05T22:27:32.151494+0000","last_unstale":"2024-04-05T22:27:32.157885+0000","last_undegraded":"2024-04-05T22:27:32.157885+0000","last_fullsized":"2024-04-05T22:27:32.157885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:36:41.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144887+0000","last_change":"2024-04-05T22:27:31.659129+0000","last_active":"2024-04-05T22:27:32.144887+0000","last_peered":"2024-04-05T22:27:32.144887+0000","last_clean":"2024-04-05T22:27:32.144887+0000","last_became_active":"2024-04-05T22:27:31.658765+0000","last_became_peered":"2024-04-05T22:27:31.658765+0000","last_unstale":"2024-04-05T22:27:32.144887+0000","last_undegraded":"2024-04-05T22:27:32.144887+0000","last_fullsized":"2024-04-05T22:27:32.144887+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:22.616723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.155319+0000","last_change":"2024-04-05T22:27:32.152698+0000","last_active":"2024-04-05T22:27:32.155319+0000","last_peered":"2024-04-05T22:27:32.155319+0000","last_clean":"2024-04-05T22:27:32.155319+0000","last_became_active":"2024-04-05T22:27:32.152323+0000","last_became_peered":"2024-04-05T22:27:32.152323+0000","last_unstale":"2024-04-05T22:27:32.155319+0000","last_undegraded":"2024-04-05T22:27:32.155319+0000","last_fullsized":"2024-04-05T22:27:32.155319+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:43.275719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.121091+0000","last_change":"2024-04-05T22:27:31.649699+0000","last_active":"2024-04-05T22:27:32.121091+0000","last_peered":"2024-04-05T22:27:32.121091+0000","last_clean":"2024-04-05T22:27:32.121091+0000","last_became_active":"2024-04-05T22:27:31.649211+0000","last_became_peered":"2024-04-05T22:27:31.649211+0000","last_unstale":"2024-04-05T22:27:32.121091+0000","last_undegraded":"2024-04-05T22:27:32.121091+0000","last_fullsized":"2024-04-05T22:27:32.121091+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:32.947166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158016+0000","last_change":"2024-04-05T22:27:32.154365+0000","last_active":"2024-04-05T22:27:32.158016+0000","last_peered":"2024-04-05T22:27:32.158016+0000","last_clean":"2024-04-05T22:27:32.158016+0000","last_became_active":"2024-04-05T22:27:32.151432+0000","last_became_peered":"2024-04-05T22:27:32.151432+0000","last_unstale":"2024-04-05T22:27:32.158016+0000","last_undegraded":"2024-04-05T22:27:32.158016+0000","last_fullsized":"2024-04-05T22:27:32.158016+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:21.246110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.118469+0000","last_change":"2024-04-05T22:27:31.650690+0000","last_active":"2024-04-05T22:27:32.118469+0000","last_peered":"2024-04-05T22:27:32.118469+0000","last_clean":"2024-04-05T22:27:32.118469+0000","last_became_active":"2024-04-05T22:27:31.650387+0000","last_became_peered":"2024-04-05T22:27:31.650387+0000","last_unstale":"2024-04-05T22:27:32.118469+0000","last_undegraded":"2024-04-05T22:27:32.118469+0000","last_fullsized":"2024-04-05T22:27:32.118469+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:28.175860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143594+0000","last_change":"2024-04-05T22:27:32.133254+0000","last_active":"2024-04-05T22:27:32.143594+0000","last_peered":"2024-04-05T22:27:32.143594+0000","last_clean":"2024-04-05T22:27:32.143594+0000","last_became_active":"2024-04-05T22:27:32.132955+0000","last_became_peered":"2024-04-05T22:27:32.132955+0000","last_unstale":"2024-04-05T22:27:32.143594+0000","last_undegraded":"2024-04-05T22:27:32.143594+0000","last_fullsized":"2024-04-05T22:27:32.143594+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:15:22.544892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113689+0000","last_change":"2024-04-05T22:27:31.658510+0000","last_active":"2024-04-05T22:27:32.113689+0000","last_peered":"2024-04-05T22:27:32.113689+0000","last_clean":"2024-04-05T22:27:32.113689+0000","last_became_active":"2024-04-05T22:27:31.658163+0000","last_became_peered":"2024-04-05T22:27:31.658163+0000","last_unstale":"2024-04-05T22:27:32.113689+0000","last_undegraded":"2024-04-05T22:27:32.113689+0000","last_fullsized":"2024-04-05T22:27:32.113689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:04:01.004689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158637+0000","last_change":"2024-04-05T22:27:32.153312+0000","last_active":"2024-04-05T22:27:32.158637+0000","last_peered":"2024-04-05T22:27:32.158637+0000","last_clean":"2024-04-05T22:27:32.158637+0000","last_became_active":"2024-04-05T22:27:32.152404+0000","last_became_peered":"2024-04-05T22:27:32.152404+0000","last_unstale":"2024-04-05T22:27:32.158637+0000","last_undegraded":"2024-04-05T22:27:32.158637+0000","last_fullsized":"2024-04-05T22:27:32.158637+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:17:12.574970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143602+0000","last_change":"2024-04-05T22:27:31.660346+0000","last_active":"2024-04-05T22:27:32.143602+0000","last_peered":"2024-04-05T22:27:32.143602+0000","last_clean":"2024-04-05T22:27:32.143602+0000","last_became_active":"2024-04-05T22:27:31.660031+0000","last_became_peered":"2024-04-05T22:27:31.660031+0000","last_unstale":"2024-04-05T22:27:32.143602+0000","last_undegraded":"2024-04-05T22:27:32.143602+0000","last_fullsized":"2024-04-05T22:27:32.143602+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:40:33.424001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143631+0000","last_change":"2024-04-05T22:27:32.138494+0000","last_active":"2024-04-05T22:27:32.143631+0000","last_peered":"2024-04-05T22:27:32.143631+0000","last_clean":"2024-04-05T22:27:32.143631+0000","last_became_active":"2024-04-05T22:27:32.137997+0000","last_became_peered":"2024-04-05T22:27:32.137997+0000","last_unstale":"2024-04-05T22:27:32.143631+0000","last_undegraded":"2024-04-05T22:27:32.143631+0000","last_fullsized":"2024-04-05T22:27:32.143631+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:42:46.145634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144915+0000","last_change":"2024-04-05T22:27:31.660207+0000","last_active":"2024-04-05T22:27:32.144915+0000","last_peered":"2024-04-05T22:27:32.144915+0000","last_clean":"2024-04-05T22:27:32.144915+0000","last_became_active":"2024-04-05T22:27:31.659953+0000","last_became_peered":"2024-04-05T22:27:31.659953+0000","last_unstale":"2024-04-05T22:27:32.144915+0000","last_undegraded":"2024-04-05T22:27:32.144915+0000","last_fullsized":"2024-04-05T22:27:32.144915+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:46:03.768005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.129609+0000","last_change":"2024-04-05T22:27:32.125090+0000","last_active":"2024-04-05T22:27:32.129609+0000","last_peered":"2024-04-05T22:27:32.129609+0000","last_clean":"2024-04-05T22:27:32.129609+0000","last_became_active":"2024-04-05T22:27:32.124832+0000","last_became_peered":"2024-04-05T22:27:32.124832+0000","last_unstale":"2024-04-05T22:27:32.129609+0000","last_undegraded":"2024-04-05T22:27:32.129609+0000","last_fullsized":"2024-04-05T22:27:32.129609+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:42:08.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.114067+0000","last_change":"2024-04-05T22:27:31.651404+0000","last_active":"2024-04-05T22:27:32.114067+0000","last_peered":"2024-04-05T22:27:32.114067+0000","last_clean":"2024-04-05T22:27:32.114067+0000","last_became_active":"2024-04-05T22:27:31.649789+0000","last_became_peered":"2024-04-05T22:27:31.649789+0000","last_unstale":"2024-04-05T22:27:32.114067+0000","last_undegraded":"2024-04-05T22:27:32.114067+0000","last_fullsized":"2024-04-05T22:27:32.114067+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:45.006358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.156588+0000","last_change":"2024-04-05T22:27:32.151727+0000","last_active":"2024-04-05T22:27:32.156588+0000","last_peered":"2024-04-05T22:27:32.156588+0000","last_clean":"2024-04-05T22:27:32.156588+0000","last_became_active":"2024-04-05T22:27:32.151305+0000","last_became_peered":"2024-04-05T22:27:32.151305+0000","last_unstale":"2024-04-05T22:27:32.156588+0000","last_undegraded":"2024-04-05T22:27:32.156588+0000","last_fullsized":"2024-04-05T22:27:32.156588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:44:23.736872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.126673+0000","last_change":"2024-04-05T22:27:31.651066+0000","last_active":"2024-04-05T22:27:32.126673+0000","last_peered":"2024-04-05T22:27:32.126673+0000","last_clean":"2024-04-05T22:27:32.126673+0000","last_became_active":"2024-04-05T22:27:31.650814+0000","last_became_peered":"2024-04-05T22:27:31.650814+0000","last_unstale":"2024-04-05T22:27:32.126673+0000","last_undegraded":"2024-04-05T22:27:32.126673+0000","last_fullsized":"2024-04-05T22:27:32.126673+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:35:34.319386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144399+0000","last_change":"2024-04-05T22:27:32.138595+0000","last_active":"2024-04-05T22:27:32.144399+0000","last_peered":"2024-04-05T22:27:32.144399+0000","last_clean":"2024-04-05T22:27:32.144399+0000","last_became_active":"2024-04-05T22:27:32.137137+0000","last_became_peered":"2024-04-05T22:27:32.137137+0000","last_unstale":"2024-04-05T22:27:32.144399+0000","last_undegraded":"2024-04-05T22:27:32.144399+0000","last_fullsized":"2024-04-05T22:27:32.144399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:40:34.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.129624+0000","last_change":"2024-04-05T22:27:31.657303+0000","last_active":"2024-04-05T22:27:32.129624+0000","last_peered":"2024-04-05T22:27:32.129624+0000","last_clean":"2024-04-05T22:27:32.129624+0000","last_became_active":"2024-04-05T22:27:31.656942+0000","last_became_peered":"2024-04-05T22:27:31.656942+0000","last_unstale":"2024-04-05T22:27:32.129624+0000","last_undegraded":"2024-04-05T22:27:32.129624+0000","last_fullsized":"2024-04-05T22:27:32.129624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:37:20.277508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144104+0000","last_change":"2024-04-05T22:27:32.133337+0000","last_active":"2024-04-05T22:27:32.144104+0000","last_peered":"2024-04-05T22:27:32.144104+0000","last_clean":"2024-04-05T22:27:32.144104+0000","last_became_active":"2024-04-05T22:27:32.132094+0000","last_became_peered":"2024-04-05T22:27:32.132094+0000","last_unstale":"2024-04-05T22:27:32.144104+0000","last_undegraded":"2024-04-05T22:27:32.144104+0000","last_fullsized":"2024-04-05T22:27:32.144104+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:36.855438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122283+0000","last_change":"2024-04-05T22:27:31.654670+0000","last_active":"2024-04-05T22:27:32.122283+0000","last_peered":"2024-04-05T22:27:32.122283+0000","last_clean":"2024-04-05T22:27:32.122283+0000","last_became_active":"2024-04-05T22:27:31.654404+0000","last_became_peered":"2024-04-05T22:27:31.654404+0000","last_unstale":"2024-04-05T22:27:32.122283+0000","last_undegraded":"2024-04-05T22:27:32.122283+0000","last_fullsized":"2024-04-05T22:27:32.122283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:24.738157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159164+0000","last_change":"2024-04-05T22:27:32.154408+0000","last_active":"2024-04-05T22:27:32.159164+0000","last_peered":"2024-04-05T22:27:32.159164+0000","last_clean":"2024-04-05T22:27:32.159164+0000","last_became_active":"2024-04-05T22:27:32.151762+0000","last_became_peered":"2024-04-05T22:27:32.151762+0000","last_unstale":"2024-04-05T22:27:32.159164+0000","last_undegraded":"2024-04-05T22:27:32.159164+0000","last_fullsized":"2024-04-05T22:27:32.159164+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:03:13.396297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122069+0000","last_change":"2024-04-05T22:27:31.653290+0000","last_active":"2024-04-05T22:27:32.122069+0000","last_peered":"2024-04-05T22:27:32.122069+0000","last_clean":"2024-04-05T22:27:32.122069+0000","last_became_active":"2024-04-05T22:27:31.652864+0000","last_became_peered":"2024-04-05T22:27:31.652864+0000","last_unstale":"2024-04-05T22:27:32.122069+0000","last_undegraded":"2024-04-05T22:27:32.122069+0000","last_fullsized":"2024-04-05T22:27:32.122069+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:28.175860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.125613+0000","last_change":"2024-04-05T22:27:32.121660+0000","last_active":"2024-04-05T22:27:32.125613+0000","last_peered":"2024-04-05T22:27:32.125613+0000","last_clean":"2024-04-05T22:27:32.125613+0000","last_became_active":"2024-04-05T22:27:32.120741+0000","last_became_peered":"2024-04-05T22:27:32.120741+0000","last_unstale":"2024-04-05T22:27:32.125613+0000","last_undegraded":"2024-04-05T22:27:32.125613+0000","last_fullsized":"2024-04-05T22:27:32.125613+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:38.523011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113633+0000","last_change":"2024-04-05T22:27:31.654989+0000","last_active":"2024-04-05T22:27:32.113633+0000","last_peered":"2024-04-05T22:27:32.113633+0000","last_clean":"2024-04-05T22:27:32.113633+0000","last_became_active":"2024-04-05T22:27:31.654581+0000","last_became_peered":"2024-04-05T22:27:31.654581+0000","last_unstale":"2024-04-05T22:27:32.113633+0000","last_undegraded":"2024-04-05T22:27:32.113633+0000","last_fullsized":"2024-04-05T22:27:32.113633+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:54:12.487510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.149881+0000","last_change":"2024-04-05T22:27:32.147959+0000","last_active":"2024-04-05T22:27:32.149881+0000","last_peered":"2024-04-05T22:27:32.149881+0000","last_clean":"2024-04-05T22:27:32.149881+0000","last_became_active":"2024-04-05T22:27:32.147664+0000","last_became_peered":"2024-04-05T22:27:32.147664+0000","last_unstale":"2024-04-05T22:27:32.149881+0000","last_undegraded":"2024-04-05T22:27:32.149881+0000","last_fullsized":"2024-04-05T22:27:32.149881+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:42:08.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.130264+0000","last_change":"2024-04-05T22:27:31.654588+0000","last_active":"2024-04-05T22:27:32.130264+0000","last_peered":"2024-04-05T22:27:32.130264+0000","last_clean":"2024-04-05T22:27:32.130264+0000","last_became_active":"2024-04-05T22:27:31.654302+0000","last_became_peered":"2024-04-05T22:27:31.654302+0000","last_unstale":"2024-04-05T22:27:32.130264+0000","last_undegraded":"2024-04-05T22:27:32.130264+0000","last_fullsized":"2024-04-05T22:27:32.130264+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:19.081446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":274,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-05T22:27:31.658083+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:27:31.658083+0000","last_peered":"2024-04-05T22:27:31.658083+0000","last_clean":"2024-04-05T22:27:31.658083+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:27:31.658083+0000","last_undegraded":"2024-04-05T22:27:31.658083+0000","last_fullsized":"2024-04-05T22:27:31.658083+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1916928,"data_stored":1167264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0700000000000001,"15min":1.034},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.6759999999999999,"5min":1.796,"15min":1.796},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0620000000000001,"15min":1.0109999999999999},"min":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3919999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1639999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.58099999999999996},"max":{"1min":1.504,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.1000000000000001,"15min":1.0680000000000001},"min":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.173,"15min":1.171},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4139999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.087},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.272,"15min":1.222},"min":{"1min":0.92400000000000004,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1299999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.208,"15min":1.139},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72299999999999998},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.712},"last":1.171},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.218},"min":{"1min":0.96899999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7849999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.234,"15min":1.2190000000000001},"min":{"1min":0.81499999999999995,"5min":0.68700000000000006,"15min":0.66200000000000003},"max":{"1min":1.6339999999999999,"5min":1.8360000000000001,"15min":1.865},"last":1.331},{"interface":"front","average":{"1min":1.333,"5min":1.3080000000000001,"15min":1.3169999999999999},"min":{"1min":0.89500000000000002,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":1.681,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3819999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.329,"15min":1.2470000000000001},"min":{"1min":0.88300000000000001,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.752,"5min":1.8440000000000001,"15min":1.9590000000000001},"last":1.224},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3799999999999999,"15min":1.3500000000000001},"min":{"1min":1,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9390000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.04}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.522,"15min":1.4650000000000001},"min":{"1min":1.25,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":2.0630000000000002,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.4890000000000001},"min":{"1min":1.2070000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0150000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.2589999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1314816,"data_stored":576280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0369999999999999,"15min":1.0580000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.5229999999999999,"5min":1.579,"15min":2.2509999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.097},"min":{"1min":0.746,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.627,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.2490000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.093,"15min":1.155},"min":{"1min":0.624,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.1100000000000001,"15min":1.1459999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.8680000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1599999999999999,"15min":1.194},"min":{"1min":0.84599999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.71299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4930000000000001,"5min":1.7749999999999999,"15min":2.2250000000000001},"last":1.361}]},{"osd":3,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1990000000000001,"15min":1.3089999999999999},"min":{"1min":0.73299999999999998,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.7769999999999999},"last":1.96},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.7310000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2390000000000001,"15min":1.321},"min":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1019999999999999},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.55},"min":{"1min":0.78700000000000003,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":3.27},"last":1.8149999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.244,"15min":1.4279999999999999},"min":{"1min":0.76200000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.927,"5min":1.927,"15min":2.3650000000000002},"last":1.708},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4219999999999999,"15min":1.544},"min":{"1min":0.81100000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.3479999999999999},"last":2.3540000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.365,"15min":1.5349999999999999},"min":{"1min":1.0229999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.99,"5min":2.2599999999999998,"15min":3.3540000000000001},"last":1.643},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.359,"15min":1.5389999999999999},"min":{"1min":0.89600000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.831,"5min":1.897,"15min":3.0920000000000001},"last":1.899}]}]},{"osd":5,"up_from":14,"seq":60129542224,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1290240,"data_stored":563230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.093,"15min":1.0189999999999999},"min":{"1min":0.72899999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.405,"5min":1.726,"15min":1.726},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.129,"15min":1.0960000000000001},"min":{"1min":0.84199999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5289999999999999,"5min":1.907,"15min":1.907},"last":1.099}]},{"osd":1,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3300000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9330000000000001,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.98399999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.196,"15min":1.161},"min":{"1min":0.82999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0099999999999998,"5min":2.544,"15min":2.544},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.22,"15min":1.25},"min":{"1min":0.89400000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.538,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.5149999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2090000000000001,"15min":1.1830000000000001},"min":{"1min":0.755,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.504,"5min":1.867,"15min":1.867},"last":1.401},{"interface":"front","average":{"1min":1.282,"5min":1.262,"15min":1.355},"min":{"1min":0.99399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8360000000000001,"5min":2.927,"15min":2.9620000000000002},"last":1.2090000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2609999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.6739999999999999,"5min":2.0099999999999998,"15min":2.1970000000000001},"last":1.151},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.264,"15min":1.3009999999999999},"min":{"1min":0.52100000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.702,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.6759999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.82899999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.496,"15min":1.496},"min":{"1min":0.77200000000000002,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":2.169,"5min":2.169,"15min":2.5350000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4039999999999999,"15min":1.452},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.028,"5min":2.028,"15min":2.2509999999999999},"last":1.571},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4630000000000001,"15min":1.603},"min":{"1min":0.66700000000000004,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":1.974,"5min":2.0609999999999999,"15min":4.4329999999999998},"last":1.4990000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.0920000000000001,"15min":1.0129999999999999},"min":{"1min":0.69699999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1000000000000001,"15min":1.081},"min":{"1min":0.73899999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7769999999999999,"5min":1.859,"15min":1.859},"last":1.135}]},{"osd":1,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1379999999999999,"15min":1.1279999999999999},"min":{"1min":0.78100000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.139,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":1.3200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.133,"15min":1.2070000000000001},"min":{"1min":0.75700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4099999999999999,"5min":1.6930000000000001,"15min":1.974},"last":1.024},{"interface":"front","average":{"1min":1.323,"5min":1.224,"15min":1.196},"min":{"1min":0.78800000000000003,"5min":0.68200000000000005,"15min":0.67400000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.1819999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.329,"15min":1.369},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.835,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.452},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3480000000000001,"15min":1.3999999999999999},"min":{"1min":0.91200000000000003,"5min":0.874,"15min":0.874},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.7509999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.29},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":2.1459999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.343,"15min":1.393},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.8200000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.351,"15min":1.4510000000000001},"min":{"1min":0.85799999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.9950000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.401,"15min":1.482},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.8899999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.496,"15min":1.534},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.962},{"interface":"front","average":{"1min":1.663,"5min":1.575,"15min":1.583},"min":{"1min":0.93500000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":2.109}]}]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.85399999999999998},"min":{"1min":0.38700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.486,"5min":1.744,"15min":1.744},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.93200000000000005,"15min":0.91300000000000003},"min":{"1min":0.501,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.85599999999999998,"15min":0.89400000000000002},"min":{"1min":0.442,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95599999999999996,"15min":1.01},"min":{"1min":0.50600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.403,"5min":1.5720000000000001,"15min":1.9410000000000001},"last":1.127}]},{"osd":2,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.98099999999999998,"15min":1.0189999999999999},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.827},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.0620000000000001,"15min":1.0780000000000001},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4530000000000001,"5min":1.675,"15min":2.2549999999999999},"last":1.2769999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.181,"15min":1.2390000000000001},"min":{"1min":0.748,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.079},{"interface":"front","average":{"1min":1.21,"5min":1.194,"15min":1.2969999999999999},"min":{"1min":0.77800000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2609999999999999,"15min":1.3580000000000001},"min":{"1min":0.83999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7050000000000001,"5min":1.7390000000000001,"15min":2.5129999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.276,"15min":1.292},"min":{"1min":0.71299999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.958,"5min":1.958,"15min":2.3239999999999998},"last":1.3340000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3109999999999999,"15min":1.363},"min":{"1min":0.82499999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.296,"15min":1.403},"min":{"1min":0.94399999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.325,"15min":1.423},"min":{"1min":0.82599999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.651,"5min":1.9410000000000001,"15min":2.468},"last":1.601},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.4239999999999999},"min":{"1min":0.86799999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.508,"5min":2.508,"15min":2.798},"last":1.319}]}]},{"osd":2,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82499999999999996,"15min":0.82899999999999996},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.506,"5min":1.621,"15min":1.621},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6359999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.131,"15min":1.125},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.694},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2829999999999999,"15min":1.2290000000000001},"min":{"1min":0.82599999999999996,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":1.6970000000000001,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.216,"15min":1.23},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":1.8320000000000001,"15min":2.9510000000000001},"last":1.2070000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.8919999999999999,"5min":2.3180000000000001,"15min":2.8370000000000002},"last":1.012},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.216,"15min":1.264},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.837,"5min":1.9530000000000001,"15min":2.3039999999999998},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.2649999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.76200000000000001},"max":{"1min":1.9079999999999999,"5min":2.0230000000000001,"15min":2.3969999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.298,"15min":1.276},"min":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.7890000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3069999999999999,"15min":1.3500000000000001},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.74099999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.6269999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3759999999999999,"15min":1.3879999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.96,"5min":1.96,"15min":2.1600000000000001},"last":1.002}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78700000000000003,"15min":0.76400000000000001},"min":{"1min":0.38400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2589999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.88300000000000001,"15min":0.85099999999999998},"min":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2450000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.873,"15min":0.83499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.488,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.998,"15min":1},"min":{"1min":0.45200000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.3300000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0840000000000001,"15min":1.1020000000000001},"min":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5029999999999999,"5min":1.75,"15min":1.75},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.032,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1659999999999999,"15min":1.141},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.855},"last":0.877},{"interface":"front","average":{"1min":1.268,"5min":1.278,"15min":1.284},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.72,"5min":1.806,"15min":2.2789999999999999},"last":1.212}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2629999999999999,"15min":1.2689999999999999},"min":{"1min":0.76000000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2509999999999999,"15min":1.244},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.843,"5min":1.849,"15min":2.173},"last":1.167}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3720000000000001,"15min":1.3380000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.857,"5min":1.917,"15min":1.917},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3220000000000001,"15min":1.361},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.347}]},{"osd":7,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.357,"15min":1.363},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.3420000000000001,"15min":1.3520000000000001},"min":{"1min":0.84599999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.9279999999999999,"5min":1.9359999999999999,"15min":2.3879999999999999},"last":1.294}]}]},{"osd":0,"up_from":14,"seq":60129542225,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1896448,"data_stored":1159873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.075,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":0.88400000000000001},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.31900000000000001},"max":{"1min":1.5640000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":1.5720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.95499999999999996,"15min":0.93200000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2689999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.625},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.98799999999999999,"15min":0.96799999999999997},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.55,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.1259999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.92000000000000004,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35599999999999998},"max":{"1min":1.5700000000000001,"5min":1.976,"15min":1.976},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.087,"15min":1.103},"min":{"1min":0.51100000000000001,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":1.6080000000000001,"5min":2.157,"15min":2.157},"last":1.448}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.21,"15min":1.1719999999999999},"min":{"1min":0.77500000000000002,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3129999999999999,"15min":1.3069999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8520000000000001,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.331}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.274,"15min":1.268},"min":{"1min":0.746,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.893,"5min":2.105,"15min":2.3879999999999999},"last":1.492},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.238,"15min":1.204},"min":{"1min":0.81299999999999994,"5min":0.76600000000000001,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.296}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.387},"min":{"1min":0.878,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.349,"15min":1.375},"min":{"1min":0.874,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8120000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.401}]},{"osd":7,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.353,"15min":1.3440000000000001},"min":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0449999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.77000000000000002,"5min":0.75,"15min":0.68999999999999995},"max":{"1min":1.7509999999999999,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:27:36.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:27:36.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.792+0000 7fd9ba200640 1 Processor -- start 2024-04-05T22:27:36.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.792+0000 7fd9ba200640 1 -- start start 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.793+0000 7fd9ba200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9b4160620 0x7fd9b40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.793+0000 7fd9ba200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9b40cb2f0 0x7fd9b40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9ba200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40cbc70 con 0x7fd9b40b1870 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9ba200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40ad880 con 0x7fd9b4160620 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9ba200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40ce5d0 con 0x7fd9b40cb2f0 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9b4160620 0x7fd9b40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9b4160620 0x7fd9b40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52062/0 (socket says 172.21.15.73:52062) 2024-04-05T22:27:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 -- 172.21.15.73:0/213944353 learned_addr learned my addr 172.21.15.73:0/213944353 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:36.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 -- 172.21.15.73:0/213944353 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40cb2f0 msgr2=0x7fd9b40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 --2- 172.21.15.73:0/213944353 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40cb2f0 0x7fd9b40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 -- 172.21.15.73:0/213944353 >> v1:172.21.15.157:6789/0 conn(0x7fd9b40b1870 legacy=0x7fd9b40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.794+0000 7fd9b3400640 1 -- 172.21.15.73:0/213944353 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40ce010 con 0x7fd9b4160620 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.795+0000 7fd9b3400640 1 --2- 172.21.15.73:0/213944353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b4160620 0x7fd9b40caa60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406f1b0 tx=0x7fd9a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8494b0fc698e7061 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9b2a00640 1 -- 172.21.15.73:0/213944353 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031870 con 0x7fd9b4160620 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9b2a00640 1 -- 172.21.15.73:0/213944353 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40319d0 con 0x7fd9b4160620 2024-04-05T22:27:36.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9b2a00640 1 -- 172.21.15.73:0/213944353 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031d60 con 0x7fd9b4160620 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 -- 172.21.15.73:0/213944353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b4160620 msgr2=0x7fd9b40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 --2- 172.21.15.73:0/213944353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b4160620 0x7fd9b40caa60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406f1b0 tx=0x7fd9a402b910 comp rx=0 tx=0).stop 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 -- 172.21.15.73:0/213944353 shutdown_connections 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 --2- 172.21.15.73:0/213944353 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40cb2f0 0x7fd9b40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 --2- 172.21.15.73:0/213944353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b4160620 0x7fd9b40caa60 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 -- 172.21.15.73:0/213944353 >> 172.21.15.73:0/213944353 conn(0x7fd9b40ce8e0 msgr2=0x7fd9b40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 -- 172.21.15.73:0/213944353 shutdown_connections 2024-04-05T22:27:36.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.796+0000 7fd9ba200640 1 -- 172.21.15.73:0/213944353 wait complete. 2024-04-05T22:27:36.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.798+0000 7fd9ba200640 1 Processor -- start 2024-04-05T22:27:36.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.798+0000 7fd9ba200640 1 -- start start 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b40cb2f0 0x7fd9b40af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b4160620 0x7fd9b40af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415c3c0 con 0x7fd9b40b1870 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415d960 con 0x7fd9b4160620 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9ba200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b7b0 con 0x7fd9b40cb2f0 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52076/0 (socket says 172.21.15.73:52076) 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.799+0000 7fd9b3400640 1 -- 172.21.15.73:0/2725505246 learned_addr learned my addr 172.21.15.73:0/2725505246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 -- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b40cb2f0 msgr2=0x7fd9b40af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b40cb2f0 0x7fd9b40af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 -- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b4160620 msgr2=0x7fd9b40af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:36.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b4160620 0x7fd9b40af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 -- 172.21.15.73:0/2725505246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40094d0 con 0x7fd9b40b1870 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9b3400640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4000c00 tx=0x7fd9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031840 con 0x7fd9b40b1870 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40443f0 con 0x7fd9b40b1870 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40447c0 con 0x7fd9b40b1870 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.800+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b415ba90 con 0x7fd9b40b1870 2024-04-05T22:27:36.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.801+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b415c050 con 0x7fd9b40b1870 2024-04-05T22:27:36.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.802+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b4066af0 con 0x7fd9b40b1870 2024-04-05T22:27:36.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.803+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fd9a404d450 con 0x7fd9b40b1870 2024-04-05T22:27:36.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.804+0000 7fd9abe00640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 0x7fd99006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:36.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.804+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9a402d040 con 0x7fd9b40b1870 2024-04-05T22:27:36.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.804+0000 7fd9aaa00640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 0x7fd99006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:36.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.805+0000 7fd9aaa00640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 0x7fd99006d210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4154bc0 tx=0x7fd99c007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:36.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.808+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd9a4076d10 con 0x7fd9b40b1870 2024-04-05T22:27:36.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:36.999+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9b40acc50 con 0x7fd99006ad50 2024-04-05T22:27:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.022+0000 7fd9abe00640 1 -- 172.21.15.73:0/2725505246 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287348 (secure 0 0 0) 0x7fd99c0050d0 con 0x7fd99006ad50 2024-04-05T22:27:37.026 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 msgr2=0x7fd99006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 0x7fd99006d210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4154bc0 tx=0x7fd99c007990 comp rx=0 tx=0).stop 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 msgr2=0x7fd9b415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4000c00 tx=0x7fd9a40046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 shutdown_connections 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd99006ad50 0x7fd99006d210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9b4160620 0x7fd9b40af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b40cb2f0 0x7fd9b40af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 --2- 172.21.15.73:0/2725505246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9b40b1870 0x7fd9b415d420 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 >> 172.21.15.73:0/2725505246 conn(0x7fd9b40ce8e0 msgr2=0x7fd9b406fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:37.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 shutdown_connections 2024-04-05T22:27:37.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.029+0000 7fd9ba200640 1 -- 172.21.15.73:0/2725505246 wait complete. 2024-04-05T22:27:37.048 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-05T22:27:35.521324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260304,"kb_used_data":11356,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714416,"statfs":{"total":773094113280,"available":772827561984,"internally_reserved":0,"allocated":11628544,"data_stored":5744667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":254791227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.913060"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158217+0000","last_change":"2024-04-05T22:27:32.157635+0000","last_active":"2024-04-05T22:27:32.158217+0000","last_peered":"2024-04-05T22:27:32.158217+0000","last_clean":"2024-04-05T22:27:32.158217+0000","last_became_active":"2024-04-05T22:27:32.157328+0000","last_became_peered":"2024-04-05T22:27:32.157328+0000","last_unstale":"2024-04-05T22:27:32.158217+0000","last_undegraded":"2024-04-05T22:27:32.158217+0000","last_fullsized":"2024-04-05T22:27:32.158217+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:28:52.775014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.135700+0000","last_change":"2024-04-05T22:27:32.128480+0000","last_active":"2024-04-05T22:27:32.135700+0000","last_peered":"2024-04-05T22:27:32.135700+0000","last_clean":"2024-04-05T22:27:32.135700+0000","last_became_active":"2024-04-05T22:27:32.127744+0000","last_became_peered":"2024-04-05T22:27:32.127744+0000","last_unstale":"2024-04-05T22:27:32.135700+0000","last_undegraded":"2024-04-05T22:27:32.135700+0000","last_fullsized":"2024-04-05T22:27:32.135700+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:21.246110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159011+0000","last_change":"2024-04-05T22:27:32.154608+0000","last_active":"2024-04-05T22:27:32.159011+0000","last_peered":"2024-04-05T22:27:32.159011+0000","last_clean":"2024-04-05T22:27:32.159011+0000","last_became_active":"2024-04-05T22:27:32.152098+0000","last_became_peered":"2024-04-05T22:27:32.152098+0000","last_unstale":"2024-04-05T22:27:32.159011+0000","last_undegraded":"2024-04-05T22:27:32.159011+0000","last_fullsized":"2024-04-05T22:27:32.159011+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:36.855438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146212+0000","last_change":"2024-04-05T22:27:32.140555+0000","last_active":"2024-04-05T22:27:32.146212+0000","last_peered":"2024-04-05T22:27:32.146212+0000","last_clean":"2024-04-05T22:27:32.146212+0000","last_became_active":"2024-04-05T22:27:32.139526+0000","last_became_peered":"2024-04-05T22:27:32.139526+0000","last_unstale":"2024-04-05T22:27:32.146212+0000","last_undegraded":"2024-04-05T22:27:32.146212+0000","last_fullsized":"2024-04-05T22:27:32.146212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:13.650988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144201+0000","last_change":"2024-04-05T22:27:32.136309+0000","last_active":"2024-04-05T22:27:32.144201+0000","last_peered":"2024-04-05T22:27:32.144201+0000","last_clean":"2024-04-05T22:27:32.144201+0000","last_became_active":"2024-04-05T22:27:32.135855+0000","last_became_peered":"2024-04-05T22:27:32.135855+0000","last_unstale":"2024-04-05T22:27:32.144201+0000","last_undegraded":"2024-04-05T22:27:32.144201+0000","last_fullsized":"2024-04-05T22:27:32.144201+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:44.881691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140088+0000","last_change":"2024-04-05T22:27:32.132707+0000","last_active":"2024-04-05T22:27:32.140088+0000","last_peered":"2024-04-05T22:27:32.140088+0000","last_clean":"2024-04-05T22:27:32.140088+0000","last_became_active":"2024-04-05T22:27:32.132464+0000","last_became_peered":"2024-04-05T22:27:32.132464+0000","last_unstale":"2024-04-05T22:27:32.140088+0000","last_undegraded":"2024-04-05T22:27:32.140088+0000","last_fullsized":"2024-04-05T22:27:32.140088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:20:01.110274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.152865+0000","last_change":"2024-04-05T22:27:32.149061+0000","last_active":"2024-04-05T22:27:32.152865+0000","last_peered":"2024-04-05T22:27:32.152865+0000","last_clean":"2024-04-05T22:27:32.152865+0000","last_became_active":"2024-04-05T22:27:32.148768+0000","last_became_peered":"2024-04-05T22:27:32.148768+0000","last_unstale":"2024-04-05T22:27:32.152865+0000","last_undegraded":"2024-04-05T22:27:32.152865+0000","last_fullsized":"2024-04-05T22:27:32.152865+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:57.461648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157998+0000","last_change":"2024-04-05T22:27:32.152096+0000","last_active":"2024-04-05T22:27:32.157998+0000","last_peered":"2024-04-05T22:27:32.157998+0000","last_clean":"2024-04-05T22:27:32.157998+0000","last_became_active":"2024-04-05T22:27:32.151133+0000","last_became_peered":"2024-04-05T22:27:32.151133+0000","last_unstale":"2024-04-05T22:27:32.157998+0000","last_undegraded":"2024-04-05T22:27:32.157998+0000","last_fullsized":"2024-04-05T22:27:32.157998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:35:08.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138637+0000","last_change":"2024-04-05T22:27:32.133509+0000","last_active":"2024-04-05T22:27:32.138637+0000","last_peered":"2024-04-05T22:27:32.138637+0000","last_clean":"2024-04-05T22:27:32.138637+0000","last_became_active":"2024-04-05T22:27:32.131818+0000","last_became_peered":"2024-04-05T22:27:32.131818+0000","last_unstale":"2024-04-05T22:27:32.138637+0000","last_undegraded":"2024-04-05T22:27:32.138637+0000","last_fullsized":"2024-04-05T22:27:32.138637+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:59:43.241703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.136324+0000","last_change":"2024-04-05T22:27:32.128788+0000","last_active":"2024-04-05T22:27:32.136324+0000","last_peered":"2024-04-05T22:27:32.136324+0000","last_clean":"2024-04-05T22:27:32.136324+0000","last_became_active":"2024-04-05T22:27:32.128100+0000","last_became_peered":"2024-04-05T22:27:32.128100+0000","last_unstale":"2024-04-05T22:27:32.136324+0000","last_undegraded":"2024-04-05T22:27:32.136324+0000","last_fullsized":"2024-04-05T22:27:32.136324+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:37:45.504548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158348+0000","last_change":"2024-04-05T22:27:32.150306+0000","last_active":"2024-04-05T22:27:32.158348+0000","last_peered":"2024-04-05T22:27:32.158348+0000","last_clean":"2024-04-05T22:27:32.158348+0000","last_became_active":"2024-04-05T22:27:32.150037+0000","last_became_peered":"2024-04-05T22:27:32.150037+0000","last_unstale":"2024-04-05T22:27:32.158348+0000","last_undegraded":"2024-04-05T22:27:32.158348+0000","last_fullsized":"2024-04-05T22:27:32.158348+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:26.515897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.141800+0000","last_change":"2024-04-05T22:27:32.130409+0000","last_active":"2024-04-05T22:27:32.141800+0000","last_peered":"2024-04-05T22:27:32.141800+0000","last_clean":"2024-04-05T22:27:32.141800+0000","last_became_active":"2024-04-05T22:27:32.129153+0000","last_became_peered":"2024-04-05T22:27:32.129153+0000","last_unstale":"2024-04-05T22:27:32.141800+0000","last_undegraded":"2024-04-05T22:27:32.141800+0000","last_fullsized":"2024-04-05T22:27:32.141800+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:16:42.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157102+0000","last_change":"2024-04-05T22:27:32.150741+0000","last_active":"2024-04-05T22:27:32.157102+0000","last_peered":"2024-04-05T22:27:32.157102+0000","last_clean":"2024-04-05T22:27:32.157102+0000","last_became_active":"2024-04-05T22:27:32.150463+0000","last_became_peered":"2024-04-05T22:27:32.150463+0000","last_unstale":"2024-04-05T22:27:32.157102+0000","last_undegraded":"2024-04-05T22:27:32.157102+0000","last_fullsized":"2024-04-05T22:27:32.157102+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:37:04.887872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157867+0000","last_change":"2024-04-05T22:27:32.150532+0000","last_active":"2024-04-05T22:27:32.157867+0000","last_peered":"2024-04-05T22:27:32.157867+0000","last_clean":"2024-04-05T22:27:32.157867+0000","last_became_active":"2024-04-05T22:27:32.150193+0000","last_became_peered":"2024-04-05T22:27:32.150193+0000","last_unstale":"2024-04-05T22:27:32.157867+0000","last_undegraded":"2024-04-05T22:27:32.157867+0000","last_fullsized":"2024-04-05T22:27:32.157867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140447+0000","last_change":"2024-04-05T22:27:32.134048+0000","last_active":"2024-04-05T22:27:32.140447+0000","last_peered":"2024-04-05T22:27:32.140447+0000","last_clean":"2024-04-05T22:27:32.140447+0000","last_became_active":"2024-04-05T22:27:32.133797+0000","last_became_peered":"2024-04-05T22:27:32.133797+0000","last_unstale":"2024-04-05T22:27:32.140447+0000","last_undegraded":"2024-04-05T22:27:32.140447+0000","last_fullsized":"2024-04-05T22:27:32.140447+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:22:57.461648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157944+0000","last_change":"2024-04-05T22:27:32.153109+0000","last_active":"2024-04-05T22:27:32.157944+0000","last_peered":"2024-04-05T22:27:32.157944+0000","last_clean":"2024-04-05T22:27:32.157944+0000","last_became_active":"2024-04-05T22:27:32.152412+0000","last_became_peered":"2024-04-05T22:27:32.152412+0000","last_unstale":"2024-04-05T22:27:32.157944+0000","last_undegraded":"2024-04-05T22:27:32.157944+0000","last_fullsized":"2024-04-05T22:27:32.157944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:35:25.167803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159705+0000","last_change":"2024-04-05T22:27:32.158134+0000","last_active":"2024-04-05T22:27:32.159705+0000","last_peered":"2024-04-05T22:27:32.159705+0000","last_clean":"2024-04-05T22:27:32.159705+0000","last_became_active":"2024-04-05T22:27:32.157852+0000","last_became_peered":"2024-04-05T22:27:32.157852+0000","last_unstale":"2024-04-05T22:27:32.159705+0000","last_undegraded":"2024-04-05T22:27:32.159705+0000","last_fullsized":"2024-04-05T22:27:32.159705+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:25.013402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.156419+0000","last_change":"2024-04-05T22:27:32.149242+0000","last_active":"2024-04-05T22:27:32.156419+0000","last_peered":"2024-04-05T22:27:32.156419+0000","last_clean":"2024-04-05T22:27:32.156419+0000","last_became_active":"2024-04-05T22:27:32.148981+0000","last_became_peered":"2024-04-05T22:27:32.148981+0000","last_unstale":"2024-04-05T22:27:32.156419+0000","last_undegraded":"2024-04-05T22:27:32.156419+0000","last_fullsized":"2024-04-05T22:27:32.156419+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:47:36.810811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159053+0000","last_change":"2024-04-05T22:27:32.155369+0000","last_active":"2024-04-05T22:27:32.159053+0000","last_peered":"2024-04-05T22:27:32.159053+0000","last_clean":"2024-04-05T22:27:32.159053+0000","last_became_active":"2024-04-05T22:27:32.154861+0000","last_became_peered":"2024-04-05T22:27:32.154861+0000","last_unstale":"2024-04-05T22:27:32.159053+0000","last_undegraded":"2024-04-05T22:27:32.159053+0000","last_fullsized":"2024-04-05T22:27:32.159053+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:17:43.310001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159061+0000","last_change":"2024-04-05T22:27:32.153912+0000","last_active":"2024-04-05T22:27:32.159061+0000","last_peered":"2024-04-05T22:27:32.159061+0000","last_clean":"2024-04-05T22:27:32.159061+0000","last_became_active":"2024-04-05T22:27:32.153633+0000","last_became_peered":"2024-04-05T22:27:32.153633+0000","last_unstale":"2024-04-05T22:27:32.159061+0000","last_undegraded":"2024-04-05T22:27:32.159061+0000","last_fullsized":"2024-04-05T22:27:32.159061+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:35:08.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144125+0000","last_change":"2024-04-05T22:27:32.134200+0000","last_active":"2024-04-05T22:27:32.144125+0000","last_peered":"2024-04-05T22:27:32.144125+0000","last_clean":"2024-04-05T22:27:32.144125+0000","last_became_active":"2024-04-05T22:27:32.132856+0000","last_became_peered":"2024-04-05T22:27:32.132856+0000","last_unstale":"2024-04-05T22:27:32.144125+0000","last_undegraded":"2024-04-05T22:27:32.144125+0000","last_fullsized":"2024-04-05T22:27:32.144125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:35:35.331299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.153023+0000","last_change":"2024-04-05T22:27:32.148948+0000","last_active":"2024-04-05T22:27:32.153023+0000","last_peered":"2024-04-05T22:27:32.153023+0000","last_clean":"2024-04-05T22:27:32.153023+0000","last_became_active":"2024-04-05T22:27:32.148720+0000","last_became_peered":"2024-04-05T22:27:32.148720+0000","last_unstale":"2024-04-05T22:27:32.153023+0000","last_undegraded":"2024-04-05T22:27:32.153023+0000","last_fullsized":"2024-04-05T22:27:32.153023+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:22:04.714703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159026+0000","last_change":"2024-04-05T22:27:32.153105+0000","last_active":"2024-04-05T22:27:32.159026+0000","last_peered":"2024-04-05T22:27:32.159026+0000","last_clean":"2024-04-05T22:27:32.159026+0000","last_became_active":"2024-04-05T22:27:32.152834+0000","last_became_peered":"2024-04-05T22:27:32.152834+0000","last_unstale":"2024-04-05T22:27:32.159026+0000","last_undegraded":"2024-04-05T22:27:32.159026+0000","last_fullsized":"2024-04-05T22:27:32.159026+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:38:58.315925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157359+0000","last_change":"2024-04-05T22:27:32.151986+0000","last_active":"2024-04-05T22:27:32.157359+0000","last_peered":"2024-04-05T22:27:32.157359+0000","last_clean":"2024-04-05T22:27:32.157359+0000","last_became_active":"2024-04-05T22:27:32.151725+0000","last_became_peered":"2024-04-05T22:27:32.151725+0000","last_unstale":"2024-04-05T22:27:32.157359+0000","last_undegraded":"2024-04-05T22:27:32.157359+0000","last_fullsized":"2024-04-05T22:27:32.157359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:49:44.804179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158218+0000","last_change":"2024-04-05T22:27:32.149943+0000","last_active":"2024-04-05T22:27:32.158218+0000","last_peered":"2024-04-05T22:27:32.158218+0000","last_clean":"2024-04-05T22:27:32.158218+0000","last_became_active":"2024-04-05T22:27:32.149602+0000","last_became_peered":"2024-04-05T22:27:32.149602+0000","last_unstale":"2024-04-05T22:27:32.158218+0000","last_undegraded":"2024-04-05T22:27:32.158218+0000","last_fullsized":"2024-04-05T22:27:32.158218+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140119+0000","last_change":"2024-04-05T22:27:32.137693+0000","last_active":"2024-04-05T22:27:32.140119+0000","last_peered":"2024-04-05T22:27:32.140119+0000","last_clean":"2024-04-05T22:27:32.140119+0000","last_became_active":"2024-04-05T22:27:32.137422+0000","last_became_peered":"2024-04-05T22:27:32.137422+0000","last_unstale":"2024-04-05T22:27:32.140119+0000","last_undegraded":"2024-04-05T22:27:32.140119+0000","last_fullsized":"2024-04-05T22:27:32.140119+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:50:49.869635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138442+0000","last_change":"2024-04-05T22:27:32.128032+0000","last_active":"2024-04-05T22:27:32.138442+0000","last_peered":"2024-04-05T22:27:32.138442+0000","last_clean":"2024-04-05T22:27:32.138442+0000","last_became_active":"2024-04-05T22:27:32.127767+0000","last_became_peered":"2024-04-05T22:27:32.127767+0000","last_unstale":"2024-04-05T22:27:32.138442+0000","last_undegraded":"2024-04-05T22:27:32.138442+0000","last_fullsized":"2024-04-05T22:27:32.138442+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:01.524104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143729+0000","last_change":"2024-04-05T22:27:32.134496+0000","last_active":"2024-04-05T22:27:32.143729+0000","last_peered":"2024-04-05T22:27:32.143729+0000","last_clean":"2024-04-05T22:27:32.143729+0000","last_became_active":"2024-04-05T22:27:32.134230+0000","last_became_peered":"2024-04-05T22:27:32.134230+0000","last_unstale":"2024-04-05T22:27:32.143729+0000","last_undegraded":"2024-04-05T22:27:32.143729+0000","last_fullsized":"2024-04-05T22:27:32.143729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:29:35.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154077+0000","last_change":"2024-04-05T22:27:32.148558+0000","last_active":"2024-04-05T22:27:32.154077+0000","last_peered":"2024-04-05T22:27:32.154077+0000","last_clean":"2024-04-05T22:27:32.154077+0000","last_became_active":"2024-04-05T22:27:32.148017+0000","last_became_peered":"2024-04-05T22:27:32.148017+0000","last_unstale":"2024-04-05T22:27:32.154077+0000","last_undegraded":"2024-04-05T22:27:32.154077+0000","last_fullsized":"2024-04-05T22:27:32.154077+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:17:43.310001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140967+0000","last_change":"2024-04-05T22:27:32.137556+0000","last_active":"2024-04-05T22:27:32.140967+0000","last_peered":"2024-04-05T22:27:32.140967+0000","last_clean":"2024-04-05T22:27:32.140967+0000","last_became_active":"2024-04-05T22:27:32.137060+0000","last_became_peered":"2024-04-05T22:27:32.137060+0000","last_unstale":"2024-04-05T22:27:32.140967+0000","last_undegraded":"2024-04-05T22:27:32.140967+0000","last_fullsized":"2024-04-05T22:27:32.140967+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:29.910437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.141350+0000","last_change":"2024-04-05T22:27:32.136505+0000","last_active":"2024-04-05T22:27:32.141350+0000","last_peered":"2024-04-05T22:27:32.141350+0000","last_clean":"2024-04-05T22:27:32.141350+0000","last_became_active":"2024-04-05T22:27:32.136266+0000","last_became_peered":"2024-04-05T22:27:32.136266+0000","last_unstale":"2024-04-05T22:27:32.141350+0000","last_undegraded":"2024-04-05T22:27:32.141350+0000","last_fullsized":"2024-04-05T22:27:32.141350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:50:48.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.140150+0000","last_change":"2024-04-05T22:27:32.133312+0000","last_active":"2024-04-05T22:27:32.140150+0000","last_peered":"2024-04-05T22:27:32.140150+0000","last_clean":"2024-04-05T22:27:32.140150+0000","last_became_active":"2024-04-05T22:27:32.132649+0000","last_became_peered":"2024-04-05T22:27:32.132649+0000","last_unstale":"2024-04-05T22:27:32.140150+0000","last_undegraded":"2024-04-05T22:27:32.140150+0000","last_fullsized":"2024-04-05T22:27:32.140150+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.139752+0000","last_change":"2024-04-05T22:27:32.129501+0000","last_active":"2024-04-05T22:27:32.139752+0000","last_peered":"2024-04-05T22:27:32.139752+0000","last_clean":"2024-04-05T22:27:32.139752+0000","last_became_active":"2024-04-05T22:27:32.129197+0000","last_became_peered":"2024-04-05T22:27:32.129197+0000","last_unstale":"2024-04-05T22:27:32.139752+0000","last_undegraded":"2024-04-05T22:27:32.139752+0000","last_fullsized":"2024-04-05T22:27:32.139752+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:36:41.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154787+0000","last_change":"2024-04-05T22:27:31.660851+0000","last_active":"2024-04-05T22:27:32.154787+0000","last_peered":"2024-04-05T22:27:32.154787+0000","last_clean":"2024-04-05T22:27:32.154787+0000","last_became_active":"2024-04-05T22:27:31.660534+0000","last_became_peered":"2024-04-05T22:27:31.660534+0000","last_unstale":"2024-04-05T22:27:32.154787+0000","last_undegraded":"2024-04-05T22:27:32.154787+0000","last_fullsized":"2024-04-05T22:27:32.154787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:32.947166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158936+0000","last_change":"2024-04-05T22:27:32.157085+0000","last_active":"2024-04-05T22:27:32.158936+0000","last_peered":"2024-04-05T22:27:32.158936+0000","last_clean":"2024-04-05T22:27:32.158936+0000","last_became_active":"2024-04-05T22:27:32.156503+0000","last_became_peered":"2024-04-05T22:27:32.156503+0000","last_unstale":"2024-04-05T22:27:32.158936+0000","last_undegraded":"2024-04-05T22:27:32.158936+0000","last_fullsized":"2024-04-05T22:27:32.158936+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:20:15.117257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.151117+0000","last_change":"2024-04-05T22:27:31.663795+0000","last_active":"2024-04-05T22:27:32.151117+0000","last_peered":"2024-04-05T22:27:32.151117+0000","last_clean":"2024-04-05T22:27:32.151117+0000","last_became_active":"2024-04-05T22:27:31.663319+0000","last_became_peered":"2024-04-05T22:27:31.663319+0000","last_unstale":"2024-04-05T22:27:32.151117+0000","last_undegraded":"2024-04-05T22:27:32.151117+0000","last_fullsized":"2024-04-05T22:27:32.151117+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:06.096111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143180+0000","last_change":"2024-04-05T22:27:32.140848+0000","last_active":"2024-04-05T22:27:32.143180+0000","last_peered":"2024-04-05T22:27:32.143180+0000","last_clean":"2024-04-05T22:27:32.143180+0000","last_became_active":"2024-04-05T22:27:32.139745+0000","last_became_peered":"2024-04-05T22:27:32.139745+0000","last_unstale":"2024-04-05T22:27:32.143180+0000","last_undegraded":"2024-04-05T22:27:32.143180+0000","last_fullsized":"2024-04-05T22:27:32.143180+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:29.910437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144536+0000","last_change":"2024-04-05T22:27:31.653855+0000","last_active":"2024-04-05T22:27:32.144536+0000","last_peered":"2024-04-05T22:27:32.144536+0000","last_clean":"2024-04-05T22:27:32.144536+0000","last_became_active":"2024-04-05T22:27:31.653563+0000","last_became_peered":"2024-04-05T22:27:31.653563+0000","last_unstale":"2024-04-05T22:27:32.144536+0000","last_undegraded":"2024-04-05T22:27:32.144536+0000","last_fullsized":"2024-04-05T22:27:32.144536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:37:20.277508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159033+0000","last_change":"2024-04-05T22:27:32.155118+0000","last_active":"2024-04-05T22:27:32.159033+0000","last_peered":"2024-04-05T22:27:32.159033+0000","last_clean":"2024-04-05T22:27:32.159033+0000","last_became_active":"2024-04-05T22:27:32.154327+0000","last_became_peered":"2024-04-05T22:27:32.154327+0000","last_unstale":"2024-04-05T22:27:32.159033+0000","last_undegraded":"2024-04-05T22:27:32.159033+0000","last_fullsized":"2024-04-05T22:27:32.159033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:38:58.315925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.145764+0000","last_change":"2024-04-05T22:27:31.649498+0000","last_active":"2024-04-05T22:27:32.145764+0000","last_peered":"2024-04-05T22:27:32.145764+0000","last_clean":"2024-04-05T22:27:32.145764+0000","last_became_active":"2024-04-05T22:27:31.647928+0000","last_became_peered":"2024-04-05T22:27:31.647928+0000","last_unstale":"2024-04-05T22:27:32.145764+0000","last_undegraded":"2024-04-05T22:27:32.145764+0000","last_fullsized":"2024-04-05T22:27:32.145764+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.139259+0000","last_change":"2024-04-05T22:27:32.133816+0000","last_active":"2024-04-05T22:27:32.139259+0000","last_peered":"2024-04-05T22:27:32.139259+0000","last_clean":"2024-04-05T22:27:32.139259+0000","last_became_active":"2024-04-05T22:27:32.133542+0000","last_became_peered":"2024-04-05T22:27:32.133542+0000","last_unstale":"2024-04-05T22:27:32.139259+0000","last_undegraded":"2024-04-05T22:27:32.139259+0000","last_fullsized":"2024-04-05T22:27:32.139259+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:46.460747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146398+0000","last_change":"2024-04-05T22:27:31.651770+0000","last_active":"2024-04-05T22:27:32.146398+0000","last_peered":"2024-04-05T22:27:32.146398+0000","last_clean":"2024-04-05T22:27:32.146398+0000","last_became_active":"2024-04-05T22:27:31.648415+0000","last_became_peered":"2024-04-05T22:27:31.648415+0000","last_unstale":"2024-04-05T22:27:32.146398+0000","last_undegraded":"2024-04-05T22:27:32.146398+0000","last_fullsized":"2024-04-05T22:27:32.146398+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144142+0000","last_change":"2024-04-05T22:27:32.134502+0000","last_active":"2024-04-05T22:27:32.144142+0000","last_peered":"2024-04-05T22:27:32.144142+0000","last_clean":"2024-04-05T22:27:32.144142+0000","last_became_active":"2024-04-05T22:27:32.133666+0000","last_became_peered":"2024-04-05T22:27:32.133666+0000","last_unstale":"2024-04-05T22:27:32.144142+0000","last_undegraded":"2024-04-05T22:27:32.144142+0000","last_fullsized":"2024-04-05T22:27:32.144142+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:52.724372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146490+0000","last_change":"2024-04-05T22:27:31.654819+0000","last_active":"2024-04-05T22:27:32.146490+0000","last_peered":"2024-04-05T22:27:32.146490+0000","last_clean":"2024-04-05T22:27:32.146490+0000","last_became_active":"2024-04-05T22:27:31.654476+0000","last_became_peered":"2024-04-05T22:27:31.654476+0000","last_unstale":"2024-04-05T22:27:32.146490+0000","last_undegraded":"2024-04-05T22:27:32.146490+0000","last_fullsized":"2024-04-05T22:27:32.146490+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:34.557930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.131616+0000","last_change":"2024-04-05T22:27:32.125645+0000","last_active":"2024-04-05T22:27:32.131616+0000","last_peered":"2024-04-05T22:27:32.131616+0000","last_clean":"2024-04-05T22:27:32.131616+0000","last_became_active":"2024-04-05T22:27:32.124429+0000","last_became_peered":"2024-04-05T22:27:32.124429+0000","last_unstale":"2024-04-05T22:27:32.131616+0000","last_undegraded":"2024-04-05T22:27:32.131616+0000","last_fullsized":"2024-04-05T22:27:32.131616+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:22:04.714703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122685+0000","last_change":"2024-04-05T22:27:31.656501+0000","last_active":"2024-04-05T22:27:32.122685+0000","last_peered":"2024-04-05T22:27:32.122685+0000","last_clean":"2024-04-05T22:27:32.122685+0000","last_became_active":"2024-04-05T22:27:31.656225+0000","last_became_peered":"2024-04-05T22:27:31.656225+0000","last_unstale":"2024-04-05T22:27:32.122685+0000","last_undegraded":"2024-04-05T22:27:32.122685+0000","last_fullsized":"2024-04-05T22:27:32.122685+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:00.512191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.152884+0000","last_change":"2024-04-05T22:27:32.149318+0000","last_active":"2024-04-05T22:27:32.152884+0000","last_peered":"2024-04-05T22:27:32.152884+0000","last_clean":"2024-04-05T22:27:32.152884+0000","last_became_active":"2024-04-05T22:27:32.148943+0000","last_became_peered":"2024-04-05T22:27:32.148943+0000","last_unstale":"2024-04-05T22:27:32.152884+0000","last_undegraded":"2024-04-05T22:27:32.152884+0000","last_fullsized":"2024-04-05T22:27:32.152884+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:20:01.110274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113566+0000","last_change":"2024-04-05T22:27:31.654830+0000","last_active":"2024-04-05T22:27:32.113566+0000","last_peered":"2024-04-05T22:27:32.113566+0000","last_clean":"2024-04-05T22:27:32.113566+0000","last_became_active":"2024-04-05T22:27:31.654260+0000","last_became_peered":"2024-04-05T22:27:31.654260+0000","last_unstale":"2024-04-05T22:27:32.113566+0000","last_undegraded":"2024-04-05T22:27:32.113566+0000","last_fullsized":"2024-04-05T22:27:32.113566+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:25.503984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.150564+0000","last_change":"2024-04-05T22:27:32.144118+0000","last_active":"2024-04-05T22:27:32.150564+0000","last_peered":"2024-04-05T22:27:32.150564+0000","last_clean":"2024-04-05T22:27:32.150564+0000","last_became_active":"2024-04-05T22:27:32.143842+0000","last_became_peered":"2024-04-05T22:27:32.143842+0000","last_unstale":"2024-04-05T22:27:32.150564+0000","last_undegraded":"2024-04-05T22:27:32.150564+0000","last_fullsized":"2024-04-05T22:27:32.150564+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:44:57.857859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.125436+0000","last_change":"2024-04-05T22:27:31.649844+0000","last_active":"2024-04-05T22:27:32.125436+0000","last_peered":"2024-04-05T22:27:32.125436+0000","last_clean":"2024-04-05T22:27:32.125436+0000","last_became_active":"2024-04-05T22:27:31.649587+0000","last_became_peered":"2024-04-05T22:27:31.649587+0000","last_unstale":"2024-04-05T22:27:32.125436+0000","last_undegraded":"2024-04-05T22:27:32.125436+0000","last_fullsized":"2024-04-05T22:27:32.125436+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.675407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157577+0000","last_change":"2024-04-05T22:27:32.152471+0000","last_active":"2024-04-05T22:27:32.157577+0000","last_peered":"2024-04-05T22:27:32.157577+0000","last_clean":"2024-04-05T22:27:32.157577+0000","last_became_active":"2024-04-05T22:27:32.152179+0000","last_became_peered":"2024-04-05T22:27:32.152179+0000","last_unstale":"2024-04-05T22:27:32.157577+0000","last_undegraded":"2024-04-05T22:27:32.157577+0000","last_fullsized":"2024-04-05T22:27:32.157577+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:36.175015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.124261+0000","last_change":"2024-04-05T22:27:31.661594+0000","last_active":"2024-04-05T22:27:32.124261+0000","last_peered":"2024-04-05T22:27:32.124261+0000","last_clean":"2024-04-05T22:27:32.124261+0000","last_became_active":"2024-04-05T22:27:31.661326+0000","last_became_peered":"2024-04-05T22:27:31.661326+0000","last_unstale":"2024-04-05T22:27:32.124261+0000","last_undegraded":"2024-04-05T22:27:32.124261+0000","last_fullsized":"2024-04-05T22:27:32.124261+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:24.001489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143873+0000","last_change":"2024-04-05T22:27:32.136804+0000","last_active":"2024-04-05T22:27:32.143873+0000","last_peered":"2024-04-05T22:27:32.143873+0000","last_clean":"2024-04-05T22:27:32.143873+0000","last_became_active":"2024-04-05T22:27:32.136577+0000","last_became_peered":"2024-04-05T22:27:32.136577+0000","last_unstale":"2024-04-05T22:27:32.143873+0000","last_undegraded":"2024-04-05T22:27:32.143873+0000","last_fullsized":"2024-04-05T22:27:32.143873+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:57:46.503662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143478+0000","last_change":"2024-04-05T22:27:31.650080+0000","last_active":"2024-04-05T22:27:32.143478+0000","last_peered":"2024-04-05T22:27:32.143478+0000","last_clean":"2024-04-05T22:27:32.143478+0000","last_became_active":"2024-04-05T22:27:31.649371+0000","last_became_peered":"2024-04-05T22:27:31.649371+0000","last_unstale":"2024-04-05T22:27:32.143478+0000","last_undegraded":"2024-04-05T22:27:32.143478+0000","last_fullsized":"2024-04-05T22:27:32.143478+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:48:34.557930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.155092+0000","last_change":"2024-04-05T22:27:32.153957+0000","last_active":"2024-04-05T22:27:32.155092+0000","last_peered":"2024-04-05T22:27:32.155092+0000","last_clean":"2024-04-05T22:27:32.155092+0000","last_became_active":"2024-04-05T22:27:32.153675+0000","last_became_peered":"2024-04-05T22:27:32.153675+0000","last_unstale":"2024-04-05T22:27:32.155092+0000","last_undegraded":"2024-04-05T22:27:32.155092+0000","last_fullsized":"2024-04-05T22:27:32.155092+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:44:44.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.150118+0000","last_change":"2024-04-05T22:27:31.652577+0000","last_active":"2024-04-05T22:27:32.150118+0000","last_peered":"2024-04-05T22:27:32.150118+0000","last_clean":"2024-04-05T22:27:32.150118+0000","last_became_active":"2024-04-05T22:27:31.652248+0000","last_became_peered":"2024-04-05T22:27:31.652248+0000","last_unstale":"2024-04-05T22:27:32.150118+0000","last_undegraded":"2024-04-05T22:27:32.150118+0000","last_fullsized":"2024-04-05T22:27:32.150118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:18:36.909692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144083+0000","last_change":"2024-04-05T22:27:32.138144+0000","last_active":"2024-04-05T22:27:32.144083+0000","last_peered":"2024-04-05T22:27:32.144083+0000","last_clean":"2024-04-05T22:27:32.144083+0000","last_became_active":"2024-04-05T22:27:32.137808+0000","last_became_peered":"2024-04-05T22:27:32.137808+0000","last_unstale":"2024-04-05T22:27:32.144083+0000","last_undegraded":"2024-04-05T22:27:32.144083+0000","last_fullsized":"2024-04-05T22:27:32.144083+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:35.096886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.153369+0000","last_change":"2024-04-05T22:27:31.662465+0000","last_active":"2024-04-05T22:27:32.153369+0000","last_peered":"2024-04-05T22:27:32.153369+0000","last_clean":"2024-04-05T22:27:32.153369+0000","last_became_active":"2024-04-05T22:27:31.661824+0000","last_became_peered":"2024-04-05T22:27:31.661824+0000","last_unstale":"2024-04-05T22:27:32.153369+0000","last_undegraded":"2024-04-05T22:27:32.153369+0000","last_fullsized":"2024-04-05T22:27:32.153369+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:57.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157724+0000","last_change":"2024-04-05T22:27:32.155119+0000","last_active":"2024-04-05T22:27:32.157724+0000","last_peered":"2024-04-05T22:27:32.157724+0000","last_clean":"2024-04-05T22:27:32.157724+0000","last_became_active":"2024-04-05T22:27:32.154632+0000","last_became_peered":"2024-04-05T22:27:32.154632+0000","last_unstale":"2024-04-05T22:27:32.157724+0000","last_undegraded":"2024-04-05T22:27:32.157724+0000","last_fullsized":"2024-04-05T22:27:32.157724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:56:14.562747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143983+0000","last_change":"2024-04-05T22:27:31.658087+0000","last_active":"2024-04-05T22:27:32.143983+0000","last_peered":"2024-04-05T22:27:32.143983+0000","last_clean":"2024-04-05T22:27:32.143983+0000","last_became_active":"2024-04-05T22:27:31.657595+0000","last_became_peered":"2024-04-05T22:27:31.657595+0000","last_unstale":"2024-04-05T22:27:32.143983+0000","last_undegraded":"2024-04-05T22:27:32.143983+0000","last_fullsized":"2024-04-05T22:27:32.143983+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:13.300136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144822+0000","last_change":"2024-04-05T22:27:32.137673+0000","last_active":"2024-04-05T22:27:32.144822+0000","last_peered":"2024-04-05T22:27:32.144822+0000","last_clean":"2024-04-05T22:27:32.144822+0000","last_became_active":"2024-04-05T22:27:32.136510+0000","last_became_peered":"2024-04-05T22:27:32.136510+0000","last_unstale":"2024-04-05T22:27:32.144822+0000","last_undegraded":"2024-04-05T22:27:32.144822+0000","last_fullsized":"2024-04-05T22:27:32.144822+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:04:06.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.147391+0000","last_change":"2024-04-05T22:27:31.658771+0000","last_active":"2024-04-05T22:27:32.147391+0000","last_peered":"2024-04-05T22:27:32.147391+0000","last_clean":"2024-04-05T22:27:32.147391+0000","last_became_active":"2024-04-05T22:27:31.657722+0000","last_became_peered":"2024-04-05T22:27:31.657722+0000","last_unstale":"2024-04-05T22:27:32.147391+0000","last_undegraded":"2024-04-05T22:27:32.147391+0000","last_fullsized":"2024-04-05T22:27:32.147391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:23:41.153422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.131968+0000","last_change":"2024-04-05T22:27:32.128073+0000","last_active":"2024-04-05T22:27:32.131968+0000","last_peered":"2024-04-05T22:27:32.131968+0000","last_clean":"2024-04-05T22:27:32.131968+0000","last_became_active":"2024-04-05T22:27:32.127693+0000","last_became_peered":"2024-04-05T22:27:32.127693+0000","last_unstale":"2024-04-05T22:27:32.131968+0000","last_undegraded":"2024-04-05T22:27:32.131968+0000","last_fullsized":"2024-04-05T22:27:32.131968+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:18:04.839710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146927+0000","last_change":"2024-04-05T22:27:31.657566+0000","last_active":"2024-04-05T22:27:32.146927+0000","last_peered":"2024-04-05T22:27:32.146927+0000","last_clean":"2024-04-05T22:27:32.146927+0000","last_became_active":"2024-04-05T22:27:31.657338+0000","last_became_peered":"2024-04-05T22:27:31.657338+0000","last_unstale":"2024-04-05T22:27:32.146927+0000","last_undegraded":"2024-04-05T22:27:32.146927+0000","last_fullsized":"2024-04-05T22:27:32.146927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:39:25.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.138690+0000","last_change":"2024-04-05T22:27:32.130139+0000","last_active":"2024-04-05T22:27:32.138690+0000","last_peered":"2024-04-05T22:27:32.138690+0000","last_clean":"2024-04-05T22:27:32.138690+0000","last_became_active":"2024-04-05T22:27:32.129813+0000","last_became_peered":"2024-04-05T22:27:32.129813+0000","last_unstale":"2024-04-05T22:27:32.138690+0000","last_undegraded":"2024-04-05T22:27:32.138690+0000","last_fullsized":"2024-04-05T22:27:32.138690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:25.750070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.146985+0000","last_change":"2024-04-05T22:27:31.656663+0000","last_active":"2024-04-05T22:27:32.146985+0000","last_peered":"2024-04-05T22:27:32.146985+0000","last_clean":"2024-04-05T22:27:32.146985+0000","last_became_active":"2024-04-05T22:27:31.656380+0000","last_became_peered":"2024-04-05T22:27:31.656380+0000","last_unstale":"2024-04-05T22:27:32.146985+0000","last_undegraded":"2024-04-05T22:27:32.146985+0000","last_fullsized":"2024-04-05T22:27:32.146985+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:22.616723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157350+0000","last_change":"2024-04-05T22:27:32.149910+0000","last_active":"2024-04-05T22:27:32.157350+0000","last_peered":"2024-04-05T22:27:32.157350+0000","last_clean":"2024-04-05T22:27:32.157350+0000","last_became_active":"2024-04-05T22:27:32.149606+0000","last_became_peered":"2024-04-05T22:27:32.149606+0000","last_unstale":"2024-04-05T22:27:32.157350+0000","last_undegraded":"2024-04-05T22:27:32.157350+0000","last_fullsized":"2024-04-05T22:27:32.157350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:47:40.767763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.154737+0000","last_change":"2024-04-05T22:27:31.653732+0000","last_active":"2024-04-05T22:27:32.154737+0000","last_peered":"2024-04-05T22:27:32.154737+0000","last_clean":"2024-04-05T22:27:32.154737+0000","last_became_active":"2024-04-05T22:27:31.653196+0000","last_became_peered":"2024-04-05T22:27:31.653196+0000","last_unstale":"2024-04-05T22:27:32.154737+0000","last_undegraded":"2024-04-05T22:27:32.154737+0000","last_fullsized":"2024-04-05T22:27:32.154737+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:38:33.507385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159072+0000","last_change":"2024-04-05T22:27:32.155550+0000","last_active":"2024-04-05T22:27:32.159072+0000","last_peered":"2024-04-05T22:27:32.159072+0000","last_clean":"2024-04-05T22:27:32.159072+0000","last_became_active":"2024-04-05T22:27:32.154935+0000","last_became_peered":"2024-04-05T22:27:32.154935+0000","last_unstale":"2024-04-05T22:27:32.159072+0000","last_undegraded":"2024-04-05T22:27:32.159072+0000","last_fullsized":"2024-04-05T22:27:32.159072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:52.724372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.128625+0000","last_change":"2024-04-05T22:27:31.658814+0000","last_active":"2024-04-05T22:27:32.128625+0000","last_peered":"2024-04-05T22:27:32.128625+0000","last_clean":"2024-04-05T22:27:32.128625+0000","last_became_active":"2024-04-05T22:27:31.658509+0000","last_became_peered":"2024-04-05T22:27:31.658509+0000","last_unstale":"2024-04-05T22:27:32.128625+0000","last_undegraded":"2024-04-05T22:27:32.128625+0000","last_fullsized":"2024-04-05T22:27:32.128625+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.675407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.157885+0000","last_change":"2024-04-05T22:27:32.151800+0000","last_active":"2024-04-05T22:27:32.157885+0000","last_peered":"2024-04-05T22:27:32.157885+0000","last_clean":"2024-04-05T22:27:32.157885+0000","last_became_active":"2024-04-05T22:27:32.151494+0000","last_became_peered":"2024-04-05T22:27:32.151494+0000","last_unstale":"2024-04-05T22:27:32.157885+0000","last_undegraded":"2024-04-05T22:27:32.157885+0000","last_fullsized":"2024-04-05T22:27:32.157885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:36:41.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144887+0000","last_change":"2024-04-05T22:27:31.659129+0000","last_active":"2024-04-05T22:27:32.144887+0000","last_peered":"2024-04-05T22:27:32.144887+0000","last_clean":"2024-04-05T22:27:32.144887+0000","last_became_active":"2024-04-05T22:27:31.658765+0000","last_became_peered":"2024-04-05T22:27:31.658765+0000","last_unstale":"2024-04-05T22:27:32.144887+0000","last_undegraded":"2024-04-05T22:27:32.144887+0000","last_fullsized":"2024-04-05T22:27:32.144887+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:45:22.616723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.155319+0000","last_change":"2024-04-05T22:27:32.152698+0000","last_active":"2024-04-05T22:27:32.155319+0000","last_peered":"2024-04-05T22:27:32.155319+0000","last_clean":"2024-04-05T22:27:32.155319+0000","last_became_active":"2024-04-05T22:27:32.152323+0000","last_became_peered":"2024-04-05T22:27:32.152323+0000","last_unstale":"2024-04-05T22:27:32.155319+0000","last_undegraded":"2024-04-05T22:27:32.155319+0000","last_fullsized":"2024-04-05T22:27:32.155319+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:08:43.275719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.121091+0000","last_change":"2024-04-05T22:27:31.649699+0000","last_active":"2024-04-05T22:27:32.121091+0000","last_peered":"2024-04-05T22:27:32.121091+0000","last_clean":"2024-04-05T22:27:32.121091+0000","last_became_active":"2024-04-05T22:27:31.649211+0000","last_became_peered":"2024-04-05T22:27:31.649211+0000","last_unstale":"2024-04-05T22:27:32.121091+0000","last_undegraded":"2024-04-05T22:27:32.121091+0000","last_fullsized":"2024-04-05T22:27:32.121091+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:32.947166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158016+0000","last_change":"2024-04-05T22:27:32.154365+0000","last_active":"2024-04-05T22:27:32.158016+0000","last_peered":"2024-04-05T22:27:32.158016+0000","last_clean":"2024-04-05T22:27:32.158016+0000","last_became_active":"2024-04-05T22:27:32.151432+0000","last_became_peered":"2024-04-05T22:27:32.151432+0000","last_unstale":"2024-04-05T22:27:32.158016+0000","last_undegraded":"2024-04-05T22:27:32.158016+0000","last_fullsized":"2024-04-05T22:27:32.158016+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:21.246110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.118469+0000","last_change":"2024-04-05T22:27:31.650690+0000","last_active":"2024-04-05T22:27:32.118469+0000","last_peered":"2024-04-05T22:27:32.118469+0000","last_clean":"2024-04-05T22:27:32.118469+0000","last_became_active":"2024-04-05T22:27:31.650387+0000","last_became_peered":"2024-04-05T22:27:31.650387+0000","last_unstale":"2024-04-05T22:27:32.118469+0000","last_undegraded":"2024-04-05T22:27:32.118469+0000","last_fullsized":"2024-04-05T22:27:32.118469+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:28.175860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143594+0000","last_change":"2024-04-05T22:27:32.133254+0000","last_active":"2024-04-05T22:27:32.143594+0000","last_peered":"2024-04-05T22:27:32.143594+0000","last_clean":"2024-04-05T22:27:32.143594+0000","last_became_active":"2024-04-05T22:27:32.132955+0000","last_became_peered":"2024-04-05T22:27:32.132955+0000","last_unstale":"2024-04-05T22:27:32.143594+0000","last_undegraded":"2024-04-05T22:27:32.143594+0000","last_fullsized":"2024-04-05T22:27:32.143594+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:15:22.544892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113689+0000","last_change":"2024-04-05T22:27:31.658510+0000","last_active":"2024-04-05T22:27:32.113689+0000","last_peered":"2024-04-05T22:27:32.113689+0000","last_clean":"2024-04-05T22:27:32.113689+0000","last_became_active":"2024-04-05T22:27:31.658163+0000","last_became_peered":"2024-04-05T22:27:31.658163+0000","last_unstale":"2024-04-05T22:27:32.113689+0000","last_undegraded":"2024-04-05T22:27:32.113689+0000","last_fullsized":"2024-04-05T22:27:32.113689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:04:01.004689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.158637+0000","last_change":"2024-04-05T22:27:32.153312+0000","last_active":"2024-04-05T22:27:32.158637+0000","last_peered":"2024-04-05T22:27:32.158637+0000","last_clean":"2024-04-05T22:27:32.158637+0000","last_became_active":"2024-04-05T22:27:32.152404+0000","last_became_peered":"2024-04-05T22:27:32.152404+0000","last_unstale":"2024-04-05T22:27:32.158637+0000","last_undegraded":"2024-04-05T22:27:32.158637+0000","last_fullsized":"2024-04-05T22:27:32.158637+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:17:12.574970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143602+0000","last_change":"2024-04-05T22:27:31.660346+0000","last_active":"2024-04-05T22:27:32.143602+0000","last_peered":"2024-04-05T22:27:32.143602+0000","last_clean":"2024-04-05T22:27:32.143602+0000","last_became_active":"2024-04-05T22:27:31.660031+0000","last_became_peered":"2024-04-05T22:27:31.660031+0000","last_unstale":"2024-04-05T22:27:32.143602+0000","last_undegraded":"2024-04-05T22:27:32.143602+0000","last_fullsized":"2024-04-05T22:27:32.143602+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:40:33.424001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.143631+0000","last_change":"2024-04-05T22:27:32.138494+0000","last_active":"2024-04-05T22:27:32.143631+0000","last_peered":"2024-04-05T22:27:32.143631+0000","last_clean":"2024-04-05T22:27:32.143631+0000","last_became_active":"2024-04-05T22:27:32.137997+0000","last_became_peered":"2024-04-05T22:27:32.137997+0000","last_unstale":"2024-04-05T22:27:32.143631+0000","last_undegraded":"2024-04-05T22:27:32.143631+0000","last_fullsized":"2024-04-05T22:27:32.143631+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:42:46.145634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144915+0000","last_change":"2024-04-05T22:27:31.660207+0000","last_active":"2024-04-05T22:27:32.144915+0000","last_peered":"2024-04-05T22:27:32.144915+0000","last_clean":"2024-04-05T22:27:32.144915+0000","last_became_active":"2024-04-05T22:27:31.659953+0000","last_became_peered":"2024-04-05T22:27:31.659953+0000","last_unstale":"2024-04-05T22:27:32.144915+0000","last_undegraded":"2024-04-05T22:27:32.144915+0000","last_fullsized":"2024-04-05T22:27:32.144915+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:46:03.768005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.129609+0000","last_change":"2024-04-05T22:27:32.125090+0000","last_active":"2024-04-05T22:27:32.129609+0000","last_peered":"2024-04-05T22:27:32.129609+0000","last_clean":"2024-04-05T22:27:32.129609+0000","last_became_active":"2024-04-05T22:27:32.124832+0000","last_became_peered":"2024-04-05T22:27:32.124832+0000","last_unstale":"2024-04-05T22:27:32.129609+0000","last_undegraded":"2024-04-05T22:27:32.129609+0000","last_fullsized":"2024-04-05T22:27:32.129609+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:42:08.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.114067+0000","last_change":"2024-04-05T22:27:31.651404+0000","last_active":"2024-04-05T22:27:32.114067+0000","last_peered":"2024-04-05T22:27:32.114067+0000","last_clean":"2024-04-05T22:27:32.114067+0000","last_became_active":"2024-04-05T22:27:31.649789+0000","last_became_peered":"2024-04-05T22:27:31.649789+0000","last_unstale":"2024-04-05T22:27:32.114067+0000","last_undegraded":"2024-04-05T22:27:32.114067+0000","last_fullsized":"2024-04-05T22:27:32.114067+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:40:45.006358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.156588+0000","last_change":"2024-04-05T22:27:32.151727+0000","last_active":"2024-04-05T22:27:32.156588+0000","last_peered":"2024-04-05T22:27:32.156588+0000","last_clean":"2024-04-05T22:27:32.156588+0000","last_became_active":"2024-04-05T22:27:32.151305+0000","last_became_peered":"2024-04-05T22:27:32.151305+0000","last_unstale":"2024-04-05T22:27:32.156588+0000","last_undegraded":"2024-04-05T22:27:32.156588+0000","last_fullsized":"2024-04-05T22:27:32.156588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:44:23.736872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.126673+0000","last_change":"2024-04-05T22:27:31.651066+0000","last_active":"2024-04-05T22:27:32.126673+0000","last_peered":"2024-04-05T22:27:32.126673+0000","last_clean":"2024-04-05T22:27:32.126673+0000","last_became_active":"2024-04-05T22:27:31.650814+0000","last_became_peered":"2024-04-05T22:27:31.650814+0000","last_unstale":"2024-04-05T22:27:32.126673+0000","last_undegraded":"2024-04-05T22:27:32.126673+0000","last_fullsized":"2024-04-05T22:27:32.126673+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:35:34.319386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144399+0000","last_change":"2024-04-05T22:27:32.138595+0000","last_active":"2024-04-05T22:27:32.144399+0000","last_peered":"2024-04-05T22:27:32.144399+0000","last_clean":"2024-04-05T22:27:32.144399+0000","last_became_active":"2024-04-05T22:27:32.137137+0000","last_became_peered":"2024-04-05T22:27:32.137137+0000","last_unstale":"2024-04-05T22:27:32.144399+0000","last_undegraded":"2024-04-05T22:27:32.144399+0000","last_fullsized":"2024-04-05T22:27:32.144399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:40:34.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.129624+0000","last_change":"2024-04-05T22:27:31.657303+0000","last_active":"2024-04-05T22:27:32.129624+0000","last_peered":"2024-04-05T22:27:32.129624+0000","last_clean":"2024-04-05T22:27:32.129624+0000","last_became_active":"2024-04-05T22:27:31.656942+0000","last_became_peered":"2024-04-05T22:27:31.656942+0000","last_unstale":"2024-04-05T22:27:32.129624+0000","last_undegraded":"2024-04-05T22:27:32.129624+0000","last_fullsized":"2024-04-05T22:27:32.129624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:37:20.277508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.144104+0000","last_change":"2024-04-05T22:27:32.133337+0000","last_active":"2024-04-05T22:27:32.144104+0000","last_peered":"2024-04-05T22:27:32.144104+0000","last_clean":"2024-04-05T22:27:32.144104+0000","last_became_active":"2024-04-05T22:27:32.132094+0000","last_became_peered":"2024-04-05T22:27:32.132094+0000","last_unstale":"2024-04-05T22:27:32.144104+0000","last_undegraded":"2024-04-05T22:27:32.144104+0000","last_fullsized":"2024-04-05T22:27:32.144104+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:45:36.855438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122283+0000","last_change":"2024-04-05T22:27:31.654670+0000","last_active":"2024-04-05T22:27:32.122283+0000","last_peered":"2024-04-05T22:27:32.122283+0000","last_clean":"2024-04-05T22:27:32.122283+0000","last_became_active":"2024-04-05T22:27:31.654404+0000","last_became_peered":"2024-04-05T22:27:31.654404+0000","last_unstale":"2024-04-05T22:27:32.122283+0000","last_undegraded":"2024-04-05T22:27:32.122283+0000","last_fullsized":"2024-04-05T22:27:32.122283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:33:24.738157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.159164+0000","last_change":"2024-04-05T22:27:32.154408+0000","last_active":"2024-04-05T22:27:32.159164+0000","last_peered":"2024-04-05T22:27:32.159164+0000","last_clean":"2024-04-05T22:27:32.159164+0000","last_became_active":"2024-04-05T22:27:32.151762+0000","last_became_peered":"2024-04-05T22:27:32.151762+0000","last_unstale":"2024-04-05T22:27:32.159164+0000","last_undegraded":"2024-04-05T22:27:32.159164+0000","last_fullsized":"2024-04-05T22:27:32.159164+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:03:13.396297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.122069+0000","last_change":"2024-04-05T22:27:31.653290+0000","last_active":"2024-04-05T22:27:32.122069+0000","last_peered":"2024-04-05T22:27:32.122069+0000","last_clean":"2024-04-05T22:27:32.122069+0000","last_became_active":"2024-04-05T22:27:31.652864+0000","last_became_peered":"2024-04-05T22:27:31.652864+0000","last_unstale":"2024-04-05T22:27:32.122069+0000","last_undegraded":"2024-04-05T22:27:32.122069+0000","last_fullsized":"2024-04-05T22:27:32.122069+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:22:28.175860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.125613+0000","last_change":"2024-04-05T22:27:32.121660+0000","last_active":"2024-04-05T22:27:32.125613+0000","last_peered":"2024-04-05T22:27:32.125613+0000","last_clean":"2024-04-05T22:27:32.125613+0000","last_became_active":"2024-04-05T22:27:32.120741+0000","last_became_peered":"2024-04-05T22:27:32.120741+0000","last_unstale":"2024-04-05T22:27:32.125613+0000","last_undegraded":"2024-04-05T22:27:32.125613+0000","last_fullsized":"2024-04-05T22:27:32.125613+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:57:38.523011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.113633+0000","last_change":"2024-04-05T22:27:31.654989+0000","last_active":"2024-04-05T22:27:32.113633+0000","last_peered":"2024-04-05T22:27:32.113633+0000","last_clean":"2024-04-05T22:27:32.113633+0000","last_became_active":"2024-04-05T22:27:31.654581+0000","last_became_peered":"2024-04-05T22:27:31.654581+0000","last_unstale":"2024-04-05T22:27:32.113633+0000","last_undegraded":"2024-04-05T22:27:32.113633+0000","last_fullsized":"2024-04-05T22:27:32.113633+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:54:12.487510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.149881+0000","last_change":"2024-04-05T22:27:32.147959+0000","last_active":"2024-04-05T22:27:32.149881+0000","last_peered":"2024-04-05T22:27:32.149881+0000","last_clean":"2024-04-05T22:27:32.149881+0000","last_became_active":"2024-04-05T22:27:32.147664+0000","last_became_peered":"2024-04-05T22:27:32.147664+0000","last_unstale":"2024-04-05T22:27:32.149881+0000","last_undegraded":"2024-04-05T22:27:32.149881+0000","last_fullsized":"2024-04-05T22:27:32.149881+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:31.607318+0000","last_clean_scrub_stamp":"2024-04-05T22:27:31.607318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:42:08.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-05T22:27:32.130264+0000","last_change":"2024-04-05T22:27:31.654588+0000","last_active":"2024-04-05T22:27:32.130264+0000","last_peered":"2024-04-05T22:27:32.130264+0000","last_clean":"2024-04-05T22:27:32.130264+0000","last_became_active":"2024-04-05T22:27:31.654302+0000","last_became_peered":"2024-04-05T22:27:31.654302+0000","last_unstale":"2024-04-05T22:27:32.130264+0000","last_undegraded":"2024-04-05T22:27:32.130264+0000","last_fullsized":"2024-04-05T22:27:32.130264+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:27:30.595405+0000","last_clean_scrub_stamp":"2024-04-05T22:27:30.595405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:36:19.081446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":274,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-05T22:27:31.658083+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:27:31.658083+0000","last_peered":"2024-04-05T22:27:31.658083+0000","last_clean":"2024-04-05T22:27:31.658083+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:27:31.658083+0000","last_undegraded":"2024-04-05T22:27:31.658083+0000","last_fullsized":"2024-04-05T22:27:31.658083+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1916928,"data_stored":1167264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0700000000000001,"15min":1.034},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.6759999999999999,"5min":1.796,"15min":1.796},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0620000000000001,"15min":1.0109999999999999},"min":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3919999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1639999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.58099999999999996},"max":{"1min":1.504,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.1000000000000001,"15min":1.0680000000000001},"min":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.173,"15min":1.171},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4139999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.087},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.272,"15min":1.222},"min":{"1min":0.92400000000000004,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.587,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1299999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.208,"15min":1.139},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72299999999999998},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.712},"last":1.171},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.218},"min":{"1min":0.96899999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7849999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.234,"15min":1.2190000000000001},"min":{"1min":0.81499999999999995,"5min":0.68700000000000006,"15min":0.66200000000000003},"max":{"1min":1.6339999999999999,"5min":1.8360000000000001,"15min":1.865},"last":1.331},{"interface":"front","average":{"1min":1.333,"5min":1.3080000000000001,"15min":1.3169999999999999},"min":{"1min":0.89500000000000002,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":1.681,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3819999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.329,"15min":1.2470000000000001},"min":{"1min":0.88300000000000001,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.752,"5min":1.8440000000000001,"15min":1.9590000000000001},"last":1.224},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3799999999999999,"15min":1.3500000000000001},"min":{"1min":1,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9390000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.04}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.522,"15min":1.4650000000000001},"min":{"1min":1.25,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":2.0630000000000002,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.4890000000000001},"min":{"1min":1.2070000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0150000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.2589999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1314816,"data_stored":576280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0369999999999999,"15min":1.0580000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.5229999999999999,"5min":1.579,"15min":2.2509999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.097},"min":{"1min":0.746,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.627,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.2490000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.093,"15min":1.155},"min":{"1min":0.624,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.1100000000000001,"15min":1.1459999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.8680000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1599999999999999,"15min":1.194},"min":{"1min":0.84599999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.71299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4930000000000001,"5min":1.7749999999999999,"15min":2.2250000000000001},"last":1.361}]},{"osd":3,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1990000000000001,"15min":1.3089999999999999},"min":{"1min":0.73299999999999998,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.7769999999999999},"last":1.96},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.7310000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2390000000000001,"15min":1.321},"min":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1019999999999999},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.55},"min":{"1min":0.78700000000000003,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":3.27},"last":1.8149999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.244,"15min":1.4279999999999999},"min":{"1min":0.76200000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.927,"5min":1.927,"15min":2.3650000000000002},"last":1.708},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4219999999999999,"15min":1.544},"min":{"1min":0.81100000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.3479999999999999},"last":2.3540000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:27:18 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.365,"15min":1.5349999999999999},"min":{"1min":1.0229999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.99,"5min":2.2599999999999998,"15min":3.3540000000000001},"last":1.643},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.359,"15min":1.5389999999999999},"min":{"1min":0.89600000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.831,"5min":1.897,"15min":3.0920000000000001},"last":1.899}]}]},{"osd":5,"up_from":14,"seq":60129542224,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1290240,"data_stored":563230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.093,"15min":1.0189999999999999},"min":{"1min":0.72899999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.405,"5min":1.726,"15min":1.726},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.129,"15min":1.0960000000000001},"min":{"1min":0.84199999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5289999999999999,"5min":1.907,"15min":1.907},"last":1.099}]},{"osd":1,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3300000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9330000000000001,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.98399999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.196,"15min":1.161},"min":{"1min":0.82999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0099999999999998,"5min":2.544,"15min":2.544},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.22,"15min":1.25},"min":{"1min":0.89400000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.538,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.5149999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2090000000000001,"15min":1.1830000000000001},"min":{"1min":0.755,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.504,"5min":1.867,"15min":1.867},"last":1.401},{"interface":"front","average":{"1min":1.282,"5min":1.262,"15min":1.355},"min":{"1min":0.99399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8360000000000001,"5min":2.927,"15min":2.9620000000000002},"last":1.2090000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2609999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.6739999999999999,"5min":2.0099999999999998,"15min":2.1970000000000001},"last":1.151},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.264,"15min":1.3009999999999999},"min":{"1min":0.52100000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.702,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.6759999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.82899999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.496,"15min":1.496},"min":{"1min":0.77200000000000002,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":2.169,"5min":2.169,"15min":2.5350000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:27:23 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4039999999999999,"15min":1.452},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.028,"5min":2.028,"15min":2.2509999999999999},"last":1.571},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4630000000000001,"15min":1.603},"min":{"1min":0.66700000000000004,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":1.974,"5min":2.0609999999999999,"15min":4.4329999999999998},"last":1.4990000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.0920000000000001,"15min":1.0129999999999999},"min":{"1min":0.69699999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1000000000000001,"15min":1.081},"min":{"1min":0.73899999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7769999999999999,"5min":1.859,"15min":1.859},"last":1.135}]},{"osd":1,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1379999999999999,"15min":1.1279999999999999},"min":{"1min":0.78100000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.139,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":1.3200000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.133,"15min":1.2070000000000001},"min":{"1min":0.75700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4099999999999999,"5min":1.6930000000000001,"15min":1.974},"last":1.024},{"interface":"front","average":{"1min":1.323,"5min":1.224,"15min":1.196},"min":{"1min":0.78800000000000003,"5min":0.68200000000000005,"15min":0.67400000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.1819999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.329,"15min":1.369},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.835,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.452},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3480000000000001,"15min":1.3999999999999999},"min":{"1min":0.91200000000000003,"5min":0.874,"15min":0.874},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.7509999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.29},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":2.1459999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.343,"15min":1.393},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.8200000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.351,"15min":1.4510000000000001},"min":{"1min":0.85799999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.9950000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.401,"15min":1.482},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.8899999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.496,"15min":1.534},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.962},{"interface":"front","average":{"1min":1.663,"5min":1.575,"15min":1.583},"min":{"1min":0.93500000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":2.109}]}]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.85399999999999998},"min":{"1min":0.38700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.486,"5min":1.744,"15min":1.744},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.93200000000000005,"15min":0.91300000000000003},"min":{"1min":0.501,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.85599999999999998,"15min":0.89400000000000002},"min":{"1min":0.442,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95599999999999996,"15min":1.01},"min":{"1min":0.50600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.403,"5min":1.5720000000000001,"15min":1.9410000000000001},"last":1.127}]},{"osd":2,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.98099999999999998,"15min":1.0189999999999999},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.827},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.0620000000000001,"15min":1.0780000000000001},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4530000000000001,"5min":1.675,"15min":2.2549999999999999},"last":1.2769999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.181,"15min":1.2390000000000001},"min":{"1min":0.748,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.079},{"interface":"front","average":{"1min":1.21,"5min":1.194,"15min":1.2969999999999999},"min":{"1min":0.77800000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2609999999999999,"15min":1.3580000000000001},"min":{"1min":0.83999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7050000000000001,"5min":1.7390000000000001,"15min":2.5129999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.276,"15min":1.292},"min":{"1min":0.71299999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.958,"5min":1.958,"15min":2.3239999999999998},"last":1.3340000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3109999999999999,"15min":1.363},"min":{"1min":0.82499999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.296,"15min":1.403},"min":{"1min":0.94399999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:27:25 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.325,"15min":1.423},"min":{"1min":0.82599999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.651,"5min":1.9410000000000001,"15min":2.468},"last":1.601},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.4239999999999999},"min":{"1min":0.86799999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.508,"5min":2.508,"15min":2.798},"last":1.319}]}]},{"osd":2,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82499999999999996,"15min":0.82899999999999996},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.506,"5min":1.621,"15min":1.621},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6359999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.131,"15min":1.125},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.694},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2829999999999999,"15min":1.2290000000000001},"min":{"1min":0.82599999999999996,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":1.6970000000000001,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.216,"15min":1.23},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":1.8320000000000001,"15min":2.9510000000000001},"last":1.2070000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.8919999999999999,"5min":2.3180000000000001,"15min":2.8370000000000002},"last":1.012},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.216,"15min":1.264},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.837,"5min":1.9530000000000001,"15min":2.3039999999999998},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.2649999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.76200000000000001},"max":{"1min":1.9079999999999999,"5min":2.0230000000000001,"15min":2.3969999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.298,"15min":1.276},"min":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.7890000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Apr 5 22:27:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3069999999999999,"15min":1.3500000000000001},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.74099999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.6269999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3759999999999999,"15min":1.3879999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.96,"5min":1.96,"15min":2.1600000000000001},"last":1.002}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1302528,"data_stored":569505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78700000000000003,"15min":0.76400000000000001},"min":{"1min":0.38400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2589999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.88300000000000001,"15min":0.85099999999999998},"min":{"1min":0.42299999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2450000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.873,"15min":0.83499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.488,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.998,"15min":1},"min":{"1min":0.45200000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.3300000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0840000000000001,"15min":1.1020000000000001},"min":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5029999999999999,"5min":1.75,"15min":1.75},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.032,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1659999999999999,"15min":1.141},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.855},"last":0.877},{"interface":"front","average":{"1min":1.268,"5min":1.278,"15min":1.284},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.72,"5min":1.806,"15min":2.2789999999999999},"last":1.212}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2629999999999999,"15min":1.2689999999999999},"min":{"1min":0.76000000000000001,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2509999999999999,"15min":1.244},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.843,"5min":1.849,"15min":2.173},"last":1.167}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3720000000000001,"15min":1.3380000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.857,"5min":1.917,"15min":1.917},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3220000000000001,"15min":1.361},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.347}]},{"osd":7,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.357,"15min":1.363},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.3420000000000001,"15min":1.3520000000000001},"min":{"1min":0.84599999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.9279999999999999,"5min":1.9359999999999999,"15min":2.3879999999999999},"last":1.294}]}]},{"osd":0,"up_from":14,"seq":60129542225,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1896448,"data_stored":1159873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.075,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":0.88400000000000001},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.31900000000000001},"max":{"1min":1.5640000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":1.5720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.95499999999999996,"15min":0.93200000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2689999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.625},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.98799999999999999,"15min":0.96799999999999997},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.55,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.1259999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.92000000000000004,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35599999999999998},"max":{"1min":1.5700000000000001,"5min":1.976,"15min":1.976},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.087,"15min":1.103},"min":{"1min":0.51100000000000001,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":1.6080000000000001,"5min":2.157,"15min":2.157},"last":1.448}]},{"osd":4,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.21,"15min":1.1719999999999999},"min":{"1min":0.77500000000000002,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3129999999999999,"15min":1.3069999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8520000000000001,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.331}]},{"osd":5,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.274,"15min":1.268},"min":{"1min":0.746,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.893,"5min":2.105,"15min":2.3879999999999999},"last":1.492},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.238,"15min":1.204},"min":{"1min":0.81299999999999994,"5min":0.76600000000000001,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.296}]},{"osd":6,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.387},"min":{"1min":0.878,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.349,"15min":1.375},"min":{"1min":0.874,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8120000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.401}]},{"osd":7,"last update":"Fri Apr 5 22:27:22 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.353,"15min":1.3440000000000001},"min":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0449999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.77000000000000002,"5min":0.75,"15min":0.68999999999999995},"max":{"1min":1.7509999999999999,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:27:37.053 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:27:37.053 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:27:37.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.315+0000 7fb972e00640 1 Processor -- start 2024-04-05T22:27:37.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.315+0000 7fb972e00640 1 -- start start 2024-04-05T22:27:37.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb96c15d070 0x7fb96c06d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb96c072040 0x7fb96c06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 0x7fb96c0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c071de0 con 0x7fb96c15d070 2024-04-05T22:27:37.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c15d450 con 0x7fb96c072040 2024-04-05T22:27:37.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb972e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c06a760 con 0x7fb96c0713f0 2024-04-05T22:27:37.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb96be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb96c15d070 0x7fb96c06d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb96be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb96c15d070 0x7fb96c06d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52090/0 (socket says 172.21.15.73:52090) 2024-04-05T22:27:37.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.317+0000 7fb96be00640 1 -- 172.21.15.73:0/1038065818 learned_addr learned my addr 172.21.15.73:0/1038065818 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:37.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96b400640 1 --2- 172.21.15.73:0/1038065818 >> v2:172.21.15.157:3300/0 conn(0x7fb96c072040 0x7fb96c06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb970e00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 0x7fb96c0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96be00640 1 -- 172.21.15.73:0/1038065818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 msgr2=0x7fb96c0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96be00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 0x7fb96c0718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96be00640 1 -- 172.21.15.73:0/1038065818 >> v2:172.21.15.157:3300/0 conn(0x7fb96c072040 msgr2=0x7fb96c06ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96be00640 1 --2- 172.21.15.73:0/1038065818 >> v2:172.21.15.157:3300/0 conn(0x7fb96c072040 0x7fb96c06ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb96be00640 1 -- 172.21.15.73:0/1038065818 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c06aa40 con 0x7fb96c15d070 2024-04-05T22:27:37.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.318+0000 7fb970e00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 0x7fb96c0718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:37.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.319+0000 7fb96be00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c15d070 0x7fb96c06d880 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0b4610 tx=0x7fb95c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3640a3bb427901a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.319+0000 7fb96aa00640 1 -- 172.21.15.73:0/1038065818 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c02ca70 con 0x7fb96c15d070 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.319+0000 7fb96aa00640 1 -- 172.21.15.73:0/1038065818 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c02cbd0 con 0x7fb96c15d070 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.319+0000 7fb96aa00640 1 -- 172.21.15.73:0/1038065818 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0343f0 con 0x7fb96c15d070 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 -- 172.21.15.73:0/1038065818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c15d070 msgr2=0x7fb96c06d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c15d070 0x7fb96c06d880 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0b4610 tx=0x7fb95c02bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 -- 172.21.15.73:0/1038065818 shutdown_connections 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c0713f0 0x7fb96c0718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 --2- 172.21.15.73:0/1038065818 >> v2:172.21.15.157:3300/0 conn(0x7fb96c072040 0x7fb96c06ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 --2- 172.21.15.73:0/1038065818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c15d070 0x7fb96c06d880 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 -- 172.21.15.73:0/1038065818 >> 172.21.15.73:0/1038065818 conn(0x7fb96c0b2c70 msgr2=0x7fb96c0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 -- 172.21.15.73:0/1038065818 shutdown_connections 2024-04-05T22:27:37.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.320+0000 7fb972e00640 1 -- 172.21.15.73:0/1038065818 wait complete. 2024-04-05T22:27:37.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.322+0000 7fb972e00640 1 Processor -- start 2024-04-05T22:27:37.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.322+0000 7fb972e00640 1 -- start start 2024-04-05T22:27:37.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96c072040 0x7fb96c157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c15d070 0x7fb96c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c09bce0 con 0x7fb96c0713f0 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c09bfc0 con 0x7fb96c072040 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb972e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c09c2a0 con 0x7fb96c15d070 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52106/0 (socket says 172.21.15.73:52106) 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 -- 172.21.15.73:0/2295859361 learned_addr learned my addr 172.21.15.73:0/2295859361 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 -- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c15d070 msgr2=0x7fb96c1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c15d070 0x7fb96c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 -- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96c072040 msgr2=0x7fb96c157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96c072040 0x7fb96c157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.323+0000 7fb96be00640 1 -- 172.21.15.73:0/2295859361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c009b30 con 0x7fb96c0713f0 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb96be00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000f80 tx=0x7fb95c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c009510 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c03e450 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c03e800 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c09c580 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.324+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c09af60 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.325+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c068160 con 0x7fb96c0713f0 2024-04-05T22:27:37.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.326+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb95c03e960 con 0x7fb96c0713f0 2024-04-05T22:27:37.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.328+0000 7fb963e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 0x7fb94806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.328+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb95c0ac260 con 0x7fb96c0713f0 2024-04-05T22:27:37.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.328+0000 7fb96b400640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 0x7fb94806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.329+0000 7fb96b400640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 0x7fb94806d1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0cb520 tx=0x7fb954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.332+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb95c0773c0 con 0x7fb96c0713f0 2024-04-05T22:27:37.590 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-05T22:27:37.591+0000 7f4d20a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:27:37.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.594+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb96c066180 con 0x7fb96c0713f0 2024-04-05T22:27:37.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.601+0000 7fb963e00640 1 -- 172.21.15.73:0/2295859361 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb95c07a9e0 con 0x7fb96c0713f0 2024-04-05T22:27:37.600 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 msgr2=0x7fb94806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 0x7fb94806d1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0cb520 tx=0x7fb954008040 comp rx=0 tx=0).stop 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 msgr2=0x7fb96c1556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000f80 tx=0x7fb95c004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 shutdown_connections 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb94806ad20 0x7fb94806d1e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb96c15d070 0x7fb96c1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb96c072040 0x7fb96c157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 --2- 172.21.15.73:0/2295859361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb96c0713f0 0x7fb96c1556c0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 >> 172.21.15.73:0/2295859361 conn(0x7fb96c0b2c70 msgr2=0x7fb96c1587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:37.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 shutdown_connections 2024-04-05T22:27:37.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.604+0000 7fb972e00640 1 -- 172.21.15.73:0/2295859361 wait complete. 2024-04-05T22:27:37.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:27:37.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.823+0000 7fe44c600640 1 Processor -- start 2024-04-05T22:27:37.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.823+0000 7fe44c600640 1 -- start start 2024-04-05T22:27:37.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.824+0000 7fe44c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 0x7fe4440b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.824+0000 7fe44c600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe44406d710 0x7fe4440cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.824+0000 7fe44c600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44415aa60 con 0x7fe44415aef0 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.824+0000 7fe44c600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b3b60 con 0x7fe44406d710 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.824+0000 7fe44c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440701c0 con 0x7fe4441541a0 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe44a600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe44415aef0 0x7fe44415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49626/0 (socket says 172.21.15.73:49626) 2024-04-05T22:27:37.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe44a600640 1 -- 172.21.15.73:0/2893916375 learned_addr learned my addr 172.21.15.73:0/2893916375 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:37.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe449c00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 0x7fe4440b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642367578 0 0) 0x7fe44415aa60 con 0x7fe44415aef0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe449200640 1 --2- 172.21.15.73:0/2893916375 >> v2:172.21.15.157:3300/0 conn(0x7fe44406d710 0x7fe4440cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c003520 con 0x7fe44415aef0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305309278 0 0) 0x7fe42c003520 con 0x7fe44415aef0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 msgr2=0x7fe4440b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 0x7fe4440b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 >> v2:172.21.15.157:3300/0 conn(0x7fe44406d710 msgr2=0x7fe4440cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 --2- 172.21.15.73:0/2893916375 >> v2:172.21.15.157:3300/0 conn(0x7fe44406d710 0x7fe4440cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440b3880 con 0x7fe44415aef0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.825+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe43c0019f0 con 0x7fe44415aef0 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 >> v1:172.21.15.73:6789/0 conn(0x7fe44415aef0 legacy=0x7fe44415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe42c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 0x7fe42c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b3b60 con 0x7fe42c003970 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440701c0 con 0x7fe42c0047c0 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c00ba30 con 0x7fe4441541a0 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe42c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 -- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 msgr2=0x7fe42c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4441541a0 0x7fe42c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 -- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42c0047c0 msgr2=0x7fe42c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.826+0000 7fe449c00640 1 -- 172.21.15.73:0/2893916375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c003520 con 0x7fe42c003970 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.827+0000 7fe449c00640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe42c003d90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ba00 tx=0x7fe42c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8b6ce41708e729 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.827+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438032620 con 0x7fe42c003970 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438032780 con 0x7fe42c003970 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe443e00640 1 -- 172.21.15.73:0/2893916375 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438032b10 con 0x7fe42c003970 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 -- 172.21.15.73:0/2893916375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 msgr2=0x7fe42c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 --2- 172.21.15.73:0/2893916375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe42c003d90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ba00 tx=0x7fe42c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44a600640 1 -- 172.21.15.73:0/2893916375 reap_dead start 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 -- 172.21.15.73:0/2893916375 shutdown_connections 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 -- 172.21.15.73:0/2893916375 >> 172.21.15.73:0/2893916375 conn(0x7fe4440ce8e0 msgr2=0x7fe4440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 -- 172.21.15.73:0/2893916375 shutdown_connections 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.828+0000 7fe44c600640 1 -- 172.21.15.73:0/2893916375 wait complete. 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.830+0000 7fe44c600640 1 Processor -- start 2024-04-05T22:27:37.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.830+0000 7fe44c600640 1 -- start start 2024-04-05T22:27:37.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4441487c0 0x7fe4440ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 0x7fe4440ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444149a90 con 0x7fe42c003970 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44406b330 con 0x7fe4441487c0 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444149c70 con 0x7fe444148c40 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe449c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe449c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52130/0 (socket says 172.21.15.73:52130) 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe449c00640 1 -- 172.21.15.73:0/3491162153 learned_addr learned my addr 172.21.15.73:0/3491162153 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe449200640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4441487c0 0x7fe4440ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.831+0000 7fe44a600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 0x7fe4440ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 -- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 msgr2=0x7fe4440ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 0x7fe4440ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 -- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4441487c0 msgr2=0x7fe4440ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4441487c0 0x7fe4440ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 -- 172.21.15.73:0/3491162153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe444149f50 con 0x7fe42c003970 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe44a600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 0x7fe4440ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe449c00640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe438035040 tx=0x7fe438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438032db0 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438004020 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4380043d0 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44414a230 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.832+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4440abbc0 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.834+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe444066af0 con 0x7fe42c003970 2024-04-05T22:27:37.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.834+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fe438043070 con 0x7fe42c003970 2024-04-05T22:27:37.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.835+0000 7fe441600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 0x7fe42406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:37.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.835+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4380ab370 con 0x7fe42c003970 2024-04-05T22:27:37.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.836+0000 7fe449200640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 0x7fe42406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:37.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.837+0000 7fe449200640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 0x7fe42406d1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe430006a90 tx=0x7fe430008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:37.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:37.840+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe438076290 con 0x7fe42c003970 2024-04-05T22:27:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.071+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe44414a3c0 con 0x7fe42c003970 2024-04-05T22:27:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.072+0000 7fe441600640 1 -- 172.21.15.73:0/3491162153 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fe4380798b0 con 0x7fe42c003970 2024-04-05T22:27:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:38.071 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:27:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 63 2024-04-05T22:27:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 msgr2=0x7fe42406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 0x7fe42406d1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe430006a90 tx=0x7fe430008040 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 msgr2=0x7fe4440a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe438035040 tx=0x7fe438004730 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 shutdown_connections 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe42406ad20 0x7fe42406d1e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe444148c40 0x7fe4440ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4441487c0 0x7fe4440ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 --2- 172.21.15.73:0/3491162153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe42c003970 0x7fe4440a8de0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 >> 172.21.15.73:0/3491162153 conn(0x7fe4440ce8e0 msgr2=0x7fe44406f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 shutdown_connections 2024-04-05T22:27:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.075+0000 7fe44c600640 1 -- 172.21.15.73:0/3491162153 wait complete. 2024-04-05T22:27:38.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:27:32.104968+0000', 'modified': '2024-04-05T22:27:35.132684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7623}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7623': {'gid': 7623, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6835/3290176955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6833', 'nonce': 3290176955}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 3290176955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:27:38.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:27:38.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-05T22:27:38.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.716+0000 7f83b6800640 1 Processor -- start 2024-04-05T22:27:38.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.716+0000 7f83b6800640 1 -- start start 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f83b01441e0 0x7f83b00cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b6800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 0x7f83b00cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b6800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b00b1b60 con 0x7f83b00b2c00 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b6800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b00b1950 con 0x7f83b01441e0 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b6800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b00b1e40 con 0x7f83b006c2d0 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f83b01441e0 0x7f83b00cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b4a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f83b00b2c00 0x7f83b006bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49638/0 (socket says 172.21.15.73:49638) 2024-04-05T22:27:38.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b4a00640 1 -- 172.21.15.73:0/2573330140 learned_addr learned my addr 172.21.15.73:0/2573330140 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b5e00640 1 --2- 172.21.15.73:0/2573330140 >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 0x7f83b00cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b5400640 1 -- 172.21.15.73:0/2573330140 >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 msgr2=0x7f83b00cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b5400640 1 --2- 172.21.15.73:0/2573330140 >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 0x7f83b00cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.718+0000 7f83b5400640 1 -- 172.21.15.73:0/2573330140 >> v1:172.21.15.73:6789/0 conn(0x7f83b00b2c00 legacy=0x7f83b006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.719+0000 7f83b5400640 1 -- 172.21.15.73:0/2573330140 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00b2120 con 0x7f83b01441e0 2024-04-05T22:27:38.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.719+0000 7f83b5e00640 1 --2- 172.21.15.73:0/2573330140 >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 0x7f83b00cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.720+0000 7f83b5400640 1 --2- 172.21.15.73:0/2573330140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b00cc050 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83b006f540 tx=0x7f83a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb11e03a190314c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.720+0000 7f83afe00640 1 -- 172.21.15.73:0/2573330140 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4031830 con 0x7f83b01441e0 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.720+0000 7f83afe00640 1 -- 172.21.15.73:0/2573330140 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4031990 con 0x7f83b01441e0 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.720+0000 7f83afe00640 1 -- 172.21.15.73:0/2573330140 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4031d20 con 0x7f83b01441e0 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 -- 172.21.15.73:0/2573330140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 msgr2=0x7f83b00cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 --2- 172.21.15.73:0/2573330140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b00cc050 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83b006f540 tx=0x7f83a402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 -- 172.21.15.73:0/2573330140 shutdown_connections 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 --2- 172.21.15.73:0/2573330140 >> v2:172.21.15.157:3300/0 conn(0x7f83b006c2d0 0x7f83b00cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 --2- 172.21.15.73:0/2573330140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b00cc050 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 -- 172.21.15.73:0/2573330140 >> 172.21.15.73:0/2573330140 conn(0x7f83b00b0830 msgr2=0x7f83b00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 -- 172.21.15.73:0/2573330140 shutdown_connections 2024-04-05T22:27:38.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.721+0000 7f83b6800640 1 -- 172.21.15.73:0/2573330140 wait complete. 2024-04-05T22:27:38.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.722+0000 7f83b6800640 1 Processor -- start 2024-04-05T22:27:38.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.723+0000 7f83b6800640 1 -- start start 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.723+0000 7f83b6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b006c2d0 0x7f83b01e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.723+0000 7f83b6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83b00b2c00 0x7f83b01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.723+0000 7f83b6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.723+0000 7f83b6800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b012ff50 con 0x7f83b01441e0 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01301f0 con 0x7f83b006c2d0 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01304d0 con 0x7f83b00b2c00 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52144/0 (socket says 172.21.15.73:52144) 2024-04-05T22:27:38.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 -- 172.21.15.73:0/1949700405 learned_addr learned my addr 172.21.15.73:0/1949700405 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5400640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b006c2d0 0x7f83b01e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b4a00640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83b00b2c00 0x7f83b01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 -- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83b00b2c00 msgr2=0x7f83b01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83b00b2c00 0x7f83b01543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 -- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b006c2d0 msgr2=0x7f83b01e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b006c2d0 0x7f83b01e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.724+0000 7f83b5e00640 1 -- 172.21.15.73:0/1949700405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009400 con 0x7f83b01441e0 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83a800cd50 tx=0x7f83a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83ad600640 1 -- 172.21.15.73:0/1949700405 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800f680 con 0x7f83b01441e0 2024-04-05T22:27:38.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83ad600640 1 -- 172.21.15.73:0/1949700405 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a800f800 con 0x7f83b01441e0 2024-04-05T22:27:38.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83ad600640 1 -- 172.21.15.73:0/1949700405 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800fbb0 con 0x7f83b01441e0 2024-04-05T22:27:38.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b01307b0 con 0x7f83b01441e0 2024-04-05T22:27:38.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.725+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b0154910 con 0x7f83b01441e0 2024-04-05T22:27:38.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.727+0000 7f83ad600640 1 -- 172.21.15.73:0/1949700405 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f83a800b040 con 0x7f83b01441e0 2024-04-05T22:27:38.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.728+0000 7f83ad600640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 0x7f839006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.728+0000 7f83ad600640 1 -- 172.21.15.73:0/1949700405 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83a8089fe0 con 0x7f83b01441e0 2024-04-05T22:27:38.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.729+0000 7f83b5400640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 0x7f839006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.729+0000 7f83bac65740 1 Processor -- start 2024-04-05T22:27:38.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.729+0000 7f83bac65740 1 -- start start 2024-04-05T22:27:38.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83b5400640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 0x7f839006d1e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83a4036000 tx=0x7f83a403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0eaf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0eaf840 0x562bc0edf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0e1f8c0 con 0x562bc0edc740 2024-04-05T22:27:38.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0e1d050 con 0x562bc0eaf840 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc0e18940 con 0x562bc0edc360 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83b4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83b4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52150/0 (socket says 172.21.15.73:52150) 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.730+0000 7f83b4a00640 1 -- 172.21.15.73:0/397695878 learned_addr learned my addr 172.21.15.73:0/397695878 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b5400640 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0eaf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 -- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 msgr2=0x562bc0eaf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0eaf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 -- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0eaf840 msgr2=0x562bc0edf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0eaf840 0x562bc0edf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 -- 172.21.15.73:0/397695878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc0de39e0 con 0x562bc0edc740 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83b4a00640 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83b0142ed0 tx=0x7f839c0019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10ffc075c9a7a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83a2000640 1 -- 172.21.15.73:0/397695878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c00b070 con 0x562bc0edc740 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83a2000640 1 -- 172.21.15.73:0/397695878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c005ce0 con 0x562bc0edc740 2024-04-05T22:27:38.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.731+0000 7f83a2000640 1 -- 172.21.15.73:0/397695878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c0072d0 con 0x562bc0edc740 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 -- 172.21.15.73:0/397695878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 msgr2=0x562bc0edf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f83b0142ed0 tx=0x7f839c0019a0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 -- 172.21.15.73:0/397695878 shutdown_connections 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0eaf840 0x562bc0edf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0edf0f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 --2- 172.21.15.73:0/397695878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0eaf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 -- 172.21.15.73:0/397695878 >> 172.21.15.73:0/397695878 conn(0x562bc0eb1570 msgr2=0x562bc0efa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 -- 172.21.15.73:0/397695878 shutdown_connections 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.732+0000 7f83bac65740 1 -- 172.21.15.73:0/397695878 wait complete. 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.733+0000 7f83bac65740 1 -- start start 2024-04-05T22:27:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.733+0000 7f83bac65740 1 Processor -- start 2024-04-05T22:27:38.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ec8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0edc740 0x562bc0ef0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0dba070 con 0x562bc0efa9e0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0db9d10 con 0x562bc0edc740 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc0b7ff20 con 0x562bc0edc360 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83bac65740 10 client.0 _ll_register_callbacks cb 0x562bc0e11fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.734+0000 7f83b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52152/0 (socket says 172.21.15.73:52152) 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 -- 172.21.15.73:0/1021715617 learned_addr learned my addr 172.21.15.73:0/1021715617 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5400640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ec8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b4a00640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0edc740 0x562bc0ef0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 -- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 msgr2=0x562bc0ec8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ec8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 -- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0edc740 msgr2=0x562bc0ef0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0edc740 0x562bc0ef0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 -- 172.21.15.73:0/1021715617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b00b1b60 con 0x562bc0efa9e0 2024-04-05T22:27:38.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f83b5e00640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f83a8007fd0 tx=0x7f83a8058cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c5bd7e29471b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.735+0000 7f8396a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80553c0 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83a8055520 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bc0e42ae0 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a80581a0 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 10 client.7764 fetch_fsmap learned FSMap version 63 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x562bc0e16fc0 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8074080 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f83a8020070 con 0x562bc0efa9e0 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 10 client.7764 fetch_fsmap finished waiting for FSMap version 63 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 10 client.7764 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 20 client.7764 populate_metadata read hostname 'smithi073' 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ec80d0 0x562bc0ede120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 4 client.7764 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:27:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.736+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 --> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc0b0c3c0 con 0x562bc0ec80d0 2024-04-05T22:27:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.737+0000 7f83b5400640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ec80d0 0x562bc0ede120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.737+0000 7f83b5400640 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ec80d0 0x562bc0ede120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3c284906036df4 server_cookie=fa224e7fdc1a03b6 in_seq=0 out_seq=0 2024-04-05T22:27:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.737+0000 7f8396a00640 10 client.7764 ms_handle_connect on v2:172.21.15.73:6833/3290176955 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f8396a00640 1 -- 172.21.15.73:0/1021715617 <== mds.0 v2:172.21.15.73:6833/3290176955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562bc0b0c3c0 con 0x562bc0ec80d0 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f8396a00640 10 client.7764 handle_command_reply: tid=0 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 client.7764 shutdown 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 msgr2=0x562bc0efbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f83a8007fd0 tx=0x7f83a8058cb0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 shutdown_connections 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ec80d0 0x562bc0ede120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0efa9e0 0x562bc0efbe80 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0edc740 0x562bc0ef0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 --2- 172.21.15.73:0/1021715617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ec8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f8396a00640 0 client.7764 ms_handle_reset on v2:172.21.15.73:6833/3290176955 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.738+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 >> 172.21.15.73:0/1021715617 conn(0x562bc0eaf840 msgr2=0x562bc0efd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.739+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 shutdown_connections 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.739+0000 7f83bac65740 1 -- 172.21.15.73:0/1021715617 wait complete. 2024-04-05T22:27:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.739+0000 7f83bac65740 20 client.7764 trim_cache size 0 max 16384 2024-04-05T22:27:38.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.755+0000 7f83bac65740 1 Processor -- start 2024-04-05T22:27:38.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.755+0000 7f83bac65740 1 -- start start 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc740 0x562bc0ed6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0eb4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0eb00d0 con 0x562bc0edc360 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0ec77b0 con 0x562bc0efa9e0 2024-04-05T22:27:38.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc0f13770 con 0x562bc0edc740 2024-04-05T22:27:38.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.756+0000 7f83b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52164/0 (socket says 172.21.15.73:52164) 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 -- 172.21.15.73:0/3571960289 learned_addr learned my addr 172.21.15.73:0/3571960289 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 -- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc740 msgr2=0x562bc0ed6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc740 0x562bc0ed6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 -- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 msgr2=0x562bc0eb4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0eb4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 -- 172.21.15.73:0/3571960289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc0e4bf80 con 0x562bc0edc360 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83b5400640 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f83a403a6b0 tx=0x7f83a4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2200db92469c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83a2a00640 1 -- 172.21.15.73:0/3571960289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a403f040 con 0x562bc0edc360 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83a2a00640 1 -- 172.21.15.73:0/3571960289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a404c040 con 0x562bc0edc360 2024-04-05T22:27:38.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.757+0000 7f83a2a00640 1 -- 172.21.15.73:0/3571960289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40515b0 con 0x562bc0edc360 2024-04-05T22:27:38.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 -- 172.21.15.73:0/3571960289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 msgr2=0x562bc0f139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f83a403a6b0 tx=0x7f83a4030000 comp rx=0 tx=0).stop 2024-04-05T22:27:38.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 -- 172.21.15.73:0/3571960289 shutdown_connections 2024-04-05T22:27:38.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0eb4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc740 0x562bc0ed6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 --2- 172.21.15.73:0/3571960289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc360 0x562bc0f139e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 -- 172.21.15.73:0/3571960289 >> 172.21.15.73:0/3571960289 conn(0x562bc0eaf840 msgr2=0x562bc0eaab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 -- 172.21.15.73:0/3571960289 shutdown_connections 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.758+0000 7f83bac65740 1 -- 172.21.15.73:0/3571960289 wait complete. 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.759+0000 7f83bac65740 1 -- start start 2024-04-05T22:27:38.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.759+0000 7f83bac65740 1 Processor -- start 2024-04-05T22:27:38.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ed5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0ee25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0ed65b0 con 0x562bc0edc740 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc0f1af80 con 0x562bc0efa9e0 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc0efd040 con 0x562bc0edc360 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83bac65740 10 client.0 _ll_register_callbacks cb 0x7f83a8054ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83b4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83b4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52180/0 (socket says 172.21.15.73:52180) 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.760+0000 7f83b4a00640 1 -- 172.21.15.73:0/3754814487 learned_addr learned my addr 172.21.15.73:0/3754814487 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b5400640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ed5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 -- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 msgr2=0x562bc0ed5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ed5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 -- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 msgr2=0x562bc0ee25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0ee25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 -- 172.21.15.73:0/3754814487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b00b2120 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b5400640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ed5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83b4a00640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f839c000f60 tx=0x7f839c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a36641128eea092 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f8396a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c007c20 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f839c013680 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.761+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f839c005ce0 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c013840 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 10 client.7770 fetch_fsmap learned FSMap version 63 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x562bc0db9280 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c005ce0 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f839c01d040 con 0x562bc0edc740 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 10 client.7770 fetch_fsmap finished waiting for FSMap version 63 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 10 client.7770 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 20 client.7770 populate_metadata read hostname 'smithi073' 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ee2b30 0x562bc0ee2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 4 client.7770 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:27:38.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 --> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562bc0b1d6c0 con 0x562bc0ee2b30 2024-04-05T22:27:38.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.762+0000 7f83b5400640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ee2b30 0x562bc0ee2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83b5400640 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ee2b30 0x562bc0ee2f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a124a958d743ef90 server_cookie=82456add2dc24c0d in_seq=0 out_seq=0 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f8396a00640 10 client.7770 ms_handle_connect on v2:172.21.15.73:6833/3290176955 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f8396a00640 1 -- 172.21.15.73:0/3754814487 <== mds.0 v2:172.21.15.73:6833/3290176955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562bc0b1d6c0 con 0x562bc0ee2b30 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f8396a00640 10 client.7770 handle_command_reply: tid=0 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 client.7770 shutdown 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 msgr2=0x562bc0ed5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f839c000f60 tx=0x7f839c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 shutdown_connections 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:6833/3290176955,v1:172.21.15.73:6835/3290176955] conn(0x562bc0ee2b30 0x562bc0ee2f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562bc0efa9e0 0x562bc0ee25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562bc0edc740 0x562bc0ed5ca0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 --2- 172.21.15.73:0/3754814487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562bc0edc360 0x562bc0ed5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 >> 172.21.15.73:0/3754814487 conn(0x562bc0eaf840 msgr2=0x562bc0efd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.763+0000 7f8396a00640 0 client.7770 ms_handle_reset on v2:172.21.15.73:6833/3290176955 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 shutdown_connections 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83bac65740 1 -- 172.21.15.73:0/3754814487 wait complete. 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83bac65740 20 client.7770 trim_cache size 0 max 16384 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:27:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 7623, 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.652692858, 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 63, 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 107, 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 107, 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 11.587939201999999 2024-04-05T22:27:38.767 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 msgr2=0x7f839006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 0x7f839006d1e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83a4036000 tx=0x7f83a403c040 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 msgr2=0x7f83b01e8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f83a800cd50 tx=0x7f83a800a650 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.764+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 shutdown_connections 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f839006ad20 0x7f839006d1e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b01441e0 0x7f83b01e8340 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83b00b2c00 0x7f83b01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 --2- 172.21.15.73:0/1949700405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b006c2d0 0x7f83b01e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 >> 172.21.15.73:0/1949700405 conn(0x7f83b00b0830 msgr2=0x7f83b00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 shutdown_connections 2024-04-05T22:27:38.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.765+0000 7f83b6800640 1 -- 172.21.15.73:0/1949700405 wait complete. 2024-04-05T22:27:38.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:27:38.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.981+0000 7ff3b2e00640 1 Processor -- start 2024-04-05T22:27:38.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.981+0000 7ff3b2e00640 1 -- start start 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac15e620 0x7ff3ac15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b2e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06b500 con 0x7ff3ac06bc70 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b2e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac0b20d0 con 0x7ff3ac158860 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac15eff0 con 0x7ff3ac15e620 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac15e620 0x7ff3ac15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac15e620 0x7ff3ac15eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52184/0 (socket says 172.21.15.73:52184) 2024-04-05T22:27:38.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b1000640 1 -- 172.21.15.73:0/2152092356 learned_addr learned my addr 172.21.15.73:0/2152092356 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b2400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff3ac06bc70 0x7ff3ac06c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49650/0 (socket says 172.21.15.73:49650) 2024-04-05T22:27:38.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b1000640 1 -- 172.21.15.73:0/2152092356 >> v1:172.21.15.157:6789/0 conn(0x7ff3ac158860 legacy=0x7ff3ac0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:38.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3b1000640 1 -- 172.21.15.73:0/2152092356 >> v1:172.21.15.73:6789/0 conn(0x7ff3ac06bc70 legacy=0x7ff3ac06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.984+0000 7ff3b1000640 1 -- 172.21.15.73:0/2152092356 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0b23a0 con 0x7ff3ac15e620 2024-04-05T22:27:38.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.983+0000 7ff3abe00640 1 -- 172.21.15.73:0/2152092356 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853617550 0 0) 0x7ff3ac06b500 con 0x7ff3ac06bc70 2024-04-05T22:27:38.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3b1000640 1 --2- 172.21.15.73:0/2152092356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac15eab0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff398009870 tx=0x7ff39802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1990638e46af0e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3abe00640 1 -- 172.21.15.73:0/2152092356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398031870 con 0x7ff3ac15e620 2024-04-05T22:27:38.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3abe00640 1 -- 172.21.15.73:0/2152092356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3980319d0 con 0x7ff3ac15e620 2024-04-05T22:27:38.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3abe00640 1 -- 172.21.15.73:0/2152092356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398031d60 con 0x7ff3ac15e620 2024-04-05T22:27:38.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3b2e00640 1 -- 172.21.15.73:0/2152092356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 msgr2=0x7ff3ac15eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/2152092356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac15eab0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff398009870 tx=0x7ff39802b910 comp rx=0 tx=0).stop 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3b2e00640 1 -- 172.21.15.73:0/2152092356 shutdown_connections 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.985+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/2152092356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac15eab0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.986+0000 7ff3b2e00640 1 -- 172.21.15.73:0/2152092356 >> 172.21.15.73:0/2152092356 conn(0x7ff3ac0b06c0 msgr2=0x7ff3ac0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.986+0000 7ff3b2e00640 1 -- 172.21.15.73:0/2152092356 shutdown_connections 2024-04-05T22:27:38.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.986+0000 7ff3b2e00640 1 -- 172.21.15.73:0/2152092356 wait complete. 2024-04-05T22:27:38.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.987+0000 7ff3b2e00640 1 Processor -- start 2024-04-05T22:27:38.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.987+0000 7ff3b2e00640 1 -- start start 2024-04-05T22:27:38.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac06bc70 0x7ff3ac1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15a360 0x7ff3ac155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac158d80 con 0x7ff3ac15e620 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac159060 con 0x7ff3ac15a360 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.988+0000 7ff3b2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac159340 con 0x7ff3ac06bc70 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52188/0 (socket says 172.21.15.73:52188) 2024-04-05T22:27:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 -- 172.21.15.73:0/3730586284 learned_addr learned my addr 172.21.15.73:0/3730586284 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:38.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1a00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac06bc70 0x7ff3ac1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 -- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac06bc70 msgr2=0x7ff3ac1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac06bc70 0x7ff3ac1537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 -- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15a360 msgr2=0x7ff3ac155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:38.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15a360 0x7ff3ac155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 -- 172.21.15.73:0/3730586284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980094d0 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.989+0000 7ff3b1000640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff3980090b0 tx=0x7ff3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.990+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39803ad30 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.990+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3980443f0 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.990+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980447a0 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.990+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac159620 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.990+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac153d00 con 0x7ff3ac15e620 2024-04-05T22:27:38.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.991+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac153ef0 con 0x7ff3ac15e620 2024-04-05T22:27:38.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.992+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7ff39804d450 con 0x7ff3ac15e620 2024-04-05T22:27:38.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.993+0000 7ff3a9600640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 0x7ff38c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:38.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.993+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff39802d040 con 0x7ff3ac15e620 2024-04-05T22:27:38.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.993+0000 7ff3b1a00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 0x7ff38c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:38.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.994+0000 7ff3b1a00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 0x7ff38c06d1e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0003d90 tx=0x7ff3a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:38.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:38.997+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff398076c60 con 0x7ff3ac15e620 2024-04-05T22:27:39.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:27:39.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:27:39.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.233+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3ac159b40 con 0x7ff3ac15e620 2024-04-05T22:27:39.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.234+0000 7ff3a9600640 1 -- 172.21.15.73:0/3730586284 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff3ac159b40 con 0x7ff3ac15e620 2024-04-05T22:27:39.233 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.236+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 msgr2=0x7ff38c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.236+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 0x7ff38c06d1e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0003d90 tx=0x7ff3a0009040 comp rx=0 tx=0).stop 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.236+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 msgr2=0x7ff3ac159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.236+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff3980090b0 tx=0x7ff3980046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 shutdown_connections 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38c06ad20 0x7ff38c06d1e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15a360 0x7ff3ac155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac15e620 0x7ff3ac159e20 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 --2- 172.21.15.73:0/3730586284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac06bc70 0x7ff3ac1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:39.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 >> 172.21.15.73:0/3730586284 conn(0x7ff3ac0b06c0 msgr2=0x7ff3ac072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:39.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 shutdown_connections 2024-04-05T22:27:39.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:39.237+0000 7ff3b2e00640 1 -- 172.21.15.73:0/3730586284 wait complete. 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:27:39.253 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:27:39.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.254 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:27:39.344 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:27:39.344 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:39.345 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:27:39.346 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:27:39.347 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.347 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:27:39.347 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:27:39.347 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:27:39.347 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:39.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.348 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:27:39.482 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:27:39.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.486 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:27:39.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:27:39.541 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:27:39.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.720 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:27:39.771 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:27:39.771 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:27:39.771 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:27:39.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:27:39.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:27:39.863 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:27:39.916 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:27:39.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:27:39.916 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:27:39.965 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:27:39.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:39.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:40.097 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:40.097 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:27:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:27:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:27:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:27:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:27:40.132 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:27:40.132 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:27:40.132 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:27:40.132 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:27:40.132 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:27:40.136 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:27:40.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:40.136 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:27:40.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:40.210 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:27:40.272 INFO:teuthology.orchestra.run.smithi073.stdout:client7782 2024-04-05T22:27:40.272 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:40.272 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:40.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:27:40.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.528+0000 7f44a1800640 1 Processor -- start 2024-04-05T22:27:40.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.528+0000 7f44a1800640 1 -- start start 2024-04-05T22:27:40.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.530+0000 7f44a1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c15af40 0x7f449c0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.530+0000 7f44a1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f449c15d1b0 0x7f449c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.530+0000 7f44a1800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c15d750 con 0x7f449c0b48f0 2024-04-05T22:27:40.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.530+0000 7f44a1800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c0b4d80 con 0x7f449c15d1b0 2024-04-05T22:27:40.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.530+0000 7f44a1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c15da30 con 0x7f449c15af40 2024-04-05T22:27:40.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f449c15d1b0 0x7f449c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f449c15d1b0 0x7f449c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52202/0 (socket says 172.21.15.73:52202) 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 -- 172.21.15.73:0/1520637214 learned_addr learned my addr 172.21.15.73:0/1520637214 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f449be00640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c15af40 0x7f449c0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 -- 172.21.15.73:0/1520637214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c15af40 msgr2=0x7f449c0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c15af40 0x7f449c0b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 -- 172.21.15.73:0/1520637214 >> v1:172.21.15.157:6789/0 conn(0x7f449c0b48f0 legacy=0x7f449c15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.531+0000 7f4493400640 1 -- 172.21.15.73:0/1520637214 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c15dd10 con 0x7f449c15d1b0 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.532+0000 7f4493400640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c0b43b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4484009870 tx=0x7f448402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf34065b845e35e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f449b400640 1 -- 172.21.15.73:0/1520637214 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448402f840 con 0x7f449c15d1b0 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f449b400640 1 -- 172.21.15.73:0/1520637214 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448402f9a0 con 0x7f449c15d1b0 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f449b400640 1 -- 172.21.15.73:0/1520637214 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448402fd30 con 0x7f449c15d1b0 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 -- 172.21.15.73:0/1520637214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 msgr2=0x7f449c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c0b43b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4484009870 tx=0x7f448402ba20 comp rx=0 tx=0).stop 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 -- 172.21.15.73:0/1520637214 shutdown_connections 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c0b43b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 --2- 172.21.15.73:0/1520637214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c15af40 0x7f449c0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 -- 172.21.15.73:0/1520637214 >> 172.21.15.73:0/1520637214 conn(0x7f449c0b5d50 msgr2=0x7f449c0b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.533+0000 7f44a1800640 1 -- 172.21.15.73:0/1520637214 shutdown_connections 2024-04-05T22:27:40.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.534+0000 7f44a1800640 1 -- 172.21.15.73:0/1520637214 wait complete. 2024-04-05T22:27:40.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.535+0000 7f44a1800640 1 Processor -- start 2024-04-05T22:27:40.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.535+0000 7f44a1800640 1 -- start start 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b48f0 0x7f449c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0cd8f0 0x7f449c1f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0cfc70 con 0x7f449c15d1b0 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0cff10 con 0x7f449c0cd8f0 2024-04-05T22:27:40.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.536+0000 7f44a1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0d01f0 con 0x7f449c0b48f0 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52210/0 (socket says 172.21.15.73:52210) 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 -- 172.21.15.73:0/961542639 learned_addr learned my addr 172.21.15.73:0/961542639 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 -- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b48f0 msgr2=0x7f449c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f449be00640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b48f0 0x7f449c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b48f0 0x7f449c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 -- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0cd8f0 msgr2=0x7f449c1f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0cd8f0 0x7f449c1f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 -- 172.21.15.73:0/961542639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840094d0 con 0x7f449c15d1b0 2024-04-05T22:27:40.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.537+0000 7f4493400640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f44840090b0 tx=0x7f4484004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.538+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448403c040 con 0x7f449c15d1b0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.538+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484037c60 con 0x7f449c15d1b0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.538+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840423f0 con 0x7f449c15d1b0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.538+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0d04d0 con 0x7f449c15d1b0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.538+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0ccbb0 con 0x7f449c15d1b0 2024-04-05T22:27:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.539+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c0d06c0 con 0x7f449c15d1b0 2024-04-05T22:27:40.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.540+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f4484042570 con 0x7f449c15d1b0 2024-04-05T22:27:40.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.541+0000 7f4498c00640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 0x7f447806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:40.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.541+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44840ab720 con 0x7f449c15d1b0 2024-04-05T22:27:40.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.541+0000 7f449be00640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 0x7f447806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:40.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.542+0000 7f449be00640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 0x7f447806d1e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f448c006fd0 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:40.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.546+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f44840766d0 con 0x7f449c15d1b0 2024-04-05T22:27:40.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.777+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f449c04e160 con 0x7f449c15d1b0 2024-04-05T22:27:40.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.777+0000 7f4498c00640 1 -- 172.21.15.73:0/961542639 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4484079cf0 con 0x7f449c15d1b0 2024-04-05T22:27:40.777 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:27:40.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 msgr2=0x7f447806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 0x7f447806d1e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f448c006fd0 tx=0x7f448c008040 comp rx=0 tx=0).stop 2024-04-05T22:27:40.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 msgr2=0x7f449c1f0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f44840090b0 tx=0x7f4484004730 comp rx=0 tx=0).stop 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 shutdown_connections 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f447806ad20 0x7f447806d1e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0cd8f0 0x7f449c1f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f449c15d1b0 0x7f449c1f0410 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 --2- 172.21.15.73:0/961542639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b48f0 0x7f449c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 >> 172.21.15.73:0/961542639 conn(0x7f449c0b5d50 msgr2=0x7f449c158740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 shutdown_connections 2024-04-05T22:27:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:40.780+0000 7f44a1800640 1 -- 172.21.15.73:0/961542639 wait complete. 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:27:40.798 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:27:40.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:40.799 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:27:40.856 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:27:40.856 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:27:40.857 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42226sec preferred_lft 42226sec 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:27:40.858 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:27:40.859 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.859 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:27:40.859 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:27:40.859 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:27:40.859 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:27:40.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:40.860 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:27:40.955 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:27:40.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:40.960 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:27:41.014 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:27:41.015 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:27:41.015 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:27:41.015 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:27:41.015 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:27:41.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.015 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:27:41.015 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:27:41.015 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:27:41.015 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:27:41.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:27:41.016 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:27:41.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.195 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:27:41.246 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:27:41.247 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:27:41.248 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:27:41.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:27:41.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:27:41.341 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:41.395 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:27:41.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:27:41.395 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:41.444 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:27:41.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:41.579 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:41.579 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:27:41.616 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:27:41.616 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:27:41.616 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:27:41.616 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:27:41.617 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:27:41.621 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:27:41.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.621 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:27:41.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:41.700 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:27:41.756 INFO:teuthology.orchestra.run.smithi157.stdout:client5363 2024-04-05T22:27:41.757 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:27:41.757 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:41.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:27:42.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.034+0000 7f5be6e00640 1 Processor -- start 2024-04-05T22:27:42.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.034+0000 7f5be6e00640 1 -- start start 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be6e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5be015ae50 0x7f5be00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be6e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be015b230 con 0x7f5be00cf3e0 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be6e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be015f4a0 con 0x7f5be00cce90 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be6e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be00cd320 con 0x7f5be015ae50 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be4e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5be00cf3e0 0x7f5be00b1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49660/0 (socket says 172.21.15.73:49660) 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.036+0000 7f5be4e00640 1 -- 172.21.15.73:0/2746435470 learned_addr learned my addr 172.21.15.73:0/2746435470 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/2746435470 >> v2:172.21.15.157:3300/0 conn(0x7f5be015ae50 0x7f5be00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002515420 0 0) 0x7f5be015b230 con 0x7f5be00cf3e0 2024-04-05T22:27:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc4003520 con 0x7f5be00cf3e0 2024-04-05T22:27:42.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544681929 0 0) 0x7f5bc4003520 con 0x7f5be00cf3e0 2024-04-05T22:27:42.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 >> v2:172.21.15.157:3300/0 conn(0x7f5be015ae50 msgr2=0x7f5be00b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 --2- 172.21.15.73:0/2746435470 >> v2:172.21.15.157:3300/0 conn(0x7f5be015ae50 0x7f5be00b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 >> v1:172.21.15.157:6789/0 conn(0x7f5be00cce90 legacy=0x7f5be00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be015f780 con 0x7f5be00cf3e0 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5bd8001670 con 0x7f5be00cf3e0 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 >> v1:172.21.15.73:6789/0 conn(0x7f5be00cf3e0 legacy=0x7f5be00b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.037+0000 7f5bdf400640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40047c0 0x7f5bc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdf400640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc4005180 0x7f5bc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc400c0f0 con 0x7f5bc4003970 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc400c260 con 0x7f5bc4005180 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc400c420 con 0x7f5bc40047c0 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40047c0 msgr2=0x7f5bc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40047c0 0x7f5bc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc4005180 msgr2=0x7f5bc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc4005180 0x7f5bc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2746435470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4003520 con 0x7f5bc4003970 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.038+0000 7f5be4e00640 1 -- 172.21.15.73:0/2746435470 reap_dead start 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.039+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009fc0 tx=0x7f5bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551c9192b8c1ab65 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.039+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd002cc60 con 0x7f5bc4003970 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.039+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000f040 con 0x7f5bc4003970 2024-04-05T22:27:42.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.039+0000 7f5bdf400640 1 -- 172.21.15.73:0/2746435470 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0034440 con 0x7f5bc4003970 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 -- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 msgr2=0x7f5bc4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009fc0 tx=0x7f5bc40057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 -- 172.21.15.73:0/2746435470 shutdown_connections 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 --2- 172.21.15.73:0/2746435470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 -- 172.21.15.73:0/2746435470 >> 172.21.15.73:0/2746435470 conn(0x7f5be006a0d0 msgr2=0x7f5be006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 -- 172.21.15.73:0/2746435470 shutdown_connections 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.040+0000 7f5be6e00640 1 -- 172.21.15.73:0/2746435470 wait complete. 2024-04-05T22:27:42.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.042+0000 7f5be6e00640 1 Processor -- start 2024-04-05T22:27:42.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.042+0000 7f5be6e00640 1 -- start start 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 0x7f5be01f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b690 0x7f5be01f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be014ec40 con 0x7f5bc4003970 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be014f850 con 0x7f5be006b690 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5be6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0070db0 con 0x7f5be0070930 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5bdfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5bdfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52232/0 (socket says 172.21.15.73:52232) 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.043+0000 7f5bdfe00640 1 -- 172.21.15.73:0/1949872762 learned_addr learned my addr 172.21.15.73:0/1949872762 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:42.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bd7400640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 0x7f5be01f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 -- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 msgr2=0x7f5be01f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 0x7f5be01f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 -- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b690 msgr2=0x7f5be01f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5be4e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b690 0x7f5be01f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b690 0x7f5be01f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 -- 172.21.15.73:0/1949872762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0071060 con 0x7f5bc4003970 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bd7400640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 0x7f5be01f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:42.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdfe00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5bd002c6d0 tx=0x7f5bd0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0034c50 con 0x7f5bc4003970 2024-04-05T22:27:42.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0034db0 con 0x7f5bc4003970 2024-04-05T22:27:42.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be0071340 con 0x7f5bc4003970 2024-04-05T22:27:42.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003b90 con 0x7f5bc4003970 2024-04-05T22:27:42.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.044+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be01f5e60 con 0x7f5bc4003970 2024-04-05T22:27:42.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.045+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0068160 con 0x7f5bc4003970 2024-04-05T22:27:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.046+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f5bd0040450 con 0x7f5bc4003970 2024-04-05T22:27:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.048+0000 7f5bdcc00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 0x7f5bbc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.048+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bd00abb10 con 0x7f5bc4003970 2024-04-05T22:27:42.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.048+0000 7f5bd7400640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 0x7f5bbc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.050+0000 7f5bd7400640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 0x7f5bbc06d1e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5be00c70c0 tx=0x7f5bc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.053+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5bd0076d90 con 0x7f5bc4003970 2024-04-05T22:27:42.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.290+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5be00af9c0 con 0x7f5bc4003970 2024-04-05T22:27:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.313+0000 7f5bdcc00640 1 -- 172.21.15.73:0/1949872762 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5bd007a3b0 con 0x7f5bc4003970 2024-04-05T22:27:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 msgr2=0x7f5bbc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 0x7f5bbc06d1e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5be00c70c0 tx=0x7f5bc8009040 comp rx=0 tx=0).stop 2024-04-05T22:27:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 msgr2=0x7f5be00c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5bd002c6d0 tx=0x7f5bd0009750 comp rx=0 tx=0).stop 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 shutdown_connections 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bbc06ad20 0x7f5bbc06d1e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b690 0x7f5be01f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0070930 0x7f5be01f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 --2- 172.21.15.73:0/1949872762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc4003970 0x7f5be00c5b20 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 >> 172.21.15.73:0/1949872762 conn(0x7f5be006a0d0 msgr2=0x7f5be00b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 shutdown_connections 2024-04-05T22:27:42.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.316+0000 7f5be6e00640 1 -- 172.21.15.73:0/1949872762 wait complete. 2024-04-05T22:27:42.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:27:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.530+0000 7f50cca00640 1 Processor -- start 2024-04-05T22:27:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.530+0000 7f50cca00640 1 -- start start 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50cca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50c8070f50 0x7f50c806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50cca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c80b2f30 0x7f50c80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50cca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c80b3c40 con 0x7f50c8146490 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50cca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c806c490 con 0x7f50c8070f50 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50cca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c806c6f0 con 0x7f50c80b2f30 2024-04-05T22:27:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50c6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50c8070f50 0x7f50c806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50c6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50c8070f50 0x7f50c806d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52246/0 (socket says 172.21.15.73:52246) 2024-04-05T22:27:42.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.532+0000 7f50c6a00640 1 -- 172.21.15.73:0/1348080011 learned_addr learned my addr 172.21.15.73:0/1348080011 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:42.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c7e00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c80b2f30 0x7f50c80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c6a00640 1 -- 172.21.15.73:0/1348080011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c80b2f30 msgr2=0x7f50c80b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c6a00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c80b2f30 0x7f50c80b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c6a00640 1 -- 172.21.15.73:0/1348080011 >> v1:172.21.15.73:6789/0 conn(0x7f50c8146490 legacy=0x7f50c8070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c6a00640 1 -- 172.21.15.73:0/1348080011 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c806c9d0 con 0x7f50c8070f50 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.533+0000 7f50c6000640 1 -- 172.21.15.73:0/1348080011 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721053434 0 0) 0x7f50b8001110 con 0x7f50c8146490 2024-04-05T22:27:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.534+0000 7f50c6a00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8070f50 0x7f50c806d4b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009870 tx=0x7f50b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a10fce71a54a258 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.534+0000 7f50c6000640 1 -- 172.21.15.73:0/1348080011 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4030840 con 0x7f50c8070f50 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.534+0000 7f50c6000640 1 -- 172.21.15.73:0/1348080011 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b40309a0 con 0x7f50c8070f50 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.534+0000 7f50c6000640 1 -- 172.21.15.73:0/1348080011 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4030d30 con 0x7f50c8070f50 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 -- 172.21.15.73:0/1348080011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8070f50 msgr2=0x7f50c806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8070f50 0x7f50c806d4b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009870 tx=0x7f50b402b980 comp rx=0 tx=0).stop 2024-04-05T22:27:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 -- 172.21.15.73:0/1348080011 shutdown_connections 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c80b2f30 0x7f50c80b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 --2- 172.21.15.73:0/1348080011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8070f50 0x7f50c806d4b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 -- 172.21.15.73:0/1348080011 >> 172.21.15.73:0/1348080011 conn(0x7f50c80b0830 msgr2=0x7f50c80b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 -- 172.21.15.73:0/1348080011 shutdown_connections 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.535+0000 7f50cca00640 1 -- 172.21.15.73:0/1348080011 wait complete. 2024-04-05T22:27:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.537+0000 7f50cca00640 1 Processor -- start 2024-04-05T22:27:42.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.537+0000 7f50cca00640 1 -- start start 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 0x7f50c813f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 0x7f50c81f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c813fe30 con 0x7f50c8146490 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8158ca0 con 0x7f50c80b2f30 2024-04-05T22:27:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50cca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8160090 con 0x7f50c8070f50 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52262/0 (socket says 172.21.15.73:52262) 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 -- 172.21.15.73:0/1170654097 learned_addr learned my addr 172.21.15.73:0/1170654097 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c6a00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 0x7f50c81f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 -- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 msgr2=0x7f50c813f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7400640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 0x7f50c813f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 0x7f50c813f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 -- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 msgr2=0x7f50c81f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 0x7f50c81f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.538+0000 7f50c7e00640 1 -- 172.21.15.73:0/1170654097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40094d0 con 0x7f50c8146490 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50c6a00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 0x7f50c81f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50c7400640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 0x7f50c813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50c7e00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007e40 tx=0x7f50c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014620 con 0x7f50c8146490 2024-04-05T22:27:42.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c000ee90 con 0x7f50c8146490 2024-04-05T22:27:42.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00149e0 con 0x7f50c8146490 2024-04-05T22:27:42.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c81f2300 con 0x7f50c8146490 2024-04-05T22:27:42.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.539+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c813d340 con 0x7f50c8146490 2024-04-05T22:27:42.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.540+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c813d4e0 con 0x7f50c8146490 2024-04-05T22:27:42.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.542+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f50c0014b40 con 0x7f50c8146490 2024-04-05T22:27:42.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.543+0000 7f50bf400640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 0x7f50a806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.543+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50c0089d40 con 0x7f50c8146490 2024-04-05T22:27:42.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.543+0000 7f50c7400640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 0x7f50a806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.545+0000 7f50c7400640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 0x7f50a806d1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50b8000f60 tx=0x7f50b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.548+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f50c0054e00 con 0x7f50c8146490 2024-04-05T22:27:42.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.767+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c8146870 con 0x7f50a806ad20 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.771+0000 7f50bf400640 1 -- 172.21.15.73:0/1170654097 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f50b8001ca0 con 0x7f50a806ad20 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:42.770 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:27:42.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 msgr2=0x7f50a806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 0x7f50a806d1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50b8000f60 tx=0x7f50b8009040 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 msgr2=0x7f50c8143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007e40 tx=0x7f50c000e4b0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 shutdown_connections 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f50a806ad20 0x7f50a806d1e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c8146490 0x7f50c8143a80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c80b2f30 0x7f50c81f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 --2- 172.21.15.73:0/1170654097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50c8070f50 0x7f50c813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 >> 172.21.15.73:0/1170654097 conn(0x7f50c80b0830 msgr2=0x7f50c8070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.773+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 shutdown_connections 2024-04-05T22:27:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.774+0000 7f50cca00640 1 -- 172.21.15.73:0/1170654097 wait complete. 2024-04-05T22:27:42.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:27:42.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.979+0000 7fb575600640 1 Processor -- start 2024-04-05T22:27:42.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.980+0000 7fb575600640 1 -- start start 2024-04-05T22:27:42.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb5700cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5700b1950 0x7fb570164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 0x7fb570161850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5701657c0 con 0x7fb5700b1950 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb570070d20 con 0x7fb570165310 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb575600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015f260 con 0x7fb57015ee80 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5700b1950 0x7fb570164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5700b1950 0x7fb570164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52272/0 (socket says 172.21.15.73:52272) 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 -- 172.21.15.73:0/3850621633 learned_addr learned my addr 172.21.15.73:0/3850621633 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb56ea00640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb5700cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb56f400640 1 --2- 172.21.15.73:0/3850621633 >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 0x7fb570161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 -- 172.21.15.73:0/3850621633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 msgr2=0x7fb5700cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb5700cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 -- 172.21.15.73:0/3850621633 >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 msgr2=0x7fb570161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:42.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 --2- 172.21.15.73:0/3850621633 >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 0x7fb570161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb566000640 1 -- 172.21.15.73:0/3850621633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57006f5b0 con 0x7fb5700b1950 2024-04-05T22:27:42.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb56ea00640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb5700cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:42.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.982+0000 7fb56f400640 1 --2- 172.21.15.73:0/3850621633 >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 0x7fb570161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.983+0000 7fb566000640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5700b1950 0x7fb570164dd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb5700cf710 tx=0x7fb55802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f6033132e39fa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.984+0000 7fb56e000640 1 -- 172.21.15.73:0/3850621633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558031870 con 0x7fb5700b1950 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.984+0000 7fb56e000640 1 -- 172.21.15.73:0/3850621633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5580319d0 con 0x7fb5700b1950 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.984+0000 7fb56e000640 1 -- 172.21.15.73:0/3850621633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558031d60 con 0x7fb5700b1950 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.984+0000 7fb575600640 1 -- 172.21.15.73:0/3850621633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5700b1950 msgr2=0x7fb570164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.984+0000 7fb575600640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5700b1950 0x7fb570164dd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb5700cf710 tx=0x7fb55802b910 comp rx=0 tx=0).stop 2024-04-05T22:27:42.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 -- 172.21.15.73:0/3850621633 shutdown_connections 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5700b1950 0x7fb570164dd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 --2- 172.21.15.73:0/3850621633 >> v2:172.21.15.157:3300/0 conn(0x7fb570165310 0x7fb570161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 --2- 172.21.15.73:0/3850621633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb5700cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 -- 172.21.15.73:0/3850621633 >> 172.21.15.73:0/3850621633 conn(0x7fb570069cb0 msgr2=0x7fb57006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 -- 172.21.15.73:0/3850621633 shutdown_connections 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.985+0000 7fb575600640 1 -- 172.21.15.73:0/3850621633 wait complete. 2024-04-05T22:27:42.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.986+0000 7fb575600640 1 Processor -- start 2024-04-05T22:27:42.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.986+0000 7fb575600640 1 -- start start 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5700b1950 0x7fb5701f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb57014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57014e240 con 0x7fb570165310 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701f9a30 con 0x7fb5700b1950 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.987+0000 7fb575600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701f8e20 con 0x7fb57015ee80 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52278/0 (socket says 172.21.15.73:52278) 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 -- 172.21.15.73:0/2454400607 learned_addr learned my addr 172.21.15.73:0/2454400607 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:42.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb566000640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb57014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 -- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 msgr2=0x7fb57014dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56ea00640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5700b1950 0x7fb5701f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb57014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 -- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5700b1950 msgr2=0x7fb5701f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5700b1950 0x7fb5701f8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 -- 172.21.15.73:0/2454400607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5580094d0 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.988+0000 7fb56f400640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb56800cd50 tx=0x7fb56800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.989+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800f680 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.989+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56800f800 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.989+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800fbb0 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.989+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5701f9100 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.989+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5701f96c0 con 0x7fb570165310 2024-04-05T22:27:42.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.990+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570068160 con 0x7fb570165310 2024-04-05T22:27:42.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.991+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7fb56800b040 con 0x7fb570165310 2024-04-05T22:27:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.992+0000 7fb567400640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 0x7fb54c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.992+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb568089fb0 con 0x7fb570165310 2024-04-05T22:27:42.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.993+0000 7fb56ea00640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 0x7fb54c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:42.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.994+0000 7fb56ea00640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 0x7fb54c06d1e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb560006fd0 tx=0x7fb560008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:42.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:42.996+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb5680bd650 con 0x7fb570165310 2024-04-05T22:27:43.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.187+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb5701f9b70 con 0x7fb570165310 2024-04-05T22:27:43.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.188+0000 7fb567400640 1 -- 172.21.15.73:0/2454400607 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb56801d020 con 0x7fb570165310 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 msgr2=0x7fb54c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 0x7fb54c06d1e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb560006fd0 tx=0x7fb560008040 comp rx=0 tx=0).stop 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 msgr2=0x7fb5700c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb56800cd50 tx=0x7fb56800a650 comp rx=0 tx=0).stop 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 shutdown_connections 2024-04-05T22:27:43.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb54c06ad20 0x7fb54c06d1e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb570165310 0x7fb5700c4340 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57015ee80 0x7fb57014dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 --2- 172.21.15.73:0/2454400607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5700b1950 0x7fb5701f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 >> 172.21.15.73:0/2454400607 conn(0x7fb570069cb0 msgr2=0x7fb57015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 shutdown_connections 2024-04-05T22:27:43.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.191+0000 7fb575600640 1 -- 172.21.15.73:0/2454400607 wait complete. 2024-04-05T22:27:43.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:43.208 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609) 2024-04-05T22:27:43.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:43.307 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/fake) 2024-04-05T22:27:43.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:43.435 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:27:43.486 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:27:43.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:43.487 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609) 2024-04-05T22:27:43.584 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:43.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol44000/.meta 2024-04-05T22:27:43.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol44000 2024-04-05T22:27:43.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.903+0000 7f1895400640 1 Processor -- start 2024-04-05T22:27:43.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.903+0000 7f1895400640 1 -- start start 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.904+0000 7f1895400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f18900cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.904+0000 7f1895400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18901654e0 0x7f189006b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.904+0000 7f1895400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189015f260 con 0x7f18900b1950 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.904+0000 7f1895400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1890070d20 con 0x7f18901654e0 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.904+0000 7f1895400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189006f5b0 con 0x7f189015ee80 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188de00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f18900b1950 0x7f1890164dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49680/0 (socket says 172.21.15.73:49680) 2024-04-05T22:27:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188de00640 1 -- 172.21.15.73:0/2415736547 learned_addr learned my addr 172.21.15.73:0/2415736547 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188f200640 1 --2- 172.21.15.73:0/2415736547 >> v2:172.21.15.157:3300/0 conn(0x7f18901654e0 0x7f189006b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188e800640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f18900cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829673926 0 0) 0x7f187c001110 con 0x7f18900b1950 2024-04-05T22:27:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.905+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1874003520 con 0x7f18900b1950 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3864722529 0 0) 0x7f1874003520 con 0x7f18900b1950 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 msgr2=0x7f18900cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f18900cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 >> v2:172.21.15.157:3300/0 conn(0x7f18901654e0 msgr2=0x7f189006b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 --2- 172.21.15.73:0/2415736547 >> v2:172.21.15.157:3300/0 conn(0x7f18901654e0 0x7f189006b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189006fb70 con 0x7f18900b1950 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f187c001800 con 0x7f18900b1950 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 >> v1:172.21.15.73:6789/0 conn(0x7f18900b1950 legacy=0x7f1890164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1874003970 0x7f1874003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18740047c0 0x7f1874004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f1874005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189006f5b0 con 0x7f18740047c0 2024-04-05T22:27:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187400b980 con 0x7f1874003970 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187400bac0 con 0x7f189015ee80 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.906+0000 7f188de00640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18740047c0 0x7f1874004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188e800640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1874003970 0x7f1874003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188de00640 1 -- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 msgr2=0x7f1874005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188f200640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f1874005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188de00640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189015ee80 0x7f1874005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188de00640 1 -- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1874003970 msgr2=0x7f1874003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188de00640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1874003970 0x7f1874003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188de00640 1 -- 172.21.15.73:0/2415736547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874003520 con 0x7f18740047c0 2024-04-05T22:27:43.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.907+0000 7f188e800640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1874003970 0x7f1874003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:27:43.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.908+0000 7f188de00640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18740047c0 0x7f1874004c40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c210 tx=0x7f18740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7220bc49c006f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:43.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c005ad0 con 0x7f18740047c0 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c005c30 con 0x7f18740047c0 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f188d400640 1 -- 172.21.15.73:0/2415736547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c02c890 con 0x7f18740047c0 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 -- 172.21.15.73:0/2415736547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18740047c0 msgr2=0x7f1874004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 --2- 172.21.15.73:0/2415736547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18740047c0 0x7f1874004c40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c210 tx=0x7f18740057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f188f200640 1 -- 172.21.15.73:0/2415736547 reap_dead start 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 -- 172.21.15.73:0/2415736547 shutdown_connections 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 -- 172.21.15.73:0/2415736547 >> 172.21.15.73:0/2415736547 conn(0x7f1890069cb0 msgr2=0x7f189006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 -- 172.21.15.73:0/2415736547 shutdown_connections 2024-04-05T22:27:43.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.909+0000 7f1895400640 1 -- 172.21.15.73:0/2415736547 wait complete. 2024-04-05T22:27:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.911+0000 7f1895400640 1 Processor -- start 2024-04-05T22:27:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.911+0000 7f1895400640 1 -- start start 2024-04-05T22:27:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18740047c0 0x7f1890163020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189014b770 0x7f189014bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900c38c0 con 0x7f18900cb350 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900cb8f0 con 0x7f18740047c0 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f1895400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900cbbd0 con 0x7f189014b770 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f188de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f188de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52292/0 (socket says 172.21.15.73:52292) 2024-04-05T22:27:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.912+0000 7f188de00640 1 -- 172.21.15.73:0/860325106 learned_addr learned my addr 172.21.15.73:0/860325106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188e800640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18740047c0 0x7f1890163020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188f200640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189014b770 0x7f189014bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 -- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189014b770 msgr2=0x7f189014bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189014b770 0x7f189014bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 -- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18740047c0 msgr2=0x7f1890163020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18740047c0 0x7f1890163020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 -- 172.21.15.73:0/860325106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900cbeb0 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.913+0000 7f188de00640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f187c02c430 tx=0x7f189006fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.914+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c039070 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.914+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c02cca0 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.914+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c03c040 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.914+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18900cc190 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.914+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18900cc950 con 0x7f18900cb350 2024-04-05T22:27:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.915+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18900c3a00 con 0x7f18900cb350 2024-04-05T22:27:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.916+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f187c032070 con 0x7f18900cb350 2024-04-05T22:27:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.917+0000 7f1886a00640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.917+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f187c0aa7a0 con 0x7f18900cb350 2024-04-05T22:27:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.918+0000 7f188e800640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.919+0000 7f188e800640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1888007360 tx=0x7f188800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:43.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:43.921+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f187c0757d0 con 0x7f18900cb350 2024-04-05T22:27:44.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.215+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7f1890071780 con 0x7f187006ad20 2024-04-05T22:27:44.224 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:44.225+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:44.225 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:44.225+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:44.225 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:44.225+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:44.225 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:44.225+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:44.225 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:44.225+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.313+0000 7f1886a00640 1 -- 172.21.15.73:0/860325106 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f188800f040 con 0x7f187006ad20 2024-04-05T22:27:44.313 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 msgr2=0x7f187006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1888007360 tx=0x7f188800b040 comp rx=0 tx=0).stop 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 msgr2=0x7f1890163560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f187c02c430 tx=0x7f189006fae0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 shutdown_connections 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f187006ad20 0x7f187006d1e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189014b770 0x7f189014bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18900cb350 0x7f1890163560 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 --2- 172.21.15.73:0/860325106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18740047c0 0x7f1890163020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 >> 172.21.15.73:0/860325106 conn(0x7f1890069cb0 msgr2=0x7f189006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 shutdown_connections 2024-04-05T22:27:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.318+0000 7f1895400640 1 -- 172.21.15.73:0/860325106 wait complete. 2024-04-05T22:27:44.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-05T22:27:44.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.551+0000 7f8566c00640 1 Processor -- start 2024-04-05T22:27:44.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.551+0000 7f8566c00640 1 -- start start 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f85601465e0 0x7f8560071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 0x7f85600adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85601469c0 con 0x7f85601465e0 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856006d000 con 0x7f8560070570 2024-04-05T22:27:44.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8566c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600709d0 con 0x7f856006c630 2024-04-05T22:27:44.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.553+0000 7f8565800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f85601465e0 0x7f8560071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f85601465e0 0x7f8560071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52298/0 (socket says 172.21.15.73:52298) 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 -- 172.21.15.73:0/238248736 learned_addr learned my addr 172.21.15.73:0/238248736 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8564e00640 1 --2- 172.21.15.73:0/238248736 >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 0x7f85600adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8566200640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 -- 172.21.15.73:0/238248736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 msgr2=0x7f856006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856006cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 -- 172.21.15.73:0/238248736 >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 msgr2=0x7f85600adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 --2- 172.21.15.73:0/238248736 >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 0x7f85600adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8565800640 1 -- 172.21.15.73:0/238248736 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600ad780 con 0x7f85601465e0 2024-04-05T22:27:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.554+0000 7f8564e00640 1 --2- 172.21.15.73:0/238248736 >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 0x7f85600adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:44.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.556+0000 7f8565800640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f8560071560 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f85600666c0 tx=0x7f855402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60bbf2fa23d5a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:44.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.556+0000 7f855fe00640 1 -- 172.21.15.73:0/238248736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554031640 con 0x7f85601465e0 2024-04-05T22:27:44.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.556+0000 7f855fe00640 1 -- 172.21.15.73:0/238248736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85540317a0 con 0x7f85601465e0 2024-04-05T22:27:44.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.556+0000 7f855fe00640 1 -- 172.21.15.73:0/238248736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554031b30 con 0x7f85601465e0 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 -- 172.21.15.73:0/238248736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 msgr2=0x7f8560071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f8560071560 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f85600666c0 tx=0x7f855402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 -- 172.21.15.73:0/238248736 shutdown_connections 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 --2- 172.21.15.73:0/238248736 >> v2:172.21.15.157:3300/0 conn(0x7f8560070570 0x7f85600adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 --2- 172.21.15.73:0/238248736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f8560071560 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 -- 172.21.15.73:0/238248736 >> 172.21.15.73:0/238248736 conn(0x7f85600b0830 msgr2=0x7f85600b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 -- 172.21.15.73:0/238248736 shutdown_connections 2024-04-05T22:27:44.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.557+0000 7f8566c00640 1 -- 172.21.15.73:0/238248736 wait complete. 2024-04-05T22:27:44.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.559+0000 7f8566c00640 1 Processor -- start 2024-04-05T22:27:44.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.559+0000 7f8566c00640 1 -- start start 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.559+0000 7f8566c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8560070570 0x7f856013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856013b2c0 con 0x7f85601465e0 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85601f2570 con 0x7f8560070570 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85601f3590 con 0x7f856006c630 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52300/0 (socket says 172.21.15.73:52300) 2024-04-05T22:27:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 -- 172.21.15.73:0/369869896 learned_addr learned my addr 172.21.15.73:0/369869896 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:44.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 -- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 msgr2=0x7f856015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 -- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8560070570 msgr2=0x7f856013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8560070570 0x7f856013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.560+0000 7f8566200640 1 -- 172.21.15.73:0/369869896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540094d0 con 0x7f85601465e0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f8566200640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8558007e90 tx=0x7f855800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558014420 con 0x7f85601465e0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85601f27d0 con 0x7f85601465e0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85601f2d90 con 0x7f85601465e0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8558014580 con 0x7f85601465e0 2024-04-05T22:27:44.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.561+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580099c0 con 0x7f85601465e0 2024-04-05T22:27:44.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.562+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856004e160 con 0x7f85601465e0 2024-04-05T22:27:44.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.563+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f85580148f0 con 0x7f85601465e0 2024-04-05T22:27:44.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.564+0000 7f855d600640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 0x7f854006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:44.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.564+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f855808adc0 con 0x7f85601465e0 2024-04-05T22:27:44.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.565+0000 7f8565800640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 0x7f854006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:44.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.566+0000 7f8565800640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 0x7f854006d1e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8554036000 tx=0x7f8554030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.567+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8558055df0 con 0x7f85601465e0 2024-04-05T22:27:44.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.791+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f85600ace50 con 0x7f854006ad20 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.801+0000 7f855d600640 1 -- 172.21.15.73:0/369869896 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f8554031990 con 0x7f854006ad20 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subvol44000" 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:44.800 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:27:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 msgr2=0x7f854006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 0x7f854006d1e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8554036000 tx=0x7f8554030000 comp rx=0 tx=0).stop 2024-04-05T22:27:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 msgr2=0x7f856013ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8558007e90 tx=0x7f855800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 shutdown_connections 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f854006ad20 0x7f854006d1e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85601465e0 0x7f856013ac40 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8560070570 0x7f856013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 --2- 172.21.15.73:0/369869896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f856006c630 0x7f856015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 >> 172.21.15.73:0/369869896 conn(0x7f85600b0830 msgr2=0x7f85600cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 shutdown_connections 2024-04-05T22:27:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:44.804+0000 7f8566c00640 1 -- 172.21.15.73:0/369869896 wait complete. 2024-04-05T22:27:44.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol44000 2024-04-05T22:27:45.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.037+0000 7f811be00640 1 Processor -- start 2024-04-05T22:27:45.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.037+0000 7f811be00640 1 -- start start 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f811c0cb0b0 0x7f811c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c160ea0 con 0x7f811c157410 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c06d800 con 0x7f811c157030 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c0cbaa0 con 0x7f811c0cb0b0 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811a000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f811c157410 0x7f811c160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49682/0 (socket says 172.21.15.73:49682) 2024-04-05T22:27:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811a000640 1 -- 172.21.15.73:0/2215115024 learned_addr learned my addr 172.21.15.73:0/2215115024 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163759737 0 0) 0x7f811c160ea0 con 0x7f811c157410 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f811b400640 1 --2- 172.21.15.73:0/2215115024 >> v2:172.21.15.157:3300/0 conn(0x7f811c0cb0b0 0x7f811c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.039+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8100003520 con 0x7f811c157410 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3646029157 0 0) 0x7f8100003520 con 0x7f811c157410 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 >> v2:172.21.15.157:3300/0 conn(0x7f811c0cb0b0 msgr2=0x7f811c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 --2- 172.21.15.73:0/2215115024 >> v2:172.21.15.157:3300/0 conn(0x7f811c0cb0b0 0x7f811c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 >> v1:172.21.15.157:6789/0 conn(0x7f811c157030 legacy=0x7f811c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c0b3090 con 0x7f811c157410 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8108001670 con 0x7f811c157410 2024-04-05T22:27:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 >> v1:172.21.15.73:6789/0 conn(0x7f811c157410 legacy=0x7f811c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8100003970 0x7f8100003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 0x7f8100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8100005180 0x7f810000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0cbaa0 con 0x7f81000047c0 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810000c120 con 0x7f8100003970 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810000c2e0 con 0x7f8100005180 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.040+0000 7f811a000640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 0x7f8100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811aa00640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8100003970 0x7f8100003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811a000640 1 -- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8100005180 msgr2=0x7f810000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811a000640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8100005180 0x7f810000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811a000640 1 -- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8100003970 msgr2=0x7f8100003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811a000640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8100003970 0x7f8100003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811a000640 1 -- 172.21.15.73:0/2215115024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100003520 con 0x7f81000047c0 2024-04-05T22:27:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.041+0000 7f811b400640 1 -- 172.21.15.73:0/2215115024 reap_dead start 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.042+0000 7f811a000640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 0x7f8100004c40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f810800c210 tx=0x7f81000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c964bf3ac4fd957a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.042+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108005ad0 con 0x7f81000047c0 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.042+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108005c50 con 0x7f81000047c0 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f8119600640 1 -- 172.21.15.73:0/2215115024 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810802c850 con 0x7f81000047c0 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 -- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 msgr2=0x7f8100004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 0x7f8100004c40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f810800c210 tx=0x7f81000057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 -- 172.21.15.73:0/2215115024 shutdown_connections 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 --2- 172.21.15.73:0/2215115024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81000047c0 0x7f8100004c40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 -- 172.21.15.73:0/2215115024 >> 172.21.15.73:0/2215115024 conn(0x7f811c069cb0 msgr2=0x7f811c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 -- 172.21.15.73:0/2215115024 shutdown_connections 2024-04-05T22:27:45.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.043+0000 7f811be00640 1 -- 172.21.15.73:0/2215115024 wait complete. 2024-04-05T22:27:45.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.045+0000 7f811be00640 1 Processor -- start 2024-04-05T22:27:45.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.045+0000 7f811be00640 1 -- start start 2024-04-05T22:27:45.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 0x7f811c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f811c153520 0x7f811c15e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1f6a40 con 0x7f811c15ec60 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c15f0e0 con 0x7f81000047c0 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1f5ec0 con 0x7f811c153520 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52328/0 (socket says 172.21.15.73:52328) 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.046+0000 7f811b400640 1 -- 172.21.15.73:0/208134587 learned_addr learned my addr 172.21.15.73:0/208134587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811aa00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 0x7f811c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 -- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f811c153520 msgr2=0x7f811c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811a000640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f811c153520 0x7f811c15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f811c153520 0x7f811c15e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 -- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 msgr2=0x7f811c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 0x7f811c158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 -- 172.21.15.73:0/208134587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c1f61a0 con 0x7f811c15ec60 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811aa00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 0x7f811c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811b400640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f811c0cf710 tx=0x7f811400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114010040 con 0x7f811c15ec60 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400bc80 con 0x7f811c15ec60 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c1f6480 con 0x7f811c15ec60 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.047+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114014400 con 0x7f811c15ec60 2024-04-05T22:27:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.048+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c158cb0 con 0x7f811c15ec60 2024-04-05T22:27:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.049+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c04e160 con 0x7f811c15ec60 2024-04-05T22:27:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.050+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 89247+0+0 (secure 0 0 0) 0x7f8114014560 con 0x7f811c15ec60 2024-04-05T22:27:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.051+0000 7f8112a00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 0x7f80fc06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.051+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8114089b50 con 0x7f811c15ec60 2024-04-05T22:27:45.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.051+0000 7f811aa00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 0x7f80fc06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.053+0000 7f811aa00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 0x7f80fc06d210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.055+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8114054b10 con 0x7f811c15ec60 2024-04-05T22:27:45.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.277+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7f811c15d2b0 con 0x7f80fc06ad50 2024-04-05T22:27:45.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.340+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f8114088070 con 0x7f811c15ec60 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.348+0000 7f8112a00640 1 -- 172.21.15.73:0/208134587 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+686 (secure 0 0 0) 0x7f810c00d050 con 0x7f80fc06ad50 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "atime": "2024-04-05 22:27:43", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_pcent": "undefined", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_quota": "infinite", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_used": 0, 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "created_at": "2024-04-05 22:27:43", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "ctime": "2024-04-05 22:27:44", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "data_pool": "cephfs_data", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "features": [ 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-clone", 2024-04-05T22:27:45.349 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-autoprotect" 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "flavor": 1, 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 0, 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": 16877, 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_addrs": [ 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.73:6789", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6789", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6790" 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "mtime": "2024-04-05 22:27:43", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_namespace": "", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "complete", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "subvolume", 2024-04-05T22:27:45.350 INFO:teuthology.orchestra.run.smithi073.stdout: "uid": 0 2024-04-05T22:27:45.351 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:45.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 msgr2=0x7f80fc06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 0x7f80fc06d210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c009040 comp rx=0 tx=0).stop 2024-04-05T22:27:45.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 msgr2=0x7f811c160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f811c0cf710 tx=0x7f811400b480 comp rx=0 tx=0).stop 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 shutdown_connections 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f80fc06ad50 0x7f80fc06d210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f811c15ec60 0x7f811c160590 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f811c153520 0x7f811c15e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 --2- 172.21.15.73:0/208134587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81000047c0 0x7f811c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 >> 172.21.15.73:0/208134587 conn(0x7f811c069cb0 msgr2=0x7f811c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 shutdown_connections 2024-04-05T22:27:45.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.352+0000 7f811be00640 1 -- 172.21.15.73:0/208134587 wait complete. 2024-04-05T22:27:45.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol44000 10485760 2024-04-05T22:27:45.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.585+0000 7f9f5f200640 1 Processor -- start 2024-04-05T22:27:45.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.585+0000 7f9f5f200640 1 -- start start 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f5f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f5815ae50 0x7f9f580b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f5f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f580cce90 0x7f9f580b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f5f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5815b230 con 0x7f9f580cecd0 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f5f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580cd320 con 0x7f9f580cce90 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f5f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580ce2f0 con 0x7f9f5815ae50 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f580cce90 0x7f9f580b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f580cce90 0x7f9f580b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52330/0 (socket says 172.21.15.73:52330) 2024-04-05T22:27:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 -- 172.21.15.73:0/1228097466 learned_addr learned my addr 172.21.15.73:0/1228097466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 -- 172.21.15.73:0/1228097466 >> v2:172.21.15.157:3300/0 conn(0x7f9f5815ae50 msgr2=0x7f9f580b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 --2- 172.21.15.73:0/1228097466 >> v2:172.21.15.157:3300/0 conn(0x7f9f5815ae50 0x7f9f580b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 -- 172.21.15.73:0/1228097466 >> v1:172.21.15.157:6789/0 conn(0x7f9f580cecd0 legacy=0x7f9f580b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:45.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.587+0000 7f9f57400640 1 -- 172.21.15.73:0/1228097466 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5815f4a0 con 0x7f9f580cce90 2024-04-05T22:27:45.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f57400640 1 --2- 172.21.15.73:0/1228097466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f580b2e00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806dbb0 tx=0x7f9f4402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e401a350bf361d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f56a00640 1 -- 172.21.15.73:0/1228097466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440317e0 con 0x7f9f580cce90 2024-04-05T22:27:45.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f56a00640 1 -- 172.21.15.73:0/1228097466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44031940 con 0x7f9f580cce90 2024-04-05T22:27:45.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f56a00640 1 -- 172.21.15.73:0/1228097466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44031cd0 con 0x7f9f580cce90 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f5f200640 1 -- 172.21.15.73:0/1228097466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 msgr2=0x7f9f580b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.589+0000 7f9f5f200640 1 --2- 172.21.15.73:0/1228097466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f580b2e00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806dbb0 tx=0x7f9f4402b880 comp rx=0 tx=0).stop 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 -- 172.21.15.73:0/1228097466 shutdown_connections 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 --2- 172.21.15.73:0/1228097466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f580b2e00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 --2- 172.21.15.73:0/1228097466 >> v2:172.21.15.157:3300/0 conn(0x7f9f5815ae50 0x7f9f580b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 -- 172.21.15.73:0/1228097466 >> 172.21.15.73:0/1228097466 conn(0x7f9f5806a0d0 msgr2=0x7f9f5806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 -- 172.21.15.73:0/1228097466 shutdown_connections 2024-04-05T22:27:45.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.590+0000 7f9f5f200640 1 -- 172.21.15.73:0/1228097466 wait complete. 2024-04-05T22:27:45.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.591+0000 7f9f5f200640 1 Processor -- start 2024-04-05T22:27:45.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.592+0000 7f9f5f200640 1 -- start start 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.592+0000 7f9f5f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.592+0000 7f9f5f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f580cecd0 0x7f9f580ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f5f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f5815ae50 0x7f9f580ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f5f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580afa30 con 0x7f9f580cce90 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f5f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580af5a0 con 0x7f9f5815ae50 2024-04-05T22:27:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f5f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580714c0 con 0x7f9f580cecd0 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52344/0 (socket says 172.21.15.73:52344) 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 -- 172.21.15.73:0/2088692461 learned_addr learned my addr 172.21.15.73:0/2088692461 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 -- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f580cecd0 msgr2=0x7f9f580ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f580cecd0 0x7f9f580ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 -- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f5815ae50 msgr2=0x7f9f580ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f5815ae50 0x7f9f580ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.593+0000 7f9f57e00640 1 -- 172.21.15.73:0/2088692461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440094d0 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f57e00640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b410 tx=0x7f9f4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48011830 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f480119b0 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48011d60 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f580aef50 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.594+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f580710e0 con 0x7f9f580cce90 2024-04-05T22:27:45.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.595+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f580aedb0 con 0x7f9f580cce90 2024-04-05T22:27:45.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.596+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f9f4801a400 con 0x7f9f580cce90 2024-04-05T22:27:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.597+0000 7f9f4fe00640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 0x7f9f3806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.597+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f48089740 con 0x7f9f580cce90 2024-04-05T22:27:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.598+0000 7f9f57400640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 0x7f9f3806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.599+0000 7f9f57400640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 0x7f9f3806d270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9f44036000 tx=0x7f9f44030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.602+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9f580aedb0 con 0x7f9f580cce90 2024-04-05T22:27:45.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.832+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol44000", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f580cb610 con 0x7f9f3806adb0 2024-04-05T22:27:45.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.900+0000 7f9f4fe00640 1 -- 172.21.15.73:0/2088692461 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7f9f44031c70 con 0x7f9f3806adb0 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_used": 0 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:45.901 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_quota": 10485760 2024-04-05T22:27:45.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:27:45.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:27:45.902 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_pcent": "0.00" 2024-04-05T22:27:45.902 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:45.902 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:27:45.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 msgr2=0x7f9f3806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 0x7f9f3806d270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9f44036000 tx=0x7f9f44030000 comp rx=0 tx=0).stop 2024-04-05T22:27:45.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 msgr2=0x7f9f58155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:45.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b410 tx=0x7f9f4800b8d0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 shutdown_connections 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f3806adb0 0x7f9f3806d270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f5815ae50 0x7f9f580ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f580cecd0 0x7f9f580ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 --2- 172.21.15.73:0/2088692461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f580cce90 0x7f9f58155600 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.903+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 >> 172.21.15.73:0/2088692461 conn(0x7f9f5806a0d0 msgr2=0x7f9f5806ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:45.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.904+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 shutdown_connections 2024-04-05T22:27:45.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:45.904+0000 7f9f5f200640 1 -- 172.21.15.73:0/2088692461 wait complete. 2024-04-05T22:27:45.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol44000 2024-04-05T22:27:46.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.140+0000 7fc710000640 1 Processor -- start 2024-04-05T22:27:46.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.140+0000 7fc710000640 1 -- start start 2024-04-05T22:27:46.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc710000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc70806b3d0 0x7fc70815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc710000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7080cec00 0x7fc708070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc710000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70815e780 con 0x7fc708159cf0 2024-04-05T22:27:46.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc710000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7080c9d70 con 0x7fc70806b3d0 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc710000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7080cb5c0 con 0x7fc7080cec00 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc70806b3d0 0x7fc70815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc70806b3d0 0x7fc70815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51164/0 (socket says 172.21.15.73:51164) 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70cc00640 1 -- 172.21.15.73:0/4207520208 learned_addr learned my addr 172.21.15.73:0/4207520208 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70e000640 1 --2- 172.21.15.73:0/4207520208 >> v2:172.21.15.157:3300/0 conn(0x7fc7080cec00 0x7fc708070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70cc00640 1 -- 172.21.15.73:0/4207520208 >> v2:172.21.15.157:3300/0 conn(0x7fc7080cec00 msgr2=0x7fc708070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.142+0000 7fc70cc00640 1 --2- 172.21.15.73:0/4207520208 >> v2:172.21.15.157:3300/0 conn(0x7fc7080cec00 0x7fc708070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.143+0000 7fc70cc00640 1 -- 172.21.15.73:0/4207520208 >> v1:172.21.15.73:6789/0 conn(0x7fc708159cf0 legacy=0x7fc70806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.143+0000 7fc70cc00640 1 -- 172.21.15.73:0/4207520208 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7080cb8a0 con 0x7fc70806b3d0 2024-04-05T22:27:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.143+0000 7fc707e00640 1 -- 172.21.15.73:0/4207520208 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664986961 0 0) 0x7fc6fc001110 con 0x7fc708159cf0 2024-04-05T22:27:46.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc70cc00640 1 --2- 172.21.15.73:0/4207520208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc70815e240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc7080676c0 tx=0x7fc6f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928ddbea748cf328 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc707e00640 1 -- 172.21.15.73:0/4207520208 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4031870 con 0x7fc70806b3d0 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc707e00640 1 -- 172.21.15.73:0/4207520208 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f40319d0 con 0x7fc70806b3d0 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc707e00640 1 -- 172.21.15.73:0/4207520208 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4031d60 con 0x7fc70806b3d0 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc710000640 1 -- 172.21.15.73:0/4207520208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 msgr2=0x7fc70815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.144+0000 7fc710000640 1 --2- 172.21.15.73:0/4207520208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc70815e240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc7080676c0 tx=0x7fc6f402b910 comp rx=0 tx=0).stop 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 -- 172.21.15.73:0/4207520208 shutdown_connections 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 --2- 172.21.15.73:0/4207520208 >> v2:172.21.15.157:3300/0 conn(0x7fc7080cec00 0x7fc708070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 --2- 172.21.15.73:0/4207520208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc70815e240 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 -- 172.21.15.73:0/4207520208 >> 172.21.15.73:0/4207520208 conn(0x7fc7080b0bb0 msgr2=0x7fc7080b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 -- 172.21.15.73:0/4207520208 shutdown_connections 2024-04-05T22:27:46.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.145+0000 7fc710000640 1 -- 172.21.15.73:0/4207520208 wait complete. 2024-04-05T22:27:46.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.146+0000 7fc710000640 1 Processor -- start 2024-04-05T22:27:46.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 -- start start 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7080cec00 0x7fc708153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc708159cf0 0x7fc7080ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708154270 con 0x7fc70806b3d0 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708154550 con 0x7fc7080cec00 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.147+0000 7fc710000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc708154830 con 0x7fc708159cf0 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51168/0 (socket says 172.21.15.73:51168) 2024-04-05T22:27:46.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 -- 172.21.15.73:0/827923526 learned_addr learned my addr 172.21.15.73:0/827923526 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:46.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 -- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc708159cf0 msgr2=0x7fc7080ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc708159cf0 0x7fc7080ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 -- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7080cec00 msgr2=0x7fc708153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7080cec00 0x7fc708153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 -- 172.21.15.73:0/827923526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40094d0 con 0x7fc70806b3d0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.148+0000 7fc70d600640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00c750 tx=0x7fc6fc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.149+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc005270 con 0x7fc70806b3d0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.149+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0053f0 con 0x7fc70806b3d0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.149+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00a880 con 0x7fc70806b3d0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.149+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708154b10 con 0x7fc70806b3d0 2024-04-05T22:27:46.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.149+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708153b80 con 0x7fc70806b3d0 2024-04-05T22:27:46.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.150+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708155030 con 0x7fc70806b3d0 2024-04-05T22:27:46.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.151+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fc6fc00a9e0 con 0x7fc70806b3d0 2024-04-05T22:27:46.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.152+0000 7fc705600640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 0x7fc6e806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.152+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6fc088a70 con 0x7fc70806b3d0 2024-04-05T22:27:46.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.152+0000 7fc70cc00640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 0x7fc6e806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.153+0000 7fc70cc00640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 0x7fc6e806d270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4036000 tx=0x7fc6f4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.156+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc708155030 con 0x7fc70806b3d0 2024-04-05T22:27:46.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.379+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7fc70815a0d0 con 0x7fc6e806adb0 2024-04-05T22:27:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.444+0000 7fc705600640 1 -- 172.21.15.73:0/827923526 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+679 (secure 0 0 0) 0x7fc6f4031b40 con 0x7fc6e806adb0 2024-04-05T22:27:46.444 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "atime": "2024-04-05 22:27:43", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_pcent": "0.00", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_quota": 10485760, 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_used": 0, 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "created_at": "2024-04-05 22:27:43", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "ctime": "2024-04-05 22:27:45", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "data_pool": "cephfs_data", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "features": [ 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-clone", 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-autoprotect" 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flavor": 1, 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 0, 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": 16877, 2024-04-05T22:27:46.445 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_addrs": [ 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.73:6789", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6789", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6790" 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "mtime": "2024-04-05 22:27:43", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_namespace": "", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "complete", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "subvolume", 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout: "uid": 0 2024-04-05T22:27:46.446 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:46.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 msgr2=0x7fc6e806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 0x7fc6e806d270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4036000 tx=0x7fc6f4030000 comp rx=0 tx=0).stop 2024-04-05T22:27:46.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 msgr2=0x7fc7080c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00c750 tx=0x7fc6fc00cc10 comp rx=0 tx=0).stop 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 shutdown_connections 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6e806adb0 0x7fc6e806d270 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc708159cf0 0x7fc7080ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7080cec00 0x7fc708153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 --2- 172.21.15.73:0/827923526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc70806b3d0 0x7fc7080c8980 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 >> 172.21.15.73:0/827923526 conn(0x7fc7080b0bb0 msgr2=0x7fc7080ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 shutdown_connections 2024-04-05T22:27:46.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.447+0000 7fc710000640 1 -- 172.21.15.73:0/827923526 wait complete. 2024-04-05T22:27:46.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol44000 --mode 777 --uid 1000 --gid 1000 2024-04-05T22:27:46.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.679+0000 7fdb96200640 1 Processor -- start 2024-04-05T22:27:46.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.679+0000 7fdb96200640 1 -- start start 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb96200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb9015abe0 0x7fdb900712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb96200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb96200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9015afc0 con 0x7fdb9006c9a0 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb96200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9006f5b0 con 0x7fdb9015abe0 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb96200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9006f890 con 0x7fdb9006bc40 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb9015abe0 0x7fdb900712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb9015abe0 0x7fdb900712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51180/0 (socket says 172.21.15.73:51180) 2024-04-05T22:27:46.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 -- 172.21.15.73:0/1187884761 learned_addr learned my addr 172.21.15.73:0/1187884761 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:46.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 -- 172.21.15.73:0/1187884761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 msgr2=0x7fdb90164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:46.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 --2- 172.21.15.73:0/1187884761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 -- 172.21.15.73:0/1187884761 >> v1:172.21.15.157:6789/0 conn(0x7fdb9006c9a0 legacy=0x7fdb9006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:46.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.681+0000 7fdb8f400640 1 -- 172.21.15.73:0/1187884761 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9006fb70 con 0x7fdb9015abe0 2024-04-05T22:27:46.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.683+0000 7fdb8f400640 1 --2- 172.21.15.73:0/1187884761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb900712e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009870 tx=0x7fdb8002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a6955ebe99e71f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.683+0000 7fdb8e000640 1 -- 172.21.15.73:0/1187884761 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80031740 con 0x7fdb9015abe0 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.683+0000 7fdb8e000640 1 -- 172.21.15.73:0/1187884761 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb800318a0 con 0x7fdb9015abe0 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.683+0000 7fdb8e000640 1 -- 172.21.15.73:0/1187884761 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80031c30 con 0x7fdb9015abe0 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 -- 172.21.15.73:0/1187884761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 msgr2=0x7fdb900712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 --2- 172.21.15.73:0/1187884761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb900712e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009870 tx=0x7fdb8002b7e0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 -- 172.21.15.73:0/1187884761 shutdown_connections 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 --2- 172.21.15.73:0/1187884761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 --2- 172.21.15.73:0/1187884761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb900712e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 -- 172.21.15.73:0/1187884761 >> 172.21.15.73:0/1187884761 conn(0x7fdb90069cb0 msgr2=0x7fdb9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:46.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 -- 172.21.15.73:0/1187884761 shutdown_connections 2024-04-05T22:27:46.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.684+0000 7fdb96200640 1 -- 172.21.15.73:0/1187884761 wait complete. 2024-04-05T22:27:46.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.686+0000 7fdb96200640 1 Processor -- start 2024-04-05T22:27:46.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.686+0000 7fdb96200640 1 -- start start 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9006c9a0 0x7fdb90144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb90159fe0 con 0x7fdb9015abe0 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb90144b70 con 0x7fdb9006c9a0 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.687+0000 7fdb96200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb90159070 con 0x7fdb9006bc40 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51192/0 (socket says 172.21.15.73:51192) 2024-04-05T22:27:46.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 -- 172.21.15.73:0/1740255724 learned_addr learned my addr 172.21.15.73:0/1740255724 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 -- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 msgr2=0x7fdb90158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 -- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9006c9a0 msgr2=0x7fdb90144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8ea00640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9006c9a0 0x7fdb90144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9006c9a0 0x7fdb90144630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.688+0000 7fdb8fe00640 1 -- 172.21.15.73:0/1740255724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800094d0 con 0x7fdb9015abe0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb8fe00640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800b9a0 tx=0x7fdb8800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800cba0 con 0x7fdb9015abe0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8800cd20 con 0x7fdb9015abe0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb90143890 con 0x7fdb9015abe0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88014680 con 0x7fdb9015abe0 2024-04-05T22:27:46.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.689+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb90143e50 con 0x7fdb9015abe0 2024-04-05T22:27:46.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.690+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb90068160 con 0x7fdb9015abe0 2024-04-05T22:27:46.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.691+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdb8801d3b0 con 0x7fdb9015abe0 2024-04-05T22:27:46.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.692+0000 7fdb87400640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 0x7fdb7006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:46.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.692+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb88089610 con 0x7fdb9015abe0 2024-04-05T22:27:46.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.693+0000 7fdb8f400640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 0x7fdb7006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:46.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.694+0000 7fdb8f400640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 0x7fdb7006d270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb80035000 tx=0x7fdb80030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:46.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.698+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdb88054560 con 0x7fdb9015abe0 2024-04-05T22:27:46.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.920+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol44000", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7fdb90071820 con 0x7fdb7006adb0 2024-04-05T22:27:46.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.991+0000 7fdb87400640 1 -- 172.21.15.73:0/1740255724 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdb80031a90 con 0x7fdb7006adb0 2024-04-05T22:27:46.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 msgr2=0x7fdb7006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 0x7fdb7006d270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb80035000 tx=0x7fdb80030000 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 msgr2=0x7fdb9015d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800b9a0 tx=0x7fdb8800be60 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 shutdown_connections 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdb7006adb0 0x7fdb7006d270 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9015abe0 0x7fdb9015d7f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9006c9a0 0x7fdb90144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 --2- 172.21.15.73:0/1740255724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006bc40 0x7fdb90158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 >> 172.21.15.73:0/1740255724 conn(0x7fdb90069cb0 msgr2=0x7fdb900cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.994+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 shutdown_connections 2024-04-05T22:27:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:46.995+0000 7fdb96200640 1 -- 172.21.15.73:0/1740255724 wait complete. 2024-04-05T22:27:47.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol44000 2024-04-05T22:27:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.233+0000 7f2baa400640 1 Processor -- start 2024-04-05T22:27:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.233+0000 7f2baa400640 1 -- start start 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2baa400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2ba415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2baa400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ba4070640 0x7f2ba406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2baa400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba406bc50 con 0x7f2ba415e580 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2baa400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba415f530 con 0x7f2ba4070640 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2baa400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415ef70 con 0x7f2ba415a850 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.234+0000 7f2b9be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ba415e580 0x7f2ba406ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53026/0 (socket says 172.21.15.73:53026) 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2b9be00640 1 -- 172.21.15.73:0/3433135260 learned_addr learned my addr 172.21.15.73:0/3433135260 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307968277 0 0) 0x7f2ba406bc50 con 0x7f2ba415e580 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b88003520 con 0x7f2ba415e580 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba9000640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2ba415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> v2:172.21.15.157:3300/0 conn(0x7f2ba4070640 0x7f2ba406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544622989 0 0) 0x7f2b88003520 con 0x7f2ba415e580 2024-04-05T22:27:47.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 msgr2=0x7f2ba415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2ba415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 >> v2:172.21.15.157:3300/0 conn(0x7f2ba4070640 msgr2=0x7f2ba406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 --2- 172.21.15.73:0/3433135260 >> v2:172.21.15.157:3300/0 conn(0x7f2ba4070640 0x7f2ba406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba415f250 con 0x7f2ba415e580 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2b8c001670 con 0x7f2ba415e580 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> v2:172.21.15.157:3300/0 conn(0x7f2ba4070640 0x7f2ba406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 >> v1:172.21.15.73:6789/0 conn(0x7f2ba415e580 legacy=0x7f2ba406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b88003970 0x7f2b88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2b880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2b88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415f530 con 0x7f2b88004d00 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415ef70 con 0x7f2b88003970 2024-04-05T22:27:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.235+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8800bb10 con 0x7f2ba415a850 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2b88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9000640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b88003970 0x7f2b88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2b9be00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2b880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 -- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 msgr2=0x7f2b880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2b880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 -- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b88003970 msgr2=0x7f2b88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b88003970 0x7f2b88003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2ba9a00640 1 -- 172.21.15.73:0/3433135260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b88003520 con 0x7f2b88004d00 2024-04-05T22:27:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.236+0000 7f2b9be00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba415a850 0x7f2b880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.237+0000 7f2ba9a00640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2b88005180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c490 tx=0x7f2b880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246d2292603d5cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c033660 con 0x7f2b88004d00 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c0337c0 con 0x7f2b88004d00 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2ba3e00640 1 -- 172.21.15.73:0/3433135260 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c033b70 con 0x7f2b88004d00 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 -- 172.21.15.73:0/3433135260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 msgr2=0x7f2b88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 --2- 172.21.15.73:0/3433135260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2b88005180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c490 tx=0x7f2b880057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2ba9a00640 1 -- 172.21.15.73:0/3433135260 reap_dead start 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 -- 172.21.15.73:0/3433135260 shutdown_connections 2024-04-05T22:27:47.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 -- 172.21.15.73:0/3433135260 >> 172.21.15.73:0/3433135260 conn(0x7f2ba4069cb0 msgr2=0x7f2ba406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 -- 172.21.15.73:0/3433135260 shutdown_connections 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.238+0000 7f2baa400640 1 -- 172.21.15.73:0/3433135260 wait complete. 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.240+0000 7f2baa400640 1 Processor -- start 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.240+0000 7f2baa400640 1 -- start start 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba414c8c0 0x7f2ba414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ba4151dd0 0x7f2ba4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba41527a0 con 0x7f2b88004d00 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba414f2c0 con 0x7f2ba4151dd0 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2baa400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba414cda0 con 0x7f2ba414c8c0 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2ba9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2ba9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51210/0 (socket says 172.21.15.73:51210) 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.241+0000 7f2ba9000640 1 -- 172.21.15.73:0/2012942374 learned_addr learned my addr 172.21.15.73:0/2012942374 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2b9be00640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba414c8c0 0x7f2ba414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 -- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba414c8c0 msgr2=0x7f2ba414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba414c8c0 0x7f2ba414bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 -- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ba4151dd0 msgr2=0x7f2ba4152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ba4151dd0 0x7f2ba4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 -- 172.21.15.73:0/2012942374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba414e7e0 con 0x7f2b88004d00 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba9000640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400f5b0 tx=0x7f2b9400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94013810 con 0x7f2b88004d00 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94013970 con 0x7f2b88004d00 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94013ce0 con 0x7f2b88004d00 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.242+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba414eac0 con 0x7f2b88004d00 2024-04-05T22:27:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.243+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba414b770 con 0x7f2b88004d00 2024-04-05T22:27:47.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.244+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba404e160 con 0x7f2b88004d00 2024-04-05T22:27:47.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.245+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f2b9401f460 con 0x7f2b88004d00 2024-04-05T22:27:47.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.245+0000 7f2ba1600640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 0x7f2b8006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:47.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.245+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b94089940 con 0x7f2b88004d00 2024-04-05T22:27:47.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.246+0000 7f2b9be00640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 0x7f2b8006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:47.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.247+0000 7f2b9be00640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 0x7f2b8006d270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0017a0 tx=0x7f2b8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:47.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.250+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2b94054930 con 0x7f2b88004d00 2024-04-05T22:27:47.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.472+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ba406c1d0 con 0x7f2b8006adb0 2024-04-05T22:27:47.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.527+0000 7f2ba1600640 1 -- 172.21.15.73:0/2012942374 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f2b8c00d040 con 0x7f2b8006adb0 2024-04-05T22:27:47.527 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609 2024-04-05T22:27:47.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 msgr2=0x7f2b8006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 0x7f2b8006d270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0017a0 tx=0x7f2b8c009040 comp rx=0 tx=0).stop 2024-04-05T22:27:47.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 msgr2=0x7f2ba414b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400f5b0 tx=0x7f2b9400fa70 comp rx=0 tx=0).stop 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 shutdown_connections 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b8006adb0 0x7f2b8006d270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ba4151dd0 0x7f2ba4152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ba414c8c0 0x7f2ba414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 --2- 172.21.15.73:0/2012942374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b88004d00 0x7f2ba414b230 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 >> 172.21.15.73:0/2012942374 conn(0x7f2ba4069cb0 msgr2=0x7f2ba415c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 shutdown_connections 2024-04-05T22:27:47.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:47.530+0000 7f2baa400640 1 -- 172.21.15.73:0/2012942374 wait complete. 2024-04-05T22:27:47.549 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol44000 with 8 files each 1MB size under directory volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609 2024-04-05T22:27:47.549 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:47.550 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:47.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:47.597 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:47.716 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:47.716 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:47.716 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232295 s, 45.1 MB/s 2024-04-05T22:27:47.718 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:47.718 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:47.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:47.762 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307914 s, 34.1 MB/s 2024-04-05T22:27:47.888 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:47.888 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:47.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:47.941 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03676 s, 28.5 MB/s 2024-04-05T22:27:48.075 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:48.075 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:48.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:48.129 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321275 s, 32.6 MB/s 2024-04-05T22:27:48.211 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:48.211 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:48.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:48.260 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251825 s, 41.6 MB/s 2024-04-05T22:27:48.385 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:48.385 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:48.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:48.438 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343372 s, 30.5 MB/s 2024-04-05T22:27:48.571 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:48.571 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:48.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:48.624 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.708 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.708 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.708 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327715 s, 32.0 MB/s 2024-04-05T22:27:48.710 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:27:48.710 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:27:48.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:48.766 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-05T22:27:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-05T22:27:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-05T22:27:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291551 s, 36.0 MB/s 2024-04-05T22:27:48.851 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol44000 subvol_snap6510 2024-04-05T22:27:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.065+0000 7f62f1200640 1 Processor -- start 2024-04-05T22:27:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.066+0000 7f62f1200640 1 -- start start 2024-04-05T22:27:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62f1200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62ec06b230 0x7f62ec167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62f1200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec15e3d0 con 0x7f62ec15d230 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62f1200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec15d7c0 con 0x7f62ec15ce50 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62f1200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec15daa0 con 0x7f62ec06b230 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62ea800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62ec06b230 0x7f62ec167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62ea800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62ec06b230 0x7f62ec167e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51224/0 (socket says 172.21.15.73:51224) 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62ea800640 1 -- 172.21.15.73:0/1243277269 learned_addr learned my addr 172.21.15.73:0/1243277269 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.067+0000 7f62e9400640 1 --1- 172.21.15.73:0/1243277269 >> v1:172.21.15.73:6789/0 conn(0x7f62ec15d230 0x7f62ec0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53028/0 (socket says 172.21.15.73:53028) 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.068+0000 7f62ea800640 1 -- 172.21.15.73:0/1243277269 >> v1:172.21.15.157:6789/0 conn(0x7f62ec15ce50 legacy=0x7f62ec1614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.068+0000 7f62ea800640 1 -- 172.21.15.73:0/1243277269 >> v1:172.21.15.73:6789/0 conn(0x7f62ec15d230 legacy=0x7f62ec0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.068+0000 7f62ea800640 1 -- 172.21.15.73:0/1243277269 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec15dd80 con 0x7f62ec06b230 2024-04-05T22:27:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.069+0000 7f62ea800640 1 --2- 172.21.15.73:0/1243277269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec06b230 0x7f62ec167e30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009e50 tx=0x7f62e402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaa829d5e243163 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.069+0000 7f62e8a00640 1 -- 172.21.15.73:0/1243277269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e402cb10 con 0x7f62ec06b230 2024-04-05T22:27:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.069+0000 7f62e8a00640 1 -- 172.21.15.73:0/1243277269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e402cc70 con 0x7f62ec06b230 2024-04-05T22:27:49.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.069+0000 7f62e8a00640 1 -- 172.21.15.73:0/1243277269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40343f0 con 0x7f62ec06b230 2024-04-05T22:27:49.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 -- 172.21.15.73:0/1243277269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec06b230 msgr2=0x7f62ec167e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 --2- 172.21.15.73:0/1243277269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec06b230 0x7f62ec167e30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009e50 tx=0x7f62e402bf70 comp rx=0 tx=0).stop 2024-04-05T22:27:49.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 -- 172.21.15.73:0/1243277269 shutdown_connections 2024-04-05T22:27:49.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 --2- 172.21.15.73:0/1243277269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec06b230 0x7f62ec167e30 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 -- 172.21.15.73:0/1243277269 >> 172.21.15.73:0/1243277269 conn(0x7f62ec0b3ed0 msgr2=0x7f62ec0b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:49.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 -- 172.21.15.73:0/1243277269 shutdown_connections 2024-04-05T22:27:49.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.070+0000 7f62f1200640 1 -- 172.21.15.73:0/1243277269 wait complete. 2024-04-05T22:27:49.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.072+0000 7f62f1200640 1 Processor -- start 2024-04-05T22:27:49.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.072+0000 7f62f1200640 1 -- start start 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 0x7f62ec0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15ce50 0x7f62ec1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec154a40 con 0x7f62ec15d230 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec153460 con 0x7f62ec06b230 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62f1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec153740 con 0x7f62ec15ce50 2024-04-05T22:27:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62ea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62ea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51236/0 (socket says 172.21.15.73:51236) 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.073+0000 7f62ea800640 1 -- 172.21.15.73:0/1038813128 learned_addr learned my addr 172.21.15.73:0/1038813128 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62e9e00640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 0x7f62ec0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62e9400640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15ce50 0x7f62ec1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 -- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15ce50 msgr2=0x7f62ec1f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15ce50 0x7f62ec1f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 -- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 msgr2=0x7f62ec0cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 0x7f62ec0cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 -- 172.21.15.73:0/1038813128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4009b30 con 0x7f62ec15d230 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62e9e00640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 0x7f62ec0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:49.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.074+0000 7f62ea800640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f62e4036040 tx=0x7f62e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.075+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e403e450 con 0x7f62ec15d230 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.075+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e403e5d0 con 0x7f62ec15d230 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.075+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec153a20 con 0x7f62ec15d230 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.075+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e403e980 con 0x7f62ec15d230 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.075+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec153fe0 con 0x7f62ec15d230 2024-04-05T22:27:49.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.076+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec1541d0 con 0x7f62ec15d230 2024-04-05T22:27:49.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.077+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f62e403eae0 con 0x7f62ec15d230 2024-04-05T22:27:49.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.078+0000 7f62e2000640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 0x7f62c806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.078+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e40ac4c0 con 0x7f62ec15d230 2024-04-05T22:27:49.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.079+0000 7f62e9e00640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 0x7f62c806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.080+0000 7f62e9e00640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 0x7f62c806d270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f62dc003eb0 tx=0x7f62dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.082+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f62e4077560 con 0x7f62ec15d230 2024-04-05T22:27:49.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.309+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "subvol_snap6510", "target": ["mon-mgr", ""]}) v1 -- 0x7f62ec1538b0 con 0x7f62c806adb0 2024-04-05T22:27:49.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.355+0000 7f62e2000640 1 -- 172.21.15.73:0/1038813128 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f62ec1538b0 con 0x7f62c806adb0 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.357+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 msgr2=0x7f62c806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.357+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 0x7f62c806d270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f62dc003eb0 tx=0x7f62dc009040 comp rx=0 tx=0).stop 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.357+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 msgr2=0x7f62ec154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.357+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f62e4036040 tx=0x7f62e4004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 shutdown_connections 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62c806adb0 0x7f62c806d270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62ec15d230 0x7f62ec154500 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15ce50 0x7f62ec1f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 --2- 172.21.15.73:0/1038813128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec06b230 0x7f62ec0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 >> 172.21.15.73:0/1038813128 conn(0x7f62ec0b3ed0 msgr2=0x7f62ec15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 shutdown_connections 2024-04-05T22:27:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.358+0000 7f62f1200640 1 -- 172.21.15.73:0/1038813128 wait complete. 2024-04-05T22:27:49.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol44000 subvol_snap6510 subvol_clone90950 2024-04-05T22:27:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.582+0000 7f7aaa200640 1 Processor -- start 2024-04-05T22:27:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.582+0000 7f7aaa200640 1 -- start start 2024-04-05T22:27:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aaa200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7aa406f5b0 0x7f7aa406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aaa200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40b2b90 con 0x7f7aa40ad720 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aaa200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa415a570 con 0x7f7aa40cb440 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aaa200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa415a850 con 0x7f7aa406f5b0 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aa9800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7aa40ad720 0x7f7aa40b3040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53034/0 (socket says 172.21.15.73:53034) 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aa9800640 1 -- 172.21.15.73:0/2553085136 learned_addr learned my addr 172.21.15.73:0/2553085136 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.584+0000 7f7aa3e00640 1 --2- 172.21.15.73:0/2553085136 >> v2:172.21.15.73:3300/0 conn(0x7f7aa406f5b0 0x7f7aa406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.585+0000 7f7aa3400640 1 -- 172.21.15.73:0/2553085136 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229594097 0 0) 0x7f7aa40b2b90 con 0x7f7aa40ad720 2024-04-05T22:27:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.585+0000 7f7aa3400640 1 -- 172.21.15.73:0/2553085136 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8c003520 con 0x7f7aa40ad720 2024-04-05T22:27:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.585+0000 7f7aa3e00640 1 -- 172.21.15.73:0/2553085136 >> v1:172.21.15.157:6789/0 conn(0x7f7aa40cb440 legacy=0x7f7aa4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.585+0000 7f7aa3e00640 1 -- 172.21.15.73:0/2553085136 >> v1:172.21.15.73:6789/0 conn(0x7f7aa40ad720 legacy=0x7f7aa40b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.585+0000 7f7aa3e00640 1 -- 172.21.15.73:0/2553085136 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa415ab30 con 0x7f7aa406f5b0 2024-04-05T22:27:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.586+0000 7f7aa3e00640 1 --2- 172.21.15.73:0/2553085136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa406f5b0 0x7f7aa406c910 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097a0 tx=0x7f7a9002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa43d476ed570b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.586+0000 7f7aa3400640 1 -- 172.21.15.73:0/2553085136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90030610 con 0x7f7aa406f5b0 2024-04-05T22:27:49.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aa3400640 1 -- 172.21.15.73:0/2553085136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90030770 con 0x7f7aa406f5b0 2024-04-05T22:27:49.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aa3400640 1 -- 172.21.15.73:0/2553085136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90030b00 con 0x7f7aa406f5b0 2024-04-05T22:27:49.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 -- 172.21.15.73:0/2553085136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa406f5b0 msgr2=0x7f7aa406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 --2- 172.21.15.73:0/2553085136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa406f5b0 0x7f7aa406c910 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097a0 tx=0x7f7a9002b960 comp rx=0 tx=0).stop 2024-04-05T22:27:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 -- 172.21.15.73:0/2553085136 shutdown_connections 2024-04-05T22:27:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 --2- 172.21.15.73:0/2553085136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa406f5b0 0x7f7aa406c910 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 -- 172.21.15.73:0/2553085136 >> 172.21.15.73:0/2553085136 conn(0x7f7aa4069cb0 msgr2=0x7f7aa406a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 -- 172.21.15.73:0/2553085136 shutdown_connections 2024-04-05T22:27:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.587+0000 7f7aaa200640 1 -- 172.21.15.73:0/2553085136 wait complete. 2024-04-05T22:27:49.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.589+0000 7f7aaa200640 1 Processor -- start 2024-04-05T22:27:49.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.589+0000 7f7aaa200640 1 -- start start 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 0x7f7aa4141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40cb440 0x7f7aa40a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41419a0 con 0x7f7aa40ad720 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4142b70 con 0x7f7aa40cb440 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aaa200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4142e50 con 0x7f7aa406f5b0 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aa3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aa3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51252/0 (socket says 172.21.15.73:51252) 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.590+0000 7f7aa3e00640 1 -- 172.21.15.73:0/1623946335 learned_addr learned my addr 172.21.15.73:0/1623946335 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa8e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 0x7f7aa4141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa9800640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40cb440 0x7f7aa40a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 -- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 msgr2=0x7f7aa4141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 0x7f7aa4141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 -- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40cb440 msgr2=0x7f7aa40a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40cb440 0x7f7aa40a8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 -- 172.21.15.73:0/1623946335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009400 con 0x7f7aa40ad720 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa8e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 0x7f7aa4141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:49.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa3e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7a9002bf70 tx=0x7f7a90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.591+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9003e040 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.592+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90039cb0 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.592+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90044630 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.592+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4143130 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.592+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa41436f0 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.593+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa404e160 con 0x7f7aa40ad720 2024-04-05T22:27:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.594+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7f7a90043390 con 0x7f7aa40ad720 2024-04-05T22:27:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.594+0000 7f7aa0c00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 0x7f7a8406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.595+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a900ab940 con 0x7f7aa40ad720 2024-04-05T22:27:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.595+0000 7f7aa8e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 0x7f7a8406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.596+0000 7f7aa8e00640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 0x7f7a8406d270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000f60 tx=0x7f7a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.599+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7a90076790 con 0x7f7aa40ad720 2024-04-05T22:27:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.830+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "subvol_snap6510", "target_sub_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aa4159b70 con 0x7f7a8406adb0 2024-04-05T22:27:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.994+0000 7f7aa0c00640 1 -- 172.21.15.73:0/1623946335 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7aa4159b70 con 0x7f7a8406adb0 2024-04-05T22:27:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 msgr2=0x7f7a8406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 0x7f7a8406d270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000f60 tx=0x7f7a98009040 comp rx=0 tx=0).stop 2024-04-05T22:27:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 msgr2=0x7f7aa40a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7a9002bf70 tx=0x7f7a90004730 comp rx=0 tx=0).stop 2024-04-05T22:27:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 shutdown_connections 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a8406adb0 0x7f7a8406d270 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40cb440 0x7f7aa40a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa40ad720 0x7f7aa40a84c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 --2- 172.21.15.73:0/1623946335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa406f5b0 0x7f7aa4141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 >> 172.21.15.73:0/1623946335 conn(0x7f7aa4069cb0 msgr2=0x7f7aa40adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 shutdown_connections 2024-04-05T22:27:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:49.998+0000 7f7aaa200640 1 -- 172.21.15.73:0/1623946335 wait complete. 2024-04-05T22:27:50.007 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.008+0000 7f1d8c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.008+0000 7f1d8c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.008+0000 7f1d8c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.008+0000 7f1d8c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.008+0000 7f1d8c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone90950 2024-04-05T22:27:50.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.035+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.035+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.035+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.035+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:27:50.035+0000 7f1d8cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:27:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.232+0000 7fdc39800640 1 Processor -- start 2024-04-05T22:27:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.232+0000 7fdc39800640 1 -- start start 2024-04-05T22:27:50.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.233+0000 7fdc39800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3415abe0 0x7fdc340712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.233+0000 7fdc39800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 0x7fdc3406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.233+0000 7fdc39800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc3406ba70 0x7fdc340cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc39800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3406bec0 con 0x7fdc3406ba70 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc39800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3415afc0 con 0x7fdc3406c9a0 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc39800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3406f5b0 con 0x7fdc3415abe0 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc3406ba70 0x7fdc340cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc3406ba70 0x7fdc340cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51260/0 (socket says 172.21.15.73:51260) 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 -- 172.21.15.73:0/837631058 learned_addr learned my addr 172.21.15.73:0/837631058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc32a00640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3415abe0 0x7fdc340712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc32000640 1 --2- 172.21.15.73:0/837631058 >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 0x7fdc3406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 -- 172.21.15.73:0/837631058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3415abe0 msgr2=0x7fdc340712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3415abe0 0x7fdc340712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 -- 172.21.15.73:0/837631058 >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 msgr2=0x7fdc3406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:50.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 --2- 172.21.15.73:0/837631058 >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 0x7fdc3406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc33400640 1 -- 172.21.15.73:0/837631058 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3406f890 con 0x7fdc3406ba70 2024-04-05T22:27:50.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.234+0000 7fdc32000640 1 --2- 172.21.15.73:0/837631058 >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 0x7fdc3406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.236+0000 7fdc33400640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3406ba70 0x7fdc340cccb0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b3cb0 tx=0x7fdc2402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be57e63001baae server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.236+0000 7fdc31600640 1 -- 172.21.15.73:0/837631058 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24031750 con 0x7fdc3406ba70 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.236+0000 7fdc31600640 1 -- 172.21.15.73:0/837631058 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc240318b0 con 0x7fdc3406ba70 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.236+0000 7fdc31600640 1 -- 172.21.15.73:0/837631058 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24031c40 con 0x7fdc3406ba70 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 -- 172.21.15.73:0/837631058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3406ba70 msgr2=0x7fdc340cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3406ba70 0x7fdc340cccb0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b3cb0 tx=0x7fdc2402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 -- 172.21.15.73:0/837631058 shutdown_connections 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3406ba70 0x7fdc340cccb0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 --2- 172.21.15.73:0/837631058 >> v2:172.21.15.157:3300/0 conn(0x7fdc3406c9a0 0x7fdc3406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 --2- 172.21.15.73:0/837631058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3415abe0 0x7fdc340712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 -- 172.21.15.73:0/837631058 >> 172.21.15.73:0/837631058 conn(0x7fdc34069cb0 msgr2=0x7fdc3406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 -- 172.21.15.73:0/837631058 shutdown_connections 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.237+0000 7fdc39800640 1 -- 172.21.15.73:0/837631058 wait complete. 2024-04-05T22:27:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.239+0000 7fdc39800640 1 Processor -- start 2024-04-05T22:27:50.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.239+0000 7fdc39800640 1 -- start start 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3406ba70 0x7fdc3415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 0x7fdc3414bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415b2c0 con 0x7fdc3415abe0 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415b5a0 con 0x7fdc3406c9a0 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc39800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415b880 con 0x7fdc3406ba70 2024-04-05T22:27:50.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc33400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc33400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51272/0 (socket says 172.21.15.73:51272) 2024-04-05T22:27:50.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc33400640 1 -- 172.21.15.73:0/1514086023 learned_addr learned my addr 172.21.15.73:0/1514086023 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc32a00640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3406ba70 0x7fdc3415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.240+0000 7fdc32000640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 0x7fdc3414bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 -- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3406ba70 msgr2=0x7fdc3415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3406ba70 0x7fdc3415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 -- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 msgr2=0x7fdc3414bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:50.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 0x7fdc3414bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 -- 172.21.15.73:0/1514086023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240094d0 con 0x7fdc3415abe0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc32000640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 0x7fdc3414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc33400640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdc2402b830 tx=0x7fdc240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24033c80 con 0x7fdc3415abe0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc24033e00 con 0x7fdc3415abe0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3415bb60 con 0x7fdc3415abe0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2403c640 con 0x7fdc3415abe0 2024-04-05T22:27:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.241+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3414c530 con 0x7fdc3415abe0 2024-04-05T22:27:50.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.243+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3415c080 con 0x7fdc3415abe0 2024-04-05T22:27:50.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.244+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 89298+0+0 (secure 0 0 0) 0x7fdc2403a040 con 0x7fdc3415abe0 2024-04-05T22:27:50.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.245+0000 7fdc2aa00640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 0x7fdc1006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:50.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.245+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc240ab4b0 con 0x7fdc3415abe0 2024-04-05T22:27:50.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.245+0000 7fdc32a00640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 0x7fdc1006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:50.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.246+0000 7fdc32a00640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 0x7fdc1006d270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c007130 tx=0x7fdc2c007060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:50.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.249+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdc3415c080 con 0x7fdc3415abe0 2024-04-05T22:27:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.491+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc34070130 con 0x7fdc1006adb0 2024-04-05T22:27:50.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.524+0000 7fdc2aa00640 1 -- 172.21.15.73:0/1514086023 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+168 (secure 0 0 0) 0x7fdc2c007aa0 con 0x7fdc1006adb0 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "status": { 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "in-progress", 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "source": { 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "volume": "cephfs", 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "subvolume": "subvol44000", 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot": "subvol_snap6510" 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:50.526 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 msgr2=0x7fdc1006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 0x7fdc1006d270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c007130 tx=0x7fdc2c007060 comp rx=0 tx=0).stop 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 msgr2=0x7fdc341f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdc2402b830 tx=0x7fdc240046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 shutdown_connections 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdc1006adb0 0x7fdc1006d270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3415abe0 0x7fdc341f33f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc3406c9a0 0x7fdc3414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 --2- 172.21.15.73:0/1514086023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc3406ba70 0x7fdc3415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.528+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 >> 172.21.15.73:0/1514086023 conn(0x7fdc34069cb0 msgr2=0x7fdc34070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.529+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 shutdown_connections 2024-04-05T22:27:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:50.529+0000 7fdc39800640 1 -- 172.21.15.73:0/1514086023 wait complete. 2024-04-05T22:27:51.549 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone90950 2024-04-05T22:27:51.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.767+0000 7fd370800640 1 Processor -- start 2024-04-05T22:27:51.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.767+0000 7fd370800640 1 -- start start 2024-04-05T22:27:51.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.768+0000 7fd370800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd36815abe0 0x7fd3680712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:51.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd370800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36815afc0 con 0x7fd36806c9a0 2024-04-05T22:27:51.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd370800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36806f5b0 con 0x7fd36806bc40 2024-04-05T22:27:51.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd370800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36806f890 con 0x7fd36815abe0 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd36de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd36815abe0 0x7fd3680712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd36de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd36815abe0 0x7fd3680712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51276/0 (socket says 172.21.15.73:51276) 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.769+0000 7fd36de00640 1 -- 172.21.15.73:0/2486333542 learned_addr learned my addr 172.21.15.73:0/2486333542 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.770+0000 7fd36de00640 1 -- 172.21.15.73:0/2486333542 >> v1:172.21.15.157:6789/0 conn(0x7fd36806bc40 legacy=0x7fd368164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.770+0000 7fd36de00640 1 -- 172.21.15.73:0/2486333542 >> v1:172.21.15.73:6789/0 conn(0x7fd36806c9a0 legacy=0x7fd36806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.770+0000 7fd36de00640 1 -- 172.21.15.73:0/2486333542 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36806fb70 con 0x7fd36815abe0 2024-04-05T22:27:51.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.770+0000 7fd36ca00640 1 -- 172.21.15.73:0/2486333542 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636655422 0 0) 0x7fd36815afc0 con 0x7fd36806c9a0 2024-04-05T22:27:51.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.771+0000 7fd36de00640 1 --2- 172.21.15.73:0/2486333542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd3680712e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89d54573faf5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:51.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.771+0000 7fd36ca00640 1 -- 172.21.15.73:0/2486333542 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0317e0 con 0x7fd36815abe0 2024-04-05T22:27:51.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.771+0000 7fd36ca00640 1 -- 172.21.15.73:0/2486333542 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c031940 con 0x7fd36815abe0 2024-04-05T22:27:51.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.771+0000 7fd36ca00640 1 -- 172.21.15.73:0/2486333542 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c031cd0 con 0x7fd36815abe0 2024-04-05T22:27:51.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 -- 172.21.15.73:0/2486333542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 msgr2=0x7fd3680712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 --2- 172.21.15.73:0/2486333542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd3680712e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02b880 comp rx=0 tx=0).stop 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 -- 172.21.15.73:0/2486333542 shutdown_connections 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 --2- 172.21.15.73:0/2486333542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd3680712e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 -- 172.21.15.73:0/2486333542 >> 172.21.15.73:0/2486333542 conn(0x7fd368069cb0 msgr2=0x7fd36806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 -- 172.21.15.73:0/2486333542 shutdown_connections 2024-04-05T22:27:51.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.772+0000 7fd370800640 1 -- 172.21.15.73:0/2486333542 wait complete. 2024-04-05T22:27:51.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.775+0000 7fd370800640 1 Processor -- start 2024-04-05T22:27:51.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.775+0000 7fd370800640 1 -- start start 2024-04-05T22:27:51.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 0x7fd3681454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:51.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd36806c9a0 0x7fd368147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:51.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:51.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368145a10 con 0x7fd36815abe0 2024-04-05T22:27:51.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681481b0 con 0x7fd36806c9a0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd370800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd368148490 con 0x7fd36806bc40 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd36e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd36e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51280/0 (socket says 172.21.15.73:51280) 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.776+0000 7fd36e800640 1 -- 172.21.15.73:0/3240023836 learned_addr learned my addr 172.21.15.73:0/3240023836 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36de00640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 0x7fd3681454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36d400640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd36806c9a0 0x7fd368147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 -- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 msgr2=0x7fd3681454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 0x7fd3681454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 -- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd36806c9a0 msgr2=0x7fd368147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd36806c9a0 0x7fd368147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 -- 172.21.15.73:0/3240023836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0094d0 con 0x7fd36815abe0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36de00640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 0x7fd3681454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd36e800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c780 tx=0x7fd36400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:51.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364004b60 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36400ce90 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36400a850 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd368148770 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.777+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd368149140 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.779+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3680cdc70 con 0x7fd36815abe0 2024-04-05T22:27:51.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.780+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fd36400a9b0 con 0x7fd36815abe0 2024-04-05T22:27:51.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.781+0000 7fd362000640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 0x7fd34c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:51.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.781+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd364089950 con 0x7fd36815abe0 2024-04-05T22:27:51.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.782+0000 7fd36de00640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 0x7fd34c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:51.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.783+0000 7fd36de00640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 0x7fd34c06d3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd35c036000 tx=0x7fd35c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:51.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:51.786+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd3640547d0 con 0x7fd36815abe0 2024-04-05T22:27:52.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.016+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7fd368071820 con 0x7fd34c06af20 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.202+0000 7fd362000640 1 -- 172.21.15.73:0/3240023836 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fd35c031c30 con 0x7fd34c06af20 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stdout: "status": { 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "complete" 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:27:52.203 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 msgr2=0x7fd34c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 0x7fd34c06d3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd35c036000 tx=0x7fd35c03d040 comp rx=0 tx=0).stop 2024-04-05T22:27:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 msgr2=0x7fd36815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c780 tx=0x7fd36400cc40 comp rx=0 tx=0).stop 2024-04-05T22:27:52.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 shutdown_connections 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd34c06af20 0x7fd34c06d3e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd36815abe0 0x7fd36815d120 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd36806c9a0 0x7fd368147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 --2- 172.21.15.73:0/3240023836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd36806bc40 0x7fd3681454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 >> 172.21.15.73:0/3240023836 conn(0x7fd368069cb0 msgr2=0x7fd3680cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 shutdown_connections 2024-04-05T22:27:52.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.207+0000 7fd370800640 1 -- 172.21.15.73:0/3240023836 wait complete. 2024-04-05T22:27:52.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:52.230 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone90950/.meta) 2024-04-05T22:27:52.316 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:27:52.316 INFO:teuthology.orchestra.run.smithi073.stdout:version = 2 2024-04-05T22:27:52.316 INFO:teuthology.orchestra.run.smithi073.stdout:type = clone 2024-04-05T22:27:52.317 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe 2024-04-05T22:27:52.317 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:27:52.317 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:27:52.320 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol44000 2024-04-05T22:27:52.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.525+0000 7fae9da00640 1 Processor -- start 2024-04-05T22:27:52.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.525+0000 7fae9da00640 1 -- start start 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae9da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 0x7fae980add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae9da00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae98070e70 con 0x7fae9806cd50 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae9da00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae980b27f0 con 0x7fae981599f0 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae9da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae980ae240 con 0x7fae98070a40 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae8e000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fae9806cd50 0x7fae98159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53046/0 (socket says 172.21.15.73:53046) 2024-04-05T22:27:52.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.527+0000 7fae8e000640 1 -- 172.21.15.73:0/2959578991 learned_addr learned my addr 172.21.15.73:0/2959578991 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae97400640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 0x7fae980add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266398074 0 0) 0x7fae98070e70 con 0x7fae9806cd50 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c003520 con 0x7fae9806cd50 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2068360638 0 0) 0x7fae7c003520 con 0x7fae9806cd50 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 msgr2=0x7fae980add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 0x7fae980add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 >> v1:172.21.15.157:6789/0 conn(0x7fae981599f0 legacy=0x7fae9806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae980b2ad0 con 0x7fae9806cd50 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fae80001690 con 0x7fae9806cd50 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 >> v1:172.21.15.73:6789/0 conn(0x7fae9806cd50 legacy=0x7fae98159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 0x7fae7c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7c003f20 0x7fae7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7c004d00 0x7fae7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980ae240 con 0x7fae7c003f20 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00ba60 con 0x7fae7c004d00 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00bc20 con 0x7fae98070a40 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.528+0000 7fae8e000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7c003f20 0x7fae7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae97400640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7c004d00 0x7fae7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae8e000640 1 -- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 msgr2=0x7fae7c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae8e000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae98070a40 0x7fae7c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae8e000640 1 -- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7c004d00 msgr2=0x7fae7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae8e000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae7c004d00 0x7fae7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.529+0000 7fae8e000640 1 -- 172.21.15.73:0/2959578991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c003520 con 0x7fae7c003f20 2024-04-05T22:27:52.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.530+0000 7fae8e000640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7c003f20 0x7fae7c0047c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fae9806d780 tx=0x7fae7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5735a5eba9a27e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.530+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80005ad0 con 0x7fae7c003f20 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.530+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80005c30 con 0x7fae7c003f20 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.530+0000 7fae96000640 1 -- 172.21.15.73:0/2959578991 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8002c890 con 0x7fae7c003f20 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 -- 172.21.15.73:0/2959578991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7c003f20 msgr2=0x7fae7c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 --2- 172.21.15.73:0/2959578991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae7c003f20 0x7fae7c0047c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fae9806d780 tx=0x7fae7c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae97400640 1 -- 172.21.15.73:0/2959578991 reap_dead start 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 -- 172.21.15.73:0/2959578991 shutdown_connections 2024-04-05T22:27:52.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 -- 172.21.15.73:0/2959578991 >> 172.21.15.73:0/2959578991 conn(0x7fae98069cb0 msgr2=0x7fae9806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:52.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 -- 172.21.15.73:0/2959578991 shutdown_connections 2024-04-05T22:27:52.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.531+0000 7fae9da00640 1 -- 172.21.15.73:0/2959578991 wait complete. 2024-04-05T22:27:52.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.533+0000 7fae9da00640 1 Processor -- start 2024-04-05T22:27:52.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.533+0000 7fae9da00640 1 -- start start 2024-04-05T22:27:52.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7c003f20 0x7fae9814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae98163140 0x7fae9814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981635c0 con 0x7fae98152420 2024-04-05T22:27:52.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9814d160 con 0x7fae98163140 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae9da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae98163a50 con 0x7fae7c003f20 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51294/0 (socket says 172.21.15.73:51294) 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 -- 172.21.15.73:0/3317772868 learned_addr learned my addr 172.21.15.73:0/3317772868 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae96a00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7c003f20 0x7fae9814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 -- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7c003f20 msgr2=0x7fae9814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7c003f20 0x7fae9814cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae97400640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae98163140 0x7fae9814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 -- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae98163140 msgr2=0x7fae9814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae98163140 0x7fae9814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.534+0000 7fae8e000640 1 -- 172.21.15.73:0/3317772868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae981530a0 con 0x7fae98152420 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae8e000640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f60 tx=0x7fae80001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80039070 con 0x7fae98152420 2024-04-05T22:27:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8002cca0 con 0x7fae98152420 2024-04-05T22:27:52.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8003c040 con 0x7fae98152420 2024-04-05T22:27:52.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae98153380 con 0x7fae98152420 2024-04-05T22:27:52.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.535+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae98163cf0 con 0x7fae98152420 2024-04-05T22:27:52.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.536+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae98163e80 con 0x7fae98152420 2024-04-05T22:27:52.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.537+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fae80032070 con 0x7fae98152420 2024-04-05T22:27:52.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.538+0000 7fae8f400640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 0x7fae7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:52.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.538+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae800aab10 con 0x7fae98152420 2024-04-05T22:27:52.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.539+0000 7fae96a00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 0x7fae7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.540+0000 7fae96a00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 0x7fae7406d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae98070270 tx=0x7fae88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:52.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.542+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae800759f0 con 0x7fae98152420 2024-04-05T22:27:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.770+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7fae981538a0 con 0x7fae7406af00 2024-04-05T22:27:52.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.808+0000 7fae8f400640 1 -- 172.21.15.73:0/3317772868 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7fae88001c40 con 0x7fae7406af00 2024-04-05T22:27:52.809 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609 2024-04-05T22:27:52.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 msgr2=0x7fae7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 0x7fae7406d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae98070270 tx=0x7fae88009040 comp rx=0 tx=0).stop 2024-04-05T22:27:52.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 msgr2=0x7fae98162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:52.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f60 tx=0x7fae80001a20 comp rx=0 tx=0).stop 2024-04-05T22:27:52.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 shutdown_connections 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae7406af00 0x7fae7406d3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae98163140 0x7fae9814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae98152420 0x7fae98162c00 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 --2- 172.21.15.73:0/3317772868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae7c003f20 0x7fae9814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 >> 172.21.15.73:0/3317772868 conn(0x7fae98069cb0 msgr2=0x7fae98151260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 shutdown_connections 2024-04-05T22:27:52.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:52.811+0000 7fae9da00640 1 -- 172.21.15.73:0/3317772868 wait complete. 2024-04-05T22:27:52.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone90950 2024-04-05T22:27:53.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.044+0000 7f9fcac00640 1 Processor -- start 2024-04-05T22:27:53.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.044+0000 7f9fcac00640 1 -- start start 2024-04-05T22:27:53.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fcac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9fc415abe0 0x7f9fc40712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fcac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fcac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc406bec0 con 0x7f9fc406ba70 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fcac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc415afc0 con 0x7f9fc415abe0 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fcac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406f5b0 con 0x7f9fc406c9a0 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fc3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9fc415abe0 0x7f9fc40712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fc3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9fc415abe0 0x7f9fc40712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51304/0 (socket says 172.21.15.73:51304) 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fc3e00640 1 -- 172.21.15.73:0/70631817 learned_addr learned my addr 172.21.15.73:0/70631817 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:53.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.046+0000 7f9fbbe00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.047+0000 7f9fc3e00640 1 -- 172.21.15.73:0/70631817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 msgr2=0x7f9fc406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.047+0000 7f9fc3e00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.047+0000 7f9fc3e00640 1 -- 172.21.15.73:0/70631817 >> v1:172.21.15.73:6789/0 conn(0x7f9fc406ba70 legacy=0x7f9fc40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.047+0000 7f9fc3e00640 1 -- 172.21.15.73:0/70631817 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc406f890 con 0x7f9fc415abe0 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.048+0000 7f9fc3e00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc415abe0 0x7f9fc40712e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009870 tx=0x7f9fb402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33591fab7326892 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.048+0000 7f9fc3400640 1 -- 172.21.15.73:0/70631817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40317c0 con 0x7f9fc415abe0 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.048+0000 7f9fc3400640 1 -- 172.21.15.73:0/70631817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4031920 con 0x7f9fc415abe0 2024-04-05T22:27:53.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.048+0000 7f9fc3400640 1 -- 172.21.15.73:0/70631817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4031cb0 con 0x7f9fc415abe0 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 -- 172.21.15.73:0/70631817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc415abe0 msgr2=0x7f9fc40712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc415abe0 0x7f9fc40712e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009870 tx=0x7f9fb402b860 comp rx=0 tx=0).stop 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 -- 172.21.15.73:0/70631817 shutdown_connections 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 --2- 172.21.15.73:0/70631817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc415abe0 0x7f9fc40712e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 -- 172.21.15.73:0/70631817 >> 172.21.15.73:0/70631817 conn(0x7f9fc4069cb0 msgr2=0x7f9fc406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:53.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 -- 172.21.15.73:0/70631817 shutdown_connections 2024-04-05T22:27:53.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.049+0000 7f9fcac00640 1 -- 172.21.15.73:0/70631817 wait complete. 2024-04-05T22:27:53.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.051+0000 7f9fcac00640 1 Processor -- start 2024-04-05T22:27:53.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.051+0000 7f9fcac00640 1 -- start start 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc415abe0 0x7f9fc4157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4158020 con 0x7f9fc406ba70 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc41582c0 con 0x7f9fc415abe0 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fcac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc41585a0 con 0x7f9fc406c9a0 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fc3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fc3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51320/0 (socket says 172.21.15.73:51320) 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fc3e00640 1 -- 172.21.15.73:0/88170807 learned_addr learned my addr 172.21.15.73:0/88170807 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.052+0000 7f9fbbe00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc8c00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc415abe0 0x7f9fc4157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 -- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 msgr2=0x7f9fc414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 -- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc415abe0 msgr2=0x7f9fc4157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc415abe0 0x7f9fc4157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 -- 172.21.15.73:0/88170807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40094d0 con 0x7f9fc406ba70 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc3e00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402b8a0 tx=0x7f9fb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4032860 con 0x7f9fc406ba70 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb40329e0 con 0x7f9fc406ba70 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4032d90 con 0x7f9fc406ba70 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc4158880 con 0x7f9fc406ba70 2024-04-05T22:27:53.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.053+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc4158e40 con 0x7f9fc406ba70 2024-04-05T22:27:53.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.054+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4159030 con 0x7f9fc406ba70 2024-04-05T22:27:53.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.055+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f9fb404e450 con 0x7f9fc406ba70 2024-04-05T22:27:53.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.056+0000 7f9fc0c00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 0x7f9fa006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.056+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fb40ab980 con 0x7f9fc406ba70 2024-04-05T22:27:53.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.057+0000 7f9fbbe00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 0x7f9fa006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.058+0000 7f9fbbe00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 0x7f9fa006d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fc406d5a0 tx=0x7f9fac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.060+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9fb4076810 con 0x7f9fc406ba70 2024-04-05T22:27:53.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.286+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fc4070130 con 0x7f9fa006aed0 2024-04-05T22:27:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.300+0000 7f9fc0c00640 1 -- 172.21.15.73:0/88170807 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+73 (secure 0 0 0) 0x7f9fac00b200 con 0x7f9fa006aed0 2024-04-05T22:27:53.300 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe 2024-04-05T22:27:53.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 msgr2=0x7f9fa006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 0x7f9fa006d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fc406d5a0 tx=0x7f9fac008040 comp rx=0 tx=0).stop 2024-04-05T22:27:53.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 msgr2=0x7f9fc41496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402b8a0 tx=0x7f9fb40046c0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 shutdown_connections 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9fa006aed0 0x7f9fa006d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc415abe0 0x7f9fc4157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc406c9a0 0x7f9fc414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 --2- 172.21.15.73:0/88170807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fc406ba70 0x7f9fc41496c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 >> 172.21.15.73:0/88170807 conn(0x7f9fc4069cb0 msgr2=0x7f9fc40b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 shutdown_connections 2024-04-05T22:27:53.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.303+0000 7f9fcac00640 1 -- 172.21.15.73:0/88170807 wait complete. 2024-04-05T22:27:53.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:53.322 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:9 2024-04-05T22:27:53.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:53.371 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe) 2024-04-05T22:27:53.470 INFO:teuthology.orchestra.run.smithi073.stdout:9 2024-04-05T22:27:53.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone90950 2024-04-05T22:27:53.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.680+0000 7fa4c7000640 1 Processor -- start 2024-04-05T22:27:53.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.680+0000 7fa4c7000640 1 -- start start 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.681+0000 7fa4c7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015a850 0x7fa4c015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c7000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4c015e580 0x7fa4c006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c7000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c006bc50 con 0x7fa4c015e9a0 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c7000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c015f530 con 0x7fa4c015e580 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c015ef70 con 0x7fa4c015a850 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4c015e580 0x7fa4c006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4c015e580 0x7fa4c006ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51332/0 (socket says 172.21.15.73:51332) 2024-04-05T22:27:53.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c5200640 1 -- 172.21.15.73:0/1177616852 learned_addr learned my addr 172.21.15.73:0/1177616852 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:53.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c5c00640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015a850 0x7fa4c015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.682+0000 7fa4c5200640 1 -- 172.21.15.73:0/1177616852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015a850 msgr2=0x7fa4c015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:53.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.683+0000 7fa4c5200640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015a850 0x7fa4c015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.683+0000 7fa4c5200640 1 -- 172.21.15.73:0/1177616852 >> v1:172.21.15.73:6789/0 conn(0x7fa4c015e9a0 legacy=0x7fa4c00b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.683+0000 7fa4c5200640 1 -- 172.21.15.73:0/1177616852 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c015f250 con 0x7fa4c015e580 2024-04-05T22:27:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.684+0000 7fa4c5200640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c006ff30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009870 tx=0x7fa4ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87611a86d2a6d4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.684+0000 7fa4bfe00640 1 -- 172.21.15.73:0/1177616852 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac030840 con 0x7fa4c015e580 2024-04-05T22:27:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.684+0000 7fa4bfe00640 1 -- 172.21.15.73:0/1177616852 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac0309a0 con 0x7fa4c015e580 2024-04-05T22:27:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.684+0000 7fa4bfe00640 1 -- 172.21.15.73:0/1177616852 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac030d30 con 0x7fa4c015e580 2024-04-05T22:27:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 -- 172.21.15.73:0/1177616852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 msgr2=0x7fa4c006ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c006ff30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009870 tx=0x7fa4ac02b980 comp rx=0 tx=0).stop 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 -- 172.21.15.73:0/1177616852 shutdown_connections 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c006ff30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1177616852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015a850 0x7fa4c015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 -- 172.21.15.73:0/1177616852 >> 172.21.15.73:0/1177616852 conn(0x7fa4c0069cb0 msgr2=0x7fa4c006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 -- 172.21.15.73:0/1177616852 shutdown_connections 2024-04-05T22:27:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.685+0000 7fa4c7000640 1 -- 172.21.15.73:0/1177616852 wait complete. 2024-04-05T22:27:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.687+0000 7fa4c7000640 1 Processor -- start 2024-04-05T22:27:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.687+0000 7fa4c7000640 1 -- start start 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 0x7fa4c014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 0x7fa4c0154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c015dee0 con 0x7fa4c015e580 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f59e0 con 0x7fa4c015a850 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c00cd820 con 0x7fa4c015e9a0 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51346/0 (socket says 172.21.15.73:51346) 2024-04-05T22:27:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 -- 172.21.15.73:0/1167545905 learned_addr learned my addr 172.21.15.73:0/1167545905 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c6600640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 0x7fa4c0154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5c00640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 0x7fa4c014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 -- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 msgr2=0x7fa4c0154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 0x7fa4c0154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 -- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 msgr2=0x7fa4c014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 0x7fa4c014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5200640 1 -- 172.21.15.73:0/1167545905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0094d0 con 0x7fa4c015e580 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c6600640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 0x7fa4c0154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.688+0000 7fa4c5c00640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 0x7fa4c014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4c5200640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000c00 tx=0x7fa4ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac039c90 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac039df0 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac043640 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c00cdb00 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.689+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c00ce0c0 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.690+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c004e160 con 0x7fa4c015e580 2024-04-05T22:27:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.691+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fa4ac0437a0 con 0x7fa4c015e580 2024-04-05T22:27:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.692+0000 7fa4bd600640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 0x7fa4a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.692+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4ac03e070 con 0x7fa4c015e580 2024-04-05T22:27:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.693+0000 7fa4c5c00640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 0x7fa4a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:53.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.694+0000 7fa4c5c00640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 0x7fa4a006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:53.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.696+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa4ac077750 con 0x7fa4c015e580 2024-04-05T22:27:53.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.920+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4c006c1d0 con 0x7fa4a006aed0 2024-04-05T22:27:53.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.939+0000 7fa4bd600640 1 -- 172.21.15.73:0/1167545905 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+724 (secure 0 0 0) 0x7fa4b40095d0 con 0x7fa4a006aed0 2024-04-05T22:27:53.939 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:27:53.939 INFO:teuthology.orchestra.run.smithi073.stdout: "atime": "2024-04-05 22:27:43", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_pcent": "80.00", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_quota": 10485760, 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "bytes_used": 8388608, 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "created_at": "2024-04-05 22:27:49", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "ctime": "2024-04-05 22:27:50", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "data_pool": "cephfs_data", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "features": [ 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-clone", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-autoprotect", 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot-retention" 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "flavor": 2, 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 1000, 2024-04-05T22:27:53.940 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": 16895, 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_addrs": [ 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.73:6789", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6789", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "172.21.15.157:6790" 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "mtime": "2024-04-05 22:27:48", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_namespace": "", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "complete", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "clone", 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout: "uid": 1000 2024-04-05T22:27:53.941 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 msgr2=0x7fa4a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 0x7fa4a006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4008040 comp rx=0 tx=0).stop 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 msgr2=0x7fa4c00cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000c00 tx=0x7fa4ac004730 comp rx=0 tx=0).stop 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 shutdown_connections 2024-04-05T22:27:53.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a006aed0 0x7fa4a006d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c015e9a0 0x7fa4c0154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c015e580 0x7fa4c00cc280 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 --2- 172.21.15.73:0/1167545905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c015a850 0x7fa4c014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.941+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 >> 172.21.15.73:0/1167545905 conn(0x7fa4c0069cb0 msgr2=0x7fa4c0157230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.942+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 shutdown_connections 2024-04-05T22:27:53.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:53.942+0000 7fa4c7000640 1 -- 172.21.15.73:0/1167545905 wait complete. 2024-04-05T22:27:53.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:53.961 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.005 INFO:teuthology.orchestra.run.smithi073.stderr:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510: ceph.quota.max_bytes: No such attribute 2024-04-05T22:27:54.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:27:54.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.006 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.101 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:27:54.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.101 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.151 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510 2024-04-05T22:27:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.2 2024-04-05T22:27:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.5 2024-04-05T22:27:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.4 2024-04-05T22:27:54.204 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.7 2024-04-05T22:27:54.204 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.6 2024-04-05T22:27:54.204 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.0 2024-04-05T22:27:54.204 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.3 2024-04-05T22:27:54.204 INFO:teuthology.orchestra.run.smithi073.stdout:volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.1 2024-04-05T22:27:54.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.205 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:41ff 2024-04-05T22:27:54.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.257 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/) 2024-04-05T22:27:54.306 INFO:teuthology.orchestra.run.smithi073.stdout:41ff 2024-04-05T22:27:54.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.307 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.357 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.358 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/) 2024-04-05T22:27:54.411 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.412 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.466 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.467 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/) 2024-04-05T22:27:54.518 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.520 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510) 2024-04-05T22:27:54.570 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:54.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.572 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/) 2024-04-05T22:27:54.624 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:54.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.625 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.2) 2024-04-05T22:27:54.676 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:54.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.678 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.2) 2024-04-05T22:27:54.734 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:54.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.735 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.2) 2024-04-05T22:27:54.786 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.788 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.2) 2024-04-05T22:27:54.836 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:54.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.838 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.2) 2024-04-05T22:27:54.886 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:54.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.887 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.2) 2024-04-05T22:27:54.934 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:54.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.935 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.2) 2024-04-05T22:27:54.984 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:54.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:54.985 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.2) 2024-04-05T22:27:55.034 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:55.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.036 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.5) 2024-04-05T22:27:55.090 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:55.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.091 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.5) 2024-04-05T22:27:55.152 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:55.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.153 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.5) 2024-04-05T22:27:55.204 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:55.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.206 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.5) 2024-04-05T22:27:55.257 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:55.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.258 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.5) 2024-04-05T22:27:55.310 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:55.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.312 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.5) 2024-04-05T22:27:55.361 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:55.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.362 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.5) 2024-04-05T22:27:55.412 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:55.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.413 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.5) 2024-04-05T22:27:55.461 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:55.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.462 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.4) 2024-04-05T22:27:55.516 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:55.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.517 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.4) 2024-04-05T22:27:55.575 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:55.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.577 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.4) 2024-04-05T22:27:55.625 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:55.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.627 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.4) 2024-04-05T22:27:55.673 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:55.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.675 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.4) 2024-04-05T22:27:55.772 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:55.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.773 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.4) 2024-04-05T22:27:55.823 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:55.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.825 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.4) 2024-04-05T22:27:55.876 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:55.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.877 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.4) 2024-04-05T22:27:55.934 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:55.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:55.935 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.7) 2024-04-05T22:27:56.000 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.002 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.7) 2024-04-05T22:27:56.060 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.062 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.7) 2024-04-05T22:27:56.113 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:56.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.114 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.7) 2024-04-05T22:27:56.169 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:56.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.170 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.7) 2024-04-05T22:27:56.220 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:56.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.221 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.7) 2024-04-05T22:27:56.270 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:56.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.272 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.7) 2024-04-05T22:27:56.327 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:56.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.328 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.7) 2024-04-05T22:27:56.380 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:56.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.382 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.6) 2024-04-05T22:27:56.435 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.437 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.6) 2024-04-05T22:27:56.487 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.488 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.6) 2024-04-05T22:27:56.535 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:56.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.536 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.6) 2024-04-05T22:27:56.585 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:56.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.587 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.6) 2024-04-05T22:27:56.638 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:56.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.639 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.6) 2024-04-05T22:27:56.690 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:56.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.691 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.6) 2024-04-05T22:27:56.745 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:56.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.746 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.6) 2024-04-05T22:27:56.798 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:56.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.800 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.0) 2024-04-05T22:27:56.851 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.853 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.0) 2024-04-05T22:27:56.902 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:56.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.904 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.0) 2024-04-05T22:27:56.950 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:56.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:56.953 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.0) 2024-04-05T22:27:56.999 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:57.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.001 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.0) 2024-04-05T22:27:57.095 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.097 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.0) 2024-04-05T22:27:57.146 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.148 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.0) 2024-04-05T22:27:57.200 INFO:teuthology.orchestra.run.smithi073.stdout:1712356067 2024-04-05T22:27:57.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.202 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.0) 2024-04-05T22:27:57.249 INFO:teuthology.orchestra.run.smithi073.stdout:1712356067 2024-04-05T22:27:57.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.251 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.3) 2024-04-05T22:27:57.303 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:57.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.304 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.3) 2024-04-05T22:27:57.353 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:57.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.355 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.3) 2024-04-05T22:27:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:57.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.404 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.3) 2024-04-05T22:27:57.451 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:57.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.453 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.3) 2024-04-05T22:27:57.505 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.507 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.3) 2024-04-05T22:27:57.554 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.556 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.3) 2024-04-05T22:27:57.609 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:57.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.611 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.3) 2024-04-05T22:27:57.661 INFO:teuthology.orchestra.run.smithi073.stdout:1712356068 2024-04-05T22:27:57.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.662 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.1) 2024-04-05T22:27:57.711 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:57.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.713 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.1) 2024-04-05T22:27:57.760 INFO:teuthology.orchestra.run.smithi073.stdout:81a4 2024-04-05T22:27:57.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.762 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.1) 2024-04-05T22:27:57.809 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:57.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.810 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.1) 2024-04-05T22:27:57.861 INFO:teuthology.orchestra.run.smithi073.stdout:1000 2024-04-05T22:27:57.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.863 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.1) 2024-04-05T22:27:57.913 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.914 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.1) 2024-04-05T22:27:57.963 INFO:teuthology.orchestra.run.smithi073.stdout:1301 2024-04-05T22:27:57.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:57.965 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609/.snap/subvol_snap6510/subvolume_file.1) 2024-04-05T22:27:58.023 INFO:teuthology.orchestra.run.smithi073.stdout:1712356067 2024-04-05T22:27:58.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:27:58.024 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone90950/80dfce96-a72b-4f81-81a3-586e6d224efe/subvolume_file.1) 2024-04-05T22:27:58.076 INFO:teuthology.orchestra.run.smithi073.stdout:1712356067 2024-04-05T22:27:58.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol44000 fake subvol_clone24791 2024-04-05T22:27:58.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.295+0000 7f1ebe800640 1 Processor -- start 2024-04-05T22:27:58.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.295+0000 7f1ebe800640 1 -- start start 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb81584e0 0x7f1eb806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1eb806b460 0x7f1eb815b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb80cda60 0x7f1eb80cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80cbd80 con 0x7f1eb80cda60 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb815be70 con 0x7f1eb806b460 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.296+0000 7f1ebe800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81588c0 con 0x7f1eb81584e0 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb80cda60 0x7f1eb80cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb80cda60 0x7f1eb80cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33020/0 (socket says 172.21.15.73:33020) 2024-04-05T22:27:58.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 -- 172.21.15.73:0/529550422 learned_addr learned my addr 172.21.15.73:0/529550422 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebd400640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb81584e0 0x7f1eb806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebca00640 1 --2- 172.21.15.73:0/529550422 >> v2:172.21.15.157:3300/0 conn(0x7f1eb806b460 0x7f1eb815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 -- 172.21.15.73:0/529550422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb81584e0 msgr2=0x7f1eb806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb81584e0 0x7f1eb806ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 -- 172.21.15.73:0/529550422 >> v2:172.21.15.157:3300/0 conn(0x7f1eb806b460 msgr2=0x7f1eb815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 --2- 172.21.15.73:0/529550422 >> v2:172.21.15.157:3300/0 conn(0x7f1eb806b460 0x7f1eb815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.297+0000 7f1ebde00640 1 -- 172.21.15.73:0/529550422 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb806b960 con 0x7f1eb80cda60 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1ebde00640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb80cda60 0x7f1eb80cdeb0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806f0f0 tx=0x7f1eb002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d23478859cedcae server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1eb7e00640 1 -- 172.21.15.73:0/529550422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb002ca70 con 0x7f1eb80cda60 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1eb7e00640 1 -- 172.21.15.73:0/529550422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb002cbd0 con 0x7f1eb80cda60 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1eb7e00640 1 -- 172.21.15.73:0/529550422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00343f0 con 0x7f1eb80cda60 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1ebe800640 1 -- 172.21.15.73:0/529550422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb80cda60 msgr2=0x7f1eb80cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.299+0000 7f1ebe800640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb80cda60 0x7f1eb80cdeb0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806f0f0 tx=0x7f1eb002bed0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 -- 172.21.15.73:0/529550422 shutdown_connections 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb80cda60 0x7f1eb80cdeb0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 --2- 172.21.15.73:0/529550422 >> v2:172.21.15.157:3300/0 conn(0x7f1eb806b460 0x7f1eb815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 --2- 172.21.15.73:0/529550422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb81584e0 0x7f1eb806ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 -- 172.21.15.73:0/529550422 >> 172.21.15.73:0/529550422 conn(0x7f1eb8069e90 msgr2=0x7f1eb806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 -- 172.21.15.73:0/529550422 shutdown_connections 2024-04-05T22:27:58.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.300+0000 7f1ebe800640 1 -- 172.21.15.73:0/529550422 wait complete. 2024-04-05T22:27:58.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.301+0000 7f1ebe800640 1 Processor -- start 2024-04-05T22:27:58.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.302+0000 7f1ebe800640 1 -- start start 2024-04-05T22:27:58.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.302+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 0x7f1eb81f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.302+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 0x7f1eb80af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebe800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81f6880 con 0x7f1eb81584e0 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebe800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ae7f0 con 0x7f1eb806b460 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebe800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80aead0 con 0x7f1eb80cda60 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33024/0 (socket says 172.21.15.73:33024) 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 -- 172.21.15.73:0/3445601574 learned_addr learned my addr 172.21.15.73:0/3445601574 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebd400640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 0x7f1eb81f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebca00640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 0x7f1eb80af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 -- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 msgr2=0x7f1eb80af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 0x7f1eb80af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 -- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 msgr2=0x7f1eb81f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 0x7f1eb81f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebde00640 1 -- 172.21.15.73:0/3445601574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009b30 con 0x7f1eb81584e0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebca00640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 0x7f1eb80af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.303+0000 7f1ebd400640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 0x7f1eb81f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1ebde00640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0036040 tx=0x7f1eb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0009510 con 0x7f1eb81584e0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb003e450 con 0x7f1eb81584e0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb80aedb0 con 0x7f1eb81584e0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb003e800 con 0x7f1eb81584e0 2024-04-05T22:27:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.304+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80af370 con 0x7f1eb81584e0 2024-04-05T22:27:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.305+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb804e160 con 0x7f1eb81584e0 2024-04-05T22:27:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.306+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f1eb003e960 con 0x7f1eb81584e0 2024-04-05T22:27:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.307+0000 7f1eb5600640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 0x7f1e9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.307+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eb00ac560 con 0x7f1eb81584e0 2024-04-05T22:27:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.308+0000 7f1ebd400640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 0x7f1e9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.309+0000 7f1ebd400640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 0x7f1e9806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1eac006fd0 tx=0x7f1eac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.311+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1eb00773f0 con 0x7f1eb81584e0 2024-04-05T22:27:58.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.542+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "fake", "target_sub_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eb806c200 con 0x7f1e9806aed0 2024-04-05T22:27:58.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.718+0000 7f1eb5600640 1 -- 172.21.15.73:0/3445601574 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1eac009530 con 0x7f1e9806aed0 2024-04-05T22:27:58.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.722+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 msgr2=0x7f1e9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.722+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 0x7f1e9806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1eac006fd0 tx=0x7f1eac008040 comp rx=0 tx=0).stop 2024-04-05T22:27:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.722+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 msgr2=0x7f1eb814bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.722+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0036040 tx=0x7f1eb0004a40 comp rx=0 tx=0).stop 2024-04-05T22:27:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 shutdown_connections 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e9806aed0 0x7f1e9806d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb81584e0 0x7f1eb814bb60 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb80cda60 0x7f1eb80af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 --2- 172.21.15.73:0/3445601574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb806b460 0x7f1eb81f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 >> 172.21.15.73:0/3445601574 conn(0x7f1eb8069e90 msgr2=0x7f1eb806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 shutdown_connections 2024-04-05T22:27:58.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.723+0000 7f1ebe800640 1 -- 172.21.15.73:0/3445601574 wait complete. 2024-04-05T22:27:58.749 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone24791 2024-04-05T22:27:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.954+0000 7fddd5400640 1 Processor -- start 2024-04-05T22:27:58.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.954+0000 7fddd5400640 1 -- start start 2024-04-05T22:27:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddd5400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddd0069cb0 0x7fddd0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddd5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 0x7fddd00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddd5400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd01612c0 con 0x7fddd006a090 2024-04-05T22:27:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddd5400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd01610d0 con 0x7fddd0069cb0 2024-04-05T22:27:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddd5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00c85d0 con 0x7fddd006be40 2024-04-05T22:27:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddc7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fddd006a090 0x7fddd015df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37060/0 (socket says 172.21.15.73:37060) 2024-04-05T22:27:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.956+0000 7fddc7e00640 1 -- 172.21.15.73:0/1127545752 learned_addr learned my addr 172.21.15.73:0/1127545752 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:27:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990216515 0 0) 0x7fddd01612c0 con 0x7fddd006a090 2024-04-05T22:27:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcf200640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 0x7fddd00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddce800640 1 --2- 172.21.15.73:0/1127545752 >> v2:172.21.15.157:3300/0 conn(0x7fddd0069cb0 0x7fddd0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb4003520 con 0x7fddd006a090 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133426133 0 0) 0x7fddb4003520 con 0x7fddd006a090 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 msgr2=0x7fddd00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 0x7fddd00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 >> v2:172.21.15.157:3300/0 conn(0x7fddd0069cb0 msgr2=0x7fddd0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 --2- 172.21.15.73:0/1127545752 >> v2:172.21.15.157:3300/0 conn(0x7fddd0069cb0 0x7fddd0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd01615a0 con 0x7fddd006a090 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fddb8001670 con 0x7fddd006a090 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 >> v1:172.21.15.73:6789/0 conn(0x7fddd006a090 legacy=0x7fddd015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 0x7fddb4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddb4003fa0 0x7fddb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb4004d00 0x7fddb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00c85d0 con 0x7fddb4003fa0 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400bac0 con 0x7fddb4004d00 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.957+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400bc80 con 0x7fddd006be40 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddb4003fa0 0x7fddb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 -- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 msgr2=0x7fddb4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddd006be40 0x7fddb4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 -- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb4004d00 msgr2=0x7fddb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb4004d00 0x7fddb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.958+0000 7fddc7e00640 1 -- 172.21.15.73:0/1127545752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4003520 con 0x7fddb4003fa0 2024-04-05T22:27:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.959+0000 7fddc7e00640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddb4003fa0 0x7fddb40047c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c240 tx=0x7fddb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddda73aeea7e51e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.959+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802cb00 con 0x7fddb4003fa0 2024-04-05T22:27:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.959+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb802cc60 con 0x7fddb4003fa0 2024-04-05T22:27:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.959+0000 7fddcde00640 1 -- 172.21.15.73:0/1127545752 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005af0 con 0x7fddb4003fa0 2024-04-05T22:27:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 -- 172.21.15.73:0/1127545752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddb4003fa0 msgr2=0x7fddb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 --2- 172.21.15.73:0/1127545752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddb4003fa0 0x7fddb40047c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c240 tx=0x7fddb40057f0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddcf200640 1 -- 172.21.15.73:0/1127545752 reap_dead start 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 -- 172.21.15.73:0/1127545752 shutdown_connections 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 -- 172.21.15.73:0/1127545752 >> 172.21.15.73:0/1127545752 conn(0x7fddd00b0ce0 msgr2=0x7fddd00b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 -- 172.21.15.73:0/1127545752 shutdown_connections 2024-04-05T22:27:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.960+0000 7fddd5400640 1 -- 172.21.15.73:0/1127545752 wait complete. 2024-04-05T22:27:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.962+0000 7fddd5400640 1 Processor -- start 2024-04-05T22:27:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.962+0000 7fddd5400640 1 -- start start 2024-04-05T22:27:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4003fa0 0x7fddd00cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddd009c150 0x7fddd01f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01f27e0 con 0x7fddd009c5d0 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00cba00 con 0x7fddd009c150 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddd5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0099fb0 con 0x7fddb4003fa0 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33038/0 (socket says 172.21.15.73:33038) 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 -- 172.21.15.73:0/4166899488 learned_addr learned my addr 172.21.15.73:0/4166899488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddc7e00640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddd009c150 0x7fddd01f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddce800640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4003fa0 0x7fddd00cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 -- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4003fa0 msgr2=0x7fddd00cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4003fa0 0x7fddd00cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 -- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddd009c150 msgr2=0x7fddd01f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddd009c150 0x7fddd01f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:27:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.963+0000 7fddcf200640 1 -- 172.21.15.73:0/4166899488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00cb440 con 0x7fddd009c5d0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddcf200640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fddd00b41b0 tx=0x7fddc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc800edc0 con 0x7fddd009c5d0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc8014420 con 0x7fddd009c5d0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80147d0 con 0x7fddd009c5d0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd00cb720 con 0x7fddd009c5d0 2024-04-05T22:27:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.964+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd00cf3a0 con 0x7fddd009c5d0 2024-04-05T22:27:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.965+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd00cf530 con 0x7fddd009c5d0 2024-04-05T22:27:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.966+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fddc8014930 con 0x7fddd009c5d0 2024-04-05T22:27:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.967+0000 7fddc6a00640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 0x7fddac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:27:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.967+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddc8089b70 con 0x7fddd009c5d0 2024-04-05T22:27:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.968+0000 7fddce800640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 0x7fddac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:27:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.969+0000 7fddce800640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 0x7fddac06d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddc00047b0 tx=0x7fddc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:27:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:58.971+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fddc8054a90 con 0x7fddd009c5d0 2024-04-05T22:27:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:27:59.202+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd006a510 con 0x7fddac06aed0 2024-04-05T22:28:02.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.219+0000 7fddc6a00640 1 -- 172.21.15.73:0/4166899488 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+153 (secure 0 0 0) 0x7fddc00094e0 con 0x7fddac06aed0 2024-04-05T22:28:02.219 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:02.219 INFO:teuthology.orchestra.run.smithi073.stdout: "status": { 2024-04-05T22:28:02.219 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "pending", 2024-04-05T22:28:02.219 INFO:teuthology.orchestra.run.smithi073.stdout: "source": { 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout: "volume": "cephfs", 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout: "subvolume": "subvol44000", 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot": "fake" 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:02.220 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 msgr2=0x7fddac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 0x7fddac06d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddc00047b0 tx=0x7fddc0008040 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 msgr2=0x7fddd01f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fddd00b41b0 tx=0x7fddc800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 shutdown_connections 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fddac06aed0 0x7fddac06d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd009c5d0 0x7fddd01f22a0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddd009c150 0x7fddd01f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 --2- 172.21.15.73:0/4166899488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4003fa0 0x7fddd00cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 >> 172.21.15.73:0/4166899488 conn(0x7fddd00b0ce0 msgr2=0x7fddd00b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 shutdown_connections 2024-04-05T22:28:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:02.222+0000 7fddd5400640 1 -- 172.21.15.73:0/4166899488 wait complete. 2024-04-05T22:28:03.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone24791 2024-04-05T22:28:03.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.470+0000 7f1df9400640 1 Processor -- start 2024-04-05T22:28:03.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.470+0000 7f1df9400640 1 -- start start 2024-04-05T22:28:03.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1df40b5890 0x7f1df406bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:03.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df9400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df406c140 con 0x7f1df40b1e10 2024-04-05T22:28:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df9400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df40b37e0 con 0x7f1df406b750 2024-04-05T22:28:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df9400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40b3ac0 con 0x7f1df40b5890 2024-04-05T22:28:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1df40b1e10 0x7f1df40cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37072/0 (socket says 172.21.15.73:37072) 2024-04-05T22:28:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df3e00640 1 -- 172.21.15.73:0/640629251 learned_addr learned my addr 172.21.15.73:0/640629251 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.472+0000 7f1df8a00640 1 --2- 172.21.15.73:0/640629251 >> v2:172.21.15.73:3300/0 conn(0x7f1df40b5890 0x7f1df406bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:03.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.473+0000 7f1df8a00640 1 -- 172.21.15.73:0/640629251 >> v1:172.21.15.157:6789/0 conn(0x7f1df406b750 legacy=0x7f1df415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:03.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.473+0000 7f1df8a00640 1 -- 172.21.15.73:0/640629251 >> v1:172.21.15.73:6789/0 conn(0x7f1df40b1e10 legacy=0x7f1df40cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:03.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.473+0000 7f1df8a00640 1 -- 172.21.15.73:0/640629251 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40b3da0 con 0x7f1df40b5890 2024-04-05T22:28:03.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.474+0000 7f1df8a00640 1 --2- 172.21.15.73:0/640629251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b5890 0x7f1df406bc00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009d90 tx=0x7f1dec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61556d18d9c1b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:03.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.474+0000 7f1df2a00640 1 -- 172.21.15.73:0/640629251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec02cbd0 con 0x7f1df40b5890 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.474+0000 7f1df2a00640 1 -- 172.21.15.73:0/640629251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec02cd30 con 0x7f1df40b5890 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.474+0000 7f1df2a00640 1 -- 172.21.15.73:0/640629251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec02cea0 con 0x7f1df40b5890 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 -- 172.21.15.73:0/640629251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b5890 msgr2=0x7f1df406bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 --2- 172.21.15.73:0/640629251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b5890 0x7f1df406bc00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009d90 tx=0x7f1dec02be20 comp rx=0 tx=0).stop 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 -- 172.21.15.73:0/640629251 shutdown_connections 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 --2- 172.21.15.73:0/640629251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b5890 0x7f1df406bc00 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 -- 172.21.15.73:0/640629251 >> 172.21.15.73:0/640629251 conn(0x7f1df40d09b0 msgr2=0x7f1df40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 -- 172.21.15.73:0/640629251 shutdown_connections 2024-04-05T22:28:03.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.475+0000 7f1df9400640 1 -- 172.21.15.73:0/640629251 wait complete. 2024-04-05T22:28:03.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.477+0000 7f1df9400640 1 Processor -- start 2024-04-05T22:28:03.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.477+0000 7f1df9400640 1 -- start start 2024-04-05T22:28:03.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 0x7f1df4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:03.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:03.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 0x7f1df40ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41f4640 con 0x7f1df40b1e10 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41580d0 con 0x7f1df40b5890 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41f39c0 con 0x7f1df406b750 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33052/0 (socket says 172.21.15.73:33052) 2024-04-05T22:28:03.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.478+0000 7f1df3400640 1 -- 172.21.15.73:0/114599594 learned_addr learned my addr 172.21.15.73:0/114599594 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df8a00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 0x7f1df40ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3e00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 0x7f1df4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 -- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 msgr2=0x7f1df4157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 0x7f1df4157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 -- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 msgr2=0x7f1df40ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 0x7f1df40ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 -- 172.21.15.73:0/114599594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009a70 con 0x7f1df40b1e10 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df8a00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 0x7f1df40ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3e00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 0x7f1df4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:03.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df3400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c430 tx=0x7f1de000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011790 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de00118f0 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011ce0 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.479+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41f3ca0 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.480+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df41f4260 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.481+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df41f4780 con 0x7f1df40b1e10 2024-04-05T22:28:03.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.482+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f1de001c460 con 0x7f1df40b1e10 2024-04-05T22:28:03.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.483+0000 7f1debe00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:03.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.483+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de00896a0 con 0x7f1df40b1e10 2024-04-05T22:28:03.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.484+0000 7f1df3e00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:03.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.485+0000 7f1df3e00640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 0x7f1dd406d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de4005880 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:03.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.489+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1df41f4780 con 0x7f1df40b1e10 2024-04-05T22:28:03.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:03.729+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) v1 -- 0x7f1df40b21f0 con 0x7f1dd406aed0 2024-04-05T22:28:07.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.202+0000 7f1debe00640 1 -- 172.21.15.73:0/114599594 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f1de40013d0 con 0x7f1dd406aed0 2024-04-05T22:28:07.202 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "status": { 2024-04-05T22:28:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "complete" 2024-04-05T22:28:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:07.203 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 msgr2=0x7f1dd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 0x7f1dd406d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de4005880 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 msgr2=0x7f1df40ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c430 tx=0x7f1de000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 shutdown_connections 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40b5890 0x7f1df40ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df40b1e10 0x7f1df40ae2a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 --2- 172.21.15.73:0/114599594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df406b750 0x7f1df4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 >> 172.21.15.73:0/114599594 conn(0x7f1df40d09b0 msgr2=0x7f1df40b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.205+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 shutdown_connections 2024-04-05T22:28:07.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.206+0000 7f1df9400640 1 -- 172.21.15.73:0/114599594 wait complete. 2024-04-05T22:28:07.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:07.228 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone24791/.meta) 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout:version = 2 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout:type = clone 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol_clone24791/8119f242-eced-417a-bace-a9bc0286985b 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:28:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:07.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol44000 subvol_snap6510 2024-04-05T22:28:07.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.537+0000 7f39c5600640 1 Processor -- start 2024-04-05T22:28:07.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.537+0000 7f39c5600640 1 -- start start 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.538+0000 7f39c5600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39c015ae50 0x7f39c00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39c5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39c00b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39c5600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c015b230 con 0x7f39c00cecd0 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39c5600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c00ce2f0 con 0x7f39c015ae50 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39c5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00cd2f0 con 0x7f39c00cce90 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39bf400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f39c00cecd0 0x7f39c00cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42460/0 (socket says 172.21.15.73:42460) 2024-04-05T22:28:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39bf400640 1 -- 172.21.15.73:0/2673321822 learned_addr learned my addr 172.21.15.73:0/2673321822 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:07.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39be000640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39c00b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> v2:172.21.15.157:3300/0 conn(0x7f39c015ae50 0x7f39c00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999698 0 0) 0x7f39c015b230 con 0x7f39c00cecd0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.539+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a4003520 con 0x7f39c00cecd0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1772278600 0 0) 0x7f39a4003520 con 0x7f39c00cecd0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 msgr2=0x7f39c00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39c00b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 >> v2:172.21.15.157:3300/0 conn(0x7f39c015ae50 msgr2=0x7f39c00b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 --2- 172.21.15.73:0/2673321822 >> v2:172.21.15.157:3300/0 conn(0x7f39c015ae50 0x7f39c00b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c015f4a0 con 0x7f39c00cecd0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f39b80019f0 con 0x7f39c00cecd0 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 >> v1:172.21.15.73:6789/0 conn(0x7f39c00cecd0 legacy=0x7f39c00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a4003970 0x7f39a4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4004d00 0x7f39a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00ce2f0 con 0x7f39a4003970 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00cd2f0 con 0x7f39a4004d00 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a400ba30 con 0x7f39c00cce90 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> v2:172.21.15.157:3300/0 conn(0x7f39c015ae50 0x7f39c00b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.540+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a4003970 0x7f39a4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bf400640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4004d00 0x7f39a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39be000640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bea00640 1 -- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 msgr2=0x7f39a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c00cce90 0x7f39a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bea00640 1 -- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4004d00 msgr2=0x7f39a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4004d00 0x7f39a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.541+0000 7f39bea00640 1 -- 172.21.15.73:0/2673321822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4003520 con 0x7f39a4003970 2024-04-05T22:28:07.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.542+0000 7f39bea00640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a4003970 0x7f39a4003db0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098a0 tx=0x7f39a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94200bdf378cfda server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:07.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.542+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0037600 con 0x7f39a4003970 2024-04-05T22:28:07.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.542+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0037760 con 0x7f39a4003970 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.542+0000 7f39bd600640 1 -- 172.21.15.73:0/2673321822 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0037ad0 con 0x7f39a4003970 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 -- 172.21.15.73:0/2673321822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a4003970 msgr2=0x7f39a4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 --2- 172.21.15.73:0/2673321822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a4003970 0x7f39a4003db0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098a0 tx=0x7f39a40057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39bf400640 1 -- 172.21.15.73:0/2673321822 reap_dead start 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 -- 172.21.15.73:0/2673321822 shutdown_connections 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 -- 172.21.15.73:0/2673321822 >> 172.21.15.73:0/2673321822 conn(0x7f39c006a0d0 msgr2=0x7f39c006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 -- 172.21.15.73:0/2673321822 shutdown_connections 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.543+0000 7f39c5600640 1 -- 172.21.15.73:0/2673321822 wait complete. 2024-04-05T22:28:07.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.545+0000 7f39c5600640 1 Processor -- start 2024-04-05T22:28:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.545+0000 7f39c5600640 1 -- start start 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 0x7f39c015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c01f27f0 0x7f39c01570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c015caa0 con 0x7f39c015ddc0 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c015cd20 con 0x7f39a4003970 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39c5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c015d000 con 0x7f39c01f27f0 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39be000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39be000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47138/0 (socket says 172.21.15.73:47138) 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.546+0000 7f39be000640 1 -- 172.21.15.73:0/1621505443 learned_addr learned my addr 172.21.15.73:0/1621505443 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39bea00640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 0x7f39c015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39bf400640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c01f27f0 0x7f39c01570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 -- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c01f27f0 msgr2=0x7f39c01570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c01f27f0 0x7f39c01570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 -- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 msgr2=0x7f39c015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 0x7f39c015c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 -- 172.21.15.73:0/1621505443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c015d2e0 con 0x7f39c015ddc0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39bea00640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 0x7f39c015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39be000640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f39ac011600 tx=0x7f39ac011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00f920 con 0x7f39c015ddc0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00fa80 con 0x7f39c015ddc0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00fdf0 con 0x7f39c015ddc0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.547+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c015d5c0 con 0x7f39c015ddc0 2024-04-05T22:28:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.548+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c00ad280 con 0x7f39c015ddc0 2024-04-05T22:28:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.549+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c006d490 con 0x7f39c015ddc0 2024-04-05T22:28:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.550+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f39ac015460 con 0x7f39c015ddc0 2024-04-05T22:28:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.550+0000 7f39b6a00640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 0x7f39a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.551+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39ac089d90 con 0x7f39c015ddc0 2024-04-05T22:28:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.551+0000 7f39bea00640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 0x7f39a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.552+0000 7f39bea00640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 0x7f39a006d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39b0036000 tx=0x7f39b0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:07.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.555+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f39ac08d030 con 0x7f39c015ddc0 2024-04-05T22:28:07.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.784+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "subvol_snap6510", "target": ["mon-mgr", ""]}) v1 -- 0x7f39c00cb610 con 0x7f39a006aed0 2024-04-05T22:28:07.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.832+0000 7f39b6a00640 1 -- 172.21.15.73:0/1621505443 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+115 (secure 0 0 0) 0x7f39b00378d0 con 0x7f39a006aed0 2024-04-05T22:28:07.833 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "created_at": "2024-04-05 22:27:49.346132", 2024-04-05T22:28:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "data_pool": "cephfs_data", 2024-04-05T22:28:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "has_pending_clones": "no" 2024-04-05T22:28:07.833 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.835+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 msgr2=0x7f39a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.835+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 0x7f39a006d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39b0036000 tx=0x7f39b0031000 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.835+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 msgr2=0x7f39c00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.835+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f39ac011600 tx=0x7f39ac011ac0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 shutdown_connections 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f39a006aed0 0x7f39a006d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c01f27f0 0x7f39c01570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39c015ddc0 0x7f39c00acd40 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 --2- 172.21.15.73:0/1621505443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39a4003970 0x7f39c015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 >> 172.21.15.73:0/1621505443 conn(0x7f39c006a0d0 msgr2=0x7f39c00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:07.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 shutdown_connections 2024-04-05T22:28:07.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:07.836+0000 7f39c5600640 1 -- 172.21.15.73:0/1621505443 wait complete. 2024-04-05T22:28:07.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol44000 2024-04-05T22:28:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.063+0000 7f0aab200640 1 Processor -- start 2024-04-05T22:28:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.063+0000 7f0aab200640 1 -- start start 2024-04-05T22:28:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aab200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0aac159990 0x7f0aac15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aab200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0aac0704a0 0x7f0aac0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aab200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac0caf80 con 0x7f0aac15dad0 2024-04-05T22:28:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aab200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac15df60 con 0x7f0aac159990 2024-04-05T22:28:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aab200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac0caa00 con 0x7f0aac0704a0 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aa9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0aac159990 0x7f0aac15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aa9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0aac159990 0x7f0aac15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47142/0 (socket says 172.21.15.73:47142) 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.065+0000 7f0aa9e00640 1 -- 172.21.15.73:0/2683311108 learned_addr learned my addr 172.21.15.73:0/2683311108 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.066+0000 7f0aaa800640 1 --2- 172.21.15.73:0/2683311108 >> v2:172.21.15.157:3300/0 conn(0x7f0aac0704a0 0x7f0aac0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.066+0000 7f0aa9e00640 1 -- 172.21.15.73:0/2683311108 >> v2:172.21.15.157:3300/0 conn(0x7f0aac0704a0 msgr2=0x7f0aac0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.066+0000 7f0aa9e00640 1 --2- 172.21.15.73:0/2683311108 >> v2:172.21.15.157:3300/0 conn(0x7f0aac0704a0 0x7f0aac0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.066+0000 7f0aa9e00640 1 -- 172.21.15.73:0/2683311108 >> v1:172.21.15.157:6789/0 conn(0x7f0aac15dad0 legacy=0x7f0aac0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:08.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.066+0000 7f0aa9e00640 1 -- 172.21.15.73:0/2683311108 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac06b610 con 0x7f0aac159990 2024-04-05T22:28:08.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.067+0000 7f0aa9e00640 1 --2- 172.21.15.73:0/2683311108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac159990 0x7f0aac15d590 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0671f0 tx=0x7f0a9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18f6fed19bf546b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.067+0000 7f0aa8a00640 1 -- 172.21.15.73:0/2683311108 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c031870 con 0x7f0aac159990 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.067+0000 7f0aa8a00640 1 -- 172.21.15.73:0/2683311108 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0319d0 con 0x7f0aac159990 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.067+0000 7f0aa8a00640 1 -- 172.21.15.73:0/2683311108 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c031d60 con 0x7f0aac159990 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 -- 172.21.15.73:0/2683311108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac159990 msgr2=0x7f0aac15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 --2- 172.21.15.73:0/2683311108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac159990 0x7f0aac15d590 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0671f0 tx=0x7f0a9c02b910 comp rx=0 tx=0).stop 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 -- 172.21.15.73:0/2683311108 shutdown_connections 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 --2- 172.21.15.73:0/2683311108 >> v2:172.21.15.157:3300/0 conn(0x7f0aac0704a0 0x7f0aac0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 --2- 172.21.15.73:0/2683311108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac159990 0x7f0aac15d590 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 -- 172.21.15.73:0/2683311108 >> 172.21.15.73:0/2683311108 conn(0x7f0aac0b0360 msgr2=0x7f0aac0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 -- 172.21.15.73:0/2683311108 shutdown_connections 2024-04-05T22:28:08.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.068+0000 7f0aab200640 1 -- 172.21.15.73:0/2683311108 wait complete. 2024-04-05T22:28:08.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.070+0000 7f0aab200640 1 Processor -- start 2024-04-05T22:28:08.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.070+0000 7f0aab200640 1 -- start start 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 0x7f0aac15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 0x7f0aac0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac148fe0 con 0x7f0aac0704a0 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac14d080 con 0x7f0aac159990 2024-04-05T22:28:08.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac15c290 con 0x7f0aac15dad0 2024-04-05T22:28:08.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aa9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aa9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47150/0 (socket says 172.21.15.73:47150) 2024-04-05T22:28:08.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.071+0000 7f0aa9e00640 1 -- 172.21.15.73:0/1715068400 learned_addr learned my addr 172.21.15.73:0/1715068400 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9400640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 0x7f0aac15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aaa800640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 0x7f0aac0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 -- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 msgr2=0x7f0aac0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 0x7f0aac0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 -- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 msgr2=0x7f0aac15bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 0x7f0aac15bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 -- 172.21.15.73:0/1715068400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0094d0 con 0x7f0aac0704a0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aaa800640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 0x7f0aac0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9400640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 0x7f0aac15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa9e00640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c000c00 tx=0x7f0a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c031840 con 0x7f0aac0704a0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0443f0 con 0x7f0aac0704a0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac14c470 con 0x7f0aac0704a0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.072+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0447a0 con 0x7f0aac0704a0 2024-04-05T22:28:08.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.073+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac14ca30 con 0x7f0aac0704a0 2024-04-05T22:28:08.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.074+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac04e160 con 0x7f0aac0704a0 2024-04-05T22:28:08.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.074+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f0a9c04d450 con 0x7f0aac0704a0 2024-04-05T22:28:08.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.075+0000 7f0aa2000640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.075+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a9c02d040 con 0x7f0aac0704a0 2024-04-05T22:28:08.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.076+0000 7f0aa9400640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.077+0000 7f0aa9400640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 0x7f0a8c06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0aac167b10 tx=0x7f0a98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.080+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0a9c076e40 con 0x7f0aac0704a0 2024-04-05T22:28:08.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.306+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aac0cac10 con 0x7f0a8c06aed0 2024-04-05T22:28:08.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.340+0000 7f0aa2000640 1 -- 172.21.15.73:0/1715068400 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+86 (secure 0 0 0) 0x7f0a9800ad60 con 0x7f0a8c06aed0 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fake" 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subvol_snap6510" 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:08.341 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.342+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 msgr2=0x7f0a8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.342+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 0x7f0a8c06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0aac167b10 tx=0x7f0a98008040 comp rx=0 tx=0).stop 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.342+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 msgr2=0x7f0aac148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.342+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c000c00 tx=0x7f0a9c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 shutdown_connections 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0aac15dad0 0x7f0aac0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0aac159990 0x7f0aac15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 --2- 172.21.15.73:0/1715068400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aac0704a0 0x7f0aac148aa0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 >> 172.21.15.73:0/1715068400 conn(0x7f0aac0b0360 msgr2=0x7f0aac0ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:08.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 shutdown_connections 2024-04-05T22:28:08.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.343+0000 7f0aab200640 1 -- 172.21.15.73:0/1715068400 wait complete. 2024-04-05T22:28:08.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol44000 subvol_snap6510 2024-04-05T22:28:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.577+0000 7f8477a00640 1 Processor -- start 2024-04-05T22:28:08.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.578+0000 7f8477a00640 1 -- start start 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8477a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84700cf3e0 0x7f84700b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8477a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8470154580 0x7f84700b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8477a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847015d900 con 0x7f84701541a0 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8477a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847015e1a0 con 0x7f84700cf3e0 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8477a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700b3490 con 0x7f8470154580 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8475000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f84701541a0 0x7f84700cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42468/0 (socket says 172.21.15.73:42468) 2024-04-05T22:28:08.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.579+0000 7f8475000640 1 -- 172.21.15.73:0/2396739731 learned_addr learned my addr 172.21.15.73:0/2396739731 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f846fe00640 1 --2- 172.21.15.73:0/2396739731 >> v2:172.21.15.73:3300/0 conn(0x7f84700cf3e0 0x7f84700b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f8475a00640 1 --2- 172.21.15.73:0/2396739731 >> v2:172.21.15.157:3300/0 conn(0x7f8470154580 0x7f84700b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f846fe00640 1 -- 172.21.15.73:0/2396739731 >> v2:172.21.15.157:3300/0 conn(0x7f8470154580 msgr2=0x7f84700b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f846fe00640 1 --2- 172.21.15.73:0/2396739731 >> v2:172.21.15.157:3300/0 conn(0x7f8470154580 0x7f84700b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f846fe00640 1 -- 172.21.15.73:0/2396739731 >> v1:172.21.15.73:6789/0 conn(0x7f84701541a0 legacy=0x7f84700cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.580+0000 7f846fe00640 1 -- 172.21.15.73:0/2396739731 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847015db50 con 0x7f84700cf3e0 2024-04-05T22:28:08.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.581+0000 7f846fe00640 1 --2- 172.21.15.73:0/2396739731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84700cf3e0 0x7f84700b2840 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f84700ae440 tx=0x7f845c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad382d99d19ce99 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.581+0000 7f846f400640 1 -- 172.21.15.73:0/2396739731 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c031830 con 0x7f84700cf3e0 2024-04-05T22:28:08.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f846f400640 1 -- 172.21.15.73:0/2396739731 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c031990 con 0x7f84700cf3e0 2024-04-05T22:28:08.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f846f400640 1 -- 172.21.15.73:0/2396739731 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c031d20 con 0x7f84700cf3e0 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 -- 172.21.15.73:0/2396739731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84700cf3e0 msgr2=0x7f84700b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 --2- 172.21.15.73:0/2396739731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84700cf3e0 0x7f84700b2840 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f84700ae440 tx=0x7f845c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 -- 172.21.15.73:0/2396739731 shutdown_connections 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 --2- 172.21.15.73:0/2396739731 >> v2:172.21.15.157:3300/0 conn(0x7f8470154580 0x7f84700b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 --2- 172.21.15.73:0/2396739731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84700cf3e0 0x7f84700b2840 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 -- 172.21.15.73:0/2396739731 >> 172.21.15.73:0/2396739731 conn(0x7f847006a0d0 msgr2=0x7f847006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 -- 172.21.15.73:0/2396739731 shutdown_connections 2024-04-05T22:28:08.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.582+0000 7f8477a00640 1 -- 172.21.15.73:0/2396739731 wait complete. 2024-04-05T22:28:08.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.584+0000 7f8477a00640 1 Processor -- start 2024-04-05T22:28:08.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.584+0000 7f8477a00640 1 -- start start 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84700cf3e0 0x7f847015c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701541a0 0x7f847015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700ccd80 con 0x7f8470154580 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470155930 con 0x7f84701541a0 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8477a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8470154b70 con 0x7f84700cf3e0 2024-04-05T22:28:08.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8475a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8475a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47166/0 (socket says 172.21.15.73:47166) 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.585+0000 7f8475a00640 1 -- 172.21.15.73:0/844124941 learned_addr learned my addr 172.21.15.73:0/844124941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475000640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84700cf3e0 0x7f847015c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f846fe00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701541a0 0x7f847015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 -- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84700cf3e0 msgr2=0x7f847015c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84700cf3e0 0x7f847015c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 -- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701541a0 msgr2=0x7f847015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701541a0 0x7f847015cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 -- 172.21.15.73:0/844124941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c009400 con 0x7f8470154580 2024-04-05T22:28:08.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f8475a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f846000c570 tx=0x7f846000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460004220 con 0x7f8470154580 2024-04-05T22:28:08.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84600043a0 con 0x7f8470154580 2024-04-05T22:28:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.586+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460003850 con 0x7f8470154580 2024-04-05T22:28:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.587+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8470154e50 con 0x7f8470154580 2024-04-05T22:28:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.587+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8470155410 con 0x7f8470154580 2024-04-05T22:28:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.588+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847006fcd0 con 0x7f8470154580 2024-04-05T22:28:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.589+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f84600039b0 con 0x7f8470154580 2024-04-05T22:28:08.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.590+0000 7f846cc00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 0x7f845006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:08.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.590+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8460089630 con 0x7f8470154580 2024-04-05T22:28:08.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.591+0000 7f8475000640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 0x7f845006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:08.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.592+0000 7f8475000640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 0x7f845006d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8468005c00 tx=0x7f8468009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:08.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.595+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f84600542e0 con 0x7f8470154580 2024-04-05T22:28:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.825+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "subvol_snap6510", "target": ["mon-mgr", ""]}) v1 -- 0x7f847006b420 con 0x7f845006aed0 2024-04-05T22:28:08.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.904+0000 7f846cc00640 1 -- 172.21.15.73:0/844124941 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f846800ec40 con 0x7f845006aed0 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 msgr2=0x7f845006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 0x7f845006d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8468005c00 tx=0x7f8468009040 comp rx=0 tx=0).stop 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 msgr2=0x7f84700cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f846000c570 tx=0x7f846000ca30 comp rx=0 tx=0).stop 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 shutdown_connections 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f845006aed0 0x7f845006d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8470154580 0x7f84700cc010 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84701541a0 0x7f847015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 --2- 172.21.15.73:0/844124941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84700cf3e0 0x7f847015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:08.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 >> 172.21.15.73:0/844124941 conn(0x7f847006a0d0 msgr2=0x7f8470152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:08.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 shutdown_connections 2024-04-05T22:28:08.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:08.907+0000 7f8477a00640 1 -- 172.21.15.73:0/844124941 wait complete. 2024-04-05T22:28:08.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol44000 fake 2024-04-05T22:28:09.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.136+0000 7fe056600640 1 Processor -- start 2024-04-05T22:28:09.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.136+0000 7fe056600640 1 -- start start 2024-04-05T22:28:09.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe056600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe05015ac00 0x7fe05006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe056600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05015afe0 con 0x7fe0500cf3e0 2024-04-05T22:28:09.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe056600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe050161760 con 0x7fe0500b36e0 2024-04-05T22:28:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe056600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05006d4c0 con 0x7fe05015ac00 2024-04-05T22:28:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe04fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe0500cf3e0 0x7fe05015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42478/0 (socket says 172.21.15.73:42478) 2024-04-05T22:28:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.138+0000 7fe04fe00640 1 -- 172.21.15.73:0/1825896807 learned_addr learned my addr 172.21.15.73:0/1825896807 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04f400640 1 --2- 172.21.15.73:0/1825896807 >> v2:172.21.15.157:3300/0 conn(0x7fe05015ac00 0x7fe05006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430498725 0 0) 0x7fe048001490 con 0x7fe0500cf3e0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe034003520 con 0x7fe0500cf3e0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815824239 0 0) 0x7fe034003520 con 0x7fe0500cf3e0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 >> v2:172.21.15.157:3300/0 conn(0x7fe05015ac00 msgr2=0x7fe05006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 --2- 172.21.15.73:0/1825896807 >> v2:172.21.15.157:3300/0 conn(0x7fe05015ac00 0x7fe05006cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 >> v1:172.21.15.157:6789/0 conn(0x7fe0500b36e0 legacy=0x7fe0500cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050160b50 con 0x7fe0500cf3e0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe048001b80 con 0x7fe0500cf3e0 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 >> v1:172.21.15.73:6789/0 conn(0x7fe0500cf3e0 legacy=0x7fe05015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034003970 0x7fe034003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe034004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0340052a0 0x7fe03400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05006d4c0 con 0x7fe0340047c0 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03400c150 con 0x7fe034003970 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03400c310 con 0x7fe0340052a0 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.139+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe034004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04f400640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034003970 0x7fe034003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04ea00640 1 -- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0340052a0 msgr2=0x7fe03400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0340052a0 0x7fe03400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04ea00640 1 -- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034003970 msgr2=0x7fe034003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034003970 0x7fe034003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04ea00640 1 -- 172.21.15.73:0/1825896807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034003520 con 0x7fe0340047c0 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.140+0000 7fe04fe00640 1 -- 172.21.15.73:0/1825896807 reap_dead start 2024-04-05T22:28:09.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.141+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe034004c40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe0500b3b90 tx=0x7fe0340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cda5f3415d94be server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.141+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c032600 con 0x7fe0340047c0 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.141+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c02e050 con 0x7fe0340047c0 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.141+0000 7fe04e000640 1 -- 172.21.15.73:0/1825896807 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0329e0 con 0x7fe0340047c0 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 -- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 msgr2=0x7fe034004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe034004c40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe0500b3b90 tx=0x7fe0340057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 -- 172.21.15.73:0/1825896807 shutdown_connections 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 --2- 172.21.15.73:0/1825896807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe034004c40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 -- 172.21.15.73:0/1825896807 >> 172.21.15.73:0/1825896807 conn(0x7fe05006a0d0 msgr2=0x7fe05006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 -- 172.21.15.73:0/1825896807 shutdown_connections 2024-04-05T22:28:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.142+0000 7fe056600640 1 -- 172.21.15.73:0/1825896807 wait complete. 2024-04-05T22:28:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.144+0000 7fe056600640 1 Processor -- start 2024-04-05T22:28:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.144+0000 7fe056600640 1 -- start start 2024-04-05T22:28:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe050147a90 0x7fe050147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050158c40 0x7fe0501590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050145ff0 con 0x7fe0340047c0 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05015e4e0 con 0x7fe050147a90 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe056600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe050145a00 con 0x7fe050158c40 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47180/0 (socket says 172.21.15.73:47180) 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 -- 172.21.15.73:0/1748132372 learned_addr learned my addr 172.21.15.73:0/1748132372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 -- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050158c40 msgr2=0x7fe0501590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050158c40 0x7fe0501590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 -- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe050147a90 msgr2=0x7fe050147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe050147a90 0x7fe050147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.145+0000 7fe04f400640 1 -- 172.21.15.73:0/1748132372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050148530 con 0x7fe0340047c0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe04f400640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe04000f600 tx=0x7fe04000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040014870 con 0x7fe0340047c0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0400149d0 con 0x7fe0340047c0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040014d40 con 0x7fe0340047c0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe050148810 con 0x7fe0340047c0 2024-04-05T22:28:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.146+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0501456b0 con 0x7fe0340047c0 2024-04-05T22:28:09.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.147+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe050068160 con 0x7fe0340047c0 2024-04-05T22:28:09.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.148+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fe04000d490 con 0x7fe0340047c0 2024-04-05T22:28:09.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.149+0000 7fe047400640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 0x7fe03006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:09.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.149+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe04008a1b0 con 0x7fe0340047c0 2024-04-05T22:28:09.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.150+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 0x7fe03006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:09.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.151+0000 7fe04ea00640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 0x7fe03006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe03c034000 tx=0x7fe03c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:09.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.153+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe0400550d0 con 0x7fe0340047c0 2024-04-05T22:28:09.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:28:09.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:28:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.382+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol44000", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7fe050159600 con 0x7fe03006aed0 2024-04-05T22:28:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.423+0000 7fe047400640 1 -- 172.21.15.73:0/1748132372 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe050159600 con 0x7fe03006aed0 2024-04-05T22:28:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.425+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 msgr2=0x7fe03006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.425+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 0x7fe03006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe03c034000 tx=0x7fe03c02f000 comp rx=0 tx=0).stop 2024-04-05T22:28:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.425+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 msgr2=0x7fe050145170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.425+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe04000f600 tx=0x7fe04000fac0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 shutdown_connections 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03006aed0 0x7fe03006d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050158c40 0x7fe0501590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe050147a90 0x7fe050147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 --2- 172.21.15.73:0/1748132372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0340047c0 0x7fe050145170 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 >> 172.21.15.73:0/1748132372 conn(0x7fe05006a0d0 msgr2=0x7fe05006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 shutdown_connections 2024-04-05T22:28:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:09.426+0000 7fe056600640 1 -- 172.21.15.73:0/1748132372 wait complete. 2024-04-05T22:28:09.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:09.444 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol44000/.meta) 2024-04-05T22:28:12.192 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:28:12.192 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:28:12.193 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:28:12.193 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol44000/df9851ae-b1df-4c3b-8418-4bee9a2cf609 2024-04-05T22:28:12.193 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:28:12.193 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:12.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol44000 2024-04-05T22:28:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.411+0000 7fe025600640 1 Processor -- start 2024-04-05T22:28:12.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.412+0000 7fe025600640 1 -- start start 2024-04-05T22:28:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.413+0000 7fe025600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe020160b90 0x7fe02015df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.413+0000 7fe025600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe020161010 con 0x7fe02006be40 2024-04-05T22:28:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.413+0000 7fe025600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0201612c0 con 0x7fe020069cb0 2024-04-05T22:28:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.413+0000 7fe025600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200c85d0 con 0x7fe020160b90 2024-04-05T22:28:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe02006be40 0x7fe0200c8a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42486/0 (socket says 172.21.15.73:42486) 2024-04-05T22:28:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01f400640 1 -- 172.21.15.73:0/3315684496 learned_addr learned my addr 172.21.15.73:0/3315684496 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe017e00640 1 --2- 172.21.15.73:0/3315684496 >> v2:172.21.15.157:3300/0 conn(0x7fe020160b90 0x7fe02015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388156386 0 0) 0x7fe020161010 con 0x7fe02006be40 2024-04-05T22:28:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe004003520 con 0x7fe02006be40 2024-04-05T22:28:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923875232 0 0) 0x7fe004003520 con 0x7fe02006be40 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 >> v2:172.21.15.157:3300/0 conn(0x7fe020160b90 msgr2=0x7fe02015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 --2- 172.21.15.73:0/3315684496 >> v2:172.21.15.157:3300/0 conn(0x7fe020160b90 0x7fe02015df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.414+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 >> v1:172.21.15.157:6789/0 conn(0x7fe020069cb0 legacy=0x7fe02006a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0201615a0 con 0x7fe02006be40 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe0180019f0 con 0x7fe02006be40 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 >> v1:172.21.15.73:6789/0 conn(0x7fe02006be40 legacy=0x7fe0200c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004003970 0x7fe004003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 0x7fe00400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c85d0 con 0x7fe004005180 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00400c120 con 0x7fe004003970 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00400c2e0 con 0x7fe0040047c0 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 0x7fe00400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01ea00640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004003970 0x7fe004003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 -- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040047c0 msgr2=0x7fe004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 -- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004003970 msgr2=0x7fe004003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004003970 0x7fe004003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.415+0000 7fe01f400640 1 -- 172.21.15.73:0/3315684496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe004003520 con 0x7fe004005180 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.416+0000 7fe01ea00640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004003970 0x7fe004003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.416+0000 7fe01f400640 1 -- 172.21.15.73:0/3315684496 reap_dead start 2024-04-05T22:28:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe01f400640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 0x7fe00400b980 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe01800a830 tx=0x7fe0040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbebd3093f3d7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180072d0 con 0x7fe004005180 2024-04-05T22:28:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01802f070 con 0x7fe004005180 2024-04-05T22:28:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe01e000640 1 -- 172.21.15.73:0/3315684496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018007690 con 0x7fe004005180 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe025600640 1 -- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 msgr2=0x7fe00400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe025600640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 0x7fe00400b980 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe01800a830 tx=0x7fe0040057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe025600640 1 -- 172.21.15.73:0/3315684496 shutdown_connections 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe025600640 1 --2- 172.21.15.73:0/3315684496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe004005180 0x7fe00400b980 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.417+0000 7fe025600640 1 -- 172.21.15.73:0/3315684496 >> 172.21.15.73:0/3315684496 conn(0x7fe0200b0ce0 msgr2=0x7fe0200b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.418+0000 7fe025600640 1 -- 172.21.15.73:0/3315684496 shutdown_connections 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.418+0000 7fe025600640 1 -- 172.21.15.73:0/3315684496 wait complete. 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.419+0000 7fe025600640 1 Processor -- start 2024-04-05T22:28:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.419+0000 7fe025600640 1 -- start start 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004005180 0x7fe020099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0201f1cd0 0x7fe0201f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009c4c0 con 0x7fe0200cc350 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201600f0 con 0x7fe004005180 2024-04-05T22:28:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.420+0000 7fe025600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009c170 con 0x7fe0201f1cd0 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47200/0 (socket says 172.21.15.73:47200) 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 -- 172.21.15.73:0/2585228951 learned_addr learned my addr 172.21.15.73:0/2585228951 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe01ea00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004005180 0x7fe020099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 -- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0201f1cd0 msgr2=0x7fe0201f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe01f400640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0201f1cd0 0x7fe0201f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0201f1cd0 0x7fe0201f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 -- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004005180 msgr2=0x7fe020099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004005180 0x7fe020099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 -- 172.21.15.73:0/2585228951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020160390 con 0x7fe0200cc350 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.421+0000 7fe017e00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe00800e7a0 tx=0x7fe00800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.422+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800eeb0 con 0x7fe0200cc350 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.422+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800c940 con 0x7fe0200cc350 2024-04-05T22:28:12.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.422+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800cd10 con 0x7fe0200cc350 2024-04-05T22:28:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.422+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0201f26c0 con 0x7fe0200cc350 2024-04-05T22:28:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.422+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0200b3520 con 0x7fe0200cc350 2024-04-05T22:28:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.423+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0201605d0 con 0x7fe0200cc350 2024-04-05T22:28:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.424+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fe008013470 con 0x7fe0200cc350 2024-04-05T22:28:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.425+0000 7fe016a00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 0x7fdffc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.425+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0080897e0 con 0x7fe0200cc350 2024-04-05T22:28:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.426+0000 7fe01ea00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 0x7fdffc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.427+0000 7fe01ea00640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 0x7fdffc06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe010000f60 tx=0x7fe010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:12.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.430+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe008054700 con 0x7fe0200cc350 2024-04-05T22:28:12.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.650+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol44000", "target": ["mon-mgr", ""]}) v1 -- 0x7fe02009bff0 con 0x7fdffc06af00 2024-04-05T22:28:12.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.756+0000 7fe016a00640 1 -- 172.21.15.73:0/2585228951 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe01000d050 con 0x7fdffc06af00 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.759+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 msgr2=0x7fdffc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 0x7fdffc06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe010000f60 tx=0x7fe010009040 comp rx=0 tx=0).stop 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 msgr2=0x7fe0200b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe00800e7a0 tx=0x7fe00800ec60 comp rx=0 tx=0).stop 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 shutdown_connections 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06af00 0x7fdffc06d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0201f1cd0 0x7fe0201f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200cc350 0x7fe0200b2fe0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 --2- 172.21.15.73:0/2585228951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004005180 0x7fe020099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 >> 172.21.15.73:0/2585228951 conn(0x7fe0200b0ce0 msgr2=0x7fe0200c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 shutdown_connections 2024-04-05T22:28:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.760+0000 7fe025600640 1 -- 172.21.15.73:0/2585228951 wait complete. 2024-04-05T22:28:12.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.767+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.767+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.767+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.767+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.767 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.767+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone90950 2024-04-05T22:28:12.793 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.794+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.793 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.794+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.793 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.794+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.793 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.794+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.793 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:12.794+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:12.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.982+0000 7f755e200640 1 Processor -- start 2024-04-05T22:28:12.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.982+0000 7f755e200640 1 -- start start 2024-04-05T22:28:12.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.983+0000 7f755e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f755815abe0 0x7f75580712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f755e200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f755806c9a0 0x7f755806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f755e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 0x7f75580cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f755e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755806bea0 con 0x7f755815abe0 2024-04-05T22:28:12.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f755e200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755815afc0 con 0x7f755806c9a0 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f755e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755806f5b0 con 0x7f755806ba70 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f755815abe0 0x7f75580712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f755815abe0 0x7f75580712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47208/0 (socket says 172.21.15.73:47208) 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557400640 1 -- 172.21.15.73:0/1605222560 learned_addr learned my addr 172.21.15.73:0/1605222560 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7556a00640 1 --2- 172.21.15.73:0/1605222560 >> v2:172.21.15.157:3300/0 conn(0x7f755806c9a0 0x7f755806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557e00640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 0x7f75580cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557400640 1 -- 172.21.15.73:0/1605222560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 msgr2=0x7f75580cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:12.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.984+0000 7f7557400640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 0x7f75580cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.985+0000 7f7557400640 1 -- 172.21.15.73:0/1605222560 >> v2:172.21.15.157:3300/0 conn(0x7f755806c9a0 msgr2=0x7f755806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:12.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.985+0000 7f7557400640 1 --2- 172.21.15.73:0/1605222560 >> v2:172.21.15.157:3300/0 conn(0x7f755806c9a0 0x7f755806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.985+0000 7f7557400640 1 -- 172.21.15.73:0/1605222560 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755806f890 con 0x7f755815abe0 2024-04-05T22:28:12.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.985+0000 7f7557e00640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 0x7f75580cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:12.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.986+0000 7f7557400640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75580712e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f755806ce30 tx=0x7f754802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5455870e26324d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:12.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.986+0000 7f7556000640 1 -- 172.21.15.73:0/1605222560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031750 con 0x7f755815abe0 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.986+0000 7f7556000640 1 -- 172.21.15.73:0/1605222560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480318b0 con 0x7f755815abe0 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.986+0000 7f7556000640 1 -- 172.21.15.73:0/1605222560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031c40 con 0x7f755815abe0 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 -- 172.21.15.73:0/1605222560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 msgr2=0x7f75580712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75580712e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f755806ce30 tx=0x7f754802b7f0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 -- 172.21.15.73:0/1605222560 shutdown_connections 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806ba70 0x7f75580cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 --2- 172.21.15.73:0/1605222560 >> v2:172.21.15.157:3300/0 conn(0x7f755806c9a0 0x7f755806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 --2- 172.21.15.73:0/1605222560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75580712e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 -- 172.21.15.73:0/1605222560 >> 172.21.15.73:0/1605222560 conn(0x7f7558069cb0 msgr2=0x7f755806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 -- 172.21.15.73:0/1605222560 shutdown_connections 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.987+0000 7f755e200640 1 -- 172.21.15.73:0/1605222560 wait complete. 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.989+0000 7f755e200640 1 Processor -- start 2024-04-05T22:28:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.990+0000 7f755e200640 1 -- start start 2024-04-05T22:28:12.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 0x7f75580c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806c9a0 0x7f7558159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755815a3f0 con 0x7f755815abe0 2024-04-05T22:28:12.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580c8cd0 con 0x7f755806ba70 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f755e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580cb4a0 con 0x7f755806c9a0 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47216/0 (socket says 172.21.15.73:47216) 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 -- 172.21.15.73:0/1220556016 learned_addr learned my addr 172.21.15.73:0/1220556016 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557400640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 0x7f75580c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 -- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806c9a0 msgr2=0x7f7558159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7556a00640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806c9a0 0x7f7558159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806c9a0 0x7f7558159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 -- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 msgr2=0x7f75580c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.991+0000 7f7557e00640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 0x7f75580c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f7557e00640 1 -- 172.21.15.73:0/1220556016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480094d0 con 0x7f755815abe0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f7557400640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 0x7f75580c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f7557e00640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f755000cef0 tx=0x7f755000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000d840 con 0x7f755815abe0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755000d9c0 con 0x7f755815abe0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75580c82b0 con 0x7f755815abe0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000dd70 con 0x7f755815abe0 2024-04-05T22:28:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.992+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75580c89f0 con 0x7f755815abe0 2024-04-05T22:28:12.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.994+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755806a190 con 0x7f755815abe0 2024-04-05T22:28:12.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.994+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f755001e3b0 con 0x7f755815abe0 2024-04-05T22:28:12.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.995+0000 7f754f400640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 0x7f753806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:12.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.995+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f755008a310 con 0x7f755815abe0 2024-04-05T22:28:12.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.996+0000 7f7557400640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 0x7f753806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:12.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:12.997+0000 7f7557400640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 0x7f753806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7548035000 tx=0x7f7548008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:13.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.000+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f75500551d0 con 0x7f755815abe0 2024-04-05T22:28:13.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.225+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone90950", "target": ["mon-mgr", ""]}) v1 -- 0x7f7558070130 con 0x7f753806aed0 2024-04-05T22:28:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.269+0000 7f754f400640 1 -- 172.21.15.73:0/1220556016 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f75480430c0 con 0x7f753806aed0 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.272+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 msgr2=0x7f753806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.272+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 0x7f753806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7548035000 tx=0x7f7548008030 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.272+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 msgr2=0x7f75581f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.272+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f755000cef0 tx=0x7f755000a710 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 shutdown_connections 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f753806aed0 0x7f753806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f755815abe0 0x7f75581f5e30 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f755806c9a0 0x7f7558159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 --2- 172.21.15.73:0/1220556016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f755806ba70 0x7f75580c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 >> 172.21.15.73:0/1220556016 conn(0x7f7558069cb0 msgr2=0x7f755815b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:13.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 shutdown_connections 2024-04-05T22:28:13.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.273+0000 7f755e200640 1 -- 172.21.15.73:0/1220556016 wait complete. 2024-04-05T22:28:13.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone24791 2024-04-05T22:28:13.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.502+0000 7f75e3200640 1 Processor -- start 2024-04-05T22:28:13.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.502+0000 7f75e3200640 1 -- start start 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75e40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75e415d310 0x7f75e40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e40d3a30 con 0x7f75e415cf30 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e40cdc50 con 0x7f75e415d310 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40cdf30 con 0x7f75e40cecf0 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e1e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f75e415cf30 0x7f75e40d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42502/0 (socket says 172.21.15.73:42502) 2024-04-05T22:28:13.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.504+0000 7f75e1e00640 1 -- 172.21.15.73:0/206165722 learned_addr learned my addr 172.21.15.73:0/206165722 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e2800640 1 --2- 172.21.15.73:0/206165722 >> v2:172.21.15.73:3300/0 conn(0x7f75e415d310 0x7f75e40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240395088 0 0) 0x7f75e40d3a30 con 0x7f75e415cf30 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8003520 con 0x7f75e415cf30 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e1400640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75e40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342091009 0 0) 0x7f75c8003520 con 0x7f75e415cf30 2024-04-05T22:28:13.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 msgr2=0x7f75e40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75e40d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 >> v2:172.21.15.73:3300/0 conn(0x7f75e415d310 msgr2=0x7f75e40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 --2- 172.21.15.73:0/206165722 >> v2:172.21.15.73:3300/0 conn(0x7f75e415d310 0x7f75e40d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40ce210 con 0x7f75e415cf30 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f75cc001670 con 0x7f75e415cf30 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 >> v1:172.21.15.73:6789/0 conn(0x7f75e415cf30 legacy=0x7f75e40d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c8003970 0x7f75c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75c8004d00 0x7f75c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40cdc50 con 0x7f75c8004d00 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40cdf30 con 0x7f75c8003970 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.505+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c800ba80 con 0x7f75e40cecf0 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75c8004d00 0x7f75c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e1e00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c8003970 0x7f75c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e1400640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 -- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 msgr2=0x7f75c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:13.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75e40cecf0 0x7f75c80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 -- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c8003970 msgr2=0x7f75c8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c8003970 0x7f75c8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e2800640 1 -- 172.21.15.73:0/206165722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8003520 con 0x7f75c8004d00 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.506+0000 7f75e1e00640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c8003970 0x7f75c8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.507+0000 7f75e2800640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75c8004d00 0x7f75c8005180 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ac0 tx=0x7f75c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6799be9e9efde7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc02c710 con 0x7f75c8004d00 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc02c870 con 0x7f75c8004d00 2024-04-05T22:28:13.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e0a00640 1 -- 172.21.15.73:0/206165722 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc02cc00 con 0x7f75c8004d00 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 -- 172.21.15.73:0/206165722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75c8004d00 msgr2=0x7f75c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 --2- 172.21.15.73:0/206165722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75c8004d00 0x7f75c8005180 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ac0 tx=0x7f75c80057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e2800640 1 -- 172.21.15.73:0/206165722 reap_dead start 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 -- 172.21.15.73:0/206165722 shutdown_connections 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 -- 172.21.15.73:0/206165722 >> 172.21.15.73:0/206165722 conn(0x7f75e409be30 msgr2=0x7f75e40d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 -- 172.21.15.73:0/206165722 shutdown_connections 2024-04-05T22:28:13.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.508+0000 7f75e3200640 1 -- 172.21.15.73:0/206165722 wait complete. 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.510+0000 7f75e3200640 1 Processor -- start 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.510+0000 7f75e3200640 1 -- start start 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75c8004d00 0x7f75e415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e415f680 0x7f75e415fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4160040 con 0x7f75e4151de0 2024-04-05T22:28:13.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40ca460 con 0x7f75e415f680 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.511+0000 7f75e3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e415e7b0 con 0x7f75c8004d00 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47240/0 (socket says 172.21.15.73:47240) 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 -- 172.21.15.73:0/614547566 learned_addr learned my addr 172.21.15.73:0/614547566 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 -- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75c8004d00 msgr2=0x7f75e415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e2800640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e415f680 0x7f75e415fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1e00640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75c8004d00 0x7f75e415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75c8004d00 0x7f75e415e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 -- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e415f680 msgr2=0x7f75e415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e415f680 0x7f75e415fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 -- 172.21.15.73:0/614547566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e415ea90 con 0x7f75e4151de0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75e1400640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f75d400da80 tx=0x7f75d400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.512+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4016440 con 0x7f75e4151de0 2024-04-05T22:28:13.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.513+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d40165c0 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.513+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4016970 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.513+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e415ed70 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.513+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e4152260 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.514+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e404e160 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.515+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f75d400e3f0 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.515+0000 7f75da000640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 0x7f75c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.516+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f75d4089fa0 con 0x7f75e4151de0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.516+0000 7f75e1e00640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 0x7f75c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:13.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.517+0000 7f75e1e00640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 0x7f75c006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0017a0 tx=0x7f75cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:13.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.520+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f75d4054da0 con 0x7f75e4151de0 2024-04-05T22:28:13.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.724+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f75d4055170 con 0x7f75e4151de0 2024-04-05T22:28:13.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.756+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) v1 -- 0x7f75e40ce7d0 con 0x7f75c006aed0 2024-04-05T22:28:13.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.796+0000 7f75da000640 1 -- 172.21.15.73:0/614547566 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f75cc00d040 con 0x7f75c006aed0 2024-04-05T22:28:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 msgr2=0x7f75c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 0x7f75c006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0017a0 tx=0x7f75cc009040 comp rx=0 tx=0).stop 2024-04-05T22:28:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 msgr2=0x7f75e40b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f75d400da80 tx=0x7f75d400df40 comp rx=0 tx=0).stop 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 shutdown_connections 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f75c006aed0 0x7f75c006d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e415f680 0x7f75e415fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75e4151de0 0x7f75e40b73a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 --2- 172.21.15.73:0/614547566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75c8004d00 0x7f75e415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.799+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 >> 172.21.15.73:0/614547566 conn(0x7f75e409be30 msgr2=0x7f75e406ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.800+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 shutdown_connections 2024-04-05T22:28:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:13.800+0000 7f75e3200640 1 -- 172.21.15.73:0/614547566 wait complete. 2024-04-05T22:28:13.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:13.819 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:28:13.871 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:28:13.873 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:28:13.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:28:13.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:28:14.081 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:28:14.082 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:14.082 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:14.138 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:14.139 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:14.194 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:14.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.406+0000 7f17a6200640 1 Processor -- start 2024-04-05T22:28:14.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.406+0000 7f17a6200640 1 -- start start 2024-04-05T22:28:14.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.408+0000 7f17a6200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f17a0160620 0x7f17a00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.408+0000 7f17a6200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a00ad880 con 0x7f17a00cb2f0 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.408+0000 7f17a6200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a00ce5d0 con 0x7f17a00b1870 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.408+0000 7f17a6200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a00ce010 con 0x7f17a0160620 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f17a0160620 0x7f17a00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f17a0160620 0x7f17a00caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47250/0 (socket says 172.21.15.73:47250) 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 -- 172.21.15.73:0/2407555153 learned_addr learned my addr 172.21.15.73:0/2407555153 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 -- 172.21.15.73:0/2407555153 >> v1:172.21.15.157:6789/0 conn(0x7f17a00b1870 legacy=0x7f17a00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:14.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 -- 172.21.15.73:0/2407555153 >> v1:172.21.15.73:6789/0 conn(0x7f17a00cb2f0 legacy=0x7f17a00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.409+0000 7f179f400640 1 -- 172.21.15.73:0/2407555153 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00ce2f0 con 0x7f17a0160620 2024-04-05T22:28:14.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.410+0000 7f179f400640 1 --2- 172.21.15.73:0/2407555153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a0160620 0x7f17a00caa60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1790009870 tx=0x7f179002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc9a0d07f959a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.410+0000 7f179e000640 1 -- 172.21.15.73:0/2407555153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790030860 con 0x7f17a0160620 2024-04-05T22:28:14.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f179e000640 1 -- 172.21.15.73:0/2407555153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17900309c0 con 0x7f17a0160620 2024-04-05T22:28:14.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f179e000640 1 -- 172.21.15.73:0/2407555153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790030d50 con 0x7f17a0160620 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f17a6200640 1 -- 172.21.15.73:0/2407555153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a0160620 msgr2=0x7f17a00caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f17a6200640 1 --2- 172.21.15.73:0/2407555153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a0160620 0x7f17a00caa60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1790009870 tx=0x7f179002b9a0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f17a6200640 1 -- 172.21.15.73:0/2407555153 shutdown_connections 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f17a6200640 1 --2- 172.21.15.73:0/2407555153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a0160620 0x7f17a00caa60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.411+0000 7f17a6200640 1 -- 172.21.15.73:0/2407555153 >> 172.21.15.73:0/2407555153 conn(0x7f17a00ce8e0 msgr2=0x7f17a00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.412+0000 7f17a6200640 1 -- 172.21.15.73:0/2407555153 shutdown_connections 2024-04-05T22:28:14.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.412+0000 7f17a6200640 1 -- 172.21.15.73:0/2407555153 wait complete. 2024-04-05T22:28:14.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.413+0000 7f17a6200640 1 Processor -- start 2024-04-05T22:28:14.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.414+0000 7f17a6200640 1 -- start start 2024-04-05T22:28:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00b1870 0x7f17a015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0160620 0x7f17a00aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a0158bc0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a015bbd0 con 0x7f17a0160620 2024-04-05T22:28:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f17a6200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a0143ce0 con 0x7f17a00b1870 2024-04-05T22:28:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f179ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f179ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47262/0 (socket says 172.21.15.73:47262) 2024-04-05T22:28:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.415+0000 7f179ea00640 1 -- 172.21.15.73:0/3183481702 learned_addr learned my addr 172.21.15.73:0/3183481702 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 -- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00b1870 msgr2=0x7f17a015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00b1870 0x7f17a015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 -- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0160620 msgr2=0x7f17a00aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0160620 0x7f17a00aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 -- 172.21.15.73:0/3183481702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900094d0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f179ea00640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f178800c410 tx=0x7f178800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17880035f0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1788003750 con 0x7f17a00cb2f0 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788003b40 con 0x7f17a00cb2f0 2024-04-05T22:28:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.416+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a015b8f0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.417+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a00ae7f0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.418+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a0066af0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.418+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f1788003ca0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.419+0000 7f1797400640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 0x7f177c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.419+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f1788088cb0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.420+0000 7f179f400640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 0x7f177c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.421+0000 7f179f400640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 0x7f177c06d4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1790036000 tx=0x7f179003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.424+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f17880538b0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.618+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17a0068160 con 0x7f17a00cb2f0 2024-04-05T22:28:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.619+0000 7f1797400640 1 -- 172.21.15.73:0/3183481702 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16193 (secure 0 0 0) 0x7f1788056ed0 con 0x7f17a00cb2f0 2024-04-05T22:28:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:14.620 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:13.699726+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 msgr2=0x7f177c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 0x7f177c06d4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1790036000 tx=0x7f179003b040 comp rx=0 tx=0).stop 2024-04-05T22:28:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 msgr2=0x7f17a015b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f178800c410 tx=0x7f178800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 shutdown_connections 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f177c06aff0 0x7f177c06d4b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0160620 0x7f17a00aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a00cb2f0 0x7f17a015b220 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 --2- 172.21.15.73:0/3183481702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00b1870 0x7f17a015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 >> 172.21.15.73:0/3183481702 conn(0x7f17a00ce8e0 msgr2=0x7f17a006f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 shutdown_connections 2024-04-05T22:28:14.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.622+0000 7f17a6200640 1 -- 172.21.15.73:0/3183481702 wait complete. 2024-04-05T22:28:14.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:27:30.588885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:27:31.104825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:28:14.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:28:14.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.854+0000 7fd68ea00640 1 Processor -- start 2024-04-05T22:28:14.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.854+0000 7fd68ea00640 1 -- start start 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.855+0000 7fd68ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd6881599f0 0x7fd688070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.855+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd68ea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6880ae190 con 0x7fd68806cb80 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd68ea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68806d010 con 0x7fd6881599f0 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd68ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880b3890 con 0x7fd6880add00 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd6881599f0 0x7fd688070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd6881599f0 0x7fd688070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47264/0 (socket says 172.21.15.73:47264) 2024-04-05T22:28:14.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 -- 172.21.15.73:0/1041974440 learned_addr learned my addr 172.21.15.73:0/1041974440 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:14.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687400640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 -- 172.21.15.73:0/1041974440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 msgr2=0x7fd68806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:14.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68806c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 -- 172.21.15.73:0/1041974440 >> v1:172.21.15.73:6789/0 conn(0x7fd68806cb80 legacy=0x7fd68815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.856+0000 7fd687e00640 1 -- 172.21.15.73:0/1041974440 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6880b27f0 con 0x7fd6881599f0 2024-04-05T22:28:14.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.857+0000 7fd687e00640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688070a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd678009e80 tx=0x7fd67802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cebc8c503e67f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd686a00640 1 -- 172.21.15.73:0/1041974440 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67802caf0 con 0x7fd6881599f0 2024-04-05T22:28:14.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd686a00640 1 -- 172.21.15.73:0/1041974440 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67802cc50 con 0x7fd6881599f0 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd686a00640 1 -- 172.21.15.73:0/1041974440 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6780343f0 con 0x7fd6881599f0 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 -- 172.21.15.73:0/1041974440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 msgr2=0x7fd688070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688070a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd678009e80 tx=0x7fd67802bf50 comp rx=0 tx=0).stop 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 -- 172.21.15.73:0/1041974440 shutdown_connections 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68806c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1041974440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688070a40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 -- 172.21.15.73:0/1041974440 >> 172.21.15.73:0/1041974440 conn(0x7fd688069cb0 msgr2=0x7fd68806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 -- 172.21.15.73:0/1041974440 shutdown_connections 2024-04-05T22:28:14.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.858+0000 7fd68ea00640 1 -- 172.21.15.73:0/1041974440 wait complete. 2024-04-05T22:28:14.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.860+0000 7fd68ea00640 1 Processor -- start 2024-04-05T22:28:14.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.860+0000 7fd68ea00640 1 -- start start 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68806cb80 0x7fd6881403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68814e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd688140930 con 0x7fd6881599f0 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68814e230 con 0x7fd68806cb80 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68814cfe0 con 0x7fd6880add00 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47272/0 (socket says 172.21.15.73:47272) 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.861+0000 7fd68ca00640 1 -- 172.21.15.73:0/1357897152 learned_addr learned my addr 172.21.15.73:0/1357897152 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd687400640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68814e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd687e00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68806cb80 0x7fd6881403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 -- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 msgr2=0x7fd68814e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68814e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 -- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68806cb80 msgr2=0x7fd6881403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68806cb80 0x7fd6881403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 -- 172.21.15.73:0/1357897152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678009b30 con 0x7fd6881599f0 2024-04-05T22:28:14.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ca00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c390 tx=0x7fd68000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6800052d0 con 0x7fd6881599f0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680005430 con 0x7fd6881599f0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68814d620 con 0x7fd6881599f0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.862+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680003810 con 0x7fd6881599f0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.863+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68814dbe0 con 0x7fd6881599f0 2024-04-05T22:28:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.864+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68814e3a0 con 0x7fd6881599f0 2024-04-05T22:28:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.865+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fd680003970 con 0x7fd6881599f0 2024-04-05T22:28:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.866+0000 7fd67fe00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:14.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.866+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fd680088f20 con 0x7fd6881599f0 2024-04-05T22:28:14.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.866+0000 7fd687e00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:14.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.867+0000 7fd687e00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 0x7fd66806d4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd678035000 tx=0x7fd678030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:14.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:14.871+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd680053b60 con 0x7fd6881599f0 2024-04-05T22:28:15.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.062+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd688068160 con 0x7fd6881599f0 2024-04-05T22:28:15.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.062+0000 7fd67fe00640 1 -- 172.21.15.73:0/1357897152 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6800170a0 con 0x7fd6881599f0 2024-04-05T22:28:15.063 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 msgr2=0x7fd66806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 0x7fd66806d4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd678035000 tx=0x7fd678030000 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 msgr2=0x7fd688153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c390 tx=0x7fd68000c850 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 shutdown_connections 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6881599f0 0x7fd688153520 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6880add00 0x7fd68814e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 --2- 172.21.15.73:0/1357897152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68806cb80 0x7fd6881403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.065+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 >> 172.21.15.73:0/1357897152 conn(0x7fd688069cb0 msgr2=0x7fd688070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.066+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 shutdown_connections 2024-04-05T22:28:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.066+0000 7fd68ea00640 1 -- 172.21.15.73:0/1357897152 wait complete. 2024-04-05T22:28:15.084 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:28:15.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.297+0000 7f0dc8c00640 1 Processor -- start 2024-04-05T22:28:15.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.297+0000 7f0dc8c00640 1 -- start start 2024-04-05T22:28:15.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.298+0000 7f0dc8c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 0x7f0dc40d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.298+0000 7f0dc8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 0x7f0dc40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.298+0000 7f0dc8c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc40cd630 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.298+0000 7f0dc8c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc40ceab0 con 0x7f0dc415cdc0 2024-04-05T22:28:15.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.298+0000 7f0dc8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc415e280 con 0x7f0dc40d1780 2024-04-05T22:28:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.299+0000 7f0dc3400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 0x7f0dc40d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.299+0000 7f0dc3400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 0x7f0dc40d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:33212/0 (socket says 172.21.15.73:33212) 2024-04-05T22:28:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.299+0000 7f0dc3400640 1 -- 172.21.15.73:0/1085731443 learned_addr learned my addr 172.21.15.73:0/1085731443 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:15.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.299+0000 7f0dc3e00640 1 --2- 172.21.15.73:0/1085731443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 0x7f0dc40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.300+0000 7f0dc3e00640 1 -- 172.21.15.73:0/1085731443 >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 msgr2=0x7f0dc40d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.300+0000 7f0dc3e00640 1 --2- 172.21.15.73:0/1085731443 >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 0x7f0dc40d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.300+0000 7f0dc3e00640 1 -- 172.21.15.73:0/1085731443 >> v1:172.21.15.157:6789/0 conn(0x7f0dc40d0bc0 legacy=0x7f0dc40d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.300+0000 7f0dc3e00640 1 -- 172.21.15.73:0/1085731443 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc415d700 con 0x7f0dc40d1780 2024-04-05T22:28:15.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.301+0000 7f0dc3e00640 1 --2- 172.21.15.73:0/1085731443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 0x7f0dc40b37c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31511c06c301c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.302+0000 7f0dc2000640 1 -- 172.21.15.73:0/1085731443 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc02cc50 con 0x7f0dc40d1780 2024-04-05T22:28:15.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.302+0000 7f0dc2000640 1 -- 172.21.15.73:0/1085731443 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc02cdb0 con 0x7f0dc40d1780 2024-04-05T22:28:15.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.302+0000 7f0dc2000640 1 -- 172.21.15.73:0/1085731443 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc034660 con 0x7f0dc40d1780 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 -- 172.21.15.73:0/1085731443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 msgr2=0x7f0dc40b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/1085731443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 0x7f0dc40b37c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc02bea0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 -- 172.21.15.73:0/1085731443 shutdown_connections 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/1085731443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc40d1780 0x7f0dc40b37c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/1085731443 >> v2:172.21.15.157:3300/0 conn(0x7f0dc415cdc0 0x7f0dc40d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 -- 172.21.15.73:0/1085731443 >> 172.21.15.73:0/1085731443 conn(0x7f0dc409be20 msgr2=0x7f0dc40d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:15.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 -- 172.21.15.73:0/1085731443 shutdown_connections 2024-04-05T22:28:15.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.303+0000 7f0dc8c00640 1 -- 172.21.15.73:0/1085731443 wait complete. 2024-04-05T22:28:15.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.305+0000 7f0dc8c00640 1 Processor -- start 2024-04-05T22:28:15.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.305+0000 7f0dc8c00640 1 -- start start 2024-04-05T22:28:15.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dc40d1780 0x7f0dc41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc415cdc0 0x7f0dc41f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc415a800 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4159bf0 con 0x7f0dc40d1780 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4159ed0 con 0x7f0dc415cdc0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47288/0 (socket says 172.21.15.73:47288) 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.306+0000 7f0dc3400640 1 -- 172.21.15.73:0/3255705684 learned_addr learned my addr 172.21.15.73:0/3255705684 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3e00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc415cdc0 0x7f0dc41f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc2a00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dc40d1780 0x7f0dc41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 -- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc415cdc0 msgr2=0x7f0dc41f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc415cdc0 0x7f0dc41f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 -- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dc40d1780 msgr2=0x7f0dc41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dc40d1780 0x7f0dc41592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 -- 172.21.15.73:0/3255705684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc009a70 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc3400640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009fd0 tx=0x7f0db400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400c940 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400caa0 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400ce30 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc415a1b0 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.307+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc415ac90 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.308+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc404e160 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.309+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f0db4013470 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.311+0000 7f0dbb400640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 0x7f0da006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.311+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f0db4089c70 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.311+0000 7f0dc2a00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 0x7f0da006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.312+0000 7f0dc2a00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 0x7f0da006d4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0dc41f5fc0 tx=0x7f0dac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.315+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0db40547e0 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.510+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0dc40d3500 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.510+0000 7f0dbb400640 1 -- 172.21.15.73:0/3255705684 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0db4017020 con 0x7f0dc40d0bc0 2024-04-05T22:28:15.510 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 msgr2=0x7f0da006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 0x7f0da006d4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0dc41f5fc0 tx=0x7f0dac009040 comp rx=0 tx=0).stop 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 msgr2=0x7f0dc4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009fd0 tx=0x7f0db400ed40 comp rx=0 tx=0).stop 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 shutdown_connections 2024-04-05T22:28:15.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0da006b020 0x7f0da006d4e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dc415cdc0 0x7f0dc41f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dc40d1780 0x7f0dc41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 --2- 172.21.15.73:0/3255705684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0dc40d0bc0 0x7f0dc4157a40 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 >> 172.21.15.73:0/3255705684 conn(0x7f0dc409be20 msgr2=0x7f0dc40b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 shutdown_connections 2024-04-05T22:28:15.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.512+0000 7f0dc8c00640 1 -- 172.21.15.73:0/3255705684 wait complete. 2024-04-05T22:28:15.530 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:28:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.740+0000 7fc6d3800640 1 Processor -- start 2024-04-05T22:28:15.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.740+0000 7fc6d3800640 1 -- start start 2024-04-05T22:28:15.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d3800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d3800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6cc0b2360 0x7fc6cc0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d3800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0cd090 con 0x7fc6cc15d970 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d3800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ce760 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d3800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0b1bd0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47292/0 (socket says 172.21.15.73:47292) 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 -- 172.21.15.73:0/3401552601 learned_addr learned my addr 172.21.15.73:0/3401552601 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 -- 172.21.15.73:0/3401552601 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc0b2360 msgr2=0x7fc6cc0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d1800640 1 --2- 172.21.15.73:0/3401552601 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc0b2360 0x7fc6cc0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 --2- 172.21.15.73:0/3401552601 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc0b2360 0x7fc6cc0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.742+0000 7fc6d0e00640 1 -- 172.21.15.73:0/3401552601 >> v1:172.21.15.157:6789/0 conn(0x7fc6cc15d970 legacy=0x7fc6cc15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:15.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.743+0000 7fc6d0e00640 1 -- 172.21.15.73:0/3401552601 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0cd2e0 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.744+0000 7fc6d0e00640 1 --2- 172.21.15.73:0/3401552601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc06b200 tx=0x7fc6c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e32789d2e50eda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.744+0000 7fc6cb400640 1 -- 172.21.15.73:0/3401552601 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0031870 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.744+0000 7fc6cb400640 1 -- 172.21.15.73:0/3401552601 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c00319d0 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.744+0000 7fc6cb400640 1 -- 172.21.15.73:0/3401552601 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0031d60 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 -- 172.21.15.73:0/3401552601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc15cdd0 msgr2=0x7fc6cc0ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 --2- 172.21.15.73:0/3401552601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc06b200 tx=0x7fc6c002b910 comp rx=0 tx=0).stop 2024-04-05T22:28:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 -- 172.21.15.73:0/3401552601 shutdown_connections 2024-04-05T22:28:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 --2- 172.21.15.73:0/3401552601 >> v2:172.21.15.157:3300/0 conn(0x7fc6cc0b2360 0x7fc6cc0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 --2- 172.21.15.73:0/3401552601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc0ce220 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 -- 172.21.15.73:0/3401552601 >> 172.21.15.73:0/3401552601 conn(0x7fc6cc0b0830 msgr2=0x7fc6cc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:15.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 -- 172.21.15.73:0/3401552601 shutdown_connections 2024-04-05T22:28:15.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.745+0000 7fc6d3800640 1 -- 172.21.15.73:0/3401552601 wait complete. 2024-04-05T22:28:15.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.746+0000 7fc6d3800640 1 Processor -- start 2024-04-05T22:28:15.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 -- start start 2024-04-05T22:28:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc15d970 0x7fc6cc1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc13d4b0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc13c830 con 0x7fc6cc15cdd0 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.747+0000 7fc6d3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc13cb10 con 0x7fc6cc15d970 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47308/0 (socket says 172.21.15.73:47308) 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 -- 172.21.15.73:0/1900116777 learned_addr learned my addr 172.21.15.73:0/1900116777 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 -- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc15d970 msgr2=0x7fc6cc1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc15d970 0x7fc6cc1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 -- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc15cdd0 msgr2=0x7fc6cc13a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 -- 172.21.15.73:0/1900116777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00094d0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.748+0000 7fc6d0e00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.749+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c003ad10 con 0x7fc6cc0b2360 2024-04-05T22:28:15.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.749+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c003ae70 con 0x7fc6cc0b2360 2024-04-05T22:28:15.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.749+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0044610 con 0x7fc6cc0b2360 2024-04-05T22:28:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.749+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc13cdf0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.749+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc13c4e0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.750+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc13d7d0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.751+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fc6c0044770 con 0x7fc6cc0b2360 2024-04-05T22:28:15.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.752+0000 7fc6c8c00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 0x7fc6ac06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:15.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.752+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fc6c002d040 con 0x7fc6cc0b2360 2024-04-05T22:28:15.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.752+0000 7fc6cbe00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 0x7fc6ac06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:15.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.753+0000 7fc6cbe00640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 0x7fc6ac06d4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0aead0 tx=0x7fc6b8007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:15.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.757+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc6cc13d7d0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.953+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6cc13d7d0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.955+0000 7fc6c8c00640 1 -- 172.21.15.73:0/1900116777 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6cc13d7d0 con 0x7fc6cc0b2360 2024-04-05T22:28:15.954 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 msgr2=0x7fc6ac06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 0x7fc6ac06d4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0aead0 tx=0x7fc6b8007990 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 msgr2=0x7fc6cc06cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 shutdown_connections 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc6ac06aff0 0x7fc6ac06d4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6cc15d970 0x7fc6cc1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc15cdd0 0x7fc6cc13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 --2- 172.21.15.73:0/1900116777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6cc0b2360 0x7fc6cc06cb70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:15.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 >> 172.21.15.73:0/1900116777 conn(0x7fc6cc0b0830 msgr2=0x7fc6cc143390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:15.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 shutdown_connections 2024-04-05T22:28:15.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:15.957+0000 7fc6d3800640 1 -- 172.21.15.73:0/1900116777 wait complete. 2024-04-05T22:28:15.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:28:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.189+0000 7f3afda00640 1 Processor -- start 2024-04-05T22:28:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.189+0000 7f3afda00640 1 -- start start 2024-04-05T22:28:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.190+0000 7f3afda00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3af815cf70 0x7f3af80d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.190+0000 7f3afda00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af80d0bd0 0x7f3af80d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.190+0000 7f3afda00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af80ceee0 con 0x7f3af815d350 2024-04-05T22:28:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.190+0000 7f3afda00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af80ce5b0 con 0x7f3af815cf70 2024-04-05T22:28:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.190+0000 7f3afda00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80ce890 con 0x7f3af80d0bd0 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3af815cf70 0x7f3af80d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3af815cf70 0x7f3af80d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60582/0 (socket says 172.21.15.73:60582) 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 -- 172.21.15.73:0/4254984235 learned_addr learned my addr 172.21.15.73:0/4254984235 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3aefe00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af80d0bd0 0x7f3af80d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 -- 172.21.15.73:0/4254984235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af80d0bd0 msgr2=0x7f3af80d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af80d0bd0 0x7f3af80d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 -- 172.21.15.73:0/4254984235 >> v1:172.21.15.157:6789/0 conn(0x7f3af815d350 legacy=0x7f3af80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.191+0000 7f3af6a00640 1 -- 172.21.15.73:0/4254984235 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80ceb70 con 0x7f3af815cf70 2024-04-05T22:28:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.192+0000 7f3af6a00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815cf70 0x7f3af80d3910 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009870 tx=0x7f3ae802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bfd427e7051bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.193+0000 7f3af6000640 1 -- 172.21.15.73:0/4254984235 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80317c0 con 0x7f3af815cf70 2024-04-05T22:28:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.193+0000 7f3af6000640 1 -- 172.21.15.73:0/4254984235 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8031920 con 0x7f3af815cf70 2024-04-05T22:28:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.193+0000 7f3af6000640 1 -- 172.21.15.73:0/4254984235 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8031cb0 con 0x7f3af815cf70 2024-04-05T22:28:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.193+0000 7f3afda00640 1 -- 172.21.15.73:0/4254984235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815cf70 msgr2=0x7f3af80d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.193+0000 7f3afda00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815cf70 0x7f3af80d3910 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009870 tx=0x7f3ae802b860 comp rx=0 tx=0).stop 2024-04-05T22:28:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 -- 172.21.15.73:0/4254984235 shutdown_connections 2024-04-05T22:28:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af80d0bd0 0x7f3af80d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 --2- 172.21.15.73:0/4254984235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815cf70 0x7f3af80d3910 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 -- 172.21.15.73:0/4254984235 >> 172.21.15.73:0/4254984235 conn(0x7f3af80d2df0 msgr2=0x7f3af80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 -- 172.21.15.73:0/4254984235 shutdown_connections 2024-04-05T22:28:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.194+0000 7f3afda00640 1 -- 172.21.15.73:0/4254984235 wait complete. 2024-04-05T22:28:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.196+0000 7f3afda00640 1 Processor -- start 2024-04-05T22:28:16.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.196+0000 7f3afda00640 1 -- start start 2024-04-05T22:28:16.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80d0bd0 0x7f3af806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af815cf70 0x7f3af806be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8150ca0 con 0x7f3af815d350 2024-04-05T22:28:16.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8150090 con 0x7f3af80d0bd0 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3afda00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8150370 con 0x7f3af815cf70 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60584/0 (socket says 172.21.15.73:60584) 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 -- 172.21.15.73:0/1668851213 learned_addr learned my addr 172.21.15.73:0/1668851213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af6a00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80d0bd0 0x7f3af806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 -- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af815cf70 msgr2=0x7f3af806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3aefe00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af815cf70 0x7f3af806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af815cf70 0x7f3af806be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 -- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80d0bd0 msgr2=0x7f3af806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80d0bd0 0x7f3af806b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.197+0000 7f3af7400640 1 -- 172.21.15.73:0/1668851213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80094d0 con 0x7f3af815d350 2024-04-05T22:28:16.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3af7400640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b990 tx=0x7f3af000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af000cba0 con 0x7f3af815d350 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af000cd00 con 0x7f3af815d350 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0014680 con 0x7f3af815d350 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af8150650 con 0x7f3af815d350 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.198+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af80b47b0 con 0x7f3af815d350 2024-04-05T22:28:16.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.199+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af8068160 con 0x7f3af815d350 2024-04-05T22:28:16.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.201+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f3af00147e0 con 0x7f3af815d350 2024-04-05T22:28:16.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.201+0000 7f3aeea00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 0x7f3ad406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.202+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f3af0089a80 con 0x7f3af815d350 2024-04-05T22:28:16.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.202+0000 7f3af6a00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 0x7f3ad406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.203+0000 7f3af6a00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 0x7f3ad406d4b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8036000 tx=0x7f3ae803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.206+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3af00545f0 con 0x7f3af815d350 2024-04-05T22:28:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.413+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3af8150de0 con 0x7f3af815d350 2024-04-05T22:28:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.413+0000 7f3aeea00640 1 -- 172.21.15.73:0/1668851213 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3af0019080 con 0x7f3af815d350 2024-04-05T22:28:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:28:16.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 msgr2=0x7f3ad406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 0x7f3ad406d4b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8036000 tx=0x7f3ae803d040 comp rx=0 tx=0).stop 2024-04-05T22:28:16.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 msgr2=0x7f3af80b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b990 tx=0x7f3af000be50 comp rx=0 tx=0).stop 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 shutdown_connections 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ad406aff0 0x7f3ad406d4b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3af815d350 0x7f3af80b4270 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3af815cf70 0x7f3af806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 --2- 172.21.15.73:0/1668851213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80d0bd0 0x7f3af806b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 >> 172.21.15.73:0/1668851213 conn(0x7f3af80d2df0 msgr2=0x7f3af815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 shutdown_connections 2024-04-05T22:28:16.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.416+0000 7f3afda00640 1 -- 172.21.15.73:0/1668851213 wait complete. 2024-04-05T22:28:16.432 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:28:16.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:16.433 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:28:16.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:28:16.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:16.554 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:28:16.610 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:16.610 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:16.650 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:16.650 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:16.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:16.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.963+0000 7f0e31e00640 1 Processor -- start 2024-04-05T22:28:16.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.963+0000 7f0e31e00640 1 -- start start 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.964+0000 7f0e31e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e2c15e620 0x7f0e2c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.964+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.964+0000 7f0e31e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c06b500 con 0x7f0e2c158860 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.964+0000 7f0e31e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c15eff0 con 0x7f0e2c15e620 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.964+0000 7f0e31e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0ccaf0 con 0x7f0e2c06bc70 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e2c15e620 0x7f0e2c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e2c15e620 0x7f0e2c15eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60592/0 (socket says 172.21.15.73:60592) 2024-04-05T22:28:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 -- 172.21.15.73:0/2454709270 learned_addr learned my addr 172.21.15.73:0/2454709270 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e31400640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 -- 172.21.15.73:0/2454709270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 msgr2=0x7f0e2c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c06c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 -- 172.21.15.73:0/2454709270 >> v1:172.21.15.157:6789/0 conn(0x7f0e2c158860 legacy=0x7f0e2c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.965+0000 7f0e2be00640 1 -- 172.21.15.73:0/2454709270 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0b20d0 con 0x7f0e2c15e620 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.966+0000 7f0e2be00640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15eab0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e50 tx=0x7f0e1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20883fff2053ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.966+0000 7f0e2b400640 1 -- 172.21.15.73:0/2454709270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402c9e0 con 0x7f0e2c15e620 2024-04-05T22:28:16.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.966+0000 7f0e2b400640 1 -- 172.21.15.73:0/2454709270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1402cb40 con 0x7f0e2c15e620 2024-04-05T22:28:16.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.966+0000 7f0e2b400640 1 -- 172.21.15.73:0/2454709270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140343f0 con 0x7f0e2c15e620 2024-04-05T22:28:16.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 -- 172.21.15.73:0/2454709270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c15e620 msgr2=0x7f0e2c15eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15eab0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e50 tx=0x7f0e1402be40 comp rx=0 tx=0).stop 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 -- 172.21.15.73:0/2454709270 shutdown_connections 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 --2- 172.21.15.73:0/2454709270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15eab0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 -- 172.21.15.73:0/2454709270 >> 172.21.15.73:0/2454709270 conn(0x7f0e2c0b06c0 msgr2=0x7f0e2c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 -- 172.21.15.73:0/2454709270 shutdown_connections 2024-04-05T22:28:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.967+0000 7f0e31e00640 1 -- 172.21.15.73:0/2454709270 wait complete. 2024-04-05T22:28:16.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.969+0000 7f0e31e00640 1 Processor -- start 2024-04-05T22:28:16.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.969+0000 7f0e31e00640 1 -- start start 2024-04-05T22:28:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15d560 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c150c50 con 0x7f0e2c15e620 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15c6b0 con 0x7f0e2c06bc70 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60602/0 (socket says 172.21.15.73:60602) 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.970+0000 7f0e31400640 1 -- 172.21.15.73:0/1115413697 learned_addr learned my addr 172.21.15.73:0/1115413697 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e30a00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e2be00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 -- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 msgr2=0x7f0e2c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c15d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 -- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2c15e620 msgr2=0x7f0e2c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 -- 172.21.15.73:0/1115413697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009b30 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e31400640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c550 tx=0x7f0e2400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004220 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24004380 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.971+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24003830 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.972+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c15c8e0 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.972+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c15cea0 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.973+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c15d6a0 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.974+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f0e24003990 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.974+0000 7f0e28c00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 0x7f0e0806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.975+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f0e24089850 con 0x7f0e2c0c88a0 2024-04-05T22:28:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.975+0000 7f0e30a00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 0x7f0e0806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.977+0000 7f0e30a00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 0x7f0e0806d4b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c003d90 tx=0x7f0e1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:16.979+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0e2c15d6a0 con 0x7f0e2c0c88a0 2024-04-05T22:28:17.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.170+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e2c15d6a0 con 0x7f0e2c0c88a0 2024-04-05T22:28:17.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.173+0000 7f0e28c00640 1 -- 172.21.15.73:0/1115413697 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16193 (secure 0 0 0) 0x7f0e2c15d6a0 con 0x7f0e2c0c88a0 2024-04-05T22:28:17.172 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:17.173 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:13.699726+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 msgr2=0x7f0e0806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 0x7f0e0806d4b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c003d90 tx=0x7f0e1c009040 comp rx=0 tx=0).stop 2024-04-05T22:28:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 msgr2=0x7f0e2c140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c550 tx=0x7f0e2400ca10 comp rx=0 tx=0).stop 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 shutdown_connections 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e0806aff0 0x7f0e0806d4b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c0c88a0 0x7f0e2c140220 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2c15e620 0x7f0e2c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 --2- 172.21.15.73:0/1115413697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e2c06bc70 0x7f0e2c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 >> 172.21.15.73:0/1115413697 conn(0x7f0e2c0b06c0 msgr2=0x7f0e2c0b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 shutdown_connections 2024-04-05T22:28:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.176+0000 7f0e31e00640 1 -- 172.21.15.73:0/1115413697 wait complete. 2024-04-05T22:28:17.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:27:30.588885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:27:31.104825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:28:17.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.405+0000 7f05b6400640 1 Processor -- start 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.406+0000 7f05b6400640 1 -- start start 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015a850 0x7f05b015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 0x7f05b006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05b015e9a0 0x7f05b0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b015f530 con 0x7f05b015e9a0 2024-04-05T22:28:17.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b006bc50 con 0x7f05b015e580 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.407+0000 7f05b6400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b015ef70 con 0x7f05b015a850 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05b015e9a0 0x7f05b0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05b015e9a0 0x7f05b0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60618/0 (socket says 172.21.15.73:60618) 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 -- 172.21.15.73:0/1346270153 learned_addr learned my addr 172.21.15.73:0/1346270153 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05afe00640 1 --2- 172.21.15.73:0/1346270153 >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 0x7f05b006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5000640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015a850 0x7f05b015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 -- 172.21.15.73:0/1346270153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015a850 msgr2=0x7f05b015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015a850 0x7f05b015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 -- 172.21.15.73:0/1346270153 >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 msgr2=0x7f05b006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 --2- 172.21.15.73:0/1346270153 >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 0x7f05b006ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05b5a00640 1 -- 172.21.15.73:0/1346270153 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b015f250 con 0x7f05b015e9a0 2024-04-05T22:28:17.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.408+0000 7f05afe00640 1 --2- 172.21.15.73:0/1346270153 >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 0x7f05b006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.409+0000 7f05b5a00640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015e9a0 0x7f05b0070470 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f05b00cf740 tx=0x7f05a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1896b0e53814edb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05af400640 1 -- 172.21.15.73:0/1346270153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8031870 con 0x7f05b015e9a0 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05af400640 1 -- 172.21.15.73:0/1346270153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a80319d0 con 0x7f05b015e9a0 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05af400640 1 -- 172.21.15.73:0/1346270153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8031d60 con 0x7f05b015e9a0 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 -- 172.21.15.73:0/1346270153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015e9a0 msgr2=0x7f05b0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015e9a0 0x7f05b0070470 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f05b00cf740 tx=0x7f05a802b910 comp rx=0 tx=0).stop 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 -- 172.21.15.73:0/1346270153 shutdown_connections 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015e9a0 0x7f05b0070470 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 --2- 172.21.15.73:0/1346270153 >> v2:172.21.15.157:3300/0 conn(0x7f05b015e580 0x7f05b006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 --2- 172.21.15.73:0/1346270153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015a850 0x7f05b015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 -- 172.21.15.73:0/1346270153 >> 172.21.15.73:0/1346270153 conn(0x7f05b0069cb0 msgr2=0x7f05b006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:17.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.410+0000 7f05b6400640 1 -- 172.21.15.73:0/1346270153 shutdown_connections 2024-04-05T22:28:17.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.411+0000 7f05b6400640 1 -- 172.21.15.73:0/1346270153 wait complete. 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.412+0000 7f05b6400640 1 Processor -- start 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.412+0000 7f05b6400640 1 -- start start 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015e580 0x7f05b015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b015e9a0 0x7f05b015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01528e0 con 0x7f05b015a850 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0158560 con 0x7f05b015e9a0 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.413+0000 7f05b6400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0158840 con 0x7f05b015e580 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60630/0 (socket says 172.21.15.73:60630) 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 -- 172.21.15.73:0/2449521147 learned_addr learned my addr 172.21.15.73:0/2449521147 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05afe00640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015e580 0x7f05b015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5a00640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b015e9a0 0x7f05b015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 -- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015e580 msgr2=0x7f05b015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015e580 0x7f05b015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 -- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b015e9a0 msgr2=0x7f05b015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b015e9a0 0x7f05b015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 -- 172.21.15.73:0/2449521147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80094d0 con 0x7f05b015a850 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.414+0000 7f05b5000640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f05a0007ef0 tx=0x7f05a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.415+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0014620 con 0x7f05b015a850 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.415+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a0014780 con 0x7f05b015a850 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.415+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0158b20 con 0x7f05b015a850 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.415+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0014b30 con 0x7f05b015a850 2024-04-05T22:28:17.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.415+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b01590e0 con 0x7f05b015a850 2024-04-05T22:28:17.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.416+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b004e160 con 0x7f05b015a850 2024-04-05T22:28:17.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.417+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f05a0014c90 con 0x7f05b015a850 2024-04-05T22:28:17.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.417+0000 7f05acc00640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 0x7f059006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.418+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f05a008a4e0 con 0x7f05b015a850 2024-04-05T22:28:17.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.418+0000 7f05afe00640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 0x7f059006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.419+0000 7f05afe00640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 0x7f059006d4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f05b00b0f70 tx=0x7f059c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.422+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f05a0055090 con 0x7f05b015a850 2024-04-05T22:28:17.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.618+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05b00b1f80 con 0x7f05b015a850 2024-04-05T22:28:17.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.619+0000 7f05acc00640 1 -- 172.21.15.73:0/2449521147 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05a00586b0 con 0x7f05b015a850 2024-04-05T22:28:17.618 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:28:17.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.621+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 msgr2=0x7f059006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.621+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 0x7f059006d4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f05b00b0f70 tx=0x7f059c008040 comp rx=0 tx=0).stop 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.621+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 msgr2=0x7f05b01523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.621+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f05a0007ef0 tx=0x7f05a000e4b0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 shutdown_connections 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f059006aff0 0x7f059006d4b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b015e9a0 0x7f05b015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05b015e580 0x7f05b015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 --2- 172.21.15.73:0/2449521147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05b015a850 0x7f05b01523a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 >> 172.21.15.73:0/2449521147 conn(0x7f05b0069cb0 msgr2=0x7f05b00b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 shutdown_connections 2024-04-05T22:28:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.622+0000 7f05b6400640 1 -- 172.21.15.73:0/2449521147 wait complete. 2024-04-05T22:28:17.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:28:17.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.852+0000 7f806ac00640 1 Processor -- start 2024-04-05T22:28:17.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.852+0000 7f806ac00640 1 -- start start 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f806ac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f806415abe0 0x7f80640712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f806ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8064164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f806ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806415afc0 con 0x7f806406c9a0 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f806ac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806406f5b0 con 0x7f806415abe0 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f806ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806406f890 con 0x7f806406bc40 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f8063400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f806406c9a0 0x7f806406b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47226/0 (socket says 172.21.15.73:47226) 2024-04-05T22:28:17.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f8063400640 1 -- 172.21.15.73:0/3413259210 learned_addr learned my addr 172.21.15.73:0/3413259210 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f8068c00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8064164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.854+0000 7f8063e00640 1 --2- 172.21.15.73:0/3413259210 >> v2:172.21.15.157:3300/0 conn(0x7f806415abe0 0x7f80640712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274311373 0 0) 0x7f806415afc0 con 0x7f806406c9a0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8048003520 con 0x7f806406c9a0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613877720 0 0) 0x7f8048003520 con 0x7f806406c9a0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 msgr2=0x7f8064164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8064164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 >> v2:172.21.15.157:3300/0 conn(0x7f806415abe0 msgr2=0x7f80640712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 --2- 172.21.15.73:0/3413259210 >> v2:172.21.15.157:3300/0 conn(0x7f806415abe0 0x7f80640712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806406fb70 con 0x7f806406c9a0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8050001670 con 0x7f806406c9a0 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 >> v1:172.21.15.73:6789/0 conn(0x7f806406c9a0 legacy=0x7f806406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8048003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80480047c0 0x7f8048004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8048005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806406f5b0 con 0x7f8048003970 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806406f890 con 0x7f80480047c0 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804800ba30 con 0x7f806406bc40 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.855+0000 7f8063e00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8048003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063400640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80480047c0 0x7f8048004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8068c00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8048005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063e00640 1 -- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 msgr2=0x7f8048005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063e00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f806406bc40 0x7f8048005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063e00640 1 -- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80480047c0 msgr2=0x7f8048004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063e00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80480047c0 0x7f8048004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063e00640 1 -- 172.21.15.73:0/3413259210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048003520 con 0x7f8048003970 2024-04-05T22:28:17.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.856+0000 7f8063400640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80480047c0 0x7f8048004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.857+0000 7f8063e00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8048003d90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ac0 tx=0x7f80480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6f4e17cb24a5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805402c710 con 0x7f8048003970 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805402c870 con 0x7f8048003970 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f8062a00640 1 -- 172.21.15.73:0/3413259210 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805402cc00 con 0x7f8048003970 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f806ac00640 1 -- 172.21.15.73:0/3413259210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 msgr2=0x7f8048003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f806ac00640 1 --2- 172.21.15.73:0/3413259210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8048003d90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ac0 tx=0x7f80480057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f8068c00640 1 -- 172.21.15.73:0/3413259210 reap_dead start 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f806ac00640 1 -- 172.21.15.73:0/3413259210 shutdown_connections 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f806ac00640 1 -- 172.21.15.73:0/3413259210 >> 172.21.15.73:0/3413259210 conn(0x7f8064069cb0 msgr2=0x7f806406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.858+0000 7f806ac00640 1 -- 172.21.15.73:0/3413259210 shutdown_connections 2024-04-05T22:28:17.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.859+0000 7f806ac00640 1 -- 172.21.15.73:0/3413259210 wait complete. 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.860+0000 7f806ac00640 1 Processor -- start 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.860+0000 7f806ac00640 1 -- start start 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8064144d60 0x7f806413a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80641f2600 0x7f80641f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8064146c60 con 0x7f8048003970 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806413a860 con 0x7f80641f2600 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.861+0000 7f806ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8064146eb0 con 0x7f8064144d60 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60652/0 (socket says 172.21.15.73:60652) 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 -- 172.21.15.73:0/2281426506 learned_addr learned my addr 172.21.15.73:0/2281426506 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063400640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8064144d60 0x7f806413a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8068c00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80641f2600 0x7f80641f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 -- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8064144d60 msgr2=0x7f806413a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8064144d60 0x7f806413a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 -- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80641f2600 msgr2=0x7f80641f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80641f2600 0x7f80641f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 -- 172.21.15.73:0/2281426506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8064147190 con 0x7f8048003970 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.862+0000 7f8063e00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f805402c2b0 tx=0x7f8054004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.863+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054003810 con 0x7f8048003970 2024-04-05T22:28:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.863+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8054003970 con 0x7f8048003970 2024-04-05T22:28:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.863+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054003d00 con 0x7f8048003970 2024-04-05T22:28:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.863+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8064147470 con 0x7f8048003970 2024-04-05T22:28:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.863+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8064147a30 con 0x7f8048003970 2024-04-05T22:28:17.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.864+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8064147bf0 con 0x7f8048003970 2024-04-05T22:28:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.865+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f8054040450 con 0x7f8048003970 2024-04-05T22:28:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.866+0000 7f805be00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 0x7f804406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.866+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f80540aba10 con 0x7f8048003970 2024-04-05T22:28:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.866+0000 7f8063400640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 0x7f804406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:17.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.867+0000 7f8063400640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 0x7f804406d4e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8050001200 tx=0x7f8050009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:17.870+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8054076580 con 0x7f8048003970 2024-04-05T22:28:18.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.062+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8064068160 con 0x7f8048003970 2024-04-05T22:28:18.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.063+0000 7f805be00640 1 -- 172.21.15.73:0/2281426506 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8054003e60 con 0x7f8048003970 2024-04-05T22:28:18.062 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 msgr2=0x7f804406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 0x7f804406d4e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8050001200 tx=0x7f8050009040 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 msgr2=0x7f8064148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f805402c2b0 tx=0x7f8054004730 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 shutdown_connections 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f804406b020 0x7f804406d4e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80641f2600 0x7f80641f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8064144d60 0x7f806413a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 --2- 172.21.15.73:0/2281426506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8048003970 0x7f8064148c20 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.065+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 >> 172.21.15.73:0/2281426506 conn(0x7f8064069cb0 msgr2=0x7f80640cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:18.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.066+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 shutdown_connections 2024-04-05T22:28:18.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.066+0000 7f806ac00640 1 -- 172.21.15.73:0/2281426506 wait complete. 2024-04-05T22:28:18.081 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:28:18.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.290+0000 7f3112c00640 1 Processor -- start 2024-04-05T22:28:18.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.290+0000 7f3112c00640 1 -- start start 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f3112c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f310c0b32b0 0x7f310c1657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f3112c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c0b1b30 con 0x7f310c06c630 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f3112c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c0b1e10 con 0x7f310c0cbe00 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f3112c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c0ce7b0 con 0x7f310c0b32b0 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f310be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f310c06c630 0x7f310c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47240/0 (socket says 172.21.15.73:47240) 2024-04-05T22:28:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f310be00640 1 -- 172.21.15.73:0/4163908711 learned_addr learned my addr 172.21.15.73:0/4163908711 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f3110c00640 1 --2- 172.21.15.73:0/4163908711 >> v2:172.21.15.157:3300/0 conn(0x7f310c0b32b0 0x7f310c1657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.292+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493750410 0 0) 0x7f310c0b1b30 con 0x7f310c06c630 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f0003520 con 0x7f310c06c630 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062402330 0 0) 0x7f30f0003520 con 0x7f310c06c630 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 >> v2:172.21.15.157:3300/0 conn(0x7f310c0b32b0 msgr2=0x7f310c1657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 --2- 172.21.15.73:0/4163908711 >> v2:172.21.15.157:3300/0 conn(0x7f310c0b32b0 0x7f310c1657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 >> v1:172.21.15.157:6789/0 conn(0x7f310c0cbe00 legacy=0x7f310c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0cc300 con 0x7f310c06c630 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f30fc001670 con 0x7f310c06c630 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f3110c00640 1 --2- 172.21.15.73:0/4163908711 >> v2:172.21.15.157:3300/0 conn(0x7f310c0b32b0 0x7f310c1657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 >> v1:172.21.15.73:6789/0 conn(0x7f310c06c630 legacy=0x7f310c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f0003970 0x7f30f0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 0x7f30f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30f0005180 0x7f30f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0ce7b0 con 0x7f30f00047c0 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f000c0b0 con 0x7f30f0003970 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f000c270 con 0x7f30f0005180 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.293+0000 7f310b400640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 0x7f30f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310b400640 1 -- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30f0005180 msgr2=0x7f30f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310b400640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30f0005180 0x7f30f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310b400640 1 -- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f0003970 msgr2=0x7f30f0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310be00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f0003970 0x7f30f0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310b400640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f0003970 0x7f30f0003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f310b400640 1 -- 172.21.15.73:0/4163908711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0003520 con 0x7f30f00047c0 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.294+0000 7f3110c00640 1 -- 172.21.15.73:0/4163908711 reap_dead start 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f310b400640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 0x7f30f0004c40 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f30f8003d80 tx=0x7f30f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b138d38b4eee1a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8032600 con 0x7f30f00047c0 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f802e050 con 0x7f30f00047c0 2024-04-05T22:28:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f310aa00640 1 -- 172.21.15.73:0/4163908711 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f80329e0 con 0x7f30f00047c0 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f3112c00640 1 -- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 msgr2=0x7f30f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.295+0000 7f3112c00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 0x7f30f0004c40 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f30f8003d80 tx=0x7f30f00057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.296+0000 7f3112c00640 1 -- 172.21.15.73:0/4163908711 shutdown_connections 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.296+0000 7f3112c00640 1 --2- 172.21.15.73:0/4163908711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30f00047c0 0x7f30f0004c40 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.296+0000 7f3112c00640 1 -- 172.21.15.73:0/4163908711 >> 172.21.15.73:0/4163908711 conn(0x7f310c0b0830 msgr2=0x7f310c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.296+0000 7f3112c00640 1 -- 172.21.15.73:0/4163908711 shutdown_connections 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.296+0000 7f3112c00640 1 -- 172.21.15.73:0/4163908711 wait complete. 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.297+0000 7f3112c00640 1 Processor -- start 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.297+0000 7f3112c00640 1 -- start start 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f00047c0 0x7f310c1f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f310c1372a0 0x7f310c0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c15add0 con 0x7f310c136e20 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1f2ee0 con 0x7f30f00047c0 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.298+0000 7f3112c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1f30f0 con 0x7f310c1372a0 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60664/0 (socket says 172.21.15.73:60664) 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 -- 172.21.15.73:0/1834384727 learned_addr learned my addr 172.21.15.73:0/1834384727 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310be00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f00047c0 0x7f310c1f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 -- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f310c1372a0 msgr2=0x7f310c0c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f310c1372a0 0x7f310c0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 -- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f00047c0 msgr2=0x7f310c1f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f00047c0 0x7f310c1f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 -- 172.21.15.73:0/1834384727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c1f33d0 con 0x7f310c136e20 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.299+0000 7f310b400640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035040 tx=0x7f30f80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.300+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f803c410 con 0x7f310c136e20 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.300+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f803c570 con 0x7f310c136e20 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.300+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f803c8e0 con 0x7f310c136e20 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.300+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c1f36b0 con 0x7f310c136e20 2024-04-05T22:28:18.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.300+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c1f3c70 con 0x7f310c136e20 2024-04-05T22:28:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.301+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c068160 con 0x7f310c136e20 2024-04-05T22:28:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.302+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f30f8039050 con 0x7f310c136e20 2024-04-05T22:28:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.303+0000 7f3103e00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 0x7f30ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.303+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f30f80ab3c0 con 0x7f310c136e20 2024-04-05T22:28:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.303+0000 7f310be00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 0x7f30ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.305+0000 7f310be00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 0x7f30ec06d4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30fc001200 tx=0x7f30fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.307+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f30f8075e20 con 0x7f310c136e20 2024-04-05T22:28:18.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.498+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f310c0adff0 con 0x7f310c136e20 2024-04-05T22:28:18.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.499+0000 7f3103e00640 1 -- 172.21.15.73:0/1834384727 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30f80370a0 con 0x7f310c136e20 2024-04-05T22:28:18.498 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 msgr2=0x7f30ec06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 0x7f30ec06d4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30fc001200 tx=0x7f30fc009040 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 msgr2=0x7f310c0c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035040 tx=0x7f30f80036f0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 shutdown_connections 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f30ec06b020 0x7f30ec06d4e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f310c1372a0 0x7f310c0c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f310c136e20 0x7f310c0c4550 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 --2- 172.21.15.73:0/1834384727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30f00047c0 0x7f310c1f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 >> 172.21.15.73:0/1834384727 conn(0x7f310c0b0830 msgr2=0x7f310c0ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:18.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 shutdown_connections 2024-04-05T22:28:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.501+0000 7f3112c00640 1 -- 172.21.15.73:0/1834384727 wait complete. 2024-04-05T22:28:18.517 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:28:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.712+0000 7f9ecf000640 1 Processor -- start 2024-04-05T22:28:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.712+0000 7f9ecf000640 1 -- start start 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.713+0000 7f9ecf000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ec81587d0 0x7f9ec80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.713+0000 7f9ecf000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec806f5b0 con 0x7f9ec815e850 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.713+0000 7f9ecf000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec8070410 con 0x7f9ec80ccd20 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.713+0000 7f9ecf000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec8158bb0 con 0x7f9ec81587d0 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.714+0000 7f9ecd000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9ec815e850 0x7f9ec80708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47244/0 (socket says 172.21.15.73:47244) 2024-04-05T22:28:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.714+0000 7f9ecd000640 1 -- 172.21.15.73:0/788366817 learned_addr learned my addr 172.21.15.73:0/788366817 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.714+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/788366817 >> v2:172.21.15.157:3300/0 conn(0x7f9ec81587d0 0x7f9ec80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.714+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934863754 0 0) 0x7f9ec806f5b0 con 0x7f9ec815e850 2024-04-05T22:28:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.714+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eac003520 con 0x7f9ec815e850 2024-04-05T22:28:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609898802 0 0) 0x7f9eac003520 con 0x7f9ec815e850 2024-04-05T22:28:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 >> v2:172.21.15.157:3300/0 conn(0x7f9ec81587d0 msgr2=0x7f9ec80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 --2- 172.21.15.73:0/788366817 >> v2:172.21.15.157:3300/0 conn(0x7f9ec81587d0 0x7f9ec80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 >> v1:172.21.15.157:6789/0 conn(0x7f9ec80ccd20 legacy=0x7f9ec806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec806f800 con 0x7f9ec815e850 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9ec00019f0 con 0x7f9ec815e850 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 >> v1:172.21.15.73:6789/0 conn(0x7f9ec815e850 legacy=0x7f9ec80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9eac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eac0047c0 0x7f9eac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac005180 0x7f9eac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8158bb0 con 0x7f9eac003970 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac00c020 con 0x7f9eac005180 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac00c1e0 con 0x7f9eac0047c0 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9eac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ecd000640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac005180 0x7f9eac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 -- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eac0047c0 msgr2=0x7f9eac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eac0047c0 0x7f9eac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 -- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac005180 msgr2=0x7f9eac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac005180 0x7f9eac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ec7e00640 1 -- 172.21.15.73:0/788366817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac003520 con 0x7f9eac003970 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ecd000640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac005180 0x7f9eac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.715+0000 7f9ecd000640 1 -- 172.21.15.73:0/788366817 reap_dead start 2024-04-05T22:28:18.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9eac003d90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009ac0 tx=0x7f9eac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38ddd63ff4ef046 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb802c6b0 con 0x7f9eac003970 2024-04-05T22:28:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb802c810 con 0x7f9eac003970 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ec6a00640 1 -- 172.21.15.73:0/788366817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb802cba0 con 0x7f9eac003970 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 -- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 msgr2=0x7f9eac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9eac003d90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009ac0 tx=0x7f9eac0057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 -- 172.21.15.73:0/788366817 shutdown_connections 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 --2- 172.21.15.73:0/788366817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9eac003d90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 -- 172.21.15.73:0/788366817 >> 172.21.15.73:0/788366817 conn(0x7f9ec80ce8e0 msgr2=0x7f9ec80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 -- 172.21.15.73:0/788366817 shutdown_connections 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.717+0000 7f9ecf000640 1 -- 172.21.15.73:0/788366817 wait complete. 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.719+0000 7f9ecf000640 1 Processor -- start 2024-04-05T22:28:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.719+0000 7f9ecf000640 1 -- start start 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec80b2a40 0x7f9ec80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec80b35d0 0x7f9ec81f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80b2620 con 0x7f9eac003970 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80b19a0 con 0x7f9ec80b2a40 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ecf000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80b1c80 con 0x7f9ec80b35d0 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ec7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ec7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60680/0 (socket says 172.21.15.73:60680) 2024-04-05T22:28:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.720+0000 7f9ec7e00640 1 -- 172.21.15.73:0/453343627 learned_addr learned my addr 172.21.15.73:0/453343627 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 -- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec80b35d0 msgr2=0x7f9ec81f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec80b35d0 0x7f9ec81f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 -- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec80b2a40 msgr2=0x7f9ec80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec80b2a40 0x7f9ec80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 -- 172.21.15.73:0/453343627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80b1f60 con 0x7f9eac003970 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ec7e00640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9eb802c250 tx=0x7f9eb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8003810 con 0x7f9eac003970 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8003970 con 0x7f9eac003970 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec80b2240 con 0x7f9eac003970 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8003d00 con 0x7f9eac003970 2024-04-05T22:28:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.721+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec80aec90 con 0x7f9eac003970 2024-04-05T22:28:18.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.723+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec8066af0 con 0x7f9eac003970 2024-04-05T22:28:18.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.724+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f9eb8047450 con 0x7f9eac003970 2024-04-05T22:28:18.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.724+0000 7f9ebfe00640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 0x7f9ea806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:18.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.725+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f9eb80af120 con 0x7f9eac003970 2024-04-05T22:28:18.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.725+0000 7f9ec7400640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 0x7f9ea806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:18.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.727+0000 7f9ec7400640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 0x7f9ea806d4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8143e40 tx=0x7f9eb40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:18.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.730+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9eb8076c00 con 0x7f9eac003970 2024-04-05T22:28:18.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.933+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ec8068160 con 0x7f9eac003970 2024-04-05T22:28:18.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.934+0000 7f9ebfe00640 1 -- 172.21.15.73:0/453343627 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9eb804c080 con 0x7f9eac003970 2024-04-05T22:28:18.933 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:28:18.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.936+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 msgr2=0x7f9ea806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.936+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 0x7f9ea806d4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8143e40 tx=0x7f9eb40079b0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.936+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 msgr2=0x7f9ec80ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.936+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9eb802c250 tx=0x7f9eb8004730 comp rx=0 tx=0).stop 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 shutdown_connections 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ea806aff0 0x7f9ea806d4b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ec80b35d0 0x7f9ec81f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ec80b2a40 0x7f9ec80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 --2- 172.21.15.73:0/453343627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac003970 0x7f9ec80ae750 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 >> 172.21.15.73:0/453343627 conn(0x7f9ec80ce8e0 msgr2=0x7f9ec8142cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 shutdown_connections 2024-04-05T22:28:18.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:18.937+0000 7f9ecf000640 1 -- 172.21.15.73:0/453343627 wait complete. 2024-04-05T22:28:18.953 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:28:18.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:18.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:28:19.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:28:19.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:19.068 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:28:19.119 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:19.120 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:19.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:28:19.173 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:19.173 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:19.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:28:19.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.442+0000 7fca9b200640 1 Processor -- start 2024-04-05T22:28:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.443+0000 7fca9b200640 1 -- start start 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.444+0000 7fca9b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca941587d0 0x7fca940cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca9b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9406f5b0 con 0x7fca940ccd20 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca9b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca94070410 con 0x7fca9415e850 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca9b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca94158bb0 con 0x7fca941587d0 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca93400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fca940ccd20 0x7fca9406cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47256/0 (socket says 172.21.15.73:47256) 2024-04-05T22:28:19.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca93400640 1 -- 172.21.15.73:0/3551160448 learned_addr learned my addr 172.21.15.73:0/3551160448 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca93e00640 1 --2- 172.21.15.73:0/3551160448 >> v2:172.21.15.157:3300/0 conn(0x7fca941587d0 0x7fca940cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206235016 0 0) 0x7fca9406f5b0 con 0x7fca940ccd20 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.445+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca78003520 con 0x7fca940ccd20 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234841464 0 0) 0x7fca78003520 con 0x7fca940ccd20 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 >> v2:172.21.15.157:3300/0 conn(0x7fca941587d0 msgr2=0x7fca940cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 --2- 172.21.15.73:0/3551160448 >> v2:172.21.15.157:3300/0 conn(0x7fca941587d0 0x7fca940cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 >> v1:172.21.15.157:6789/0 conn(0x7fca9415e850 legacy=0x7fca940708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9406f800 con 0x7fca940ccd20 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fca800019f0 con 0x7fca940ccd20 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 >> v1:172.21.15.73:6789/0 conn(0x7fca940ccd20 legacy=0x7fca9406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 0x7fca78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780047c0 0x7fca78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78005180 0x7fca7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca94158bb0 con 0x7fca78003970 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800c020 con 0x7fca78005180 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800c1e0 con 0x7fca780047c0 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.446+0000 7fca93e00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 0x7fca78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca93e00640 1 -- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780047c0 msgr2=0x7fca78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca93e00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780047c0 0x7fca78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca93e00640 1 -- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78005180 msgr2=0x7fca7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca99200640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78005180 0x7fca7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca93e00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78005180 0x7fca7800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca93e00640 1 -- 172.21.15.73:0/3551160448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78003520 con 0x7fca78003970 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca99200640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78005180 0x7fca7800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.447+0000 7fca99200640 1 -- 172.21.15.73:0/3551160448 reap_dead start 2024-04-05T22:28:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca93e00640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 0x7fca78003d90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fca84009ac0 tx=0x7fca780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b3961553b8cbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8402c6b0 con 0x7fca78003970 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8402c810 con 0x7fca78003970 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca92a00640 1 -- 172.21.15.73:0/3551160448 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8402cba0 con 0x7fca78003970 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca9b200640 1 -- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 msgr2=0x7fca78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.448+0000 7fca9b200640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 0x7fca78003d90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fca84009ac0 tx=0x7fca780057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.449+0000 7fca9b200640 1 -- 172.21.15.73:0/3551160448 shutdown_connections 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.449+0000 7fca9b200640 1 --2- 172.21.15.73:0/3551160448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca78003970 0x7fca78003d90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.449+0000 7fca9b200640 1 -- 172.21.15.73:0/3551160448 >> 172.21.15.73:0/3551160448 conn(0x7fca940ce8e0 msgr2=0x7fca940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.449+0000 7fca9b200640 1 -- 172.21.15.73:0/3551160448 shutdown_connections 2024-04-05T22:28:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.449+0000 7fca9b200640 1 -- 172.21.15.73:0/3551160448 wait complete. 2024-04-05T22:28:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.450+0000 7fca9b200640 1 Processor -- start 2024-04-05T22:28:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 -- start start 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca78003970 0x7fca94157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca940ae4e0 0x7fca940ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415acb0 con 0x7fca94159f80 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca940aeea0 con 0x7fca940ae4e0 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.451+0000 7fca9b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca940b17a0 con 0x7fca78003970 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60708/0 (socket says 172.21.15.73:60708) 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 -- 172.21.15.73:0/928692110 learned_addr learned my addr 172.21.15.73:0/928692110 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca93e00640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca78003970 0x7fca94157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca93400640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca940ae4e0 0x7fca940ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 -- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca78003970 msgr2=0x7fca94157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca78003970 0x7fca94157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 -- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca940ae4e0 msgr2=0x7fca940ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca940ae4e0 0x7fca940ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 -- 172.21.15.73:0/928692110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940b1a40 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.452+0000 7fca99200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fca9415c270 tx=0x7fca8c00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.453+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00b070 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.453+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c0052d0 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.453+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0035f0 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.453+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca940b1d20 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.453+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9415a940 con 0x7fca94159f80 2024-04-05T22:28:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.454+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca94066af0 con 0x7fca94159f80 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.455+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fca8c003750 con 0x7fca94159f80 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.456+0000 7fca8be00640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 0x7fca7406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.456+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fca8c089150 con 0x7fca94159f80 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.456+0000 7fca93e00640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 0x7fca7406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.457+0000 7fca93e00640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 0x7fca7406d4b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca84036000 tx=0x7fca84030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.460+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fca8c053bf0 con 0x7fca94159f80 2024-04-05T22:28:19.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.700+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca940b4b40 con 0x7fca94159f80 2024-04-05T22:28:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.701+0000 7fca8be00640 1 -- 172.21.15.73:0/928692110 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fca8c057210 con 0x7fca94159f80 2024-04-05T22:28:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:19.700 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:28:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.703+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 msgr2=0x7fca7406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.703+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 0x7fca7406d4b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca84036000 tx=0x7fca84030000 comp rx=0 tx=0).stop 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.703+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 msgr2=0x7fca9415a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.703+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fca9415c270 tx=0x7fca8c00d5e0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 shutdown_connections 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca7406aff0 0x7fca7406d4b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca94159f80 0x7fca9415a400 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca940ae4e0 0x7fca940ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 --2- 172.21.15.73:0/928692110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca78003970 0x7fca94157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 >> 172.21.15.73:0/928692110 conn(0x7fca940ce8e0 msgr2=0x7fca9406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 shutdown_connections 2024-04-05T22:28:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.704+0000 7fca9b200640 1 -- 172.21.15.73:0/928692110 wait complete. 2024-04-05T22:28:19.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:19.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.931+0000 7fdd4b200640 1 Processor -- start 2024-04-05T22:28:19.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.931+0000 7fdd4b200640 1 -- start start 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd4b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd440cb2f0 0x7fdd440cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd4b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440cbc70 con 0x7fdd440b1870 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd4b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440ad880 con 0x7fdd44160620 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd4b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd440ce5d0 con 0x7fdd440cb2f0 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd49200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd440cb2f0 0x7fdd440cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd49200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd440cb2f0 0x7fdd440cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60710/0 (socket says 172.21.15.73:60710) 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd49200640 1 -- 172.21.15.73:0/3118512976 learned_addr learned my addr 172.21.15.73:0/3118512976 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.933+0000 7fdd43400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdd440b1870 0x7fdd440ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47264/0 (socket says 172.21.15.73:47264) 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.934+0000 7fdd49200640 1 -- 172.21.15.73:0/3118512976 >> v1:172.21.15.157:6789/0 conn(0x7fdd44160620 legacy=0x7fdd440caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.934+0000 7fdd49200640 1 -- 172.21.15.73:0/3118512976 >> v1:172.21.15.73:6789/0 conn(0x7fdd440b1870 legacy=0x7fdd440ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.934+0000 7fdd49200640 1 -- 172.21.15.73:0/3118512976 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440ce010 con 0x7fdd440cb2f0 2024-04-05T22:28:19.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.934+0000 7fdd42a00640 1 -- 172.21.15.73:0/3118512976 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567779931 0 0) 0x7fdd440cbc70 con 0x7fdd440b1870 2024-04-05T22:28:19.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.935+0000 7fdd49200640 1 --2- 172.21.15.73:0/3118512976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440cb2f0 0x7fdd440cb730 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009e50 tx=0x7fdd3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9796c865fa59188 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.935+0000 7fdd42a00640 1 -- 172.21.15.73:0/3118512976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c02cb20 con 0x7fdd440cb2f0 2024-04-05T22:28:19.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.935+0000 7fdd42a00640 1 -- 172.21.15.73:0/3118512976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c02cc80 con 0x7fdd440cb2f0 2024-04-05T22:28:19.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.935+0000 7fdd42a00640 1 -- 172.21.15.73:0/3118512976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0343f0 con 0x7fdd440cb2f0 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 -- 172.21.15.73:0/3118512976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440cb2f0 msgr2=0x7fdd440cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 --2- 172.21.15.73:0/3118512976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440cb2f0 0x7fdd440cb730 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009e50 tx=0x7fdd3c02bf80 comp rx=0 tx=0).stop 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 -- 172.21.15.73:0/3118512976 shutdown_connections 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 --2- 172.21.15.73:0/3118512976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440cb2f0 0x7fdd440cb730 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 -- 172.21.15.73:0/3118512976 >> 172.21.15.73:0/3118512976 conn(0x7fdd440ce8e0 msgr2=0x7fdd440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:19.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 -- 172.21.15.73:0/3118512976 shutdown_connections 2024-04-05T22:28:19.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.936+0000 7fdd4b200640 1 -- 172.21.15.73:0/3118512976 wait complete. 2024-04-05T22:28:19.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.938+0000 7fdd4b200640 1 Processor -- start 2024-04-05T22:28:19.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.938+0000 7fdd4b200640 1 -- start start 2024-04-05T22:28:19.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd440cb2f0 0x7fdd4406f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd44160620 0x7fdd4406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441f6290 con 0x7fdd440b1870 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441f5980 con 0x7fdd44160620 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd4b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440b0050 con 0x7fdd440cb2f0 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60726/0 (socket says 172.21.15.73:60726) 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 -- 172.21.15.73:0/1574225682 learned_addr learned my addr 172.21.15.73:0/1574225682 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 -- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd440cb2f0 msgr2=0x7fdd4406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd440cb2f0 0x7fdd4406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 -- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd44160620 msgr2=0x7fdd4406fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd44160620 0x7fdd4406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.939+0000 7fdd43e00640 1 -- 172.21.15.73:0/1574225682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009b30 con 0x7fdd440b1870 2024-04-05T22:28:19.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd43e00640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b3f0 tx=0x7fdd3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34011830 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34011990 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34011d40 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd441f5bd0 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.940+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440c5b20 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.941+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44066af0 con 0x7fdd440b1870 2024-04-05T22:28:19.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.942+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fdd3401a400 con 0x7fdd440b1870 2024-04-05T22:28:19.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.943+0000 7fdd3be00640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 0x7fdd2406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:19.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.943+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fdd340898e0 con 0x7fdd440b1870 2024-04-05T22:28:19.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.944+0000 7fdd43400640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 0x7fdd2406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:19.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.945+0000 7fdd43400640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 0x7fdd2406d4b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd30007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:19.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:19.947+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdd340544e0 con 0x7fdd440b1870 2024-04-05T22:28:20.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.144+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd441f63d0 con 0x7fdd440b1870 2024-04-05T22:28:20.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.146+0000 7fdd3be00640 1 -- 172.21.15.73:0/1574225682 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16193 (secure 0 0 0) 0x7fdd34057b00 con 0x7fdd440b1870 2024-04-05T22:28:20.145 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:20.146 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:13.699726+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 msgr2=0x7fdd2406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 0x7fdd2406d4b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd30007920 comp rx=0 tx=0).stop 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 msgr2=0x7fdd440c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b3f0 tx=0x7fdd3400b8b0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 shutdown_connections 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd2406aff0 0x7fdd2406d4b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd44160620 0x7fdd4406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd440cb2f0 0x7fdd4406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 --2- 172.21.15.73:0/1574225682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd440b1870 0x7fdd440c55e0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 >> 172.21.15.73:0/1574225682 conn(0x7fdd440ce8e0 msgr2=0x7fdd440caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 shutdown_connections 2024-04-05T22:28:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.149+0000 7fdd4b200640 1 -- 172.21.15.73:0/1574225682 wait complete. 2024-04-05T22:28:20.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:27:30.588885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:27:31.104825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:28:20.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:28:20.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.376+0000 7fcd8a200640 1 Processor -- start 2024-04-05T22:28:20.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.376+0000 7fcd8a200640 1 -- start start 2024-04-05T22:28:20.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd8a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd8415a2a0 0x7fcd840abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd8a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd840cefa0 con 0x7fcd84159ec0 2024-04-05T22:28:20.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd8a200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd840ce9e0 con 0x7fcd8406d7c0 2024-04-05T22:28:20.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd8a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8406b690 con 0x7fcd8415a2a0 2024-04-05T22:28:20.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd83400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcd84159ec0 0x7fcd8406aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47280/0 (socket says 172.21.15.73:47280) 2024-04-05T22:28:20.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.378+0000 7fcd83400640 1 -- 172.21.15.73:0/2678610772 learned_addr learned my addr 172.21.15.73:0/2678610772 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2678610772 >> v2:172.21.15.157:3300/0 conn(0x7fcd8415a2a0 0x7fcd840abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501603115 0 0) 0x7fcd840cefa0 con 0x7fcd84159ec0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd68003520 con 0x7fcd84159ec0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458333792 0 0) 0x7fcd68003520 con 0x7fcd84159ec0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 >> v2:172.21.15.157:3300/0 conn(0x7fcd8415a2a0 msgr2=0x7fcd840abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 --2- 172.21.15.73:0/2678610772 >> v2:172.21.15.157:3300/0 conn(0x7fcd8415a2a0 0x7fcd840abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 >> v1:172.21.15.157:6789/0 conn(0x7fcd8406d7c0 legacy=0x7fcd84069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840cecc0 con 0x7fcd84159ec0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fcd74001670 con 0x7fcd84159ec0 2024-04-05T22:28:20.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 >> v1:172.21.15.73:6789/0 conn(0x7fcd84159ec0 legacy=0x7fcd8406aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd68003970 0x7fcd68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 0x7fcd68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd68005180 0x7fcd6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8406b690 con 0x7fcd680047c0 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6800c0d0 con 0x7fcd68003970 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.379+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6800e4a0 con 0x7fcd68005180 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 0x7fcd68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd83400640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd68003970 0x7fcd68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd83e00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd68005180 0x7fcd6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 -- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd68005180 msgr2=0x7fcd6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd68005180 0x7fcd6800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 -- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd68003970 msgr2=0x7fcd68003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd68003970 0x7fcd68003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd7be00640 1 -- 172.21.15.73:0/2678610772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68003520 con 0x7fcd680047c0 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd83e00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd68005180 0x7fcd6800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:20.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.380+0000 7fcd83e00640 1 -- 172.21.15.73:0/2678610772 reap_dead start 2024-04-05T22:28:20.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.381+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 0x7fcd68004c40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ba00 tx=0x7fcd680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67075c628cc6aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.381+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c032620 con 0x7fcd680047c0 2024-04-05T22:28:20.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.381+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c032780 con 0x7fcd680047c0 2024-04-05T22:28:20.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.381+0000 7fcd82a00640 1 -- 172.21.15.73:0/2678610772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c032b10 con 0x7fcd680047c0 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 -- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 msgr2=0x7fcd68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 0x7fcd68004c40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ba00 tx=0x7fcd680057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 -- 172.21.15.73:0/2678610772 shutdown_connections 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2678610772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680047c0 0x7fcd68004c40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 -- 172.21.15.73:0/2678610772 >> 172.21.15.73:0/2678610772 conn(0x7fcd840b0ce0 msgr2=0x7fcd840b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 -- 172.21.15.73:0/2678610772 shutdown_connections 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.382+0000 7fcd8a200640 1 -- 172.21.15.73:0/2678610772 wait complete. 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.384+0000 7fcd8a200640 1 Processor -- start 2024-04-05T22:28:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.384+0000 7fcd8a200640 1 -- start start 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd680047c0 0x7fcd8414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8414f0a0 0x7fcd840b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840ca1a0 con 0x7fcd841560e0 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8406fe80 con 0x7fcd8414f0a0 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd8a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840703d0 con 0x7fcd680047c0 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60756/0 (socket says 172.21.15.73:60756) 2024-04-05T22:28:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 -- 172.21.15.73:0/2839964516 learned_addr learned my addr 172.21.15.73:0/2839964516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 -- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd680047c0 msgr2=0x7fcd8414f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd680047c0 0x7fcd8414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 -- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8414f0a0 msgr2=0x7fcd840b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8414f0a0 0x7fcd840b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.385+0000 7fcd7be00640 1 -- 172.21.15.73:0/2839964516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840700f0 con 0x7fcd841560e0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd7be00640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02bea0 tx=0x7fcd6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03b420 con 0x7fcd841560e0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c03b580 con 0x7fcd841560e0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd84154fd0 con 0x7fcd841560e0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03b910 con 0x7fcd841560e0 2024-04-05T22:28:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.386+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd84071f80 con 0x7fcd841560e0 2024-04-05T22:28:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.387+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84154e30 con 0x7fcd841560e0 2024-04-05T22:28:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.388+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fcd6c04e450 con 0x7fcd841560e0 2024-04-05T22:28:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.389+0000 7fcd7b400640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 0x7fcd6006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.389+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fcd6c0abcc0 con 0x7fcd841560e0 2024-04-05T22:28:20.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.390+0000 7fcd83400640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 0x7fcd6006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.391+0000 7fcd83400640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 0x7fcd6006d530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd74001050 tx=0x7fcd74001d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.394+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcd84154e30 con 0x7fcd841560e0 2024-04-05T22:28:20.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.591+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd84154e30 con 0x7fcd841560e0 2024-04-05T22:28:20.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.592+0000 7fcd7b400640 1 -- 172.21.15.73:0/2839964516 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd84154e30 con 0x7fcd841560e0 2024-04-05T22:28:20.591 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 msgr2=0x7fcd6006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 0x7fcd6006d530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd74001050 tx=0x7fcd74001d40 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 msgr2=0x7fcd84071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02bea0 tx=0x7fcd6c004730 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 shutdown_connections 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcd6006b070 0x7fcd6006d530 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8414f0a0 0x7fcd840b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd841560e0 0x7fcd84071a40 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 --2- 172.21.15.73:0/2839964516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd680047c0 0x7fcd8414f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.594+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 >> 172.21.15.73:0/2839964516 conn(0x7fcd840b0ce0 msgr2=0x7fcd8415db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:20.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.595+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 shutdown_connections 2024-04-05T22:28:20.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.595+0000 7fcd8a200640 1 -- 172.21.15.73:0/2839964516 wait complete. 2024-04-05T22:28:20.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:28:20.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.826+0000 7fa7b5c00640 1 Processor -- start 2024-04-05T22:28:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.826+0000 7fa7b5c00640 1 -- start start 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7b5c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7b01654e0 0x7fa7b016c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7b5c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b015f260 con 0x7fa7b00b1950 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7b5c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b006f5b0 con 0x7fa7b015ee80 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7b5c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b0070d20 con 0x7fa7b01654e0 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7ae000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa7b00b1950 0x7fa7b0164dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47296/0 (socket says 172.21.15.73:47296) 2024-04-05T22:28:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7ae000640 1 -- 172.21.15.73:0/583011746 learned_addr learned my addr 172.21.15.73:0/583011746 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.828+0000 7fa7af400640 1 --2- 172.21.15.73:0/583011746 >> v2:172.21.15.157:3300/0 conn(0x7fa7b01654e0 0x7fa7b016c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105620749 0 0) 0x7fa79c001110 con 0x7fa7b00b1950 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa794003520 con 0x7fa7b00b1950 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947357014 0 0) 0x7fa794003520 con 0x7fa7b00b1950 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 >> v2:172.21.15.157:3300/0 conn(0x7fa7b01654e0 msgr2=0x7fa7b016c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 --2- 172.21.15.73:0/583011746 >> v2:172.21.15.157:3300/0 conn(0x7fa7b01654e0 0x7fa7b016c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 >> v1:172.21.15.157:6789/0 conn(0x7fa7b015ee80 legacy=0x7fa7b00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b006fb70 con 0x7fa7b00b1950 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa79c001800 con 0x7fa7b00b1950 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 >> v1:172.21.15.73:6789/0 conn(0x7fa7b00b1950 legacy=0x7fa7b0164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa794003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940047c0 0x7fa794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa794005180 0x7fa79400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79400c050 con 0x7fa7940039c0 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79400c1c0 con 0x7fa7940047c0 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.829+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79400c380 con 0x7fa794005180 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa794003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7ae000640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940047c0 0x7fa794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7af400640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa794005180 0x7fa79400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 -- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa794005180 msgr2=0x7fa79400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa794005180 0x7fa79400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 -- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940047c0 msgr2=0x7fa794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940047c0 0x7fa794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7aea00640 1 -- 172.21.15.73:0/583011746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794003520 con 0x7fa7940039c0 2024-04-05T22:28:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.830+0000 7fa7af400640 1 -- 172.21.15.73:0/583011746 reap_dead start 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.831+0000 7fa7aea00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa794003de0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0001750 tx=0x7fa7940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e314b6aa3d798634 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.831+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00052d0 con 0x7fa7940039c0 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.831+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a002e050 con 0x7fa7940039c0 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.831+0000 7fa7ad600640 1 -- 172.21.15.73:0/583011746 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0003830 con 0x7fa7940039c0 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 -- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 msgr2=0x7fa794003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa794003de0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0001750 tx=0x7fa7940057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 -- 172.21.15.73:0/583011746 shutdown_connections 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/583011746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa794003de0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 -- 172.21.15.73:0/583011746 >> 172.21.15.73:0/583011746 conn(0x7fa7b0069cb0 msgr2=0x7fa7b006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 -- 172.21.15.73:0/583011746 shutdown_connections 2024-04-05T22:28:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.832+0000 7fa7b5c00640 1 -- 172.21.15.73:0/583011746 wait complete. 2024-04-05T22:28:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.834+0000 7fa7b5c00640 1 Processor -- start 2024-04-05T22:28:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.834+0000 7fa7b5c00640 1 -- start start 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7b00cb350 0x7fa7b0146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7b0155880 0x7fa7b0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b00aedd0 con 0x7fa7940039c0 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0155150 con 0x7fa7b0155880 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7b5c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0154250 con 0x7fa7b00cb350 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60774/0 (socket says 172.21.15.73:60774) 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7aea00640 1 -- 172.21.15.73:0/3487010196 learned_addr learned my addr 172.21.15.73:0/3487010196 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7ae000640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7b00cb350 0x7fa7b0146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.835+0000 7fa7af400640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7b0155880 0x7fa7b0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 -- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7b00cb350 msgr2=0x7fa7b0146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7b00cb350 0x7fa7b0146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 -- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7b0155880 msgr2=0x7fa7b0155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7b0155880 0x7fa7b0155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 -- 172.21.15.73:0/3487010196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b00ae5c0 con 0x7fa7940039c0 2024-04-05T22:28:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7aea00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005040 tx=0x7fa7a0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00076c0 con 0x7fa7940039c0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a00069e0 con 0x7fa7940039c0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b00ae810 con 0x7fa7940039c0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0006d90 con 0x7fa7940039c0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.836+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b0155360 con 0x7fa7940039c0 2024-04-05T22:28:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.838+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b00cb1c0 con 0x7fa7940039c0 2024-04-05T22:28:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.839+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fa7a00035f0 con 0x7fa7940039c0 2024-04-05T22:28:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.839+0000 7fa7a6a00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 0x7fa79006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.840+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa7a00ab830 con 0x7fa7940039c0 2024-04-05T22:28:20.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.840+0000 7fa7ae000640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 0x7fa79006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:20.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.841+0000 7fa7ae000640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 0x7fa79006d4b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa79c001050 tx=0x7fa79c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:20.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:20.844+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa7a0076350 con 0x7fa7940039c0 2024-04-05T22:28:21.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.039+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7b00aed30 con 0x7fa7940039c0 2024-04-05T22:28:21.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.040+0000 7fa7a6a00640 1 -- 172.21.15.73:0/3487010196 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7a0079970 con 0x7fa7940039c0 2024-04-05T22:28:21.040 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:28:21.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 msgr2=0x7fa79006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 0x7fa79006d4b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa79c001050 tx=0x7fa79c009040 comp rx=0 tx=0).stop 2024-04-05T22:28:21.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 msgr2=0x7fa7b01643b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005040 tx=0x7fa7a0001120 comp rx=0 tx=0).stop 2024-04-05T22:28:21.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 shutdown_connections 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa79006aff0 0x7fa79006d4b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7b0155880 0x7fa7b0155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7b00cb350 0x7fa7b0146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 --2- 172.21.15.73:0/3487010196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7940039c0 0x7fa7b01643b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 >> 172.21.15.73:0/3487010196 conn(0x7fa7b0069cb0 msgr2=0x7fa7b006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 shutdown_connections 2024-04-05T22:28:21.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.043+0000 7fa7b5c00640 1 -- 172.21.15.73:0/3487010196 wait complete. 2024-04-05T22:28:21.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:28:21.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.266+0000 7fe5bb600640 1 Processor -- start 2024-04-05T22:28:21.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.266+0000 7fe5bb600640 1 -- start start 2024-04-05T22:28:21.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.267+0000 7fe5bb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 0x7fe5b41657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.267+0000 7fe5bb600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b40b1b30 con 0x7fe5b406c630 2024-04-05T22:28:21.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.267+0000 7fe5bb600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b40b1e10 con 0x7fe5b40cbe00 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.267+0000 7fe5bb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40ce7b0 con 0x7fe5b40b32b0 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b8c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe5b406c630 0x7fe5b406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47308/0 (socket says 172.21.15.73:47308) 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b8c00640 1 -- 172.21.15.73:0/2367073025 learned_addr learned my addr 172.21.15.73:0/2367073025 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973800118 0 0) 0x7fe5b40b1b30 con 0x7fe5b406c630 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c003520 con 0x7fe5b406c630 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b9600640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 0x7fe5b41657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.268+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108122538 0 0) 0x7fe59c003520 con 0x7fe5b406c630 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 msgr2=0x7fe5b41657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 0x7fe5b41657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 >> v1:172.21.15.157:6789/0 conn(0x7fe5b40cbe00 legacy=0x7fe5b40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40cc300 con 0x7fe5b406c630 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe5a8001670 con 0x7fe5b406c630 2024-04-05T22:28:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 >> v1:172.21.15.73:6789/0 conn(0x7fe5b406c630 legacy=0x7fe5b406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59c0039a0 0x7fe59c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe59c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 0x7fe59c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40ce7b0 con 0x7fe59c0047c0 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c00ba90 con 0x7fe59c0039a0 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c00bc50 con 0x7fe5b40b32b0 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe59c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 -- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 msgr2=0x7fe59c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b40b32b0 0x7fe59c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 -- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59c0039a0 msgr2=0x7fe59c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59c0039a0 0x7fe59c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.269+0000 7fe5b3e00640 1 -- 172.21.15.73:0/2367073025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c003520 con 0x7fe59c0047c0 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.270+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe59c004c40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001750 tx=0x7fe59c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cab55f316a6044 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00052d0 con 0x7fe59c0047c0 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a002e050 con 0x7fe59c0047c0 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5b3400640 1 -- 172.21.15.73:0/2367073025 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003610 con 0x7fe59c0047c0 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 -- 172.21.15.73:0/2367073025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 msgr2=0x7fe59c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2367073025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe59c004c40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001750 tx=0x7fe59c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5b9600640 1 -- 172.21.15.73:0/2367073025 reap_dead start 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 -- 172.21.15.73:0/2367073025 shutdown_connections 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 -- 172.21.15.73:0/2367073025 >> 172.21.15.73:0/2367073025 conn(0x7fe5b40b0830 msgr2=0x7fe5b40b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 -- 172.21.15.73:0/2367073025 shutdown_connections 2024-04-05T22:28:21.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.271+0000 7fe5bb600640 1 -- 172.21.15.73:0/2367073025 wait complete. 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.273+0000 7fe5bb600640 1 Processor -- start 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.273+0000 7fe5bb600640 1 -- start start 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4142b90 0x7fe5b4141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b4141570 0x7fe5b413e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b4141af0 con 0x7fe59c0047c0 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b4141dd0 con 0x7fe5b4142b90 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5bb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b41420b0 con 0x7fe5b4141570 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.274+0000 7fe5b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60796/0 (socket says 172.21.15.73:60796) 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 -- 172.21.15.73:0/2535458812 learned_addr learned my addr 172.21.15.73:0/2535458812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 -- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b4141570 msgr2=0x7fe5b413e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b4141570 0x7fe5b413e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 -- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4142b90 msgr2=0x7fe5b4141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4142b90 0x7fe5b4141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 -- 172.21.15.73:0/2535458812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4142390 con 0x7fe59c0047c0 2024-04-05T22:28:21.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.275+0000 7fe5b8c00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800c7a0 tx=0x7fe5a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.276+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80074e0 con 0x7fe59c0047c0 2024-04-05T22:28:21.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.276+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8007640 con 0x7fe59c0047c0 2024-04-05T22:28:21.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.276+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8004a00 con 0x7fe59c0047c0 2024-04-05T22:28:21.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.276+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b4142670 con 0x7fe59c0047c0 2024-04-05T22:28:21.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.276+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b40aef50 con 0x7fe59c0047c0 2024-04-05T22:28:21.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.277+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b41428b0 con 0x7fe59c0047c0 2024-04-05T22:28:21.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.278+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fe5a8004b60 con 0x7fe59c0047c0 2024-04-05T22:28:21.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.279+0000 7fe5b0c00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 0x7fe59406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.279+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe5a8089100 con 0x7fe59c0047c0 2024-04-05T22:28:21.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.280+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 0x7fe59406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.281+0000 7fe5b3e00640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 0x7fe59406d4e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0033000 tx=0x7fe5a002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.284+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe5b41428b0 con 0x7fe59c0047c0 2024-04-05T22:28:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.478+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5b41428b0 con 0x7fe59c0047c0 2024-04-05T22:28:21.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.479+0000 7fe5b0c00640 1 -- 172.21.15.73:0/2535458812 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5b41428b0 con 0x7fe59c0047c0 2024-04-05T22:28:21.480 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:28:21.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 msgr2=0x7fe59406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 0x7fe59406d4e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0033000 tx=0x7fe5a002f000 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 msgr2=0x7fe5b40aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800c7a0 tx=0x7fe5a800cc60 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 shutdown_connections 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe59406b020 0x7fe59406d4e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b4141570 0x7fe5b413e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4142b90 0x7fe5b4141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 --2- 172.21.15.73:0/2535458812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c0047c0 0x7fe5b40aea10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 >> 172.21.15.73:0/2535458812 conn(0x7fe5b40b0830 msgr2=0x7fe5b40ada30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 shutdown_connections 2024-04-05T22:28:21.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.482+0000 7fe5bb600640 1 -- 172.21.15.73:0/2535458812 wait complete. 2024-04-05T22:28:21.500 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:21.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.710+0000 7fb9c1c00640 1 Processor -- start 2024-04-05T22:28:21.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.711+0000 7fb9c1c00640 1 -- start start 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9c1c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9bc15cf70 0x7fb9bc0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9c1c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9c1c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0ceee0 con 0x7fb9bc15d350 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9c1c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc0ce5b0 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9c1c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc0ce890 con 0x7fb9bc15cf70 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9ba000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9ba000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60798/0 (socket says 172.21.15.73:60798) 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.712+0000 7fb9ba000640 1 -- 172.21.15.73:0/174810930 learned_addr learned my addr 172.21.15.73:0/174810930 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.713+0000 7fb9baa00640 1 --2- 172.21.15.73:0/174810930 >> v2:172.21.15.157:3300/0 conn(0x7fb9bc15cf70 0x7fb9bc0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.713+0000 7fb9ba000640 1 -- 172.21.15.73:0/174810930 >> v2:172.21.15.157:3300/0 conn(0x7fb9bc15cf70 msgr2=0x7fb9bc0d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:21.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.713+0000 7fb9ba000640 1 --2- 172.21.15.73:0/174810930 >> v2:172.21.15.157:3300/0 conn(0x7fb9bc15cf70 0x7fb9bc0d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.713+0000 7fb9ba000640 1 -- 172.21.15.73:0/174810930 >> v1:172.21.15.157:6789/0 conn(0x7fb9bc15d350 legacy=0x7fb9bc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:21.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.713+0000 7fb9ba000640 1 -- 172.21.15.73:0/174810930 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0ceb70 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.714+0000 7fb9ba000640 1 --2- 172.21.15.73:0/174810930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0d08f0 tx=0x7fb9ac02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eff0db359bf705 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.714+0000 7fb9b9600640 1 -- 172.21.15.73:0/174810930 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac02c9f0 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.714+0000 7fb9b9600640 1 -- 172.21.15.73:0/174810930 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac02cb50 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.714+0000 7fb9b9600640 1 -- 172.21.15.73:0/174810930 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0343f0 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 -- 172.21.15.73:0/174810930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc0d0bd0 msgr2=0x7fb9bc0d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/174810930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0d08f0 tx=0x7fb9ac02be50 comp rx=0 tx=0).stop 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 -- 172.21.15.73:0/174810930 shutdown_connections 2024-04-05T22:28:21.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/174810930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0d1220 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/174810930 >> v2:172.21.15.157:3300/0 conn(0x7fb9bc15cf70 0x7fb9bc0d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 -- 172.21.15.73:0/174810930 >> 172.21.15.73:0/174810930 conn(0x7fb9bc0d2df0 msgr2=0x7fb9bc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 -- 172.21.15.73:0/174810930 shutdown_connections 2024-04-05T22:28:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.715+0000 7fb9c1c00640 1 -- 172.21.15.73:0/174810930 wait complete. 2024-04-05T22:28:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.717+0000 7fb9c1c00640 1 Processor -- start 2024-04-05T22:28:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.717+0000 7fb9c1c00640 1 -- start start 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 0x7fb9bc09ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0c8e20 con 0x7fb9bc15d350 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc09eaf0 con 0x7fb9bc0d0bd0 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.718+0000 7fb9c1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc09e4f0 con 0x7fb9bc15cf70 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60804/0 (socket says 172.21.15.73:60804) 2024-04-05T22:28:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 -- 172.21.15.73:0/3237092992 learned_addr learned my addr 172.21.15.73:0/3237092992 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9baa00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 -- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 msgr2=0x7fb9bc09ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9ba000640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 0x7fb9bc09ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 0x7fb9bc09ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 -- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 msgr2=0x7fb9bc0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9bb400640 1 -- 172.21.15.73:0/3237092992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009b30 con 0x7fb9bc15d350 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9ba000640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 0x7fb9bc09ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.719+0000 7fb9baa00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9bb400640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c390 tx=0x7fb9b400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40035f0 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4003750 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4003b00 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc09f4c0 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.720+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc0b86a0 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.721+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc068160 con 0x7fb9bc15d350 2024-04-05T22:28:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.722+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fb9b4003c60 con 0x7fb9bc15d350 2024-04-05T22:28:21.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.723+0000 7fb9b2a00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 0x7fb99806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:21.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.723+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb9b4088b70 con 0x7fb9bc15d350 2024-04-05T22:28:21.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.724+0000 7fb9baa00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 0x7fb99806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.725+0000 7fb9baa00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 0x7fb99806d4e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4008890 tx=0x7fb9a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:21.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.727+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb9b408ae60 con 0x7fb9bc15d350 2024-04-05T22:28:21.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.960+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9bc0b0c90 con 0x7fb9bc15d350 2024-04-05T22:28:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.961+0000 7fb9b2a00640 1 -- 172.21.15.73:0/3237092992 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fb9b40537a0 con 0x7fb9bc15d350 2024-04-05T22:28:21.960 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:21.960 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:28:21.960 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2024-04-05T22:28:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.963+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 msgr2=0x7fb99806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.963+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 0x7fb99806d4e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4008890 tx=0x7fb9a4008040 comp rx=0 tx=0).stop 2024-04-05T22:28:21.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.963+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 msgr2=0x7fb9bc0a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.963+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c390 tx=0x7fb9b400c850 comp rx=0 tx=0).stop 2024-04-05T22:28:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 shutdown_connections 2024-04-05T22:28:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb99806b020 0x7fb99806d4e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9bc15d350 0x7fb9bc0a04a0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9bc15cf70 0x7fb9bc09ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 --2- 172.21.15.73:0/3237092992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc0d0bd0 0x7fb9bc0c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 >> 172.21.15.73:0/3237092992 conn(0x7fb9bc0d2df0 msgr2=0x7fb9bc0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 shutdown_connections 2024-04-05T22:28:21.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:21.964+0000 7fb9c1c00640 1 -- 172.21.15.73:0/3237092992 wait complete. 2024-04-05T22:28:21.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.202+0000 7f0dacc00640 1 Processor -- start 2024-04-05T22:28:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.202+0000 7f0dacc00640 1 -- start start 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0dacc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0da815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0dacc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0da80d35e0 0x7f0da80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0dacc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da80b2420 con 0x7f0da80d0ef0 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0dacc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da80b2770 con 0x7f0da80d35e0 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0dacc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80ce380 con 0x7f0da815ccd0 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0da80d0ef0 0x7f0da80cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47314/0 (socket says 172.21.15.73:47314) 2024-04-05T22:28:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da7e00640 1 -- 172.21.15.73:0/379892122 learned_addr learned my addr 172.21.15.73:0/379892122 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da6a00640 1 --2- 172.21.15.73:0/379892122 >> v2:172.21.15.157:3300/0 conn(0x7f0da80d35e0 0x7f0da80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da7400640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0da815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395326196 0 0) 0x7f0da80b2420 con 0x7f0da80d0ef0 2024-04-05T22:28:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.204+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c003520 con 0x7f0da80d0ef0 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4206671450 0 0) 0x7f0d8c003520 con 0x7f0da80d0ef0 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 msgr2=0x7f0da815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0da815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 >> v2:172.21.15.157:3300/0 conn(0x7f0da80d35e0 msgr2=0x7f0da80d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 --2- 172.21.15.73:0/379892122 >> v2:172.21.15.157:3300/0 conn(0x7f0da80d35e0 0x7f0da80d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80ce660 con 0x7f0da80d0ef0 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0da00019f0 con 0x7f0da80d0ef0 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 >> v1:172.21.15.73:6789/0 conn(0x7f0da80d0ef0 legacy=0x7f0da80cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0d8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0d8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c004d00 0x7f0d8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80b2770 con 0x7f0d8c003970 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80ce380 con 0x7f0d8c004d00 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c00ba00 con 0x7f0da815ccd0 2024-04-05T22:28:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.205+0000 7f0da7400640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0d8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da6a00640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0d8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7400640 1 -- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 msgr2=0x7f0d8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7e00640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c004d00 0x7f0d8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7400640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0d8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7400640 1 -- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c004d00 msgr2=0x7f0d8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7400640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c004d00 0x7f0d8c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da7400640 1 -- 172.21.15.73:0/379892122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c003520 con 0x7f0d8c003970 2024-04-05T22:28:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.206+0000 7f0da6a00640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da815ccd0 0x7f0d8c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.207+0000 7f0da7400640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0d8c003d90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ac0 tx=0x7f0d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f14a6990d74d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.207+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9802c6b0 con 0x7f0d8c003970 2024-04-05T22:28:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.207+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9802c810 con 0x7f0d8c003970 2024-04-05T22:28:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.207+0000 7f0da6000640 1 -- 172.21.15.73:0/379892122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9802cba0 con 0x7f0d8c003970 2024-04-05T22:28:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 -- 172.21.15.73:0/379892122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 msgr2=0x7f0d8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 --2- 172.21.15.73:0/379892122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0d8c003d90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ac0 tx=0x7f0d8c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0da7e00640 1 -- 172.21.15.73:0/379892122 reap_dead start 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 -- 172.21.15.73:0/379892122 shutdown_connections 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 -- 172.21.15.73:0/379892122 >> 172.21.15.73:0/379892122 conn(0x7f0da80d2ac0 msgr2=0x7f0da80d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 -- 172.21.15.73:0/379892122 shutdown_connections 2024-04-05T22:28:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.208+0000 7f0dacc00640 1 -- 172.21.15.73:0/379892122 wait complete. 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.210+0000 7f0dacc00640 1 Processor -- start 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.210+0000 7f0dacc00640 1 -- start start 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 0x7f0da80c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 0x7f0da8154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80c7ea0 con 0x7f0d8c003970 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8154850 con 0x7f0da8153e90 2024-04-05T22:28:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0dacc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80c6f70 con 0x7f0da80c65d0 2024-04-05T22:28:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60830/0 (socket says 172.21.15.73:60830) 2024-04-05T22:28:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da7400640 1 -- 172.21.15.73:0/1523710391 learned_addr learned my addr 172.21.15.73:0/1523710391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da7400640 1 -- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 msgr2=0x7f0da80c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da7e00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 0x7f0da8154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.211+0000 7f0da6a00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 0x7f0da80c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7400640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 0x7f0da80c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7400640 1 -- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 msgr2=0x7f0da8154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7400640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 0x7f0da8154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7400640 1 -- 172.21.15.73:0/1523710391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80c7220 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da6a00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 0x7f0da80c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7e00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 0x7f0da8154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0da7400640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ac0 tx=0x7f0d98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98003810 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98003970 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98003d00 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da80c7500 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.212+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da80c7ac0 con 0x7f0d8c003970 2024-04-05T22:28:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.213+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da8067250 con 0x7f0d8c003970 2024-04-05T22:28:22.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.214+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f0d98040450 con 0x7f0d8c003970 2024-04-05T22:28:22.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.216+0000 7f0d9f400640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 0x7f0d8806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.216+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0d980abaa0 con 0x7f0d8c003970 2024-04-05T22:28:22.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.216+0000 7f0da6a00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 0x7f0d8806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.218+0000 7f0da6a00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 0x7f0d8806d4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da80c9610 tx=0x7f0d94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.220+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0d98076640 con 0x7f0d8c003970 2024-04-05T22:28:22.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.413+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0da804e160 con 0x7f0d8c003970 2024-04-05T22:28:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.416+0000 7f0d9f400640 1 -- 172.21.15.73:0/1523710391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16126 (secure 0 0 0) 0x7f0d980044e0 con 0x7f0d8c003970 2024-04-05T22:28:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:22.416 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:20.408998+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 msgr2=0x7f0d8806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 0x7f0d8806d4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da80c9610 tx=0x7f0d94008040 comp rx=0 tx=0).stop 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 msgr2=0x7f0da806b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ac0 tx=0x7f0d98004730 comp rx=0 tx=0).stop 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 shutdown_connections 2024-04-05T22:28:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d8806aff0 0x7f0d8806d4b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da8153e90 0x7f0da8154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0da80c65d0 0x7f0da80c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 --2- 172.21.15.73:0/1523710391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c003970 0x7f0da806b050 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 >> 172.21.15.73:0/1523710391 conn(0x7f0da80d2ac0 msgr2=0x7f0da80afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 shutdown_connections 2024-04-05T22:28:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.419+0000 7f0dacc00640 1 -- 172.21.15.73:0/1523710391 wait complete. 2024-04-05T22:28:22.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:28:22.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.642+0000 7f5a02c00640 1 Processor -- start 2024-04-05T22:28:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.643+0000 7f5a02c00640 1 -- start start 2024-04-05T22:28:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.644+0000 7f5a02c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.644+0000 7f5a02c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59fc15e850 0x7f59fc15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.644+0000 7f5a02c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc06f5b0 con 0x7f59fc0ccd20 2024-04-05T22:28:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.644+0000 7f5a02c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc15f240 con 0x7f59fc15e850 2024-04-05T22:28:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.644+0000 7f5a02c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc070410 con 0x7f59fc1587d0 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.645+0000 7f59fbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.645+0000 7f5a00c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59fc15e850 0x7f59fc15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.645+0000 7f59fbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:45592/0 (socket says 172.21.15.73:45592) 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.645+0000 7f59fbe00640 1 -- 172.21.15.73:0/734324515 learned_addr learned my addr 172.21.15.73:0/734324515 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.646+0000 7f59fbe00640 1 -- 172.21.15.73:0/734324515 >> v2:172.21.15.157:3300/0 conn(0x7f59fc15e850 msgr2=0x7f59fc15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.646+0000 7f59fbe00640 1 --2- 172.21.15.73:0/734324515 >> v2:172.21.15.157:3300/0 conn(0x7f59fc15e850 0x7f59fc15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.646+0000 7f59fbe00640 1 -- 172.21.15.73:0/734324515 >> v1:172.21.15.157:6789/0 conn(0x7f59fc0ccd20 legacy=0x7f59fc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.646+0000 7f59fbe00640 1 -- 172.21.15.73:0/734324515 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc158bb0 con 0x7f59fc1587d0 2024-04-05T22:28:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.647+0000 7f59fbe00640 1 --2- 172.21.15.73:0/734324515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37806bafc23aed1b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.648+0000 7f59faa00640 1 -- 172.21.15.73:0/734324515 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec030840 con 0x7f59fc1587d0 2024-04-05T22:28:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.648+0000 7f59faa00640 1 -- 172.21.15.73:0/734324515 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec0309a0 con 0x7f59fc1587d0 2024-04-05T22:28:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.648+0000 7f59faa00640 1 -- 172.21.15.73:0/734324515 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec030d30 con 0x7f59fc1587d0 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.648+0000 7f5a02c00640 1 -- 172.21.15.73:0/734324515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 msgr2=0x7f59fc0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.648+0000 7f5a02c00640 1 --2- 172.21.15.73:0/734324515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec02b980 comp rx=0 tx=0).stop 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 -- 172.21.15.73:0/734324515 shutdown_connections 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 --2- 172.21.15.73:0/734324515 >> v2:172.21.15.157:3300/0 conn(0x7f59fc15e850 0x7f59fc15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 --2- 172.21.15.73:0/734324515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc1587d0 0x7f59fc0cc7e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 -- 172.21.15.73:0/734324515 >> 172.21.15.73:0/734324515 conn(0x7f59fc0ce8e0 msgr2=0x7f59fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 -- 172.21.15.73:0/734324515 shutdown_connections 2024-04-05T22:28:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.649+0000 7f5a02c00640 1 -- 172.21.15.73:0/734324515 wait complete. 2024-04-05T22:28:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.650+0000 7f5a02c00640 1 Processor -- start 2024-04-05T22:28:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 -- start start 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc15e850 0x7f59fc154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 0x7f59fc1f0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1f0aa0 con 0x7f59fc1587d0 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0ac810 con 0x7f59fc15e850 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.651+0000 7f5a02c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0acaf0 con 0x7f59fc15a5d0 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60842/0 (socket says 172.21.15.73:60842) 2024-04-05T22:28:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 -- 172.21.15.73:0/4191541500 learned_addr learned my addr 172.21.15.73:0/4191541500 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f5a00c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 0x7f59fc1f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fb400640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc15e850 0x7f59fc154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 -- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 msgr2=0x7f59fc1f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 0x7f59fc1f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 -- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc15e850 msgr2=0x7f59fc154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc15e850 0x7f59fc154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f59fbe00640 1 -- 172.21.15.73:0/4191541500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0094d0 con 0x7f59fc1587d0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.652+0000 7f5a00c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 0x7f59fc1f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f59fbe00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec039c90 con 0x7f59fc1587d0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec039df0 con 0x7f59fc1587d0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec043640 con 0x7f59fc1587d0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc0acdd0 con 0x7f59fc1587d0 2024-04-05T22:28:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.653+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc1597b0 con 0x7f59fc1587d0 2024-04-05T22:28:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.654+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc066af0 con 0x7f59fc1587d0 2024-04-05T22:28:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.655+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f59ec0437a0 con 0x7f59fc1587d0 2024-04-05T22:28:22.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.656+0000 7f59f3e00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 0x7f59d806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:22.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.656+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f59ec03e070 con 0x7f59fc1587d0 2024-04-05T22:28:22.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.657+0000 7f59fb400640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 0x7f59d806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:22.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.658+0000 7f59fb400640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 0x7f59d806d4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59e4007fd0 tx=0x7f59e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:22.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.660+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f59ec0776f0 con 0x7f59fc1587d0 2024-04-05T22:28:22.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.891+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59fc159940 con 0x7f59fc1587d0 2024-04-05T22:28:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.892+0000 7f59f3e00640 1 -- 172.21.15.73:0/4191541500 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f59ec07ad10 con 0x7f59fc1587d0 2024-04-05T22:28:22.891 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:22.891 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:28:22.892 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.894+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 msgr2=0x7f59d806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.894+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 0x7f59d806d4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59e4007fd0 tx=0x7f59e4009040 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.894+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 msgr2=0x7f59fc0ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.894+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec004730 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 shutdown_connections 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59d806aff0 0x7f59d806d4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59fc15a5d0 0x7f59fc1f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59fc15e850 0x7f59fc154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 --2- 172.21.15.73:0/4191541500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59fc1587d0 0x7f59fc0ac210 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:22.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 >> 172.21.15.73:0/4191541500 conn(0x7f59fc0ce8e0 msgr2=0x7f59fc06c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:22.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 shutdown_connections 2024-04-05T22:28:22.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:22.895+0000 7f5a02c00640 1 -- 172.21.15.73:0/4191541500 wait complete. 2024-04-05T22:28:22.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.119+0000 7f7fe9e00640 1 Processor -- start 2024-04-05T22:28:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.119+0000 7f7fe9e00640 1 -- start start 2024-04-05T22:28:23.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fe40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe9e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fe40b2ba0 0x7f7fe40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe9e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe41627f0 con 0x7f7fe4161f90 2024-04-05T22:28:23.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe9e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe4158790 con 0x7f7fe40b2ba0 2024-04-05T22:28:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40b3570 con 0x7f7fe4162370 2024-04-05T22:28:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe2a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7fe4161f90 0x7f7fe4164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47322/0 (socket says 172.21.15.73:47322) 2024-04-05T22:28:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.121+0000 7f7fe2a00640 1 -- 172.21.15.73:0/662745533 learned_addr learned my addr 172.21.15.73:0/662745533 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe3400640 1 --2- 172.21.15.73:0/662745533 >> v2:172.21.15.157:3300/0 conn(0x7f7fe40b2ba0 0x7f7fe40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe2000640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fe40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054706768 0 0) 0x7f7fd4001490 con 0x7f7fe4161f90 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc8003520 con 0x7f7fe4161f90 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852259603 0 0) 0x7f7fc8003520 con 0x7f7fe4161f90 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 msgr2=0x7f7fe40b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fe40b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 >> v2:172.21.15.157:3300/0 conn(0x7f7fe40b2ba0 msgr2=0x7f7fe40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 --2- 172.21.15.73:0/662745533 >> v2:172.21.15.157:3300/0 conn(0x7f7fe40b2ba0 0x7f7fe40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4165cf0 con 0x7f7fe4161f90 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7fd4001b80 con 0x7f7fe4161f90 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 >> v1:172.21.15.73:6789/0 conn(0x7f7fe4161f90 legacy=0x7f7fe4164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.122+0000 7f7fe1600640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fc8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe1600640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8003ef0 0x7f7fc80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe1600640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc8004d00 0x7f7fc8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4158790 con 0x7f7fc8004d00 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40b3570 con 0x7f7fc8003ef0 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800b980 con 0x7f7fe4162370 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc8004d00 0x7f7fc8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe2a00640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fc8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe2000640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8003ef0 0x7f7fc80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 -- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 msgr2=0x7f7fc8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4162370 0x7f7fc8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 -- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8003ef0 msgr2=0x7f7fc80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8003ef0 0x7f7fc80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.123+0000 7f7fe3400640 1 -- 172.21.15.73:0/662745533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8003520 con 0x7f7fc8004d00 2024-04-05T22:28:23.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe3400640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc8004d00 0x7f7fc8005180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009ac0 tx=0x7f7fc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550ce5d72f5914a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc02c6d0 con 0x7f7fc8004d00 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc02c830 con 0x7f7fc8004d00 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe1600640 1 -- 172.21.15.73:0/662745533 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc02cbc0 con 0x7f7fc8004d00 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe9e00640 1 -- 172.21.15.73:0/662745533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc8004d00 msgr2=0x7f7fc8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/662745533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc8004d00 0x7f7fc8005180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009ac0 tx=0x7f7fc80057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.125+0000 7f7fe3400640 1 -- 172.21.15.73:0/662745533 reap_dead start 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.126+0000 7f7fe9e00640 1 -- 172.21.15.73:0/662745533 shutdown_connections 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.126+0000 7f7fe9e00640 1 -- 172.21.15.73:0/662745533 >> 172.21.15.73:0/662745533 conn(0x7f7fe40ce8e0 msgr2=0x7f7fe40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.126+0000 7f7fe9e00640 1 -- 172.21.15.73:0/662745533 shutdown_connections 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.126+0000 7f7fe9e00640 1 -- 172.21.15.73:0/662745533 wait complete. 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.127+0000 7f7fe9e00640 1 Processor -- start 2024-04-05T22:28:23.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.128+0000 7f7fe9e00640 1 -- start start 2024-04-05T22:28:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe4156c00 0x7f7fe4146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4146580 0x7f7fe4147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4157080 con 0x7f7fe4145c60 2024-04-05T22:28:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4145660 con 0x7f7fe4156c00 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4148de0 con 0x7f7fe4146580 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60862/0 (socket says 172.21.15.73:60862) 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 -- 172.21.15.73:0/896165375 learned_addr learned my addr 172.21.15.73:0/896165375 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 -- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4146580 msgr2=0x7f7fe4147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4146580 0x7f7fe4147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 -- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe4156c00 msgr2=0x7f7fe4146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe4156c00 0x7f7fe4146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.129+0000 7f7fe2a00640 1 -- 172.21.15.73:0/896165375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4146f80 con 0x7f7fe4145c60 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fe2a00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4007c40 tx=0x7f7fd400e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400ee10 con 0x7f7fe4145c60 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4013420 con 0x7f7fe4145c60 2024-04-05T22:28:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40137d0 con 0x7f7fe4145c60 2024-04-05T22:28:23.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe4147260 con 0x7f7fe4145c60 2024-04-05T22:28:23.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.130+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe4149030 con 0x7f7fe4145c60 2024-04-05T22:28:23.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.131+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe4066af0 con 0x7f7fe4145c60 2024-04-05T22:28:23.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.132+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f7fd4013930 con 0x7f7fe4145c60 2024-04-05T22:28:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.133+0000 7f7fdb400640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 0x7f7fc406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.133+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7fd408a330 con 0x7f7fe4145c60 2024-04-05T22:28:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.134+0000 7f7fe2000640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 0x7f7fc406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.135+0000 7f7fe2000640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 0x7f7fc406d4e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4069fc0 tx=0x7f7fd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.138+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7fd4054f60 con 0x7f7fe4145c60 2024-04-05T22:28:23.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.372+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe41627f0 con 0x7f7fe4145c60 2024-04-05T22:28:23.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.373+0000 7f7fdb400640 1 -- 172.21.15.73:0/896165375 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f7fe41627f0 con 0x7f7fe4145c60 2024-04-05T22:28:23.372 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:23.372 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:27:32.104968+0000","modified":"2024-04-05T22:27:35.132684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/3290176955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3290176955},{"type":"v1","addr":"172.21.15.73:6835","nonce":3290176955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-05T22:28:23.373 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2024-04-05T22:28:23.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 msgr2=0x7f7fc406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 0x7f7fc406d4e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4069fc0 tx=0x7f7fd0008040 comp rx=0 tx=0).stop 2024-04-05T22:28:23.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 msgr2=0x7f7fe41566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4007c40 tx=0x7f7fd400e450 comp rx=0 tx=0).stop 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 shutdown_connections 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7fc406b020 0x7f7fc406d4e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe4146580 0x7f7fe4147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe4156c00 0x7f7fe4146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 --2- 172.21.15.73:0/896165375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fe4145c60 0x7f7fe41566c0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 >> 172.21.15.73:0/896165375 conn(0x7f7fe40ce8e0 msgr2=0x7f7fe406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 shutdown_connections 2024-04-05T22:28:23.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.376+0000 7f7fe9e00640 1 -- 172.21.15.73:0/896165375 wait complete. 2024-04-05T22:28:23.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:23.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.606+0000 7fa0d4800640 1 Processor -- start 2024-04-05T22:28:23.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.606+0000 7fa0d4800640 1 -- start start 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.607+0000 7fa0d4800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0cc1587d0 0x7fa0cc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.607+0000 7fa0d4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.607+0000 7fa0d4800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc06f5b0 con 0x7fa0cc06d0c0 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.607+0000 7fa0d4800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc070410 con 0x7fa0cc1587d0 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.607+0000 7fa0d4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc158bb0 con 0x7fa0cc0ccd20 2024-04-05T22:28:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.608+0000 7fa0d1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.608+0000 7fa0d1e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0cc1587d0 0x7fa0cc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.608+0000 7fa0d1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:45648/0 (socket says 172.21.15.73:45648) 2024-04-05T22:28:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.608+0000 7fa0d1400640 1 -- 172.21.15.73:0/808459757 learned_addr learned my addr 172.21.15.73:0/808459757 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.609+0000 7fa0d1e00640 1 -- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 msgr2=0x7fa0cc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.609+0000 7fa0d1e00640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.609+0000 7fa0d1e00640 1 -- 172.21.15.73:0/808459757 >> v1:172.21.15.157:6789/0 conn(0x7fa0cc06d0c0 legacy=0x7fa0cc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.609+0000 7fa0d1e00640 1 -- 172.21.15.73:0/808459757 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc06f800 con 0x7fa0cc1587d0 2024-04-05T22:28:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.609+0000 7fa0d1400640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:28:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.610+0000 7fa0d1e00640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0cc7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00097a0 tx=0x7fa0c002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1949daa69a3cc338 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.611+0000 7fa0d0a00640 1 -- 172.21.15.73:0/808459757 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c002d040 con 0x7fa0cc1587d0 2024-04-05T22:28:23.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.611+0000 7fa0d0a00640 1 -- 172.21.15.73:0/808459757 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0031810 con 0x7fa0cc1587d0 2024-04-05T22:28:23.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.611+0000 7fa0d0a00640 1 -- 172.21.15.73:0/808459757 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0031ba0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.611+0000 7fa0d4800640 1 -- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc1587d0 msgr2=0x7fa0cc0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.611+0000 7fa0d4800640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0cc7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00097a0 tx=0x7fa0c002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 -- 172.21.15.73:0/808459757 shutdown_connections 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc0ccd20 0x7fa0cc06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 --2- 172.21.15.73:0/808459757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0cc7e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 -- 172.21.15.73:0/808459757 >> 172.21.15.73:0/808459757 conn(0x7fa0cc0ce8e0 msgr2=0x7fa0cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 -- 172.21.15.73:0/808459757 shutdown_connections 2024-04-05T22:28:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.612+0000 7fa0d4800640 1 -- 172.21.15.73:0/808459757 wait complete. 2024-04-05T22:28:23.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.613+0000 7fa0d4800640 1 Processor -- start 2024-04-05T22:28:23.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 -- start start 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc06d0c0 0x7fa0cc1f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc15df10 0x7fa0cc0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1f2440 con 0x7fa0cc1587d0 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc140000 con 0x7fa0cc15df10 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.614+0000 7fa0d4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1402e0 con 0x7fa0cc06d0c0 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60870/0 (socket says 172.21.15.73:60870) 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 -- 172.21.15.73:0/2940453525 learned_addr learned my addr 172.21.15.73:0/2940453525 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1e00640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc06d0c0 0x7fa0cc1f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d2800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc15df10 0x7fa0cc0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 -- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc06d0c0 msgr2=0x7fa0cc1f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc06d0c0 0x7fa0cc1f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 -- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc15df10 msgr2=0x7fa0cc0ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc15df10 0x7fa0cc0ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.615+0000 7fa0d1400640 1 -- 172.21.15.73:0/2940453525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0094d0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0d1400640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00e9d0 tx=0x7fa0bc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00ca60 con 0x7fa0cc1587d0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00cbc0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc013470 con 0x7fa0cc1587d0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0009400 con 0x7fa0cc1587d0 2024-04-05T22:28:23.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.616+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1405c0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.617+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc066af0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.618+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fa0bc0135d0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.619+0000 7fa0c6000640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 0x7fa0b006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:23.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.619+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa0bc089eb0 con 0x7fa0cc1587d0 2024-04-05T22:28:23.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.619+0000 7fa0d1e00640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 0x7fa0b006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:23.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.620+0000 7fa0d1e00640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 0x7fa0b006d4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0036000 tx=0x7fa0c003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:23.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.623+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa0bc054a50 con 0x7fa0cc1587d0 2024-04-05T22:28:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.821+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0cc068160 con 0x7fa0cc1587d0 2024-04-05T22:28:23.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.824+0000 7fa0c6000640 1 -- 172.21.15.73:0/2940453525 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16126 (secure 0 0 0) 0x7fa0bc017090 con 0x7fa0cc1587d0 2024-04-05T22:28:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:23.824 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:20.408998+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:27:30.588885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-05T22:27:31.104825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 msgr2=0x7fa0b006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 0x7fa0b006d4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0036000 tx=0x7fa0c003c040 comp rx=0 tx=0).stop 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 msgr2=0x7fa0cc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00e9d0 tx=0x7fa0bc00ee90 comp rx=0 tx=0).stop 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 shutdown_connections 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa0b006aff0 0x7fa0b006d4b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0cc15df10 0x7fa0cc0ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0cc1587d0 0x7fa0cc0ac8d0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 --2- 172.21.15.73:0/2940453525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0cc06d0c0 0x7fa0cc1f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 >> 172.21.15.73:0/2940453525 conn(0x7fa0cc0ce8e0 msgr2=0x7fa0cc0add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 shutdown_connections 2024-04-05T22:28:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:23.827+0000 7fa0d4800640 1 -- 172.21.15.73:0/2940453525 wait complete. 2024-04-05T22:28:23.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:28:24.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.059+0000 7fbda7200640 1 Processor -- start 2024-04-05T22:28:24.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.059+0000 7fbda7200640 1 -- start start 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbda81441e0 0x7fbda80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda80b2c00 0x7fbda80b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbda806bbc0 0x7fbda806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80b1950 con 0x7fbda806bbc0 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda806c570 con 0x7fbda81441e0 2024-04-05T22:28:24.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80b1b60 con 0x7fbda80b2c00 2024-04-05T22:28:24.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbda806bbc0 0x7fbda806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbda806bbc0 0x7fbda806c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60872/0 (socket says 172.21.15.73:60872) 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.061+0000 7fbda6800640 1 -- 172.21.15.73:0/3386380922 learned_addr learned my addr 172.21.15.73:0/3386380922 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbd9d400640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda80b2c00 0x7fbda80b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda5e00640 1 --2- 172.21.15.73:0/3386380922 >> v2:172.21.15.157:3300/0 conn(0x7fbda81441e0 0x7fbda80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda6800640 1 -- 172.21.15.73:0/3386380922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda80b2c00 msgr2=0x7fbda80b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda6800640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda80b2c00 0x7fbda80b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda6800640 1 -- 172.21.15.73:0/3386380922 >> v2:172.21.15.157:3300/0 conn(0x7fbda81441e0 msgr2=0x7fbda80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda6800640 1 --2- 172.21.15.73:0/3386380922 >> v2:172.21.15.157:3300/0 conn(0x7fbda81441e0 0x7fbda80cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.062+0000 7fbda6800640 1 -- 172.21.15.73:0/3386380922 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80b1e40 con 0x7fbda806bbc0 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.063+0000 7fbda6800640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda806bbc0 0x7fbda806c030 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbda80b39a0 tx=0x7fbda002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b88d7377f5299 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.063+0000 7fbda5400640 1 -- 172.21.15.73:0/3386380922 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0031870 con 0x7fbda806bbc0 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.063+0000 7fbda5400640 1 -- 172.21.15.73:0/3386380922 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda00319d0 con 0x7fbda806bbc0 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.063+0000 7fbda5400640 1 -- 172.21.15.73:0/3386380922 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0031d60 con 0x7fbda806bbc0 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 -- 172.21.15.73:0/3386380922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda806bbc0 msgr2=0x7fbda806c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda806bbc0 0x7fbda806c030 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbda80b39a0 tx=0x7fbda002b910 comp rx=0 tx=0).stop 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 -- 172.21.15.73:0/3386380922 shutdown_connections 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda806bbc0 0x7fbda806c030 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 --2- 172.21.15.73:0/3386380922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda80b2c00 0x7fbda80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 --2- 172.21.15.73:0/3386380922 >> v2:172.21.15.157:3300/0 conn(0x7fbda81441e0 0x7fbda80cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 -- 172.21.15.73:0/3386380922 >> 172.21.15.73:0/3386380922 conn(0x7fbda80b0830 msgr2=0x7fbda80b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 -- 172.21.15.73:0/3386380922 shutdown_connections 2024-04-05T22:28:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.064+0000 7fbda7200640 1 -- 172.21.15.73:0/3386380922 wait complete. 2024-04-05T22:28:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.066+0000 7fbda7200640 1 Processor -- start 2024-04-05T22:28:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.066+0000 7fbda7200640 1 -- start start 2024-04-05T22:28:24.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbda806bbc0 0x7fbda8142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 0x7fbda8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80ac4a0 con 0x7fbda80b2c00 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80ad460 con 0x7fbda806bbc0 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbda7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80ac010 con 0x7fbda81441e0 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbd9d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbd9d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60876/0 (socket says 172.21.15.73:60876) 2024-04-05T22:28:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.067+0000 7fbd9d400640 1 -- 172.21.15.73:0/2858870129 learned_addr learned my addr 172.21.15.73:0/2858870129 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbda6800640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 0x7fbda8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbda5e00640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbda806bbc0 0x7fbda8142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 -- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 msgr2=0x7fbda8070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 0x7fbda8070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 -- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbda806bbc0 msgr2=0x7fbda8142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbda806bbc0 0x7fbda8142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 -- 172.21.15.73:0/2858870129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00094d0 con 0x7fbda80b2c00 2024-04-05T22:28:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbda6800640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 0x7fbda8070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9d400640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d800 tx=0x7fbd9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000b6c0 con 0x7fbda80b2c00 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000b820 con 0x7fbda80b2c00 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.068+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000bbb0 con 0x7fbda80b2c00 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.069+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda81429b0 con 0x7fbda80b2c00 2024-04-05T22:28:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.069+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80ad180 con 0x7fbda80b2c00 2024-04-05T22:28:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.070+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda80ac180 con 0x7fbda80b2c00 2024-04-05T22:28:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.071+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fbd9000bd10 con 0x7fbda80b2c00 2024-04-05T22:28:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.072+0000 7fbd9ea00640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 0x7fbd8406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.072+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbd9008a010 con 0x7fbda80b2c00 2024-04-05T22:28:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.072+0000 7fbda5e00640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 0x7fbd8406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.073+0000 7fbda5e00640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 0x7fbd8406d4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbda8067d00 tx=0x7fbd9800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.077+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbd90054c40 con 0x7fbda80b2c00 2024-04-05T22:28:24.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.313+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbda804e160 con 0x7fbda80b2c00 2024-04-05T22:28:24.315 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:28:24.316+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:28:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.331+0000 7fbd9ea00640 1 -- 172.21.15.73:0/2858870129 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd90058260 con 0x7fbda80b2c00 2024-04-05T22:28:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:28:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 msgr2=0x7fbd8406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 0x7fbd8406d4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbda8067d00 tx=0x7fbd9800b040 comp rx=0 tx=0).stop 2024-04-05T22:28:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 msgr2=0x7fbda80a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d800 tx=0x7fbd9000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 shutdown_connections 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd8406aff0 0x7fbd8406d4b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbda81441e0 0x7fbda8070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbda80b2c00 0x7fbda80a84c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 --2- 172.21.15.73:0/2858870129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbda806bbc0 0x7fbda8142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 >> 172.21.15.73:0/2858870129 conn(0x7fbda80b0830 msgr2=0x7fbda80cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 shutdown_connections 2024-04-05T22:28:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.334+0000 7fbda7200640 1 -- 172.21.15.73:0/2858870129 wait complete. 2024-04-05T22:28:24.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:28:24.400 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:28:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.541+0000 7fcfe7800640 1 Processor -- start 2024-04-05T22:28:24.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.541+0000 7fcfe7800640 1 -- start start 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe7800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcfe015dd90 0x7fcfe0071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe7800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe0146ac0 con 0x7fcfe006c630 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe7800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00acd30 con 0x7fcfe00ac8d0 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe7800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe0070720 con 0x7fcfe015dd90 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe4e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcfe006c630 0x7fcfe006ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47348/0 (socket says 172.21.15.73:47348) 2024-04-05T22:28:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe4e00640 1 -- 172.21.15.73:0/846024542 learned_addr learned my addr 172.21.15.73:0/846024542 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080284086 0 0) 0x7fcfe0146ac0 con 0x7fcfe006c630 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfe5800640 1 --2- 172.21.15.73:0/846024542 >> v2:172.21.15.157:3300/0 conn(0x7fcfe015dd90 0x7fcfe0071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.543+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc4003520 con 0x7fcfe006c630 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077056389 0 0) 0x7fcfc4003520 con 0x7fcfe006c630 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 >> v2:172.21.15.157:3300/0 conn(0x7fcfe015dd90 msgr2=0x7fcfe0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 --2- 172.21.15.73:0/846024542 >> v2:172.21.15.157:3300/0 conn(0x7fcfe015dd90 0x7fcfe0071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 >> v1:172.21.15.157:6789/0 conn(0x7fcfe00ac8d0 legacy=0x7fcfe0070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00acf60 con 0x7fcfe006c630 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fcfd0001670 con 0x7fcfe006c630 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 >> v1:172.21.15.73:6789/0 conn(0x7fcfe006c630 legacy=0x7fcfe006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc4003970 0x7fcfc4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc40047c0 0x7fcfc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0070720 con 0x7fcfc4005180 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400c0b0 con 0x7fcfc4003970 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400c270 con 0x7fcfc40047c0 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfe5800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.544+0000 7fcfe4e00640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc4003970 0x7fcfc4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfdfe00640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc40047c0 0x7fcfc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 -- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc40047c0 msgr2=0x7fcfc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc40047c0 0x7fcfc4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 -- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc4003970 msgr2=0x7fcfc4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc4003970 0x7fcfc4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 -- 172.21.15.73:0/846024542 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4003520 con 0x7fcfc4005180 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.545+0000 7fcfe5800640 1 -- 172.21.15.73:0/846024542 reap_dead start 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfe5800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfc400b980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007330 tx=0x7fcfc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60c90cc8dbe4f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd802ccd0 con 0x7fcfc4005180 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd802ce30 con 0x7fcfc4005180 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfdf400640 1 -- 172.21.15.73:0/846024542 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8003bd0 con 0x7fcfc4005180 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfe7800640 1 -- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 msgr2=0x7fcfc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.546+0000 7fcfe7800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfc400b980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007330 tx=0x7fcfc40057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.547+0000 7fcfe7800640 1 -- 172.21.15.73:0/846024542 shutdown_connections 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.547+0000 7fcfe7800640 1 --2- 172.21.15.73:0/846024542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfc400b980 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.547+0000 7fcfe7800640 1 -- 172.21.15.73:0/846024542 >> 172.21.15.73:0/846024542 conn(0x7fcfe00b0830 msgr2=0x7fcfe00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.547+0000 7fcfe7800640 1 -- 172.21.15.73:0/846024542 shutdown_connections 2024-04-05T22:28:24.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.547+0000 7fcfe7800640 1 -- 172.21.15.73:0/846024542 wait complete. 2024-04-05T22:28:24.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.548+0000 7fcfe7800640 1 Processor -- start 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.548+0000 7fcfe7800640 1 -- start start 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfe01f5d60 0x7fcfe00b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe01f61e0 0x7fcfe00b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01f6d00 con 0x7fcfc4005180 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01336c0 con 0x7fcfe01f61e0 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.549+0000 7fcfe7800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01f2650 con 0x7fcfe01f5d60 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60940/0 (socket says 172.21.15.73:60940) 2024-04-05T22:28:24.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 -- 172.21.15.73:0/2285654706 learned_addr learned my addr 172.21.15.73:0/2285654706 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:24.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 -- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfe01f5d60 msgr2=0x7fcfe00b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfe01f5d60 0x7fcfe00b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 -- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe01f61e0 msgr2=0x7fcfe00b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:24.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe01f61e0 0x7fcfe00b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 -- 172.21.15.73:0/2285654706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe01f6740 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfe4e00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e740 tx=0x7fcfd000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.550+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000ee50 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.551+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000c8d0 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.551+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe01f6a20 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.551+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000cc80 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.551+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0135e10 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.552+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0133140 con 0x7fcfc4005180 2024-04-05T22:28:24.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.553+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fcfd0013420 con 0x7fcfc4005180 2024-04-05T22:28:24.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.553+0000 7fcfdcc00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 0x7fcfbc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:24.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.554+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcfd00896f0 con 0x7fcfc4005180 2024-04-05T22:28:24.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.554+0000 7fcfdfe00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 0x7fcfbc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:24.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.555+0000 7fcfdfe00640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 0x7fcfbc06d4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0136360 tx=0x7fcfc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:24.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.558+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcfe0133140 con 0x7fcfc4005180 2024-04-05T22:28:24.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:24.791+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcfe0133140 con 0x7fcfc4005180 2024-04-05T22:28:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.335+0000 7fcfdcc00640 1 -- 172.21.15.73:0/2285654706 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcfe0133140 con 0x7fcfc4005180 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 msgr2=0x7fcfbc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 0x7fcfbc06d4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0136360 tx=0x7fcfc8009040 comp rx=0 tx=0).stop 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 msgr2=0x7fcfe01f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e740 tx=0x7fcfd000ec00 comp rx=0 tx=0).stop 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 shutdown_connections 2024-04-05T22:28:25.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcfbc06b020 0x7fcfbc06d4e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe01f61e0 0x7fcfe00b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfe01f5d60 0x7fcfe00b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 --2- 172.21.15.73:0/2285654706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc4005180 0x7fcfe01f2110 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 >> 172.21.15.73:0/2285654706 conn(0x7fcfe00b0830 msgr2=0x7fcfe00713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 shutdown_connections 2024-04-05T22:28:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.339+0000 7fcfe7800640 1 -- 172.21.15.73:0/2285654706 wait complete. 2024-04-05T22:28:25.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:28:25.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.614+0000 7f3296600640 1 Processor -- start 2024-04-05T22:28:25.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.614+0000 7f3296600640 1 -- start start 2024-04-05T22:28:25.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f3296600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32900cf3e0 0x7f32900b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f3296600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 0x7f32900b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f3296600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329015d900 con 0x7f32901541a0 2024-04-05T22:28:25.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f3296600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329015e1a0 con 0x7f32900cf3e0 2024-04-05T22:28:25.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f3296600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900b3490 con 0x7f3290154580 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f328ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32900cf3e0 0x7f32900b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f328f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f32901541a0 0x7f32900cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47358/0 (socket says 172.21.15.73:47358) 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f328f400640 1 -- 172.21.15.73:0/2890807276 learned_addr learned my addr 172.21.15.73:0/2890807276 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f328fe00640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 0x7f32900b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.616+0000 7f328ea00640 1 -- 172.21.15.73:0/2890807276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 msgr2=0x7f32900b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.617+0000 7f328ea00640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 0x7f32900b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.617+0000 7f328ea00640 1 -- 172.21.15.73:0/2890807276 >> v1:172.21.15.73:6789/0 conn(0x7f32901541a0 legacy=0x7f32900cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.617+0000 7f328ea00640 1 -- 172.21.15.73:0/2890807276 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329015db50 con 0x7f32900cf3e0 2024-04-05T22:28:25.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.617+0000 7f328fe00640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 0x7f32900b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:25.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.617+0000 7f328e000640 1 -- 172.21.15.73:0/2890807276 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374422222 0 0) 0x7f329015d900 con 0x7f32901541a0 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.618+0000 7f328ea00640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32900cf3e0 0x7f32900b2840 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097a0 tx=0x7f327c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e280ad7351f7b6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.618+0000 7f328e000640 1 -- 172.21.15.73:0/2890807276 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c02d040 con 0x7f32900cf3e0 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.618+0000 7f328e000640 1 -- 172.21.15.73:0/2890807276 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c031810 con 0x7f32900cf3e0 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.618+0000 7f328e000640 1 -- 172.21.15.73:0/2890807276 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c031ba0 con 0x7f32900cf3e0 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 -- 172.21.15.73:0/2890807276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32900cf3e0 msgr2=0x7f32900b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32900cf3e0 0x7f32900b2840 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097a0 tx=0x7f327c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 -- 172.21.15.73:0/2890807276 shutdown_connections 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3290154580 0x7f32900b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 --2- 172.21.15.73:0/2890807276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32900cf3e0 0x7f32900b2840 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 -- 172.21.15.73:0/2890807276 >> 172.21.15.73:0/2890807276 conn(0x7f329006a0d0 msgr2=0x7f329006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 -- 172.21.15.73:0/2890807276 shutdown_connections 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.619+0000 7f3296600640 1 -- 172.21.15.73:0/2890807276 wait complete. 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.620+0000 7f3296600640 1 Processor -- start 2024-04-05T22:28:25.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.621+0000 7f3296600640 1 -- start start 2024-04-05T22:28:25.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32900cf3e0 0x7f32900b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3290154580 0x7f329014c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290153820 con 0x7f32901541a0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32900af880 con 0x7f3290154580 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f3296600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900afb60 con 0x7f32900cf3e0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60956/0 (socket says 172.21.15.73:60956) 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 -- 172.21.15.73:0/3709169159 learned_addr learned my addr 172.21.15.73:0/3709169159 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 -- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32900cf3e0 msgr2=0x7f32900b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328f400640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32900cf3e0 0x7f32900b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32900cf3e0 0x7f32900b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 -- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3290154580 msgr2=0x7f329014c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328fe00640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3290154580 0x7f329014c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3290154580 0x7f329014c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.622+0000 7f328ea00640 1 -- 172.21.15.73:0/3709169159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009400 con 0x7f32901541a0 2024-04-05T22:28:25.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f328ea00640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c03bae0 con 0x7f32901541a0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c03bc40 con 0x7f32901541a0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c004030 con 0x7f32901541a0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32900afe40 con 0x7f32901541a0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.623+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32900b0400 con 0x7f32901541a0 2024-04-05T22:28:25.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.624+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3290068160 con 0x7f32901541a0 2024-04-05T22:28:25.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.625+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f327c043070 con 0x7f32901541a0 2024-04-05T22:28:25.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.626+0000 7f3287400640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 0x7f327006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:25.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.626+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f327c0abb30 con 0x7f32901541a0 2024-04-05T22:28:25.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.627+0000 7f328f400640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 0x7f327006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:25.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.628+0000 7f328f400640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 0x7f327006d4b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3280007fd0 tx=0x7f3280009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:25.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.631+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f327c076610 con 0x7f32901541a0 2024-04-05T22:28:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:25.825+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32900b2060 con 0x7f32901541a0 2024-04-05T22:28:26.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.784+0000 7f3287400640 1 -- 172.21.15.73:0/3709169159 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7f327c035020 con 0x7f32901541a0 2024-04-05T22:28:26.784 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 msgr2=0x7f327006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 0x7f327006d4b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3280007fd0 tx=0x7f3280009040 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 msgr2=0x7f32900acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 shutdown_connections 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f327006aff0 0x7f327006d4b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3290154580 0x7f329014c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32901541a0 0x7f32900acb40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 --2- 172.21.15.73:0/3709169159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32900cf3e0 0x7f32900b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:26.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.790+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 >> 172.21.15.73:0/3709169159 conn(0x7f329006a0d0 msgr2=0x7f32900cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.792+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 shutdown_connections 2024-04-05T22:28:26.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:26.793+0000 7f3296600640 1 -- 172.21.15.73:0/3709169159 wait complete. 2024-04-05T22:28:26.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:28:27.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.017+0000 7ff0bc800640 1 Processor -- start 2024-04-05T22:28:27.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.018+0000 7ff0bc800640 1 -- start start 2024-04-05T22:28:27.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.019+0000 7ff0bc800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0b415ac00 0x7ff0b406cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.019+0000 7ff0bc800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0b415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.019+0000 7ff0bc800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b415afe0 con 0x7ff0b40b36e0 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.019+0000 7ff0bc800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b4161760 con 0x7ff0b415ac00 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.019+0000 7ff0bc800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b406d4c0 con 0x7ff0b40cf3e0 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0b9400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff0b40b36e0 0x7ff0b40cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48536/0 (socket says 172.21.15.73:48536) 2024-04-05T22:28:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0b9400640 1 -- 172.21.15.73:0/3329794934 learned_addr learned my addr 172.21.15.73:0/3329794934 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0b9e00640 1 --2- 172.21.15.73:0/3329794934 >> v2:172.21.15.157:3300/0 conn(0x7ff0b415ac00 0x7ff0b406cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0ba800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0b415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239855707 0 0) 0x7ff0a4001110 con 0x7ff0b40b36e0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.020+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a0003520 con 0x7ff0b40b36e0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1183178615 0 0) 0x7ff0a0003520 con 0x7ff0b40b36e0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 msgr2=0x7ff0b415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0b415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 >> v2:172.21.15.157:3300/0 conn(0x7ff0b415ac00 msgr2=0x7ff0b406cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 --2- 172.21.15.73:0/3329794934 >> v2:172.21.15.157:3300/0 conn(0x7ff0b415ac00 0x7ff0b406cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4160b50 con 0x7ff0b40b36e0 2024-04-05T22:28:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff0a4001800 con 0x7ff0b40b36e0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 >> v1:172.21.15.73:6789/0 conn(0x7ff0b40b36e0 legacy=0x7ff0b40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0003ef0 0x7ff0a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0004d00 0x7ff0a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4161760 con 0x7ff0a0004d00 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b406d4c0 con 0x7ff0a0003ef0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a000b9d0 con 0x7ff0b40cf3e0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0ba800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0004d00 0x7ff0a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b9e00640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.021+0000 7ff0b9400640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0003ef0 0x7ff0a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.022+0000 7ff0ba800640 1 -- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 msgr2=0x7ff0a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.022+0000 7ff0ba800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b40cf3e0 0x7ff0a0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.022+0000 7ff0ba800640 1 -- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0003ef0 msgr2=0x7ff0a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.022+0000 7ff0ba800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0a0003ef0 0x7ff0a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.022+0000 7ff0ba800640 1 -- 172.21.15.73:0/3329794934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0003520 con 0x7ff0a0004d00 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0ba800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0004d00 0x7ff0a0005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009a90 tx=0x7ff0a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52b11c2a88d14de server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac02c660 con 0x7ff0a0004d00 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac02c7c0 con 0x7ff0a0004d00 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0b8a00640 1 -- 172.21.15.73:0/3329794934 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac02cb70 con 0x7ff0a0004d00 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0bc800640 1 -- 172.21.15.73:0/3329794934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0004d00 msgr2=0x7ff0a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.023+0000 7ff0bc800640 1 --2- 172.21.15.73:0/3329794934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0004d00 0x7ff0a0005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009a90 tx=0x7ff0a00057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.024+0000 7ff0ba800640 1 -- 172.21.15.73:0/3329794934 reap_dead start 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.024+0000 7ff0bc800640 1 -- 172.21.15.73:0/3329794934 shutdown_connections 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.024+0000 7ff0bc800640 1 -- 172.21.15.73:0/3329794934 >> 172.21.15.73:0/3329794934 conn(0x7ff0b406a0d0 msgr2=0x7ff0b406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.024+0000 7ff0bc800640 1 -- 172.21.15.73:0/3329794934 shutdown_connections 2024-04-05T22:28:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.024+0000 7ff0bc800640 1 -- 172.21.15.73:0/3329794934 wait complete. 2024-04-05T22:28:27.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.025+0000 7ff0bc800640 1 Processor -- start 2024-04-05T22:28:27.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 -- start start 2024-04-05T22:28:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a0004d00 0x7ff0b415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b41f43e0 0x7ff0b41f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b41f2770 con 0x7ff0b41f3080 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b41f49a0 con 0x7ff0b41f43e0 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.026+0000 7ff0bc800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b41f4c80 con 0x7ff0a0004d00 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36178/0 (socket says 172.21.15.73:36178) 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 -- 172.21.15.73:0/4024552821 learned_addr learned my addr 172.21.15.73:0/4024552821 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 -- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a0004d00 msgr2=0x7ff0b415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a0004d00 0x7ff0b415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 -- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b41f43e0 msgr2=0x7ff0b41f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b41f43e0 0x7ff0b41f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 -- 172.21.15.73:0/4024552821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b41f4f60 con 0x7ff0b41f3080 2024-04-05T22:28:27.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.027+0000 7ff0ba800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac037040 tx=0x7ff0ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.028+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03e850 con 0x7ff0b41f3080 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.028+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac03e9b0 con 0x7ff0b41f3080 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.028+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03ed60 con 0x7ff0b41f3080 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.028+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b41f5240 con 0x7ff0b41f3080 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.028+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41f5a40 con 0x7ff0b41f3080 2024-04-05T22:28:27.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.029+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b41f5760 con 0x7ff0b41f3080 2024-04-05T22:28:27.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.030+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7ff0ac034440 con 0x7ff0b41f3080 2024-04-05T22:28:27.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.031+0000 7ff0aa000640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 0x7ff09806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:27.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.031+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5357+0+0 (secure 0 0 0) 0x7ff0ac000e40 con 0x7ff0b41f3080 2024-04-05T22:28:27.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.031+0000 7ff0b9e00640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 0x7ff09806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:27.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.032+0000 7ff0b9e00640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 0x7ff09806d4e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0b406c730 tx=0x7ff0b0009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:27.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.035+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff0b41f5760 con 0x7ff0b41f3080 2024-04-05T22:28:27.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.228+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0b41f5760 con 0x7ff0b41f3080 2024-04-05T22:28:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.795+0000 7ff0aa000640 1 -- 172.21.15.73:0/4024552821 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0b41f5760 con 0x7ff0b41f3080 2024-04-05T22:28:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:28:27.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 msgr2=0x7ff09806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 0x7ff09806d4e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0b406c730 tx=0x7ff0b0009750 comp rx=0 tx=0).stop 2024-04-05T22:28:27.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 msgr2=0x7ff0b415d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:27.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac037040 tx=0x7ff0ac004730 comp rx=0 tx=0).stop 2024-04-05T22:28:27.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 shutdown_connections 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff09806b020 0x7ff09806d4e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0b41f3080 0x7ff0b415d990 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b41f43e0 0x7ff0b41f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.801+0000 7ff0bc800640 1 --2- 172.21.15.73:0/4024552821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0a0004d00 0x7ff0b415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.802+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 >> 172.21.15.73:0/4024552821 conn(0x7ff0b406a0d0 msgr2=0x7ff0b40ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.802+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 shutdown_connections 2024-04-05T22:28:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:27.802+0000 7ff0bc800640 1 -- 172.21.15.73:0/4024552821 wait complete. 2024-04-05T22:28:27.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:28:28.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.017+0000 7f19ad400640 1 Processor -- start 2024-04-05T22:28:28.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.018+0000 7f19ad400640 1 -- start start 2024-04-05T22:28:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.019+0000 7f19ad400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19a8072040 0x7f19a80713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.019+0000 7f19ad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.019+0000 7f19ad400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a815d450 con 0x7f19a815d070 2024-04-05T22:28:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.019+0000 7f19ad400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a8071dc0 con 0x7f19a8072040 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.019+0000 7f19ad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a806a760 con 0x7f19a8071930 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.020+0000 7f19a5e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19a8072040 0x7f19a80713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.020+0000 7f19a7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.020+0000 7f19a7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58060/0 (socket says 172.21.15.73:58060) 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.020+0000 7f19a7200640 1 -- 172.21.15.73:0/793264707 learned_addr learned my addr 172.21.15.73:0/793264707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.020+0000 7f19a6800640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f19a815d070 0x7f19a806d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:35604/0 (socket says 172.21.15.73:35604) 2024-04-05T22:28:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.021+0000 7f19a5e00640 1 -- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 msgr2=0x7f19a80ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.021+0000 7f19a5e00640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.021+0000 7f19a5e00640 1 -- 172.21.15.73:0/793264707 >> v1:172.21.15.157:6789/0 conn(0x7f19a815d070 legacy=0x7f19a806d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.021+0000 7f19a5e00640 1 -- 172.21.15.73:0/793264707 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a806aa40 con 0x7f19a8072040 2024-04-05T22:28:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.021+0000 7f19a7200640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:28:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.022+0000 7f19a5e00640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a80713f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19940097a0 tx=0x7f199402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b89073eb451c385 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.022+0000 7f19a5400640 1 -- 172.21.15.73:0/793264707 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199402d040 con 0x7f19a8072040 2024-04-05T22:28:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.022+0000 7f19a5400640 1 -- 172.21.15.73:0/793264707 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994031810 con 0x7f19a8072040 2024-04-05T22:28:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.022+0000 7f19a5400640 1 -- 172.21.15.73:0/793264707 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994031ba0 con 0x7f19a8072040 2024-04-05T22:28:28.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 -- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 msgr2=0x7f19a80713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a80713f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19940097a0 tx=0x7f199402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 -- 172.21.15.73:0/793264707 shutdown_connections 2024-04-05T22:28:28.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 --2- 172.21.15.73:0/793264707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a80713f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 -- 172.21.15.73:0/793264707 >> 172.21.15.73:0/793264707 conn(0x7f19a80b2c70 msgr2=0x7f19a80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:28.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 -- 172.21.15.73:0/793264707 shutdown_connections 2024-04-05T22:28:28.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.023+0000 7f19ad400640 1 -- 172.21.15.73:0/793264707 wait complete. 2024-04-05T22:28:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.025+0000 7f19ad400640 1 Processor -- start 2024-04-05T22:28:28.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.025+0000 7f19ad400640 1 -- start start 2024-04-05T22:28:28.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a81493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8156c20 con 0x7f19a815d070 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80c93b0 con 0x7f19a8072040 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19ad400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8154d10 con 0x7f19a8071930 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19a7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19a7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36190/0 (socket says 172.21.15.73:36190) 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19a7200640 1 -- 172.21.15.73:0/2115382417 learned_addr learned my addr 172.21.15.73:0/2115382417 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.026+0000 7f19a5e00640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a81493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a6800640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 -- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 msgr2=0x7f19a80c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 -- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 msgr2=0x7f19a81493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a81493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 -- 172.21.15.73:0/2115382417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0009b30 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19a7200640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f19a000efc0 tx=0x7f19a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0018070 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a000f040 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000ce00 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1994009400 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.027+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81552d0 con 0x7f19a815d070 2024-04-05T22:28:28.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.028+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a8156d60 con 0x7f19a815d070 2024-04-05T22:28:28.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.029+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f19a0014480 con 0x7f19a815d070 2024-04-05T22:28:28.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.030+0000 7f199ea00640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 0x7f198806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.030+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19a0089bd0 con 0x7f19a815d070 2024-04-05T22:28:28.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.031+0000 7f19a6800640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 0x7f198806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.032+0000 7f19a6800640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 0x7f198806d4b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1998000f60 tx=0x7f1998009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.035+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f19a8156d60 con 0x7f19a815d070 2024-04-05T22:28:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.215+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f19a8156d60 con 0x7f19a815d070 2024-04-05T22:28:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.221+0000 7f199ea00640 1 -- 172.21.15.73:0/2115382417 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19a8156d60 con 0x7f19a815d070 2024-04-05T22:28:28.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.223+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 msgr2=0x7f198806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.223+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 0x7f198806d4b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1998000f60 tx=0x7f1998009040 comp rx=0 tx=0).stop 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.223+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 msgr2=0x7f19a8154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.223+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f19a000efc0 tx=0x7f19a000c490 comp rx=0 tx=0).stop 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 shutdown_connections 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f198806aff0 0x7f198806d4b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19a815d070 0x7f19a8154270 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a8072040 0x7f19a81493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 --2- 172.21.15.73:0/2115382417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071930 0x7f19a80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 >> 172.21.15.73:0/2115382417 conn(0x7f19a80b2c70 msgr2=0x7f19a815f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:28.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 shutdown_connections 2024-04-05T22:28:28.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.224+0000 7f19ad400640 1 -- 172.21.15.73:0/2115382417 wait complete. 2024-04-05T22:28:28.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-05T22:28:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.434+0000 7f6c16800640 1 Processor -- start 2024-04-05T22:28:28.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.434+0000 7f6c16800640 1 -- start start 2024-04-05T22:28:28.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.435+0000 7f6c16800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6c101587d0 0x7f6c100cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.435+0000 7f6c16800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1006f5b0 con 0x7f6c1015e850 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.435+0000 7f6c16800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c10070410 con 0x7f6c100ccd20 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.435+0000 7f6c16800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c10158bb0 con 0x7f6c101587d0 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6c101587d0 0x7f6c100cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6c1015e850 0x7f6c100708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48542/0 (socket says 172.21.15.73:48542) 2024-04-05T22:28:28.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0fe00640 1 -- 172.21.15.73:0/2529656050 learned_addr learned my addr 172.21.15.73:0/2529656050 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0f400640 1 -- 172.21.15.73:0/2529656050 >> v1:172.21.15.157:6789/0 conn(0x7f6c100ccd20 legacy=0x7f6c1006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0f400640 1 -- 172.21.15.73:0/2529656050 >> v1:172.21.15.73:6789/0 conn(0x7f6c1015e850 legacy=0x7f6c100708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.436+0000 7f6c0f400640 1 -- 172.21.15.73:0/2529656050 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1006f800 con 0x7f6c101587d0 2024-04-05T22:28:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.437+0000 7f6c0f400640 1 --2- 172.21.15.73:0/2529656050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c101587d0 0x7f6c100cc7e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6c000097d0 tx=0x7f6c0002b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbbd7bf83b4b023 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.437+0000 7f6c0ea00640 1 -- 172.21.15.73:0/2529656050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00030710 con 0x7f6c101587d0 2024-04-05T22:28:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.437+0000 7f6c0ea00640 1 -- 172.21.15.73:0/2529656050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0002e050 con 0x7f6c101587d0 2024-04-05T22:28:28.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.437+0000 7f6c0ea00640 1 -- 172.21.15.73:0/2529656050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00030a90 con 0x7f6c101587d0 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 -- 172.21.15.73:0/2529656050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c101587d0 msgr2=0x7f6c100cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 --2- 172.21.15.73:0/2529656050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c101587d0 0x7f6c100cc7e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6c000097d0 tx=0x7f6c0002b6f0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 -- 172.21.15.73:0/2529656050 shutdown_connections 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 --2- 172.21.15.73:0/2529656050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c101587d0 0x7f6c100cc7e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 -- 172.21.15.73:0/2529656050 >> 172.21.15.73:0/2529656050 conn(0x7f6c100ce8e0 msgr2=0x7f6c100cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 -- 172.21.15.73:0/2529656050 shutdown_connections 2024-04-05T22:28:28.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.438+0000 7f6c16800640 1 -- 172.21.15.73:0/2529656050 wait complete. 2024-04-05T22:28:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.440+0000 7f6c16800640 1 Processor -- start 2024-04-05T22:28:28.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.440+0000 7f6c16800640 1 -- start start 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c101587d0 0x7f6c101efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c1015e850 0x7f6c10159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10143a20 con 0x7f6c1015a3b0 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1015a860 con 0x7f6c101587d0 2024-04-05T22:28:28.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c16800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101f0430 con 0x7f6c1015e850 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36214/0 (socket says 172.21.15.73:36214) 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 -- 172.21.15.73:0/3083892002 learned_addr learned my addr 172.21.15.73:0/3083892002 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 -- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c1015e850 msgr2=0x7f6c10159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c1015e850 0x7f6c10159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 -- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c101587d0 msgr2=0x7f6c101efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c101587d0 0x7f6c101efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.441+0000 7f6c0fe00640 1 -- 172.21.15.73:0/3083892002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00009400 con 0x7f6c1015a3b0 2024-04-05T22:28:28.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c0fe00640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ccd0 tx=0x7f6c0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08012400 con 0x7f6c1015a3b0 2024-04-05T22:28:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c08012560 con 0x7f6c1015a3b0 2024-04-05T22:28:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c080128d0 con 0x7f6c1015a3b0 2024-04-05T22:28:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10142da0 con 0x7f6c1015a3b0 2024-04-05T22:28:28.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.442+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c10143360 con 0x7f6c1015a3b0 2024-04-05T22:28:28.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.443+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c10066af0 con 0x7f6c1015a3b0 2024-04-05T22:28:28.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.444+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f6c08007070 con 0x7f6c1015a3b0 2024-04-05T22:28:28.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.445+0000 7f6c07400640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 0x7f6bec06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:28.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.445+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c08089730 con 0x7f6c1015a3b0 2024-04-05T22:28:28.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.445+0000 7f6c0f400640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 0x7f6bec06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:28.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.447+0000 7f6c0f400640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 0x7f6bec06d4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c00034000 tx=0x7f6c0002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:28.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.449+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6c08054730 con 0x7f6c1015a3b0 2024-04-05T22:28:28.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.711+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f6c10143b60 con 0x7f6c1015a3b0 2024-04-05T22:28:28.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.796+0000 7f6c07400640 1 -- 172.21.15.73:0/3083892002 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v385) v1 ==== 149+0+0 (secure 0 0 0) 0x7f6c10143b60 con 0x7f6c1015a3b0 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.798+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 msgr2=0x7f6bec06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.798+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 0x7f6bec06d4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c00034000 tx=0x7f6c0002f000 comp rx=0 tx=0).stop 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.798+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 msgr2=0x7f6c10153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ccd0 tx=0x7f6c0800a430 comp rx=0 tx=0).stop 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 shutdown_connections 2024-04-05T22:28:28.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6bec06aff0 0x7f6bec06d4b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c1015a3b0 0x7f6c10153ae0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c1015e850 0x7f6c10159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 --2- 172.21.15.73:0/3083892002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c101587d0 0x7f6c101efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 >> 172.21.15.73:0/3083892002 conn(0x7f6c100ce8e0 msgr2=0x7f6c1015ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 shutdown_connections 2024-04-05T22:28:28.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:28.799+0000 7f6c16800640 1 -- 172.21.15.73:0/3083892002 wait complete. 2024-04-05T22:28:28.818 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-05T22:28:28.819 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-05T22:28:28.819 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:28:28.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.026+0000 7f3c92c00640 1 Processor -- start 2024-04-05T22:28:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.027+0000 7f3c92c00640 1 -- start start 2024-04-05T22:28:29.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c92c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c8c1541a0 0x7f3c8c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c92c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c92c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c15aa60 con 0x7f3c8c15aef0 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c92c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c0b3b60 con 0x7f3c8c1541a0 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c92c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0701c0 con 0x7f3c8c06d710 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c8be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c8c1541a0 0x7f3c8c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.028+0000 7f3c90c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3c8c15aef0 0x7f3c8c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48556/0 (socket says 172.21.15.73:48556) 2024-04-05T22:28:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c90c00640 1 -- 172.21.15.73:0/3759901487 learned_addr learned my addr 172.21.15.73:0/3759901487 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c8be00640 1 -- 172.21.15.73:0/3759901487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 msgr2=0x7f3c8c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c8be00640 1 --2- 172.21.15.73:0/3759901487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c8be00640 1 -- 172.21.15.73:0/3759901487 >> v1:172.21.15.73:6789/0 conn(0x7f3c8c15aef0 legacy=0x7f3c8c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c8be00640 1 -- 172.21.15.73:0/3759901487 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0b3880 con 0x7f3c8c1541a0 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.029+0000 7f3c8b400640 1 -- 172.21.15.73:0/3759901487 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296855844 0 0) 0x7f3c8c15aa60 con 0x7f3c8c15aef0 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.030+0000 7f3c8be00640 1 --2- 172.21.15.73:0/3759901487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c0b3050 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a5fd1b12824f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.030+0000 7f3c8b400640 1 -- 172.21.15.73:0/3759901487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c031830 con 0x7f3c8c1541a0 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.030+0000 7f3c8b400640 1 -- 172.21.15.73:0/3759901487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c031990 con 0x7f3c8c1541a0 2024-04-05T22:28:29.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.030+0000 7f3c8b400640 1 -- 172.21.15.73:0/3759901487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c031d20 con 0x7f3c8c1541a0 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 -- 172.21.15.73:0/3759901487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 msgr2=0x7f3c8c0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 --2- 172.21.15.73:0/3759901487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c0b3050 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 -- 172.21.15.73:0/3759901487 shutdown_connections 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 --2- 172.21.15.73:0/3759901487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 --2- 172.21.15.73:0/3759901487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c0b3050 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 -- 172.21.15.73:0/3759901487 >> 172.21.15.73:0/3759901487 conn(0x7f3c8c0ce8e0 msgr2=0x7f3c8c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 -- 172.21.15.73:0/3759901487 shutdown_connections 2024-04-05T22:28:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.031+0000 7f3c92c00640 1 -- 172.21.15.73:0/3759901487 wait complete. 2024-04-05T22:28:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.033+0000 7f3c92c00640 1 Processor -- start 2024-04-05T22:28:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.033+0000 7f3c92c00640 1 -- start start 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c8c15aef0 0x7f3c8c1e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c151480 con 0x7f3c8c1541a0 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c156cd0 con 0x7f3c8c15aef0 2024-04-05T22:28:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c92c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c150900 con 0x7f3c8c06d710 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36240/0 (socket says 172.21.15.73:36240) 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 -- 172.21.15.73:0/514115253 learned_addr learned my addr 172.21.15.73:0/514115253 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c8be00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 -- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 msgr2=0x7f3c8c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c90c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c8c15aef0 0x7f3c8c1e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c14d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 -- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c8c15aef0 msgr2=0x7f3c8c1e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.034+0000 7f3c83400640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c8c15aef0 0x7f3c8c1e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c83400640 1 -- 172.21.15.73:0/514115253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c009400 con 0x7f3c8c1541a0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c83400640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400ba50 tx=0x7f3c7400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400cbe0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7400cd60 con 0x7f3c8c1541a0 2024-04-05T22:28:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74014680 con 0x7f3c8c1541a0 2024-04-05T22:28:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c150be0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.035+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c1511a0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.037+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c066af0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.037+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f3c7401d3b0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.038+0000 7f3c88c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 0x7f3c6806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.038+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3c740896c0 con 0x7f3c8c1541a0 2024-04-05T22:28:29.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.039+0000 7f3c8be00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 0x7f3c6806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.040+0000 7f3c8be00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 0x7f3c6806d4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c036000 tx=0x7f3c7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.042+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3c74054650 con 0x7f3c8c1541a0 2024-04-05T22:28:29.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.241+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c8c156e90 con 0x7f3c8c1541a0 2024-04-05T22:28:29.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.243+0000 7f3c88c00640 1 -- 172.21.15.73:0/514115253 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+12528 (secure 0 0 0) 0x7f3c74019090 con 0x7f3c8c1541a0 2024-04-05T22:28:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:29.244 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":113,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:27.775943+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6835/3290176955":"2024-04-06T22:28:24.314934+0000","172.21.15.73:6833/3290176955":"2024-04-06T22:28:24.314934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.246+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 msgr2=0x7f3c6806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.246+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 0x7f3c6806d4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c036000 tx=0x7f3c7c008030 comp rx=0 tx=0).stop 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.246+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 msgr2=0x7f3c8c1e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.246+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400ba50 tx=0x7f3c7400bf10 comp rx=0 tx=0).stop 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 shutdown_connections 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c6806aff0 0x7f3c6806d4b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c8c15aef0 0x7f3c8c1e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c8c1541a0 0x7f3c8c1e6780 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 --2- 172.21.15.73:0/514115253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c8c06d710 0x7f3c8c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 >> 172.21.15.73:0/514115253 conn(0x7f3c8c0ce8e0 msgr2=0x7f3c8c151d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:29.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 shutdown_connections 2024-04-05T22:28:29.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.247+0000 7f3c92c00640 1 -- 172.21.15.73:0/514115253 wait complete. 2024-04-05T22:28:29.265 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:28:29.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:28:29.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.473+0000 7fb648c00640 1 Processor -- start 2024-04-05T22:28:29.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.473+0000 7fb648c00640 1 -- start start 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.474+0000 7fb648c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb64406c9a0 0x7fb64406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.474+0000 7fb648c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb64406ba70 0x7fb644164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb648c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64406bea0 con 0x7fb64415abe0 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb648c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64415afc0 con 0x7fb64406c9a0 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb648c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64406f5b0 con 0x7fb64406ba70 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb64415abe0 0x7fb6440712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48570/0 (socket says 172.21.15.73:48570) 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb64406c9a0 0x7fb64406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641e00640 1 -- 172.21.15.73:0/2140329070 learned_addr learned my addr 172.21.15.73:0/2140329070 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb642800640 1 --2- 172.21.15.73:0/2140329070 >> v2:172.21.15.157:3300/0 conn(0x7fb64406ba70 0x7fb644164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb640a00640 1 -- 172.21.15.73:0/2140329070 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819026392 0 0) 0x7fb64406bea0 con 0x7fb64415abe0 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641400640 1 -- 172.21.15.73:0/2140329070 >> v2:172.21.15.157:3300/0 conn(0x7fb64406ba70 msgr2=0x7fb644164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641400640 1 --2- 172.21.15.73:0/2140329070 >> v2:172.21.15.157:3300/0 conn(0x7fb64406ba70 0x7fb644164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641400640 1 -- 172.21.15.73:0/2140329070 >> v1:172.21.15.73:6789/0 conn(0x7fb64415abe0 legacy=0x7fb6440712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.475+0000 7fb641400640 1 -- 172.21.15.73:0/2140329070 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64406f890 con 0x7fb64406c9a0 2024-04-05T22:28:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.477+0000 7fb641400640 1 --2- 172.21.15.73:0/2140329070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64406b530 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb64406c900 tx=0x7fb63002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2565873443895ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.477+0000 7fb640a00640 1 -- 172.21.15.73:0/2140329070 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630031710 con 0x7fb64406c9a0 2024-04-05T22:28:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.477+0000 7fb640a00640 1 -- 172.21.15.73:0/2140329070 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63002f050 con 0x7fb64406c9a0 2024-04-05T22:28:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.477+0000 7fb640a00640 1 -- 172.21.15.73:0/2140329070 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630031a90 con 0x7fb64406c9a0 2024-04-05T22:28:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 -- 172.21.15.73:0/2140329070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 msgr2=0x7fb64406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 --2- 172.21.15.73:0/2140329070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64406b530 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb64406c900 tx=0x7fb63002b7b0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 -- 172.21.15.73:0/2140329070 shutdown_connections 2024-04-05T22:28:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 --2- 172.21.15.73:0/2140329070 >> v2:172.21.15.157:3300/0 conn(0x7fb64406ba70 0x7fb644164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 --2- 172.21.15.73:0/2140329070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64406b530 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 -- 172.21.15.73:0/2140329070 >> 172.21.15.73:0/2140329070 conn(0x7fb644069cb0 msgr2=0x7fb64406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 -- 172.21.15.73:0/2140329070 shutdown_connections 2024-04-05T22:28:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.478+0000 7fb648c00640 1 -- 172.21.15.73:0/2140329070 wait complete. 2024-04-05T22:28:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.480+0000 7fb648c00640 1 Processor -- start 2024-04-05T22:28:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.480+0000 7fb648c00640 1 -- start start 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb64406ba70 0x7fb64414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 0x7fb6441f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb644150160 con 0x7fb64406c9a0 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64414f040 con 0x7fb64415abe0 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb648c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb644154cf0 con 0x7fb64406ba70 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb641400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb641400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36252/0 (socket says 172.21.15.73:36252) 2024-04-05T22:28:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb641400640 1 -- 172.21.15.73:0/2687594204 learned_addr learned my addr 172.21.15.73:0/2687594204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb641e00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb64406ba70 0x7fb64414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.481+0000 7fb642800640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 0x7fb6441f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 -- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb64406ba70 msgr2=0x7fb64414fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb64406ba70 0x7fb64414fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 -- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 msgr2=0x7fb6441f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 0x7fb6441f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 -- 172.21.15.73:0/2687594204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630009400 con 0x7fb64406c9a0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb642800640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 0x7fb6441f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb641400640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb630000c00 tx=0x7fb6300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630033910 con 0x7fb64406c9a0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630033a70 con 0x7fb64406c9a0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644154fd0 con 0x7fb64406c9a0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.482+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630033e20 con 0x7fb64406c9a0 2024-04-05T22:28:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.483+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb644155590 con 0x7fb64406c9a0 2024-04-05T22:28:29.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.484+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644154e30 con 0x7fb64406c9a0 2024-04-05T22:28:29.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.485+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fb63003a040 con 0x7fb64406c9a0 2024-04-05T22:28:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.486+0000 7fb63a000640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 0x7fb62406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.486+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6300aaef0 con 0x7fb64406c9a0 2024-04-05T22:28:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.487+0000 7fb641e00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 0x7fb62406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.488+0000 7fb641e00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 0x7fb62406d4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb634000f60 tx=0x7fb634009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.490+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb644154e30 con 0x7fb64406c9a0 2024-04-05T22:28:29.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.683+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb644154e30 con 0x7fb64406c9a0 2024-04-05T22:28:29.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.684+0000 7fb63a000640 1 -- 172.21.15.73:0/2687594204 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb644154e30 con 0x7fb64406c9a0 2024-04-05T22:28:29.684 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:28:29.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 msgr2=0x7fb62406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 0x7fb62406d4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb634000f60 tx=0x7fb634009040 comp rx=0 tx=0).stop 2024-04-05T22:28:29.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 msgr2=0x7fb64414eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb630000c00 tx=0x7fb6300046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 shutdown_connections 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb62406aff0 0x7fb62406d4b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb64415abe0 0x7fb6441f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64406c9a0 0x7fb64414eb00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 --2- 172.21.15.73:0/2687594204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb64406ba70 0x7fb64414fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 >> 172.21.15.73:0/2687594204 conn(0x7fb644069cb0 msgr2=0x7fb644070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 shutdown_connections 2024-04-05T22:28:29.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.687+0000 7fb648c00640 1 -- 172.21.15.73:0/2687594204 wait complete. 2024-04-05T22:28:29.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-05T22:28:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.915+0000 7f4eba200640 1 Processor -- start 2024-04-05T22:28:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.916+0000 7f4eba200640 1 -- start start 2024-04-05T22:28:29.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eba200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4eb415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eba200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb40b2420 con 0x7f4eb415ccd0 2024-04-05T22:28:29.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eba200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb40ce380 con 0x7f4eb40d35e0 2024-04-05T22:28:29.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eba200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40b2770 con 0x7f4eb40d1600 2024-04-05T22:28:29.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eb8e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4eb415ccd0 0x7f4eb40d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48574/0 (socket says 172.21.15.73:48574) 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.917+0000 7f4eb8e00640 1 -- 172.21.15.73:0/1721381294 learned_addr learned my addr 172.21.15.73:0/1721381294 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916689331 0 0) 0x7f4eb40b2420 con 0x7f4eb415ccd0 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4eb415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e98003520 con 0x7f4eb415ccd0 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3739647964 0 0) 0x7f4e98003520 con 0x7f4eb415ccd0 2024-04-05T22:28:29.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 msgr2=0x7f4eb415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4eb415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 >> v1:172.21.15.157:6789/0 conn(0x7f4eb40d35e0 legacy=0x7f4eb40b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40ce660 con 0x7f4eb415ccd0 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.918+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4ea40019f0 con 0x7f4eb415ccd0 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 >> v1:172.21.15.73:6789/0 conn(0x7f4eb415ccd0 legacy=0x7f4eb40d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980039a0 0x7f4e98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4e980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e98004d00 0x7f4e98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40b2770 con 0x7f4e980039a0 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9800bab0 con 0x7f4e98004d00 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9800bc70 con 0x7f4eb40d1600 2024-04-05T22:28:29.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980039a0 0x7f4e98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4e980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.919+0000 7f4eb9800640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e98004d00 0x7f4e98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb8e00640 1 -- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 msgr2=0x7f4e980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4e980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb8e00640 1 -- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e98004d00 msgr2=0x7f4e98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:29.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e98004d00 0x7f4e98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb8e00640 1 -- 172.21.15.73:0/1721381294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98003520 con 0x7f4e980039a0 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.920+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb40d1600 0x7f4e980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980039a0 0x7f4e98003dc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400a830 tx=0x7f4e980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f521454ae43ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40072d0 con 0x7f4e980039a0 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea402f070 con 0x7f4e980039a0 2024-04-05T22:28:29.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eb3400640 1 -- 172.21.15.73:0/1721381294 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4007690 con 0x7f4e980039a0 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eba200640 1 -- 172.21.15.73:0/1721381294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980039a0 msgr2=0x7f4e98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eba200640 1 --2- 172.21.15.73:0/1721381294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980039a0 0x7f4e98003dc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400a830 tx=0x7f4e980057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.921+0000 7f4eb9800640 1 -- 172.21.15.73:0/1721381294 reap_dead start 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.922+0000 7f4eba200640 1 -- 172.21.15.73:0/1721381294 shutdown_connections 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.922+0000 7f4eba200640 1 -- 172.21.15.73:0/1721381294 >> 172.21.15.73:0/1721381294 conn(0x7f4eb40d2ac0 msgr2=0x7f4eb40d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:29.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.922+0000 7f4eba200640 1 -- 172.21.15.73:0/1721381294 shutdown_connections 2024-04-05T22:28:29.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.922+0000 7f4eba200640 1 -- 172.21.15.73:0/1721381294 wait complete. 2024-04-05T22:28:29.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.923+0000 7f4eba200640 1 Processor -- start 2024-04-05T22:28:29.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.924+0000 7f4eba200640 1 -- start start 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.924+0000 7f4eba200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e980039a0 0x7f4eb41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eba200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eba200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb41f15f0 0x7f4eb41f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eba200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4155100 con 0x7f4eb415abb0 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eba200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40c65d0 con 0x7f4e980039a0 2024-04-05T22:28:29.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eba200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40c68b0 con 0x7f4eb41f15f0 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36284/0 (socket says 172.21.15.73:36284) 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 -- 172.21.15.73:0/1452559835 learned_addr learned my addr 172.21.15.73:0/1452559835 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 -- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb41f15f0 msgr2=0x7f4eb41f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb41f15f0 0x7f4eb41f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 -- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e980039a0 msgr2=0x7f4eb41589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e980039a0 0x7f4eb41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.925+0000 7f4eb3e00640 1 -- 172.21.15.73:0/1452559835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4157b10 con 0x7f4eb415abb0 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eb3e00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c011700 tx=0x7f4e9c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00f960 con 0x7f4eb415abb0 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00fac0 con 0x7f4eb415abb0 2024-04-05T22:28:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00fe70 con 0x7f4eb415abb0 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb4157df0 con 0x7f4eb415abb0 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.926+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb41f1f90 con 0x7f4eb415abb0 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.927+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb4067250 con 0x7f4eb415abb0 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.928+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f4e9c015460 con 0x7f4eb415abb0 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.929+0000 7f4eb0c00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 0x7f4e9006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.929+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e9c089f90 con 0x7f4eb415abb0 2024-04-05T22:28:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.930+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 0x7f4e9006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.931+0000 7f4eb8e00640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 0x7f4e9006d4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:29.933+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4e9c054e50 con 0x7f4eb415abb0 2024-04-05T22:28:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.199+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f4eb40b2a90 con 0x7f4eb415abb0 2024-04-05T22:28:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.808+0000 7f4eb0c00640 1 -- 172.21.15.73:0/1452559835 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v387) v1 ==== 151+0+0 (secure 0 0 0) 0x7f4e9c0220d0 con 0x7f4eb415abb0 2024-04-05T22:28:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 msgr2=0x7f4e9006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 0x7f4e9006d4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea402f000 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 msgr2=0x7f4eb406ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c011700 tx=0x7f4e9c011bc0 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 shutdown_connections 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e9006aff0 0x7f4e9006d4b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb41f15f0 0x7f4eb41f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eb415abb0 0x7f4eb406ace0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 --2- 172.21.15.73:0/1452559835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e980039a0 0x7f4eb41589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 >> 172.21.15.73:0/1452559835 conn(0x7f4eb40d2ac0 msgr2=0x7f4eb40b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 shutdown_connections 2024-04-05T22:28:30.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:30.811+0000 7f4eba200640 1 -- 172.21.15.73:0/1452559835 wait complete. 2024-04-05T22:28:30.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:28:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.046+0000 7f87ce000640 1 Processor -- start 2024-04-05T22:28:31.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.047+0000 7f87ce000640 1 -- start start 2024-04-05T22:28:31.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87ce000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87c8160580 0x7f87c80d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87ce000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87c80b31f0 0x7f87c80af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87ce000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c80d1d50 con 0x7f87c80d1900 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87ce000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c8160960 con 0x7f87c8160580 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87ce000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c80b3800 con 0x7f87c80b31f0 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87c7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87c8160580 0x7f87c80d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87c7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87c8160580 0x7f87c80d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36294/0 (socket says 172.21.15.73:36294) 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.048+0000 7f87c7400640 1 -- 172.21.15.73:0/1275455654 learned_addr learned my addr 172.21.15.73:0/1275455654 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.049+0000 7f87c7400640 1 -- 172.21.15.73:0/1275455654 >> v2:172.21.15.157:3300/0 conn(0x7f87c80b31f0 msgr2=0x7f87c80af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.049+0000 7f87c7e00640 1 --2- 172.21.15.73:0/1275455654 >> v2:172.21.15.157:3300/0 conn(0x7f87c80b31f0 0x7f87c80af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.049+0000 7f87c7400640 1 --2- 172.21.15.73:0/1275455654 >> v2:172.21.15.157:3300/0 conn(0x7f87c80b31f0 0x7f87c80af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.049+0000 7f87c7400640 1 -- 172.21.15.73:0/1275455654 >> v1:172.21.15.157:6789/0 conn(0x7f87c80d1900 legacy=0x7f87c80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.049+0000 7f87c7400640 1 -- 172.21.15.73:0/1275455654 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80b3ae0 con 0x7f87c8160580 2024-04-05T22:28:31.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.050+0000 7f87c7400640 1 --2- 172.21.15.73:0/1275455654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c8160580 0x7f87c80d13c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f87c808be50 tx=0x7f87b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aa2a10f8ba447e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.050+0000 7f87c6000640 1 -- 172.21.15.73:0/1275455654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80317e0 con 0x7f87c8160580 2024-04-05T22:28:31.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.050+0000 7f87c6000640 1 -- 172.21.15.73:0/1275455654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8031940 con 0x7f87c8160580 2024-04-05T22:28:31.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.050+0000 7f87c6000640 1 -- 172.21.15.73:0/1275455654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8031cd0 con 0x7f87c8160580 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 -- 172.21.15.73:0/1275455654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c8160580 msgr2=0x7f87c80d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 --2- 172.21.15.73:0/1275455654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c8160580 0x7f87c80d13c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f87c808be50 tx=0x7f87b802b880 comp rx=0 tx=0).stop 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 -- 172.21.15.73:0/1275455654 shutdown_connections 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 --2- 172.21.15.73:0/1275455654 >> v2:172.21.15.157:3300/0 conn(0x7f87c80b31f0 0x7f87c80af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 --2- 172.21.15.73:0/1275455654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c8160580 0x7f87c80d13c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 -- 172.21.15.73:0/1275455654 >> 172.21.15.73:0/1275455654 conn(0x7f87c80d2c60 msgr2=0x7f87c80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 -- 172.21.15.73:0/1275455654 shutdown_connections 2024-04-05T22:28:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.051+0000 7f87ce000640 1 -- 172.21.15.73:0/1275455654 wait complete. 2024-04-05T22:28:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.053+0000 7f87ce000640 1 Processor -- start 2024-04-05T22:28:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.053+0000 7f87ce000640 1 -- start start 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80d1900 0x7f87c80b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87c8160580 0x7f87c80b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80b74c0 con 0x7f87c80b31f0 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c81f3060 con 0x7f87c80d1900 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87ce000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c81f65d0 con 0x7f87c8160580 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36306/0 (socket says 172.21.15.73:36306) 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.054+0000 7f87c7400640 1 -- 172.21.15.73:0/4070854170 learned_addr learned my addr 172.21.15.73:0/4070854170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c6a00640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80d1900 0x7f87c80b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 -- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87c8160580 msgr2=0x7f87c80b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87c8160580 0x7f87c80b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 -- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80d1900 msgr2=0x7f87c80b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80d1900 0x7f87c80b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 -- 172.21.15.73:0/4070854170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80094d0 con 0x7f87c80b31f0 2024-04-05T22:28:31.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87c7400640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f87b802b8c0 tx=0x7f87b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80329c0 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8032b40 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c814e5f0 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80453f0 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.055+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c81f6b00 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.057+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c8068160 con 0x7f87c80b31f0 2024-04-05T22:28:31.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.057+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f87b804e450 con 0x7f87c80b31f0 2024-04-05T22:28:31.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.058+0000 7f87bf400640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 0x7f87a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.058+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f87b80ab970 con 0x7f87c80b31f0 2024-04-05T22:28:31.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.059+0000 7f87c6a00640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 0x7f87a406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.060+0000 7f87c6a00640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 0x7f87a406d4e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c80c94b0 tx=0x7f87b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.063+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f87b80767f0 con 0x7f87c80b31f0 2024-04-05T22:28:31.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.258+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f87c80d1d50 con 0x7f87c80b31f0 2024-04-05T22:28:31.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.266+0000 7f87bf400640 1 -- 172.21.15.73:0/4070854170 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87c80d1d50 con 0x7f87c80b31f0 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 msgr2=0x7f87a406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 0x7f87a406d4e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c80c94b0 tx=0x7f87b0009040 comp rx=0 tx=0).stop 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 msgr2=0x7f87c814e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f87b802b8c0 tx=0x7f87b80046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 shutdown_connections 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87a406b020 0x7f87a406d4e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87c8160580 0x7f87c80b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80d1900 0x7f87c80b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 --2- 172.21.15.73:0/4070854170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c80b31f0 0x7f87c814e0b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.269+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 >> 172.21.15.73:0/4070854170 conn(0x7f87c80d2c60 msgr2=0x7f87c80cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:31.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.270+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 shutdown_connections 2024-04-05T22:28:31.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.270+0000 7f87ce000640 1 -- 172.21.15.73:0/4070854170 wait complete. 2024-04-05T22:28:31.290 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:28:31.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:28:31.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:28:31.521 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:28:31.522 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:31.522 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:31.579 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:28:31.579 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:31.579 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:31.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:28:31.635 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:31.635 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:31.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:28:31.693 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:31.693 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:31.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:28:31.750 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:28:31.750 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:31.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.964+0000 7f93c2c00640 1 Processor -- start 2024-04-05T22:28:31.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.964+0000 7f93c2c00640 1 -- start start 2024-04-05T22:28:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.966+0000 7f93c2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 0x7f93bc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.966+0000 7f93c2c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93bc0ad6b0 0x7f93bc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.966+0000 7f93c2c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0ce5d0 con 0x7f93bc160620 2024-04-05T22:28:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.966+0000 7f93c2c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc0ce010 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.966+0000 7f93c2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc0ce2f0 con 0x7f93bc0b1870 2024-04-05T22:28:31.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.967+0000 7f93b3400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 0x7f93bc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.967+0000 7f93c0c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93bc0ad6b0 0x7f93bc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.967+0000 7f93b3400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 0x7f93bc0ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58150/0 (socket says 172.21.15.73:58150) 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.967+0000 7f93b3400640 1 -- 172.21.15.73:0/1168227690 learned_addr learned my addr 172.21.15.73:0/1168227690 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.968+0000 7f93c0c00640 1 -- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 msgr2=0x7f93bc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.968+0000 7f93c0c00640 1 --2- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 0x7f93bc0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.968+0000 7f93c0c00640 1 -- 172.21.15.73:0/1168227690 >> v1:172.21.15.157:6789/0 conn(0x7f93bc160620 legacy=0x7f93bc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.968+0000 7f93c0c00640 1 -- 172.21.15.73:0/1168227690 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc0ac970 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.969+0000 7f93c0c00640 1 --2- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc0cb2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009d90 tx=0x7f93b402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1552403440a1c433 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.969+0000 7f93bb400640 1 -- 172.21.15.73:0/1168227690 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b402cc50 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.969+0000 7f93bb400640 1 -- 172.21.15.73:0/1168227690 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b402cdb0 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.969+0000 7f93bb400640 1 -- 172.21.15.73:0/1168227690 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4034660 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 -- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 msgr2=0x7f93bc0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 --2- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc0cb2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009d90 tx=0x7f93b402bea0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 -- 172.21.15.73:0/1168227690 shutdown_connections 2024-04-05T22:28:31.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 --2- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc0cb2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 --2- 172.21.15.73:0/1168227690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc0b1870 0x7f93bc0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 -- 172.21.15.73:0/1168227690 >> 172.21.15.73:0/1168227690 conn(0x7f93bc0ce8e0 msgr2=0x7f93bc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:31.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 -- 172.21.15.73:0/1168227690 shutdown_connections 2024-04-05T22:28:31.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.970+0000 7f93c2c00640 1 -- 172.21.15.73:0/1168227690 wait complete. 2024-04-05T22:28:31.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.972+0000 7f93c2c00640 1 Processor -- start 2024-04-05T22:28:31.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.972+0000 7f93c2c00640 1 -- start start 2024-04-05T22:28:31.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 0x7f93bc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1590f0 con 0x7f93bc0b1870 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc0b0080 con 0x7f93bc0ad6b0 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93c2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc159800 con 0x7f93bc160620 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36310/0 (socket says 172.21.15.73:36310) 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.973+0000 7f93b3400640 1 -- 172.21.15.73:0/308649112 learned_addr learned my addr 172.21.15.73:0/308649112 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93c0c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 0x7f93bc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93bbe00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 -- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 msgr2=0x7f93bc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 0x7f93bc15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 -- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 msgr2=0x7f93bc144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 -- 172.21.15.73:0/308649112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009a70 con 0x7f93bc0b1870 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93c0c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 0x7f93bc15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93bbe00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b3400640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e8b0 tx=0x7f93a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400c940 con 0x7f93bc0b1870 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a400caa0 con 0x7f93bc0b1870 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1448a0 con 0x7f93bc0b1870 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.974+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400ce30 con 0x7f93bc0b1870 2024-04-05T22:28:31.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.975+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc158b30 con 0x7f93bc0b1870 2024-04-05T22:28:31.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.976+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc066af0 con 0x7f93bc0b1870 2024-04-05T22:28:31.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.977+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f93a4013470 con 0x7f93bc0b1870 2024-04-05T22:28:31.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.978+0000 7f93b8c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 0x7f939806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:31.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.978+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f93a40898f0 con 0x7f93bc0b1870 2024-04-05T22:28:31.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.978+0000 7f93bbe00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 0x7f939806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:31.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.979+0000 7f93bbe00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 0x7f939806d4b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000f60 tx=0x7f93ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:31.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:31.982+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f93a4054840 con 0x7f93bc0b1870 2024-04-05T22:28:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.212+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93bc1595d0 con 0x7f93bc0b1870 2024-04-05T22:28:32.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.213+0000 7f93b8c00640 1 -- 172.21.15.73:0/308649112 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f93a4017de0 con 0x7f93bc0b1870 2024-04-05T22:28:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:32.213 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2275719751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2275719751},{"type":"v1","addr":"172.21.15.157:6833","nonce":2275719751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5321,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3523491824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3523491824},{"type":"v1","addr":"172.21.15.157:6835","nonce":3523491824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/773338661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":773338661},{"type":"v1","addr":"172.21.15.73:6834","nonce":773338661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1674363716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1674363716},{"type":"v1","addr":"172.21.15.73:6835","nonce":1674363716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[]} 2024-04-05T22:28:32.213 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 66 2024-04-05T22:28:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 msgr2=0x7f939806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 0x7f939806d4b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000f60 tx=0x7f93ac009040 comp rx=0 tx=0).stop 2024-04-05T22:28:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 msgr2=0x7f93bc15e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e8b0 tx=0x7f93a400ed70 comp rx=0 tx=0).stop 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 shutdown_connections 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f939806aff0 0x7f939806d4b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93bc160620 0x7f93bc15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93bc0b1870 0x7f93bc15e360 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 --2- 172.21.15.73:0/308649112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93bc0ad6b0 0x7f93bc144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 >> 172.21.15.73:0/308649112 conn(0x7f93bc0ce8e0 msgr2=0x7f93bc0caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 shutdown_connections 2024-04-05T22:28:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:32.216+0000 7f93c2c00640 1 -- 172.21.15.73:0/308649112 wait complete. 2024-04-05T22:28:32.237 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:28:32.237 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:28:32.237 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:28:32.238 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:28:32.238 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:28:32.238 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:28:32.238 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:28:32.238 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:28:32.238 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:28:32.238 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:28:32.239 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:28:32.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:28:32.239 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:28:32.239 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:28:32.239 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:28:32.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:28:32.259 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:28:32.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:28:32.260 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:28:32.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:28:32.269 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:28:32.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:28:32.283 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:28:32.284 DEBUG:teuthology.parallel:result is None 2024-04-05T22:28:32.288 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:28:32.289 DEBUG:teuthology.parallel:result is None 2024-04-05T22:28:32.319 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:28:32.319 DEBUG:teuthology.parallel:result is None 2024-04-05T22:28:32.441 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:28:32.445 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:28:32.502 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:28:38.266 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:28:38.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:28:38.275 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:28:38.275 DEBUG:teuthology.parallel:result is None 2024-04-05T22:28:38.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:28:38.433 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:28:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.468+0000 7fb8fb400640 1 Processor -- start 2024-04-05T22:28:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.468+0000 7fb8fb400640 1 -- start start 2024-04-05T22:28:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8fb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8f40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8fb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40ad880 con 0x7fb8f40b1870 2024-04-05T22:28:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8fb400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40ce5d0 con 0x7fb8f40cb2f0 2024-04-05T22:28:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8fb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40ce010 con 0x7fb8f4160620 2024-04-05T22:28:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8ebe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb8f40b1870 0x7fb8f40ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44034/0 (socket says 172.21.15.73:44034) 2024-04-05T22:28:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.470+0000 7fb8ebe00640 1 -- 172.21.15.73:0/2562226305 learned_addr learned my addr 172.21.15.73:0/2562226305 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8f40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100753647 0 0) 0x7fb8f40ad880 con 0x7fb8f40b1870 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d8003520 con 0x7fb8f40b1870 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 11801467 0 0) 0x7fb8d8003520 con 0x7fb8f40b1870 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 msgr2=0x7fb8f40caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8f40caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 >> v1:172.21.15.157:6789/0 conn(0x7fb8f40cb2f0 legacy=0x7fb8f40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40ce2f0 con 0x7fb8f40b1870 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb8dc001670 con 0x7fb8f40b1870 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 >> v1:172.21.15.73:6789/0 conn(0x7fb8f40b1870 legacy=0x7fb8f40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8d8003f20 0x7fb8d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40ce010 con 0x7fb8d8004d00 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d800ba60 con 0x7fb8d8003f20 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.471+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d800bc20 con 0x7fb8f4160620 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8ebe00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8d8003f20 0x7fb8d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 -- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 msgr2=0x7fb8d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8d80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 -- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8d8003f20 msgr2=0x7fb8d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8d8003f20 0x7fb8d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f9400640 1 -- 172.21.15.73:0/2562226305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8003520 con 0x7fb8d8004d00 2024-04-05T22:28:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8ebe00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8d8003f20 0x7fb8d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.472+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f4160620 0x7fb8d80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.473+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8d8005180 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001e50 tx=0x7fb8d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8246025d7ba070f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.473+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec003b70 con 0x7fb8d8004d00 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.473+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec003cd0 con 0x7fb8d8004d00 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.473+0000 7fb8f3e00640 1 -- 172.21.15.73:0/2562226305 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec005310 con 0x7fb8d8004d00 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 -- 172.21.15.73:0/2562226305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 msgr2=0x7fb8d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2562226305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8d8005180 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001e50 tx=0x7fb8d80057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8f9400640 1 -- 172.21.15.73:0/2562226305 reap_dead start 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 -- 172.21.15.73:0/2562226305 shutdown_connections 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 -- 172.21.15.73:0/2562226305 >> 172.21.15.73:0/2562226305 conn(0x7fb8f40ce8e0 msgr2=0x7fb8f40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 -- 172.21.15.73:0/2562226305 shutdown_connections 2024-04-05T22:28:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.474+0000 7fb8fb400640 1 -- 172.21.15.73:0/2562226305 wait complete. 2024-04-05T22:28:38.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.476+0000 7fb8fb400640 1 Processor -- start 2024-04-05T22:28:38.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.476+0000 7fb8fb400640 1 -- start start 2024-04-05T22:28:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f40af430 0x7fb8f40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 0x7fb8f415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40afdf0 con 0x7fb8d8004d00 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41f5cb0 con 0x7fb8f415c3e0 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8fb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41f5f00 con 0x7fb8f40af430 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47054/0 (socket says 172.21.15.73:47054) 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.477+0000 7fb8f8a00640 1 -- 172.21.15.73:0/2989371637 learned_addr learned my addr 172.21.15.73:0/2989371637 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8ebe00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f40af430 0x7fb8f40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 0x7fb8f415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 -- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f40af430 msgr2=0x7fb8f40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f40af430 0x7fb8f40af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 -- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 msgr2=0x7fb8f415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 0x7fb8f415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 -- 172.21.15.73:0/2989371637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f41f61e0 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f9400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 0x7fb8f415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f8a00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40108e0 tx=0x7fb8e4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400eb00 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400ec60 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e401c400 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.478+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f41f64c0 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.479+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f4149230 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.480+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4066af0 con 0x7fb8d8004d00 2024-04-05T22:28:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.481+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fb8e401c560 con 0x7fb8d8004d00 2024-04-05T22:28:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.482+0000 7fb8f1600640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 0x7fb8d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.482+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8e4089cd0 con 0x7fb8d8004d00 2024-04-05T22:28:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.482+0000 7fb8ebe00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 0x7fb8d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.484+0000 7fb8ebe00640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 0x7fb8d006d4e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc001200 tx=0x7fb8dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:38.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-05T22:28:38.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.487+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb8e4054cd0 con 0x7fb8d8004d00 2024-04-05T22:28:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:38.696+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb8f4068160 con 0x7fb8d8004d00 2024-04-05T22:28:39.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:28:39.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:28:39.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.633+0000 7fb8f1600640 1 -- 172.21.15.73:0/2989371637 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb8e40582f0 con 0x7fb8d8004d00 2024-04-05T22:28:39.633 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.636+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 msgr2=0x7fb8d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.636+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 0x7fb8d006d4e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc001200 tx=0x7fb8dc009040 comp rx=0 tx=0).stop 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.636+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 msgr2=0x7fb8f4148cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.636+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40108e0 tx=0x7fb8e4010da0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 shutdown_connections 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb8d006b020 0x7fb8d006d4e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415c3e0 0x7fb8f415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f40af430 0x7fb8f40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 --2- 172.21.15.73:0/2989371637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8d8004d00 0x7fb8f4148cf0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 >> 172.21.15.73:0/2989371637 conn(0x7fb8f40ce8e0 msgr2=0x7fb8f415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 shutdown_connections 2024-04-05T22:28:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.637+0000 7fb8fb400640 1 -- 172.21.15.73:0/2989371637 wait complete. 2024-04-05T22:28:39.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:28:39.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.912+0000 7f95d1000640 1 Processor -- start 2024-04-05T22:28:39.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.912+0000 7f95d1000640 1 -- start start 2024-04-05T22:28:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95d1000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95cc0cb2f0 0x7f95cc0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95d1000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0cbc70 con 0x7f95cc160620 2024-04-05T22:28:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95d1000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0ad880 con 0x7f95cc0b1870 2024-04-05T22:28:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95d1000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0ce5d0 con 0x7f95cc0cb2f0 2024-04-05T22:28:39.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95c9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f95cc160620 0x7f95cc0caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44046/0 (socket says 172.21.15.73:44046) 2024-04-05T22:28:39.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.914+0000 7f95c9e00640 1 -- 172.21.15.73:0/1117575696 learned_addr learned my addr 172.21.15.73:0/1117575696 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95ca800640 1 --2- 172.21.15.73:0/1117575696 >> v2:172.21.15.157:3300/0 conn(0x7f95cc0cb2f0 0x7f95cc0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520396411 0 0) 0x7f95cc0cbc70 con 0x7f95cc160620 2024-04-05T22:28:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0003520 con 0x7f95cc160620 2024-04-05T22:28:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043887013 0 0) 0x7f95b0003520 con 0x7f95cc160620 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 >> v2:172.21.15.157:3300/0 conn(0x7f95cc0cb2f0 msgr2=0x7f95cc0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 --2- 172.21.15.73:0/1117575696 >> v2:172.21.15.157:3300/0 conn(0x7f95cc0cb2f0 0x7f95cc0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 >> v1:172.21.15.157:6789/0 conn(0x7f95cc0b1870 legacy=0x7f95cc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0ce010 con 0x7f95cc160620 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f95bc001670 con 0x7f95cc160620 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 >> v1:172.21.15.73:6789/0 conn(0x7f95cc160620 legacy=0x7f95cc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0003970 0x7f95b0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0ce5d0 con 0x7f95b0005180 2024-04-05T22:28:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b000c000 con 0x7f95b0003970 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.915+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b000c1c0 con 0x7f95b00047c0 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95c9e00640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0003970 0x7f95b0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95c3e00640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 -- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00047c0 msgr2=0x7f95b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 -- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0003970 msgr2=0x7f95b0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0003970 0x7f95b0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95ca800640 1 -- 172.21.15.73:0/1117575696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0003520 con 0x7f95b0005180 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95c9e00640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0003970 0x7f95b0003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.916+0000 7f95c3e00640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:39.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.917+0000 7f95ca800640 1 -- 172.21.15.73:0/1117575696 reap_dead start 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.917+0000 7f95ca800640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 0x7f95b000b980 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009fc0 tx=0x7f95b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf785d6ff09e23f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.917+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c402cc30 con 0x7f95b0005180 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.917+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c400f040 con 0x7f95b0005180 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.917+0000 7f95c9400640 1 -- 172.21.15.73:0/1117575696 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4003990 con 0x7f95b0005180 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 -- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 msgr2=0x7f95b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 0x7f95b000b980 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009fc0 tx=0x7f95b00057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 -- 172.21.15.73:0/1117575696 shutdown_connections 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 --2- 172.21.15.73:0/1117575696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 -- 172.21.15.73:0/1117575696 >> 172.21.15.73:0/1117575696 conn(0x7f95cc0ce8e0 msgr2=0x7f95cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 -- 172.21.15.73:0/1117575696 shutdown_connections 2024-04-05T22:28:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.918+0000 7f95d1000640 1 -- 172.21.15.73:0/1117575696 wait complete. 2024-04-05T22:28:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.920+0000 7f95d1000640 1 Processor -- start 2024-04-05T22:28:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.920+0000 7f95d1000640 1 -- start start 2024-04-05T22:28:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b0005180 0x7f95cc154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95cc157310 0x7f95cc0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15b6c0 con 0x7f95cc15c490 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc157b30 con 0x7f95cc157310 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95d1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc157850 con 0x7f95b0005180 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47084/0 (socket says 172.21.15.73:47084) 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.921+0000 7f95c3e00640 1 -- 172.21.15.73:0/2882369449 learned_addr learned my addr 172.21.15.73:0/2882369449 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c9e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b0005180 0x7f95cc154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95ca800640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95cc157310 0x7f95cc0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 -- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b0005180 msgr2=0x7f95cc154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b0005180 0x7f95cc154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 -- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95cc157310 msgr2=0x7f95cc0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95cc157310 0x7f95cc0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:39.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 -- 172.21.15.73:0/2882369449 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc15b9b0 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c3e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d590 tx=0x7f95b400da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007220 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b40073a0 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b40067d0 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc15bc90 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.922+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc15c910 con 0x7f95cc15c490 2024-04-05T22:28:39.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.924+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc066af0 con 0x7f95cc15c490 2024-04-05T22:28:39.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.925+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f95b4006930 con 0x7f95cc15c490 2024-04-05T22:28:39.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.925+0000 7f95c2000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 0x7f95a806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:39.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.925+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95b4089580 con 0x7f95cc15c490 2024-04-05T22:28:39.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.926+0000 7f95c9e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 0x7f95a806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:39.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.927+0000 7f95c9e00640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 0x7f95a806d4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001eb0 tx=0x7f95bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:39.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:39.930+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f95b4054410 con 0x7f95cc15c490 2024-04-05T22:28:40.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.197+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f95cc068160 con 0x7f95cc15c490 2024-04-05T22:28:40.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.198+0000 7f95c2000640 1 -- 172.21.15.73:0/2882369449 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f95b4057a30 con 0x7f95cc15c490 2024-04-05T22:28:40.198 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:40.198 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:40.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:28:40.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.199 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.201 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:28:40.203 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.205 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:28:40.206 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:28:40.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:28:40.208 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:28:40.209 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:28:40.210 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:28:40.211 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:40.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 msgr2=0x7f95a806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 0x7f95a806d4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001eb0 tx=0x7f95bc009040 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 msgr2=0x7f95cc156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d590 tx=0x7f95b400da50 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 shutdown_connections 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f95a806b020 0x7f95a806d4e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95cc157310 0x7f95cc0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95cc15c490 0x7f95cc156dd0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 --2- 172.21.15.73:0/2882369449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b0005180 0x7f95cc154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 >> 172.21.15.73:0/2882369449 conn(0x7f95cc0ce8e0 msgr2=0x7f95cc06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 shutdown_connections 2024-04-05T22:28:40.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.201+0000 7f95d1000640 1 -- 172.21.15.73:0/2882369449 wait complete. 2024-04-05T22:28:40.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:40.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.418+0000 7f76db200640 1 Processor -- start 2024-04-05T22:28:40.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.418+0000 7f76db200640 1 -- start start 2024-04-05T22:28:40.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76db200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76d41587d0 0x7f76d40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76db200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76d40ccd20 0x7f76d406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76db200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d406f5b0 con 0x7f76d406d0c0 2024-04-05T22:28:40.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76db200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d4070410 con 0x7f76d40ccd20 2024-04-05T22:28:40.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76db200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d4158bb0 con 0x7f76d41587d0 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.420+0000 7f76cbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76d40ccd20 0x7f76d406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76d40ccd20 0x7f76d406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47090/0 (socket says 172.21.15.73:47090) 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 -- 172.21.15.73:0/368416067 learned_addr learned my addr 172.21.15.73:0/368416067 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 -- 172.21.15.73:0/368416067 >> v2:172.21.15.157:3300/0 conn(0x7f76d41587d0 msgr2=0x7f76d40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 --2- 172.21.15.73:0/368416067 >> v2:172.21.15.157:3300/0 conn(0x7f76d41587d0 0x7f76d40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 -- 172.21.15.73:0/368416067 >> v1:172.21.15.157:6789/0 conn(0x7f76d406d0c0 legacy=0x7f76d415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.421+0000 7f76cbe00640 1 -- 172.21.15.73:0/368416067 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d406f800 con 0x7f76d40ccd20 2024-04-05T22:28:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.422+0000 7f76cbe00640 1 --2- 172.21.15.73:0/368416067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d40ccd20 0x7f76d406cb80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009870 tx=0x7f76bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5277ff3940e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76d3400640 1 -- 172.21.15.73:0/368416067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc031870 con 0x7f76d40ccd20 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76d3400640 1 -- 172.21.15.73:0/368416067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc0319d0 con 0x7f76d40ccd20 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76d3400640 1 -- 172.21.15.73:0/368416067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc031d60 con 0x7f76d40ccd20 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 -- 172.21.15.73:0/368416067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d40ccd20 msgr2=0x7f76d406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 --2- 172.21.15.73:0/368416067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d40ccd20 0x7f76d406cb80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009870 tx=0x7f76bc02b910 comp rx=0 tx=0).stop 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 -- 172.21.15.73:0/368416067 shutdown_connections 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 --2- 172.21.15.73:0/368416067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d40ccd20 0x7f76d406cb80 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 --2- 172.21.15.73:0/368416067 >> v2:172.21.15.157:3300/0 conn(0x7f76d41587d0 0x7f76d40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 -- 172.21.15.73:0/368416067 >> 172.21.15.73:0/368416067 conn(0x7f76d40ce8e0 msgr2=0x7f76d40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 -- 172.21.15.73:0/368416067 shutdown_connections 2024-04-05T22:28:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.423+0000 7f76db200640 1 -- 172.21.15.73:0/368416067 wait complete. 2024-04-05T22:28:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.425+0000 7f76db200640 1 Processor -- start 2024-04-05T22:28:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.425+0000 7f76db200640 1 -- start start 2024-04-05T22:28:40.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 0x7f76d40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76d40ace10 0x7f76d4149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40ad2a0 con 0x7f76d406d0c0 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d414b030 con 0x7f76d40ace10 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76db200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4149f90 con 0x7f76d41587d0 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47102/0 (socket says 172.21.15.73:47102) 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.426+0000 7f76d3e00640 1 -- 172.21.15.73:0/4093478085 learned_addr learned my addr 172.21.15.73:0/4093478085 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76cbe00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 0x7f76d40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d9200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76d40ace10 0x7f76d4149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 -- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 msgr2=0x7f76d40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 0x7f76d40ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 -- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76d40ace10 msgr2=0x7f76d4149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76d40ace10 0x7f76d4149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 -- 172.21.15.73:0/4093478085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0094d0 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76cbe00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 0x7f76d40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d3e00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b440 tx=0x7f76c400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.427+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4011890 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.428+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c40119f0 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.428+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4011da0 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.428+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d414a270 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.428+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d414a830 con 0x7f76d406d0c0 2024-04-05T22:28:40.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.429+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d4066af0 con 0x7f76d406d0c0 2024-04-05T22:28:40.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.430+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f76c401a400 con 0x7f76d406d0c0 2024-04-05T22:28:40.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.431+0000 7f76d0c00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 0x7f76b006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.431+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76c4089a60 con 0x7f76d406d0c0 2024-04-05T22:28:40.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.432+0000 7f76cbe00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 0x7f76b006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.433+0000 7f76cbe00640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 0x7f76b006d4b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76bc036000 tx=0x7f76bc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.436+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f76c4054970 con 0x7f76d406d0c0 2024-04-05T22:28:40.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.615+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76d406ce90 con 0x7f76d406d0c0 2024-04-05T22:28:40.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.617+0000 7f76d0c00640 1 -- 172.21.15.73:0/4093478085 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+12399 (secure 0 0 0) 0x7f76c4015020 con 0x7f76d406d0c0 2024-04-05T22:28:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:40.617 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":118,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:39.616390+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 msgr2=0x7f76b006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 0x7f76b006d4b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76bc036000 tx=0x7f76bc03c040 comp rx=0 tx=0).stop 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 msgr2=0x7f76d414f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b440 tx=0x7f76c400b900 comp rx=0 tx=0).stop 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 shutdown_connections 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76b006aff0 0x7f76b006d4b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76d40ace10 0x7f76d4149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76d41587d0 0x7f76d40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 --2- 172.21.15.73:0/4093478085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d406d0c0 0x7f76d414f000 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.619+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 >> 172.21.15.73:0/4093478085 conn(0x7f76d40ce8e0 msgr2=0x7f76d4159170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:40.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.620+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 shutdown_connections 2024-04-05T22:28:40.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.620+0000 7f76db200640 1 -- 172.21.15.73:0/4093478085 wait complete. 2024-04-05T22:28:40.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:28:40.639 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:28:40.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.849+0000 7f5b83600640 1 Processor -- start 2024-04-05T22:28:40.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.850+0000 7f5b83600640 1 -- start start 2024-04-05T22:28:40.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.851+0000 7f5b83600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.851+0000 7f5b83600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c15f260 con 0x7f5b7c1654e0 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.851+0000 7f5b83600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c06f5b0 con 0x7f5b7c0b1950 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.851+0000 7f5b83600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c070d20 con 0x7f5b7c15ee80 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.851+0000 7f5b80c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.852+0000 7f5b80c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47110/0 (socket says 172.21.15.73:47110) 2024-04-05T22:28:40.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.852+0000 7f5b80c00640 1 -- 172.21.15.73:0/1237507105 learned_addr learned my addr 172.21.15.73:0/1237507105 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.852+0000 7f5b80c00640 1 -- 172.21.15.73:0/1237507105 >> v1:172.21.15.157:6789/0 conn(0x7f5b7c0b1950 legacy=0x7f5b7c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.852+0000 7f5b80c00640 1 -- 172.21.15.73:0/1237507105 >> v1:172.21.15.73:6789/0 conn(0x7f5b7c1654e0 legacy=0x7f5b7c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.852+0000 7f5b80c00640 1 -- 172.21.15.73:0/1237507105 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c06fb70 con 0x7f5b7c15ee80 2024-04-05T22:28:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.853+0000 7f5b80c00640 1 --2- 172.21.15.73:0/1237507105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd12ddfbef2fd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.853+0000 7f5b7be00640 1 -- 172.21.15.73:0/1237507105 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64030860 con 0x7f5b7c15ee80 2024-04-05T22:28:40.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.853+0000 7f5b7be00640 1 -- 172.21.15.73:0/1237507105 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640309c0 con 0x7f5b7c15ee80 2024-04-05T22:28:40.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.853+0000 7f5b7be00640 1 -- 172.21.15.73:0/1237507105 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64030d50 con 0x7f5b7c15ee80 2024-04-05T22:28:40.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 -- 172.21.15.73:0/1237507105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c15ee80 msgr2=0x7f5b7c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 --2- 172.21.15.73:0/1237507105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b6402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 -- 172.21.15.73:0/1237507105 shutdown_connections 2024-04-05T22:28:40.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 --2- 172.21.15.73:0/1237507105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c0cecc0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 -- 172.21.15.73:0/1237507105 >> 172.21.15.73:0/1237507105 conn(0x7f5b7c069cb0 msgr2=0x7f5b7c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:40.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 -- 172.21.15.73:0/1237507105 shutdown_connections 2024-04-05T22:28:40.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.854+0000 7f5b83600640 1 -- 172.21.15.73:0/1237507105 wait complete. 2024-04-05T22:28:40.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.856+0000 7f5b83600640 1 Processor -- start 2024-04-05T22:28:40.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.856+0000 7f5b83600640 1 -- start start 2024-04-05T22:28:40.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b7c0b1950 0x7f5b7c14ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1502c0 con 0x7f5b7c1654e0 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c14cf70 con 0x7f5b7c15ee80 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b83600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c14ffe0 con 0x7f5b7c0b1950 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b81600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b81600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47122/0 (socket says 172.21.15.73:47122) 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.857+0000 7f5b81600640 1 -- 172.21.15.73:0/3131220102 learned_addr learned my addr 172.21.15.73:0/3131220102 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b80c00640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b7c0b1950 0x7f5b7c14ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b73e00640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 -- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b7c0b1950 msgr2=0x7f5b7c14ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b7c0b1950 0x7f5b7c14ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 -- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c15ee80 msgr2=0x7f5b7c1603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c1603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 -- 172.21.15.73:0/3131220102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640094d0 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b81600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c730 tx=0x7f5b7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74006d50 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c1fa480 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74003970 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74003d60 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.858+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c1faa40 con 0x7f5b7c1654e0 2024-04-05T22:28:40.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.859+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c1fa2e0 con 0x7f5b7c1654e0 2024-04-05T22:28:40.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.860+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f5b740050d0 con 0x7f5b7c1654e0 2024-04-05T22:28:40.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.861+0000 7f5b79600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 0x7f5b5806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:40.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.861+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b74089160 con 0x7f5b7c1654e0 2024-04-05T22:28:40.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.862+0000 7f5b80c00640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 0x7f5b5806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:40.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.863+0000 7f5b80c00640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 0x7f5b5806d4b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b64036000 tx=0x7f5b64008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:40.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:40.865+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5b7c1fa2e0 con 0x7f5b7c1654e0 2024-04-05T22:28:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.058+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b7c15f260 con 0x7f5b7c1654e0 2024-04-05T22:28:41.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.059+0000 7f5b79600640 1 -- 172.21.15.73:0/3131220102 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b7c15f260 con 0x7f5b7c1654e0 2024-04-05T22:28:41.059 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.061+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 msgr2=0x7f5b5806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.061+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 0x7f5b5806d4b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b64036000 tx=0x7f5b64008030 comp rx=0 tx=0).stop 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 msgr2=0x7f5b7c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c730 tx=0x7f5b7400cbf0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 shutdown_connections 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5b5806aff0 0x7f5b5806d4b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b7c1654e0 0x7f5b7c160900 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c15ee80 0x7f5b7c1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 --2- 172.21.15.73:0/3131220102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b7c0b1950 0x7f5b7c14ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 >> 172.21.15.73:0/3131220102 conn(0x7f5b7c069cb0 msgr2=0x7f5b7c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 shutdown_connections 2024-04-05T22:28:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.062+0000 7f5b83600640 1 -- 172.21.15.73:0/3131220102 wait complete. 2024-04-05T22:28:41.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:28:41.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.285+0000 7fe722600640 1 Processor -- start 2024-04-05T22:28:41.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.286+0000 7fe722600640 1 -- start start 2024-04-05T22:28:41.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.287+0000 7fe722600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 0x7fe71c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.287+0000 7fe722600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.287+0000 7fe722600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c165cf0 con 0x7fe71c164e50 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.287+0000 7fe722600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c158790 con 0x7fe71c161f90 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.287+0000 7fe722600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0b2820 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.288+0000 7fe71b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 0x7fe71c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.288+0000 7fe71be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.288+0000 7fe712a00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe71c164e50 0x7fe71c0b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:50646/0 (socket says 172.21.15.73:50646) 2024-04-05T22:28:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.288+0000 7fe712a00640 1 -- 172.21.15.73:0/2767217834 learned_addr learned my addr 172.21.15.73:0/2767217834 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.289+0000 7fe71be00640 1 -- 172.21.15.73:0/2767217834 >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 msgr2=0x7fe71c162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.289+0000 7fe71be00640 1 --2- 172.21.15.73:0/2767217834 >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 0x7fe71c162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.289+0000 7fe71be00640 1 -- 172.21.15.73:0/2767217834 >> v1:172.21.15.157:6789/0 conn(0x7fe71c164e50 legacy=0x7fe71c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.289+0000 7fe71be00640 1 -- 172.21.15.73:0/2767217834 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c165730 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.289+0000 7fe71b400640 1 --2- 172.21.15.73:0/2767217834 >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 0x7fe71c162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:28:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.290+0000 7fe71be00640 1 --2- 172.21.15.73:0/2767217834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0b3010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d90 tx=0x7fe71402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c63183e4a45335 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.290+0000 7fe71aa00640 1 -- 172.21.15.73:0/2767217834 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71402cc60 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe71aa00640 1 -- 172.21.15.73:0/2767217834 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71402cdc0 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe71aa00640 1 -- 172.21.15.73:0/2767217834 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714034660 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 -- 172.21.15.73:0/2767217834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 msgr2=0x7fe71c0b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 --2- 172.21.15.73:0/2767217834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0b3010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d90 tx=0x7fe71402beb0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 -- 172.21.15.73:0/2767217834 shutdown_connections 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 --2- 172.21.15.73:0/2767217834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0b3010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 --2- 172.21.15.73:0/2767217834 >> v2:172.21.15.157:3300/0 conn(0x7fe71c161f90 0x7fe71c162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 -- 172.21.15.73:0/2767217834 >> 172.21.15.73:0/2767217834 conn(0x7fe71c0ce8e0 msgr2=0x7fe71c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:41.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 -- 172.21.15.73:0/2767217834 shutdown_connections 2024-04-05T22:28:41.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.291+0000 7fe722600640 1 -- 172.21.15.73:0/2767217834 wait complete. 2024-04-05T22:28:41.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.293+0000 7fe722600640 1 Processor -- start 2024-04-05T22:28:41.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.293+0000 7fe722600640 1 -- start start 2024-04-05T22:28:41.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 0x7fe71c14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14b8a0 con 0x7fe71c164e50 2024-04-05T22:28:41.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14a220 con 0x7fe71c161f90 2024-04-05T22:28:41.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.294+0000 7fe722600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c149610 con 0x7fe71c0b2ba0 2024-04-05T22:28:41.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe712a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 0x7fe71c14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47128/0 (socket says 172.21.15.73:47128) 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 -- 172.21.15.73:0/3718912172 learned_addr learned my addr 172.21.15.73:0/3718912172 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 -- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 msgr2=0x7fe71c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 -- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 msgr2=0x7fe71c14a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 0x7fe71c14a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71be00640 1 -- 172.21.15.73:0/3718912172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009400 con 0x7fe71c164e50 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe71b400640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.295+0000 7fe712a00640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 0x7fe71c14a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe71be00640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe714009eb0 tx=0x7fe714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714040650 con 0x7fe71c164e50 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7140407b0 con 0x7fe71c164e50 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714040b60 con 0x7fe71c164e50 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe714009a70 con 0x7fe71c164e50 2024-04-05T22:28:41.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.296+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c149bd0 con 0x7fe71c164e50 2024-04-05T22:28:41.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.297+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c066af0 con 0x7fe71c164e50 2024-04-05T22:28:41.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.298+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fe714040cc0 con 0x7fe71c164e50 2024-04-05T22:28:41.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.299+0000 7fe713e00640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 0x7fe6f806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.299+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7140ac4c0 con 0x7fe71c164e50 2024-04-05T22:28:41.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.300+0000 7fe71b400640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 0x7fe6f806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.301+0000 7fe71b400640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 0x7fe6f806d4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009d20 tx=0x7fe70c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.303+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe7140773d0 con 0x7fe71c164e50 2024-04-05T22:28:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.498+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe71c068160 con 0x7fe71c164e50 2024-04-05T22:28:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.499+0000 7fe713e00640 1 -- 172.21.15.73:0/3718912172 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe714009450 con 0x7fe71c164e50 2024-04-05T22:28:41.499 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:41.499 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:28:41.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:41.500 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:28:41.500 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:28:41.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:41.500 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:28:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 msgr2=0x7fe6f806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 0x7fe6f806d4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009d20 tx=0x7fe70c008040 comp rx=0 tx=0).stop 2024-04-05T22:28:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 msgr2=0x7fe71c0ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe714009eb0 tx=0x7fe714004a40 comp rx=0 tx=0).stop 2024-04-05T22:28:41.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 shutdown_connections 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6f806aff0 0x7fe6f806d4b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c164e50 0x7fe71c0ac350 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c161f90 0x7fe71c14a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 --2- 172.21.15.73:0/3718912172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b2ba0 0x7fe71c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 >> 172.21.15.73:0/3718912172 conn(0x7fe71c0ce8e0 msgr2=0x7fe71c157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 shutdown_connections 2024-04-05T22:28:41.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.502+0000 7fe722600640 1 -- 172.21.15.73:0/3718912172 wait complete. 2024-04-05T22:28:41.520 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:28:41.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:28:41.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.736+0000 7f2270a00640 1 Processor -- start 2024-04-05T22:28:41.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.737+0000 7f2270a00640 1 -- start start 2024-04-05T22:28:41.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c15aa80 0x7f226c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f226c0714b0 0x7f226c06b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f226c06b9d0 0x7f226c0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0cd5f0 con 0x7f226c06b9d0 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0ccc50 con 0x7f226c0714b0 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.738+0000 7f2270a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0ccf30 con 0x7f226c15aa80 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f226c06b9d0 0x7f226c0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f226c06b9d0 0x7f226c0cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47144/0 (socket says 172.21.15.73:47144) 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 -- 172.21.15.73:0/2638803262 learned_addr learned my addr 172.21.15.73:0/2638803262 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226a800640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c15aa80 0x7f226c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f2269e00640 1 --2- 172.21.15.73:0/2638803262 >> v2:172.21.15.157:3300/0 conn(0x7f226c0714b0 0x7f226c06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 -- 172.21.15.73:0/2638803262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c15aa80 msgr2=0x7f226c06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c15aa80 0x7f226c06fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 -- 172.21.15.73:0/2638803262 >> v2:172.21.15.157:3300/0 conn(0x7f226c0714b0 msgr2=0x7f226c06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 --2- 172.21.15.73:0/2638803262 >> v2:172.21.15.157:3300/0 conn(0x7f226c0714b0 0x7f226c06b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.739+0000 7f226b200640 1 -- 172.21.15.73:0/2638803262 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0cd210 con 0x7f226c06b9d0 2024-04-05T22:28:41.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.740+0000 7f226b200640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c06b9d0 0x7f226c0cda10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f226c067930 tx=0x7f226402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d75aadefa89ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.741+0000 7f2269400640 1 -- 172.21.15.73:0/2638803262 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226402c950 con 0x7f226c06b9d0 2024-04-05T22:28:41.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.741+0000 7f2269400640 1 -- 172.21.15.73:0/2638803262 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226402cab0 con 0x7f226c06b9d0 2024-04-05T22:28:41.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.741+0000 7f2269400640 1 -- 172.21.15.73:0/2638803262 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226402ce40 con 0x7f226c06b9d0 2024-04-05T22:28:41.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.741+0000 7f2270a00640 1 -- 172.21.15.73:0/2638803262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c06b9d0 msgr2=0x7f226c0cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.741+0000 7f2270a00640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c06b9d0 0x7f226c0cda10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f226c067930 tx=0x7f226402bdb0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 -- 172.21.15.73:0/2638803262 shutdown_connections 2024-04-05T22:28:41.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c06b9d0 0x7f226c0cda10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 --2- 172.21.15.73:0/2638803262 >> v2:172.21.15.157:3300/0 conn(0x7f226c0714b0 0x7f226c06b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 --2- 172.21.15.73:0/2638803262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c15aa80 0x7f226c06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 -- 172.21.15.73:0/2638803262 >> 172.21.15.73:0/2638803262 conn(0x7f226c0b0360 msgr2=0x7f226c0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 -- 172.21.15.73:0/2638803262 shutdown_connections 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.742+0000 7f2270a00640 1 -- 172.21.15.73:0/2638803262 wait complete. 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.743+0000 7f2270a00640 1 Processor -- start 2024-04-05T22:28:41.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.743+0000 7f2270a00640 1 -- start start 2024-04-05T22:28:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c06b9d0 0x7f226c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f226c0714b0 0x7f226c14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0b37f0 con 0x7f226c15aa80 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c151360 con 0x7f226c0714b0 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.744+0000 7f2270a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c1580d0 con 0x7f226c06b9d0 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47146/0 (socket says 172.21.15.73:47146) 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 -- 172.21.15.73:0/3526787073 learned_addr learned my addr 172.21.15.73:0/3526787073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f2269e00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f226c0714b0 0x7f226c14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 -- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c06b9d0 msgr2=0x7f226c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c06b9d0 0x7f226c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 -- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f226c0714b0 msgr2=0x7f226c14d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f226c0714b0 0x7f226c14d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 -- 172.21.15.73:0/3526787073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009b30 con 0x7f226c15aa80 2024-04-05T22:28:41.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.745+0000 7f226b200640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2264036040 tx=0x7f2264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.746+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264009510 con 0x7f226c15aa80 2024-04-05T22:28:41.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.746+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226403e450 con 0x7f226c15aa80 2024-04-05T22:28:41.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.746+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226403e800 con 0x7f226c15aa80 2024-04-05T22:28:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.746+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c0b3510 con 0x7f226c15aa80 2024-04-05T22:28:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.746+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c158650 con 0x7f226c15aa80 2024-04-05T22:28:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.747+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c157f30 con 0x7f226c15aa80 2024-04-05T22:28:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.748+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f226403e960 con 0x7f226c15aa80 2024-04-05T22:28:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.749+0000 7f2262a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 0x7f224c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.749+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22640ac730 con 0x7f226c15aa80 2024-04-05T22:28:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.749+0000 7f226a800640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 0x7f224c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:41.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.750+0000 7f226a800640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 0x7f224c06d4b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f225c006fd0 tx=0x7f225c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.753+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f226c157f30 con 0x7f226c15aa80 2024-04-05T22:28:41.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:41.946+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f226c157f30 con 0x7f226c15aa80 2024-04-05T22:28:42.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.659+0000 7f2262a00640 1 -- 172.21.15.73:0/3526787073 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) v1 ==== 138+0+0 (secure 0 0 0) 0x7f226c157f30 con 0x7f226c15aa80 2024-04-05T22:28:42.660 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:28:42.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.665+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 msgr2=0x7f224c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.665+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 0x7f224c06d4b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f225c006fd0 tx=0x7f225c008040 comp rx=0 tx=0).stop 2024-04-05T22:28:42.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.665+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 msgr2=0x7f226c0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.665+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2264036040 tx=0x7f2264004a40 comp rx=0 tx=0).stop 2024-04-05T22:28:42.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 shutdown_connections 2024-04-05T22:28:42.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f224c06aff0 0x7f224c06d4b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c15aa80 0x7f226c0b29a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f226c0714b0 0x7f226c14d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 --2- 172.21.15.73:0/3526787073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f226c06b9d0 0x7f226c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 >> 172.21.15.73:0/3526787073 conn(0x7f226c0b0360 msgr2=0x7f226c0ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 shutdown_connections 2024-04-05T22:28:42.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.667+0000 7f2270a00640 1 -- 172.21.15.73:0/3526787073 wait complete. 2024-04-05T22:28:42.690 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:28:42.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.888+0000 7f4054400640 1 Processor -- start 2024-04-05T22:28:42.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.888+0000 7f4054400640 1 -- start start 2024-04-05T22:28:42.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4054400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f404c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4054400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c15afc0 con 0x7f404c06c9a0 2024-04-05T22:28:42.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4054400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c06f5b0 con 0x7f404c06bc40 2024-04-05T22:28:42.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4054400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c06f890 con 0x7f404c15abe0 2024-04-05T22:28:42.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4051000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f404c06c9a0 0x7f404c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44060/0 (socket says 172.21.15.73:44060) 2024-04-05T22:28:42.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4051000640 1 -- 172.21.15.73:0/1968710656 learned_addr learned my addr 172.21.15.73:0/1968710656 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:42.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.890+0000 7f4051a00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f404c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479503960 0 0) 0x7f404c15afc0 con 0x7f404c06c9a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4034003520 con 0x7f404c06c9a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2910923697 0 0) 0x7f4034003520 con 0x7f404c06c9a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 msgr2=0x7f404c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f404c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 >> v1:172.21.15.157:6789/0 conn(0x7f404c06bc40 legacy=0x7f404c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c06fb70 con 0x7f404c06c9a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f4038001670 con 0x7f404c06c9a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 >> v1:172.21.15.73:6789/0 conn(0x7f404c06c9a0 legacy=0x7f404c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f4034003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40340047c0 0x7f4034004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f4034005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c06f890 con 0x7f40340039a0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403400bb40 con 0x7f40340047c0 2024-04-05T22:28:42.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.891+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403400bd00 con 0x7f404c15abe0 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f4034003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051000640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40340047c0 0x7f4034004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4052400640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f4034005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 -- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 msgr2=0x7f4034005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f4034005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 -- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40340047c0 msgr2=0x7f4034004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40340047c0 0x7f4034004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4051a00640 1 -- 172.21.15.73:0/1968710656 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034003520 con 0x7f40340039a0 2024-04-05T22:28:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.892+0000 7f4052400640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c15abe0 0x7f4034005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:42.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.893+0000 7f4051a00640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f4034003dc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4040009ac0 tx=0x7f40340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe0af50c8abee52 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:42.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404002c6d0 con 0x7f40340039a0 2024-04-05T22:28:42.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404002c830 con 0x7f40340039a0 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f404be00640 1 -- 172.21.15.73:0/1968710656 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404002cbc0 con 0x7f40340039a0 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 -- 172.21.15.73:0/1968710656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 msgr2=0x7f4034003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 --2- 172.21.15.73:0/1968710656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f4034003dc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4040009ac0 tx=0x7f40340057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4052400640 1 -- 172.21.15.73:0/1968710656 reap_dead start 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 -- 172.21.15.73:0/1968710656 shutdown_connections 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 -- 172.21.15.73:0/1968710656 >> 172.21.15.73:0/1968710656 conn(0x7f404c069cb0 msgr2=0x7f404c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 -- 172.21.15.73:0/1968710656 shutdown_connections 2024-04-05T22:28:42.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.894+0000 7f4054400640 1 -- 172.21.15.73:0/1968710656 wait complete. 2024-04-05T22:28:42.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.896+0000 7f4054400640 1 Processor -- start 2024-04-05T22:28:42.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.896+0000 7f4054400640 1 -- start start 2024-04-05T22:28:42.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c0c4fe0 0x7f404c1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f404c0b24a0 0x7f404c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0b2e90 con 0x7f40340039a0 2024-04-05T22:28:42.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1574c0 con 0x7f404c0b24a0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.897+0000 7f4054400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1577a0 con 0x7f404c0c4fe0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47164/0 (socket says 172.21.15.73:47164) 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 -- 172.21.15.73:0/2628555812 learned_addr learned my addr 172.21.15.73:0/2628555812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051000640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c0c4fe0 0x7f404c1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4052400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f404c0b24a0 0x7f404c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 -- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c0c4fe0 msgr2=0x7f404c1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c0c4fe0 0x7f404c1f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 -- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f404c0b24a0 msgr2=0x7f404c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f404c0b24a0 0x7f404c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 -- 172.21.15.73:0/2628555812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c157a80 con 0x7f40340039a0 2024-04-05T22:28:42.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.898+0000 7f4051a00640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f404002c270 tx=0x7f4040004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:42.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.899+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040003810 con 0x7f40340039a0 2024-04-05T22:28:42.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.899+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040003970 con 0x7f40340039a0 2024-04-05T22:28:42.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.899+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c157d60 con 0x7f40340039a0 2024-04-05T22:28:42.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.899+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040003ce0 con 0x7f40340039a0 2024-04-05T22:28:42.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.899+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c0c5440 con 0x7f40340039a0 2024-04-05T22:28:42.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.900+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c0c5630 con 0x7f40340039a0 2024-04-05T22:28:42.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.901+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f4040051480 con 0x7f40340039a0 2024-04-05T22:28:42.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.902+0000 7f4049600640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 0x7f402c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:42.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.902+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f40400abc70 con 0x7f40340039a0 2024-04-05T22:28:42.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.903+0000 7f4051000640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 0x7f402c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:42.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.904+0000 7f4051000640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 0x7f402c06d4e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4038001120 tx=0x7f4038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:42.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:42.906+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4040076cf0 con 0x7f40340039a0 2024-04-05T22:28:43.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.097+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f404c068160 con 0x7f40340039a0 2024-04-05T22:28:43.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.668+0000 7f4049600640 1 -- 172.21.15.73:0/2628555812 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) v1 ==== 170+0+0 (secure 0 0 0) 0x7f40400044e0 con 0x7f40340039a0 2024-04-05T22:28:43.669 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:28:43.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.675+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 msgr2=0x7f402c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:43.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.675+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 0x7f402c06d4e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4038001120 tx=0x7f4038009040 comp rx=0 tx=0).stop 2024-04-05T22:28:43.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.675+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 msgr2=0x7f404c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:43.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.675+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f404002c270 tx=0x7f4040004730 comp rx=0 tx=0).stop 2024-04-05T22:28:43.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 shutdown_connections 2024-04-05T22:28:43.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f402c06b020 0x7f402c06d4e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f404c0b24a0 0x7f404c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f404c0c4fe0 0x7f404c1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 --2- 172.21.15.73:0/2628555812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40340039a0 0x7f404c156e70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.677+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 >> 172.21.15.73:0/2628555812 conn(0x7f404c069cb0 msgr2=0x7f404c0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:43.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.678+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 shutdown_connections 2024-04-05T22:28:43.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.679+0000 7f4054400640 1 -- 172.21.15.73:0/2628555812 wait complete. 2024-04-05T22:28:43.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:28:43.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.888+0000 7f27ad000640 1 Processor -- start 2024-04-05T22:28:43.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.888+0000 7f27ad000640 1 -- start start 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.889+0000 7f27ad000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a815ee80 0x7f27a80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.889+0000 7f27ad000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27a80b1950 0x7f27a8164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.889+0000 7f27ad000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27a8165310 0x7f27a8161850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.889+0000 7f27ad000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a81657c0 con 0x7f27a80b1950 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27ad000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a8070d20 con 0x7f27a8165310 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27ad000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a815f260 con 0x7f27a815ee80 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27a80b1950 0x7f27a8164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27a80b1950 0x7f27a8164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47168/0 (socket says 172.21.15.73:47168) 2024-04-05T22:28:43.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 -- 172.21.15.73:0/687319251 learned_addr learned my addr 172.21.15.73:0/687319251 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a6800640 1 --2- 172.21.15.73:0/687319251 >> v2:172.21.15.157:3300/0 conn(0x7f27a8165310 0x7f27a8161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5e00640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a815ee80 0x7f27a80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 -- 172.21.15.73:0/687319251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a815ee80 msgr2=0x7f27a80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a815ee80 0x7f27a80cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 -- 172.21.15.73:0/687319251 >> v2:172.21.15.157:3300/0 conn(0x7f27a8165310 msgr2=0x7f27a8161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:43.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 --2- 172.21.15.73:0/687319251 >> v2:172.21.15.157:3300/0 conn(0x7f27a8165310 0x7f27a8161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.890+0000 7f27a5400640 1 -- 172.21.15.73:0/687319251 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a806f5b0 con 0x7f27a80b1950 2024-04-05T22:28:43.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.892+0000 7f27a5400640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a8164dd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f27a80cf710 tx=0x7f279402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14d86f72c7ccf94 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:43.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.892+0000 7f27a4a00640 1 -- 172.21.15.73:0/687319251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279402ca70 con 0x7f27a80b1950 2024-04-05T22:28:43.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.892+0000 7f27a4a00640 1 -- 172.21.15.73:0/687319251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279402cbd0 con 0x7f27a80b1950 2024-04-05T22:28:43.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.892+0000 7f27a4a00640 1 -- 172.21.15.73:0/687319251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940343f0 con 0x7f27a80b1950 2024-04-05T22:28:43.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 -- 172.21.15.73:0/687319251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 msgr2=0x7f27a8164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:43.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a8164dd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f27a80cf710 tx=0x7f279402bed0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 -- 172.21.15.73:0/687319251 shutdown_connections 2024-04-05T22:28:43.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a8164dd0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 --2- 172.21.15.73:0/687319251 >> v2:172.21.15.157:3300/0 conn(0x7f27a8165310 0x7f27a8161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 --2- 172.21.15.73:0/687319251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a815ee80 0x7f27a80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 -- 172.21.15.73:0/687319251 >> 172.21.15.73:0/687319251 conn(0x7f27a8069cb0 msgr2=0x7f27a806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 -- 172.21.15.73:0/687319251 shutdown_connections 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.893+0000 7f27ad000640 1 -- 172.21.15.73:0/687319251 wait complete. 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.894+0000 7f27ad000640 1 Processor -- start 2024-04-05T22:28:43.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.895+0000 7f27ad000640 1 -- start start 2024-04-05T22:28:43.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.895+0000 7f27ad000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.895+0000 7f27ad000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27a815ee80 0x7f27a80c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27ad000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a8165310 0x7f27a814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27ad000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81f9de0 con 0x7f27a80b1950 2024-04-05T22:28:43.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27ad000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80c6b60 con 0x7f27a815ee80 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27ad000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a814f4a0 con 0x7f27a8165310 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47170/0 (socket says 172.21.15.73:47170) 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 -- 172.21.15.73:0/3417705777 learned_addr learned my addr 172.21.15.73:0/3417705777 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 -- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a8165310 msgr2=0x7f27a814c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5400640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27a815ee80 0x7f27a80c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a8165310 0x7f27a814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 -- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27a815ee80 msgr2=0x7f27a80c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27a815ee80 0x7f27a80c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.896+0000 7f27a5e00640 1 -- 172.21.15.73:0/3417705777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794009b30 con 0x7f27a80b1950 2024-04-05T22:28:43.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f27a5e00640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f27a000d800 tx=0x7f27a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00096a0 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0009800 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0009b90 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a814f780 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.897+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a814ca90 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.898+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a8068160 con 0x7f27a80b1950 2024-04-05T22:28:43.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.899+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f27a0009cf0 con 0x7f27a80b1950 2024-04-05T22:28:43.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.900+0000 7f279e000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 0x7f278806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:43.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.900+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f27a0089ee0 con 0x7f27a80b1950 2024-04-05T22:28:43.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.901+0000 7f27a5400640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 0x7f278806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:43.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.902+0000 7f27a5400640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 0x7f278806d4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2794035000 tx=0x7f2794030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:43.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:43.905+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f27a0054b00 con 0x7f27a80b1950 2024-04-05T22:28:44.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.142+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f27a80c6d90 con 0x7f27a80b1950 2024-04-05T22:28:44.156 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 18 and data pool 19 2024-04-05T22:28:44.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.153+0000 7f279e000640 1 -- 172.21.15.73:0/3417705777 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7f27a000b440 con 0x7f27a80b1950 2024-04-05T22:28:44.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.158+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 msgr2=0x7f278806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.158+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 0x7f278806d4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2794035000 tx=0x7f2794030000 comp rx=0 tx=0).stop 2024-04-05T22:28:44.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.159+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 msgr2=0x7f27a814eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.159+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f27a000d800 tx=0x7f27a000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 shutdown_connections 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f278806aff0 0x7f278806d4b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27a8165310 0x7f27a814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27a815ee80 0x7f27a80c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 --2- 172.21.15.73:0/3417705777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27a80b1950 0x7f27a814eec0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 >> 172.21.15.73:0/3417705777 conn(0x7f27a8069cb0 msgr2=0x7f27a80ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:44.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 shutdown_connections 2024-04-05T22:28:44.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.162+0000 7f27ad000640 1 -- 172.21.15.73:0/3417705777 wait complete. 2024-04-05T22:28:44.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:44.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.367+0000 7f8c20600640 1 Processor -- start 2024-04-05T22:28:44.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.367+0000 7f8c20600640 1 -- start start 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c20600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c1815acc0 0x7f8c180ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c20600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1815f5f0 con 0x7f8c1815a8e0 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c20600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1815f350 con 0x7f8c180ca9f0 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c20600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180c3830 con 0x7f8c1815acc0 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c1dc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8c1815a8e0 0x7f8c180c3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44074/0 (socket says 172.21.15.73:44074) 2024-04-05T22:28:44.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c1dc00640 1 -- 172.21.15.73:0/336974730 learned_addr learned my addr 172.21.15.73:0/336974730 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009275587 0 0) 0x7f8c1815f5f0 con 0x7f8c1815a8e0 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c1d200640 1 --2- 172.21.15.73:0/336974730 >> v2:172.21.15.157:3300/0 conn(0x7f8c1815acc0 0x7f8c180ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.369+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bfc003520 con 0x7f8c1815a8e0 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1700341161 0 0) 0x7f8bfc003520 con 0x7f8c1815a8e0 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 >> v2:172.21.15.157:3300/0 conn(0x7f8c1815acc0 msgr2=0x7f8c180ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 --2- 172.21.15.73:0/336974730 >> v2:172.21.15.157:3300/0 conn(0x7f8c1815acc0 0x7f8c180ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 >> v1:172.21.15.157:6789/0 conn(0x7f8c180ca9f0 legacy=0x7f8c1815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180c3ad0 con 0x7f8c1815a8e0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8c08001670 con 0x7f8c1815a8e0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 >> v1:172.21.15.73:6789/0 conn(0x7f8c1815a8e0 legacy=0x7f8c180c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 0x7f8bfc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc0047c0 0x7f8bfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc0052a0 0x7f8bfc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180c3830 con 0x7f8bfc003970 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc00c170 con 0x7f8bfc0047c0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc00c330 con 0x7f8bfc0052a0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.370+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 0x7f8bfc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1d200640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc0047c0 0x7f8bfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1dc00640 1 -- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc0052a0 msgr2=0x7f8bfc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc0052a0 0x7f8bfc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1dc00640 1 -- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc0047c0 msgr2=0x7f8bfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:44.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bfc0047c0 0x7f8bfc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1dc00640 1 -- 172.21.15.73:0/336974730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc003520 con 0x7f8bfc003970 2024-04-05T22:28:44.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.371+0000 7f8c1e600640 1 -- 172.21.15.73:0/336974730 reap_dead start 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 0x7f8bfc003d90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007fd0 tx=0x7f8bfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e7e03bcac63acf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08005ad0 con 0x7f8bfc003970 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08005c30 con 0x7f8bfc003970 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c17e00640 1 -- 172.21.15.73:0/336974730 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0802ca90 con 0x7f8bfc003970 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c20600640 1 -- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 msgr2=0x7f8bfc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c20600640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 0x7f8bfc003d90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007fd0 tx=0x7f8bfc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c20600640 1 -- 172.21.15.73:0/336974730 shutdown_connections 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c20600640 1 --2- 172.21.15.73:0/336974730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bfc003970 0x7f8bfc003d90 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.372+0000 7f8c20600640 1 -- 172.21.15.73:0/336974730 >> 172.21.15.73:0/336974730 conn(0x7f8c180b0490 msgr2=0x7f8c180b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.373+0000 7f8c20600640 1 -- 172.21.15.73:0/336974730 shutdown_connections 2024-04-05T22:28:44.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.373+0000 7f8c20600640 1 -- 172.21.15.73:0/336974730 wait complete. 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.374+0000 7f8c20600640 1 Processor -- start 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.374+0000 7f8c20600640 1 -- start start 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc003970 0x7f8c180b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1814e7e0 0x7f8c180aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180cc5b0 con 0x7f8c1814e360 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180cc890 con 0x7f8c1814e7e0 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.375+0000 7f8c20600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180ccb70 con 0x7f8bfc003970 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47196/0 (socket says 172.21.15.73:47196) 2024-04-05T22:28:44.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 -- 172.21.15.73:0/1883732090 learned_addr learned my addr 172.21.15.73:0/1883732090 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:44.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc003970 0x7f8c180b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 -- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc003970 msgr2=0x7f8c180b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc003970 0x7f8c180b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 -- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1814e7e0 msgr2=0x7f8c180aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1814e7e0 0x7f8c180aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 -- 172.21.15.73:0/1883732090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180cce50 con 0x7f8c1814e360 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.376+0000 7f8c1d200640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f8c040108e0 tx=0x7f8c04010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.377+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400eb60 con 0x7f8c1814e360 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.377+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0400ecc0 con 0x7f8c1814e360 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.377+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0401c400 con 0x7f8c1814e360 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.377+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c180b2b10 con 0x7f8c1814e360 2024-04-05T22:28:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.377+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1815ce80 con 0x7f8c1814e360 2024-04-05T22:28:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.378+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18068160 con 0x7f8c1814e360 2024-04-05T22:28:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.379+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f8c0401c560 con 0x7f8c1814e360 2024-04-05T22:28:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.380+0000 7f8c15600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 0x7f8bf806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.380+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c0408a080 con 0x7f8c1814e360 2024-04-05T22:28:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.380+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 0x7f8bf806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.381+0000 7f8c1dc00640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 0x7f8bf806d4e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c08034000 tx=0x7f8c08009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.384+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8c040bd650 con 0x7f8c1814e360 2024-04-05T22:28:44.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.571+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c180ccfe0 con 0x7f8c1814e360 2024-04-05T22:28:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.574+0000 7f8c15600640 1 -- 172.21.15.73:0/1883732090 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16124 (secure 0 0 0) 0x7f8c04054d30 con 0x7f8c1814e360 2024-04-05T22:28:44.574 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:44.574 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":121,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:44.144681+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 msgr2=0x7f8bf806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 0x7f8bf806d4e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c08034000 tx=0x7f8c08009030 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 msgr2=0x7f8c1815c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f8c040108e0 tx=0x7f8c04010da0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 shutdown_connections 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bf806b020 0x7f8bf806d4e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1814e7e0 0x7f8c180aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c1814e360 0x7f8c1815c940 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 --2- 172.21.15.73:0/1883732090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bfc003970 0x7f8c180b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 >> 172.21.15.73:0/1883732090 conn(0x7f8c180b0490 msgr2=0x7f8c1815e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.576+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 shutdown_connections 2024-04-05T22:28:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.577+0000 7f8c20600640 1 -- 172.21.15.73:0/1883732090 wait complete. 2024-04-05T22:28:44.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:44.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.792+0000 7fc170e00640 1 Processor -- start 2024-04-05T22:28:44.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.792+0000 7fc170e00640 1 -- start start 2024-04-05T22:28:44.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.793+0000 7fc170e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc16c160620 0x7fc16c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc170e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0cb2f0 0x7fc16c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc170e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c0ad880 con 0x7fc16c0b1870 2024-04-05T22:28:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc170e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c0cbcc0 con 0x7fc16c160620 2024-04-05T22:28:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc170e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0ce5d0 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc169e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc16c160620 0x7fc16c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc16a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0cb2f0 0x7fc16c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc169400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc16c0b1870 0x7fc16c0ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:50672/0 (socket says 172.21.15.73:50672) 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc169e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc16c160620 0x7fc16c0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:40676/0 (socket says 172.21.15.73:40676) 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.794+0000 7fc169400640 1 -- 172.21.15.73:0/765438935 learned_addr learned my addr 172.21.15.73:0/765438935 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.795+0000 7fc169e00640 1 -- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0cb2f0 msgr2=0x7fc16c0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:44.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.795+0000 7fc169e00640 1 --2- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0cb2f0 0x7fc16c0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.795+0000 7fc169e00640 1 -- 172.21.15.73:0/765438935 >> v1:172.21.15.157:6789/0 conn(0x7fc16c0b1870 legacy=0x7fc16c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.795+0000 7fc169e00640 1 -- 172.21.15.73:0/765438935 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0ce010 con 0x7fc16c160620 2024-04-05T22:28:44.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.797+0000 7fc169e00640 1 --2- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0caa60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0097a0 tx=0x7fc15c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abce8d7c5c3eea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.797+0000 7fc168a00640 1 -- 172.21.15.73:0/765438935 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02d040 con 0x7fc16c160620 2024-04-05T22:28:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.797+0000 7fc168a00640 1 -- 172.21.15.73:0/765438935 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c031810 con 0x7fc16c160620 2024-04-05T22:28:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.797+0000 7fc168a00640 1 -- 172.21.15.73:0/765438935 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c031ba0 con 0x7fc16c160620 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 -- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 msgr2=0x7fc16c0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 --2- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0caa60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0097a0 tx=0x7fc15c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 -- 172.21.15.73:0/765438935 shutdown_connections 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 --2- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0cb2f0 0x7fc16c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 --2- 172.21.15.73:0/765438935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0caa60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 -- 172.21.15.73:0/765438935 >> 172.21.15.73:0/765438935 conn(0x7fc16c0ce8e0 msgr2=0x7fc16c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 -- 172.21.15.73:0/765438935 shutdown_connections 2024-04-05T22:28:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.798+0000 7fc170e00640 1 -- 172.21.15.73:0/765438935 wait complete. 2024-04-05T22:28:44.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.800+0000 7fc170e00640 1 Processor -- start 2024-04-05T22:28:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.800+0000 7fc170e00640 1 -- start start 2024-04-05T22:28:44.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0b1870 0x7fc16c1f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0c4500 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1f3780 con 0x7fc16c160620 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc170e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0c3880 con 0x7fc16c0b1870 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47210/0 (socket says 172.21.15.73:47210) 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 -- 172.21.15.73:0/742369583 learned_addr learned my addr 172.21.15.73:0/742369583 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0b1870 0x7fc16c1f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 -- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0b1870 msgr2=0x7fc16c1f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0b1870 0x7fc16c1f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 -- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 msgr2=0x7fc16c0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.801+0000 7fc169400640 1 -- 172.21.15.73:0/742369583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009400 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc169400640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc15800c450 tx=0x7fc15800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158005ce0 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158005e40 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0c3b60 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158007310 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.802+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0c4120 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.803+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c066af0 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.804+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fc158007470 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.805+0000 7fc162000640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 0x7fc14c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:44.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.805+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc158088fd0 con 0x7fc16c0cb2f0 2024-04-05T22:28:44.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.806+0000 7fc169e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 0x7fc14c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:44.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.807+0000 7fc169e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 0x7fc14c06d4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc15c036000 tx=0x7fc15c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:44.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:44.809+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc158053c60 con 0x7fc16c0cb2f0 2024-04-05T22:28:45.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.006+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc16c0c4640 con 0x7fc16c0cb2f0 2024-04-05T22:28:45.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.008+0000 7fc162000640 1 -- 172.21.15.73:0/742369583 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16124 (secure 0 0 0) 0x7fc16c0c4640 con 0x7fc16c0cb2f0 2024-04-05T22:28:45.008 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:45.008 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":121,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:44.144681+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 msgr2=0x7fc14c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 0x7fc14c06d4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc15c036000 tx=0x7fc15c03c040 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 msgr2=0x7fc16c15a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc15800c450 tx=0x7fc15800c910 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 shutdown_connections 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc14c06aff0 0x7fc14c06d4b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c160620 0x7fc16c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc16c0cb2f0 0x7fc16c15a8e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 --2- 172.21.15.73:0/742369583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc16c0b1870 0x7fc16c1f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 >> 172.21.15.73:0/742369583 conn(0x7fc16c0ce8e0 msgr2=0x7fc16c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.010+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 shutdown_connections 2024-04-05T22:28:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.011+0000 7fc170e00640 1 -- 172.21.15.73:0/742369583 wait complete. 2024-04-05T22:28:45.029 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:28:45.199 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-05T22:28:45.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.237+0000 7f5bcb000640 1 Processor -- start 2024-04-05T22:28:45.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.237+0000 7f5bcb000640 1 -- start start 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bcb000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5bc415aa70 0x7f5bc4070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bcb000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc415ae50 con 0x7f5bc40714a0 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bcb000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc415e3b0 con 0x7f5bc415dd90 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bcb000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc4071930 con 0x7f5bc415aa70 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bc3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5bc415aa70 0x7f5bc4070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bc3400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5bc40714a0 0x7f5bc415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44086/0 (socket says 172.21.15.73:44086) 2024-04-05T22:28:45.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bc3400640 1 -- 172.21.15.73:0/2927751220 learned_addr learned my addr 172.21.15.73:0/2927751220 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:45.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.239+0000 7f5bc2a00640 1 -- 172.21.15.73:0/2927751220 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007234880 0 0) 0x7f5bc415ae50 con 0x7f5bc40714a0 2024-04-05T22:28:45.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.240+0000 7f5bc3e00640 1 -- 172.21.15.73:0/2927751220 >> v1:172.21.15.157:6789/0 conn(0x7f5bc415dd90 legacy=0x7f5bc406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:45.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.240+0000 7f5bc3e00640 1 -- 172.21.15.73:0/2927751220 >> v1:172.21.15.73:6789/0 conn(0x7f5bc40714a0 legacy=0x7f5bc415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.240+0000 7f5bc3e00640 1 -- 172.21.15.73:0/2927751220 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40ac9e0 con 0x7f5bc415aa70 2024-04-05T22:28:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bc3e00640 1 --2- 172.21.15.73:0/2927751220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4070f60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40097a0 tx=0x7f5bb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436da7201a38fb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bc2a00640 1 -- 172.21.15.73:0/2927751220 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4031830 con 0x7f5bc415aa70 2024-04-05T22:28:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bc2a00640 1 -- 172.21.15.73:0/2927751220 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4031990 con 0x7f5bc415aa70 2024-04-05T22:28:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bc2a00640 1 -- 172.21.15.73:0/2927751220 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4031d20 con 0x7f5bc415aa70 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bcb000640 1 -- 172.21.15.73:0/2927751220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 msgr2=0x7f5bc4070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.241+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2927751220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4070f60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40097a0 tx=0x7f5bb402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.242+0000 7f5bcb000640 1 -- 172.21.15.73:0/2927751220 shutdown_connections 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.242+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2927751220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4070f60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.242+0000 7f5bcb000640 1 -- 172.21.15.73:0/2927751220 >> 172.21.15.73:0/2927751220 conn(0x7f5bc4069cb0 msgr2=0x7f5bc406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.242+0000 7f5bcb000640 1 -- 172.21.15.73:0/2927751220 shutdown_connections 2024-04-05T22:28:45.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.242+0000 7f5bcb000640 1 -- 172.21.15.73:0/2927751220 wait complete. 2024-04-05T22:28:45.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.243+0000 7f5bcb000640 1 Processor -- start 2024-04-05T22:28:45.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.243+0000 7f5bcb000640 1 -- start start 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40714a0 0x7f5bc41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc415dd90 0x7f5bc415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4159530 con 0x7f5bc415aa70 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40b0430 con 0x7f5bc415dd90 2024-04-05T22:28:45.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.244+0000 7f5bcb000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4156f20 con 0x7f5bc40714a0 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47218/0 (socket says 172.21.15.73:47218) 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 -- 172.21.15.73:0/2827220426 learned_addr learned my addr 172.21.15.73:0/2827220426 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 -- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40714a0 msgr2=0x7f5bc41568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc9000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc415dd90 0x7f5bc415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3e00640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40714a0 0x7f5bc41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40714a0 0x7f5bc41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 -- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc415dd90 msgr2=0x7f5bc415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc415dd90 0x7f5bc415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:45.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 -- 172.21.15.73:0/2827220426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4009400 con 0x7f5bc415aa70 2024-04-05T22:28:45.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.245+0000 7f5bc3400640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4d0 tx=0x7f5bb000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:45.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.246+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00074e0 con 0x7f5bc415aa70 2024-04-05T22:28:45.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.246+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0007660 con 0x7f5bc415aa70 2024-04-05T22:28:45.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.246+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004a20 con 0x7f5bc415aa70 2024-04-05T22:28:45.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.246+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc4157200 con 0x7f5bc415aa70 2024-04-05T22:28:45.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.246+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc41577c0 con 0x7f5bc415aa70 2024-04-05T22:28:45.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.247+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc4068160 con 0x7f5bc415aa70 2024-04-05T22:28:45.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.248+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f5bb0004b80 con 0x7f5bc415aa70 2024-04-05T22:28:45.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.249+0000 7f5bbbe00640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 0x7f5ba406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:45.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.249+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb0088c10 con 0x7f5bc415aa70 2024-04-05T22:28:45.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.249+0000 7f5bc3e00640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 0x7f5ba406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:45.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.251+0000 7f5bc3e00640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 0x7f5ba406d4b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4036000 tx=0x7f5bb4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:45.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.253+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5bb0053860 con 0x7f5bc415aa70 2024-04-05T22:28:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:45.484+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5bc406fc90 con 0x7f5bc415aa70 2024-04-05T22:28:46.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.185+0000 7f5bbbe00640 1 -- 172.21.15.73:0/2827220426 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5bb0018020 con 0x7f5bc415aa70 2024-04-05T22:28:46.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 msgr2=0x7f5ba406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:46.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 0x7f5ba406d4b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4036000 tx=0x7f5bb4008030 comp rx=0 tx=0).stop 2024-04-05T22:28:46.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 msgr2=0x7f5bc4158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4d0 tx=0x7f5bb000c990 comp rx=0 tx=0).stop 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 shutdown_connections 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5ba406aff0 0x7f5ba406d4b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc415dd90 0x7f5bc415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bc415aa70 0x7f5bc4158ff0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 --2- 172.21.15.73:0/2827220426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc40714a0 0x7f5bc41568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.188+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 >> 172.21.15.73:0/2827220426 conn(0x7f5bc4069cb0 msgr2=0x7f5bc415d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.189+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 shutdown_connections 2024-04-05T22:28:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.189+0000 7f5bcb000640 1 -- 172.21.15.73:0/2827220426 wait complete. 2024-04-05T22:28:46.208 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:28:46.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:28:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.408+0000 7f9b50600640 1 Processor -- start 2024-04-05T22:28:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.408+0000 7f9b50600640 1 -- start start 2024-04-05T22:28:46.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4815abe0 0x7f9b480712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b4806c9a0 0x7f9b4806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b4806ba70 0x7f9b480cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4806bec0 con 0x7f9b4806c9a0 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4815afc0 con 0x7f9b4806ba70 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b50600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4806f5b0 con 0x7f9b4815abe0 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b4806c9a0 0x7f9b4806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b4806c9a0 0x7f9b4806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48486/0 (socket says 172.21.15.73:48486) 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 -- 172.21.15.73:0/4214104717 learned_addr learned my addr 172.21.15.73:0/4214104717 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 -- 172.21.15.73:0/4214104717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4815abe0 msgr2=0x7f9b480712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4dc00640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4815abe0 0x7f9b480712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4e600640 1 --2- 172.21.15.73:0/4214104717 >> v2:172.21.15.157:3300/0 conn(0x7f9b4806ba70 0x7f9b480cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4815abe0 0x7f9b480712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 -- 172.21.15.73:0/4214104717 >> v2:172.21.15.157:3300/0 conn(0x7f9b4806ba70 msgr2=0x7f9b480cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:46.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 --2- 172.21.15.73:0/4214104717 >> v2:172.21.15.157:3300/0 conn(0x7f9b4806ba70 0x7f9b480cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.410+0000 7f9b4d200640 1 -- 172.21.15.73:0/4214104717 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4806f890 con 0x7f9b4806c9a0 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b4d200640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4806c9a0 0x7f9b4806b530 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9b48070bc0 tx=0x7f9b3402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5603d1715148fe23 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b47e00640 1 -- 172.21.15.73:0/4214104717 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34031740 con 0x7f9b4806c9a0 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b47e00640 1 -- 172.21.15.73:0/4214104717 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b340318a0 con 0x7f9b4806c9a0 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b47e00640 1 -- 172.21.15.73:0/4214104717 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34031c30 con 0x7f9b4806c9a0 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b50600640 1 -- 172.21.15.73:0/4214104717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4806c9a0 msgr2=0x7f9b4806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.412+0000 7f9b50600640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4806c9a0 0x7f9b4806b530 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9b48070bc0 tx=0x7f9b3402b7e0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 -- 172.21.15.73:0/4214104717 shutdown_connections 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4806c9a0 0x7f9b4806b530 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 --2- 172.21.15.73:0/4214104717 >> v2:172.21.15.157:3300/0 conn(0x7f9b4806ba70 0x7f9b480cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 --2- 172.21.15.73:0/4214104717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4815abe0 0x7f9b480712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 -- 172.21.15.73:0/4214104717 >> 172.21.15.73:0/4214104717 conn(0x7f9b48069cb0 msgr2=0x7f9b4806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 -- 172.21.15.73:0/4214104717 shutdown_connections 2024-04-05T22:28:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.413+0000 7f9b50600640 1 -- 172.21.15.73:0/4214104717 wait complete. 2024-04-05T22:28:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.414+0000 7f9b50600640 1 Processor -- start 2024-04-05T22:28:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.414+0000 7f9b50600640 1 -- start start 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4806ba70 0x7f9b48146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4806c9a0 0x7f9b48156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48154c50 con 0x7f9b4815abe0 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48159230 con 0x7f9b4806c9a0 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.415+0000 7f9b50600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48158f00 con 0x7f9b4806ba70 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48498/0 (socket says 172.21.15.73:48498) 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 -- 172.21.15.73:0/1551693816 learned_addr learned my addr 172.21.15.73:0/1551693816 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4dc00640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4806ba70 0x7f9b48146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 -- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4806ba70 msgr2=0x7f9b48146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4806ba70 0x7f9b48146230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 -- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4806c9a0 msgr2=0x7f9b48156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4806c9a0 0x7f9b48156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 -- 172.21.15.73:0/1551693816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b340094d0 con 0x7f9b4815abe0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.416+0000 7f9b4e600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cc30 tx=0x7f9b4000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.417+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000ce40 con 0x7f9b4815abe0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.417+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40003990 con 0x7f9b4815abe0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.417+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40003d20 con 0x7f9b4815abe0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.417+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b48154e30 con 0x7f9b4815abe0 2024-04-05T22:28:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.417+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b481516d0 con 0x7f9b4815abe0 2024-04-05T22:28:46.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.418+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48068160 con 0x7f9b4815abe0 2024-04-05T22:28:46.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.419+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f9b40003e80 con 0x7f9b4815abe0 2024-04-05T22:28:46.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.420+0000 7f9b45600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 0x7f9b2806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:46.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.420+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b40089fa0 con 0x7f9b4815abe0 2024-04-05T22:28:46.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.421+0000 7f9b4dc00640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 0x7f9b2806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:46.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.422+0000 7f9b4dc00640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 0x7f9b2806d4b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006fd0 tx=0x7f9b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:46.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.425+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9b40054a90 con 0x7f9b4815abe0 2024-04-05T22:28:46.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:46.640+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9b48159370 con 0x7f9b4815abe0 2024-04-05T22:28:47.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.189+0000 7f9b45600640 1 -- 172.21.15.73:0/1551693816 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b4001d070 con 0x7f9b4815abe0 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.192+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 msgr2=0x7f9b2806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 0x7f9b2806d4b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006fd0 tx=0x7f9b3c008040 comp rx=0 tx=0).stop 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 msgr2=0x7f9b480b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cc30 tx=0x7f9b4000a5c0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 shutdown_connections 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b2806aff0 0x7f9b2806d4b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b4815abe0 0x7f9b480b2ad0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4806c9a0 0x7f9b48156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 --2- 172.21.15.73:0/1551693816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b4806ba70 0x7f9b48146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 >> 172.21.15.73:0/1551693816 conn(0x7f9b48069cb0 msgr2=0x7f9b481502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 shutdown_connections 2024-04-05T22:28:47.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.193+0000 7f9b50600640 1 -- 172.21.15.73:0/1551693816 wait complete. 2024-04-05T22:28:47.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:28:47.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:47.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.419+0000 7f7cbac00640 1 Processor -- start 2024-04-05T22:28:47.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.419+0000 7f7cbac00640 1 -- start start 2024-04-05T22:28:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cbac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7cb415ef60 0x7f7cb406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cbac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb406f5b0 con 0x7f7cb41587d0 2024-04-05T22:28:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cbac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb4070410 con 0x7f7cb406cb80 2024-04-05T22:28:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cbac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb406d500 con 0x7f7cb415ef60 2024-04-05T22:28:47.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cb3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7cb41587d0 0x7f7cb40cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55404/0 (socket says 172.21.15.73:55404) 2024-04-05T22:28:47.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cb3e00640 1 -- 172.21.15.73:0/3613118382 learned_addr learned my addr 172.21.15.73:0/3613118382 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:47.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.421+0000 7f7cb8c00640 1 --2- 172.21.15.73:0/3613118382 >> v2:172.21.15.73:3300/0 conn(0x7f7cb415ef60 0x7f7cb406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.422+0000 7f7cb2a00640 1 -- 172.21.15.73:0/3613118382 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326530178 0 0) 0x7f7cb406f5b0 con 0x7f7cb41587d0 2024-04-05T22:28:47.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.422+0000 7f7cb2a00640 1 -- 172.21.15.73:0/3613118382 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c98003520 con 0x7f7cb41587d0 2024-04-05T22:28:47.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.422+0000 7f7cb8c00640 1 -- 172.21.15.73:0/3613118382 >> v1:172.21.15.157:6789/0 conn(0x7f7cb406cb80 legacy=0x7f7cb415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:47.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.422+0000 7f7cb8c00640 1 -- 172.21.15.73:0/3613118382 >> v1:172.21.15.73:6789/0 conn(0x7f7cb41587d0 legacy=0x7f7cb40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.422+0000 7f7cb8c00640 1 -- 172.21.15.73:0/3613118382 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40ccef0 con 0x7f7cb415ef60 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.423+0000 7f7cb8c00640 1 --2- 172.21.15.73:0/3613118382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb415ef60 0x7f7cb406cfc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009d90 tx=0x7f7cac02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f90379dec264fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.423+0000 7f7cb2a00640 1 -- 172.21.15.73:0/3613118382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac02cc60 con 0x7f7cb415ef60 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.423+0000 7f7cb2a00640 1 -- 172.21.15.73:0/3613118382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac02cdc0 con 0x7f7cb415ef60 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.423+0000 7f7cb2a00640 1 -- 172.21.15.73:0/3613118382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac034660 con 0x7f7cb415ef60 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 -- 172.21.15.73:0/3613118382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb415ef60 msgr2=0x7f7cb406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 --2- 172.21.15.73:0/3613118382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb415ef60 0x7f7cb406cfc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009d90 tx=0x7f7cac02beb0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 -- 172.21.15.73:0/3613118382 shutdown_connections 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 --2- 172.21.15.73:0/3613118382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb415ef60 0x7f7cb406cfc0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 -- 172.21.15.73:0/3613118382 >> 172.21.15.73:0/3613118382 conn(0x7f7cb40ce8e0 msgr2=0x7f7cb40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:47.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 -- 172.21.15.73:0/3613118382 shutdown_connections 2024-04-05T22:28:47.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.424+0000 7f7cbac00640 1 -- 172.21.15.73:0/3613118382 wait complete. 2024-04-05T22:28:47.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.426+0000 7f7cbac00640 1 Processor -- start 2024-04-05T22:28:47.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.426+0000 7f7cbac00640 1 -- start start 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb406cb80 0x7f7cb40c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cb415ef60 0x7f7cb4070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4148bc0 con 0x7f7cb41587d0 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4149700 con 0x7f7cb406cb80 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cbac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4148dd0 con 0x7f7cb415ef60 2024-04-05T22:28:47.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cb3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cb3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48506/0 (socket says 172.21.15.73:48506) 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cb3400640 1 -- 172.21.15.73:0/1821599999 learned_addr learned my addr 172.21.15.73:0/1821599999 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.427+0000 7f7cb3e00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb406cb80 0x7f7cb40c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 -- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cb415ef60 msgr2=0x7f7cb4070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cb415ef60 0x7f7cb4070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 -- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb406cb80 msgr2=0x7f7cb40c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb406cb80 0x7f7cb40c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 -- 172.21.15.73:0/1821599999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac009a70 con 0x7f7cb41587d0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cb3400640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c430 tx=0x7f7ca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011790 con 0x7f7cb41587d0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca00118f0 con 0x7f7cb41587d0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011ce0 con 0x7f7cb41587d0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb41490b0 con 0x7f7cb41587d0 2024-04-05T22:28:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.428+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40c6400 con 0x7f7cb41587d0 2024-04-05T22:28:47.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.429+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4066af0 con 0x7f7cb41587d0 2024-04-05T22:28:47.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.430+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f7ca001c460 con 0x7f7cb41587d0 2024-04-05T22:28:47.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.431+0000 7f7cabe00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 0x7f7c9406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.431+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ca00899d0 con 0x7f7cb41587d0 2024-04-05T22:28:47.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.432+0000 7f7cb3e00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 0x7f7c9406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.433+0000 7f7cb3e00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 0x7f7c9406d4e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40011e0 tx=0x7f7ca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.435+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7ca0054590 con 0x7f7cb41587d0 2024-04-05T22:28:47.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.671+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cb4149870 con 0x7f7cb41587d0 2024-04-05T22:28:47.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.672+0000 7f7cabe00640 1 -- 172.21.15.73:0/1821599999 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f7ca0057bb0 con 0x7f7cb41587d0 2024-04-05T22:28:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:47.673 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:28:47.673 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 73 2024-04-05T22:28:47.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 msgr2=0x7f7c9406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 0x7f7c9406d4e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40011e0 tx=0x7f7ca4009040 comp rx=0 tx=0).stop 2024-04-05T22:28:47.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 msgr2=0x7f7cb40708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c430 tx=0x7f7ca000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 shutdown_connections 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7c9406b020 0x7f7c9406d4e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cb415ef60 0x7f7cb4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb41587d0 0x7f7cb40708a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 --2- 172.21.15.73:0/1821599999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb406cb80 0x7f7cb40c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 >> 172.21.15.73:0/1821599999 conn(0x7f7cb40ce8e0 msgr2=0x7f7cb4159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 shutdown_connections 2024-04-05T22:28:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.675+0000 7f7cbac00640 1 -- 172.21.15.73:0/1821599999 wait complete. 2024-04-05T22:28:47.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:28:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.904+0000 7f8843c00640 1 Processor -- start 2024-04-05T22:28:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.904+0000 7f8843c00640 1 -- start start 2024-04-05T22:28:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8843c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f883c1587d0 0x7f883c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8843c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8843c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883c06f5b0 con 0x7f883c0ccd20 2024-04-05T22:28:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8843c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c15f240 con 0x7f883c1587d0 2024-04-05T22:28:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8843c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c070410 con 0x7f883c15e850 2024-04-05T22:28:47.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8841200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f883c1587d0 0x7f883c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8841200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f883c1587d0 0x7f883c0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48516/0 (socket says 172.21.15.73:48516) 2024-04-05T22:28:47.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8841200640 1 -- 172.21.15.73:0/3307285847 learned_addr learned my addr 172.21.15.73:0/3307285847 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:47.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.906+0000 7f8841c00640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.907+0000 7f8841200640 1 -- 172.21.15.73:0/3307285847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 msgr2=0x7f883c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:47.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.907+0000 7f8841200640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.907+0000 7f8841200640 1 -- 172.21.15.73:0/3307285847 >> v1:172.21.15.157:6789/0 conn(0x7f883c0ccd20 legacy=0x7f883c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.907+0000 7f8841200640 1 -- 172.21.15.73:0/3307285847 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c158bb0 con 0x7f883c1587d0 2024-04-05T22:28:47.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.907+0000 7f8841c00640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:47.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.908+0000 7f8841200640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c1587d0 0x7f883c0cc7e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8830009870 tx=0x7f883002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec767620f8ecf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.909+0000 7f883b400640 1 -- 172.21.15.73:0/3307285847 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830030840 con 0x7f883c1587d0 2024-04-05T22:28:47.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.909+0000 7f883b400640 1 -- 172.21.15.73:0/3307285847 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88300309a0 con 0x7f883c1587d0 2024-04-05T22:28:47.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.909+0000 7f883b400640 1 -- 172.21.15.73:0/3307285847 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830030d30 con 0x7f883c1587d0 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.909+0000 7f8843c00640 1 -- 172.21.15.73:0/3307285847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c1587d0 msgr2=0x7f883c0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.909+0000 7f8843c00640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c1587d0 0x7f883c0cc7e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8830009870 tx=0x7f883002b980 comp rx=0 tx=0).stop 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 -- 172.21.15.73:0/3307285847 shutdown_connections 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 --2- 172.21.15.73:0/3307285847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c1587d0 0x7f883c0cc7e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 -- 172.21.15.73:0/3307285847 >> 172.21.15.73:0/3307285847 conn(0x7f883c0ce8e0 msgr2=0x7f883c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:47.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 -- 172.21.15.73:0/3307285847 shutdown_connections 2024-04-05T22:28:47.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.910+0000 7f8843c00640 1 -- 172.21.15.73:0/3307285847 wait complete. 2024-04-05T22:28:47.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.912+0000 7f8843c00640 1 Processor -- start 2024-04-05T22:28:47.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.912+0000 7f8843c00640 1 -- start start 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1587d0 0x7f883c143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c15a790 con 0x7f883c159dc0 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c155190 con 0x7f883c1587d0 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8843c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c144160 con 0x7f883c15e850 2024-04-05T22:28:47.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48532/0 (socket says 172.21.15.73:48532) 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 -- 172.21.15.73:0/934228099 learned_addr learned my addr 172.21.15.73:0/934228099 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841200640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1587d0 0x7f883c143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f883be00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 -- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 msgr2=0x7f883c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 -- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1587d0 msgr2=0x7f883c143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1587d0 0x7f883c143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f8841c00640 1 -- 172.21.15.73:0/934228099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300094d0 con 0x7f883c159dc0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.913+0000 7f883be00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8841c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8834007df0 tx=0x7f883400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834003990 con 0x7f883c159dc0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883400f040 con 0x7f883c159dc0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c154eb0 con 0x7f883c159dc0 2024-04-05T22:28:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834003d30 con 0x7f883c159dc0 2024-04-05T22:28:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.914+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c141540 con 0x7f883c159dc0 2024-04-05T22:28:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.915+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c066af0 con 0x7f883c159dc0 2024-04-05T22:28:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.916+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f8834004030 con 0x7f883c159dc0 2024-04-05T22:28:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.917+0000 7f8838c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 0x7f881c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.917+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8834089f10 con 0x7f883c159dc0 2024-04-05T22:28:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.918+0000 7f8841200640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 0x7f881c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:47.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.919+0000 7f8841200640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 0x7f881c06d4b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8830036000 tx=0x7f883003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:47.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:47.921+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f88340549c0 con 0x7f883c159dc0 2024-04-05T22:28:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.117+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f883c15dab0 con 0x7f883c159dc0 2024-04-05T22:28:48.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.119+0000 7f8838c00640 1 -- 172.21.15.73:0/934228099 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f883401a090 con 0x7f883c159dc0 2024-04-05T22:28:48.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:48.119 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":122,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:28:45.149453+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.121+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 msgr2=0x7f881c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.121+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 0x7f881c06d4b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8830036000 tx=0x7f883003b040 comp rx=0 tx=0).stop 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.121+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 msgr2=0x7f883c15a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.121+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8834007df0 tx=0x7f883400d4a0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 shutdown_connections 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f881c06aff0 0x7f881c06d4b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f883c159dc0 0x7f883c15a250 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c15e850 0x7f883c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 --2- 172.21.15.73:0/934228099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1587d0 0x7f883c143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 >> 172.21.15.73:0/934228099 conn(0x7f883c0ce8e0 msgr2=0x7f883c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:48.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 shutdown_connections 2024-04-05T22:28:48.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.122+0000 7f8843c00640 1 -- 172.21.15.73:0/934228099 wait complete. 2024-04-05T22:28:48.141 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:28:48.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:28:48.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.349+0000 7f1238800640 1 Processor -- start 2024-04-05T22:28:48.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.349+0000 7f1238800640 1 -- start start 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1238800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f123006cb80 0x7f123006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1238800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123006f5b0 con 0x7f12301587d0 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1238800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1230070410 con 0x7f123015e850 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1238800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123006d500 con 0x7f123006cb80 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1235e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f12301587d0 0x7f12300cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55412/0 (socket says 172.21.15.73:55412) 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1235400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f123006cb80 0x7f123006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.351+0000 7f1235e00640 1 -- 172.21.15.73:0/4214991062 learned_addr learned my addr 172.21.15.73:0/4214991062 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:48.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.352+0000 7f1235400640 1 -- 172.21.15.73:0/4214991062 >> v1:172.21.15.157:6789/0 conn(0x7f123015e850 legacy=0x7f12300708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:48.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.352+0000 7f1235400640 1 -- 172.21.15.73:0/4214991062 >> v1:172.21.15.73:6789/0 conn(0x7f12301587d0 legacy=0x7f12300cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.352+0000 7f1235400640 1 -- 172.21.15.73:0/4214991062 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300ccef0 con 0x7f123006cb80 2024-04-05T22:28:48.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.352+0000 7f1234a00640 1 -- 172.21.15.73:0/4214991062 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585078363 0 0) 0x7f123006f5b0 con 0x7f12301587d0 2024-04-05T22:28:48.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.353+0000 7f1235400640 1 --2- 172.21.15.73:0/4214991062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f123006cfc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f12200097a0 tx=0x7f122002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7f78c74da87508 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.353+0000 7f1234a00640 1 -- 172.21.15.73:0/4214991062 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220030610 con 0x7f123006cb80 2024-04-05T22:28:48.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.353+0000 7f1234a00640 1 -- 172.21.15.73:0/4214991062 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220030770 con 0x7f123006cb80 2024-04-05T22:28:48.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.353+0000 7f1234a00640 1 -- 172.21.15.73:0/4214991062 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220030b00 con 0x7f123006cb80 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 -- 172.21.15.73:0/4214991062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 msgr2=0x7f123006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 --2- 172.21.15.73:0/4214991062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f123006cfc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f12200097a0 tx=0x7f122002b960 comp rx=0 tx=0).stop 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 -- 172.21.15.73:0/4214991062 shutdown_connections 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 --2- 172.21.15.73:0/4214991062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f123006cfc0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 -- 172.21.15.73:0/4214991062 >> 172.21.15.73:0/4214991062 conn(0x7f12300ce8e0 msgr2=0x7f12300cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 -- 172.21.15.73:0/4214991062 shutdown_connections 2024-04-05T22:28:48.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.354+0000 7f1238800640 1 -- 172.21.15.73:0/4214991062 wait complete. 2024-04-05T22:28:48.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.355+0000 7f1238800640 1 Processor -- start 2024-04-05T22:28:48.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 -- start start 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12301587d0 0x7f12301f0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123015e850 0x7f12301f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12301f1750 con 0x7f123006cb80 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12301f10c0 con 0x7f123015e850 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.356+0000 7f1238800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12301f13a0 con 0x7f12301587d0 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48554/0 (socket says 172.21.15.73:48554) 2024-04-05T22:28:48.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 -- 172.21.15.73:0/3069342516 learned_addr learned my addr 172.21.15.73:0/3069342516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:48.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235400640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12301587d0 0x7f12301f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1236800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123015e850 0x7f12301f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 -- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12301587d0 msgr2=0x7f12301f0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12301587d0 0x7f12301f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 -- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123015e850 msgr2=0x7f12301f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123015e850 0x7f12301f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.357+0000 7f1235e00640 1 -- 172.21.15.73:0/3069342516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220009400 con 0x7f123006cb80 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f1235e00640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f122400c4a0 tx=0x7f122400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240074e0 con 0x7f123006cb80 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224007640 con 0x7f123006cb80 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12301f1980 con 0x7f123006cb80 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224004a00 con 0x7f123006cb80 2024-04-05T22:28:48.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.358+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1230143bb0 con 0x7f123006cb80 2024-04-05T22:28:48.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.359+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1230066af0 con 0x7f123006cb80 2024-04-05T22:28:48.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.360+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f1224004b60 con 0x7f123006cb80 2024-04-05T22:28:48.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.361+0000 7f122a000640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 0x7f121406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.361+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1224088c40 con 0x7f123006cb80 2024-04-05T22:28:48.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.361+0000 7f1235400640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 0x7f121406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.362+0000 7f1235400640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 0x7f121406d4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f122003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.365+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1224053800 con 0x7f123006cb80 2024-04-05T22:28:48.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.559+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f123006fae0 con 0x7f121406aff0 2024-04-05T22:28:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.584+0000 7f122a000640 1 -- 172.21.15.73:0/3069342516 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287507 (secure 0 0 0) 0x7f1220030960 con 0x7f121406aff0 2024-04-05T22:28:48.588 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:48.590 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 msgr2=0x7f121406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 0x7f121406d4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f122003b040 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 msgr2=0x7f1230143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f122400c4a0 tx=0x7f122400c960 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 shutdown_connections 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f121406aff0 0x7f121406d4b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f123015e850 0x7f12301f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12301587d0 0x7f12301f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 --2- 172.21.15.73:0/3069342516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f123006cb80 0x7f1230143670 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 >> 172.21.15.73:0/3069342516 conn(0x7f12300ce8e0 msgr2=0x7f1230070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 shutdown_connections 2024-04-05T22:28:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.589+0000 7f1238800640 1 -- 172.21.15.73:0/3069342516 wait complete. 2024-04-05T22:28:48.609 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-05T22:28:47.547459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270220,"kb_used_data":13060,"kb_used_omap":12,"kb_used_meta":257011,"kb_avail":754704500,"statfs":{"total":773094113280,"available":772817408000,"internally_reserved":0,"allocated":13373440,"data_stored":6525440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":263179835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.897078"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221745+0000","last_change":"2024-04-05T22:28:44.185034+0000","last_active":"2024-04-05T22:28:47.221745+0000","last_peered":"2024-04-05T22:28:47.221745+0000","last_clean":"2024-04-05T22:28:47.221745+0000","last_became_active":"2024-04-05T22:28:44.184797+0000","last_became_peered":"2024-04-05T22:28:44.184797+0000","last_unstale":"2024-04-05T22:28:47.221745+0000","last_undegraded":"2024-04-05T22:28:47.221745+0000","last_fullsized":"2024-04-05T22:28:47.221745+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:03.899196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160475+0000","last_change":"2024-04-05T22:28:44.189507+0000","last_active":"2024-04-05T22:28:45.160475+0000","last_peered":"2024-04-05T22:28:45.160475+0000","last_clean":"2024-04-05T22:28:45.160475+0000","last_became_active":"2024-04-05T22:28:44.189197+0000","last_became_peered":"2024-04-05T22:28:44.189197+0000","last_unstale":"2024-04-05T22:28:45.160475+0000","last_undegraded":"2024-04-05T22:28:45.160475+0000","last_fullsized":"2024-04-05T22:28:45.160475+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:43.214897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226738+0000","last_change":"2024-04-05T22:28:44.184126+0000","last_active":"2024-04-05T22:28:47.226738+0000","last_peered":"2024-04-05T22:28:47.226738+0000","last_clean":"2024-04-05T22:28:47.226738+0000","last_became_active":"2024-04-05T22:28:44.183879+0000","last_became_peered":"2024-04-05T22:28:44.183879+0000","last_unstale":"2024-04-05T22:28:47.226738+0000","last_undegraded":"2024-04-05T22:28:47.226738+0000","last_fullsized":"2024-04-05T22:28:47.226738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:42.578935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221343+0000","last_change":"2024-04-05T22:28:44.188137+0000","last_active":"2024-04-05T22:28:47.221343+0000","last_peered":"2024-04-05T22:28:47.221343+0000","last_clean":"2024-04-05T22:28:47.221343+0000","last_became_active":"2024-04-05T22:28:44.187462+0000","last_became_peered":"2024-04-05T22:28:44.187462+0000","last_unstale":"2024-04-05T22:28:47.221343+0000","last_undegraded":"2024-04-05T22:28:47.221343+0000","last_fullsized":"2024-04-05T22:28:47.221343+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:30:48.351327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226464+0000","last_change":"2024-04-05T22:28:44.193392+0000","last_active":"2024-04-05T22:28:47.226464+0000","last_peered":"2024-04-05T22:28:47.226464+0000","last_clean":"2024-04-05T22:28:47.226464+0000","last_became_active":"2024-04-05T22:28:44.192983+0000","last_became_peered":"2024-04-05T22:28:44.192983+0000","last_unstale":"2024-04-05T22:28:47.226464+0000","last_undegraded":"2024-04-05T22:28:47.226464+0000","last_fullsized":"2024-04-05T22:28:47.226464+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:19.650150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160969+0000","last_change":"2024-04-05T22:28:44.184613+0000","last_active":"2024-04-05T22:28:45.160969+0000","last_peered":"2024-04-05T22:28:45.160969+0000","last_clean":"2024-04-05T22:28:45.160969+0000","last_became_active":"2024-04-05T22:28:44.184090+0000","last_became_peered":"2024-04-05T22:28:44.184090+0000","last_unstale":"2024-04-05T22:28:45.160969+0000","last_undegraded":"2024-04-05T22:28:45.160969+0000","last_fullsized":"2024-04-05T22:28:45.160969+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:35.058574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160391+0000","last_change":"2024-04-05T22:28:44.197608+0000","last_active":"2024-04-05T22:28:45.160391+0000","last_peered":"2024-04-05T22:28:45.160391+0000","last_clean":"2024-04-05T22:28:45.160391+0000","last_became_active":"2024-04-05T22:28:44.197268+0000","last_became_peered":"2024-04-05T22:28:44.197268+0000","last_unstale":"2024-04-05T22:28:45.160391+0000","last_undegraded":"2024-04-05T22:28:45.160391+0000","last_fullsized":"2024-04-05T22:28:45.160391+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:55.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224901+0000","last_change":"2024-04-05T22:28:44.185281+0000","last_active":"2024-04-05T22:28:47.224901+0000","last_peered":"2024-04-05T22:28:47.224901+0000","last_clean":"2024-04-05T22:28:47.224901+0000","last_became_active":"2024-04-05T22:28:44.184951+0000","last_became_peered":"2024-04-05T22:28:44.184951+0000","last_unstale":"2024-04-05T22:28:47.224901+0000","last_undegraded":"2024-04-05T22:28:47.224901+0000","last_fullsized":"2024-04-05T22:28:47.224901+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:54.584812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227302+0000","last_change":"2024-04-05T22:28:44.192736+0000","last_active":"2024-04-05T22:28:47.227302+0000","last_peered":"2024-04-05T22:28:47.227302+0000","last_clean":"2024-04-05T22:28:47.227302+0000","last_became_active":"2024-04-05T22:28:44.192379+0000","last_became_peered":"2024-04-05T22:28:44.192379+0000","last_unstale":"2024-04-05T22:28:47.227302+0000","last_undegraded":"2024-04-05T22:28:47.227302+0000","last_fullsized":"2024-04-05T22:28:47.227302+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:18:39.385137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160570+0000","last_change":"2024-04-05T22:28:44.189802+0000","last_active":"2024-04-05T22:28:45.160570+0000","last_peered":"2024-04-05T22:28:45.160570+0000","last_clean":"2024-04-05T22:28:45.160570+0000","last_became_active":"2024-04-05T22:28:44.189428+0000","last_became_peered":"2024-04-05T22:28:44.189428+0000","last_unstale":"2024-04-05T22:28:45.160570+0000","last_undegraded":"2024-04-05T22:28:45.160570+0000","last_fullsized":"2024-04-05T22:28:45.160570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:32:00.250959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161335+0000","last_change":"2024-04-05T22:28:44.173845+0000","last_active":"2024-04-05T22:28:45.161335+0000","last_peered":"2024-04-05T22:28:45.161335+0000","last_clean":"2024-04-05T22:28:45.161335+0000","last_became_active":"2024-04-05T22:28:44.172098+0000","last_became_peered":"2024-04-05T22:28:44.172098+0000","last_unstale":"2024-04-05T22:28:45.161335+0000","last_undegraded":"2024-04-05T22:28:45.161335+0000","last_fullsized":"2024-04-05T22:28:45.161335+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:10:42.606996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226112+0000","last_change":"2024-04-05T22:28:44.194324+0000","last_active":"2024-04-05T22:28:47.226112+0000","last_peered":"2024-04-05T22:28:47.226112+0000","last_clean":"2024-04-05T22:28:47.226112+0000","last_became_active":"2024-04-05T22:28:44.194003+0000","last_became_peered":"2024-04-05T22:28:44.194003+0000","last_unstale":"2024-04-05T22:28:47.226112+0000","last_undegraded":"2024-04-05T22:28:47.226112+0000","last_fullsized":"2024-04-05T22:28:47.226112+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:21:08.265973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221881+0000","last_change":"2024-04-05T22:28:44.195706+0000","last_active":"2024-04-05T22:28:47.221881+0000","last_peered":"2024-04-05T22:28:47.221881+0000","last_clean":"2024-04-05T22:28:47.221881+0000","last_became_active":"2024-04-05T22:28:44.195396+0000","last_became_peered":"2024-04-05T22:28:44.195396+0000","last_unstale":"2024-04-05T22:28:47.221881+0000","last_undegraded":"2024-04-05T22:28:47.221881+0000","last_fullsized":"2024-04-05T22:28:47.221881+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:14:17.268003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160176+0000","last_change":"2024-04-05T22:28:44.164931+0000","last_active":"2024-04-05T22:28:45.160176+0000","last_peered":"2024-04-05T22:28:45.160176+0000","last_clean":"2024-04-05T22:28:45.160176+0000","last_became_active":"2024-04-05T22:28:44.163769+0000","last_became_peered":"2024-04-05T22:28:44.163769+0000","last_unstale":"2024-04-05T22:28:45.160176+0000","last_undegraded":"2024-04-05T22:28:45.160176+0000","last_fullsized":"2024-04-05T22:28:45.160176+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:10.066966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221553+0000","last_change":"2024-04-05T22:28:44.187388+0000","last_active":"2024-04-05T22:28:47.221553+0000","last_peered":"2024-04-05T22:28:47.221553+0000","last_clean":"2024-04-05T22:28:47.221553+0000","last_became_active":"2024-04-05T22:28:44.187090+0000","last_became_peered":"2024-04-05T22:28:44.187090+0000","last_unstale":"2024-04-05T22:28:47.221553+0000","last_undegraded":"2024-04-05T22:28:47.221553+0000","last_fullsized":"2024-04-05T22:28:47.221553+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:41.839280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701153+0000","last_change":"2024-04-05T22:28:44.169157+0000","last_active":"2024-04-05T22:28:46.701153+0000","last_peered":"2024-04-05T22:28:46.701153+0000","last_clean":"2024-04-05T22:28:46.701153+0000","last_became_active":"2024-04-05T22:28:44.168731+0000","last_became_peered":"2024-04-05T22:28:44.168731+0000","last_unstale":"2024-04-05T22:28:46.701153+0000","last_undegraded":"2024-04-05T22:28:46.701153+0000","last_fullsized":"2024-04-05T22:28:46.701153+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:30.805385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161629+0000","last_change":"2024-04-05T22:28:44.184002+0000","last_active":"2024-04-05T22:28:45.161629+0000","last_peered":"2024-04-05T22:28:45.161629+0000","last_clean":"2024-04-05T22:28:45.161629+0000","last_became_active":"2024-04-05T22:28:44.183723+0000","last_became_peered":"2024-04-05T22:28:44.183723+0000","last_unstale":"2024-04-05T22:28:45.161629+0000","last_undegraded":"2024-04-05T22:28:45.161629+0000","last_fullsized":"2024-04-05T22:28:45.161629+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:19.650150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701742+0000","last_change":"2024-04-05T22:28:44.190030+0000","last_active":"2024-04-05T22:28:46.701742+0000","last_peered":"2024-04-05T22:28:46.701742+0000","last_clean":"2024-04-05T22:28:46.701742+0000","last_became_active":"2024-04-05T22:28:44.189727+0000","last_became_peered":"2024-04-05T22:28:44.189727+0000","last_unstale":"2024-04-05T22:28:46.701742+0000","last_undegraded":"2024-04-05T22:28:46.701742+0000","last_fullsized":"2024-04-05T22:28:46.701742+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:31.237845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161580+0000","last_change":"2024-04-05T22:28:44.185176+0000","last_active":"2024-04-05T22:28:45.161580+0000","last_peered":"2024-04-05T22:28:45.161580+0000","last_clean":"2024-04-05T22:28:45.161580+0000","last_became_active":"2024-04-05T22:28:44.184706+0000","last_became_peered":"2024-04-05T22:28:44.184706+0000","last_unstale":"2024-04-05T22:28:45.161580+0000","last_undegraded":"2024-04-05T22:28:45.161580+0000","last_fullsized":"2024-04-05T22:28:45.161580+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:16:17.575933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701375+0000","last_change":"2024-04-05T22:28:44.169437+0000","last_active":"2024-04-05T22:28:46.701375+0000","last_peered":"2024-04-05T22:28:46.701375+0000","last_clean":"2024-04-05T22:28:46.701375+0000","last_became_active":"2024-04-05T22:28:44.169065+0000","last_became_peered":"2024-04-05T22:28:44.169065+0000","last_unstale":"2024-04-05T22:28:46.701375+0000","last_undegraded":"2024-04-05T22:28:46.701375+0000","last_fullsized":"2024-04-05T22:28:46.701375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:14:41.423570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226063+0000","last_change":"2024-04-05T22:28:44.193054+0000","last_active":"2024-04-05T22:28:47.226063+0000","last_peered":"2024-04-05T22:28:47.226063+0000","last_clean":"2024-04-05T22:28:47.226063+0000","last_became_active":"2024-04-05T22:28:44.192791+0000","last_became_peered":"2024-04-05T22:28:44.192791+0000","last_unstale":"2024-04-05T22:28:47.226063+0000","last_undegraded":"2024-04-05T22:28:47.226063+0000","last_fullsized":"2024-04-05T22:28:47.226063+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:44:14.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161951+0000","last_change":"2024-04-05T22:28:44.174096+0000","last_active":"2024-04-05T22:28:45.161951+0000","last_peered":"2024-04-05T22:28:45.161951+0000","last_clean":"2024-04-05T22:28:45.161951+0000","last_became_active":"2024-04-05T22:28:44.172311+0000","last_became_peered":"2024-04-05T22:28:44.172311+0000","last_unstale":"2024-04-05T22:28:45.161951+0000","last_undegraded":"2024-04-05T22:28:45.161951+0000","last_fullsized":"2024-04-05T22:28:45.161951+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:11:44.750953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225326+0000","last_change":"2024-04-05T22:28:44.194477+0000","last_active":"2024-04-05T22:28:47.225326+0000","last_peered":"2024-04-05T22:28:47.225326+0000","last_clean":"2024-04-05T22:28:47.225326+0000","last_became_active":"2024-04-05T22:28:44.193843+0000","last_became_peered":"2024-04-05T22:28:44.193843+0000","last_unstale":"2024-04-05T22:28:47.225326+0000","last_undegraded":"2024-04-05T22:28:47.225326+0000","last_fullsized":"2024-04-05T22:28:47.225326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:59:47.779915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160207+0000","last_change":"2024-04-05T22:28:44.181232+0000","last_active":"2024-04-05T22:28:45.160207+0000","last_peered":"2024-04-05T22:28:45.160207+0000","last_clean":"2024-04-05T22:28:45.160207+0000","last_became_active":"2024-04-05T22:28:44.180961+0000","last_became_peered":"2024-04-05T22:28:44.180961+0000","last_unstale":"2024-04-05T22:28:45.160207+0000","last_undegraded":"2024-04-05T22:28:45.160207+0000","last_fullsized":"2024-04-05T22:28:45.160207+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:56:10.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.162238+0000","last_change":"2024-04-05T22:28:44.179721+0000","last_active":"2024-04-05T22:28:45.162238+0000","last_peered":"2024-04-05T22:28:45.162238+0000","last_clean":"2024-04-05T22:28:45.162238+0000","last_became_active":"2024-04-05T22:28:44.179455+0000","last_became_peered":"2024-04-05T22:28:44.179455+0000","last_unstale":"2024-04-05T22:28:45.162238+0000","last_undegraded":"2024-04-05T22:28:45.162238+0000","last_fullsized":"2024-04-05T22:28:45.162238+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:33.344547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701481+0000","last_change":"2024-04-05T22:28:44.170754+0000","last_active":"2024-04-05T22:28:46.701481+0000","last_peered":"2024-04-05T22:28:46.701481+0000","last_clean":"2024-04-05T22:28:46.701481+0000","last_became_active":"2024-04-05T22:28:44.170505+0000","last_became_peered":"2024-04-05T22:28:44.170505+0000","last_unstale":"2024-04-05T22:28:46.701481+0000","last_undegraded":"2024-04-05T22:28:46.701481+0000","last_fullsized":"2024-04-05T22:28:46.701481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:17:58.027150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221606+0000","last_change":"2024-04-05T22:28:44.184792+0000","last_active":"2024-04-05T22:28:47.221606+0000","last_peered":"2024-04-05T22:28:47.221606+0000","last_clean":"2024-04-05T22:28:47.221606+0000","last_became_active":"2024-04-05T22:28:44.184495+0000","last_became_peered":"2024-04-05T22:28:44.184495+0000","last_unstale":"2024-04-05T22:28:47.221606+0000","last_undegraded":"2024-04-05T22:28:47.221606+0000","last_fullsized":"2024-04-05T22:28:47.221606+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:18.277608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224312+0000","last_change":"2024-04-05T22:28:44.173575+0000","last_active":"2024-04-05T22:28:47.224312+0000","last_peered":"2024-04-05T22:28:47.224312+0000","last_clean":"2024-04-05T22:28:47.224312+0000","last_became_active":"2024-04-05T22:28:44.173149+0000","last_became_peered":"2024-04-05T22:28:44.173149+0000","last_unstale":"2024-04-05T22:28:47.224312+0000","last_undegraded":"2024-04-05T22:28:47.224312+0000","last_fullsized":"2024-04-05T22:28:47.224312+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:18.277608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221326+0000","last_change":"2024-04-05T22:28:44.192388+0000","last_active":"2024-04-05T22:28:47.221326+0000","last_peered":"2024-04-05T22:28:47.221326+0000","last_clean":"2024-04-05T22:28:47.221326+0000","last_became_active":"2024-04-05T22:28:44.192069+0000","last_became_peered":"2024-04-05T22:28:44.192069+0000","last_unstale":"2024-04-05T22:28:47.221326+0000","last_undegraded":"2024-04-05T22:28:47.221326+0000","last_fullsized":"2024-04-05T22:28:47.221326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:25.604715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221717+0000","last_change":"2024-04-05T22:28:44.186601+0000","last_active":"2024-04-05T22:28:47.221717+0000","last_peered":"2024-04-05T22:28:47.221717+0000","last_clean":"2024-04-05T22:28:47.221717+0000","last_became_active":"2024-04-05T22:28:44.186321+0000","last_became_peered":"2024-04-05T22:28:44.186321+0000","last_unstale":"2024-04-05T22:28:47.221717+0000","last_undegraded":"2024-04-05T22:28:47.221717+0000","last_fullsized":"2024-04-05T22:28:47.221717+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:42.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224731+0000","last_change":"2024-04-05T22:28:44.188798+0000","last_active":"2024-04-05T22:28:47.224731+0000","last_peered":"2024-04-05T22:28:47.224731+0000","last_clean":"2024-04-05T22:28:47.224731+0000","last_became_active":"2024-04-05T22:28:44.188457+0000","last_became_peered":"2024-04-05T22:28:44.188457+0000","last_unstale":"2024-04-05T22:28:47.224731+0000","last_undegraded":"2024-04-05T22:28:47.224731+0000","last_fullsized":"2024-04-05T22:28:47.224731+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:38.836024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160376+0000","last_change":"2024-04-05T22:28:44.183434+0000","last_active":"2024-04-05T22:28:45.160376+0000","last_peered":"2024-04-05T22:28:45.160376+0000","last_clean":"2024-04-05T22:28:45.160376+0000","last_became_active":"2024-04-05T22:28:44.183149+0000","last_became_peered":"2024-04-05T22:28:44.183149+0000","last_unstale":"2024-04-05T22:28:45.160376+0000","last_undegraded":"2024-04-05T22:28:45.160376+0000","last_fullsized":"2024-04-05T22:28:45.160376+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:51.150781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701720+0000","last_change":"2024-04-05T22:28:44.184148+0000","last_active":"2024-04-05T22:28:46.701720+0000","last_peered":"2024-04-05T22:28:46.701720+0000","last_clean":"2024-04-05T22:28:46.701720+0000","last_became_active":"2024-04-05T22:28:44.183885+0000","last_became_peered":"2024-04-05T22:28:44.183885+0000","last_unstale":"2024-04-05T22:28:46.701720+0000","last_undegraded":"2024-04-05T22:28:46.701720+0000","last_fullsized":"2024-04-05T22:28:46.701720+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:57:19.146937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227343+0000","last_change":"2024-04-05T22:28:43.700302+0000","last_active":"2024-04-05T22:28:47.227343+0000","last_peered":"2024-04-05T22:28:47.227343+0000","last_clean":"2024-04-05T22:28:47.227343+0000","last_became_active":"2024-04-05T22:28:43.699939+0000","last_became_peered":"2024-04-05T22:28:43.699939+0000","last_unstale":"2024-04-05T22:28:47.227343+0000","last_undegraded":"2024-04-05T22:28:47.227343+0000","last_fullsized":"2024-04-05T22:28:47.227343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:33.892027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227490+0000","last_change":"2024-04-05T22:28:44.183599+0000","last_active":"2024-04-05T22:28:47.227490+0000","last_peered":"2024-04-05T22:28:47.227490+0000","last_clean":"2024-04-05T22:28:47.227490+0000","last_became_active":"2024-04-05T22:28:44.183302+0000","last_became_peered":"2024-04-05T22:28:44.183302+0000","last_unstale":"2024-04-05T22:28:47.227490+0000","last_undegraded":"2024-04-05T22:28:47.227490+0000","last_fullsized":"2024-04-05T22:28:47.227490+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:08.159332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224228+0000","last_change":"2024-04-05T22:28:43.688913+0000","last_active":"2024-04-05T22:28:47.224228+0000","last_peered":"2024-04-05T22:28:47.224228+0000","last_clean":"2024-04-05T22:28:47.224228+0000","last_became_active":"2024-04-05T22:28:43.687888+0000","last_became_peered":"2024-04-05T22:28:43.687888+0000","last_unstale":"2024-04-05T22:28:47.224228+0000","last_undegraded":"2024-04-05T22:28:47.224228+0000","last_fullsized":"2024-04-05T22:28:47.224228+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:22.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226069+0000","last_change":"2024-04-05T22:28:44.197344+0000","last_active":"2024-04-05T22:28:47.226069+0000","last_peered":"2024-04-05T22:28:47.226069+0000","last_clean":"2024-04-05T22:28:47.226069+0000","last_became_active":"2024-04-05T22:28:44.196436+0000","last_became_peered":"2024-04-05T22:28:44.196436+0000","last_unstale":"2024-04-05T22:28:47.226069+0000","last_undegraded":"2024-04-05T22:28:47.226069+0000","last_fullsized":"2024-04-05T22:28:47.226069+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:35:56.811067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161801+0000","last_change":"2024-04-05T22:28:43.690568+0000","last_active":"2024-04-05T22:28:45.161801+0000","last_peered":"2024-04-05T22:28:45.161801+0000","last_clean":"2024-04-05T22:28:45.161801+0000","last_became_active":"2024-04-05T22:28:43.686622+0000","last_became_peered":"2024-04-05T22:28:43.686622+0000","last_unstale":"2024-04-05T22:28:45.161801+0000","last_undegraded":"2024-04-05T22:28:45.161801+0000","last_fullsized":"2024-04-05T22:28:45.161801+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:07.964416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225617+0000","last_change":"2024-04-05T22:28:44.196617+0000","last_active":"2024-04-05T22:28:47.225617+0000","last_peered":"2024-04-05T22:28:47.225617+0000","last_clean":"2024-04-05T22:28:47.225617+0000","last_became_active":"2024-04-05T22:28:44.196334+0000","last_became_peered":"2024-04-05T22:28:44.196334+0000","last_unstale":"2024-04-05T22:28:47.225617+0000","last_undegraded":"2024-04-05T22:28:47.225617+0000","last_fullsized":"2024-04-05T22:28:47.225617+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:54.589924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160548+0000","last_change":"2024-04-05T22:28:43.689070+0000","last_active":"2024-04-05T22:28:45.160548+0000","last_peered":"2024-04-05T22:28:45.160548+0000","last_clean":"2024-04-05T22:28:45.160548+0000","last_became_active":"2024-04-05T22:28:43.688812+0000","last_became_peered":"2024-04-05T22:28:43.688812+0000","last_unstale":"2024-04-05T22:28:45.160548+0000","last_undegraded":"2024-04-05T22:28:45.160548+0000","last_fullsized":"2024-04-05T22:28:45.160548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:22.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701014+0000","last_change":"2024-04-05T22:28:44.187314+0000","last_active":"2024-04-05T22:28:46.701014+0000","last_peered":"2024-04-05T22:28:46.701014+0000","last_clean":"2024-04-05T22:28:46.701014+0000","last_became_active":"2024-04-05T22:28:44.187003+0000","last_became_peered":"2024-04-05T22:28:44.187003+0000","last_unstale":"2024-04-05T22:28:46.701014+0000","last_undegraded":"2024-04-05T22:28:46.701014+0000","last_fullsized":"2024-04-05T22:28:46.701014+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:54.429884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160707+0000","last_change":"2024-04-05T22:28:43.684119+0000","last_active":"2024-04-05T22:28:45.160707+0000","last_peered":"2024-04-05T22:28:45.160707+0000","last_clean":"2024-04-05T22:28:45.160707+0000","last_became_active":"2024-04-05T22:28:43.682897+0000","last_became_peered":"2024-04-05T22:28:43.682897+0000","last_unstale":"2024-04-05T22:28:45.160707+0000","last_undegraded":"2024-04-05T22:28:45.160707+0000","last_fullsized":"2024-04-05T22:28:45.160707+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:27.205824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160679+0000","last_change":"2024-04-05T22:28:44.185696+0000","last_active":"2024-04-05T22:28:45.160679+0000","last_peered":"2024-04-05T22:28:45.160679+0000","last_clean":"2024-04-05T22:28:45.160679+0000","last_became_active":"2024-04-05T22:28:44.185365+0000","last_became_peered":"2024-04-05T22:28:44.185365+0000","last_unstale":"2024-04-05T22:28:45.160679+0000","last_undegraded":"2024-04-05T22:28:45.160679+0000","last_fullsized":"2024-04-05T22:28:45.160679+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:44:14.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.700960+0000","last_change":"2024-04-05T22:28:43.690959+0000","last_active":"2024-04-05T22:28:46.700960+0000","last_peered":"2024-04-05T22:28:46.700960+0000","last_clean":"2024-04-05T22:28:46.700960+0000","last_became_active":"2024-04-05T22:28:43.688011+0000","last_became_peered":"2024-04-05T22:28:43.688011+0000","last_unstale":"2024-04-05T22:28:46.700960+0000","last_undegraded":"2024-04-05T22:28:46.700960+0000","last_fullsized":"2024-04-05T22:28:46.700960+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:49.481207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224816+0000","last_change":"2024-04-05T22:28:44.180916+0000","last_active":"2024-04-05T22:28:47.224816+0000","last_peered":"2024-04-05T22:28:47.224816+0000","last_clean":"2024-04-05T22:28:47.224816+0000","last_became_active":"2024-04-05T22:28:44.180424+0000","last_became_peered":"2024-04-05T22:28:44.180424+0000","last_unstale":"2024-04-05T22:28:47.224816+0000","last_undegraded":"2024-04-05T22:28:47.224816+0000","last_fullsized":"2024-04-05T22:28:47.224816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:57:42.094078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160647+0000","last_change":"2024-04-05T22:28:43.688163+0000","last_active":"2024-04-05T22:28:45.160647+0000","last_peered":"2024-04-05T22:28:45.160647+0000","last_clean":"2024-04-05T22:28:45.160647+0000","last_became_active":"2024-04-05T22:28:43.687897+0000","last_became_peered":"2024-04-05T22:28:43.687897+0000","last_unstale":"2024-04-05T22:28:45.160647+0000","last_undegraded":"2024-04-05T22:28:45.160647+0000","last_fullsized":"2024-04-05T22:28:45.160647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:02.893895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159816+0000","last_change":"2024-04-05T22:28:44.170252+0000","last_active":"2024-04-05T22:28:45.159816+0000","last_peered":"2024-04-05T22:28:45.159816+0000","last_clean":"2024-04-05T22:28:45.159816+0000","last_became_active":"2024-04-05T22:28:44.169136+0000","last_became_peered":"2024-04-05T22:28:44.169136+0000","last_unstale":"2024-04-05T22:28:45.159816+0000","last_undegraded":"2024-04-05T22:28:45.159816+0000","last_fullsized":"2024-04-05T22:28:45.159816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:48.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160230+0000","last_change":"2024-04-05T22:28:43.680522+0000","last_active":"2024-04-05T22:28:45.160230+0000","last_peered":"2024-04-05T22:28:45.160230+0000","last_clean":"2024-04-05T22:28:45.160230+0000","last_became_active":"2024-04-05T22:28:43.680091+0000","last_became_peered":"2024-04-05T22:28:43.680091+0000","last_unstale":"2024-04-05T22:28:45.160230+0000","last_undegraded":"2024-04-05T22:28:45.160230+0000","last_fullsized":"2024-04-05T22:28:45.160230+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:30.327300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221623+0000","last_change":"2024-04-05T22:28:44.168500+0000","last_active":"2024-04-05T22:28:47.221623+0000","last_peered":"2024-04-05T22:28:47.221623+0000","last_clean":"2024-04-05T22:28:47.221623+0000","last_became_active":"2024-04-05T22:28:44.168209+0000","last_became_peered":"2024-04-05T22:28:44.168209+0000","last_unstale":"2024-04-05T22:28:47.221623+0000","last_undegraded":"2024-04-05T22:28:47.221623+0000","last_fullsized":"2024-04-05T22:28:47.221623+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:09.569386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226519+0000","last_change":"2024-04-05T22:28:43.699235+0000","last_active":"2024-04-05T22:28:47.226519+0000","last_peered":"2024-04-05T22:28:47.226519+0000","last_clean":"2024-04-05T22:28:47.226519+0000","last_became_active":"2024-04-05T22:28:43.698946+0000","last_became_peered":"2024-04-05T22:28:43.698946+0000","last_unstale":"2024-04-05T22:28:47.226519+0000","last_undegraded":"2024-04-05T22:28:47.226519+0000","last_fullsized":"2024-04-05T22:28:47.226519+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:24.599414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225867+0000","last_change":"2024-04-05T22:28:44.172491+0000","last_active":"2024-04-05T22:28:47.225867+0000","last_peered":"2024-04-05T22:28:47.225867+0000","last_clean":"2024-04-05T22:28:47.225867+0000","last_became_active":"2024-04-05T22:28:44.171409+0000","last_became_peered":"2024-04-05T22:28:44.171409+0000","last_unstale":"2024-04-05T22:28:47.225867+0000","last_undegraded":"2024-04-05T22:28:47.225867+0000","last_fullsized":"2024-04-05T22:28:47.225867+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:07.474759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701887+0000","last_change":"2024-04-05T22:28:43.695784+0000","last_active":"2024-04-05T22:28:46.701887+0000","last_peered":"2024-04-05T22:28:46.701887+0000","last_clean":"2024-04-05T22:28:46.701887+0000","last_became_active":"2024-04-05T22:28:43.691131+0000","last_became_peered":"2024-04-05T22:28:43.691131+0000","last_unstale":"2024-04-05T22:28:46.701887+0000","last_undegraded":"2024-04-05T22:28:46.701887+0000","last_fullsized":"2024-04-05T22:28:46.701887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:35:55.805766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226320+0000","last_change":"2024-04-05T22:28:44.195246+0000","last_active":"2024-04-05T22:28:47.226320+0000","last_peered":"2024-04-05T22:28:47.226320+0000","last_clean":"2024-04-05T22:28:47.226320+0000","last_became_active":"2024-04-05T22:28:44.194977+0000","last_became_peered":"2024-04-05T22:28:44.194977+0000","last_unstale":"2024-04-05T22:28:47.226320+0000","last_undegraded":"2024-04-05T22:28:47.226320+0000","last_fullsized":"2024-04-05T22:28:47.226320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:55.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.223880+0000","last_change":"2024-04-05T22:28:43.687200+0000","last_active":"2024-04-05T22:28:47.223880+0000","last_peered":"2024-04-05T22:28:47.223880+0000","last_clean":"2024-04-05T22:28:47.223880+0000","last_became_active":"2024-04-05T22:28:43.681369+0000","last_became_peered":"2024-04-05T22:28:43.681369+0000","last_unstale":"2024-04-05T22:28:47.223880+0000","last_undegraded":"2024-04-05T22:28:47.223880+0000","last_fullsized":"2024-04-05T22:28:47.223880+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:15.286554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225916+0000","last_change":"2024-04-05T22:28:44.202113+0000","last_active":"2024-04-05T22:28:47.225916+0000","last_peered":"2024-04-05T22:28:47.225916+0000","last_clean":"2024-04-05T22:28:47.225916+0000","last_became_active":"2024-04-05T22:28:44.201844+0000","last_became_peered":"2024-04-05T22:28:44.201844+0000","last_unstale":"2024-04-05T22:28:47.225916+0000","last_undegraded":"2024-04-05T22:28:47.225916+0000","last_fullsized":"2024-04-05T22:28:47.225916+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:38.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226351+0000","last_change":"2024-04-05T22:28:43.690910+0000","last_active":"2024-04-05T22:28:47.226351+0000","last_peered":"2024-04-05T22:28:47.226351+0000","last_clean":"2024-04-05T22:28:47.226351+0000","last_became_active":"2024-04-05T22:28:43.690605+0000","last_became_peered":"2024-04-05T22:28:43.690605+0000","last_unstale":"2024-04-05T22:28:47.226351+0000","last_undegraded":"2024-04-05T22:28:47.226351+0000","last_fullsized":"2024-04-05T22:28:47.226351+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:00.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221511+0000","last_change":"2024-04-05T22:28:44.173697+0000","last_active":"2024-04-05T22:28:47.221511+0000","last_peered":"2024-04-05T22:28:47.221511+0000","last_clean":"2024-04-05T22:28:47.221511+0000","last_became_active":"2024-04-05T22:28:44.173435+0000","last_became_peered":"2024-04-05T22:28:44.173435+0000","last_unstale":"2024-04-05T22:28:47.221511+0000","last_undegraded":"2024-04-05T22:28:47.221511+0000","last_fullsized":"2024-04-05T22:28:47.221511+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:07.474759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221638+0000","last_change":"2024-04-05T22:28:43.697534+0000","last_active":"2024-04-05T22:28:47.221638+0000","last_peered":"2024-04-05T22:28:47.221638+0000","last_clean":"2024-04-05T22:28:47.221638+0000","last_became_active":"2024-04-05T22:28:43.697192+0000","last_became_peered":"2024-04-05T22:28:43.697192+0000","last_unstale":"2024-04-05T22:28:47.221638+0000","last_undegraded":"2024-04-05T22:28:47.221638+0000","last_fullsized":"2024-04-05T22:28:47.221638+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:09:55.749993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161579+0000","last_change":"2024-04-05T22:28:44.189614+0000","last_active":"2024-04-05T22:28:45.161579+0000","last_peered":"2024-04-05T22:28:45.161579+0000","last_clean":"2024-04-05T22:28:45.161579+0000","last_became_active":"2024-04-05T22:28:44.188681+0000","last_became_peered":"2024-04-05T22:28:44.188681+0000","last_unstale":"2024-04-05T22:28:45.161579+0000","last_undegraded":"2024-04-05T22:28:45.161579+0000","last_fullsized":"2024-04-05T22:28:45.161579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:01:44.090875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159827+0000","last_change":"2024-04-05T22:28:43.690306+0000","last_active":"2024-04-05T22:28:45.159827+0000","last_peered":"2024-04-05T22:28:45.159827+0000","last_clean":"2024-04-05T22:28:45.159827+0000","last_became_active":"2024-04-05T22:28:43.690037+0000","last_became_peered":"2024-04-05T22:28:43.690037+0000","last_unstale":"2024-04-05T22:28:45.159827+0000","last_undegraded":"2024-04-05T22:28:45.159827+0000","last_fullsized":"2024-04-05T22:28:45.159827+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:15.286554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224922+0000","last_change":"2024-04-05T22:28:44.194647+0000","last_active":"2024-04-05T22:28:47.224922+0000","last_peered":"2024-04-05T22:28:47.224922+0000","last_clean":"2024-04-05T22:28:47.224922+0000","last_became_active":"2024-04-05T22:28:44.194237+0000","last_became_peered":"2024-04-05T22:28:44.194237+0000","last_unstale":"2024-04-05T22:28:47.224922+0000","last_undegraded":"2024-04-05T22:28:47.224922+0000","last_fullsized":"2024-04-05T22:28:47.224922+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:49.689552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":19,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225090+0000","last_change":"2024-04-05T22:28:43.682159+0000","last_active":"2024-04-05T22:28:47.225090+0000","last_peered":"2024-04-05T22:28:47.225090+0000","last_clean":"2024-04-05T22:28:47.225090+0000","last_became_active":"2024-04-05T22:28:43.681671+0000","last_became_peered":"2024-04-05T22:28:43.681671+0000","last_unstale":"2024-04-05T22:28:47.225090+0000","last_undegraded":"2024-04-05T22:28:47.225090+0000","last_fullsized":"2024-04-05T22:28:47.225090+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:32:07.315190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227178+0000","last_change":"2024-04-05T22:28:44.188056+0000","last_active":"2024-04-05T22:28:47.227178+0000","last_peered":"2024-04-05T22:28:47.227178+0000","last_clean":"2024-04-05T22:28:47.227178+0000","last_became_active":"2024-04-05T22:28:44.187672+0000","last_became_peered":"2024-04-05T22:28:44.187672+0000","last_unstale":"2024-04-05T22:28:47.227178+0000","last_undegraded":"2024-04-05T22:28:47.227178+0000","last_fullsized":"2024-04-05T22:28:47.227178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:15.880043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226703+0000","last_change":"2024-04-05T22:28:43.694561+0000","last_active":"2024-04-05T22:28:47.226703+0000","last_peered":"2024-04-05T22:28:47.226703+0000","last_clean":"2024-04-05T22:28:47.226703+0000","last_became_active":"2024-04-05T22:28:43.694280+0000","last_became_peered":"2024-04-05T22:28:43.694280+0000","last_unstale":"2024-04-05T22:28:47.226703+0000","last_undegraded":"2024-04-05T22:28:47.226703+0000","last_fullsized":"2024-04-05T22:28:47.226703+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:27.205824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226119+0000","last_change":"2024-04-05T22:28:44.172744+0000","last_active":"2024-04-05T22:28:47.226119+0000","last_peered":"2024-04-05T22:28:47.226119+0000","last_clean":"2024-04-05T22:28:47.226119+0000","last_became_active":"2024-04-05T22:28:44.172337+0000","last_became_peered":"2024-04-05T22:28:44.172337+0000","last_unstale":"2024-04-05T22:28:47.226119+0000","last_undegraded":"2024-04-05T22:28:47.226119+0000","last_fullsized":"2024-04-05T22:28:47.226119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:09.569386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161034+0000","last_change":"2024-04-05T22:28:43.694643+0000","last_active":"2024-04-05T22:28:45.161034+0000","last_peered":"2024-04-05T22:28:45.161034+0000","last_clean":"2024-04-05T22:28:45.161034+0000","last_became_active":"2024-04-05T22:28:43.693079+0000","last_became_peered":"2024-04-05T22:28:43.693079+0000","last_unstale":"2024-04-05T22:28:45.161034+0000","last_undegraded":"2024-04-05T22:28:45.161034+0000","last_fullsized":"2024-04-05T22:28:45.161034+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:25.081503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701503+0000","last_change":"2024-04-05T22:28:44.170760+0000","last_active":"2024-04-05T22:28:46.701503+0000","last_peered":"2024-04-05T22:28:46.701503+0000","last_clean":"2024-04-05T22:28:46.701503+0000","last_became_active":"2024-04-05T22:28:44.169908+0000","last_became_peered":"2024-04-05T22:28:44.169908+0000","last_unstale":"2024-04-05T22:28:46.701503+0000","last_undegraded":"2024-04-05T22:28:46.701503+0000","last_fullsized":"2024-04-05T22:28:46.701503+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:43:44.127033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161644+0000","last_change":"2024-04-05T22:28:43.696440+0000","last_active":"2024-04-05T22:28:45.161644+0000","last_peered":"2024-04-05T22:28:45.161644+0000","last_clean":"2024-04-05T22:28:45.161644+0000","last_became_active":"2024-04-05T22:28:43.696036+0000","last_became_peered":"2024-04-05T22:28:43.696036+0000","last_unstale":"2024-04-05T22:28:45.161644+0000","last_undegraded":"2024-04-05T22:28:45.161644+0000","last_fullsized":"2024-04-05T22:28:45.161644+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:06.469458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161251+0000","last_change":"2024-04-05T22:28:44.169833+0000","last_active":"2024-04-05T22:28:45.161251+0000","last_peered":"2024-04-05T22:28:45.161251+0000","last_clean":"2024-04-05T22:28:45.161251+0000","last_became_active":"2024-04-05T22:28:44.169504+0000","last_became_peered":"2024-04-05T22:28:44.169504+0000","last_unstale":"2024-04-05T22:28:45.161251+0000","last_undegraded":"2024-04-05T22:28:45.161251+0000","last_fullsized":"2024-04-05T22:28:45.161251+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:15:42.237466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224560+0000","last_change":"2024-04-05T22:28:43.689495+0000","last_active":"2024-04-05T22:28:47.224560+0000","last_peered":"2024-04-05T22:28:47.224560+0000","last_clean":"2024-04-05T22:28:47.224560+0000","last_became_active":"2024-04-05T22:28:43.688435+0000","last_became_peered":"2024-04-05T22:28:43.688435+0000","last_unstale":"2024-04-05T22:28:47.224560+0000","last_undegraded":"2024-04-05T22:28:47.224560+0000","last_fullsized":"2024-04-05T22:28:47.224560+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:23.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221336+0000","last_change":"2024-04-05T22:28:44.184923+0000","last_active":"2024-04-05T22:28:47.221336+0000","last_peered":"2024-04-05T22:28:47.221336+0000","last_clean":"2024-04-05T22:28:47.221336+0000","last_became_active":"2024-04-05T22:28:44.184657+0000","last_became_peered":"2024-04-05T22:28:44.184657+0000","last_unstale":"2024-04-05T22:28:47.221336+0000","last_undegraded":"2024-04-05T22:28:47.221336+0000","last_fullsized":"2024-04-05T22:28:47.221336+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:55:22.095840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224460+0000","last_change":"2024-04-05T22:28:43.687043+0000","last_active":"2024-04-05T22:28:47.224460+0000","last_peered":"2024-04-05T22:28:47.224460+0000","last_clean":"2024-04-05T22:28:47.224460+0000","last_became_active":"2024-04-05T22:28:43.686727+0000","last_became_peered":"2024-04-05T22:28:43.686727+0000","last_unstale":"2024-04-05T22:28:47.224460+0000","last_undegraded":"2024-04-05T22:28:47.224460+0000","last_fullsized":"2024-04-05T22:28:47.224460+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:30.327300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160849+0000","last_change":"2024-04-05T22:28:44.197663+0000","last_active":"2024-04-05T22:28:45.160849+0000","last_peered":"2024-04-05T22:28:45.160849+0000","last_clean":"2024-04-05T22:28:45.160849+0000","last_became_active":"2024-04-05T22:28:44.197416+0000","last_became_peered":"2024-04-05T22:28:44.197416+0000","last_unstale":"2024-04-05T22:28:45.160849+0000","last_undegraded":"2024-04-05T22:28:45.160849+0000","last_fullsized":"2024-04-05T22:28:45.160849+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:22:16.495636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701199+0000","last_change":"2024-04-05T22:28:43.694398+0000","last_active":"2024-04-05T22:28:46.701199+0000","last_peered":"2024-04-05T22:28:46.701199+0000","last_clean":"2024-04-05T22:28:46.701199+0000","last_became_active":"2024-04-05T22:28:43.694094+0000","last_became_peered":"2024-04-05T22:28:43.694094+0000","last_unstale":"2024-04-05T22:28:46.701199+0000","last_undegraded":"2024-04-05T22:28:46.701199+0000","last_fullsized":"2024-04-05T22:28:46.701199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:37.830723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159979+0000","last_change":"2024-04-05T22:28:44.169860+0000","last_active":"2024-04-05T22:28:45.159979+0000","last_peered":"2024-04-05T22:28:45.159979+0000","last_clean":"2024-04-05T22:28:45.159979+0000","last_became_active":"2024-04-05T22:28:44.169583+0000","last_became_peered":"2024-04-05T22:28:44.169583+0000","last_unstale":"2024-04-05T22:28:45.159979+0000","last_undegraded":"2024-04-05T22:28:45.159979+0000","last_fullsized":"2024-04-05T22:28:45.159979+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:13.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.700888+0000","last_change":"2024-04-05T22:28:43.693478+0000","last_active":"2024-04-05T22:28:46.700888+0000","last_peered":"2024-04-05T22:28:46.700888+0000","last_clean":"2024-04-05T22:28:46.700888+0000","last_became_active":"2024-04-05T22:28:43.693211+0000","last_became_peered":"2024-04-05T22:28:43.693211+0000","last_unstale":"2024-04-05T22:28:46.700888+0000","last_undegraded":"2024-04-05T22:28:46.700888+0000","last_fullsized":"2024-04-05T22:28:46.700888+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:42.534652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.223816+0000","last_change":"2024-04-05T22:28:44.187533+0000","last_active":"2024-04-05T22:28:47.223816+0000","last_peered":"2024-04-05T22:28:47.223816+0000","last_clean":"2024-04-05T22:28:47.223816+0000","last_became_active":"2024-04-05T22:28:44.187160+0000","last_became_peered":"2024-04-05T22:28:44.187160+0000","last_unstale":"2024-04-05T22:28:47.223816+0000","last_undegraded":"2024-04-05T22:28:47.223816+0000","last_fullsized":"2024-04-05T22:28:47.223816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:50.486508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160446+0000","last_change":"2024-04-05T22:28:43.689907+0000","last_active":"2024-04-05T22:28:45.160446+0000","last_peered":"2024-04-05T22:28:45.160446+0000","last_clean":"2024-04-05T22:28:45.160446+0000","last_became_active":"2024-04-05T22:28:43.689623+0000","last_became_peered":"2024-04-05T22:28:43.689623+0000","last_unstale":"2024-04-05T22:28:45.160446+0000","last_undegraded":"2024-04-05T22:28:45.160446+0000","last_fullsized":"2024-04-05T22:28:45.160446+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:37:15.970010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161585+0000","last_change":"2024-04-05T22:28:44.201321+0000","last_active":"2024-04-05T22:28:45.161585+0000","last_peered":"2024-04-05T22:28:45.161585+0000","last_clean":"2024-04-05T22:28:45.161585+0000","last_became_active":"2024-04-05T22:28:44.200821+0000","last_became_peered":"2024-04-05T22:28:44.200821+0000","last_unstale":"2024-04-05T22:28:45.161585+0000","last_undegraded":"2024-04-05T22:28:45.161585+0000","last_fullsized":"2024-04-05T22:28:45.161585+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:38.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221828+0000","last_change":"2024-04-05T22:28:43.697526+0000","last_active":"2024-04-05T22:28:47.221828+0000","last_peered":"2024-04-05T22:28:47.221828+0000","last_clean":"2024-04-05T22:28:47.221828+0000","last_became_active":"2024-04-05T22:28:43.696839+0000","last_became_peered":"2024-04-05T22:28:43.696839+0000","last_unstale":"2024-04-05T22:28:47.221828+0000","last_undegraded":"2024-04-05T22:28:47.221828+0000","last_fullsized":"2024-04-05T22:28:47.221828+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:49:14.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221752+0000","last_change":"2024-04-05T22:28:44.192621+0000","last_active":"2024-04-05T22:28:47.221752+0000","last_peered":"2024-04-05T22:28:47.221752+0000","last_clean":"2024-04-05T22:28:47.221752+0000","last_became_active":"2024-04-05T22:28:44.192391+0000","last_became_peered":"2024-04-05T22:28:44.192391+0000","last_unstale":"2024-04-05T22:28:47.221752+0000","last_undegraded":"2024-04-05T22:28:47.221752+0000","last_fullsized":"2024-04-05T22:28:47.221752+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:31.332600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227146+0000","last_change":"2024-04-05T22:28:43.697556+0000","last_active":"2024-04-05T22:28:47.227146+0000","last_peered":"2024-04-05T22:28:47.227146+0000","last_clean":"2024-04-05T22:28:47.227146+0000","last_became_active":"2024-04-05T22:28:43.697044+0000","last_became_peered":"2024-04-05T22:28:43.697044+0000","last_unstale":"2024-04-05T22:28:47.227146+0000","last_undegraded":"2024-04-05T22:28:47.227146+0000","last_fullsized":"2024-04-05T22:28:47.227146+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:08:21.023561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160372+0000","last_change":"2024-04-05T22:28:44.188078+0000","last_active":"2024-04-05T22:28:45.160372+0000","last_peered":"2024-04-05T22:28:45.160372+0000","last_clean":"2024-04-05T22:28:45.160372+0000","last_became_active":"2024-04-05T22:28:44.187550+0000","last_became_peered":"2024-04-05T22:28:44.187550+0000","last_unstale":"2024-04-05T22:28:45.160372+0000","last_undegraded":"2024-04-05T22:28:45.160372+0000","last_fullsized":"2024-04-05T22:28:45.160372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:54.589924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161431+0000","last_change":"2024-04-05T22:28:43.683572+0000","last_active":"2024-04-05T22:28:45.161431+0000","last_peered":"2024-04-05T22:28:45.161431+0000","last_clean":"2024-04-05T22:28:45.161431+0000","last_became_active":"2024-04-05T22:28:43.683163+0000","last_became_peered":"2024-04-05T22:28:43.683163+0000","last_unstale":"2024-04-05T22:28:45.161431+0000","last_undegraded":"2024-04-05T22:28:45.161431+0000","last_fullsized":"2024-04-05T22:28:45.161431+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:23.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161502+0000","last_change":"2024-04-05T22:28:44.185403+0000","last_active":"2024-04-05T22:28:45.161502+0000","last_peered":"2024-04-05T22:28:45.161502+0000","last_clean":"2024-04-05T22:28:45.161502+0000","last_became_active":"2024-04-05T22:28:44.183322+0000","last_became_peered":"2024-04-05T22:28:44.183322+0000","last_unstale":"2024-04-05T22:28:45.161502+0000","last_undegraded":"2024-04-05T22:28:45.161502+0000","last_fullsized":"2024-04-05T22:28:45.161502+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:33:06.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160656+0000","last_change":"2024-04-05T22:28:43.694345+0000","last_active":"2024-04-05T22:28:45.160656+0000","last_peered":"2024-04-05T22:28:45.160656+0000","last_clean":"2024-04-05T22:28:45.160656+0000","last_became_active":"2024-04-05T22:28:43.693514+0000","last_became_peered":"2024-04-05T22:28:43.693514+0000","last_unstale":"2024-04-05T22:28:45.160656+0000","last_undegraded":"2024-04-05T22:28:45.160656+0000","last_fullsized":"2024-04-05T22:28:45.160656+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:29.800084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225024+0000","last_change":"2024-04-05T22:28:44.186635+0000","last_active":"2024-04-05T22:28:47.225024+0000","last_peered":"2024-04-05T22:28:47.225024+0000","last_clean":"2024-04-05T22:28:47.225024+0000","last_became_active":"2024-04-05T22:28:44.186325+0000","last_became_peered":"2024-04-05T22:28:44.186325+0000","last_unstale":"2024-04-05T22:28:47.225024+0000","last_undegraded":"2024-04-05T22:28:47.225024+0000","last_fullsized":"2024-04-05T22:28:47.225024+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:52.294626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160670+0000","last_change":"2024-04-05T22:28:43.691684+0000","last_active":"2024-04-05T22:28:45.160670+0000","last_peered":"2024-04-05T22:28:45.160670+0000","last_clean":"2024-04-05T22:28:45.160670+0000","last_became_active":"2024-04-05T22:28:43.691425+0000","last_became_peered":"2024-04-05T22:28:43.691425+0000","last_unstale":"2024-04-05T22:28:45.160670+0000","last_undegraded":"2024-04-05T22:28:45.160670+0000","last_fullsized":"2024-04-05T22:28:45.160670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:24.599414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224337+0000","last_change":"2024-04-05T22:28:44.173708+0000","last_active":"2024-04-05T22:28:47.224337+0000","last_peered":"2024-04-05T22:28:47.224337+0000","last_clean":"2024-04-05T22:28:47.224337+0000","last_became_active":"2024-04-05T22:28:44.173219+0000","last_became_peered":"2024-04-05T22:28:44.173219+0000","last_unstale":"2024-04-05T22:28:47.224337+0000","last_undegraded":"2024-04-05T22:28:47.224337+0000","last_fullsized":"2024-04-05T22:28:47.224337+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:03.899196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225741+0000","last_change":"2024-04-05T22:28:43.698675+0000","last_active":"2024-04-05T22:28:47.225741+0000","last_peered":"2024-04-05T22:28:47.225741+0000","last_clean":"2024-04-05T22:28:47.225741+0000","last_became_active":"2024-04-05T22:28:43.698405+0000","last_became_peered":"2024-04-05T22:28:43.698405+0000","last_unstale":"2024-04-05T22:28:47.225741+0000","last_undegraded":"2024-04-05T22:28:47.225741+0000","last_fullsized":"2024-04-05T22:28:47.225741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:01:43.085574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701498+0000","last_change":"2024-04-05T22:28:44.200946+0000","last_active":"2024-04-05T22:28:46.701498+0000","last_peered":"2024-04-05T22:28:46.701498+0000","last_clean":"2024-04-05T22:28:46.701498+0000","last_became_active":"2024-04-05T22:28:44.200687+0000","last_became_peered":"2024-04-05T22:28:44.200687+0000","last_unstale":"2024-04-05T22:28:46.701498+0000","last_undegraded":"2024-04-05T22:28:46.701498+0000","last_fullsized":"2024-04-05T22:28:46.701498+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:06:49.784818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701179+0000","last_change":"2024-04-05T22:28:43.689450+0000","last_active":"2024-04-05T22:28:46.701179+0000","last_peered":"2024-04-05T22:28:46.701179+0000","last_clean":"2024-04-05T22:28:46.701179+0000","last_became_active":"2024-04-05T22:28:43.689085+0000","last_became_peered":"2024-04-05T22:28:43.689085+0000","last_unstale":"2024-04-05T22:28:46.701179+0000","last_undegraded":"2024-04-05T22:28:46.701179+0000","last_fullsized":"2024-04-05T22:28:46.701179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:54.463598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159728+0000","last_change":"2024-04-05T22:28:44.178842+0000","last_active":"2024-04-05T22:28:45.159728+0000","last_peered":"2024-04-05T22:28:45.159728+0000","last_clean":"2024-04-05T22:28:45.159728+0000","last_became_active":"2024-04-05T22:28:44.178438+0000","last_became_peered":"2024-04-05T22:28:44.178438+0000","last_unstale":"2024-04-05T22:28:45.159728+0000","last_undegraded":"2024-04-05T22:28:45.159728+0000","last_fullsized":"2024-04-05T22:28:45.159728+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:55.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160655+0000","last_change":"2024-04-05T22:28:43.696617+0000","last_active":"2024-04-05T22:28:45.160655+0000","last_peered":"2024-04-05T22:28:45.160655+0000","last_clean":"2024-04-05T22:28:45.160655+0000","last_became_active":"2024-04-05T22:28:43.696371+0000","last_became_peered":"2024-04-05T22:28:43.696371+0000","last_unstale":"2024-04-05T22:28:45.160655+0000","last_undegraded":"2024-04-05T22:28:45.160655+0000","last_fullsized":"2024-04-05T22:28:45.160655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:17:57.021850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224726+0000","last_change":"2024-04-05T22:28:44.183893+0000","last_active":"2024-04-05T22:28:47.224726+0000","last_peered":"2024-04-05T22:28:47.224726+0000","last_clean":"2024-04-05T22:28:47.224726+0000","last_became_active":"2024-04-05T22:28:44.182460+0000","last_became_peered":"2024-04-05T22:28:44.182460+0000","last_unstale":"2024-04-05T22:28:47.224726+0000","last_undegraded":"2024-04-05T22:28:47.224726+0000","last_fullsized":"2024-04-05T22:28:47.224726+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:35.058574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226700+0000","last_change":"2024-04-05T22:28:43.699966+0000","last_active":"2024-04-05T22:28:47.226700+0000","last_peered":"2024-04-05T22:28:47.226700+0000","last_clean":"2024-04-05T22:28:47.226700+0000","last_became_active":"2024-04-05T22:28:43.699652+0000","last_became_peered":"2024-04-05T22:28:43.699652+0000","last_unstale":"2024-04-05T22:28:47.226700+0000","last_undegraded":"2024-04-05T22:28:47.226700+0000","last_fullsized":"2024-04-05T22:28:47.226700+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:06.469458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":288,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-05T22:28:44.173813+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:28:44.173813+0000","last_peered":"2024-04-05T22:28:44.173813+0000","last_clean":"2024-04-05T22:28:44.173813+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:28:44.173813+0000","last_undegraded":"2024-04-05T22:28:44.173813+0000","last_fullsized":"2024-04-05T22:28:44.173813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2138112,"data_stored":1262430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0609999999999999,"15min":1.0329999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6639999999999999,"5min":1.6759999999999999,"15min":1.796},"last":1.288},{"interface":"front","average":{"1min":1.046,"5min":1.0529999999999999,"15min":1.0149999999999999},"min":{"1min":0.67100000000000004,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.4750000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.169,"15min":1.0700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":1.5940000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.087,"15min":1.0660000000000001},"min":{"1min":0.76900000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.542,"5min":1.819,"15min":1.819},"last":0.92500000000000004}]},{"osd":2,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.157,"15min":1.169},"min":{"1min":0.81599999999999995,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":1.502,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.446},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.226},"min":{"1min":0.92000000000000004,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.5669999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1899999999999999,"15min":1.143},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.72299999999999998},"max":{"1min":1.544,"5min":1.667,"15min":1.712},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.214},"min":{"1min":0.71999999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.46,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.171}]},{"osd":4,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.212,"15min":1.2190000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.653,"5min":1.8360000000000001,"15min":1.865},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.296,"15min":1.3140000000000001},"min":{"1min":0.73399999999999999,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":1.7669999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3200000000000001,"15min":1.256},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.9590000000000001},"last":1.339},{"interface":"front","average":{"1min":1.331,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.88600000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.764,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.5269999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4990000000000001,"15min":1.468},"min":{"1min":0.95799999999999996,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":1.927,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.476,"15min":1.49},"min":{"1min":0.79700000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.8799999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.371}]}]},{"osd":6,"up_from":14,"seq":60129542240,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":665124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.054,"15min":1.0580000000000001},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":1.127},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.081,"15min":1.087},"min":{"1min":0.71899999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.337,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.095,"15min":1.1439999999999999},"min":{"1min":0.73699999999999999,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.5620000000000001,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.155},{"interface":"front","average":{"1min":1.137,"5min":1.121,"15min":1.145},"min":{"1min":0.753,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.123,"15min":1.1799999999999999},"min":{"1min":0.76500000000000001,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.319,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1699999999999999,"15min":1.2},"min":{"1min":0.84799999999999998,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.484,"5min":1.7749999999999999,"15min":2.2250000000000001},"last":1.306}]},{"osd":3,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.208,"15min":1.302},"min":{"1min":0.80500000000000005,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.29,"15min":1.379},"min":{"1min":0.79200000000000004,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.7310000000000001,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.085}]},{"osd":4,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.2749999999999999,"15min":1.323},"min":{"1min":0.998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3899999999999999,"15min":1.5349999999999999},"min":{"1min":1.091,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9259999999999999,"15min":3.27},"last":1.375}]},{"osd":5,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.274,"15min":1.4159999999999999},"min":{"1min":1.079,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.708,"5min":1.927,"15min":2.3650000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.456,"15min":1.542},"min":{"1min":1.0529999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.238}]},{"osd":7,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.365,"15min":1.518},"min":{"1min":1.1220000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.643,"5min":1.99,"15min":3.3540000000000001},"last":1.268},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3879999999999999,"15min":1.5289999999999999},"min":{"1min":1.0089999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.899,"5min":1.899,"15min":3.0920000000000001},"last":1.5009999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1515520,"data_stored":663672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.115,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.466},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.097},"min":{"1min":0.68300000000000005,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.5900000000000001,"5min":1.907,"15min":1.907},"last":1.506}]},{"osd":1,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.76600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3919999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.98399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6359999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.177,"15min":1.1599999999999999},"min":{"1min":0.82599999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3759999999999999,"5min":2.544,"15min":2.544},"last":1.421},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.89900000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5840000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.327}]},{"osd":3,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1859999999999999,"15min":1.181},"min":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7529999999999999,"5min":1.867,"15min":1.867},"last":1.286},{"interface":"front","average":{"1min":1.268,"5min":1.2909999999999999,"15min":1.3460000000000001},"min":{"1min":1.03,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.649,"5min":2.927,"15min":2.9620000000000002},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7050000000000001,"5min":1.911,"15min":2.1970000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2969999999999999,"15min":1.302},"min":{"1min":0.89900000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.784,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.597}]},{"osd":6,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.79,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.377},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5129999999999999,"15min":1.5},"min":{"1min":1.284,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.9079999999999999,"5min":2.169,"15min":2.5350000000000001},"last":1.5569999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.464,"15min":1.456},"min":{"1min":1.2010000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.919,"5min":2.028,"15min":2.2509999999999999},"last":1.631},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5,"15min":1.597},"min":{"1min":1.1799999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":4.4329999999999998},"last":1.657}]}]},{"osd":4,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1523712,"data_stored":669930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1100000000000001,"15min":1.0249999999999999},"min":{"1min":0.82599999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.466,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.085},"min":{"1min":0.73199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5369999999999999,"5min":1.859,"15min":1.859},"last":1.016}]},{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1279999999999999},"min":{"1min":0.88100000000000001,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.087},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":1.0009999999999999,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.659,"5min":2.577,"15min":2.577},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1319999999999999,"15min":1.202},"min":{"1min":0.75,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.974},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2450000000000001,"15min":1.2},"min":{"1min":1.0389999999999999,"5min":0.76200000000000001,"15min":0.67400000000000004},"max":{"1min":1.657,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3200000000000001,"15min":1.3640000000000001},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7889999999999999,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3919999999999999,"15min":1.403},"min":{"1min":1.196,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.7509999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":1.0529999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1459999999999999,"5min":2.665,"15min":2.665},"last":1.236},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.375,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.915,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.4139999999999999,"15min":1.4550000000000001},"min":{"1min":1.0229999999999999,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":1.9950000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":1.212,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.8899999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.524,"15min":1.5389999999999999},"min":{"1min":1.0760000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0019999999999998,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6259999999999999,"15min":1.591},"min":{"1min":1.3839999999999999,"5min":0.91300000000000003,"15min":0.72399999999999998},"max":{"1min":2.109,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.3839999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1527808,"data_stored":670020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0229999999999999,"5min":1.744,"15min":1.744},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5860000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.5369999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":1.006},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.0720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0580000000000001,"15min":1.073},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.391,"5min":1.675,"15min":2.2549999999999999},"last":1.629}]},{"osd":4,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1899999999999999,"15min":1.2290000000000001},"min":{"1min":0.82099999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5289999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.294},"min":{"1min":0.75900000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.286,"15min":1.351},"min":{"1min":0.73699999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.288,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6459999999999999,"5min":1.958,"15min":2.3239999999999998},"last":1.448}]},{"osd":6,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.3009999999999999,"15min":1.3500000000000001},"min":{"1min":0.875,"5min":0.505,"15min":0.505},"max":{"1min":1.6240000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.347,"15min":1.401},"min":{"1min":0.97899999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.298,"15min":2.298},"last":1.375}]},{"osd":7,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3400000000000001,"15min":1.415},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":2.468},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.377,"15min":1.415},"min":{"1min":0.93700000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.73,"5min":2.508,"15min":2.798},"last":1.3260000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542238,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1515520,"data_stored":664176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.159,"5min":0.97499999999999998,"15min":0.92900000000000005},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8979999999999997,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.91700000000000004,"15min":0.89000000000000001},"min":{"1min":0.49199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.97999999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.149,"15min":1.1279999999999999},"min":{"1min":0.71099999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3029999999999999,"15min":1.2430000000000001},"min":{"1min":1.0740000000000001,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.9630000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.78100000000000003,"5min":0.625,"15min":0.625},"max":{"1min":5.8639999999999999,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.2190000000000001,"15min":1.2529999999999999},"min":{"1min":0.82399999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2250000000000001,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.385}]},{"osd":6,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.27},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.2949999999999999,"15min":1.2769999999999999},"min":{"1min":0.88600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.242,"5min":2.242,"15min":2.7890000000000001},"last":1.196}]},{"osd":7,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3280000000000001,"15min":1.357},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.7169999999999996,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.542},{"interface":"front","average":{"1min":1.454,"5min":1.385,"15min":1.3919999999999999},"min":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.478}]}]},{"osd":1,"up_from":14,"seq":60129542240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1519616,"data_stored":669860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.363,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.89900000000000002,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.42299999999999999,"15min":0.36799999999999999},"max":{"1min":1.554,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85399999999999998,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.206,"5min":1.488,"15min":1.5389999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.999,"15min":0.99399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0880000000000001,"15min":1.097},"min":{"1min":0.53600000000000003,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.484,"5min":1.5509999999999999,"15min":1.75},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.3959999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6679999999999999,"5min":1.806,"15min":2.2789999999999999},"last":1.141}]},{"osd":5,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.258,"15min":1.262},"min":{"1min":0.84099999999999997,"5min":0.76000000000000001,"15min":0.68100000000000005},"max":{"1min":1.706,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.022},{"interface":"front","average":{"1min":1.123,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.873,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.5429999999999999,"5min":1.843,"15min":2.173},"last":1.292}]},{"osd":6,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.369,"15min":1.3340000000000001},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6439999999999999,"5min":1.9059999999999999,"15min":1.917},"last":1.258},{"interface":"front","average":{"1min":1.292,"5min":1.345,"15min":1.355},"min":{"1min":0.92200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.825,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.6319999999999999,"5min":1.9279999999999999,"15min":2.3879999999999999},"last":1.165}]}]},{"osd":0,"up_from":14,"seq":60129542240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2113536,"data_stored":1260228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.1479999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.92600000000000005,"15min":0.92000000000000004},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2569999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.2729999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.25,"5min":1.976,"15min":1.976},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0840000000000001,"15min":1.093},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5900000000000001,"5min":2.157,"15min":2.157},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2,"15min":1.1699999999999999},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.429,"5min":1.931,"15min":1.931},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3129999999999999,"15min":1.302},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.786,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2549999999999999,"15min":1.2609999999999999},"min":{"1min":0.75700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.524,"5min":2.105,"15min":2.3879999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.2130000000000001,"15min":1.198},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.357,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.878,"15min":0.65500000000000003},"max":{"1min":1.8340000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3280000000000001,"15min":1.365},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.585,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.1459999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3320000000000001,"15min":1.337},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.22},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.6080000000000001,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.022}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:28:48.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:28:48.622 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-05T22:28:48.622+0000 7f272ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:28:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.835+0000 7fa907200640 1 Processor -- start 2024-04-05T22:28:48.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.835+0000 7fa907200640 1 -- start start 2024-04-05T22:28:48.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa907200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa90815ccd0 0x7fa90815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa907200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9080b2420 con 0x7fa9080d1600 2024-04-05T22:28:48.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa907200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9080ce380 con 0x7fa9080d35e0 2024-04-05T22:28:48.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa907200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080b2770 con 0x7fa90815ccd0 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa905e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa90815ccd0 0x7fa90815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa906800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa9080d1600 0x7fa9080b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55424/0 (socket says 172.21.15.73:55424) 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.837+0000 7fa906800640 1 -- 172.21.15.73:0/2928928902 learned_addr learned my addr 172.21.15.73:0/2928928902 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.838+0000 7fa905e00640 1 -- 172.21.15.73:0/2928928902 >> v1:172.21.15.157:6789/0 conn(0x7fa9080d35e0 legacy=0x7fa9080d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.838+0000 7fa905e00640 1 -- 172.21.15.73:0/2928928902 >> v1:172.21.15.73:6789/0 conn(0x7fa9080d1600 legacy=0x7fa9080b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.838+0000 7fa905e00640 1 -- 172.21.15.73:0/2928928902 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080ce660 con 0x7fa90815ccd0 2024-04-05T22:28:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.839+0000 7fa905e00640 1 --2- 172.21.15.73:0/2928928902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa90815d0b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80097a0 tx=0x7fa8f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcea66748e36e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.839+0000 7fa905400640 1 -- 172.21.15.73:0/2928928902 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f802e070 con 0x7fa90815ccd0 2024-04-05T22:28:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.839+0000 7fa905400640 1 -- 172.21.15.73:0/2928928902 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80315a0 con 0x7fa90815ccd0 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.839+0000 7fa905400640 1 -- 172.21.15.73:0/2928928902 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8031930 con 0x7fa90815ccd0 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 -- 172.21.15.73:0/2928928902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 msgr2=0x7fa90815d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 --2- 172.21.15.73:0/2928928902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa90815d0b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80097a0 tx=0x7fa8f802b840 comp rx=0 tx=0).stop 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 -- 172.21.15.73:0/2928928902 shutdown_connections 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 --2- 172.21.15.73:0/2928928902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa90815d0b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 -- 172.21.15.73:0/2928928902 >> 172.21.15.73:0/2928928902 conn(0x7fa9080d2ac0 msgr2=0x7fa9080d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 -- 172.21.15.73:0/2928928902 shutdown_connections 2024-04-05T22:28:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.840+0000 7fa907200640 1 -- 172.21.15.73:0/2928928902 wait complete. 2024-04-05T22:28:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.842+0000 7fa907200640 1 Processor -- start 2024-04-05T22:28:48.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.842+0000 7fa907200640 1 -- start start 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9080d1600 0x7fa9081538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 0x7fa9080ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b67b0 con 0x7fa90815ccd0 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b5b30 con 0x7fa9080d35e0 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa907200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b5e10 con 0x7fa9080d1600 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48578/0 (socket says 172.21.15.73:48578) 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 -- 172.21.15.73:0/891718825 learned_addr learned my addr 172.21.15.73:0/891718825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa905e00640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9080d1600 0x7fa9081538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa8ffe00640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 0x7fa9080ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 -- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9080d1600 msgr2=0x7fa9081538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9080d1600 0x7fa9081538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 -- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 msgr2=0x7fa9080ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 0x7fa9080ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa906800640 1 -- 172.21.15.73:0/891718825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8009400 con 0x7fa90815ccd0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.843+0000 7fa8ffe00640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 0x7fa9080ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa906800640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa90000caa0 tx=0x7fa90000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000ab40 con 0x7fa90815ccd0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000aca0 con 0x7fa90815ccd0 2024-04-05T22:28:48.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900012600 con 0x7fa90815ccd0 2024-04-05T22:28:48.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080b60f0 con 0x7fa90815ccd0 2024-04-05T22:28:48.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.844+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908153e30 con 0x7fa90815ccd0 2024-04-05T22:28:48.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.845+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908067250 con 0x7fa90815ccd0 2024-04-05T22:28:48.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.846+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fa900007070 con 0x7fa90815ccd0 2024-04-05T22:28:48.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.847+0000 7fa8fe000640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 0x7fa8e406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:48.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.847+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9000896f0 con 0x7fa90815ccd0 2024-04-05T22:28:48.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.848+0000 7fa905e00640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 0x7fa8e406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.849+0000 7fa905e00640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 0x7fa8e406d4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8036000 tx=0x7fa8f803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:48.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:48.851+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa900054450 con 0x7fa90815ccd0 2024-04-05T22:28:49.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.043+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa90804e160 con 0x7fa8e406aff0 2024-04-05T22:28:49.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.066+0000 7fa8fe000640 1 -- 172.21.15.73:0/891718825 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287507 (secure 0 0 0) 0x7fa90804e160 con 0x7fa8e406aff0 2024-04-05T22:28:49.069 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:49.072 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:28:49.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 msgr2=0x7fa8e406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 0x7fa8e406d4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8036000 tx=0x7fa8f803d040 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 msgr2=0x7fa9080b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa90000caa0 tx=0x7fa90000cf60 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 shutdown_connections 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa8e406aff0 0x7fa8e406d4b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa90815ccd0 0x7fa9080b0250 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9080d35e0 0x7fa9080ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 --2- 172.21.15.73:0/891718825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9080d1600 0x7fa9081538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.072+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 >> 172.21.15.73:0/891718825 conn(0x7fa9080d2ac0 msgr2=0x7fa90806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.073+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 shutdown_connections 2024-04-05T22:28:49.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.073+0000 7fa907200640 1 -- 172.21.15.73:0/891718825 wait complete. 2024-04-05T22:28:49.092 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-05T22:28:47.547459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270220,"kb_used_data":13060,"kb_used_omap":12,"kb_used_meta":257011,"kb_avail":754704500,"statfs":{"total":773094113280,"available":772817408000,"internally_reserved":0,"allocated":13373440,"data_stored":6525440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":263179835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.897078"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221745+0000","last_change":"2024-04-05T22:28:44.185034+0000","last_active":"2024-04-05T22:28:47.221745+0000","last_peered":"2024-04-05T22:28:47.221745+0000","last_clean":"2024-04-05T22:28:47.221745+0000","last_became_active":"2024-04-05T22:28:44.184797+0000","last_became_peered":"2024-04-05T22:28:44.184797+0000","last_unstale":"2024-04-05T22:28:47.221745+0000","last_undegraded":"2024-04-05T22:28:47.221745+0000","last_fullsized":"2024-04-05T22:28:47.221745+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:03.899196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160475+0000","last_change":"2024-04-05T22:28:44.189507+0000","last_active":"2024-04-05T22:28:45.160475+0000","last_peered":"2024-04-05T22:28:45.160475+0000","last_clean":"2024-04-05T22:28:45.160475+0000","last_became_active":"2024-04-05T22:28:44.189197+0000","last_became_peered":"2024-04-05T22:28:44.189197+0000","last_unstale":"2024-04-05T22:28:45.160475+0000","last_undegraded":"2024-04-05T22:28:45.160475+0000","last_fullsized":"2024-04-05T22:28:45.160475+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:43.214897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226738+0000","last_change":"2024-04-05T22:28:44.184126+0000","last_active":"2024-04-05T22:28:47.226738+0000","last_peered":"2024-04-05T22:28:47.226738+0000","last_clean":"2024-04-05T22:28:47.226738+0000","last_became_active":"2024-04-05T22:28:44.183879+0000","last_became_peered":"2024-04-05T22:28:44.183879+0000","last_unstale":"2024-04-05T22:28:47.226738+0000","last_undegraded":"2024-04-05T22:28:47.226738+0000","last_fullsized":"2024-04-05T22:28:47.226738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:42.578935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221343+0000","last_change":"2024-04-05T22:28:44.188137+0000","last_active":"2024-04-05T22:28:47.221343+0000","last_peered":"2024-04-05T22:28:47.221343+0000","last_clean":"2024-04-05T22:28:47.221343+0000","last_became_active":"2024-04-05T22:28:44.187462+0000","last_became_peered":"2024-04-05T22:28:44.187462+0000","last_unstale":"2024-04-05T22:28:47.221343+0000","last_undegraded":"2024-04-05T22:28:47.221343+0000","last_fullsized":"2024-04-05T22:28:47.221343+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:30:48.351327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226464+0000","last_change":"2024-04-05T22:28:44.193392+0000","last_active":"2024-04-05T22:28:47.226464+0000","last_peered":"2024-04-05T22:28:47.226464+0000","last_clean":"2024-04-05T22:28:47.226464+0000","last_became_active":"2024-04-05T22:28:44.192983+0000","last_became_peered":"2024-04-05T22:28:44.192983+0000","last_unstale":"2024-04-05T22:28:47.226464+0000","last_undegraded":"2024-04-05T22:28:47.226464+0000","last_fullsized":"2024-04-05T22:28:47.226464+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:19.650150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160969+0000","last_change":"2024-04-05T22:28:44.184613+0000","last_active":"2024-04-05T22:28:45.160969+0000","last_peered":"2024-04-05T22:28:45.160969+0000","last_clean":"2024-04-05T22:28:45.160969+0000","last_became_active":"2024-04-05T22:28:44.184090+0000","last_became_peered":"2024-04-05T22:28:44.184090+0000","last_unstale":"2024-04-05T22:28:45.160969+0000","last_undegraded":"2024-04-05T22:28:45.160969+0000","last_fullsized":"2024-04-05T22:28:45.160969+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:35.058574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160391+0000","last_change":"2024-04-05T22:28:44.197608+0000","last_active":"2024-04-05T22:28:45.160391+0000","last_peered":"2024-04-05T22:28:45.160391+0000","last_clean":"2024-04-05T22:28:45.160391+0000","last_became_active":"2024-04-05T22:28:44.197268+0000","last_became_peered":"2024-04-05T22:28:44.197268+0000","last_unstale":"2024-04-05T22:28:45.160391+0000","last_undegraded":"2024-04-05T22:28:45.160391+0000","last_fullsized":"2024-04-05T22:28:45.160391+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:55.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224901+0000","last_change":"2024-04-05T22:28:44.185281+0000","last_active":"2024-04-05T22:28:47.224901+0000","last_peered":"2024-04-05T22:28:47.224901+0000","last_clean":"2024-04-05T22:28:47.224901+0000","last_became_active":"2024-04-05T22:28:44.184951+0000","last_became_peered":"2024-04-05T22:28:44.184951+0000","last_unstale":"2024-04-05T22:28:47.224901+0000","last_undegraded":"2024-04-05T22:28:47.224901+0000","last_fullsized":"2024-04-05T22:28:47.224901+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:40:54.584812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227302+0000","last_change":"2024-04-05T22:28:44.192736+0000","last_active":"2024-04-05T22:28:47.227302+0000","last_peered":"2024-04-05T22:28:47.227302+0000","last_clean":"2024-04-05T22:28:47.227302+0000","last_became_active":"2024-04-05T22:28:44.192379+0000","last_became_peered":"2024-04-05T22:28:44.192379+0000","last_unstale":"2024-04-05T22:28:47.227302+0000","last_undegraded":"2024-04-05T22:28:47.227302+0000","last_fullsized":"2024-04-05T22:28:47.227302+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:18:39.385137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160570+0000","last_change":"2024-04-05T22:28:44.189802+0000","last_active":"2024-04-05T22:28:45.160570+0000","last_peered":"2024-04-05T22:28:45.160570+0000","last_clean":"2024-04-05T22:28:45.160570+0000","last_became_active":"2024-04-05T22:28:44.189428+0000","last_became_peered":"2024-04-05T22:28:44.189428+0000","last_unstale":"2024-04-05T22:28:45.160570+0000","last_undegraded":"2024-04-05T22:28:45.160570+0000","last_fullsized":"2024-04-05T22:28:45.160570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:32:00.250959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161335+0000","last_change":"2024-04-05T22:28:44.173845+0000","last_active":"2024-04-05T22:28:45.161335+0000","last_peered":"2024-04-05T22:28:45.161335+0000","last_clean":"2024-04-05T22:28:45.161335+0000","last_became_active":"2024-04-05T22:28:44.172098+0000","last_became_peered":"2024-04-05T22:28:44.172098+0000","last_unstale":"2024-04-05T22:28:45.161335+0000","last_undegraded":"2024-04-05T22:28:45.161335+0000","last_fullsized":"2024-04-05T22:28:45.161335+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:10:42.606996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226112+0000","last_change":"2024-04-05T22:28:44.194324+0000","last_active":"2024-04-05T22:28:47.226112+0000","last_peered":"2024-04-05T22:28:47.226112+0000","last_clean":"2024-04-05T22:28:47.226112+0000","last_became_active":"2024-04-05T22:28:44.194003+0000","last_became_peered":"2024-04-05T22:28:44.194003+0000","last_unstale":"2024-04-05T22:28:47.226112+0000","last_undegraded":"2024-04-05T22:28:47.226112+0000","last_fullsized":"2024-04-05T22:28:47.226112+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:21:08.265973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221881+0000","last_change":"2024-04-05T22:28:44.195706+0000","last_active":"2024-04-05T22:28:47.221881+0000","last_peered":"2024-04-05T22:28:47.221881+0000","last_clean":"2024-04-05T22:28:47.221881+0000","last_became_active":"2024-04-05T22:28:44.195396+0000","last_became_peered":"2024-04-05T22:28:44.195396+0000","last_unstale":"2024-04-05T22:28:47.221881+0000","last_undegraded":"2024-04-05T22:28:47.221881+0000","last_fullsized":"2024-04-05T22:28:47.221881+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:14:17.268003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160176+0000","last_change":"2024-04-05T22:28:44.164931+0000","last_active":"2024-04-05T22:28:45.160176+0000","last_peered":"2024-04-05T22:28:45.160176+0000","last_clean":"2024-04-05T22:28:45.160176+0000","last_became_active":"2024-04-05T22:28:44.163769+0000","last_became_peered":"2024-04-05T22:28:44.163769+0000","last_unstale":"2024-04-05T22:28:45.160176+0000","last_undegraded":"2024-04-05T22:28:45.160176+0000","last_fullsized":"2024-04-05T22:28:45.160176+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:55:10.066966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221553+0000","last_change":"2024-04-05T22:28:44.187388+0000","last_active":"2024-04-05T22:28:47.221553+0000","last_peered":"2024-04-05T22:28:47.221553+0000","last_clean":"2024-04-05T22:28:47.221553+0000","last_became_active":"2024-04-05T22:28:44.187090+0000","last_became_peered":"2024-04-05T22:28:44.187090+0000","last_unstale":"2024-04-05T22:28:47.221553+0000","last_undegraded":"2024-04-05T22:28:47.221553+0000","last_fullsized":"2024-04-05T22:28:47.221553+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:41.839280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701153+0000","last_change":"2024-04-05T22:28:44.169157+0000","last_active":"2024-04-05T22:28:46.701153+0000","last_peered":"2024-04-05T22:28:46.701153+0000","last_clean":"2024-04-05T22:28:46.701153+0000","last_became_active":"2024-04-05T22:28:44.168731+0000","last_became_peered":"2024-04-05T22:28:44.168731+0000","last_unstale":"2024-04-05T22:28:46.701153+0000","last_undegraded":"2024-04-05T22:28:46.701153+0000","last_fullsized":"2024-04-05T22:28:46.701153+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:30.805385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161629+0000","last_change":"2024-04-05T22:28:44.184002+0000","last_active":"2024-04-05T22:28:45.161629+0000","last_peered":"2024-04-05T22:28:45.161629+0000","last_clean":"2024-04-05T22:28:45.161629+0000","last_became_active":"2024-04-05T22:28:44.183723+0000","last_became_peered":"2024-04-05T22:28:44.183723+0000","last_unstale":"2024-04-05T22:28:45.161629+0000","last_undegraded":"2024-04-05T22:28:45.161629+0000","last_fullsized":"2024-04-05T22:28:45.161629+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:19.650150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701742+0000","last_change":"2024-04-05T22:28:44.190030+0000","last_active":"2024-04-05T22:28:46.701742+0000","last_peered":"2024-04-05T22:28:46.701742+0000","last_clean":"2024-04-05T22:28:46.701742+0000","last_became_active":"2024-04-05T22:28:44.189727+0000","last_became_peered":"2024-04-05T22:28:44.189727+0000","last_unstale":"2024-04-05T22:28:46.701742+0000","last_undegraded":"2024-04-05T22:28:46.701742+0000","last_fullsized":"2024-04-05T22:28:46.701742+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:31.237845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161580+0000","last_change":"2024-04-05T22:28:44.185176+0000","last_active":"2024-04-05T22:28:45.161580+0000","last_peered":"2024-04-05T22:28:45.161580+0000","last_clean":"2024-04-05T22:28:45.161580+0000","last_became_active":"2024-04-05T22:28:44.184706+0000","last_became_peered":"2024-04-05T22:28:44.184706+0000","last_unstale":"2024-04-05T22:28:45.161580+0000","last_undegraded":"2024-04-05T22:28:45.161580+0000","last_fullsized":"2024-04-05T22:28:45.161580+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:16:17.575933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701375+0000","last_change":"2024-04-05T22:28:44.169437+0000","last_active":"2024-04-05T22:28:46.701375+0000","last_peered":"2024-04-05T22:28:46.701375+0000","last_clean":"2024-04-05T22:28:46.701375+0000","last_became_active":"2024-04-05T22:28:44.169065+0000","last_became_peered":"2024-04-05T22:28:44.169065+0000","last_unstale":"2024-04-05T22:28:46.701375+0000","last_undegraded":"2024-04-05T22:28:46.701375+0000","last_fullsized":"2024-04-05T22:28:46.701375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:14:41.423570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226063+0000","last_change":"2024-04-05T22:28:44.193054+0000","last_active":"2024-04-05T22:28:47.226063+0000","last_peered":"2024-04-05T22:28:47.226063+0000","last_clean":"2024-04-05T22:28:47.226063+0000","last_became_active":"2024-04-05T22:28:44.192791+0000","last_became_peered":"2024-04-05T22:28:44.192791+0000","last_unstale":"2024-04-05T22:28:47.226063+0000","last_undegraded":"2024-04-05T22:28:47.226063+0000","last_fullsized":"2024-04-05T22:28:47.226063+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:44:14.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161951+0000","last_change":"2024-04-05T22:28:44.174096+0000","last_active":"2024-04-05T22:28:45.161951+0000","last_peered":"2024-04-05T22:28:45.161951+0000","last_clean":"2024-04-05T22:28:45.161951+0000","last_became_active":"2024-04-05T22:28:44.172311+0000","last_became_peered":"2024-04-05T22:28:44.172311+0000","last_unstale":"2024-04-05T22:28:45.161951+0000","last_undegraded":"2024-04-05T22:28:45.161951+0000","last_fullsized":"2024-04-05T22:28:45.161951+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:11:44.750953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225326+0000","last_change":"2024-04-05T22:28:44.194477+0000","last_active":"2024-04-05T22:28:47.225326+0000","last_peered":"2024-04-05T22:28:47.225326+0000","last_clean":"2024-04-05T22:28:47.225326+0000","last_became_active":"2024-04-05T22:28:44.193843+0000","last_became_peered":"2024-04-05T22:28:44.193843+0000","last_unstale":"2024-04-05T22:28:47.225326+0000","last_undegraded":"2024-04-05T22:28:47.225326+0000","last_fullsized":"2024-04-05T22:28:47.225326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:59:47.779915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160207+0000","last_change":"2024-04-05T22:28:44.181232+0000","last_active":"2024-04-05T22:28:45.160207+0000","last_peered":"2024-04-05T22:28:45.160207+0000","last_clean":"2024-04-05T22:28:45.160207+0000","last_became_active":"2024-04-05T22:28:44.180961+0000","last_became_peered":"2024-04-05T22:28:44.180961+0000","last_unstale":"2024-04-05T22:28:45.160207+0000","last_undegraded":"2024-04-05T22:28:45.160207+0000","last_fullsized":"2024-04-05T22:28:45.160207+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:56:10.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.162238+0000","last_change":"2024-04-05T22:28:44.179721+0000","last_active":"2024-04-05T22:28:45.162238+0000","last_peered":"2024-04-05T22:28:45.162238+0000","last_clean":"2024-04-05T22:28:45.162238+0000","last_became_active":"2024-04-05T22:28:44.179455+0000","last_became_peered":"2024-04-05T22:28:44.179455+0000","last_unstale":"2024-04-05T22:28:45.162238+0000","last_undegraded":"2024-04-05T22:28:45.162238+0000","last_fullsized":"2024-04-05T22:28:45.162238+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:46:33.344547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701481+0000","last_change":"2024-04-05T22:28:44.170754+0000","last_active":"2024-04-05T22:28:46.701481+0000","last_peered":"2024-04-05T22:28:46.701481+0000","last_clean":"2024-04-05T22:28:46.701481+0000","last_became_active":"2024-04-05T22:28:44.170505+0000","last_became_peered":"2024-04-05T22:28:44.170505+0000","last_unstale":"2024-04-05T22:28:46.701481+0000","last_undegraded":"2024-04-05T22:28:46.701481+0000","last_fullsized":"2024-04-05T22:28:46.701481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:17:58.027150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221606+0000","last_change":"2024-04-05T22:28:44.184792+0000","last_active":"2024-04-05T22:28:47.221606+0000","last_peered":"2024-04-05T22:28:47.221606+0000","last_clean":"2024-04-05T22:28:47.221606+0000","last_became_active":"2024-04-05T22:28:44.184495+0000","last_became_peered":"2024-04-05T22:28:44.184495+0000","last_unstale":"2024-04-05T22:28:47.221606+0000","last_undegraded":"2024-04-05T22:28:47.221606+0000","last_fullsized":"2024-04-05T22:28:47.221606+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:18.277608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224312+0000","last_change":"2024-04-05T22:28:44.173575+0000","last_active":"2024-04-05T22:28:47.224312+0000","last_peered":"2024-04-05T22:28:47.224312+0000","last_clean":"2024-04-05T22:28:47.224312+0000","last_became_active":"2024-04-05T22:28:44.173149+0000","last_became_peered":"2024-04-05T22:28:44.173149+0000","last_unstale":"2024-04-05T22:28:47.224312+0000","last_undegraded":"2024-04-05T22:28:47.224312+0000","last_fullsized":"2024-04-05T22:28:47.224312+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:18.277608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221326+0000","last_change":"2024-04-05T22:28:44.192388+0000","last_active":"2024-04-05T22:28:47.221326+0000","last_peered":"2024-04-05T22:28:47.221326+0000","last_clean":"2024-04-05T22:28:47.221326+0000","last_became_active":"2024-04-05T22:28:44.192069+0000","last_became_peered":"2024-04-05T22:28:44.192069+0000","last_unstale":"2024-04-05T22:28:47.221326+0000","last_undegraded":"2024-04-05T22:28:47.221326+0000","last_fullsized":"2024-04-05T22:28:47.221326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:25.604715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221717+0000","last_change":"2024-04-05T22:28:44.186601+0000","last_active":"2024-04-05T22:28:47.221717+0000","last_peered":"2024-04-05T22:28:47.221717+0000","last_clean":"2024-04-05T22:28:47.221717+0000","last_became_active":"2024-04-05T22:28:44.186321+0000","last_became_peered":"2024-04-05T22:28:44.186321+0000","last_unstale":"2024-04-05T22:28:47.221717+0000","last_undegraded":"2024-04-05T22:28:47.221717+0000","last_fullsized":"2024-04-05T22:28:47.221717+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:05:42.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224731+0000","last_change":"2024-04-05T22:28:44.188798+0000","last_active":"2024-04-05T22:28:47.224731+0000","last_peered":"2024-04-05T22:28:47.224731+0000","last_clean":"2024-04-05T22:28:47.224731+0000","last_became_active":"2024-04-05T22:28:44.188457+0000","last_became_peered":"2024-04-05T22:28:44.188457+0000","last_unstale":"2024-04-05T22:28:47.224731+0000","last_undegraded":"2024-04-05T22:28:47.224731+0000","last_fullsized":"2024-04-05T22:28:47.224731+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:38.836024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160376+0000","last_change":"2024-04-05T22:28:44.183434+0000","last_active":"2024-04-05T22:28:45.160376+0000","last_peered":"2024-04-05T22:28:45.160376+0000","last_clean":"2024-04-05T22:28:45.160376+0000","last_became_active":"2024-04-05T22:28:44.183149+0000","last_became_peered":"2024-04-05T22:28:44.183149+0000","last_unstale":"2024-04-05T22:28:45.160376+0000","last_undegraded":"2024-04-05T22:28:45.160376+0000","last_fullsized":"2024-04-05T22:28:45.160376+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:51.150781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701720+0000","last_change":"2024-04-05T22:28:44.184148+0000","last_active":"2024-04-05T22:28:46.701720+0000","last_peered":"2024-04-05T22:28:46.701720+0000","last_clean":"2024-04-05T22:28:46.701720+0000","last_became_active":"2024-04-05T22:28:44.183885+0000","last_became_peered":"2024-04-05T22:28:44.183885+0000","last_unstale":"2024-04-05T22:28:46.701720+0000","last_undegraded":"2024-04-05T22:28:46.701720+0000","last_fullsized":"2024-04-05T22:28:46.701720+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:57:19.146937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227343+0000","last_change":"2024-04-05T22:28:43.700302+0000","last_active":"2024-04-05T22:28:47.227343+0000","last_peered":"2024-04-05T22:28:47.227343+0000","last_clean":"2024-04-05T22:28:47.227343+0000","last_became_active":"2024-04-05T22:28:43.699939+0000","last_became_peered":"2024-04-05T22:28:43.699939+0000","last_unstale":"2024-04-05T22:28:47.227343+0000","last_undegraded":"2024-04-05T22:28:47.227343+0000","last_fullsized":"2024-04-05T22:28:47.227343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:33.892027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227490+0000","last_change":"2024-04-05T22:28:44.183599+0000","last_active":"2024-04-05T22:28:47.227490+0000","last_peered":"2024-04-05T22:28:47.227490+0000","last_clean":"2024-04-05T22:28:47.227490+0000","last_became_active":"2024-04-05T22:28:44.183302+0000","last_became_peered":"2024-04-05T22:28:44.183302+0000","last_unstale":"2024-04-05T22:28:47.227490+0000","last_undegraded":"2024-04-05T22:28:47.227490+0000","last_fullsized":"2024-04-05T22:28:47.227490+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:08.159332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224228+0000","last_change":"2024-04-05T22:28:43.688913+0000","last_active":"2024-04-05T22:28:47.224228+0000","last_peered":"2024-04-05T22:28:47.224228+0000","last_clean":"2024-04-05T22:28:47.224228+0000","last_became_active":"2024-04-05T22:28:43.687888+0000","last_became_peered":"2024-04-05T22:28:43.687888+0000","last_unstale":"2024-04-05T22:28:47.224228+0000","last_undegraded":"2024-04-05T22:28:47.224228+0000","last_fullsized":"2024-04-05T22:28:47.224228+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:22.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226069+0000","last_change":"2024-04-05T22:28:44.197344+0000","last_active":"2024-04-05T22:28:47.226069+0000","last_peered":"2024-04-05T22:28:47.226069+0000","last_clean":"2024-04-05T22:28:47.226069+0000","last_became_active":"2024-04-05T22:28:44.196436+0000","last_became_peered":"2024-04-05T22:28:44.196436+0000","last_unstale":"2024-04-05T22:28:47.226069+0000","last_undegraded":"2024-04-05T22:28:47.226069+0000","last_fullsized":"2024-04-05T22:28:47.226069+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:35:56.811067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161801+0000","last_change":"2024-04-05T22:28:43.690568+0000","last_active":"2024-04-05T22:28:45.161801+0000","last_peered":"2024-04-05T22:28:45.161801+0000","last_clean":"2024-04-05T22:28:45.161801+0000","last_became_active":"2024-04-05T22:28:43.686622+0000","last_became_peered":"2024-04-05T22:28:43.686622+0000","last_unstale":"2024-04-05T22:28:45.161801+0000","last_undegraded":"2024-04-05T22:28:45.161801+0000","last_fullsized":"2024-04-05T22:28:45.161801+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:55:07.964416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225617+0000","last_change":"2024-04-05T22:28:44.196617+0000","last_active":"2024-04-05T22:28:47.225617+0000","last_peered":"2024-04-05T22:28:47.225617+0000","last_clean":"2024-04-05T22:28:47.225617+0000","last_became_active":"2024-04-05T22:28:44.196334+0000","last_became_peered":"2024-04-05T22:28:44.196334+0000","last_unstale":"2024-04-05T22:28:47.225617+0000","last_undegraded":"2024-04-05T22:28:47.225617+0000","last_fullsized":"2024-04-05T22:28:47.225617+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:54.589924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160548+0000","last_change":"2024-04-05T22:28:43.689070+0000","last_active":"2024-04-05T22:28:45.160548+0000","last_peered":"2024-04-05T22:28:45.160548+0000","last_clean":"2024-04-05T22:28:45.160548+0000","last_became_active":"2024-04-05T22:28:43.688812+0000","last_became_peered":"2024-04-05T22:28:43.688812+0000","last_unstale":"2024-04-05T22:28:45.160548+0000","last_undegraded":"2024-04-05T22:28:45.160548+0000","last_fullsized":"2024-04-05T22:28:45.160548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:50:22.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701014+0000","last_change":"2024-04-05T22:28:44.187314+0000","last_active":"2024-04-05T22:28:46.701014+0000","last_peered":"2024-04-05T22:28:46.701014+0000","last_clean":"2024-04-05T22:28:46.701014+0000","last_became_active":"2024-04-05T22:28:44.187003+0000","last_became_peered":"2024-04-05T22:28:44.187003+0000","last_unstale":"2024-04-05T22:28:46.701014+0000","last_undegraded":"2024-04-05T22:28:46.701014+0000","last_fullsized":"2024-04-05T22:28:46.701014+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:54.429884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160707+0000","last_change":"2024-04-05T22:28:43.684119+0000","last_active":"2024-04-05T22:28:45.160707+0000","last_peered":"2024-04-05T22:28:45.160707+0000","last_clean":"2024-04-05T22:28:45.160707+0000","last_became_active":"2024-04-05T22:28:43.682897+0000","last_became_peered":"2024-04-05T22:28:43.682897+0000","last_unstale":"2024-04-05T22:28:45.160707+0000","last_undegraded":"2024-04-05T22:28:45.160707+0000","last_fullsized":"2024-04-05T22:28:45.160707+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:27.205824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160679+0000","last_change":"2024-04-05T22:28:44.185696+0000","last_active":"2024-04-05T22:28:45.160679+0000","last_peered":"2024-04-05T22:28:45.160679+0000","last_clean":"2024-04-05T22:28:45.160679+0000","last_became_active":"2024-04-05T22:28:44.185365+0000","last_became_peered":"2024-04-05T22:28:44.185365+0000","last_unstale":"2024-04-05T22:28:45.160679+0000","last_undegraded":"2024-04-05T22:28:45.160679+0000","last_fullsized":"2024-04-05T22:28:45.160679+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:44:14.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.700960+0000","last_change":"2024-04-05T22:28:43.690959+0000","last_active":"2024-04-05T22:28:46.700960+0000","last_peered":"2024-04-05T22:28:46.700960+0000","last_clean":"2024-04-05T22:28:46.700960+0000","last_became_active":"2024-04-05T22:28:43.688011+0000","last_became_peered":"2024-04-05T22:28:43.688011+0000","last_unstale":"2024-04-05T22:28:46.700960+0000","last_undegraded":"2024-04-05T22:28:46.700960+0000","last_fullsized":"2024-04-05T22:28:46.700960+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:49.481207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224816+0000","last_change":"2024-04-05T22:28:44.180916+0000","last_active":"2024-04-05T22:28:47.224816+0000","last_peered":"2024-04-05T22:28:47.224816+0000","last_clean":"2024-04-05T22:28:47.224816+0000","last_became_active":"2024-04-05T22:28:44.180424+0000","last_became_peered":"2024-04-05T22:28:44.180424+0000","last_unstale":"2024-04-05T22:28:47.224816+0000","last_undegraded":"2024-04-05T22:28:47.224816+0000","last_fullsized":"2024-04-05T22:28:47.224816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:57:42.094078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160647+0000","last_change":"2024-04-05T22:28:43.688163+0000","last_active":"2024-04-05T22:28:45.160647+0000","last_peered":"2024-04-05T22:28:45.160647+0000","last_clean":"2024-04-05T22:28:45.160647+0000","last_became_active":"2024-04-05T22:28:43.687897+0000","last_became_peered":"2024-04-05T22:28:43.687897+0000","last_unstale":"2024-04-05T22:28:45.160647+0000","last_undegraded":"2024-04-05T22:28:45.160647+0000","last_fullsized":"2024-04-05T22:28:45.160647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:02.893895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159816+0000","last_change":"2024-04-05T22:28:44.170252+0000","last_active":"2024-04-05T22:28:45.159816+0000","last_peered":"2024-04-05T22:28:45.159816+0000","last_clean":"2024-04-05T22:28:45.159816+0000","last_became_active":"2024-04-05T22:28:44.169136+0000","last_became_peered":"2024-04-05T22:28:44.169136+0000","last_unstale":"2024-04-05T22:28:45.159816+0000","last_undegraded":"2024-04-05T22:28:45.159816+0000","last_fullsized":"2024-04-05T22:28:45.159816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:48.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160230+0000","last_change":"2024-04-05T22:28:43.680522+0000","last_active":"2024-04-05T22:28:45.160230+0000","last_peered":"2024-04-05T22:28:45.160230+0000","last_clean":"2024-04-05T22:28:45.160230+0000","last_became_active":"2024-04-05T22:28:43.680091+0000","last_became_peered":"2024-04-05T22:28:43.680091+0000","last_unstale":"2024-04-05T22:28:45.160230+0000","last_undegraded":"2024-04-05T22:28:45.160230+0000","last_fullsized":"2024-04-05T22:28:45.160230+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:30.327300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221623+0000","last_change":"2024-04-05T22:28:44.168500+0000","last_active":"2024-04-05T22:28:47.221623+0000","last_peered":"2024-04-05T22:28:47.221623+0000","last_clean":"2024-04-05T22:28:47.221623+0000","last_became_active":"2024-04-05T22:28:44.168209+0000","last_became_peered":"2024-04-05T22:28:44.168209+0000","last_unstale":"2024-04-05T22:28:47.221623+0000","last_undegraded":"2024-04-05T22:28:47.221623+0000","last_fullsized":"2024-04-05T22:28:47.221623+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:09.569386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226519+0000","last_change":"2024-04-05T22:28:43.699235+0000","last_active":"2024-04-05T22:28:47.226519+0000","last_peered":"2024-04-05T22:28:47.226519+0000","last_clean":"2024-04-05T22:28:47.226519+0000","last_became_active":"2024-04-05T22:28:43.698946+0000","last_became_peered":"2024-04-05T22:28:43.698946+0000","last_unstale":"2024-04-05T22:28:47.226519+0000","last_undegraded":"2024-04-05T22:28:47.226519+0000","last_fullsized":"2024-04-05T22:28:47.226519+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:24.599414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225867+0000","last_change":"2024-04-05T22:28:44.172491+0000","last_active":"2024-04-05T22:28:47.225867+0000","last_peered":"2024-04-05T22:28:47.225867+0000","last_clean":"2024-04-05T22:28:47.225867+0000","last_became_active":"2024-04-05T22:28:44.171409+0000","last_became_peered":"2024-04-05T22:28:44.171409+0000","last_unstale":"2024-04-05T22:28:47.225867+0000","last_undegraded":"2024-04-05T22:28:47.225867+0000","last_fullsized":"2024-04-05T22:28:47.225867+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:07.474759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701887+0000","last_change":"2024-04-05T22:28:43.695784+0000","last_active":"2024-04-05T22:28:46.701887+0000","last_peered":"2024-04-05T22:28:46.701887+0000","last_clean":"2024-04-05T22:28:46.701887+0000","last_became_active":"2024-04-05T22:28:43.691131+0000","last_became_peered":"2024-04-05T22:28:43.691131+0000","last_unstale":"2024-04-05T22:28:46.701887+0000","last_undegraded":"2024-04-05T22:28:46.701887+0000","last_fullsized":"2024-04-05T22:28:46.701887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:35:55.805766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226320+0000","last_change":"2024-04-05T22:28:44.195246+0000","last_active":"2024-04-05T22:28:47.226320+0000","last_peered":"2024-04-05T22:28:47.226320+0000","last_clean":"2024-04-05T22:28:47.226320+0000","last_became_active":"2024-04-05T22:28:44.194977+0000","last_became_peered":"2024-04-05T22:28:44.194977+0000","last_unstale":"2024-04-05T22:28:47.226320+0000","last_undegraded":"2024-04-05T22:28:47.226320+0000","last_fullsized":"2024-04-05T22:28:47.226320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:55.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.223880+0000","last_change":"2024-04-05T22:28:43.687200+0000","last_active":"2024-04-05T22:28:47.223880+0000","last_peered":"2024-04-05T22:28:47.223880+0000","last_clean":"2024-04-05T22:28:47.223880+0000","last_became_active":"2024-04-05T22:28:43.681369+0000","last_became_peered":"2024-04-05T22:28:43.681369+0000","last_unstale":"2024-04-05T22:28:47.223880+0000","last_undegraded":"2024-04-05T22:28:47.223880+0000","last_fullsized":"2024-04-05T22:28:47.223880+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:15.286554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225916+0000","last_change":"2024-04-05T22:28:44.202113+0000","last_active":"2024-04-05T22:28:47.225916+0000","last_peered":"2024-04-05T22:28:47.225916+0000","last_clean":"2024-04-05T22:28:47.225916+0000","last_became_active":"2024-04-05T22:28:44.201844+0000","last_became_peered":"2024-04-05T22:28:44.201844+0000","last_unstale":"2024-04-05T22:28:47.225916+0000","last_undegraded":"2024-04-05T22:28:47.225916+0000","last_fullsized":"2024-04-05T22:28:47.225916+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:38.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226351+0000","last_change":"2024-04-05T22:28:43.690910+0000","last_active":"2024-04-05T22:28:47.226351+0000","last_peered":"2024-04-05T22:28:47.226351+0000","last_clean":"2024-04-05T22:28:47.226351+0000","last_became_active":"2024-04-05T22:28:43.690605+0000","last_became_peered":"2024-04-05T22:28:43.690605+0000","last_unstale":"2024-04-05T22:28:47.226351+0000","last_undegraded":"2024-04-05T22:28:47.226351+0000","last_fullsized":"2024-04-05T22:28:47.226351+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:00.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221511+0000","last_change":"2024-04-05T22:28:44.173697+0000","last_active":"2024-04-05T22:28:47.221511+0000","last_peered":"2024-04-05T22:28:47.221511+0000","last_clean":"2024-04-05T22:28:47.221511+0000","last_became_active":"2024-04-05T22:28:44.173435+0000","last_became_peered":"2024-04-05T22:28:44.173435+0000","last_unstale":"2024-04-05T22:28:47.221511+0000","last_undegraded":"2024-04-05T22:28:47.221511+0000","last_fullsized":"2024-04-05T22:28:47.221511+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:07.474759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221638+0000","last_change":"2024-04-05T22:28:43.697534+0000","last_active":"2024-04-05T22:28:47.221638+0000","last_peered":"2024-04-05T22:28:47.221638+0000","last_clean":"2024-04-05T22:28:47.221638+0000","last_became_active":"2024-04-05T22:28:43.697192+0000","last_became_peered":"2024-04-05T22:28:43.697192+0000","last_unstale":"2024-04-05T22:28:47.221638+0000","last_undegraded":"2024-04-05T22:28:47.221638+0000","last_fullsized":"2024-04-05T22:28:47.221638+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:09:55.749993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161579+0000","last_change":"2024-04-05T22:28:44.189614+0000","last_active":"2024-04-05T22:28:45.161579+0000","last_peered":"2024-04-05T22:28:45.161579+0000","last_clean":"2024-04-05T22:28:45.161579+0000","last_became_active":"2024-04-05T22:28:44.188681+0000","last_became_peered":"2024-04-05T22:28:44.188681+0000","last_unstale":"2024-04-05T22:28:45.161579+0000","last_undegraded":"2024-04-05T22:28:45.161579+0000","last_fullsized":"2024-04-05T22:28:45.161579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:01:44.090875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159827+0000","last_change":"2024-04-05T22:28:43.690306+0000","last_active":"2024-04-05T22:28:45.159827+0000","last_peered":"2024-04-05T22:28:45.159827+0000","last_clean":"2024-04-05T22:28:45.159827+0000","last_became_active":"2024-04-05T22:28:43.690037+0000","last_became_peered":"2024-04-05T22:28:43.690037+0000","last_unstale":"2024-04-05T22:28:45.159827+0000","last_undegraded":"2024-04-05T22:28:45.159827+0000","last_fullsized":"2024-04-05T22:28:45.159827+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:15.286554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224922+0000","last_change":"2024-04-05T22:28:44.194647+0000","last_active":"2024-04-05T22:28:47.224922+0000","last_peered":"2024-04-05T22:28:47.224922+0000","last_clean":"2024-04-05T22:28:47.224922+0000","last_became_active":"2024-04-05T22:28:44.194237+0000","last_became_peered":"2024-04-05T22:28:44.194237+0000","last_unstale":"2024-04-05T22:28:47.224922+0000","last_undegraded":"2024-04-05T22:28:47.224922+0000","last_fullsized":"2024-04-05T22:28:47.224922+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:49.689552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":19,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225090+0000","last_change":"2024-04-05T22:28:43.682159+0000","last_active":"2024-04-05T22:28:47.225090+0000","last_peered":"2024-04-05T22:28:47.225090+0000","last_clean":"2024-04-05T22:28:47.225090+0000","last_became_active":"2024-04-05T22:28:43.681671+0000","last_became_peered":"2024-04-05T22:28:43.681671+0000","last_unstale":"2024-04-05T22:28:47.225090+0000","last_undegraded":"2024-04-05T22:28:47.225090+0000","last_fullsized":"2024-04-05T22:28:47.225090+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:32:07.315190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227178+0000","last_change":"2024-04-05T22:28:44.188056+0000","last_active":"2024-04-05T22:28:47.227178+0000","last_peered":"2024-04-05T22:28:47.227178+0000","last_clean":"2024-04-05T22:28:47.227178+0000","last_became_active":"2024-04-05T22:28:44.187672+0000","last_became_peered":"2024-04-05T22:28:44.187672+0000","last_unstale":"2024-04-05T22:28:47.227178+0000","last_undegraded":"2024-04-05T22:28:47.227178+0000","last_fullsized":"2024-04-05T22:28:47.227178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:15.880043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226703+0000","last_change":"2024-04-05T22:28:43.694561+0000","last_active":"2024-04-05T22:28:47.226703+0000","last_peered":"2024-04-05T22:28:47.226703+0000","last_clean":"2024-04-05T22:28:47.226703+0000","last_became_active":"2024-04-05T22:28:43.694280+0000","last_became_peered":"2024-04-05T22:28:43.694280+0000","last_unstale":"2024-04-05T22:28:47.226703+0000","last_undegraded":"2024-04-05T22:28:47.226703+0000","last_fullsized":"2024-04-05T22:28:47.226703+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:27.205824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226119+0000","last_change":"2024-04-05T22:28:44.172744+0000","last_active":"2024-04-05T22:28:47.226119+0000","last_peered":"2024-04-05T22:28:47.226119+0000","last_clean":"2024-04-05T22:28:47.226119+0000","last_became_active":"2024-04-05T22:28:44.172337+0000","last_became_peered":"2024-04-05T22:28:44.172337+0000","last_unstale":"2024-04-05T22:28:47.226119+0000","last_undegraded":"2024-04-05T22:28:47.226119+0000","last_fullsized":"2024-04-05T22:28:47.226119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:09.569386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161034+0000","last_change":"2024-04-05T22:28:43.694643+0000","last_active":"2024-04-05T22:28:45.161034+0000","last_peered":"2024-04-05T22:28:45.161034+0000","last_clean":"2024-04-05T22:28:45.161034+0000","last_became_active":"2024-04-05T22:28:43.693079+0000","last_became_peered":"2024-04-05T22:28:43.693079+0000","last_unstale":"2024-04-05T22:28:45.161034+0000","last_undegraded":"2024-04-05T22:28:45.161034+0000","last_fullsized":"2024-04-05T22:28:45.161034+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:25.081503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701503+0000","last_change":"2024-04-05T22:28:44.170760+0000","last_active":"2024-04-05T22:28:46.701503+0000","last_peered":"2024-04-05T22:28:46.701503+0000","last_clean":"2024-04-05T22:28:46.701503+0000","last_became_active":"2024-04-05T22:28:44.169908+0000","last_became_peered":"2024-04-05T22:28:44.169908+0000","last_unstale":"2024-04-05T22:28:46.701503+0000","last_undegraded":"2024-04-05T22:28:46.701503+0000","last_fullsized":"2024-04-05T22:28:46.701503+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:43:44.127033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161644+0000","last_change":"2024-04-05T22:28:43.696440+0000","last_active":"2024-04-05T22:28:45.161644+0000","last_peered":"2024-04-05T22:28:45.161644+0000","last_clean":"2024-04-05T22:28:45.161644+0000","last_became_active":"2024-04-05T22:28:43.696036+0000","last_became_peered":"2024-04-05T22:28:43.696036+0000","last_unstale":"2024-04-05T22:28:45.161644+0000","last_undegraded":"2024-04-05T22:28:45.161644+0000","last_fullsized":"2024-04-05T22:28:45.161644+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:06.469458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161251+0000","last_change":"2024-04-05T22:28:44.169833+0000","last_active":"2024-04-05T22:28:45.161251+0000","last_peered":"2024-04-05T22:28:45.161251+0000","last_clean":"2024-04-05T22:28:45.161251+0000","last_became_active":"2024-04-05T22:28:44.169504+0000","last_became_peered":"2024-04-05T22:28:44.169504+0000","last_unstale":"2024-04-05T22:28:45.161251+0000","last_undegraded":"2024-04-05T22:28:45.161251+0000","last_fullsized":"2024-04-05T22:28:45.161251+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:15:42.237466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224560+0000","last_change":"2024-04-05T22:28:43.689495+0000","last_active":"2024-04-05T22:28:47.224560+0000","last_peered":"2024-04-05T22:28:47.224560+0000","last_clean":"2024-04-05T22:28:47.224560+0000","last_became_active":"2024-04-05T22:28:43.688435+0000","last_became_peered":"2024-04-05T22:28:43.688435+0000","last_unstale":"2024-04-05T22:28:47.224560+0000","last_undegraded":"2024-04-05T22:28:47.224560+0000","last_fullsized":"2024-04-05T22:28:47.224560+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:23.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221336+0000","last_change":"2024-04-05T22:28:44.184923+0000","last_active":"2024-04-05T22:28:47.221336+0000","last_peered":"2024-04-05T22:28:47.221336+0000","last_clean":"2024-04-05T22:28:47.221336+0000","last_became_active":"2024-04-05T22:28:44.184657+0000","last_became_peered":"2024-04-05T22:28:44.184657+0000","last_unstale":"2024-04-05T22:28:47.221336+0000","last_undegraded":"2024-04-05T22:28:47.221336+0000","last_fullsized":"2024-04-05T22:28:47.221336+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:55:22.095840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224460+0000","last_change":"2024-04-05T22:28:43.687043+0000","last_active":"2024-04-05T22:28:47.224460+0000","last_peered":"2024-04-05T22:28:47.224460+0000","last_clean":"2024-04-05T22:28:47.224460+0000","last_became_active":"2024-04-05T22:28:43.686727+0000","last_became_peered":"2024-04-05T22:28:43.686727+0000","last_unstale":"2024-04-05T22:28:47.224460+0000","last_undegraded":"2024-04-05T22:28:47.224460+0000","last_fullsized":"2024-04-05T22:28:47.224460+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:30.327300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160849+0000","last_change":"2024-04-05T22:28:44.197663+0000","last_active":"2024-04-05T22:28:45.160849+0000","last_peered":"2024-04-05T22:28:45.160849+0000","last_clean":"2024-04-05T22:28:45.160849+0000","last_became_active":"2024-04-05T22:28:44.197416+0000","last_became_peered":"2024-04-05T22:28:44.197416+0000","last_unstale":"2024-04-05T22:28:45.160849+0000","last_undegraded":"2024-04-05T22:28:45.160849+0000","last_fullsized":"2024-04-05T22:28:45.160849+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:22:16.495636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701199+0000","last_change":"2024-04-05T22:28:43.694398+0000","last_active":"2024-04-05T22:28:46.701199+0000","last_peered":"2024-04-05T22:28:46.701199+0000","last_clean":"2024-04-05T22:28:46.701199+0000","last_became_active":"2024-04-05T22:28:43.694094+0000","last_became_peered":"2024-04-05T22:28:43.694094+0000","last_unstale":"2024-04-05T22:28:46.701199+0000","last_undegraded":"2024-04-05T22:28:46.701199+0000","last_fullsized":"2024-04-05T22:28:46.701199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:43:37.830723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159979+0000","last_change":"2024-04-05T22:28:44.169860+0000","last_active":"2024-04-05T22:28:45.159979+0000","last_peered":"2024-04-05T22:28:45.159979+0000","last_clean":"2024-04-05T22:28:45.159979+0000","last_became_active":"2024-04-05T22:28:44.169583+0000","last_became_peered":"2024-04-05T22:28:44.169583+0000","last_unstale":"2024-04-05T22:28:45.159979+0000","last_undegraded":"2024-04-05T22:28:45.159979+0000","last_fullsized":"2024-04-05T22:28:45.159979+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:13.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.700888+0000","last_change":"2024-04-05T22:28:43.693478+0000","last_active":"2024-04-05T22:28:46.700888+0000","last_peered":"2024-04-05T22:28:46.700888+0000","last_clean":"2024-04-05T22:28:46.700888+0000","last_became_active":"2024-04-05T22:28:43.693211+0000","last_became_peered":"2024-04-05T22:28:43.693211+0000","last_unstale":"2024-04-05T22:28:46.700888+0000","last_undegraded":"2024-04-05T22:28:46.700888+0000","last_fullsized":"2024-04-05T22:28:46.700888+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:45:42.534652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.223816+0000","last_change":"2024-04-05T22:28:44.187533+0000","last_active":"2024-04-05T22:28:47.223816+0000","last_peered":"2024-04-05T22:28:47.223816+0000","last_clean":"2024-04-05T22:28:47.223816+0000","last_became_active":"2024-04-05T22:28:44.187160+0000","last_became_peered":"2024-04-05T22:28:44.187160+0000","last_unstale":"2024-04-05T22:28:47.223816+0000","last_undegraded":"2024-04-05T22:28:47.223816+0000","last_fullsized":"2024-04-05T22:28:47.223816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:48:50.486508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160446+0000","last_change":"2024-04-05T22:28:43.689907+0000","last_active":"2024-04-05T22:28:45.160446+0000","last_peered":"2024-04-05T22:28:45.160446+0000","last_clean":"2024-04-05T22:28:45.160446+0000","last_became_active":"2024-04-05T22:28:43.689623+0000","last_became_peered":"2024-04-05T22:28:43.689623+0000","last_unstale":"2024-04-05T22:28:45.160446+0000","last_undegraded":"2024-04-05T22:28:45.160446+0000","last_fullsized":"2024-04-05T22:28:45.160446+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:37:15.970010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161585+0000","last_change":"2024-04-05T22:28:44.201321+0000","last_active":"2024-04-05T22:28:45.161585+0000","last_peered":"2024-04-05T22:28:45.161585+0000","last_clean":"2024-04-05T22:28:45.161585+0000","last_became_active":"2024-04-05T22:28:44.200821+0000","last_became_peered":"2024-04-05T22:28:44.200821+0000","last_unstale":"2024-04-05T22:28:45.161585+0000","last_undegraded":"2024-04-05T22:28:45.161585+0000","last_fullsized":"2024-04-05T22:28:45.161585+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:38.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221828+0000","last_change":"2024-04-05T22:28:43.697526+0000","last_active":"2024-04-05T22:28:47.221828+0000","last_peered":"2024-04-05T22:28:47.221828+0000","last_clean":"2024-04-05T22:28:47.221828+0000","last_became_active":"2024-04-05T22:28:43.696839+0000","last_became_peered":"2024-04-05T22:28:43.696839+0000","last_unstale":"2024-04-05T22:28:47.221828+0000","last_undegraded":"2024-04-05T22:28:47.221828+0000","last_fullsized":"2024-04-05T22:28:47.221828+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:49:14.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.221752+0000","last_change":"2024-04-05T22:28:44.192621+0000","last_active":"2024-04-05T22:28:47.221752+0000","last_peered":"2024-04-05T22:28:47.221752+0000","last_clean":"2024-04-05T22:28:47.221752+0000","last_became_active":"2024-04-05T22:28:44.192391+0000","last_became_peered":"2024-04-05T22:28:44.192391+0000","last_unstale":"2024-04-05T22:28:47.221752+0000","last_undegraded":"2024-04-05T22:28:47.221752+0000","last_fullsized":"2024-04-05T22:28:47.221752+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:31.332600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.227146+0000","last_change":"2024-04-05T22:28:43.697556+0000","last_active":"2024-04-05T22:28:47.227146+0000","last_peered":"2024-04-05T22:28:47.227146+0000","last_clean":"2024-04-05T22:28:47.227146+0000","last_became_active":"2024-04-05T22:28:43.697044+0000","last_became_peered":"2024-04-05T22:28:43.697044+0000","last_unstale":"2024-04-05T22:28:47.227146+0000","last_undegraded":"2024-04-05T22:28:47.227146+0000","last_fullsized":"2024-04-05T22:28:47.227146+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:08:21.023561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160372+0000","last_change":"2024-04-05T22:28:44.188078+0000","last_active":"2024-04-05T22:28:45.160372+0000","last_peered":"2024-04-05T22:28:45.160372+0000","last_clean":"2024-04-05T22:28:45.160372+0000","last_became_active":"2024-04-05T22:28:44.187550+0000","last_became_peered":"2024-04-05T22:28:44.187550+0000","last_unstale":"2024-04-05T22:28:45.160372+0000","last_undegraded":"2024-04-05T22:28:45.160372+0000","last_fullsized":"2024-04-05T22:28:45.160372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:05:54.589924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161431+0000","last_change":"2024-04-05T22:28:43.683572+0000","last_active":"2024-04-05T22:28:45.161431+0000","last_peered":"2024-04-05T22:28:45.161431+0000","last_clean":"2024-04-05T22:28:45.161431+0000","last_became_active":"2024-04-05T22:28:43.683163+0000","last_became_peered":"2024-04-05T22:28:43.683163+0000","last_unstale":"2024-04-05T22:28:45.161431+0000","last_undegraded":"2024-04-05T22:28:45.161431+0000","last_fullsized":"2024-04-05T22:28:45.161431+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:23.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.161502+0000","last_change":"2024-04-05T22:28:44.185403+0000","last_active":"2024-04-05T22:28:45.161502+0000","last_peered":"2024-04-05T22:28:45.161502+0000","last_clean":"2024-04-05T22:28:45.161502+0000","last_became_active":"2024-04-05T22:28:44.183322+0000","last_became_peered":"2024-04-05T22:28:44.183322+0000","last_unstale":"2024-04-05T22:28:45.161502+0000","last_undegraded":"2024-04-05T22:28:45.161502+0000","last_fullsized":"2024-04-05T22:28:45.161502+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:33:06.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160656+0000","last_change":"2024-04-05T22:28:43.694345+0000","last_active":"2024-04-05T22:28:45.160656+0000","last_peered":"2024-04-05T22:28:45.160656+0000","last_clean":"2024-04-05T22:28:45.160656+0000","last_became_active":"2024-04-05T22:28:43.693514+0000","last_became_peered":"2024-04-05T22:28:43.693514+0000","last_unstale":"2024-04-05T22:28:45.160656+0000","last_undegraded":"2024-04-05T22:28:45.160656+0000","last_fullsized":"2024-04-05T22:28:45.160656+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:29.800084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225024+0000","last_change":"2024-04-05T22:28:44.186635+0000","last_active":"2024-04-05T22:28:47.225024+0000","last_peered":"2024-04-05T22:28:47.225024+0000","last_clean":"2024-04-05T22:28:47.225024+0000","last_became_active":"2024-04-05T22:28:44.186325+0000","last_became_peered":"2024-04-05T22:28:44.186325+0000","last_unstale":"2024-04-05T22:28:47.225024+0000","last_undegraded":"2024-04-05T22:28:47.225024+0000","last_fullsized":"2024-04-05T22:28:47.225024+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:32:52.294626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160670+0000","last_change":"2024-04-05T22:28:43.691684+0000","last_active":"2024-04-05T22:28:45.160670+0000","last_peered":"2024-04-05T22:28:45.160670+0000","last_clean":"2024-04-05T22:28:45.160670+0000","last_became_active":"2024-04-05T22:28:43.691425+0000","last_became_peered":"2024-04-05T22:28:43.691425+0000","last_unstale":"2024-04-05T22:28:45.160670+0000","last_undegraded":"2024-04-05T22:28:45.160670+0000","last_fullsized":"2024-04-05T22:28:45.160670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:20:24.599414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224337+0000","last_change":"2024-04-05T22:28:44.173708+0000","last_active":"2024-04-05T22:28:47.224337+0000","last_peered":"2024-04-05T22:28:47.224337+0000","last_clean":"2024-04-05T22:28:47.224337+0000","last_became_active":"2024-04-05T22:28:44.173219+0000","last_became_peered":"2024-04-05T22:28:44.173219+0000","last_unstale":"2024-04-05T22:28:47.224337+0000","last_undegraded":"2024-04-05T22:28:47.224337+0000","last_fullsized":"2024-04-05T22:28:47.224337+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:04:03.899196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.225741+0000","last_change":"2024-04-05T22:28:43.698675+0000","last_active":"2024-04-05T22:28:47.225741+0000","last_peered":"2024-04-05T22:28:47.225741+0000","last_clean":"2024-04-05T22:28:47.225741+0000","last_became_active":"2024-04-05T22:28:43.698405+0000","last_became_peered":"2024-04-05T22:28:43.698405+0000","last_unstale":"2024-04-05T22:28:47.225741+0000","last_undegraded":"2024-04-05T22:28:47.225741+0000","last_fullsized":"2024-04-05T22:28:47.225741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:01:43.085574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701498+0000","last_change":"2024-04-05T22:28:44.200946+0000","last_active":"2024-04-05T22:28:46.701498+0000","last_peered":"2024-04-05T22:28:46.701498+0000","last_clean":"2024-04-05T22:28:46.701498+0000","last_became_active":"2024-04-05T22:28:44.200687+0000","last_became_peered":"2024-04-05T22:28:44.200687+0000","last_unstale":"2024-04-05T22:28:46.701498+0000","last_undegraded":"2024-04-05T22:28:46.701498+0000","last_fullsized":"2024-04-05T22:28:46.701498+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:06:49.784818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:46.701179+0000","last_change":"2024-04-05T22:28:43.689450+0000","last_active":"2024-04-05T22:28:46.701179+0000","last_peered":"2024-04-05T22:28:46.701179+0000","last_clean":"2024-04-05T22:28:46.701179+0000","last_became_active":"2024-04-05T22:28:43.689085+0000","last_became_peered":"2024-04-05T22:28:43.689085+0000","last_unstale":"2024-04-05T22:28:46.701179+0000","last_undegraded":"2024-04-05T22:28:46.701179+0000","last_fullsized":"2024-04-05T22:28:46.701179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:12:54.463598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.159728+0000","last_change":"2024-04-05T22:28:44.178842+0000","last_active":"2024-04-05T22:28:45.159728+0000","last_peered":"2024-04-05T22:28:45.159728+0000","last_clean":"2024-04-05T22:28:45.159728+0000","last_became_active":"2024-04-05T22:28:44.178438+0000","last_became_peered":"2024-04-05T22:28:44.178438+0000","last_unstale":"2024-04-05T22:28:45.159728+0000","last_undegraded":"2024-04-05T22:28:45.159728+0000","last_fullsized":"2024-04-05T22:28:45.159728+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:48:55.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:45.160655+0000","last_change":"2024-04-05T22:28:43.696617+0000","last_active":"2024-04-05T22:28:45.160655+0000","last_peered":"2024-04-05T22:28:45.160655+0000","last_clean":"2024-04-05T22:28:45.160655+0000","last_became_active":"2024-04-05T22:28:43.696371+0000","last_became_peered":"2024-04-05T22:28:43.696371+0000","last_unstale":"2024-04-05T22:28:45.160655+0000","last_undegraded":"2024-04-05T22:28:45.160655+0000","last_fullsized":"2024-04-05T22:28:45.160655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:17:57.021850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.224726+0000","last_change":"2024-04-05T22:28:44.183893+0000","last_active":"2024-04-05T22:28:47.224726+0000","last_peered":"2024-04-05T22:28:47.224726+0000","last_clean":"2024-04-05T22:28:47.224726+0000","last_became_active":"2024-04-05T22:28:44.182460+0000","last_became_peered":"2024-04-05T22:28:44.182460+0000","last_unstale":"2024-04-05T22:28:47.224726+0000","last_undegraded":"2024-04-05T22:28:47.224726+0000","last_fullsized":"2024-04-05T22:28:47.224726+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:43.647378+0000","last_clean_scrub_stamp":"2024-04-05T22:28:43.647378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:22:35.058574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-05T22:28:47.226700+0000","last_change":"2024-04-05T22:28:43.699966+0000","last_active":"2024-04-05T22:28:47.226700+0000","last_peered":"2024-04-05T22:28:47.226700+0000","last_clean":"2024-04-05T22:28:47.226700+0000","last_became_active":"2024-04-05T22:28:43.699652+0000","last_became_peered":"2024-04-05T22:28:43.699652+0000","last_unstale":"2024-04-05T22:28:47.226700+0000","last_undegraded":"2024-04-05T22:28:47.226700+0000","last_fullsized":"2024-04-05T22:28:47.226700+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:28:42.642078+0000","last_clean_scrub_stamp":"2024-04-05T22:28:42.642078+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:06.469458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":288,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-05T22:28:44.173813+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:28:44.173813+0000","last_peered":"2024-04-05T22:28:44.173813+0000","last_clean":"2024-04-05T22:28:44.173813+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:28:44.173813+0000","last_undegraded":"2024-04-05T22:28:44.173813+0000","last_fullsized":"2024-04-05T22:28:44.173813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2138112,"data_stored":1262430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0609999999999999,"15min":1.0329999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6639999999999999,"5min":1.6759999999999999,"15min":1.796},"last":1.288},{"interface":"front","average":{"1min":1.046,"5min":1.0529999999999999,"15min":1.0149999999999999},"min":{"1min":0.67100000000000004,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.4750000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.169,"15min":1.0700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":1.5940000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.087,"15min":1.0660000000000001},"min":{"1min":0.76900000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.542,"5min":1.819,"15min":1.819},"last":0.92500000000000004}]},{"osd":2,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.157,"15min":1.169},"min":{"1min":0.81599999999999995,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":1.502,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.446},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.226},"min":{"1min":0.92000000000000004,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.5669999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1899999999999999,"15min":1.143},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.72299999999999998},"max":{"1min":1.544,"5min":1.667,"15min":1.712},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.214},"min":{"1min":0.71999999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.46,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.171}]},{"osd":4,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.212,"15min":1.2190000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.653,"5min":1.8360000000000001,"15min":1.865},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.296,"15min":1.3140000000000001},"min":{"1min":0.73399999999999999,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":1.7669999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3200000000000001,"15min":1.256},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.9590000000000001},"last":1.339},{"interface":"front","average":{"1min":1.331,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.88600000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.764,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.5269999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4990000000000001,"15min":1.468},"min":{"1min":0.95799999999999996,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":1.927,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.476,"15min":1.49},"min":{"1min":0.79700000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.8799999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.371}]}]},{"osd":6,"up_from":14,"seq":60129542240,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":665124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.054,"15min":1.0580000000000001},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":1.127},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.081,"15min":1.087},"min":{"1min":0.71899999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.337,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.095,"15min":1.1439999999999999},"min":{"1min":0.73699999999999999,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.5620000000000001,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.155},{"interface":"front","average":{"1min":1.137,"5min":1.121,"15min":1.145},"min":{"1min":0.753,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.123,"15min":1.1799999999999999},"min":{"1min":0.76500000000000001,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.319,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1699999999999999,"15min":1.2},"min":{"1min":0.84799999999999998,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.484,"5min":1.7749999999999999,"15min":2.2250000000000001},"last":1.306}]},{"osd":3,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.208,"15min":1.302},"min":{"1min":0.80500000000000005,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.29,"15min":1.379},"min":{"1min":0.79200000000000004,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.7310000000000001,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.085}]},{"osd":4,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.2749999999999999,"15min":1.323},"min":{"1min":0.998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3899999999999999,"15min":1.5349999999999999},"min":{"1min":1.091,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9259999999999999,"15min":3.27},"last":1.375}]},{"osd":5,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.274,"15min":1.4159999999999999},"min":{"1min":1.079,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.708,"5min":1.927,"15min":2.3650000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.456,"15min":1.542},"min":{"1min":1.0529999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.238}]},{"osd":7,"last update":"Fri Apr 5 22:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.365,"15min":1.518},"min":{"1min":1.1220000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.643,"5min":1.99,"15min":3.3540000000000001},"last":1.268},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3879999999999999,"15min":1.5289999999999999},"min":{"1min":1.0089999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.899,"5min":1.899,"15min":3.0920000000000001},"last":1.5009999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1515520,"data_stored":663672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.115,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.466},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.097},"min":{"1min":0.68300000000000005,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.5900000000000001,"5min":1.907,"15min":1.907},"last":1.506}]},{"osd":1,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.76600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3919999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.98399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6359999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.177,"15min":1.1599999999999999},"min":{"1min":0.82599999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3759999999999999,"5min":2.544,"15min":2.544},"last":1.421},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.89900000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5840000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.327}]},{"osd":3,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1859999999999999,"15min":1.181},"min":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7529999999999999,"5min":1.867,"15min":1.867},"last":1.286},{"interface":"front","average":{"1min":1.268,"5min":1.2909999999999999,"15min":1.3460000000000001},"min":{"1min":1.03,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.649,"5min":2.927,"15min":2.9620000000000002},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7050000000000001,"5min":1.911,"15min":2.1970000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2969999999999999,"15min":1.302},"min":{"1min":0.89900000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.784,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.597}]},{"osd":6,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.79,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.377},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5129999999999999,"15min":1.5},"min":{"1min":1.284,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.9079999999999999,"5min":2.169,"15min":2.5350000000000001},"last":1.5569999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.464,"15min":1.456},"min":{"1min":1.2010000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.919,"5min":2.028,"15min":2.2509999999999999},"last":1.631},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5,"15min":1.597},"min":{"1min":1.1799999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":4.4329999999999998},"last":1.657}]}]},{"osd":4,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1523712,"data_stored":669930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1100000000000001,"15min":1.0249999999999999},"min":{"1min":0.82599999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.466,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.085},"min":{"1min":0.73199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5369999999999999,"5min":1.859,"15min":1.859},"last":1.016}]},{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1279999999999999},"min":{"1min":0.88100000000000001,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.087},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":1.0009999999999999,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.659,"5min":2.577,"15min":2.577},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1319999999999999,"15min":1.202},"min":{"1min":0.75,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.974},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2450000000000001,"15min":1.2},"min":{"1min":1.0389999999999999,"5min":0.76200000000000001,"15min":0.67400000000000004},"max":{"1min":1.657,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3200000000000001,"15min":1.3640000000000001},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7889999999999999,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3919999999999999,"15min":1.403},"min":{"1min":1.196,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.7509999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":1.0529999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1459999999999999,"5min":2.665,"15min":2.665},"last":1.236},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.375,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.915,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.4139999999999999,"15min":1.4550000000000001},"min":{"1min":1.0229999999999999,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":1.9950000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":1.212,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.8899999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.524,"15min":1.5389999999999999},"min":{"1min":1.0760000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0019999999999998,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6259999999999999,"15min":1.591},"min":{"1min":1.3839999999999999,"5min":0.91300000000000003,"15min":0.72399999999999998},"max":{"1min":2.109,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.3839999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1527808,"data_stored":670020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0229999999999999,"5min":1.744,"15min":1.744},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5860000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.5369999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":1.006},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.0720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0580000000000001,"15min":1.073},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.391,"5min":1.675,"15min":2.2549999999999999},"last":1.629}]},{"osd":4,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1899999999999999,"15min":1.2290000000000001},"min":{"1min":0.82099999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5289999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.294},"min":{"1min":0.75900000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.286,"15min":1.351},"min":{"1min":0.73699999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.288,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6459999999999999,"5min":1.958,"15min":2.3239999999999998},"last":1.448}]},{"osd":6,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.3009999999999999,"15min":1.3500000000000001},"min":{"1min":0.875,"5min":0.505,"15min":0.505},"max":{"1min":1.6240000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.347,"15min":1.401},"min":{"1min":0.97899999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.298,"15min":2.298},"last":1.375}]},{"osd":7,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3400000000000001,"15min":1.415},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":2.468},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.377,"15min":1.415},"min":{"1min":0.93700000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.73,"5min":2.508,"15min":2.798},"last":1.3260000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542238,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1515520,"data_stored":664176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85599999999999998,"15min":0.83199999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.159,"5min":0.97499999999999998,"15min":0.92900000000000005},"min":{"1min":0.57899999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8979999999999997,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.91700000000000004,"15min":0.89000000000000001},"min":{"1min":0.49199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.97999999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.149,"15min":1.1279999999999999},"min":{"1min":0.71099999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3029999999999999,"15min":1.2430000000000001},"min":{"1min":1.0740000000000001,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.9630000000000001,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.78100000000000003,"5min":0.625,"15min":0.625},"max":{"1min":5.8639999999999999,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.2190000000000001,"15min":1.2529999999999999},"min":{"1min":0.82399999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2250000000000001,"15min":1.266},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.385}]},{"osd":6,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.27},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.2949999999999999,"15min":1.2769999999999999},"min":{"1min":0.88600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.242,"5min":2.242,"15min":2.7890000000000001},"last":1.196}]},{"osd":7,"last update":"Fri Apr 5 22:28:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3280000000000001,"15min":1.357},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.7169999999999996,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.542},{"interface":"front","average":{"1min":1.454,"5min":1.385,"15min":1.3919999999999999},"min":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.478}]}]},{"osd":1,"up_from":14,"seq":60129542240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1519616,"data_stored":669860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.363,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.89900000000000002,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.42299999999999999,"15min":0.36799999999999999},"max":{"1min":1.554,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85399999999999998,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.206,"5min":1.488,"15min":1.5389999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.999,"15min":0.99399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0880000000000001,"15min":1.097},"min":{"1min":0.53600000000000003,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.484,"5min":1.5509999999999999,"15min":1.75},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.3959999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6679999999999999,"5min":1.806,"15min":2.2789999999999999},"last":1.141}]},{"osd":5,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.258,"15min":1.262},"min":{"1min":0.84099999999999997,"5min":0.76000000000000001,"15min":0.68100000000000005},"max":{"1min":1.706,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.022},{"interface":"front","average":{"1min":1.123,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.873,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.5429999999999999,"5min":1.843,"15min":2.173},"last":1.292}]},{"osd":6,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.369,"15min":1.3340000000000001},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6439999999999999,"5min":1.9059999999999999,"15min":1.917},"last":1.258},{"interface":"front","average":{"1min":1.292,"5min":1.345,"15min":1.355},"min":{"1min":0.92200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.825,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.6319999999999999,"5min":1.9279999999999999,"15min":2.3879999999999999},"last":1.165}]}]},{"osd":0,"up_from":14,"seq":60129542240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2113536,"data_stored":1260228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.1479999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.92600000000000005,"15min":0.92000000000000004},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2569999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.2729999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.25,"5min":1.976,"15min":1.976},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0840000000000001,"15min":1.093},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5900000000000001,"5min":2.157,"15min":2.157},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2,"15min":1.1699999999999999},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.429,"5min":1.931,"15min":1.931},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3129999999999999,"15min":1.302},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.786,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2549999999999999,"15min":1.2609999999999999},"min":{"1min":0.75700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.524,"5min":2.105,"15min":2.3879999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.2130000000000001,"15min":1.198},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.357,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.878,"15min":0.65500000000000003},"max":{"1min":1.8340000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3280000000000001,"15min":1.365},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.585,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.1459999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3320000000000001,"15min":1.337},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.22},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.6080000000000001,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.022}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:28:49.097 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:28:49.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:28:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.357+0000 7fc2ce800640 1 Processor -- start 2024-04-05T22:28:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.357+0000 7fc2ce800640 1 -- start start 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2ce800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2c80cdbb0 0x7fc2c80cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2ce800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c806b7a0 con 0x7fc2c81435f0 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2ce800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c806ba80 con 0x7fc2c8144260 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2ce800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c806bd60 con 0x7fc2c80cdbb0 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2cd400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc2c81435f0 0x7fc2c806c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55438/0 (socket says 172.21.15.73:55438) 2024-04-05T22:28:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2cd400640 1 -- 172.21.15.73:0/3827320962 learned_addr learned my addr 172.21.15.73:0/3827320962 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2cde00640 1 --2- 172.21.15.73:0/3827320962 >> v2:172.21.15.157:3300/0 conn(0x7fc2c80cdbb0 0x7fc2c80cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.359+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251592641 0 0) 0x7fc2c806b7a0 con 0x7fc2c81435f0 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b0003520 con 0x7fc2c81435f0 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368120334 0 0) 0x7fc2b0003520 con 0x7fc2c81435f0 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 >> v2:172.21.15.157:3300/0 conn(0x7fc2c80cdbb0 msgr2=0x7fc2c80cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 --2- 172.21.15.73:0/3827320962 >> v2:172.21.15.157:3300/0 conn(0x7fc2c80cdbb0 0x7fc2c80cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 >> v1:172.21.15.157:6789/0 conn(0x7fc2c8144260 legacy=0x7fc2c81439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c806c040 con 0x7fc2c81435f0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc2bc001670 con 0x7fc2c81435f0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 >> v1:172.21.15.73:6789/0 conn(0x7fc2c81435f0 legacy=0x7fc2c806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2b0003970 0x7fc2b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2b00047c0 0x7fc2b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c806bd60 con 0x7fc2b0005180 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000c020 con 0x7fc2b0003970 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000c1e0 con 0x7fc2b00047c0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.360+0000 7fc2cde00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cd400640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2b0003970 0x7fc2b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cca00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2b00047c0 0x7fc2b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cde00640 1 -- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2b00047c0 msgr2=0x7fc2b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:49.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cde00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2b00047c0 0x7fc2b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cde00640 1 -- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2b0003970 msgr2=0x7fc2b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cde00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2b0003970 0x7fc2b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.361+0000 7fc2cde00640 1 -- 172.21.15.73:0/3827320962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0003520 con 0x7fc2b0005180 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.362+0000 7fc2cde00640 1 -- 172.21.15.73:0/3827320962 reap_dead start 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.362+0000 7fc2cde00640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2b000b980 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b5e74b7dfeac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.362+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002cc60 con 0x7fc2b0005180 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.362+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c000f040 con 0x7fc2b0005180 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.362+0000 7fc2c7e00640 1 -- 172.21.15.73:0/3827320962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003990 con 0x7fc2b0005180 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 -- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 msgr2=0x7fc2b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2b000b980 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2b00057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 -- 172.21.15.73:0/3827320962 shutdown_connections 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 --2- 172.21.15.73:0/3827320962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2b000b980 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 -- 172.21.15.73:0/3827320962 >> 172.21.15.73:0/3827320962 conn(0x7fc2c80b0830 msgr2=0x7fc2c80b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 -- 172.21.15.73:0/3827320962 shutdown_connections 2024-04-05T22:28:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.363+0000 7fc2ce800640 1 -- 172.21.15.73:0/3827320962 wait complete. 2024-04-05T22:28:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.365+0000 7fc2ce800640 1 Processor -- start 2024-04-05T22:28:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.365+0000 7fc2ce800640 1 -- start start 2024-04-05T22:28:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 0x7fc2c80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c815c4d0 0x7fc2c815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c815def0 con 0x7fc2b0005180 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80af0b0 con 0x7fc2c815c4d0 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.366+0000 7fc2ce800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c815d270 con 0x7fc2c80ae6f0 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48594/0 (socket says 172.21.15.73:48594) 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 -- 172.21.15.73:0/647970317 learned_addr learned my addr 172.21.15.73:0/647970317 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cde00640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c815c4d0 0x7fc2c815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cca00640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 0x7fc2c80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 -- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 msgr2=0x7fc2c80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 0x7fc2c80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 -- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c815c4d0 msgr2=0x7fc2c815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c815c4d0 0x7fc2c815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cd400640 1 -- 172.21.15.73:0/647970317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c815d550 con 0x7fc2b0005180 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.367+0000 7fc2cca00640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 0x7fc2c80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:28:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2cd400640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e7b0 tx=0x7fc2bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00eec0 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc00c8d0 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c815d830 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00cc60 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.368+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c815a8f0 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.369+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c815aab0 con 0x7fc2b0005180 2024-04-05T22:28:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.370+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fc2bc01d460 con 0x7fc2b0005180 2024-04-05T22:28:49.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.371+0000 7fc2c5600640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 0x7fc2ac06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.371+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2bc089ae0 con 0x7fc2b0005180 2024-04-05T22:28:49.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.372+0000 7fc2cca00640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 0x7fc2ac06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.373+0000 7fc2cca00640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 0x7fc2ac06d4b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2c81f15b0 tx=0x7fc2b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.375+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc2bc058130 con 0x7fc2b0005180 2024-04-05T22:28:49.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.640+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc2c804e160 con 0x7fc2b0005180 2024-04-05T22:28:49.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.648+0000 7fc2c5600640 1 -- 172.21.15.73:0/647970317 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc2bc057cc0 con 0x7fc2b0005180 2024-04-05T22:28:49.648 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.650+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 msgr2=0x7fc2ac06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.650+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 0x7fc2ac06d4b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2c81f15b0 tx=0x7fc2b8009040 comp rx=0 tx=0).stop 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 msgr2=0x7fc2c815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e7b0 tx=0x7fc2bc00ec70 comp rx=0 tx=0).stop 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 shutdown_connections 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc2ac06aff0 0x7fc2ac06d4b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c815c4d0 0x7fc2c815c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c80ae6f0 0x7fc2c80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 --2- 172.21.15.73:0/647970317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b0005180 0x7fc2c815a3b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 >> 172.21.15.73:0/647970317 conn(0x7fc2c80b0830 msgr2=0x7fc2c80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 shutdown_connections 2024-04-05T22:28:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.651+0000 7fc2ce800640 1 -- 172.21.15.73:0/647970317 wait complete. 2024-04-05T22:28:49.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:28:49.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.874+0000 7f5623000640 1 Processor -- start 2024-04-05T22:28:49.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.874+0000 7f5623000640 1 -- start start 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f5623000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f561c1587d0 0x7f561c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f5623000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f561c0ccd20 0x7f561c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f5623000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c06f5b0 con 0x7f561c06d0c0 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f5623000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c070410 con 0x7f561c1587d0 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f5623000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c158bb0 con 0x7f561c0ccd20 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f561be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f561c1587d0 0x7f561c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f561be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f561c1587d0 0x7f561c0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48598/0 (socket says 172.21.15.73:48598) 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.876+0000 7f561be00640 1 -- 172.21.15.73:0/628315851 learned_addr learned my addr 172.21.15.73:0/628315851 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.877+0000 7f561be00640 1 -- 172.21.15.73:0/628315851 >> v2:172.21.15.157:3300/0 conn(0x7f561c0ccd20 msgr2=0x7f561c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.877+0000 7f5613400640 1 --2- 172.21.15.73:0/628315851 >> v2:172.21.15.157:3300/0 conn(0x7f561c0ccd20 0x7f561c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.877+0000 7f561be00640 1 --2- 172.21.15.73:0/628315851 >> v2:172.21.15.157:3300/0 conn(0x7f561c0ccd20 0x7f561c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.877+0000 7f561be00640 1 -- 172.21.15.73:0/628315851 >> v1:172.21.15.157:6789/0 conn(0x7f561c06d0c0 legacy=0x7f561c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:49.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.877+0000 7f561be00640 1 -- 172.21.15.73:0/628315851 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c06f800 con 0x7f561c1587d0 2024-04-05T22:28:49.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.878+0000 7f561be00640 1 --2- 172.21.15.73:0/628315851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0cc7e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f560c009870 tx=0x7f560c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fc138196db3ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.878+0000 7f561b400640 1 -- 172.21.15.73:0/628315851 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c031860 con 0x7f561c1587d0 2024-04-05T22:28:49.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.878+0000 7f561b400640 1 -- 172.21.15.73:0/628315851 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c0319c0 con 0x7f561c1587d0 2024-04-05T22:28:49.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.878+0000 7f561b400640 1 -- 172.21.15.73:0/628315851 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c031d50 con 0x7f561c1587d0 2024-04-05T22:28:49.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 -- 172.21.15.73:0/628315851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 msgr2=0x7f561c0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 --2- 172.21.15.73:0/628315851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0cc7e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f560c009870 tx=0x7f560c02b900 comp rx=0 tx=0).stop 2024-04-05T22:28:49.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 -- 172.21.15.73:0/628315851 shutdown_connections 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 --2- 172.21.15.73:0/628315851 >> v2:172.21.15.157:3300/0 conn(0x7f561c0ccd20 0x7f561c06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 --2- 172.21.15.73:0/628315851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0cc7e0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 -- 172.21.15.73:0/628315851 >> 172.21.15.73:0/628315851 conn(0x7f561c0ce8e0 msgr2=0x7f561c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 -- 172.21.15.73:0/628315851 shutdown_connections 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.879+0000 7f5623000640 1 -- 172.21.15.73:0/628315851 wait complete. 2024-04-05T22:28:49.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.881+0000 7f5623000640 1 Processor -- start 2024-04-05T22:28:49.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.881+0000 7f5623000640 1 -- start start 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c06d0c0 0x7f561c148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c0c3970 0x7f561c0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c149580 con 0x7f561c1587d0 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c149270 con 0x7f561c06d0c0 2024-04-05T22:28:49.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5623000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c149790 con 0x7f561c0c3970 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48610/0 (socket says 172.21.15.73:48610) 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 -- 172.21.15.73:0/2603249047 learned_addr learned my addr 172.21.15.73:0/2603249047 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 -- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c0c3970 msgr2=0x7f561c0c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c0c3970 0x7f561c0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 -- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c06d0c0 msgr2=0x7f561c148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f561be00640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c06d0c0 0x7f561c148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c06d0c0 0x7f561c148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.882+0000 7f5613400640 1 -- 172.21.15.73:0/2603249047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0094d0 con 0x7f561c1587d0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5613400640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f560400c7f0 tx=0x7f560400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604004220 con 0x7f561c1587d0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040043a0 con 0x7f561c1587d0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604007860 con 0x7f561c1587d0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c149a70 con 0x7f561c1587d0 2024-04-05T22:28:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.883+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c14a030 con 0x7f561c1587d0 2024-04-05T22:28:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.884+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c066af0 con 0x7f561c1587d0 2024-04-05T22:28:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.885+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f5604007bd0 con 0x7f561c1587d0 2024-04-05T22:28:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.886+0000 7f5618c00640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 0x7f55f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.886+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5604089c70 con 0x7f561c1587d0 2024-04-05T22:28:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.887+0000 7f561be00640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 0x7f55f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:49.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.888+0000 7f561be00640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 0x7f55f806d4e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f560c036000 tx=0x7f560c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:49.890+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5604054690 con 0x7f561c1587d0 2024-04-05T22:28:50.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.106+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f561c14a220 con 0x7f561c1587d0 2024-04-05T22:28:50.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.107+0000 7f5618c00640 1 -- 172.21.15.73:0/2603249047 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f560401e070 con 0x7f561c1587d0 2024-04-05T22:28:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:28:50.108 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:28:50.108 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 73 2024-04-05T22:28:50.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 msgr2=0x7f55f806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 0x7f55f806d4e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f560c036000 tx=0x7f560c03c040 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 msgr2=0x7f561c0c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f560400c7f0 tx=0x7f560400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 shutdown_connections 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f55f806b020 0x7f55f806d4e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c0c3970 0x7f561c0c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561c1587d0 0x7f561c0c6390 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 --2- 172.21.15.73:0/2603249047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c06d0c0 0x7f561c148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 >> 172.21.15.73:0/2603249047 conn(0x7f561c0ce8e0 msgr2=0x7f561c158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 shutdown_connections 2024-04-05T22:28:50.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.110+0000 7f5623000640 1 -- 172.21.15.73:0/2603249047 wait complete. 2024-04-05T22:28:50.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:28:44.145472+0000', 'modified': '2024-04-05T22:28:47.182754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8217': {'gid': 8217, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6835/651015208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 651015208}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 651015208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:28:50.533 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:28:50.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-05T22:28:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.750+0000 7f6765e00640 1 Processor -- start 2024-04-05T22:28:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.750+0000 7f6765e00640 1 -- start start 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.751+0000 7f6765e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f6760162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6765e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f67600b2ba0 0x7f67600b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6765e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6760165cf0 con 0x7f6760164e50 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6765e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6760158790 con 0x7f67600b2ba0 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6765e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600b2820 con 0x7f6760161f90 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6756000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6760164e50 0x7f67600b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55450/0 (socket says 172.21.15.73:55450) 2024-04-05T22:28:50.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f6756000640 1 -- 172.21.15.73:0/158201422 learned_addr learned my addr 172.21.15.73:0/158201422 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f675f400640 1 --2- 172.21.15.73:0/158201422 >> v2:172.21.15.73:3300/0 conn(0x7f67600b2ba0 0x7f67600b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f675ea00640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f6760162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f675e000640 1 -- 172.21.15.73:0/158201422 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632733171 0 0) 0x7f6760165cf0 con 0x7f6760164e50 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.752+0000 7f675e000640 1 -- 172.21.15.73:0/158201422 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6744003520 con 0x7f6760164e50 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.753+0000 7f675f400640 1 -- 172.21.15.73:0/158201422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 msgr2=0x7f6760162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.753+0000 7f675f400640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f6760162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.753+0000 7f675f400640 1 -- 172.21.15.73:0/158201422 >> v1:172.21.15.73:6789/0 conn(0x7f6760164e50 legacy=0x7f67600b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.753+0000 7f675f400640 1 -- 172.21.15.73:0/158201422 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760165730 con 0x7f67600b2ba0 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f675f400640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67600b2ba0 0x7f67600b3030 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6758009d90 tx=0x7f675802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe883c494a3ee0f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f675e000640 1 -- 172.21.15.73:0/158201422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675802cc50 con 0x7f67600b2ba0 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f675e000640 1 -- 172.21.15.73:0/158201422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675802cdb0 con 0x7f67600b2ba0 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f675e000640 1 -- 172.21.15.73:0/158201422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758034660 con 0x7f67600b2ba0 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f6765e00640 1 -- 172.21.15.73:0/158201422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67600b2ba0 msgr2=0x7f67600b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.754+0000 7f6765e00640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67600b2ba0 0x7f67600b3030 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6758009d90 tx=0x7f675802bea0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 -- 172.21.15.73:0/158201422 shutdown_connections 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67600b2ba0 0x7f67600b3030 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 --2- 172.21.15.73:0/158201422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f6760162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 -- 172.21.15.73:0/158201422 >> 172.21.15.73:0/158201422 conn(0x7f67600ce8e0 msgr2=0x7f67600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 -- 172.21.15.73:0/158201422 shutdown_connections 2024-04-05T22:28:50.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.755+0000 7f6765e00640 1 -- 172.21.15.73:0/158201422 wait complete. 2024-04-05T22:28:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.756+0000 7f6765e00640 1 Processor -- start 2024-04-05T22:28:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.756+0000 7f6765e00640 1 -- start start 2024-04-05T22:28:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67600b2ba0 0x7f67600ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f676014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600b0400 con 0x7f6760164e50 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600abb90 con 0x7f67600b2ba0 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.757+0000 7f6765e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600ac110 con 0x7f6760161f90 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48620/0 (socket says 172.21.15.73:48620) 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 -- 172.21.15.73:0/1902604363 learned_addr learned my addr 172.21.15.73:0/1902604363 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 -- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 msgr2=0x7f676014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f6756000640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f676014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675ea00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67600b2ba0 0x7f67600ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f676014a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 -- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67600b2ba0 msgr2=0x7f67600ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67600b2ba0 0x7f67600ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.758+0000 7f675f400640 1 -- 172.21.15.73:0/1902604363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758009a70 con 0x7f6760164e50 2024-04-05T22:28:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f675f400640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6758009eb0 tx=0x7f6758004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f6757e00640 1 -- 172.21.15.73:0/1902604363 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758040650 con 0x7f6760164e50 2024-04-05T22:28:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f6757e00640 1 -- 172.21.15.73:0/1902604363 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580407b0 con 0x7f6760164e50 2024-04-05T22:28:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f6757e00640 1 -- 172.21.15.73:0/1902604363 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758040b60 con 0x7f6760164e50 2024-04-05T22:28:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67600ac3f0 con 0x7f6760164e50 2024-04-05T22:28:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.759+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600ac9b0 con 0x7f6760164e50 2024-04-05T22:28:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.761+0000 7f6757e00640 1 -- 172.21.15.73:0/1902604363 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f6758040cc0 con 0x7f6760164e50 2024-04-05T22:28:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.762+0000 7f6757e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 0x7f673c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.763+0000 7f6757e00640 1 -- 172.21.15.73:0/1902604363 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67580ac4c0 con 0x7f6760164e50 2024-04-05T22:28:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.763+0000 7f6768dda740 1 Processor -- start 2024-04-05T22:28:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.763+0000 7f675ea00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 0x7f673c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.763+0000 7f6768dda740 1 -- start start 2024-04-05T22:28:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb91f0 0x556c6fc8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb95d0 0x556c6fcbbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbfc8c0 con 0x556c6fc8c6d0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbfa050 con 0x556c6fcb91f0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbf5940 con 0x556c6fcb95d0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f675ea00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 0x7f673c06d4e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6750006fd0 tx=0x7f6750008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48630/0 (socket says 172.21.15.73:48630) 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.764+0000 7f675f400640 1 -- 172.21.15.73:0/1704928423 learned_addr learned my addr 172.21.15.73:0/1704928423 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675ea00640 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb91f0 0x556c6fc8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f6756000640 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb95d0 0x556c6fcbbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 -- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb95d0 msgr2=0x556c6fcbbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb95d0 0x556c6fcbbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 -- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb91f0 msgr2=0x556c6fc8c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb91f0 0x556c6fc8c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 -- 172.21.15.73:0/1704928423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c6fbc09e0 con 0x556c6fc8c6d0 2024-04-05T22:28:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f675f400640 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f675802bee0 tx=0x7f675807a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f36f03082d5d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f674fe00640 1 -- 172.21.15.73:0/1704928423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675807b5f0 con 0x556c6fc8c6d0 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f674fe00640 1 -- 172.21.15.73:0/1704928423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675807aef0 con 0x556c6fc8c6d0 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.765+0000 7f674fe00640 1 -- 172.21.15.73:0/1704928423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675807ed20 con 0x556c6fc8c6d0 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 -- 172.21.15.73:0/1704928423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 msgr2=0x556c6fcbc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f675802bee0 tx=0x7f675807a850 comp rx=0 tx=0).stop 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 -- 172.21.15.73:0/1704928423 shutdown_connections 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fc8c6d0 0x556c6fcbc4c0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb95d0 0x556c6fcbbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 --2- 172.21.15.73:0/1704928423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb91f0 0x556c6fc8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 -- 172.21.15.73:0/1704928423 >> 172.21.15.73:0/1704928423 conn(0x556c6fc8e400 msgr2=0x556c6fcd6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 -- 172.21.15.73:0/1704928423 shutdown_connections 2024-04-05T22:28:50.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.766+0000 7f6768dda740 1 -- 172.21.15.73:0/1704928423 wait complete. 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.767+0000 7f6768dda740 1 -- start start 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.767+0000 7f6768dda740 1 Processor -- start 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcbb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 0x556c6fcbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fb97070 con 0x556c6fcb95d0 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fb96d10 con 0x556c6fcbefb0 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c6f95cf20 con 0x556c6fcb91f0 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6768dda740 10 client.0 _ll_register_callbacks cb 0x556c6fbeefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6756000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6756000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48636/0 (socket says 172.21.15.73:48636) 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f6756000640 1 -- 172.21.15.73:0/2343221734 learned_addr learned my addr 172.21.15.73:0/2343221734 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f675ea00640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcbb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.768+0000 7f675f400640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 0x556c6fcbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 -- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 msgr2=0x556c6fcbb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcbb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 -- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 msgr2=0x556c6fcbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 0x556c6fcbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 -- 172.21.15.73:0/2343221734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600ac9b0 con 0x556c6fcb95d0 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f675f400640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 0x556c6fcbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6756000640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6748001d70 tx=0x7f674800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfac132bb5cc866 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:28:50.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748005a50 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6748005bb0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c6fc1fae0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6748007da0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6768dda740 10 client.8346 fetch_fsmap learned FSMap version 73 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.769+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556c6fbf3fc0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c6fc1fae0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f67480070d0 con 0x556c6fcb95d0 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 10 client.8346 fetch_fsmap finished waiting for FSMap version 73 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 10 client.8346 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 20 client.8346 populate_metadata read hostname 'smithi073' 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fcea090 0x556c6fcea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 4 client.8346 mds_command: new command op to 8217 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 --> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c6f8e93c0 con 0x556c6fcea090 2024-04-05T22:28:50.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f675ea00640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fcea090 0x556c6fcea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f675ea00640 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fcea090 0x556c6fcea470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a5d80709814fb server_cookie=bd7ad918229d1a22 in_seq=0 out_seq=0 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.770+0000 7f674cc00640 10 client.8346 ms_handle_connect on v2:172.21.15.73:6834/651015208 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f674cc00640 1 -- 172.21.15.73:0/2343221734 <== mds.0 v2:172.21.15.73:6834/651015208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c6f8e93c0 con 0x556c6fcea090 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f674cc00640 10 client.8346 handle_command_reply: tid=0 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 client.8346 shutdown 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 msgr2=0x556c6fce7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6748001d70 tx=0x7f674800e590 comp rx=0 tx=0).stop 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 shutdown_connections 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fcea090 0x556c6fcea470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcbefb0 0x556c6fcbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb95d0 0x556c6fce7050 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f674cc00640 0 client.8346 ms_handle_reset on v2:172.21.15.73:6834/651015208 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 --2- 172.21.15.73:0/2343221734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcbb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.771+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 >> 172.21.15.73:0/2343221734 conn(0x556c6fc8c6d0 msgr2=0x556c6fc90070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.772+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 shutdown_connections 2024-04-05T22:28:50.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.772+0000 7f6768dda740 1 -- 172.21.15.73:0/2343221734 wait complete. 2024-04-05T22:28:50.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.772+0000 7f6768dda740 20 client.8346 trim_cache size 0 max 16384 2024-04-05T22:28:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.788+0000 7f6768dda740 1 Processor -- start 2024-04-05T22:28:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.788+0000 7f6768dda740 1 -- start start 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fc92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcbefb0 0x556c6fcda490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fcdae10 con 0x556c6fcb91f0 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fcda9d0 con 0x556c6fcb95d0 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbee860 con 0x556c6fcbefb0 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f675ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f675ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48648/0 (socket says 172.21.15.73:48648) 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.789+0000 7f675ea00640 1 -- 172.21.15.73:0/3682934328 learned_addr learned my addr 172.21.15.73:0/3682934328 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 -- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcbefb0 msgr2=0x556c6fcda490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcbefb0 0x556c6fcda490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 -- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 msgr2=0x556c6fc92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fc92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 -- 172.21.15.73:0/3682934328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c6fc28f80 con 0x556c6fcb91f0 2024-04-05T22:28:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f675ea00640 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x556c6fcbdb70 tx=0x7f6750014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606ba05af22ddec server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f6754c00640 1 -- 172.21.15.73:0/3682934328 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750027e70 con 0x556c6fcb91f0 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f6754c00640 1 -- 172.21.15.73:0/3682934328 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000ea30 con 0x556c6fcb91f0 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.790+0000 7f6754c00640 1 -- 172.21.15.73:0/3682934328 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000eda0 con 0x556c6fcb91f0 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 -- 172.21.15.73:0/3682934328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 msgr2=0x556c6fc8d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x556c6fcbdb70 tx=0x7f6750014000 comp rx=0 tx=0).stop 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 -- 172.21.15.73:0/3682934328 shutdown_connections 2024-04-05T22:28:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcbefb0 0x556c6fcda490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fc92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 --2- 172.21.15.73:0/3682934328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcb91f0 0x556c6fc8d460 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 -- 172.21.15.73:0/3682934328 >> 172.21.15.73:0/3682934328 conn(0x556c6fc8c6d0 msgr2=0x556c6fcbff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 -- 172.21.15.73:0/3682934328 shutdown_connections 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.791+0000 7f6768dda740 1 -- 172.21.15.73:0/3682934328 wait complete. 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.792+0000 7f6768dda740 1 -- start start 2024-04-05T22:28:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.792+0000 7f6768dda740 1 Processor -- start 2024-04-05T22:28:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcb7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fcb7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fce5730 con 0x556c6fcbefb0 2024-04-05T22:28:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fcf8540 con 0x556c6fcb95d0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c6fcb9da0 con 0x556c6fcb91f0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f6768dda740 10 client.0 _ll_register_callbacks cb 0x556c6fc8c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f675f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48654/0 (socket says 172.21.15.73:48654) 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f675f400640 1 -- 172.21.15.73:0/2571629615 learned_addr learned my addr 172.21.15.73:0/2571629615 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f675f400640 1 -- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 msgr2=0x556c6fcb7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.793+0000 7f675ea00640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f6756000640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fcb7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f675f400640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcb7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f675f400640 1 -- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 msgr2=0x556c6fcb7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f675f400640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fcb7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f675f400640 1 -- 172.21.15.73:0/2571629615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675001d040 con 0x556c6fcbefb0 2024-04-05T22:28:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f675f400640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f675807ae80 tx=0x7f6758034f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1487024b055caad server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758081040 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f675807b5f0 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c6fc8c4b0 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67580832e0 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f6768dda740 10 client.8352 fetch_fsmap learned FSMap version 73 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.794+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556c6fb96280 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c6fc8c4b0 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f675807b150 con 0x556c6fcbefb0 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 10 client.8352 fetch_fsmap finished waiting for FSMap version 73 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 10 client.8352 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 20 client.8352 populate_metadata read hostname 'smithi073' 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fc8b940 0x556c6fcf7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:50.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 4 client.8352 mds_command: new command op to 8217 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 --> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f675807aef0 con 0x556c6fc8b940 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f675ea00640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fc8b940 0x556c6fcf7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f674cc00640 10 client.8352 ms_handle_connect on v2:172.21.15.73:6834/651015208 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.795+0000 7f675ea00640 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fc8b940 0x556c6fcf7a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f876858e78b1d50 server_cookie=598cb77ce4936396 in_seq=0 out_seq=0 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f674cc00640 1 -- 172.21.15.73:0/2571629615 <== mds.0 v2:172.21.15.73:6834/651015208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f675807aef0 con 0x556c6fc8b940 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f674cc00640 10 client.8352 handle_command_reply: tid=0 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 client.8352 shutdown 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 msgr2=0x556c6fcf7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f675807ae80 tx=0x7f6758034f70 comp rx=0 tx=0).stop 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 shutdown_connections 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:6834/651015208,v1:172.21.15.73:6835/651015208] conn(0x556c6fc8b940 0x556c6fcf7a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c6fcbefb0 0x556c6fcf7510 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c6fcb95d0 0x556c6fcb7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 --2- 172.21.15.73:0/2571629615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556c6fcb91f0 0x556c6fcb7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f674cc00640 0 client.8352 ms_handle_reset on v2:172.21.15.73:6834/651015208 2024-04-05T22:28:50.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 >> 172.21.15.73:0/2571629615 conn(0x556c6fc8c6d0 msgr2=0x556c6fc8b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 shutdown_connections 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 1 -- 172.21.15.73:0/2571629615 wait complete. 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.796+0000 7f6768dda740 20 client.8352 trim_cache size 0 max 16384 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 8217, 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.6464831440000003, 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 73, 2024-04-05T22:28:50.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 122, 2024-04-05T22:28:50.802 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 122, 2024-04-05T22:28:50.802 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 12.356914842 2024-04-05T22:28:50.802 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 msgr2=0x7f673c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 0x7f673c06d4e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6750006fd0 tx=0x7f6750008040 comp rx=0 tx=0).stop 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 msgr2=0x7f676014ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6758009eb0 tx=0x7f6758004a40 comp rx=0 tx=0).stop 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 shutdown_connections 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f673c06b020 0x7f673c06d4e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6760164e50 0x7f676014ac80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6760161f90 0x7f676014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 --2- 172.21.15.73:0/1902604363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67600b2ba0 0x7f67600ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 >> 172.21.15.73:0/1902604363 conn(0x7f67600ce8e0 msgr2=0x7f67600cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 shutdown_connections 2024-04-05T22:28:50.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.797+0000 7f6765e00640 1 -- 172.21.15.73:0/1902604363 wait complete. 2024-04-05T22:28:50.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:28:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.999+0000 7ff244800640 1 Processor -- start 2024-04-05T22:28:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:50.999+0000 7ff244800640 1 -- start start 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.000+0000 7ff244800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 0x7ff23c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.000+0000 7ff244800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff23c15e850 0x7ff23c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.000+0000 7ff244800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c06f5b0 con 0x7ff23c0ccd20 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff244800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c15f240 con 0x7ff23c15e850 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff244800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c070410 con 0x7ff23c1587d0 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff241400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff23c0ccd20 0x7ff23c06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55456/0 (socket says 172.21.15.73:55456) 2024-04-05T22:28:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff241400640 1 -- 172.21.15.73:0/2370450572 learned_addr learned my addr 172.21.15.73:0/2370450572 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff242800640 1 --2- 172.21.15.73:0/2370450572 >> v2:172.21.15.157:3300/0 conn(0x7ff23c15e850 0x7ff23c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff241e00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 0x7ff23c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119065331 0 0) 0x7ff23c06f5b0 con 0x7ff23c0ccd20 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.001+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff224003520 con 0x7ff23c0ccd20 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804677146 0 0) 0x7ff224003520 con 0x7ff23c0ccd20 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 msgr2=0x7ff23c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 0x7ff23c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 >> v2:172.21.15.157:3300/0 conn(0x7ff23c15e850 msgr2=0x7ff23c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 --2- 172.21.15.73:0/2370450572 >> v2:172.21.15.157:3300/0 conn(0x7ff23c15e850 0x7ff23c15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c158bb0 con 0x7ff23c0ccd20 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff22c001670 con 0x7ff23c0ccd20 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 >> v1:172.21.15.73:6789/0 conn(0x7ff23c0ccd20 legacy=0x7ff23c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff224003970 0x7ff224003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff224004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 0x7ff224005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c15f240 con 0x7ff2240047c0 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c070410 con 0x7ff224003970 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22400b980 con 0x7ff23c1587d0 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff241400640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff224004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.002+0000 7ff241e00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff224003970 0x7ff224003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241400640 1 -- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 msgr2=0x7ff224005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241400640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c1587d0 0x7ff224005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241400640 1 -- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff224003970 msgr2=0x7ff224003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241400640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff224003970 0x7ff224003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241400640 1 -- 172.21.15.73:0/2370450572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224003520 con 0x7ff2240047c0 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.003+0000 7ff241e00640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff224003970 0x7ff224003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:51.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff241400640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff224004c40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c210 tx=0x7ff2240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceee6f52d3e70bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:51.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c02cae0 con 0x7ff2240047c0 2024-04-05T22:28:51.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c02cc40 con 0x7ff2240047c0 2024-04-05T22:28:51.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff240a00640 1 -- 172.21.15.73:0/2370450572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0058d0 con 0x7ff2240047c0 2024-04-05T22:28:51.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff244800640 1 -- 172.21.15.73:0/2370450572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 msgr2=0x7ff224004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff244800640 1 --2- 172.21.15.73:0/2370450572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff224004c40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c210 tx=0x7ff2240057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.004+0000 7ff242800640 1 -- 172.21.15.73:0/2370450572 reap_dead start 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.005+0000 7ff244800640 1 -- 172.21.15.73:0/2370450572 shutdown_connections 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.005+0000 7ff244800640 1 -- 172.21.15.73:0/2370450572 >> 172.21.15.73:0/2370450572 conn(0x7ff23c0ce8e0 msgr2=0x7ff23c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.005+0000 7ff244800640 1 -- 172.21.15.73:0/2370450572 shutdown_connections 2024-04-05T22:28:51.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.005+0000 7ff244800640 1 -- 172.21.15.73:0/2370450572 wait complete. 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.006+0000 7ff244800640 1 Processor -- start 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.006+0000 7ff244800640 1 -- start start 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 0x7ff23c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c0a9420 0x7ff23c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0af280 con 0x7ff2240047c0 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0a8f90 con 0x7ff23c0aff70 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff244800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0a8660 con 0x7ff23c0a9420 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.007+0000 7ff241e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48684/0 (socket says 172.21.15.73:48684) 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 -- 172.21.15.73:0/2012547946 learned_addr learned my addr 172.21.15.73:0/2012547946 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:51.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241400640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 0x7ff23c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 -- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c0a9420 msgr2=0x7ff23c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff242800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c0a9420 0x7ff23c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c0a9420 0x7ff23c15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 -- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 msgr2=0x7ff23c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 0x7ff23c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 -- 172.21.15.73:0/2012547946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0a8940 con 0x7ff2240047c0 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241400640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 0x7ff23c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.008+0000 7ff241e00640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff2300108b0 tx=0x7ff230010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.009+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000eb60 con 0x7ff2240047c0 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.009+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000ecc0 con 0x7ff2240047c0 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.009+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0a8c20 con 0x7ff2240047c0 2024-04-05T22:28:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.009+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23001c400 con 0x7ff2240047c0 2024-04-05T22:28:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.009+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c13f9a0 con 0x7ff2240047c0 2024-04-05T22:28:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.010+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c066af0 con 0x7ff2240047c0 2024-04-05T22:28:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.011+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7ff23001c560 con 0x7ff2240047c0 2024-04-05T22:28:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.012+0000 7ff236000640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 0x7ff22006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.012+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff230089f30 con 0x7ff2240047c0 2024-04-05T22:28:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.013+0000 7ff241400640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 0x7ff22006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.014+0000 7ff241400640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 0x7ff22006d4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff22c034000 tx=0x7ff22c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.017+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff230054c30 con 0x7ff2240047c0 2024-04-05T22:28:51.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.235+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff23c068160 con 0x7ff2240047c0 2024-04-05T22:28:51.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.236+0000 7ff236000640 1 -- 172.21.15.73:0/2012547946 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff230058250 con 0x7ff2240047c0 2024-04-05T22:28:51.236 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.238+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 msgr2=0x7ff22006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.238+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 0x7ff22006d4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff22c034000 tx=0x7ff22c007000 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.238+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 msgr2=0x7ff23c13f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.238+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff2300108b0 tx=0x7ff230010d70 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 shutdown_connections 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff22006aff0 0x7ff22006d4b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff23c0a9420 0x7ff23c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff23c0aff70 0x7ff23c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 --2- 172.21.15.73:0/2012547946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2240047c0 0x7ff23c13f280 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 >> 172.21.15.73:0/2012547946 conn(0x7ff23c0ce8e0 msgr2=0x7ff23c158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:51.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 shutdown_connections 2024-04-05T22:28:51.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:51.239+0000 7ff244800640 1 -- 172.21.15.73:0/2012547946 wait complete. 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:28:51.256 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:28:51.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.257 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:28:51.304 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:28:51.305 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42132sec preferred_lft 42132sec 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.306 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:51.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.307 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:28:51.390 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:28:51.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.395 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:28:51.447 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:28:51.447 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:28:51.447 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:28:51.447 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:28:51.448 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:28:51.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:28:51.448 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:28:51.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.622 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:28:51.671 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:28:51.671 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:28:51.672 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:28:51.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:28:51.768 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:28:51.769 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:28:51.821 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:28:51.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:28:51.821 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:28:51.868 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:28:51.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:51.869 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:52.049 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:52.049 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:28:52.085 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:52.085 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:28:52.085 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:28:52.085 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:28:52.085 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:28:52.089 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:28:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:28:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:28:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:28:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:28:52.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:28:52.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:52.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:28:52.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:52.178 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:28:52.241 INFO:teuthology.orchestra.run.smithi073.stdout:client5549 2024-04-05T22:28:52.242 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:52.242 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:52.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:28:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.549+0000 7f6f91c00640 1 Processor -- start 2024-04-05T22:28:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.549+0000 7f6f91c00640 1 -- start start 2024-04-05T22:28:52.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.550+0000 7f6f91c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:52.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.550+0000 7f6f91c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c06f5b0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.550+0000 7f6f91c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c070410 con 0x7f6f8c15e850 2024-04-05T22:28:52.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.550+0000 7f6f91c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8c06d500 con 0x7f6f8c06cb80 2024-04-05T22:28:52.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48686/0 (socket says 172.21.15.73:48686) 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f8aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6f8c1587d0 0x7f6f8c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55460/0 (socket says 172.21.15.73:55460) 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 -- 172.21.15.73:0/413136170 learned_addr learned my addr 172.21.15.73:0/413136170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 -- 172.21.15.73:0/413136170 >> v1:172.21.15.157:6789/0 conn(0x7f6f8c15e850 legacy=0x7f6f8c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 -- 172.21.15.73:0/413136170 >> v1:172.21.15.73:6789/0 conn(0x7f6f8c1587d0 legacy=0x7f6f8c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.551+0000 7f6f83e00640 1 -- 172.21.15.73:0/413136170 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0ccef0 con 0x7f6f8c06cb80 2024-04-05T22:28:52.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.552+0000 7f6f83e00640 1 --2- 172.21.15.73:0/413136170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009870 tx=0x7f6f7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f746ab6a2428dc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:52.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f8a000640 1 -- 172.21.15.73:0/413136170 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74030860 con 0x7f6f8c06cb80 2024-04-05T22:28:52.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f8a000640 1 -- 172.21.15.73:0/413136170 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f740309c0 con 0x7f6f8c06cb80 2024-04-05T22:28:52.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f8a000640 1 -- 172.21.15.73:0/413136170 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74030d50 con 0x7f6f8c06cb80 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 -- 172.21.15.73:0/413136170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c06cb80 msgr2=0x7f6f8c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 --2- 172.21.15.73:0/413136170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009870 tx=0x7f6f7402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 -- 172.21.15.73:0/413136170 shutdown_connections 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 --2- 172.21.15.73:0/413136170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c06cb80 0x7f6f8c06cfc0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 -- 172.21.15.73:0/413136170 >> 172.21.15.73:0/413136170 conn(0x7f6f8c0ce8e0 msgr2=0x7f6f8c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 -- 172.21.15.73:0/413136170 shutdown_connections 2024-04-05T22:28:52.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.553+0000 7f6f91c00640 1 -- 172.21.15.73:0/413136170 wait complete. 2024-04-05T22:28:52.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.555+0000 7f6f91c00640 1 Processor -- start 2024-04-05T22:28:52.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.555+0000 7f6f91c00640 1 -- start start 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f8c06cb80 0x7f6f8c1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 0x7f6f8c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1f2330 con 0x7f6f8c1587d0 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1f2610 con 0x7f6f8c15e850 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f91c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1f28f0 con 0x7f6f8c06cb80 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48688/0 (socket says 172.21.15.73:48688) 2024-04-05T22:28:52.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.556+0000 7f6f83e00640 1 -- 172.21.15.73:0/2812430674 learned_addr learned my addr 172.21.15.73:0/2812430674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f8aa00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f8c06cb80 0x7f6f8c1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f8b400640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 0x7f6f8c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 -- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f8c06cb80 msgr2=0x7f6f8c1f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f8c06cb80 0x7f6f8c1f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 -- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 msgr2=0x7f6f8c15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 0x7f6f8c15eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 -- 172.21.15.73:0/2812430674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740094d0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f8b400640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 0x7f6f8c15eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:52.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f83e00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009870 tx=0x7f6f74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.557+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74030810 con 0x7f6f8c1587d0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.558+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f74043400 con 0x7f6f8c1587d0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.558+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c1f2bd0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.558+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f740437b0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.558+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c0ad210 con 0x7f6f8c1587d0 2024-04-05T22:28:52.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.559+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c066af0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.560+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f6f74043910 con 0x7f6f8c1587d0 2024-04-05T22:28:52.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.561+0000 7f6f82a00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 0x7f6f6806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:52.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.561+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f7403e070 con 0x7f6f8c1587d0 2024-04-05T22:28:52.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.561+0000 7f6f8aa00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 0x7f6f6806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:52.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.562+0000 7f6f8aa00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 0x7f6f6806d4e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:52.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.565+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6f74077b50 con 0x7f6f8c1587d0 2024-04-05T22:28:52.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.798+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6f8c1f30f0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.798+0000 7f6f82a00640 1 -- 172.21.15.73:0/2812430674 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6f8c1f30f0 con 0x7f6f8c1587d0 2024-04-05T22:28:52.799 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:28:52.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 msgr2=0x7f6f6806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 0x7f6f6806d4e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c009040 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 msgr2=0x7f6f8c0accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009870 tx=0x7f6f74004730 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 shutdown_connections 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f6806b020 0x7f6f6806d4e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c15e850 0x7f6f8c15eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f8c1587d0 0x7f6f8c0accd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 --2- 172.21.15.73:0/2812430674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f8c06cb80 0x7f6f8c1f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 >> 172.21.15.73:0/2812430674 conn(0x7f6f8c0ce8e0 msgr2=0x7f6f8c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 shutdown_connections 2024-04-05T22:28:52.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:52.801+0000 7f6f91c00640 1 -- 172.21.15.73:0/2812430674 wait complete. 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:28:52.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:28:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:52.819 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:28:52.877 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42154sec preferred_lft 42154sec 2024-04-05T22:28:52.878 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:28:52.879 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:28:52.880 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:28:52.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:52.881 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:28:52.972 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:28:52.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:52.977 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:28:53.031 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:28:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:28:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:28:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:28:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:28:53.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.032 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:28:53.032 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:28:53.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:28:53.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:28:53.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:28:53.033 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:28:53.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.211 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:28:53.270 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:28:53.271 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:28:53.274 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:28:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:28:53.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:28:53.380 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:28:53.442 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:28:53.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:28:53.443 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:28:53.496 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:28:53.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:53.634 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:53.634 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:28:53.665 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:28:53.665 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:28:53.665 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:28:53.666 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:28:53.666 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:28:53.667 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:28:53.667 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:28:53.667 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:28:53.667 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:28:53.667 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:28:53.671 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:28:53.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.671 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:28:53.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:53.755 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:28:53.807 INFO:teuthology.orchestra.run.smithi157.stdout:client5552 2024-04-05T22:28:53.807 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:28:53.807 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:28:53.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:28:54.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.073+0000 7efc8cc00640 1 Processor -- start 2024-04-05T22:28:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.073+0000 7efc8cc00640 1 -- start start 2024-04-05T22:28:54.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc8cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc880b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc8cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8815d900 con 0x7efc881541a0 2024-04-05T22:28:54.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc8cc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8815e1a0 con 0x7efc88154580 2024-04-05T22:28:54.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc8cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc880b3490 con 0x7efc880cf3e0 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc85e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efc881541a0 0x7efc880cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55474/0 (socket says 172.21.15.73:55474) 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc85e00640 1 -- 172.21.15.73:0/1882489186 learned_addr learned my addr 172.21.15.73:0/1882489186 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507163576 0 0) 0x7efc8815d900 con 0x7efc881541a0 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.075+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc6c005730 con 0x7efc881541a0 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc85400640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc880b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132907747 0 0) 0x7efc6c005730 con 0x7efc881541a0 2024-04-05T22:28:54.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 msgr2=0x7efc880b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc880b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 >> v1:172.21.15.157:6789/0 conn(0x7efc88154580 legacy=0x7efc8815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8815db50 con 0x7efc881541a0 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7efc70001670 con 0x7efc881541a0 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 >> v1:172.21.15.73:6789/0 conn(0x7efc881541a0 legacy=0x7efc880cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c005bb0 0x7efc6c005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc6c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f10 0x7efc6c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc880b3490 con 0x7efc6c006f10 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c00dcc0 con 0x7efc6c005bb0 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c00de80 con 0x7efc880cf3e0 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.076+0000 7efc86800640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f10 0x7efc6c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc85e00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c005bb0 0x7efc6c005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc85400640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc6c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc86800640 1 -- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 msgr2=0x7efc6c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc86800640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc6c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:54.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc86800640 1 -- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c005bb0 msgr2=0x7efc6c005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc86800640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c005bb0 0x7efc6c005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc86800640 1 -- 172.21.15.73:0/1882489186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c005730 con 0x7efc6c006f10 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.077+0000 7efc85400640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc880cf3e0 0x7efc6c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.078+0000 7efc86800640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f10 0x7efc6c007390 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7efc80003d80 tx=0x7efc6c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de59e01b43ac4f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.078+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80032600 con 0x7efc6c006f10 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.078+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8002e050 con 0x7efc6c006f10 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.078+0000 7efc84a00640 1 -- 172.21.15.73:0/1882489186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800329a0 con 0x7efc6c006f10 2024-04-05T22:28:54.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 -- 172.21.15.73:0/1882489186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f10 msgr2=0x7efc6c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 --2- 172.21.15.73:0/1882489186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f10 0x7efc6c007390 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7efc80003d80 tx=0x7efc6c007a00 comp rx=0 tx=0).stop 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc86800640 1 -- 172.21.15.73:0/1882489186 reap_dead start 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 -- 172.21.15.73:0/1882489186 shutdown_connections 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 -- 172.21.15.73:0/1882489186 >> 172.21.15.73:0/1882489186 conn(0x7efc8806a0d0 msgr2=0x7efc8806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 -- 172.21.15.73:0/1882489186 shutdown_connections 2024-04-05T22:28:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.079+0000 7efc8cc00640 1 -- 172.21.15.73:0/1882489186 wait complete. 2024-04-05T22:28:54.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.081+0000 7efc8cc00640 1 Processor -- start 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.081+0000 7efc8cc00640 1 -- start start 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 0x7efc88155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc8814c120 0x7efc8806c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc881564f0 con 0x7efc8814c5a0 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc881567d0 con 0x7efc8814c120 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc8cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc88156ab0 con 0x7efc6c006f10 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48694/0 (socket says 172.21.15.73:48694) 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 -- 172.21.15.73:0/3017182723 learned_addr learned my addr 172.21.15.73:0/3017182723 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc85400640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc8814c120 0x7efc8806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc85e00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 0x7efc88155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 -- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 msgr2=0x7efc88155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 0x7efc88155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 -- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc8814c120 msgr2=0x7efc8806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc8814c120 0x7efc8806c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.082+0000 7efc86800640 1 -- 172.21.15.73:0/3017182723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88156d90 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc85e00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 0x7efc88155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc86800640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7efc80035040 tx=0x7efc800036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800069e0 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80006b60 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc88156100 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80007030 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.083+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc880cc000 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.084+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8815b0a0 con 0x7efc8814c5a0 2024-04-05T22:28:54.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.085+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7efc80007190 con 0x7efc8814c5a0 2024-04-05T22:28:54.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.087+0000 7efc7e000640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 0x7efc6406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:54.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.087+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc80000e40 con 0x7efc8814c5a0 2024-04-05T22:28:54.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.087+0000 7efc85e00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 0x7efc6406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:54.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.088+0000 7efc85e00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 0x7efc6406d4b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc70000f60 tx=0x7efc70000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:54.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.091+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7efc80075540 con 0x7efc8814c5a0 2024-04-05T22:28:54.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:54.324+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efc88156f80 con 0x7efc8814c5a0 2024-04-05T22:28:55.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.325+0000 7efc7e000640 1 -- 172.21.15.73:0/3017182723 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 136+0+0 (secure 0 0 0) 0x7efc80078b60 con 0x7efc8814c5a0 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 msgr2=0x7efc6406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 0x7efc6406d4b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc70000f60 tx=0x7efc70000ed0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 msgr2=0x7efc8806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7efc80035040 tx=0x7efc800036f0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 shutdown_connections 2024-04-05T22:28:55.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efc6406aff0 0x7efc6406d4b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc8814c5a0 0x7efc8806c9b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc8814c120 0x7efc8806c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 --2- 172.21.15.73:0/3017182723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc6c006f10 0x7efc88155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 >> 172.21.15.73:0/3017182723 conn(0x7efc8806a0d0 msgr2=0x7efc88158940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 shutdown_connections 2024-04-05T22:28:55.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.329+0000 7efc8cc00640 1 -- 172.21.15.73:0/3017182723 wait complete. 2024-04-05T22:28:55.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:28:55.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.553+0000 7fa79f000640 1 Processor -- start 2024-04-05T22:28:55.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.553+0000 7fa79f000640 1 -- start start 2024-04-05T22:28:55.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa79f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7981587d0 0x7fa7980cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa79f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa79815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa79f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79806f5b0 con 0x7fa7980ccd20 2024-04-05T22:28:55.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa79f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79815f240 con 0x7fa7981587d0 2024-04-05T22:28:55.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa79f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa798070410 con 0x7fa79815e850 2024-04-05T22:28:55.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.554+0000 7fa78fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa7980ccd20 0x7fa79806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55478/0 (socket says 172.21.15.73:55478) 2024-04-05T22:28:55.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa78fe00640 1 -- 172.21.15.73:0/4247160011 learned_addr learned my addr 172.21.15.73:0/4247160011 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658440439 0 0) 0x7fa79806f5b0 con 0x7fa7980ccd20 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa77c003520 con 0x7fa7980ccd20 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797e00640 1 --2- 172.21.15.73:0/4247160011 >> v2:172.21.15.157:3300/0 conn(0x7fa7981587d0 0x7fa7980cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa79d000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa79815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920968600 0 0) 0x7fa77c003520 con 0x7fa7980ccd20 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 msgr2=0x7fa79815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa79815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 >> v2:172.21.15.157:3300/0 conn(0x7fa7981587d0 msgr2=0x7fa7980cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 --2- 172.21.15.73:0/4247160011 >> v2:172.21.15.157:3300/0 conn(0x7fa7981587d0 0x7fa7980cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa798158bb0 con 0x7fa7980ccd20 2024-04-05T22:28:55.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa780001670 con 0x7fa7980ccd20 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 >> v1:172.21.15.73:6789/0 conn(0x7fa7980ccd20 legacy=0x7fa79806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa77c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c003fa0 0x7fa77c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.555+0000 7fa797400640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77c004d00 0x7fa77c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79815f240 con 0x7fa77c004d00 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798070410 con 0x7fa77c003fa0 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c00ba80 con 0x7fa79815e850 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77c004d00 0x7fa77c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa797e00640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa77c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa78fe00640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c003fa0 0x7fa77c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 -- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 msgr2=0x7fa77c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79815e850 0x7fa77c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 -- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c003fa0 msgr2=0x7fa77c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:55.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c003fa0 0x7fa77c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.556+0000 7fa79d000640 1 -- 172.21.15.73:0/4247160011 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c003520 con 0x7fa77c004d00 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.557+0000 7fa79d000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77c004d00 0x7fa77c005180 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa790007360 tx=0x7fa77c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c552d6186f36f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.557+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79002ccd0 con 0x7fa77c004d00 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79002ce30 con 0x7fa77c004d00 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa797400640 1 -- 172.21.15.73:0/4247160011 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790034660 con 0x7fa77c004d00 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 -- 172.21.15.73:0/4247160011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77c004d00 msgr2=0x7fa77c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 --2- 172.21.15.73:0/4247160011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77c004d00 0x7fa77c005180 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa790007360 tx=0x7fa77c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79d000640 1 -- 172.21.15.73:0/4247160011 reap_dead start 2024-04-05T22:28:55.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 -- 172.21.15.73:0/4247160011 shutdown_connections 2024-04-05T22:28:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 -- 172.21.15.73:0/4247160011 >> 172.21.15.73:0/4247160011 conn(0x7fa7980ce8e0 msgr2=0x7fa7980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 -- 172.21.15.73:0/4247160011 shutdown_connections 2024-04-05T22:28:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.558+0000 7fa79f000640 1 -- 172.21.15.73:0/4247160011 wait complete. 2024-04-05T22:28:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.560+0000 7fa79f000640 1 Processor -- start 2024-04-05T22:28:55.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.560+0000 7fa79f000640 1 -- start start 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c004d00 0x7fa7980c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7981f5f70 0x7fa7981f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981574d0 con 0x7fa798156b10 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980c47f0 con 0x7fa77c004d00 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa79f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b1780 con 0x7fa7981f5f70 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48704/0 (socket says 172.21.15.73:48704) 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.561+0000 7fa78fe00640 1 -- 172.21.15.73:0/459255109 learned_addr learned my addr 172.21.15.73:0/459255109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:55.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa79d000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7981f5f70 0x7fa7981f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa797e00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c004d00 0x7fa7980c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 -- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7981f5f70 msgr2=0x7fa7981f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7981f5f70 0x7fa7981f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 -- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c004d00 msgr2=0x7fa7980c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c004d00 0x7fa7980c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 -- 172.21.15.73:0/459255109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980b1a60 con 0x7fa798156b10 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa78fe00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa780000f60 tx=0x7fa78000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800074e0 con 0x7fa798156b10 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780007640 con 0x7fa798156b10 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780004a00 con 0x7fa798156b10 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7980b1d40 con 0x7fa798156b10 2024-04-05T22:28:55.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.562+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981f6960 con 0x7fa798156b10 2024-04-05T22:28:55.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.563+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa798066af0 con 0x7fa798156b10 2024-04-05T22:28:55.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.564+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fa780004b60 con 0x7fa798156b10 2024-04-05T22:28:55.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.565+0000 7fa794c00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 0x7fa77406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:55.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.565+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa780089550 con 0x7fa798156b10 2024-04-05T22:28:55.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.566+0000 7fa797e00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 0x7fa77406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:55.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.567+0000 7fa797e00640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 0x7fa77406d4b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa788006fd0 tx=0x7fa788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:55.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.570+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa780054090 con 0x7fa798156b10 2024-04-05T22:28:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.783+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa79806fdc0 con 0x7fa77406aff0 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.787+0000 7fa794c00640 1 -- 172.21.15.73:0/459255109 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa78800be00 con 0x7fa77406aff0 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:55.787 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:28:55.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 msgr2=0x7fa77406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 0x7fa77406d4b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa788006fd0 tx=0x7fa788008040 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 msgr2=0x7fa798156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa780000f60 tx=0x7fa78000dd40 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 shutdown_connections 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa77406aff0 0x7fa77406d4b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7981f5f70 0x7fa7981f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa798156b10 0x7fa798156f90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 --2- 172.21.15.73:0/459255109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c004d00 0x7fa7980c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 >> 172.21.15.73:0/459255109 conn(0x7fa7980ce8e0 msgr2=0x7fa79815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 shutdown_connections 2024-04-05T22:28:55.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:55.789+0000 7fa79f000640 1 -- 172.21.15.73:0/459255109 wait complete. 2024-04-05T22:28:55.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:28:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.011+0000 7f752b800640 1 Processor -- start 2024-04-05T22:28:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.011+0000 7f752b800640 1 -- start start 2024-04-05T22:28:56.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f752b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752415abe0 0x7f75240712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f752b800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f752406bc40 0x7f7524164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f752b800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752415afc0 con 0x7f752406c9a0 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f752b800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752406f5b0 con 0x7f752406bc40 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f752b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752406f890 con 0x7f752415abe0 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f752406bc40 0x7f7524164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f752406bc40 0x7f7524164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60732/0 (socket says 172.21.15.73:60732) 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 -- 172.21.15.73:0/4031653091 learned_addr learned my addr 172.21.15.73:0/4031653091 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 -- 172.21.15.73:0/4031653091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752415abe0 msgr2=0x7f75240712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 --2- 172.21.15.73:0/4031653091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752415abe0 0x7f75240712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 -- 172.21.15.73:0/4031653091 >> v1:172.21.15.157:6789/0 conn(0x7f752406c9a0 legacy=0x7f752406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.013+0000 7f7529800640 1 -- 172.21.15.73:0/4031653091 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752406fb70 con 0x7f752406bc40 2024-04-05T22:28:56.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.014+0000 7f7529800640 1 --2- 172.21.15.73:0/4031653091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752406bc40 0x7f7524164740 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f751c009e50 tx=0x7f751c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7aae89b6ca9700 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:56.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.015+0000 7f7523400640 1 -- 172.21.15.73:0/4031653091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c02c950 con 0x7f752406bc40 2024-04-05T22:28:56.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.015+0000 7f7523400640 1 -- 172.21.15.73:0/4031653091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c02cab0 con 0x7f752406bc40 2024-04-05T22:28:56.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.015+0000 7f7523400640 1 -- 172.21.15.73:0/4031653091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c02ce40 con 0x7f752406bc40 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.015+0000 7f752b800640 1 -- 172.21.15.73:0/4031653091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752406bc40 msgr2=0x7f7524164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.015+0000 7f752b800640 1 --2- 172.21.15.73:0/4031653091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752406bc40 0x7f7524164740 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f751c009e50 tx=0x7f751c02bdb0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 -- 172.21.15.73:0/4031653091 shutdown_connections 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 --2- 172.21.15.73:0/4031653091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752406bc40 0x7f7524164740 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 --2- 172.21.15.73:0/4031653091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752415abe0 0x7f75240712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 -- 172.21.15.73:0/4031653091 >> 172.21.15.73:0/4031653091 conn(0x7f7524069cb0 msgr2=0x7f752406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:56.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 -- 172.21.15.73:0/4031653091 shutdown_connections 2024-04-05T22:28:56.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.016+0000 7f752b800640 1 -- 172.21.15.73:0/4031653091 wait complete. 2024-04-05T22:28:56.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.017+0000 7f752b800640 1 Processor -- start 2024-04-05T22:28:56.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.017+0000 7f752b800640 1 -- start start 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752406bc40 0x7f75241475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752406c9a0 0x7f752414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75240b1aa0 con 0x7f752415abe0 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75240ada30 con 0x7f752406c9a0 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.018+0000 7f752b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75240add10 con 0x7f752406bc40 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60738/0 (socket says 172.21.15.73:60738) 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 -- 172.21.15.73:0/1533242036 learned_addr learned my addr 172.21.15.73:0/1533242036 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 -- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752406bc40 msgr2=0x7f75241475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752406bc40 0x7f75241475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 -- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752406c9a0 msgr2=0x7f752414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752406c9a0 0x7f752414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 -- 172.21.15.73:0/1533242036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009b30 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7529800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f751c036040 tx=0x7f751c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.019+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c03e450 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.020+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c03e5d0 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.020+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c03e980 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.020+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75240b1d60 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.020+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752414bdc0 con 0x7f752415abe0 2024-04-05T22:28:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.021+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7524156e80 con 0x7f752415abe0 2024-04-05T22:28:56.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.022+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f751c03eae0 con 0x7f752415abe0 2024-04-05T22:28:56.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.022+0000 7f7520c00640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 0x7f750406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:56.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.023+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f751c0ac920 con 0x7f752415abe0 2024-04-05T22:28:56.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.023+0000 7f7528e00640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 0x7f750406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.024+0000 7f7528e00640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 0x7f750406d4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7518009710 tx=0x7f7518006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:56.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.027+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f751c077520 con 0x7f752415abe0 2024-04-05T22:28:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.216+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75240b1f50 con 0x7f752415abe0 2024-04-05T22:28:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.217+0000 7f7520c00640 1 -- 172.21.15.73:0/1533242036 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f751c07ab40 con 0x7f752415abe0 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.219+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 msgr2=0x7f750406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.219+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 0x7f750406d4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7518009710 tx=0x7f7518006c90 comp rx=0 tx=0).stop 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.219+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 msgr2=0x7f752414b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.219+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f751c036040 tx=0x7f751c004a40 comp rx=0 tx=0).stop 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 shutdown_connections 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f750406aff0 0x7f750406d4b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f752415abe0 0x7f752414b6f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f752406c9a0 0x7f752414b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 --2- 172.21.15.73:0/1533242036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f752406bc40 0x7f75241475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:56.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 >> 172.21.15.73:0/1533242036 conn(0x7f7524069cb0 msgr2=0x7f75240af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:56.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 shutdown_connections 2024-04-05T22:28:56.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:56.220+0000 7f752b800640 1 -- 172.21.15.73:0/1533242036 wait complete. 2024-04-05T22:28:56.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.238 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol98550/3a5a0911-5e0e-42a9-ab7a-dc744e988651) 2024-04-05T22:28:56.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.337 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol98550/3a5a0911-5e0e-42a9-ab7a-dc744e988651/.snap/fake) 2024-04-05T22:28:56.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.469 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:28:56.523 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:28:56.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.523 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol98550/3a5a0911-5e0e-42a9-ab7a-dc744e988651) 2024-04-05T22:28:56.621 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:56.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol98550/.meta 2024-04-05T22:28:56.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.717 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33700/subvol86631/cbd646cf-be6f-4240-a067-1b6848b0875b) 2024-04-05T22:28:56.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.817 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33700/subvol86631/cbd646cf-be6f-4240-a067-1b6848b0875b/.snap/fake) 2024-04-05T22:28:56.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.942 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:28:56.997 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:28:56.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:56.998 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp33700/subvol86631/cbd646cf-be6f-4240-a067-1b6848b0875b) 2024-04-05T22:28:57.095 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:57.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp33700/subvol86631/.meta 2024-04-05T22:28:57.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:57.192 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol95432/48691a8f-ac18-4550-ae92-c6d1c6d2b26f) 2024-04-05T22:28:57.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:57.331 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:28:57.385 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:28:57.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:28:57.385 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol95432/48691a8f-ac18-4550-ae92-c6d1c6d2b26f) 2024-04-05T22:28:57.486 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:28:57.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol95432/.meta 2024-04-05T22:28:57.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98550 2024-04-05T22:28:57.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.841+0000 7f3198c00640 1 Processor -- start 2024-04-05T22:28:57.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.841+0000 7f3198c00640 1 -- start start 2024-04-05T22:28:57.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3198c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f319406c9a0 0x7f319406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3198c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406ba70 0x7f3194164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3198c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319406bea0 con 0x7f319415abe0 2024-04-05T22:28:57.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3198c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319415afc0 con 0x7f319406c9a0 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3198c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319406f5b0 con 0x7f319406ba70 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3189400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f319406c9a0 0x7f319406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3189400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f319406c9a0 0x7f319406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60754/0 (socket says 172.21.15.73:60754) 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.843+0000 7f3189400640 1 -- 172.21.15.73:0/670561039 learned_addr learned my addr 172.21.15.73:0/670561039 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.844+0000 7f3192800640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406ba70 0x7f3194164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.844+0000 7f3189400640 1 -- 172.21.15.73:0/670561039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406ba70 msgr2=0x7f3194164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.844+0000 7f3189400640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406ba70 0x7f3194164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.844+0000 7f3189400640 1 -- 172.21.15.73:0/670561039 >> v1:172.21.15.157:6789/0 conn(0x7f319415abe0 legacy=0x7f31940712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:57.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.844+0000 7f3189400640 1 -- 172.21.15.73:0/670561039 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319406f890 con 0x7f319406c9a0 2024-04-05T22:28:57.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.845+0000 7f3189400640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f319406b530 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f319406ce30 tx=0x7f317c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f996e1b05b1f675 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.845+0000 7f3191400640 1 -- 172.21.15.73:0/670561039 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c02f050 con 0x7f319406c9a0 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.845+0000 7f3191400640 1 -- 172.21.15.73:0/670561039 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c031700 con 0x7f319406c9a0 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.845+0000 7f3191400640 1 -- 172.21.15.73:0/670561039 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c031a90 con 0x7f319406c9a0 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 -- 172.21.15.73:0/670561039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 msgr2=0x7f319406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f319406b530 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f319406ce30 tx=0x7f317c02b7a0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 -- 172.21.15.73:0/670561039 shutdown_connections 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406ba70 0x7f3194164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 --2- 172.21.15.73:0/670561039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f319406b530 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 -- 172.21.15.73:0/670561039 >> 172.21.15.73:0/670561039 conn(0x7f3194069cb0 msgr2=0x7f319406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 -- 172.21.15.73:0/670561039 shutdown_connections 2024-04-05T22:28:57.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.846+0000 7f3198c00640 1 -- 172.21.15.73:0/670561039 wait complete. 2024-04-05T22:28:57.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.848+0000 7f3198c00640 1 Processor -- start 2024-04-05T22:28:57.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.848+0000 7f3198c00640 1 -- start start 2024-04-05T22:28:57.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319406ba70 0x7f31941374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319415abe0 0x7f3194159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31941f61f0 con 0x7f319406c9a0 2024-04-05T22:28:57.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31941f64d0 con 0x7f319406ba70 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.849+0000 7f3198c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3194136cf0 con 0x7f319415abe0 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60762/0 (socket says 172.21.15.73:60762) 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 -- 172.21.15.73:0/2988007276 learned_addr learned my addr 172.21.15.73:0/2988007276 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3191e00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319406ba70 0x7f31941374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 -- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319415abe0 msgr2=0x7f3194159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3192800640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319415abe0 0x7f3194159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319415abe0 0x7f3194159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 -- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319406ba70 msgr2=0x7f31941374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319406ba70 0x7f31941374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:57.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.850+0000 7f3189400640 1 -- 172.21.15.73:0/2988007276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0094d0 con 0x7f319406c9a0 2024-04-05T22:28:57.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f3189400640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f317c000c00 tx=0x7f317c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:57.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0339b0 con 0x7f319406c9a0 2024-04-05T22:28:57.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c033b30 con 0x7f319406c9a0 2024-04-05T22:28:57.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c03c400 con 0x7f319406c9a0 2024-04-05T22:28:57.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3194136fd0 con 0x7f319406c9a0 2024-04-05T22:28:57.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.851+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319414d2e0 con 0x7f319406c9a0 2024-04-05T22:28:57.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.852+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3194068160 con 0x7f319406c9a0 2024-04-05T22:28:57.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.853+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f317c03a040 con 0x7f319406c9a0 2024-04-05T22:28:57.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.854+0000 7f318aa00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 0x7f317006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:57.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.854+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f317c0ab580 con 0x7f319406c9a0 2024-04-05T22:28:57.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.855+0000 7f3191e00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 0x7f317006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:57.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.856+0000 7f3191e00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 0x7f317006d4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3184007fd0 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:57.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:57.858+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f317c076030 con 0x7f319406c9a0 2024-04-05T22:28:58.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.082+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98550", "target": ["mon-mgr", ""]}) v1 -- 0x7f319414bc30 con 0x7f317006aff0 2024-04-05T22:28:58.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:58.096+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:58.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:58.096+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:58.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:58.096+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:58.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:58.096+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:58.095 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:28:58.096+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:28:58.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.190+0000 7f318aa00640 1 -- 172.21.15.73:0/2988007276 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f319414bc30 con 0x7f317006aff0 2024-04-05T22:28:58.190 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol98550/3a5a0911-5e0e-42a9-ab7a-dc744e988651 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.193+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 msgr2=0x7f317006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.193+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 0x7f317006d4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3184007fd0 tx=0x7f3184009040 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.193+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 msgr2=0x7f31941f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.193+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f317c000c00 tx=0x7f317c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 shutdown_connections 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317006aff0 0x7f317006d4b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319415abe0 0x7f3194159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f319406c9a0 0x7f31941f57a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 --2- 172.21.15.73:0/2988007276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319406ba70 0x7f31941374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 >> 172.21.15.73:0/2988007276 conn(0x7f3194069cb0 msgr2=0x7f31940ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:58.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 shutdown_connections 2024-04-05T22:28:58.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.194+0000 7f3198c00640 1 -- 172.21.15.73:0/2988007276 wait complete. 2024-04-05T22:28:58.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol86631 subvol_grp33700 2024-04-05T22:28:58.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.429+0000 7f6069000640 1 Processor -- start 2024-04-05T22:28:58.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.430+0000 7f6069000640 1 -- start start 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6069000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f606415d000 0x7f606406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6069000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 0x7f60640b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6069000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60640b4e60 con 0x7f60640b1ca0 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6069000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60640b1a70 con 0x7f606415d000 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6069000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b5d50 con 0x7f606406bcf0 2024-04-05T22:28:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6061400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f60640b1ca0 0x7f60640adcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60636/0 (socket says 172.21.15.73:60636) 2024-04-05T22:28:58.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.432+0000 7f6061400640 1 -- 172.21.15.73:0/2782611923 learned_addr learned my addr 172.21.15.73:0/2782611923 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:58.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6061e00640 1 --2- 172.21.15.73:0/2782611923 >> v2:172.21.15.157:3300/0 conn(0x7f606415d000 0x7f606406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6062800640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 0x7f60640b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137362209 0 0) 0x7f60640b4e60 con 0x7f60640b1ca0 2024-04-05T22:28:58.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6044003520 con 0x7f60640b1ca0 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054550898 0 0) 0x7f6044003520 con 0x7f60640b1ca0 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 msgr2=0x7f60640b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 0x7f60640b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 >> v2:172.21.15.157:3300/0 conn(0x7f606415d000 msgr2=0x7f606406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 --2- 172.21.15.73:0/2782611923 >> v2:172.21.15.157:3300/0 conn(0x7f606415d000 0x7f606406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640b5140 con 0x7f60640b1ca0 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f604c0019f0 con 0x7f60640b1ca0 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 >> v1:172.21.15.73:6789/0 conn(0x7f60640b1ca0 legacy=0x7f60640adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 0x7f6044003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044003ef0 0x7f60440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6044004d00 0x7f6044005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b5d50 con 0x7f6044004d00 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604400b980 con 0x7f6044003ef0 2024-04-05T22:28:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.433+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604400bb10 con 0x7f606406bcf0 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6044004d00 0x7f6044005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 -- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 msgr2=0x7f6044003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606406bcf0 0x7f6044003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 -- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044003ef0 msgr2=0x7f60440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044003ef0 0x7f60440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.434+0000 7f6062800640 1 -- 172.21.15.73:0/2782611923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044003520 con 0x7f6044004d00 2024-04-05T22:28:58.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.435+0000 7f6062800640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6044004d00 0x7f6044005180 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f605c009af0 tx=0x7f60440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b615ed87db3920 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:58.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.435+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02c710 con 0x7f6044004d00 2024-04-05T22:28:58.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.435+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c02c870 con 0x7f6044004d00 2024-04-05T22:28:58.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.435+0000 7f6060a00640 1 -- 172.21.15.73:0/2782611923 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02cc00 con 0x7f6044004d00 2024-04-05T22:28:58.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 -- 172.21.15.73:0/2782611923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6044004d00 msgr2=0x7f6044005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 --2- 172.21.15.73:0/2782611923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6044004d00 0x7f6044005180 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f605c009af0 tx=0x7f60440057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6062800640 1 -- 172.21.15.73:0/2782611923 reap_dead start 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 -- 172.21.15.73:0/2782611923 shutdown_connections 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 -- 172.21.15.73:0/2782611923 >> 172.21.15.73:0/2782611923 conn(0x7f60640b5ed0 msgr2=0x7f60640b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 -- 172.21.15.73:0/2782611923 shutdown_connections 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.436+0000 7f6069000640 1 -- 172.21.15.73:0/2782611923 wait complete. 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.438+0000 7f6069000640 1 Processor -- start 2024-04-05T22:28:58.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.438+0000 7f6069000640 1 -- start start 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044004d00 0x7f60641f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606414f750 0x7f60640aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640cb360 con 0x7f60641f39f0 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606415e7f0 con 0x7f6044004d00 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6069000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606415ea90 con 0x7f606414f750 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6061400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6061400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60772/0 (socket says 172.21.15.73:60772) 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.439+0000 7f6061400640 1 -- 172.21.15.73:0/2862421195 learned_addr learned my addr 172.21.15.73:0/2862421195 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6062800640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606414f750 0x7f60640aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061e00640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044004d00 0x7f60641f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 -- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606414f750 msgr2=0x7f60640aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606414f750 0x7f60640aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 -- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044004d00 msgr2=0x7f60641f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044004d00 0x7f60641f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 -- 172.21.15.73:0/2862421195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640cb600 con 0x7f60641f39f0 2024-04-05T22:28:58.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6061400640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f604c00cb90 tx=0x7f604c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c004d60 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c00ab40 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c0070d0 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.440+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60640cb8e0 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.441+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606414fc00 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.442+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6064067290 con 0x7f60641f39f0 2024-04-05T22:28:58.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.443+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7f604c007230 con 0x7f60641f39f0 2024-04-05T22:28:58.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.444+0000 7f605a000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 0x7f604006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.444+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f604c089410 con 0x7f60641f39f0 2024-04-05T22:28:58.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.445+0000 7f6061e00640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 0x7f604006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:58.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.446+0000 7f6061e00640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 0x7f604006d4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6054004770 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:58.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.448+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f604c054010 con 0x7f60641f39f0 2024-04-05T22:28:58.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.669+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol86631", "group_name": "subvol_grp33700", "target": ["mon-mgr", ""]}) v1 -- 0x7f60640b5700 con 0x7f604006aff0 2024-04-05T22:28:58.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.768+0000 7f605a000640 1 -- 172.21.15.73:0/2862421195 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f6054009620 con 0x7f604006aff0 2024-04-05T22:28:58.768 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp33700/subvol86631/cbd646cf-be6f-4240-a067-1b6848b0875b 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.771+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 msgr2=0x7f604006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.771+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 0x7f604006d4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6054004770 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.771+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 msgr2=0x7f606415e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.771+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f604c00cb90 tx=0x7f604c00a430 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 shutdown_connections 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f604006aff0 0x7f604006d4b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f606414f750 0x7f60640aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60641f39f0 0x7f606415e2b0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 --2- 172.21.15.73:0/2862421195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6044004d00 0x7f60641f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 >> 172.21.15.73:0/2862421195 conn(0x7f60640b5ed0 msgr2=0x7f6064159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 shutdown_connections 2024-04-05T22:28:58.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.772+0000 7f6069000640 1 -- 172.21.15.73:0/2862421195 wait complete. 2024-04-05T22:28:58.793 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol95432 2024-04-05T22:28:58.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.998+0000 7fe893e00640 1 Processor -- start 2024-04-05T22:28:58.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:58.998+0000 7fe893e00640 1 -- start start 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe89415ac30 0x7fe89406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89406bc50 con 0x7fe894070470 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89415ef70 con 0x7fe89415a850 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89415f530 con 0x7fe89415ac30 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe894070470 0x7fe89406c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60640/0 (socket says 172.21.15.73:60640) 2024-04-05T22:28:58.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe893400640 1 -- 172.21.15.73:0/2492285749 learned_addr learned my addr 172.21.15.73:0/2492285749 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe88be00640 1 --2- 172.21.15.73:0/2492285749 >> v2:172.21.15.157:3300/0 conn(0x7fe89415ac30 0x7fe89406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.000+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269026307 0 0) 0x7fe89406bc50 con 0x7fe894070470 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe878003520 con 0x7fe894070470 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2276696324 0 0) 0x7fe878003520 con 0x7fe894070470 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 >> v2:172.21.15.157:3300/0 conn(0x7fe89415ac30 msgr2=0x7fe89406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 --2- 172.21.15.73:0/2492285749 >> v2:172.21.15.157:3300/0 conn(0x7fe89415ac30 0x7fe89406ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 >> v1:172.21.15.157:6789/0 conn(0x7fe89415a850 legacy=0x7fe89415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89415f250 con 0x7fe894070470 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe88c0019f0 con 0x7fe894070470 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 >> v1:172.21.15.73:6789/0 conn(0x7fe894070470 legacy=0x7fe89406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878003970 0x7fe878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe878004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe878005180 0x7fe87800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89415f530 con 0x7fe8780047c0 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87800c020 con 0x7fe878003970 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87800c1e0 con 0x7fe878005180 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.001+0000 7fe88be00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe878004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe892a00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878003970 0x7fe878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe88be00640 1 -- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe878005180 msgr2=0x7fe87800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe88be00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe878005180 0x7fe87800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe88be00640 1 -- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878003970 msgr2=0x7fe878003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe88be00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878003970 0x7fe878003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe88be00640 1 -- 172.21.15.73:0/2492285749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe878003520 con 0x7fe8780047c0 2024-04-05T22:28:59.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe892a00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878003970 0x7fe878003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:28:59.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.002+0000 7fe893400640 1 -- 172.21.15.73:0/2492285749 reap_dead start 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.003+0000 7fe88be00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe878004c40 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009ac0 tx=0x7fe8780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c8a6e60d6b5e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.003+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c02c6d0 con 0x7fe8780047c0 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.003+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c02c830 con 0x7fe8780047c0 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.003+0000 7fe892000640 1 -- 172.21.15.73:0/2492285749 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c02cbc0 con 0x7fe8780047c0 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 -- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 msgr2=0x7fe878004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe878004c40 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009ac0 tx=0x7fe8780057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 -- 172.21.15.73:0/2492285749 shutdown_connections 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 --2- 172.21.15.73:0/2492285749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe878004c40 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 -- 172.21.15.73:0/2492285749 >> 172.21.15.73:0/2492285749 conn(0x7fe894069cb0 msgr2=0x7fe89406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 -- 172.21.15.73:0/2492285749 shutdown_connections 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.004+0000 7fe893e00640 1 -- 172.21.15.73:0/2492285749 wait complete. 2024-04-05T22:28:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.006+0000 7fe893e00640 1 Processor -- start 2024-04-05T22:28:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.006+0000 7fe893e00640 1 -- start start 2024-04-05T22:28:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940ab5d0 0x7fe8941f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941f1980 0x7fe8940a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940aba50 con 0x7fe8780047c0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940abcb0 con 0x7fe8941f1980 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe893e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940abf90 con 0x7fe8940ab5d0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60790/0 (socket says 172.21.15.73:60790) 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 -- 172.21.15.73:0/1711029502 learned_addr learned my addr 172.21.15.73:0/1711029502 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 -- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940ab5d0 msgr2=0x7fe8941f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940ab5d0 0x7fe8941f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 -- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941f1980 msgr2=0x7fe8940a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941f1980 0x7fe8940a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.007+0000 7fe892a00640 1 -- 172.21.15.73:0/1711029502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940ac270 con 0x7fe8780047c0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe892a00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe88400d480 tx=0x7fe88400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884005480 con 0x7fe8780047c0 2024-04-05T22:28:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884003610 con 0x7fe8780047c0 2024-04-05T22:28:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884003a00 con 0x7fe8780047c0 2024-04-05T22:28:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8940ac550 con 0x7fe8780047c0 2024-04-05T22:28:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.008+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe894153810 con 0x7fe8780047c0 2024-04-05T22:28:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.009+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89404e160 con 0x7fe8780047c0 2024-04-05T22:28:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.010+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 89502+0+0 (secure 0 0 0) 0x7fe884003b60 con 0x7fe8780047c0 2024-04-05T22:28:59.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.011+0000 7fe88b400640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 0x7fe87006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.011+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8840899f0 con 0x7fe8780047c0 2024-04-05T22:28:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.012+0000 7fe88be00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 0x7fe87006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.013+0000 7fe88be00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 0x7fe87006d4e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe87c036000 tx=0x7fe87c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.015+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe884054630 con 0x7fe8780047c0 2024-04-05T22:28:59.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.248+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol95432", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8941539a0 con 0x7fe87006b020 2024-04-05T22:28:59.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.325+0000 7fe88b400640 1 -- 172.21.15.73:0/1711029502 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fe8941539a0 con 0x7fe87006b020 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "status": { 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "pending", 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "source": { 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "volume": "fake", 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "subvolume": "fake", 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot": "fake" 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:28:59.325 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 msgr2=0x7fe87006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 0x7fe87006d4e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe87c036000 tx=0x7fe87c02e000 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 msgr2=0x7fe8941f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe88400d480 tx=0x7fe88400d940 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 shutdown_connections 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe87006b020 0x7fe87006d4e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8941f1980 0x7fe8940a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8940ab5d0 0x7fe8941f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 --2- 172.21.15.73:0/1711029502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8780047c0 0x7fe8941f0270 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.328+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 >> 172.21.15.73:0/1711029502 conn(0x7fe894069cb0 msgr2=0x7fe89406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.329+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 shutdown_connections 2024-04-05T22:28:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.329+0000 7fe893e00640 1 -- 172.21.15.73:0/1711029502 wait complete. 2024-04-05T22:28:59.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol98550 fake 2024-04-05T22:28:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.560+0000 7f93f9000640 1 Processor -- start 2024-04-05T22:28:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.560+0000 7f93f9000640 1 -- start start 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f9000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93f40d1600 0x7f93f40cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93f40cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f9000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40ce400 con 0x7f93f415af90 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f9000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f40cdea0 con 0x7f93f40d1600 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cc790 con 0x7f93f40cd4d0 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f1e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f93f415af90 0x7f93f40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60652/0 (socket says 172.21.15.73:60652) 2024-04-05T22:28:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.562+0000 7f93f1e00640 1 -- 172.21.15.73:0/1302058238 learned_addr learned my addr 172.21.15.73:0/1302058238 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:28:59.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f1400640 1 --2- 172.21.15.73:0/1302058238 >> v2:172.21.15.157:3300/0 conn(0x7f93f40d1600 0x7f93f40cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f2800640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93f40cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636097855 0 0) 0x7f93f40ce400 con 0x7f93f415af90 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d8003520 con 0x7f93f415af90 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1532741711 0 0) 0x7f93d8003520 con 0x7f93f415af90 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 msgr2=0x7f93f40cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93f40cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 >> v2:172.21.15.157:3300/0 conn(0x7f93f40d1600 msgr2=0x7f93f40cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 --2- 172.21.15.73:0/1302058238 >> v2:172.21.15.157:3300/0 conn(0x7f93f40d1600 0x7f93f40cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40ce850 con 0x7f93f415af90 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f93e4001670 con 0x7f93f415af90 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 >> v1:172.21.15.73:6789/0 conn(0x7f93f415af90 legacy=0x7f93f40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d8003970 0x7f93d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d80047c0 0x7f93d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cdea0 con 0x7f93d8003970 2024-04-05T22:28:59.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cc790 con 0x7f93d80047c0 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d800b980 con 0x7f93f40cd4d0 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.563+0000 7f93f1e00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d8003970 0x7f93d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1400640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d80047c0 0x7f93d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f2800640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1e00640 1 -- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 msgr2=0x7f93d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1e00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93f40cd4d0 0x7f93d8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1e00640 1 -- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d80047c0 msgr2=0x7f93d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1e00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d80047c0 0x7f93d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.564+0000 7f93f1e00640 1 -- 172.21.15.73:0/1302058238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8003520 con 0x7f93d8003970 2024-04-05T22:28:59.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.565+0000 7f93f1e00640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d8003970 0x7f93d8003d90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c240 tx=0x7f93d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd6677ba2ef7096 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.565+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e402cb00 con 0x7f93d8003970 2024-04-05T22:28:59.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.565+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e402cc60 con 0x7f93d8003970 2024-04-05T22:28:59.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.565+0000 7f93f0a00640 1 -- 172.21.15.73:0/1302058238 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4005af0 con 0x7f93d8003970 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 -- 172.21.15.73:0/1302058238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d8003970 msgr2=0x7f93d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 --2- 172.21.15.73:0/1302058238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d8003970 0x7f93d8003d90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c240 tx=0x7f93d80057f0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f2800640 1 -- 172.21.15.73:0/1302058238 reap_dead start 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 -- 172.21.15.73:0/1302058238 shutdown_connections 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 -- 172.21.15.73:0/1302058238 >> 172.21.15.73:0/1302058238 conn(0x7f93f40d0ae0 msgr2=0x7f93f40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 -- 172.21.15.73:0/1302058238 shutdown_connections 2024-04-05T22:28:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.566+0000 7f93f9000640 1 -- 172.21.15.73:0/1302058238 wait complete. 2024-04-05T22:28:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.568+0000 7f93f9000640 1 Processor -- start 2024-04-05T22:28:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.568+0000 7f93f9000640 1 -- start start 2024-04-05T22:28:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93d8003970 0x7f93f4155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93f40b0de0 0x7f93f40b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4155690 con 0x7f93f40c9200 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40b17b0 con 0x7f93f40b0de0 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f415d420 con 0x7f93d8003970 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60810/0 (socket says 172.21.15.73:60810) 2024-04-05T22:28:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.569+0000 7f93f1400640 1 -- 172.21.15.73:0/2472720054 learned_addr learned my addr 172.21.15.73:0/2472720054 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:28:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f2800640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93f40b0de0 0x7f93f40b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1e00640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93d8003970 0x7f93f4155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 -- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93d8003970 msgr2=0x7f93f4155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93d8003970 0x7f93f4155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 -- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93f40b0de0 msgr2=0x7f93f40b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93f40b0de0 0x7f93f40b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 -- 172.21.15.73:0/2472720054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f415e480 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f1400640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f93e00115d0 tx=0x7f93e0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0015870 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e00159d0 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f415d6c0 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0015d80 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.570+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f415dc80 con 0x7f93f40c9200 2024-04-05T22:28:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.572+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4068160 con 0x7f93f40c9200 2024-04-05T22:28:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.573+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f93e000f480 con 0x7f93f40c9200 2024-04-05T22:28:59.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.574+0000 7f93ea000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 0x7f93d406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:28:59.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.574+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93e008a310 con 0x7f93f40c9200 2024-04-05T22:28:59.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.574+0000 7f93f1e00640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 0x7f93d406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:28:59.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.576+0000 7f93f1e00640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 0x7f93d406d540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f93e4034000 tx=0x7f93e4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:28:59.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.578+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f93e0054df0 con 0x7f93f40c9200 2024-04-05T22:28:59.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.809+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol98550", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f40cfa40 con 0x7f93d406b080 2024-04-05T22:28:59.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.941+0000 7f93ea000640 1 -- 172.21.15.73:0/2472720054 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93e402ca00 con 0x7f93d406b080 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 msgr2=0x7f93d406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 0x7f93d406d540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f93e4034000 tx=0x7f93e4007000 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 msgr2=0x7f93f415c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f93e00115d0 tx=0x7f93e0011a90 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 shutdown_connections 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f93d406b080 0x7f93d406d540 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93f40b0de0 0x7f93f40b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.944+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93f40c9200 0x7f93f415c8f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.945+0000 7f93f9000640 1 --2- 172.21.15.73:0/2472720054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93d8003970 0x7f93f4155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.945+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 >> 172.21.15.73:0/2472720054 conn(0x7f93f40d0ae0 msgr2=0x7f93f40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:28:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.945+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 shutdown_connections 2024-04-05T22:28:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:28:59.945+0000 7f93f9000640 1 -- 172.21.15.73:0/2472720054 wait complete. 2024-04-05T22:28:59.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol86631 fake subvol_grp33700 2024-04-05T22:29:00.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.183+0000 7f378b200640 1 Processor -- start 2024-04-05T22:29:00.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.184+0000 7f378b200640 1 -- start start 2024-04-05T22:29:00.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.185+0000 7f378b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f378c153f30 0x7f378c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.185+0000 7f378b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c0b2b30 0x7f378c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.185+0000 7f378b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c15e4d0 con 0x7f378c06adc0 2024-04-05T22:29:00.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.185+0000 7f378b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c0b3a70 con 0x7f378c153f30 2024-04-05T22:29:00.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.185+0000 7f378b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0b3d50 con 0x7f378c0b2b30 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f378c153f30 0x7f378c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f378c153f30 0x7f378c0b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60826/0 (socket says 172.21.15.73:60826) 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 -- 172.21.15.73:0/429146481 learned_addr learned my addr 172.21.15.73:0/429146481 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3781400640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c0b2b30 0x7f378c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 -- 172.21.15.73:0/429146481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c0b2b30 msgr2=0x7f378c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c0b2b30 0x7f378c15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 -- 172.21.15.73:0/429146481 >> v1:172.21.15.157:6789/0 conn(0x7f378c06adc0 legacy=0x7f378c15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:00.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.186+0000 7f3789e00640 1 -- 172.21.15.73:0/429146481 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c15d790 con 0x7f378c153f30 2024-04-05T22:29:00.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.187+0000 7f3789e00640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c153f30 0x7f378c0b3390 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f377c009870 tx=0x7f377c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223a440f35abffbe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.188+0000 7f3789400640 1 -- 172.21.15.73:0/429146481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0317c0 con 0x7f378c153f30 2024-04-05T22:29:00.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.188+0000 7f3789400640 1 -- 172.21.15.73:0/429146481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c031920 con 0x7f378c153f30 2024-04-05T22:29:00.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.188+0000 7f3789400640 1 -- 172.21.15.73:0/429146481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c031cb0 con 0x7f378c153f30 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 -- 172.21.15.73:0/429146481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c153f30 msgr2=0x7f378c0b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c153f30 0x7f378c0b3390 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f377c009870 tx=0x7f377c02b860 comp rx=0 tx=0).stop 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 -- 172.21.15.73:0/429146481 shutdown_connections 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c0b2b30 0x7f378c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 --2- 172.21.15.73:0/429146481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c153f30 0x7f378c0b3390 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 -- 172.21.15.73:0/429146481 >> 172.21.15.73:0/429146481 conn(0x7f378c0b1580 msgr2=0x7f378c0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 -- 172.21.15.73:0/429146481 shutdown_connections 2024-04-05T22:29:00.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.189+0000 7f378b200640 1 -- 172.21.15.73:0/429146481 wait complete. 2024-04-05T22:29:00.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.191+0000 7f378b200640 1 Processor -- start 2024-04-05T22:29:00.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.191+0000 7f378b200640 1 -- start start 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c06adc0 0x7f378c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378c153f30 0x7f378c0cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0cd730 con 0x7f378c0b2b30 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0ccb20 con 0x7f378c153f30 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f378b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0cce00 con 0x7f378c06adc0 2024-04-05T22:29:00.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f3781400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f3781400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60832/0 (socket says 172.21.15.73:60832) 2024-04-05T22:29:00.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f3781400640 1 -- 172.21.15.73:0/524891833 learned_addr learned my addr 172.21.15.73:0/524891833 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.192+0000 7f3789e00640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c06adc0 0x7f378c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 -- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c06adc0 msgr2=0x7f378c15c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c06adc0 0x7f378c15c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 -- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378c153f30 msgr2=0x7f378c0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378c153f30 0x7f378c0cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 -- 172.21.15.73:0/524891833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c0094d0 con 0x7f378c0b2b30 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3781400640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f377400c7f0 tx=0x7f377400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774004220 con 0x7f378c0b2b30 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740043a0 con 0x7f378c0b2b30 2024-04-05T22:29:00.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774007860 con 0x7f378c0b2b30 2024-04-05T22:29:00.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c0cd0e0 con 0x7f378c0b2b30 2024-04-05T22:29:00.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.193+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c149880 con 0x7f378c0b2b30 2024-04-05T22:29:00.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.195+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c0cd8a0 con 0x7f378c0b2b30 2024-04-05T22:29:00.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.195+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f3774007bd0 con 0x7f378c0b2b30 2024-04-05T22:29:00.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.196+0000 7f3782a00640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 0x7f376806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.196+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3774089c60 con 0x7f378c0b2b30 2024-04-05T22:29:00.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.197+0000 7f3789e00640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 0x7f376806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.199+0000 7f3789e00640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 0x7f376806d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f377c036000 tx=0x7f377c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.201+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3774054700 con 0x7f378c0b2b30 2024-04-05T22:29:00.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.436+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol86631", "snap_name": "fake", "group_name": "subvol_grp33700", "target": ["mon-mgr", ""]}) v1 -- 0x7f378c0b2f90 con 0x7f376806b080 2024-04-05T22:29:00.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.551+0000 7f3782a00640 1 -- 172.21.15.73:0/524891833 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f377c031b90 con 0x7f376806b080 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 msgr2=0x7f376806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 0x7f376806d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f377c036000 tx=0x7f377c03d040 comp rx=0 tx=0).stop 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 msgr2=0x7f378c15c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f377400c7f0 tx=0x7f377400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 shutdown_connections 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f376806b080 0x7f376806d540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.554+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378c153f30 0x7f378c0cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.555+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c0b2b30 0x7f378c15c840 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.555+0000 7f378b200640 1 --2- 172.21.15.73:0/524891833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378c06adc0 0x7f378c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.555+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 >> 172.21.15.73:0/524891833 conn(0x7f378c0b1580 msgr2=0x7f378c06cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:00.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.555+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 shutdown_connections 2024-04-05T22:29:00.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.555+0000 7f378b200640 1 -- 172.21.15.73:0/524891833 wait complete. 2024-04-05T22:29:00.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:00.576 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol98550/.meta) 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol98550/3a5a0911-5e0e-42a9-ab7a-dc744e988651 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:29:00.669 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:00.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:00.673 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp33700/subvol86631/.meta) 2024-04-05T22:29:00.764 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:29:00.765 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:29:00.765 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:29:00.765 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/subvol_grp33700/subvol86631/cbd646cf-be6f-4240-a067-1b6848b0875b 2024-04-05T22:29:00.765 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:29:00.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:00.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol98550 2024-04-05T22:29:00.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.978+0000 7f2442200640 1 Processor -- start 2024-04-05T22:29:00.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.978+0000 7f2442200640 1 -- start start 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f2442200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f243c06c9a0 0x7f243c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f2442200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c06bea0 con 0x7f243c15abe0 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f2442200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c15afc0 con 0x7f243c06ba70 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f2442200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c06f5b0 con 0x7f243c06c9a0 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f243b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f243c15abe0 0x7f243c0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60660/0 (socket says 172.21.15.73:60660) 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f243b400640 1 -- 172.21.15.73:0/1506955943 learned_addr learned my addr 172.21.15.73:0/1506955943 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:00.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f243aa00640 1 --2- 172.21.15.73:0/1506955943 >> v2:172.21.15.73:3300/0 conn(0x7f243c06c9a0 0x7f243c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.980+0000 7f243a000640 1 -- 172.21.15.73:0/1506955943 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480044496 0 0) 0x7f243c06bea0 con 0x7f243c15abe0 2024-04-05T22:29:00.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.981+0000 7f243a000640 1 -- 172.21.15.73:0/1506955943 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2420003520 con 0x7f243c15abe0 2024-04-05T22:29:00.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.981+0000 7f243aa00640 1 -- 172.21.15.73:0/1506955943 >> v1:172.21.15.157:6789/0 conn(0x7f243c06ba70 legacy=0x7f243c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:00.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.981+0000 7f243aa00640 1 -- 172.21.15.73:0/1506955943 >> v1:172.21.15.73:6789/0 conn(0x7f243c15abe0 legacy=0x7f243c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.981+0000 7f243aa00640 1 -- 172.21.15.73:0/1506955943 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c06f890 con 0x7f243c06c9a0 2024-04-05T22:29:00.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.982+0000 7f243aa00640 1 --2- 172.21.15.73:0/1506955943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c06c9a0 0x7f243c06b530 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f24280097a0 tx=0x7f242802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb13d79f3b50793 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.982+0000 7f243a000640 1 -- 172.21.15.73:0/1506955943 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242802f050 con 0x7f243c06c9a0 2024-04-05T22:29:00.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.982+0000 7f243a000640 1 -- 172.21.15.73:0/1506955943 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2428031700 con 0x7f243c06c9a0 2024-04-05T22:29:00.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.982+0000 7f243a000640 1 -- 172.21.15.73:0/1506955943 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428031a90 con 0x7f243c06c9a0 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 -- 172.21.15.73:0/1506955943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c06c9a0 msgr2=0x7f243c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 --2- 172.21.15.73:0/1506955943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c06c9a0 0x7f243c06b530 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f24280097a0 tx=0x7f242802b7a0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 -- 172.21.15.73:0/1506955943 shutdown_connections 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 --2- 172.21.15.73:0/1506955943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c06c9a0 0x7f243c06b530 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 -- 172.21.15.73:0/1506955943 >> 172.21.15.73:0/1506955943 conn(0x7f243c069cb0 msgr2=0x7f243c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 -- 172.21.15.73:0/1506955943 shutdown_connections 2024-04-05T22:29:00.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.983+0000 7f2442200640 1 -- 172.21.15.73:0/1506955943 wait complete. 2024-04-05T22:29:00.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.984+0000 7f2442200640 1 Processor -- start 2024-04-05T22:29:00.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.985+0000 7f2442200640 1 -- start start 2024-04-05T22:29:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 0x7f243c1443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f243c06c9a0 0x7f243c14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0b1c50 con 0x7f243c15abe0 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c14bbd0 con 0x7f243c06ba70 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f2442200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c1448e0 con 0x7f243c06c9a0 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60838/0 (socket says 172.21.15.73:60838) 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 -- 172.21.15.73:0/2814498928 learned_addr learned my addr 172.21.15.73:0/2814498928 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243b400640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 0x7f243c1443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 -- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f243c06c9a0 msgr2=0x7f243c14e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f243c06c9a0 0x7f243c14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 -- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 msgr2=0x7f243c1443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 0x7f243c1443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243be00640 1 -- 172.21.15.73:0/2814498928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009400 con 0x7f243c15abe0 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.986+0000 7f243b400640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 0x7f243c1443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f243be00640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f243400c7b0 tx=0x7f243400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434004b60 con 0x7f243c15abe0 2024-04-05T22:29:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434004cc0 con 0x7f243c15abe0 2024-04-05T22:29:00.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400aa90 con 0x7f243c15abe0 2024-04-05T22:29:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c0b1ea0 con 0x7f243c15abe0 2024-04-05T22:29:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.987+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c0b2460 con 0x7f243c15abe0 2024-04-05T22:29:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.988+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c0cdc40 con 0x7f243c15abe0 2024-04-05T22:29:00.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.989+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f243400abf0 con 0x7f243c15abe0 2024-04-05T22:29:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.990+0000 7f2433400640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 0x7f241c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.991+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2434089dd0 con 0x7f243c15abe0 2024-04-05T22:29:00.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.991+0000 7f243b400640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 0x7f241c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:00.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.992+0000 7f243b400640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 0x7f241c06d540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f242c005880 tx=0x7f242c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:00.996+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f24340548b0 con 0x7f243c15abe0 2024-04-05T22:29:01.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.216+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol98550", "target": ["mon-mgr", ""]}) v1 -- 0x7f243c070130 con 0x7f241c06b080 2024-04-05T22:29:01.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.318+0000 7f2433400640 1 -- 172.21.15.73:0/2814498928 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f242c001370 con 0x7f241c06b080 2024-04-05T22:29:01.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 msgr2=0x7f241c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 0x7f241c06d540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f242c005880 tx=0x7f242c009040 comp rx=0 tx=0).stop 2024-04-05T22:29:01.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 msgr2=0x7f243c153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f243400c7b0 tx=0x7f243400cc70 comp rx=0 tx=0).stop 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 shutdown_connections 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f241c06b080 0x7f241c06d540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243c15abe0 0x7f243c153c40 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f243c06c9a0 0x7f243c14e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 --2- 172.21.15.73:0/2814498928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c06ba70 0x7f243c1443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 >> 172.21.15.73:0/2814498928 conn(0x7f243c069cb0 msgr2=0x7f243c0af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 shutdown_connections 2024-04-05T22:29:01.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.322+0000 7f2442200640 1 -- 172.21.15.73:0/2814498928 wait complete. 2024-04-05T22:29:01.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.334+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.334+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.334+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.334+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.334+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol86631 subvol_grp33700 2024-04-05T22:29:01.361 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.362+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.362 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.362+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.362 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.362+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.362 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.362+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.362 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:01.362+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.556+0000 7f14b1e00640 1 Processor -- start 2024-04-05T22:29:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.556+0000 7f14b1e00640 1 -- start start 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14b1e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14ac144400 0x7f14ac0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14b1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14ac15f1f0 0x7f14ac069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14b1e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac06a620 con 0x7f14ac06a1f0 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14b1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0b3090 con 0x7f14ac15f1f0 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14b1e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0b3370 con 0x7f14ac144400 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14aa000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14ac15f1f0 0x7f14ac069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14aa000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14ac15f1f0 0x7f14ac069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60850/0 (socket says 172.21.15.73:60850) 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14aa000640 1 -- 172.21.15.73:0/1884162125 learned_addr learned my addr 172.21.15.73:0/1884162125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14ab400640 1 --1- 172.21.15.73:0/1884162125 >> v1:172.21.15.73:6789/0 conn(0x7f14ac06a1f0 0x7f14ac070820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60662/0 (socket says 172.21.15.73:60662) 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.558+0000 7f14aaa00640 1 --2- 172.21.15.73:0/1884162125 >> v2:172.21.15.157:3300/0 conn(0x7f14ac144400 0x7f14ac0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.559+0000 7f14aa000640 1 -- 172.21.15.73:0/1884162125 >> v2:172.21.15.157:3300/0 conn(0x7f14ac144400 msgr2=0x7f14ac0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.559+0000 7f14aa000640 1 --2- 172.21.15.73:0/1884162125 >> v2:172.21.15.157:3300/0 conn(0x7f14ac144400 0x7f14ac0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.559+0000 7f14aa000640 1 -- 172.21.15.73:0/1884162125 >> v1:172.21.15.73:6789/0 conn(0x7f14ac06a1f0 legacy=0x7f14ac070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.559+0000 7f14aa000640 1 -- 172.21.15.73:0/1884162125 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0b3650 con 0x7f14ac15f1f0 2024-04-05T22:29:01.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.559+0000 7f14a9600640 1 -- 172.21.15.73:0/1884162125 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848617084 0 0) 0x7f14ac06a620 con 0x7f14ac06a1f0 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.560+0000 7f14aa000640 1 --2- 172.21.15.73:0/1884162125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac069cb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f14ac067a30 tx=0x7f149802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80fa68ab6c20f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.560+0000 7f14a9600640 1 -- 172.21.15.73:0/1884162125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149802f850 con 0x7f14ac15f1f0 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.560+0000 7f14a9600640 1 -- 172.21.15.73:0/1884162125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149802f9b0 con 0x7f14ac15f1f0 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.560+0000 7f14a9600640 1 -- 172.21.15.73:0/1884162125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149802fd40 con 0x7f14ac15f1f0 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.560+0000 7f14b1e00640 1 -- 172.21.15.73:0/1884162125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 msgr2=0x7f14ac069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 --2- 172.21.15.73:0/1884162125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac069cb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f14ac067a30 tx=0x7f149802ba30 comp rx=0 tx=0).stop 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 -- 172.21.15.73:0/1884162125 shutdown_connections 2024-04-05T22:29:01.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 --2- 172.21.15.73:0/1884162125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac069cb0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 --2- 172.21.15.73:0/1884162125 >> v2:172.21.15.157:3300/0 conn(0x7f14ac144400 0x7f14ac0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 -- 172.21.15.73:0/1884162125 >> 172.21.15.73:0/1884162125 conn(0x7f14ac0b17b0 msgr2=0x7f14ac0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 -- 172.21.15.73:0/1884162125 shutdown_connections 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.561+0000 7f14b1e00640 1 -- 172.21.15.73:0/1884162125 wait complete. 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.562+0000 7f14b1e00640 1 Processor -- start 2024-04-05T22:29:01.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.562+0000 7f14b1e00640 1 -- start start 2024-04-05T22:29:01.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14ac06a1f0 0x7f14ac158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 0x7f14ac13bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac06e3f0 con 0x7f14ac15f1f0 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac143280 con 0x7f14ac06a1f0 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.563+0000 7f14b1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac142670 con 0x7f14ac144400 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60866/0 (socket says 172.21.15.73:60866) 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 -- 172.21.15.73:0/417092093 learned_addr learned my addr 172.21.15.73:0/417092093 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14aa000640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 0x7f14ac13bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14aaa00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14ac06a1f0 0x7f14ac158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 -- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 msgr2=0x7f14ac13bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 0x7f14ac13bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 -- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14ac06a1f0 msgr2=0x7f14ac158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14ac06a1f0 0x7f14ac158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 -- 172.21.15.73:0/417092093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980094d0 con 0x7f14ac15f1f0 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14aa000640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 0x7f14ac13bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:01.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.564+0000 7f14ab400640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001560 tx=0x7f14a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.565+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400a640 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.565+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400a7a0 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.565+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400ab30 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.565+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac142950 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.565+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac142f10 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.566+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac0ad550 con 0x7f14ac15f1f0 2024-04-05T22:29:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.567+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f14a400ac90 con 0x7f14ac15f1f0 2024-04-05T22:29:01.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.568+0000 7f14a3400640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 0x7f148c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:01.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.568+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a4089400 con 0x7f14ac15f1f0 2024-04-05T22:29:01.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.569+0000 7f14aaa00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 0x7f148c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:01.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.570+0000 7f14aaa00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 0x7f148c06d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f149c006fd0 tx=0x7f149c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:01.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.572+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f14a4053fb0 con 0x7f14ac15f1f0 2024-04-05T22:29:01.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.795+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol86631", "group_name": "subvol_grp33700", "target": ["mon-mgr", ""]}) v1 -- 0x7f14ac15f660 con 0x7f148c06b080 2024-04-05T22:29:01.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.893+0000 7f14a3400640 1 -- 172.21.15.73:0/417092093 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f149c00b600 con 0x7f148c06b080 2024-04-05T22:29:01.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 msgr2=0x7f148c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 0x7f148c06d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f149c006fd0 tx=0x7f149c008040 comp rx=0 tx=0).stop 2024-04-05T22:29:01.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 msgr2=0x7f14ac13c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001560 tx=0x7f14a400cc70 comp rx=0 tx=0).stop 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 shutdown_connections 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f148c06b080 0x7f148c06d540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ac15f1f0 0x7f14ac13c2c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14ac144400 0x7f14ac13bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 --2- 172.21.15.73:0/417092093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14ac06a1f0 0x7f14ac158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 >> 172.21.15.73:0/417092093 conn(0x7f14ac0b17b0 msgr2=0x7f14ac159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 shutdown_connections 2024-04-05T22:29:01.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:01.896+0000 7f14b1e00640 1 -- 172.21.15.73:0/417092093 wait complete. 2024-04-05T22:29:01.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol95432 2024-04-05T22:29:02.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.122+0000 7f5850e00640 1 Processor -- start 2024-04-05T22:29:02.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.122+0000 7f5850e00640 1 -- start start 2024-04-05T22:29:02.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.123+0000 7f5850e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c15abe0 0x7f584c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.123+0000 7f5850e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f584c06bc40 0x7f584c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5850e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584c15afc0 con 0x7f584c06c9a0 2024-04-05T22:29:02.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5850e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c06f5b0 con 0x7f584c06bc40 2024-04-05T22:29:02.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5850e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06f890 con 0x7f584c15abe0 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5841400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f584c06c9a0 0x7f584c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60664/0 (socket says 172.21.15.73:60664) 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5841400640 1 -- 172.21.15.73:0/366994586 learned_addr learned my addr 172.21.15.73:0/366994586 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f584a800640 1 --2- 172.21.15.73:0/366994586 >> v2:172.21.15.73:3300/0 conn(0x7f584c06bc40 0x7f584c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5849e00640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c15abe0 0x7f584c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5849400640 1 -- 172.21.15.73:0/366994586 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451790331 0 0) 0x7f584c15afc0 con 0x7f584c06c9a0 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f5849400640 1 -- 172.21.15.73:0/366994586 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5830003520 con 0x7f584c06c9a0 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.124+0000 7f584a800640 1 -- 172.21.15.73:0/366994586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c15abe0 msgr2=0x7f584c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:02.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.125+0000 7f584a800640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c15abe0 0x7f584c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.125+0000 7f584a800640 1 -- 172.21.15.73:0/366994586 >> v1:172.21.15.73:6789/0 conn(0x7f584c06c9a0 legacy=0x7f584c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.125+0000 7f584a800640 1 -- 172.21.15.73:0/366994586 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c06fb70 con 0x7f584c06bc40 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f584a800640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c06bc40 0x7f584c164740 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5844009d90 tx=0x7f584402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e862a6b8a7d85b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f5849400640 1 -- 172.21.15.73:0/366994586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584402cb30 con 0x7f584c06bc40 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f5849400640 1 -- 172.21.15.73:0/366994586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584402cc90 con 0x7f584c06bc40 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f5849400640 1 -- 172.21.15.73:0/366994586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844034440 con 0x7f584c06bc40 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f5850e00640 1 -- 172.21.15.73:0/366994586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c06bc40 msgr2=0x7f584c164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.126+0000 7f5850e00640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c06bc40 0x7f584c164740 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5844009d90 tx=0x7f584402bd80 comp rx=0 tx=0).stop 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 -- 172.21.15.73:0/366994586 shutdown_connections 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c06bc40 0x7f584c164740 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 --2- 172.21.15.73:0/366994586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c15abe0 0x7f584c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 -- 172.21.15.73:0/366994586 >> 172.21.15.73:0/366994586 conn(0x7f584c069cb0 msgr2=0x7f584c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:02.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 -- 172.21.15.73:0/366994586 shutdown_connections 2024-04-05T22:29:02.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.127+0000 7f5850e00640 1 -- 172.21.15.73:0/366994586 wait complete. 2024-04-05T22:29:02.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.129+0000 7f5850e00640 1 Processor -- start 2024-04-05T22:29:02.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.129+0000 7f5850e00640 1 -- start start 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c06bc40 0x7f584c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f584c06c9a0 0x7f584c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c15abe0 0x7f584c14a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c15ce20 con 0x7f584c15abe0 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1f31c0 con 0x7f584c06c9a0 2024-04-05T22:29:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f5850e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c14b2f0 con 0x7f584c06bc40 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f584a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c15abe0 0x7f584c14a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f584a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c15abe0 0x7f584c14a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60880/0 (socket says 172.21.15.73:60880) 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.130+0000 7f584a800640 1 -- 172.21.15.73:0/1235863195 learned_addr learned my addr 172.21.15.73:0/1235863195 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5849e00640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c06bc40 0x7f584c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5841400640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f584c06c9a0 0x7f584c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 -- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c06bc40 msgr2=0x7f584c15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f584c06bc40 0x7f584c15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 -- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f584c06c9a0 msgr2=0x7f584c149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f584c06c9a0 0x7f584c149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 -- 172.21.15.73:0/1235863195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844009a70 con 0x7f584c15abe0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5841400640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f584c06c9a0 0x7f584c149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f584a800640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f584c15abe0 0x7f584c14a100 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5844009d90 tx=0x7f5844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844009450 con 0x7f584c15abe0 2024-04-05T22:29:02.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844040450 con 0x7f584c15abe0 2024-04-05T22:29:02.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844040800 con 0x7f584c15abe0 2024-04-05T22:29:02.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.131+0000 7f5850e00640 1 -- 172.21.15.73:0/1235863195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c14b5d0 con 0x7f584c15abe0 2024-04-05T22:29:02.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.132+0000 7f5850e00640 1 -- 172.21.15.73:0/1235863195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c14bb90 con 0x7f584c15abe0 2024-04-05T22:29:02.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.133+0000 7f5850e00640 1 -- 172.21.15.73:0/1235863195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c070370 con 0x7f584c15abe0 2024-04-05T22:29:02.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.133+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f5844040960 con 0x7f584c15abe0 2024-04-05T22:29:02.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.135+0000 7f5842a00640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f582806b080 0x7f582806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.135+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58440ac5f0 con 0x7f584c15abe0 2024-04-05T22:29:02.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.135+0000 7f5849e00640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f582806b080 0x7f582806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.137+0000 7f5849e00640 1 --2- 172.21.15.73:0/1235863195 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f582806b080 0x7f582806d540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f583c006fd0 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.140+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f58440772a0 con 0x7f584c15abe0 2024-04-05T22:29:02.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.363+0000 7f5850e00640 1 -- 172.21.15.73:0/1235863195 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol95432", "target": ["mon-mgr", ""]}) v1 -- 0x7f584c071820 con 0x7f582806b080 2024-04-05T22:29:02.424 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:02.425+0000 7f1d85400640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol95432' is not ready for operation rm (use --force to override) 2024-04-05T22:29:02.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.426+0000 7f5842a00640 1 -- 172.21.15.73:0/1235863195 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol95432' is not ready for operation rm (use --force to override)) v1 ==== 87+0+0 (secure 0 0 0) 0x7f583c00b600 con 0x7f582806b080 2024-04-05T22:29:02.425 INFO:teuthology.orchestra.run.smithi073.stderr:Error EAGAIN: subvolume 'subvol95432' is not ready for operation rm (use --force to override) 2024-04-05T22:29:02.440 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-04-05T22:29:02.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:02.441 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol95432/.meta) 2024-04-05T22:29:02.529 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:29:02.529 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:29:02.529 INFO:teuthology.orchestra.run.smithi073.stdout:type = clone 2024-04-05T22:29:02.529 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol95432/48691a8f-ac18-4550-ae92-c6d1c6d2b26f 2024-04-05T22:29:02.530 INFO:teuthology.orchestra.run.smithi073.stdout:state = pending 2024-04-05T22:29:02.530 INFO:teuthology.orchestra.run.smithi073.stdout:[source] 2024-04-05T22:29:02.530 INFO:teuthology.orchestra.run.smithi073.stdout:volume = fake 2024-04-05T22:29:02.530 INFO:teuthology.orchestra.run.smithi073.stdout:subvolume = fake 2024-04-05T22:29:02.530 INFO:teuthology.orchestra.run.smithi073.stdout:snapshot = fake 2024-04-05T22:29:02.534 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol95432 --force 2024-04-05T22:29:02.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.746+0000 7f2e63200640 1 Processor -- start 2024-04-05T22:29:02.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.746+0000 7f2e63200640 1 -- start start 2024-04-05T22:29:02.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e63200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6415aa80 0x7f2e6406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e63200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e640714b0 0x7f2e6406b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e63200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640ccc50 con 0x7f2e6406b9d0 2024-04-05T22:29:02.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e63200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640cd5f0 con 0x7f2e640714b0 2024-04-05T22:29:02.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e63200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640ccf30 con 0x7f2e6415aa80 2024-04-05T22:29:02.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e62800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2e6406b9d0 0x7f2e640cda10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60670/0 (socket says 172.21.15.73:60670) 2024-04-05T22:29:02.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e62800640 1 -- 172.21.15.73:0/3003447318 learned_addr learned my addr 172.21.15.73:0/3003447318 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:02.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.748+0000 7f2e59400640 1 --2- 172.21.15.73:0/3003447318 >> v2:172.21.15.73:3300/0 conn(0x7f2e640714b0 0x7f2e6406b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e61e00640 1 --2- 172.21.15.73:0/3003447318 >> v2:172.21.15.157:3300/0 conn(0x7f2e6415aa80 0x7f2e6406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e61400640 1 -- 172.21.15.73:0/3003447318 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097714355 0 0) 0x7f2e640ccc50 con 0x7f2e6406b9d0 2024-04-05T22:29:02.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e61400640 1 -- 172.21.15.73:0/3003447318 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e48003520 con 0x7f2e6406b9d0 2024-04-05T22:29:02.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e59400640 1 -- 172.21.15.73:0/3003447318 >> v2:172.21.15.157:3300/0 conn(0x7f2e6415aa80 msgr2=0x7f2e6406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:02.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e59400640 1 --2- 172.21.15.73:0/3003447318 >> v2:172.21.15.157:3300/0 conn(0x7f2e6415aa80 0x7f2e6406fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e59400640 1 -- 172.21.15.73:0/3003447318 >> v1:172.21.15.73:6789/0 conn(0x7f2e6406b9d0 legacy=0x7f2e640cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.749+0000 7f2e59400640 1 -- 172.21.15.73:0/3003447318 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640cd210 con 0x7f2e640714b0 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.750+0000 7f2e59400640 1 --2- 172.21.15.73:0/3003447318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e640714b0 0x7f2e6406b490 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2e6406b2b0 tx=0x7f2e4c02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a89e73e1d33eb92 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.750+0000 7f2e61400640 1 -- 172.21.15.73:0/3003447318 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c031520 con 0x7f2e640714b0 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.750+0000 7f2e61400640 1 -- 172.21.15.73:0/3003447318 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c031680 con 0x7f2e640714b0 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.750+0000 7f2e61400640 1 -- 172.21.15.73:0/3003447318 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c031a10 con 0x7f2e640714b0 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 -- 172.21.15.73:0/3003447318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e640714b0 msgr2=0x7f2e6406b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 --2- 172.21.15.73:0/3003447318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e640714b0 0x7f2e6406b490 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2e6406b2b0 tx=0x7f2e4c02b7c0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 -- 172.21.15.73:0/3003447318 shutdown_connections 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 --2- 172.21.15.73:0/3003447318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e640714b0 0x7f2e6406b490 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 --2- 172.21.15.73:0/3003447318 >> v2:172.21.15.157:3300/0 conn(0x7f2e6415aa80 0x7f2e6406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 -- 172.21.15.73:0/3003447318 >> 172.21.15.73:0/3003447318 conn(0x7f2e640b0360 msgr2=0x7f2e640b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 -- 172.21.15.73:0/3003447318 shutdown_connections 2024-04-05T22:29:02.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.751+0000 7f2e63200640 1 -- 172.21.15.73:0/3003447318 wait complete. 2024-04-05T22:29:02.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.753+0000 7f2e63200640 1 Processor -- start 2024-04-05T22:29:02.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.753+0000 7f2e63200640 1 -- start start 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e640714b0 0x7f2e64157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 0x7f2e641eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640aad90 con 0x7f2e6406b9d0 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640a97b0 con 0x7f2e6415aa80 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e63200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640a9a90 con 0x7f2e640714b0 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e61e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e61e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60894/0 (socket says 172.21.15.73:60894) 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.754+0000 7f2e61e00640 1 -- 172.21.15.73:0/1998538609 learned_addr learned my addr 172.21.15.73:0/1998538609 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:02.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e62800640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 0x7f2e641eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e59400640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e640714b0 0x7f2e64157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 -- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e640714b0 msgr2=0x7f2e64157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e640714b0 0x7f2e64157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 -- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 msgr2=0x7f2e641eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 0x7f2e641eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 -- 172.21.15.73:0/1998538609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c009400 con 0x7f2e6406b9d0 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e62800640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 0x7f2e641eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:02.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e61e00640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007ef0 tx=0x7f2e5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54014620 con 0x7f2e6406b9d0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54014780 con 0x7f2e6406b9d0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54014b30 con 0x7f2e6406b9d0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.755+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e640a9d70 con 0x7f2e6406b9d0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.756+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e640aa330 con 0x7f2e6406b9d0 2024-04-05T22:29:02.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.756+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6404e160 con 0x7f2e6406b9d0 2024-04-05T22:29:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.757+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 89553+0+0 (secure 0 0 0) 0x7f2e54014c90 con 0x7f2e6406b9d0 2024-04-05T22:29:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.758+0000 7f2e5aa00640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 0x7f2e4006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:02.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.758+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e5408a4e0 con 0x7f2e6406b9d0 2024-04-05T22:29:02.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.759+0000 7f2e59400640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 0x7f2e4006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:02.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.760+0000 7f2e59400640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 0x7f2e4006d540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c035000 tx=0x7f2e4c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:02.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.763+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2e54055130 con 0x7f2e6406b9d0 2024-04-05T22:29:02.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:02.996+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol95432", "force": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2e640cc870 con 0x7f2e4006b080 2024-04-05T22:29:03.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.465+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f2e54055500 con 0x7f2e6406b9d0 2024-04-05T22:29:03.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.497+0000 7f2e5aa00640 1 -- 172.21.15.73:0/1998538609 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2e4c006e10 con 0x7f2e4006b080 2024-04-05T22:29:03.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 msgr2=0x7f2e4006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 0x7f2e4006d540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c035000 tx=0x7f2e4c030000 comp rx=0 tx=0).stop 2024-04-05T22:29:03.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 msgr2=0x7f2e640aa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007ef0 tx=0x7f2e5400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 shutdown_connections 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e4006b080 0x7f2e4006d540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6415aa80 0x7f2e641eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e640714b0 0x7f2e64157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 --2- 172.21.15.73:0/1998538609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e6406b9d0 0x7f2e640aa850 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 >> 172.21.15.73:0/1998538609 conn(0x7f2e640b0360 msgr2=0x7f2e64158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 shutdown_connections 2024-04-05T22:29:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.501+0000 7f2e63200640 1 -- 172.21.15.73:0/1998538609 wait complete. 2024-04-05T22:29:03.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-05T22:29:03.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.732+0000 7f7ac2c00640 1 Processor -- start 2024-04-05T22:29:03.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.732+0000 7f7ac2c00640 1 -- start start 2024-04-05T22:29:03.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.733+0000 7f7ac2c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7abc06c630 0x7f7abc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.733+0000 7f7ac2c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7abc070c50 0x7f7abc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac2c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc146ac0 con 0x7f7abc15dd90 2024-04-05T22:29:03.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac2c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc070720 con 0x7f7abc070c50 2024-04-05T22:29:03.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac2c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc0acf60 con 0x7f7abc06c630 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7abc070c50 0x7f7abc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7abc070c50 0x7f7abc0710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60896/0 (socket says 172.21.15.73:60896) 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 -- 172.21.15.73:0/3381648113 learned_addr learned my addr 172.21.15.73:0/3381648113 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7abbe00640 1 --2- 172.21.15.73:0/3381648113 >> v2:172.21.15.157:3300/0 conn(0x7f7abc06c630 0x7f7abc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 -- 172.21.15.73:0/3381648113 >> v2:172.21.15.157:3300/0 conn(0x7f7abc06c630 msgr2=0x7f7abc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 --2- 172.21.15.73:0/3381648113 >> v2:172.21.15.157:3300/0 conn(0x7f7abc06c630 0x7f7abc0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.734+0000 7f7ac0c00640 1 -- 172.21.15.73:0/3381648113 >> v1:172.21.15.157:6789/0 conn(0x7f7abc15dd90 legacy=0x7f7abc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:03.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.735+0000 7f7ac0c00640 1 -- 172.21.15.73:0/3381648113 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0ad240 con 0x7f7abc070c50 2024-04-05T22:29:03.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.736+0000 7f7ac0c00640 1 --2- 172.21.15.73:0/3381648113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc070c50 0x7f7abc0710e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06b1d0 tx=0x7f7ab402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcf98252cdffd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:03.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.736+0000 7f7abb400640 1 -- 172.21.15.73:0/3381648113 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab402ca80 con 0x7f7abc070c50 2024-04-05T22:29:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.736+0000 7f7abb400640 1 -- 172.21.15.73:0/3381648113 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab402cbe0 con 0x7f7abc070c50 2024-04-05T22:29:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.736+0000 7f7abb400640 1 -- 172.21.15.73:0/3381648113 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40343f0 con 0x7f7abc070c50 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 -- 172.21.15.73:0/3381648113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc070c50 msgr2=0x7f7abc0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/3381648113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc070c50 0x7f7abc0710e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06b1d0 tx=0x7f7ab402bee0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 -- 172.21.15.73:0/3381648113 shutdown_connections 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/3381648113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc070c50 0x7f7abc0710e0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/3381648113 >> v2:172.21.15.157:3300/0 conn(0x7f7abc06c630 0x7f7abc0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 -- 172.21.15.73:0/3381648113 >> 172.21.15.73:0/3381648113 conn(0x7f7abc0b0830 msgr2=0x7f7abc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 -- 172.21.15.73:0/3381648113 shutdown_connections 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.737+0000 7f7ac2c00640 1 -- 172.21.15.73:0/3381648113 wait complete. 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.739+0000 7f7ac2c00640 1 Processor -- start 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.739+0000 7f7ac2c00640 1 -- start start 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 0x7f7abc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7abc15dd90 0x7f7abc15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0ce3c0 con 0x7f7abc06c630 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1f2b10 con 0x7f7abc070c50 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7ac2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1f2df0 con 0x7f7abc15dd90 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60902/0 (socket says 172.21.15.73:60902) 2024-04-05T22:29:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.740+0000 7f7abbe00640 1 -- 172.21.15.73:0/1373340068 learned_addr learned my addr 172.21.15.73:0/1373340068 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ab3e00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 0x7f7abc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 -- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7abc15dd90 msgr2=0x7f7abc15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7abc15dd90 0x7f7abc15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 -- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 msgr2=0x7f7abc15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 0x7f7abc15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 -- 172.21.15.73:0/1373340068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009b30 con 0x7f7abc06c630 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ab3e00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 0x7f7abc15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7abbe00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d5f0 tx=0x7f7aac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0125f0 con 0x7f7abc06c630 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac012750 con 0x7f7abc06c630 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac012b00 con 0x7f7abc06c630 2024-04-05T22:29:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc1f30d0 con 0x7f7abc06c630 2024-04-05T22:29:03.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.741+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1f2800 con 0x7f7abc06c630 2024-04-05T22:29:03.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.743+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc15e1d0 con 0x7f7abc06c630 2024-04-05T22:29:03.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.744+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f7aac012c60 con 0x7f7abc06c630 2024-04-05T22:29:03.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.744+0000 7f7ab8c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 0x7f7a9806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:03.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.744+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aac08a220 con 0x7f7abc06c630 2024-04-05T22:29:03.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.745+0000 7f7ab3e00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 0x7f7a9806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.746+0000 7f7ab3e00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 0x7f7a9806d660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0ca170 tx=0x7f7aa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:03.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.749+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7abc15e1d0 con 0x7f7abc06c630 2024-04-05T22:29:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.971+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f7abc0ad800 con 0x7f7a9806b1a0 2024-04-05T22:29:03.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.981+0000 7f7ab8c00640 1 -- 172.21.15.73:0/1373340068 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f7aa4005030 con 0x7f7a9806b1a0 2024-04-05T22:29:03.981 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-04-05T22:29:03.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 msgr2=0x7f7a9806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 0x7f7a9806d660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0ca170 tx=0x7f7aa4009040 comp rx=0 tx=0).stop 2024-04-05T22:29:03.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 msgr2=0x7f7abc13e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:03.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d5f0 tx=0x7f7aac00dab0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 shutdown_connections 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7a9806b1a0 0x7f7a9806d660 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7abc15dd90 0x7f7abc15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7abc070c50 0x7f7abc15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 --2- 172.21.15.73:0/1373340068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7abc06c630 0x7f7abc13e360 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 >> 172.21.15.73:0/1373340068 conn(0x7f7abc0b0830 msgr2=0x7f7abc0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 shutdown_connections 2024-04-05T22:29:03.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:03.984+0000 7f7ac2c00640 1 -- 172.21.15.73:0/1373340068 wait complete. 2024-04-05T22:29:04.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:04.001 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:29:04.064 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:29:04.065 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:29:04.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:04.181 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:04.300 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:29:04.300 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:04.300 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:04.358 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:04.358 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:04.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.618+0000 7f0ba1c00640 1 Processor -- start 2024-04-05T22:29:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.618+0000 7f0ba1c00640 1 -- start start 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b9c15ce20 0x7f0b9c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15d200 0x7f0b9c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b9c0ca050 0x7f0b9c160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c09d110 con 0x7f0b9c0ca050 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c06aa40 con 0x7f0b9c15ce20 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0ba1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06ad20 con 0x7f0b9c15d200 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b9c0ca050 0x7f0b9c160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b9c0ca050 0x7f0b9c160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60916/0 (socket says 172.21.15.73:60916) 2024-04-05T22:29:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 -- 172.21.15.73:0/1867090295 learned_addr learned my addr 172.21.15.73:0/1867090295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:04.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 -- 172.21.15.73:0/1867090295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15d200 msgr2=0x7f0b9c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:04.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 --2- 172.21.15.73:0/1867090295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15d200 0x7f0b9c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 -- 172.21.15.73:0/1867090295 >> v2:172.21.15.157:3300/0 conn(0x7f0b9c15ce20 msgr2=0x7f0b9c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:04.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 --2- 172.21.15.73:0/1867090295 >> v2:172.21.15.157:3300/0 conn(0x7f0b9c15ce20 0x7f0b9c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.620+0000 7f0b9b400640 1 -- 172.21.15.73:0/1867090295 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0cde30 con 0x7f0b9c0ca050 2024-04-05T22:29:04.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.621+0000 7f0b9b400640 1 --2- 172.21.15.73:0/1867090295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c160b70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0cf0f0 tx=0x7f0b9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4328f5f85fb64d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0b9a000640 1 -- 172.21.15.73:0/1867090295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9402ca70 con 0x7f0b9c0ca050 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0b9a000640 1 -- 172.21.15.73:0/1867090295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9402cbd0 con 0x7f0b9c0ca050 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0b9a000640 1 -- 172.21.15.73:0/1867090295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b940343f0 con 0x7f0b9c0ca050 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 -- 172.21.15.73:0/1867090295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 msgr2=0x7f0b9c160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/1867090295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c160b70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0cf0f0 tx=0x7f0b9402bed0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 -- 172.21.15.73:0/1867090295 shutdown_connections 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/1867090295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c160b70 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/1867090295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15d200 0x7f0b9c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/1867090295 >> v2:172.21.15.157:3300/0 conn(0x7f0b9c15ce20 0x7f0b9c0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.622+0000 7f0ba1c00640 1 -- 172.21.15.73:0/1867090295 >> 172.21.15.73:0/1867090295 conn(0x7f0b9c0cf190 msgr2=0x7f0b9c0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.623+0000 7f0ba1c00640 1 -- 172.21.15.73:0/1867090295 shutdown_connections 2024-04-05T22:29:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.623+0000 7f0ba1c00640 1 -- 172.21.15.73:0/1867090295 wait complete. 2024-04-05T22:29:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.624+0000 7f0ba1c00640 1 Processor -- start 2024-04-05T22:29:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.624+0000 7f0ba1c00640 1 -- start start 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15ce20 0x7f0b9c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b9c15d200 0x7f0b9c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c67a0 con 0x7f0b9c0ca050 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1604d0 con 0x7f0b9c15d200 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0ba1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c099b70 con 0x7f0b9c15ce20 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.625+0000 7f0b9aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60920/0 (socket says 172.21.15.73:60920) 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 -- 172.21.15.73:0/297533212 learned_addr learned my addr 172.21.15.73:0/297533212 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 -- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15ce20 msgr2=0x7f0b9c155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15ce20 0x7f0b9c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 -- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b9c15d200 msgr2=0x7f0b9c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b9c15d200 0x7f0b9c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 -- 172.21.15.73:0/297533212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94009b30 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b9aa00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00b4a0 tx=0x7f0b8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c010610 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c010770 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.626+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c010b20 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.627+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c099e50 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.627+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c09a410 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.628+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c09a630 con 0x7f0b9c0ca050 2024-04-05T22:29:04.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.629+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f0b8c010c80 con 0x7f0b9c0ca050 2024-04-05T22:29:04.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.629+0000 7f0b93400640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 0x7f0b7806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:04.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.630+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0b8c089b10 con 0x7f0b9c0ca050 2024-04-05T22:29:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.630+0000 7f0b92000640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 0x7f0b7806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.631+0000 7f0b92000640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 0x7f0b7806d660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c06dbe0 tx=0x7f0b84006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:04.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.634+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0b8c054370 con 0x7f0b9c0ca050 2024-04-05T22:29:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.831+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b9c06a690 con 0x7f0b9c0ca050 2024-04-05T22:29:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.833+0000 7f0b93400640 1 -- 172.21.15.73:0/297533212 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16222 (secure 0 0 0) 0x7f0b8c01e020 con 0x7f0b9c0ca050 2024-04-05T22:29:04.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:04.833 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":123,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:04.441444+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.836+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 msgr2=0x7f0b7806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.836+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 0x7f0b7806d660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c06dbe0 tx=0x7f0b84006bf0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.836+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 msgr2=0x7f0b9c0c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.836+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00b4a0 tx=0x7f0b8c00b960 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 shutdown_connections 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0b7806b1a0 0x7f0b7806d660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b9c15d200 0x7f0b9c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9c15ce20 0x7f0b9c155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 --2- 172.21.15.73:0/297533212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b9c0ca050 0x7f0b9c0c6260 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 >> 172.21.15.73:0/297533212 conn(0x7f0b9c0cf190 msgr2=0x7f0b9c0cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 shutdown_connections 2024-04-05T22:29:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:04.837+0000 7f0ba1c00640 1 -- 172.21.15.73:0/297533212 wait complete. 2024-04-05T22:29:04.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:28:42.000556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:28:43.153067+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:04.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:05.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.063+0000 7fe7ce000640 1 Processor -- start 2024-04-05T22:29:05.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.063+0000 7fe7ce000640 1 -- start start 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7c81599f0 0x7fe7c8070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c80add00 0x7fe7c806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7c806cb80 0x7fe7c815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c806d010 con 0x7fe7c806cb80 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c80b3890 con 0x7fe7c81599f0 2024-04-05T22:29:05.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7ce000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80ae190 con 0x7fe7c80add00 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7c806cb80 0x7fe7c815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe7c806cb80 0x7fe7c815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60926/0 (socket says 172.21.15.73:60926) 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7c7e00640 1 -- 172.21.15.73:0/169994770 learned_addr learned my addr 172.21.15.73:0/169994770 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.065+0000 7fe7c7400640 1 --2- 172.21.15.73:0/169994770 >> v2:172.21.15.157:3300/0 conn(0x7fe7c81599f0 0x7fe7c8070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7bea00640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c80add00 0x7fe7c806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7c7e00640 1 -- 172.21.15.73:0/169994770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c80add00 msgr2=0x7fe7c806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7c7e00640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c80add00 0x7fe7c806c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7c7e00640 1 -- 172.21.15.73:0/169994770 >> v2:172.21.15.157:3300/0 conn(0x7fe7c81599f0 msgr2=0x7fe7c8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7c7e00640 1 --2- 172.21.15.73:0/169994770 >> v2:172.21.15.157:3300/0 conn(0x7fe7c81599f0 0x7fe7c8070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.066+0000 7fe7c7e00640 1 -- 172.21.15.73:0/169994770 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c80b27f0 con 0x7fe7c806cb80 2024-04-05T22:29:05.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.067+0000 7fe7c7e00640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c806cb80 0x7fe7c815d6c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806d300 tx=0x7fe7c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5354d8b3816bda76 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.067+0000 7fe7c6a00640 1 -- 172.21.15.73:0/169994770 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c002ca70 con 0x7fe7c806cb80 2024-04-05T22:29:05.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.067+0000 7fe7c6a00640 1 -- 172.21.15.73:0/169994770 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c002cbd0 con 0x7fe7c806cb80 2024-04-05T22:29:05.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.067+0000 7fe7c6a00640 1 -- 172.21.15.73:0/169994770 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00343f0 con 0x7fe7c806cb80 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 -- 172.21.15.73:0/169994770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c806cb80 msgr2=0x7fe7c815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c806cb80 0x7fe7c815d6c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806d300 tx=0x7fe7c002bed0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 -- 172.21.15.73:0/169994770 shutdown_connections 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c806cb80 0x7fe7c815d6c0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 --2- 172.21.15.73:0/169994770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c80add00 0x7fe7c806c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 --2- 172.21.15.73:0/169994770 >> v2:172.21.15.157:3300/0 conn(0x7fe7c81599f0 0x7fe7c8070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 -- 172.21.15.73:0/169994770 >> 172.21.15.73:0/169994770 conn(0x7fe7c8069cb0 msgr2=0x7fe7c806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:05.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 -- 172.21.15.73:0/169994770 shutdown_connections 2024-04-05T22:29:05.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.068+0000 7fe7ce000640 1 -- 172.21.15.73:0/169994770 wait complete. 2024-04-05T22:29:05.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.070+0000 7fe7ce000640 1 Processor -- start 2024-04-05T22:29:05.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.070+0000 7fe7ce000640 1 -- start start 2024-04-05T22:29:05.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c806cb80 0x7fe7c81585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c81599f0 0x7fe7c814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c814ad60 con 0x7fe7c80add00 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c814a4c0 con 0x7fe7c81599f0 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.071+0000 7fe7ce000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c814a7a0 con 0x7fe7c806cb80 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60932/0 (socket says 172.21.15.73:60932) 2024-04-05T22:29:05.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 -- 172.21.15.73:0/1698259454 learned_addr learned my addr 172.21.15.73:0/1698259454 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7c7e00640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c81599f0 0x7fe7c814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7c7400640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c806cb80 0x7fe7c81585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 -- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c806cb80 msgr2=0x7fe7c81585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c806cb80 0x7fe7c81585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 -- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c81599f0 msgr2=0x7fe7c814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c81599f0 0x7fe7c814e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 -- 172.21.15.73:0/1698259454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0009b30 con 0x7fe7c80add00 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.072+0000 7fe7bea00640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000d800 tx=0x7fe7b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.073+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000b6c0 con 0x7fe7c80add00 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.073+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b000b820 con 0x7fe7c80add00 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.073+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c814aa80 con 0x7fe7c80add00 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.073+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000bbb0 con 0x7fe7c80add00 2024-04-05T22:29:05.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.073+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c814e020 con 0x7fe7c80add00 2024-04-05T22:29:05.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.074+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c814fea0 con 0x7fe7c80add00 2024-04-05T22:29:05.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.075+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fe7b000bd10 con 0x7fe7c80add00 2024-04-05T22:29:05.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.076+0000 7fe7bfe00640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 0x7fe7a406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.076+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe7b008a3d0 con 0x7fe7c80add00 2024-04-05T22:29:05.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.076+0000 7fe7c7400640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 0x7fe7a406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.077+0000 7fe7c7400640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 0x7fe7a406d660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8006fd0 tx=0x7fe7b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.080+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe7c814fea0 con 0x7fe7c80add00 2024-04-05T22:29:05.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.272+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7c814fea0 con 0x7fe7c80add00 2024-04-05T22:29:05.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.273+0000 7fe7bfe00640 1 -- 172.21.15.73:0/1698259454 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7c814fea0 con 0x7fe7c80add00 2024-04-05T22:29:05.272 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:05.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 msgr2=0x7fe7a406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 0x7fe7a406d660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8006fd0 tx=0x7fe7b8008040 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 msgr2=0x7fe7c8158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000d800 tx=0x7fe7b000dcc0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 shutdown_connections 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe7a406b1a0 0x7fe7a406d660 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c81599f0 0x7fe7c814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c80add00 0x7fe7c8158ae0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 --2- 172.21.15.73:0/1698259454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7c806cb80 0x7fe7c81585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 >> 172.21.15.73:0/1698259454 conn(0x7fe7c8069cb0 msgr2=0x7fe7c806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 shutdown_connections 2024-04-05T22:29:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.276+0000 7fe7ce000640 1 -- 172.21.15.73:0/1698259454 wait complete. 2024-04-05T22:29:05.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:29:05.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.502+0000 7ff115c00640 1 Processor -- start 2024-04-05T22:29:05.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.502+0000 7ff115c00640 1 -- start start 2024-04-05T22:29:05.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff11015e580 0x7ff11006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff11015e9a0 0x7ff110070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11015f530 con 0x7ff11015e580 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11006bc50 con 0x7ff11015e9a0 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.503+0000 7ff115c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015ef70 con 0x7ff11015a850 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff11015e580 0x7ff11006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff11015e580 0x7ff11006ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60946/0 (socket says 172.21.15.73:60946) 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 -- 172.21.15.73:0/2179907503 learned_addr learned my addr 172.21.15.73:0/2179907503 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 -- 172.21.15.73:0/2179907503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 msgr2=0x7ff11015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff115200640 1 --2- 172.21.15.73:0/2179907503 >> v2:172.21.15.157:3300/0 conn(0x7ff11015e9a0 0x7ff110070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 --2- 172.21.15.73:0/2179907503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 -- 172.21.15.73:0/2179907503 >> v2:172.21.15.157:3300/0 conn(0x7ff11015e9a0 msgr2=0x7ff110070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 --2- 172.21.15.73:0/2179907503 >> v2:172.21.15.157:3300/0 conn(0x7ff11015e9a0 0x7ff110070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.504+0000 7ff107400640 1 -- 172.21.15.73:0/2179907503 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11015f250 con 0x7ff11015e580 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.505+0000 7ff107400640 1 --2- 172.21.15.73:0/2179907503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff11006ff30 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff1100cc210 tx=0x7ff0f802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb3d7ad475c7e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.506+0000 7ff10f400640 1 -- 172.21.15.73:0/2179907503 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8031880 con 0x7ff11015e580 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.506+0000 7ff10f400640 1 -- 172.21.15.73:0/2179907503 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f80319e0 con 0x7ff11015e580 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.506+0000 7ff10f400640 1 -- 172.21.15.73:0/2179907503 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8031d70 con 0x7ff11015e580 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.506+0000 7ff115c00640 1 -- 172.21.15.73:0/2179907503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 msgr2=0x7ff11006ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.506+0000 7ff115c00640 1 --2- 172.21.15.73:0/2179907503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff11006ff30 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff1100cc210 tx=0x7ff0f802b920 comp rx=0 tx=0).stop 2024-04-05T22:29:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 -- 172.21.15.73:0/2179907503 shutdown_connections 2024-04-05T22:29:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 --2- 172.21.15.73:0/2179907503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff11006ff30 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 --2- 172.21.15.73:0/2179907503 >> v2:172.21.15.157:3300/0 conn(0x7ff11015e9a0 0x7ff110070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 --2- 172.21.15.73:0/2179907503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 -- 172.21.15.73:0/2179907503 >> 172.21.15.73:0/2179907503 conn(0x7ff110069cb0 msgr2=0x7ff11006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:05.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 -- 172.21.15.73:0/2179907503 shutdown_connections 2024-04-05T22:29:05.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.507+0000 7ff115c00640 1 -- 172.21.15.73:0/2179907503 wait complete. 2024-04-05T22:29:05.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.508+0000 7ff115c00640 1 Processor -- start 2024-04-05T22:29:05.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 -- start start 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11013a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11015e9a0 0x7ff11014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110153710 con 0x7ff11015e580 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101539f0 con 0x7ff11015e9a0 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.509+0000 7ff115c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff110153cd0 con 0x7ff11015a850 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60948/0 (socket says 172.21.15.73:60948) 2024-04-05T22:29:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 -- 172.21.15.73:0/1296290149 learned_addr learned my addr 172.21.15.73:0/1296290149 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 -- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 msgr2=0x7ff11013a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff10fe00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11013a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff115200640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11015e9a0 0x7ff11014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11013a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 -- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11015e9a0 msgr2=0x7ff11014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11015e9a0 0x7ff11014c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.510+0000 7ff107400640 1 -- 172.21.15.73:0/1296290149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80094d0 con 0x7ff11015e580 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff107400640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8031850 con 0x7ff11015e580 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f80443f0 con 0x7ff11015e580 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80447a0 con 0x7ff11015e580 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff110153fb0 con 0x7ff11015e580 2024-04-05T22:29:05.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.511+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11014e380 con 0x7ff11015e580 2024-04-05T22:29:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.512+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1101544d0 con 0x7ff11015e580 2024-04-05T22:29:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.513+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff0f80303d0 con 0x7ff11015e580 2024-04-05T22:29:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.514+0000 7ff10cc00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 0x7ff0ec06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.515+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff0f80ac8e0 con 0x7ff11015e580 2024-04-05T22:29:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.515+0000 7ff10fe00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 0x7ff0ec06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.516+0000 7ff10fe00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 0x7ff0ec06d660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff100007dc0 tx=0x7ff1000072f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.520+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff1101544d0 con 0x7ff11015e580 2024-04-05T22:29:05.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.714+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1101544d0 con 0x7ff11015e580 2024-04-05T22:29:05.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.715+0000 7ff10cc00640 1 -- 172.21.15.73:0/1296290149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1101544d0 con 0x7ff11015e580 2024-04-05T22:29:05.714 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:29:05.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 msgr2=0x7ff0ec06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 0x7ff0ec06d660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff100007dc0 tx=0x7ff1000072f0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 msgr2=0x7ff110152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f80046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 shutdown_connections 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff0ec06b1a0 0x7ff0ec06d660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11015e9a0 0x7ff11014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff11015e580 0x7ff110152c10 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 --2- 172.21.15.73:0/1296290149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11015a850 0x7ff11013a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 >> 172.21.15.73:0/1296290149 conn(0x7ff110069cb0 msgr2=0x7ff110071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 shutdown_connections 2024-04-05T22:29:05.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.718+0000 7ff115c00640 1 -- 172.21.15.73:0/1296290149 wait complete. 2024-04-05T22:29:05.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:29:05.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.950+0000 7fc2a3000640 1 Processor -- start 2024-04-05T22:29:05.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.951+0000 7fc2a3000640 1 -- start start 2024-04-05T22:29:05.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a3000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc29c0ae170 0x7fc29c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc29c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a3000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c1613d0 con 0x7fc29c160ff0 2024-04-05T22:29:05.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a3000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c0cc070 con 0x7fc29c0ae170 2024-04-05T22:29:05.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c06b4f0 con 0x7fc29c1654e0 2024-04-05T22:29:05.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a1c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc29c160ff0 0x7fc29c0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41228/0 (socket says 172.21.15.73:41228) 2024-04-05T22:29:05.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.952+0000 7fc2a1c00640 1 -- 172.21.15.73:0/4075727391 learned_addr learned my addr 172.21.15.73:0/4075727391 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956376452 0 0) 0x7fc29c1613d0 con 0x7fc29c160ff0 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc284003520 con 0x7fc29c160ff0 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc2a1200640 1 --2- 172.21.15.73:0/4075727391 >> v2:172.21.15.157:3300/0 conn(0x7fc29c0ae170 0x7fc29c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc2a2600640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc29c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358322381 0 0) 0x7fc284003520 con 0x7fc29c160ff0 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 msgr2=0x7fc29c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc29c0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 >> v2:172.21.15.157:3300/0 conn(0x7fc29c0ae170 msgr2=0x7fc29c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 --2- 172.21.15.73:0/4075727391 >> v2:172.21.15.157:3300/0 conn(0x7fc29c0ae170 0x7fc29c0b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c06b760 con 0x7fc29c160ff0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc290001670 con 0x7fc29c160ff0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 >> v1:172.21.15.73:6789/0 conn(0x7fc29c160ff0 legacy=0x7fc29c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc284003970 0x7fc284003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2840047c0 0x7fc284004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc284005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0cc070 con 0x7fc2840047c0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c06b4f0 con 0x7fc284003970 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.953+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28400bab0 con 0x7fc29c1654e0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2840047c0 0x7fc284004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1c00640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc284003970 0x7fc284003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 -- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 msgr2=0x7fc284005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a2600640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc284005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c1654e0 0x7fc284005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 -- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc284003970 msgr2=0x7fc284003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc284003970 0x7fc284003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.954+0000 7fc2a1200640 1 -- 172.21.15.73:0/4075727391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc284003520 con 0x7fc2840047c0 2024-04-05T22:29:05.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.955+0000 7fc2a1200640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2840047c0 0x7fc284004c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00ba00 tx=0x7fc2840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e72de6b9513043 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.955+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c032630 con 0x7fc2840047c0 2024-04-05T22:29:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.955+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c032790 con 0x7fc2840047c0 2024-04-05T22:29:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.955+0000 7fc29be00640 1 -- 172.21.15.73:0/4075727391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c032b20 con 0x7fc2840047c0 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 -- 172.21.15.73:0/4075727391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2840047c0 msgr2=0x7fc284004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 --2- 172.21.15.73:0/4075727391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2840047c0 0x7fc284004c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00ba00 tx=0x7fc2840057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a2600640 1 -- 172.21.15.73:0/4075727391 reap_dead start 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 -- 172.21.15.73:0/4075727391 shutdown_connections 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 -- 172.21.15.73:0/4075727391 >> 172.21.15.73:0/4075727391 conn(0x7fc29c0b0830 msgr2=0x7fc29c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 -- 172.21.15.73:0/4075727391 shutdown_connections 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.956+0000 7fc2a3000640 1 -- 172.21.15.73:0/4075727391 wait complete. 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.958+0000 7fc2a3000640 1 Processor -- start 2024-04-05T22:29:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.958+0000 7fc2a3000640 1 -- start start 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2840047c0 0x7fc29c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c146a90 0x7fc29c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c164490 con 0x7fc29c147560 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c162eb0 con 0x7fc29c146a90 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c163190 con 0x7fc2840047c0 2024-04-05T22:29:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51444/0 (socket says 172.21.15.73:51444) 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.959+0000 7fc2a1200640 1 -- 172.21.15.73:0/2981594867 learned_addr learned my addr 172.21.15.73:0/2981594867 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1c00640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2840047c0 0x7fc29c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 -- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2840047c0 msgr2=0x7fc29c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2840047c0 0x7fc29c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 -- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c146a90 msgr2=0x7fc29c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a2600640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c146a90 0x7fc29c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c146a90 0x7fc29c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 -- 172.21.15.73:0/2981594867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c163470 con 0x7fc29c147560 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a1200640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc28c02beb0 tx=0x7fc28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c03ba30 con 0x7fc29c147560 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c03bb90 con 0x7fc29c147560 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c030400 con 0x7fc29c147560 2024-04-05T22:29:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c163750 con 0x7fc29c147560 2024-04-05T22:29:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.960+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c162b10 con 0x7fc29c147560 2024-04-05T22:29:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.962+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c04e160 con 0x7fc29c147560 2024-04-05T22:29:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.962+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fc28c030560 con 0x7fc29c147560 2024-04-05T22:29:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.963+0000 7fc299600640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 0x7fc28006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.964+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc28c0ac2d0 con 0x7fc29c147560 2024-04-05T22:29:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.964+0000 7fc2a1c00640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 0x7fc28006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.965+0000 7fc2a1c00640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 0x7fc28006d660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2900017a0 tx=0x7fc290009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:05.967+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc28c07a580 con 0x7fc29c147560 2024-04-05T22:29:06.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.161+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc29c164870 con 0x7fc29c147560 2024-04-05T22:29:06.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.162+0000 7fc299600640 1 -- 172.21.15.73:0/2981594867 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc28c07a110 con 0x7fc29c147560 2024-04-05T22:29:06.161 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 msgr2=0x7fc28006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 0x7fc28006d660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2900017a0 tx=0x7fc290009040 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 msgr2=0x7fc29c163f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc28c02beb0 tx=0x7fc28c004730 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 shutdown_connections 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc28006b1a0 0x7fc28006d660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c146a90 0x7fc29c0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c147560 0x7fc29c163f50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 --2- 172.21.15.73:0/2981594867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2840047c0 0x7fc29c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.164+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 >> 172.21.15.73:0/2981594867 conn(0x7fc29c0b0830 msgr2=0x7fc29c0af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.165+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 shutdown_connections 2024-04-05T22:29:06.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.165+0000 7fc2a3000640 1 -- 172.21.15.73:0/2981594867 wait complete. 2024-04-05T22:29:06.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:29:06.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.391+0000 7f7773a00640 1 Processor -- start 2024-04-05T22:29:06.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.391+0000 7f7773a00640 1 -- start start 2024-04-05T22:29:06.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.393+0000 7f7773a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.393+0000 7f7773a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 0x7f776c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.393+0000 7f7773a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c06f5b0 con 0x7f776c1587d0 2024-04-05T22:29:06.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.393+0000 7f7773a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c06d500 con 0x7f776c15e850 2024-04-05T22:29:06.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.393+0000 7f7773a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c15f220 con 0x7f776c06cb80 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f7771a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 0x7f776c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f776be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f7771a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 0x7f776c15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:50884/0 (socket says 172.21.15.73:50884) 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f776be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:54720/0 (socket says 172.21.15.73:54720) 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f7771a00640 1 -- 172.21.15.73:0/3293766102 learned_addr learned my addr 172.21.15.73:0/3293766102 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:06.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f776be00640 1 -- 172.21.15.73:0/3293766102 >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 msgr2=0x7f776c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:06.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f776be00640 1 --2- 172.21.15.73:0/3293766102 >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 0x7f776c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.394+0000 7f776be00640 1 -- 172.21.15.73:0/3293766102 >> v1:172.21.15.157:6789/0 conn(0x7f776c1587d0 legacy=0x7f776c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.395+0000 7f776be00640 1 -- 172.21.15.73:0/3293766102 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c070410 con 0x7f776c06cb80 2024-04-05T22:29:06.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.396+0000 7f776be00640 1 --2- 172.21.15.73:0/3293766102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f77580097a0 tx=0x7f775802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bf06b3c7a928ca server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:06.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.396+0000 7f776b400640 1 -- 172.21.15.73:0/3293766102 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758031830 con 0x7f776c06cb80 2024-04-05T22:29:06.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f776b400640 1 -- 172.21.15.73:0/3293766102 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758031990 con 0x7f776c06cb80 2024-04-05T22:29:06.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f776b400640 1 -- 172.21.15.73:0/3293766102 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758031d20 con 0x7f776c06cb80 2024-04-05T22:29:06.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 -- 172.21.15.73:0/3293766102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 msgr2=0x7f776c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 --2- 172.21.15.73:0/3293766102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f77580097a0 tx=0x7f775802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 -- 172.21.15.73:0/3293766102 shutdown_connections 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 --2- 172.21.15.73:0/3293766102 >> v2:172.21.15.157:3300/0 conn(0x7f776c15e850 0x7f776c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 --2- 172.21.15.73:0/3293766102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c06cb80 0x7f776c06cfc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 -- 172.21.15.73:0/3293766102 >> 172.21.15.73:0/3293766102 conn(0x7f776c0ce8e0 msgr2=0x7f776c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.397+0000 7f7773a00640 1 -- 172.21.15.73:0/3293766102 shutdown_connections 2024-04-05T22:29:06.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.398+0000 7f7773a00640 1 -- 172.21.15.73:0/3293766102 wait complete. 2024-04-05T22:29:06.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.400+0000 7f7773a00640 1 Processor -- start 2024-04-05T22:29:06.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.400+0000 7f7773a00640 1 -- start start 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f776c06cb80 0x7f776c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 0x7f776c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0af470 con 0x7f776c1587d0 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0b02e0 con 0x7f776c06cb80 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7773a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0ae860 con 0x7f776c15e850 2024-04-05T22:29:06.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f776be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f776be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51460/0 (socket says 172.21.15.73:51460) 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f776be00640 1 -- 172.21.15.73:0/3060443664 learned_addr learned my addr 172.21.15.73:0/3060443664 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.401+0000 7f7771000640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f776c06cb80 0x7f776c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7771a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 0x7f776c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 -- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 msgr2=0x7f776c14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 0x7f776c14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 -- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f776c06cb80 msgr2=0x7f776c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f776c06cb80 0x7f776c0afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 -- 172.21.15.73:0/3060443664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758009400 con 0x7f776c1587d0 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7771a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 0x7f776c14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:06.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f776be00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f77580098c0 tx=0x7f77580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775803a6f0 con 0x7f776c1587d0 2024-04-05T22:29:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775803a870 con 0x7f776c1587d0 2024-04-05T22:29:06.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c0aeb40 con 0x7f776c1587d0 2024-04-05T22:29:06.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.402+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775803ac20 con 0x7f776c1587d0 2024-04-05T22:29:06.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.403+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c0af100 con 0x7f776c1587d0 2024-04-05T22:29:06.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.404+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c066af0 con 0x7f776c1587d0 2024-04-05T22:29:06.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.405+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f7758043410 con 0x7f776c1587d0 2024-04-05T22:29:06.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.405+0000 7f7768c00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 0x7f774c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:06.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.406+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f77580abaa0 con 0x7f776c1587d0 2024-04-05T22:29:06.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.406+0000 7f7771000640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 0x7f774c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:06.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.407+0000 7f7771000640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 0x7f774c06d660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f775c0012e0 tx=0x7f775c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:06.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.410+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f77580762c0 con 0x7f776c1587d0 2024-04-05T22:29:06.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.618+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f776c068160 con 0x7f776c1587d0 2024-04-05T22:29:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.619+0000 7f7768c00640 1 -- 172.21.15.73:0/3060443664 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7758035080 con 0x7f776c1587d0 2024-04-05T22:29:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.622+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 msgr2=0x7f774c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.622+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 0x7f774c06d660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f775c0012e0 tx=0x7f775c009040 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.622+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 msgr2=0x7f776c1f1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.622+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f77580098c0 tx=0x7f77580046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 shutdown_connections 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f774c06b1a0 0x7f774c06d660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f776c15e850 0x7f776c14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f776c1587d0 0x7f776c1f1d00 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 --2- 172.21.15.73:0/3060443664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f776c06cb80 0x7f776c0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:06.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 >> 172.21.15.73:0/3060443664 conn(0x7f776c0ce8e0 msgr2=0x7f776c069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:06.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 shutdown_connections 2024-04-05T22:29:06.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:06.623+0000 7f7773a00640 1 -- 172.21.15.73:0/3060443664 wait complete. 2024-04-05T22:29:06.640 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:29:06.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:06.641 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:06.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:29:06.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:06.775 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:29:06.830 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:06.830 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:06.871 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:06.871 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:06.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:07.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.163+0000 7f13a8600640 1 Processor -- start 2024-04-05T22:29:07.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.163+0000 7f13a8600640 1 -- start start 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.164+0000 7f13a8600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13a01541a0 0x7f13a00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.164+0000 7f13a8600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13a00ce000 0x7f13a015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.164+0000 7f13a8600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a015aa60 con 0x7f13a006d710 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.164+0000 7f13a8600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00b3b60 con 0x7f13a01541a0 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.164+0000 7f13a8600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00701c0 con 0x7f13a00ce000 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f13a006d710 0x7f13a00cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41236/0 (socket says 172.21.15.73:41236) 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5200640 1 -- 172.21.15.73:0/4237373718 learned_addr learned my addr 172.21.15.73:0/4237373718 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:07.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13a01541a0 0x7f13a00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5c00640 1 -- 172.21.15.73:0/4237373718 >> v2:172.21.15.157:3300/0 conn(0x7f13a00ce000 msgr2=0x7f13a015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:07.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5c00640 1 --2- 172.21.15.73:0/4237373718 >> v2:172.21.15.157:3300/0 conn(0x7f13a00ce000 0x7f13a015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5c00640 1 -- 172.21.15.73:0/4237373718 >> v1:172.21.15.73:6789/0 conn(0x7f13a006d710 legacy=0x7f13a00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.165+0000 7f13a5c00640 1 -- 172.21.15.73:0/4237373718 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00b3880 con 0x7f13a01541a0 2024-04-05T22:29:07.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.166+0000 7f13a5c00640 1 --2- 172.21.15.73:0/4237373718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a00b3050 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f13a015a9d0 tx=0x7f139402b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3e5722ee9c7ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.166+0000 7f139fe00640 1 -- 172.21.15.73:0/4237373718 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394030710 con 0x7f13a01541a0 2024-04-05T22:29:07.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.166+0000 7f139fe00640 1 -- 172.21.15.73:0/4237373718 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139402e050 con 0x7f13a01541a0 2024-04-05T22:29:07.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.166+0000 7f139fe00640 1 -- 172.21.15.73:0/4237373718 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394030a90 con 0x7f13a01541a0 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 -- 172.21.15.73:0/4237373718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 msgr2=0x7f13a00b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 --2- 172.21.15.73:0/4237373718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a00b3050 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f13a015a9d0 tx=0x7f139402b680 comp rx=0 tx=0).stop 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 -- 172.21.15.73:0/4237373718 shutdown_connections 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 --2- 172.21.15.73:0/4237373718 >> v2:172.21.15.157:3300/0 conn(0x7f13a00ce000 0x7f13a015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 --2- 172.21.15.73:0/4237373718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a00b3050 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 -- 172.21.15.73:0/4237373718 >> 172.21.15.73:0/4237373718 conn(0x7f13a00ce8e0 msgr2=0x7f13a00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 -- 172.21.15.73:0/4237373718 shutdown_connections 2024-04-05T22:29:07.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.167+0000 7f13a8600640 1 -- 172.21.15.73:0/4237373718 wait complete. 2024-04-05T22:29:07.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.169+0000 7f13a8600640 1 Processor -- start 2024-04-05T22:29:07.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.169+0000 7f13a8600640 1 -- start start 2024-04-05T22:29:07.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a006d710 0x7f13a00b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00ce000 0x7f13a00b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0159880 con 0x7f13a01541a0 2024-04-05T22:29:07.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0159b60 con 0x7f13a00ce000 2024-04-05T22:29:07.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00abb00 con 0x7f13a006d710 2024-04-05T22:29:07.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51482/0 (socket says 172.21.15.73:51482) 2024-04-05T22:29:07.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a6600640 1 -- 172.21.15.73:0/2969737968 learned_addr learned my addr 172.21.15.73:0/2969737968 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.170+0000 7f13a5c00640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a006d710 0x7f13a00b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a5200640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00ce000 0x7f13a00b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 -- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a006d710 msgr2=0x7f13a00b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a006d710 0x7f13a00b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 -- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00ce000 msgr2=0x7f13a00b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00ce000 0x7f13a00b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 -- 172.21.15.73:0/2969737968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394009400 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a6600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f139800b970 tx=0x7f139800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800cb40 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139800ccc0 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398014680 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a00ab7f0 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.171+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a006c380 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.173+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a0066af0 con 0x7f13a01541a0 2024-04-05T22:29:07.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.174+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f139801d3b0 con 0x7f13a01541a0 2024-04-05T22:29:07.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.174+0000 7f139d600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 0x7f138006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.175+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1398089cf0 con 0x7f13a01541a0 2024-04-05T22:29:07.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.175+0000 7f13a5c00640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 0x7f138006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.176+0000 7f13a5c00640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 0x7f138006d660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f1394008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.179+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f13980546b0 con 0x7f13a01541a0 2024-04-05T22:29:07.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.373+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13a01599c0 con 0x7f13a01541a0 2024-04-05T22:29:07.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.374+0000 7f139d600640 1 -- 172.21.15.73:0/2969737968 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16222 (secure 0 0 0) 0x7f13a01599c0 con 0x7f13a01541a0 2024-04-05T22:29:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:07.374 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":123,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:04.441444+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:07.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 msgr2=0x7f138006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 0x7f138006d660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f1394008030 comp rx=0 tx=0).stop 2024-04-05T22:29:07.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 msgr2=0x7f13a0148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f139800b970 tx=0x7f139800be30 comp rx=0 tx=0).stop 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 shutdown_connections 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f138006b1a0 0x7f138006d660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01541a0 0x7f13a0148730 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00ce000 0x7f13a00b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 --2- 172.21.15.73:0/2969737968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a006d710 0x7f13a00b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 >> 172.21.15.73:0/2969737968 conn(0x7f13a00ce8e0 msgr2=0x7f13a00af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 shutdown_connections 2024-04-05T22:29:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.377+0000 7f13a8600640 1 -- 172.21.15.73:0/2969737968 wait complete. 2024-04-05T22:29:07.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:28:42.000556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:28:43.153067+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:07.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:07.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.600+0000 7fed2c200640 1 Processor -- start 2024-04-05T22:29:07.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.600+0000 7fed2c200640 1 -- start start 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2c200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 0x7fed2406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2c200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fed2406ba70 0x7fed24164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2c200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2406bea0 con 0x7fed2415abe0 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2c200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2415afc0 con 0x7fed2406ba70 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2c200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2406f5b0 con 0x7fed2406c9a0 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fed2406ba70 0x7fed24164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fed2406ba70 0x7fed24164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51486/0 (socket says 172.21.15.73:51486) 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.602+0000 7fed2a200640 1 -- 172.21.15.73:0/2423317497 learned_addr learned my addr 172.21.15.73:0/2423317497 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed28e00640 1 --2- 172.21.15.73:0/2423317497 >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 0x7fed2406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed2a200640 1 -- 172.21.15.73:0/2423317497 >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 msgr2=0x7fed2406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:07.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed2a200640 1 --2- 172.21.15.73:0/2423317497 >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 0x7fed2406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed2a200640 1 -- 172.21.15.73:0/2423317497 >> v1:172.21.15.157:6789/0 conn(0x7fed2415abe0 legacy=0x7fed240712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed2a200640 1 -- 172.21.15.73:0/2423317497 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2406f890 con 0x7fed2406ba70 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.603+0000 7fed28e00640 1 --2- 172.21.15.73:0/2423317497 >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 0x7fed2406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.604+0000 7fed2a200640 1 --2- 172.21.15.73:0/2423317497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406ba70 0x7fed24164740 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fed2406c900 tx=0x7fed1c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f7d6a7aaa9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.604+0000 7fed23e00640 1 -- 172.21.15.73:0/2423317497 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c02c950 con 0x7fed2406ba70 2024-04-05T22:29:07.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.604+0000 7fed23e00640 1 -- 172.21.15.73:0/2423317497 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c02cab0 con 0x7fed2406ba70 2024-04-05T22:29:07.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.604+0000 7fed23e00640 1 -- 172.21.15.73:0/2423317497 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c02ce40 con 0x7fed2406ba70 2024-04-05T22:29:07.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 -- 172.21.15.73:0/2423317497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406ba70 msgr2=0x7fed24164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 --2- 172.21.15.73:0/2423317497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406ba70 0x7fed24164740 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fed2406c900 tx=0x7fed1c02bdb0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 -- 172.21.15.73:0/2423317497 shutdown_connections 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 --2- 172.21.15.73:0/2423317497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406ba70 0x7fed24164740 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 --2- 172.21.15.73:0/2423317497 >> v2:172.21.15.157:3300/0 conn(0x7fed2406c9a0 0x7fed2406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 -- 172.21.15.73:0/2423317497 >> 172.21.15.73:0/2423317497 conn(0x7fed24069cb0 msgr2=0x7fed2406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 -- 172.21.15.73:0/2423317497 shutdown_connections 2024-04-05T22:29:07.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.605+0000 7fed2c200640 1 -- 172.21.15.73:0/2423317497 wait complete. 2024-04-05T22:29:07.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.607+0000 7fed2c200640 1 Processor -- start 2024-04-05T22:29:07.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.607+0000 7fed2c200640 1 -- start start 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed2406ba70 0x7fed240b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed2415abe0 0x7fed241537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2414c820 con 0x7fed2406c9a0 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240b3890 con 0x7fed2415abe0 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2414bc10 con 0x7fed2406ba70 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed28e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed28e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51500/0 (socket says 172.21.15.73:51500) 2024-04-05T22:29:07.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed28e00640 1 -- 172.21.15.73:0/3367070831 learned_addr learned my addr 172.21.15.73:0/3367070831 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.608+0000 7fed2a200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed2415abe0 0x7fed241537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 -- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed2406ba70 msgr2=0x7fed240b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed2406ba70 0x7fed240b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 -- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed2415abe0 msgr2=0x7fed241537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed2415abe0 0x7fed241537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 -- 172.21.15.73:0/3367070831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009b30 con 0x7fed2406c9a0 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed28e00640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c7f0 tx=0x7fed1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10007a30 con 0x7fed2406c9a0 2024-04-05T22:29:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10007b90 con 0x7fed2406c9a0 2024-04-05T22:29:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1000ce40 con 0x7fed2406c9a0 2024-04-05T22:29:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2414bef0 con 0x7fed2406c9a0 2024-04-05T22:29:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.609+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2414c4b0 con 0x7fed2406c9a0 2024-04-05T22:29:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.611+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24068160 con 0x7fed2406c9a0 2024-04-05T22:29:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.611+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fed100113f0 con 0x7fed2406c9a0 2024-04-05T22:29:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.612+0000 7fed21600640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 0x7fed0406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.612+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fed1008a100 con 0x7fed2406c9a0 2024-04-05T22:29:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.613+0000 7fed29800640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 0x7fed0406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.614+0000 7fed29800640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 0x7fed0406d660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed241f3920 tx=0x7fed18007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:07.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.617+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fed100bd650 con 0x7fed2406c9a0 2024-04-05T22:29:07.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.816+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed2414c960 con 0x7fed2406c9a0 2024-04-05T22:29:07.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.817+0000 7fed21600640 1 -- 172.21.15.73:0/3367070831 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed10054920 con 0x7fed2406c9a0 2024-04-05T22:29:07.816 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 msgr2=0x7fed0406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 0x7fed0406d660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed241f3920 tx=0x7fed18007990 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 msgr2=0x7fed241532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c7f0 tx=0x7fed1000cbf0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 shutdown_connections 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fed0406b1a0 0x7fed0406d660 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed2415abe0 0x7fed241537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2406c9a0 0x7fed241532b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 --2- 172.21.15.73:0/3367070831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed2406ba70 0x7fed240b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:07.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.819+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 >> 172.21.15.73:0/3367070831 conn(0x7fed24069cb0 msgr2=0x7fed240af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:07.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.820+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 shutdown_connections 2024-04-05T22:29:07.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:07.820+0000 7fed2c200640 1 -- 172.21.15.73:0/3367070831 wait complete. 2024-04-05T22:29:07.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:29:08.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.049+0000 7f2b4d400640 1 Processor -- start 2024-04-05T22:29:08.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.050+0000 7f2b4d400640 1 -- start start 2024-04-05T22:29:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b4d400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b48071670 0x7f2b4815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b4d400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4815ae50 con 0x7f2b4815aa70 2024-04-05T22:29:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b4d400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4815e3b0 con 0x7f2b4815dbc0 2024-04-05T22:29:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b4d400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480ac9e0 con 0x7f2b48071670 2024-04-05T22:29:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b46800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2b4815aa70 0x7f2b48070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41242/0 (socket says 172.21.15.73:41242) 2024-04-05T22:29:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.051+0000 7f2b46800640 1 -- 172.21.15.73:0/2164211755 learned_addr learned my addr 172.21.15.73:0/2164211755 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:08.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b3de00640 1 --2- 172.21.15.73:0/2164211755 >> v2:172.21.15.157:3300/0 conn(0x7f2b48071670 0x7f2b4815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794678324 0 0) 0x7f2b4815ae50 con 0x7f2b4815aa70 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2c003520 con 0x7f2b4815aa70 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270085191 0 0) 0x7f2b2c003520 con 0x7f2b4815aa70 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 >> v2:172.21.15.157:3300/0 conn(0x7f2b48071670 msgr2=0x7f2b4815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 --2- 172.21.15.73:0/2164211755 >> v2:172.21.15.157:3300/0 conn(0x7f2b48071670 0x7f2b4815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 >> v1:172.21.15.157:6789/0 conn(0x7f2b4815dbc0 legacy=0x7f2b4806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480accc0 con 0x7f2b4815aa70 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2b38001670 con 0x7f2b4815aa70 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.052+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 >> v1:172.21.15.73:6789/0 conn(0x7f2b4815aa70 legacy=0x7f2b48070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c005180 0x7f2b2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480ac9e0 con 0x7f2b2c003970 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00c020 con 0x7f2b2c005180 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00c1e0 con 0x7f2b2c0047c0 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b47200640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c005180 0x7f2b2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 -- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0047c0 msgr2=0x7f2b2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b2c0047c0 0x7f2b2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 -- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c005180 msgr2=0x7f2b2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c005180 0x7f2b2c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b46800640 1 -- 172.21.15.73:0/2164211755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c003520 con 0x7f2b2c003970 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b47200640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b2c005180 0x7f2b2c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.053+0000 7f2b47200640 1 -- 172.21.15.73:0/2164211755 reap_dead start 2024-04-05T22:29:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b46800640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b2c003d90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c190 tx=0x7f2b2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626d9e548bb6af22 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38005ad0 con 0x7f2b2c003970 2024-04-05T22:29:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b38005c30 con 0x7f2b2c003970 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b45e00640 1 -- 172.21.15.73:0/2164211755 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3802cae0 con 0x7f2b2c003970 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b4d400640 1 -- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 msgr2=0x7f2b2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.055+0000 7f2b4d400640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b2c003d90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c190 tx=0x7f2b2c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.056+0000 7f2b4d400640 1 -- 172.21.15.73:0/2164211755 shutdown_connections 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.056+0000 7f2b4d400640 1 --2- 172.21.15.73:0/2164211755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b2c003d90 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.056+0000 7f2b4d400640 1 -- 172.21.15.73:0/2164211755 >> 172.21.15.73:0/2164211755 conn(0x7f2b48069cb0 msgr2=0x7f2b4806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.056+0000 7f2b4d400640 1 -- 172.21.15.73:0/2164211755 shutdown_connections 2024-04-05T22:29:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.056+0000 7f2b4d400640 1 -- 172.21.15.73:0/2164211755 wait complete. 2024-04-05T22:29:08.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.057+0000 7f2b4d400640 1 Processor -- start 2024-04-05T22:29:08.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.057+0000 7f2b4d400640 1 -- start start 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b481569b0 0x7f2b481521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b481f2480 0x7f2b481f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48157170 con 0x7f2b2c003970 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48157450 con 0x7f2b481569b0 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.058+0000 7f2b4d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48157730 con 0x7f2b481f2480 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51512/0 (socket says 172.21.15.73:51512) 2024-04-05T22:29:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 -- 172.21.15.73:0/216838911 learned_addr learned my addr 172.21.15.73:0/216838911 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 -- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b481f2480 msgr2=0x7f2b481f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b481f2480 0x7f2b481f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 -- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b481569b0 msgr2=0x7f2b481521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b481569b0 0x7f2b481521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 -- 172.21.15.73:0/216838911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48157a10 con 0x7f2b2c003970 2024-04-05T22:29:08.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.059+0000 7f2b46800640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f2b38001930 tx=0x7f2b3802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.060+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3803c040 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.060+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3802ccb0 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.060+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380072b0 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.060+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b48157cf0 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.060+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b48152720 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.061+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b48157870 con 0x7f2b2c003970 2024-04-05T22:29:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.062+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f2b38032070 con 0x7f2b2c003970 2024-04-05T22:29:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.063+0000 7f2b3f400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 0x7f2b2406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.063+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2b380ab2d0 con 0x7f2b2c003970 2024-04-05T22:29:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.063+0000 7f2b3de00640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 0x7f2b2406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.064+0000 7f2b3de00640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 0x7f2b2406d660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2b300093d0 tx=0x7f2b30009300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.067+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2b48157870 con 0x7f2b2c003970 2024-04-05T22:29:08.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.265+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b48157870 con 0x7f2b2c003970 2024-04-05T22:29:08.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.266+0000 7f2b3f400640 1 -- 172.21.15.73:0/216838911 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b48157870 con 0x7f2b2c003970 2024-04-05T22:29:08.266 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.269+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 msgr2=0x7f2b2406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.269+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 0x7f2b2406d660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2b300093d0 tx=0x7f2b30009300 comp rx=0 tx=0).stop 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.269+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 msgr2=0x7f2b480afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.269+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f2b38001930 tx=0x7f2b3802e070 comp rx=0 tx=0).stop 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 shutdown_connections 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2b2406b1a0 0x7f2b2406d660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b481f2480 0x7f2b481f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b481569b0 0x7f2b481521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 --2- 172.21.15.73:0/216838911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b2c003970 0x7f2b480afed0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 >> 172.21.15.73:0/216838911 conn(0x7f2b48069cb0 msgr2=0x7f2b4806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:08.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 shutdown_connections 2024-04-05T22:29:08.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.270+0000 7f2b4d400640 1 -- 172.21.15.73:0/216838911 wait complete. 2024-04-05T22:29:08.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:29:08.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.488+0000 7fec74e00640 1 Processor -- start 2024-04-05T22:29:08.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.488+0000 7fec74e00640 1 -- start start 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.489+0000 7fec74e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec7015a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.489+0000 7fec74e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fec70070660 0x7fec70071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.489+0000 7fec74e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec70071b80 con 0x7fec70143660 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.489+0000 7fec74e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700cb8c0 con 0x7fec70070660 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.489+0000 7fec74e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec70070e70 con 0x7fec7006cd40 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.490+0000 7fec6de00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fec70143660 0x7fec7006c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41246/0 (socket says 172.21.15.73:41246) 2024-04-05T22:29:08.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.490+0000 7fec6de00640 1 -- 172.21.15.73:0/3664688016 learned_addr learned my addr 172.21.15.73:0/3664688016 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.490+0000 7fec6d400640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec7015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.490+0000 7fec6e800640 1 --2- 172.21.15.73:0/3664688016 >> v2:172.21.15.157:3300/0 conn(0x7fec70070660 0x7fec70071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.491+0000 7fec6e800640 1 -- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 msgr2=0x7fec7015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.491+0000 7fec6e800640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec7015a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.491+0000 7fec6e800640 1 -- 172.21.15.73:0/3664688016 >> v1:172.21.15.73:6789/0 conn(0x7fec70143660 legacy=0x7fec7006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.491+0000 7fec6e800640 1 -- 172.21.15.73:0/3664688016 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700cceb0 con 0x7fec70070660 2024-04-05T22:29:08.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.491+0000 7fec6d400640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec7015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:08.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.493+0000 7fec6e800640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec70071640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fec68009d90 tx=0x7fec6802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a268d880991b0a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.498+0000 7fec6ca00640 1 -- 172.21.15.73:0/3664688016 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6802cbd0 con 0x7fec70070660 2024-04-05T22:29:08.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.498+0000 7fec6ca00640 1 -- 172.21.15.73:0/3664688016 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6802cd30 con 0x7fec70070660 2024-04-05T22:29:08.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.498+0000 7fec6ca00640 1 -- 172.21.15.73:0/3664688016 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6802cea0 con 0x7fec70070660 2024-04-05T22:29:08.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 -- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 msgr2=0x7fec70071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec70071640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fec68009d90 tx=0x7fec6802be20 comp rx=0 tx=0).stop 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 -- 172.21.15.73:0/3664688016 shutdown_connections 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec70071640 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 --2- 172.21.15.73:0/3664688016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec7015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 -- 172.21.15.73:0/3664688016 >> 172.21.15.73:0/3664688016 conn(0x7fec700b0830 msgr2=0x7fec700b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 -- 172.21.15.73:0/3664688016 shutdown_connections 2024-04-05T22:29:08.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.499+0000 7fec74e00640 1 -- 172.21.15.73:0/3664688016 wait complete. 2024-04-05T22:29:08.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.501+0000 7fec74e00640 1 Processor -- start 2024-04-05T22:29:08.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.501+0000 7fec74e00640 1 -- start start 2024-04-05T22:29:08.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec701f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec701361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7013c7c0 con 0x7fec70143660 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701f4540 con 0x7fec70070660 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec74e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec701f4820 con 0x7fec7006cd40 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec6e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec6e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51524/0 (socket says 172.21.15.73:51524) 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec6e800640 1 -- 172.21.15.73:0/2113707871 learned_addr learned my addr 172.21.15.73:0/2113707871 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec6de00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec701f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.502+0000 7fec6d400640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec701361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 -- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 msgr2=0x7fec701f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec701f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 -- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 msgr2=0x7fec701361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec701361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 -- 172.21.15.73:0/2113707871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009400 con 0x7fec70143660 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6de00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec701f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6d400640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec701361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec6e800640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fec68009d90 tx=0x7fec68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6802cea0 con 0x7fec70143660 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68040650 con 0x7fec70143660 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68040a00 con 0x7fec70143660 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec68009a70 con 0x7fec70143660 2024-04-05T22:29:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.503+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701f4de0 con 0x7fec70143660 2024-04-05T22:29:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.505+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70068160 con 0x7fec70143660 2024-04-05T22:29:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.505+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fec68040b60 con 0x7fec70143660 2024-04-05T22:29:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.506+0000 7fec66000640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 0x7fec5006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.506+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fec680acbf0 con 0x7fec70143660 2024-04-05T22:29:08.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.507+0000 7fec6de00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 0x7fec5006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.508+0000 7fec6de00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 0x7fec5006d660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec60005880 tx=0x7fec60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.511+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fec68077570 con 0x7fec70143660 2024-04-05T22:29:08.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.702+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec7006d620 con 0x7fec70143660 2024-04-05T22:29:08.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.703+0000 7fec66000640 1 -- 172.21.15.73:0/2113707871 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec68039020 con 0x7fec70143660 2024-04-05T22:29:08.703 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:29:08.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.705+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 msgr2=0x7fec5006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.705+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 0x7fec5006d660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec60005880 tx=0x7fec60009040 comp rx=0 tx=0).stop 2024-04-05T22:29:08.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.705+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 msgr2=0x7fec700a8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.705+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fec68009d90 tx=0x7fec68004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:08.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 shutdown_connections 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fec5006b1a0 0x7fec5006d660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec70143660 0x7fec700a8dd0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec70070660 0x7fec701361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 --2- 172.21.15.73:0/2113707871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec7006cd40 0x7fec701f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 >> 172.21.15.73:0/2113707871 conn(0x7fec700b0830 msgr2=0x7fec700b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 shutdown_connections 2024-04-05T22:29:08.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.706+0000 7fec74e00640 1 -- 172.21.15.73:0/2113707871 wait complete. 2024-04-05T22:29:08.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:29:08.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.930+0000 7f9650c00640 1 Processor -- start 2024-04-05T22:29:08.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.930+0000 7f9650c00640 1 -- start start 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9650c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f964c160620 0x7f964c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9650c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f964c0b1870 0x7f964c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9650c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c0ce5d0 con 0x7f964c0ad6b0 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9650c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c0ce010 con 0x7f964c0b1870 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9650c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c0ce2f0 con 0x7f964c160620 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f964a800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f964c0ad6b0 0x7f964c0cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41252/0 (socket says 172.21.15.73:41252) 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9641400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f964c0b1870 0x7f964c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f964a800640 1 -- 172.21.15.73:0/3964302180 learned_addr learned my addr 172.21.15.73:0/3964302180 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9649e00640 1 --2- 172.21.15.73:0/3964302180 >> v2:172.21.15.157:3300/0 conn(0x7f964c160620 0x7f964c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.932+0000 7f9649400640 1 -- 172.21.15.73:0/3964302180 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601958025 0 0) 0x7f964c0ce5d0 con 0x7f964c0ad6b0 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.933+0000 7f9641400640 1 -- 172.21.15.73:0/3964302180 >> v2:172.21.15.157:3300/0 conn(0x7f964c160620 msgr2=0x7f964c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.933+0000 7f9641400640 1 --2- 172.21.15.73:0/3964302180 >> v2:172.21.15.157:3300/0 conn(0x7f964c160620 0x7f964c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.933+0000 7f9641400640 1 -- 172.21.15.73:0/3964302180 >> v1:172.21.15.73:6789/0 conn(0x7f964c0ad6b0 legacy=0x7f964c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.933+0000 7f9641400640 1 -- 172.21.15.73:0/3964302180 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c0ac970 con 0x7f964c0b1870 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9641400640 1 --2- 172.21.15.73:0/3964302180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c0b1870 0x7f964c0ad170 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f964c0667b0 tx=0x7f963402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d5dfe55766b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9649400640 1 -- 172.21.15.73:0/3964302180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634031830 con 0x7f964c0b1870 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9649400640 1 -- 172.21.15.73:0/3964302180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634031990 con 0x7f964c0b1870 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9649400640 1 -- 172.21.15.73:0/3964302180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634031d20 con 0x7f964c0b1870 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9650c00640 1 -- 172.21.15.73:0/3964302180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c0b1870 msgr2=0x7f964c0ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.934+0000 7f9650c00640 1 --2- 172.21.15.73:0/3964302180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c0b1870 0x7f964c0ad170 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f964c0667b0 tx=0x7f963402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 -- 172.21.15.73:0/3964302180 shutdown_connections 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 --2- 172.21.15.73:0/3964302180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c0b1870 0x7f964c0ad170 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 --2- 172.21.15.73:0/3964302180 >> v2:172.21.15.157:3300/0 conn(0x7f964c160620 0x7f964c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 -- 172.21.15.73:0/3964302180 >> 172.21.15.73:0/3964302180 conn(0x7f964c0ce8e0 msgr2=0x7f964c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:08.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 -- 172.21.15.73:0/3964302180 shutdown_connections 2024-04-05T22:29:08.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.935+0000 7f9650c00640 1 -- 172.21.15.73:0/3964302180 wait complete. 2024-04-05T22:29:08.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.936+0000 7f9650c00640 1 Processor -- start 2024-04-05T22:29:08.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.937+0000 7f9650c00640 1 -- start start 2024-04-05T22:29:08.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.937+0000 7f9650c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f964c0ad6b0 0x7f964c1f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.937+0000 7f9650c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964c0b1870 0x7f964c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.937+0000 7f9650c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.937+0000 7f9650c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c15d1f0 con 0x7f964c160620 2024-04-05T22:29:08.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f9650c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1f34e0 con 0x7f964c0b1870 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f9650c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1f2970 con 0x7f964c0ad6b0 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51530/0 (socket says 172.21.15.73:51530) 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 -- 172.21.15.73:0/4144876966 learned_addr learned my addr 172.21.15.73:0/4144876966 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f9649e00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f964c0ad6b0 0x7f964c1f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 -- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f964c0ad6b0 msgr2=0x7f964c1f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f964c0ad6b0 0x7f964c1f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 -- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964c0b1870 msgr2=0x7f964c14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964c0b1870 0x7f964c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.938+0000 7f964a800640 1 -- 172.21.15.73:0/4144876966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009400 con 0x7f964c160620 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f964a800640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f964400c7a0 tx=0x7f964400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400a640 con 0x7f964c160620 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964400a7a0 con 0x7f964c160620 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400ab90 con 0x7f964c160620 2024-04-05T22:29:08.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c1f2c40 con 0x7f964c160620 2024-04-05T22:29:08.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.939+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1f3200 con 0x7f964c160620 2024-04-05T22:29:08.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.940+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c066af0 con 0x7f964c160620 2024-04-05T22:29:08.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.941+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f964400acf0 con 0x7f964c160620 2024-04-05T22:29:08.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.942+0000 7f9642a00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 0x7f962806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:08.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.942+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9644089400 con 0x7f964c160620 2024-04-05T22:29:08.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.943+0000 7f9649e00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 0x7f962806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.944+0000 7f9649e00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 0x7f962806d660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f963c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:08.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:08.947+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9644053cf0 con 0x7f964c160620 2024-04-05T22:29:09.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.155+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f964c0caf70 con 0x7f964c160620 2024-04-05T22:29:09.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.156+0000 7f9642a00640 1 -- 172.21.15.73:0/4144876966 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9644018080 con 0x7f964c160620 2024-04-05T22:29:09.156 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:29:09.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.158+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 msgr2=0x7f962806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.158+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 0x7f962806d660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f963c008040 comp rx=0 tx=0).stop 2024-04-05T22:29:09.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 msgr2=0x7f964c157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f964400c7a0 tx=0x7f964400cc60 comp rx=0 tx=0).stop 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 shutdown_connections 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f962806b1a0 0x7f962806d660 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c160620 0x7f964c157960 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964c0b1870 0x7f964c14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 --2- 172.21.15.73:0/4144876966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f964c0ad6b0 0x7f964c1f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 >> 172.21.15.73:0/4144876966 conn(0x7f964c0ce8e0 msgr2=0x7f964c06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 shutdown_connections 2024-04-05T22:29:09.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.159+0000 7f9650c00640 1 -- 172.21.15.73:0/4144876966 wait complete. 2024-04-05T22:29:09.177 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:29:09.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:09.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:29:09.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:29:09.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:09.280 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:29:09.327 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:09.327 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:09.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:29:09.384 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:09.384 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:09.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:29:09.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:09.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:29:09.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:29:09.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.647+0000 7f3527a00640 1 Processor -- start 2024-04-05T22:29:09.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.647+0000 7f3527a00640 1 -- start start 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.648+0000 7f3527a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f352015cf50 0x7f352006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.648+0000 7f3527a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f35200cd7a0 0x7f35200d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.648+0000 7f3527a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200cb950 con 0x7f35200d09b0 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.648+0000 7f3527a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200ce990 con 0x7f35200cd7a0 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.648+0000 7f3527a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200d1350 con 0x7f352015cf50 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f35200cd7a0 0x7f35200d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f35200cd7a0 0x7f35200d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51532/0 (socket says 172.21.15.73:51532) 2024-04-05T22:29:09.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 -- 172.21.15.73:0/1287651046 learned_addr learned my addr 172.21.15.73:0/1287651046 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:09.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 -- 172.21.15.73:0/1287651046 >> v2:172.21.15.157:3300/0 conn(0x7f352015cf50 msgr2=0x7f352006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:09.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 --2- 172.21.15.73:0/1287651046 >> v2:172.21.15.157:3300/0 conn(0x7f352015cf50 0x7f352006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 -- 172.21.15.73:0/1287651046 >> v1:172.21.15.157:6789/0 conn(0x7f35200d09b0 legacy=0x7f35200cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:09.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.649+0000 7f3525a00640 1 -- 172.21.15.73:0/1287651046 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200cdce0 con 0x7f35200cd7a0 2024-04-05T22:29:09.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.650+0000 7f3525a00640 1 --2- 172.21.15.73:0/1287651046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f35200d0e10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f35200ce590 tx=0x7f351802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704453a4ea8f5819 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:09.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f351f400640 1 -- 172.21.15.73:0/1287651046 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351802ca80 con 0x7f35200cd7a0 2024-04-05T22:29:09.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f351f400640 1 -- 172.21.15.73:0/1287651046 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351802cbe0 con 0x7f35200cd7a0 2024-04-05T22:29:09.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f351f400640 1 -- 172.21.15.73:0/1287651046 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180343f0 con 0x7f35200cd7a0 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 -- 172.21.15.73:0/1287651046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 msgr2=0x7f35200d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 --2- 172.21.15.73:0/1287651046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f35200d0e10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f35200ce590 tx=0x7f351802bee0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 -- 172.21.15.73:0/1287651046 shutdown_connections 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 --2- 172.21.15.73:0/1287651046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f35200d0e10 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 --2- 172.21.15.73:0/1287651046 >> v2:172.21.15.157:3300/0 conn(0x7f352015cf50 0x7f352006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 -- 172.21.15.73:0/1287651046 >> 172.21.15.73:0/1287651046 conn(0x7f3520099a60 msgr2=0x7f3520099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 -- 172.21.15.73:0/1287651046 shutdown_connections 2024-04-05T22:29:09.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.651+0000 7f3527a00640 1 -- 172.21.15.73:0/1287651046 wait complete. 2024-04-05T22:29:09.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.653+0000 7f3527a00640 1 Processor -- start 2024-04-05T22:29:09.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.653+0000 7f3527a00640 1 -- start start 2024-04-05T22:29:09.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35200d09b0 0x7f35200b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352015cf50 0x7f35200b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b4cd0 con 0x7f35200cd7a0 2024-04-05T22:29:09.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b3c30 con 0x7f35200d09b0 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3527a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b3f10 con 0x7f352015cf50 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3525000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3525000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51534/0 (socket says 172.21.15.73:51534) 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.654+0000 7f3525000640 1 -- 172.21.15.73:0/2573467739 learned_addr learned my addr 172.21.15.73:0/2573467739 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f351fe00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35200d09b0 0x7f35200b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352015cf50 0x7f35200b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 -- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352015cf50 msgr2=0x7f35200b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352015cf50 0x7f35200b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 -- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35200d09b0 msgr2=0x7f35200b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35200d09b0 0x7f35200b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 -- 172.21.15.73:0/2573467739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009b30 con 0x7f35200cd7a0 2024-04-05T22:29:09.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.655+0000 7f3525000640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f351400b410 tx=0x7f351400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:09.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.656+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514011830 con 0x7f35200cd7a0 2024-04-05T22:29:09.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.656+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35140119b0 con 0x7f35200cd7a0 2024-04-05T22:29:09.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.656+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35200b41f0 con 0x7f35200cd7a0 2024-04-05T22:29:09.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.656+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514011d60 con 0x7f35200cd7a0 2024-04-05T22:29:09.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.656+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35200b47b0 con 0x7f35200cd7a0 2024-04-05T22:29:09.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.657+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520068160 con 0x7f35200cd7a0 2024-04-05T22:29:09.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.658+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f351401a400 con 0x7f35200cd7a0 2024-04-05T22:29:09.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.659+0000 7f351cc00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 0x7f350006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:09.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.659+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3514089ff0 con 0x7f35200cd7a0 2024-04-05T22:29:09.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.659+0000 7f351fe00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 0x7f350006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:09.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.660+0000 7f351fe00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 0x7f350006d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f352015c6f0 tx=0x7f350c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:09.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.664+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f35140548e0 con 0x7f35200cd7a0 2024-04-05T22:29:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.898+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35200cf2d0 con 0x7f35200cd7a0 2024-04-05T22:29:09.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.900+0000 7f351cc00640 1 -- 172.21.15.73:0/2573467739 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f3514015020 con 0x7f35200cd7a0 2024-04-05T22:29:09.900 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:09.900 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:29:09.901 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 74 2024-04-05T22:29:09.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 msgr2=0x7f350006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 0x7f350006d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f352015c6f0 tx=0x7f350c009040 comp rx=0 tx=0).stop 2024-04-05T22:29:09.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 msgr2=0x7f352015ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:09.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f351400b410 tx=0x7f351400b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 shutdown_connections 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f350006b1a0 0x7f350006d660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352015cf50 0x7f35200b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35200d09b0 0x7f35200b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 --2- 172.21.15.73:0/2573467739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35200cd7a0 0x7f352015ba40 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 >> 172.21.15.73:0/2573467739 conn(0x7f3520099a60 msgr2=0x7f352006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 shutdown_connections 2024-04-05T22:29:09.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:09.903+0000 7f3527a00640 1 -- 172.21.15.73:0/2573467739 wait complete. 2024-04-05T22:29:09.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.116+0000 7fd7f5200640 1 Processor -- start 2024-04-05T22:29:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.116+0000 7fd7f5200640 1 -- start start 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7f5200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7f01587d0 0x7fd7f00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7f5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7f015e850 0x7fd7f015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7f5200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f006f5b0 con 0x7fd7f00ccd20 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7f5200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f015f240 con 0x7fd7f015e850 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7f5200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f0070410 con 0x7fd7f01587d0 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7ed400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd7f00ccd20 0x7fd7f006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41256/0 (socket says 172.21.15.73:41256) 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7ed400640 1 -- 172.21.15.73:0/1579243840 learned_addr learned my addr 172.21.15.73:0/1579243840 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7ee800640 1 --2- 172.21.15.73:0/1579243840 >> v2:172.21.15.73:3300/0 conn(0x7fd7f015e850 0x7fd7f015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7ede00640 1 --2- 172.21.15.73:0/1579243840 >> v2:172.21.15.157:3300/0 conn(0x7fd7f01587d0 0x7fd7f00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.118+0000 7fd7eca00640 1 -- 172.21.15.73:0/1579243840 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293202322 0 0) 0x7fd7f006f5b0 con 0x7fd7f00ccd20 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.119+0000 7fd7eca00640 1 -- 172.21.15.73:0/1579243840 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d4003520 con 0x7fd7f00ccd20 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.119+0000 7fd7ee800640 1 -- 172.21.15.73:0/1579243840 >> v2:172.21.15.157:3300/0 conn(0x7fd7f01587d0 msgr2=0x7fd7f00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.119+0000 7fd7ee800640 1 --2- 172.21.15.73:0/1579243840 >> v2:172.21.15.157:3300/0 conn(0x7fd7f01587d0 0x7fd7f00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.119+0000 7fd7ee800640 1 -- 172.21.15.73:0/1579243840 >> v1:172.21.15.73:6789/0 conn(0x7fd7f00ccd20 legacy=0x7fd7f006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.119+0000 7fd7ee800640 1 -- 172.21.15.73:0/1579243840 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f0158bb0 con 0x7fd7f015e850 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.120+0000 7fd7ee800640 1 --2- 172.21.15.73:0/1579243840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015ed00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009d90 tx=0x7fd7e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f2c389a83f3cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.120+0000 7fd7eca00640 1 -- 172.21.15.73:0/1579243840 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e802cc60 con 0x7fd7f015e850 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.120+0000 7fd7eca00640 1 -- 172.21.15.73:0/1579243840 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e802cdc0 con 0x7fd7f015e850 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.120+0000 7fd7eca00640 1 -- 172.21.15.73:0/1579243840 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8034660 con 0x7fd7f015e850 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 -- 172.21.15.73:0/1579243840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 msgr2=0x7fd7f015ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 --2- 172.21.15.73:0/1579243840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015ed00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009d90 tx=0x7fd7e802beb0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 -- 172.21.15.73:0/1579243840 shutdown_connections 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 --2- 172.21.15.73:0/1579243840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015ed00 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 --2- 172.21.15.73:0/1579243840 >> v2:172.21.15.157:3300/0 conn(0x7fd7f01587d0 0x7fd7f00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 -- 172.21.15.73:0/1579243840 >> 172.21.15.73:0/1579243840 conn(0x7fd7f00ce8e0 msgr2=0x7fd7f00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 -- 172.21.15.73:0/1579243840 shutdown_connections 2024-04-05T22:29:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.121+0000 7fd7f5200640 1 -- 172.21.15.73:0/1579243840 wait complete. 2024-04-05T22:29:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.123+0000 7fd7f5200640 1 Processor -- start 2024-04-05T22:29:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.123+0000 7fd7f5200640 1 -- start start 2024-04-05T22:29:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7f01587d0 0x7fd7f013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f015d920 0x7fd7f00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c4ec0 con 0x7fd7f015e850 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f015ddd0 con 0x7fd7f015d920 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7f5200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c59d0 con 0x7fd7f01587d0 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51544/0 (socket says 172.21.15.73:51544) 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 -- 172.21.15.73:0/30257121 learned_addr learned my addr 172.21.15.73:0/30257121 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ede00640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7f01587d0 0x7fd7f013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ee800640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f015d920 0x7fd7f00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 -- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7f01587d0 msgr2=0x7fd7f013e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7f01587d0 0x7fd7f013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 -- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f015d920 msgr2=0x7fd7f00af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f015d920 0x7fd7f00af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.124+0000 7fd7ed400640 1 -- 172.21.15.73:0/30257121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009a70 con 0x7fd7f015e850 2024-04-05T22:29:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7ed400640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00c500 tx=0x7fd7dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc005ce0 con 0x7fd7f015e850 2024-04-05T22:29:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc005e40 con 0x7fd7f015e850 2024-04-05T22:29:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc007330 con 0x7fd7f015e850 2024-04-05T22:29:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00c5e60 con 0x7fd7f015e850 2024-04-05T22:29:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.125+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00c5380 con 0x7fd7f015e850 2024-04-05T22:29:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.126+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0066af0 con 0x7fd7f015e850 2024-04-05T22:29:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.127+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fd7dc007550 con 0x7fd7f015e850 2024-04-05T22:29:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.128+0000 7fd7e6000640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 0x7fd7d006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.128+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd7dc0893d0 con 0x7fd7f015e850 2024-04-05T22:29:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.128+0000 7fd7ede00640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 0x7fd7d006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.130+0000 7fd7ede00640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 0x7fd7d006d660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.132+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd7dc053e50 con 0x7fd7f015e850 2024-04-05T22:29:10.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.326+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f0068160 con 0x7fd7f015e850 2024-04-05T22:29:10.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.329+0000 7fd7e6000640 1 -- 172.21.15.73:0/30257121 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16222 (secure 0 0 0) 0x7fd7dc057470 con 0x7fd7f015e850 2024-04-05T22:29:10.329 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:10.329 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":123,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:04.441444+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 msgr2=0x7fd7d006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 0x7fd7d006d660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 msgr2=0x7fd7f015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00c500 tx=0x7fd7dc00c9c0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 shutdown_connections 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd7d006b1a0 0x7fd7d006d660 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f015d920 0x7fd7f00af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f015e850 0x7fd7f015d3e0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 --2- 172.21.15.73:0/30257121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7f01587d0 0x7fd7f013e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.331+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 >> 172.21.15.73:0/30257121 conn(0x7fd7f00ce8e0 msgr2=0x7fd7f0069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:10.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.332+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 shutdown_connections 2024-04-05T22:29:10.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.332+0000 7fd7f5200640 1 -- 172.21.15.73:0/30257121 wait complete. 2024-04-05T22:29:10.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:28:42.000556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:28:43.153067+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:10.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:10.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.559+0000 7f6310a00640 1 Processor -- start 2024-04-05T22:29:10.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.560+0000 7f6310a00640 1 -- start start 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6310a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f630c15abe0 0x7f630c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6310a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6310a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c15afc0 con 0x7f630c06c9a0 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6310a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c06f5b0 con 0x7f630c15abe0 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6310a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c06f890 con 0x7f630c06bc40 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.561+0000 7f6309e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f630c15abe0 0x7f630c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f630c15abe0 0x7f630c0712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51550/0 (socket says 172.21.15.73:51550) 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 -- 172.21.15.73:0/2809025013 learned_addr learned my addr 172.21.15.73:0/2809025013 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f630a800640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 -- 172.21.15.73:0/2809025013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 msgr2=0x7f630c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 -- 172.21.15.73:0/2809025013 >> v1:172.21.15.157:6789/0 conn(0x7f630c06c9a0 legacy=0x7f630c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f6309e00640 1 -- 172.21.15.73:0/2809025013 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c06fb70 con 0x7f630c15abe0 2024-04-05T22:29:10.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.562+0000 7f630a800640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:10.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.563+0000 7f6309e00640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c0712e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009870 tx=0x7f62fc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e173a99d5b818a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6308a00640 1 -- 172.21.15.73:0/2809025013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc031740 con 0x7f630c15abe0 2024-04-05T22:29:10.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6308a00640 1 -- 172.21.15.73:0/2809025013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0318a0 con 0x7f630c15abe0 2024-04-05T22:29:10.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6308a00640 1 -- 172.21.15.73:0/2809025013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc031c30 con 0x7f630c15abe0 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 -- 172.21.15.73:0/2809025013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 msgr2=0x7f630c0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c0712e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009870 tx=0x7f62fc02b7e0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 -- 172.21.15.73:0/2809025013 shutdown_connections 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 --2- 172.21.15.73:0/2809025013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c0712e0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 -- 172.21.15.73:0/2809025013 >> 172.21.15.73:0/2809025013 conn(0x7f630c069cb0 msgr2=0x7f630c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.564+0000 7f6310a00640 1 -- 172.21.15.73:0/2809025013 shutdown_connections 2024-04-05T22:29:10.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.565+0000 7f6310a00640 1 -- 172.21.15.73:0/2809025013 wait complete. 2024-04-05T22:29:10.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.566+0000 7f6310a00640 1 Processor -- start 2024-04-05T22:29:10.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.566+0000 7f6310a00640 1 -- start start 2024-04-05T22:29:10.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.567+0000 7f6310a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.567+0000 7f6310a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c06c9a0 0x7f630c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6310a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6310a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c157e10 con 0x7f630c15abe0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6310a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c1571d0 con 0x7f630c06c9a0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6310a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c157470 con 0x7f630c06bc40 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6309e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6309400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c06c9a0 0x7f630c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6309e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:54856/0 (socket says 172.21.15.73:54856) 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f6309e00640 1 -- 172.21.15.73:0/3304872989 learned_addr learned my addr 172.21.15.73:0/3304872989 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 -- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 msgr2=0x7f630c1495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c1495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 -- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c06c9a0 msgr2=0x7f630c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c06c9a0 0x7f630c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.568+0000 7f630a800640 1 -- 172.21.15.73:0/3304872989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0094d0 con 0x7f630c15abe0 2024-04-05T22:29:10.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f630a800640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c158770 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f630400c5c0 tx=0x7f630400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304011800 con 0x7f630c15abe0 2024-04-05T22:29:10.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304011960 con 0x7f630c15abe0 2024-04-05T22:29:10.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c157750 con 0x7f630c15abe0 2024-04-05T22:29:10.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304011d50 con 0x7f630c15abe0 2024-04-05T22:29:10.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.569+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c149ae0 con 0x7f630c15abe0 2024-04-05T22:29:10.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.570+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c157f50 con 0x7f630c15abe0 2024-04-05T22:29:10.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.571+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f6304022350 con 0x7f630c15abe0 2024-04-05T22:29:10.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.572+0000 7f6302000640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 0x7f62e806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.572+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6304089fa0 con 0x7f630c15abe0 2024-04-05T22:29:10.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.573+0000 7f6309e00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 0x7f62e806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.574+0000 7f6309e00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 0x7f62e806d660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0090b0 tx=0x7f62fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.577+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f630c157f50 con 0x7f630c15abe0 2024-04-05T22:29:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.757+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f630c157f50 con 0x7f630c15abe0 2024-04-05T22:29:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.758+0000 7f6302000640 1 -- 172.21.15.73:0/3304872989 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f630c157f50 con 0x7f630c15abe0 2024-04-05T22:29:10.758 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 msgr2=0x7f62e806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 0x7f62e806d660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0090b0 tx=0x7f62fc008030 comp rx=0 tx=0).stop 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 msgr2=0x7f630c158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c158770 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f630400c5c0 tx=0x7f630400ca80 comp rx=0 tx=0).stop 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 shutdown_connections 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f62e806b1a0 0x7f62e806d660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f630c15abe0 0x7f630c158770 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c06c9a0 0x7f630c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 --2- 172.21.15.73:0/3304872989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f630c06bc40 0x7f630c1495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.760+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 >> 172.21.15.73:0/3304872989 conn(0x7f630c069cb0 msgr2=0x7f630c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:10.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.761+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 shutdown_connections 2024-04-05T22:29:10.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.761+0000 7f6310a00640 1 -- 172.21.15.73:0/3304872989 wait complete. 2024-04-05T22:29:10.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:29:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.984+0000 7f4aaac00640 1 Processor -- start 2024-04-05T22:29:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.984+0000 7f4aaac00640 1 -- start start 2024-04-05T22:29:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.986+0000 7f4aaac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 0x7f4aa40ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.986+0000 7f4aaac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aa4157410 0x7f4aa4160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.986+0000 7f4aaac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa406d800 con 0x7f4aa40cb0b0 2024-04-05T22:29:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.986+0000 7f4aaac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa40b3090 con 0x7f4aa4157410 2024-04-05T22:29:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.986+0000 7f4aaac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40b3370 con 0x7f4aa4157030 2024-04-05T22:29:10.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aa4157410 0x7f4aa4160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 0x7f4aa40ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aa4157410 0x7f4aa4160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:51030/0 (socket says 172.21.15.73:51030) 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 -- 172.21.15.73:0/2104493058 learned_addr learned my addr 172.21.15.73:0/2104493058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 -- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 msgr2=0x7f4aa40ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 0x7f4aa40ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 -- 172.21.15.73:0/2104493058 >> v1:172.21.15.157:6789/0 conn(0x7f4aa40cb0b0 legacy=0x7f4aa415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa8e00640 1 -- 172.21.15.73:0/2104493058 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40b3650 con 0x7f4aa4157410 2024-04-05T22:29:10.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.987+0000 7f4aa9800640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 0x7f4aa40ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.989+0000 7f4aa8e00640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa4157410 0x7f4aa4160790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c014598c6fb7dbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.989+0000 7f4aa3e00640 1 -- 172.21.15.73:0/2104493058 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90030840 con 0x7f4aa4157410 2024-04-05T22:29:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aa3e00640 1 -- 172.21.15.73:0/2104493058 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a900309a0 con 0x7f4aa4157410 2024-04-05T22:29:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aa3e00640 1 -- 172.21.15.73:0/2104493058 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90030d30 con 0x7f4aa4157410 2024-04-05T22:29:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 -- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa4157410 msgr2=0x7f4aa4160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa4157410 0x7f4aa4160790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002b980 comp rx=0 tx=0).stop 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 -- 172.21.15.73:0/2104493058 shutdown_connections 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa4157410 0x7f4aa4160790 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 --2- 172.21.15.73:0/2104493058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157030 0x7f4aa40ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 -- 172.21.15.73:0/2104493058 >> 172.21.15.73:0/2104493058 conn(0x7f4aa4069cb0 msgr2=0x7f4aa406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 -- 172.21.15.73:0/2104493058 shutdown_connections 2024-04-05T22:29:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.990+0000 7f4aaac00640 1 -- 172.21.15.73:0/2104493058 wait complete. 2024-04-05T22:29:10.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.992+0000 7f4aaac00640 1 Processor -- start 2024-04-05T22:29:10.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.992+0000 7f4aaac00640 1 -- start start 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.993+0000 7f4aaac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa40cb0b0 0x7f4aa41502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.993+0000 7f4aaac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.993+0000 7f4aaac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 0x7f4aa406b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aaac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41534a0 con 0x7f4aa4157030 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aaac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4154a70 con 0x7f4aa40cb0b0 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aaac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4150830 con 0x7f4aa4157410 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51568/0 (socket says 172.21.15.73:51568) 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 -- 172.21.15.73:0/140544867 learned_addr learned my addr 172.21.15.73:0/140544867 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 -- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 msgr2=0x7f4aa406b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aaa200640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 0x7f4aa406b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa9800640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa40cb0b0 0x7f4aa41502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 0x7f4aa406b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 -- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa40cb0b0 msgr2=0x7f4aa41502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa40cb0b0 0x7f4aa41502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aa8e00640 1 -- 172.21.15.73:0/140544867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009a70 con 0x7f4aa4157030 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.994+0000 7f4aaa200640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 0x7f4aa406b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aa8e00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4a900090b0 tx=0x7f4a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9003e070 con 0x7f4aa4157030 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a90039a10 con 0x7f4aa4157030 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90039de0 con 0x7f4aa4157030 2024-04-05T22:29:10.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a900094d0 con 0x7f4aa4157030 2024-04-05T22:29:10.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.995+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa4154790 con 0x7f4aa4157030 2024-04-05T22:29:10.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.996+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa4154c70 con 0x7f4aa4157030 2024-04-05T22:29:10.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.997+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f4a9004b400 con 0x7f4aa4157030 2024-04-05T22:29:10.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.998+0000 7f4aa1600640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 0x7f4a8406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:10.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.998+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a900ac430 con 0x7f4aa4157030 2024-04-05T22:29:10.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:10.999+0000 7f4aa9800640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 0x7f4a8406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.000+0000 7f4aa9800640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 0x7f4a8406d660 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007580 tx=0x7f4a9400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.003+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4a90076c70 con 0x7f4aa4157030 2024-04-05T22:29:11.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.195+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4aa404e160 con 0x7f4aa4157030 2024-04-05T22:29:11.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.195+0000 7f4aa1600640 1 -- 172.21.15.73:0/140544867 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a9007a290 con 0x7f4aa4157030 2024-04-05T22:29:11.196 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:29:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 msgr2=0x7f4a8406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 0x7f4a8406d660 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007580 tx=0x7f4a9400b040 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 msgr2=0x7f4aa415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4a900090b0 tx=0x7f4a900046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 shutdown_connections 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a8406b1a0 0x7f4a8406d660 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa4157410 0x7f4aa406b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aa4157030 0x7f4aa415c9e0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 --2- 172.21.15.73:0/140544867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa40cb0b0 0x7f4aa41502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 >> 172.21.15.73:0/140544867 conn(0x7f4aa4069cb0 msgr2=0x7f4aa406f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 shutdown_connections 2024-04-05T22:29:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.198+0000 7f4aaac00640 1 -- 172.21.15.73:0/140544867 wait complete. 2024-04-05T22:29:11.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:29:11.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.415+0000 7f85d4a00640 1 Processor -- start 2024-04-05T22:29:11.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.415+0000 7f85d4a00640 1 -- start start 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.416+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85d00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85d4a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00b29f0 con 0x7f85d0070ee0 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85d4a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d0161700 con 0x7f85d01459d0 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85d4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0160e60 con 0x7f85d01455f0 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85cf200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f85d0070ee0 0x7f85d00b3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41264/0 (socket says 172.21.15.73:41264) 2024-04-05T22:29:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85cf200640 1 -- 172.21.15.73:0/491542917 learned_addr learned my addr 172.21.15.73:0/491542917 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:11.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85ce800640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85d00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543866845 0 0) 0x7f85d00b29f0 con 0x7f85d0070ee0 2024-04-05T22:29:11.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.417+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b4003520 con 0x7f85d0070ee0 2024-04-05T22:29:11.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356109279 0 0) 0x7f85b4003520 con 0x7f85d0070ee0 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 msgr2=0x7f85d00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85d00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 >> v1:172.21.15.157:6789/0 conn(0x7f85d01459d0 legacy=0x7f85d00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0161140 con 0x7f85d0070ee0 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f85c80019f0 con 0x7f85d0070ee0 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 >> v1:172.21.15.73:6789/0 conn(0x7f85d0070ee0 legacy=0x7f85d00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85b40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b4003f20 0x7f85b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b4004d00 0x7f85b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0160e60 con 0x7f85b4003f20 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b400ba60 con 0x7f85b4004d00 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b400bc20 con 0x7f85d01455f0 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.418+0000 7f85c7e00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b4003f20 0x7f85b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85ce800640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85b40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85cf200640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b4004d00 0x7f85b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85c7e00640 1 -- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 msgr2=0x7f85b40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85c7e00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85d01455f0 0x7f85b40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85c7e00640 1 -- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b4004d00 msgr2=0x7f85b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85c7e00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b4004d00 0x7f85b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85c7e00640 1 -- 172.21.15.73:0/491542917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4003520 con 0x7f85b4003f20 2024-04-05T22:29:11.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.419+0000 7f85cf200640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b4004d00 0x7f85b4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:11.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.420+0000 7f85c7e00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b4003f20 0x7f85b40047c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001750 tx=0x7f85b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2addaf8454e63912 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.420+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80052d0 con 0x7f85b4003f20 2024-04-05T22:29:11.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.420+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b802e050 con 0x7f85b4003f20 2024-04-05T22:29:11.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.420+0000 7f85cde00640 1 -- 172.21.15.73:0/491542917 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8003830 con 0x7f85b4003f20 2024-04-05T22:29:11.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 -- 172.21.15.73:0/491542917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b4003f20 msgr2=0x7f85b40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 --2- 172.21.15.73:0/491542917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b4003f20 0x7f85b40047c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001750 tx=0x7f85b40057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85cf200640 1 -- 172.21.15.73:0/491542917 reap_dead start 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 -- 172.21.15.73:0/491542917 shutdown_connections 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 -- 172.21.15.73:0/491542917 >> 172.21.15.73:0/491542917 conn(0x7f85d00b0830 msgr2=0x7f85d00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 -- 172.21.15.73:0/491542917 shutdown_connections 2024-04-05T22:29:11.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.421+0000 7f85d4a00640 1 -- 172.21.15.73:0/491542917 wait complete. 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.423+0000 7f85d4a00640 1 Processor -- start 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.423+0000 7f85d4a00640 1 -- start start 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b4003f20 0x7f85d00ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85d013b330 0x7f85d013b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015f450 con 0x7f85d0138ec0 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015f730 con 0x7f85d013b330 2024-04-05T22:29:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85d4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015fa10 con 0x7f85b4003f20 2024-04-05T22:29:11.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51582/0 (socket says 172.21.15.73:51582) 2024-04-05T22:29:11.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 -- 172.21.15.73:0/1800432344 learned_addr learned my addr 172.21.15.73:0/1800432344 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 -- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b4003f20 msgr2=0x7f85d00ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b4003f20 0x7f85d00ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 -- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85d013b330 msgr2=0x7f85d013b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85d013b330 0x7f85d013b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.424+0000 7f85cf200640 1 -- 172.21.15.73:0/1800432344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d015fcf0 con 0x7f85d0138ec0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85cf200640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000f80 tx=0x7f85c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8004d60 con 0x7f85d0138ec0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c800bb80 con 0x7f85d0138ec0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8007070 con 0x7f85d0138ec0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d015ffd0 con 0x7f85d0138ec0 2024-04-05T22:29:11.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.425+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d00ad590 con 0x7f85d0138ec0 2024-04-05T22:29:11.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.427+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d00712e0 con 0x7f85d0138ec0 2024-04-05T22:29:11.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.428+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f85c80071d0 con 0x7f85d0138ec0 2024-04-05T22:29:11.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.428+0000 7f85c6a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 0x7f85ac06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.429+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f85c808a2c0 con 0x7f85d0138ec0 2024-04-05T22:29:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.429+0000 7f85ce800640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 0x7f85ac06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.430+0000 7f85ce800640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 0x7f85ac06d690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009910 tx=0x7f85c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.433+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f85c8054b00 con 0x7f85d0138ec0 2024-04-05T22:29:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.626+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85d00ad780 con 0x7f85d0138ec0 2024-04-05T22:29:11.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.627+0000 7f85c6a00640 1 -- 172.21.15.73:0/1800432344 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85c8058120 con 0x7f85d0138ec0 2024-04-05T22:29:11.627 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:29:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 msgr2=0x7f85ac06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 0x7f85ac06d690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009910 tx=0x7f85c0008040 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 msgr2=0x7f85d0139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000f80 tx=0x7f85c800b4a0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 shutdown_connections 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f85ac06b1d0 0x7f85ac06d690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d0138ec0 0x7f85d0139340 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85d013b330 0x7f85d013b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 --2- 172.21.15.73:0/1800432344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b4003f20 0x7f85d00ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 >> 172.21.15.73:0/1800432344 conn(0x7f85d00b0830 msgr2=0x7f85d00ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 shutdown_connections 2024-04-05T22:29:11.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.629+0000 7f85d4a00640 1 -- 172.21.15.73:0/1800432344 wait complete. 2024-04-05T22:29:11.649 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:11.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.853+0000 7fa3b7c00640 1 Processor -- start 2024-04-05T22:29:11.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.854+0000 7fa3b7c00640 1 -- start start 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b7c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3b01587d0 0x7fa3b00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b7c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3b015e850 0x7fa3b015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b7c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b006f5b0 con 0x7fa3b00ccd20 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b7c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b015f240 con 0x7fa3b01587d0 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b7c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b0070410 con 0x7fa3b015e850 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3b01587d0 0x7fa3b00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3b01587d0 0x7fa3b00cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51588/0 (socket says 172.21.15.73:51588) 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.855+0000 7fa3b5200640 1 -- 172.21.15.73:0/3576146958 learned_addr learned my addr 172.21.15.73:0/3576146958 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.856+0000 7fa3b5200640 1 -- 172.21.15.73:0/3576146958 >> v2:172.21.15.157:3300/0 conn(0x7fa3b015e850 msgr2=0x7fa3b015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.856+0000 7fa3b5200640 1 --2- 172.21.15.73:0/3576146958 >> v2:172.21.15.157:3300/0 conn(0x7fa3b015e850 0x7fa3b015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.856+0000 7fa3b5200640 1 -- 172.21.15.73:0/3576146958 >> v1:172.21.15.157:6789/0 conn(0x7fa3b00ccd20 legacy=0x7fa3b006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:11.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.856+0000 7fa3b5200640 1 -- 172.21.15.73:0/3576146958 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0158bb0 con 0x7fa3b01587d0 2024-04-05T22:29:11.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.857+0000 7fa3b5200640 1 --2- 172.21.15.73:0/3576146958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b01587d0 0x7fa3b00cc7e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbd16ebb163d3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.857+0000 7fa3af400640 1 -- 172.21.15.73:0/3576146958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4031860 con 0x7fa3b01587d0 2024-04-05T22:29:11.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.857+0000 7fa3af400640 1 -- 172.21.15.73:0/3576146958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a40319c0 con 0x7fa3b01587d0 2024-04-05T22:29:11.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.857+0000 7fa3af400640 1 -- 172.21.15.73:0/3576146958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4031d50 con 0x7fa3b01587d0 2024-04-05T22:29:11.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 -- 172.21.15.73:0/3576146958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b01587d0 msgr2=0x7fa3b00cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/3576146958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b01587d0 0x7fa3b00cc7e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402b900 comp rx=0 tx=0).stop 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 -- 172.21.15.73:0/3576146958 shutdown_connections 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/3576146958 >> v2:172.21.15.157:3300/0 conn(0x7fa3b015e850 0x7fa3b015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/3576146958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b01587d0 0x7fa3b00cc7e0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 -- 172.21.15.73:0/3576146958 >> 172.21.15.73:0/3576146958 conn(0x7fa3b00ce8e0 msgr2=0x7fa3b00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 -- 172.21.15.73:0/3576146958 shutdown_connections 2024-04-05T22:29:11.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.858+0000 7fa3b7c00640 1 -- 172.21.15.73:0/3576146958 wait complete. 2024-04-05T22:29:11.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.860+0000 7fa3b7c00640 1 Processor -- start 2024-04-05T22:29:11.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.860+0000 7fa3b7c00640 1 -- start start 2024-04-05T22:29:11.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3b01587d0 0x7fa3b0155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b015d3e0 0x7fa3b015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b015ddd0 con 0x7fa3b015e850 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00af490 con 0x7fa3b015d3e0 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3b7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0156340 con 0x7fa3b01587d0 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51598/0 (socket says 172.21.15.73:51598) 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.861+0000 7fa3afe00640 1 -- 172.21.15.73:0/176733888 learned_addr learned my addr 172.21.15.73:0/176733888 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 -- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3b01587d0 msgr2=0x7fa3b0155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3b01587d0 0x7fa3b0155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 -- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b015d3e0 msgr2=0x7fa3b015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b015d3e0 0x7fa3b015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 -- 172.21.15.73:0/176733888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40094d0 con 0x7fa3b015e850 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3afe00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c440 tx=0x7fa39c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c005480 con 0x7fa3b015e850 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0035f0 con 0x7fa3b015e850 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0039c0 con 0x7fa3b015e850 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b0154d60 con 0x7fa3b015e850 2024-04-05T22:29:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.862+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b0155320 con 0x7fa3b015e850 2024-04-05T22:29:11.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.863+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b0066af0 con 0x7fa3b015e850 2024-04-05T22:29:11.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.864+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa39c003b20 con 0x7fa3b015e850 2024-04-05T22:29:11.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.865+0000 7fa3acc00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 0x7fa39006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:11.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.865+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa39c0890b0 con 0x7fa3b015e850 2024-04-05T22:29:11.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.866+0000 7fa3b5200640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 0x7fa39006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:11.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.867+0000 7fa3b5200640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 0x7fa39006d660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4036000 tx=0x7fa3a4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:11.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:11.869+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa39c053930 con 0x7fa3b015e850 2024-04-05T22:29:12.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.084+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b0068160 con 0x7fa3b015e850 2024-04-05T22:29:12.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.084+0000 7fa3acc00640 1 -- 172.21.15.73:0/176733888 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fa39c017070 con 0x7fa3b015e850 2024-04-05T22:29:12.085 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:12.086 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:29:12.086 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 74 2024-04-05T22:29:12.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 msgr2=0x7fa39006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 0x7fa39006d660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4036000 tx=0x7fa3a4030000 comp rx=0 tx=0).stop 2024-04-05T22:29:12.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 msgr2=0x7fa3b0150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c440 tx=0x7fa39c00c900 comp rx=0 tx=0).stop 2024-04-05T22:29:12.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 shutdown_connections 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa39006b1a0 0x7fa39006d660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b015d3e0 0x7fa3b015d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b015e850 0x7fa3b0150370 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 --2- 172.21.15.73:0/176733888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3b01587d0 0x7fa3b0155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 >> 172.21.15.73:0/176733888 conn(0x7fa3b00ce8e0 msgr2=0x7fa3b00ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 shutdown_connections 2024-04-05T22:29:12.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.087+0000 7fa3b7c00640 1 -- 172.21.15.73:0/176733888 wait complete. 2024-04-05T22:29:12.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:12.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.305+0000 7fef0d000640 1 Processor -- start 2024-04-05T22:29:12.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.305+0000 7fef0d000640 1 -- start start 2024-04-05T22:29:12.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef0d000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef0d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 0x7fef081429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef0d000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef08143330 con 0x7fef0806b500 2024-04-05T22:29:12.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef0d000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0806d3a0 con 0x7fef0813fa60 2024-04-05T22:29:12.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef0d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef08143050 con 0x7fef0813fe40 2024-04-05T22:29:12.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef06a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 0x7fef081429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef07400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.307+0000 7fef07400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:51108/0 (socket says 172.21.15.73:51108) 2024-04-05T22:29:12.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef07400640 1 -- 172.21.15.73:0/2307689243 learned_addr learned my addr 172.21.15.73:0/2307689243 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:12.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef06a00640 1 -- 172.21.15.73:0/2307689243 >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 msgr2=0x7fef08142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:12.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef06a00640 1 --2- 172.21.15.73:0/2307689243 >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef06a00640 1 -- 172.21.15.73:0/2307689243 >> v1:172.21.15.157:6789/0 conn(0x7fef0806b500 legacy=0x7fef0806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef06a00640 1 -- 172.21.15.73:0/2307689243 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080ac970 con 0x7fef0813fe40 2024-04-05T22:29:12.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.308+0000 7fef07400640 1 --2- 172.21.15.73:0/2307689243 >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:12.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.310+0000 7fef06a00640 1 --2- 172.21.15.73:0/2307689243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 0x7fef081429d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feef40097a0 tx=0x7feef402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e405cad3e07a97aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.310+0000 7fef06000640 1 -- 172.21.15.73:0/2307689243 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef402d040 con 0x7fef0813fe40 2024-04-05T22:29:12.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.310+0000 7fef06000640 1 -- 172.21.15.73:0/2307689243 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4031810 con 0x7fef0813fe40 2024-04-05T22:29:12.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.310+0000 7fef06000640 1 -- 172.21.15.73:0/2307689243 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4031ba0 con 0x7fef0813fe40 2024-04-05T22:29:12.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 -- 172.21.15.73:0/2307689243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 msgr2=0x7fef081429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 --2- 172.21.15.73:0/2307689243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 0x7fef081429d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feef40097a0 tx=0x7feef402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 -- 172.21.15.73:0/2307689243 shutdown_connections 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 --2- 172.21.15.73:0/2307689243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fe40 0x7fef081429d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 --2- 172.21.15.73:0/2307689243 >> v2:172.21.15.157:3300/0 conn(0x7fef0813fa60 0x7fef08142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 -- 172.21.15.73:0/2307689243 >> 172.21.15.73:0/2307689243 conn(0x7fef080b06c0 msgr2=0x7fef080b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 -- 172.21.15.73:0/2307689243 shutdown_connections 2024-04-05T22:29:12.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.311+0000 7fef0d000640 1 -- 172.21.15.73:0/2307689243 wait complete. 2024-04-05T22:29:12.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.313+0000 7fef0d000640 1 Processor -- start 2024-04-05T22:29:12.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.313+0000 7fef0d000640 1 -- start start 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fa60 0x7fef0813b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0813fe40 0x7fef0813b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef080cdbf0 con 0x7fef0806b500 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0813bec0 con 0x7fef0813fe40 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef0d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0815ff70 con 0x7fef0813fa60 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef07400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51604/0 (socket says 172.21.15.73:51604) 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.314+0000 7fef07400640 1 -- 172.21.15.73:0/3112552915 learned_addr learned my addr 172.21.15.73:0/3112552915 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef06a00640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fa60 0x7fef0813b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07e00640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0813fe40 0x7fef0813b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 -- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fa60 msgr2=0x7fef0813b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fa60 0x7fef0813b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 -- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0813fe40 msgr2=0x7fef0813b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0813fe40 0x7fef0813b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 -- 172.21.15.73:0/3112552915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef80094d0 con 0x7fef0806b500 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7fef07400640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7feef800e9b0 tx=0x7feef800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.315+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef800c9a0 con 0x7fef0806b500 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.316+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef800cb00 con 0x7fef0806b500 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.316+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef800ce70 con 0x7fef0806b500 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.316+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4009400 con 0x7fef0806b500 2024-04-05T22:29:12.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.316+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0815fc90 con 0x7fef0806b500 2024-04-05T22:29:12.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.317+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef080cde60 con 0x7fef0806b500 2024-04-05T22:29:12.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.318+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7feef8013470 con 0x7fef0806b500 2024-04-05T22:29:12.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.319+0000 7feeff400640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 0x7feee806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.319+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7feef808a3d0 con 0x7fef0806b500 2024-04-05T22:29:12.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.319+0000 7fef06a00640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 0x7feee806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.320+0000 7fef06a00640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 0x7feee806d660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feef4036000 tx=0x7feef403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.324+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feef8054ce0 con 0x7fef0806b500 2024-04-05T22:29:12.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.518+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef0804e160 con 0x7fef0806b500 2024-04-05T22:29:12.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.526+0000 7feeff400640 1 -- 172.21.15.73:0/3112552915 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16178 (secure 0 0 0) 0x7feef8058300 con 0x7fef0806b500 2024-04-05T22:29:12.527 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:12.527 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":124,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:10.425535+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:12.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 msgr2=0x7feee806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 0x7feee806d660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feef4036000 tx=0x7feef403c040 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 msgr2=0x7fef080cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7feef800e9b0 tx=0x7feef800ee70 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 shutdown_connections 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feee806b1a0 0x7feee806d660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0813fe40 0x7fef0813b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef0813fa60 0x7fef0813b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 --2- 172.21.15.73:0/3112552915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef0806b500 0x7fef080cce20 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 >> 172.21.15.73:0/3112552915 conn(0x7fef080b06c0 msgr2=0x7fef080ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 shutdown_connections 2024-04-05T22:29:12.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.529+0000 7fef0d000640 1 -- 172.21.15.73:0/3112552915 wait complete. 2024-04-05T22:29:12.547 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:29:12.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.760+0000 7fa276800640 1 Processor -- start 2024-04-05T22:29:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.760+0000 7fa276800640 1 -- start start 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.761+0000 7fa276800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa2701587d0 0x7fa2700cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.761+0000 7fa276800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa27015e850 0x7fa27015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.761+0000 7fa276800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27006f5b0 con 0x7fa2700ccd20 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.761+0000 7fa276800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27015f240 con 0x7fa2701587d0 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.761+0000 7fa276800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa270070410 con 0x7fa27015e850 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa2701587d0 0x7fa2700cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa266a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa2700ccd20 0x7fa27006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41266/0 (socket says 172.21.15.73:41266) 2024-04-05T22:29:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa266a00640 1 -- 172.21.15.73:0/153492115 learned_addr learned my addr 172.21.15.73:0/153492115 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26f400640 1 -- 172.21.15.73:0/153492115 >> v2:172.21.15.157:3300/0 conn(0x7fa27015e850 msgr2=0x7fa27015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26f400640 1 --2- 172.21.15.73:0/153492115 >> v2:172.21.15.157:3300/0 conn(0x7fa27015e850 0x7fa27015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26f400640 1 -- 172.21.15.73:0/153492115 >> v1:172.21.15.73:6789/0 conn(0x7fa2700ccd20 legacy=0x7fa27006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26ea00640 1 -- 172.21.15.73:0/153492115 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188987702 0 0) 0x7fa27006f5b0 con 0x7fa2700ccd20 2024-04-05T22:29:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.762+0000 7fa26f400640 1 -- 172.21.15.73:0/153492115 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270158bb0 con 0x7fa2701587d0 2024-04-05T22:29:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.763+0000 7fa26f400640 1 --2- 172.21.15.73:0/153492115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2701587d0 0x7fa2700cc7e0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa2600097a0 tx=0x7fa26002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef612b8d6b63c715 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa26ea00640 1 -- 172.21.15.73:0/153492115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260031830 con 0x7fa2701587d0 2024-04-05T22:29:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa26ea00640 1 -- 172.21.15.73:0/153492115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260031990 con 0x7fa2701587d0 2024-04-05T22:29:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa26ea00640 1 -- 172.21.15.73:0/153492115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260031d20 con 0x7fa2701587d0 2024-04-05T22:29:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 -- 172.21.15.73:0/153492115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2701587d0 msgr2=0x7fa2700cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 --2- 172.21.15.73:0/153492115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2701587d0 0x7fa2700cc7e0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa2600097a0 tx=0x7fa26002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 -- 172.21.15.73:0/153492115 shutdown_connections 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 --2- 172.21.15.73:0/153492115 >> v2:172.21.15.157:3300/0 conn(0x7fa27015e850 0x7fa27015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 --2- 172.21.15.73:0/153492115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2701587d0 0x7fa2700cc7e0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 -- 172.21.15.73:0/153492115 >> 172.21.15.73:0/153492115 conn(0x7fa2700ce8e0 msgr2=0x7fa2700cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 -- 172.21.15.73:0/153492115 shutdown_connections 2024-04-05T22:29:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.764+0000 7fa276800640 1 -- 172.21.15.73:0/153492115 wait complete. 2024-04-05T22:29:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.766+0000 7fa276800640 1 Processor -- start 2024-04-05T22:29:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.766+0000 7fa276800640 1 -- start start 2024-04-05T22:29:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2701587d0 0x7fa2700af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27015e850 0x7fa2700c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c5850 con 0x7fa2700c4730 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c4bd0 con 0x7fa27015e850 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa276800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c4eb0 con 0x7fa2701587d0 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa26fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa26fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51622/0 (socket says 172.21.15.73:51622) 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.767+0000 7fa26fe00640 1 -- 172.21.15.73:0/3071970175 learned_addr learned my addr 172.21.15.73:0/3071970175 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 -- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2701587d0 msgr2=0x7fa2700af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2701587d0 0x7fa2700af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 -- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27015e850 msgr2=0x7fa2700c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27015e850 0x7fa2700c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 -- 172.21.15.73:0/3071970175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260009400 con 0x7fa2700c4730 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa26fe00640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b970 tx=0x7fa26800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800cb40 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800ccc0 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268014680 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2700c5190 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.768+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2700c5c50 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.769+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa270066af0 con 0x7fa2700c4730 2024-04-05T22:29:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.770+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa26801d3b0 con 0x7fa2700c4730 2024-04-05T22:29:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.771+0000 7fa267e00640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 0x7fa24c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.771+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa268089d80 con 0x7fa2700c4730 2024-04-05T22:29:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.772+0000 7fa26f400640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 0x7fa24c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.773+0000 7fa26f400640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 0x7fa24c06d660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa260036000 tx=0x7fa260030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.776+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa2680546d0 con 0x7fa2700c4730 2024-04-05T22:29:12.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.987+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa270068160 con 0x7fa2700c4730 2024-04-05T22:29:12.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.987+0000 7fa267e00640 1 -- 172.21.15.73:0/3071970175 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fa268019020 con 0x7fa2700c4730 2024-04-05T22:29:12.988 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:12.988 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:29:12.988 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 74 2024-04-05T22:29:12.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 msgr2=0x7fa24c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 0x7fa24c06d660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa260036000 tx=0x7fa260030000 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 msgr2=0x7fa2700b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b970 tx=0x7fa26800be30 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 shutdown_connections 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa24c06b1a0 0x7fa24c06d660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2700c4730 0x7fa2700b1600 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa27015e850 0x7fa2700c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 --2- 172.21.15.73:0/3071970175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2701587d0 0x7fa2700af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 >> 172.21.15.73:0/3071970175 conn(0x7fa2700ce8e0 msgr2=0x7fa27006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 shutdown_connections 2024-04-05T22:29:12.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:12.990+0000 7fa276800640 1 -- 172.21.15.73:0/3071970175 wait complete. 2024-04-05T22:29:13.008 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.215+0000 7fe024a00640 1 Processor -- start 2024-04-05T22:29:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.216+0000 7fe024a00640 1 -- start start 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.217+0000 7fe024a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0200d1600 0x7fe02015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.217+0000 7fe024a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe0200d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.217+0000 7fe024a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200b2420 con 0x7fe02015ccd0 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.217+0000 7fe024a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200b2770 con 0x7fe0200d1600 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.217+0000 7fe024a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200ce380 con 0x7fe0200d35e0 2024-04-05T22:29:13.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.218+0000 7fe01de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0200d1600 0x7fe02015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.218+0000 7fe01de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0200d1600 0x7fe02015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:51140/0 (socket says 172.21.15.73:51140) 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.218+0000 7fe01de00640 1 -- 172.21.15.73:0/1234413448 learned_addr learned my addr 172.21.15.73:0/1234413448 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.218+0000 7fe01f200640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe0200d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.218+0000 7fe01de00640 1 -- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 msgr2=0x7fe0200d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.219+0000 7fe01de00640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe0200d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.219+0000 7fe01de00640 1 -- 172.21.15.73:0/1234413448 >> v1:172.21.15.157:6789/0 conn(0x7fe02015ccd0 legacy=0x7fe0200d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.219+0000 7fe01de00640 1 -- 172.21.15.73:0/1234413448 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200ce660 con 0x7fe0200d1600 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.220+0000 7fe01de00640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200d1600 0x7fe02015d0b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e50 tx=0x7fe00802be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82e8538aba1c9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe01d400640 1 -- 172.21.15.73:0/1234413448 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00802c9d0 con 0x7fe0200d1600 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe01d400640 1 -- 172.21.15.73:0/1234413448 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00802cb30 con 0x7fe0200d1600 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe01d400640 1 -- 172.21.15.73:0/1234413448 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080343f0 con 0x7fe0200d1600 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 -- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200d1600 msgr2=0x7fe02015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200d1600 0x7fe02015d0b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e50 tx=0x7fe00802be30 comp rx=0 tx=0).stop 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 -- 172.21.15.73:0/1234413448 shutdown_connections 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe0200d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 --2- 172.21.15.73:0/1234413448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0200d1600 0x7fe02015d0b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.221+0000 7fe024a00640 1 -- 172.21.15.73:0/1234413448 >> 172.21.15.73:0/1234413448 conn(0x7fe0200d2ac0 msgr2=0x7fe0200d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.222+0000 7fe024a00640 1 -- 172.21.15.73:0/1234413448 shutdown_connections 2024-04-05T22:29:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.222+0000 7fe024a00640 1 -- 172.21.15.73:0/1234413448 wait complete. 2024-04-05T22:29:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.224+0000 7fe024a00640 1 Processor -- start 2024-04-05T22:29:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.224+0000 7fe024a00640 1 -- start start 2024-04-05T22:29:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe02009d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02015ccd0 0x7fe02006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009da50 con 0x7fe0200d1600 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009f900 con 0x7fe02015ccd0 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe024a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009fb40 con 0x7fe0200d35e0 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe01e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe01e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51632/0 (socket says 172.21.15.73:51632) 2024-04-05T22:29:13.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.225+0000 7fe01e800640 1 -- 172.21.15.73:0/3976182415 learned_addr learned my addr 172.21.15.73:0/3976182415 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01de00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe02009d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01f200640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02015ccd0 0x7fe02006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 -- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 msgr2=0x7fe02009d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe02009d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 -- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02015ccd0 msgr2=0x7fe02006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02015ccd0 0x7fe02006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 -- 172.21.15.73:0/3976182415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009b30 con 0x7fe0200d1600 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe01e800640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c430 tx=0x7fe01000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.226+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100052d0 con 0x7fe0200d1600 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.227+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010005430 con 0x7fe0200d1600 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.227+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010003830 con 0x7fe0200d1600 2024-04-05T22:29:13.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.227+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02009fe20 con 0x7fe0200d1600 2024-04-05T22:29:13.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.227+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0200a03e0 con 0x7fe0200d1600 2024-04-05T22:29:13.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.228+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020067250 con 0x7fe0200d1600 2024-04-05T22:29:13.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.229+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fe010003990 con 0x7fe0200d1600 2024-04-05T22:29:13.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.230+0000 7fe016a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 0x7fdffc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.230+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe0100892d0 con 0x7fe0200d1600 2024-04-05T22:29:13.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.230+0000 7fe01de00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 0x7fdffc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.232+0000 7fe01de00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 0x7fdffc06d660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe008035000 tx=0x7fe008030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.234+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe010053c70 con 0x7fe0200d1600 2024-04-05T22:29:13.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.464+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe02004e160 con 0x7fe0200d1600 2024-04-05T22:29:13.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.465+0000 7fe016a00640 1 -- 172.21.15.73:0/3976182415 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fe010057290 con 0x7fe0200d1600 2024-04-05T22:29:13.466 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:13.466 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:28:44.145472+0000","modified":"2024-04-05T22:28:47.182754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8217":{"gid":8217,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/651015208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":651015208},{"type":"v1","addr":"172.21.15.73:6835","nonce":651015208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-05T22:29:13.466 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 74 2024-04-05T22:29:13.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 msgr2=0x7fdffc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 0x7fdffc06d660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe008035000 tx=0x7fe008030000 comp rx=0 tx=0).stop 2024-04-05T22:29:13.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 msgr2=0x7fe020150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c430 tx=0x7fe01000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 shutdown_connections 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdffc06b1a0 0x7fdffc06d660 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02015ccd0 0x7fe02006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0200d35e0 0x7fe02009d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 --2- 172.21.15.73:0/3976182415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0200d1600 0x7fe020150ed0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 >> 172.21.15.73:0/3976182415 conn(0x7fe0200d2ac0 msgr2=0x7fe0200b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 shutdown_connections 2024-04-05T22:29:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.468+0000 7fe024a00640 1 -- 172.21.15.73:0/3976182415 wait complete. 2024-04-05T22:29:13.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:13.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.687+0000 7f9c5d200640 1 Processor -- start 2024-04-05T22:29:13.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.688+0000 7f9c5d200640 1 -- start start 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c5d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c581587d0 0x7f9c580cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c5d200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9c5815e850 0x7f9c5815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c5d200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5806f5b0 con 0x7f9c580ccd20 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c5d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5815f240 con 0x7f9c581587d0 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c5d200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c58070410 con 0x7f9c5815e850 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c55e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c581587d0 0x7f9c580cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c55e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c581587d0 0x7f9c580cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51646/0 (socket says 172.21.15.73:51646) 2024-04-05T22:29:13.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.689+0000 7f9c55e00640 1 -- 172.21.15.73:0/2903762329 learned_addr learned my addr 172.21.15.73:0/2903762329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.690+0000 7f9c55e00640 1 -- 172.21.15.73:0/2903762329 >> v2:172.21.15.157:3300/0 conn(0x7f9c5815e850 msgr2=0x7f9c5815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.690+0000 7f9c55e00640 1 --2- 172.21.15.73:0/2903762329 >> v2:172.21.15.157:3300/0 conn(0x7f9c5815e850 0x7f9c5815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.690+0000 7f9c55e00640 1 -- 172.21.15.73:0/2903762329 >> v1:172.21.15.157:6789/0 conn(0x7f9c580ccd20 legacy=0x7f9c5806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.690+0000 7f9c55e00640 1 -- 172.21.15.73:0/2903762329 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58158bb0 con 0x7f9c581587d0 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.691+0000 7f9c55e00640 1 --2- 172.21.15.73:0/2903762329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c581587d0 0x7f9c580cc7e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009870 tx=0x7f9c4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bc7674a11bef20 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.691+0000 7f9c55400640 1 -- 172.21.15.73:0/2903762329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48031870 con 0x7f9c581587d0 2024-04-05T22:29:13.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.691+0000 7f9c55400640 1 -- 172.21.15.73:0/2903762329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c480319d0 con 0x7f9c581587d0 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.691+0000 7f9c55400640 1 -- 172.21.15.73:0/2903762329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48031d60 con 0x7f9c581587d0 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 -- 172.21.15.73:0/2903762329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c581587d0 msgr2=0x7f9c580cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 --2- 172.21.15.73:0/2903762329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c581587d0 0x7f9c580cc7e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009870 tx=0x7f9c4802b910 comp rx=0 tx=0).stop 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 -- 172.21.15.73:0/2903762329 shutdown_connections 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 --2- 172.21.15.73:0/2903762329 >> v2:172.21.15.157:3300/0 conn(0x7f9c5815e850 0x7f9c5815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 --2- 172.21.15.73:0/2903762329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c581587d0 0x7f9c580cc7e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 -- 172.21.15.73:0/2903762329 >> 172.21.15.73:0/2903762329 conn(0x7f9c580ce8e0 msgr2=0x7f9c580cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 -- 172.21.15.73:0/2903762329 shutdown_connections 2024-04-05T22:29:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.692+0000 7f9c5d200640 1 -- 172.21.15.73:0/2903762329 wait complete. 2024-04-05T22:29:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.694+0000 7f9c5d200640 1 Processor -- start 2024-04-05T22:29:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.694+0000 7f9c5d200640 1 -- start start 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c581587d0 0x7f9c5814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5815e850 0x7f9c580cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580ce410 con 0x7f9c580cdf60 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580af410 con 0x7f9c5815e850 2024-04-05T22:29:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c5d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580aee70 con 0x7f9c581587d0 2024-04-05T22:29:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51660/0 (socket says 172.21.15.73:51660) 2024-04-05T22:29:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 -- 172.21.15.73:0/1707644765 learned_addr learned my addr 172.21.15.73:0/1707644765 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:13.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 -- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c581587d0 msgr2=0x7f9c5814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c4fe00640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5815e850 0x7f9c580cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c581587d0 0x7f9c5814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 -- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5815e850 msgr2=0x7f9c580cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5815e850 0x7f9c580cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.695+0000 7f9c56800640 1 -- 172.21.15.73:0/1707644765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480094d0 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c56800640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000b9a0 tx=0x7f9c5000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000cba0 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5000cd20 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c580aeb90 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50014680 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.696+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c581543b0 con 0x7f9c580cdf60 2024-04-05T22:29:13.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.697+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58066af0 con 0x7f9c580cdf60 2024-04-05T22:29:13.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.698+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f9c5001d3b0 con 0x7f9c580cdf60 2024-04-05T22:29:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.699+0000 7f9c4e000640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 0x7f9c3406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.699+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9c50089db0 con 0x7f9c580cdf60 2024-04-05T22:29:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.700+0000 7f9c55e00640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 0x7f9c3406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.701+0000 7f9c55e00640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 0x7f9c3406d660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9c48036000 tx=0x7f9c48030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.703+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9c50054670 con 0x7f9c580cdf60 2024-04-05T22:29:13.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.882+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c58068160 con 0x7f9c580cdf60 2024-04-05T22:29:13.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.885+0000 7f9c4e000640 1 -- 172.21.15.73:0/1707644765 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16178 (secure 0 0 0) 0x7f9c50019070 con 0x7f9c580cdf60 2024-04-05T22:29:13.886 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:13.886 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":124,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:10.425535+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:28:42.000556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-05T22:28:43.153067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:13.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 msgr2=0x7f9c3406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 0x7f9c3406d660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9c48036000 tx=0x7f9c48030000 comp rx=0 tx=0).stop 2024-04-05T22:29:13.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 msgr2=0x7f9c580adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000b9a0 tx=0x7f9c5000be60 comp rx=0 tx=0).stop 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 shutdown_connections 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9c3406b1a0 0x7f9c3406d660 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c580cdf60 0x7f9c580adf40 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5815e850 0x7f9c580cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 --2- 172.21.15.73:0/1707644765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c581587d0 0x7f9c5814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:13.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.887+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 >> 172.21.15.73:0/1707644765 conn(0x7f9c580ce8e0 msgr2=0x7f9c58159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:13.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.888+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 shutdown_connections 2024-04-05T22:29:13.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:13.888+0000 7f9c5d200640 1 -- 172.21.15.73:0/1707644765 wait complete. 2024-04-05T22:29:13.907 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:29:14.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.117+0000 7fae27200640 1 Processor -- start 2024-04-05T22:29:14.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.117+0000 7fae27200640 1 -- start start 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 0x7fae200cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae200ccd20 0x7fae2006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae2006d0c0 0x7fae2015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2006f5b0 con 0x7fae200ccd20 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae20070410 con 0x7fae201587d0 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae27200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae20158bb0 con 0x7fae2006d0c0 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae17400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae200ccd20 0x7fae2006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae17400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae200ccd20 0x7fae2006cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51662/0 (socket says 172.21.15.73:51662) 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.119+0000 7fae17400640 1 -- 172.21.15.73:0/3430422322 learned_addr learned my addr 172.21.15.73:0/3430422322 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae1fe00640 1 --2- 172.21.15.73:0/3430422322 >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 0x7fae200cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae25200640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae2006d0c0 0x7fae2015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae17400640 1 -- 172.21.15.73:0/3430422322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae2006d0c0 msgr2=0x7fae2015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae17400640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae2006d0c0 0x7fae2015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae17400640 1 -- 172.21.15.73:0/3430422322 >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 msgr2=0x7fae200cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae17400640 1 --2- 172.21.15.73:0/3430422322 >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 0x7fae200cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae17400640 1 -- 172.21.15.73:0/3430422322 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2006f800 con 0x7fae200ccd20 2024-04-05T22:29:14.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.120+0000 7fae1fe00640 1 --2- 172.21.15.73:0/3430422322 >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 0x7fae200cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:14.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.121+0000 7fae17400640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae200ccd20 0x7fae2006cb80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fae200667b0 tx=0x7fae0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bc709e3636784 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.121+0000 7fae1f400640 1 -- 172.21.15.73:0/3430422322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08031870 con 0x7fae200ccd20 2024-04-05T22:29:14.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.121+0000 7fae1f400640 1 -- 172.21.15.73:0/3430422322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae080319d0 con 0x7fae200ccd20 2024-04-05T22:29:14.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.121+0000 7fae1f400640 1 -- 172.21.15.73:0/3430422322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08031d60 con 0x7fae200ccd20 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 -- 172.21.15.73:0/3430422322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae200ccd20 msgr2=0x7fae2006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae200ccd20 0x7fae2006cb80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fae200667b0 tx=0x7fae0802b910 comp rx=0 tx=0).stop 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 -- 172.21.15.73:0/3430422322 shutdown_connections 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae200ccd20 0x7fae2006cb80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 --2- 172.21.15.73:0/3430422322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae2006d0c0 0x7fae2015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 --2- 172.21.15.73:0/3430422322 >> v2:172.21.15.157:3300/0 conn(0x7fae201587d0 0x7fae200cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 -- 172.21.15.73:0/3430422322 >> 172.21.15.73:0/3430422322 conn(0x7fae200ce8e0 msgr2=0x7fae200cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 -- 172.21.15.73:0/3430422322 shutdown_connections 2024-04-05T22:29:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.122+0000 7fae27200640 1 -- 172.21.15.73:0/3430422322 wait complete. 2024-04-05T22:29:14.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.124+0000 7fae27200640 1 Processor -- start 2024-04-05T22:29:14.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.124+0000 7fae27200640 1 -- start start 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae2006d0c0 0x7fae201e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae201587d0 0x7fae201e7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200af3a0 con 0x7fae20146310 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200af680 con 0x7fae2006d0c0 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.125+0000 7fae27200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae200af960 con 0x7fae201587d0 2024-04-05T22:29:14.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51676/0 (socket says 172.21.15.73:51676) 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 -- 172.21.15.73:0/3415802030 learned_addr learned my addr 172.21.15.73:0/3415802030 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae1fe00640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae2006d0c0 0x7fae201e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae17400640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae201587d0 0x7fae201e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 -- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae201587d0 msgr2=0x7fae201e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae201587d0 0x7fae201e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 -- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae2006d0c0 msgr2=0x7fae201e6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae2006d0c0 0x7fae201e6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 -- 172.21.15.73:0/3415802030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080094d0 con 0x7fae20146310 2024-04-05T22:29:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae25200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ccf0 tx=0x7fae1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.126+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18003b90 con 0x7fae20146310 2024-04-05T22:29:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.127+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18003cf0 con 0x7fae20146310 2024-04-05T22:29:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.127+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18004240 con 0x7fae20146310 2024-04-05T22:29:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.127+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae20144320 con 0x7fae20146310 2024-04-05T22:29:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.127+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae20157a90 con 0x7fae20146310 2024-04-05T22:29:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.128+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20066af0 con 0x7fae20146310 2024-04-05T22:29:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.129+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fae180043a0 con 0x7fae20146310 2024-04-05T22:29:14.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.129+0000 7fae1cc00640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 0x7fadfc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.130+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fae18012070 con 0x7fae20146310 2024-04-05T22:29:14.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.130+0000 7fae1fe00640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 0x7fadfc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.131+0000 7fae1fe00640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 0x7fadfc06d660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae1000a950 tx=0x7fae10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.134+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae180550d0 con 0x7fae20146310 2024-04-05T22:29:14.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.371+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fae2006c830 con 0x7fae20146310 2024-04-05T22:29:14.373 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:29:14.373+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:29:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.387+0000 7fae1cc00640 1 -- 172.21.15.73:0/3415802030 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7fae180586f0 con 0x7fae20146310 2024-04-05T22:29:14.387 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:29:14.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 msgr2=0x7fadfc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 0x7fadfc06d660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae1000a950 tx=0x7fae10008040 comp rx=0 tx=0).stop 2024-04-05T22:29:14.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 msgr2=0x7fae20146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ccf0 tx=0x7fae1800a660 comp rx=0 tx=0).stop 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 shutdown_connections 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fadfc06b1a0 0x7fadfc06d660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae20146310 0x7fae20146790 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae201587d0 0x7fae201e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 --2- 172.21.15.73:0/3415802030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae2006d0c0 0x7fae201e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 >> 172.21.15.73:0/3415802030 conn(0x7fae200ce8e0 msgr2=0x7fae200cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 shutdown_connections 2024-04-05T22:29:14.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.390+0000 7fae27200640 1 -- 172.21.15.73:0/3415802030 wait complete. 2024-04-05T22:29:14.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:29:14.512 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:29:14.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.616+0000 7fee6a800640 1 Processor -- start 2024-04-05T22:29:14.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.616+0000 7fee6a800640 1 -- start start 2024-04-05T22:29:14.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.617+0000 7fee6a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee640cbe00 0x7fee640ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee6a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee640b32b0 0x7fee641647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee6a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640b1b30 con 0x7fee6406c630 2024-04-05T22:29:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee6a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640b1e10 con 0x7fee640b32b0 2024-04-05T22:29:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee6a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640cc300 con 0x7fee640cbe00 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee640b32b0 0x7fee641647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fee6406c630 0x7fee6406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41276/0 (socket says 172.21.15.73:41276) 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63400640 1 -- 172.21.15.73:0/2993978739 learned_addr learned my addr 172.21.15.73:0/2993978739 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee62a00640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee640cbe00 0x7fee640ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63e00640 1 -- 172.21.15.73:0/2993978739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee640cbe00 msgr2=0x7fee640ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63e00640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee640cbe00 0x7fee640ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63e00640 1 -- 172.21.15.73:0/2993978739 >> v1:172.21.15.73:6789/0 conn(0x7fee6406c630 legacy=0x7fee6406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.618+0000 7fee63e00640 1 -- 172.21.15.73:0/2993978739 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640cc5e0 con 0x7fee640b32b0 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.619+0000 7fee63e00640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640b32b0 0x7fee641647c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa248b1ae1c1c285 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee62000640 1 -- 172.21.15.73:0/2993978739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c02cc60 con 0x7fee640b32b0 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee62000640 1 -- 172.21.15.73:0/2993978739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c02cdc0 con 0x7fee640b32b0 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee62000640 1 -- 172.21.15.73:0/2993978739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c034660 con 0x7fee640b32b0 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 -- 172.21.15.73:0/2993978739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640b32b0 msgr2=0x7fee641647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640b32b0 0x7fee641647c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 -- 172.21.15.73:0/2993978739 shutdown_connections 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee640cbe00 0x7fee640ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 --2- 172.21.15.73:0/2993978739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640b32b0 0x7fee641647c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.620+0000 7fee6a800640 1 -- 172.21.15.73:0/2993978739 >> 172.21.15.73:0/2993978739 conn(0x7fee640b0830 msgr2=0x7fee640b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:14.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.621+0000 7fee6a800640 1 -- 172.21.15.73:0/2993978739 shutdown_connections 2024-04-05T22:29:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.621+0000 7fee6a800640 1 -- 172.21.15.73:0/2993978739 wait complete. 2024-04-05T22:29:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.622+0000 7fee6a800640 1 Processor -- start 2024-04-05T22:29:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.622+0000 7fee6a800640 1 -- start start 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6406c630 0x7fee640c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee640b32b0 0x7fee641f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640ad760 con 0x7fee640cbe00 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640ada40 con 0x7fee640b32b0 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.623+0000 7fee6a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640add20 con 0x7fee6406c630 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51698/0 (socket says 172.21.15.73:51698) 2024-04-05T22:29:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 -- 172.21.15.73:0/265266268 learned_addr learned my addr 172.21.15.73:0/265266268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 -- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6406c630 msgr2=0x7fee640c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6406c630 0x7fee640c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 -- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee640b32b0 msgr2=0x7fee641f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee640b32b0 0x7fee641f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 -- 172.21.15.73:0/265266268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009a70 con 0x7fee640cbe00 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.624+0000 7fee63e00640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.625+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c040650 con 0x7fee640cbe00 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.625+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c0407d0 con 0x7fee640cbe00 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.625+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c040b80 con 0x7fee640cbe00 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.625+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640ae000 con 0x7fee640cbe00 2024-04-05T22:29:14.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.625+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee641f21b0 con 0x7fee640cbe00 2024-04-05T22:29:14.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.626+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee641f23a0 con 0x7fee640cbe00 2024-04-05T22:29:14.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.627+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fee5c040ce0 con 0x7fee640cbe00 2024-04-05T22:29:14.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.628+0000 7fee5b400640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 0x7fee4406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:14.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.628+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fee5c0acca0 con 0x7fee640cbe00 2024-04-05T22:29:14.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.628+0000 7fee63400640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 0x7fee4406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:14.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.629+0000 7fee63400640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 0x7fee4406d660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee540015f0 tx=0x7fee54007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:14.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.632+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fee5c077690 con 0x7fee640cbe00 2024-04-05T22:29:14.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:14.861+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee64068160 con 0x7fee640cbe00 2024-04-05T22:29:15.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.393+0000 7fee5b400640 1 -- 172.21.15.73:0/265266268 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee5c07acb0 con 0x7fee640cbe00 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 msgr2=0x7fee4406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 0x7fee4406d660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee540015f0 tx=0x7fee54007a40 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 msgr2=0x7fee640c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 shutdown_connections 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee4406b1a0 0x7fee4406d660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee640cbe00 0x7fee640c4480 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee640b32b0 0x7fee641f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 --2- 172.21.15.73:0/265266268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6406c630 0x7fee640c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 >> 172.21.15.73:0/265266268 conn(0x7fee640b0830 msgr2=0x7fee6415d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 shutdown_connections 2024-04-05T22:29:15.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.397+0000 7fee6a800640 1 -- 172.21.15.73:0/265266268 wait complete. 2024-04-05T22:29:15.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:29:15.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.628+0000 7fa8c7a00640 1 Processor -- start 2024-04-05T22:29:15.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.628+0000 7fa8c7a00640 1 -- start start 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.629+0000 7fa8c7a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8c00d1600 0x7fa8c00cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.629+0000 7fa8c7a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 0x7fa8c00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.629+0000 7fa8c7a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00cde60 con 0x7fa8c015af90 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.629+0000 7fa8c7a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c00ce400 con 0x7fa8c00d1600 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.629+0000 7fa8c7a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c00cc790 con 0x7fa8c00cd4d0 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8c00d1600 0x7fa8c00cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8c00d1600 0x7fa8c00cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51712/0 (socket says 172.21.15.73:51712) 2024-04-05T22:29:15.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 -- 172.21.15.73:0/3120851283 learned_addr learned my addr 172.21.15.73:0/3120851283 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 -- 172.21.15.73:0/3120851283 >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 msgr2=0x7fa8c00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8c5a00640 1 --2- 172.21.15.73:0/3120851283 >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 0x7fa8c00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 --2- 172.21.15.73:0/3120851283 >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 0x7fa8c00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 -- 172.21.15.73:0/3120851283 >> v1:172.21.15.157:6789/0 conn(0x7fa8c015af90 legacy=0x7fa8c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8bfe00640 1 -- 172.21.15.73:0/3120851283 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00ce850 con 0x7fa8c00d1600 2024-04-05T22:29:15.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.630+0000 7fa8c5a00640 1 --2- 172.21.15.73:0/3120851283 >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 0x7fa8c00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:15.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.631+0000 7fa8bfe00640 1 --2- 172.21.15.73:0/3120851283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c00cc0d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa8c015fbb0 tx=0x7fa8a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ee0fa488d407f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8bf400640 1 -- 172.21.15.73:0/3120851283 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a802ca80 con 0x7fa8c00d1600 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8bf400640 1 -- 172.21.15.73:0/3120851283 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a802cbe0 con 0x7fa8c00d1600 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8bf400640 1 -- 172.21.15.73:0/3120851283 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80343f0 con 0x7fa8c00d1600 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 -- 172.21.15.73:0/3120851283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 msgr2=0x7fa8c00cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/3120851283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c00cc0d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa8c015fbb0 tx=0x7fa8a802bee0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 -- 172.21.15.73:0/3120851283 shutdown_connections 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/3120851283 >> v2:172.21.15.157:3300/0 conn(0x7fa8c00cd4d0 0x7fa8c00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/3120851283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c00cc0d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.632+0000 7fa8c7a00640 1 -- 172.21.15.73:0/3120851283 >> 172.21.15.73:0/3120851283 conn(0x7fa8c00d0ae0 msgr2=0x7fa8c00d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.633+0000 7fa8c7a00640 1 -- 172.21.15.73:0/3120851283 shutdown_connections 2024-04-05T22:29:15.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.633+0000 7fa8c7a00640 1 -- 172.21.15.73:0/3120851283 wait complete. 2024-04-05T22:29:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.634+0000 7fa8c7a00640 1 Processor -- start 2024-04-05T22:29:15.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.634+0000 7fa8c7a00640 1 -- start start 2024-04-05T22:29:15.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8c00cd4d0 0x7fa8c015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c015af90 0x7fa8c015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0157920 con 0x7fa8c00d1600 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015bb60 con 0x7fa8c015af90 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.635+0000 7fa8c7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015be40 con 0x7fa8c00cd4d0 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51722/0 (socket says 172.21.15.73:51722) 2024-04-05T22:29:15.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 -- 172.21.15.73:0/1264768943 learned_addr learned my addr 172.21.15.73:0/1264768943 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:15.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 -- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8c00cd4d0 msgr2=0x7fa8c015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:15.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8c00cd4d0 0x7fa8c015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 -- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c015af90 msgr2=0x7fa8c015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c015af90 0x7fa8c015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 -- 172.21.15.73:0/1264768943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8009b30 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.636+0000 7fa8bfe00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000f80 tx=0x7fa8a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.637+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a803e450 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.637+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a803e5f0 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.637+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a803e9a0 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.637+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c015c120 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.637+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c015dc90 con 0x7fa8c00d1600 2024-04-05T22:29:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.638+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c01f0910 con 0x7fa8c00d1600 2024-04-05T22:29:15.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.639+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa8a802cd50 con 0x7fa8c00d1600 2024-04-05T22:29:15.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.640+0000 7fa8bcc00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 0x7fa89c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:15.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.640+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa8a80ac760 con 0x7fa8c00d1600 2024-04-05T22:29:15.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.640+0000 7fa8c5000640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 0x7fa89c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:15.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.642+0000 7fa8c5000640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 0x7fa89c06d660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009650 tx=0x7fa8b0006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:15.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.645+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa8a80772c0 con 0x7fa8c00d1600 2024-04-05T22:29:15.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:15.842+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8c00cde60 con 0x7fa8c00d1600 2024-04-05T22:29:16.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.543+0000 7fa8bcc00640 1 -- 172.21.15.73:0/1264768943 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8c00cde60 con 0x7fa8c00d1600 2024-04-05T22:29:16.544 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:29:16.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 msgr2=0x7fa89c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:16.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 0x7fa89c06d660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009650 tx=0x7fa8b0006ba0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 msgr2=0x7fa8c01f03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:16.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000f80 tx=0x7fa8a8004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 shutdown_connections 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa89c06b1a0 0x7fa89c06d660 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c015af90 0x7fa8c015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c00d1600 0x7fa8c01f03b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 --2- 172.21.15.73:0/1264768943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8c00cd4d0 0x7fa8c015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.549+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 >> 172.21.15.73:0/1264768943 conn(0x7fa8c00d0ae0 msgr2=0x7fa8c00d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.550+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 shutdown_connections 2024-04-05T22:29:16.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.550+0000 7fa8c7a00640 1 -- 172.21.15.73:0/1264768943 wait complete. 2024-04-05T22:29:16.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:29:16.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.773+0000 7fdd36a00640 1 Processor -- start 2024-04-05T22:29:16.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.773+0000 7fdd36a00640 1 -- start start 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd36a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd300cf3e0 0x7fdd300b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 0x7fdd300b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd36a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3015b230 con 0x7fdd3015ae50 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd36a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd300cd320 con 0x7fdd300cf3e0 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd36a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300ce2f0 con 0x7fdd300cce90 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd27400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd300cf3e0 0x7fdd300b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd27400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd300cf3e0 0x7fdd300b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47754/0 (socket says 172.21.15.73:47754) 2024-04-05T22:29:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.775+0000 7fdd27400640 1 -- 172.21.15.73:0/447188241 learned_addr learned my addr 172.21.15.73:0/447188241 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd34a00640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 0x7fdd300b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd27400640 1 -- 172.21.15.73:0/447188241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 msgr2=0x7fdd300b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd27400640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 0x7fdd300b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd27400640 1 -- 172.21.15.73:0/447188241 >> v1:172.21.15.157:6789/0 conn(0x7fdd3015ae50 legacy=0x7fdd300cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd27400640 1 -- 172.21.15.73:0/447188241 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3015f4a0 con 0x7fdd300cf3e0 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.776+0000 7fdd34a00640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 0x7fdd300b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.777+0000 7fdd27400640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cf3e0 0x7fdd300b3aa0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009870 tx=0x7fdd1802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23790d453301375 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.777+0000 7fdd2f400640 1 -- 172.21.15.73:0/447188241 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd180317c0 con 0x7fdd300cf3e0 2024-04-05T22:29:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.777+0000 7fdd2f400640 1 -- 172.21.15.73:0/447188241 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18031920 con 0x7fdd300cf3e0 2024-04-05T22:29:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd2f400640 1 -- 172.21.15.73:0/447188241 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18031cb0 con 0x7fdd300cf3e0 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 -- 172.21.15.73:0/447188241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cf3e0 msgr2=0x7fdd300b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cf3e0 0x7fdd300b3aa0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009870 tx=0x7fdd1802b860 comp rx=0 tx=0).stop 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 -- 172.21.15.73:0/447188241 shutdown_connections 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300cce90 0x7fdd300b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 --2- 172.21.15.73:0/447188241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cf3e0 0x7fdd300b3aa0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 -- 172.21.15.73:0/447188241 >> 172.21.15.73:0/447188241 conn(0x7fdd3006a0d0 msgr2=0x7fdd3006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 -- 172.21.15.73:0/447188241 shutdown_connections 2024-04-05T22:29:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.778+0000 7fdd36a00640 1 -- 172.21.15.73:0/447188241 wait complete. 2024-04-05T22:29:16.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.780+0000 7fdd36a00640 1 Processor -- start 2024-04-05T22:29:16.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.780+0000 7fdd36a00640 1 -- start start 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cf3e0 0x7fdd300af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd3015ae50 0x7fdd300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300ca600 con 0x7fdd300cce90 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301512d0 con 0x7fdd300cf3e0 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd36a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300afef0 con 0x7fdd3015ae50 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47760/0 (socket says 172.21.15.73:47760) 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.781+0000 7fdd2fe00640 1 -- 172.21.15.73:0/310544525 learned_addr learned my addr 172.21.15.73:0/310544525 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd27400640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cf3e0 0x7fdd300af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd34a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd3015ae50 0x7fdd300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 -- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd3015ae50 msgr2=0x7fdd300caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd3015ae50 0x7fdd300caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 -- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cf3e0 msgr2=0x7fdd300af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cf3e0 0x7fdd300af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 -- 172.21.15.73:0/310544525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180094d0 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2fe00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b410 tx=0x7fdd2000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20011830 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20011990 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20011d40 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.782+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd300b0170 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.783+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd300c9fb0 con 0x7fdd300cce90 2024-04-05T22:29:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.784+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd30068160 con 0x7fdd300cce90 2024-04-05T22:29:16.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.785+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fdd2001a410 con 0x7fdd300cce90 2024-04-05T22:29:16.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.785+0000 7fdd2cc00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 0x7fdd0c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:16.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.786+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fdd20089910 con 0x7fdd300cce90 2024-04-05T22:29:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.786+0000 7fdd27400640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 0x7fdd0c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.787+0000 7fdd27400640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 0x7fdd0c06d690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd18036000 tx=0x7fdd1803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.790+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdd200543a0 con 0x7fdd300cce90 2024-04-05T22:29:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:16.983+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdd3006a6c0 con 0x7fdd300cce90 2024-04-05T22:29:17.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.556+0000 7fdd2cc00640 1 -- 172.21.15.73:0/310544525 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdd200579c0 con 0x7fdd300cce90 2024-04-05T22:29:17.557 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:29:17.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.564+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 msgr2=0x7fdd0c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.564+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 0x7fdd0c06d690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd18036000 tx=0x7fdd1803d040 comp rx=0 tx=0).stop 2024-04-05T22:29:17.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.564+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 msgr2=0x7fdd30150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.564+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b410 tx=0x7fdd2000b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 shutdown_connections 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd0c06b1d0 0x7fdd0c06d690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd3015ae50 0x7fdd300caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cf3e0 0x7fdd300af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 --2- 172.21.15.73:0/310544525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cce90 0x7fdd30150d90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.566+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 >> 172.21.15.73:0/310544525 conn(0x7fdd3006a0d0 msgr2=0x7fdd300b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:17.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.567+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 shutdown_connections 2024-04-05T22:29:17.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.568+0000 7fdd36a00640 1 -- 172.21.15.73:0/310544525 wait complete. 2024-04-05T22:29:17.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:29:17.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.784+0000 7ff163a00640 1 Processor -- start 2024-04-05T22:29:17.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.784+0000 7ff163a00640 1 -- start start 2024-04-05T22:29:17.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.785+0000 7ff163a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 0x7ff15c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.785+0000 7ff163a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff15c0b32b0 0x7ff15c1647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.785+0000 7ff163a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0b1b30 con 0x7ff15c0cbe00 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.785+0000 7ff163a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0b1e10 con 0x7ff15c0b32b0 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.785+0000 7ff163a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0ce7b0 con 0x7ff15c06c630 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff15c0b32b0 0x7ff15c1647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff153e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff15c0cbe00 0x7ff15c0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49902/0 (socket says 172.21.15.73:49902) 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff153e00640 1 -- 172.21.15.73:0/3343138280 learned_addr learned my addr 172.21.15.73:0/3343138280 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161000640 1 --2- 172.21.15.73:0/3343138280 >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 0x7ff15c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161a00640 1 -- 172.21.15.73:0/3343138280 >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 msgr2=0x7ff15c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161a00640 1 --2- 172.21.15.73:0/3343138280 >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 0x7ff15c06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161a00640 1 -- 172.21.15.73:0/3343138280 >> v1:172.21.15.73:6789/0 conn(0x7ff15c0cbe00 legacy=0x7ff15c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161a00640 1 -- 172.21.15.73:0/3343138280 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c06cf50 con 0x7ff15c0b32b0 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff15be00640 1 -- 172.21.15.73:0/3343138280 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935722361 0 0) 0x7ff15c0b1b30 con 0x7ff15c0cbe00 2024-04-05T22:29:17.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.786+0000 7ff161000640 1 --2- 172.21.15.73:0/3343138280 >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 0x7ff15c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:17.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.787+0000 7ff161a00640 1 --2- 172.21.15.73:0/3343138280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1647c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff15c06f360 tx=0x7ff15402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498ff3c46d5b565b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:17.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff15be00640 1 -- 172.21.15.73:0/3343138280 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15402cc70 con 0x7ff15c0b32b0 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff15be00640 1 -- 172.21.15.73:0/3343138280 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15402cdd0 con 0x7ff15c0b32b0 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff15be00640 1 -- 172.21.15.73:0/3343138280 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154034660 con 0x7ff15c0b32b0 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 -- 172.21.15.73:0/3343138280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 msgr2=0x7ff15c1647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 --2- 172.21.15.73:0/3343138280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1647c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff15c06f360 tx=0x7ff15402bec0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 -- 172.21.15.73:0/3343138280 shutdown_connections 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 --2- 172.21.15.73:0/3343138280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1647c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 --2- 172.21.15.73:0/3343138280 >> v2:172.21.15.157:3300/0 conn(0x7ff15c06c630 0x7ff15c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 -- 172.21.15.73:0/3343138280 >> 172.21.15.73:0/3343138280 conn(0x7ff15c0b0830 msgr2=0x7ff15c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.788+0000 7ff163a00640 1 -- 172.21.15.73:0/3343138280 shutdown_connections 2024-04-05T22:29:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.789+0000 7ff163a00640 1 -- 172.21.15.73:0/3343138280 wait complete. 2024-04-05T22:29:17.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.790+0000 7ff163a00640 1 Processor -- start 2024-04-05T22:29:17.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.790+0000 7ff163a00640 1 -- start start 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff15c06c630 0x7ff15c071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15c0cbe00 0x7ff15c140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c070e70 con 0x7ff15c0b32b0 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c0717d0 con 0x7ff15c0cbe00 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.791+0000 7ff163a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c070230 con 0x7ff15c06c630 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47780/0 (socket says 172.21.15.73:47780) 2024-04-05T22:29:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 -- 172.21.15.73:0/1863735299 learned_addr learned my addr 172.21.15.73:0/1863735299 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff161000640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff15c06c630 0x7ff15c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 -- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff15c06c630 msgr2=0x7ff15c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff15c06c630 0x7ff15c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff161a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15c0cbe00 0x7ff15c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 -- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15c0cbe00 msgr2=0x7ff15c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15c0cbe00 0x7ff15c140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 -- 172.21.15.73:0/1863735299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154009a70 con 0x7ff15c0b32b0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.792+0000 7ff153e00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c430 tx=0x7ff14400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.793+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1440074e0 con 0x7ff15c0b32b0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.793+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144007660 con 0x7ff15c0b32b0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.793+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c0704d0 con 0x7ff15c0b32b0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.793+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144004a20 con 0x7ff15c0b32b0 2024-04-05T22:29:17.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.793+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c070a90 con 0x7ff15c0b32b0 2024-04-05T22:29:17.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.794+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c070fb0 con 0x7ff15c0b32b0 2024-04-05T22:29:17.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.795+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff144004b80 con 0x7ff15c0b32b0 2024-04-05T22:29:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.796+0000 7ff159600640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 0x7ff13806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.796+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff144089890 con 0x7ff15c0b32b0 2024-04-05T22:29:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.796+0000 7ff161000640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 0x7ff13806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.798+0000 7ff161000640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 0x7ff13806d660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff14c008850 tx=0x7ff14c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:17.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:17.800+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff15c070fb0 con 0x7ff15c0b32b0 2024-04-05T22:29:18.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.000+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff15c070fb0 con 0x7ff15c0b32b0 2024-04-05T22:29:18.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.009+0000 7ff159600640 1 -- 172.21.15.73:0/1863735299 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff15c070fb0 con 0x7ff15c0b32b0 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.011+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 msgr2=0x7ff13806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.011+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 0x7ff13806d660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff14c008850 tx=0x7ff14c008040 comp rx=0 tx=0).stop 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 msgr2=0x7ff15c1341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c430 tx=0x7ff14400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 shutdown_connections 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff13806b1a0 0x7ff13806d660 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15c0cbe00 0x7ff15c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff15c0b32b0 0x7ff15c1341a0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 --2- 172.21.15.73:0/1863735299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff15c06c630 0x7ff15c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 >> 172.21.15.73:0/1863735299 conn(0x7ff15c0b0830 msgr2=0x7ff15c0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:18.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 shutdown_connections 2024-04-05T22:29:18.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.012+0000 7ff163a00640 1 -- 172.21.15.73:0/1863735299 wait complete. 2024-04-05T22:29:18.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-05T22:29:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.233+0000 7f1284000640 1 Processor -- start 2024-04-05T22:29:18.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.233+0000 7f1284000640 1 -- start start 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1284000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c15cf50 0x7f127c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1284000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f127c0d09b0 0x7f127c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1284000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c0cb950 con 0x7f127c0cd090 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1284000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c0ce990 con 0x7f127c0d09b0 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1284000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0d1350 con 0x7f127c15cf50 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1280c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f127c0d09b0 0x7f127c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1280c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f127c0d09b0 0x7f127c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47790/0 (socket says 172.21.15.73:47790) 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.235+0000 7f1280c00640 1 -- 172.21.15.73:0/1433045040 learned_addr learned my addr 172.21.15.73:0/1433045040 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.236+0000 7f1281600640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c15cf50 0x7f127c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.236+0000 7f1280c00640 1 -- 172.21.15.73:0/1433045040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c15cf50 msgr2=0x7f127c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.236+0000 7f1280c00640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c15cf50 0x7f127c06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.236+0000 7f1280c00640 1 -- 172.21.15.73:0/1433045040 >> v1:172.21.15.157:6789/0 conn(0x7f127c0cd090 legacy=0x7f127c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.236+0000 7f1280c00640 1 -- 172.21.15.73:0/1433045040 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0cdce0 con 0x7f127c0d09b0 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.237+0000 7f1280c00640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c0d09b0 0x7f127c0d0e10 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f1268009870 tx=0x7f126802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5e3fa257cf3e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.237+0000 7f127be00640 1 -- 172.21.15.73:0/1433045040 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268031640 con 0x7f127c0d09b0 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.237+0000 7f127be00640 1 -- 172.21.15.73:0/1433045040 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12680317a0 con 0x7f127c0d09b0 2024-04-05T22:29:18.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.237+0000 7f127be00640 1 -- 172.21.15.73:0/1433045040 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268031b30 con 0x7f127c0d09b0 2024-04-05T22:29:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 -- 172.21.15.73:0/1433045040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c0d09b0 msgr2=0x7f127c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c0d09b0 0x7f127c0d0e10 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f1268009870 tx=0x7f126802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 -- 172.21.15.73:0/1433045040 shutdown_connections 2024-04-05T22:29:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c0d09b0 0x7f127c0d0e10 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 --2- 172.21.15.73:0/1433045040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c15cf50 0x7f127c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 -- 172.21.15.73:0/1433045040 >> 172.21.15.73:0/1433045040 conn(0x7f127c099a60 msgr2=0x7f127c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 -- 172.21.15.73:0/1433045040 shutdown_connections 2024-04-05T22:29:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.238+0000 7f1284000640 1 -- 172.21.15.73:0/1433045040 wait complete. 2024-04-05T22:29:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.240+0000 7f1284000640 1 Processor -- start 2024-04-05T22:29:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.241+0000 7f1284000640 1 -- start start 2024-04-05T22:29:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c0cd090 0x7f127c0c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c0d09b0 0x7f127c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0c5fb0 con 0x7f127c15cf50 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0c54e0 con 0x7f127c0d09b0 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1284000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0b0520 con 0x7f127c0cd090 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1282000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1282000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47806/0 (socket says 172.21.15.73:47806) 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1282000640 1 -- 172.21.15.73:0/1841543982 learned_addr learned my addr 172.21.15.73:0/1841543982 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1280c00640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c0d09b0 0x7f127c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.242+0000 7f1281600640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c0cd090 0x7f127c0c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 -- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c0cd090 msgr2=0x7f127c0c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c0cd090 0x7f127c0c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 -- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c0d09b0 msgr2=0x7f127c1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c0d09b0 0x7f127c1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 -- 172.21.15.73:0/1841543982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680094d0 con 0x7f127c15cf50 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1282000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7b0 tx=0x7f127400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274004b60 con 0x7f127c15cf50 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274004cc0 con 0x7f127c15cf50 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0c5710 con 0x7f127c15cf50 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127400aa90 con 0x7f127c15cf50 2024-04-05T22:29:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.243+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c0c5cd0 con 0x7f127c15cf50 2024-04-05T22:29:18.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.245+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c0b02f0 con 0x7f127c15cf50 2024-04-05T22:29:18.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.245+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f127400abf0 con 0x7f127c15cf50 2024-04-05T22:29:18.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.246+0000 7f1279600640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 0x7f125c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.246+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1274089ca0 con 0x7f127c15cf50 2024-04-05T22:29:18.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.247+0000 7f1281600640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 0x7f125c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.248+0000 7f1281600640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 0x7f125c06d690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1270006fd0 tx=0x7f1270008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.251+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f127c0b02f0 con 0x7f127c15cf50 2024-04-05T22:29:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.520+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f127c0b02f0 con 0x7f127c15cf50 2024-04-05T22:29:18.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.576+0000 7f1279600640 1 -- 172.21.15.73:0/1841543982 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v431) v1 ==== 148+0+0 (secure 0 0 0) 0x7f1274054910 con 0x7f127c15cf50 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.578+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 msgr2=0x7f125c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.578+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 0x7f125c06d690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1270006fd0 tx=0x7f1270008040 comp rx=0 tx=0).stop 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.578+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 msgr2=0x7f127c0b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.578+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7b0 tx=0x7f127400cc70 comp rx=0 tx=0).stop 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 shutdown_connections 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f125c06b1d0 0x7f125c06d690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f127c15cf50 0x7f127c0b27f0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f127c0d09b0 0x7f127c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 --2- 172.21.15.73:0/1841543982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f127c0cd090 0x7f127c0c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 >> 172.21.15.73:0/1841543982 conn(0x7f127c099a60 msgr2=0x7f127c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:18.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 shutdown_connections 2024-04-05T22:29:18.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.579+0000 7f1284000640 1 -- 172.21.15.73:0/1841543982 wait complete. 2024-04-05T22:29:18.599 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-05T22:29:18.599 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:29:18.600 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:18.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.806+0000 7fd09b200640 1 Processor -- start 2024-04-05T22:29:18.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.807+0000 7fd09b200640 1 -- start start 2024-04-05T22:29:18.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.808+0000 7fd09b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd094160620 0x7fd0940caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.808+0000 7fd09b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0940cb2f0 0x7fd0940cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.808+0000 7fd09b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940ad880 con 0x7fd0940b1870 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.808+0000 7fd09b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940cbcc0 con 0x7fd0940cb2f0 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.808+0000 7fd09b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940ce5d0 con 0x7fd094160620 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0940cb2f0 0x7fd0940cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0940cb2f0 0x7fd0940cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47814/0 (socket says 172.21.15.73:47814) 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 -- 172.21.15.73:0/2217261301 learned_addr learned my addr 172.21.15.73:0/2217261301 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:18.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd093e00640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd094160620 0x7fd0940caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 -- 172.21.15.73:0/2217261301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd094160620 msgr2=0x7fd0940caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:18.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd094160620 0x7fd0940caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 -- 172.21.15.73:0/2217261301 >> v1:172.21.15.157:6789/0 conn(0x7fd0940b1870 legacy=0x7fd0940ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.809+0000 7fd099200640 1 -- 172.21.15.73:0/2217261301 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940ce010 con 0x7fd0940cb2f0 2024-04-05T22:29:18.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.811+0000 7fd099200640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0940cb2f0 0x7fd0940cb780 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38958dd1e2ce320b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.811+0000 7fd093400640 1 -- 172.21.15.73:0/2217261301 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c02cb00 con 0x7fd0940cb2f0 2024-04-05T22:29:18.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.811+0000 7fd093400640 1 -- 172.21.15.73:0/2217261301 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c02cc60 con 0x7fd0940cb2f0 2024-04-05T22:29:18.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.811+0000 7fd093400640 1 -- 172.21.15.73:0/2217261301 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0343f0 con 0x7fd0940cb2f0 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 -- 172.21.15.73:0/2217261301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0940cb2f0 msgr2=0x7fd0940cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0940cb2f0 0x7fd0940cb780 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c02bf60 comp rx=0 tx=0).stop 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 -- 172.21.15.73:0/2217261301 shutdown_connections 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0940cb2f0 0x7fd0940cb780 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 --2- 172.21.15.73:0/2217261301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd094160620 0x7fd0940caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 -- 172.21.15.73:0/2217261301 >> 172.21.15.73:0/2217261301 conn(0x7fd0940ce8e0 msgr2=0x7fd0940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 -- 172.21.15.73:0/2217261301 shutdown_connections 2024-04-05T22:29:18.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.812+0000 7fd09b200640 1 -- 172.21.15.73:0/2217261301 wait complete. 2024-04-05T22:29:18.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.814+0000 7fd09b200640 1 Processor -- start 2024-04-05T22:29:18.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.814+0000 7fd09b200640 1 -- start start 2024-04-05T22:29:18.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 0x7fd094144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 0x7fd09415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415ba00 con 0x7fd094160620 2024-04-05T22:29:18.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415b3f0 con 0x7fd0940b1870 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd09b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094158ca0 con 0x7fd0940cb2f0 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd099200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd099200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47828/0 (socket says 172.21.15.73:47828) 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd099200640 1 -- 172.21.15.73:0/3051627975 learned_addr learned my addr 172.21.15.73:0/3051627975 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd093e00640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 0x7fd094144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.815+0000 7fd08b400640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 0x7fd09415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 -- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 msgr2=0x7fd09415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 0x7fd09415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 -- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 msgr2=0x7fd094144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 0x7fd094144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 -- 172.21.15.73:0/3051627975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c009b30 con 0x7fd094160620 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd093e00640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 0x7fd094144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd08b400640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 0x7fd09415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd099200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c009510 con 0x7fd094160620 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c03e450 con 0x7fd094160620 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941589c0 con 0x7fd094160620 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c03e800 con 0x7fd094160620 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.816+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd094158650 con 0x7fd094160620 2024-04-05T22:29:18.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.817+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094066af0 con 0x7fd094160620 2024-04-05T22:29:18.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.818+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fd08c03e960 con 0x7fd094160620 2024-04-05T22:29:18.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.819+0000 7fd090c00640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 0x7fd07006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:18.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.819+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd08c0ac850 con 0x7fd094160620 2024-04-05T22:29:18.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.820+0000 7fd093e00640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 0x7fd07006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:18.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.821+0000 7fd093e00640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 0x7fd07006d660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd084006fd0 tx=0x7fd084008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:18.824+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd08c0774c0 con 0x7fd094160620 2024-04-05T22:29:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.021+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd09415c5e0 con 0x7fd094160620 2024-04-05T22:29:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.023+0000 7fd090c00640 1 -- 172.21.15.73:0/3051627975 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+12526 (secure 0 0 0) 0x7fd09415c5e0 con 0x7fd094160620 2024-04-05T22:29:19.024 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:19.024 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":127,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:17.548463+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6835/651015208":"2024-04-06T22:29:14.373826+0000","172.21.15.73:6834/651015208":"2024-04-06T22:29:14.373826+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:19.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.026+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 msgr2=0x7fd07006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 0x7fd07006d660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd084006fd0 tx=0x7fd084008040 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 msgr2=0x7fd09415ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 shutdown_connections 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd07006b1a0 0x7fd07006d660 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd094160620 0x7fd09415ce00 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0940cb2f0 0x7fd09415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 --2- 172.21.15.73:0/3051627975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0940b1870 0x7fd094144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 >> 172.21.15.73:0/3051627975 conn(0x7fd0940ce8e0 msgr2=0x7fd09406cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 shutdown_connections 2024-04-05T22:29:19.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.027+0000 7fd09b200640 1 -- 172.21.15.73:0/3051627975 wait complete. 2024-04-05T22:29:19.045 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:19.046 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:19.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.258+0000 7f15f5a00640 1 Processor -- start 2024-04-05T22:29:19.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.259+0000 7f15f5a00640 1 -- start start 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.260+0000 7f15f5a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15f00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.260+0000 7f15f5a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f015afc0 con 0x7f15f006c9a0 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.260+0000 7f15f5a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f006f5b0 con 0x7f15f006bc40 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.260+0000 7f15f5a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f006f890 con 0x7f15f015abe0 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.261+0000 7f15e7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f15f006c9a0 0x7f15f006b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49914/0 (socket says 172.21.15.73:49914) 2024-04-05T22:29:19.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.261+0000 7f15e7e00640 1 -- 172.21.15.73:0/4004080280 learned_addr learned my addr 172.21.15.73:0/4004080280 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:19.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.261+0000 7f15eea00640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15f00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.261+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268070245 0 0) 0x7f15f015afc0 con 0x7f15f006c9a0 2024-04-05T22:29:19.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.261+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d4003520 con 0x7f15f006c9a0 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2858782716 0 0) 0x7f15d4003520 con 0x7f15f006c9a0 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 msgr2=0x7f15f00712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15f00712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 >> v1:172.21.15.157:6789/0 conn(0x7f15f006bc40 legacy=0x7f15f0164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f006fb70 con 0x7f15f006c9a0 2024-04-05T22:29:19.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f15d8001670 con 0x7f15f006c9a0 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 >> v1:172.21.15.73:6789/0 conn(0x7f15f006c9a0 legacy=0x7f15f006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15d40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4003f20 0x7f15d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15d4004d00 0x7f15d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f006f890 con 0x7f15d4004d00 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d400bbc0 con 0x7f15d4003f20 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d400bd80 con 0x7f15f015abe0 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15ef400640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15d4004d00 0x7f15d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15e7e00640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4003f20 0x7f15d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.262+0000 7f15eea00640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15d40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15ef400640 1 -- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 msgr2=0x7f15d40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15ef400640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15f015abe0 0x7f15d40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15ef400640 1 -- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4003f20 msgr2=0x7f15d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:19.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15ef400640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4003f20 0x7f15d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15ef400640 1 -- 172.21.15.73:0/4004080280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d4003520 con 0x7f15d4004d00 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.263+0000 7f15e7e00640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4003f20 0x7f15d40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.264+0000 7f15ef400640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15d4004d00 0x7f15d4005180 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001e50 tx=0x7f15d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992f68d38a5af04f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.264+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003b70 con 0x7f15d4004d00 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.264+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8003cd0 con 0x7f15d4004d00 2024-04-05T22:29:19.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.264+0000 7f15ee000640 1 -- 172.21.15.73:0/4004080280 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8005310 con 0x7f15d4004d00 2024-04-05T22:29:19.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 -- 172.21.15.73:0/4004080280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15d4004d00 msgr2=0x7f15d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 --2- 172.21.15.73:0/4004080280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15d4004d00 0x7f15d4005180 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001e50 tx=0x7f15d40057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15ef400640 1 -- 172.21.15.73:0/4004080280 reap_dead start 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 -- 172.21.15.73:0/4004080280 shutdown_connections 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 -- 172.21.15.73:0/4004080280 >> 172.21.15.73:0/4004080280 conn(0x7f15f0069cb0 msgr2=0x7f15f006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 -- 172.21.15.73:0/4004080280 shutdown_connections 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.265+0000 7f15f5a00640 1 -- 172.21.15.73:0/4004080280 wait complete. 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.267+0000 7f15f5a00640 1 Processor -- start 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.267+0000 7f15f5a00640 1 -- start start 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 0x7f15f00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f01f0320 0x7f15f00af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0153770 con 0x7f15f013f040 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f01f0820 con 0x7f15f01f0320 2024-04-05T22:29:19.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15f5a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f01f0b00 con 0x7f15d4004d00 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47844/0 (socket says 172.21.15.73:47844) 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 -- 172.21.15.73:0/3763508720 learned_addr learned my addr 172.21.15.73:0/3763508720 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15e7e00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f01f0320 0x7f15f00af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15eea00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 0x7f15f00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 -- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 msgr2=0x7f15f00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 0x7f15f00aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 -- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f01f0320 msgr2=0x7f15f00af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f01f0320 0x7f15f00af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15ef400640 1 -- 172.21.15.73:0/3763508720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f01f0de0 con 0x7f15f013f040 2024-04-05T22:29:19.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.268+0000 7f15eea00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 0x7f15f00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15ef400640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001f70 tx=0x7f15e80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e803e070 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8003e40 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f01f10c0 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8006d90 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.269+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f013fbe0 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.270+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f0068160 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.271+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f15e8007030 con 0x7f15f013f040 2024-04-05T22:29:19.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.272+0000 7f15e6a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 0x7f15cc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.272+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f15e80aae30 con 0x7f15f013f040 2024-04-05T22:29:19.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.273+0000 7f15eea00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 0x7f15cc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.274+0000 7f15eea00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 0x7f15cc06d660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009a00 tx=0x7f15e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.276+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f15e8075990 con 0x7f15f013f040 2024-04-05T22:29:19.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.468+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15f0153770 con 0x7f15f013f040 2024-04-05T22:29:19.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.469+0000 7f15e6a00640 1 -- 172.21.15.73:0/3763508720 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15e8048020 con 0x7f15f013f040 2024-04-05T22:29:19.469 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 msgr2=0x7f15cc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 0x7f15cc06d660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009a00 tx=0x7f15e0008040 comp rx=0 tx=0).stop 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 msgr2=0x7f15f013f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001f70 tx=0x7f15e80014a0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 shutdown_connections 2024-04-05T22:29:19.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f15cc06b1a0 0x7f15cc06d660 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f013f040 0x7f15f013f4c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f01f0320 0x7f15f00af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 --2- 172.21.15.73:0/3763508720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d4004d00 0x7f15f00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 >> 172.21.15.73:0/3763508720 conn(0x7f15f0069cb0 msgr2=0x7f15f015b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 shutdown_connections 2024-04-05T22:29:19.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.472+0000 7f15f5a00640 1 -- 172.21.15.73:0/3763508720 wait complete. 2024-04-05T22:29:19.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-05T22:29:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.704+0000 7f778cc00640 1 Processor -- start 2024-04-05T22:29:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.704+0000 7f778cc00640 1 -- start start 2024-04-05T22:29:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.705+0000 7f778cc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77880b20a0 0x7f7788070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.705+0000 7f778cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.705+0000 7f778cc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778806d3e0 con 0x7f77880b2480 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.705+0000 7f778cc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778806d610 con 0x7f77880b20a0 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.705+0000 7f778cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7788070230 con 0x7f77880b1b50 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.706+0000 7f777ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.706+0000 7f7787400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77880b20a0 0x7f7788070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.706+0000 7f777ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:33134/0 (socket says 172.21.15.73:33134) 2024-04-05T22:29:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.706+0000 7f777ea00640 1 -- 172.21.15.73:0/826055912 learned_addr learned my addr 172.21.15.73:0/826055912 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.707+0000 7f7787400640 1 -- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 msgr2=0x7f778806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.707+0000 7f7787400640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.707+0000 7f7787400640 1 -- 172.21.15.73:0/826055912 >> v1:172.21.15.157:6789/0 conn(0x7f77880b2480 legacy=0x7f77880b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.707+0000 7f7787400640 1 -- 172.21.15.73:0/826055912 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788070510 con 0x7f77880b20a0 2024-04-05T22:29:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.707+0000 7f777ea00640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.708+0000 7f7787400640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f7788070ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7778009d90 tx=0x7f777802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9673046fd886fe3e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.708+0000 7f7786a00640 1 -- 172.21.15.73:0/826055912 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777802cb40 con 0x7f77880b20a0 2024-04-05T22:29:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f7786a00640 1 -- 172.21.15.73:0/826055912 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777802cca0 con 0x7f77880b20a0 2024-04-05T22:29:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f7786a00640 1 -- 172.21.15.73:0/826055912 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778034660 con 0x7f77880b20a0 2024-04-05T22:29:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 -- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 msgr2=0x7f7788070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f7788070ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7778009d90 tx=0x7f777802bd90 comp rx=0 tx=0).stop 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 -- 172.21.15.73:0/826055912 shutdown_connections 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b1b50 0x7f778806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 --2- 172.21.15.73:0/826055912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f7788070ff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 -- 172.21.15.73:0/826055912 >> 172.21.15.73:0/826055912 conn(0x7f77880b06c0 msgr2=0x7f77880b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 -- 172.21.15.73:0/826055912 shutdown_connections 2024-04-05T22:29:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.709+0000 7f778cc00640 1 -- 172.21.15.73:0/826055912 wait complete. 2024-04-05T22:29:19.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.711+0000 7f778cc00640 1 Processor -- start 2024-04-05T22:29:19.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.711+0000 7f778cc00640 1 -- start start 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f77881f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b2480 0x7f77881f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7788150aa0 con 0x7f77880b1b50 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778815b770 con 0x7f77880b20a0 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f778cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778815ba50 con 0x7f77880b2480 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f7787400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f7787400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47856/0 (socket says 172.21.15.73:47856) 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.712+0000 7f7787400640 1 -- 172.21.15.73:0/2700800192 learned_addr learned my addr 172.21.15.73:0/2700800192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f777ea00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f77881f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787e00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b2480 0x7f77881f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 -- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b2480 msgr2=0x7f77881f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b2480 0x7f77881f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 -- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 msgr2=0x7f77881f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f77881f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:19.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 -- 172.21.15.73:0/2700800192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700094d0 con 0x7f77880b1b50 2024-04-05T22:29:19.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.713+0000 7f7787400640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7778009d90 tx=0x7f7778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.714+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778040650 con 0x7f77880b1b50 2024-04-05T22:29:19.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.714+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77780407b0 con 0x7f77880b1b50 2024-04-05T22:29:19.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.714+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7778009a70 con 0x7f77880b1b50 2024-04-05T22:29:19.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.714+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778040b60 con 0x7f77880b1b50 2024-04-05T22:29:19.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.714+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778815c010 con 0x7f77880b1b50 2024-04-05T22:29:19.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.715+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778804e160 con 0x7f77880b1b50 2024-04-05T22:29:19.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.716+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f7778040cc0 con 0x7f77880b1b50 2024-04-05T22:29:19.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.717+0000 7f777fe00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 0x7f776406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:19.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.717+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f77780ac940 con 0x7f77880b1b50 2024-04-05T22:29:19.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.717+0000 7f777ea00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 0x7f776406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:19.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.718+0000 7f777ea00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 0x7f776406d660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad340 tx=0x7f7770008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:19.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.721+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7778077760 con 0x7f77880b1b50 2024-04-05T22:29:19.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:19.986+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7f77880b2d30 con 0x7f77880b1b50 2024-04-05T22:29:20.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.588+0000 7f777fe00640 1 -- 172.21.15.73:0/2700800192 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v433) v1 ==== 154+0+0 (secure 0 0 0) 0x7f7778081020 con 0x7f77880b1b50 2024-04-05T22:29:20.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.591+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 msgr2=0x7f776406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.591+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 0x7f776406d660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad340 tx=0x7f7770008040 comp rx=0 tx=0).stop 2024-04-05T22:29:20.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.591+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 msgr2=0x7f77880ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.591+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7778009d90 tx=0x7f7778004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:20.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 shutdown_connections 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f776406b1a0 0x7f776406d660 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77880b2480 0x7f77881f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77880b20a0 0x7f77881f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 --2- 172.21.15.73:0/2700800192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77880b1b50 0x7f77880ad570 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 >> 172.21.15.73:0/2700800192 conn(0x7f77880b06c0 msgr2=0x7f7788070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 shutdown_connections 2024-04-05T22:29:20.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.592+0000 7f778cc00640 1 -- 172.21.15.73:0/2700800192 wait complete. 2024-04-05T22:29:20.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:29:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.827+0000 7f7323200640 1 Processor -- start 2024-04-05T22:29:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.827+0000 7f7323200640 1 -- start start 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7323200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415a850 0x7f732415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7323200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f732415e580 0x7f732406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7323200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732406bc50 con 0x7f732415e9a0 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7323200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732415f530 con 0x7f732415e580 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7323200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732415ef70 con 0x7f732415a850 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7322800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f732415e9a0 0x7f73240b26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49924/0 (socket says 172.21.15.73:49924) 2024-04-05T22:29:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7322800640 1 -- 172.21.15.73:0/785440971 learned_addr learned my addr 172.21.15.73:0/785440971 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7321400640 1 --2- 172.21.15.73:0/785440971 >> v2:172.21.15.73:3300/0 conn(0x7f732415e580 0x7f732406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7321e00640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415a850 0x7f732415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.829+0000 7f7320a00640 1 -- 172.21.15.73:0/785440971 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187242020 0 0) 0x7f732406bc50 con 0x7f732415e9a0 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.830+0000 7f7320a00640 1 -- 172.21.15.73:0/785440971 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7308003520 con 0x7f732415e9a0 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.830+0000 7f7321400640 1 -- 172.21.15.73:0/785440971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415a850 msgr2=0x7f732415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.830+0000 7f7321400640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415a850 0x7f732415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.830+0000 7f7321400640 1 -- 172.21.15.73:0/785440971 >> v1:172.21.15.73:6789/0 conn(0x7f732415e9a0 legacy=0x7f73240b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.830+0000 7f7321400640 1 -- 172.21.15.73:0/785440971 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732415f250 con 0x7f732415e580 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7321400640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f732406ff30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f73100097a0 tx=0x7f731002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd1b9b944ac6e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7320a00640 1 -- 172.21.15.73:0/785440971 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731002d040 con 0x7f732415e580 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7320a00640 1 -- 172.21.15.73:0/785440971 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310031810 con 0x7f732415e580 2024-04-05T22:29:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7320a00640 1 -- 172.21.15.73:0/785440971 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310031b80 con 0x7f732415e580 2024-04-05T22:29:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7323200640 1 -- 172.21.15.73:0/785440971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 msgr2=0x7f732406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.831+0000 7f7323200640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f732406ff30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f73100097a0 tx=0x7f731002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 -- 172.21.15.73:0/785440971 shutdown_connections 2024-04-05T22:29:20.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f732406ff30 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 --2- 172.21.15.73:0/785440971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415a850 0x7f732415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 -- 172.21.15.73:0/785440971 >> 172.21.15.73:0/785440971 conn(0x7f7324069cb0 msgr2=0x7f732406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 -- 172.21.15.73:0/785440971 shutdown_connections 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.832+0000 7f7323200640 1 -- 172.21.15.73:0/785440971 wait complete. 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.833+0000 7f7323200640 1 Processor -- start 2024-04-05T22:29:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.834+0000 7f7323200640 1 -- start start 2024-04-05T22:29:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732415a850 0x7f7324143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415e9a0 0x7f7324154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324143660 con 0x7f732415e580 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240ce2a0 con 0x7f732415a850 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7323200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732415d290 con 0x7f732415e9a0 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47866/0 (socket says 172.21.15.73:47866) 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 -- 172.21.15.73:0/3106172116 learned_addr learned my addr 172.21.15.73:0/3106172116 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321e00640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732415a850 0x7f7324143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 -- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415e9a0 msgr2=0x7f7324154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7322800640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415e9a0 0x7f7324154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415e9a0 0x7f7324154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 -- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732415a850 msgr2=0x7f7324143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732415a850 0x7f7324143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:20.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.835+0000 7f7321400640 1 -- 172.21.15.73:0/3106172116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310009400 con 0x7f732415e580 2024-04-05T22:29:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f7321400640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f731002b8f0 tx=0x7f73100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310006e10 con 0x7f732415e580 2024-04-05T22:29:20.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731003ad60 con 0x7f732415e580 2024-04-05T22:29:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732415d570 con 0x7f732415e580 2024-04-05T22:29:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310044610 con 0x7f732415e580 2024-04-05T22:29:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.836+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732415db30 con 0x7f732415e580 2024-04-05T22:29:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.837+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7324157e40 con 0x7f732415e580 2024-04-05T22:29:20.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.838+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f731004d450 con 0x7f732415e580 2024-04-05T22:29:20.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.839+0000 7f731a000640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 0x7f730406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:20.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.839+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f73100abe20 con 0x7f732415e580 2024-04-05T22:29:20.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.840+0000 7f7321e00640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 0x7f730406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:20.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.841+0000 7f7321e00640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 0x7f730406d660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7314006fd0 tx=0x7f7314008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:20.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:20.844+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7324157e40 con 0x7f732415e580 2024-04-05T22:29:21.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.035+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7324157e40 con 0x7f732415e580 2024-04-05T22:29:21.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.041+0000 7f731a000640 1 -- 172.21.15.73:0/3106172116 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7324157e40 con 0x7f732415e580 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 msgr2=0x7f730406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 0x7f730406d660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7314006fd0 tx=0x7f7314008040 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 msgr2=0x7f73240cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f731002b8f0 tx=0x7f73100046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 shutdown_connections 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f730406b1a0 0x7f730406d660 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f732415e9a0 0x7f7324154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f732415e580 0x7f73240cdd60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 --2- 172.21.15.73:0/3106172116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732415a850 0x7f7324143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.043+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 >> 172.21.15.73:0/3106172116 conn(0x7f7324069cb0 msgr2=0x7f732406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:21.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.044+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 shutdown_connections 2024-04-05T22:29:21.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.044+0000 7f7323200640 1 -- 172.21.15.73:0/3106172116 wait complete. 2024-04-05T22:29:21.064 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:29:21.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:21.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:21.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:29:21.295 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:21.295 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:21.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:29:21.352 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:21.352 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:21.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:29:21.409 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:21.409 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:21.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:29:21.465 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:21.465 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:21.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:29:21.522 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:29:21.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:21.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.738+0000 7f1992c00640 1 Processor -- start 2024-04-05T22:29:21.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.738+0000 7f1992c00640 1 -- start start 2024-04-05T22:29:21.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f1992c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f198c06cb80 0x7f198c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f1992c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c15e850 0x7f198c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f1992c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c06f5b0 con 0x7f198c1587d0 2024-04-05T22:29:21.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f1992c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c06d500 con 0x7f198c06cb80 2024-04-05T22:29:21.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f1992c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c15f220 con 0x7f198c15e850 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f198c06cb80 0x7f198c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f198c06cb80 0x7f198c06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47874/0 (socket says 172.21.15.73:47874) 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 -- 172.21.15.73:0/994458727 learned_addr learned my addr 172.21.15.73:0/994458727 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 -- 172.21.15.73:0/994458727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c15e850 msgr2=0x7f198c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 --2- 172.21.15.73:0/994458727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c15e850 0x7f198c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 -- 172.21.15.73:0/994458727 >> v1:172.21.15.157:6789/0 conn(0x7f198c1587d0 legacy=0x7f198c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:21.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.740+0000 7f198b400640 1 -- 172.21.15.73:0/994458727 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c070410 con 0x7f198c06cb80 2024-04-05T22:29:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.741+0000 7f198b400640 1 --2- 172.21.15.73:0/994458727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c06cb80 0x7f198c06cfc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f19780098a0 tx=0x7f197802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568594fa5dda9f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f198aa00640 1 -- 172.21.15.73:0/994458727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978031640 con 0x7f198c06cb80 2024-04-05T22:29:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f198aa00640 1 -- 172.21.15.73:0/994458727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19780317a0 con 0x7f198c06cb80 2024-04-05T22:29:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f198aa00640 1 -- 172.21.15.73:0/994458727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978031b30 con 0x7f198c06cb80 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 -- 172.21.15.73:0/994458727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c06cb80 msgr2=0x7f198c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 --2- 172.21.15.73:0/994458727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c06cb80 0x7f198c06cfc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f19780098a0 tx=0x7f197802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 -- 172.21.15.73:0/994458727 shutdown_connections 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 --2- 172.21.15.73:0/994458727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c15e850 0x7f198c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 --2- 172.21.15.73:0/994458727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c06cb80 0x7f198c06cfc0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 -- 172.21.15.73:0/994458727 >> 172.21.15.73:0/994458727 conn(0x7f198c0ce8e0 msgr2=0x7f198c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 -- 172.21.15.73:0/994458727 shutdown_connections 2024-04-05T22:29:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.742+0000 7f1992c00640 1 -- 172.21.15.73:0/994458727 wait complete. 2024-04-05T22:29:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.744+0000 7f1992c00640 1 Processor -- start 2024-04-05T22:29:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.744+0000 7f1992c00640 1 -- start start 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 0x7f198c1439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c1587d0 0x7f198c146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c143ee0 con 0x7f198c15e850 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1444f0 con 0x7f198c06cb80 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.745+0000 7f1992c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1441f0 con 0x7f198c1587d0 2024-04-05T22:29:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47876/0 (socket says 172.21.15.73:47876) 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 -- 172.21.15.73:0/51646320 learned_addr learned my addr 172.21.15.73:0/51646320 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f198b400640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c1587d0 0x7f198c146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f198be00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 0x7f198c1439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 -- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c1587d0 msgr2=0x7f198c146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c1587d0 0x7f198c146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 -- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 msgr2=0x7f198c1439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 0x7f198c1439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f1990c00640 1 -- 172.21.15.73:0/51646320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780094d0 con 0x7f198c15e850 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.746+0000 7f198be00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 0x7f198c1439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1990c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f198400b9a0 tx=0x7f198400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400cba0 con 0x7f198c15e850 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400cd20 con 0x7f198c15e850 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984014680 con 0x7f198c15e850 2024-04-05T22:29:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c146aa0 con 0x7f198c15e850 2024-04-05T22:29:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.747+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c147060 con 0x7f198c15e850 2024-04-05T22:29:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.748+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c066af0 con 0x7f198c15e850 2024-04-05T22:29:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.749+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f198401d3b0 con 0x7f198c15e850 2024-04-05T22:29:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.750+0000 7f1983e00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 0x7f196c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.750+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1984089a90 con 0x7f198c15e850 2024-04-05T22:29:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.750+0000 7f198be00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 0x7f196c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.752+0000 7f198be00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 0x7f196c06d660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f197c007fd0 tx=0x7f197c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.754+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f19840546c0 con 0x7f198c15e850 2024-04-05T22:29:21.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.987+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c068160 con 0x7f198c15e850 2024-04-05T22:29:21.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.988+0000 7f1983e00640 1 -- 172.21.15.73:0/51646320 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f1984019020 con 0x7f198c15e850 2024-04-05T22:29:21.988 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:21.988 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5920,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3418052291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3418052291},{"type":"v1","addr":"172.21.15.157:6833","nonce":3418052291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3318349280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3318349280},{"type":"v1","addr":"172.21.15.157:6835","nonce":3318349280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1964372064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1964372064},{"type":"v1","addr":"172.21.15.73:6833","nonce":1964372064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/100765240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":100765240},{"type":"v1","addr":"172.21.15.73:6835","nonce":100765240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2024-04-05T22:29:21.989 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2024-04-05T22:29:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 msgr2=0x7f196c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 0x7f196c06d660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f197c007fd0 tx=0x7f197c009040 comp rx=0 tx=0).stop 2024-04-05T22:29:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 msgr2=0x7f198c15a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f198400b9a0 tx=0x7f198400be60 comp rx=0 tx=0).stop 2024-04-05T22:29:21.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 shutdown_connections 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f196c06b1a0 0x7f196c06d660 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f198c15e850 0x7f198c15a240 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f198c1587d0 0x7f198c146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 --2- 172.21.15.73:0/51646320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c06cb80 0x7f198c1439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 >> 172.21.15.73:0/51646320 conn(0x7f198c0ce8e0 msgr2=0x7f198c158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 shutdown_connections 2024-04-05T22:29:21.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:21.991+0000 7f1992c00640 1 -- 172.21.15.73:0/51646320 wait complete. 2024-04-05T22:29:22.010 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:29:22.010 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:29:22.010 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:29:22.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:29:22.011 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:29:22.011 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:29:22.011 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:29:22.011 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:29:22.011 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:29:22.011 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:29:22.012 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:29:22.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:29:22.012 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:29:22.012 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:29:22.012 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:29:22.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:29:22.028 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:29:22.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:29:22.029 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:29:22.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:29:22.032 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:29:22.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:29:22.033 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:29:22.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:29:22.075 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:29:22.075 DEBUG:teuthology.parallel:result is None 2024-04-05T22:29:22.080 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:29:22.080 DEBUG:teuthology.parallel:result is None 2024-04-05T22:29:22.081 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:29:22.082 DEBUG:teuthology.parallel:result is None 2024-04-05T22:29:22.085 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:29:22.086 DEBUG:teuthology.parallel:result is None 2024-04-05T22:29:22.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:29:22.227 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:29:22.229 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:29:22.246 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:29:22.246 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:29:22.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.277+0000 7fe642200640 1 Processor -- start 2024-04-05T22:29:22.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.277+0000 7fe642200640 1 -- start start 2024-04-05T22:29:22.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.278+0000 7fe642200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe63c1605d0 0x7fe63c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.278+0000 7fe642200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe63c0d09d0 0x7fe63c0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe642200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0af6d0 con 0x7fe63c0d1630 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe642200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0d1ae0 con 0x7fe63c1605d0 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe642200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0b22b0 con 0x7fe63c0d09d0 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe640e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe63c1605d0 0x7fe63c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe641800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe63c0d1630 0x7fe63c0cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49960/0 (socket says 172.21.15.73:49960) 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe641800640 1 -- 172.21.15.73:0/444296327 learned_addr learned my addr 172.21.15.73:0/444296327 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe63be00640 1 --2- 172.21.15.73:0/444296327 >> v2:172.21.15.157:3300/0 conn(0x7fe63c0d09d0 0x7fe63c0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe640e00640 1 -- 172.21.15.73:0/444296327 >> v2:172.21.15.157:3300/0 conn(0x7fe63c0d09d0 msgr2=0x7fe63c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe640e00640 1 --2- 172.21.15.73:0/444296327 >> v2:172.21.15.157:3300/0 conn(0x7fe63c0d09d0 0x7fe63c0d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe640e00640 1 -- 172.21.15.73:0/444296327 >> v1:172.21.15.73:6789/0 conn(0x7fe63c0d1630 legacy=0x7fe63c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe640e00640 1 -- 172.21.15.73:0/444296327 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0b2590 con 0x7fe63c1605d0 2024-04-05T22:29:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.279+0000 7fe63b400640 1 -- 172.21.15.73:0/444296327 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609311611 0 0) 0x7fe63c0af6d0 con 0x7fe63c0d1630 2024-04-05T22:29:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.280+0000 7fe640e00640 1 --2- 172.21.15.73:0/444296327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c1605d0 0x7fe63c0d3630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe63c08ba50 tx=0x7fe63002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5788c9384cad40f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe63b400640 1 -- 172.21.15.73:0/444296327 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630030610 con 0x7fe63c1605d0 2024-04-05T22:29:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe63b400640 1 -- 172.21.15.73:0/444296327 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe630030770 con 0x7fe63c1605d0 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe63b400640 1 -- 172.21.15.73:0/444296327 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630030b00 con 0x7fe63c1605d0 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 -- 172.21.15.73:0/444296327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c1605d0 msgr2=0x7fe63c0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 --2- 172.21.15.73:0/444296327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c1605d0 0x7fe63c0d3630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe63c08ba50 tx=0x7fe63002b960 comp rx=0 tx=0).stop 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 -- 172.21.15.73:0/444296327 shutdown_connections 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 --2- 172.21.15.73:0/444296327 >> v2:172.21.15.157:3300/0 conn(0x7fe63c0d09d0 0x7fe63c0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 --2- 172.21.15.73:0/444296327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c1605d0 0x7fe63c0d3630 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 -- 172.21.15.73:0/444296327 >> 172.21.15.73:0/444296327 conn(0x7fe63c0d2b10 msgr2=0x7fe63c0d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 -- 172.21.15.73:0/444296327 shutdown_connections 2024-04-05T22:29:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.281+0000 7fe642200640 1 -- 172.21.15.73:0/444296327 wait complete. 2024-04-05T22:29:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.283+0000 7fe642200640 1 Processor -- start 2024-04-05T22:29:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.283+0000 7fe642200640 1 -- start start 2024-04-05T22:29:22.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 0x7fe63c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 0x7fe63c0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1f23d0 con 0x7fe63c0d1630 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0c9580 con 0x7fe63c0d09d0 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe642200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0c8780 con 0x7fe63c1605d0 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe63be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe63be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47932/0 (socket says 172.21.15.73:47932) 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.284+0000 7fe63be00640 1 -- 172.21.15.73:0/1625326480 learned_addr learned my addr 172.21.15.73:0/1625326480 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe641800640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 0x7fe63c0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 -- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 msgr2=0x7fe63c0af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe640e00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 0x7fe63c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 0x7fe63c0af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 -- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 msgr2=0x7fe63c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 0x7fe63c0b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 -- 172.21.15.73:0/1625326480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe630009400 con 0x7fe63c0d1630 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe640e00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 0x7fe63c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe641800640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 0x7fe63c0af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe63be00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d830 tx=0x7fe62800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800b6c0 con 0x7fe63c0d1630 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62800b820 con 0x7fe63c0d1630 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c0c8a00 con 0x7fe63c0d1630 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.285+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800bbb0 con 0x7fe63c0d1630 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.286+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c0c8fc0 con 0x7fe63c0d1630 2024-04-05T22:29:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.287+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c04e160 con 0x7fe63c0d1630 2024-04-05T22:29:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.288+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fe62801e460 con 0x7fe63c0d1630 2024-04-05T22:29:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.288+0000 7fe638c00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 0x7fe61c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.289+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe62808a210 con 0x7fe63c0d1630 2024-04-05T22:29:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.289+0000 7fe640e00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 0x7fe61c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.290+0000 7fe640e00640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 0x7fe61c06d690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe630036000 tx=0x7fe630008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.293+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe628054ed0 con 0x7fe63c0d1630 2024-04-05T22:29:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:22.497+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe63c0d0ed0 con 0x7fe63c0d1630 2024-04-05T22:29:23.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.274+0000 7fe638c00640 1 -- 172.21.15.73:0/1625326480 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe6280584f0 con 0x7fe63c0d1630 2024-04-05T22:29:23.275 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 msgr2=0x7fe61c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 0x7fe61c06d690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe630036000 tx=0x7fe630008030 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 msgr2=0x7fe63c0c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d830 tx=0x7fe62800dcf0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 shutdown_connections 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe61c06b1d0 0x7fe61c06d690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe63c1605d0 0x7fe63c0af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe63c0d1630 0x7fe63c0c6b70 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 --2- 172.21.15.73:0/1625326480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe63c0d09d0 0x7fe63c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.278+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 >> 172.21.15.73:0/1625326480 conn(0x7fe63c0d2b10 msgr2=0x7fe63c0b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:23.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.279+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 shutdown_connections 2024-04-05T22:29:23.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.279+0000 7fe642200640 1 -- 172.21.15.73:0/1625326480 wait complete. 2024-04-05T22:29:23.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:29:23.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.508+0000 7fa800a00640 1 Processor -- start 2024-04-05T22:29:23.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.508+0000 7fa800a00640 1 -- start start 2024-04-05T22:29:23.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa800a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7fc0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa800a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc1627f0 con 0x7fa7fc162370 2024-04-05T22:29:23.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa800a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc165cf0 con 0x7fa7fc161f90 2024-04-05T22:29:23.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa800a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc158790 con 0x7fa7fc0b2ba0 2024-04-05T22:29:23.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa7f1400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa7fc162370 0x7fa7fc0b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49964/0 (socket says 172.21.15.73:49964) 2024-04-05T22:29:23.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.510+0000 7fa7f1400640 1 -- 172.21.15.73:0/652985237 learned_addr learned my addr 172.21.15.73:0/652985237 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:23.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312560443 0 0) 0x7fa7e4001110 con 0x7fa7fc162370 2024-04-05T22:29:23.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e0003520 con 0x7fa7fc162370 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7fa800640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7fc0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340530376 0 0) 0x7fa7e0003520 con 0x7fa7fc162370 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 msgr2=0x7fa7fc0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7fc0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 >> v1:172.21.15.157:6789/0 conn(0x7fa7fc161f90 legacy=0x7fa7fc164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc165730 con 0x7fa7fc162370 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa7e4001800 con 0x7fa7fc162370 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 >> v1:172.21.15.73:6789/0 conn(0x7fa7fc162370 legacy=0x7fa7fc0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e00039a0 0x7fa7e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.511+0000 7fa7f9400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7e0004d00 0x7fa7e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc158790 con 0x7fa7e0004d00 2024-04-05T22:29:23.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e000bb10 con 0x7fa7e00039a0 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e000bca0 con 0x7fa7fc0b2ba0 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7e0004d00 0x7fa7e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9e00640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e00039a0 0x7fa7e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f1400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 -- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 msgr2=0x7fa7e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 -- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e00039a0 msgr2=0x7fa7e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e00039a0 0x7fa7e0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7fa800640 1 -- 172.21.15.73:0/652985237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0003520 con 0x7fa7e0004d00 2024-04-05T22:29:23.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f9e00640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e00039a0 0x7fa7e0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:23.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.512+0000 7fa7f1400640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7fc0b2ba0 0x7fa7e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:23.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.513+0000 7fa7fa800640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7e0004d00 0x7fa7e0005180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400ba60 tx=0x7fa7e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4380a28257a93f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.513+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4032620 con 0x7fa7e0004d00 2024-04-05T22:29:23.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.513+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f4032780 con 0x7fa7e0004d00 2024-04-05T22:29:23.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa7f9400640 1 -- 172.21.15.73:0/652985237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4032b10 con 0x7fa7e0004d00 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 -- 172.21.15.73:0/652985237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7e0004d00 msgr2=0x7fa7e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 --2- 172.21.15.73:0/652985237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7e0004d00 0x7fa7e0005180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400ba60 tx=0x7fa7e00057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa7fa800640 1 -- 172.21.15.73:0/652985237 reap_dead start 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 -- 172.21.15.73:0/652985237 shutdown_connections 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 -- 172.21.15.73:0/652985237 >> 172.21.15.73:0/652985237 conn(0x7fa7fc0ce8e0 msgr2=0x7fa7fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 -- 172.21.15.73:0/652985237 shutdown_connections 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.514+0000 7fa800a00640 1 -- 172.21.15.73:0/652985237 wait complete. 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.516+0000 7fa800a00640 1 Processor -- start 2024-04-05T22:29:23.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.516+0000 7fa800a00640 1 -- start start 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7e0004d00 0x7fa7fc14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 0x7fa7fc06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc06d2e0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc159da0 con 0x7fa7fc06c920 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa800a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc15a080 con 0x7fa7e0004d00 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa7f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa7f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47948/0 (socket says 172.21.15.73:47948) 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.517+0000 7fa7f1400640 1 -- 172.21.15.73:0/4038932467 learned_addr learned my addr 172.21.15.73:0/4038932467 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f9e00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7e0004d00 0x7fa7fc14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7fa800640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 0x7fa7fc06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 -- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7e0004d00 msgr2=0x7fa7fc14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7e0004d00 0x7fa7fc14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 -- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 msgr2=0x7fa7fc06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 0x7fa7fc06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 -- 172.21.15.73:0/4038932467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc14daa0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7fa800640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 0x7fa7fc06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f1400640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400d860 tx=0x7fa7e400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40074e0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e4007640 con 0x7fa7fc1592e0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40049e0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc14dd80 con 0x7fa7fc1592e0 2024-04-05T22:29:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.518+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc14cd90 con 0x7fa7fc1592e0 2024-04-05T22:29:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.519+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc066af0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.521+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa7e4004b40 con 0x7fa7fc1592e0 2024-04-05T22:29:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.521+0000 7fa7f2a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 0x7fa7d806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.521+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa7e4089220 con 0x7fa7fc1592e0 2024-04-05T22:29:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.522+0000 7fa7f9e00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 0x7fa7d806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.523+0000 7fa7f9e00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 0x7fa7d806d660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc1fa370 tx=0x7fa7ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.526+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa7e4053ed0 con 0x7fa7fc1592e0 2024-04-05T22:29:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.769+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa7fc14d900 con 0x7fa7fc1592e0 2024-04-05T22:29:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.770+0000 7fa7f2a00640 1 -- 172.21.15.73:0/4038932467 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa7fc14d900 con 0x7fa7fc1592e0 2024-04-05T22:29:23.770 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:23.770 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:29:23.770 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:29:23.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:29:23.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.773 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:29:23.775 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.776 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.777 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.779 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.781 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:29:23.783 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 msgr2=0x7fa7d806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 0x7fa7d806d660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc1fa370 tx=0x7fa7ec009040 comp rx=0 tx=0).stop 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 msgr2=0x7fa7fc159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400d860 tx=0x7fa7e400dd20 comp rx=0 tx=0).stop 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 shutdown_connections 2024-04-05T22:29:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa7d806b1a0 0x7fa7d806d660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7fc06c920 0x7fa7fc06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7fc1592e0 0x7fa7fc159760 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 --2- 172.21.15.73:0/4038932467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7e0004d00 0x7fa7fc14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.772+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 >> 172.21.15.73:0/4038932467 conn(0x7fa7fc0ce8e0 msgr2=0x7fa7fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.773+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 shutdown_connections 2024-04-05T22:29:23.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.773+0000 7fa800a00640 1 -- 172.21.15.73:0/4038932467 wait complete. 2024-04-05T22:29:23.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:23.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.980+0000 7f4eb3200640 1 Processor -- start 2024-04-05T22:29:23.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.981+0000 7f4eb3200640 1 -- start start 2024-04-05T22:29:23.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.982+0000 7f4eb3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4eac1587d0 0x7f4eac0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.982+0000 7f4eb3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0ccd20 0x7f4eac06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.982+0000 7f4eb3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac06f5b0 con 0x7f4eac06d0c0 2024-04-05T22:29:23.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.982+0000 7f4eb3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac070410 con 0x7f4eac1587d0 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.982+0000 7f4eb3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac158bb0 con 0x7f4eac0ccd20 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eabe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4eac1587d0 0x7f4eac0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eb1200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4eac06d0c0 0x7f4eac15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49978/0 (socket says 172.21.15.73:49978) 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eb1200640 1 -- 172.21.15.73:0/77227374 learned_addr learned my addr 172.21.15.73:0/77227374 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4ea3400640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0ccd20 0x7f4eac06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eabe00640 1 -- 172.21.15.73:0/77227374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0ccd20 msgr2=0x7f4eac06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eabe00640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0ccd20 0x7f4eac06cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eabe00640 1 -- 172.21.15.73:0/77227374 >> v1:172.21.15.73:6789/0 conn(0x7f4eac06d0c0 legacy=0x7f4eac15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eabe00640 1 -- 172.21.15.73:0/77227374 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac06f800 con 0x7f4eac1587d0 2024-04-05T22:29:23.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.983+0000 7f4eab400640 1 -- 172.21.15.73:0/77227374 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124322673 0 0) 0x7f4eac06f5b0 con 0x7f4eac06d0c0 2024-04-05T22:29:23.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.984+0000 7f4eabe00640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac0cc7e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009d90 tx=0x7f4e9c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd21a8d865256dde server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eab400640 1 -- 172.21.15.73:0/77227374 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c02cc50 con 0x7f4eac1587d0 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eab400640 1 -- 172.21.15.73:0/77227374 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c02cdb0 con 0x7f4eac1587d0 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eab400640 1 -- 172.21.15.73:0/77227374 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c034660 con 0x7f4eac1587d0 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 -- 172.21.15.73:0/77227374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 msgr2=0x7f4eac0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac0cc7e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009d90 tx=0x7f4e9c02bea0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 -- 172.21.15.73:0/77227374 shutdown_connections 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0ccd20 0x7f4eac06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 --2- 172.21.15.73:0/77227374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac0cc7e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 -- 172.21.15.73:0/77227374 >> 172.21.15.73:0/77227374 conn(0x7f4eac0ce8e0 msgr2=0x7f4eac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 -- 172.21.15.73:0/77227374 shutdown_connections 2024-04-05T22:29:23.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.985+0000 7f4eb3200640 1 -- 172.21.15.73:0/77227374 wait complete. 2024-04-05T22:29:23.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.987+0000 7f4eb3200640 1 Processor -- start 2024-04-05T22:29:23.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.988+0000 7f4eb3200640 1 -- start start 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac06d0c0 0x7f4eac0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0cd980 0x7f4eac0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0ce350 con 0x7f4eac1587d0 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0b2520 con 0x7f4eac06d0c0 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0b2800 con 0x7f4eac0cd980 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47972/0 (socket says 172.21.15.73:47972) 2024-04-05T22:29:23.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 -- 172.21.15.73:0/2127462975 learned_addr learned my addr 172.21.15.73:0/2127462975 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eb1200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0cd980 0x7f4eac0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4eabe00640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac06d0c0 0x7f4eac0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 -- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0cd980 msgr2=0x7f4eac0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0cd980 0x7f4eac0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 -- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac06d0c0 msgr2=0x7f4eac0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac06d0c0 0x7f4eac0b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.989+0000 7f4ea3400640 1 -- 172.21.15.73:0/2127462975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009a70 con 0x7f4eac1587d0 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4ea3400640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400d800 tx=0x7f4e9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400b6c0 con 0x7f4eac1587d0 2024-04-05T22:29:23.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400b820 con 0x7f4eac1587d0 2024-04-05T22:29:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400bbb0 con 0x7f4eac1587d0 2024-04-05T22:29:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac0b2ae0 con 0x7f4eac1587d0 2024-04-05T22:29:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.990+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac0b30a0 con 0x7f4eac1587d0 2024-04-05T22:29:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.991+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac0b3260 con 0x7f4eac1587d0 2024-04-05T22:29:23.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.992+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f4e9400bd10 con 0x7f4eac1587d0 2024-04-05T22:29:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.993+0000 7f4ea8c00640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 0x7f4e8806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.993+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e9408a090 con 0x7f4eac1587d0 2024-04-05T22:29:23.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.994+0000 7f4eabe00640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 0x7f4e8806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:23.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.995+0000 7f4eabe00640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 0x7f4e8806d660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c036000 tx=0x7f4e9c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:23.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:23.997+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4e94054c80 con 0x7f4eac1587d0 2024-04-05T22:29:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.192+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eac066af0 con 0x7f4eac1587d0 2024-04-05T22:29:24.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.194+0000 7f4ea8c00640 1 -- 172.21.15.73:0/2127462975 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+12399 (secure 0 0 0) 0x7f4e940582a0 con 0x7f4eac1587d0 2024-04-05T22:29:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:24.195 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":132,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:23.267225+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 msgr2=0x7f4e8806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 0x7f4e8806d660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c036000 tx=0x7f4e9c030000 comp rx=0 tx=0).stop 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 msgr2=0x7f4eac1576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400d800 tx=0x7f4e9400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 shutdown_connections 2024-04-05T22:29:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4e8806b1a0 0x7f4e8806d660 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac0cd980 0x7f4eac0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4eac1587d0 0x7f4eac1576b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 --2- 172.21.15.73:0/2127462975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac06d0c0 0x7f4eac0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 >> 172.21.15.73:0/2127462975 conn(0x7f4eac0ce8e0 msgr2=0x7f4eac0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 shutdown_connections 2024-04-05T22:29:24.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.197+0000 7f4eb3200640 1 -- 172.21.15.73:0/2127462975 wait complete. 2024-04-05T22:29:24.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:24.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:24.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.438+0000 7f0053600640 1 Processor -- start 2024-04-05T22:29:24.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.439+0000 7f0053600640 1 -- start start 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0053600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f004c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0053600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f004c0b1950 0x7f004c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0053600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c15f260 con 0x7f004c165310 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0053600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c1657c0 con 0x7f004c0b1950 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0053600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c070d20 con 0x7f004c15ee80 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0051600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f004c165310 0x7f004c161850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49984/0 (socket says 172.21.15.73:49984) 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.440+0000 7f0051600640 1 -- 172.21.15.73:0/3938444812 learned_addr learned my addr 172.21.15.73:0/3938444812 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f0050c00640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f004c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004be00640 1 --2- 172.21.15.73:0/3938444812 >> v2:172.21.15.157:3300/0 conn(0x7f004c0b1950 0x7f004c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251707763 0 0) 0x7f0044001490 con 0x7f004c165310 2024-04-05T22:29:24.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0030003520 con 0x7f004c165310 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 143502703 0 0) 0x7f0030003520 con 0x7f004c165310 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 msgr2=0x7f004c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f004c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 >> v2:172.21.15.157:3300/0 conn(0x7f004c0b1950 msgr2=0x7f004c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 --2- 172.21.15.73:0/3938444812 >> v2:172.21.15.157:3300/0 conn(0x7f004c0b1950 0x7f004c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c06f5b0 con 0x7f004c165310 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0044001b80 con 0x7f004c165310 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 >> v1:172.21.15.73:6789/0 conn(0x7f004c165310 legacy=0x7f004c161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.441+0000 7f004b400640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0030003970 0x7f0030003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004b400640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f00300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004b400640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0030004d00 0x7f0030005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1657c0 con 0x7f0030004d00 2024-04-05T22:29:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003000b980 con 0x7f0030003970 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003000bac0 con 0x7f004c15ee80 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0030004d00 0x7f0030005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f004be00640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f00300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0050c00640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0030003970 0x7f0030003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 -- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 msgr2=0x7f00300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c15ee80 0x7f00300047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 -- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0030003970 msgr2=0x7f0030003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0030003970 0x7f0030003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.442+0000 7f0051600640 1 -- 172.21.15.73:0/3938444812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0030003520 con 0x7f0030004d00 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.443+0000 7f0051600640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0030004d00 0x7f0030005180 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f004400a830 tx=0x7f00300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab55df6b78295483 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044034630 con 0x7f0030004d00 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004402f070 con 0x7f0030004d00 2024-04-05T22:29:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f004b400640 1 -- 172.21.15.73:0/3938444812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440349f0 con 0x7f0030004d00 2024-04-05T22:29:24.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 -- 172.21.15.73:0/3938444812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0030004d00 msgr2=0x7f0030005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 --2- 172.21.15.73:0/3938444812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0030004d00 0x7f0030005180 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f004400a830 tx=0x7f00300057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0051600640 1 -- 172.21.15.73:0/3938444812 reap_dead start 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 -- 172.21.15.73:0/3938444812 shutdown_connections 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 -- 172.21.15.73:0/3938444812 >> 172.21.15.73:0/3938444812 conn(0x7f004c069cb0 msgr2=0x7f004c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 -- 172.21.15.73:0/3938444812 shutdown_connections 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.444+0000 7f0053600640 1 -- 172.21.15.73:0/3938444812 wait complete. 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.446+0000 7f0053600640 1 Processor -- start 2024-04-05T22:29:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.446+0000 7f0053600640 1 -- start start 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c0add70 0x7f004c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c0ae150 0x7f004c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0cda70 con 0x7f004c0cd510 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0ae660 con 0x7f004c0add70 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0053600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0ae940 con 0x7f004c0ae150 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0051600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0051600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47976/0 (socket says 172.21.15.73:47976) 2024-04-05T22:29:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.447+0000 7f0051600640 1 -- 172.21.15.73:0/1269653460 learned_addr learned my addr 172.21.15.73:0/1269653460 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0050c00640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c0add70 0x7f004c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f004be00640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c0ae150 0x7f004c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 -- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c0ae150 msgr2=0x7f004c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c0ae150 0x7f004c15cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 -- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c0add70 msgr2=0x7f004c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c0add70 0x7f004c0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 -- 172.21.15.73:0/1269653460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0aec20 con 0x7f004c0cd510 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0051600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f004400a0b0 tx=0x7f004c06fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004403f670 con 0x7f004c0cd510 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004403f7d0 con 0x7f004c0cd510 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.448+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c1485d0 con 0x7f004c0cd510 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.449+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004403fb40 con 0x7f004c0cd510 2024-04-05T22:29:24.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.449+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c0cb510 con 0x7f004c0cd510 2024-04-05T22:29:24.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.450+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c0aee10 con 0x7f004c0cd510 2024-04-05T22:29:24.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.451+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f0044031030 con 0x7f004c0cd510 2024-04-05T22:29:24.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.452+0000 7f0048c00640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 0x7f002c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.452+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00440ab9e0 con 0x7f004c0cd510 2024-04-05T22:29:24.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.452+0000 7f0050c00640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 0x7f002c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.454+0000 7f0050c00640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 0x7f002c06d690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f003c009a10 tx=0x7f003c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.456+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00440766b0 con 0x7f004c0cd510 2024-04-05T22:29:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.648+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f004c15f260 con 0x7f004c0cd510 2024-04-05T22:29:24.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.649+0000 7f0048c00640 1 -- 172.21.15.73:0/1269653460 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f004c15f260 con 0x7f004c0cd510 2024-04-05T22:29:24.650 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.652+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 msgr2=0x7f002c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.652+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 0x7f002c06d690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f003c009a10 tx=0x7f003c008040 comp rx=0 tx=0).stop 2024-04-05T22:29:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.652+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 msgr2=0x7f004c15d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.652+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f004400a0b0 tx=0x7f004c06fae0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 shutdown_connections 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f002c06b1d0 0x7f002c06d690 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f004c0cd510 0x7f004c15d110 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f004c0ae150 0x7f004c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 --2- 172.21.15.73:0/1269653460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c0add70 0x7f004c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 >> 172.21.15.73:0/1269653460 conn(0x7f004c069cb0 msgr2=0x7f004c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:24.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 shutdown_connections 2024-04-05T22:29:24.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.653+0000 7f0053600640 1 -- 172.21.15.73:0/1269653460 wait complete. 2024-04-05T22:29:24.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:29:24.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.884+0000 7fa019600640 1 Processor -- start 2024-04-05T22:29:24.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.884+0000 7fa019600640 1 -- start start 2024-04-05T22:29:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 0x7fa014071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 0x7fa0140b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa01406e920 0x7fa0140b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01415d500 con 0x7fa01406e920 2024-04-05T22:29:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140b4ac0 con 0x7fa01406ede0 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa019600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01406bbc0 con 0x7fa01415d120 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa013400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa01406e920 0x7fa0140b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa013400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa01406e920 0x7fa0140b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47990/0 (socket says 172.21.15.73:47990) 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.886+0000 7fa013400640 1 -- 172.21.15.73:0/1538372572 learned_addr learned my addr 172.21.15.73:0/1538372572 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa012000640 1 --2- 172.21.15.73:0/1538372572 >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 0x7fa0140b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa012a00640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 0x7fa014071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa013400640 1 -- 172.21.15.73:0/1538372572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 msgr2=0x7fa014071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa013400640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 0x7fa014071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa013400640 1 -- 172.21.15.73:0/1538372572 >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 msgr2=0x7fa0140b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa013400640 1 --2- 172.21.15.73:0/1538372572 >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 0x7fa0140b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa013400640 1 -- 172.21.15.73:0/1538372572 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01406b8e0 con 0x7fa01406e920 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa012000640 1 --2- 172.21.15.73:0/1538372572 >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 0x7fa0140b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:24.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.887+0000 7fa012a00640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 0x7fa014071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:24.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.888+0000 7fa013400640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0140b4580 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa014067930 tx=0x7fa00c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32aa5901d0ca77d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.888+0000 7fa011600640 1 -- 172.21.15.73:0/1538372572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c031880 con 0x7fa01406e920 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.888+0000 7fa011600640 1 -- 172.21.15.73:0/1538372572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0319e0 con 0x7fa01406e920 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.888+0000 7fa011600640 1 -- 172.21.15.73:0/1538372572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c031d70 con 0x7fa01406e920 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 -- 172.21.15.73:0/1538372572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 msgr2=0x7fa0140b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0140b4580 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa014067930 tx=0x7fa00c02b920 comp rx=0 tx=0).stop 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 -- 172.21.15.73:0/1538372572 shutdown_connections 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0140b4580 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 --2- 172.21.15.73:0/1538372572 >> v2:172.21.15.157:3300/0 conn(0x7fa01406ede0 0x7fa0140b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 --2- 172.21.15.73:0/1538372572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415d120 0x7fa014071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 -- 172.21.15.73:0/1538372572 >> 172.21.15.73:0/1538372572 conn(0x7fa01406adc0 msgr2=0x7fa01406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 -- 172.21.15.73:0/1538372572 shutdown_connections 2024-04-05T22:29:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.889+0000 7fa019600640 1 -- 172.21.15.73:0/1538372572 wait complete. 2024-04-05T22:29:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.891+0000 7fa019600640 1 Processor -- start 2024-04-05T22:29:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.891+0000 7fa019600640 1 -- start start 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa01415d120 0x7fa01415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415e590 0x7fa0140b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014154320 con 0x7fa01406e920 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415f740 con 0x7fa01415d120 2024-04-05T22:29:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa019600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415b1e0 con 0x7fa01415e590 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa012a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa012a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48006/0 (socket says 172.21.15.73:48006) 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.892+0000 7fa012a00640 1 -- 172.21.15.73:0/2981630770 learned_addr learned my addr 172.21.15.73:0/2981630770 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa013400640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415e590 0x7fa0140b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 -- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415e590 msgr2=0x7fa0140b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415e590 0x7fa0140b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 -- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa01415d120 msgr2=0x7fa01415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012000640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa01415d120 0x7fa01415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa01415d120 0x7fa01415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 -- 172.21.15.73:0/2981630770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0094d0 con 0x7fa01406e920 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa012a00640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa00400ce00 tx=0x7fa00400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400f890 con 0x7fa01406e920 2024-04-05T22:29:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00400f9f0 con 0x7fa01406e920 2024-04-05T22:29:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400fdc0 con 0x7fa01406e920 2024-04-05T22:29:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01415ebc0 con 0x7fa01406e920 2024-04-05T22:29:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.893+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01415f180 con 0x7fa01406e920 2024-04-05T22:29:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.895+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014066af0 con 0x7fa01406e920 2024-04-05T22:29:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.896+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa00400b040 con 0x7fa01406e920 2024-04-05T22:29:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.896+0000 7fa00aa00640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 0x7f9ff406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.897+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa00408a610 con 0x7fa01406e920 2024-04-05T22:29:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.897+0000 7fa012000640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 0x7f9ff406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.898+0000 7fa012000640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 0x7f9ff406d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa000006fd0 tx=0x7fa000008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:24.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:24.901+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa004055180 con 0x7fa01406e920 2024-04-05T22:29:25.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.095+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa014154460 con 0x7fa01406e920 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.096+0000 7fa00aa00640 1 -- 172.21.15.73:0/2981630770 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa00401d060 con 0x7fa01406e920 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:25.096 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:29:25.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 msgr2=0x7f9ff406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:25.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 0x7f9ff406d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa000006fd0 tx=0x7fa000008040 comp rx=0 tx=0).stop 2024-04-05T22:29:25.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 msgr2=0x7fa0141535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:25.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa00400ce00 tx=0x7fa00400a650 comp rx=0 tx=0).stop 2024-04-05T22:29:25.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 shutdown_connections 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9ff406b1a0 0x7f9ff406d660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa01415e590 0x7fa0140b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa01415d120 0x7fa01415e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.098+0000 7fa019600640 1 --2- 172.21.15.73:0/2981630770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa01406e920 0x7fa0141535d0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.099+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 >> 172.21.15.73:0/2981630770 conn(0x7fa01406adc0 msgr2=0x7fa01406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.099+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 shutdown_connections 2024-04-05T22:29:25.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.099+0000 7fa019600640 1 -- 172.21.15.73:0/2981630770 wait complete. 2024-04-05T22:29:25.118 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:29:25.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:29:25.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.321+0000 7f7da2200640 1 Processor -- start 2024-04-05T22:29:25.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.321+0000 7f7da2200640 1 -- start start 2024-04-05T22:29:25.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d9c15aa80 0x7f7d9c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0cda10 0x7f7d9c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da2200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0ce380 con 0x7f7d9c0714b0 2024-04-05T22:29:25.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0cd5f0 con 0x7f7d9c15aa80 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0ccc50 con 0x7f7d9c0cda10 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d9c15aa80 0x7f7d9c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d9c15aa80 0x7f7d9c06fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48016/0 (socket says 172.21.15.73:48016) 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.323+0000 7f7da0e00640 1 -- 172.21.15.73:0/1532135649 learned_addr learned my addr 172.21.15.73:0/1532135649 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.324+0000 7f7da1800640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0cda10 0x7f7d9c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.324+0000 7f7da0e00640 1 -- 172.21.15.73:0/1532135649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0cda10 msgr2=0x7f7d9c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:25.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.324+0000 7f7da0e00640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0cda10 0x7f7d9c0cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.324+0000 7f7da0e00640 1 -- 172.21.15.73:0/1532135649 >> v1:172.21.15.157:6789/0 conn(0x7f7d9c0714b0 legacy=0x7f7d9c06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.324+0000 7f7da0e00640 1 -- 172.21.15.73:0/1532135649 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0ccf30 con 0x7f7d9c15aa80 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.325+0000 7f7da0e00640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c06fea0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009e50 tx=0x7f7d8c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a902c715c6b04744 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.325+0000 7f7d9b400640 1 -- 172.21.15.73:0/1532135649 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c02c950 con 0x7f7d9c15aa80 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.325+0000 7f7d9b400640 1 -- 172.21.15.73:0/1532135649 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c02cab0 con 0x7f7d9c15aa80 2024-04-05T22:29:25.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.325+0000 7f7d9b400640 1 -- 172.21.15.73:0/1532135649 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c02ce40 con 0x7f7d9c15aa80 2024-04-05T22:29:25.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 -- 172.21.15.73:0/1532135649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c15aa80 msgr2=0x7f7d9c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c06fea0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009e50 tx=0x7f7d8c02bdb0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 -- 172.21.15.73:0/1532135649 shutdown_connections 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0cda10 0x7f7d9c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 --2- 172.21.15.73:0/1532135649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c06fea0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 -- 172.21.15.73:0/1532135649 >> 172.21.15.73:0/1532135649 conn(0x7f7d9c0b0360 msgr2=0x7f7d9c0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 -- 172.21.15.73:0/1532135649 shutdown_connections 2024-04-05T22:29:25.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.326+0000 7f7da2200640 1 -- 172.21.15.73:0/1532135649 wait complete. 2024-04-05T22:29:25.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.328+0000 7f7da2200640 1 Processor -- start 2024-04-05T22:29:25.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.329+0000 7f7da2200640 1 -- start start 2024-04-05T22:29:25.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0714b0 0x7f7d9c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c1f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c154330 con 0x7f7d9c0cda10 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b1800 con 0x7f7d9c15aa80 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7da2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b1aa0 con 0x7f7d9c0714b0 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48028/0 (socket says 172.21.15.73:48028) 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.330+0000 7f7d9be00640 1 -- 172.21.15.73:0/284449004 learned_addr learned my addr 172.21.15.73:0/284449004 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7da0e00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0714b0 0x7f7d9c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 -- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0714b0 msgr2=0x7f7d9c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0714b0 0x7f7d9c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7da1800640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c1f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 -- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 msgr2=0x7f7d9c1f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c1f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 -- 172.21.15.73:0/284449004 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c009b30 con 0x7f7d9c0cda10 2024-04-05T22:29:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7da1800640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d9be00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b3c0 tx=0x7f7d8800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88011830 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d88011990 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88011d80 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.331+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c0b1d80 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.332+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c0b2340 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.333+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c0a8c80 con 0x7f7d9c0cda10 2024-04-05T22:29:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.333+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f7d8801b460 con 0x7f7d9c0cda10 2024-04-05T22:29:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.334+0000 7f7d98c00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 0x7f7d7c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.334+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7d88089620 con 0x7f7d9c0cda10 2024-04-05T22:29:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.335+0000 7f7da0e00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 0x7f7d7c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:25.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.336+0000 7f7da0e00640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 0x7f7d7c06d660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c035000 tx=0x7f7d8c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:25.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.339+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7d880542a0 con 0x7f7d9c0cda10 2024-04-05T22:29:25.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:25.536+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7d9c04e160 con 0x7f7d9c0cda10 2024-04-05T22:29:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.076+0000 7f7d98c00640 1 -- 172.21.15.73:0/284449004 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7d880578c0 con 0x7f7d9c0cda10 2024-04-05T22:29:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:29:26.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.081+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 msgr2=0x7f7d7c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.081+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 0x7f7d7c06d660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c035000 tx=0x7f7d8c030000 comp rx=0 tx=0).stop 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 msgr2=0x7f7d9c0b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b3c0 tx=0x7f7d8800b880 comp rx=0 tx=0).stop 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 shutdown_connections 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7d7c06b1a0 0x7f7d7c06d660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d9c15aa80 0x7f7d9c1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d9c0cda10 0x7f7d9c0b2860 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 --2- 172.21.15.73:0/284449004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9c0714b0 0x7f7d9c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.082+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 >> 172.21.15.73:0/284449004 conn(0x7f7d9c0b0360 msgr2=0x7f7d9c0ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:26.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.083+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 shutdown_connections 2024-04-05T22:29:26.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.083+0000 7f7da2200640 1 -- 172.21.15.73:0/284449004 wait complete. 2024-04-05T22:29:26.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:29:26.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.310+0000 7f1a09600640 1 Processor -- start 2024-04-05T22:29:26.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.310+0000 7f1a09600640 1 -- start start 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a09600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a0415abe0 0x7f1a040712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a09600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a0406bc40 0x7f1a04164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a09600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0415afc0 con 0x7f1a0406c9a0 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a09600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0406f5b0 con 0x7f1a0415abe0 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a09600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0406f890 con 0x7f1a0406bc40 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a02a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a0415abe0 0x7f1a040712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a02a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a0415abe0 0x7f1a040712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34334/0 (socket says 172.21.15.73:34334) 2024-04-05T22:29:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.312+0000 7f1a02a00640 1 -- 172.21.15.73:0/2505478595 learned_addr learned my addr 172.21.15.73:0/2505478595 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.313+0000 7f1a03400640 1 --2- 172.21.15.73:0/2505478595 >> v2:172.21.15.157:3300/0 conn(0x7f1a0406bc40 0x7f1a04164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.313+0000 7f1a02a00640 1 -- 172.21.15.73:0/2505478595 >> v2:172.21.15.157:3300/0 conn(0x7f1a0406bc40 msgr2=0x7f1a04164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.313+0000 7f1a02a00640 1 --2- 172.21.15.73:0/2505478595 >> v2:172.21.15.157:3300/0 conn(0x7f1a0406bc40 0x7f1a04164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.313+0000 7f1a02a00640 1 -- 172.21.15.73:0/2505478595 >> v1:172.21.15.157:6789/0 conn(0x7f1a0406c9a0 legacy=0x7f1a0406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.313+0000 7f1a02a00640 1 -- 172.21.15.73:0/2505478595 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0406fb70 con 0x7f1a0415abe0 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.314+0000 7f1a02a00640 1 --2- 172.21.15.73:0/2505478595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0415abe0 0x7f1a040712e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1a0406c900 tx=0x7f19f402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2f2fa635097d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.314+0000 7f1a01600640 1 -- 172.21.15.73:0/2505478595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f402c960 con 0x7f1a0415abe0 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.314+0000 7f1a01600640 1 -- 172.21.15.73:0/2505478595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f402cac0 con 0x7f1a0415abe0 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.314+0000 7f1a01600640 1 -- 172.21.15.73:0/2505478595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f402ce50 con 0x7f1a0415abe0 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 -- 172.21.15.73:0/2505478595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0415abe0 msgr2=0x7f1a040712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 --2- 172.21.15.73:0/2505478595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0415abe0 0x7f1a040712e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1a0406c900 tx=0x7f19f402bdc0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 -- 172.21.15.73:0/2505478595 shutdown_connections 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 --2- 172.21.15.73:0/2505478595 >> v2:172.21.15.157:3300/0 conn(0x7f1a0406bc40 0x7f1a04164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 --2- 172.21.15.73:0/2505478595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0415abe0 0x7f1a040712e0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 -- 172.21.15.73:0/2505478595 >> 172.21.15.73:0/2505478595 conn(0x7f1a04069cb0 msgr2=0x7f1a0406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 -- 172.21.15.73:0/2505478595 shutdown_connections 2024-04-05T22:29:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.315+0000 7f1a09600640 1 -- 172.21.15.73:0/2505478595 wait complete. 2024-04-05T22:29:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.317+0000 7f1a09600640 1 Processor -- start 2024-04-05T22:29:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.317+0000 7f1a09600640 1 -- start start 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0406bc40 0x7f1a040b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0415abe0 0x7f1a04148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04145a10 con 0x7f1a0406c9a0 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04148f70 con 0x7f1a0406bc40 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a09600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040b3120 con 0x7f1a0415abe0 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a02000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a02000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34336/0 (socket says 172.21.15.73:34336) 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a02000640 1 -- 172.21.15.73:0/3208560116 learned_addr learned my addr 172.21.15.73:0/3208560116 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.318+0000 7f1a03400640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0415abe0 0x7f1a04148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02a00640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0406bc40 0x7f1a040b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 -- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0415abe0 msgr2=0x7f1a04148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:26.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0415abe0 0x7f1a04148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 -- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0406bc40 msgr2=0x7f1a040b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0406bc40 0x7f1a040b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 -- 172.21.15.73:0/3208560116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009b30 con 0x7f1a0406c9a0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a02000640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c460 tx=0x7f19ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec012830 con 0x7f1a0406c9a0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0129d0 con 0x7f1a0406c9a0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec012d80 con 0x7f1a0406c9a0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.319+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0414b0a0 con 0x7f1a0406c9a0 2024-04-05T22:29:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.320+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a040b2e40 con 0x7f1a0406c9a0 2024-04-05T22:29:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.320+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0414af00 con 0x7f1a0406c9a0 2024-04-05T22:29:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.323+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f19ec01b400 con 0x7f1a0406c9a0 2024-04-05T22:29:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.324+0000 7f19faa00640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 0x7f19e006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.324+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f19ec089ea0 con 0x7f1a0406c9a0 2024-04-05T22:29:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.324+0000 7f1a02a00640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 0x7f19e006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.326+0000 7f1a02a00640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 0x7f19e006d690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f19f4035000 tx=0x7f19f4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:26.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.328+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f19ec054ac0 con 0x7f1a0406c9a0 2024-04-05T22:29:26.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:26.521+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a0414af00 con 0x7f1a0406c9a0 2024-04-05T22:29:27.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.085+0000 7f19faa00640 1 -- 172.21.15.73:0/3208560116 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a0414af00 con 0x7f1a0406c9a0 2024-04-05T22:29:27.087 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:29:27.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.091+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 msgr2=0x7f19e006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.091+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 0x7f19e006d690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f19f4035000 tx=0x7f19f4030000 comp rx=0 tx=0).stop 2024-04-05T22:29:27.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.092+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 msgr2=0x7f1a041454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.092+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c460 tx=0x7f19ec00c920 comp rx=0 tx=0).stop 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 shutdown_connections 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f19e006b1d0 0x7f19e006d690 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0415abe0 0x7f1a04148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a0406c9a0 0x7f1a041454d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 --2- 172.21.15.73:0/3208560116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a0406bc40 0x7f1a040b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.093+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 >> 172.21.15.73:0/3208560116 conn(0x7f1a04069cb0 msgr2=0x7f1a040afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:27.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.094+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 shutdown_connections 2024-04-05T22:29:27.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.095+0000 7f1a09600640 1 -- 172.21.15.73:0/3208560116 wait complete. 2024-04-05T22:29:27.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:29:27.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.308+0000 7f0862600640 1 Processor -- start 2024-04-05T22:29:27.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.308+0000 7f0862600640 1 -- start start 2024-04-05T22:29:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f085c15aa70 0x7f085c070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f085c0714a0 0x7f085c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c071930 con 0x7f085c0714a0 2024-04-05T22:29:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c15ae50 con 0x7f085c15aa70 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f0862600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c15e3b0 con 0x7f085c15dbc0 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f085aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f085c0714a0 0x7f085c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f085aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f085c0714a0 0x7f085c15d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34344/0 (socket says 172.21.15.73:34344) 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f085aa00640 1 -- 172.21.15.73:0/991555792 learned_addr learned my addr 172.21.15.73:0/991555792 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f085b400640 1 --2- 172.21.15.73:0/991555792 >> v2:172.21.15.157:3300/0 conn(0x7f085c15aa70 0x7f085c070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.310+0000 7f085be00640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085aa00640 1 -- 172.21.15.73:0/991555792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 msgr2=0x7f085c1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085aa00640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085aa00640 1 -- 172.21.15.73:0/991555792 >> v2:172.21.15.157:3300/0 conn(0x7f085c15aa70 msgr2=0x7f085c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085aa00640 1 --2- 172.21.15.73:0/991555792 >> v2:172.21.15.157:3300/0 conn(0x7f085c15aa70 0x7f085c070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085aa00640 1 -- 172.21.15.73:0/991555792 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0ac9e0 con 0x7f085c0714a0 2024-04-05T22:29:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.311+0000 7f085be00640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:27.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.312+0000 7f085aa00640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c15d680 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f085c0b16a0 tx=0x7f084802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1814b7948fee68c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.312+0000 7f085a000640 1 -- 172.21.15.73:0/991555792 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848031870 con 0x7f085c0714a0 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.312+0000 7f085a000640 1 -- 172.21.15.73:0/991555792 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08480319d0 con 0x7f085c0714a0 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.312+0000 7f085a000640 1 -- 172.21.15.73:0/991555792 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848031d60 con 0x7f085c0714a0 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 -- 172.21.15.73:0/991555792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 msgr2=0x7f085c15d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c15d680 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f085c0b16a0 tx=0x7f084802b910 comp rx=0 tx=0).stop 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 -- 172.21.15.73:0/991555792 shutdown_connections 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c15d680 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 --2- 172.21.15.73:0/991555792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 --2- 172.21.15.73:0/991555792 >> v2:172.21.15.157:3300/0 conn(0x7f085c15aa70 0x7f085c070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 -- 172.21.15.73:0/991555792 >> 172.21.15.73:0/991555792 conn(0x7f085c069cb0 msgr2=0x7f085c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 -- 172.21.15.73:0/991555792 shutdown_connections 2024-04-05T22:29:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.313+0000 7f0862600640 1 -- 172.21.15.73:0/991555792 wait complete. 2024-04-05T22:29:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.315+0000 7f0862600640 1 Processor -- start 2024-04-05T22:29:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.315+0000 7f0862600640 1 -- start start 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c15aa70 0x7f085c153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0ae3e0 con 0x7f085c0714a0 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0aec80 con 0x7f085c15aa70 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.316+0000 7f0862600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0b0260 con 0x7f085c15dbc0 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34350/0 (socket says 172.21.15.73:34350) 2024-04-05T22:29:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 -- 172.21.15.73:0/809285239 learned_addr learned my addr 172.21.15.73:0/809285239 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 -- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 msgr2=0x7f085c154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 -- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c15aa70 msgr2=0x7f085c153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085aa00640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c15aa70 0x7f085c153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c15aa70 0x7f085c153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 -- 172.21.15.73:0/809285239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480094d0 con 0x7f085c0714a0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.317+0000 7f085b400640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f084c00d830 tx=0x7f084c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.318+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0096a0 con 0x7f085c0714a0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.318+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c009800 con 0x7f085c0714a0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.318+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c0ae630 con 0x7f085c0714a0 2024-04-05T22:29:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.318+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c009b90 con 0x7f085c0714a0 2024-04-05T22:29:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.318+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c152820 con 0x7f085c0714a0 2024-04-05T22:29:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.319+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c068160 con 0x7f085c0714a0 2024-04-05T22:29:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.321+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f084c026430 con 0x7f085c0714a0 2024-04-05T22:29:27.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.321+0000 7f0853400640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 0x7f083c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.322+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f084c08a710 con 0x7f085c0714a0 2024-04-05T22:29:27.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.322+0000 7f085aa00640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 0x7f083c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.323+0000 7f085aa00640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 0x7f083c06d660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0848036000 tx=0x7f084803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.326+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f084c0551e0 con 0x7f085c0714a0 2024-04-05T22:29:27.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.551+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f085c0ae1c0 con 0x7f085c0714a0 2024-04-05T22:29:27.564 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 20 and data pool 21 2024-04-05T22:29:27.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.564+0000 7f0853400640 1 -- 172.21.15.73:0/809285239 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f084c00b440 con 0x7f085c0714a0 2024-04-05T22:29:27.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.568+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 msgr2=0x7f083c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.568+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 0x7f083c06d660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0848036000 tx=0x7f084803c040 comp rx=0 tx=0).stop 2024-04-05T22:29:27.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.568+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 msgr2=0x7f085c1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.568+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f084c00d830 tx=0x7f084c00dcf0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 shutdown_connections 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f083c06b1a0 0x7f083c06d660 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f085c15dbc0 0x7f085c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c15aa70 0x7f085c153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 --2- 172.21.15.73:0/809285239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c0714a0 0x7f085c1522e0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.572+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 >> 172.21.15.73:0/809285239 conn(0x7f085c069cb0 msgr2=0x7f085c1552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.573+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 shutdown_connections 2024-04-05T22:29:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.574+0000 7f0862600640 1 -- 172.21.15.73:0/809285239 wait complete. 2024-04-05T22:29:27.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.777+0000 7f5f45400640 1 Processor -- start 2024-04-05T22:29:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.777+0000 7f5f45400640 1 -- start start 2024-04-05T22:29:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.779+0000 7f5f45400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f401587d0 0x7f5f400cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.779+0000 7f5f45400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.779+0000 7f5f45400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4006f5b0 con 0x7f5f400ccd20 2024-04-05T22:29:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.779+0000 7f5f45400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4015f240 con 0x7f5f401587d0 2024-04-05T22:29:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.779+0000 7f5f45400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40070410 con 0x7f5f4015e850 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:44578/0 (socket says 172.21.15.73:44578) 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3e800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f401587d0 0x7f5f400cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 -- 172.21.15.73:0/2175227141 learned_addr learned my addr 172.21.15.73:0/2175227141 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 -- 172.21.15.73:0/2175227141 >> v2:172.21.15.157:3300/0 conn(0x7f5f401587d0 msgr2=0x7f5f400cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:27.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 --2- 172.21.15.73:0/2175227141 >> v2:172.21.15.157:3300/0 conn(0x7f5f401587d0 0x7f5f400cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 -- 172.21.15.73:0/2175227141 >> v1:172.21.15.157:6789/0 conn(0x7f5f400ccd20 legacy=0x7f5f4006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.780+0000 7f5f3f200640 1 -- 172.21.15.73:0/2175227141 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40158bb0 con 0x7f5f4015e850 2024-04-05T22:29:27.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.782+0000 7f5f3f200640 1 --2- 172.21.15.73:0/2175227141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e50 tx=0x7f5f3802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c0c7c5adf62768 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.783+0000 7f5f3de00640 1 -- 172.21.15.73:0/2175227141 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3802cb00 con 0x7f5f4015e850 2024-04-05T22:29:27.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.783+0000 7f5f3de00640 1 -- 172.21.15.73:0/2175227141 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3802cc60 con 0x7f5f4015e850 2024-04-05T22:29:27.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.783+0000 7f5f3de00640 1 -- 172.21.15.73:0/2175227141 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380343f0 con 0x7f5f4015e850 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 -- 172.21.15.73:0/2175227141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 msgr2=0x7f5f4015ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 --2- 172.21.15.73:0/2175227141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e50 tx=0x7f5f3802bf60 comp rx=0 tx=0).stop 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 -- 172.21.15.73:0/2175227141 shutdown_connections 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 --2- 172.21.15.73:0/2175227141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f4015e850 0x7f5f4015ed00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 --2- 172.21.15.73:0/2175227141 >> v2:172.21.15.157:3300/0 conn(0x7f5f401587d0 0x7f5f400cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 -- 172.21.15.73:0/2175227141 >> 172.21.15.73:0/2175227141 conn(0x7f5f400ce8e0 msgr2=0x7f5f400cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 -- 172.21.15.73:0/2175227141 shutdown_connections 2024-04-05T22:29:27.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.784+0000 7f5f45400640 1 -- 172.21.15.73:0/2175227141 wait complete. 2024-04-05T22:29:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.786+0000 7f5f45400640 1 Processor -- start 2024-04-05T22:29:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.786+0000 7f5f45400640 1 -- start start 2024-04-05T22:29:27.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 0x7f5f401422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f400b3070 0x7f5f401f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400c4960 con 0x7f5f401587d0 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400b2330 con 0x7f5f4015e850 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f45400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40142810 con 0x7f5f400b3070 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34360/0 (socket says 172.21.15.73:34360) 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f3e800640 1 -- 172.21.15.73:0/2601623456 learned_addr learned my addr 172.21.15.73:0/2601623456 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f3e800640 1 -- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f400b3070 msgr2=0x7f5f401f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.787+0000 7f5f3f200640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f400b3070 0x7f5f401f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f3e800640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f400b3070 0x7f5f401f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f35e00640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 0x7f5f401422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f3e800640 1 -- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 msgr2=0x7f5f401422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f3e800640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 0x7f5f401422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f3e800640 1 -- 172.21.15.73:0/2601623456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38009b30 con 0x7f5f401587d0 2024-04-05T22:29:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f35e00640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 0x7f5f401422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f3e800640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e9b0 tx=0x7f5f3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30009670 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3000c9a0 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000cd50 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f400c4680 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.788+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f400b2870 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.789+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f40066af0 con 0x7f5f401587d0 2024-04-05T22:29:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.791+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f5f30013470 con 0x7f5f401587d0 2024-04-05T22:29:27.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.791+0000 7f5f37400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 0x7f5f1c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:27.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.792+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3008a0a0 con 0x7f5f401587d0 2024-04-05T22:29:27.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.792+0000 7f5f35e00640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 0x7f5f1c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:27.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.793+0000 7f5f35e00640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 0x7f5f1c06d660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f400743e0 tx=0x7f5f28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.796+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5f300549d0 con 0x7f5f401587d0 2024-04-05T22:29:27.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.980+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f40068160 con 0x7f5f401587d0 2024-04-05T22:29:27.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.982+0000 7f5f37400640 1 -- 172.21.15.73:0/2601623456 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16124 (secure 0 0 0) 0x7f5f30017090 con 0x7f5f401587d0 2024-04-05T22:29:27.982 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:27.983 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":135,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:27.553743+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:27.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.984+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 msgr2=0x7f5f1c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 0x7f5f1c06d660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f400743e0 tx=0x7f5f28009040 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 msgr2=0x7f5f400c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e9b0 tx=0x7f5f3000ee70 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 shutdown_connections 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5f1c06b1a0 0x7f5f1c06d660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f400b3070 0x7f5f401f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f4015e850 0x7f5f401422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 --2- 172.21.15.73:0/2601623456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f401587d0 0x7f5f400c37e0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 >> 172.21.15.73:0/2601623456 conn(0x7f5f400ce8e0 msgr2=0x7f5f400ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 shutdown_connections 2024-04-05T22:29:27.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:27.985+0000 7f5f45400640 1 -- 172.21.15.73:0/2601623456 wait complete. 2024-04-05T22:29:28.003 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:28.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.207+0000 7f50ac400640 1 Processor -- start 2024-04-05T22:29:28.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.207+0000 7f50ac400640 1 -- start start 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50ac400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50a41587d0 0x7f50a40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50ac400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a406f5b0 con 0x7f50a415e850 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50ac400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a4070410 con 0x7f50a40ccd20 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50ac400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a4158bb0 con 0x7f50a41587d0 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50aa400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f50a415e850 0x7f50a40708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53158/0 (socket says 172.21.15.73:53158) 2024-04-05T22:29:28.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50aa400640 1 -- 172.21.15.73:0/528145298 learned_addr learned my addr 172.21.15.73:0/528145298 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:28.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387258160 0 0) 0x7f50a406f5b0 con 0x7f50a415e850 2024-04-05T22:29:28.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c003520 con 0x7f50a415e850 2024-04-05T22:29:28.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.209+0000 7f50a9a00640 1 --2- 172.21.15.73:0/528145298 >> v2:172.21.15.157:3300/0 conn(0x7f50a41587d0 0x7f50a40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796491434 0 0) 0x7f508c003520 con 0x7f50a415e850 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 >> v2:172.21.15.157:3300/0 conn(0x7f50a41587d0 msgr2=0x7f50a40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 --2- 172.21.15.73:0/528145298 >> v2:172.21.15.157:3300/0 conn(0x7f50a41587d0 0x7f50a40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 >> v1:172.21.15.157:6789/0 conn(0x7f50a40ccd20 legacy=0x7f50a406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a406f800 con 0x7f50a415e850 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f509c0019f0 con 0x7f50a415e850 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 >> v1:172.21.15.73:6789/0 conn(0x7f50a415e850 legacy=0x7f50a40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f508c003970 0x7f508c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0047c0 0x7f508c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f508c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4158bb0 con 0x7f508c005180 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c00c0d0 con 0x7f508c0047c0 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c00c290 con 0x7f508c003970 2024-04-05T22:29:28.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50aa400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f508c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.210+0000 7f50a9a00640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f508c003970 0x7f508c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50a9000640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0047c0 0x7f508c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50aa400640 1 -- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f508c003970 msgr2=0x7f508c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50aa400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f508c003970 0x7f508c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50aa400640 1 -- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0047c0 msgr2=0x7f508c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50aa400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0047c0 0x7f508c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.211+0000 7f50aa400640 1 -- 172.21.15.73:0/528145298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c003520 con 0x7f508c005180 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.212+0000 7f50aa400640 1 -- 172.21.15.73:0/528145298 reap_dead start 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.212+0000 7f50aa400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f508c00b980 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f509c00a7c0 tx=0x7f508c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae3c84e55597605 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.212+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c034630 con 0x7f508c005180 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.212+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c02f070 con 0x7f508c005180 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.212+0000 7f50a3e00640 1 -- 172.21.15.73:0/528145298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c034a10 con 0x7f508c005180 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 -- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 msgr2=0x7f508c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f508c00b980 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f509c00a7c0 tx=0x7f508c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 -- 172.21.15.73:0/528145298 shutdown_connections 2024-04-05T22:29:28.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 --2- 172.21.15.73:0/528145298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f508c00b980 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 -- 172.21.15.73:0/528145298 >> 172.21.15.73:0/528145298 conn(0x7f50a40ce8e0 msgr2=0x7f50a40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:28.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 -- 172.21.15.73:0/528145298 shutdown_connections 2024-04-05T22:29:28.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.213+0000 7f50ac400640 1 -- 172.21.15.73:0/528145298 wait complete. 2024-04-05T22:29:28.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.215+0000 7f50ac400640 1 Processor -- start 2024-04-05T22:29:28.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.215+0000 7f50ac400640 1 -- start start 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50a41f5e40 0x7f50a41429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 0x7f50a413d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4142f00 con 0x7f508c005180 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41f6dc0 con 0x7f50a4069f20 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50ac400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41f62a0 con 0x7f50a41f5e40 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34386/0 (socket says 172.21.15.73:34386) 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 -- 172.21.15.73:0/2659041419 learned_addr learned my addr 172.21.15.73:0/2659041419 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50aa400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 0x7f50a413d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 -- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50a41f5e40 msgr2=0x7f50a41429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9000640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50a41f5e40 0x7f50a41429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50a41f5e40 0x7f50a41429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 -- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 msgr2=0x7f50a413d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 0x7f50a413d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50a9a00640 1 -- 172.21.15.73:0/2659041419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a41f6520 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.216+0000 7f50aa400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 0x7f50a413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50a9a00640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f509800f5b0 tx=0x7f509800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098013810 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5098013970 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098013ce0 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a41f6800 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.217+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a413e5c0 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.218+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a413e780 con 0x7f508c005180 2024-04-05T22:29:28.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.219+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f509801f460 con 0x7f508c005180 2024-04-05T22:29:28.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.220+0000 7f50a1600640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 0x7f508406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.220+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f509808a120 con 0x7f508c005180 2024-04-05T22:29:28.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.221+0000 7f50a9000640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 0x7f508406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.222+0000 7f50a9000640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 0x7f508406d690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5090007fd0 tx=0x7f5090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.224+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f50980bd650 con 0x7f508c005180 2024-04-05T22:29:28.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.418+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50a4066af0 con 0x7f508c005180 2024-04-05T22:29:28.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.420+0000 7f50a1600640 1 -- 172.21.15.73:0/2659041419 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16124 (secure 0 0 0) 0x7f5098054ae0 con 0x7f508c005180 2024-04-05T22:29:28.421 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:28.421 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":135,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:27.553743+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:28.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 msgr2=0x7f508406d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 0x7f508406d690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5090007fd0 tx=0x7f5090009040 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 msgr2=0x7f50a4158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f509800f5b0 tx=0x7f509800fa70 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 shutdown_connections 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f508406b1d0 0x7f508406d690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50a4069f20 0x7f50a413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50a41f5e40 0x7f50a41429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 --2- 172.21.15.73:0/2659041419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c005180 0x7f50a4158e80 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 >> 172.21.15.73:0/2659041419 conn(0x7f50a40ce8e0 msgr2=0x7f50a415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 shutdown_connections 2024-04-05T22:29:28.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.423+0000 7f50ac400640 1 -- 172.21.15.73:0/2659041419 wait complete. 2024-04-05T22:29:28.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:29:28.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.650+0000 7f678f200640 1 Processor -- start 2024-04-05T22:29:28.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.650+0000 7f678f200640 1 -- start start 2024-04-05T22:29:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.651+0000 7f678f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f678815abe0 0x7f67880712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.651+0000 7f678f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 0x7f678806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.651+0000 7f678f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678806bea0 con 0x7f678806ba70 2024-04-05T22:29:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.651+0000 7f678f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678815afc0 con 0x7f678815abe0 2024-04-05T22:29:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.651+0000 7f678f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678806f5b0 con 0x7f678806c9a0 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f678815abe0 0x7f67880712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f678d200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f678806ba70 0x7f67880cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53164/0 (socket says 172.21.15.73:53164) 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f678d200640 1 -- 172.21.15.73:0/4198589999 learned_addr learned my addr 172.21.15.73:0/4198589999 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787400640 1 --2- 172.21.15.73:0/4198589999 >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 0x7f678806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787e00640 1 -- 172.21.15.73:0/4198589999 >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 msgr2=0x7f678806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787e00640 1 --2- 172.21.15.73:0/4198589999 >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 0x7f678806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787e00640 1 -- 172.21.15.73:0/4198589999 >> v1:172.21.15.73:6789/0 conn(0x7f678806ba70 legacy=0x7f67880cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787e00640 1 -- 172.21.15.73:0/4198589999 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678806f890 con 0x7f678815abe0 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6786a00640 1 -- 172.21.15.73:0/4198589999 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993116557 0 0) 0x7f678806bea0 con 0x7f678806ba70 2024-04-05T22:29:28.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.652+0000 7f6787400640 1 --2- 172.21.15.73:0/4198589999 >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 0x7f678806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.653+0000 7f6787e00640 1 --2- 172.21.15.73:0/4198589999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67880712e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f678806ce30 tx=0x7f677802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defb5e1295d05fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.653+0000 7f6786a00640 1 -- 172.21.15.73:0/4198589999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778031710 con 0x7f678815abe0 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f6786a00640 1 -- 172.21.15.73:0/4198589999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677802f050 con 0x7f678815abe0 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f6786a00640 1 -- 172.21.15.73:0/4198589999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778031a90 con 0x7f678815abe0 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 -- 172.21.15.73:0/4198589999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 msgr2=0x7f67880712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 --2- 172.21.15.73:0/4198589999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67880712e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f678806ce30 tx=0x7f677802b7b0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 -- 172.21.15.73:0/4198589999 shutdown_connections 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 --2- 172.21.15.73:0/4198589999 >> v2:172.21.15.157:3300/0 conn(0x7f678806c9a0 0x7f678806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 --2- 172.21.15.73:0/4198589999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67880712e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 -- 172.21.15.73:0/4198589999 >> 172.21.15.73:0/4198589999 conn(0x7f6788069cb0 msgr2=0x7f678806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:28.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 -- 172.21.15.73:0/4198589999 shutdown_connections 2024-04-05T22:29:28.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.654+0000 7f678f200640 1 -- 172.21.15.73:0/4198589999 wait complete. 2024-04-05T22:29:28.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.656+0000 7f678f200640 1 Processor -- start 2024-04-05T22:29:28.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.656+0000 7f678f200640 1 -- start start 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 0x7f67880adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678806c9a0 0x7f678814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678814c3c0 con 0x7f678815abe0 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678814c6a0 con 0x7f678806ba70 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678814c980 con 0x7f678806c9a0 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34398/0 (socket says 172.21.15.73:34398) 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.657+0000 7f678d200640 1 -- 172.21.15.73:0/1390668959 learned_addr learned my addr 172.21.15.73:0/1390668959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f6787400640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678806c9a0 0x7f678814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f6787e00640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 0x7f67880adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 -- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678806c9a0 msgr2=0x7f678814d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678806c9a0 0x7f678814d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 -- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 msgr2=0x7f67880adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 0x7f67880adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 -- 172.21.15.73:0/1390668959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778009400 con 0x7f678815abe0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f6787e00640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 0x7f67880adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678d200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f678000c7e0 tx=0x7f678000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000a640 con 0x7f678815abe0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000a7c0 con 0x7f678815abe0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000ab70 con 0x7f678815abe0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.658+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678814cc60 con 0x7f678815abe0 2024-04-05T22:29:28.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.659+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678814d9a0 con 0x7f678815abe0 2024-04-05T22:29:28.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.660+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678814d180 con 0x7f678815abe0 2024-04-05T22:29:28.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.661+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f678000acd0 con 0x7f678815abe0 2024-04-05T22:29:28.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.661+0000 7f677fe00640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 0x7f676806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:28.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.661+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67800893f0 con 0x7f678815abe0 2024-04-05T22:29:28.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.662+0000 7f6787e00640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 0x7f676806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:28.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.663+0000 7f6787e00640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 0x7f676806d660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6778035000 tx=0x7f6778008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:28.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.666+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f678814d180 con 0x7f678815abe0 2024-04-05T22:29:28.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:28.896+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f678814d180 con 0x7f678815abe0 2024-04-05T22:29:29.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.587+0000 7f677fe00640 1 -- 172.21.15.73:0/1390668959 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7f678814d180 con 0x7f678815abe0 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.590+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 msgr2=0x7f676806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.590+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 0x7f676806d660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6778035000 tx=0x7f6778008030 comp rx=0 tx=0).stop 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 msgr2=0x7f67881484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f678000c7e0 tx=0x7f678000cca0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 shutdown_connections 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f676806b1a0 0x7f676806d660 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f678815abe0 0x7f67881484a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f678806c9a0 0x7f678814d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 --2- 172.21.15.73:0/1390668959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678806ba70 0x7f67880adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 >> 172.21.15.73:0/1390668959 conn(0x7f6788069cb0 msgr2=0x7f678815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 shutdown_connections 2024-04-05T22:29:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.591+0000 7f678f200640 1 -- 172.21.15.73:0/1390668959 wait complete. 2024-04-05T22:29:29.611 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:29:29.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:29:29.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.819+0000 7f0552a00640 1 Processor -- start 2024-04-05T22:29:29.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.819+0000 7f0552a00640 1 -- start start 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0552a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f054c0b1950 0x7f054c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0552a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f054c06b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0552a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c15f260 con 0x7f054c15ee80 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0552a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c1657c0 con 0x7f054c0b1950 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0552a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c070d20 con 0x7f054c165310 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f054be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f054c15ee80 0x7f054c0cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53166/0 (socket says 172.21.15.73:53166) 2024-04-05T22:29:29.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f054be00640 1 -- 172.21.15.73:0/713538407 learned_addr learned my addr 172.21.15.73:0/713538407 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0543400640 1 --2- 172.21.15.73:0/713538407 >> v2:172.21.15.157:3300/0 conn(0x7f054c0b1950 0x7f054c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123168217 0 0) 0x7f053c001110 con 0x7f054c15ee80 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0530003520 con 0x7f054c15ee80 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.821+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f054c06b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901208104 0 0) 0x7f0530003520 con 0x7f054c15ee80 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 msgr2=0x7f054c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:29.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f054c06b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 >> v2:172.21.15.157:3300/0 conn(0x7f054c0b1950 msgr2=0x7f054c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 --2- 172.21.15.73:0/713538407 >> v2:172.21.15.157:3300/0 conn(0x7f054c0b1950 0x7f054c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c06f5b0 con 0x7f054c15ee80 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f053c001800 con 0x7f054c15ee80 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f054c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 >> v1:172.21.15.73:6789/0 conn(0x7f054c15ee80 legacy=0x7f054c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f0530003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530003eb0 0x7f05300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0530004d00 0x7f0530005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1657c0 con 0x7f0530004d00 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053000b980 con 0x7f0530003eb0 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053000baf0 con 0x7f054c165310 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.822+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0530004d00 0x7f0530005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f054be00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f0530003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f0550a00640 1 -- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 msgr2=0x7f0530003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:29.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c165310 0x7f0530003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f0550a00640 1 -- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530003eb0 msgr2=0x7f05300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:29.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530003eb0 0x7f05300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.823+0000 7f0550a00640 1 -- 172.21.15.73:0/713538407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530003520 con 0x7f0530004d00 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.824+0000 7f0550a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0530004d00 0x7f0530005180 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f05440073e0 tx=0x7f05300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e31a07f3fa56b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.824+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054402cd30 con 0x7f0530004d00 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.824+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054402ce90 con 0x7f0530004d00 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.824+0000 7f054b400640 1 -- 172.21.15.73:0/713538407 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544034640 con 0x7f0530004d00 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 -- 172.21.15.73:0/713538407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0530004d00 msgr2=0x7f0530005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 --2- 172.21.15.73:0/713538407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0530004d00 0x7f0530005180 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f05440073e0 tx=0x7f05300057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0550a00640 1 -- 172.21.15.73:0/713538407 reap_dead start 2024-04-05T22:29:29.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 -- 172.21.15.73:0/713538407 shutdown_connections 2024-04-05T22:29:29.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 -- 172.21.15.73:0/713538407 >> 172.21.15.73:0/713538407 conn(0x7f054c069cb0 msgr2=0x7f054c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:29.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 -- 172.21.15.73:0/713538407 shutdown_connections 2024-04-05T22:29:29.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.825+0000 7f0552a00640 1 -- 172.21.15.73:0/713538407 wait complete. 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.827+0000 7f0552a00640 1 Processor -- start 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.827+0000 7f0552a00640 1 -- start start 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530004d00 0x7f054c0c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c0ce7f0 0x7f054c0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0ceca0 con 0x7f054c14eed0 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c3a40 con 0x7f0530004d00 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0552a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c3d20 con 0x7f054c0ce7f0 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34428/0 (socket says 172.21.15.73:34428) 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 -- 172.21.15.73:0/2451866142 learned_addr learned my addr 172.21.15.73:0/2451866142 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 -- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c0ce7f0 msgr2=0x7f054c0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c0ce7f0 0x7f054c0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 -- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530004d00 msgr2=0x7f054c0c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530004d00 0x7f054c0c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.828+0000 7f0543400640 1 -- 172.21.15.73:0/2451866142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0c4000 con 0x7f054c14eed0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0543400640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f053400e830 tx=0x7f053400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534009570 con 0x7f054c14eed0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400c7e0 con 0x7f054c14eed0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c0c42e0 con 0x7f054c14eed0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400cb70 con 0x7f054c14eed0 2024-04-05T22:29:29.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.829+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c0c5020 con 0x7f054c14eed0 2024-04-05T22:29:29.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.830+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c068160 con 0x7f054c14eed0 2024-04-05T22:29:29.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.831+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f053400ccd0 con 0x7f054c14eed0 2024-04-05T22:29:29.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.832+0000 7f0548c00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 0x7f052806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:29.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.832+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0534089de0 con 0x7f054c14eed0 2024-04-05T22:29:29.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.833+0000 7f054be00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 0x7f052806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:29.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.834+0000 7f054be00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 0x7f052806d690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f053c000ff0 tx=0x7f053c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:29.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:29.837+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0534054750 con 0x7f054c14eed0 2024-04-05T22:29:30.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.073+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f054c06ce20 con 0x7f054c14eed0 2024-04-05T22:29:30.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.590+0000 7f0548c00640 1 -- 172.21.15.73:0/2451866142 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05340210c0 con 0x7f054c14eed0 2024-04-05T22:29:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 msgr2=0x7f052806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 0x7f052806d690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f053c000ff0 tx=0x7f053c009040 comp rx=0 tx=0).stop 2024-04-05T22:29:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 msgr2=0x7f054c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f053400e830 tx=0x7f053400ecf0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 shutdown_connections 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f052806b1d0 0x7f052806d690 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f054c0ce7f0 0x7f054c0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f054c14eed0 0x7f054c0ce2b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.593+0000 7f0552a00640 1 --2- 172.21.15.73:0/2451866142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0530004d00 0x7f054c0c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.594+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 >> 172.21.15.73:0/2451866142 conn(0x7f054c069cb0 msgr2=0x7f054c15e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.594+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 shutdown_connections 2024-04-05T22:29:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.594+0000 7f0552a00640 1 -- 172.21.15.73:0/2451866142 wait complete. 2024-04-05T22:29:30.618 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:29:30.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:30.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.822+0000 7f230ec00640 1 Processor -- start 2024-04-05T22:29:30.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.822+0000 7f230ec00640 1 -- start start 2024-04-05T22:29:30.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.824+0000 7f230ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f230815cee0 0x7f23080cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.824+0000 7f230ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23080ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.824+0000 7f230ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23080ace90 con 0x7f2308160990 2024-04-05T22:29:30.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.824+0000 7f230ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230815d2c0 con 0x7f230815cee0 2024-04-05T22:29:30.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.824+0000 7f230ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23080ad170 con 0x7f23080c3700 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.825+0000 7f230cc00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2308160990 0x7f23080cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53172/0 (socket says 172.21.15.73:53172) 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.825+0000 7f230cc00640 1 -- 172.21.15.73:0/1309906746 learned_addr learned my addr 172.21.15.73:0/1309906746 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.825+0000 7f2307e00640 1 --2- 172.21.15.73:0/1309906746 >> v2:172.21.15.73:3300/0 conn(0x7f230815cee0 0x7f23080cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.825+0000 7f2307400640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23080ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.826+0000 7f2307e00640 1 -- 172.21.15.73:0/1309906746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 msgr2=0x7f23080ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.826+0000 7f2307e00640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23080ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.826+0000 7f2307e00640 1 -- 172.21.15.73:0/1309906746 >> v1:172.21.15.73:6789/0 conn(0x7f2308160990 legacy=0x7f23080cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.826+0000 7f2307e00640 1 -- 172.21.15.73:0/1309906746 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080ad450 con 0x7f230815cee0 2024-04-05T22:29:30.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.826+0000 7f2306a00640 1 -- 172.21.15.73:0/1309906746 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119989086 0 0) 0x7f23080ace90 con 0x7f2308160990 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f2307e00640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f23080cc6b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f22f80097a0 tx=0x7f22f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58efef347cc5298 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f2306a00640 1 -- 172.21.15.73:0/1309906746 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8031830 con 0x7f230815cee0 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f2306a00640 1 -- 172.21.15.73:0/1309906746 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f8031990 con 0x7f230815cee0 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f2306a00640 1 -- 172.21.15.73:0/1309906746 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8031d20 con 0x7f230815cee0 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f230ec00640 1 -- 172.21.15.73:0/1309906746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 msgr2=0x7f23080cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.827+0000 7f230ec00640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f23080cc6b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f22f80097a0 tx=0x7f22f802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 -- 172.21.15.73:0/1309906746 shutdown_connections 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23080ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 --2- 172.21.15.73:0/1309906746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f23080cc6b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 -- 172.21.15.73:0/1309906746 >> 172.21.15.73:0/1309906746 conn(0x7f23080b0490 msgr2=0x7f23080b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 -- 172.21.15.73:0/1309906746 shutdown_connections 2024-04-05T22:29:30.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.828+0000 7f230ec00640 1 -- 172.21.15.73:0/1309906746 wait complete. 2024-04-05T22:29:30.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.829+0000 7f230ec00640 1 Processor -- start 2024-04-05T22:29:30.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.830+0000 7f230ec00640 1 -- start start 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.830+0000 7f230ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23081479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f230ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f230ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308160990 0x7f23081449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f230ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23081574b0 con 0x7f230815cee0 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f230ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23081568a0 con 0x7f2308160990 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f230ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2308156b80 con 0x7f23080c3700 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:30.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34446/0 (socket says 172.21.15.73:34446) 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 -- 172.21.15.73:0/1122484814 learned_addr learned my addr 172.21.15.73:0/1122484814 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 -- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 msgr2=0x7f23081479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23081479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 -- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308160990 msgr2=0x7f23081449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308160990 0x7f23081449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.831+0000 7f2307400640 1 -- 172.21.15.73:0/1122484814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009400 con 0x7f230815cee0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f2307400640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e9b0 tx=0x7f22f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0009670 con 0x7f230815cee0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f000c9a0 con 0x7f230815cee0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f000cd50 con 0x7f230815cee0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2308156e60 con 0x7f230815cee0 2024-04-05T22:29:30.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.832+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23081559a0 con 0x7f230815cee0 2024-04-05T22:29:30.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.833+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2308068160 con 0x7f230815cee0 2024-04-05T22:29:30.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.834+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f22f0013470 con 0x7f230815cee0 2024-04-05T22:29:30.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.835+0000 7f22ffe00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 0x7f22e406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:30.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.835+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22f008a050 con 0x7f230815cee0 2024-04-05T22:29:30.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.835+0000 7f2307e00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 0x7f22e406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:30.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.837+0000 7f2307e00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 0x7f22e406d660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f22f8036000 tx=0x7f22f8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:30.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:30.840+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f22f00549c0 con 0x7f230815cee0 2024-04-05T22:29:31.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.069+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23081575f0 con 0x7f230815cee0 2024-04-05T22:29:31.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.070+0000 7f22ffe00640 1 -- 172.21.15.73:0/1122484814 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f22f0017090 con 0x7f230815cee0 2024-04-05T22:29:31.070 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:31.070 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:29:31.071 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 82 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 msgr2=0x7f22e406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 0x7f22e406d660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f22f8036000 tx=0x7f22f8030000 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 msgr2=0x7f2308158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e9b0 tx=0x7f22f000ee70 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 shutdown_connections 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f22e406b1a0 0x7f22e406d660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.072+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308160990 0x7f23081449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.073+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230815cee0 0x7f2308158cb0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.073+0000 7f230ec00640 1 --2- 172.21.15.73:0/1122484814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23080c3700 0x7f23081479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.073+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 >> 172.21.15.73:0/1122484814 conn(0x7f23080b0490 msgr2=0x7f23080ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.073+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 shutdown_connections 2024-04-05T22:29:31.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.073+0000 7f230ec00640 1 -- 172.21.15.73:0/1122484814 wait complete. 2024-04-05T22:29:31.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:31.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.301+0000 7fcf9c800640 1 Processor -- start 2024-04-05T22:29:31.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.301+0000 7fcf9c800640 1 -- start start 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf9c800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf9415ef60 0x7fcf9406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf9c800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9406f5b0 con 0x7fcf9406cb80 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf9c800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf94070410 con 0x7fcf941587d0 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf9c800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9406d500 con 0x7fcf9415ef60 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf99400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcf9406cb80 0x7fcf9415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53186/0 (socket says 172.21.15.73:53186) 2024-04-05T22:29:31.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf99400640 1 -- 172.21.15.73:0/1704986663 learned_addr learned my addr 172.21.15.73:0/1704986663 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238660662 0 0) 0x7fcf9406f5b0 con 0x7fcf9406cb80 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf9a800640 1 --2- 172.21.15.73:0/1704986663 >> v2:172.21.15.157:3300/0 conn(0x7fcf9415ef60 0x7fcf9406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.303+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf7c003520 con 0x7fcf9406cb80 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3395396850 0 0) 0x7fcf7c003520 con 0x7fcf9406cb80 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 >> v2:172.21.15.157:3300/0 conn(0x7fcf9415ef60 msgr2=0x7fcf9406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 --2- 172.21.15.73:0/1704986663 >> v2:172.21.15.157:3300/0 conn(0x7fcf9415ef60 0x7fcf9406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 >> v1:172.21.15.157:6789/0 conn(0x7fcf941587d0 legacy=0x7fcf940cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940ccef0 con 0x7fcf9406cb80 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fcf84001670 con 0x7fcf9406cb80 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 >> v1:172.21.15.73:6789/0 conn(0x7fcf9406cb80 legacy=0x7fcf9415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c003970 0x7fcf7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c005180 0x7fcf7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9406d500 con 0x7fcf7c0047c0 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00c020 con 0x7fcf7c005180 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00c1e0 con 0x7fcf7c003970 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.304+0000 7fcf99400640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99e00640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c003970 0x7fcf7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99400640 1 -- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c003970 msgr2=0x7fcf7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99400640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c003970 0x7fcf7c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99400640 1 -- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c005180 msgr2=0x7fcf7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99400640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c005180 0x7fcf7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf9a800640 1 -- 172.21.15.73:0/1704986663 reap_dead start 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.305+0000 7fcf99400640 1 -- 172.21.15.73:0/1704986663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c003520 con 0x7fcf7c0047c0 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf99400640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf7c004c40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c240 tx=0x7fcf7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8462d768dc496d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84005ad0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84005c30 con 0x7fcf7c0047c0 2024-04-05T22:29:31.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf98a00640 1 -- 172.21.15.73:0/1704986663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8402caf0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf9c800640 1 -- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 msgr2=0x7fcf7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.306+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf7c004c40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c240 tx=0x7fcf7c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.307+0000 7fcf9c800640 1 -- 172.21.15.73:0/1704986663 shutdown_connections 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.307+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1704986663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf7c004c40 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.307+0000 7fcf9c800640 1 -- 172.21.15.73:0/1704986663 >> 172.21.15.73:0/1704986663 conn(0x7fcf940ce8e0 msgr2=0x7fcf940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.307+0000 7fcf9c800640 1 -- 172.21.15.73:0/1704986663 shutdown_connections 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.307+0000 7fcf9c800640 1 -- 172.21.15.73:0/1704986663 wait complete. 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.308+0000 7fcf9c800640 1 Processor -- start 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 -- start start 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf941f5e10 0x7fcf941f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf94158ff0 0x7fcf94159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941f67b0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940c37e0 con 0x7fcf941f5e10 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.309+0000 7fcf9c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94159d60 con 0x7fcf94158ff0 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34464/0 (socket says 172.21.15.73:34464) 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 -- 172.21.15.73:0/1074321014 learned_addr learned my addr 172.21.15.73:0/1074321014 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 -- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf94158ff0 msgr2=0x7fcf94159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf94158ff0 0x7fcf94159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 -- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf941f5e10 msgr2=0x7fcf941f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf941f5e10 0x7fcf941f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 -- 172.21.15.73:0/1074321014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9415a040 con 0x7fcf7c0047c0 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.310+0000 7fcf99e00640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800d4a0 tx=0x7fcf8800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.311+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800b070 con 0x7fcf7c0047c0 2024-04-05T22:29:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.311+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf880054e0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.311+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88003810 con 0x7fcf7c0047c0 2024-04-05T22:29:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.311+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9415a320 con 0x7fcf7c0047c0 2024-04-05T22:29:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.311+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf941599b0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.312+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf94066af0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.313+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fcf88003970 con 0x7fcf7c0047c0 2024-04-05T22:29:31.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.314+0000 7fcf8e000640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 0x7fcf7806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.314+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf880896f0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.314+0000 7fcf99400640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 0x7fcf7806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.315+0000 7fcf99400640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 0x7fcf7806d660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf84034000 tx=0x7fcf84039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.318+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcf88053ff0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.517+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf940c39a0 con 0x7fcf7c0047c0 2024-04-05T22:29:31.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.520+0000 7fcf8e000640 1 -- 172.21.15.73:0/1074321014 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7fcf88057610 con 0x7fcf7c0047c0 2024-04-05T22:29:31.521 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:31.521 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 msgr2=0x7fcf7806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 0x7fcf7806d660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf84034000 tx=0x7fcf84039040 comp rx=0 tx=0).stop 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 msgr2=0x7fcf940cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800d4a0 tx=0x7fcf8800d960 comp rx=0 tx=0).stop 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 shutdown_connections 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcf7806b1a0 0x7fcf7806d660 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf94158ff0 0x7fcf94159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf941f5e10 0x7fcf941f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 --2- 172.21.15.73:0/1074321014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf7c0047c0 0x7fcf940cb240 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.523+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 >> 172.21.15.73:0/1074321014 conn(0x7fcf940ce8e0 msgr2=0x7fcf940cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:31.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.524+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 shutdown_connections 2024-04-05T22:29:31.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.524+0000 7fcf9c800640 1 -- 172.21.15.73:0/1074321014 wait complete. 2024-04-05T22:29:31.543 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:29:31.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:29:31.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.750+0000 7f1e13200640 1 Processor -- start 2024-04-05T22:29:31.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.750+0000 7f1e13200640 1 -- start start 2024-04-05T22:29:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e13200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e14158860 0x7f1e140b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e13200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e13200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1406b500 con 0x7f1e140b1f00 2024-04-05T22:29:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e13200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e140ccaf0 con 0x7f1e14158860 2024-04-05T22:29:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e13200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140b23a0 con 0x7f1e1415ed30 2024-04-05T22:29:31.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.752+0000 7f1e12800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:44696/0 (socket says 172.21.15.73:44696) 2024-04-05T22:29:31.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 -- 172.21.15.73:0/1416595552 learned_addr learned my addr 172.21.15.73:0/1416595552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:31.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e0be00640 1 --1- 172.21.15.73:0/1416595552 >> v1:172.21.15.157:6789/0 conn(0x7f1e140b1f00 0x7f1e1415e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:59656/0 (socket says 172.21.15.73:59656) 2024-04-05T22:29:31.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e11e00640 1 --2- 172.21.15.73:0/1416595552 >> v2:172.21.15.157:3300/0 conn(0x7f1e14158860 0x7f1e140b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 -- 172.21.15.73:0/1416595552 >> v2:172.21.15.157:3300/0 conn(0x7f1e14158860 msgr2=0x7f1e140b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:31.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 --2- 172.21.15.73:0/1416595552 >> v2:172.21.15.157:3300/0 conn(0x7f1e14158860 0x7f1e140b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 -- 172.21.15.73:0/1416595552 >> v1:172.21.15.157:6789/0 conn(0x7f1e140b1f00 legacy=0x7f1e1415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.753+0000 7f1e12800640 1 -- 172.21.15.73:0/1416595552 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140b2680 con 0x7f1e1415ed30 2024-04-05T22:29:31.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.755+0000 7f1e12800640 1 --2- 172.21.15.73:0/1416595552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009e50 tx=0x7f1e0c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a5e83addcb56d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.755+0000 7f1e11400640 1 -- 172.21.15.73:0/1416595552 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c02ca70 con 0x7f1e1415ed30 2024-04-05T22:29:31.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.755+0000 7f1e11400640 1 -- 172.21.15.73:0/1416595552 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c02cbd0 con 0x7f1e1415ed30 2024-04-05T22:29:31.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.755+0000 7f1e11400640 1 -- 172.21.15.73:0/1416595552 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0343f0 con 0x7f1e1415ed30 2024-04-05T22:29:31.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 -- 172.21.15.73:0/1416595552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 msgr2=0x7f1e1406bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 --2- 172.21.15.73:0/1416595552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009e50 tx=0x7f1e0c02bed0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 -- 172.21.15.73:0/1416595552 shutdown_connections 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 --2- 172.21.15.73:0/1416595552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415ed30 0x7f1e1406bc70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 --2- 172.21.15.73:0/1416595552 >> v2:172.21.15.157:3300/0 conn(0x7f1e14158860 0x7f1e140b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 -- 172.21.15.73:0/1416595552 >> 172.21.15.73:0/1416595552 conn(0x7f1e140b06c0 msgr2=0x7f1e140b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 -- 172.21.15.73:0/1416595552 shutdown_connections 2024-04-05T22:29:31.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.756+0000 7f1e13200640 1 -- 172.21.15.73:0/1416595552 wait complete. 2024-04-05T22:29:31.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.758+0000 7f1e13200640 1 Processor -- start 2024-04-05T22:29:31.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.758+0000 7f1e13200640 1 -- start start 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 0x7f1e1415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415d150 0x7f1e141512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1415d5d0 con 0x7f1e140b1f00 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14153040 con 0x7f1e1415ed30 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e13200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14151fa0 con 0x7f1e1415d150 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e11e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e11e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34468/0 (socket says 172.21.15.73:34468) 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.759+0000 7f1e11e00640 1 -- 172.21.15.73:0/477160094 learned_addr learned my addr 172.21.15.73:0/477160094 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e12800640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415d150 0x7f1e141512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e0be00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 0x7f1e1415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 -- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415d150 msgr2=0x7f1e141512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415d150 0x7f1e141512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 -- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 msgr2=0x7f1e1415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 0x7f1e1415cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 -- 172.21.15.73:0/477160094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c009b30 con 0x7f1e140b1f00 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e0be00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 0x7f1e1415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e11e00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400e9b0 tx=0x7f1e0400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.760+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04009670 con 0x7f1e140b1f00 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.761+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400c9a0 con 0x7f1e140b1f00 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.761+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400cd50 con 0x7f1e140b1f00 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.761+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e14152280 con 0x7f1e140b1f00 2024-04-05T22:29:31.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.761+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e14152840 con 0x7f1e140b1f00 2024-04-05T22:29:31.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.762+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e140c8ec0 con 0x7f1e140b1f00 2024-04-05T22:29:31.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.763+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f1e04013470 con 0x7f1e140b1f00 2024-04-05T22:29:31.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.763+0000 7f1e0a000640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 0x7f1df006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:31.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.764+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e0408a100 con 0x7f1e140b1f00 2024-04-05T22:29:31.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.764+0000 7f1e0be00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 0x7f1df006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:31.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.765+0000 7f1e0be00640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 0x7f1df006d660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e141583e0 tx=0x7f1dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:31.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.768+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1e040bd650 con 0x7f1e140b1f00 2024-04-05T22:29:31.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.957+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e1406b750 con 0x7f1df006b1a0 2024-04-05T22:29:31.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.980+0000 7f1e0a000640 1 -- 172.21.15.73:0/477160094 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284513 (secure 0 0 0) 0x7f1dfc001cc0 con 0x7f1df006b1a0 2024-04-05T22:29:31.985 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:31.987 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 msgr2=0x7f1df006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 0x7f1df006d660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e141583e0 tx=0x7f1dfc009040 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 msgr2=0x7f1e14149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400e9b0 tx=0x7f1e0400ee70 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 shutdown_connections 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1df006b1a0 0x7f1df006d660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1415d150 0x7f1e141512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1415ed30 0x7f1e1415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 --2- 172.21.15.73:0/477160094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e140b1f00 0x7f1e14149930 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 >> 172.21.15.73:0/477160094 conn(0x7f1e140b06c0 msgr2=0x7f1e140ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:31.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.987+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 shutdown_connections 2024-04-05T22:29:31.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:31.988+0000 7f1e13200640 1 -- 172.21.15.73:0/477160094 wait complete. 2024-04-05T22:29:32.006 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-05T22:29:31.564078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279508,"kb_used_data":14188,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754695212,"statfs":{"total":773094113280,"available":772807897088,"internally_reserved":0,"allocated":14528512,"data_stored":7050553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494764"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081084+0000","last_change":"2024-04-05T22:29:27.081084+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081084+0000","last_undegraded":"2024-04-05T22:29:27.081084+0000","last_fullsized":"2024-04-05T22:29:27.081084+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092518+0000","last_change":"2024-04-05T22:29:27.092518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092518+0000","last_undegraded":"2024-04-05T22:29:27.092518+0000","last_fullsized":"2024-04-05T22:29:27.092518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091256+0000","last_change":"2024-04-05T22:29:27.091256+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091256+0000","last_undegraded":"2024-04-05T22:29:27.091256+0000","last_fullsized":"2024-04-05T22:29:27.091256+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082145+0000","last_change":"2024-04-05T22:29:27.082145+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082145+0000","last_undegraded":"2024-04-05T22:29:27.082145+0000","last_fullsized":"2024-04-05T22:29:27.082145+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097597+0000","last_change":"2024-04-05T22:29:27.097597+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097597+0000","last_undegraded":"2024-04-05T22:29:27.097597+0000","last_fullsized":"2024-04-05T22:29:27.097597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082968+0000","last_change":"2024-04-05T22:29:27.082968+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082968+0000","last_undegraded":"2024-04-05T22:29:27.082968+0000","last_fullsized":"2024-04-05T22:29:27.082968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096950+0000","last_change":"2024-04-05T22:29:27.096950+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096950+0000","last_undegraded":"2024-04-05T22:29:27.096950+0000","last_fullsized":"2024-04-05T22:29:27.096950+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081332+0000","last_change":"2024-04-05T22:29:27.081332+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081332+0000","last_undegraded":"2024-04-05T22:29:27.081332+0000","last_fullsized":"2024-04-05T22:29:27.081332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092768+0000","last_change":"2024-04-05T22:29:27.092768+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092768+0000","last_undegraded":"2024-04-05T22:29:27.092768+0000","last_fullsized":"2024-04-05T22:29:27.092768+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.102673+0000","last_change":"2024-04-05T22:29:27.102673+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.102673+0000","last_undegraded":"2024-04-05T22:29:27.102673+0000","last_fullsized":"2024-04-05T22:29:27.102673+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089407+0000","last_change":"2024-04-05T22:29:27.089407+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089407+0000","last_undegraded":"2024-04-05T22:29:27.089407+0000","last_fullsized":"2024-04-05T22:29:27.089407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096067+0000","last_change":"2024-04-05T22:29:27.096067+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096067+0000","last_undegraded":"2024-04-05T22:29:27.096067+0000","last_fullsized":"2024-04-05T22:29:27.096067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094222+0000","last_change":"2024-04-05T22:29:27.094222+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094222+0000","last_undegraded":"2024-04-05T22:29:27.094222+0000","last_fullsized":"2024-04-05T22:29:27.094222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086244+0000","last_change":"2024-04-05T22:29:27.086244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086244+0000","last_undegraded":"2024-04-05T22:29:27.086244+0000","last_fullsized":"2024-04-05T22:29:27.086244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095219+0000","last_change":"2024-04-05T22:29:27.095219+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095219+0000","last_undegraded":"2024-04-05T22:29:27.095219+0000","last_fullsized":"2024-04-05T22:29:27.095219+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096108+0000","last_change":"2024-04-05T22:29:27.096108+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096108+0000","last_undegraded":"2024-04-05T22:29:27.096108+0000","last_fullsized":"2024-04-05T22:29:27.096108+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097011+0000","last_change":"2024-04-05T22:29:27.097011+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097011+0000","last_undegraded":"2024-04-05T22:29:27.097011+0000","last_fullsized":"2024-04-05T22:29:27.097011+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098055+0000","last_change":"2024-04-05T22:29:27.098055+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098055+0000","last_undegraded":"2024-04-05T22:29:27.098055+0000","last_fullsized":"2024-04-05T22:29:27.098055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084360+0000","last_change":"2024-04-05T22:29:27.084360+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084360+0000","last_undegraded":"2024-04-05T22:29:27.084360+0000","last_fullsized":"2024-04-05T22:29:27.084360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093190+0000","last_change":"2024-04-05T22:29:27.093190+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093190+0000","last_undegraded":"2024-04-05T22:29:27.093190+0000","last_fullsized":"2024-04-05T22:29:27.093190+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099854+0000","last_change":"2024-04-05T22:29:27.099854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099854+0000","last_undegraded":"2024-04-05T22:29:27.099854+0000","last_fullsized":"2024-04-05T22:29:27.099854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084747+0000","last_change":"2024-04-05T22:29:27.084747+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084747+0000","last_undegraded":"2024-04-05T22:29:27.084747+0000","last_fullsized":"2024-04-05T22:29:27.084747+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082244+0000","last_change":"2024-04-05T22:29:27.082244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082244+0000","last_undegraded":"2024-04-05T22:29:27.082244+0000","last_fullsized":"2024-04-05T22:29:27.082244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094229+0000","last_change":"2024-04-05T22:29:27.094229+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094229+0000","last_undegraded":"2024-04-05T22:29:27.094229+0000","last_fullsized":"2024-04-05T22:29:27.094229+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095176+0000","last_change":"2024-04-05T22:29:27.095176+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095176+0000","last_undegraded":"2024-04-05T22:29:27.095176+0000","last_fullsized":"2024-04-05T22:29:27.095176+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081906+0000","last_change":"2024-04-05T22:29:27.081906+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081906+0000","last_undegraded":"2024-04-05T22:29:27.081906+0000","last_fullsized":"2024-04-05T22:29:27.081906+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096447+0000","last_change":"2024-04-05T22:29:27.096447+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096447+0000","last_undegraded":"2024-04-05T22:29:27.096447+0000","last_fullsized":"2024-04-05T22:29:27.096447+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100820+0000","last_change":"2024-04-05T22:29:27.100820+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100820+0000","last_undegraded":"2024-04-05T22:29:27.100820+0000","last_fullsized":"2024-04-05T22:29:27.100820+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081518+0000","last_change":"2024-04-05T22:29:27.081518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081518+0000","last_undegraded":"2024-04-05T22:29:27.081518+0000","last_fullsized":"2024-04-05T22:29:27.081518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.083464+0000","last_change":"2024-04-05T22:29:27.083464+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.083464+0000","last_undegraded":"2024-04-05T22:29:27.083464+0000","last_fullsized":"2024-04-05T22:29:27.083464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090854+0000","last_change":"2024-04-05T22:29:27.090854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090854+0000","last_undegraded":"2024-04-05T22:29:27.090854+0000","last_fullsized":"2024-04-05T22:29:27.090854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094333+0000","last_change":"2024-04-05T22:29:27.094333+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094333+0000","last_undegraded":"2024-04-05T22:29:27.094333+0000","last_fullsized":"2024-04-05T22:29:27.094333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101884+0000","last_change":"2024-04-05T22:29:27.101884+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101884+0000","last_undegraded":"2024-04-05T22:29:27.101884+0000","last_fullsized":"2024-04-05T22:29:27.101884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116115+0000","last_change":"2024-04-05T22:29:27.113665+0000","last_active":"2024-04-05T22:29:27.116115+0000","last_peered":"2024-04-05T22:29:27.116115+0000","last_clean":"2024-04-05T22:29:27.116115+0000","last_became_active":"2024-04-05T22:29:27.113368+0000","last_became_peered":"2024-04-05T22:29:27.113368+0000","last_unstale":"2024-04-05T22:29:27.116115+0000","last_undegraded":"2024-04-05T22:29:27.116115+0000","last_fullsized":"2024-04-05T22:29:27.116115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:29.247549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091093+0000","last_change":"2024-04-05T22:29:27.091093+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091093+0000","last_undegraded":"2024-04-05T22:29:27.091093+0000","last_fullsized":"2024-04-05T22:29:27.091093+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116613+0000","last_change":"2024-04-05T22:29:27.115636+0000","last_active":"2024-04-05T22:29:27.116613+0000","last_peered":"2024-04-05T22:29:27.116613+0000","last_clean":"2024-04-05T22:29:27.116613+0000","last_became_active":"2024-04-05T22:29:27.115170+0000","last_became_peered":"2024-04-05T22:29:27.115170+0000","last_unstale":"2024-04-05T22:29:27.116613+0000","last_undegraded":"2024-04-05T22:29:27.116613+0000","last_fullsized":"2024-04-05T22:29:27.116613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:49.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081733+0000","last_change":"2024-04-05T22:29:27.081733+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081733+0000","last_undegraded":"2024-04-05T22:29:27.081733+0000","last_fullsized":"2024-04-05T22:29:27.081733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109271+0000","last_change":"2024-04-05T22:29:27.106460+0000","last_active":"2024-04-05T22:29:27.109271+0000","last_peered":"2024-04-05T22:29:27.109271+0000","last_clean":"2024-04-05T22:29:27.109271+0000","last_became_active":"2024-04-05T22:29:27.106113+0000","last_became_peered":"2024-04-05T22:29:27.106113+0000","last_unstale":"2024-04-05T22:29:27.109271+0000","last_undegraded":"2024-04-05T22:29:27.109271+0000","last_fullsized":"2024-04-05T22:29:27.109271+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:47.380812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095782+0000","last_change":"2024-04-05T22:29:27.095782+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095782+0000","last_undegraded":"2024-04-05T22:29:27.095782+0000","last_fullsized":"2024-04-05T22:29:27.095782+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100802+0000","last_change":"2024-04-05T22:29:27.098092+0000","last_active":"2024-04-05T22:29:27.100802+0000","last_peered":"2024-04-05T22:29:27.100802+0000","last_clean":"2024-04-05T22:29:27.100802+0000","last_became_active":"2024-04-05T22:29:27.097449+0000","last_became_peered":"2024-04-05T22:29:27.097449+0000","last_unstale":"2024-04-05T22:29:27.100802+0000","last_undegraded":"2024-04-05T22:29:27.100802+0000","last_fullsized":"2024-04-05T22:29:27.100802+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:40.984286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092341+0000","last_change":"2024-04-05T22:29:27.092341+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092341+0000","last_undegraded":"2024-04-05T22:29:27.092341+0000","last_fullsized":"2024-04-05T22:29:27.092341+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110010+0000","last_change":"2024-04-05T22:29:27.106850+0000","last_active":"2024-04-05T22:29:27.110010+0000","last_peered":"2024-04-05T22:29:27.110010+0000","last_clean":"2024-04-05T22:29:27.110010+0000","last_became_active":"2024-04-05T22:29:27.106525+0000","last_became_peered":"2024-04-05T22:29:27.106525+0000","last_unstale":"2024-04-05T22:29:27.110010+0000","last_undegraded":"2024-04-05T22:29:27.110010+0000","last_fullsized":"2024-04-05T22:29:27.110010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:30.113756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082197+0000","last_change":"2024-04-05T22:29:27.082197+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082197+0000","last_undegraded":"2024-04-05T22:29:27.082197+0000","last_fullsized":"2024-04-05T22:29:27.082197+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.117465+0000","last_change":"2024-04-05T22:29:27.113913+0000","last_active":"2024-04-05T22:29:27.117465+0000","last_peered":"2024-04-05T22:29:27.117465+0000","last_clean":"2024-04-05T22:29:27.117465+0000","last_became_active":"2024-04-05T22:29:27.112452+0000","last_became_peered":"2024-04-05T22:29:27.112452+0000","last_unstale":"2024-04-05T22:29:27.117465+0000","last_undegraded":"2024-04-05T22:29:27.117465+0000","last_fullsized":"2024-04-05T22:29:27.117465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:13:41.450507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089823+0000","last_change":"2024-04-05T22:29:27.089823+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089823+0000","last_undegraded":"2024-04-05T22:29:27.089823+0000","last_fullsized":"2024-04-05T22:29:27.089823+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.108934+0000","last_change":"2024-04-05T22:29:27.105830+0000","last_active":"2024-04-05T22:29:27.108934+0000","last_peered":"2024-04-05T22:29:27.108934+0000","last_clean":"2024-04-05T22:29:27.108934+0000","last_became_active":"2024-04-05T22:29:27.104458+0000","last_became_peered":"2024-04-05T22:29:27.104458+0000","last_unstale":"2024-04-05T22:29:27.108934+0000","last_undegraded":"2024-04-05T22:29:27.108934+0000","last_fullsized":"2024-04-05T22:29:27.108934+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:34.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096482+0000","last_change":"2024-04-05T22:29:27.096482+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096482+0000","last_undegraded":"2024-04-05T22:29:27.096482+0000","last_fullsized":"2024-04-05T22:29:27.096482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109450+0000","last_change":"2024-04-05T22:29:27.107183+0000","last_active":"2024-04-05T22:29:27.109450+0000","last_peered":"2024-04-05T22:29:27.109450+0000","last_clean":"2024-04-05T22:29:27.109450+0000","last_became_active":"2024-04-05T22:29:27.106873+0000","last_became_peered":"2024-04-05T22:29:27.106873+0000","last_unstale":"2024-04-05T22:29:27.109450+0000","last_undegraded":"2024-04-05T22:29:27.109450+0000","last_fullsized":"2024-04-05T22:29:27.109450+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:31.435759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098271+0000","last_change":"2024-04-05T22:29:27.098271+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098271+0000","last_undegraded":"2024-04-05T22:29:27.098271+0000","last_fullsized":"2024-04-05T22:29:27.098271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100170+0000","last_change":"2024-04-05T22:29:27.098118+0000","last_active":"2024-04-05T22:29:27.100170+0000","last_peered":"2024-04-05T22:29:27.100170+0000","last_clean":"2024-04-05T22:29:27.100170+0000","last_became_active":"2024-04-05T22:29:27.097772+0000","last_became_peered":"2024-04-05T22:29:27.097772+0000","last_unstale":"2024-04-05T22:29:27.100170+0000","last_undegraded":"2024-04-05T22:29:27.100170+0000","last_fullsized":"2024-04-05T22:29:27.100170+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:04.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098638+0000","last_change":"2024-04-05T22:29:27.098638+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098638+0000","last_undegraded":"2024-04-05T22:29:27.098638+0000","last_fullsized":"2024-04-05T22:29:27.098638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118528+0000","last_change":"2024-04-05T22:29:27.117368+0000","last_active":"2024-04-05T22:29:27.118528+0000","last_peered":"2024-04-05T22:29:27.118528+0000","last_clean":"2024-04-05T22:29:27.118528+0000","last_became_active":"2024-04-05T22:29:27.117006+0000","last_became_peered":"2024-04-05T22:29:27.117006+0000","last_unstale":"2024-04-05T22:29:27.118528+0000","last_undegraded":"2024-04-05T22:29:27.118528+0000","last_fullsized":"2024-04-05T22:29:27.118528+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:26.585990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093113+0000","last_change":"2024-04-05T22:29:27.093113+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093113+0000","last_undegraded":"2024-04-05T22:29:27.093113+0000","last_fullsized":"2024-04-05T22:29:27.093113+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109634+0000","last_change":"2024-04-05T22:29:27.107727+0000","last_active":"2024-04-05T22:29:27.109634+0000","last_peered":"2024-04-05T22:29:27.109634+0000","last_clean":"2024-04-05T22:29:27.109634+0000","last_became_active":"2024-04-05T22:29:27.107409+0000","last_became_peered":"2024-04-05T22:29:27.107409+0000","last_unstale":"2024-04-05T22:29:27.109634+0000","last_undegraded":"2024-04-05T22:29:27.109634+0000","last_fullsized":"2024-04-05T22:29:27.109634+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:36:16.139153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096703+0000","last_change":"2024-04-05T22:29:27.096703+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096703+0000","last_undegraded":"2024-04-05T22:29:27.096703+0000","last_fullsized":"2024-04-05T22:29:27.096703+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112336+0000","last_change":"2024-04-05T22:29:27.109281+0000","last_active":"2024-04-05T22:29:27.112336+0000","last_peered":"2024-04-05T22:29:27.112336+0000","last_clean":"2024-04-05T22:29:27.112336+0000","last_became_active":"2024-04-05T22:29:27.108992+0000","last_became_peered":"2024-04-05T22:29:27.108992+0000","last_unstale":"2024-04-05T22:29:27.112336+0000","last_undegraded":"2024-04-05T22:29:27.112336+0000","last_fullsized":"2024-04-05T22:29:27.112336+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:33:12.557477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091308+0000","last_change":"2024-04-05T22:29:27.091308+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091308+0000","last_undegraded":"2024-04-05T22:29:27.091308+0000","last_fullsized":"2024-04-05T22:29:27.091308+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112144+0000","last_change":"2024-04-05T22:29:27.109031+0000","last_active":"2024-04-05T22:29:27.112144+0000","last_peered":"2024-04-05T22:29:27.112144+0000","last_clean":"2024-04-05T22:29:27.112144+0000","last_became_active":"2024-04-05T22:29:27.108620+0000","last_became_peered":"2024-04-05T22:29:27.108620+0000","last_unstale":"2024-04-05T22:29:27.112144+0000","last_undegraded":"2024-04-05T22:29:27.112144+0000","last_fullsized":"2024-04-05T22:29:27.112144+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:46.523577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092972+0000","last_change":"2024-04-05T22:29:27.092972+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092972+0000","last_undegraded":"2024-04-05T22:29:27.092972+0000","last_fullsized":"2024-04-05T22:29:27.092972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107366+0000","last_change":"2024-04-05T22:29:27.103901+0000","last_active":"2024-04-05T22:29:27.107366+0000","last_peered":"2024-04-05T22:29:27.107366+0000","last_clean":"2024-04-05T22:29:27.107366+0000","last_became_active":"2024-04-05T22:29:27.103615+0000","last_became_peered":"2024-04-05T22:29:27.103615+0000","last_unstale":"2024-04-05T22:29:27.107366+0000","last_undegraded":"2024-04-05T22:29:27.107366+0000","last_fullsized":"2024-04-05T22:29:27.107366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:46.499208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095841+0000","last_change":"2024-04-05T22:29:27.095841+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095841+0000","last_undegraded":"2024-04-05T22:29:27.095841+0000","last_fullsized":"2024-04-05T22:29:27.095841+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106936+0000","last_change":"2024-04-05T22:29:27.104205+0000","last_active":"2024-04-05T22:29:27.106936+0000","last_peered":"2024-04-05T22:29:27.106936+0000","last_clean":"2024-04-05T22:29:27.106936+0000","last_became_active":"2024-04-05T22:29:27.102618+0000","last_became_peered":"2024-04-05T22:29:27.102618+0000","last_unstale":"2024-04-05T22:29:27.106936+0000","last_undegraded":"2024-04-05T22:29:27.106936+0000","last_fullsized":"2024-04-05T22:29:27.106936+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:05.414218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096852+0000","last_change":"2024-04-05T22:29:27.096852+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096852+0000","last_undegraded":"2024-04-05T22:29:27.096852+0000","last_fullsized":"2024-04-05T22:29:27.096852+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116552+0000","last_change":"2024-04-05T22:29:27.113619+0000","last_active":"2024-04-05T22:29:27.116552+0000","last_peered":"2024-04-05T22:29:27.116552+0000","last_clean":"2024-04-05T22:29:27.116552+0000","last_became_active":"2024-04-05T22:29:27.113367+0000","last_became_peered":"2024-04-05T22:29:27.113367+0000","last_unstale":"2024-04-05T22:29:27.116552+0000","last_undegraded":"2024-04-05T22:29:27.116552+0000","last_fullsized":"2024-04-05T22:29:27.116552+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:26:24.419813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093979+0000","last_change":"2024-04-05T22:29:27.093979+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093979+0000","last_undegraded":"2024-04-05T22:29:27.093979+0000","last_fullsized":"2024-04-05T22:29:27.093979+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106232+0000","last_change":"2024-04-05T22:29:27.103848+0000","last_active":"2024-04-05T22:29:27.106232+0000","last_peered":"2024-04-05T22:29:27.106232+0000","last_clean":"2024-04-05T22:29:27.106232+0000","last_became_active":"2024-04-05T22:29:27.103586+0000","last_became_peered":"2024-04-05T22:29:27.103586+0000","last_unstale":"2024-04-05T22:29:27.106232+0000","last_undegraded":"2024-04-05T22:29:27.106232+0000","last_fullsized":"2024-04-05T22:29:27.106232+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:11.324558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092715+0000","last_change":"2024-04-05T22:29:27.092715+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092715+0000","last_undegraded":"2024-04-05T22:29:27.092715+0000","last_fullsized":"2024-04-05T22:29:27.092715+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109481+0000","last_change":"2024-04-05T22:29:27.106062+0000","last_active":"2024-04-05T22:29:27.109481+0000","last_peered":"2024-04-05T22:29:27.109481+0000","last_clean":"2024-04-05T22:29:27.109481+0000","last_became_active":"2024-04-05T22:29:27.105748+0000","last_became_peered":"2024-04-05T22:29:27.105748+0000","last_unstale":"2024-04-05T22:29:27.109481+0000","last_undegraded":"2024-04-05T22:29:27.109481+0000","last_fullsized":"2024-04-05T22:29:27.109481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:48.841874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086784+0000","last_change":"2024-04-05T22:29:27.086784+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086784+0000","last_undegraded":"2024-04-05T22:29:27.086784+0000","last_fullsized":"2024-04-05T22:29:27.086784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.105147+0000","last_change":"2024-04-05T22:29:27.102360+0000","last_active":"2024-04-05T22:29:27.105147+0000","last_peered":"2024-04-05T22:29:27.105147+0000","last_clean":"2024-04-05T22:29:27.105147+0000","last_became_active":"2024-04-05T22:29:27.102051+0000","last_became_peered":"2024-04-05T22:29:27.102051+0000","last_unstale":"2024-04-05T22:29:27.105147+0000","last_undegraded":"2024-04-05T22:29:27.105147+0000","last_fullsized":"2024-04-05T22:29:27.105147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:47.073194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081980+0000","last_change":"2024-04-05T22:29:27.081980+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081980+0000","last_undegraded":"2024-04-05T22:29:27.081980+0000","last_fullsized":"2024-04-05T22:29:27.081980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112449+0000","last_change":"2024-04-05T22:29:27.111825+0000","last_active":"2024-04-05T22:29:27.112449+0000","last_peered":"2024-04-05T22:29:27.112449+0000","last_clean":"2024-04-05T22:29:27.112449+0000","last_became_active":"2024-04-05T22:29:27.111542+0000","last_became_peered":"2024-04-05T22:29:27.111542+0000","last_unstale":"2024-04-05T22:29:27.112449+0000","last_undegraded":"2024-04-05T22:29:27.112449+0000","last_fullsized":"2024-04-05T22:29:27.112449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:22.051262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100124+0000","last_change":"2024-04-05T22:29:27.100124+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100124+0000","last_undegraded":"2024-04-05T22:29:27.100124+0000","last_fullsized":"2024-04-05T22:29:27.100124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.114102+0000","last_change":"2024-04-05T22:29:27.111984+0000","last_active":"2024-04-05T22:29:27.114102+0000","last_peered":"2024-04-05T22:29:27.114102+0000","last_clean":"2024-04-05T22:29:27.114102+0000","last_became_active":"2024-04-05T22:29:27.111570+0000","last_became_peered":"2024-04-05T22:29:27.111570+0000","last_unstale":"2024-04-05T22:29:27.114102+0000","last_undegraded":"2024-04-05T22:29:27.114102+0000","last_fullsized":"2024-04-05T22:29:27.114102+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:02:55.012896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081843+0000","last_change":"2024-04-05T22:29:27.081843+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081843+0000","last_undegraded":"2024-04-05T22:29:27.081843+0000","last_fullsized":"2024-04-05T22:29:27.081843+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118512+0000","last_change":"2024-04-05T22:29:27.117354+0000","last_active":"2024-04-05T22:29:27.118512+0000","last_peered":"2024-04-05T22:29:27.118512+0000","last_clean":"2024-04-05T22:29:27.118512+0000","last_became_active":"2024-04-05T22:29:27.116968+0000","last_became_peered":"2024-04-05T22:29:27.116968+0000","last_unstale":"2024-04-05T22:29:27.118512+0000","last_undegraded":"2024-04-05T22:29:27.118512+0000","last_fullsized":"2024-04-05T22:29:27.118512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:32:03.796707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101316+0000","last_change":"2024-04-05T22:29:27.101316+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101316+0000","last_undegraded":"2024-04-05T22:29:27.101316+0000","last_fullsized":"2024-04-05T22:29:27.101316+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110905+0000","last_change":"2024-04-05T22:29:27.104579+0000","last_active":"2024-04-05T22:29:27.110905+0000","last_peered":"2024-04-05T22:29:27.110905+0000","last_clean":"2024-04-05T22:29:27.110905+0000","last_became_active":"2024-04-05T22:29:27.104293+0000","last_became_peered":"2024-04-05T22:29:27.104293+0000","last_unstale":"2024-04-05T22:29:27.110905+0000","last_undegraded":"2024-04-05T22:29:27.110905+0000","last_fullsized":"2024-04-05T22:29:27.110905+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:13:51.613749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097362+0000","last_change":"2024-04-05T22:29:27.097362+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097362+0000","last_undegraded":"2024-04-05T22:29:27.097362+0000","last_fullsized":"2024-04-05T22:29:27.097362+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116901+0000","last_change":"2024-04-05T22:29:27.115301+0000","last_active":"2024-04-05T22:29:27.116901+0000","last_peered":"2024-04-05T22:29:27.116901+0000","last_clean":"2024-04-05T22:29:27.116901+0000","last_became_active":"2024-04-05T22:29:27.114776+0000","last_became_peered":"2024-04-05T22:29:27.114776+0000","last_unstale":"2024-04-05T22:29:27.116901+0000","last_undegraded":"2024-04-05T22:29:27.116901+0000","last_fullsized":"2024-04-05T22:29:27.116901+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:53.380117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099085+0000","last_change":"2024-04-05T22:29:27.099085+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099085+0000","last_undegraded":"2024-04-05T22:29:27.099085+0000","last_fullsized":"2024-04-05T22:29:27.099085+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107334+0000","last_change":"2024-04-05T22:29:27.105399+0000","last_active":"2024-04-05T22:29:27.107334+0000","last_peered":"2024-04-05T22:29:27.107334+0000","last_clean":"2024-04-05T22:29:27.107334+0000","last_became_active":"2024-04-05T22:29:27.105109+0000","last_became_peered":"2024-04-05T22:29:27.105109+0000","last_unstale":"2024-04-05T22:29:27.107334+0000","last_undegraded":"2024-04-05T22:29:27.107334+0000","last_fullsized":"2024-04-05T22:29:27.107334+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:28.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090127+0000","last_change":"2024-04-05T22:29:27.090127+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090127+0000","last_undegraded":"2024-04-05T22:29:27.090127+0000","last_fullsized":"2024-04-05T22:29:27.090127+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097140+0000","last_change":"2024-04-05T22:29:27.094375+0000","last_active":"2024-04-05T22:29:27.097140+0000","last_peered":"2024-04-05T22:29:27.097140+0000","last_clean":"2024-04-05T22:29:27.097140+0000","last_became_active":"2024-04-05T22:29:27.094043+0000","last_became_peered":"2024-04-05T22:29:27.094043+0000","last_unstale":"2024-04-05T22:29:27.097140+0000","last_undegraded":"2024-04-05T22:29:27.097140+0000","last_fullsized":"2024-04-05T22:29:27.097140+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:36.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.085671+0000","last_change":"2024-04-05T22:29:27.085671+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.085671+0000","last_undegraded":"2024-04-05T22:29:27.085671+0000","last_fullsized":"2024-04-05T22:29:27.085671+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097067+0000","last_change":"2024-04-05T22:29:27.092419+0000","last_active":"2024-04-05T22:29:27.097067+0000","last_peered":"2024-04-05T22:29:27.097067+0000","last_clean":"2024-04-05T22:29:27.097067+0000","last_became_active":"2024-04-05T22:29:27.091178+0000","last_became_peered":"2024-04-05T22:29:27.091178+0000","last_unstale":"2024-04-05T22:29:27.097067+0000","last_undegraded":"2024-04-05T22:29:27.097067+0000","last_fullsized":"2024-04-05T22:29:27.097067+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:31.397216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092775+0000","last_change":"2024-04-05T22:29:27.092775+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092775+0000","last_undegraded":"2024-04-05T22:29:27.092775+0000","last_fullsized":"2024-04-05T22:29:27.092775+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109970+0000","last_change":"2024-04-05T22:29:27.108317+0000","last_active":"2024-04-05T22:29:27.109970+0000","last_peered":"2024-04-05T22:29:27.109970+0000","last_clean":"2024-04-05T22:29:27.109970+0000","last_became_active":"2024-04-05T22:29:27.107888+0000","last_became_peered":"2024-04-05T22:29:27.107888+0000","last_unstale":"2024-04-05T22:29:27.109970+0000","last_undegraded":"2024-04-05T22:29:27.109970+0000","last_fullsized":"2024-04-05T22:29:27.109970+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:42.291460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092874+0000","last_change":"2024-04-05T22:29:27.092874+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092874+0000","last_undegraded":"2024-04-05T22:29:27.092874+0000","last_fullsized":"2024-04-05T22:29:27.092874+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116811+0000","last_change":"2024-04-05T22:29:27.114605+0000","last_active":"2024-04-05T22:29:27.116811+0000","last_peered":"2024-04-05T22:29:27.116811+0000","last_clean":"2024-04-05T22:29:27.116811+0000","last_became_active":"2024-04-05T22:29:27.114311+0000","last_became_peered":"2024-04-05T22:29:27.114311+0000","last_unstale":"2024-04-05T22:29:27.116811+0000","last_undegraded":"2024-04-05T22:29:27.116811+0000","last_fullsized":"2024-04-05T22:29:27.116811+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:06.440778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092986+0000","last_change":"2024-04-05T22:29:27.092986+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092986+0000","last_undegraded":"2024-04-05T22:29:27.092986+0000","last_fullsized":"2024-04-05T22:29:27.092986+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112429+0000","last_change":"2024-04-05T22:29:27.109061+0000","last_active":"2024-04-05T22:29:27.112429+0000","last_peered":"2024-04-05T22:29:27.112429+0000","last_clean":"2024-04-05T22:29:27.112429+0000","last_became_active":"2024-04-05T22:29:27.108742+0000","last_became_peered":"2024-04-05T22:29:27.108742+0000","last_unstale":"2024-04-05T22:29:27.112429+0000","last_undegraded":"2024-04-05T22:29:27.112429+0000","last_fullsized":"2024-04-05T22:29:27.112429+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:26:35.912596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094988+0000","last_change":"2024-04-05T22:29:27.094988+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094988+0000","last_undegraded":"2024-04-05T22:29:27.094988+0000","last_fullsized":"2024-04-05T22:29:27.094988+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116214+0000","last_change":"2024-04-05T22:29:27.115510+0000","last_active":"2024-04-05T22:29:27.116214+0000","last_peered":"2024-04-05T22:29:27.116214+0000","last_clean":"2024-04-05T22:29:27.116214+0000","last_became_active":"2024-04-05T22:29:27.115162+0000","last_became_peered":"2024-04-05T22:29:27.115162+0000","last_unstale":"2024-04-05T22:29:27.116214+0000","last_undegraded":"2024-04-05T22:29:27.116214+0000","last_fullsized":"2024-04-05T22:29:27.116214+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:44.332750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096436+0000","last_change":"2024-04-05T22:29:27.096436+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096436+0000","last_undegraded":"2024-04-05T22:29:27.096436+0000","last_fullsized":"2024-04-05T22:29:27.096436+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.111421+0000","last_change":"2024-04-05T22:29:27.107244+0000","last_active":"2024-04-05T22:29:27.111421+0000","last_peered":"2024-04-05T22:29:27.111421+0000","last_clean":"2024-04-05T22:29:27.111421+0000","last_became_active":"2024-04-05T22:29:27.106979+0000","last_became_peered":"2024-04-05T22:29:27.106979+0000","last_unstale":"2024-04-05T22:29:27.111421+0000","last_undegraded":"2024-04-05T22:29:27.111421+0000","last_fullsized":"2024-04-05T22:29:27.111421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:28:57.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":299,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-05T22:29:26.085340+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:29:26.085340+0000","last_peered":"2024-04-05T22:29:26.085340+0000","last_clean":"2024-04-05T22:29:26.085340+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:29:26.085340+0000","last_undegraded":"2024-04-05T22:29:26.085340+0000","last_fullsized":"2024-04-05T22:29:26.085340+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.21}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.085},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.401}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.597},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.6339999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":1.026},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.534},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.488},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1.6870000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.6679999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.1499999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":1.0880000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.196},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.343}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.29},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.377}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.258},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.431}]}]},{"osd":5,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.115,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.097},"min":{"1min":0.68300000000000005,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.5900000000000001,"5min":1.907,"15min":1.907},"last":0.999}]},{"osd":1,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.76600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3919999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.98399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6359999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.177,"15min":1.1599999999999999},"min":{"1min":0.82599999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3759999999999999,"5min":2.544,"15min":2.544},"last":1.25},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.89900000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5840000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0589999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1859999999999999,"15min":1.181},"min":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7529999999999999,"5min":1.867,"15min":1.867},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.268,"5min":1.2909999999999999,"15min":1.3460000000000001},"min":{"1min":1.03,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.649,"5min":2.927,"15min":2.9620000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7050000000000001,"5min":1.911,"15min":2.1970000000000001},"last":1.405},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2969999999999999,"15min":1.302},"min":{"1min":0.89900000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.784,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.79,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5129999999999999,"15min":1.5},"min":{"1min":1.284,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.9079999999999999,"5min":2.169,"15min":2.5350000000000001},"last":1.599}]},{"osd":7,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.464,"15min":1.456},"min":{"1min":1.2010000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.919,"5min":2.028,"15min":2.2509999999999999},"last":1.448},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5,"15min":1.597},"min":{"1min":1.1799999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":4.4329999999999998},"last":1.5169999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1100000000000001,"15min":1.0249999999999999},"min":{"1min":0.82599999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.466,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.085},"min":{"1min":0.73199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5369999999999999,"5min":1.859,"15min":1.859},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1279999999999999},"min":{"1min":0.88100000000000001,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":1.0009999999999999,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.659,"5min":2.577,"15min":2.577},"last":1.5029999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1319999999999999,"15min":1.202},"min":{"1min":0.75,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.974},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2450000000000001,"15min":1.2},"min":{"1min":1.0389999999999999,"5min":0.76200000000000001,"15min":0.67400000000000004},"max":{"1min":1.657,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.745}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3200000000000001,"15min":1.3640000000000001},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7889999999999999,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3919999999999999,"15min":1.403},"min":{"1min":1.196,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.7509999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.573}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":1.0529999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1459999999999999,"5min":2.665,"15min":2.665},"last":1.282},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.375,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.915,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.6180000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.4139999999999999,"15min":1.4550000000000001},"min":{"1min":1.0229999999999999,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":1.9950000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":1.212,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.8899999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.524,"15min":1.5389999999999999},"min":{"1min":1.0760000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0019999999999998,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.764},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6259999999999999,"15min":1.591},"min":{"1min":1.3839999999999999,"5min":0.91300000000000003,"15min":0.72399999999999998},"max":{"1min":2.109,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":2.0409999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1675264,"data_stored":737649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0229999999999999,"5min":1.744,"15min":1.744},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5860000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.111}]},{"osd":1,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":1.006},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.3720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.325},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0580000000000001,"15min":1.073},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.391,"5min":1.675,"15min":2.2549999999999999},"last":1.274}]},{"osd":4,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1899999999999999,"15min":1.2290000000000001},"min":{"1min":0.82099999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5289999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.294},"min":{"1min":0.75900000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.399}]},{"osd":5,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.286,"15min":1.351},"min":{"1min":0.73699999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.482},{"interface":"front","average":{"1min":1.246,"5min":1.288,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6459999999999999,"5min":1.958,"15min":2.3239999999999998},"last":1.6579999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.3009999999999999,"15min":1.3500000000000001},"min":{"1min":0.875,"5min":0.505,"15min":0.505},"max":{"1min":1.6240000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.347,"15min":1.401},"min":{"1min":0.97899999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.298,"15min":2.298},"last":1.7}]},{"osd":7,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3400000000000001,"15min":1.415},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":2.468},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.377,"15min":1.415},"min":{"1min":0.93700000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.73,"5min":2.508,"15min":2.798},"last":1.8160000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542246,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.4319999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.496}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.165}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2030000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.363,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.89900000000000002,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.42299999999999999,"15min":0.36799999999999999},"max":{"1min":1.554,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85399999999999998,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.206,"5min":1.488,"15min":1.5389999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.999,"15min":0.99399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0880000000000001,"15min":1.097},"min":{"1min":0.53600000000000003,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.484,"5min":1.5509999999999999,"15min":1.75},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.3959999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":1.125},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6679999999999999,"5min":1.806,"15min":2.2789999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.258,"15min":1.262},"min":{"1min":0.84099999999999997,"5min":0.76000000000000001,"15min":0.68100000000000005},"max":{"1min":1.706,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.873,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.5429999999999999,"5min":1.843,"15min":2.173},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.369,"15min":1.3340000000000001},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6439999999999999,"5min":1.9059999999999999,"15min":1.917},"last":1.264},{"interface":"front","average":{"1min":1.292,"5min":1.345,"15min":1.355},"min":{"1min":0.92200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.825,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.407},{"interface":"front","average":{"1min":1.196,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.6319999999999999,"5min":1.9279999999999999,"15min":2.3879999999999999},"last":1.429}]}]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.1479999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.92600000000000005,"15min":0.92000000000000004},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2569999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.2729999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.25,"5min":1.976,"15min":1.976},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0840000000000001,"15min":1.093},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5900000000000001,"5min":2.157,"15min":2.157},"last":0.749}]},{"osd":4,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2,"15min":1.1699999999999999},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.429,"5min":1.931,"15min":1.931},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3129999999999999,"15min":1.302},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.786,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.0549999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2549999999999999,"15min":1.2609999999999999},"min":{"1min":0.75700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.524,"5min":2.105,"15min":2.3879999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.2130000000000001,"15min":1.198},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.357,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.878,"15min":0.65500000000000003},"max":{"1min":1.8340000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3280000000000001,"15min":1.365},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.585,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3320000000000001,"15min":1.337},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.6080000000000001,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.151}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:29:32.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:29:32.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.259+0000 7f8da1200640 1 Processor -- start 2024-04-05T22:29:32.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.260+0000 7f8da1200640 1 -- start start 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.261+0000 7f8da1200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d9c1587d0 0x7f8d9c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.261+0000 7f8da1200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d9c15e850 0x7f8d9c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.261+0000 7f8da1200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c06f5b0 con 0x7f8d9c0ccd20 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.261+0000 7f8da1200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c15f240 con 0x7f8d9c15e850 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.261+0000 7f8da1200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c070410 con 0x7f8d9c1587d0 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d93e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8d9c0ccd20 0x7f8d9c06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53194/0 (socket says 172.21.15.73:53194) 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d93e00640 1 -- 172.21.15.73:0/2844787977 learned_addr learned my addr 172.21.15.73:0/2844787977 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:32.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d9a800640 1 --2- 172.21.15.73:0/2844787977 >> v2:172.21.15.73:3300/0 conn(0x7f8d9c15e850 0x7f8d9c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d99e00640 1 --2- 172.21.15.73:0/2844787977 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c1587d0 0x7f8d9c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d9a800640 1 -- 172.21.15.73:0/2844787977 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c1587d0 msgr2=0x7f8d9c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d9a800640 1 --2- 172.21.15.73:0/2844787977 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c1587d0 0x7f8d9c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d9a800640 1 -- 172.21.15.73:0/2844787977 >> v1:172.21.15.73:6789/0 conn(0x7f8d9c0ccd20 legacy=0x7f8d9c06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d99400640 1 -- 172.21.15.73:0/2844787977 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732674937 0 0) 0x7f8d9c06f5b0 con 0x7f8d9c0ccd20 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.262+0000 7f8d9a800640 1 -- 172.21.15.73:0/2844787977 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c158bb0 con 0x7f8d9c15e850 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.263+0000 7f8d9a800640 1 --2- 172.21.15.73:0/2844787977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c15e850 0x7f8d9c15ed00 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009d90 tx=0x7f8d9402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c00085d3d1a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8d99400640 1 -- 172.21.15.73:0/2844787977 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9402cc60 con 0x7f8d9c15e850 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8d99400640 1 -- 172.21.15.73:0/2844787977 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9402cdc0 con 0x7f8d9c15e850 2024-04-05T22:29:32.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8d99400640 1 -- 172.21.15.73:0/2844787977 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94034660 con 0x7f8d9c15e850 2024-04-05T22:29:32.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 -- 172.21.15.73:0/2844787977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c15e850 msgr2=0x7f8d9c15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 --2- 172.21.15.73:0/2844787977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c15e850 0x7f8d9c15ed00 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009d90 tx=0x7f8d9402beb0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 -- 172.21.15.73:0/2844787977 shutdown_connections 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 --2- 172.21.15.73:0/2844787977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c15e850 0x7f8d9c15ed00 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 --2- 172.21.15.73:0/2844787977 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c1587d0 0x7f8d9c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 -- 172.21.15.73:0/2844787977 >> 172.21.15.73:0/2844787977 conn(0x7f8d9c0ce8e0 msgr2=0x7f8d9c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 -- 172.21.15.73:0/2844787977 shutdown_connections 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.264+0000 7f8da1200640 1 -- 172.21.15.73:0/2844787977 wait complete. 2024-04-05T22:29:32.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.266+0000 7f8da1200640 1 Processor -- start 2024-04-05T22:29:32.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.266+0000 7f8da1200640 1 -- start start 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15e850 0x7f8d9c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c1475d0 0x7f8d9c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1440e0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1442f0 con 0x7f8d9c15e850 2024-04-05T22:29:32.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8da1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1445d0 con 0x7f8d9c1475d0 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8d99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8d99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34482/0 (socket says 172.21.15.73:34482) 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.267+0000 7f8d99e00640 1 -- 172.21.15.73:0/1917271683 learned_addr learned my addr 172.21.15.73:0/1917271683 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 -- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c1475d0 msgr2=0x7f8d9c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c1475d0 0x7f8d9c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 -- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15e850 msgr2=0x7f8d9c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15e850 0x7f8d9c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 -- 172.21.15.73:0/1917271683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d94009a70 con 0x7f8d9c1587d0 2024-04-05T22:29:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.268+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d800 tx=0x7f8d8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.269+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00b6c0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.269+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c00b820 con 0x7f8d9c1587d0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.269+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00bb90 con 0x7f8d9c1587d0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.269+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c1448b0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.269+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c146440 con 0x7f8d9c1587d0 2024-04-05T22:29:32.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.270+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c066af0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.271+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f8d8c00bcf0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.272+0000 7f8d92000640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 0x7f8d7806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.272+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d8c08a390 con 0x7f8d9c1587d0 2024-04-05T22:29:32.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.273+0000 7f8d93e00640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 0x7f8d7806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.274+0000 7f8d93e00640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 0x7f8d7806d6e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007fd0 tx=0x7f8d84007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.276+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8d8c054cb0 con 0x7f8d9c1587d0 2024-04-05T22:29:32.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.465+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d9c06fdc0 con 0x7f8d7806b220 2024-04-05T22:29:32.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.487+0000 7f8d92000640 1 -- 172.21.15.73:0/1917271683 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284513 (secure 0 0 0) 0x7f8d84007070 con 0x7f8d7806b220 2024-04-05T22:29:32.492 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:32.493 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:29:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 msgr2=0x7f8d7806d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 0x7f8d7806d6e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007fd0 tx=0x7f8d84007920 comp rx=0 tx=0).stop 2024-04-05T22:29:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 msgr2=0x7f8d9c1f01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d800 tx=0x7f8d8c00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 shutdown_connections 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7806b220 0x7f8d7806d6e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c1475d0 0x7f8d9c0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15e850 0x7f8d9c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.494+0000 7f8da1200640 1 --2- 172.21.15.73:0/1917271683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c1587d0 0x7f8d9c1f01a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.495+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 >> 172.21.15.73:0/1917271683 conn(0x7f8d9c0ce8e0 msgr2=0x7f8d9c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.495+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 shutdown_connections 2024-04-05T22:29:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.495+0000 7f8da1200640 1 -- 172.21.15.73:0/1917271683 wait complete. 2024-04-05T22:29:32.513 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-05T22:29:31.564078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279508,"kb_used_data":14188,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754695212,"statfs":{"total":773094113280,"available":772807897088,"internally_reserved":0,"allocated":14528512,"data_stored":7050553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494764"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081084+0000","last_change":"2024-04-05T22:29:27.081084+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081084+0000","last_undegraded":"2024-04-05T22:29:27.081084+0000","last_fullsized":"2024-04-05T22:29:27.081084+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092518+0000","last_change":"2024-04-05T22:29:27.092518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092518+0000","last_undegraded":"2024-04-05T22:29:27.092518+0000","last_fullsized":"2024-04-05T22:29:27.092518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091256+0000","last_change":"2024-04-05T22:29:27.091256+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091256+0000","last_undegraded":"2024-04-05T22:29:27.091256+0000","last_fullsized":"2024-04-05T22:29:27.091256+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082145+0000","last_change":"2024-04-05T22:29:27.082145+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082145+0000","last_undegraded":"2024-04-05T22:29:27.082145+0000","last_fullsized":"2024-04-05T22:29:27.082145+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097597+0000","last_change":"2024-04-05T22:29:27.097597+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097597+0000","last_undegraded":"2024-04-05T22:29:27.097597+0000","last_fullsized":"2024-04-05T22:29:27.097597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082968+0000","last_change":"2024-04-05T22:29:27.082968+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082968+0000","last_undegraded":"2024-04-05T22:29:27.082968+0000","last_fullsized":"2024-04-05T22:29:27.082968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096950+0000","last_change":"2024-04-05T22:29:27.096950+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096950+0000","last_undegraded":"2024-04-05T22:29:27.096950+0000","last_fullsized":"2024-04-05T22:29:27.096950+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081332+0000","last_change":"2024-04-05T22:29:27.081332+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081332+0000","last_undegraded":"2024-04-05T22:29:27.081332+0000","last_fullsized":"2024-04-05T22:29:27.081332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092768+0000","last_change":"2024-04-05T22:29:27.092768+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092768+0000","last_undegraded":"2024-04-05T22:29:27.092768+0000","last_fullsized":"2024-04-05T22:29:27.092768+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.102673+0000","last_change":"2024-04-05T22:29:27.102673+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.102673+0000","last_undegraded":"2024-04-05T22:29:27.102673+0000","last_fullsized":"2024-04-05T22:29:27.102673+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089407+0000","last_change":"2024-04-05T22:29:27.089407+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089407+0000","last_undegraded":"2024-04-05T22:29:27.089407+0000","last_fullsized":"2024-04-05T22:29:27.089407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096067+0000","last_change":"2024-04-05T22:29:27.096067+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096067+0000","last_undegraded":"2024-04-05T22:29:27.096067+0000","last_fullsized":"2024-04-05T22:29:27.096067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094222+0000","last_change":"2024-04-05T22:29:27.094222+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094222+0000","last_undegraded":"2024-04-05T22:29:27.094222+0000","last_fullsized":"2024-04-05T22:29:27.094222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086244+0000","last_change":"2024-04-05T22:29:27.086244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086244+0000","last_undegraded":"2024-04-05T22:29:27.086244+0000","last_fullsized":"2024-04-05T22:29:27.086244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095219+0000","last_change":"2024-04-05T22:29:27.095219+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095219+0000","last_undegraded":"2024-04-05T22:29:27.095219+0000","last_fullsized":"2024-04-05T22:29:27.095219+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096108+0000","last_change":"2024-04-05T22:29:27.096108+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096108+0000","last_undegraded":"2024-04-05T22:29:27.096108+0000","last_fullsized":"2024-04-05T22:29:27.096108+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097011+0000","last_change":"2024-04-05T22:29:27.097011+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097011+0000","last_undegraded":"2024-04-05T22:29:27.097011+0000","last_fullsized":"2024-04-05T22:29:27.097011+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098055+0000","last_change":"2024-04-05T22:29:27.098055+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098055+0000","last_undegraded":"2024-04-05T22:29:27.098055+0000","last_fullsized":"2024-04-05T22:29:27.098055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084360+0000","last_change":"2024-04-05T22:29:27.084360+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084360+0000","last_undegraded":"2024-04-05T22:29:27.084360+0000","last_fullsized":"2024-04-05T22:29:27.084360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093190+0000","last_change":"2024-04-05T22:29:27.093190+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093190+0000","last_undegraded":"2024-04-05T22:29:27.093190+0000","last_fullsized":"2024-04-05T22:29:27.093190+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099854+0000","last_change":"2024-04-05T22:29:27.099854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099854+0000","last_undegraded":"2024-04-05T22:29:27.099854+0000","last_fullsized":"2024-04-05T22:29:27.099854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084747+0000","last_change":"2024-04-05T22:29:27.084747+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084747+0000","last_undegraded":"2024-04-05T22:29:27.084747+0000","last_fullsized":"2024-04-05T22:29:27.084747+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082244+0000","last_change":"2024-04-05T22:29:27.082244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082244+0000","last_undegraded":"2024-04-05T22:29:27.082244+0000","last_fullsized":"2024-04-05T22:29:27.082244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094229+0000","last_change":"2024-04-05T22:29:27.094229+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094229+0000","last_undegraded":"2024-04-05T22:29:27.094229+0000","last_fullsized":"2024-04-05T22:29:27.094229+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095176+0000","last_change":"2024-04-05T22:29:27.095176+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095176+0000","last_undegraded":"2024-04-05T22:29:27.095176+0000","last_fullsized":"2024-04-05T22:29:27.095176+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081906+0000","last_change":"2024-04-05T22:29:27.081906+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081906+0000","last_undegraded":"2024-04-05T22:29:27.081906+0000","last_fullsized":"2024-04-05T22:29:27.081906+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096447+0000","last_change":"2024-04-05T22:29:27.096447+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096447+0000","last_undegraded":"2024-04-05T22:29:27.096447+0000","last_fullsized":"2024-04-05T22:29:27.096447+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100820+0000","last_change":"2024-04-05T22:29:27.100820+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100820+0000","last_undegraded":"2024-04-05T22:29:27.100820+0000","last_fullsized":"2024-04-05T22:29:27.100820+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081518+0000","last_change":"2024-04-05T22:29:27.081518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081518+0000","last_undegraded":"2024-04-05T22:29:27.081518+0000","last_fullsized":"2024-04-05T22:29:27.081518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.083464+0000","last_change":"2024-04-05T22:29:27.083464+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.083464+0000","last_undegraded":"2024-04-05T22:29:27.083464+0000","last_fullsized":"2024-04-05T22:29:27.083464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090854+0000","last_change":"2024-04-05T22:29:27.090854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090854+0000","last_undegraded":"2024-04-05T22:29:27.090854+0000","last_fullsized":"2024-04-05T22:29:27.090854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094333+0000","last_change":"2024-04-05T22:29:27.094333+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094333+0000","last_undegraded":"2024-04-05T22:29:27.094333+0000","last_fullsized":"2024-04-05T22:29:27.094333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101884+0000","last_change":"2024-04-05T22:29:27.101884+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101884+0000","last_undegraded":"2024-04-05T22:29:27.101884+0000","last_fullsized":"2024-04-05T22:29:27.101884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116115+0000","last_change":"2024-04-05T22:29:27.113665+0000","last_active":"2024-04-05T22:29:27.116115+0000","last_peered":"2024-04-05T22:29:27.116115+0000","last_clean":"2024-04-05T22:29:27.116115+0000","last_became_active":"2024-04-05T22:29:27.113368+0000","last_became_peered":"2024-04-05T22:29:27.113368+0000","last_unstale":"2024-04-05T22:29:27.116115+0000","last_undegraded":"2024-04-05T22:29:27.116115+0000","last_fullsized":"2024-04-05T22:29:27.116115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:29.247549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091093+0000","last_change":"2024-04-05T22:29:27.091093+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091093+0000","last_undegraded":"2024-04-05T22:29:27.091093+0000","last_fullsized":"2024-04-05T22:29:27.091093+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116613+0000","last_change":"2024-04-05T22:29:27.115636+0000","last_active":"2024-04-05T22:29:27.116613+0000","last_peered":"2024-04-05T22:29:27.116613+0000","last_clean":"2024-04-05T22:29:27.116613+0000","last_became_active":"2024-04-05T22:29:27.115170+0000","last_became_peered":"2024-04-05T22:29:27.115170+0000","last_unstale":"2024-04-05T22:29:27.116613+0000","last_undegraded":"2024-04-05T22:29:27.116613+0000","last_fullsized":"2024-04-05T22:29:27.116613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:49.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081733+0000","last_change":"2024-04-05T22:29:27.081733+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081733+0000","last_undegraded":"2024-04-05T22:29:27.081733+0000","last_fullsized":"2024-04-05T22:29:27.081733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109271+0000","last_change":"2024-04-05T22:29:27.106460+0000","last_active":"2024-04-05T22:29:27.109271+0000","last_peered":"2024-04-05T22:29:27.109271+0000","last_clean":"2024-04-05T22:29:27.109271+0000","last_became_active":"2024-04-05T22:29:27.106113+0000","last_became_peered":"2024-04-05T22:29:27.106113+0000","last_unstale":"2024-04-05T22:29:27.109271+0000","last_undegraded":"2024-04-05T22:29:27.109271+0000","last_fullsized":"2024-04-05T22:29:27.109271+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:47.380812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095782+0000","last_change":"2024-04-05T22:29:27.095782+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095782+0000","last_undegraded":"2024-04-05T22:29:27.095782+0000","last_fullsized":"2024-04-05T22:29:27.095782+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100802+0000","last_change":"2024-04-05T22:29:27.098092+0000","last_active":"2024-04-05T22:29:27.100802+0000","last_peered":"2024-04-05T22:29:27.100802+0000","last_clean":"2024-04-05T22:29:27.100802+0000","last_became_active":"2024-04-05T22:29:27.097449+0000","last_became_peered":"2024-04-05T22:29:27.097449+0000","last_unstale":"2024-04-05T22:29:27.100802+0000","last_undegraded":"2024-04-05T22:29:27.100802+0000","last_fullsized":"2024-04-05T22:29:27.100802+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:40.984286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092341+0000","last_change":"2024-04-05T22:29:27.092341+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092341+0000","last_undegraded":"2024-04-05T22:29:27.092341+0000","last_fullsized":"2024-04-05T22:29:27.092341+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110010+0000","last_change":"2024-04-05T22:29:27.106850+0000","last_active":"2024-04-05T22:29:27.110010+0000","last_peered":"2024-04-05T22:29:27.110010+0000","last_clean":"2024-04-05T22:29:27.110010+0000","last_became_active":"2024-04-05T22:29:27.106525+0000","last_became_peered":"2024-04-05T22:29:27.106525+0000","last_unstale":"2024-04-05T22:29:27.110010+0000","last_undegraded":"2024-04-05T22:29:27.110010+0000","last_fullsized":"2024-04-05T22:29:27.110010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:30.113756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082197+0000","last_change":"2024-04-05T22:29:27.082197+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082197+0000","last_undegraded":"2024-04-05T22:29:27.082197+0000","last_fullsized":"2024-04-05T22:29:27.082197+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.117465+0000","last_change":"2024-04-05T22:29:27.113913+0000","last_active":"2024-04-05T22:29:27.117465+0000","last_peered":"2024-04-05T22:29:27.117465+0000","last_clean":"2024-04-05T22:29:27.117465+0000","last_became_active":"2024-04-05T22:29:27.112452+0000","last_became_peered":"2024-04-05T22:29:27.112452+0000","last_unstale":"2024-04-05T22:29:27.117465+0000","last_undegraded":"2024-04-05T22:29:27.117465+0000","last_fullsized":"2024-04-05T22:29:27.117465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:13:41.450507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089823+0000","last_change":"2024-04-05T22:29:27.089823+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089823+0000","last_undegraded":"2024-04-05T22:29:27.089823+0000","last_fullsized":"2024-04-05T22:29:27.089823+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.108934+0000","last_change":"2024-04-05T22:29:27.105830+0000","last_active":"2024-04-05T22:29:27.108934+0000","last_peered":"2024-04-05T22:29:27.108934+0000","last_clean":"2024-04-05T22:29:27.108934+0000","last_became_active":"2024-04-05T22:29:27.104458+0000","last_became_peered":"2024-04-05T22:29:27.104458+0000","last_unstale":"2024-04-05T22:29:27.108934+0000","last_undegraded":"2024-04-05T22:29:27.108934+0000","last_fullsized":"2024-04-05T22:29:27.108934+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:34.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096482+0000","last_change":"2024-04-05T22:29:27.096482+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096482+0000","last_undegraded":"2024-04-05T22:29:27.096482+0000","last_fullsized":"2024-04-05T22:29:27.096482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109450+0000","last_change":"2024-04-05T22:29:27.107183+0000","last_active":"2024-04-05T22:29:27.109450+0000","last_peered":"2024-04-05T22:29:27.109450+0000","last_clean":"2024-04-05T22:29:27.109450+0000","last_became_active":"2024-04-05T22:29:27.106873+0000","last_became_peered":"2024-04-05T22:29:27.106873+0000","last_unstale":"2024-04-05T22:29:27.109450+0000","last_undegraded":"2024-04-05T22:29:27.109450+0000","last_fullsized":"2024-04-05T22:29:27.109450+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:31.435759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098271+0000","last_change":"2024-04-05T22:29:27.098271+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098271+0000","last_undegraded":"2024-04-05T22:29:27.098271+0000","last_fullsized":"2024-04-05T22:29:27.098271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100170+0000","last_change":"2024-04-05T22:29:27.098118+0000","last_active":"2024-04-05T22:29:27.100170+0000","last_peered":"2024-04-05T22:29:27.100170+0000","last_clean":"2024-04-05T22:29:27.100170+0000","last_became_active":"2024-04-05T22:29:27.097772+0000","last_became_peered":"2024-04-05T22:29:27.097772+0000","last_unstale":"2024-04-05T22:29:27.100170+0000","last_undegraded":"2024-04-05T22:29:27.100170+0000","last_fullsized":"2024-04-05T22:29:27.100170+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:04.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098638+0000","last_change":"2024-04-05T22:29:27.098638+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098638+0000","last_undegraded":"2024-04-05T22:29:27.098638+0000","last_fullsized":"2024-04-05T22:29:27.098638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118528+0000","last_change":"2024-04-05T22:29:27.117368+0000","last_active":"2024-04-05T22:29:27.118528+0000","last_peered":"2024-04-05T22:29:27.118528+0000","last_clean":"2024-04-05T22:29:27.118528+0000","last_became_active":"2024-04-05T22:29:27.117006+0000","last_became_peered":"2024-04-05T22:29:27.117006+0000","last_unstale":"2024-04-05T22:29:27.118528+0000","last_undegraded":"2024-04-05T22:29:27.118528+0000","last_fullsized":"2024-04-05T22:29:27.118528+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:26.585990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093113+0000","last_change":"2024-04-05T22:29:27.093113+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093113+0000","last_undegraded":"2024-04-05T22:29:27.093113+0000","last_fullsized":"2024-04-05T22:29:27.093113+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109634+0000","last_change":"2024-04-05T22:29:27.107727+0000","last_active":"2024-04-05T22:29:27.109634+0000","last_peered":"2024-04-05T22:29:27.109634+0000","last_clean":"2024-04-05T22:29:27.109634+0000","last_became_active":"2024-04-05T22:29:27.107409+0000","last_became_peered":"2024-04-05T22:29:27.107409+0000","last_unstale":"2024-04-05T22:29:27.109634+0000","last_undegraded":"2024-04-05T22:29:27.109634+0000","last_fullsized":"2024-04-05T22:29:27.109634+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:36:16.139153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096703+0000","last_change":"2024-04-05T22:29:27.096703+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096703+0000","last_undegraded":"2024-04-05T22:29:27.096703+0000","last_fullsized":"2024-04-05T22:29:27.096703+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112336+0000","last_change":"2024-04-05T22:29:27.109281+0000","last_active":"2024-04-05T22:29:27.112336+0000","last_peered":"2024-04-05T22:29:27.112336+0000","last_clean":"2024-04-05T22:29:27.112336+0000","last_became_active":"2024-04-05T22:29:27.108992+0000","last_became_peered":"2024-04-05T22:29:27.108992+0000","last_unstale":"2024-04-05T22:29:27.112336+0000","last_undegraded":"2024-04-05T22:29:27.112336+0000","last_fullsized":"2024-04-05T22:29:27.112336+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:33:12.557477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091308+0000","last_change":"2024-04-05T22:29:27.091308+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091308+0000","last_undegraded":"2024-04-05T22:29:27.091308+0000","last_fullsized":"2024-04-05T22:29:27.091308+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112144+0000","last_change":"2024-04-05T22:29:27.109031+0000","last_active":"2024-04-05T22:29:27.112144+0000","last_peered":"2024-04-05T22:29:27.112144+0000","last_clean":"2024-04-05T22:29:27.112144+0000","last_became_active":"2024-04-05T22:29:27.108620+0000","last_became_peered":"2024-04-05T22:29:27.108620+0000","last_unstale":"2024-04-05T22:29:27.112144+0000","last_undegraded":"2024-04-05T22:29:27.112144+0000","last_fullsized":"2024-04-05T22:29:27.112144+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:46.523577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092972+0000","last_change":"2024-04-05T22:29:27.092972+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092972+0000","last_undegraded":"2024-04-05T22:29:27.092972+0000","last_fullsized":"2024-04-05T22:29:27.092972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107366+0000","last_change":"2024-04-05T22:29:27.103901+0000","last_active":"2024-04-05T22:29:27.107366+0000","last_peered":"2024-04-05T22:29:27.107366+0000","last_clean":"2024-04-05T22:29:27.107366+0000","last_became_active":"2024-04-05T22:29:27.103615+0000","last_became_peered":"2024-04-05T22:29:27.103615+0000","last_unstale":"2024-04-05T22:29:27.107366+0000","last_undegraded":"2024-04-05T22:29:27.107366+0000","last_fullsized":"2024-04-05T22:29:27.107366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:46.499208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095841+0000","last_change":"2024-04-05T22:29:27.095841+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095841+0000","last_undegraded":"2024-04-05T22:29:27.095841+0000","last_fullsized":"2024-04-05T22:29:27.095841+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106936+0000","last_change":"2024-04-05T22:29:27.104205+0000","last_active":"2024-04-05T22:29:27.106936+0000","last_peered":"2024-04-05T22:29:27.106936+0000","last_clean":"2024-04-05T22:29:27.106936+0000","last_became_active":"2024-04-05T22:29:27.102618+0000","last_became_peered":"2024-04-05T22:29:27.102618+0000","last_unstale":"2024-04-05T22:29:27.106936+0000","last_undegraded":"2024-04-05T22:29:27.106936+0000","last_fullsized":"2024-04-05T22:29:27.106936+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:05.414218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096852+0000","last_change":"2024-04-05T22:29:27.096852+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096852+0000","last_undegraded":"2024-04-05T22:29:27.096852+0000","last_fullsized":"2024-04-05T22:29:27.096852+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116552+0000","last_change":"2024-04-05T22:29:27.113619+0000","last_active":"2024-04-05T22:29:27.116552+0000","last_peered":"2024-04-05T22:29:27.116552+0000","last_clean":"2024-04-05T22:29:27.116552+0000","last_became_active":"2024-04-05T22:29:27.113367+0000","last_became_peered":"2024-04-05T22:29:27.113367+0000","last_unstale":"2024-04-05T22:29:27.116552+0000","last_undegraded":"2024-04-05T22:29:27.116552+0000","last_fullsized":"2024-04-05T22:29:27.116552+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:26:24.419813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093979+0000","last_change":"2024-04-05T22:29:27.093979+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093979+0000","last_undegraded":"2024-04-05T22:29:27.093979+0000","last_fullsized":"2024-04-05T22:29:27.093979+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106232+0000","last_change":"2024-04-05T22:29:27.103848+0000","last_active":"2024-04-05T22:29:27.106232+0000","last_peered":"2024-04-05T22:29:27.106232+0000","last_clean":"2024-04-05T22:29:27.106232+0000","last_became_active":"2024-04-05T22:29:27.103586+0000","last_became_peered":"2024-04-05T22:29:27.103586+0000","last_unstale":"2024-04-05T22:29:27.106232+0000","last_undegraded":"2024-04-05T22:29:27.106232+0000","last_fullsized":"2024-04-05T22:29:27.106232+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:11.324558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092715+0000","last_change":"2024-04-05T22:29:27.092715+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092715+0000","last_undegraded":"2024-04-05T22:29:27.092715+0000","last_fullsized":"2024-04-05T22:29:27.092715+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109481+0000","last_change":"2024-04-05T22:29:27.106062+0000","last_active":"2024-04-05T22:29:27.109481+0000","last_peered":"2024-04-05T22:29:27.109481+0000","last_clean":"2024-04-05T22:29:27.109481+0000","last_became_active":"2024-04-05T22:29:27.105748+0000","last_became_peered":"2024-04-05T22:29:27.105748+0000","last_unstale":"2024-04-05T22:29:27.109481+0000","last_undegraded":"2024-04-05T22:29:27.109481+0000","last_fullsized":"2024-04-05T22:29:27.109481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:48.841874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086784+0000","last_change":"2024-04-05T22:29:27.086784+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086784+0000","last_undegraded":"2024-04-05T22:29:27.086784+0000","last_fullsized":"2024-04-05T22:29:27.086784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.105147+0000","last_change":"2024-04-05T22:29:27.102360+0000","last_active":"2024-04-05T22:29:27.105147+0000","last_peered":"2024-04-05T22:29:27.105147+0000","last_clean":"2024-04-05T22:29:27.105147+0000","last_became_active":"2024-04-05T22:29:27.102051+0000","last_became_peered":"2024-04-05T22:29:27.102051+0000","last_unstale":"2024-04-05T22:29:27.105147+0000","last_undegraded":"2024-04-05T22:29:27.105147+0000","last_fullsized":"2024-04-05T22:29:27.105147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:47.073194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081980+0000","last_change":"2024-04-05T22:29:27.081980+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081980+0000","last_undegraded":"2024-04-05T22:29:27.081980+0000","last_fullsized":"2024-04-05T22:29:27.081980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112449+0000","last_change":"2024-04-05T22:29:27.111825+0000","last_active":"2024-04-05T22:29:27.112449+0000","last_peered":"2024-04-05T22:29:27.112449+0000","last_clean":"2024-04-05T22:29:27.112449+0000","last_became_active":"2024-04-05T22:29:27.111542+0000","last_became_peered":"2024-04-05T22:29:27.111542+0000","last_unstale":"2024-04-05T22:29:27.112449+0000","last_undegraded":"2024-04-05T22:29:27.112449+0000","last_fullsized":"2024-04-05T22:29:27.112449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:22.051262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100124+0000","last_change":"2024-04-05T22:29:27.100124+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100124+0000","last_undegraded":"2024-04-05T22:29:27.100124+0000","last_fullsized":"2024-04-05T22:29:27.100124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.114102+0000","last_change":"2024-04-05T22:29:27.111984+0000","last_active":"2024-04-05T22:29:27.114102+0000","last_peered":"2024-04-05T22:29:27.114102+0000","last_clean":"2024-04-05T22:29:27.114102+0000","last_became_active":"2024-04-05T22:29:27.111570+0000","last_became_peered":"2024-04-05T22:29:27.111570+0000","last_unstale":"2024-04-05T22:29:27.114102+0000","last_undegraded":"2024-04-05T22:29:27.114102+0000","last_fullsized":"2024-04-05T22:29:27.114102+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:02:55.012896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081843+0000","last_change":"2024-04-05T22:29:27.081843+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081843+0000","last_undegraded":"2024-04-05T22:29:27.081843+0000","last_fullsized":"2024-04-05T22:29:27.081843+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118512+0000","last_change":"2024-04-05T22:29:27.117354+0000","last_active":"2024-04-05T22:29:27.118512+0000","last_peered":"2024-04-05T22:29:27.118512+0000","last_clean":"2024-04-05T22:29:27.118512+0000","last_became_active":"2024-04-05T22:29:27.116968+0000","last_became_peered":"2024-04-05T22:29:27.116968+0000","last_unstale":"2024-04-05T22:29:27.118512+0000","last_undegraded":"2024-04-05T22:29:27.118512+0000","last_fullsized":"2024-04-05T22:29:27.118512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:32:03.796707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101316+0000","last_change":"2024-04-05T22:29:27.101316+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101316+0000","last_undegraded":"2024-04-05T22:29:27.101316+0000","last_fullsized":"2024-04-05T22:29:27.101316+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110905+0000","last_change":"2024-04-05T22:29:27.104579+0000","last_active":"2024-04-05T22:29:27.110905+0000","last_peered":"2024-04-05T22:29:27.110905+0000","last_clean":"2024-04-05T22:29:27.110905+0000","last_became_active":"2024-04-05T22:29:27.104293+0000","last_became_peered":"2024-04-05T22:29:27.104293+0000","last_unstale":"2024-04-05T22:29:27.110905+0000","last_undegraded":"2024-04-05T22:29:27.110905+0000","last_fullsized":"2024-04-05T22:29:27.110905+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:13:51.613749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097362+0000","last_change":"2024-04-05T22:29:27.097362+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097362+0000","last_undegraded":"2024-04-05T22:29:27.097362+0000","last_fullsized":"2024-04-05T22:29:27.097362+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116901+0000","last_change":"2024-04-05T22:29:27.115301+0000","last_active":"2024-04-05T22:29:27.116901+0000","last_peered":"2024-04-05T22:29:27.116901+0000","last_clean":"2024-04-05T22:29:27.116901+0000","last_became_active":"2024-04-05T22:29:27.114776+0000","last_became_peered":"2024-04-05T22:29:27.114776+0000","last_unstale":"2024-04-05T22:29:27.116901+0000","last_undegraded":"2024-04-05T22:29:27.116901+0000","last_fullsized":"2024-04-05T22:29:27.116901+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:53.380117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099085+0000","last_change":"2024-04-05T22:29:27.099085+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099085+0000","last_undegraded":"2024-04-05T22:29:27.099085+0000","last_fullsized":"2024-04-05T22:29:27.099085+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107334+0000","last_change":"2024-04-05T22:29:27.105399+0000","last_active":"2024-04-05T22:29:27.107334+0000","last_peered":"2024-04-05T22:29:27.107334+0000","last_clean":"2024-04-05T22:29:27.107334+0000","last_became_active":"2024-04-05T22:29:27.105109+0000","last_became_peered":"2024-04-05T22:29:27.105109+0000","last_unstale":"2024-04-05T22:29:27.107334+0000","last_undegraded":"2024-04-05T22:29:27.107334+0000","last_fullsized":"2024-04-05T22:29:27.107334+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:28.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090127+0000","last_change":"2024-04-05T22:29:27.090127+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090127+0000","last_undegraded":"2024-04-05T22:29:27.090127+0000","last_fullsized":"2024-04-05T22:29:27.090127+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097140+0000","last_change":"2024-04-05T22:29:27.094375+0000","last_active":"2024-04-05T22:29:27.097140+0000","last_peered":"2024-04-05T22:29:27.097140+0000","last_clean":"2024-04-05T22:29:27.097140+0000","last_became_active":"2024-04-05T22:29:27.094043+0000","last_became_peered":"2024-04-05T22:29:27.094043+0000","last_unstale":"2024-04-05T22:29:27.097140+0000","last_undegraded":"2024-04-05T22:29:27.097140+0000","last_fullsized":"2024-04-05T22:29:27.097140+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:36.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.085671+0000","last_change":"2024-04-05T22:29:27.085671+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.085671+0000","last_undegraded":"2024-04-05T22:29:27.085671+0000","last_fullsized":"2024-04-05T22:29:27.085671+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097067+0000","last_change":"2024-04-05T22:29:27.092419+0000","last_active":"2024-04-05T22:29:27.097067+0000","last_peered":"2024-04-05T22:29:27.097067+0000","last_clean":"2024-04-05T22:29:27.097067+0000","last_became_active":"2024-04-05T22:29:27.091178+0000","last_became_peered":"2024-04-05T22:29:27.091178+0000","last_unstale":"2024-04-05T22:29:27.097067+0000","last_undegraded":"2024-04-05T22:29:27.097067+0000","last_fullsized":"2024-04-05T22:29:27.097067+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:31.397216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092775+0000","last_change":"2024-04-05T22:29:27.092775+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092775+0000","last_undegraded":"2024-04-05T22:29:27.092775+0000","last_fullsized":"2024-04-05T22:29:27.092775+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109970+0000","last_change":"2024-04-05T22:29:27.108317+0000","last_active":"2024-04-05T22:29:27.109970+0000","last_peered":"2024-04-05T22:29:27.109970+0000","last_clean":"2024-04-05T22:29:27.109970+0000","last_became_active":"2024-04-05T22:29:27.107888+0000","last_became_peered":"2024-04-05T22:29:27.107888+0000","last_unstale":"2024-04-05T22:29:27.109970+0000","last_undegraded":"2024-04-05T22:29:27.109970+0000","last_fullsized":"2024-04-05T22:29:27.109970+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:42.291460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092874+0000","last_change":"2024-04-05T22:29:27.092874+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092874+0000","last_undegraded":"2024-04-05T22:29:27.092874+0000","last_fullsized":"2024-04-05T22:29:27.092874+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116811+0000","last_change":"2024-04-05T22:29:27.114605+0000","last_active":"2024-04-05T22:29:27.116811+0000","last_peered":"2024-04-05T22:29:27.116811+0000","last_clean":"2024-04-05T22:29:27.116811+0000","last_became_active":"2024-04-05T22:29:27.114311+0000","last_became_peered":"2024-04-05T22:29:27.114311+0000","last_unstale":"2024-04-05T22:29:27.116811+0000","last_undegraded":"2024-04-05T22:29:27.116811+0000","last_fullsized":"2024-04-05T22:29:27.116811+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:06.440778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092986+0000","last_change":"2024-04-05T22:29:27.092986+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092986+0000","last_undegraded":"2024-04-05T22:29:27.092986+0000","last_fullsized":"2024-04-05T22:29:27.092986+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112429+0000","last_change":"2024-04-05T22:29:27.109061+0000","last_active":"2024-04-05T22:29:27.112429+0000","last_peered":"2024-04-05T22:29:27.112429+0000","last_clean":"2024-04-05T22:29:27.112429+0000","last_became_active":"2024-04-05T22:29:27.108742+0000","last_became_peered":"2024-04-05T22:29:27.108742+0000","last_unstale":"2024-04-05T22:29:27.112429+0000","last_undegraded":"2024-04-05T22:29:27.112429+0000","last_fullsized":"2024-04-05T22:29:27.112429+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:26:35.912596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094988+0000","last_change":"2024-04-05T22:29:27.094988+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094988+0000","last_undegraded":"2024-04-05T22:29:27.094988+0000","last_fullsized":"2024-04-05T22:29:27.094988+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116214+0000","last_change":"2024-04-05T22:29:27.115510+0000","last_active":"2024-04-05T22:29:27.116214+0000","last_peered":"2024-04-05T22:29:27.116214+0000","last_clean":"2024-04-05T22:29:27.116214+0000","last_became_active":"2024-04-05T22:29:27.115162+0000","last_became_peered":"2024-04-05T22:29:27.115162+0000","last_unstale":"2024-04-05T22:29:27.116214+0000","last_undegraded":"2024-04-05T22:29:27.116214+0000","last_fullsized":"2024-04-05T22:29:27.116214+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:44.332750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096436+0000","last_change":"2024-04-05T22:29:27.096436+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096436+0000","last_undegraded":"2024-04-05T22:29:27.096436+0000","last_fullsized":"2024-04-05T22:29:27.096436+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.111421+0000","last_change":"2024-04-05T22:29:27.107244+0000","last_active":"2024-04-05T22:29:27.111421+0000","last_peered":"2024-04-05T22:29:27.111421+0000","last_clean":"2024-04-05T22:29:27.111421+0000","last_became_active":"2024-04-05T22:29:27.106979+0000","last_became_peered":"2024-04-05T22:29:27.106979+0000","last_unstale":"2024-04-05T22:29:27.111421+0000","last_undegraded":"2024-04-05T22:29:27.111421+0000","last_fullsized":"2024-04-05T22:29:27.111421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:28:57.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":299,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-05T22:29:26.085340+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:29:26.085340+0000","last_peered":"2024-04-05T22:29:26.085340+0000","last_clean":"2024-04-05T22:29:26.085340+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:29:26.085340+0000","last_undegraded":"2024-04-05T22:29:26.085340+0000","last_fullsized":"2024-04-05T22:29:26.085340+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.21}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.085},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.401}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.597},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.6339999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":1.026},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.534},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.488},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1.6870000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.6679999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.1499999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":1.0880000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.196},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.343}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.29},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.377}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.258},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.431}]}]},{"osd":5,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.115,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.097},"min":{"1min":0.68300000000000005,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.5900000000000001,"5min":1.907,"15min":1.907},"last":0.999}]},{"osd":1,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.76600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3919999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.98399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6359999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.177,"15min":1.1599999999999999},"min":{"1min":0.82599999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3759999999999999,"5min":2.544,"15min":2.544},"last":1.25},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.89900000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5840000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0589999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1859999999999999,"15min":1.181},"min":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7529999999999999,"5min":1.867,"15min":1.867},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.268,"5min":1.2909999999999999,"15min":1.3460000000000001},"min":{"1min":1.03,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.649,"5min":2.927,"15min":2.9620000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7050000000000001,"5min":1.911,"15min":2.1970000000000001},"last":1.405},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2969999999999999,"15min":1.302},"min":{"1min":0.89900000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.784,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.79,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5129999999999999,"15min":1.5},"min":{"1min":1.284,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.9079999999999999,"5min":2.169,"15min":2.5350000000000001},"last":1.599}]},{"osd":7,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.464,"15min":1.456},"min":{"1min":1.2010000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.919,"5min":2.028,"15min":2.2509999999999999},"last":1.448},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5,"15min":1.597},"min":{"1min":1.1799999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":4.4329999999999998},"last":1.5169999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1100000000000001,"15min":1.0249999999999999},"min":{"1min":0.82599999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.466,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.085},"min":{"1min":0.73199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5369999999999999,"5min":1.859,"15min":1.859},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1279999999999999},"min":{"1min":0.88100000000000001,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":1.0009999999999999,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.659,"5min":2.577,"15min":2.577},"last":1.5029999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1319999999999999,"15min":1.202},"min":{"1min":0.75,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.974},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2450000000000001,"15min":1.2},"min":{"1min":1.0389999999999999,"5min":0.76200000000000001,"15min":0.67400000000000004},"max":{"1min":1.657,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.745}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3200000000000001,"15min":1.3640000000000001},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7889999999999999,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3919999999999999,"15min":1.403},"min":{"1min":1.196,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.7509999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.573}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":1.0529999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1459999999999999,"5min":2.665,"15min":2.665},"last":1.282},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.375,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.915,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.6180000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.4139999999999999,"15min":1.4550000000000001},"min":{"1min":1.0229999999999999,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":1.9950000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":1.212,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.8899999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.524,"15min":1.5389999999999999},"min":{"1min":1.0760000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0019999999999998,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.764},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6259999999999999,"15min":1.591},"min":{"1min":1.3839999999999999,"5min":0.91300000000000003,"15min":0.72399999999999998},"max":{"1min":2.109,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":2.0409999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1675264,"data_stored":737649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0229999999999999,"5min":1.744,"15min":1.744},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5860000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.111}]},{"osd":1,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":1.006},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.3720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.325},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0580000000000001,"15min":1.073},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.391,"5min":1.675,"15min":2.2549999999999999},"last":1.274}]},{"osd":4,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1899999999999999,"15min":1.2290000000000001},"min":{"1min":0.82099999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5289999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.294},"min":{"1min":0.75900000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.399}]},{"osd":5,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.286,"15min":1.351},"min":{"1min":0.73699999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.482},{"interface":"front","average":{"1min":1.246,"5min":1.288,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6459999999999999,"5min":1.958,"15min":2.3239999999999998},"last":1.6579999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.3009999999999999,"15min":1.3500000000000001},"min":{"1min":0.875,"5min":0.505,"15min":0.505},"max":{"1min":1.6240000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.347,"15min":1.401},"min":{"1min":0.97899999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.298,"15min":2.298},"last":1.7}]},{"osd":7,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3400000000000001,"15min":1.415},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":2.468},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.377,"15min":1.415},"min":{"1min":0.93700000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.73,"5min":2.508,"15min":2.798},"last":1.8160000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542246,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.4319999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.496}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.165}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2030000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.363,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.89900000000000002,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.42299999999999999,"15min":0.36799999999999999},"max":{"1min":1.554,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85399999999999998,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.206,"5min":1.488,"15min":1.5389999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.999,"15min":0.99399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0880000000000001,"15min":1.097},"min":{"1min":0.53600000000000003,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.484,"5min":1.5509999999999999,"15min":1.75},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.3959999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":1.125},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6679999999999999,"5min":1.806,"15min":2.2789999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.258,"15min":1.262},"min":{"1min":0.84099999999999997,"5min":0.76000000000000001,"15min":0.68100000000000005},"max":{"1min":1.706,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.873,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.5429999999999999,"5min":1.843,"15min":2.173},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.369,"15min":1.3340000000000001},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6439999999999999,"5min":1.9059999999999999,"15min":1.917},"last":1.264},{"interface":"front","average":{"1min":1.292,"5min":1.345,"15min":1.355},"min":{"1min":0.92200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.825,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.407},{"interface":"front","average":{"1min":1.196,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.6319999999999999,"5min":1.9279999999999999,"15min":2.3879999999999999},"last":1.429}]}]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.1479999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.92600000000000005,"15min":0.92000000000000004},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2569999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.2729999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.25,"5min":1.976,"15min":1.976},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0840000000000001,"15min":1.093},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5900000000000001,"5min":2.157,"15min":2.157},"last":0.749}]},{"osd":4,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2,"15min":1.1699999999999999},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.429,"5min":1.931,"15min":1.931},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3129999999999999,"15min":1.302},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.786,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.0549999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2549999999999999,"15min":1.2609999999999999},"min":{"1min":0.75700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.524,"5min":2.105,"15min":2.3879999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.2130000000000001,"15min":1.198},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.357,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.878,"15min":0.65500000000000003},"max":{"1min":1.8340000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3280000000000001,"15min":1.365},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.585,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3320000000000001,"15min":1.337},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.6080000000000001,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.151}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:29:32.518 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:29:32.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.769+0000 7f5300c00640 1 Processor -- start 2024-04-05T22:29:32.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.769+0000 7f5300c00640 1 -- start start 2024-04-05T22:29:32.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f5300c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52fc1587d0 0x7f52fc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f5300c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f52fc0ccd20 0x7f52fc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f5300c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc06f5b0 con 0x7f52fc06d0c0 2024-04-05T22:29:32.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f5300c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc070410 con 0x7f52fc0ccd20 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f5300c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc158bb0 con 0x7f52fc1587d0 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f52f1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f52fc0ccd20 0x7f52fc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f52f1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f52fc0ccd20 0x7f52fc06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34492/0 (socket says 172.21.15.73:34492) 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.771+0000 7f52f1400640 1 -- 172.21.15.73:0/3162030621 learned_addr learned my addr 172.21.15.73:0/3162030621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.772+0000 7f52f9e00640 1 --2- 172.21.15.73:0/3162030621 >> v2:172.21.15.157:3300/0 conn(0x7f52fc1587d0 0x7f52fc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.772+0000 7f52f1400640 1 -- 172.21.15.73:0/3162030621 >> v2:172.21.15.157:3300/0 conn(0x7f52fc1587d0 msgr2=0x7f52fc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:32.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.772+0000 7f52f1400640 1 --2- 172.21.15.73:0/3162030621 >> v2:172.21.15.157:3300/0 conn(0x7f52fc1587d0 0x7f52fc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.772+0000 7f52f1400640 1 -- 172.21.15.73:0/3162030621 >> v1:172.21.15.73:6789/0 conn(0x7f52fc06d0c0 legacy=0x7f52fc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.772+0000 7f52f1400640 1 -- 172.21.15.73:0/3162030621 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc06f800 con 0x7f52fc0ccd20 2024-04-05T22:29:32.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.773+0000 7f52f1400640 1 --2- 172.21.15.73:0/3162030621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc0ccd20 0x7f52fc06cb80 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009870 tx=0x7f52e402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c831cadd594b9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f52f9400640 1 -- 172.21.15.73:0/3162030621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4030850 con 0x7f52fc0ccd20 2024-04-05T22:29:32.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f52f9400640 1 -- 172.21.15.73:0/3162030621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e40309b0 con 0x7f52fc0ccd20 2024-04-05T22:29:32.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f52f9400640 1 -- 172.21.15.73:0/3162030621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4030d40 con 0x7f52fc0ccd20 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 -- 172.21.15.73:0/3162030621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc0ccd20 msgr2=0x7f52fc06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 --2- 172.21.15.73:0/3162030621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc0ccd20 0x7f52fc06cb80 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009870 tx=0x7f52e402b990 comp rx=0 tx=0).stop 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 -- 172.21.15.73:0/3162030621 shutdown_connections 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 --2- 172.21.15.73:0/3162030621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc0ccd20 0x7f52fc06cb80 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 --2- 172.21.15.73:0/3162030621 >> v2:172.21.15.157:3300/0 conn(0x7f52fc1587d0 0x7f52fc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.774+0000 7f5300c00640 1 -- 172.21.15.73:0/3162030621 >> 172.21.15.73:0/3162030621 conn(0x7f52fc0ce8e0 msgr2=0x7f52fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.775+0000 7f5300c00640 1 -- 172.21.15.73:0/3162030621 shutdown_connections 2024-04-05T22:29:32.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.775+0000 7f5300c00640 1 -- 172.21.15.73:0/3162030621 wait complete. 2024-04-05T22:29:32.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.776+0000 7f5300c00640 1 Processor -- start 2024-04-05T22:29:32.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.776+0000 7f5300c00640 1 -- start start 2024-04-05T22:29:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc1587d0 0x7f52fc1efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc0afde0 0x7f52fc153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc14ac70 con 0x7f52fc06d0c0 2024-04-05T22:29:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1f0430 con 0x7f52fc0afde0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.777+0000 7f5300c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0aede0 con 0x7f52fc1587d0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34498/0 (socket says 172.21.15.73:34498) 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 -- 172.21.15.73:0/3516792390 learned_addr learned my addr 172.21.15.73:0/3516792390 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f1400640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc1587d0 0x7f52fc1efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 -- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc1587d0 msgr2=0x7f52fc1efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52fa800640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc0afde0 0x7f52fc153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc1587d0 0x7f52fc1efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 -- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc0afde0 msgr2=0x7f52fc153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc0afde0 0x7f52fc153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.778+0000 7f52f9e00640 1 -- 172.21.15.73:0/3516792390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40094d0 con 0x7f52fc06d0c0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f52f9e00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00d5f0 tx=0x7f52ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0125f0 con 0x7f52fc06d0c0 2024-04-05T22:29:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec012750 con 0x7f52fc06d0c0 2024-04-05T22:29:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc0af0c0 con 0x7f52fc06d0c0 2024-04-05T22:29:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec012b00 con 0x7f52fc06d0c0 2024-04-05T22:29:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.779+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0af680 con 0x7f52fc06d0c0 2024-04-05T22:29:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.780+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc152080 con 0x7f52fc06d0c0 2024-04-05T22:29:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.781+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f52ec012c60 con 0x7f52fc06d0c0 2024-04-05T22:29:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.782+0000 7f52f2a00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 0x7f52d806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.782+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52ec08a2e0 con 0x7f52fc06d0c0 2024-04-05T22:29:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.783+0000 7f52f1400640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 0x7f52d806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.784+0000 7f52f1400640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 0x7f52d806d660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f52e4036000 tx=0x7f52e403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:32.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.786+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f52ec054c50 con 0x7f52fc06d0c0 2024-04-05T22:29:32.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.976+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52fc0700a0 con 0x7f52d806b1a0 2024-04-05T22:29:32.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:32.999+0000 7f52f2a00640 1 -- 172.21.15.73:0/3516792390 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284513 (secure 0 0 0) 0x7f52e4030ba0 con 0x7f52d806b1a0 2024-04-05T22:29:33.004 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:33.006 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:29:33.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.005+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 msgr2=0x7f52d806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.005+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 0x7f52d806d660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f52e4036000 tx=0x7f52e403b040 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.005+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 msgr2=0x7f52fc0ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.005+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00d5f0 tx=0x7f52ec00dab0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 shutdown_connections 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52d806b1a0 0x7f52d806d660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc0afde0 0x7f52fc153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc1587d0 0x7f52fc1efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 --2- 172.21.15.73:0/3516792390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52fc06d0c0 0x7f52fc0ae7b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 >> 172.21.15.73:0/3516792390 conn(0x7f52fc0ce8e0 msgr2=0x7f52fc15b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 shutdown_connections 2024-04-05T22:29:33.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.006+0000 7f5300c00640 1 -- 172.21.15.73:0/3516792390 wait complete. 2024-04-05T22:29:33.025 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-05T22:29:31.564078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279508,"kb_used_data":14188,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754695212,"statfs":{"total":773094113280,"available":772807897088,"internally_reserved":0,"allocated":14528512,"data_stored":7050553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494764"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081084+0000","last_change":"2024-04-05T22:29:27.081084+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081084+0000","last_undegraded":"2024-04-05T22:29:27.081084+0000","last_fullsized":"2024-04-05T22:29:27.081084+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092518+0000","last_change":"2024-04-05T22:29:27.092518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092518+0000","last_undegraded":"2024-04-05T22:29:27.092518+0000","last_fullsized":"2024-04-05T22:29:27.092518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091256+0000","last_change":"2024-04-05T22:29:27.091256+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091256+0000","last_undegraded":"2024-04-05T22:29:27.091256+0000","last_fullsized":"2024-04-05T22:29:27.091256+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082145+0000","last_change":"2024-04-05T22:29:27.082145+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082145+0000","last_undegraded":"2024-04-05T22:29:27.082145+0000","last_fullsized":"2024-04-05T22:29:27.082145+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097597+0000","last_change":"2024-04-05T22:29:27.097597+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097597+0000","last_undegraded":"2024-04-05T22:29:27.097597+0000","last_fullsized":"2024-04-05T22:29:27.097597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082968+0000","last_change":"2024-04-05T22:29:27.082968+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082968+0000","last_undegraded":"2024-04-05T22:29:27.082968+0000","last_fullsized":"2024-04-05T22:29:27.082968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096950+0000","last_change":"2024-04-05T22:29:27.096950+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096950+0000","last_undegraded":"2024-04-05T22:29:27.096950+0000","last_fullsized":"2024-04-05T22:29:27.096950+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081332+0000","last_change":"2024-04-05T22:29:27.081332+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081332+0000","last_undegraded":"2024-04-05T22:29:27.081332+0000","last_fullsized":"2024-04-05T22:29:27.081332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092768+0000","last_change":"2024-04-05T22:29:27.092768+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092768+0000","last_undegraded":"2024-04-05T22:29:27.092768+0000","last_fullsized":"2024-04-05T22:29:27.092768+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.102673+0000","last_change":"2024-04-05T22:29:27.102673+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.102673+0000","last_undegraded":"2024-04-05T22:29:27.102673+0000","last_fullsized":"2024-04-05T22:29:27.102673+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089407+0000","last_change":"2024-04-05T22:29:27.089407+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089407+0000","last_undegraded":"2024-04-05T22:29:27.089407+0000","last_fullsized":"2024-04-05T22:29:27.089407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096067+0000","last_change":"2024-04-05T22:29:27.096067+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096067+0000","last_undegraded":"2024-04-05T22:29:27.096067+0000","last_fullsized":"2024-04-05T22:29:27.096067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094222+0000","last_change":"2024-04-05T22:29:27.094222+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094222+0000","last_undegraded":"2024-04-05T22:29:27.094222+0000","last_fullsized":"2024-04-05T22:29:27.094222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086244+0000","last_change":"2024-04-05T22:29:27.086244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086244+0000","last_undegraded":"2024-04-05T22:29:27.086244+0000","last_fullsized":"2024-04-05T22:29:27.086244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095219+0000","last_change":"2024-04-05T22:29:27.095219+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095219+0000","last_undegraded":"2024-04-05T22:29:27.095219+0000","last_fullsized":"2024-04-05T22:29:27.095219+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096108+0000","last_change":"2024-04-05T22:29:27.096108+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096108+0000","last_undegraded":"2024-04-05T22:29:27.096108+0000","last_fullsized":"2024-04-05T22:29:27.096108+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097011+0000","last_change":"2024-04-05T22:29:27.097011+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097011+0000","last_undegraded":"2024-04-05T22:29:27.097011+0000","last_fullsized":"2024-04-05T22:29:27.097011+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098055+0000","last_change":"2024-04-05T22:29:27.098055+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098055+0000","last_undegraded":"2024-04-05T22:29:27.098055+0000","last_fullsized":"2024-04-05T22:29:27.098055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084360+0000","last_change":"2024-04-05T22:29:27.084360+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084360+0000","last_undegraded":"2024-04-05T22:29:27.084360+0000","last_fullsized":"2024-04-05T22:29:27.084360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093190+0000","last_change":"2024-04-05T22:29:27.093190+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093190+0000","last_undegraded":"2024-04-05T22:29:27.093190+0000","last_fullsized":"2024-04-05T22:29:27.093190+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099854+0000","last_change":"2024-04-05T22:29:27.099854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099854+0000","last_undegraded":"2024-04-05T22:29:27.099854+0000","last_fullsized":"2024-04-05T22:29:27.099854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.084747+0000","last_change":"2024-04-05T22:29:27.084747+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.084747+0000","last_undegraded":"2024-04-05T22:29:27.084747+0000","last_fullsized":"2024-04-05T22:29:27.084747+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082244+0000","last_change":"2024-04-05T22:29:27.082244+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082244+0000","last_undegraded":"2024-04-05T22:29:27.082244+0000","last_fullsized":"2024-04-05T22:29:27.082244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094229+0000","last_change":"2024-04-05T22:29:27.094229+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094229+0000","last_undegraded":"2024-04-05T22:29:27.094229+0000","last_fullsized":"2024-04-05T22:29:27.094229+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095176+0000","last_change":"2024-04-05T22:29:27.095176+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095176+0000","last_undegraded":"2024-04-05T22:29:27.095176+0000","last_fullsized":"2024-04-05T22:29:27.095176+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081906+0000","last_change":"2024-04-05T22:29:27.081906+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081906+0000","last_undegraded":"2024-04-05T22:29:27.081906+0000","last_fullsized":"2024-04-05T22:29:27.081906+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096447+0000","last_change":"2024-04-05T22:29:27.096447+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096447+0000","last_undegraded":"2024-04-05T22:29:27.096447+0000","last_fullsized":"2024-04-05T22:29:27.096447+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100820+0000","last_change":"2024-04-05T22:29:27.100820+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100820+0000","last_undegraded":"2024-04-05T22:29:27.100820+0000","last_fullsized":"2024-04-05T22:29:27.100820+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081518+0000","last_change":"2024-04-05T22:29:27.081518+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081518+0000","last_undegraded":"2024-04-05T22:29:27.081518+0000","last_fullsized":"2024-04-05T22:29:27.081518+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.083464+0000","last_change":"2024-04-05T22:29:27.083464+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.083464+0000","last_undegraded":"2024-04-05T22:29:27.083464+0000","last_fullsized":"2024-04-05T22:29:27.083464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090854+0000","last_change":"2024-04-05T22:29:27.090854+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090854+0000","last_undegraded":"2024-04-05T22:29:27.090854+0000","last_fullsized":"2024-04-05T22:29:27.090854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094333+0000","last_change":"2024-04-05T22:29:27.094333+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094333+0000","last_undegraded":"2024-04-05T22:29:27.094333+0000","last_fullsized":"2024-04-05T22:29:27.094333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101884+0000","last_change":"2024-04-05T22:29:27.101884+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101884+0000","last_undegraded":"2024-04-05T22:29:27.101884+0000","last_fullsized":"2024-04-05T22:29:27.101884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116115+0000","last_change":"2024-04-05T22:29:27.113665+0000","last_active":"2024-04-05T22:29:27.116115+0000","last_peered":"2024-04-05T22:29:27.116115+0000","last_clean":"2024-04-05T22:29:27.116115+0000","last_became_active":"2024-04-05T22:29:27.113368+0000","last_became_peered":"2024-04-05T22:29:27.113368+0000","last_unstale":"2024-04-05T22:29:27.116115+0000","last_undegraded":"2024-04-05T22:29:27.116115+0000","last_fullsized":"2024-04-05T22:29:27.116115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:29.247549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091093+0000","last_change":"2024-04-05T22:29:27.091093+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091093+0000","last_undegraded":"2024-04-05T22:29:27.091093+0000","last_fullsized":"2024-04-05T22:29:27.091093+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116613+0000","last_change":"2024-04-05T22:29:27.115636+0000","last_active":"2024-04-05T22:29:27.116613+0000","last_peered":"2024-04-05T22:29:27.116613+0000","last_clean":"2024-04-05T22:29:27.116613+0000","last_became_active":"2024-04-05T22:29:27.115170+0000","last_became_peered":"2024-04-05T22:29:27.115170+0000","last_unstale":"2024-04-05T22:29:27.116613+0000","last_undegraded":"2024-04-05T22:29:27.116613+0000","last_fullsized":"2024-04-05T22:29:27.116613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:49.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081733+0000","last_change":"2024-04-05T22:29:27.081733+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081733+0000","last_undegraded":"2024-04-05T22:29:27.081733+0000","last_fullsized":"2024-04-05T22:29:27.081733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109271+0000","last_change":"2024-04-05T22:29:27.106460+0000","last_active":"2024-04-05T22:29:27.109271+0000","last_peered":"2024-04-05T22:29:27.109271+0000","last_clean":"2024-04-05T22:29:27.109271+0000","last_became_active":"2024-04-05T22:29:27.106113+0000","last_became_peered":"2024-04-05T22:29:27.106113+0000","last_unstale":"2024-04-05T22:29:27.109271+0000","last_undegraded":"2024-04-05T22:29:27.109271+0000","last_fullsized":"2024-04-05T22:29:27.109271+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:47.380812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095782+0000","last_change":"2024-04-05T22:29:27.095782+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095782+0000","last_undegraded":"2024-04-05T22:29:27.095782+0000","last_fullsized":"2024-04-05T22:29:27.095782+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100802+0000","last_change":"2024-04-05T22:29:27.098092+0000","last_active":"2024-04-05T22:29:27.100802+0000","last_peered":"2024-04-05T22:29:27.100802+0000","last_clean":"2024-04-05T22:29:27.100802+0000","last_became_active":"2024-04-05T22:29:27.097449+0000","last_became_peered":"2024-04-05T22:29:27.097449+0000","last_unstale":"2024-04-05T22:29:27.100802+0000","last_undegraded":"2024-04-05T22:29:27.100802+0000","last_fullsized":"2024-04-05T22:29:27.100802+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:40.984286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092341+0000","last_change":"2024-04-05T22:29:27.092341+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092341+0000","last_undegraded":"2024-04-05T22:29:27.092341+0000","last_fullsized":"2024-04-05T22:29:27.092341+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110010+0000","last_change":"2024-04-05T22:29:27.106850+0000","last_active":"2024-04-05T22:29:27.110010+0000","last_peered":"2024-04-05T22:29:27.110010+0000","last_clean":"2024-04-05T22:29:27.110010+0000","last_became_active":"2024-04-05T22:29:27.106525+0000","last_became_peered":"2024-04-05T22:29:27.106525+0000","last_unstale":"2024-04-05T22:29:27.110010+0000","last_undegraded":"2024-04-05T22:29:27.110010+0000","last_fullsized":"2024-04-05T22:29:27.110010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:30.113756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.082197+0000","last_change":"2024-04-05T22:29:27.082197+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.082197+0000","last_undegraded":"2024-04-05T22:29:27.082197+0000","last_fullsized":"2024-04-05T22:29:27.082197+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.117465+0000","last_change":"2024-04-05T22:29:27.113913+0000","last_active":"2024-04-05T22:29:27.117465+0000","last_peered":"2024-04-05T22:29:27.117465+0000","last_clean":"2024-04-05T22:29:27.117465+0000","last_became_active":"2024-04-05T22:29:27.112452+0000","last_became_peered":"2024-04-05T22:29:27.112452+0000","last_unstale":"2024-04-05T22:29:27.117465+0000","last_undegraded":"2024-04-05T22:29:27.117465+0000","last_fullsized":"2024-04-05T22:29:27.117465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:13:41.450507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.089823+0000","last_change":"2024-04-05T22:29:27.089823+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.089823+0000","last_undegraded":"2024-04-05T22:29:27.089823+0000","last_fullsized":"2024-04-05T22:29:27.089823+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.108934+0000","last_change":"2024-04-05T22:29:27.105830+0000","last_active":"2024-04-05T22:29:27.108934+0000","last_peered":"2024-04-05T22:29:27.108934+0000","last_clean":"2024-04-05T22:29:27.108934+0000","last_became_active":"2024-04-05T22:29:27.104458+0000","last_became_peered":"2024-04-05T22:29:27.104458+0000","last_unstale":"2024-04-05T22:29:27.108934+0000","last_undegraded":"2024-04-05T22:29:27.108934+0000","last_fullsized":"2024-04-05T22:29:27.108934+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:34.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096482+0000","last_change":"2024-04-05T22:29:27.096482+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096482+0000","last_undegraded":"2024-04-05T22:29:27.096482+0000","last_fullsized":"2024-04-05T22:29:27.096482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109450+0000","last_change":"2024-04-05T22:29:27.107183+0000","last_active":"2024-04-05T22:29:27.109450+0000","last_peered":"2024-04-05T22:29:27.109450+0000","last_clean":"2024-04-05T22:29:27.109450+0000","last_became_active":"2024-04-05T22:29:27.106873+0000","last_became_peered":"2024-04-05T22:29:27.106873+0000","last_unstale":"2024-04-05T22:29:27.109450+0000","last_undegraded":"2024-04-05T22:29:27.109450+0000","last_fullsized":"2024-04-05T22:29:27.109450+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:31.435759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098271+0000","last_change":"2024-04-05T22:29:27.098271+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098271+0000","last_undegraded":"2024-04-05T22:29:27.098271+0000","last_fullsized":"2024-04-05T22:29:27.098271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.100170+0000","last_change":"2024-04-05T22:29:27.098118+0000","last_active":"2024-04-05T22:29:27.100170+0000","last_peered":"2024-04-05T22:29:27.100170+0000","last_clean":"2024-04-05T22:29:27.100170+0000","last_became_active":"2024-04-05T22:29:27.097772+0000","last_became_peered":"2024-04-05T22:29:27.097772+0000","last_unstale":"2024-04-05T22:29:27.100170+0000","last_undegraded":"2024-04-05T22:29:27.100170+0000","last_fullsized":"2024-04-05T22:29:27.100170+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:04.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.098638+0000","last_change":"2024-04-05T22:29:27.098638+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.098638+0000","last_undegraded":"2024-04-05T22:29:27.098638+0000","last_fullsized":"2024-04-05T22:29:27.098638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118528+0000","last_change":"2024-04-05T22:29:27.117368+0000","last_active":"2024-04-05T22:29:27.118528+0000","last_peered":"2024-04-05T22:29:27.118528+0000","last_clean":"2024-04-05T22:29:27.118528+0000","last_became_active":"2024-04-05T22:29:27.117006+0000","last_became_peered":"2024-04-05T22:29:27.117006+0000","last_unstale":"2024-04-05T22:29:27.118528+0000","last_undegraded":"2024-04-05T22:29:27.118528+0000","last_fullsized":"2024-04-05T22:29:27.118528+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:26.585990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093113+0000","last_change":"2024-04-05T22:29:27.093113+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093113+0000","last_undegraded":"2024-04-05T22:29:27.093113+0000","last_fullsized":"2024-04-05T22:29:27.093113+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109634+0000","last_change":"2024-04-05T22:29:27.107727+0000","last_active":"2024-04-05T22:29:27.109634+0000","last_peered":"2024-04-05T22:29:27.109634+0000","last_clean":"2024-04-05T22:29:27.109634+0000","last_became_active":"2024-04-05T22:29:27.107409+0000","last_became_peered":"2024-04-05T22:29:27.107409+0000","last_unstale":"2024-04-05T22:29:27.109634+0000","last_undegraded":"2024-04-05T22:29:27.109634+0000","last_fullsized":"2024-04-05T22:29:27.109634+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:36:16.139153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096703+0000","last_change":"2024-04-05T22:29:27.096703+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096703+0000","last_undegraded":"2024-04-05T22:29:27.096703+0000","last_fullsized":"2024-04-05T22:29:27.096703+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112336+0000","last_change":"2024-04-05T22:29:27.109281+0000","last_active":"2024-04-05T22:29:27.112336+0000","last_peered":"2024-04-05T22:29:27.112336+0000","last_clean":"2024-04-05T22:29:27.112336+0000","last_became_active":"2024-04-05T22:29:27.108992+0000","last_became_peered":"2024-04-05T22:29:27.108992+0000","last_unstale":"2024-04-05T22:29:27.112336+0000","last_undegraded":"2024-04-05T22:29:27.112336+0000","last_fullsized":"2024-04-05T22:29:27.112336+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:33:12.557477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.091308+0000","last_change":"2024-04-05T22:29:27.091308+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.091308+0000","last_undegraded":"2024-04-05T22:29:27.091308+0000","last_fullsized":"2024-04-05T22:29:27.091308+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112144+0000","last_change":"2024-04-05T22:29:27.109031+0000","last_active":"2024-04-05T22:29:27.112144+0000","last_peered":"2024-04-05T22:29:27.112144+0000","last_clean":"2024-04-05T22:29:27.112144+0000","last_became_active":"2024-04-05T22:29:27.108620+0000","last_became_peered":"2024-04-05T22:29:27.108620+0000","last_unstale":"2024-04-05T22:29:27.112144+0000","last_undegraded":"2024-04-05T22:29:27.112144+0000","last_fullsized":"2024-04-05T22:29:27.112144+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:46.523577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092972+0000","last_change":"2024-04-05T22:29:27.092972+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092972+0000","last_undegraded":"2024-04-05T22:29:27.092972+0000","last_fullsized":"2024-04-05T22:29:27.092972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107366+0000","last_change":"2024-04-05T22:29:27.103901+0000","last_active":"2024-04-05T22:29:27.107366+0000","last_peered":"2024-04-05T22:29:27.107366+0000","last_clean":"2024-04-05T22:29:27.107366+0000","last_became_active":"2024-04-05T22:29:27.103615+0000","last_became_peered":"2024-04-05T22:29:27.103615+0000","last_unstale":"2024-04-05T22:29:27.107366+0000","last_undegraded":"2024-04-05T22:29:27.107366+0000","last_fullsized":"2024-04-05T22:29:27.107366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:46.499208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.095841+0000","last_change":"2024-04-05T22:29:27.095841+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.095841+0000","last_undegraded":"2024-04-05T22:29:27.095841+0000","last_fullsized":"2024-04-05T22:29:27.095841+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106936+0000","last_change":"2024-04-05T22:29:27.104205+0000","last_active":"2024-04-05T22:29:27.106936+0000","last_peered":"2024-04-05T22:29:27.106936+0000","last_clean":"2024-04-05T22:29:27.106936+0000","last_became_active":"2024-04-05T22:29:27.102618+0000","last_became_peered":"2024-04-05T22:29:27.102618+0000","last_unstale":"2024-04-05T22:29:27.106936+0000","last_undegraded":"2024-04-05T22:29:27.106936+0000","last_fullsized":"2024-04-05T22:29:27.106936+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:05.414218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096852+0000","last_change":"2024-04-05T22:29:27.096852+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096852+0000","last_undegraded":"2024-04-05T22:29:27.096852+0000","last_fullsized":"2024-04-05T22:29:27.096852+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116552+0000","last_change":"2024-04-05T22:29:27.113619+0000","last_active":"2024-04-05T22:29:27.116552+0000","last_peered":"2024-04-05T22:29:27.116552+0000","last_clean":"2024-04-05T22:29:27.116552+0000","last_became_active":"2024-04-05T22:29:27.113367+0000","last_became_peered":"2024-04-05T22:29:27.113367+0000","last_unstale":"2024-04-05T22:29:27.116552+0000","last_undegraded":"2024-04-05T22:29:27.116552+0000","last_fullsized":"2024-04-05T22:29:27.116552+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:26:24.419813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.093979+0000","last_change":"2024-04-05T22:29:27.093979+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.093979+0000","last_undegraded":"2024-04-05T22:29:27.093979+0000","last_fullsized":"2024-04-05T22:29:27.093979+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.106232+0000","last_change":"2024-04-05T22:29:27.103848+0000","last_active":"2024-04-05T22:29:27.106232+0000","last_peered":"2024-04-05T22:29:27.106232+0000","last_clean":"2024-04-05T22:29:27.106232+0000","last_became_active":"2024-04-05T22:29:27.103586+0000","last_became_peered":"2024-04-05T22:29:27.103586+0000","last_unstale":"2024-04-05T22:29:27.106232+0000","last_undegraded":"2024-04-05T22:29:27.106232+0000","last_fullsized":"2024-04-05T22:29:27.106232+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:11.324558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092715+0000","last_change":"2024-04-05T22:29:27.092715+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092715+0000","last_undegraded":"2024-04-05T22:29:27.092715+0000","last_fullsized":"2024-04-05T22:29:27.092715+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109481+0000","last_change":"2024-04-05T22:29:27.106062+0000","last_active":"2024-04-05T22:29:27.109481+0000","last_peered":"2024-04-05T22:29:27.109481+0000","last_clean":"2024-04-05T22:29:27.109481+0000","last_became_active":"2024-04-05T22:29:27.105748+0000","last_became_peered":"2024-04-05T22:29:27.105748+0000","last_unstale":"2024-04-05T22:29:27.109481+0000","last_undegraded":"2024-04-05T22:29:27.109481+0000","last_fullsized":"2024-04-05T22:29:27.109481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:48.841874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.086784+0000","last_change":"2024-04-05T22:29:27.086784+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.086784+0000","last_undegraded":"2024-04-05T22:29:27.086784+0000","last_fullsized":"2024-04-05T22:29:27.086784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.105147+0000","last_change":"2024-04-05T22:29:27.102360+0000","last_active":"2024-04-05T22:29:27.105147+0000","last_peered":"2024-04-05T22:29:27.105147+0000","last_clean":"2024-04-05T22:29:27.105147+0000","last_became_active":"2024-04-05T22:29:27.102051+0000","last_became_peered":"2024-04-05T22:29:27.102051+0000","last_unstale":"2024-04-05T22:29:27.105147+0000","last_undegraded":"2024-04-05T22:29:27.105147+0000","last_fullsized":"2024-04-05T22:29:27.105147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:47.073194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081980+0000","last_change":"2024-04-05T22:29:27.081980+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081980+0000","last_undegraded":"2024-04-05T22:29:27.081980+0000","last_fullsized":"2024-04-05T22:29:27.081980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112449+0000","last_change":"2024-04-05T22:29:27.111825+0000","last_active":"2024-04-05T22:29:27.112449+0000","last_peered":"2024-04-05T22:29:27.112449+0000","last_clean":"2024-04-05T22:29:27.112449+0000","last_became_active":"2024-04-05T22:29:27.111542+0000","last_became_peered":"2024-04-05T22:29:27.111542+0000","last_unstale":"2024-04-05T22:29:27.112449+0000","last_undegraded":"2024-04-05T22:29:27.112449+0000","last_fullsized":"2024-04-05T22:29:27.112449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:22.051262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.100124+0000","last_change":"2024-04-05T22:29:27.100124+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.100124+0000","last_undegraded":"2024-04-05T22:29:27.100124+0000","last_fullsized":"2024-04-05T22:29:27.100124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.114102+0000","last_change":"2024-04-05T22:29:27.111984+0000","last_active":"2024-04-05T22:29:27.114102+0000","last_peered":"2024-04-05T22:29:27.114102+0000","last_clean":"2024-04-05T22:29:27.114102+0000","last_became_active":"2024-04-05T22:29:27.111570+0000","last_became_peered":"2024-04-05T22:29:27.111570+0000","last_unstale":"2024-04-05T22:29:27.114102+0000","last_undegraded":"2024-04-05T22:29:27.114102+0000","last_fullsized":"2024-04-05T22:29:27.114102+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:02:55.012896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.081843+0000","last_change":"2024-04-05T22:29:27.081843+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.081843+0000","last_undegraded":"2024-04-05T22:29:27.081843+0000","last_fullsized":"2024-04-05T22:29:27.081843+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.118512+0000","last_change":"2024-04-05T22:29:27.117354+0000","last_active":"2024-04-05T22:29:27.118512+0000","last_peered":"2024-04-05T22:29:27.118512+0000","last_clean":"2024-04-05T22:29:27.118512+0000","last_became_active":"2024-04-05T22:29:27.116968+0000","last_became_peered":"2024-04-05T22:29:27.116968+0000","last_unstale":"2024-04-05T22:29:27.118512+0000","last_undegraded":"2024-04-05T22:29:27.118512+0000","last_fullsized":"2024-04-05T22:29:27.118512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:32:03.796707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.101316+0000","last_change":"2024-04-05T22:29:27.101316+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.101316+0000","last_undegraded":"2024-04-05T22:29:27.101316+0000","last_fullsized":"2024-04-05T22:29:27.101316+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.110905+0000","last_change":"2024-04-05T22:29:27.104579+0000","last_active":"2024-04-05T22:29:27.110905+0000","last_peered":"2024-04-05T22:29:27.110905+0000","last_clean":"2024-04-05T22:29:27.110905+0000","last_became_active":"2024-04-05T22:29:27.104293+0000","last_became_peered":"2024-04-05T22:29:27.104293+0000","last_unstale":"2024-04-05T22:29:27.110905+0000","last_undegraded":"2024-04-05T22:29:27.110905+0000","last_fullsized":"2024-04-05T22:29:27.110905+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:13:51.613749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.097362+0000","last_change":"2024-04-05T22:29:27.097362+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.097362+0000","last_undegraded":"2024-04-05T22:29:27.097362+0000","last_fullsized":"2024-04-05T22:29:27.097362+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116901+0000","last_change":"2024-04-05T22:29:27.115301+0000","last_active":"2024-04-05T22:29:27.116901+0000","last_peered":"2024-04-05T22:29:27.116901+0000","last_clean":"2024-04-05T22:29:27.116901+0000","last_became_active":"2024-04-05T22:29:27.114776+0000","last_became_peered":"2024-04-05T22:29:27.114776+0000","last_unstale":"2024-04-05T22:29:27.116901+0000","last_undegraded":"2024-04-05T22:29:27.116901+0000","last_fullsized":"2024-04-05T22:29:27.116901+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:53.380117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.099085+0000","last_change":"2024-04-05T22:29:27.099085+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.099085+0000","last_undegraded":"2024-04-05T22:29:27.099085+0000","last_fullsized":"2024-04-05T22:29:27.099085+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.107334+0000","last_change":"2024-04-05T22:29:27.105399+0000","last_active":"2024-04-05T22:29:27.107334+0000","last_peered":"2024-04-05T22:29:27.107334+0000","last_clean":"2024-04-05T22:29:27.107334+0000","last_became_active":"2024-04-05T22:29:27.105109+0000","last_became_peered":"2024-04-05T22:29:27.105109+0000","last_unstale":"2024-04-05T22:29:27.107334+0000","last_undegraded":"2024-04-05T22:29:27.107334+0000","last_fullsized":"2024-04-05T22:29:27.107334+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:28.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.090127+0000","last_change":"2024-04-05T22:29:27.090127+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.090127+0000","last_undegraded":"2024-04-05T22:29:27.090127+0000","last_fullsized":"2024-04-05T22:29:27.090127+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097140+0000","last_change":"2024-04-05T22:29:27.094375+0000","last_active":"2024-04-05T22:29:27.097140+0000","last_peered":"2024-04-05T22:29:27.097140+0000","last_clean":"2024-04-05T22:29:27.097140+0000","last_became_active":"2024-04-05T22:29:27.094043+0000","last_became_peered":"2024-04-05T22:29:27.094043+0000","last_unstale":"2024-04-05T22:29:27.097140+0000","last_undegraded":"2024-04-05T22:29:27.097140+0000","last_fullsized":"2024-04-05T22:29:27.097140+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:36.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.085671+0000","last_change":"2024-04-05T22:29:27.085671+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.085671+0000","last_undegraded":"2024-04-05T22:29:27.085671+0000","last_fullsized":"2024-04-05T22:29:27.085671+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.097067+0000","last_change":"2024-04-05T22:29:27.092419+0000","last_active":"2024-04-05T22:29:27.097067+0000","last_peered":"2024-04-05T22:29:27.097067+0000","last_clean":"2024-04-05T22:29:27.097067+0000","last_became_active":"2024-04-05T22:29:27.091178+0000","last_became_peered":"2024-04-05T22:29:27.091178+0000","last_unstale":"2024-04-05T22:29:27.097067+0000","last_undegraded":"2024-04-05T22:29:27.097067+0000","last_fullsized":"2024-04-05T22:29:27.097067+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:31.397216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092775+0000","last_change":"2024-04-05T22:29:27.092775+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092775+0000","last_undegraded":"2024-04-05T22:29:27.092775+0000","last_fullsized":"2024-04-05T22:29:27.092775+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.109970+0000","last_change":"2024-04-05T22:29:27.108317+0000","last_active":"2024-04-05T22:29:27.109970+0000","last_peered":"2024-04-05T22:29:27.109970+0000","last_clean":"2024-04-05T22:29:27.109970+0000","last_became_active":"2024-04-05T22:29:27.107888+0000","last_became_peered":"2024-04-05T22:29:27.107888+0000","last_unstale":"2024-04-05T22:29:27.109970+0000","last_undegraded":"2024-04-05T22:29:27.109970+0000","last_fullsized":"2024-04-05T22:29:27.109970+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:42.291460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092874+0000","last_change":"2024-04-05T22:29:27.092874+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092874+0000","last_undegraded":"2024-04-05T22:29:27.092874+0000","last_fullsized":"2024-04-05T22:29:27.092874+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116811+0000","last_change":"2024-04-05T22:29:27.114605+0000","last_active":"2024-04-05T22:29:27.116811+0000","last_peered":"2024-04-05T22:29:27.116811+0000","last_clean":"2024-04-05T22:29:27.116811+0000","last_became_active":"2024-04-05T22:29:27.114311+0000","last_became_peered":"2024-04-05T22:29:27.114311+0000","last_unstale":"2024-04-05T22:29:27.116811+0000","last_undegraded":"2024-04-05T22:29:27.116811+0000","last_fullsized":"2024-04-05T22:29:27.116811+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:06.440778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.092986+0000","last_change":"2024-04-05T22:29:27.092986+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.092986+0000","last_undegraded":"2024-04-05T22:29:27.092986+0000","last_fullsized":"2024-04-05T22:29:27.092986+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.112429+0000","last_change":"2024-04-05T22:29:27.109061+0000","last_active":"2024-04-05T22:29:27.112429+0000","last_peered":"2024-04-05T22:29:27.112429+0000","last_clean":"2024-04-05T22:29:27.112429+0000","last_became_active":"2024-04-05T22:29:27.108742+0000","last_became_peered":"2024-04-05T22:29:27.108742+0000","last_unstale":"2024-04-05T22:29:27.112429+0000","last_undegraded":"2024-04-05T22:29:27.112429+0000","last_fullsized":"2024-04-05T22:29:27.112429+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:26:35.912596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.094988+0000","last_change":"2024-04-05T22:29:27.094988+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.094988+0000","last_undegraded":"2024-04-05T22:29:27.094988+0000","last_fullsized":"2024-04-05T22:29:27.094988+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.116214+0000","last_change":"2024-04-05T22:29:27.115510+0000","last_active":"2024-04-05T22:29:27.116214+0000","last_peered":"2024-04-05T22:29:27.116214+0000","last_clean":"2024-04-05T22:29:27.116214+0000","last_became_active":"2024-04-05T22:29:27.115162+0000","last_became_peered":"2024-04-05T22:29:27.115162+0000","last_unstale":"2024-04-05T22:29:27.116214+0000","last_undegraded":"2024-04-05T22:29:27.116214+0000","last_fullsized":"2024-04-05T22:29:27.116214+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:44.332750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-04-05T22:29:27.096436+0000","last_change":"2024-04-05T22:29:27.096436+0000","last_active":"2024-04-05T22:29:27.066361+0000","last_peered":"2024-04-05T22:29:27.066361+0000","last_clean":"2024-04-05T22:29:27.066361+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:29:27.096436+0000","last_undegraded":"2024-04-05T22:29:27.096436+0000","last_fullsized":"2024-04-05T22:29:27.096436+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.111421+0000","last_change":"2024-04-05T22:29:27.107244+0000","last_active":"2024-04-05T22:29:27.111421+0000","last_peered":"2024-04-05T22:29:27.111421+0000","last_clean":"2024-04-05T22:29:27.111421+0000","last_became_active":"2024-04-05T22:29:27.106979+0000","last_became_peered":"2024-04-05T22:29:27.106979+0000","last_unstale":"2024-04-05T22:29:27.111421+0000","last_undegraded":"2024-04-05T22:29:27.111421+0000","last_fullsized":"2024-04-05T22:29:27.111421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:28:57.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":299,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-05T22:29:26.085340+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:29:26.085340+0000","last_peered":"2024-04-05T22:29:26.085340+0000","last_clean":"2024-04-05T22:29:26.085340+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:29:26.085340+0000","last_undegraded":"2024-04-05T22:29:26.085340+0000","last_fullsized":"2024-04-05T22:29:26.085340+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.21}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.085},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.401}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.597},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.6339999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":1.026},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.534},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.488},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1.6870000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.6679999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.1499999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":1.0880000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.196},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.343}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.29},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.377}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.258},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.431}]}]},{"osd":5,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.115,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.097},"min":{"1min":0.68300000000000005,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.5900000000000001,"5min":1.907,"15min":1.907},"last":0.999}]},{"osd":1,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.76600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3919999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.98399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6359999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.177,"15min":1.1599999999999999},"min":{"1min":0.82599999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3759999999999999,"5min":2.544,"15min":2.544},"last":1.25},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.89900000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5840000000000001,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0589999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1859999999999999,"15min":1.181},"min":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7529999999999999,"5min":1.867,"15min":1.867},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.268,"5min":1.2909999999999999,"15min":1.3460000000000001},"min":{"1min":1.03,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.649,"5min":2.927,"15min":2.9620000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7050000000000001,"5min":1.911,"15min":2.1970000000000001},"last":1.405},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2969999999999999,"15min":1.302},"min":{"1min":0.89900000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.784,"5min":1.8300000000000001,"15min":2.0499999999999998},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.79,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5129999999999999,"15min":1.5},"min":{"1min":1.284,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.9079999999999999,"5min":2.169,"15min":2.5350000000000001},"last":1.599}]},{"osd":7,"last update":"Fri Apr 5 22:28:25 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.464,"15min":1.456},"min":{"1min":1.2010000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.919,"5min":2.028,"15min":2.2509999999999999},"last":1.448},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5,"15min":1.597},"min":{"1min":1.1799999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":4.4329999999999998},"last":1.5169999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1100000000000001,"15min":1.0249999999999999},"min":{"1min":0.82599999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.466,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.085},"min":{"1min":0.73199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5369999999999999,"5min":1.859,"15min":1.859},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1279999999999999},"min":{"1min":0.88100000000000001,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":1.0009999999999999,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.659,"5min":2.577,"15min":2.577},"last":1.5029999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1319999999999999,"15min":1.202},"min":{"1min":0.75,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.974},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2450000000000001,"15min":1.2},"min":{"1min":1.0389999999999999,"5min":0.76200000000000001,"15min":0.67400000000000004},"max":{"1min":1.657,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.745}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3200000000000001,"15min":1.3640000000000001},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7889999999999999,"5min":1.8799999999999999,"15min":2.1840000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3919999999999999,"15min":1.403},"min":{"1min":1.196,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.7509999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.573}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":1.0529999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1459999999999999,"5min":2.665,"15min":2.665},"last":1.282},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.375,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.915,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.6180000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.4139999999999999,"15min":1.4550000000000001},"min":{"1min":1.0229999999999999,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":1.9950000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":1.212,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.8899999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.524,"15min":1.5389999999999999},"min":{"1min":1.0760000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0019999999999998,"5min":2.1880000000000002,"15min":2.4630000000000001},"last":1.764},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6259999999999999,"15min":1.591},"min":{"1min":1.3839999999999999,"5min":0.91300000000000003,"15min":0.72399999999999998},"max":{"1min":2.109,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":2.0409999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1675264,"data_stored":737649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0229999999999999,"5min":1.744,"15min":1.744},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5860000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.111}]},{"osd":1,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":1.006},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.3720000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.325},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0580000000000001,"15min":1.073},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.391,"5min":1.675,"15min":2.2549999999999999},"last":1.274}]},{"osd":4,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1899999999999999,"15min":1.2290000000000001},"min":{"1min":0.82099999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5289999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.294},"min":{"1min":0.75900000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.399}]},{"osd":5,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.286,"15min":1.351},"min":{"1min":0.73699999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.482},{"interface":"front","average":{"1min":1.246,"5min":1.288,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6459999999999999,"5min":1.958,"15min":2.3239999999999998},"last":1.6579999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.3009999999999999,"15min":1.3500000000000001},"min":{"1min":0.875,"5min":0.505,"15min":0.505},"max":{"1min":1.6240000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.347,"15min":1.401},"min":{"1min":0.97899999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.298,"15min":2.298},"last":1.7}]},{"osd":7,"last update":"Fri Apr 5 22:28:27 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3400000000000001,"15min":1.415},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":2.468},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.377,"15min":1.415},"min":{"1min":0.93700000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.73,"5min":2.508,"15min":2.798},"last":1.8160000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542246,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.4319999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.496}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.399},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.165}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2030000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1662976,"data_stored":731374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.363,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.89900000000000002,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.42299999999999999,"15min":0.36799999999999999},"max":{"1min":1.554,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85399999999999998,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.206,"5min":1.488,"15min":1.5389999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.999,"15min":0.99399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0880000000000001,"15min":1.097},"min":{"1min":0.53600000000000003,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.484,"5min":1.5509999999999999,"15min":1.75},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.3959999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":1.125},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6679999999999999,"5min":1.806,"15min":2.2789999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.258,"15min":1.262},"min":{"1min":0.84099999999999997,"5min":0.76000000000000001,"15min":0.68100000000000005},"max":{"1min":1.706,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.873,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.5429999999999999,"5min":1.843,"15min":2.173},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.369,"15min":1.3340000000000001},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6439999999999999,"5min":1.9059999999999999,"15min":1.917},"last":1.264},{"interface":"front","average":{"1min":1.292,"5min":1.345,"15min":1.355},"min":{"1min":0.92200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:28:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.825,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.407},{"interface":"front","average":{"1min":1.196,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.6319999999999999,"5min":1.9279999999999999,"15min":2.3879999999999999},"last":1.429}]}]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1328017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.1479999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.92600000000000005,"15min":0.92000000000000004},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2569999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.2729999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.25,"5min":1.976,"15min":1.976},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0840000000000001,"15min":1.093},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5900000000000001,"5min":2.157,"15min":2.157},"last":0.749}]},{"osd":4,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.2,"15min":1.1699999999999999},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.429,"5min":1.931,"15min":1.931},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3129999999999999,"15min":1.302},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.786,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.0549999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2549999999999999,"15min":1.2609999999999999},"min":{"1min":0.75700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.524,"5min":2.105,"15min":2.3879999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.2130000000000001,"15min":1.198},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.357,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.878,"15min":0.65500000000000003},"max":{"1min":1.8340000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3280000000000001,"15min":1.365},"min":{"1min":0.87,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.585,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:28:26 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3320000000000001,"15min":1.337},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.6080000000000001,"5min":1.8180000000000001,"15min":1.9810000000000001},"last":1.151}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:29:33.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081084+0000', 'last_change': '2024-04-05T22:29:27.081084+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081084+0000', 'last_undegraded': '2024-04-05T22:29:27.081084+0000', 'last_fullsized': '2024-04-05T22:29:27.081084+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092518+0000', 'last_change': '2024-04-05T22:29:27.092518+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092518+0000', 'last_undegraded': '2024-04-05T22:29:27.092518+0000', 'last_fullsized': '2024-04-05T22:29:27.092518+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.091256+0000', 'last_change': '2024-04-05T22:29:27.091256+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.091256+0000', 'last_undegraded': '2024-04-05T22:29:27.091256+0000', 'last_fullsized': '2024-04-05T22:29:27.091256+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.082145+0000', 'last_change': '2024-04-05T22:29:27.082145+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.082145+0000', 'last_undegraded': '2024-04-05T22:29:27.082145+0000', 'last_fullsized': '2024-04-05T22:29:27.082145+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2024-04-05T22:29:33.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.097597+0000', 'last_change': '2024-04-05T22:29:27.097597+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.097597+0000', 'last_undegraded': '2024-04-05T22:29:27.097597+0000', 'last_fullsized': '2024-04-05T22:29:27.097597+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.082968+0000', 'last_change': '2024-04-05T22:29:27.082968+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.082968+0000', 'last_undegraded': '2024-04-05T22:29:27.082968+0000', 'last_fullsized': '2024-04-05T22:29:27.082968+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096950+0000', 'last_change': '2024-04-05T22:29:27.096950+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096950+0000', 'last_undegraded': '2024-04-05T22:29:27.096950+0000', 'last_fullsized': '2024-04-05T22:29:27.096950+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2024-04-05T22:29:33.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081332+0000', 'last_change': '2024-04-05T22:29:27.081332+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081332+0000', 'last_undegraded': '2024-04-05T22:29:27.081332+0000', 'last_fullsized': '2024-04-05T22:29:27.081332+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092768+0000', 'last_change': '2024-04-05T22:29:27.092768+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092768+0000', 'last_undegraded': '2024-04-05T22:29:27.092768+0000', 'last_fullsized': '2024-04-05T22:29:27.092768+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.102673+0000', 'last_change': '2024-04-05T22:29:27.102673+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.102673+0000', 'last_undegraded': '2024-04-05T22:29:27.102673+0000', 'last_fullsized': '2024-04-05T22:29:27.102673+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.089407+0000', 'last_change': '2024-04-05T22:29:27.089407+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.089407+0000', 'last_undegraded': '2024-04-05T22:29:27.089407+0000', 'last_fullsized': '2024-04-05T22:29:27.089407+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096067+0000', 'last_change': '2024-04-05T22:29:27.096067+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096067+0000', 'last_undegraded': '2024-04-05T22:29:27.096067+0000', 'last_fullsized': '2024-04-05T22:29:27.096067+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2024-04-05T22:29:33.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.094222+0000', 'last_change': '2024-04-05T22:29:27.094222+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.094222+0000', 'last_undegraded': '2024-04-05T22:29:27.094222+0000', 'last_fullsized': '2024-04-05T22:29:27.094222+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.086244+0000', 'last_change': '2024-04-05T22:29:27.086244+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.086244+0000', 'last_undegraded': '2024-04-05T22:29:27.086244+0000', 'last_fullsized': '2024-04-05T22:29:27.086244+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.095219+0000', 'last_change': '2024-04-05T22:29:27.095219+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.095219+0000', 'last_undegraded': '2024-04-05T22:29:27.095219+0000', 'last_fullsized': '2024-04-05T22:29:27.095219+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096108+0000', 'last_change': '2024-04-05T22:29:27.096108+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096108+0000', 'last_undegraded': '2024-04-05T22:29:27.096108+0000', 'last_fullsized': '2024-04-05T22:29:27.096108+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.097011+0000', 'last_change': '2024-04-05T22:29:27.097011+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.097011+0000', 'last_undegraded': '2024-04-05T22:29:27.097011+0000', 'last_fullsized': '2024-04-05T22:29:27.097011+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.098055+0000', 'last_change': '2024-04-05T22:29:27.098055+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.098055+0000', 'last_undegraded': '2024-04-05T22:29:27.098055+0000', 'last_fullsized': '2024-04-05T22:29:27.098055+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2024-04-05T22:29:33.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.084360+0000', 'last_change': '2024-04-05T22:29:27.084360+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.084360+0000', 'last_undegraded': '2024-04-05T22:29:27.084360+0000', 'last_fullsized': '2024-04-05T22:29:27.084360+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2024-04-05T22:29:33.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.093190+0000', 'last_change': '2024-04-05T22:29:27.093190+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.093190+0000', 'last_undegraded': '2024-04-05T22:29:27.093190+0000', 'last_fullsized': '2024-04-05T22:29:27.093190+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2024-04-05T22:29:33.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.099854+0000', 'last_change': '2024-04-05T22:29:27.099854+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.099854+0000', 'last_undegraded': '2024-04-05T22:29:27.099854+0000', 'last_fullsized': '2024-04-05T22:29:27.099854+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.084747+0000', 'last_change': '2024-04-05T22:29:27.084747+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.084747+0000', 'last_undegraded': '2024-04-05T22:29:27.084747+0000', 'last_fullsized': '2024-04-05T22:29:27.084747+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.082244+0000', 'last_change': '2024-04-05T22:29:27.082244+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.082244+0000', 'last_undegraded': '2024-04-05T22:29:27.082244+0000', 'last_fullsized': '2024-04-05T22:29:27.082244+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.094229+0000', 'last_change': '2024-04-05T22:29:27.094229+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.094229+0000', 'last_undegraded': '2024-04-05T22:29:27.094229+0000', 'last_fullsized': '2024-04-05T22:29:27.094229+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2024-04-05T22:29:33.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.095176+0000', 'last_change': '2024-04-05T22:29:27.095176+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.095176+0000', 'last_undegraded': '2024-04-05T22:29:27.095176+0000', 'last_fullsized': '2024-04-05T22:29:27.095176+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081906+0000', 'last_change': '2024-04-05T22:29:27.081906+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081906+0000', 'last_undegraded': '2024-04-05T22:29:27.081906+0000', 'last_fullsized': '2024-04-05T22:29:27.081906+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096447+0000', 'last_change': '2024-04-05T22:29:27.096447+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096447+0000', 'last_undegraded': '2024-04-05T22:29:27.096447+0000', 'last_fullsized': '2024-04-05T22:29:27.096447+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.100820+0000', 'last_change': '2024-04-05T22:29:27.100820+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.100820+0000', 'last_undegraded': '2024-04-05T22:29:27.100820+0000', 'last_fullsized': '2024-04-05T22:29:27.100820+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081518+0000', 'last_change': '2024-04-05T22:29:27.081518+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081518+0000', 'last_undegraded': '2024-04-05T22:29:27.081518+0000', 'last_fullsized': '2024-04-05T22:29:27.081518+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.083464+0000', 'last_change': '2024-04-05T22:29:27.083464+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.083464+0000', 'last_undegraded': '2024-04-05T22:29:27.083464+0000', 'last_fullsized': '2024-04-05T22:29:27.083464+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2024-04-05T22:29:33.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.090854+0000', 'last_change': '2024-04-05T22:29:27.090854+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.090854+0000', 'last_undegraded': '2024-04-05T22:29:27.090854+0000', 'last_fullsized': '2024-04-05T22:29:27.090854+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.094333+0000', 'last_change': '2024-04-05T22:29:27.094333+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.094333+0000', 'last_undegraded': '2024-04-05T22:29:27.094333+0000', 'last_fullsized': '2024-04-05T22:29:27.094333+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.101884+0000', 'last_change': '2024-04-05T22:29:27.101884+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.101884+0000', 'last_undegraded': '2024-04-05T22:29:27.101884+0000', 'last_fullsized': '2024-04-05T22:29:27.101884+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.091093+0000', 'last_change': '2024-04-05T22:29:27.091093+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.091093+0000', 'last_undegraded': '2024-04-05T22:29:27.091093+0000', 'last_fullsized': '2024-04-05T22:29:27.091093+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081733+0000', 'last_change': '2024-04-05T22:29:27.081733+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081733+0000', 'last_undegraded': '2024-04-05T22:29:27.081733+0000', 'last_fullsized': '2024-04-05T22:29:27.081733+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.095782+0000', 'last_change': '2024-04-05T22:29:27.095782+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.095782+0000', 'last_undegraded': '2024-04-05T22:29:27.095782+0000', 'last_fullsized': '2024-04-05T22:29:27.095782+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2024-04-05T22:29:33.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092341+0000', 'last_change': '2024-04-05T22:29:27.092341+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092341+0000', 'last_undegraded': '2024-04-05T22:29:27.092341+0000', 'last_fullsized': '2024-04-05T22:29:27.092341+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.082197+0000', 'last_change': '2024-04-05T22:29:27.082197+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.082197+0000', 'last_undegraded': '2024-04-05T22:29:27.082197+0000', 'last_fullsized': '2024-04-05T22:29:27.082197+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.089823+0000', 'last_change': '2024-04-05T22:29:27.089823+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.089823+0000', 'last_undegraded': '2024-04-05T22:29:27.089823+0000', 'last_fullsized': '2024-04-05T22:29:27.089823+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096482+0000', 'last_change': '2024-04-05T22:29:27.096482+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096482+0000', 'last_undegraded': '2024-04-05T22:29:27.096482+0000', 'last_fullsized': '2024-04-05T22:29:27.096482+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.098271+0000', 'last_change': '2024-04-05T22:29:27.098271+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.098271+0000', 'last_undegraded': '2024-04-05T22:29:27.098271+0000', 'last_fullsized': '2024-04-05T22:29:27.098271+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.098638+0000', 'last_change': '2024-04-05T22:29:27.098638+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.098638+0000', 'last_undegraded': '2024-04-05T22:29:27.098638+0000', 'last_fullsized': '2024-04-05T22:29:27.098638+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.093113+0000', 'last_change': '2024-04-05T22:29:27.093113+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.093113+0000', 'last_undegraded': '2024-04-05T22:29:27.093113+0000', 'last_fullsized': '2024-04-05T22:29:27.093113+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096703+0000', 'last_change': '2024-04-05T22:29:27.096703+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096703+0000', 'last_undegraded': '2024-04-05T22:29:27.096703+0000', 'last_fullsized': '2024-04-05T22:29:27.096703+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.091308+0000', 'last_change': '2024-04-05T22:29:27.091308+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.091308+0000', 'last_undegraded': '2024-04-05T22:29:27.091308+0000', 'last_fullsized': '2024-04-05T22:29:27.091308+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092972+0000', 'last_change': '2024-04-05T22:29:27.092972+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092972+0000', 'last_undegraded': '2024-04-05T22:29:27.092972+0000', 'last_fullsized': '2024-04-05T22:29:27.092972+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.095841+0000', 'last_change': '2024-04-05T22:29:27.095841+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.095841+0000', 'last_undegraded': '2024-04-05T22:29:27.095841+0000', 'last_fullsized': '2024-04-05T22:29:27.095841+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2024-04-05T22:29:33.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096852+0000', 'last_change': '2024-04-05T22:29:27.096852+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096852+0000', 'last_undegraded': '2024-04-05T22:29:27.096852+0000', 'last_fullsized': '2024-04-05T22:29:27.096852+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.093979+0000', 'last_change': '2024-04-05T22:29:27.093979+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.093979+0000', 'last_undegraded': '2024-04-05T22:29:27.093979+0000', 'last_fullsized': '2024-04-05T22:29:27.093979+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092715+0000', 'last_change': '2024-04-05T22:29:27.092715+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092715+0000', 'last_undegraded': '2024-04-05T22:29:27.092715+0000', 'last_fullsized': '2024-04-05T22:29:27.092715+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.086784+0000', 'last_change': '2024-04-05T22:29:27.086784+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.086784+0000', 'last_undegraded': '2024-04-05T22:29:27.086784+0000', 'last_fullsized': '2024-04-05T22:29:27.086784+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081980+0000', 'last_change': '2024-04-05T22:29:27.081980+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081980+0000', 'last_undegraded': '2024-04-05T22:29:27.081980+0000', 'last_fullsized': '2024-04-05T22:29:27.081980+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.100124+0000', 'last_change': '2024-04-05T22:29:27.100124+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.100124+0000', 'last_undegraded': '2024-04-05T22:29:27.100124+0000', 'last_fullsized': '2024-04-05T22:29:27.100124+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2024-04-05T22:29:33.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.081843+0000', 'last_change': '2024-04-05T22:29:27.081843+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.081843+0000', 'last_undegraded': '2024-04-05T22:29:27.081843+0000', 'last_fullsized': '2024-04-05T22:29:27.081843+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.101316+0000', 'last_change': '2024-04-05T22:29:27.101316+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.101316+0000', 'last_undegraded': '2024-04-05T22:29:27.101316+0000', 'last_fullsized': '2024-04-05T22:29:27.101316+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.097362+0000', 'last_change': '2024-04-05T22:29:27.097362+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.097362+0000', 'last_undegraded': '2024-04-05T22:29:27.097362+0000', 'last_fullsized': '2024-04-05T22:29:27.097362+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.099085+0000', 'last_change': '2024-04-05T22:29:27.099085+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.099085+0000', 'last_undegraded': '2024-04-05T22:29:27.099085+0000', 'last_fullsized': '2024-04-05T22:29:27.099085+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.090127+0000', 'last_change': '2024-04-05T22:29:27.090127+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.090127+0000', 'last_undegraded': '2024-04-05T22:29:27.090127+0000', 'last_fullsized': '2024-04-05T22:29:27.090127+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.085671+0000', 'last_change': '2024-04-05T22:29:27.085671+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.085671+0000', 'last_undegraded': '2024-04-05T22:29:27.085671+0000', 'last_fullsized': '2024-04-05T22:29:27.085671+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2024-04-05T22:29:33.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092775+0000', 'last_change': '2024-04-05T22:29:27.092775+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092775+0000', 'last_undegraded': '2024-04-05T22:29:27.092775+0000', 'last_fullsized': '2024-04-05T22:29:27.092775+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092874+0000', 'last_change': '2024-04-05T22:29:27.092874+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092874+0000', 'last_undegraded': '2024-04-05T22:29:27.092874+0000', 'last_fullsized': '2024-04-05T22:29:27.092874+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.092986+0000', 'last_change': '2024-04-05T22:29:27.092986+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.092986+0000', 'last_undegraded': '2024-04-05T22:29:27.092986+0000', 'last_fullsized': '2024-04-05T22:29:27.092986+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.094988+0000', 'last_change': '2024-04-05T22:29:27.094988+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.094988+0000', 'last_undegraded': '2024-04-05T22:29:27.094988+0000', 'last_fullsized': '2024-04-05T22:29:27.094988+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2024-04-05T22:29:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-04-05T22:29:27.096436+0000', 'last_change': '2024-04-05T22:29:27.096436+0000', 'last_active': '2024-04-05T22:29:27.066361+0000', 'last_peered': '2024-04-05T22:29:27.066361+0000', 'last_clean': '2024-04-05T22:29:27.066361+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-05T22:29:27.096436+0000', 'last_undegraded': '2024-04-05T22:29:27.096436+0000', 'last_fullsized': '2024-04-05T22:29:27.096436+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'last_clean_scrub_stamp': '2024-04-05T22:29:27.066361+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-05T22:29:33.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-05T22:29:33.058 INFO:tasks.ceph.mds.a.smithi073.stderr:2024-04-05T22:29:33.058+0000 7f45ca600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:29:33.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.269+0000 7f66ef400640 1 Processor -- start 2024-04-05T22:29:33.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.269+0000 7f66ef400640 1 -- start start 2024-04-05T22:29:33.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66ef400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66e806cb80 0x7f66e806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66ef400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66e815e850 0x7f66e815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66ef400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e806f5b0 con 0x7f66e81587d0 2024-04-05T22:29:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66ef400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e806d500 con 0x7f66e806cb80 2024-04-05T22:29:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66ef400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e815f220 con 0x7f66e815e850 2024-04-05T22:29:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66e806cb80 0x7f66e806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66dfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66e806cb80 0x7f66e806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34504/0 (socket says 172.21.15.73:34504) 2024-04-05T22:29:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.271+0000 7f66dfe00640 1 -- 172.21.15.73:0/1877169833 learned_addr learned my addr 172.21.15.73:0/1877169833 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.272+0000 7f66ed400640 1 --2- 172.21.15.73:0/1877169833 >> v2:172.21.15.157:3300/0 conn(0x7f66e815e850 0x7f66e815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.272+0000 7f66dfe00640 1 -- 172.21.15.73:0/1877169833 >> v2:172.21.15.157:3300/0 conn(0x7f66e815e850 msgr2=0x7f66e815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.272+0000 7f66dfe00640 1 --2- 172.21.15.73:0/1877169833 >> v2:172.21.15.157:3300/0 conn(0x7f66e815e850 0x7f66e815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.272+0000 7f66dfe00640 1 -- 172.21.15.73:0/1877169833 >> v1:172.21.15.157:6789/0 conn(0x7f66e81587d0 legacy=0x7f66e80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.272+0000 7f66dfe00640 1 -- 172.21.15.73:0/1877169833 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8070410 con 0x7f66e806cb80 2024-04-05T22:29:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.273+0000 7f66dfe00640 1 --2- 172.21.15.73:0/1877169833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e806cfc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f66e815c270 tx=0x7f66d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5990f0530f09d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.274+0000 7f66e7e00640 1 -- 172.21.15.73:0/1877169833 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0031870 con 0x7f66e806cb80 2024-04-05T22:29:33.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.274+0000 7f66e7e00640 1 -- 172.21.15.73:0/1877169833 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00319d0 con 0x7f66e806cb80 2024-04-05T22:29:33.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.274+0000 7f66e7e00640 1 -- 172.21.15.73:0/1877169833 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0031d60 con 0x7f66e806cb80 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.274+0000 7f66ef400640 1 -- 172.21.15.73:0/1877169833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 msgr2=0x7f66e806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.274+0000 7f66ef400640 1 --2- 172.21.15.73:0/1877169833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e806cfc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f66e815c270 tx=0x7f66d002b910 comp rx=0 tx=0).stop 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 -- 172.21.15.73:0/1877169833 shutdown_connections 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 --2- 172.21.15.73:0/1877169833 >> v2:172.21.15.157:3300/0 conn(0x7f66e815e850 0x7f66e815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 --2- 172.21.15.73:0/1877169833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e806cfc0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 -- 172.21.15.73:0/1877169833 >> 172.21.15.73:0/1877169833 conn(0x7f66e80ce8e0 msgr2=0x7f66e80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 -- 172.21.15.73:0/1877169833 shutdown_connections 2024-04-05T22:29:33.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.275+0000 7f66ef400640 1 -- 172.21.15.73:0/1877169833 wait complete. 2024-04-05T22:29:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.277+0000 7f66ef400640 1 Processor -- start 2024-04-05T22:29:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.277+0000 7f66ef400640 1 -- start start 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e81587d0 0x7f66e80afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e815e850 0x7f66e80adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80ae960 con 0x7f66e806cb80 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80aec40 con 0x7f66e815e850 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66ef400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80aef20 con 0x7f66e81587d0 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.278+0000 7f66eca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34512/0 (socket says 172.21.15.73:34512) 2024-04-05T22:29:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 -- 172.21.15.73:0/358851335 learned_addr learned my addr 172.21.15.73:0/358851335 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66ed400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e815e850 0x7f66e80adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66dfe00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e81587d0 0x7f66e80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 -- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e81587d0 msgr2=0x7f66e80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e81587d0 0x7f66e80afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 -- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e815e850 msgr2=0x7f66e80adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e815e850 0x7f66e80adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 -- 172.21.15.73:0/358851335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00094d0 con 0x7f66e806cb80 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.279+0000 7f66eca00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f66d800c430 tx=0x7f66d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.280+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8011790 con 0x7f66e806cb80 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.280+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d80118f0 con 0x7f66e806cb80 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.280+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8011ce0 con 0x7f66e806cb80 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.280+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e80af200 con 0x7f66e806cb80 2024-04-05T22:29:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.280+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e8151e50 con 0x7f66e806cb80 2024-04-05T22:29:33.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.281+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e8066af0 con 0x7f66e806cb80 2024-04-05T22:29:33.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.282+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f66d801c460 con 0x7f66e806cb80 2024-04-05T22:29:33.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.283+0000 7f66e5600640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 0x7f66c406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.283+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66d8089e30 con 0x7f66e806cb80 2024-04-05T22:29:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.283+0000 7f66dfe00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 0x7f66c406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.285+0000 7f66dfe00640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 0x7f66c406d660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66d0036000 tx=0x7f66d003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.287+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f66d80546d0 con 0x7f66e806cb80 2024-04-05T22:29:33.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.552+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f66e8068160 con 0x7f66e806cb80 2024-04-05T22:29:33.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.552+0000 7f66e5600640 1 -- 172.21.15.73:0/358851335 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1198 (secure 0 0 0) 0x7f66d8057cf0 con 0x7f66e806cb80 2024-04-05T22:29:33.553 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:33.553 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"38861c87-67cf-4180-a71f-97401becb77d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":518,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1712355667,"num_in_osds":8,"osd_in_since":1712355659,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":286216192,"bytes_avail":772807897088,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":82,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":8685}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-04-05T22:28:29.542444+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 msgr2=0x7f66c406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 0x7f66c406d660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66d0036000 tx=0x7f66d003c040 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 msgr2=0x7f66e8151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f66d800c430 tx=0x7f66d800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 shutdown_connections 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f66c406b1a0 0x7f66c406d660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e815e850 0x7f66e80adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e81587d0 0x7f66e80afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 --2- 172.21.15.73:0/358851335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e806cb80 0x7f66e8151910 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 >> 172.21.15.73:0/358851335 conn(0x7f66e80ce8e0 msgr2=0x7f66e80708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 shutdown_connections 2024-04-05T22:29:33.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.555+0000 7f66ef400640 1 -- 172.21.15.73:0/358851335 wait complete. 2024-04-05T22:29:33.575 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-05T22:29:33.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:29:33.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.783+0000 7f0eafe00640 1 Processor -- start 2024-04-05T22:29:33.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.783+0000 7f0eafe00640 1 -- start start 2024-04-05T22:29:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.784+0000 7f0eafe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0eb00b20a0 0x7f0eb0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.784+0000 7f0eafe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b1b50 0x7f0eb006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.784+0000 7f0eafe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb006d3e0 con 0x7f0eb00b2480 2024-04-05T22:29:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eafe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb006d610 con 0x7f0eb00b20a0 2024-04-05T22:29:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eafe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0070230 con 0x7f0eb00b1b50 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0eb00b20a0 0x7f0eb0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaf400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0eb00b2480 0x7f0eb00b28d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53212/0 (socket says 172.21.15.73:53212) 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaf400640 1 -- 172.21.15.73:0/1443459939 learned_addr learned my addr 172.21.15.73:0/1443459939 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaea00640 1 -- 172.21.15.73:0/1443459939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b1b50 msgr2=0x7f0eb006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaea00640 1 --2- 172.21.15.73:0/1443459939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b1b50 0x7f0eb006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaea00640 1 -- 172.21.15.73:0/1443459939 >> v1:172.21.15.73:6789/0 conn(0x7f0eb00b2480 legacy=0x7f0eb00b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eaea00640 1 -- 172.21.15.73:0/1443459939 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0070510 con 0x7f0eb00b20a0 2024-04-05T22:29:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.785+0000 7f0eae000640 1 -- 172.21.15.73:0/1443459939 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844704386 0 0) 0x7f0eb006d3e0 con 0x7f0eb00b2480 2024-04-05T22:29:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.786+0000 7f0eaea00640 1 --2- 172.21.15.73:0/1443459939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b20a0 0x7f0eb0070ff0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097a0 tx=0x7f0ea002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3eabef261cc593 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.787+0000 7f0eae000640 1 -- 172.21.15.73:0/1443459939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00316f0 con 0x7f0eb00b20a0 2024-04-05T22:29:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.787+0000 7f0eae000640 1 -- 172.21.15.73:0/1443459939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0031850 con 0x7f0eb00b20a0 2024-04-05T22:29:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.787+0000 7f0eae000640 1 -- 172.21.15.73:0/1443459939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0031be0 con 0x7f0eb00b20a0 2024-04-05T22:29:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.787+0000 7f0eafe00640 1 -- 172.21.15.73:0/1443459939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b20a0 msgr2=0x7f0eb0070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.787+0000 7f0eafe00640 1 --2- 172.21.15.73:0/1443459939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b20a0 0x7f0eb0070ff0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097a0 tx=0x7f0ea002b790 comp rx=0 tx=0).stop 2024-04-05T22:29:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 -- 172.21.15.73:0/1443459939 shutdown_connections 2024-04-05T22:29:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 --2- 172.21.15.73:0/1443459939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b1b50 0x7f0eb006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 --2- 172.21.15.73:0/1443459939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b20a0 0x7f0eb0070ff0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 -- 172.21.15.73:0/1443459939 >> 172.21.15.73:0/1443459939 conn(0x7f0eb00b06c0 msgr2=0x7f0eb00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 -- 172.21.15.73:0/1443459939 shutdown_connections 2024-04-05T22:29:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.788+0000 7f0eafe00640 1 -- 172.21.15.73:0/1443459939 wait complete. 2024-04-05T22:29:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.789+0000 7f0eafe00640 1 Processor -- start 2024-04-05T22:29:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 -- start start 2024-04-05T22:29:33.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b20a0 0x7f0eb0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb00b2480 0x7f0eb015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb015b6d0 con 0x7f0eb00b1b50 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb015aac0 con 0x7f0eb00b2480 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.790+0000 7f0eafe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb015ada0 con 0x7f0eb00b20a0 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34530/0 (socket says 172.21.15.73:34530) 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 -- 172.21.15.73:0/4188954251 learned_addr learned my addr 172.21.15.73:0/4188954251 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 -- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b20a0 msgr2=0x7f0eb0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b20a0 0x7f0eb0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 -- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb00b2480 msgr2=0x7f0eb015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb00b2480 0x7f0eb015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 -- 172.21.15.73:0/4188954251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009400 con 0x7f0eb00b1b50 2024-04-05T22:29:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.791+0000 7f0eaea00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000c00 tx=0x7f0ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.792+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004230 con 0x7f0eb00b1b50 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.792+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0031f00 con 0x7f0eb00b1b50 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.792+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea003f690 con 0x7f0eb00b1b50 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.792+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb015b080 con 0x7f0eb00b1b50 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.792+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb0143140 con 0x7f0eb00b1b50 2024-04-05T22:29:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.793+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0143330 con 0x7f0eb00b1b50 2024-04-05T22:29:33.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.794+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f0ea003b040 con 0x7f0eb00b1b50 2024-04-05T22:29:33.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.795+0000 7f0ea7400640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 0x7f0e8c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:33.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.796+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea00abff0 con 0x7f0eb00b1b50 2024-04-05T22:29:33.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.796+0000 7f0ea6000640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 0x7f0e8c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:33.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.797+0000 7f0ea6000640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 0x7f0e8c06d660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00ae580 tx=0x7f0e98006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:33.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.801+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0ea0076930 con 0x7f0eb00b1b50 2024-04-05T22:29:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:33.992+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eb0070ad0 con 0x7f0e8c06b1a0 2024-04-05T22:29:34.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.015+0000 7f0ea7400640 1 -- 172.21.15.73:0/4188954251 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287806 (secure 0 0 0) 0x7f0e98012390 con 0x7f0e8c06b1a0 2024-04-05T22:29:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:34.021 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 msgr2=0x7f0e8c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 0x7f0e8c06d660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00ae580 tx=0x7f0e98006bf0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 msgr2=0x7f0eb0142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000c00 tx=0x7f0ea00046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 shutdown_connections 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e8c06b1a0 0x7f0e8c06d660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb00b2480 0x7f0eb015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0eb00b20a0 0x7f0eb0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 --2- 172.21.15.73:0/4188954251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb00b1b50 0x7f0eb0142c00 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:34.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 >> 172.21.15.73:0/4188954251 conn(0x7f0eb00b06c0 msgr2=0x7f0eb00cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:34.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 shutdown_connections 2024-04-05T22:29:34.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:34.022+0000 7f0eafe00640 1 -- 172.21.15.73:0/4188954251 wait complete. 2024-04-05T22:29:34.040 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-05T22:29:33.564688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279816,"kb_used_data":14504,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754694904,"statfs":{"total":773094113280,"available":772807581696,"internally_reserved":0,"allocated":14852096,"data_stored":7191252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.488802"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584861+0000","last_change":"2024-04-05T22:29:27.581218+0000","last_active":"2024-04-05T22:29:27.584861+0000","last_peered":"2024-04-05T22:29:27.584861+0000","last_clean":"2024-04-05T22:29:27.584861+0000","last_became_active":"2024-04-05T22:29:27.580977+0000","last_became_peered":"2024-04-05T22:29:27.580977+0000","last_unstale":"2024-04-05T22:29:27.584861+0000","last_undegraded":"2024-04-05T22:29:27.584861+0000","last_fullsized":"2024-04-05T22:29:27.584861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582488+0000","last_change":"2024-04-05T22:29:27.581336+0000","last_active":"2024-04-05T22:29:27.582488+0000","last_peered":"2024-04-05T22:29:27.582488+0000","last_clean":"2024-04-05T22:29:27.582488+0000","last_became_active":"2024-04-05T22:29:27.580747+0000","last_became_peered":"2024-04-05T22:29:27.580747+0000","last_unstale":"2024-04-05T22:29:27.582488+0000","last_undegraded":"2024-04-05T22:29:27.582488+0000","last_fullsized":"2024-04-05T22:29:27.582488+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.604225+0000","last_change":"2024-04-05T22:29:27.601388+0000","last_active":"2024-04-05T22:29:27.604225+0000","last_peered":"2024-04-05T22:29:27.604225+0000","last_clean":"2024-04-05T22:29:27.604225+0000","last_became_active":"2024-04-05T22:29:27.600096+0000","last_became_peered":"2024-04-05T22:29:27.600096+0000","last_unstale":"2024-04-05T22:29:27.604225+0000","last_undegraded":"2024-04-05T22:29:27.604225+0000","last_fullsized":"2024-04-05T22:29:27.604225+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608572+0000","last_change":"2024-04-05T22:29:27.607311+0000","last_active":"2024-04-05T22:29:27.608572+0000","last_peered":"2024-04-05T22:29:27.608572+0000","last_clean":"2024-04-05T22:29:27.608572+0000","last_became_active":"2024-04-05T22:29:27.606387+0000","last_became_peered":"2024-04-05T22:29:27.606387+0000","last_unstale":"2024-04-05T22:29:27.608572+0000","last_undegraded":"2024-04-05T22:29:27.608572+0000","last_fullsized":"2024-04-05T22:29:27.608572+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609841+0000","last_change":"2024-04-05T22:29:27.603416+0000","last_active":"2024-04-05T22:29:27.609841+0000","last_peered":"2024-04-05T22:29:27.609841+0000","last_clean":"2024-04-05T22:29:27.609841+0000","last_became_active":"2024-04-05T22:29:27.603098+0000","last_became_peered":"2024-04-05T22:29:27.603098+0000","last_unstale":"2024-04-05T22:29:27.609841+0000","last_undegraded":"2024-04-05T22:29:27.609841+0000","last_fullsized":"2024-04-05T22:29:27.609841+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603750+0000","last_change":"2024-04-05T22:29:27.600686+0000","last_active":"2024-04-05T22:29:27.603750+0000","last_peered":"2024-04-05T22:29:27.603750+0000","last_clean":"2024-04-05T22:29:27.603750+0000","last_became_active":"2024-04-05T22:29:27.599951+0000","last_became_peered":"2024-04-05T22:29:27.599951+0000","last_unstale":"2024-04-05T22:29:27.603750+0000","last_undegraded":"2024-04-05T22:29:27.603750+0000","last_fullsized":"2024-04-05T22:29:27.603750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595725+0000","last_change":"2024-04-05T22:29:27.592245+0000","last_active":"2024-04-05T22:29:27.595725+0000","last_peered":"2024-04-05T22:29:27.595725+0000","last_clean":"2024-04-05T22:29:27.595725+0000","last_became_active":"2024-04-05T22:29:27.591834+0000","last_became_peered":"2024-04-05T22:29:27.591834+0000","last_unstale":"2024-04-05T22:29:27.595725+0000","last_undegraded":"2024-04-05T22:29:27.595725+0000","last_fullsized":"2024-04-05T22:29:27.595725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603756+0000","last_change":"2024-04-05T22:29:27.600786+0000","last_active":"2024-04-05T22:29:27.603756+0000","last_peered":"2024-04-05T22:29:27.603756+0000","last_clean":"2024-04-05T22:29:27.603756+0000","last_became_active":"2024-04-05T22:29:27.600517+0000","last_became_peered":"2024-04-05T22:29:27.600517+0000","last_unstale":"2024-04-05T22:29:27.603756+0000","last_undegraded":"2024-04-05T22:29:27.603756+0000","last_fullsized":"2024-04-05T22:29:27.603756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577610+0000","last_change":"2024-04-05T22:29:27.574937+0000","last_active":"2024-04-05T22:29:27.577610+0000","last_peered":"2024-04-05T22:29:27.577610+0000","last_clean":"2024-04-05T22:29:27.577610+0000","last_became_active":"2024-04-05T22:29:27.574693+0000","last_became_peered":"2024-04-05T22:29:27.574693+0000","last_unstale":"2024-04-05T22:29:27.577610+0000","last_undegraded":"2024-04-05T22:29:27.577610+0000","last_fullsized":"2024-04-05T22:29:27.577610+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612953+0000","last_change":"2024-04-05T22:29:27.611130+0000","last_active":"2024-04-05T22:29:27.612953+0000","last_peered":"2024-04-05T22:29:27.612953+0000","last_clean":"2024-04-05T22:29:27.612953+0000","last_became_active":"2024-04-05T22:29:27.610809+0000","last_became_peered":"2024-04-05T22:29:27.610809+0000","last_unstale":"2024-04-05T22:29:27.612953+0000","last_undegraded":"2024-04-05T22:29:27.612953+0000","last_fullsized":"2024-04-05T22:29:27.612953+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.597766+0000","last_change":"2024-04-05T22:29:27.597089+0000","last_active":"2024-04-05T22:29:27.597766+0000","last_peered":"2024-04-05T22:29:27.597766+0000","last_clean":"2024-04-05T22:29:27.597766+0000","last_became_active":"2024-04-05T22:29:27.596762+0000","last_became_peered":"2024-04-05T22:29:27.596762+0000","last_unstale":"2024-04-05T22:29:27.597766+0000","last_undegraded":"2024-04-05T22:29:27.597766+0000","last_fullsized":"2024-04-05T22:29:27.597766+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607370+0000","last_change":"2024-04-05T22:29:27.604313+0000","last_active":"2024-04-05T22:29:27.607370+0000","last_peered":"2024-04-05T22:29:27.607370+0000","last_clean":"2024-04-05T22:29:27.607370+0000","last_became_active":"2024-04-05T22:29:27.602775+0000","last_became_peered":"2024-04-05T22:29:27.602775+0000","last_unstale":"2024-04-05T22:29:27.607370+0000","last_undegraded":"2024-04-05T22:29:27.607370+0000","last_fullsized":"2024-04-05T22:29:27.607370+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609822+0000","last_change":"2024-04-05T22:29:27.603249+0000","last_active":"2024-04-05T22:29:27.609822+0000","last_peered":"2024-04-05T22:29:27.609822+0000","last_clean":"2024-04-05T22:29:27.609822+0000","last_became_active":"2024-04-05T22:29:27.602935+0000","last_became_peered":"2024-04-05T22:29:27.602935+0000","last_unstale":"2024-04-05T22:29:27.609822+0000","last_undegraded":"2024-04-05T22:29:27.609822+0000","last_fullsized":"2024-04-05T22:29:27.609822+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606150+0000","last_change":"2024-04-05T22:29:27.601511+0000","last_active":"2024-04-05T22:29:27.606150+0000","last_peered":"2024-04-05T22:29:27.606150+0000","last_clean":"2024-04-05T22:29:27.606150+0000","last_became_active":"2024-04-05T22:29:27.601160+0000","last_became_peered":"2024-04-05T22:29:27.601160+0000","last_unstale":"2024-04-05T22:29:27.606150+0000","last_undegraded":"2024-04-05T22:29:27.606150+0000","last_fullsized":"2024-04-05T22:29:27.606150+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582328+0000","last_change":"2024-04-05T22:29:27.578913+0000","last_active":"2024-04-05T22:29:27.582328+0000","last_peered":"2024-04-05T22:29:27.582328+0000","last_clean":"2024-04-05T22:29:27.582328+0000","last_became_active":"2024-04-05T22:29:27.578418+0000","last_became_peered":"2024-04-05T22:29:27.578418+0000","last_unstale":"2024-04-05T22:29:27.582328+0000","last_undegraded":"2024-04-05T22:29:27.582328+0000","last_fullsized":"2024-04-05T22:29:27.582328+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.586696+0000","last_change":"2024-04-05T22:29:27.584911+0000","last_active":"2024-04-05T22:29:27.586696+0000","last_peered":"2024-04-05T22:29:27.586696+0000","last_clean":"2024-04-05T22:29:27.586696+0000","last_became_active":"2024-04-05T22:29:27.584589+0000","last_became_peered":"2024-04-05T22:29:27.584589+0000","last_unstale":"2024-04-05T22:29:27.586696+0000","last_undegraded":"2024-04-05T22:29:27.586696+0000","last_fullsized":"2024-04-05T22:29:27.586696+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.591583+0000","last_change":"2024-04-05T22:29:27.588186+0000","last_active":"2024-04-05T22:29:27.591583+0000","last_peered":"2024-04-05T22:29:27.591583+0000","last_clean":"2024-04-05T22:29:27.591583+0000","last_became_active":"2024-04-05T22:29:27.587695+0000","last_became_peered":"2024-04-05T22:29:27.587695+0000","last_unstale":"2024-04-05T22:29:27.591583+0000","last_undegraded":"2024-04-05T22:29:27.591583+0000","last_fullsized":"2024-04-05T22:29:27.591583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599139+0000","last_change":"2024-04-05T22:29:27.593565+0000","last_active":"2024-04-05T22:29:27.599139+0000","last_peered":"2024-04-05T22:29:27.599139+0000","last_clean":"2024-04-05T22:29:27.599139+0000","last_became_active":"2024-04-05T22:29:27.593152+0000","last_became_peered":"2024-04-05T22:29:27.593152+0000","last_unstale":"2024-04-05T22:29:27.599139+0000","last_undegraded":"2024-04-05T22:29:27.599139+0000","last_fullsized":"2024-04-05T22:29:27.599139+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584753+0000","last_change":"2024-04-05T22:29:27.578997+0000","last_active":"2024-04-05T22:29:27.584753+0000","last_peered":"2024-04-05T22:29:27.584753+0000","last_clean":"2024-04-05T22:29:27.584753+0000","last_became_active":"2024-04-05T22:29:27.578672+0000","last_became_peered":"2024-04-05T22:29:27.578672+0000","last_unstale":"2024-04-05T22:29:27.584753+0000","last_undegraded":"2024-04-05T22:29:27.584753+0000","last_fullsized":"2024-04-05T22:29:27.584753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582743+0000","last_change":"2024-04-05T22:29:27.578944+0000","last_active":"2024-04-05T22:29:27.582743+0000","last_peered":"2024-04-05T22:29:27.582743+0000","last_clean":"2024-04-05T22:29:27.582743+0000","last_became_active":"2024-04-05T22:29:27.578714+0000","last_became_peered":"2024-04-05T22:29:27.578714+0000","last_unstale":"2024-04-05T22:29:27.582743+0000","last_undegraded":"2024-04-05T22:29:27.582743+0000","last_fullsized":"2024-04-05T22:29:27.582743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601222+0000","last_change":"2024-04-05T22:29:27.595591+0000","last_active":"2024-04-05T22:29:27.601222+0000","last_peered":"2024-04-05T22:29:27.601222+0000","last_clean":"2024-04-05T22:29:27.601222+0000","last_became_active":"2024-04-05T22:29:27.595293+0000","last_became_peered":"2024-04-05T22:29:27.595293+0000","last_unstale":"2024-04-05T22:29:27.601222+0000","last_undegraded":"2024-04-05T22:29:27.601222+0000","last_fullsized":"2024-04-05T22:29:27.601222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601374+0000","last_change":"2024-04-05T22:29:27.594918+0000","last_active":"2024-04-05T22:29:27.601374+0000","last_peered":"2024-04-05T22:29:27.601374+0000","last_clean":"2024-04-05T22:29:27.601374+0000","last_became_active":"2024-04-05T22:29:27.592311+0000","last_became_peered":"2024-04-05T22:29:27.592311+0000","last_unstale":"2024-04-05T22:29:27.601374+0000","last_undegraded":"2024-04-05T22:29:27.601374+0000","last_fullsized":"2024-04-05T22:29:27.601374+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.598333+0000","last_change":"2024-04-05T22:29:27.592484+0000","last_active":"2024-04-05T22:29:27.598333+0000","last_peered":"2024-04-05T22:29:27.598333+0000","last_clean":"2024-04-05T22:29:27.598333+0000","last_became_active":"2024-04-05T22:29:27.591147+0000","last_became_peered":"2024-04-05T22:29:27.591147+0000","last_unstale":"2024-04-05T22:29:27.598333+0000","last_undegraded":"2024-04-05T22:29:27.598333+0000","last_fullsized":"2024-04-05T22:29:27.598333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602870+0000","last_change":"2024-04-05T22:29:27.594629+0000","last_active":"2024-04-05T22:29:27.602870+0000","last_peered":"2024-04-05T22:29:27.602870+0000","last_clean":"2024-04-05T22:29:27.602870+0000","last_became_active":"2024-04-05T22:29:27.594355+0000","last_became_peered":"2024-04-05T22:29:27.594355+0000","last_unstale":"2024-04-05T22:29:27.602870+0000","last_undegraded":"2024-04-05T22:29:27.602870+0000","last_fullsized":"2024-04-05T22:29:27.602870+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.587211+0000","last_change":"2024-04-05T22:29:27.585562+0000","last_active":"2024-04-05T22:29:27.587211+0000","last_peered":"2024-04-05T22:29:27.587211+0000","last_clean":"2024-04-05T22:29:27.587211+0000","last_became_active":"2024-04-05T22:29:27.585305+0000","last_became_peered":"2024-04-05T22:29:27.585305+0000","last_unstale":"2024-04-05T22:29:27.587211+0000","last_undegraded":"2024-04-05T22:29:27.587211+0000","last_fullsized":"2024-04-05T22:29:27.587211+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.585049+0000","last_change":"2024-04-05T22:29:27.581233+0000","last_active":"2024-04-05T22:29:27.585049+0000","last_peered":"2024-04-05T22:29:27.585049+0000","last_clean":"2024-04-05T22:29:27.585049+0000","last_became_active":"2024-04-05T22:29:27.580350+0000","last_became_peered":"2024-04-05T22:29:27.580350+0000","last_unstale":"2024-04-05T22:29:27.585049+0000","last_undegraded":"2024-04-05T22:29:27.585049+0000","last_fullsized":"2024-04-05T22:29:27.585049+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612146+0000","last_change":"2024-04-05T22:29:27.611763+0000","last_active":"2024-04-05T22:29:27.612146+0000","last_peered":"2024-04-05T22:29:27.612146+0000","last_clean":"2024-04-05T22:29:27.612146+0000","last_became_active":"2024-04-05T22:29:27.611516+0000","last_became_peered":"2024-04-05T22:29:27.611516+0000","last_unstale":"2024-04-05T22:29:27.612146+0000","last_undegraded":"2024-04-05T22:29:27.612146+0000","last_fullsized":"2024-04-05T22:29:27.612146+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603620+0000","last_change":"2024-04-05T22:29:27.601645+0000","last_active":"2024-04-05T22:29:27.603620+0000","last_peered":"2024-04-05T22:29:27.603620+0000","last_clean":"2024-04-05T22:29:27.603620+0000","last_became_active":"2024-04-05T22:29:27.601266+0000","last_became_peered":"2024-04-05T22:29:27.601266+0000","last_unstale":"2024-04-05T22:29:27.603620+0000","last_undegraded":"2024-04-05T22:29:27.603620+0000","last_fullsized":"2024-04-05T22:29:27.603620+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607083+0000","last_change":"2024-04-05T22:29:27.605434+0000","last_active":"2024-04-05T22:29:27.607083+0000","last_peered":"2024-04-05T22:29:27.607083+0000","last_clean":"2024-04-05T22:29:27.607083+0000","last_became_active":"2024-04-05T22:29:27.605032+0000","last_became_peered":"2024-04-05T22:29:27.605032+0000","last_unstale":"2024-04-05T22:29:27.607083+0000","last_undegraded":"2024-04-05T22:29:27.607083+0000","last_fullsized":"2024-04-05T22:29:27.607083+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595831+0000","last_change":"2024-04-05T22:29:27.588374+0000","last_active":"2024-04-05T22:29:27.595831+0000","last_peered":"2024-04-05T22:29:27.595831+0000","last_clean":"2024-04-05T22:29:27.595831+0000","last_became_active":"2024-04-05T22:29:27.588072+0000","last_became_peered":"2024-04-05T22:29:27.588072+0000","last_unstale":"2024-04-05T22:29:27.595831+0000","last_undegraded":"2024-04-05T22:29:27.595831+0000","last_fullsized":"2024-04-05T22:29:27.595831+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607320+0000","last_change":"2024-04-05T22:29:27.605960+0000","last_active":"2024-04-05T22:29:27.607320+0000","last_peered":"2024-04-05T22:29:27.607320+0000","last_clean":"2024-04-05T22:29:27.607320+0000","last_became_active":"2024-04-05T22:29:27.605718+0000","last_became_peered":"2024-04-05T22:29:27.605718+0000","last_unstale":"2024-04-05T22:29:27.607320+0000","last_undegraded":"2024-04-05T22:29:27.607320+0000","last_fullsized":"2024-04-05T22:29:27.607320+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601795+0000","last_change":"2024-04-05T22:29:27.600750+0000","last_active":"2024-04-05T22:29:27.601795+0000","last_peered":"2024-04-05T22:29:27.601795+0000","last_clean":"2024-04-05T22:29:27.601795+0000","last_became_active":"2024-04-05T22:29:27.600487+0000","last_became_peered":"2024-04-05T22:29:27.600487+0000","last_unstale":"2024-04-05T22:29:27.601795+0000","last_undegraded":"2024-04-05T22:29:27.601795+0000","last_fullsized":"2024-04-05T22:29:27.601795+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602970+0000","last_change":"2024-04-05T22:29:27.596691+0000","last_active":"2024-04-05T22:29:27.602970+0000","last_peered":"2024-04-05T22:29:27.602970+0000","last_clean":"2024-04-05T22:29:27.602970+0000","last_became_active":"2024-04-05T22:29:27.596339+0000","last_became_peered":"2024-04-05T22:29:27.596339+0000","last_unstale":"2024-04-05T22:29:27.602970+0000","last_undegraded":"2024-04-05T22:29:27.602970+0000","last_fullsized":"2024-04-05T22:29:27.602970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599357+0000","last_change":"2024-04-05T22:29:27.113665+0000","last_active":"2024-04-05T22:29:27.599357+0000","last_peered":"2024-04-05T22:29:27.599357+0000","last_clean":"2024-04-05T22:29:27.599357+0000","last_became_active":"2024-04-05T22:29:27.113368+0000","last_became_peered":"2024-04-05T22:29:27.113368+0000","last_unstale":"2024-04-05T22:29:27.599357+0000","last_undegraded":"2024-04-05T22:29:27.599357+0000","last_fullsized":"2024-04-05T22:29:27.599357+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:29.247549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.586396+0000","last_change":"2024-04-05T22:29:27.584808+0000","last_active":"2024-04-05T22:29:27.586396+0000","last_peered":"2024-04-05T22:29:27.586396+0000","last_clean":"2024-04-05T22:29:27.586396+0000","last_became_active":"2024-04-05T22:29:27.584301+0000","last_became_peered":"2024-04-05T22:29:27.584301+0000","last_unstale":"2024-04-05T22:29:27.586396+0000","last_undegraded":"2024-04-05T22:29:27.586396+0000","last_fullsized":"2024-04-05T22:29:27.586396+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.583065+0000","last_change":"2024-04-05T22:29:27.115636+0000","last_active":"2024-04-05T22:29:27.583065+0000","last_peered":"2024-04-05T22:29:27.583065+0000","last_clean":"2024-04-05T22:29:27.583065+0000","last_became_active":"2024-04-05T22:29:27.115170+0000","last_became_peered":"2024-04-05T22:29:27.115170+0000","last_unstale":"2024-04-05T22:29:27.583065+0000","last_undegraded":"2024-04-05T22:29:27.583065+0000","last_fullsized":"2024-04-05T22:29:27.583065+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:49.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607075+0000","last_change":"2024-04-05T22:29:27.604863+0000","last_active":"2024-04-05T22:29:27.607075+0000","last_peered":"2024-04-05T22:29:27.607075+0000","last_clean":"2024-04-05T22:29:27.607075+0000","last_became_active":"2024-04-05T22:29:27.604550+0000","last_became_peered":"2024-04-05T22:29:27.604550+0000","last_unstale":"2024-04-05T22:29:27.607075+0000","last_undegraded":"2024-04-05T22:29:27.607075+0000","last_fullsized":"2024-04-05T22:29:27.607075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600459+0000","last_change":"2024-04-05T22:29:27.106460+0000","last_active":"2024-04-05T22:29:27.600459+0000","last_peered":"2024-04-05T22:29:27.600459+0000","last_clean":"2024-04-05T22:29:27.600459+0000","last_became_active":"2024-04-05T22:29:27.106113+0000","last_became_peered":"2024-04-05T22:29:27.106113+0000","last_unstale":"2024-04-05T22:29:27.600459+0000","last_undegraded":"2024-04-05T22:29:27.600459+0000","last_fullsized":"2024-04-05T22:29:27.600459+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:47.380812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600371+0000","last_change":"2024-04-05T22:29:27.595378+0000","last_active":"2024-04-05T22:29:27.600371+0000","last_peered":"2024-04-05T22:29:27.600371+0000","last_clean":"2024-04-05T22:29:27.600371+0000","last_became_active":"2024-04-05T22:29:27.595083+0000","last_became_peered":"2024-04-05T22:29:27.595083+0000","last_unstale":"2024-04-05T22:29:27.600371+0000","last_undegraded":"2024-04-05T22:29:27.600371+0000","last_fullsized":"2024-04-05T22:29:27.600371+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608741+0000","last_change":"2024-04-05T22:29:27.098092+0000","last_active":"2024-04-05T22:29:27.608741+0000","last_peered":"2024-04-05T22:29:27.608741+0000","last_clean":"2024-04-05T22:29:27.608741+0000","last_became_active":"2024-04-05T22:29:27.097449+0000","last_became_peered":"2024-04-05T22:29:27.097449+0000","last_unstale":"2024-04-05T22:29:27.608741+0000","last_undegraded":"2024-04-05T22:29:27.608741+0000","last_fullsized":"2024-04-05T22:29:27.608741+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:40.984286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602125+0000","last_change":"2024-04-05T22:29:27.600242+0000","last_active":"2024-04-05T22:29:27.602125+0000","last_peered":"2024-04-05T22:29:27.602125+0000","last_clean":"2024-04-05T22:29:27.602125+0000","last_became_active":"2024-04-05T22:29:27.600010+0000","last_became_peered":"2024-04-05T22:29:27.600010+0000","last_unstale":"2024-04-05T22:29:27.602125+0000","last_undegraded":"2024-04-05T22:29:27.602125+0000","last_fullsized":"2024-04-05T22:29:27.602125+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.593666+0000","last_change":"2024-04-05T22:29:27.106850+0000","last_active":"2024-04-05T22:29:27.593666+0000","last_peered":"2024-04-05T22:29:27.593666+0000","last_clean":"2024-04-05T22:29:27.593666+0000","last_became_active":"2024-04-05T22:29:27.106525+0000","last_became_peered":"2024-04-05T22:29:27.106525+0000","last_unstale":"2024-04-05T22:29:27.593666+0000","last_undegraded":"2024-04-05T22:29:27.593666+0000","last_fullsized":"2024-04-05T22:29:27.593666+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:30.113756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608020+0000","last_change":"2024-04-05T22:29:27.605931+0000","last_active":"2024-04-05T22:29:27.608020+0000","last_peered":"2024-04-05T22:29:27.608020+0000","last_clean":"2024-04-05T22:29:27.608020+0000","last_became_active":"2024-04-05T22:29:27.605340+0000","last_became_peered":"2024-04-05T22:29:27.605340+0000","last_unstale":"2024-04-05T22:29:27.608020+0000","last_undegraded":"2024-04-05T22:29:27.608020+0000","last_fullsized":"2024-04-05T22:29:27.608020+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602826+0000","last_change":"2024-04-05T22:29:27.113913+0000","last_active":"2024-04-05T22:29:27.602826+0000","last_peered":"2024-04-05T22:29:27.602826+0000","last_clean":"2024-04-05T22:29:27.602826+0000","last_became_active":"2024-04-05T22:29:27.112452+0000","last_became_peered":"2024-04-05T22:29:27.112452+0000","last_unstale":"2024-04-05T22:29:27.602826+0000","last_undegraded":"2024-04-05T22:29:27.602826+0000","last_fullsized":"2024-04-05T22:29:27.602826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:13:41.450507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602146+0000","last_change":"2024-04-05T22:29:27.600361+0000","last_active":"2024-04-05T22:29:27.602146+0000","last_peered":"2024-04-05T22:29:27.602146+0000","last_clean":"2024-04-05T22:29:27.602146+0000","last_became_active":"2024-04-05T22:29:27.600076+0000","last_became_peered":"2024-04-05T22:29:27.600076+0000","last_unstale":"2024-04-05T22:29:27.602146+0000","last_undegraded":"2024-04-05T22:29:27.602146+0000","last_fullsized":"2024-04-05T22:29:27.602146+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577538+0000","last_change":"2024-04-05T22:29:27.105830+0000","last_active":"2024-04-05T22:29:27.577538+0000","last_peered":"2024-04-05T22:29:27.577538+0000","last_clean":"2024-04-05T22:29:27.577538+0000","last_became_active":"2024-04-05T22:29:27.104458+0000","last_became_peered":"2024-04-05T22:29:27.104458+0000","last_unstale":"2024-04-05T22:29:27.577538+0000","last_undegraded":"2024-04-05T22:29:27.577538+0000","last_fullsized":"2024-04-05T22:29:27.577538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:34.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599890+0000","last_change":"2024-04-05T22:29:27.595916+0000","last_active":"2024-04-05T22:29:27.599890+0000","last_peered":"2024-04-05T22:29:27.599890+0000","last_clean":"2024-04-05T22:29:27.599890+0000","last_became_active":"2024-04-05T22:29:27.595159+0000","last_became_peered":"2024-04-05T22:29:27.595159+0000","last_unstale":"2024-04-05T22:29:27.599890+0000","last_undegraded":"2024-04-05T22:29:27.599890+0000","last_fullsized":"2024-04-05T22:29:27.599890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.588007+0000","last_change":"2024-04-05T22:29:27.107183+0000","last_active":"2024-04-05T22:29:27.588007+0000","last_peered":"2024-04-05T22:29:27.588007+0000","last_clean":"2024-04-05T22:29:27.588007+0000","last_became_active":"2024-04-05T22:29:27.106873+0000","last_became_peered":"2024-04-05T22:29:27.106873+0000","last_unstale":"2024-04-05T22:29:27.588007+0000","last_undegraded":"2024-04-05T22:29:27.588007+0000","last_fullsized":"2024-04-05T22:29:27.588007+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:31.435759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599553+0000","last_change":"2024-04-05T22:29:27.595541+0000","last_active":"2024-04-05T22:29:27.599553+0000","last_peered":"2024-04-05T22:29:27.599553+0000","last_clean":"2024-04-05T22:29:27.599553+0000","last_became_active":"2024-04-05T22:29:27.595188+0000","last_became_peered":"2024-04-05T22:29:27.595188+0000","last_unstale":"2024-04-05T22:29:27.599553+0000","last_undegraded":"2024-04-05T22:29:27.599553+0000","last_fullsized":"2024-04-05T22:29:27.599553+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.562143+0000","last_change":"2024-04-05T22:29:27.098118+0000","last_active":"2024-04-05T22:29:27.562143+0000","last_peered":"2024-04-05T22:29:27.562143+0000","last_clean":"2024-04-05T22:29:27.562143+0000","last_became_active":"2024-04-05T22:29:27.097772+0000","last_became_peered":"2024-04-05T22:29:27.097772+0000","last_unstale":"2024-04-05T22:29:27.562143+0000","last_undegraded":"2024-04-05T22:29:27.562143+0000","last_fullsized":"2024-04-05T22:29:27.562143+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:04.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.591563+0000","last_change":"2024-04-05T22:29:27.587994+0000","last_active":"2024-04-05T22:29:27.591563+0000","last_peered":"2024-04-05T22:29:27.591563+0000","last_clean":"2024-04-05T22:29:27.591563+0000","last_became_active":"2024-04-05T22:29:27.585579+0000","last_became_peered":"2024-04-05T22:29:27.585579+0000","last_unstale":"2024-04-05T22:29:27.591563+0000","last_undegraded":"2024-04-05T22:29:27.591563+0000","last_fullsized":"2024-04-05T22:29:27.591563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.574245+0000","last_change":"2024-04-05T22:29:27.117368+0000","last_active":"2024-04-05T22:29:27.574245+0000","last_peered":"2024-04-05T22:29:27.574245+0000","last_clean":"2024-04-05T22:29:27.574245+0000","last_became_active":"2024-04-05T22:29:27.117006+0000","last_became_peered":"2024-04-05T22:29:27.117006+0000","last_unstale":"2024-04-05T22:29:27.574245+0000","last_undegraded":"2024-04-05T22:29:27.574245+0000","last_fullsized":"2024-04-05T22:29:27.574245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:26.585990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.587240+0000","last_change":"2024-04-05T22:29:27.586030+0000","last_active":"2024-04-05T22:29:27.587240+0000","last_peered":"2024-04-05T22:29:27.587240+0000","last_clean":"2024-04-05T22:29:27.587240+0000","last_became_active":"2024-04-05T22:29:27.585778+0000","last_became_peered":"2024-04-05T22:29:27.585778+0000","last_unstale":"2024-04-05T22:29:27.587240+0000","last_undegraded":"2024-04-05T22:29:27.587240+0000","last_fullsized":"2024-04-05T22:29:27.587240+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"135'6","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.597489+0000","last_change":"2024-04-05T22:29:27.107727+0000","last_active":"2024-04-05T22:29:27.597489+0000","last_peered":"2024-04-05T22:29:27.597489+0000","last_clean":"2024-04-05T22:29:27.597489+0000","last_became_active":"2024-04-05T22:29:27.107409+0000","last_became_peered":"2024-04-05T22:29:27.107409+0000","last_unstale":"2024-04-05T22:29:27.597489+0000","last_undegraded":"2024-04-05T22:29:27.597489+0000","last_fullsized":"2024-04-05T22:29:27.597489+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:36:16.139153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607928+0000","last_change":"2024-04-05T22:29:27.605515+0000","last_active":"2024-04-05T22:29:27.607928+0000","last_peered":"2024-04-05T22:29:27.607928+0000","last_clean":"2024-04-05T22:29:27.607928+0000","last_became_active":"2024-04-05T22:29:27.605233+0000","last_became_peered":"2024-04-05T22:29:27.605233+0000","last_unstale":"2024-04-05T22:29:27.607928+0000","last_undegraded":"2024-04-05T22:29:27.607928+0000","last_fullsized":"2024-04-05T22:29:27.607928+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610980+0000","last_change":"2024-04-05T22:29:27.109281+0000","last_active":"2024-04-05T22:29:27.610980+0000","last_peered":"2024-04-05T22:29:27.610980+0000","last_clean":"2024-04-05T22:29:27.610980+0000","last_became_active":"2024-04-05T22:29:27.108992+0000","last_became_peered":"2024-04-05T22:29:27.108992+0000","last_unstale":"2024-04-05T22:29:27.610980+0000","last_undegraded":"2024-04-05T22:29:27.610980+0000","last_fullsized":"2024-04-05T22:29:27.610980+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:33:12.557477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577592+0000","last_change":"2024-04-05T22:29:27.574052+0000","last_active":"2024-04-05T22:29:27.577592+0000","last_peered":"2024-04-05T22:29:27.577592+0000","last_clean":"2024-04-05T22:29:27.577592+0000","last_became_active":"2024-04-05T22:29:27.573784+0000","last_became_peered":"2024-04-05T22:29:27.573784+0000","last_unstale":"2024-04-05T22:29:27.577592+0000","last_undegraded":"2024-04-05T22:29:27.577592+0000","last_fullsized":"2024-04-05T22:29:27.577592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601011+0000","last_change":"2024-04-05T22:29:27.109031+0000","last_active":"2024-04-05T22:29:27.601011+0000","last_peered":"2024-04-05T22:29:27.601011+0000","last_clean":"2024-04-05T22:29:27.601011+0000","last_became_active":"2024-04-05T22:29:27.108620+0000","last_became_peered":"2024-04-05T22:29:27.108620+0000","last_unstale":"2024-04-05T22:29:27.601011+0000","last_undegraded":"2024-04-05T22:29:27.601011+0000","last_fullsized":"2024-04-05T22:29:27.601011+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:46.523577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609746+0000","last_change":"2024-04-05T22:29:27.604046+0000","last_active":"2024-04-05T22:29:27.609746+0000","last_peered":"2024-04-05T22:29:27.609746+0000","last_clean":"2024-04-05T22:29:27.609746+0000","last_became_active":"2024-04-05T22:29:27.603778+0000","last_became_peered":"2024-04-05T22:29:27.603778+0000","last_unstale":"2024-04-05T22:29:27.609746+0000","last_undegraded":"2024-04-05T22:29:27.609746+0000","last_fullsized":"2024-04-05T22:29:27.609746+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.569223+0000","last_change":"2024-04-05T22:29:27.103901+0000","last_active":"2024-04-05T22:29:27.569223+0000","last_peered":"2024-04-05T22:29:27.569223+0000","last_clean":"2024-04-05T22:29:27.569223+0000","last_became_active":"2024-04-05T22:29:27.103615+0000","last_became_peered":"2024-04-05T22:29:27.103615+0000","last_unstale":"2024-04-05T22:29:27.569223+0000","last_undegraded":"2024-04-05T22:29:27.569223+0000","last_fullsized":"2024-04-05T22:29:27.569223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:46.499208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601581+0000","last_change":"2024-04-05T22:29:27.596527+0000","last_active":"2024-04-05T22:29:27.601581+0000","last_peered":"2024-04-05T22:29:27.601581+0000","last_clean":"2024-04-05T22:29:27.601581+0000","last_became_active":"2024-04-05T22:29:27.596245+0000","last_became_peered":"2024-04-05T22:29:27.596245+0000","last_unstale":"2024-04-05T22:29:27.601581+0000","last_undegraded":"2024-04-05T22:29:27.601581+0000","last_fullsized":"2024-04-05T22:29:27.601581+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"135'8","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596819+0000","last_change":"2024-04-05T22:29:27.104205+0000","last_active":"2024-04-05T22:29:27.596819+0000","last_peered":"2024-04-05T22:29:27.596819+0000","last_clean":"2024-04-05T22:29:27.596819+0000","last_became_active":"2024-04-05T22:29:27.102618+0000","last_became_peered":"2024-04-05T22:29:27.102618+0000","last_unstale":"2024-04-05T22:29:27.596819+0000","last_undegraded":"2024-04-05T22:29:27.596819+0000","last_fullsized":"2024-04-05T22:29:27.596819+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:05.414218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590709+0000","last_change":"2024-04-05T22:29:27.588741+0000","last_active":"2024-04-05T22:29:27.590709+0000","last_peered":"2024-04-05T22:29:27.590709+0000","last_clean":"2024-04-05T22:29:27.590709+0000","last_became_active":"2024-04-05T22:29:27.588448+0000","last_became_peered":"2024-04-05T22:29:27.588448+0000","last_unstale":"2024-04-05T22:29:27.590709+0000","last_undegraded":"2024-04-05T22:29:27.590709+0000","last_fullsized":"2024-04-05T22:29:27.590709+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600722+0000","last_change":"2024-04-05T22:29:27.113619+0000","last_active":"2024-04-05T22:29:27.600722+0000","last_peered":"2024-04-05T22:29:27.600722+0000","last_clean":"2024-04-05T22:29:27.600722+0000","last_became_active":"2024-04-05T22:29:27.113367+0000","last_became_peered":"2024-04-05T22:29:27.113367+0000","last_unstale":"2024-04-05T22:29:27.600722+0000","last_undegraded":"2024-04-05T22:29:27.600722+0000","last_fullsized":"2024-04-05T22:29:27.600722+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:26:24.419813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596410+0000","last_change":"2024-04-05T22:29:27.594065+0000","last_active":"2024-04-05T22:29:27.596410+0000","last_peered":"2024-04-05T22:29:27.596410+0000","last_clean":"2024-04-05T22:29:27.596410+0000","last_became_active":"2024-04-05T22:29:27.593786+0000","last_became_peered":"2024-04-05T22:29:27.593786+0000","last_unstale":"2024-04-05T22:29:27.596410+0000","last_undegraded":"2024-04-05T22:29:27.596410+0000","last_fullsized":"2024-04-05T22:29:27.596410+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584249+0000","last_change":"2024-04-05T22:29:27.103848+0000","last_active":"2024-04-05T22:29:27.584249+0000","last_peered":"2024-04-05T22:29:27.584249+0000","last_clean":"2024-04-05T22:29:27.584249+0000","last_became_active":"2024-04-05T22:29:27.103586+0000","last_became_peered":"2024-04-05T22:29:27.103586+0000","last_unstale":"2024-04-05T22:29:27.584249+0000","last_undegraded":"2024-04-05T22:29:27.584249+0000","last_fullsized":"2024-04-05T22:29:27.584249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:11.324558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612482+0000","last_change":"2024-04-05T22:29:27.611143+0000","last_active":"2024-04-05T22:29:27.612482+0000","last_peered":"2024-04-05T22:29:27.612482+0000","last_clean":"2024-04-05T22:29:27.612482+0000","last_became_active":"2024-04-05T22:29:27.610705+0000","last_became_peered":"2024-04-05T22:29:27.610705+0000","last_unstale":"2024-04-05T22:29:27.612482+0000","last_undegraded":"2024-04-05T22:29:27.612482+0000","last_fullsized":"2024-04-05T22:29:27.612482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590513+0000","last_change":"2024-04-05T22:29:27.106062+0000","last_active":"2024-04-05T22:29:27.590513+0000","last_peered":"2024-04-05T22:29:27.590513+0000","last_clean":"2024-04-05T22:29:27.590513+0000","last_became_active":"2024-04-05T22:29:27.105748+0000","last_became_peered":"2024-04-05T22:29:27.105748+0000","last_unstale":"2024-04-05T22:29:27.590513+0000","last_undegraded":"2024-04-05T22:29:27.590513+0000","last_fullsized":"2024-04-05T22:29:27.590513+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:48.841874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610084+0000","last_change":"2024-04-05T22:29:27.609052+0000","last_active":"2024-04-05T22:29:27.610084+0000","last_peered":"2024-04-05T22:29:27.610084+0000","last_clean":"2024-04-05T22:29:27.610084+0000","last_became_active":"2024-04-05T22:29:27.608791+0000","last_became_peered":"2024-04-05T22:29:27.608791+0000","last_unstale":"2024-04-05T22:29:27.610084+0000","last_undegraded":"2024-04-05T22:29:27.610084+0000","last_fullsized":"2024-04-05T22:29:27.610084+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.562374+0000","last_change":"2024-04-05T22:29:27.102360+0000","last_active":"2024-04-05T22:29:27.562374+0000","last_peered":"2024-04-05T22:29:27.562374+0000","last_clean":"2024-04-05T22:29:27.562374+0000","last_became_active":"2024-04-05T22:29:27.102051+0000","last_became_peered":"2024-04-05T22:29:27.102051+0000","last_unstale":"2024-04-05T22:29:27.562374+0000","last_undegraded":"2024-04-05T22:29:27.562374+0000","last_fullsized":"2024-04-05T22:29:27.562374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:47.073194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595812+0000","last_change":"2024-04-05T22:29:27.587480+0000","last_active":"2024-04-05T22:29:27.595812+0000","last_peered":"2024-04-05T22:29:27.595812+0000","last_clean":"2024-04-05T22:29:27.595812+0000","last_became_active":"2024-04-05T22:29:27.586979+0000","last_became_peered":"2024-04-05T22:29:27.586979+0000","last_unstale":"2024-04-05T22:29:27.595812+0000","last_undegraded":"2024-04-05T22:29:27.595812+0000","last_fullsized":"2024-04-05T22:29:27.595812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.594553+0000","last_change":"2024-04-05T22:29:27.111825+0000","last_active":"2024-04-05T22:29:27.594553+0000","last_peered":"2024-04-05T22:29:27.594553+0000","last_clean":"2024-04-05T22:29:27.594553+0000","last_became_active":"2024-04-05T22:29:27.111542+0000","last_became_peered":"2024-04-05T22:29:27.111542+0000","last_unstale":"2024-04-05T22:29:27.594553+0000","last_undegraded":"2024-04-05T22:29:27.594553+0000","last_fullsized":"2024-04-05T22:29:27.594553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:22.051262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599579+0000","last_change":"2024-04-05T22:29:27.599050+0000","last_active":"2024-04-05T22:29:27.599579+0000","last_peered":"2024-04-05T22:29:27.599579+0000","last_clean":"2024-04-05T22:29:27.599579+0000","last_became_active":"2024-04-05T22:29:27.598726+0000","last_became_peered":"2024-04-05T22:29:27.598726+0000","last_unstale":"2024-04-05T22:29:27.599579+0000","last_undegraded":"2024-04-05T22:29:27.599579+0000","last_fullsized":"2024-04-05T22:29:27.599579+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567974+0000","last_change":"2024-04-05T22:29:27.111984+0000","last_active":"2024-04-05T22:29:27.567974+0000","last_peered":"2024-04-05T22:29:27.567974+0000","last_clean":"2024-04-05T22:29:27.567974+0000","last_became_active":"2024-04-05T22:29:27.111570+0000","last_became_peered":"2024-04-05T22:29:27.111570+0000","last_unstale":"2024-04-05T22:29:27.567974+0000","last_undegraded":"2024-04-05T22:29:27.567974+0000","last_fullsized":"2024-04-05T22:29:27.567974+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:02:55.012896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596428+0000","last_change":"2024-04-05T22:29:27.587732+0000","last_active":"2024-04-05T22:29:27.596428+0000","last_peered":"2024-04-05T22:29:27.596428+0000","last_clean":"2024-04-05T22:29:27.596428+0000","last_became_active":"2024-04-05T22:29:27.587061+0000","last_became_peered":"2024-04-05T22:29:27.587061+0000","last_unstale":"2024-04-05T22:29:27.596428+0000","last_undegraded":"2024-04-05T22:29:27.596428+0000","last_fullsized":"2024-04-05T22:29:27.596428+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.566714+0000","last_change":"2024-04-05T22:29:27.117354+0000","last_active":"2024-04-05T22:29:27.566714+0000","last_peered":"2024-04-05T22:29:27.566714+0000","last_clean":"2024-04-05T22:29:27.566714+0000","last_became_active":"2024-04-05T22:29:27.116968+0000","last_became_peered":"2024-04-05T22:29:27.116968+0000","last_unstale":"2024-04-05T22:29:27.566714+0000","last_undegraded":"2024-04-05T22:29:27.566714+0000","last_fullsized":"2024-04-05T22:29:27.566714+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:32:03.796707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595743+0000","last_change":"2024-04-05T22:29:27.592556+0000","last_active":"2024-04-05T22:29:27.595743+0000","last_peered":"2024-04-05T22:29:27.595743+0000","last_clean":"2024-04-05T22:29:27.595743+0000","last_became_active":"2024-04-05T22:29:27.592061+0000","last_became_peered":"2024-04-05T22:29:27.592061+0000","last_unstale":"2024-04-05T22:29:27.595743+0000","last_undegraded":"2024-04-05T22:29:27.595743+0000","last_fullsized":"2024-04-05T22:29:27.595743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.604157+0000","last_change":"2024-04-05T22:29:27.104579+0000","last_active":"2024-04-05T22:29:27.604157+0000","last_peered":"2024-04-05T22:29:27.604157+0000","last_clean":"2024-04-05T22:29:27.604157+0000","last_became_active":"2024-04-05T22:29:27.104293+0000","last_became_peered":"2024-04-05T22:29:27.104293+0000","last_unstale":"2024-04-05T22:29:27.604157+0000","last_undegraded":"2024-04-05T22:29:27.604157+0000","last_fullsized":"2024-04-05T22:29:27.604157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:13:51.613749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610310+0000","last_change":"2024-04-05T22:29:27.606580+0000","last_active":"2024-04-05T22:29:27.610310+0000","last_peered":"2024-04-05T22:29:27.610310+0000","last_clean":"2024-04-05T22:29:27.610310+0000","last_became_active":"2024-04-05T22:29:27.606239+0000","last_became_peered":"2024-04-05T22:29:27.606239+0000","last_unstale":"2024-04-05T22:29:27.610310+0000","last_undegraded":"2024-04-05T22:29:27.610310+0000","last_fullsized":"2024-04-05T22:29:27.610310+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600620+0000","last_change":"2024-04-05T22:29:27.115301+0000","last_active":"2024-04-05T22:29:27.600620+0000","last_peered":"2024-04-05T22:29:27.600620+0000","last_clean":"2024-04-05T22:29:27.600620+0000","last_became_active":"2024-04-05T22:29:27.114776+0000","last_became_peered":"2024-04-05T22:29:27.114776+0000","last_unstale":"2024-04-05T22:29:27.600620+0000","last_undegraded":"2024-04-05T22:29:27.600620+0000","last_fullsized":"2024-04-05T22:29:27.600620+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:53.380117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607801+0000","last_change":"2024-04-05T22:29:27.605387+0000","last_active":"2024-04-05T22:29:27.607801+0000","last_peered":"2024-04-05T22:29:27.607801+0000","last_clean":"2024-04-05T22:29:27.607801+0000","last_became_active":"2024-04-05T22:29:27.605136+0000","last_became_peered":"2024-04-05T22:29:27.605136+0000","last_unstale":"2024-04-05T22:29:27.607801+0000","last_undegraded":"2024-04-05T22:29:27.607801+0000","last_fullsized":"2024-04-05T22:29:27.607801+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-05T22:29:28.567987+0000","last_change":"2024-04-05T22:29:27.105399+0000","last_active":"2024-04-05T22:29:28.567987+0000","last_peered":"2024-04-05T22:29:28.567987+0000","last_clean":"2024-04-05T22:29:28.567987+0000","last_became_active":"2024-04-05T22:29:27.105109+0000","last_became_peered":"2024-04-05T22:29:27.105109+0000","last_unstale":"2024-04-05T22:29:28.567987+0000","last_undegraded":"2024-04-05T22:29:28.567987+0000","last_fullsized":"2024-04-05T22:29:28.567987+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:28.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606646+0000","last_change":"2024-04-05T22:29:27.605029+0000","last_active":"2024-04-05T22:29:27.606646+0000","last_peered":"2024-04-05T22:29:27.606646+0000","last_clean":"2024-04-05T22:29:27.606646+0000","last_became_active":"2024-04-05T22:29:27.604719+0000","last_became_peered":"2024-04-05T22:29:27.604719+0000","last_unstale":"2024-04-05T22:29:27.606646+0000","last_undegraded":"2024-04-05T22:29:27.606646+0000","last_fullsized":"2024-04-05T22:29:27.606646+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.571145+0000","last_change":"2024-04-05T22:29:27.094375+0000","last_active":"2024-04-05T22:29:27.571145+0000","last_peered":"2024-04-05T22:29:27.571145+0000","last_clean":"2024-04-05T22:29:27.571145+0000","last_became_active":"2024-04-05T22:29:27.094043+0000","last_became_peered":"2024-04-05T22:29:27.094043+0000","last_unstale":"2024-04-05T22:29:27.571145+0000","last_undegraded":"2024-04-05T22:29:27.571145+0000","last_fullsized":"2024-04-05T22:29:27.571145+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:36.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603753+0000","last_change":"2024-04-05T22:29:27.601202+0000","last_active":"2024-04-05T22:29:27.603753+0000","last_peered":"2024-04-05T22:29:27.603753+0000","last_clean":"2024-04-05T22:29:27.603753+0000","last_became_active":"2024-04-05T22:29:27.600373+0000","last_became_peered":"2024-04-05T22:29:27.600373+0000","last_unstale":"2024-04-05T22:29:27.603753+0000","last_undegraded":"2024-04-05T22:29:27.603753+0000","last_fullsized":"2024-04-05T22:29:27.603753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.570652+0000","last_change":"2024-04-05T22:29:27.092419+0000","last_active":"2024-04-05T22:29:27.570652+0000","last_peered":"2024-04-05T22:29:27.570652+0000","last_clean":"2024-04-05T22:29:27.570652+0000","last_became_active":"2024-04-05T22:29:27.091178+0000","last_became_peered":"2024-04-05T22:29:27.091178+0000","last_unstale":"2024-04-05T22:29:27.570652+0000","last_undegraded":"2024-04-05T22:29:27.570652+0000","last_fullsized":"2024-04-05T22:29:27.570652+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:31.397216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596367+0000","last_change":"2024-04-05T22:29:27.593941+0000","last_active":"2024-04-05T22:29:27.596367+0000","last_peered":"2024-04-05T22:29:27.596367+0000","last_clean":"2024-04-05T22:29:27.596367+0000","last_became_active":"2024-04-05T22:29:27.593510+0000","last_became_peered":"2024-04-05T22:29:27.593510+0000","last_unstale":"2024-04-05T22:29:27.596367+0000","last_undegraded":"2024-04-05T22:29:27.596367+0000","last_fullsized":"2024-04-05T22:29:27.596367+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.572196+0000","last_change":"2024-04-05T22:29:27.108317+0000","last_active":"2024-04-05T22:29:27.572196+0000","last_peered":"2024-04-05T22:29:27.572196+0000","last_clean":"2024-04-05T22:29:27.572196+0000","last_became_active":"2024-04-05T22:29:27.107888+0000","last_became_peered":"2024-04-05T22:29:27.107888+0000","last_unstale":"2024-04-05T22:29:27.572196+0000","last_undegraded":"2024-04-05T22:29:27.572196+0000","last_fullsized":"2024-04-05T22:29:27.572196+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:42.291460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610341+0000","last_change":"2024-04-05T22:29:27.607201+0000","last_active":"2024-04-05T22:29:27.610341+0000","last_peered":"2024-04-05T22:29:27.610341+0000","last_clean":"2024-04-05T22:29:27.610341+0000","last_became_active":"2024-04-05T22:29:27.606880+0000","last_became_peered":"2024-04-05T22:29:27.606880+0000","last_unstale":"2024-04-05T22:29:27.610341+0000","last_undegraded":"2024-04-05T22:29:27.610341+0000","last_fullsized":"2024-04-05T22:29:27.610341+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567876+0000","last_change":"2024-04-05T22:29:27.114605+0000","last_active":"2024-04-05T22:29:27.567876+0000","last_peered":"2024-04-05T22:29:27.567876+0000","last_clean":"2024-04-05T22:29:27.567876+0000","last_became_active":"2024-04-05T22:29:27.114311+0000","last_became_peered":"2024-04-05T22:29:27.114311+0000","last_unstale":"2024-04-05T22:29:27.567876+0000","last_undegraded":"2024-04-05T22:29:27.567876+0000","last_fullsized":"2024-04-05T22:29:27.567876+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:06.440778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590111+0000","last_change":"2024-04-05T22:29:27.582598+0000","last_active":"2024-04-05T22:29:27.590111+0000","last_peered":"2024-04-05T22:29:27.590111+0000","last_clean":"2024-04-05T22:29:27.590111+0000","last_became_active":"2024-04-05T22:29:27.579892+0000","last_became_peered":"2024-04-05T22:29:27.579892+0000","last_unstale":"2024-04-05T22:29:27.590111+0000","last_undegraded":"2024-04-05T22:29:27.590111+0000","last_fullsized":"2024-04-05T22:29:27.590111+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600538+0000","last_change":"2024-04-05T22:29:27.109061+0000","last_active":"2024-04-05T22:29:27.600538+0000","last_peered":"2024-04-05T22:29:27.600538+0000","last_clean":"2024-04-05T22:29:27.600538+0000","last_became_active":"2024-04-05T22:29:27.108742+0000","last_became_peered":"2024-04-05T22:29:27.108742+0000","last_unstale":"2024-04-05T22:29:27.600538+0000","last_undegraded":"2024-04-05T22:29:27.600538+0000","last_fullsized":"2024-04-05T22:29:27.600538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:26:35.912596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599871+0000","last_change":"2024-04-05T22:29:27.595437+0000","last_active":"2024-04-05T22:29:27.599871+0000","last_peered":"2024-04-05T22:29:27.599871+0000","last_clean":"2024-04-05T22:29:27.599871+0000","last_became_active":"2024-04-05T22:29:27.594151+0000","last_became_peered":"2024-04-05T22:29:27.594151+0000","last_unstale":"2024-04-05T22:29:27.599871+0000","last_undegraded":"2024-04-05T22:29:27.599871+0000","last_fullsized":"2024-04-05T22:29:27.599871+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.561663+0000","last_change":"2024-04-05T22:29:27.115510+0000","last_active":"2024-04-05T22:29:27.561663+0000","last_peered":"2024-04-05T22:29:27.561663+0000","last_clean":"2024-04-05T22:29:27.561663+0000","last_became_active":"2024-04-05T22:29:27.115162+0000","last_became_peered":"2024-04-05T22:29:27.115162+0000","last_unstale":"2024-04-05T22:29:27.561663+0000","last_undegraded":"2024-04-05T22:29:27.561663+0000","last_fullsized":"2024-04-05T22:29:27.561663+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:44.332750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606977+0000","last_change":"2024-04-05T22:29:27.606046+0000","last_active":"2024-04-05T22:29:27.606977+0000","last_peered":"2024-04-05T22:29:27.606977+0000","last_clean":"2024-04-05T22:29:27.606977+0000","last_became_active":"2024-04-05T22:29:27.605661+0000","last_became_peered":"2024-04-05T22:29:27.605661+0000","last_unstale":"2024-04-05T22:29:27.606977+0000","last_undegraded":"2024-04-05T22:29:27.606977+0000","last_fullsized":"2024-04-05T22:29:27.606977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610812+0000","last_change":"2024-04-05T22:29:27.107244+0000","last_active":"2024-04-05T22:29:27.610812+0000","last_peered":"2024-04-05T22:29:27.610812+0000","last_clean":"2024-04-05T22:29:27.610812+0000","last_became_active":"2024-04-05T22:29:27.106979+0000","last_became_peered":"2024-04-05T22:29:27.106979+0000","last_unstale":"2024-04-05T22:29:27.610812+0000","last_undegraded":"2024-04-05T22:29:27.610812+0000","last_fullsized":"2024-04-05T22:29:27.610812+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:28:57.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":301,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567658+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:29:27.567658+0000","last_peered":"2024-04-05T22:29:27.567658+0000","last_clean":"2024-04-05T22:29:27.567658+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:29:27.567658+0000","last_undegraded":"2024-04-05T22:29:27.567658+0000","last_fullsized":"2024-04-05T22:29:27.567658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542248,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2301952,"data_stored":1341744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.097},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.26}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.88},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.492},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.175}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.583}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.24},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.3899999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542249,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1712128,"data_stored":750870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.006}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.87},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":0.96499999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.333},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.2170000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.2370000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.284}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1703936,"data_stored":750746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.1799999999999999,"15min":1.0580000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":7.2329999999999997,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.458,"5min":1.2,"15min":1.1220000000000001},"min":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2330000000000001,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.2770000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.089},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.4039999999999999,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":17.381,"5min":17.381,"15min":17.381},"last":1.0600000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.4399999999999999,"15min":1.2529999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":24.52,"5min":24.52,"15min":24.52},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.3109999999999999,"15min":1.2729999999999999},"min":{"1min":0.99199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":7.5179999999999998,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.381},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.329,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.4000000000000004,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.149}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.429,"15min":1.321},"min":{"1min":0.96399999999999997,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":14.933,"5min":14.933,"15min":14.933},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.85499999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.413,"15min":1.355},"min":{"1min":1.0289999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.1959999999999997,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.24},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.4079999999999999,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":3.782,"5min":3.782,"15min":3.782},"last":1.716}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":1.5389999999999999,"15min":1.4790000000000001},"min":{"1min":1.2509999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.78,"5min":1.5580000000000001,"15min":1.6040000000000001},"min":{"1min":1.278,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":5.1429999999999998,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.607}]}]},{"osd":4,"up_from":14,"seq":60129542247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1699840,"data_stored":750724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1020000000000001,"15min":1.0289999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.59399999999999997},"max":{"1min":1.407,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0880000000000001},"min":{"1min":0.752,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5309999999999999,"5min":1.859,"15min":1.859},"last":1.288}]},{"osd":1,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1459999999999999,"15min":1.131},"min":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.728,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.51},{"interface":"front","average":{"1min":1.286,"5min":1.2150000000000001,"15min":1.181},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.645,"5min":2.577,"15min":2.577},"last":1.4239999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6100000000000001,"5min":1.6579999999999999,"15min":1.974},"last":1.476},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2490000000000001,"15min":1.2050000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.67400000000000004},"max":{"1min":1.7869999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.171}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.325,"15min":1.3620000000000001},"min":{"1min":0.82899999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7689999999999999,"5min":1.843,"15min":2.1840000000000002},"last":1.593},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4079999999999999,"15min":1.4039999999999999},"min":{"1min":1.018,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.8899999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.627}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3029999999999999,"15min":1.3},"min":{"1min":0.78800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1120000000000001,"5min":2.665,"15min":2.665},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4139999999999999,"15min":1.399},"min":{"1min":0.90200000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7490000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.3999999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4550000000000001,"15min":1.458},"min":{"1min":0.85099999999999998,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":2.0800000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.472,"15min":1.4870000000000001},"min":{"1min":0.93700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.3540000000000001,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.3500000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.5680000000000001,"15min":1.5449999999999999},"min":{"1min":1.0660000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.53},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.651,"15min":1.599},"min":{"1min":1.296,"5min":0.93500000000000005,"15min":0.72399999999999998},"max":{"1min":2.2530000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.4510000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1703936,"data_stored":750814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.83899999999999997},"min":{"1min":0.53600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.302,"5min":1.4910000000000001,"15min":1.744},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5369999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.91200000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4279999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.008,"5min":0.95899999999999996,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3720000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.0549999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.028,"15min":1.0209999999999999},"min":{"1min":0.59699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4510000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.288},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.069,"15min":1.0800000000000001},"min":{"1min":0.66700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.956,"5min":1.956,"15min":2.2549999999999999},"last":1.956}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1719999999999999,"15min":1.222},"min":{"1min":0.76300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5509999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.248,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7649999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.3779999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2809999999999999,"15min":1.343},"min":{"1min":0.84299999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.71,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.639},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3029999999999999,"15min":1.294},"min":{"1min":0.98199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.895,"5min":1.958,"15min":2.3239999999999998},"last":1.7470000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.79200000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.91400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7,"5min":2.298,"15min":2.298},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.341,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.899,"5min":1.9410000000000001,"15min":2.468},"last":1.899},{"interface":"front","average":{"1min":1.343,"5min":1.3620000000000001,"15min":1.4079999999999999},"min":{"1min":0.92100000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8160000000000001,"5min":2.508,"15min":2.798},"last":1.5980000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1716224,"data_stored":752358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.433}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.0509999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.077}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.0289999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1703936,"data_stored":751306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3680000000000001,"5min":1.379,"15min":1.6000000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86499999999999999,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1779999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.752},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97999999999999998,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4570000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.139}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.054,"15min":1.095},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.75},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0149999999999999,"15min":0.97599999999999998},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.4530000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.171,"15min":1.141},"min":{"1min":0.84199999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.716,"5min":1.8500000000000001,"15min":1.855},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.26,"5min":1.28,"15min":1.2809999999999999},"min":{"1min":0.94499999999999995,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.575,"5min":1.806,"15min":2.2789999999999999},"last":1.367}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.8200000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.004},{"interface":"front","average":{"1min":1.323,"5min":1.2649999999999999,"15min":1.2410000000000001},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.173},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.367,"15min":1.341},"min":{"1min":0.88900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.675,"5min":1.9059999999999999,"15min":1.917},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.357,"15min":1.3560000000000001},"min":{"1min":0.86899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.405,"15min":1.369},"min":{"1min":0.95899999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0259999999999998,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.208},{"interface":"front","average":{"1min":1.383,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.2829999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2310144,"data_stored":1342690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.4339999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.90900000000000003,"15min":0.88600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.8680000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.54,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95499999999999996,"15min":0.96399999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.6639999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.244,"5min":1.976,"15min":1.976},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0580000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.581,"5min":2.157,"15min":2.157},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1919999999999999,"15min":1.1719999999999999},"min":{"1min":0.80400000000000005,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.5429999999999999,"5min":1.931,"15min":1.931},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.298,"15min":1.3},"min":{"1min":0.75900000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7569999999999999,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.181}]},{"osd":5,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2450000000000001,"15min":1.2609999999999999},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0739999999999998,"5min":2.105,"15min":2.3879999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2130000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.7010000000000001,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.3740000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.7470000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.3640000000000001},"min":{"1min":0.90100000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8720000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.3080000000000001},"min":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.9810000000000001},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:29:37.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:29:37.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.266+0000 7fb4c2600640 1 Processor -- start 2024-04-05T22:29:37.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.266+0000 7fb4c2600640 1 -- start start 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.267+0000 7fb4c2600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4bc0cc170 0x7fb4bc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.267+0000 7fb4c2600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc06cf30 con 0x7fb4bc15e010 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.267+0000 7fb4c2600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0cc550 con 0x7fb4bc070c00 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.267+0000 7fb4c2600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0710b0 con 0x7fb4bc0cc170 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4bc0cc170 0x7fb4bc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bbe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb4bc15e010 0x7fb4bc06d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36508/0 (socket says 172.21.15.73:36508) 2024-04-05T22:29:37.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bbe00640 1 -- 172.21.15.73:0/2535305643 learned_addr learned my addr 172.21.15.73:0/2535305643 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4ba000640 1 -- 172.21.15.73:0/2535305643 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372666393 0 0) 0x7fb4bc06cf30 con 0x7fb4bc15e010 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4ba000640 1 -- 172.21.15.73:0/2535305643 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a0003520 con 0x7fb4bc15e010 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bb400640 1 -- 172.21.15.73:0/2535305643 >> v1:172.21.15.157:6789/0 conn(0x7fb4bc070c00 legacy=0x7fb4bc15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bb400640 1 -- 172.21.15.73:0/2535305643 >> v1:172.21.15.73:6789/0 conn(0x7fb4bc15e010 legacy=0x7fb4bc06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.268+0000 7fb4bb400640 1 -- 172.21.15.73:0/2535305643 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc06c600 con 0x7fb4bc0cc170 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.269+0000 7fb4bb400640 1 --2- 172.21.15.73:0/2535305643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc0706c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0097a0 tx=0x7fb4ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326a21bbe1f1ab5b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4ba000640 1 -- 172.21.15.73:0/2535305643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac031830 con 0x7fb4bc0cc170 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4ba000640 1 -- 172.21.15.73:0/2535305643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac031990 con 0x7fb4bc0cc170 2024-04-05T22:29:37.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4ba000640 1 -- 172.21.15.73:0/2535305643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac031d20 con 0x7fb4bc0cc170 2024-04-05T22:29:37.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4c2600640 1 -- 172.21.15.73:0/2535305643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc0cc170 msgr2=0x7fb4bc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4c2600640 1 --2- 172.21.15.73:0/2535305643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc0706c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0097a0 tx=0x7fb4ac02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4c2600640 1 -- 172.21.15.73:0/2535305643 shutdown_connections 2024-04-05T22:29:37.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4c2600640 1 --2- 172.21.15.73:0/2535305643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc0706c0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.270+0000 7fb4c2600640 1 -- 172.21.15.73:0/2535305643 >> 172.21.15.73:0/2535305643 conn(0x7fb4bc0b16d0 msgr2=0x7fb4bc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:37.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.271+0000 7fb4c2600640 1 -- 172.21.15.73:0/2535305643 shutdown_connections 2024-04-05T22:29:37.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.271+0000 7fb4c2600640 1 -- 172.21.15.73:0/2535305643 wait complete. 2024-04-05T22:29:37.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.272+0000 7fb4c2600640 1 Processor -- start 2024-04-05T22:29:37.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.272+0000 7fb4c2600640 1 -- start start 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 0x7fb4bc1474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0ae3f0 con 0x7fb4bc15e010 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0ad7e0 con 0x7fb4bc0cc170 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.273+0000 7fb4c2600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0adac0 con 0x7fb4bc070c00 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55316/0 (socket says 172.21.15.73:55316) 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 -- 172.21.15.73:0/4145163883 learned_addr learned my addr 172.21.15.73:0/4145163883 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 -- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 msgr2=0x7fb4bc1474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bb400640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 0x7fb4bc1474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4baa00640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 0x7fb4bc1474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 -- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc0cc170 msgr2=0x7fb4bc159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 -- 172.21.15.73:0/4145163883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac009400 con 0x7fb4bc15e010 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bb400640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 0x7fb4bc1474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.274+0000 7fb4bbe00640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400caa0 tx=0x7fb4b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.275+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400ac00 con 0x7fb4bc15e010 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.275+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400ad60 con 0x7fb4bc15e010 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.275+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc0adda0 con 0x7fb4bc15e010 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.275+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4012620 con 0x7fb4bc15e010 2024-04-05T22:29:37.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.275+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc159550 con 0x7fb4bc15e010 2024-04-05T22:29:37.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.276+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc0ae5a0 con 0x7fb4bc15e010 2024-04-05T22:29:37.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.277+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fb4b4007070 con 0x7fb4bc15e010 2024-04-05T22:29:37.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.278+0000 7fb4b3400640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 0x7fb49c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.278+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4b4089d70 con 0x7fb4bc15e010 2024-04-05T22:29:37.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.278+0000 7fb4bb400640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 0x7fb49c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.280+0000 7fb4bb400640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 0x7fb49c06d690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac036000 tx=0x7fb4ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.282+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb4bc0ae5a0 con 0x7fb4bc15e010 2024-04-05T22:29:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.472+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4bc0adc30 con 0x7fb49c06b1d0 2024-04-05T22:29:37.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.496+0000 7fb4b3400640 1 -- 172.21.15.73:0/4145163883 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287806 (secure 0 0 0) 0x7fb4bc0adc30 con 0x7fb49c06b1d0 2024-04-05T22:29:37.501 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:37.502 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 msgr2=0x7fb49c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 0x7fb49c06d690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac036000 tx=0x7fb4ac008030 comp rx=0 tx=0).stop 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 msgr2=0x7fb4bc15bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400caa0 tx=0x7fb4b400cf60 comp rx=0 tx=0).stop 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 shutdown_connections 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb49c06b1d0 0x7fb49c06d690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4bc15e010 0x7fb4bc15bd60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc0cc170 0x7fb4bc159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 --2- 172.21.15.73:0/4145163883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc070c00 0x7fb4bc1474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 >> 172.21.15.73:0/4145163883 conn(0x7fb4bc0b16d0 msgr2=0x7fb4bc0b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:37.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 shutdown_connections 2024-04-05T22:29:37.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.503+0000 7fb4c2600640 1 -- 172.21.15.73:0/4145163883 wait complete. 2024-04-05T22:29:37.522 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-05T22:29:35.565739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279816,"kb_used_data":14504,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754694904,"statfs":{"total":773094113280,"available":772807581696,"internally_reserved":0,"allocated":14852096,"data_stored":7191252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004682"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584861+0000","last_change":"2024-04-05T22:29:27.581218+0000","last_active":"2024-04-05T22:29:27.584861+0000","last_peered":"2024-04-05T22:29:27.584861+0000","last_clean":"2024-04-05T22:29:27.584861+0000","last_became_active":"2024-04-05T22:29:27.580977+0000","last_became_peered":"2024-04-05T22:29:27.580977+0000","last_unstale":"2024-04-05T22:29:27.584861+0000","last_undegraded":"2024-04-05T22:29:27.584861+0000","last_fullsized":"2024-04-05T22:29:27.584861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582488+0000","last_change":"2024-04-05T22:29:27.581336+0000","last_active":"2024-04-05T22:29:27.582488+0000","last_peered":"2024-04-05T22:29:27.582488+0000","last_clean":"2024-04-05T22:29:27.582488+0000","last_became_active":"2024-04-05T22:29:27.580747+0000","last_became_peered":"2024-04-05T22:29:27.580747+0000","last_unstale":"2024-04-05T22:29:27.582488+0000","last_undegraded":"2024-04-05T22:29:27.582488+0000","last_fullsized":"2024-04-05T22:29:27.582488+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:00.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.604225+0000","last_change":"2024-04-05T22:29:27.601388+0000","last_active":"2024-04-05T22:29:27.604225+0000","last_peered":"2024-04-05T22:29:27.604225+0000","last_clean":"2024-04-05T22:29:27.604225+0000","last_became_active":"2024-04-05T22:29:27.600096+0000","last_became_peered":"2024-04-05T22:29:27.600096+0000","last_unstale":"2024-04-05T22:29:27.604225+0000","last_undegraded":"2024-04-05T22:29:27.604225+0000","last_fullsized":"2024-04-05T22:29:27.604225+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:13:51.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608572+0000","last_change":"2024-04-05T22:29:27.607311+0000","last_active":"2024-04-05T22:29:27.608572+0000","last_peered":"2024-04-05T22:29:27.608572+0000","last_clean":"2024-04-05T22:29:27.608572+0000","last_became_active":"2024-04-05T22:29:27.606387+0000","last_became_peered":"2024-04-05T22:29:27.606387+0000","last_unstale":"2024-04-05T22:29:27.608572+0000","last_undegraded":"2024-04-05T22:29:27.608572+0000","last_fullsized":"2024-04-05T22:29:27.608572+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:08.653634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609841+0000","last_change":"2024-04-05T22:29:27.603416+0000","last_active":"2024-04-05T22:29:27.609841+0000","last_peered":"2024-04-05T22:29:27.609841+0000","last_clean":"2024-04-05T22:29:27.609841+0000","last_became_active":"2024-04-05T22:29:27.603098+0000","last_became_peered":"2024-04-05T22:29:27.603098+0000","last_unstale":"2024-04-05T22:29:27.609841+0000","last_undegraded":"2024-04-05T22:29:27.609841+0000","last_fullsized":"2024-04-05T22:29:27.609841+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603750+0000","last_change":"2024-04-05T22:29:27.600686+0000","last_active":"2024-04-05T22:29:27.603750+0000","last_peered":"2024-04-05T22:29:27.603750+0000","last_clean":"2024-04-05T22:29:27.603750+0000","last_became_active":"2024-04-05T22:29:27.599951+0000","last_became_peered":"2024-04-05T22:29:27.599951+0000","last_unstale":"2024-04-05T22:29:27.603750+0000","last_undegraded":"2024-04-05T22:29:27.603750+0000","last_fullsized":"2024-04-05T22:29:27.603750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595725+0000","last_change":"2024-04-05T22:29:27.592245+0000","last_active":"2024-04-05T22:29:27.595725+0000","last_peered":"2024-04-05T22:29:27.595725+0000","last_clean":"2024-04-05T22:29:27.595725+0000","last_became_active":"2024-04-05T22:29:27.591834+0000","last_became_peered":"2024-04-05T22:29:27.591834+0000","last_unstale":"2024-04-05T22:29:27.595725+0000","last_undegraded":"2024-04-05T22:29:27.595725+0000","last_fullsized":"2024-04-05T22:29:27.595725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603756+0000","last_change":"2024-04-05T22:29:27.600786+0000","last_active":"2024-04-05T22:29:27.603756+0000","last_peered":"2024-04-05T22:29:27.603756+0000","last_clean":"2024-04-05T22:29:27.603756+0000","last_became_active":"2024-04-05T22:29:27.600517+0000","last_became_peered":"2024-04-05T22:29:27.600517+0000","last_unstale":"2024-04-05T22:29:27.603756+0000","last_undegraded":"2024-04-05T22:29:27.603756+0000","last_fullsized":"2024-04-05T22:29:27.603756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577610+0000","last_change":"2024-04-05T22:29:27.574937+0000","last_active":"2024-04-05T22:29:27.577610+0000","last_peered":"2024-04-05T22:29:27.577610+0000","last_clean":"2024-04-05T22:29:27.577610+0000","last_became_active":"2024-04-05T22:29:27.574693+0000","last_became_peered":"2024-04-05T22:29:27.574693+0000","last_unstale":"2024-04-05T22:29:27.577610+0000","last_undegraded":"2024-04-05T22:29:27.577610+0000","last_fullsized":"2024-04-05T22:29:27.577610+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:00:20.347899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612953+0000","last_change":"2024-04-05T22:29:27.611130+0000","last_active":"2024-04-05T22:29:27.612953+0000","last_peered":"2024-04-05T22:29:27.612953+0000","last_clean":"2024-04-05T22:29:27.612953+0000","last_became_active":"2024-04-05T22:29:27.610809+0000","last_became_peered":"2024-04-05T22:29:27.610809+0000","last_unstale":"2024-04-05T22:29:27.612953+0000","last_undegraded":"2024-04-05T22:29:27.612953+0000","last_fullsized":"2024-04-05T22:29:27.612953+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:07.426970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.597766+0000","last_change":"2024-04-05T22:29:27.597089+0000","last_active":"2024-04-05T22:29:27.597766+0000","last_peered":"2024-04-05T22:29:27.597766+0000","last_clean":"2024-04-05T22:29:27.597766+0000","last_became_active":"2024-04-05T22:29:27.596762+0000","last_became_peered":"2024-04-05T22:29:27.596762+0000","last_unstale":"2024-04-05T22:29:27.597766+0000","last_undegraded":"2024-04-05T22:29:27.597766+0000","last_fullsized":"2024-04-05T22:29:27.597766+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607370+0000","last_change":"2024-04-05T22:29:27.604313+0000","last_active":"2024-04-05T22:29:27.607370+0000","last_peered":"2024-04-05T22:29:27.607370+0000","last_clean":"2024-04-05T22:29:27.607370+0000","last_became_active":"2024-04-05T22:29:27.602775+0000","last_became_peered":"2024-04-05T22:29:27.602775+0000","last_unstale":"2024-04-05T22:29:27.607370+0000","last_undegraded":"2024-04-05T22:29:27.607370+0000","last_fullsized":"2024-04-05T22:29:27.607370+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:41:27.270000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609822+0000","last_change":"2024-04-05T22:29:27.603249+0000","last_active":"2024-04-05T22:29:27.609822+0000","last_peered":"2024-04-05T22:29:27.609822+0000","last_clean":"2024-04-05T22:29:27.609822+0000","last_became_active":"2024-04-05T22:29:27.602935+0000","last_became_peered":"2024-04-05T22:29:27.602935+0000","last_unstale":"2024-04-05T22:29:27.609822+0000","last_undegraded":"2024-04-05T22:29:27.609822+0000","last_fullsized":"2024-04-05T22:29:27.609822+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:37.324655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606150+0000","last_change":"2024-04-05T22:29:27.601511+0000","last_active":"2024-04-05T22:29:27.606150+0000","last_peered":"2024-04-05T22:29:27.606150+0000","last_clean":"2024-04-05T22:29:27.606150+0000","last_became_active":"2024-04-05T22:29:27.601160+0000","last_became_peered":"2024-04-05T22:29:27.601160+0000","last_unstale":"2024-04-05T22:29:27.606150+0000","last_undegraded":"2024-04-05T22:29:27.606150+0000","last_fullsized":"2024-04-05T22:29:27.606150+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582328+0000","last_change":"2024-04-05T22:29:27.578913+0000","last_active":"2024-04-05T22:29:27.582328+0000","last_peered":"2024-04-05T22:29:27.582328+0000","last_clean":"2024-04-05T22:29:27.582328+0000","last_became_active":"2024-04-05T22:29:27.578418+0000","last_became_peered":"2024-04-05T22:29:27.578418+0000","last_unstale":"2024-04-05T22:29:27.582328+0000","last_undegraded":"2024-04-05T22:29:27.582328+0000","last_fullsized":"2024-04-05T22:29:27.582328+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:12.301185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.586696+0000","last_change":"2024-04-05T22:29:27.584911+0000","last_active":"2024-04-05T22:29:27.586696+0000","last_peered":"2024-04-05T22:29:27.586696+0000","last_clean":"2024-04-05T22:29:27.586696+0000","last_became_active":"2024-04-05T22:29:27.584589+0000","last_became_peered":"2024-04-05T22:29:27.584589+0000","last_unstale":"2024-04-05T22:29:27.586696+0000","last_undegraded":"2024-04-05T22:29:27.586696+0000","last_fullsized":"2024-04-05T22:29:27.586696+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:16:54.058169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.591583+0000","last_change":"2024-04-05T22:29:27.588186+0000","last_active":"2024-04-05T22:29:27.591583+0000","last_peered":"2024-04-05T22:29:27.591583+0000","last_clean":"2024-04-05T22:29:27.591583+0000","last_became_active":"2024-04-05T22:29:27.587695+0000","last_became_peered":"2024-04-05T22:29:27.587695+0000","last_unstale":"2024-04-05T22:29:27.591583+0000","last_undegraded":"2024-04-05T22:29:27.591583+0000","last_fullsized":"2024-04-05T22:29:27.591583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:45.346519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599139+0000","last_change":"2024-04-05T22:29:27.593565+0000","last_active":"2024-04-05T22:29:27.599139+0000","last_peered":"2024-04-05T22:29:27.599139+0000","last_clean":"2024-04-05T22:29:27.599139+0000","last_became_active":"2024-04-05T22:29:27.593152+0000","last_became_peered":"2024-04-05T22:29:27.593152+0000","last_unstale":"2024-04-05T22:29:27.599139+0000","last_undegraded":"2024-04-05T22:29:27.599139+0000","last_fullsized":"2024-04-05T22:29:27.599139+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:03:41.154597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584753+0000","last_change":"2024-04-05T22:29:27.578997+0000","last_active":"2024-04-05T22:29:27.584753+0000","last_peered":"2024-04-05T22:29:27.584753+0000","last_clean":"2024-04-05T22:29:27.584753+0000","last_became_active":"2024-04-05T22:29:27.578672+0000","last_became_peered":"2024-04-05T22:29:27.578672+0000","last_unstale":"2024-04-05T22:29:27.584753+0000","last_undegraded":"2024-04-05T22:29:27.584753+0000","last_fullsized":"2024-04-05T22:29:27.584753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.582743+0000","last_change":"2024-04-05T22:29:27.578944+0000","last_active":"2024-04-05T22:29:27.582743+0000","last_peered":"2024-04-05T22:29:27.582743+0000","last_clean":"2024-04-05T22:29:27.582743+0000","last_became_active":"2024-04-05T22:29:27.578714+0000","last_became_peered":"2024-04-05T22:29:27.578714+0000","last_unstale":"2024-04-05T22:29:27.582743+0000","last_undegraded":"2024-04-05T22:29:27.582743+0000","last_fullsized":"2024-04-05T22:29:27.582743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601222+0000","last_change":"2024-04-05T22:29:27.595591+0000","last_active":"2024-04-05T22:29:27.601222+0000","last_peered":"2024-04-05T22:29:27.601222+0000","last_clean":"2024-04-05T22:29:27.601222+0000","last_became_active":"2024-04-05T22:29:27.595293+0000","last_became_peered":"2024-04-05T22:29:27.595293+0000","last_unstale":"2024-04-05T22:29:27.601222+0000","last_undegraded":"2024-04-05T22:29:27.601222+0000","last_fullsized":"2024-04-05T22:29:27.601222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601374+0000","last_change":"2024-04-05T22:29:27.594918+0000","last_active":"2024-04-05T22:29:27.601374+0000","last_peered":"2024-04-05T22:29:27.601374+0000","last_clean":"2024-04-05T22:29:27.601374+0000","last_became_active":"2024-04-05T22:29:27.592311+0000","last_became_peered":"2024-04-05T22:29:27.592311+0000","last_unstale":"2024-04-05T22:29:27.601374+0000","last_undegraded":"2024-04-05T22:29:27.601374+0000","last_fullsized":"2024-04-05T22:29:27.601374+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.598333+0000","last_change":"2024-04-05T22:29:27.592484+0000","last_active":"2024-04-05T22:29:27.598333+0000","last_peered":"2024-04-05T22:29:27.598333+0000","last_clean":"2024-04-05T22:29:27.598333+0000","last_became_active":"2024-04-05T22:29:27.591147+0000","last_became_peered":"2024-04-05T22:29:27.591147+0000","last_unstale":"2024-04-05T22:29:27.598333+0000","last_undegraded":"2024-04-05T22:29:27.598333+0000","last_fullsized":"2024-04-05T22:29:27.598333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:50:54.351757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602870+0000","last_change":"2024-04-05T22:29:27.594629+0000","last_active":"2024-04-05T22:29:27.602870+0000","last_peered":"2024-04-05T22:29:27.602870+0000","last_clean":"2024-04-05T22:29:27.602870+0000","last_became_active":"2024-04-05T22:29:27.594355+0000","last_became_peered":"2024-04-05T22:29:27.594355+0000","last_unstale":"2024-04-05T22:29:27.602870+0000","last_undegraded":"2024-04-05T22:29:27.602870+0000","last_fullsized":"2024-04-05T22:29:27.602870+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:12.338327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.587211+0000","last_change":"2024-04-05T22:29:27.585562+0000","last_active":"2024-04-05T22:29:27.587211+0000","last_peered":"2024-04-05T22:29:27.587211+0000","last_clean":"2024-04-05T22:29:27.587211+0000","last_became_active":"2024-04-05T22:29:27.585305+0000","last_became_peered":"2024-04-05T22:29:27.585305+0000","last_unstale":"2024-04-05T22:29:27.587211+0000","last_undegraded":"2024-04-05T22:29:27.587211+0000","last_fullsized":"2024-04-05T22:29:27.587211+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:45:28.678555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.585049+0000","last_change":"2024-04-05T22:29:27.581233+0000","last_active":"2024-04-05T22:29:27.585049+0000","last_peered":"2024-04-05T22:29:27.585049+0000","last_clean":"2024-04-05T22:29:27.585049+0000","last_became_active":"2024-04-05T22:29:27.580350+0000","last_became_peered":"2024-04-05T22:29:27.580350+0000","last_unstale":"2024-04-05T22:29:27.585049+0000","last_undegraded":"2024-04-05T22:29:27.585049+0000","last_fullsized":"2024-04-05T22:29:27.585049+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:54.393886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612146+0000","last_change":"2024-04-05T22:29:27.611763+0000","last_active":"2024-04-05T22:29:27.612146+0000","last_peered":"2024-04-05T22:29:27.612146+0000","last_clean":"2024-04-05T22:29:27.612146+0000","last_became_active":"2024-04-05T22:29:27.611516+0000","last_became_peered":"2024-04-05T22:29:27.611516+0000","last_unstale":"2024-04-05T22:29:27.612146+0000","last_undegraded":"2024-04-05T22:29:27.612146+0000","last_fullsized":"2024-04-05T22:29:27.612146+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:29.952383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603620+0000","last_change":"2024-04-05T22:29:27.601645+0000","last_active":"2024-04-05T22:29:27.603620+0000","last_peered":"2024-04-05T22:29:27.603620+0000","last_clean":"2024-04-05T22:29:27.603620+0000","last_became_active":"2024-04-05T22:29:27.601266+0000","last_became_peered":"2024-04-05T22:29:27.601266+0000","last_unstale":"2024-04-05T22:29:27.603620+0000","last_undegraded":"2024-04-05T22:29:27.603620+0000","last_fullsized":"2024-04-05T22:29:27.603620+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:19:40.288096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607083+0000","last_change":"2024-04-05T22:29:27.605434+0000","last_active":"2024-04-05T22:29:27.607083+0000","last_peered":"2024-04-05T22:29:27.607083+0000","last_clean":"2024-04-05T22:29:27.607083+0000","last_became_active":"2024-04-05T22:29:27.605032+0000","last_became_peered":"2024-04-05T22:29:27.605032+0000","last_unstale":"2024-04-05T22:29:27.607083+0000","last_undegraded":"2024-04-05T22:29:27.607083+0000","last_fullsized":"2024-04-05T22:29:27.607083+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:36.161892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595831+0000","last_change":"2024-04-05T22:29:27.588374+0000","last_active":"2024-04-05T22:29:27.595831+0000","last_peered":"2024-04-05T22:29:27.595831+0000","last_clean":"2024-04-05T22:29:27.595831+0000","last_became_active":"2024-04-05T22:29:27.588072+0000","last_became_peered":"2024-04-05T22:29:27.588072+0000","last_unstale":"2024-04-05T22:29:27.595831+0000","last_undegraded":"2024-04-05T22:29:27.595831+0000","last_fullsized":"2024-04-05T22:29:27.595831+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:01:58.455030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607320+0000","last_change":"2024-04-05T22:29:27.605960+0000","last_active":"2024-04-05T22:29:27.607320+0000","last_peered":"2024-04-05T22:29:27.607320+0000","last_clean":"2024-04-05T22:29:27.607320+0000","last_became_active":"2024-04-05T22:29:27.605718+0000","last_became_peered":"2024-04-05T22:29:27.605718+0000","last_unstale":"2024-04-05T22:29:27.607320+0000","last_undegraded":"2024-04-05T22:29:27.607320+0000","last_fullsized":"2024-04-05T22:29:27.607320+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:51:01.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601795+0000","last_change":"2024-04-05T22:29:27.600750+0000","last_active":"2024-04-05T22:29:27.601795+0000","last_peered":"2024-04-05T22:29:27.601795+0000","last_clean":"2024-04-05T22:29:27.601795+0000","last_became_active":"2024-04-05T22:29:27.600487+0000","last_became_peered":"2024-04-05T22:29:27.600487+0000","last_unstale":"2024-04-05T22:29:27.601795+0000","last_undegraded":"2024-04-05T22:29:27.601795+0000","last_fullsized":"2024-04-05T22:29:27.601795+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:48.394581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602970+0000","last_change":"2024-04-05T22:29:27.596691+0000","last_active":"2024-04-05T22:29:27.602970+0000","last_peered":"2024-04-05T22:29:27.602970+0000","last_clean":"2024-04-05T22:29:27.602970+0000","last_became_active":"2024-04-05T22:29:27.596339+0000","last_became_peered":"2024-04-05T22:29:27.596339+0000","last_unstale":"2024-04-05T22:29:27.602970+0000","last_undegraded":"2024-04-05T22:29:27.602970+0000","last_fullsized":"2024-04-05T22:29:27.602970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:39:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599357+0000","last_change":"2024-04-05T22:29:27.113665+0000","last_active":"2024-04-05T22:29:27.599357+0000","last_peered":"2024-04-05T22:29:27.599357+0000","last_clean":"2024-04-05T22:29:27.599357+0000","last_became_active":"2024-04-05T22:29:27.113368+0000","last_became_peered":"2024-04-05T22:29:27.113368+0000","last_unstale":"2024-04-05T22:29:27.599357+0000","last_undegraded":"2024-04-05T22:29:27.599357+0000","last_fullsized":"2024-04-05T22:29:27.599357+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:29.247549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.586396+0000","last_change":"2024-04-05T22:29:27.584808+0000","last_active":"2024-04-05T22:29:27.586396+0000","last_peered":"2024-04-05T22:29:27.586396+0000","last_clean":"2024-04-05T22:29:27.586396+0000","last_became_active":"2024-04-05T22:29:27.584301+0000","last_became_peered":"2024-04-05T22:29:27.584301+0000","last_unstale":"2024-04-05T22:29:27.586396+0000","last_undegraded":"2024-04-05T22:29:27.586396+0000","last_fullsized":"2024-04-05T22:29:27.586396+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:07:00.736085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.583065+0000","last_change":"2024-04-05T22:29:27.115636+0000","last_active":"2024-04-05T22:29:27.583065+0000","last_peered":"2024-04-05T22:29:27.583065+0000","last_clean":"2024-04-05T22:29:27.583065+0000","last_became_active":"2024-04-05T22:29:27.115170+0000","last_became_peered":"2024-04-05T22:29:27.115170+0000","last_unstale":"2024-04-05T22:29:27.583065+0000","last_undegraded":"2024-04-05T22:29:27.583065+0000","last_fullsized":"2024-04-05T22:29:27.583065+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:49.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607075+0000","last_change":"2024-04-05T22:29:27.604863+0000","last_active":"2024-04-05T22:29:27.607075+0000","last_peered":"2024-04-05T22:29:27.607075+0000","last_clean":"2024-04-05T22:29:27.607075+0000","last_became_active":"2024-04-05T22:29:27.604550+0000","last_became_peered":"2024-04-05T22:29:27.604550+0000","last_unstale":"2024-04-05T22:29:27.607075+0000","last_undegraded":"2024-04-05T22:29:27.607075+0000","last_fullsized":"2024-04-05T22:29:27.607075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:30.261318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600459+0000","last_change":"2024-04-05T22:29:27.106460+0000","last_active":"2024-04-05T22:29:27.600459+0000","last_peered":"2024-04-05T22:29:27.600459+0000","last_clean":"2024-04-05T22:29:27.600459+0000","last_became_active":"2024-04-05T22:29:27.106113+0000","last_became_peered":"2024-04-05T22:29:27.106113+0000","last_unstale":"2024-04-05T22:29:27.600459+0000","last_undegraded":"2024-04-05T22:29:27.600459+0000","last_fullsized":"2024-04-05T22:29:27.600459+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:32:47.380812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600371+0000","last_change":"2024-04-05T22:29:27.595378+0000","last_active":"2024-04-05T22:29:27.600371+0000","last_peered":"2024-04-05T22:29:27.600371+0000","last_clean":"2024-04-05T22:29:27.600371+0000","last_became_active":"2024-04-05T22:29:27.595083+0000","last_became_peered":"2024-04-05T22:29:27.595083+0000","last_unstale":"2024-04-05T22:29:27.600371+0000","last_undegraded":"2024-04-05T22:29:27.600371+0000","last_fullsized":"2024-04-05T22:29:27.600371+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:12.810137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608741+0000","last_change":"2024-04-05T22:29:27.098092+0000","last_active":"2024-04-05T22:29:27.608741+0000","last_peered":"2024-04-05T22:29:27.608741+0000","last_clean":"2024-04-05T22:29:27.608741+0000","last_became_active":"2024-04-05T22:29:27.097449+0000","last_became_peered":"2024-04-05T22:29:27.097449+0000","last_unstale":"2024-04-05T22:29:27.608741+0000","last_undegraded":"2024-04-05T22:29:27.608741+0000","last_fullsized":"2024-04-05T22:29:27.608741+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:40.984286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602125+0000","last_change":"2024-04-05T22:29:27.600242+0000","last_active":"2024-04-05T22:29:27.602125+0000","last_peered":"2024-04-05T22:29:27.602125+0000","last_clean":"2024-04-05T22:29:27.602125+0000","last_became_active":"2024-04-05T22:29:27.600010+0000","last_became_peered":"2024-04-05T22:29:27.600010+0000","last_unstale":"2024-04-05T22:29:27.602125+0000","last_undegraded":"2024-04-05T22:29:27.602125+0000","last_fullsized":"2024-04-05T22:29:27.602125+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:14:10.438034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.593666+0000","last_change":"2024-04-05T22:29:27.106850+0000","last_active":"2024-04-05T22:29:27.593666+0000","last_peered":"2024-04-05T22:29:27.593666+0000","last_clean":"2024-04-05T22:29:27.593666+0000","last_became_active":"2024-04-05T22:29:27.106525+0000","last_became_peered":"2024-04-05T22:29:27.106525+0000","last_unstale":"2024-04-05T22:29:27.593666+0000","last_undegraded":"2024-04-05T22:29:27.593666+0000","last_fullsized":"2024-04-05T22:29:27.593666+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:30.113756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.608020+0000","last_change":"2024-04-05T22:29:27.605931+0000","last_active":"2024-04-05T22:29:27.608020+0000","last_peered":"2024-04-05T22:29:27.608020+0000","last_clean":"2024-04-05T22:29:27.608020+0000","last_became_active":"2024-04-05T22:29:27.605340+0000","last_became_peered":"2024-04-05T22:29:27.605340+0000","last_unstale":"2024-04-05T22:29:27.608020+0000","last_undegraded":"2024-04-05T22:29:27.608020+0000","last_fullsized":"2024-04-05T22:29:27.608020+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:51:19.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602826+0000","last_change":"2024-04-05T22:29:27.113913+0000","last_active":"2024-04-05T22:29:27.602826+0000","last_peered":"2024-04-05T22:29:27.602826+0000","last_clean":"2024-04-05T22:29:27.602826+0000","last_became_active":"2024-04-05T22:29:27.112452+0000","last_became_peered":"2024-04-05T22:29:27.112452+0000","last_unstale":"2024-04-05T22:29:27.602826+0000","last_undegraded":"2024-04-05T22:29:27.602826+0000","last_fullsized":"2024-04-05T22:29:27.602826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:13:41.450507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.602146+0000","last_change":"2024-04-05T22:29:27.600361+0000","last_active":"2024-04-05T22:29:27.602146+0000","last_peered":"2024-04-05T22:29:27.602146+0000","last_clean":"2024-04-05T22:29:27.602146+0000","last_became_active":"2024-04-05T22:29:27.600076+0000","last_became_peered":"2024-04-05T22:29:27.600076+0000","last_unstale":"2024-04-05T22:29:27.602146+0000","last_undegraded":"2024-04-05T22:29:27.602146+0000","last_fullsized":"2024-04-05T22:29:27.602146+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:51:39.593037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577538+0000","last_change":"2024-04-05T22:29:27.105830+0000","last_active":"2024-04-05T22:29:27.577538+0000","last_peered":"2024-04-05T22:29:27.577538+0000","last_clean":"2024-04-05T22:29:27.577538+0000","last_became_active":"2024-04-05T22:29:27.104458+0000","last_became_peered":"2024-04-05T22:29:27.104458+0000","last_unstale":"2024-04-05T22:29:27.577538+0000","last_undegraded":"2024-04-05T22:29:27.577538+0000","last_fullsized":"2024-04-05T22:29:27.577538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:34.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599890+0000","last_change":"2024-04-05T22:29:27.595916+0000","last_active":"2024-04-05T22:29:27.599890+0000","last_peered":"2024-04-05T22:29:27.599890+0000","last_clean":"2024-04-05T22:29:27.599890+0000","last_became_active":"2024-04-05T22:29:27.595159+0000","last_became_peered":"2024-04-05T22:29:27.595159+0000","last_unstale":"2024-04-05T22:29:27.599890+0000","last_undegraded":"2024-04-05T22:29:27.599890+0000","last_fullsized":"2024-04-05T22:29:27.599890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:27:00.354971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.588007+0000","last_change":"2024-04-05T22:29:27.107183+0000","last_active":"2024-04-05T22:29:27.588007+0000","last_peered":"2024-04-05T22:29:27.588007+0000","last_clean":"2024-04-05T22:29:27.588007+0000","last_became_active":"2024-04-05T22:29:27.106873+0000","last_became_peered":"2024-04-05T22:29:27.106873+0000","last_unstale":"2024-04-05T22:29:27.588007+0000","last_undegraded":"2024-04-05T22:29:27.588007+0000","last_fullsized":"2024-04-05T22:29:27.588007+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:31.435759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599553+0000","last_change":"2024-04-05T22:29:27.595541+0000","last_active":"2024-04-05T22:29:27.599553+0000","last_peered":"2024-04-05T22:29:27.599553+0000","last_clean":"2024-04-05T22:29:27.599553+0000","last_became_active":"2024-04-05T22:29:27.595188+0000","last_became_peered":"2024-04-05T22:29:27.595188+0000","last_unstale":"2024-04-05T22:29:27.599553+0000","last_undegraded":"2024-04-05T22:29:27.599553+0000","last_fullsized":"2024-04-05T22:29:27.599553+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:12.002317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.562143+0000","last_change":"2024-04-05T22:29:27.098118+0000","last_active":"2024-04-05T22:29:27.562143+0000","last_peered":"2024-04-05T22:29:27.562143+0000","last_clean":"2024-04-05T22:29:27.562143+0000","last_became_active":"2024-04-05T22:29:27.097772+0000","last_became_peered":"2024-04-05T22:29:27.097772+0000","last_unstale":"2024-04-05T22:29:27.562143+0000","last_undegraded":"2024-04-05T22:29:27.562143+0000","last_fullsized":"2024-04-05T22:29:27.562143+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:04.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.591563+0000","last_change":"2024-04-05T22:29:27.587994+0000","last_active":"2024-04-05T22:29:27.591563+0000","last_peered":"2024-04-05T22:29:27.591563+0000","last_clean":"2024-04-05T22:29:27.591563+0000","last_became_active":"2024-04-05T22:29:27.585579+0000","last_became_peered":"2024-04-05T22:29:27.585579+0000","last_unstale":"2024-04-05T22:29:27.591563+0000","last_undegraded":"2024-04-05T22:29:27.591563+0000","last_fullsized":"2024-04-05T22:29:27.591563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:12:05.814796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.574245+0000","last_change":"2024-04-05T22:29:27.117368+0000","last_active":"2024-04-05T22:29:27.574245+0000","last_peered":"2024-04-05T22:29:27.574245+0000","last_clean":"2024-04-05T22:29:27.574245+0000","last_became_active":"2024-04-05T22:29:27.117006+0000","last_became_peered":"2024-04-05T22:29:27.117006+0000","last_unstale":"2024-04-05T22:29:27.574245+0000","last_undegraded":"2024-04-05T22:29:27.574245+0000","last_fullsized":"2024-04-05T22:29:27.574245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:26.585990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.587240+0000","last_change":"2024-04-05T22:29:27.586030+0000","last_active":"2024-04-05T22:29:27.587240+0000","last_peered":"2024-04-05T22:29:27.587240+0000","last_clean":"2024-04-05T22:29:27.587240+0000","last_became_active":"2024-04-05T22:29:27.585778+0000","last_became_peered":"2024-04-05T22:29:27.585778+0000","last_unstale":"2024-04-05T22:29:27.587240+0000","last_undegraded":"2024-04-05T22:29:27.587240+0000","last_fullsized":"2024-04-05T22:29:27.587240+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:52:41.998055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"135'6","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.597489+0000","last_change":"2024-04-05T22:29:27.107727+0000","last_active":"2024-04-05T22:29:27.597489+0000","last_peered":"2024-04-05T22:29:27.597489+0000","last_clean":"2024-04-05T22:29:27.597489+0000","last_became_active":"2024-04-05T22:29:27.107409+0000","last_became_peered":"2024-04-05T22:29:27.107409+0000","last_unstale":"2024-04-05T22:29:27.597489+0000","last_undegraded":"2024-04-05T22:29:27.597489+0000","last_fullsized":"2024-04-05T22:29:27.597489+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:36:16.139153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607928+0000","last_change":"2024-04-05T22:29:27.605515+0000","last_active":"2024-04-05T22:29:27.607928+0000","last_peered":"2024-04-05T22:29:27.607928+0000","last_clean":"2024-04-05T22:29:27.607928+0000","last_became_active":"2024-04-05T22:29:27.605233+0000","last_became_peered":"2024-04-05T22:29:27.605233+0000","last_unstale":"2024-04-05T22:29:27.607928+0000","last_undegraded":"2024-04-05T22:29:27.607928+0000","last_fullsized":"2024-04-05T22:29:27.607928+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:24.706614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610980+0000","last_change":"2024-04-05T22:29:27.109281+0000","last_active":"2024-04-05T22:29:27.610980+0000","last_peered":"2024-04-05T22:29:27.610980+0000","last_clean":"2024-04-05T22:29:27.610980+0000","last_became_active":"2024-04-05T22:29:27.108992+0000","last_became_peered":"2024-04-05T22:29:27.108992+0000","last_unstale":"2024-04-05T22:29:27.610980+0000","last_undegraded":"2024-04-05T22:29:27.610980+0000","last_fullsized":"2024-04-05T22:29:27.610980+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:33:12.557477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.577592+0000","last_change":"2024-04-05T22:29:27.574052+0000","last_active":"2024-04-05T22:29:27.577592+0000","last_peered":"2024-04-05T22:29:27.577592+0000","last_clean":"2024-04-05T22:29:27.577592+0000","last_became_active":"2024-04-05T22:29:27.573784+0000","last_became_peered":"2024-04-05T22:29:27.573784+0000","last_unstale":"2024-04-05T22:29:27.577592+0000","last_undegraded":"2024-04-05T22:29:27.577592+0000","last_fullsized":"2024-04-05T22:29:27.577592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601011+0000","last_change":"2024-04-05T22:29:27.109031+0000","last_active":"2024-04-05T22:29:27.601011+0000","last_peered":"2024-04-05T22:29:27.601011+0000","last_clean":"2024-04-05T22:29:27.601011+0000","last_became_active":"2024-04-05T22:29:27.108620+0000","last_became_peered":"2024-04-05T22:29:27.108620+0000","last_unstale":"2024-04-05T22:29:27.601011+0000","last_undegraded":"2024-04-05T22:29:27.601011+0000","last_fullsized":"2024-04-05T22:29:27.601011+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:18:46.523577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.609746+0000","last_change":"2024-04-05T22:29:27.604046+0000","last_active":"2024-04-05T22:29:27.609746+0000","last_peered":"2024-04-05T22:29:27.609746+0000","last_clean":"2024-04-05T22:29:27.609746+0000","last_became_active":"2024-04-05T22:29:27.603778+0000","last_became_peered":"2024-04-05T22:29:27.603778+0000","last_unstale":"2024-04-05T22:29:27.609746+0000","last_undegraded":"2024-04-05T22:29:27.609746+0000","last_fullsized":"2024-04-05T22:29:27.609746+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:23:59.083073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.569223+0000","last_change":"2024-04-05T22:29:27.103901+0000","last_active":"2024-04-05T22:29:27.569223+0000","last_peered":"2024-04-05T22:29:27.569223+0000","last_clean":"2024-04-05T22:29:27.569223+0000","last_became_active":"2024-04-05T22:29:27.103615+0000","last_became_peered":"2024-04-05T22:29:27.103615+0000","last_unstale":"2024-04-05T22:29:27.569223+0000","last_undegraded":"2024-04-05T22:29:27.569223+0000","last_fullsized":"2024-04-05T22:29:27.569223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:46.499208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.601581+0000","last_change":"2024-04-05T22:29:27.596527+0000","last_active":"2024-04-05T22:29:27.601581+0000","last_peered":"2024-04-05T22:29:27.601581+0000","last_clean":"2024-04-05T22:29:27.601581+0000","last_became_active":"2024-04-05T22:29:27.596245+0000","last_became_peered":"2024-04-05T22:29:27.596245+0000","last_unstale":"2024-04-05T22:29:27.601581+0000","last_undegraded":"2024-04-05T22:29:27.601581+0000","last_fullsized":"2024-04-05T22:29:27.601581+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:44.440426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"135'8","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596819+0000","last_change":"2024-04-05T22:29:27.104205+0000","last_active":"2024-04-05T22:29:27.596819+0000","last_peered":"2024-04-05T22:29:27.596819+0000","last_clean":"2024-04-05T22:29:27.596819+0000","last_became_active":"2024-04-05T22:29:27.102618+0000","last_became_peered":"2024-04-05T22:29:27.102618+0000","last_unstale":"2024-04-05T22:29:27.596819+0000","last_undegraded":"2024-04-05T22:29:27.596819+0000","last_fullsized":"2024-04-05T22:29:27.596819+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:19:05.414218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590709+0000","last_change":"2024-04-05T22:29:27.588741+0000","last_active":"2024-04-05T22:29:27.590709+0000","last_peered":"2024-04-05T22:29:27.590709+0000","last_clean":"2024-04-05T22:29:27.590709+0000","last_became_active":"2024-04-05T22:29:27.588448+0000","last_became_peered":"2024-04-05T22:29:27.588448+0000","last_unstale":"2024-04-05T22:29:27.590709+0000","last_undegraded":"2024-04-05T22:29:27.590709+0000","last_fullsized":"2024-04-05T22:29:27.590709+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:48.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600722+0000","last_change":"2024-04-05T22:29:27.113619+0000","last_active":"2024-04-05T22:29:27.600722+0000","last_peered":"2024-04-05T22:29:27.600722+0000","last_clean":"2024-04-05T22:29:27.600722+0000","last_became_active":"2024-04-05T22:29:27.113367+0000","last_became_peered":"2024-04-05T22:29:27.113367+0000","last_unstale":"2024-04-05T22:29:27.600722+0000","last_undegraded":"2024-04-05T22:29:27.600722+0000","last_fullsized":"2024-04-05T22:29:27.600722+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:26:24.419813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596410+0000","last_change":"2024-04-05T22:29:27.594065+0000","last_active":"2024-04-05T22:29:27.596410+0000","last_peered":"2024-04-05T22:29:27.596410+0000","last_clean":"2024-04-05T22:29:27.596410+0000","last_became_active":"2024-04-05T22:29:27.593786+0000","last_became_peered":"2024-04-05T22:29:27.593786+0000","last_unstale":"2024-04-05T22:29:27.596410+0000","last_undegraded":"2024-04-05T22:29:27.596410+0000","last_fullsized":"2024-04-05T22:29:27.596410+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:12:32.449528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.584249+0000","last_change":"2024-04-05T22:29:27.103848+0000","last_active":"2024-04-05T22:29:27.584249+0000","last_peered":"2024-04-05T22:29:27.584249+0000","last_clean":"2024-04-05T22:29:27.584249+0000","last_became_active":"2024-04-05T22:29:27.103586+0000","last_became_peered":"2024-04-05T22:29:27.103586+0000","last_unstale":"2024-04-05T22:29:27.584249+0000","last_undegraded":"2024-04-05T22:29:27.584249+0000","last_fullsized":"2024-04-05T22:29:27.584249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:18:11.324558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.612482+0000","last_change":"2024-04-05T22:29:27.611143+0000","last_active":"2024-04-05T22:29:27.612482+0000","last_peered":"2024-04-05T22:29:27.612482+0000","last_clean":"2024-04-05T22:29:27.612482+0000","last_became_active":"2024-04-05T22:29:27.610705+0000","last_became_peered":"2024-04-05T22:29:27.610705+0000","last_unstale":"2024-04-05T22:29:27.612482+0000","last_undegraded":"2024-04-05T22:29:27.612482+0000","last_fullsized":"2024-04-05T22:29:27.612482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:00:02.922714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590513+0000","last_change":"2024-04-05T22:29:27.106062+0000","last_active":"2024-04-05T22:29:27.590513+0000","last_peered":"2024-04-05T22:29:27.590513+0000","last_clean":"2024-04-05T22:29:27.590513+0000","last_became_active":"2024-04-05T22:29:27.105748+0000","last_became_peered":"2024-04-05T22:29:27.105748+0000","last_unstale":"2024-04-05T22:29:27.590513+0000","last_undegraded":"2024-04-05T22:29:27.590513+0000","last_fullsized":"2024-04-05T22:29:27.590513+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:48.841874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610084+0000","last_change":"2024-04-05T22:29:27.609052+0000","last_active":"2024-04-05T22:29:27.610084+0000","last_peered":"2024-04-05T22:29:27.610084+0000","last_clean":"2024-04-05T22:29:27.610084+0000","last_became_active":"2024-04-05T22:29:27.608791+0000","last_became_peered":"2024-04-05T22:29:27.608791+0000","last_unstale":"2024-04-05T22:29:27.610084+0000","last_undegraded":"2024-04-05T22:29:27.610084+0000","last_fullsized":"2024-04-05T22:29:27.610084+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:25:46.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.562374+0000","last_change":"2024-04-05T22:29:27.102360+0000","last_active":"2024-04-05T22:29:27.562374+0000","last_peered":"2024-04-05T22:29:27.562374+0000","last_clean":"2024-04-05T22:29:27.562374+0000","last_became_active":"2024-04-05T22:29:27.102051+0000","last_became_peered":"2024-04-05T22:29:27.102051+0000","last_unstale":"2024-04-05T22:29:27.562374+0000","last_undegraded":"2024-04-05T22:29:27.562374+0000","last_fullsized":"2024-04-05T22:29:27.562374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:24:47.073194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595812+0000","last_change":"2024-04-05T22:29:27.587480+0000","last_active":"2024-04-05T22:29:27.595812+0000","last_peered":"2024-04-05T22:29:27.595812+0000","last_clean":"2024-04-05T22:29:27.595812+0000","last_became_active":"2024-04-05T22:29:27.586979+0000","last_became_peered":"2024-04-05T22:29:27.586979+0000","last_unstale":"2024-04-05T22:29:27.595812+0000","last_undegraded":"2024-04-05T22:29:27.595812+0000","last_fullsized":"2024-04-05T22:29:27.595812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:20:27.599759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.594553+0000","last_change":"2024-04-05T22:29:27.111825+0000","last_active":"2024-04-05T22:29:27.594553+0000","last_peered":"2024-04-05T22:29:27.594553+0000","last_clean":"2024-04-05T22:29:27.594553+0000","last_became_active":"2024-04-05T22:29:27.111542+0000","last_became_peered":"2024-04-05T22:29:27.111542+0000","last_unstale":"2024-04-05T22:29:27.594553+0000","last_undegraded":"2024-04-05T22:29:27.594553+0000","last_fullsized":"2024-04-05T22:29:27.594553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:57:22.051262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599579+0000","last_change":"2024-04-05T22:29:27.599050+0000","last_active":"2024-04-05T22:29:27.599579+0000","last_peered":"2024-04-05T22:29:27.599579+0000","last_clean":"2024-04-05T22:29:27.599579+0000","last_became_active":"2024-04-05T22:29:27.598726+0000","last_became_peered":"2024-04-05T22:29:27.598726+0000","last_unstale":"2024-04-05T22:29:27.599579+0000","last_undegraded":"2024-04-05T22:29:27.599579+0000","last_fullsized":"2024-04-05T22:29:27.599579+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:36:52.568979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567974+0000","last_change":"2024-04-05T22:29:27.111984+0000","last_active":"2024-04-05T22:29:27.567974+0000","last_peered":"2024-04-05T22:29:27.567974+0000","last_clean":"2024-04-05T22:29:27.567974+0000","last_became_active":"2024-04-05T22:29:27.111570+0000","last_became_peered":"2024-04-05T22:29:27.111570+0000","last_unstale":"2024-04-05T22:29:27.567974+0000","last_undegraded":"2024-04-05T22:29:27.567974+0000","last_fullsized":"2024-04-05T22:29:27.567974+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:02:55.012896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596428+0000","last_change":"2024-04-05T22:29:27.587732+0000","last_active":"2024-04-05T22:29:27.596428+0000","last_peered":"2024-04-05T22:29:27.596428+0000","last_clean":"2024-04-05T22:29:27.596428+0000","last_became_active":"2024-04-05T22:29:27.587061+0000","last_became_peered":"2024-04-05T22:29:27.587061+0000","last_unstale":"2024-04-05T22:29:27.596428+0000","last_undegraded":"2024-04-05T22:29:27.596428+0000","last_fullsized":"2024-04-05T22:29:27.596428+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:34:56.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.566714+0000","last_change":"2024-04-05T22:29:27.117354+0000","last_active":"2024-04-05T22:29:27.566714+0000","last_peered":"2024-04-05T22:29:27.566714+0000","last_clean":"2024-04-05T22:29:27.566714+0000","last_became_active":"2024-04-05T22:29:27.116968+0000","last_became_peered":"2024-04-05T22:29:27.116968+0000","last_unstale":"2024-04-05T22:29:27.566714+0000","last_undegraded":"2024-04-05T22:29:27.566714+0000","last_fullsized":"2024-04-05T22:29:27.566714+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:32:03.796707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.595743+0000","last_change":"2024-04-05T22:29:27.592556+0000","last_active":"2024-04-05T22:29:27.595743+0000","last_peered":"2024-04-05T22:29:27.595743+0000","last_clean":"2024-04-05T22:29:27.595743+0000","last_became_active":"2024-04-05T22:29:27.592061+0000","last_became_peered":"2024-04-05T22:29:27.592061+0000","last_unstale":"2024-04-05T22:29:27.595743+0000","last_undegraded":"2024-04-05T22:29:27.595743+0000","last_fullsized":"2024-04-05T22:29:27.595743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:25:13.015640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.604157+0000","last_change":"2024-04-05T22:29:27.104579+0000","last_active":"2024-04-05T22:29:27.604157+0000","last_peered":"2024-04-05T22:29:27.604157+0000","last_clean":"2024-04-05T22:29:27.604157+0000","last_became_active":"2024-04-05T22:29:27.104293+0000","last_became_peered":"2024-04-05T22:29:27.104293+0000","last_unstale":"2024-04-05T22:29:27.604157+0000","last_undegraded":"2024-04-05T22:29:27.604157+0000","last_fullsized":"2024-04-05T22:29:27.604157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:13:51.613749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610310+0000","last_change":"2024-04-05T22:29:27.606580+0000","last_active":"2024-04-05T22:29:27.610310+0000","last_peered":"2024-04-05T22:29:27.610310+0000","last_clean":"2024-04-05T22:29:27.610310+0000","last_became_active":"2024-04-05T22:29:27.606239+0000","last_became_peered":"2024-04-05T22:29:27.606239+0000","last_unstale":"2024-04-05T22:29:27.610310+0000","last_undegraded":"2024-04-05T22:29:27.610310+0000","last_fullsized":"2024-04-05T22:29:27.610310+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:05:47.512977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600620+0000","last_change":"2024-04-05T22:29:27.115301+0000","last_active":"2024-04-05T22:29:27.600620+0000","last_peered":"2024-04-05T22:29:27.600620+0000","last_clean":"2024-04-05T22:29:27.600620+0000","last_became_active":"2024-04-05T22:29:27.114776+0000","last_became_peered":"2024-04-05T22:29:27.114776+0000","last_unstale":"2024-04-05T22:29:27.600620+0000","last_undegraded":"2024-04-05T22:29:27.600620+0000","last_fullsized":"2024-04-05T22:29:27.600620+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:59:53.380117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.607801+0000","last_change":"2024-04-05T22:29:27.605387+0000","last_active":"2024-04-05T22:29:27.607801+0000","last_peered":"2024-04-05T22:29:27.607801+0000","last_clean":"2024-04-05T22:29:27.607801+0000","last_became_active":"2024-04-05T22:29:27.605136+0000","last_became_peered":"2024-04-05T22:29:27.605136+0000","last_unstale":"2024-04-05T22:29:27.607801+0000","last_undegraded":"2024-04-05T22:29:27.607801+0000","last_fullsized":"2024-04-05T22:29:27.607801+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:52:51.751427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-05T22:29:28.567987+0000","last_change":"2024-04-05T22:29:27.105399+0000","last_active":"2024-04-05T22:29:28.567987+0000","last_peered":"2024-04-05T22:29:28.567987+0000","last_clean":"2024-04-05T22:29:28.567987+0000","last_became_active":"2024-04-05T22:29:27.105109+0000","last_became_peered":"2024-04-05T22:29:27.105109+0000","last_unstale":"2024-04-05T22:29:28.567987+0000","last_undegraded":"2024-04-05T22:29:28.567987+0000","last_fullsized":"2024-04-05T22:29:28.567987+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:09:28.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606646+0000","last_change":"2024-04-05T22:29:27.605029+0000","last_active":"2024-04-05T22:29:27.606646+0000","last_peered":"2024-04-05T22:29:27.606646+0000","last_clean":"2024-04-05T22:29:27.606646+0000","last_became_active":"2024-04-05T22:29:27.604719+0000","last_became_peered":"2024-04-05T22:29:27.604719+0000","last_unstale":"2024-04-05T22:29:27.606646+0000","last_undegraded":"2024-04-05T22:29:27.606646+0000","last_fullsized":"2024-04-05T22:29:27.606646+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:10.981726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.571145+0000","last_change":"2024-04-05T22:29:27.094375+0000","last_active":"2024-04-05T22:29:27.571145+0000","last_peered":"2024-04-05T22:29:27.571145+0000","last_clean":"2024-04-05T22:29:27.571145+0000","last_became_active":"2024-04-05T22:29:27.094043+0000","last_became_peered":"2024-04-05T22:29:27.094043+0000","last_unstale":"2024-04-05T22:29:27.571145+0000","last_undegraded":"2024-04-05T22:29:27.571145+0000","last_fullsized":"2024-04-05T22:29:27.571145+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:56:36.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.603753+0000","last_change":"2024-04-05T22:29:27.601202+0000","last_active":"2024-04-05T22:29:27.603753+0000","last_peered":"2024-04-05T22:29:27.603753+0000","last_clean":"2024-04-05T22:29:27.603753+0000","last_became_active":"2024-04-05T22:29:27.600373+0000","last_became_peered":"2024-04-05T22:29:27.600373+0000","last_unstale":"2024-04-05T22:29:27.603753+0000","last_undegraded":"2024-04-05T22:29:27.603753+0000","last_fullsized":"2024-04-05T22:29:27.603753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:07.454547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.570652+0000","last_change":"2024-04-05T22:29:27.092419+0000","last_active":"2024-04-05T22:29:27.570652+0000","last_peered":"2024-04-05T22:29:27.570652+0000","last_clean":"2024-04-05T22:29:27.570652+0000","last_became_active":"2024-04-05T22:29:27.091178+0000","last_became_peered":"2024-04-05T22:29:27.091178+0000","last_unstale":"2024-04-05T22:29:27.570652+0000","last_undegraded":"2024-04-05T22:29:27.570652+0000","last_fullsized":"2024-04-05T22:29:27.570652+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:31.397216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.596367+0000","last_change":"2024-04-05T22:29:27.593941+0000","last_active":"2024-04-05T22:29:27.596367+0000","last_peered":"2024-04-05T22:29:27.596367+0000","last_clean":"2024-04-05T22:29:27.596367+0000","last_became_active":"2024-04-05T22:29:27.593510+0000","last_became_peered":"2024-04-05T22:29:27.593510+0000","last_unstale":"2024-04-05T22:29:27.596367+0000","last_undegraded":"2024-04-05T22:29:27.596367+0000","last_fullsized":"2024-04-05T22:29:27.596367+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:32.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.572196+0000","last_change":"2024-04-05T22:29:27.108317+0000","last_active":"2024-04-05T22:29:27.572196+0000","last_peered":"2024-04-05T22:29:27.572196+0000","last_clean":"2024-04-05T22:29:27.572196+0000","last_became_active":"2024-04-05T22:29:27.107888+0000","last_became_peered":"2024-04-05T22:29:27.107888+0000","last_unstale":"2024-04-05T22:29:27.572196+0000","last_undegraded":"2024-04-05T22:29:27.572196+0000","last_fullsized":"2024-04-05T22:29:27.572196+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:42.291460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610341+0000","last_change":"2024-04-05T22:29:27.607201+0000","last_active":"2024-04-05T22:29:27.610341+0000","last_peered":"2024-04-05T22:29:27.610341+0000","last_clean":"2024-04-05T22:29:27.610341+0000","last_became_active":"2024-04-05T22:29:27.606880+0000","last_became_peered":"2024-04-05T22:29:27.606880+0000","last_unstale":"2024-04-05T22:29:27.610341+0000","last_undegraded":"2024-04-05T22:29:27.610341+0000","last_fullsized":"2024-04-05T22:29:27.610341+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:40:57.622525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567876+0000","last_change":"2024-04-05T22:29:27.114605+0000","last_active":"2024-04-05T22:29:27.567876+0000","last_peered":"2024-04-05T22:29:27.567876+0000","last_clean":"2024-04-05T22:29:27.567876+0000","last_became_active":"2024-04-05T22:29:27.114311+0000","last_became_peered":"2024-04-05T22:29:27.114311+0000","last_unstale":"2024-04-05T22:29:27.567876+0000","last_undegraded":"2024-04-05T22:29:27.567876+0000","last_fullsized":"2024-04-05T22:29:27.567876+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:36:06.440778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.590111+0000","last_change":"2024-04-05T22:29:27.582598+0000","last_active":"2024-04-05T22:29:27.590111+0000","last_peered":"2024-04-05T22:29:27.590111+0000","last_clean":"2024-04-05T22:29:27.590111+0000","last_became_active":"2024-04-05T22:29:27.579892+0000","last_became_peered":"2024-04-05T22:29:27.579892+0000","last_unstale":"2024-04-05T22:29:27.590111+0000","last_undegraded":"2024-04-05T22:29:27.590111+0000","last_fullsized":"2024-04-05T22:29:27.590111+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:04:50.972985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.600538+0000","last_change":"2024-04-05T22:29:27.109061+0000","last_active":"2024-04-05T22:29:27.600538+0000","last_peered":"2024-04-05T22:29:27.600538+0000","last_clean":"2024-04-05T22:29:27.600538+0000","last_became_active":"2024-04-05T22:29:27.108742+0000","last_became_peered":"2024-04-05T22:29:27.108742+0000","last_unstale":"2024-04-05T22:29:27.600538+0000","last_undegraded":"2024-04-05T22:29:27.600538+0000","last_fullsized":"2024-04-05T22:29:27.600538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:26:35.912596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.599871+0000","last_change":"2024-04-05T22:29:27.595437+0000","last_active":"2024-04-05T22:29:27.599871+0000","last_peered":"2024-04-05T22:29:27.599871+0000","last_clean":"2024-04-05T22:29:27.599871+0000","last_became_active":"2024-04-05T22:29:27.594151+0000","last_became_peered":"2024-04-05T22:29:27.594151+0000","last_unstale":"2024-04-05T22:29:27.599871+0000","last_undegraded":"2024-04-05T22:29:27.599871+0000","last_fullsized":"2024-04-05T22:29:27.599871+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:08:58.603398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.561663+0000","last_change":"2024-04-05T22:29:27.115510+0000","last_active":"2024-04-05T22:29:27.561663+0000","last_peered":"2024-04-05T22:29:27.561663+0000","last_clean":"2024-04-05T22:29:27.561663+0000","last_became_active":"2024-04-05T22:29:27.115162+0000","last_became_peered":"2024-04-05T22:29:27.115162+0000","last_unstale":"2024-04-05T22:29:27.561663+0000","last_undegraded":"2024-04-05T22:29:27.561663+0000","last_fullsized":"2024-04-05T22:29:27.561663+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:30:44.332750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.606977+0000","last_change":"2024-04-05T22:29:27.606046+0000","last_active":"2024-04-05T22:29:27.606977+0000","last_peered":"2024-04-05T22:29:27.606977+0000","last_clean":"2024-04-05T22:29:27.606977+0000","last_became_active":"2024-04-05T22:29:27.605661+0000","last_became_peered":"2024-04-05T22:29:27.605661+0000","last_unstale":"2024-04-05T22:29:27.606977+0000","last_undegraded":"2024-04-05T22:29:27.606977+0000","last_fullsized":"2024-04-05T22:29:27.606977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:27.066361+0000","last_clean_scrub_stamp":"2024-04-05T22:29:27.066361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:20.485876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.610812+0000","last_change":"2024-04-05T22:29:27.107244+0000","last_active":"2024-04-05T22:29:27.610812+0000","last_peered":"2024-04-05T22:29:27.610812+0000","last_clean":"2024-04-05T22:29:27.610812+0000","last_became_active":"2024-04-05T22:29:27.106979+0000","last_became_peered":"2024-04-05T22:29:27.106979+0000","last_unstale":"2024-04-05T22:29:27.610812+0000","last_undegraded":"2024-04-05T22:29:27.610812+0000","last_fullsized":"2024-04-05T22:29:27.610812+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:29:26.052592+0000","last_clean_scrub_stamp":"2024-04-05T22:29:26.052592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:28:57.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":301,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-05T22:29:27.567658+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:29:27.567658+0000","last_peered":"2024-04-05T22:29:27.567658+0000","last_clean":"2024-04-05T22:29:27.567658+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:29:27.567658+0000","last_undegraded":"2024-04-05T22:29:27.567658+0000","last_fullsized":"2024-04-05T22:29:27.567658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542248,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2301952,"data_stored":1341744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.097},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.26}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.88},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.492},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.175}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.583}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.24},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.3899999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542249,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1712128,"data_stored":750870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.006}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":0.87},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":0.96499999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.333},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.2170000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.2370000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.284}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1703936,"data_stored":750746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.1799999999999999,"15min":1.0580000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":7.2329999999999997,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.458,"5min":1.2,"15min":1.1220000000000001},"min":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2330000000000001,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.2770000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.089},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.4039999999999999,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":17.381,"5min":17.381,"15min":17.381},"last":1.0600000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.4399999999999999,"15min":1.2529999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":24.52,"5min":24.52,"15min":24.52},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.3109999999999999,"15min":1.2729999999999999},"min":{"1min":0.99199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":7.5179999999999998,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.381},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.329,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.4000000000000004,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.149}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.429,"15min":1.321},"min":{"1min":0.96399999999999997,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":14.933,"5min":14.933,"15min":14.933},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.85499999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.413,"15min":1.355},"min":{"1min":1.0289999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.1959999999999997,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.24},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.4079999999999999,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":3.782,"5min":3.782,"15min":3.782},"last":1.716}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":1.5389999999999999,"15min":1.4790000000000001},"min":{"1min":1.2509999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.78,"5min":1.5580000000000001,"15min":1.6040000000000001},"min":{"1min":1.278,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":5.1429999999999998,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.607}]}]},{"osd":4,"up_from":14,"seq":60129542247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1699840,"data_stored":750724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1020000000000001,"15min":1.0289999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.59399999999999997},"max":{"1min":1.407,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0880000000000001},"min":{"1min":0.752,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5309999999999999,"5min":1.859,"15min":1.859},"last":1.288}]},{"osd":1,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1459999999999999,"15min":1.131},"min":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.728,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.51},{"interface":"front","average":{"1min":1.286,"5min":1.2150000000000001,"15min":1.181},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.645,"5min":2.577,"15min":2.577},"last":1.4239999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6100000000000001,"5min":1.6579999999999999,"15min":1.974},"last":1.476},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2490000000000001,"15min":1.2050000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.67400000000000004},"max":{"1min":1.7869999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.171}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.325,"15min":1.3620000000000001},"min":{"1min":0.82899999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7689999999999999,"5min":1.843,"15min":2.1840000000000002},"last":1.593},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4079999999999999,"15min":1.4039999999999999},"min":{"1min":1.018,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.8899999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.627}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3029999999999999,"15min":1.3},"min":{"1min":0.78800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1120000000000001,"5min":2.665,"15min":2.665},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4139999999999999,"15min":1.399},"min":{"1min":0.90200000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7490000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.3999999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4550000000000001,"15min":1.458},"min":{"1min":0.85099999999999998,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":2.0800000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.472,"15min":1.4870000000000001},"min":{"1min":0.93700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.3540000000000001,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.3500000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.5680000000000001,"15min":1.5449999999999999},"min":{"1min":1.0660000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.53},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.651,"15min":1.599},"min":{"1min":1.296,"5min":0.93500000000000005,"15min":0.72399999999999998},"max":{"1min":2.2530000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.4510000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1703936,"data_stored":750814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.83899999999999997},"min":{"1min":0.53600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.302,"5min":1.4910000000000001,"15min":1.744},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5369999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.91200000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4279999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.008,"5min":0.95899999999999996,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3720000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.0549999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.028,"15min":1.0209999999999999},"min":{"1min":0.59699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4510000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.288},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.069,"15min":1.0800000000000001},"min":{"1min":0.66700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.956,"5min":1.956,"15min":2.2549999999999999},"last":1.956}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1719999999999999,"15min":1.222},"min":{"1min":0.76300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5509999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.248,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7649999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.3779999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2809999999999999,"15min":1.343},"min":{"1min":0.84299999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.71,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.639},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3029999999999999,"15min":1.294},"min":{"1min":0.98199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.895,"5min":1.958,"15min":2.3239999999999998},"last":1.7470000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.79200000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.91400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7,"5min":2.298,"15min":2.298},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.341,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.899,"5min":1.9410000000000001,"15min":2.468},"last":1.899},{"interface":"front","average":{"1min":1.343,"5min":1.3620000000000001,"15min":1.4079999999999999},"min":{"1min":0.92100000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8160000000000001,"5min":2.508,"15min":2.798},"last":1.5980000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1716224,"data_stored":752358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.433}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.0509999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.077}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.0289999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1703936,"data_stored":751306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3680000000000001,"5min":1.379,"15min":1.6000000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86499999999999999,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1779999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.752},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97999999999999998,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4570000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.139}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.054,"15min":1.095},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.75},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0149999999999999,"15min":0.97599999999999998},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.4530000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.171,"15min":1.141},"min":{"1min":0.84199999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.716,"5min":1.8500000000000001,"15min":1.855},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.26,"5min":1.28,"15min":1.2809999999999999},"min":{"1min":0.94499999999999995,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.575,"5min":1.806,"15min":2.2789999999999999},"last":1.367}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.8200000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.004},{"interface":"front","average":{"1min":1.323,"5min":1.2649999999999999,"15min":1.2410000000000001},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.173},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.367,"15min":1.341},"min":{"1min":0.88900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.675,"5min":1.9059999999999999,"15min":1.917},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.357,"15min":1.3560000000000001},"min":{"1min":0.86899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.405,"15min":1.369},"min":{"1min":0.95899999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0259999999999998,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.208},{"interface":"front","average":{"1min":1.383,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.2829999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2310144,"data_stored":1342690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.4339999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.90900000000000003,"15min":0.88600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.8680000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.54,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95499999999999996,"15min":0.96399999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.6639999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.244,"5min":1.976,"15min":1.976},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0580000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.581,"5min":2.157,"15min":2.157},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1919999999999999,"15min":1.1719999999999999},"min":{"1min":0.80400000000000005,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.5429999999999999,"5min":1.931,"15min":1.931},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.298,"15min":1.3},"min":{"1min":0.75900000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7569999999999999,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.181}]},{"osd":5,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2450000000000001,"15min":1.2609999999999999},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0739999999999998,"5min":2.105,"15min":2.3879999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2130000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.7010000000000001,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.3740000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.7470000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.3640000000000001},"min":{"1min":0.90100000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8720000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.3080000000000001},"min":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.9810000000000001},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:29:37.527 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:29:37.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:29:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.784+0000 7f27ed200640 1 Processor -- start 2024-04-05T22:29:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.784+0000 7f27ed200640 1 -- start start 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27ed200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27e80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27ed200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80b8c80 con 0x7f27e815ce20 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27ed200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80d3b50 con 0x7f27e815d200 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27ed200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b67f0 con 0x7f27e80d0480 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e5e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f27e815ce20 0x7f27e80cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36520/0 (socket says 172.21.15.73:36520) 2024-04-05T22:29:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e5e00640 1 -- 172.21.15.73:0/2179193664 learned_addr learned my addr 172.21.15.73:0/2179193664 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e5400640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27e80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538697185 0 0) 0x7f27e80b8c80 con 0x7f27e815ce20 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc003520 con 0x7f27e815ce20 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 950479062 0 0) 0x7f27cc003520 con 0x7f27e815ce20 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 msgr2=0x7f27e80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27e80d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 >> v1:172.21.15.157:6789/0 conn(0x7f27e815d200 legacy=0x7f27e80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80b5c20 con 0x7f27e815ce20 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f27d0001670 con 0x7f27e815ce20 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 >> v1:172.21.15.73:6789/0 conn(0x7f27e815ce20 legacy=0x7f27e80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0039a0 0x7f27cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27cc0047c0 0x7f27cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27cc0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b67f0 con 0x7f27cc0047c0 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00ba90 con 0x7f27cc0039a0 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.786+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00bc50 con 0x7f27e80d0480 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27cc0047c0 0x7f27cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5e00640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0039a0 0x7f27cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e6800640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27cc0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 -- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 msgr2=0x7f27cc0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e80d0480 0x7f27cc0051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 -- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0039a0 msgr2=0x7f27cc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0039a0 0x7f27cc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.787+0000 7f27e5400640 1 -- 172.21.15.73:0/2179193664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc003520 con 0x7f27cc0047c0 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.788+0000 7f27e5400640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27cc0047c0 0x7f27cc004c40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f27e808c060 tx=0x7f27cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363180a0ff224c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.788+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8033680 con 0x7f27cc0047c0 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.788+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d80337e0 con 0x7f27cc0047c0 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.788+0000 7f27e4a00640 1 -- 172.21.15.73:0/2179193664 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8033b90 con 0x7f27cc0047c0 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 -- 172.21.15.73:0/2179193664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27cc0047c0 msgr2=0x7f27cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 --2- 172.21.15.73:0/2179193664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27cc0047c0 0x7f27cc004c40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f27e808c060 tx=0x7f27cc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27e6800640 1 -- 172.21.15.73:0/2179193664 reap_dead start 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 -- 172.21.15.73:0/2179193664 shutdown_connections 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 -- 172.21.15.73:0/2179193664 >> 172.21.15.73:0/2179193664 conn(0x7f27e80d2d30 msgr2=0x7f27e80d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 -- 172.21.15.73:0/2179193664 shutdown_connections 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.789+0000 7f27ed200640 1 -- 172.21.15.73:0/2179193664 wait complete. 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.791+0000 7f27ed200640 1 Processor -- start 2024-04-05T22:29:37.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.791+0000 7f27ed200640 1 -- start start 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0047c0 0x7f27e81f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e815e740 0x7f27e815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e809fa50 con 0x7f27e809f090 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e815f400 con 0x7f27cc0047c0 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27ed200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e815f6e0 con 0x7f27e815e740 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27e5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27e5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55336/0 (socket says 172.21.15.73:55336) 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27e5400640 1 -- 172.21.15.73:0/2267832418 learned_addr learned my addr 172.21.15.73:0/2267832418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.792+0000 7f27e5e00640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0047c0 0x7f27e81f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e6800640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e815e740 0x7f27e815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 -- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e815e740 msgr2=0x7f27e815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e815e740 0x7f27e815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 -- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0047c0 msgr2=0x7f27e81f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0047c0 0x7f27e81f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 -- 172.21.15.73:0/2267832418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e815f9c0 con 0x7f27e809f090 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27e5400640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f27d802be60 tx=0x7f27d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d803c800 con 0x7f27e809f090 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8040070 con 0x7f27e809f090 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d803cb60 con 0x7f27e809f090 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.793+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e815fca0 con 0x7f27e809f090 2024-04-05T22:29:37.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.794+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e815f100 con 0x7f27e809f090 2024-04-05T22:29:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.794+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e814e010 con 0x7f27e809f090 2024-04-05T22:29:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.795+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f27d803ccc0 con 0x7f27e809f090 2024-04-05T22:29:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.796+0000 7f27de000640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 0x7f27c406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:37.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.796+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27d80abfa0 con 0x7f27e809f090 2024-04-05T22:29:37.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.797+0000 7f27e5e00640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 0x7f27c406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:37.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.798+0000 7f27e5e00640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 0x7f27c406d660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:37.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:37.801+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f27d807a360 con 0x7f27e809f090 2024-04-05T22:29:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.070+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f27e8068160 con 0x7f27e809f090 2024-04-05T22:29:38.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.080+0000 7f27de000640 1 -- 172.21.15.73:0/2267832418 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f27d8042020 con 0x7f27e809f090 2024-04-05T22:29:38.080 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 msgr2=0x7f27c406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 0x7f27c406d660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d0009040 comp rx=0 tx=0).stop 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 msgr2=0x7f27e809f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f27d802be60 tx=0x7f27d8004730 comp rx=0 tx=0).stop 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 shutdown_connections 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f27c406b1a0 0x7f27c406d660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e815e740 0x7f27e815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27e809f090 0x7f27e809f510 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 --2- 172.21.15.73:0/2267832418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27cc0047c0 0x7f27e81f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 >> 172.21.15.73:0/2267832418 conn(0x7f27e80d2d30 msgr2=0x7f27e80d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 shutdown_connections 2024-04-05T22:29:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.083+0000 7f27ed200640 1 -- 172.21.15.73:0/2267832418 wait complete. 2024-04-05T22:29:38.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:29:38.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.305+0000 7faa15400640 1 Processor -- start 2024-04-05T22:29:38.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.305+0000 7faa15400640 1 -- start start 2024-04-05T22:29:38.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa15400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa1006cb80 0x7faa1006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa15400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faa1015e850 0x7faa1015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa15400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1006f5b0 con 0x7faa101587d0 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa15400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1006d500 con 0x7faa1015e850 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa15400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1015f220 con 0x7faa1006cb80 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faa1015e850 0x7faa1015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0e800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faa101587d0 0x7faa100cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36522/0 (socket says 172.21.15.73:36522) 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0e800640 1 -- 172.21.15.73:0/824371975 learned_addr learned my addr 172.21.15.73:0/824371975 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa05e00640 1 --2- 172.21.15.73:0/824371975 >> v2:172.21.15.157:3300/0 conn(0x7faa1006cb80 0x7faa1006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0f200640 1 -- 172.21.15.73:0/824371975 >> v2:172.21.15.157:3300/0 conn(0x7faa1006cb80 msgr2=0x7faa1006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0de00640 1 -- 172.21.15.73:0/824371975 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296266162 0 0) 0x7faa1006f5b0 con 0x7faa101587d0 2024-04-05T22:29:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0f200640 1 --2- 172.21.15.73:0/824371975 >> v2:172.21.15.157:3300/0 conn(0x7faa1006cb80 0x7faa1006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0f200640 1 -- 172.21.15.73:0/824371975 >> v1:172.21.15.73:6789/0 conn(0x7faa101587d0 legacy=0x7faa100cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.307+0000 7faa0f200640 1 -- 172.21.15.73:0/824371975 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10070410 con 0x7faa1015e850 2024-04-05T22:29:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.308+0000 7faa0f200640 1 --2- 172.21.15.73:0/824371975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa1015e850 0x7faa1015ece0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7faa1006b560 tx=0x7faa0802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9559e9a7ae90e74e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa0de00640 1 -- 172.21.15.73:0/824371975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0802cc70 con 0x7faa1015e850 2024-04-05T22:29:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa0de00640 1 -- 172.21.15.73:0/824371975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0802cdd0 con 0x7faa1015e850 2024-04-05T22:29:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa0de00640 1 -- 172.21.15.73:0/824371975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08034660 con 0x7faa1015e850 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 -- 172.21.15.73:0/824371975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa1015e850 msgr2=0x7faa1015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 --2- 172.21.15.73:0/824371975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa1015e850 0x7faa1015ece0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7faa1006b560 tx=0x7faa0802bec0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 -- 172.21.15.73:0/824371975 shutdown_connections 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 --2- 172.21.15.73:0/824371975 >> v2:172.21.15.157:3300/0 conn(0x7faa1006cb80 0x7faa1006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 --2- 172.21.15.73:0/824371975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa1015e850 0x7faa1015ece0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 -- 172.21.15.73:0/824371975 >> 172.21.15.73:0/824371975 conn(0x7faa100ce8e0 msgr2=0x7faa100cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 -- 172.21.15.73:0/824371975 shutdown_connections 2024-04-05T22:29:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.309+0000 7faa15400640 1 -- 172.21.15.73:0/824371975 wait complete. 2024-04-05T22:29:38.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.311+0000 7faa15400640 1 Processor -- start 2024-04-05T22:29:38.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.311+0000 7faa15400640 1 -- start start 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1006cb80 0x7faa10155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1015e850 0x7faa10155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa101f2400 con 0x7faa101587d0 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa101599d0 con 0x7faa1015e850 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa15400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1015db20 con 0x7faa1006cb80 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa05e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa05e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55362/0 (socket says 172.21.15.73:55362) 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.312+0000 7faa05e00640 1 -- 172.21.15.73:0/1098684659 learned_addr learned my addr 172.21.15.73:0/1098684659 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 -- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1006cb80 msgr2=0x7faa10155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1006cb80 0x7faa10155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 -- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1015e850 msgr2=0x7faa10155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1015e850 0x7faa10155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 -- 172.21.15.73:0/1098684659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa08009a70 con 0x7faa101587d0 2024-04-05T22:29:38.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa05e00640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800e9e0 tx=0x7fa9f800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8009670 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800c9d0 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1015d1f0 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.313+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800cd80 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.314+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1015d7b0 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.314+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa10066af0 con 0x7faa101587d0 2024-04-05T22:29:38.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.315+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa9f8013470 con 0x7faa101587d0 2024-04-05T22:29:38.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.316+0000 7faa07400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 0x7fa9ec06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:38.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.316+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9f808a050 con 0x7faa101587d0 2024-04-05T22:29:38.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.317+0000 7faa0e800640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 0x7fa9ec06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:38.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.318+0000 7faa0e800640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 0x7fa9ec06d690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa00007fd0 tx=0x7faa00007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:38.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.321+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9f8054980 con 0x7faa101587d0 2024-04-05T22:29:38.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.555+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa10068160 con 0x7faa101587d0 2024-04-05T22:29:38.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.557+0000 7faa07400640 1 -- 172.21.15.73:0/1098684659 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fa9f8017090 con 0x7faa101587d0 2024-04-05T22:29:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:38.557 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:29:38.557 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 82 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 msgr2=0x7fa9ec06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 0x7fa9ec06d690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa00007fd0 tx=0x7faa00007920 comp rx=0 tx=0).stop 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 msgr2=0x7faa10155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800e9e0 tx=0x7fa9f800eea0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 shutdown_connections 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9ec06b1d0 0x7fa9ec06d690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1015e850 0x7faa10155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa101587d0 0x7faa10155580 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 --2- 172.21.15.73:0/1098684659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1006cb80 0x7faa10155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:38.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 >> 172.21.15.73:0/1098684659 conn(0x7faa100ce8e0 msgr2=0x7faa100cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:38.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 shutdown_connections 2024-04-05T22:29:38.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:38.560+0000 7faa15400640 1 -- 172.21.15.73:0/1098684659 wait complete. 2024-04-05T22:29:38.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:29:27.554050+0000', 'modified': '2024-04-05T22:29:30.583853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8685}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8685': {'gid': 8685, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6835/1637066621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6833', 'nonce': 1637066621}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 1637066621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:29:38.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:29:38.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-05T22:29:39.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.195+0000 7f042fc00640 1 Processor -- start 2024-04-05T22:29:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.195+0000 7f042fc00640 1 -- start start 2024-04-05T22:29:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.197+0000 7f042fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.197+0000 7f042fc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0428070c00 0x7f042815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.197+0000 7f042fc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042806cf30 con 0x7f042815de40 2024-04-05T22:29:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.197+0000 7f042fc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0428071080 con 0x7f0428070c00 2024-04-05T22:29:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.197+0000 7f042fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04280cc550 con 0x7f04280cc170 2024-04-05T22:29:39.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58488/0 (socket says 172.21.15.73:58488) 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 -- 172.21.15.73:0/3739754915 learned_addr learned my addr 172.21.15.73:0/3739754915 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f0427e00640 1 --2- 172.21.15.73:0/3739754915 >> v2:172.21.15.157:3300/0 conn(0x7f0428070c00 0x7f042815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 -- 172.21.15.73:0/3739754915 >> v2:172.21.15.157:3300/0 conn(0x7f0428070c00 msgr2=0x7f042815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 --2- 172.21.15.73:0/3739754915 >> v2:172.21.15.157:3300/0 conn(0x7f0428070c00 0x7f042815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.198+0000 7f042d200640 1 -- 172.21.15.73:0/3739754915 >> v1:172.21.15.157:6789/0 conn(0x7f042815de40 legacy=0x7f04280b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.199+0000 7f042d200640 1 -- 172.21.15.73:0/3739754915 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042806c600 con 0x7f04280cc170 2024-04-05T22:29:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.200+0000 7f042d200640 1 --2- 172.21.15.73:0/3739754915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f041800bb10 tx=0x7f041802e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68994058283b54f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.200+0000 7f0427400640 1 -- 172.21.15.73:0/3739754915 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418036440 con 0x7f04280cc170 2024-04-05T22:29:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.200+0000 7f0427400640 1 -- 172.21.15.73:0/3739754915 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04180365a0 con 0x7f04280cc170 2024-04-05T22:29:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.200+0000 7f0427400640 1 -- 172.21.15.73:0/3739754915 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418036930 con 0x7f04280cc170 2024-04-05T22:29:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 -- 172.21.15.73:0/3739754915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 msgr2=0x7f04280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 --2- 172.21.15.73:0/3739754915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f041800bb10 tx=0x7f041802e150 comp rx=0 tx=0).stop 2024-04-05T22:29:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 -- 172.21.15.73:0/3739754915 shutdown_connections 2024-04-05T22:29:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 --2- 172.21.15.73:0/3739754915 >> v2:172.21.15.157:3300/0 conn(0x7f0428070c00 0x7f042815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 --2- 172.21.15.73:0/3739754915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04280cc170 0x7f04280706c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 -- 172.21.15.73:0/3739754915 >> 172.21.15.73:0/3739754915 conn(0x7f04280b16d0 msgr2=0x7f04280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 -- 172.21.15.73:0/3739754915 shutdown_connections 2024-04-05T22:29:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.201+0000 7f042fc00640 1 -- 172.21.15.73:0/3739754915 wait complete. 2024-04-05T22:29:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.203+0000 7f042fc00640 1 Processor -- start 2024-04-05T22:29:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.204+0000 7f042fc00640 1 -- start start 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0428070c00 0x7f042814af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f042815de40 0x7f042815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042814b440 con 0x7f04280cc170 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042815b3f0 con 0x7f0428070c00 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042814c4f0 con 0x7f042815de40 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55364/0 (socket says 172.21.15.73:55364) 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f0427e00640 1 -- 172.21.15.73:0/1982086352 learned_addr learned my addr 172.21.15.73:0/1982086352 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042d200640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0428070c00 0x7f042814af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.205+0000 7f042dc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f042815de40 0x7f042815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 -- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f042815de40 msgr2=0x7f042815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f042815de40 0x7f042815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 -- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0428070c00 msgr2=0x7f042814af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0428070c00 0x7f042814af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 -- 172.21.15.73:0/1982086352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041800b770 con 0x7f04280cc170 2024-04-05T22:29:39.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0427e00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f041000e8a0 tx=0x7f041000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0424c00640 1 -- 172.21.15.73:0/1982086352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100094f0 con 0x7f04280cc170 2024-04-05T22:29:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0424c00640 1 -- 172.21.15.73:0/1982086352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000c940 con 0x7f04280cc170 2024-04-05T22:29:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f0424c00640 1 -- 172.21.15.73:0/1982086352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000ccf0 con 0x7f04280cc170 2024-04-05T22:29:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042814c790 con 0x7f04280cc170 2024-04-05T22:29:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.206+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042815b920 con 0x7f04280cc170 2024-04-05T22:29:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.210+0000 7f0424c00640 1 -- 172.21.15.73:0/1982086352 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f0410013470 con 0x7f04280cc170 2024-04-05T22:29:39.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.211+0000 7f0424c00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 0x7f040406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.211+0000 7f0424c00640 1 -- 172.21.15.73:0/1982086352 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0410089e80 con 0x7f04280cc170 2024-04-05T22:29:39.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.211+0000 7f04329ae740 1 Processor -- start 2024-04-05T22:29:39.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.212+0000 7f042d200640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 0x7f040406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.212+0000 7f04329ae740 1 -- start start 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.212+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618435f71f0 0x5618435ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f04329ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435ca6d0 0x5618435fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f04329ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184353a8c0 con 0x5618435f75d0 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561843538050 con 0x5618435f71f0 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x561843533940 con 0x5618435ca6d0 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55376/0 (socket says 172.21.15.73:55376) 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 -- 172.21.15.73:0/3199054238 learned_addr learned my addr 172.21.15.73:0/3199054238 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f042d200640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 0x7f040406d660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0418038000 tx=0x7f0418032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f042dc00640 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435ca6d0 0x5618435fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f042d200640 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618435f71f0 0x5618435ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 -- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435ca6d0 msgr2=0x5618435fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435ca6d0 0x5618435fa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 -- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618435f71f0 msgr2=0x5618435ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618435f71f0 0x5618435ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.213+0000 7f0427e00640 1 -- 172.21.15.73:0/3199054238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618434fe9e0 con 0x5618435f75d0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f0427e00640 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0410058170 tx=0x7f04100548e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a256b60dcc3c3c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f041d600640 1 -- 172.21.15.73:0/3199054238 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100879d0 con 0x5618435f75d0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f041d600640 1 -- 172.21.15.73:0/3199054238 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410060860 con 0x5618435f75d0 2024-04-05T22:29:39.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f041d600640 1 -- 172.21.15.73:0/3199054238 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000ce50 con 0x5618435f75d0 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 -- 172.21.15.73:0/3199054238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 msgr2=0x5618435f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0410058170 tx=0x7f04100548e0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 -- 172.21.15.73:0/3199054238 shutdown_connections 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435ca6d0 0x5618435fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x5618435f9f80 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 --2- 172.21.15.73:0/3199054238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618435f71f0 0x5618435ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.214+0000 7f04329ae740 1 -- 172.21.15.73:0/3199054238 >> 172.21.15.73:0/3199054238 conn(0x5618435cc400 msgr2=0x561843614f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.215+0000 7f04329ae740 1 -- 172.21.15.73:0/3199054238 shutdown_connections 2024-04-05T22:29:39.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.215+0000 7f04329ae740 1 -- 172.21.15.73:0/3199054238 wait complete. 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.215+0000 7f04329ae740 1 -- start start 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.215+0000 7f04329ae740 1 Processor -- start 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.216+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x5618435eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.216+0000 7f04329ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.216+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x56184360e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f04329ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618434d5070 con 0x5618435f75d0 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618434d4d10 con 0x56184360dc40 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56184329af20 con 0x5618435f71f0 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f04329ae740 10 client.0 _ll_register_callbacks cb 0x56184352cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:29:39.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55388/0 (socket says 172.21.15.73:55388) 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 -- 172.21.15.73:0/4105383073 learned_addr learned my addr 172.21.15.73:0/4105383073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 -- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 msgr2=0x5618435eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x5618435eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 -- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 msgr2=0x56184360e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x56184360e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 -- 172.21.15.73:0/4105383073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042815b920 con 0x5618435f75d0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0427e00640 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0410054970 tx=0x7f0410009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbdce082f4ca990 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.217+0000 7f0416000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000ce50 con 0x5618435f75d0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f041005e230 con 0x5618435f75d0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56184355dae0 con 0x5618435f75d0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f041005e3f0 con 0x5618435f75d0 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 10 client.8856 fetch_fsmap learned FSMap version 82 2024-04-05T22:29:39.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x561843531fc0 con 0x5618435f75d0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56184355dae0 con 0x5618435f75d0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f041007a060 con 0x5618435f75d0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 10 client.8856 fetch_fsmap finished waiting for FSMap version 82 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 10 client.8856 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 20 client.8856 populate_metadata read hostname 'smithi073' 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x561843624e10 0x5618435fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 4 client.8856 mds_command: new command op to 8685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.218+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 --> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618432273c0 con 0x561843624e10 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.219+0000 7f042d200640 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x561843624e10 0x5618435fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.219+0000 7f0416000640 10 client.8856 ms_handle_connect on v2:172.21.15.73:6833/1637066621 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.219+0000 7f042d200640 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x561843624e10 0x5618435fbfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0dee34d0ec1d40e server_cookie=d47bcaebc4a2d6aa in_seq=0 out_seq=0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f0416000640 1 -- 172.21.15.73:0/4105383073 <== mds.0 v2:172.21.15.73:6833/1637066621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5618432273c0 con 0x561843624e10 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f0416000640 10 client.8856 handle_command_reply: tid=0 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 client.8856 shutdown 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 msgr2=0x56184360c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0410054970 tx=0x7f0410009f20 comp rx=0 tx=0).stop 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 shutdown_connections 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x561843624e10 0x5618435fbfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x56184360e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360c0e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f0416000640 0 client.8856 ms_handle_reset on v2:172.21.15.73:6833/1637066621 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 --2- 172.21.15.73:0/4105383073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x5618435eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 >> 172.21.15.73:0/4105383073 conn(0x5618435ca6d0 msgr2=0x5618435f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 shutdown_connections 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.220+0000 7f04329ae740 1 -- 172.21.15.73:0/4105383073 wait complete. 2024-04-05T22:29:39.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.221+0000 7f04329ae740 20 client.8856 trim_cache size 0 max 16384 2024-04-05T22:29:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.237+0000 7f04329ae740 1 Processor -- start 2024-04-05T22:29:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.237+0000 7f04329ae740 1 -- start start 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x56184360e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561843625560 con 0x5618435f75d0 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618435c9a20 con 0x56184360dc40 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618435cd990 con 0x5618435f71f0 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f0427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55390/0 (socket says 172.21.15.73:55390) 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.238+0000 7f0427e00640 1 -- 172.21.15.73:0/3357285828 learned_addr learned my addr 172.21.15.73:0/3357285828 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 -- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 msgr2=0x56184360e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x56184360e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 -- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 msgr2=0x5618435e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 -- 172.21.15.73:0/3357285828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561843566f80 con 0x5618435f75d0 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f0427e00640 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f0410058630 tx=0x7f0410083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142d4318fe8faf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f041e000640 1 -- 172.21.15.73:0/3357285828 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000e070 con 0x5618435f75d0 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f041e000640 1 -- 172.21.15.73:0/3357285828 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410057fb0 con 0x5618435f75d0 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.239+0000 7f041e000640 1 -- 172.21.15.73:0/3357285828 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410054480 con 0x5618435f75d0 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 -- 172.21.15.73:0/3357285828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 msgr2=0x56184360e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f0410058630 tx=0x7f0410083040 comp rx=0 tx=0).stop 2024-04-05T22:29:39.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 -- 172.21.15.73:0/3357285828 shutdown_connections 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f75d0 0x56184360e5e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 --2- 172.21.15.73:0/3357285828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f71f0 0x56184360e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 -- 172.21.15.73:0/3357285828 >> 172.21.15.73:0/3357285828 conn(0x5618435ca6d0 msgr2=0x5618435c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 -- 172.21.15.73:0/3357285828 shutdown_connections 2024-04-05T22:29:39.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.240+0000 7f04329ae740 1 -- 172.21.15.73:0/3357285828 wait complete. 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.241+0000 7f04329ae740 1 -- start start 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.241+0000 7f04329ae740 1 Processor -- start 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f75d0 0x5618435cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618434fa100 con 0x5618435f71f0 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618435f66f0 con 0x56184360dc40 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56184360e8f0 con 0x5618435f75d0 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f04329ae740 10 client.0 _ll_register_callbacks cb 0x7f0410057e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55394/0 (socket says 172.21.15.73:55394) 2024-04-05T22:29:39.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 -- 172.21.15.73:0/3172953597 learned_addr learned my addr 172.21.15.73:0/3172953597 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 -- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f75d0 msgr2=0x5618435cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f75d0 0x5618435cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 -- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 msgr2=0x5618435ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.242+0000 7f042d200640 1 -- 172.21.15.73:0/3172953597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561843531fc0 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f042d200640 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0418043040 tx=0x7f0418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9813674be3f060 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180542d0 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0418054430 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561843610750 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0418054820 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f04329ae740 10 client.8862 fetch_fsmap learned FSMap version 82 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5618434d4280 con 0x5618435f71f0 2024-04-05T22:29:39.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561843610750 con 0x5618435f71f0 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.243+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0418048040 con 0x5618435f71f0 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 10 client.8862 fetch_fsmap finished waiting for FSMap version 82 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 10 client.8862 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 20 client.8862 populate_metadata read hostname 'smithi073' 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x56184360ac40 0x56184360b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 4 client.8862 mds_command: new command op to 8685 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 --> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0410060860 con 0x56184360ac40 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f0427e00640 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x56184360ac40 0x56184360b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f0427e00640 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x56184360ac40 0x56184360b020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc8182a8c367296 server_cookie=9c1db264d2e1abe0 in_seq=0 out_seq=0 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.244+0000 7f0416000640 10 client.8862 ms_handle_connect on v2:172.21.15.73:6833/1637066621 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f0416000640 1 -- 172.21.15.73:0/3172953597 <== mds.0 v2:172.21.15.73:6833/1637066621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f0410060860 con 0x56184360ac40 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f0416000640 10 client.8862 handle_command_reply: tid=0 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 client.8862 shutdown 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 msgr2=0x561843676740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0418043040 tx=0x7f0418004730 comp rx=0 tx=0).stop 2024-04-05T22:29:39.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 shutdown_connections 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:6833/1637066621,v1:172.21.15.73:6835/1637066621] conn(0x56184360ac40 0x56184360b020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56184360dc40 0x5618435ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5618435f75d0 0x5618435cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 --2- 172.21.15.73:0/3172953597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5618435f71f0 0x561843676740 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f0416000640 0 client.8862 ms_handle_reset on v2:172.21.15.73:6833/1637066621 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 >> 172.21.15.73:0/3172953597 conn(0x5618435ca6d0 msgr2=0x561843618a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 shutdown_connections 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 1 -- 172.21.15.73:0/3172953597 wait complete. 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.245+0000 7f04329ae740 20 client.8862 trim_cache size 0 max 16384 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:29:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 8685, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 11.686520723999999, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 82, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 136, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 136, 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 17.010341569000001 2024-04-05T22:29:39.250 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:29:39.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 msgr2=0x7f040406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 0x7f040406d660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0418038000 tx=0x7f0418032000 comp rx=0 tx=0).stop 2024-04-05T22:29:39.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 msgr2=0x7f042814bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f041000e8a0 tx=0x7f041000ed60 comp rx=0 tx=0).stop 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 shutdown_connections 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f040406b1a0 0x7f040406d660 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f042815de40 0x7f042815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04280cc170 0x7f042814bfb0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 --2- 172.21.15.73:0/1982086352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0428070c00 0x7f042814af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 >> 172.21.15.73:0/1982086352 conn(0x7f04280b16d0 msgr2=0x7f04280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 shutdown_connections 2024-04-05T22:29:39.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.246+0000 7f042fc00640 1 -- 172.21.15.73:0/1982086352 wait complete. 2024-04-05T22:29:39.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:29:39.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.443+0000 7fe609400640 1 Processor -- start 2024-04-05T22:29:39.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.444+0000 7fe609400640 1 -- start start 2024-04-05T22:29:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.445+0000 7fe609400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6040b1870 0x7fe6040ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.445+0000 7fe609400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.445+0000 7fe609400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6040ce5d0 con 0x7fe604160620 2024-04-05T22:29:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.445+0000 7fe609400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6040ce010 con 0x7fe6040b1870 2024-04-05T22:29:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.445+0000 7fe609400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040ce2f0 con 0x7fe6040ad6b0 2024-04-05T22:29:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.446+0000 7fe603200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.446+0000 7fe603200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58578/0 (socket says 172.21.15.73:58578) 2024-04-05T22:29:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.446+0000 7fe603200640 1 -- 172.21.15.73:0/3310922322 learned_addr learned my addr 172.21.15.73:0/3310922322 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.446+0000 7fe5f9e00640 1 --2- 172.21.15.73:0/3310922322 >> v2:172.21.15.157:3300/0 conn(0x7fe6040b1870 0x7fe6040ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.447+0000 7fe603200640 1 -- 172.21.15.73:0/3310922322 >> v2:172.21.15.157:3300/0 conn(0x7fe6040b1870 msgr2=0x7fe6040ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:39.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.447+0000 7fe603200640 1 --2- 172.21.15.73:0/3310922322 >> v2:172.21.15.157:3300/0 conn(0x7fe6040b1870 0x7fe6040ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.447+0000 7fe603200640 1 -- 172.21.15.73:0/3310922322 >> v1:172.21.15.157:6789/0 conn(0x7fe604160620 legacy=0x7fe6040caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.447+0000 7fe603200640 1 -- 172.21.15.73:0/3310922322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040ac970 con 0x7fe6040ad6b0 2024-04-05T22:29:39.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.447+0000 7fe601e00640 1 -- 172.21.15.73:0/3310922322 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658290729 0 0) 0x7fe6040ce5d0 con 0x7fe604160620 2024-04-05T22:29:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.448+0000 7fe603200640 1 --2- 172.21.15.73:0/3310922322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009e50 tx=0x7fe5fc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79aa15b93449f92a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.448+0000 7fe601e00640 1 -- 172.21.15.73:0/3310922322 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc02cb00 con 0x7fe6040ad6b0 2024-04-05T22:29:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.448+0000 7fe601e00640 1 -- 172.21.15.73:0/3310922322 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc02cc60 con 0x7fe6040ad6b0 2024-04-05T22:29:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.448+0000 7fe601e00640 1 -- 172.21.15.73:0/3310922322 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0343f0 con 0x7fe6040ad6b0 2024-04-05T22:29:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 -- 172.21.15.73:0/3310922322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 msgr2=0x7fe6040cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 --2- 172.21.15.73:0/3310922322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009e50 tx=0x7fe5fc02bf60 comp rx=0 tx=0).stop 2024-04-05T22:29:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 -- 172.21.15.73:0/3310922322 shutdown_connections 2024-04-05T22:29:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 --2- 172.21.15.73:0/3310922322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040ad6b0 0x7fe6040cb2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 --2- 172.21.15.73:0/3310922322 >> v2:172.21.15.157:3300/0 conn(0x7fe6040b1870 0x7fe6040ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 -- 172.21.15.73:0/3310922322 >> 172.21.15.73:0/3310922322 conn(0x7fe6040ce8e0 msgr2=0x7fe6040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 -- 172.21.15.73:0/3310922322 shutdown_connections 2024-04-05T22:29:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.449+0000 7fe609400640 1 -- 172.21.15.73:0/3310922322 wait complete. 2024-04-05T22:29:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.451+0000 7fe609400640 1 Processor -- start 2024-04-05T22:29:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.451+0000 7fe609400640 1 -- start start 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6040ad6b0 0x7fe60414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040b1870 0x7fe6040c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604152770 con 0x7fe604160620 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040c5240 con 0x7fe6040ad6b0 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe609400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040c5520 con 0x7fe6040b1870 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe603200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe603200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55400/0 (socket says 172.21.15.73:55400) 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe603200640 1 -- 172.21.15.73:0/3887533147 learned_addr learned my addr 172.21.15.73:0/3887533147 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe5f9e00640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040b1870 0x7fe6040c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe602800640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6040ad6b0 0x7fe60414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe603200640 1 -- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040b1870 msgr2=0x7fe6040c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.452+0000 7fe603200640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040b1870 0x7fe6040c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe603200640 1 -- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6040ad6b0 msgr2=0x7fe60414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe603200640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6040ad6b0 0x7fe60414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe603200640 1 -- 172.21.15.73:0/3887533147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc009b30 con 0x7fe604160620 2024-04-05T22:29:39.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe603200640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc036040 tx=0x7fe5fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc009510 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc03e450 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc03e7e0 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6040c5800 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.453+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6040c5dc0 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.454+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604066af0 con 0x7fe604160620 2024-04-05T22:29:39.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.455+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fe5fc03e940 con 0x7fe604160620 2024-04-05T22:29:39.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.456+0000 7fe5fb400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 0x7fe5e006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:39.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.457+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5fc0acb80 con 0x7fe604160620 2024-04-05T22:29:39.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.457+0000 7fe602800640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 0x7fe5e006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:39.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.458+0000 7fe602800640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 0x7fe5e006d660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4007fd0 tx=0x7fe5f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:39.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.461+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe5fc077420 con 0x7fe604160620 2024-04-05T22:29:39.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.676+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe604068160 con 0x7fe604160620 2024-04-05T22:29:39.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.677+0000 7fe5fb400640 1 -- 172.21.15.73:0/3887533147 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5fc043070 con 0x7fe604160620 2024-04-05T22:29:39.676 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.679+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 msgr2=0x7fe5e006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.679+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 0x7fe5e006d660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4007fd0 tx=0x7fe5f4009040 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.679+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 msgr2=0x7fe604069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.679+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc036040 tx=0x7fe5fc004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 shutdown_connections 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5e006b1a0 0x7fe5e006d660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe604160620 0x7fe604069cb0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6040b1870 0x7fe6040c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 --2- 172.21.15.73:0/3887533147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6040ad6b0 0x7fe60414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 >> 172.21.15.73:0/3887533147 conn(0x7fe6040ce8e0 msgr2=0x7fe6040cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 shutdown_connections 2024-04-05T22:29:39.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:39.680+0000 7fe609400640 1 -- 172.21.15.73:0/3887533147 wait complete. 2024-04-05T22:29:39.697 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:29:39.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:29:39.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:29:39.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:29:39.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:39.698 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:29:39.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:29:39.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:29:39.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:29:39.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:39.698 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:29:39.746 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:29:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:29:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:29:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:29:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:29:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:39.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:39.816 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:29:39.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:29:39.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:29:39.893 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:29:39.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:39.897 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:29:40.032 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:29:40.032 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:29:40.032 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:29:40.032 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:29:40.032 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:29:40.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:29:40.033 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:29:40.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.200 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:29:40.249 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:29:40.249 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:29:40.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:29:40.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:29:40.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:29:40.339 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:40.394 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:29:40.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:29:40.395 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:40.446 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:29:40.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:40.580 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:40.581 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:29:40.612 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:40.613 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:29:40.613 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:29:40.613 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:29:40.613 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:29:40.615 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:29:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:29:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:29:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:29:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:29:40.619 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:29:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:40.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:40.707 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:29:40.767 INFO:teuthology.orchestra.run.smithi073.stdout:client6208 2024-04-05T22:29:40.767 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:40.767 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:40.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:29:41.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.015+0000 7f381de00640 1 Processor -- start 2024-04-05T22:29:41.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.015+0000 7f381de00640 1 -- start start 2024-04-05T22:29:41.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.016+0000 7f381de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38181587d0 0x7f38180cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.016+0000 7f381de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f381de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381806f5b0 con 0x7f38180ccd20 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f381de00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381815f240 con 0x7f38181587d0 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f381de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818070410 con 0x7f381815e850 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f3817400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f3816a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38181587d0 0x7f38180cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f3817400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58592/0 (socket says 172.21.15.73:58592) 2024-04-05T22:29:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f3817400640 1 -- 172.21.15.73:0/1566466553 learned_addr learned my addr 172.21.15.73:0/1566466553 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.017+0000 7f380fe00640 1 --1- 172.21.15.73:0/1566466553 >> v1:172.21.15.157:6789/0 conn(0x7f38180ccd20 0x7f381806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:40316/0 (socket says 172.21.15.73:40316) 2024-04-05T22:29:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.018+0000 7f3816a00640 1 -- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 msgr2=0x7f381815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.018+0000 7f3816a00640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.018+0000 7f3816a00640 1 -- 172.21.15.73:0/1566466553 >> v1:172.21.15.157:6789/0 conn(0x7f38180ccd20 legacy=0x7f381806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.018+0000 7f3816a00640 1 -- 172.21.15.73:0/1566466553 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818158bb0 con 0x7f38181587d0 2024-04-05T22:29:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.018+0000 7f3817400640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.020+0000 7f3816a00640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38181587d0 0x7f38180cc7e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38080097a0 tx=0x7f380802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f231208725ef824 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:41.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.020+0000 7f3816000640 1 -- 172.21.15.73:0/1566466553 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380802d040 con 0x7f38181587d0 2024-04-05T22:29:41.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.020+0000 7f3816000640 1 -- 172.21.15.73:0/1566466553 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808031810 con 0x7f38181587d0 2024-04-05T22:29:41.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.020+0000 7f3816000640 1 -- 172.21.15.73:0/1566466553 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808031ba0 con 0x7f38181587d0 2024-04-05T22:29:41.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 -- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38181587d0 msgr2=0x7f38180cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38181587d0 0x7f38180cc7e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38080097a0 tx=0x7f380802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 -- 172.21.15.73:0/1566466553 shutdown_connections 2024-04-05T22:29:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f381815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 --2- 172.21.15.73:0/1566466553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38181587d0 0x7f38180cc7e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 -- 172.21.15.73:0/1566466553 >> 172.21.15.73:0/1566466553 conn(0x7f38180ce8e0 msgr2=0x7f38180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 -- 172.21.15.73:0/1566466553 shutdown_connections 2024-04-05T22:29:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.021+0000 7f381de00640 1 -- 172.21.15.73:0/1566466553 wait complete. 2024-04-05T22:29:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.023+0000 7f381de00640 1 Processor -- start 2024-04-05T22:29:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.023+0000 7f381de00640 1 -- start start 2024-04-05T22:29:41.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.024+0000 7f381de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f381de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f3818149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f381de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381814a520 0x7f38180b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f381de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381814a9b0 con 0x7f38181587d0 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f381de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b38e0 con 0x7f381814a520 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f381de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381814abf0 con 0x7f381815e850 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55414/0 (socket says 172.21.15.73:55414) 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 -- 172.21.15.73:0/1571319742 learned_addr learned my addr 172.21.15.73:0/1571319742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f380fe00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f3818149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3817400640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381814a520 0x7f38180b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 -- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 msgr2=0x7f3818149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f3818149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 -- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381814a520 msgr2=0x7f38180b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381814a520 0x7f38180b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f3816a00640 1 -- 172.21.15.73:0/1571319742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009b30 con 0x7f38181587d0 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.025+0000 7f380fe00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f3818149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f3816a00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f38080098c0 tx=0x7f38080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808006e10 con 0x7f38181587d0 2024-04-05T22:29:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380803ad60 con 0x7f38181587d0 2024-04-05T22:29:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808044620 con 0x7f38181587d0 2024-04-05T22:29:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3808009400 con 0x7f38181587d0 2024-04-05T22:29:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.026+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381814b1b0 con 0x7f38181587d0 2024-04-05T22:29:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.027+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818066af0 con 0x7f38181587d0 2024-04-05T22:29:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.028+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f380804d440 con 0x7f38181587d0 2024-04-05T22:29:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.029+0000 7f380f400640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 0x7f37f406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.029+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38080ac1c0 con 0x7f38181587d0 2024-04-05T22:29:41.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.030+0000 7f380fe00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 0x7f37f406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:41.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.031+0000 7f380fe00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 0x7f37f406d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38180ad790 tx=0x7f3800009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:41.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.034+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f38080769e0 con 0x7f38181587d0 2024-04-05T22:29:41.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.266+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f381806fac0 con 0x7f38181587d0 2024-04-05T22:29:41.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.267+0000 7f380f400640 1 -- 172.21.15.73:0/1571319742 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f381806fac0 con 0x7f38181587d0 2024-04-05T22:29:41.266 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:29:41.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 msgr2=0x7f37f406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 0x7f37f406d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38180ad790 tx=0x7f3800009040 comp rx=0 tx=0).stop 2024-04-05T22:29:41.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 msgr2=0x7f38180acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:41.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f38080098c0 tx=0x7f38080046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 shutdown_connections 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f37f406b1a0 0x7f37f406d660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381814a520 0x7f38180b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381815e850 0x7f3818149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 --2- 172.21.15.73:0/1571319742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38181587d0 0x7f38180acb40 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 >> 172.21.15.73:0/1571319742 conn(0x7f38180ce8e0 msgr2=0x7f3818158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 shutdown_connections 2024-04-05T22:29:41.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:41.270+0000 7f381de00640 1 -- 172.21.15.73:0/1571319742 wait complete. 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:29:41.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:29:41.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.289 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:41.348 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42106sec preferred_lft 42106sec 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:29:41.350 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:29:41.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.352 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:29:41.446 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:29:41.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.450 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:29:41.500 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:29:41.501 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:29:41.501 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:29:41.501 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:29:41.501 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:29:41.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.501 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:29:41.501 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:29:41.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:29:41.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:29:41.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:29:41.502 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:29:41.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.684 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:29:41.734 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:29:41.734 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:29:41.734 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:29:41.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:29:41.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:29:41.825 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:29:41.879 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:29:41.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:29:41.880 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:29:41.927 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:29:41.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:41.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:42.059 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:42.059 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:29:42.094 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:29:42.094 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:29:42.094 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:29:42.094 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:29:42.094 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:29:42.097 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:29:42.097 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:29:42.097 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:29:42.097 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:29:42.097 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:29:42.101 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:29:42.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:42.101 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:29:42.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:42.190 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:29:42.247 INFO:teuthology.orchestra.run.smithi157.stdout:client5732 2024-04-05T22:29:42.247 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:42.247 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:42.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:29:42.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.524+0000 7f6268a00640 1 Processor -- start 2024-04-05T22:29:42.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.524+0000 7f6268a00640 1 -- start start 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6268a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f626406f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6268a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640caf80 con 0x7f6264159990 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6268a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626406b610 con 0x7f62640b3230 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6268a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640caa00 con 0x7f626415dca0 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6263400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6264159990 0x7f626415d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36530/0 (socket says 172.21.15.73:36530) 2024-04-05T22:29:42.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6263400640 1 -- 172.21.15.73:0/4270251704 learned_addr learned my addr 172.21.15.73:0/4270251704 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:42.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6262a00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f626406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.526+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925135521 0 0) 0x7f62640caf80 con 0x7f6264159990 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6248003520 con 0x7f6264159990 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104731512 0 0) 0x7f6248003520 con 0x7f6264159990 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 msgr2=0x7f626406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f626406f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 >> v1:172.21.15.157:6789/0 conn(0x7f62640b3230 legacy=0x7f626406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626406b8f0 con 0x7f6264159990 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f62540019f0 con 0x7f6264159990 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 >> v1:172.21.15.73:6789/0 conn(0x7f6264159990 legacy=0x7f626415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f62480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6248003f20 0x7f62480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248004d00 0x7f6248005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640caa00 con 0x7f6248004d00 2024-04-05T22:29:42.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624800ba60 con 0x7f6248003f20 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.527+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624800de30 con 0x7f626415dca0 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248004d00 0x7f6248005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263400640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f62480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6262a00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6248003f20 0x7f62480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 -- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 msgr2=0x7f62480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f626415dca0 0x7f62480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 -- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6248003f20 msgr2=0x7f62480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6248003f20 0x7f62480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.528+0000 7f6263e00640 1 -- 172.21.15.73:0/4270251704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248003520 con 0x7f6248004d00 2024-04-05T22:29:42.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.529+0000 7f6263e00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248004d00 0x7f6248005180 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f625c001720 tx=0x7f62480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e758af1fc1437d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:42.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0052d0 con 0x7f6248004d00 2024-04-05T22:29:42.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c02e050 con 0x7f6248004d00 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6262000640 1 -- 172.21.15.73:0/4270251704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c003830 con 0x7f6248004d00 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 -- 172.21.15.73:0/4270251704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248004d00 msgr2=0x7f6248005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 --2- 172.21.15.73:0/4270251704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248004d00 0x7f6248005180 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f625c001720 tx=0x7f62480057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6263e00640 1 -- 172.21.15.73:0/4270251704 reap_dead start 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 -- 172.21.15.73:0/4270251704 shutdown_connections 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 -- 172.21.15.73:0/4270251704 >> 172.21.15.73:0/4270251704 conn(0x7f62640b0360 msgr2=0x7f62640b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 -- 172.21.15.73:0/4270251704 shutdown_connections 2024-04-05T22:29:42.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.530+0000 7f6268a00640 1 -- 172.21.15.73:0/4270251704 wait complete. 2024-04-05T22:29:42.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.532+0000 7f6268a00640 1 Processor -- start 2024-04-05T22:29:42.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.532+0000 7f6268a00640 1 -- start start 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6248004d00 0x7f626414b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62640c3f70 0x7f62640c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c4930 con 0x7f62640cc9b0 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626414d350 con 0x7f62640c3f70 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.533+0000 7f6268a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626414b680 con 0x7f6248004d00 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55428/0 (socket says 172.21.15.73:55428) 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 -- 172.21.15.73:0/688762057 learned_addr learned my addr 172.21.15.73:0/688762057 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263400640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6248004d00 0x7f626414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6262a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62640c3f70 0x7f62640c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 -- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6248004d00 msgr2=0x7f626414b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6248004d00 0x7f626414b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 -- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62640c3f70 msgr2=0x7f62640c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62640c3f70 0x7f62640c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:42.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 -- 172.21.15.73:0/688762057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640cd350 con 0x7f62640cc9b0 2024-04-05T22:29:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.534+0000 7f6263e00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f625c001840 tx=0x7f625c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.535+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c007a30 con 0x7f62640cc9b0 2024-04-05T22:29:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.535+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c0067d0 con 0x7f62640cc9b0 2024-04-05T22:29:42.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.535+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c006b60 con 0x7f62640cc9b0 2024-04-05T22:29:42.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.535+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626414d590 con 0x7f62640cc9b0 2024-04-05T22:29:42.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.535+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626414db50 con 0x7f62640cc9b0 2024-04-05T22:29:42.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.536+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6264066f00 con 0x7f62640cc9b0 2024-04-05T22:29:42.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.537+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f625c006cc0 con 0x7f62640cc9b0 2024-04-05T22:29:42.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.538+0000 7f625b400640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 0x7f624006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:42.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.538+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f625c0ab5d0 con 0x7f62640cc9b0 2024-04-05T22:29:42.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.539+0000 7f6263400640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 0x7f624006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:42.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.539+0000 7f6263400640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 0x7f624006d690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6254004d00 tx=0x7f6254006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:42.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.543+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f625c075e80 con 0x7f62640cc9b0 2024-04-05T22:29:42.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:42.774+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62640cb2e0 con 0x7f62640cc9b0 2024-04-05T22:29:43.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.662+0000 7f625b400640 1 -- 172.21.15.73:0/688762057 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 136+0+0 (secure 0 0 0) 0x7f625c0794a0 con 0x7f62640cc9b0 2024-04-05T22:29:43.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 msgr2=0x7f624006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 0x7f624006d690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6254004d00 tx=0x7f6254006040 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 msgr2=0x7f62640cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f625c001840 tx=0x7f625c001120 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 shutdown_connections 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f624006b1d0 0x7f624006d690 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62640cc9b0 0x7f62640cce10 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62640c3f70 0x7f62640c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 --2- 172.21.15.73:0/688762057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6248004d00 0x7f626414b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 >> 172.21.15.73:0/688762057 conn(0x7f62640b0360 msgr2=0x7f626406c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 shutdown_connections 2024-04-05T22:29:43.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.665+0000 7f6268a00640 1 -- 172.21.15.73:0/688762057 wait complete. 2024-04-05T22:29:43.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:29:43.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.902+0000 7ff19ec00640 1 Processor -- start 2024-04-05T22:29:43.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.902+0000 7ff19ec00640 1 -- start start 2024-04-05T22:29:43.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.903+0000 7ff19ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff19806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.903+0000 7ff19ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff19815e850 0x7ff19815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.903+0000 7ff19ec00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19806f5b0 con 0x7ff1981587d0 2024-04-05T22:29:43.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.903+0000 7ff19ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19806d500 con 0x7ff19815e850 2024-04-05T22:29:43.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.903+0000 7ff19ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19815f220 con 0x7ff19806cb80 2024-04-05T22:29:43.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.904+0000 7ff19cc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff19815e850 0x7ff19815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.904+0000 7ff19cc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff19815e850 0x7ff19815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:52278/0 (socket says 172.21.15.73:52278) 2024-04-05T22:29:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.904+0000 7ff19cc00640 1 -- 172.21.15.73:0/1136037630 learned_addr learned my addr 172.21.15.73:0/1136037630 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.904+0000 7ff18f400640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff19806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.905+0000 7ff19cc00640 1 -- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 msgr2=0x7ff19806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.905+0000 7ff19cc00640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff19806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.905+0000 7ff19cc00640 1 -- 172.21.15.73:0/1136037630 >> v1:172.21.15.157:6789/0 conn(0x7ff1981587d0 legacy=0x7ff1980cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.905+0000 7ff19cc00640 1 -- 172.21.15.73:0/1136037630 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff198070410 con 0x7ff19815e850 2024-04-05T22:29:43.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.906+0000 7ff19cc00640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff19815e850 0x7ff19815ece0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e80 tx=0x7ff19002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b2a1cb8e71010 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.907+0000 7ff197400640 1 -- 172.21.15.73:0/1136037630 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19002caf0 con 0x7ff19815e850 2024-04-05T22:29:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.907+0000 7ff197400640 1 -- 172.21.15.73:0/1136037630 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19002cc50 con 0x7ff19815e850 2024-04-05T22:29:43.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.907+0000 7ff197400640 1 -- 172.21.15.73:0/1136037630 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900343f0 con 0x7ff19815e850 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 -- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff19815e850 msgr2=0x7ff19815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff19815e850 0x7ff19815ece0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e80 tx=0x7ff19002bf50 comp rx=0 tx=0).stop 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 -- 172.21.15.73:0/1136037630 shutdown_connections 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff19815e850 0x7ff19815ece0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 --2- 172.21.15.73:0/1136037630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff19806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 -- 172.21.15.73:0/1136037630 >> 172.21.15.73:0/1136037630 conn(0x7ff1980ce8e0 msgr2=0x7ff1980cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 -- 172.21.15.73:0/1136037630 shutdown_connections 2024-04-05T22:29:43.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.908+0000 7ff19ec00640 1 -- 172.21.15.73:0/1136037630 wait complete. 2024-04-05T22:29:43.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.910+0000 7ff19ec00640 1 Processor -- start 2024-04-05T22:29:43.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.910+0000 7ff19ec00640 1 -- start start 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff1980ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981587d0 0x7ff1980a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b00b0 con 0x7ff19815e850 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980af240 con 0x7ff1981587d0 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980af4a0 con 0x7ff19806cb80 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.911+0000 7ff19cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55438/0 (socket says 172.21.15.73:55438) 2024-04-05T22:29:43.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 -- 172.21.15.73:0/2475178255 learned_addr learned my addr 172.21.15.73:0/2475178255 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff197e00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff1980ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff18f400640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981587d0 0x7ff1980a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 -- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 msgr2=0x7ff1980ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff1980ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 -- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981587d0 msgr2=0x7ff1980a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981587d0 0x7ff1980a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 -- 172.21.15.73:0/2475178255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009b30 con 0x7ff19815e850 2024-04-05T22:29:43.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff197e00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff1980ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.912+0000 7ff19cc00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff190036040 tx=0x7ff190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.913+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190040670 con 0x7ff19815e850 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.913+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190040810 con 0x7ff19815e850 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.913+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190040bc0 con 0x7ff19815e850 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.913+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1980af780 con 0x7ff19815e850 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.913+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1980afd40 con 0x7ff19815e850 2024-04-05T22:29:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.914+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff198066af0 con 0x7ff19815e850 2024-04-05T22:29:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.915+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff190040d20 con 0x7ff19815e850 2024-04-05T22:29:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.916+0000 7ff194c00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 0x7ff17406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.916+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1900acd70 con 0x7ff19815e850 2024-04-05T22:29:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.917+0000 7ff197e00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 0x7ff17406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.918+0000 7ff197e00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 0x7ff17406d660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff19815bd00 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:43.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:43.922+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff1900776a0 con 0x7ff19815e850 2024-04-05T22:29:44.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.139+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff19806f800 con 0x7ff17406b1a0 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.143+0000 7ff194c00640 1 -- 172.21.15.73:0/2475178255 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff188001cd0 con 0x7ff17406b1a0 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:29:44.142 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 msgr2=0x7ff17406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 0x7ff17406d660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff19815bd00 tx=0x7ff188009040 comp rx=0 tx=0).stop 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 msgr2=0x7ff1980aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff190036040 tx=0x7ff190004a40 comp rx=0 tx=0).stop 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 shutdown_connections 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff17406b1a0 0x7ff17406d660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff19815e850 0x7ff1980aed00 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981587d0 0x7ff1980a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 --2- 172.21.15.73:0/2475178255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff19806cb80 0x7ff1980ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.145+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 >> 172.21.15.73:0/2475178255 conn(0x7ff1980ce8e0 msgr2=0x7ff19815dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:44.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.146+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 shutdown_connections 2024-04-05T22:29:44.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.146+0000 7ff19ec00640 1 -- 172.21.15.73:0/2475178255 wait complete. 2024-04-05T22:29:44.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:29:44.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.378+0000 7ff82f600640 1 Processor -- start 2024-04-05T22:29:44.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.378+0000 7ff82f600640 1 -- start start 2024-04-05T22:29:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff82f600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8281599f0 0x7ff828070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff82f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8280add00 0x7ff82806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff82f600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82806cfb0 con 0x7ff82806cb80 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff82f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8280b3890 con 0x7ff8280add00 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff82f600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8280ae190 con 0x7ff8281599f0 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff81fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8280add00 0x7ff82806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.380+0000 7ff81fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8280add00 0x7ff82806c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55452/0 (socket says 172.21.15.73:55452) 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff81fe00640 1 -- 172.21.15.73:0/1915381707 learned_addr learned my addr 172.21.15.73:0/1915381707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff82cc00640 1 --2- 172.21.15.73:0/1915381707 >> v2:172.21.15.157:3300/0 conn(0x7ff8281599f0 0x7ff828070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff81fe00640 1 -- 172.21.15.73:0/1915381707 >> v2:172.21.15.157:3300/0 conn(0x7ff8281599f0 msgr2=0x7ff828070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff81fe00640 1 --2- 172.21.15.73:0/1915381707 >> v2:172.21.15.157:3300/0 conn(0x7ff8281599f0 0x7ff828070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff81fe00640 1 -- 172.21.15.73:0/1915381707 >> v1:172.21.15.157:6789/0 conn(0x7ff82806cb80 legacy=0x7ff82815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.381+0000 7ff81fe00640 1 -- 172.21.15.73:0/1915381707 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280b27f0 con 0x7ff8280add00 2024-04-05T22:29:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.383+0000 7ff81fe00640 1 --2- 172.21.15.73:0/1915381707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff82806c640 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ff82806d300 tx=0x7ff81002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e44f198a8cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.383+0000 7ff827e00640 1 -- 172.21.15.73:0/1915381707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810031830 con 0x7ff8280add00 2024-04-05T22:29:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.383+0000 7ff827e00640 1 -- 172.21.15.73:0/1915381707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810031990 con 0x7ff8280add00 2024-04-05T22:29:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.383+0000 7ff827e00640 1 -- 172.21.15.73:0/1915381707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810031d20 con 0x7ff8280add00 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 -- 172.21.15.73:0/1915381707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 msgr2=0x7ff82806c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 --2- 172.21.15.73:0/1915381707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff82806c640 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ff82806d300 tx=0x7ff81002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 -- 172.21.15.73:0/1915381707 shutdown_connections 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 --2- 172.21.15.73:0/1915381707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff82806c640 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 --2- 172.21.15.73:0/1915381707 >> v2:172.21.15.157:3300/0 conn(0x7ff8281599f0 0x7ff828070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 -- 172.21.15.73:0/1915381707 >> 172.21.15.73:0/1915381707 conn(0x7ff828069cb0 msgr2=0x7ff82806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 -- 172.21.15.73:0/1915381707 shutdown_connections 2024-04-05T22:29:44.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.384+0000 7ff82f600640 1 -- 172.21.15.73:0/1915381707 wait complete. 2024-04-05T22:29:44.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.386+0000 7ff82f600640 1 Processor -- start 2024-04-05T22:29:44.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.386+0000 7ff82f600640 1 -- start start 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff82806cb80 0x7ff82815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281599f0 0x7ff8280cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82815a2d0 con 0x7ff8280add00 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82815a5b0 con 0x7ff8281599f0 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82815a890 con 0x7ff82806cb80 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55468/0 (socket says 172.21.15.73:55468) 2024-04-05T22:29:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 -- 172.21.15.73:0/666956451 learned_addr learned my addr 172.21.15.73:0/666956451 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 -- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff82806cb80 msgr2=0x7ff82815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff82806cb80 0x7ff82815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 -- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281599f0 msgr2=0x7ff8280cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff82d600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281599f0 0x7ff8280cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281599f0 0x7ff8280cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.387+0000 7ff81fe00640 1 -- 172.21.15.73:0/666956451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100094d0 con 0x7ff8280add00 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff81fe00640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100317d0 con 0x7ff8280add00 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81003aa60 con 0x7ff8280add00 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81003ae10 con 0x7ff8280add00 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82815ab70 con 0x7ff8280add00 2024-04-05T22:29:44.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.388+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8281456c0 con 0x7ff8280add00 2024-04-05T22:29:44.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.389+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8280cb610 con 0x7ff8280add00 2024-04-05T22:29:44.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.390+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff81004d450 con 0x7ff8280add00 2024-04-05T22:29:44.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.391+0000 7ff825600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 0x7ff80406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.391+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8100abfe0 con 0x7ff8280add00 2024-04-05T22:29:44.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.392+0000 7ff82cc00640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 0x7ff80406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.393+0000 7ff82cc00640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 0x7ff80406d660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.396+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff810076800 con 0x7ff8280add00 2024-04-05T22:29:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.590+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff82815b090 con 0x7ff8280add00 2024-04-05T22:29:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.591+0000 7ff825600640 1 -- 172.21.15.73:0/666956451 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff82815b090 con 0x7ff8280add00 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 msgr2=0x7ff80406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 0x7ff80406d660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 msgr2=0x7ff828145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 shutdown_connections 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff80406b1a0 0x7ff80406d660 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281599f0 0x7ff8280cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8280add00 0x7ff828145180 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 --2- 172.21.15.73:0/666956451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff82806cb80 0x7ff82815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 >> 172.21.15.73:0/666956451 conn(0x7ff828069cb0 msgr2=0x7ff82806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.594+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 shutdown_connections 2024-04-05T22:29:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.595+0000 7ff82f600640 1 -- 172.21.15.73:0/666956451 wait complete. 2024-04-05T22:29:44.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-04-05T22:29:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.822+0000 7ff12e800640 1 Processor -- start 2024-04-05T22:29:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.823+0000 7ff12e800640 1 -- start start 2024-04-05T22:29:44.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff12e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1281599f0 0x7ff128070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff12e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff12806cd50 0x7ff128159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff12e800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280b27f0 con 0x7ff1280add00 2024-04-05T22:29:44.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff12e800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1280b3890 con 0x7ff12806cd50 2024-04-05T22:29:44.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff12e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280b2ad0 con 0x7ff1281599f0 2024-04-05T22:29:44.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff11ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1280add00 0x7ff12806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36544/0 (socket says 172.21.15.73:36544) 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.824+0000 7ff11ea00640 1 -- 172.21.15.73:0/2355739054 learned_addr learned my addr 172.21.15.73:0/2355739054 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127e00640 1 --2- 172.21.15.73:0/2355739054 >> v2:172.21.15.73:3300/0 conn(0x7ff12806cd50 0x7ff128159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127400640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1281599f0 0x7ff128070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff126a00640 1 -- 172.21.15.73:0/2355739054 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719695552 0 0) 0x7ff1280b27f0 con 0x7ff1280add00 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff126a00640 1 -- 172.21.15.73:0/2355739054 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff10c003520 con 0x7ff1280add00 2024-04-05T22:29:44.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127e00640 1 -- 172.21.15.73:0/2355739054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1281599f0 msgr2=0x7ff128070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:44.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127e00640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1281599f0 0x7ff128070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127e00640 1 -- 172.21.15.73:0/2355739054 >> v1:172.21.15.73:6789/0 conn(0x7ff1280add00 legacy=0x7ff12806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.825+0000 7ff127e00640 1 -- 172.21.15.73:0/2355739054 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280b2db0 con 0x7ff12806cd50 2024-04-05T22:29:44.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.826+0000 7ff127e00640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff12806cd50 0x7ff128159dd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff120009d90 tx=0x7ff12002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d29452fcc7bc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.826+0000 7ff126a00640 1 -- 172.21.15.73:0/2355739054 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12002cc50 con 0x7ff12806cd50 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff126a00640 1 -- 172.21.15.73:0/2355739054 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12002cdb0 con 0x7ff12806cd50 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff126a00640 1 -- 172.21.15.73:0/2355739054 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120034660 con 0x7ff12806cd50 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 -- 172.21.15.73:0/2355739054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff12806cd50 msgr2=0x7ff128159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff12806cd50 0x7ff128159dd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff120009d90 tx=0x7ff12002bea0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 -- 172.21.15.73:0/2355739054 shutdown_connections 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff12806cd50 0x7ff128159dd0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 --2- 172.21.15.73:0/2355739054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1281599f0 0x7ff128070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 -- 172.21.15.73:0/2355739054 >> 172.21.15.73:0/2355739054 conn(0x7ff128069cb0 msgr2=0x7ff12806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 -- 172.21.15.73:0/2355739054 shutdown_connections 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.827+0000 7ff12e800640 1 -- 172.21.15.73:0/2355739054 wait complete. 2024-04-05T22:29:44.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.829+0000 7ff12e800640 1 Processor -- start 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.829+0000 7ff12e800640 1 -- start start 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff12806cd50 0x7ff1280cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1280add00 0x7ff1281f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1281599f0 0x7ff1280ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281f5a60 con 0x7ff1281599f0 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281f2c10 con 0x7ff1280add00 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff12e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281f2ef0 con 0x7ff12806cd50 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff127e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1281599f0 0x7ff1280ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff127e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1281599f0 0x7ff1280ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55492/0 (socket says 172.21.15.73:55492) 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.830+0000 7ff127e00640 1 -- 172.21.15.73:0/2154135097 learned_addr learned my addr 172.21.15.73:0/2154135097 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127400640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff12806cd50 0x7ff1280cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 -- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff12806cd50 msgr2=0x7ff1280cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff12806cd50 0x7ff1280cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff11ea00640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1280add00 0x7ff1281f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 -- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1280add00 msgr2=0x7ff1281f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1280add00 0x7ff1281f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 -- 172.21.15.73:0/2154135097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009a70 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127400640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff12806cd50 0x7ff1280cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff127e00640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1281599f0 0x7ff1280ac8d0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff120009d90 tx=0x7ff120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120040650 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1200407b0 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120040b60 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.831+0000 7ff12e800640 1 -- 172.21.15.73:0/2154135097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1281f31d0 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.832+0000 7ff12e800640 1 -- 172.21.15.73:0/2154135097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1281f66b0 con 0x7ff1281599f0 2024-04-05T22:29:44.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.833+0000 7ff12e800640 1 -- 172.21.15.73:0/2154135097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1281f33c0 con 0x7ff1281599f0 2024-04-05T22:29:44.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.834+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff120040cc0 con 0x7ff1281599f0 2024-04-05T22:29:44.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.835+0000 7ff11fe00640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff10406b1a0 0x7ff10406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:44.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.835+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1200acba0 con 0x7ff1281599f0 2024-04-05T22:29:44.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.835+0000 7ff127400640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff10406b1a0 0x7ff10406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:44.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.837+0000 7ff127400640 1 --2- 172.21.15.73:0/2154135097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff10406b1a0 0x7ff10406d660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff118006fd0 tx=0x7ff118008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:44.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:44.839+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff1200774d0 con 0x7ff1281599f0 2024-04-05T22:29:45.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.059+0000 7ff12e800640 1 -- 172.21.15.73:0/2154135097 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1280ae160 con 0x7ff10406b1a0 2024-04-05T22:29:45.061 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:45.062+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.063+0000 7ff11fe00640 1 -- 172.21.15.73:0/2154135097 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7ff118009530 con 0x7ff10406b1a0 2024-04-05T22:29:45.062 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.077 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:45.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-04-05T22:29:45.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.296+0000 7ff753000640 1 Processor -- start 2024-04-05T22:29:45.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.296+0000 7ff753000640 1 -- start start 2024-04-05T22:29:45.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff753000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff753000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff74c15dd90 0x7ff74c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff753000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74c071900 con 0x7ff74c0714a0 2024-04-05T22:29:45.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff753000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74c15ae50 con 0x7ff74c15dd90 2024-04-05T22:29:45.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff753000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c15e3b0 con 0x7ff74c15aa70 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff74be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.298+0000 7ff751000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff74c15dd90 0x7ff74c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.299+0000 7ff751000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff74c15dd90 0x7ff74c06ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:52320/0 (socket says 172.21.15.73:52320) 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.299+0000 7ff74be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:58672/0 (socket says 172.21.15.73:58672) 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.299+0000 7ff751000640 1 -- 172.21.15.73:0/3823528957 learned_addr learned my addr 172.21.15.73:0/3823528957 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:45.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.299+0000 7ff74b400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff74c0714a0 0x7ff74c15d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:40352/0 (socket says 172.21.15.73:40352) 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff751000640 1 -- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 msgr2=0x7ff74c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff751000640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff751000640 1 -- 172.21.15.73:0/3823528957 >> v1:172.21.15.157:6789/0 conn(0x7ff74c0714a0 legacy=0x7ff74c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff74aa00640 1 -- 172.21.15.73:0/3823528957 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320388320 0 0) 0x7ff74c071900 con 0x7ff74c0714a0 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff751000640 1 -- 172.21.15.73:0/3823528957 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c0ac9e0 con 0x7ff74c15dd90 2024-04-05T22:29:45.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.300+0000 7ff74be00640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:29:45.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.301+0000 7ff751000640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15dd90 0x7ff74c06ba40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff744009e50 tx=0x7ff74402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18eed819eda020be server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff74aa00640 1 -- 172.21.15.73:0/3823528957 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74402cb00 con 0x7ff74c15dd90 2024-04-05T22:29:45.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff74aa00640 1 -- 172.21.15.73:0/3823528957 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74402cc60 con 0x7ff74c15dd90 2024-04-05T22:29:45.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff74aa00640 1 -- 172.21.15.73:0/3823528957 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440343f0 con 0x7ff74c15dd90 2024-04-05T22:29:45.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 -- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15dd90 msgr2=0x7ff74c06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15dd90 0x7ff74c06ba40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff744009e50 tx=0x7ff74402bf60 comp rx=0 tx=0).stop 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 -- 172.21.15.73:0/3823528957 shutdown_connections 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15dd90 0x7ff74c06ba40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 --2- 172.21.15.73:0/3823528957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.302+0000 7ff753000640 1 -- 172.21.15.73:0/3823528957 >> 172.21.15.73:0/3823528957 conn(0x7ff74c069cb0 msgr2=0x7ff74c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.303+0000 7ff753000640 1 -- 172.21.15.73:0/3823528957 shutdown_connections 2024-04-05T22:29:45.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.303+0000 7ff753000640 1 -- 172.21.15.73:0/3823528957 wait complete. 2024-04-05T22:29:45.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.304+0000 7ff753000640 1 Processor -- start 2024-04-05T22:29:45.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.304+0000 7ff753000640 1 -- start start 2024-04-05T22:29:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c0714a0 0x7ff74c151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74c15dd90 0x7ff74c0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1542d0 con 0x7ff74c15dd90 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c0af4b0 con 0x7ff74c0714a0 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.305+0000 7ff753000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c154520 con 0x7ff74c15aa70 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74c15dd90 0x7ff74c0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74c15dd90 0x7ff74c0aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55500/0 (socket says 172.21.15.73:55500) 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 -- 172.21.15.73:0/3073439445 learned_addr learned my addr 172.21.15.73:0/3073439445 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff74b400640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff74be00640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c0714a0 0x7ff74c151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 -- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 msgr2=0x7ff74c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15aa70 0x7ff74c153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 -- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c0714a0 msgr2=0x7ff74c151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c0714a0 0x7ff74c151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 -- 172.21.15.73:0/3073439445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009400 con 0x7ff74c15dd90 2024-04-05T22:29:45.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.306+0000 7ff751000640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74c15dd90 0x7ff74c0aef70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ff744036040 tx=0x7ff744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.307+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744009510 con 0x7ff74c15dd90 2024-04-05T22:29:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.307+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74403e450 con 0x7ff74c15dd90 2024-04-05T22:29:45.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.307+0000 7ff753000640 1 -- 172.21.15.73:0/3073439445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff744009b30 con 0x7ff74c15dd90 2024-04-05T22:29:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.307+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74403e5e0 con 0x7ff74c15dd90 2024-04-05T22:29:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.307+0000 7ff753000640 1 -- 172.21.15.73:0/3073439445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c154ae0 con 0x7ff74c15dd90 2024-04-05T22:29:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.308+0000 7ff753000640 1 -- 172.21.15.73:0/3073439445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c068160 con 0x7ff74c15dd90 2024-04-05T22:29:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.309+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff74403e740 con 0x7ff74c15dd90 2024-04-05T22:29:45.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.310+0000 7ff743e00640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff72c06b1a0 0x7ff72c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.310+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7440ac5e0 con 0x7ff74c15dd90 2024-04-05T22:29:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.310+0000 7ff74be00640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff72c06b1a0 0x7ff72c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.312+0000 7ff74be00640 1 --2- 172.21.15.73:0/3073439445 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff72c06b1a0 0x7ff72c06d660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff74c161120 tx=0x7ff73c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.314+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff744077050 con 0x7ff74c15dd90 2024-04-05T22:29:45.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.534+0000 7ff753000640 1 -- 172.21.15.73:0/3073439445 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff74c0ad280 con 0x7ff72c06b1a0 2024-04-05T22:29:45.536 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:45.537+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.537+0000 7ff743e00640 1 -- 172.21.15.73:0/3073439445 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7ff73c011e90 con 0x7ff72c06b1a0 2024-04-05T22:29:45.536 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.551 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:45.551 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-04-05T22:29:45.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.751+0000 7fc469e00640 1 Processor -- start 2024-04-05T22:29:45.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.752+0000 7fc469e00640 1 -- start start 2024-04-05T22:29:45.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.753+0000 7fc469e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc46415a850 0x7fc46415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.753+0000 7fc469e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc464070640 0x7fc46406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.753+0000 7fc469e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46406bc50 con 0x7fc46415e580 2024-04-05T22:29:45.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.753+0000 7fc469e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46415f530 con 0x7fc464070640 2024-04-05T22:29:45.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.753+0000 7fc469e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46415ef70 con 0x7fc46415a850 2024-04-05T22:29:45.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc463e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc46415e580 0x7fc46406ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36546/0 (socket says 172.21.15.73:36546) 2024-04-05T22:29:45.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc463e00640 1 -- 172.21.15.73:0/1783410590 learned_addr learned my addr 172.21.15.73:0/1783410590 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc469400640 1 --2- 172.21.15.73:0/1783410590 >> v2:172.21.15.73:3300/0 conn(0x7fc464070640 0x7fc46406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc468a00640 1 --2- 172.21.15.73:0/1783410590 >> v2:172.21.15.157:3300/0 conn(0x7fc46415a850 0x7fc46415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc463400640 1 -- 172.21.15.73:0/1783410590 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571756749 0 0) 0x7fc46406bc50 con 0x7fc46415e580 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc463400640 1 -- 172.21.15.73:0/1783410590 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc44c003520 con 0x7fc46415e580 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc469400640 1 -- 172.21.15.73:0/1783410590 >> v2:172.21.15.157:3300/0 conn(0x7fc46415a850 msgr2=0x7fc46415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc469400640 1 --2- 172.21.15.73:0/1783410590 >> v2:172.21.15.157:3300/0 conn(0x7fc46415a850 0x7fc46415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc469400640 1 -- 172.21.15.73:0/1783410590 >> v1:172.21.15.73:6789/0 conn(0x7fc46415e580 legacy=0x7fc46406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.754+0000 7fc469400640 1 -- 172.21.15.73:0/1783410590 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46415f250 con 0x7fc464070640 2024-04-05T22:29:45.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.755+0000 7fc469400640 1 --2- 172.21.15.73:0/1783410590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464070640 0x7fc46406c6f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc4640cbf30 tx=0x7fc45c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f9f2f06192ba38 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.755+0000 7fc463400640 1 -- 172.21.15.73:0/1783410590 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02cbf0 con 0x7fc464070640 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc463400640 1 -- 172.21.15.73:0/1783410590 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c02cd50 con 0x7fc464070640 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc463400640 1 -- 172.21.15.73:0/1783410590 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c034660 con 0x7fc464070640 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 -- 172.21.15.73:0/1783410590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464070640 msgr2=0x7fc46406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 --2- 172.21.15.73:0/1783410590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464070640 0x7fc46406c6f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc4640cbf30 tx=0x7fc45c02be40 comp rx=0 tx=0).stop 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 -- 172.21.15.73:0/1783410590 shutdown_connections 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 --2- 172.21.15.73:0/1783410590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464070640 0x7fc46406c6f0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 --2- 172.21.15.73:0/1783410590 >> v2:172.21.15.157:3300/0 conn(0x7fc46415a850 0x7fc46415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 -- 172.21.15.73:0/1783410590 >> 172.21.15.73:0/1783410590 conn(0x7fc464069cb0 msgr2=0x7fc46406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 -- 172.21.15.73:0/1783410590 shutdown_connections 2024-04-05T22:29:45.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.756+0000 7fc469e00640 1 -- 172.21.15.73:0/1783410590 wait complete. 2024-04-05T22:29:45.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.758+0000 7fc469e00640 1 Processor -- start 2024-04-05T22:29:45.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.758+0000 7fc469e00640 1 -- start start 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc464070640 0x7fc46414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46415a850 0x7fc4640cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc46415e580 0x7fc4640cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640aeea0 con 0x7fc46415e580 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640ae2b0 con 0x7fc46415a850 2024-04-05T22:29:45.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640ae500 con 0x7fc464070640 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc46415e580 0x7fc4640cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc46415e580 0x7fc4640cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55504/0 (socket says 172.21.15.73:55504) 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.759+0000 7fc469400640 1 -- 172.21.15.73:0/1422786100 learned_addr learned my addr 172.21.15.73:0/1422786100 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc468a00640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc464070640 0x7fc46414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc463e00640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46415a850 0x7fc4640cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 -- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc464070640 msgr2=0x7fc46414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc464070640 0x7fc46414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 -- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46415a850 msgr2=0x7fc4640cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46415a850 0x7fc4640cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 -- 172.21.15.73:0/1422786100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009a70 con 0x7fc46415e580 2024-04-05T22:29:45.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.760+0000 7fc469400640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc46415e580 0x7fc4640cdd20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc45c02c450 tx=0x7fc45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.761+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c040650 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.761+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c0407b0 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.761+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c040b60 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.761+0000 7fc469e00640 1 -- 172.21.15.73:0/1422786100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4640ae7e0 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.761+0000 7fc469e00640 1 -- 172.21.15.73:0/1422786100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641f3230 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.762+0000 7fc469e00640 1 -- 172.21.15.73:0/1422786100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4640aefe0 con 0x7fc46415e580 2024-04-05T22:29:45.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.763+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fc45c040cc0 con 0x7fc46415e580 2024-04-05T22:29:45.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.764+0000 7fc460c00640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc44406b1a0 0x7fc44406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:45.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.764+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc45c0acd20 con 0x7fc46415e580 2024-04-05T22:29:45.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.765+0000 7fc468a00640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc44406b1a0 0x7fc44406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.766+0000 7fc468a00640 1 --2- 172.21.15.73:0/1422786100 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc44406b1a0 0x7fc44406d660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc458006fd0 tx=0x7fc458008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:45.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.769+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc4640aefe0 con 0x7fc46415e580 2024-04-05T22:29:45.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.975+0000 7fc469e00640 1 -- 172.21.15.73:0/1422786100 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4641f33f0 con 0x7fc44406b1a0 2024-04-05T22:29:45.978 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:45.978+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:45.979+0000 7fc460c00640 1 -- 172.21.15.73:0/1422786100 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fc4641f33f0 con 0x7fc44406b1a0 2024-04-05T22:29:45.978 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:45.991 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:45.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-04-05T22:29:46.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.197+0000 7f585a600640 1 Processor -- start 2024-04-05T22:29:46.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.197+0000 7f585a600640 1 -- start start 2024-04-05T22:29:46.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.198+0000 7f585a600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 0x7f585415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.198+0000 7f585a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f585a600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585415ae50 con 0x7f585415aa70 2024-04-05T22:29:46.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f585a600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585415e3b0 con 0x7f5854071670 2024-04-05T22:29:46.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f585a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540ac9e0 con 0x7f585415dbc0 2024-04-05T22:29:46.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f584be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 0x7f585415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f5853e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f5853e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:35196/0 (socket says 172.21.15.73:35196) 2024-04-05T22:29:46.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f5853e00640 1 -- 172.21.15.73:0/85011948 learned_addr learned my addr 172.21.15.73:0/85011948 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:46.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.199+0000 7f584be00640 1 --2- 172.21.15.73:0/85011948 >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 0x7f585415d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:56116/0 (socket says 172.21.15.73:56116) 2024-04-05T22:29:46.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.200+0000 7f5853e00640 1 -- 172.21.15.73:0/85011948 >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 msgr2=0x7f585415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:46.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.200+0000 7f5853e00640 1 --2- 172.21.15.73:0/85011948 >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 0x7f585415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.200+0000 7f5853e00640 1 -- 172.21.15.73:0/85011948 >> v1:172.21.15.157:6789/0 conn(0x7f585415aa70 legacy=0x7f5854070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.200+0000 7f5853e00640 1 -- 172.21.15.73:0/85011948 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540accc0 con 0x7f585415dbc0 2024-04-05T22:29:46.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.202+0000 7f5853e00640 1 --2- 172.21.15.73:0/85011948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f584c009e50 tx=0x7f584c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dded1975fa94323d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.202+0000 7f5852a00640 1 -- 172.21.15.73:0/85011948 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c02cb00 con 0x7f585415dbc0 2024-04-05T22:29:46.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.202+0000 7f5852a00640 1 -- 172.21.15.73:0/85011948 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c02cc60 con 0x7f585415dbc0 2024-04-05T22:29:46.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.202+0000 7f5852a00640 1 -- 172.21.15.73:0/85011948 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0343f0 con 0x7f585415dbc0 2024-04-05T22:29:46.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 -- 172.21.15.73:0/85011948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 msgr2=0x7f585406ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 --2- 172.21.15.73:0/85011948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f584c009e50 tx=0x7f584c02bf60 comp rx=0 tx=0).stop 2024-04-05T22:29:46.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 -- 172.21.15.73:0/85011948 shutdown_connections 2024-04-05T22:29:46.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 --2- 172.21.15.73:0/85011948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415dbc0 0x7f585406ba40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 --2- 172.21.15.73:0/85011948 >> v2:172.21.15.157:3300/0 conn(0x7f5854071670 0x7f585415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 -- 172.21.15.73:0/85011948 >> 172.21.15.73:0/85011948 conn(0x7f5854069cb0 msgr2=0x7f585406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:46.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 -- 172.21.15.73:0/85011948 shutdown_connections 2024-04-05T22:29:46.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.203+0000 7f585a600640 1 -- 172.21.15.73:0/85011948 wait complete. 2024-04-05T22:29:46.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.205+0000 7f585a600640 1 Processor -- start 2024-04-05T22:29:46.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.205+0000 7f585a600640 1 -- start start 2024-04-05T22:29:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5854071670 0x7f5854159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415aa70 0x7f58540cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f585415dbc0 0x7f58540ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585415a0a0 con 0x7f5854071670 2024-04-05T22:29:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854158ac0 con 0x7f585415dbc0 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f585a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5854158da0 con 0x7f585415aa70 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5854071670 0x7f5854159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5854071670 0x7f5854159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60034/0 (socket says 172.21.15.73:60034) 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 -- 172.21.15.73:0/3919878091 learned_addr learned my addr 172.21.15.73:0/3919878091 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853e00640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f585415dbc0 0x7f58540ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f584be00640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415aa70 0x7f58540cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 -- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415aa70 msgr2=0x7f58540cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f585415aa70 0x7f58540cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 -- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f585415dbc0 msgr2=0x7f58540ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f585415dbc0 0x7f58540ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.207+0000 7f5853400640 1 -- 172.21.15.73:0/3919878091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c009b30 con 0x7f5854071670 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f5853400640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5854071670 0x7f5854159b60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f584400c450 tx=0x7f584400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440052d0 con 0x7f5854071670 2024-04-05T22:29:46.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844005430 con 0x7f5854071670 2024-04-05T22:29:46.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844012870 con 0x7f5854071670 2024-04-05T22:29:46.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f585a600640 1 -- 172.21.15.73:0/3919878091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854159080 con 0x7f5854071670 2024-04-05T22:29:46.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.208+0000 7f585a600640 1 -- 172.21.15.73:0/3919878091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5854159640 con 0x7f5854071670 2024-04-05T22:29:46.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.209+0000 7f585a600640 1 -- 172.21.15.73:0/3919878091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5854068160 con 0x7f5854071670 2024-04-05T22:29:46.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.210+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f58440129d0 con 0x7f5854071670 2024-04-05T22:29:46.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.211+0000 7f584b400640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f583006b1d0 0x7f583006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.211+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58440898f0 con 0x7f5854071670 2024-04-05T22:29:46.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.212+0000 7f584be00640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f583006b1d0 0x7f583006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.213+0000 7f584be00640 1 --2- 172.21.15.73:0/3919878091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f583006b1d0 0x7f583006d690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58541566d0 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.215+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5844054220 con 0x7f5854071670 2024-04-05T22:29:46.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.420+0000 7f585a600640 1 -- 172.21.15.73:0/3919878091 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f58540ad7a0 con 0x7f583006b1d0 2024-04-05T22:29:46.422 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:46.423+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:46.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.424+0000 7f584b400640 1 -- 172.21.15.73:0/3919878091 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f583c010840 con 0x7f583006b1d0 2024-04-05T22:29:46.423 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:46.437 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:46.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-04-05T22:29:46.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.647+0000 7f46dce00640 1 Processor -- start 2024-04-05T22:29:46.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.647+0000 7f46dce00640 1 -- start start 2024-04-05T22:29:46.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.649+0000 7f46dce00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 0x7f46d806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.649+0000 7f46dce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46d80b36e0 0x7f46d80cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.649+0000 7f46dce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d815e570 con 0x7f46d80cf210 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.649+0000 7f46dce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80cf6a0 con 0x7f46d80b36e0 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.649+0000 7f46dce00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d815afe0 con 0x7f46d815ac00 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46d6800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46d80cf210 0x7f46d80ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51146/0 (socket says 172.21.15.73:51146) 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46d6800640 1 -- 172.21.15.73:0/1571756712 learned_addr learned my addr 172.21.15.73:0/1571756712 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46cfe00640 1 --2- 172.21.15.73:0/1571756712 >> v2:172.21.15.73:3300/0 conn(0x7f46d80b36e0 0x7f46d80cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46d5e00640 1 --2- 172.21.15.73:0/1571756712 >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 0x7f46d806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46cfe00640 1 -- 172.21.15.73:0/1571756712 >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 msgr2=0x7f46d806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46cfe00640 1 --2- 172.21.15.73:0/1571756712 >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 0x7f46d806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46cfe00640 1 -- 172.21.15.73:0/1571756712 >> v1:172.21.15.73:6789/0 conn(0x7f46d80cf210 legacy=0x7f46d80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46cfe00640 1 -- 172.21.15.73:0/1571756712 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8161760 con 0x7f46d80b36e0 2024-04-05T22:29:46.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46d5e00640 1 --2- 172.21.15.73:0/1571756712 >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 0x7f46d806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:46.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.650+0000 7f46d5400640 1 -- 172.21.15.73:0/1571756712 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187836089 0 0) 0x7f46d815e570 con 0x7f46d80cf210 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.651+0000 7f46cfe00640 1 --2- 172.21.15.73:0/1571756712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80b36e0 0x7f46d80cecd0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f46d80cf890 tx=0x7f46c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5532327550561474 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.651+0000 7f46d5400640 1 -- 172.21.15.73:0/1571756712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c002e070 con 0x7f46d80b36e0 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.651+0000 7f46d5400640 1 -- 172.21.15.73:0/1571756712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00315a0 con 0x7f46d80b36e0 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.651+0000 7f46d5400640 1 -- 172.21.15.73:0/1571756712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0031930 con 0x7f46d80b36e0 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 -- 172.21.15.73:0/1571756712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80b36e0 msgr2=0x7f46d80cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 --2- 172.21.15.73:0/1571756712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80b36e0 0x7f46d80cecd0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f46d80cf890 tx=0x7f46c002b840 comp rx=0 tx=0).stop 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 -- 172.21.15.73:0/1571756712 shutdown_connections 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 --2- 172.21.15.73:0/1571756712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80b36e0 0x7f46d80cecd0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 --2- 172.21.15.73:0/1571756712 >> v2:172.21.15.157:3300/0 conn(0x7f46d815ac00 0x7f46d806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 -- 172.21.15.73:0/1571756712 >> 172.21.15.73:0/1571756712 conn(0x7f46d806a0d0 msgr2=0x7f46d806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:46.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 -- 172.21.15.73:0/1571756712 shutdown_connections 2024-04-05T22:29:46.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.652+0000 7f46dce00640 1 -- 172.21.15.73:0/1571756712 wait complete. 2024-04-05T22:29:46.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.654+0000 7f46dce00640 1 Processor -- start 2024-04-05T22:29:46.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.654+0000 7f46dce00640 1 -- start start 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d80b36e0 0x7f46d815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80cf210 0x7f46d80cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d815ac00 0x7f46d8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80ccda0 con 0x7f46d80cf210 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80cd070 con 0x7f46d80b36e0 2024-04-05T22:29:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46dce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80cd350 con 0x7f46d815ac00 2024-04-05T22:29:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80cf210 0x7f46d80cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80cf210 0x7f46d80cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60042/0 (socket says 172.21.15.73:60042) 2024-04-05T22:29:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.655+0000 7f46cfe00640 1 -- 172.21.15.73:0/3111459461 learned_addr learned my addr 172.21.15.73:0/3111459461 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46d5e00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d80b36e0 0x7f46d815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 -- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d815ac00 msgr2=0x7f46d8152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d815ac00 0x7f46d8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 -- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d80b36e0 msgr2=0x7f46d815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d80b36e0 0x7f46d815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 -- 172.21.15.73:0/3111459461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009400 con 0x7f46d80cf210 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46d5e00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d80b36e0 0x7f46d815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46cfe00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80cf210 0x7f46d80cc860 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009890 tx=0x7f46c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0032840 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00329a0 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0032d50 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46dce00640 1 -- 172.21.15.73:0/3111459461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d80cd630 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.656+0000 7f46dce00640 1 -- 172.21.15.73:0/3111459461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d80cdbf0 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.658+0000 7f46dce00640 1 -- 172.21.15.73:0/3111459461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d80cdde0 con 0x7f46d80cf210 2024-04-05T22:29:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.658+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f46c003b3f0 con 0x7f46d80cf210 2024-04-05T22:29:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.659+0000 7f46ce000640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f46b406b1a0 0x7f46b406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.659+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46c00abe40 con 0x7f46d80cf210 2024-04-05T22:29:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.660+0000 7f46d5e00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f46b406b1a0 0x7f46b406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.661+0000 7f46d5e00640 1 --2- 172.21.15.73:0/3111459461 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f46b406b1a0 0x7f46b406d660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006fd0 tx=0x7f46c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.664+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f46c00766f0 con 0x7f46d80cf210 2024-04-05T22:29:46.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.898+0000 7f46dce00640 1 -- 172.21.15.73:0/3111459461 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d8160e30 con 0x7f46b406b1a0 2024-04-05T22:29:46.901 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:46.901+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:46.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:46.902+0000 7f46ce000640 1 -- 172.21.15.73:0/3111459461 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f46c8009d70 con 0x7f46b406b1a0 2024-04-05T22:29:46.902 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:46.918 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:46.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-04-05T22:29:47.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.123+0000 7f2120600640 1 Processor -- start 2024-04-05T22:29:47.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.124+0000 7f2120600640 1 -- start start 2024-04-05T22:29:47.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.125+0000 7f2120600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f211806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.125+0000 7f2120600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211815afe0 con 0x7f21180cf3e0 2024-04-05T22:29:47.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.125+0000 7f2120600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2118161760 con 0x7f21180b36e0 2024-04-05T22:29:47.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.125+0000 7f2120600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211806d4c0 con 0x7f211815ac00 2024-04-05T22:29:47.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.126+0000 7f211e600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21180cf3e0 0x7f211815d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51158/0 (socket says 172.21.15.73:51158) 2024-04-05T22:29:47.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.126+0000 7f211e600640 1 -- 172.21.15.73:0/2228911552 learned_addr learned my addr 172.21.15.73:0/2228911552 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.126+0000 7f211dc00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f211806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.126+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468946395 0 0) 0x7f2110001490 con 0x7f21180cf3e0 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.126+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2100003520 con 0x7f21180cf3e0 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187544374 0 0) 0x7f2100003520 con 0x7f21180cf3e0 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 msgr2=0x7f211806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f211806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 >> v1:172.21.15.157:6789/0 conn(0x7f21180b36e0 legacy=0x7f21180cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2118160b50 con 0x7f21180cf3e0 2024-04-05T22:29:47.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2110001b80 con 0x7f21180cf3e0 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 >> v1:172.21.15.73:6789/0 conn(0x7f21180cf3e0 legacy=0x7f211815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21000039a0 0x7f2100003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f21000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2100004d00 0x7f2100005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211806d4c0 con 0x7f2100004d00 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210000bab0 con 0x7f21000039a0 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210000bc70 con 0x7f211815ac00 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211e600640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2100004d00 0x7f2100005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211dc00640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21000039a0 0x7f2100003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211d200640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f21000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211e600640 1 -- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 msgr2=0x7f21000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211e600640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211815ac00 0x7f21000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.127+0000 7f211e600640 1 -- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21000039a0 msgr2=0x7f2100003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:47.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.128+0000 7f211e600640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21000039a0 0x7f2100003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.128+0000 7f211e600640 1 -- 172.21.15.73:0/2228911552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100003520 con 0x7f2100004d00 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f211e600640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2100004d00 0x7f2100005180 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f211806fba0 tx=0x7f21000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa66cc15ded217 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110034630 con 0x7f2100004d00 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211002f070 con 0x7f2100004d00 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f2117e00640 1 -- 172.21.15.73:0/2228911552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100349f0 con 0x7f2100004d00 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f2120600640 1 -- 172.21.15.73:0/2228911552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2100004d00 msgr2=0x7f2100005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f2120600640 1 --2- 172.21.15.73:0/2228911552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2100004d00 0x7f2100005180 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f211806fba0 tx=0x7f21000057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.129+0000 7f211e600640 1 -- 172.21.15.73:0/2228911552 reap_dead start 2024-04-05T22:29:47.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.130+0000 7f2120600640 1 -- 172.21.15.73:0/2228911552 shutdown_connections 2024-04-05T22:29:47.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.130+0000 7f2120600640 1 -- 172.21.15.73:0/2228911552 >> 172.21.15.73:0/2228911552 conn(0x7f211806a0d0 msgr2=0x7f211806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:47.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.130+0000 7f2120600640 1 -- 172.21.15.73:0/2228911552 shutdown_connections 2024-04-05T22:29:47.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.130+0000 7f2120600640 1 -- 172.21.15.73:0/2228911552 wait complete. 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.131+0000 7f2120600640 1 Processor -- start 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 -- start start 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2100004d00 0x7f21181483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211814b4c0 0x7f21181594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211806fc10 0x7f2118070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180705d0 con 0x7f211814b4c0 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.132+0000 7f2120600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181488e0 con 0x7f2100004d00 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f2120600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2118148e90 con 0x7f211806fc10 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211814b4c0 0x7f21181594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211814b4c0 0x7f21181594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60062/0 (socket says 172.21.15.73:60062) 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 -- 172.21.15.73:0/271389523 learned_addr learned my addr 172.21.15.73:0/271389523 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211dc00640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2100004d00 0x7f21181483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211e600640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211806fc10 0x7f2118070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 -- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211806fc10 msgr2=0x7f2118070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f211806fc10 0x7f2118070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 -- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2100004d00 msgr2=0x7f21181483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2100004d00 0x7f21181483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.133+0000 7f211d200640 1 -- 172.21.15.73:0/271389523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2118149130 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f211d200640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211814b4c0 0x7f21181594d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f210400d570 tx=0x7f210400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104005480 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104003610 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104003980 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f2120600640 1 -- 172.21.15.73:0/271389523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2118149410 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.134+0000 7f2120600640 1 -- 172.21.15.73:0/271389523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21181499d0 con 0x7f211814b4c0 2024-04-05T22:29:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.135+0000 7f2120600640 1 -- 172.21.15.73:0/271389523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2118149b60 con 0x7f211814b4c0 2024-04-05T22:29:47.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.136+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f2104003ae0 con 0x7f211814b4c0 2024-04-05T22:29:47.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.137+0000 7f2115600640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f20f806b1d0 0x7f20f806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.137+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2104089d60 con 0x7f211814b4c0 2024-04-05T22:29:47.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.138+0000 7f211dc00640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f20f806b1d0 0x7f20f806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.139+0000 7f211dc00640 1 --2- 172.21.15.73:0/271389523 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f20f806b1d0 0x7f20f806d690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f210c009a10 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.142+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2104054650 con 0x7f211814b4c0 2024-04-05T22:29:47.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.371+0000 7f2120600640 1 -- 172.21.15.73:0/271389523 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f21181613f0 con 0x7f20f806b1d0 2024-04-05T22:29:47.374 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:47.375+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:47.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.375+0000 7f2115600640 1 -- 172.21.15.73:0/271389523 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f210c004020 con 0x7f20f806b1d0 2024-04-05T22:29:47.375 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:47.395 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:47.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-04-05T22:29:47.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.614+0000 7fd560200640 1 Processor -- start 2024-04-05T22:29:47.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.615+0000 7fd560200640 1 -- start start 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.616+0000 7fd560200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558160490 0x7fd5580cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.616+0000 7fd560200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd5580b3c70 0x7fd5580cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.616+0000 7fd560200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580ce460 con 0x7fd5580b5be0 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.616+0000 7fd560200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5580ce740 con 0x7fd5580b3c70 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.616+0000 7fd560200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580cea20 con 0x7fd558160490 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd5580b3c70 0x7fd5580cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd5580b3c70 0x7fd5580cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60074/0 (socket says 172.21.15.73:60074) 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 -- 172.21.15.73:0/3772328388 learned_addr learned my addr 172.21.15.73:0/3772328388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 -- 172.21.15.73:0/3772328388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558160490 msgr2=0x7fd5580cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 --2- 172.21.15.73:0/3772328388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558160490 0x7fd5580cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 -- 172.21.15.73:0/3772328388 >> v1:172.21.15.157:6789/0 conn(0x7fd5580b5be0 legacy=0x7fd558160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:47.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.617+0000 7fd55ce00640 1 -- 172.21.15.73:0/3772328388 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580ced00 con 0x7fd5580b3c70 2024-04-05T22:29:47.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.618+0000 7fd55ce00640 1 --2- 172.21.15.73:0/3772328388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b3c70 0x7fd5580cf660 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd54402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2eea8411951b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.619+0000 7fd557e00640 1 -- 172.21.15.73:0/3772328388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544031640 con 0x7fd5580b3c70 2024-04-05T22:29:47.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.619+0000 7fd557e00640 1 -- 172.21.15.73:0/3772328388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5440317a0 con 0x7fd5580b3c70 2024-04-05T22:29:47.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.619+0000 7fd557e00640 1 -- 172.21.15.73:0/3772328388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544031b30 con 0x7fd5580b3c70 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.619+0000 7fd560200640 1 -- 172.21.15.73:0/3772328388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b3c70 msgr2=0x7fd5580cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.619+0000 7fd560200640 1 --2- 172.21.15.73:0/3772328388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b3c70 0x7fd5580cf660 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd54402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 -- 172.21.15.73:0/3772328388 shutdown_connections 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 --2- 172.21.15.73:0/3772328388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b3c70 0x7fd5580cf660 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 --2- 172.21.15.73:0/3772328388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558160490 0x7fd5580cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 -- 172.21.15.73:0/3772328388 >> 172.21.15.73:0/3772328388 conn(0x7fd55809c020 msgr2=0x7fd5580d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 -- 172.21.15.73:0/3772328388 shutdown_connections 2024-04-05T22:29:47.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.620+0000 7fd560200640 1 -- 172.21.15.73:0/3772328388 wait complete. 2024-04-05T22:29:47.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.621+0000 7fd560200640 1 Processor -- start 2024-04-05T22:29:47.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.622+0000 7fd560200640 1 -- start start 2024-04-05T22:29:47.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.622+0000 7fd560200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5580b3c70 0x7fd55815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.622+0000 7fd560200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b5be0 0x7fd5581f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.622+0000 7fd560200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558160490 0x7fd55815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd560200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c9e60 con 0x7fd5580b5be0 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd560200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580cae70 con 0x7fd558160490 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd560200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580ca0b0 con 0x7fd5580b3c70 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b5be0 0x7fd5581f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b5be0 0x7fd5581f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60088/0 (socket says 172.21.15.73:60088) 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 -- 172.21.15.73:0/2202935964 learned_addr learned my addr 172.21.15.73:0/2202935964 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55e200640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558160490 0x7fd55815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 -- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5580b3c70 msgr2=0x7fd55815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55d800640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5580b3c70 0x7fd55815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5580b3c70 0x7fd55815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 -- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558160490 msgr2=0x7fd55815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:47.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558160490 0x7fd55815ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55ce00640 1 -- 172.21.15.73:0/2202935964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440094d0 con 0x7fd5580b5be0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55e200640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558160490 0x7fd55815ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.623+0000 7fd55d800640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5580b3c70 0x7fd55815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd55ce00640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5580b5be0 0x7fd5581f2220 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd5440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54403ac60 con 0x7fd5580b5be0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54403adc0 con 0x7fd5580b5be0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544044630 con 0x7fd5580b5be0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd560200640 1 -- 172.21.15.73:0/2202935964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580ca390 con 0x7fd5580b5be0 2024-04-05T22:29:47.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.624+0000 7fd560200640 1 -- 172.21.15.73:0/2202935964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5580ca950 con 0x7fd5580b5be0 2024-04-05T22:29:47.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.625+0000 7fd560200640 1 -- 172.21.15.73:0/2202935964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55809c4c0 con 0x7fd5580b5be0 2024-04-05T22:29:47.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.626+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fd54402f050 con 0x7fd5580b5be0 2024-04-05T22:29:47.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.627+0000 7fd555600640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd53806b1a0 0x7fd53806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:47.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.628+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5440abf80 con 0x7fd5580b5be0 2024-04-05T22:29:47.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.628+0000 7fd55d800640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd53806b1a0 0x7fd53806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:47.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.629+0000 7fd55d800640 1 --2- 172.21.15.73:0/2202935964 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd53806b1a0 0x7fd53806d660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00a870 tx=0x7fd54c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:47.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.632+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd544076750 con 0x7fd5580b5be0 2024-04-05T22:29:47.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.858+0000 7fd560200640 1 -- 172.21.15.73:0/2202935964 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5580cdee0 con 0x7fd53806b1a0 2024-04-05T22:29:47.861 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:47.861+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:47.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:47.862+0000 7fd555600640 1 -- 172.21.15.73:0/2202935964 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fd54c00c7d0 con 0x7fd53806b1a0 2024-04-05T22:29:47.861 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:47.876 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:47.876 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-04-05T22:29:48.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.091+0000 7fbf3d600640 1 Processor -- start 2024-04-05T22:29:48.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.091+0000 7fbf3d600640 1 -- start start 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf3d600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf38154580 0x7fbf38164c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf3d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3815d900 con 0x7fbf381541a0 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf3d600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3815e1a0 con 0x7fbf380cf3e0 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf3d600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf380b3490 con 0x7fbf38154580 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf36a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbf381541a0 0x7fbf380cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51168/0 (socket says 172.21.15.73:51168) 2024-04-05T22:29:48.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.093+0000 7fbf36a00640 1 -- 172.21.15.73:0/1253117087 learned_addr learned my addr 172.21.15.73:0/1253117087 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:48.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf37400640 1 --2- 172.21.15.73:0/1253117087 >> v2:172.21.15.157:3300/0 conn(0x7fbf38154580 0x7fbf38164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055624793 0 0) 0x7fbf3815d900 con 0x7fbf381541a0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf18003520 con 0x7fbf381541a0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902834435 0 0) 0x7fbf18003520 con 0x7fbf381541a0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 >> v2:172.21.15.157:3300/0 conn(0x7fbf38154580 msgr2=0x7fbf38164c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 --2- 172.21.15.73:0/1253117087 >> v2:172.21.15.157:3300/0 conn(0x7fbf38154580 0x7fbf38164c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 >> v1:172.21.15.157:6789/0 conn(0x7fbf380cf3e0 legacy=0x7fbf380b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3815db50 con 0x7fbf381541a0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbf28001670 con 0x7fbf381541a0 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf37400640 1 --2- 172.21.15.73:0/1253117087 >> v2:172.21.15.157:3300/0 conn(0x7fbf38154580 0x7fbf38164c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 >> v1:172.21.15.73:6789/0 conn(0x7fbf381541a0 legacy=0x7fbf380cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf180039c0 0x7fbf18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 0x7fbf18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180051d0 0x7fbf1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380b3490 con 0x7fbf180047c0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1800c120 con 0x7fbf180051d0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.094+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1800c2e0 con 0x7fbf180039c0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 0x7fbf18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36a00640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf180039c0 0x7fbf18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 -- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf180039c0 msgr2=0x7fbf18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf180039c0 0x7fbf18003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 -- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180051d0 msgr2=0x7fbf1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180051d0 0x7fbf1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf36000640 1 -- 172.21.15.73:0/1253117087 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18003520 con 0x7fbf180047c0 2024-04-05T22:29:48.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.095+0000 7fbf37400640 1 -- 172.21.15.73:0/1253117087 reap_dead start 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.096+0000 7fbf36000640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 0x7fbf18004c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbf20001e50 tx=0x7fbf180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcea5e11c9e5da4c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.096+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20003b70 con 0x7fbf180047c0 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.096+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20003cd0 con 0x7fbf180047c0 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.096+0000 7fbf35600640 1 -- 172.21.15.73:0/1253117087 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20005310 con 0x7fbf180047c0 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 -- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 msgr2=0x7fbf18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 0x7fbf18004c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbf20001e50 tx=0x7fbf180057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 -- 172.21.15.73:0/1253117087 shutdown_connections 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 --2- 172.21.15.73:0/1253117087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf180047c0 0x7fbf18004c40 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 -- 172.21.15.73:0/1253117087 >> 172.21.15.73:0/1253117087 conn(0x7fbf3806a0d0 msgr2=0x7fbf3806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 -- 172.21.15.73:0/1253117087 shutdown_connections 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.097+0000 7fbf3d600640 1 -- 172.21.15.73:0/1253117087 wait complete. 2024-04-05T22:29:48.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.099+0000 7fbf3d600640 1 Processor -- start 2024-04-05T22:29:48.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.099+0000 7fbf3d600640 1 -- start start 2024-04-05T22:29:48.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180047c0 0x7fbf38146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf38156bb0 0x7fbf380ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf380ac8d0 0x7fbf380acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38156a30 con 0x7fbf380ac8d0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380ad290 con 0x7fbf180047c0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf3d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380ae980 con 0x7fbf38156bb0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf380ac8d0 0x7fbf380acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf380ac8d0 0x7fbf380acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60110/0 (socket says 172.21.15.73:60110) 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 -- 172.21.15.73:0/3157420704 learned_addr learned my addr 172.21.15.73:0/3157420704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 -- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf38156bb0 msgr2=0x7fbf380ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf38156bb0 0x7fbf380ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 -- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180047c0 msgr2=0x7fbf38146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf36a00640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180047c0 0x7fbf38146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180047c0 0x7fbf38146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf37400640 1 -- 172.21.15.73:0/3157420704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380aebb0 con 0x7fbf380ac8d0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.100+0000 7fbf36a00640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf180047c0 0x7fbf38146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf37400640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf380ac8d0 0x7fbf380acd50 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006fd0 tx=0x7fbf3000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000c070 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30013610 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf3d600640 1 -- 172.21.15.73:0/3157420704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf380aee90 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300139c0 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.101+0000 7fbf3d600640 1 -- 172.21.15.73:0/3157420704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380af450 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.102+0000 7fbf3d600640 1 -- 172.21.15.73:0/3157420704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf38068160 con 0x7fbf380ac8d0 2024-04-05T22:29:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.103+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fbf30013b20 con 0x7fbf380ac8d0 2024-04-05T22:29:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.104+0000 7fbf2ea00640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf1406b1a0 0x7fbf1406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.104+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf30089260 con 0x7fbf380ac8d0 2024-04-05T22:29:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.105+0000 7fbf36a00640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf1406b1a0 0x7fbf1406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.106+0000 7fbf36a00640 1 --2- 172.21.15.73:0/3157420704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf1406b1a0 0x7fbf1406d660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf28001eb0 tx=0x7fbf28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.108+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbf30053a80 con 0x7fbf380ac8d0 2024-04-05T22:29:48.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.332+0000 7fbf3d600640 1 -- 172.21.15.73:0/3157420704 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf380cecd0 con 0x7fbf1406b1a0 2024-04-05T22:29:48.334 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:48.334+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:48.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.335+0000 7fbf2ea00640 1 -- 172.21.15.73:0/3157420704 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fbf2800d040 con 0x7fbf1406b1a0 2024-04-05T22:29:48.334 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:48.350 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:48.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-04-05T22:29:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.572+0000 7f7b09600640 1 Processor -- start 2024-04-05T22:29:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.573+0000 7f7b09600640 1 -- start start 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.574+0000 7f7b09600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b041541a0 0x7f7b040cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.574+0000 7f7b09600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b040b2f50 0x7f7b040cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.574+0000 7f7b09600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0415d900 con 0x7f7b04154580 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.574+0000 7f7b09600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0415e1a0 con 0x7f7b040b2f50 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.574+0000 7f7b09600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040b3490 con 0x7f7b041541a0 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b02000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7b04154580 0x7f7b040b2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51172/0 (socket says 172.21.15.73:51172) 2024-04-05T22:29:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b02000640 1 -- 172.21.15.73:0/607234957 learned_addr learned my addr 172.21.15.73:0/607234957 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b03400640 1 --2- 172.21.15.73:0/607234957 >> v2:172.21.15.73:3300/0 conn(0x7f7b040b2f50 0x7f7b040cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b02a00640 1 --2- 172.21.15.73:0/607234957 >> v2:172.21.15.157:3300/0 conn(0x7f7b041541a0 0x7f7b040cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b03400640 1 -- 172.21.15.73:0/607234957 >> v2:172.21.15.157:3300/0 conn(0x7f7b041541a0 msgr2=0x7f7b040cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b01600640 1 -- 172.21.15.73:0/607234957 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410897246 0 0) 0x7f7b0415d900 con 0x7f7b04154580 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b03400640 1 --2- 172.21.15.73:0/607234957 >> v2:172.21.15.157:3300/0 conn(0x7f7b041541a0 0x7f7b040cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b03400640 1 -- 172.21.15.73:0/607234957 >> v1:172.21.15.73:6789/0 conn(0x7f7b04154580 legacy=0x7f7b040b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.575+0000 7f7b03400640 1 -- 172.21.15.73:0/607234957 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0415db50 con 0x7f7b040b2f50 2024-04-05T22:29:48.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.576+0000 7f7b03400640 1 --2- 172.21.15.73:0/607234957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b040cf210 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7b04070020 tx=0x7f7afc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaa25232631fee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.576+0000 7f7b01600640 1 -- 172.21.15.73:0/607234957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc031830 con 0x7f7b040b2f50 2024-04-05T22:29:48.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b01600640 1 -- 172.21.15.73:0/607234957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc031990 con 0x7f7b040b2f50 2024-04-05T22:29:48.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b01600640 1 -- 172.21.15.73:0/607234957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc031d20 con 0x7f7b040b2f50 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 -- 172.21.15.73:0/607234957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 msgr2=0x7f7b040cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 --2- 172.21.15.73:0/607234957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b040cf210 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7b04070020 tx=0x7f7afc02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 -- 172.21.15.73:0/607234957 shutdown_connections 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 --2- 172.21.15.73:0/607234957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b040cf210 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 --2- 172.21.15.73:0/607234957 >> v2:172.21.15.157:3300/0 conn(0x7f7b041541a0 0x7f7b040cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 -- 172.21.15.73:0/607234957 >> 172.21.15.73:0/607234957 conn(0x7f7b0406a0d0 msgr2=0x7f7b0406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.577+0000 7f7b09600640 1 -- 172.21.15.73:0/607234957 shutdown_connections 2024-04-05T22:29:48.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.578+0000 7f7b09600640 1 -- 172.21.15.73:0/607234957 wait complete. 2024-04-05T22:29:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.580+0000 7f7b09600640 1 Processor -- start 2024-04-05T22:29:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.580+0000 7f7b09600640 1 -- start start 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b0414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7b041596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b04154580 0x7f7b040ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0414c9b0 con 0x7f7b040b2f50 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040af380 con 0x7f7b04154580 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b09600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040ae770 con 0x7f7b041541a0 2024-04-05T22:29:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b02a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b0414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b02a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b0414c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60124/0 (socket says 172.21.15.73:60124) 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.581+0000 7f7b02a00640 1 -- 172.21.15.73:0/1762786660 learned_addr learned my addr 172.21.15.73:0/1762786660 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 -- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 msgr2=0x7f7b041596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7b041596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 -- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b04154580 msgr2=0x7f7b040ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b04154580 0x7f7b040ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 -- 172.21.15.73:0/1762786660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009400 con 0x7f7b040b2f50 2024-04-05T22:29:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7b02a00640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b040b2f50 0x7f7b0414c470 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d800 tx=0x7f7af400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400b6c0 con 0x7f7b040b2f50 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af400b820 con 0x7f7b040b2f50 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.582+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400bbb0 con 0x7f7b040b2f50 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.583+0000 7f7b09600640 1 -- 172.21.15.73:0/1762786660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040aea50 con 0x7f7b040b2f50 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.583+0000 7f7b09600640 1 -- 172.21.15.73:0/1762786660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040af010 con 0x7f7b040b2f50 2024-04-05T22:29:48.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.584+0000 7f7b09600640 1 -- 172.21.15.73:0/1762786660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04068160 con 0x7f7b040b2f50 2024-04-05T22:29:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.585+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f7af400bd10 con 0x7f7b040b2f50 2024-04-05T22:29:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.586+0000 7f7afaa00640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b1a0 0x7f7ae006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.586+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af408a2d0 con 0x7f7b040b2f50 2024-04-05T22:29:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.586+0000 7f7b02000640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b1a0 0x7f7ae006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:48.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.587+0000 7f7b02000640 1 --2- 172.21.15.73:0/1762786660 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b1a0 0x7f7ae006d660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007fd0 tx=0x7f7aec007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:48.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.590+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7af4054c90 con 0x7f7b040b2f50 2024-04-05T22:29:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.819+0000 7f7b09600640 1 -- 172.21.15.73:0/1762786660 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0415a310 con 0x7f7ae006b1a0 2024-04-05T22:29:48.821 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:48.822+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:48.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:48.823+0000 7f7afaa00640 1 -- 172.21.15.73:0/1762786660 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f7aec00c730 con 0x7f7ae006b1a0 2024-04-05T22:29:48.822 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:48.834 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:48.835 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-04-05T22:29:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.049+0000 7f7904e00640 1 Processor -- start 2024-04-05T22:29:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.050+0000 7f7904e00640 1 -- start start 2024-04-05T22:29:49.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.051+0000 7f7904e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79000b3aa0 0x7f79000cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.051+0000 7f7904e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790015b230 con 0x7f790015ae50 2024-04-05T22:29:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.051+0000 7f7904e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790015f4a0 con 0x7f79000cf3e0 2024-04-05T22:29:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.051+0000 7f7904e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79000b3f30 con 0x7f79000b3aa0 2024-04-05T22:29:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fde00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f790015ae50 0x7f79000cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51182/0 (socket says 172.21.15.73:51182) 2024-04-05T22:29:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fde00640 1 -- 172.21.15.73:0/2444269238 learned_addr learned my addr 172.21.15.73:0/2444269238 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891070325 0 0) 0x7f790015b230 con 0x7f790015ae50 2024-04-05T22:29:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fe800640 1 --2- 172.21.15.73:0/2444269238 >> v2:172.21.15.157:3300/0 conn(0x7f79000b3aa0 0x7f79000cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e4003520 con 0x7f790015ae50 2024-04-05T22:29:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995599113 0 0) 0x7f78e4003520 con 0x7f790015ae50 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 >> v2:172.21.15.157:3300/0 conn(0x7f79000b3aa0 msgr2=0x7f79000cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 --2- 172.21.15.73:0/2444269238 >> v2:172.21.15.157:3300/0 conn(0x7f79000b3aa0 0x7f79000cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.052+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 >> v1:172.21.15.157:6789/0 conn(0x7f79000cf3e0 legacy=0x7f79000b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790015f780 con 0x7f790015ae50 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f78f0001670 con 0x7f790015ae50 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 >> v1:172.21.15.73:6789/0 conn(0x7f790015ae50 legacy=0x7f79000cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e4003970 0x7f78e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78e40052a0 0x7f78e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000b3f30 con 0x7f78e40047c0 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e400c170 con 0x7f78e40052a0 2024-04-05T22:29:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e400c330 con 0x7f78e4003970 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fd400640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fde00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e4003970 0x7f78e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fe800640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78e40052a0 0x7f78e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fd400640 1 -- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e4003970 msgr2=0x7f78e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fd400640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e4003970 0x7f78e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fd400640 1 -- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78e40052a0 msgr2=0x7f78e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.053+0000 7f78fd400640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78e40052a0 0x7f78e400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.054+0000 7f78fd400640 1 -- 172.21.15.73:0/2444269238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4003520 con 0x7f78e40047c0 2024-04-05T22:29:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.054+0000 7f78fe800640 1 -- 172.21.15.73:0/2444269238 reap_dead start 2024-04-05T22:29:49.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.055+0000 7f78fd400640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f79000ae060 tx=0x7f78e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce0df01cb1f9063 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.055+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80052d0 con 0x7f78e40047c0 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.055+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e802e050 con 0x7f78e40047c0 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.055+0000 7f78fca00640 1 -- 172.21.15.73:0/2444269238 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8003830 con 0x7f78e40047c0 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 -- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 msgr2=0x7f78e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f79000ae060 tx=0x7f78e40057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 -- 172.21.15.73:0/2444269238 shutdown_connections 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 --2- 172.21.15.73:0/2444269238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 -- 172.21.15.73:0/2444269238 >> 172.21.15.73:0/2444269238 conn(0x7f790006a0d0 msgr2=0x7f790006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 -- 172.21.15.73:0/2444269238 shutdown_connections 2024-04-05T22:29:49.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.056+0000 7f7904e00640 1 -- 172.21.15.73:0/2444269238 wait complete. 2024-04-05T22:29:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.058+0000 7f7904e00640 1 Processor -- start 2024-04-05T22:29:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.058+0000 7f7904e00640 1 -- start start 2024-04-05T22:29:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e40047c0 0x7f79000b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79000ad2a0 0x7f79001f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79000c5170 0x7f79000c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000c5b30 con 0x7f79000c5170 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790015d6c0 con 0x7f79000ad2a0 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f7904e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790015d9a0 con 0x7f78e40047c0 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f78fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79000c5170 0x7f79000c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f78fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79000c5170 0x7f79000c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60146/0 (socket says 172.21.15.73:60146) 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f78fe800640 1 -- 172.21.15.73:0/1142297761 learned_addr learned my addr 172.21.15.73:0/1142297761 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:49.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f78fde00640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e40047c0 0x7f79000b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.060+0000 7f78fd400640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79000ad2a0 0x7f79001f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 -- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e40047c0 msgr2=0x7f79000b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78e40047c0 0x7f79000b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 -- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79000ad2a0 msgr2=0x7f79001f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79000ad2a0 0x7f79001f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 -- 172.21.15.73:0/1142297761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790015dc80 con 0x7f79000c5170 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78fe800640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79000c5170 0x7f79000c55f0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f79000cf870 tx=0x7f78f8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8007c90 con 0x7f79000c5170 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f8007df0 con 0x7f79000c5170 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8014640 con 0x7f79000c5170 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.061+0000 7f7904e00640 1 -- 172.21.15.73:0/1142297761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790015df60 con 0x7f79000c5170 2024-04-05T22:29:49.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.062+0000 7f7904e00640 1 -- 172.21.15.73:0/1142297761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790015d390 con 0x7f79000c5170 2024-04-05T22:29:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.063+0000 7f7904e00640 1 -- 172.21.15.73:0/1142297761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790014cc00 con 0x7f79000c5170 2024-04-05T22:29:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.064+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f78f801d3b0 con 0x7f79000c5170 2024-04-05T22:29:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.065+0000 7f78f6000640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f78dc06b1a0 0x7f78dc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.065+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78f8089b50 con 0x7f79000c5170 2024-04-05T22:29:49.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.065+0000 7f78fde00640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f78dc06b1a0 0x7f78dc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.066+0000 7f78fde00640 1 --2- 172.21.15.73:0/1142297761 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f78dc06b1a0 0x7f78dc06d660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f78f0001c90 tx=0x7f78f0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.069+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f790014cc00 con 0x7f79000c5170 2024-04-05T22:29:49.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.298+0000 7f7904e00640 1 -- 172.21.15.73:0/1142297761 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f79000cb610 con 0x7f78dc06b1a0 2024-04-05T22:29:49.301 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:49.302+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:49.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.302+0000 7f78f6000640 1 -- 172.21.15.73:0/1142297761 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f78f00119a0 con 0x7f78dc06b1a0 2024-04-05T22:29:49.302 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:49.318 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:49.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-04-05T22:29:49.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.537+0000 7f03cb200640 1 Processor -- start 2024-04-05T22:29:49.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.537+0000 7f03cb200640 1 -- start start 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.538+0000 7f03cb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03cc0b3180 0x7f03cc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.538+0000 7f03cb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc0b3560 con 0x7f03cc06fdb0 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.538+0000 7f03cb200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc0b2670 con 0x7f03cc0adc70 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.538+0000 7f03cb200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03cc15a720 con 0x7f03cc0b3180 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03c9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03cc0b3180 0x7f03cc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03ca800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f03cc06fdb0 0x7f03cc14f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51190/0 (socket says 172.21.15.73:51190) 2024-04-05T22:29:49.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03ca800640 1 -- 172.21.15.73:0/3637672599 learned_addr learned my addr 172.21.15.73:0/3637672599 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:49.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03c9e00640 1 -- 172.21.15.73:0/3637672599 >> v1:172.21.15.157:6789/0 conn(0x7f03cc0adc70 legacy=0x7f03cc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:49.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03c9e00640 1 -- 172.21.15.73:0/3637672599 >> v1:172.21.15.73:6789/0 conn(0x7f03cc06fdb0 legacy=0x7f03cc14f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03c9e00640 1 -- 172.21.15.73:0/3637672599 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc06bea0 con 0x7f03cc0b3180 2024-04-05T22:29:49.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.539+0000 7f03c9400640 1 -- 172.21.15.73:0/3637672599 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048176767 0 0) 0x7f03cc0b3560 con 0x7f03cc06fdb0 2024-04-05T22:29:49.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.540+0000 7f03c9e00640 1 --2- 172.21.15.73:0/3637672599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc15a1e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0097a0 tx=0x7f03bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a14563691ff7db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03c9400640 1 -- 172.21.15.73:0/3637672599 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc031830 con 0x7f03cc0b3180 2024-04-05T22:29:49.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03c9400640 1 -- 172.21.15.73:0/3637672599 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc031990 con 0x7f03cc0b3180 2024-04-05T22:29:49.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03c9400640 1 -- 172.21.15.73:0/3637672599 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc031d20 con 0x7f03cc0b3180 2024-04-05T22:29:49.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 -- 172.21.15.73:0/3637672599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 msgr2=0x7f03cc15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 --2- 172.21.15.73:0/3637672599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc15a1e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0097a0 tx=0x7f03bc02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 -- 172.21.15.73:0/3637672599 shutdown_connections 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 --2- 172.21.15.73:0/3637672599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc15a1e0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 -- 172.21.15.73:0/3637672599 >> 172.21.15.73:0/3637672599 conn(0x7f03cc0b0360 msgr2=0x7f03cc0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 -- 172.21.15.73:0/3637672599 shutdown_connections 2024-04-05T22:29:49.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.541+0000 7f03cb200640 1 -- 172.21.15.73:0/3637672599 wait complete. 2024-04-05T22:29:49.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.543+0000 7f03cb200640 1 Processor -- start 2024-04-05T22:29:49.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.543+0000 7f03cb200640 1 -- start start 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03cc06fdb0 0x7f03cc140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03cc0adc70 0x7f03cc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc142070 con 0x7f03cc0b3180 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc154510 con 0x7f03cc0adc70 2024-04-05T22:29:49.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.544+0000 7f03cb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1f2530 con 0x7f03cc06fdb0 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60158/0 (socket says 172.21.15.73:60158) 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 -- 172.21.15.73:0/2448507256 learned_addr learned my addr 172.21.15.73:0/2448507256 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03c9e00640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03cc06fdb0 0x7f03cc140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03c1400640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03cc0adc70 0x7f03cc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 -- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03cc06fdb0 msgr2=0x7f03cc140a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03cc06fdb0 0x7f03cc140a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 -- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03cc0adc70 msgr2=0x7f03cc15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03cc0adc70 0x7f03cc15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 -- 172.21.15.73:0/2448507256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc009400 con 0x7f03cc0b3180 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03c1400640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03cc0adc70 0x7f03cc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.545+0000 7f03ca800640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03cc0b3180 0x7f03cc1445b0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c700 tx=0x7f03c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.546+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400ce10 con 0x7f03cc0b3180 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.546+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c400a640 con 0x7f03cc0b3180 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.546+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400a9f0 con 0x7f03cc0b3180 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.546+0000 7f03cb200640 1 -- 172.21.15.73:0/2448507256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc1413f0 con 0x7f03cc0b3180 2024-04-05T22:29:49.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.546+0000 7f03cb200640 1 -- 172.21.15.73:0/2448507256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc1419b0 con 0x7f03cc0b3180 2024-04-05T22:29:49.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.547+0000 7f03cb200640 1 -- 172.21.15.73:0/2448507256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc1421b0 con 0x7f03cc0b3180 2024-04-05T22:29:49.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.548+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f03c400ab50 con 0x7f03cc0b3180 2024-04-05T22:29:49.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.549+0000 7f03c2a00640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f03a806b1a0 0x7f03a806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:49.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.549+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03c40891a0 con 0x7f03cc0b3180 2024-04-05T22:29:49.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.550+0000 7f03c9e00640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f03a806b1a0 0x7f03a806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:49.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.551+0000 7f03c9e00640 1 --2- 172.21.15.73:0/2448507256 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f03a806b1a0 0x7f03a806d660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036000 tx=0x7f03bc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:49.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.553+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f03cc1421b0 con 0x7f03cc0b3180 2024-04-05T22:29:49.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.783+0000 7f03cb200640 1 -- 172.21.15.73:0/2448507256 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f03cc06c740 con 0x7f03a806b1a0 2024-04-05T22:29:49.785 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:49.786+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:49.787+0000 7f03c2a00640 1 -- 172.21.15.73:0/2448507256 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f03bc031d20 con 0x7f03a806b1a0 2024-04-05T22:29:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:49.800 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:49.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-04-05T22:29:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.002+0000 7fe0be600640 1 Processor -- start 2024-04-05T22:29:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.002+0000 7fe0be600640 1 -- start start 2024-04-05T22:29:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0be600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe0b80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0be600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b80cd2e0 con 0x7fe0b815ae50 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0be600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b815b230 con 0x7fe0b80cce90 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0be600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80ce2f0 con 0x7fe0b80cf3e0 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0b7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe0b815ae50 0x7fe0b80cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51204/0 (socket says 172.21.15.73:51204) 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0b7400640 1 -- 172.21.15.73:0/3736047806 learned_addr learned my addr 172.21.15.73:0/3736047806 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.004+0000 7fe0afe00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe0b80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82185822 0 0) 0x7fe0b80cd2e0 con 0x7fe0b815ae50 2024-04-05T22:29:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c003520 con 0x7fe0b815ae50 2024-04-05T22:29:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300219042 0 0) 0x7fe09c003520 con 0x7fe0b815ae50 2024-04-05T22:29:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 msgr2=0x7fe0b80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe0b80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 >> v1:172.21.15.157:6789/0 conn(0x7fe0b80cce90 legacy=0x7fe0b80b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b815f4a0 con 0x7fe0b815ae50 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe0a8001690 con 0x7fe0b815ae50 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 >> v1:172.21.15.73:6789/0 conn(0x7fe0b815ae50 legacy=0x7fe0b80cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe09c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c003f20 0x7fe09c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c004d00 0x7fe09c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80ce2f0 con 0x7fe09c003f20 2024-04-05T22:29:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00bb10 con 0x7fe09c004d00 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00bcd0 con 0x7fe0b80cf3e0 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.005+0000 7fe0afe00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c003f20 0x7fe09c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0b7400640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe09c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0b7e00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c004d00 0x7fe09c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0afe00640 1 -- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 msgr2=0x7fe09c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0afe00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe09c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0afe00640 1 -- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c004d00 msgr2=0x7fe09c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0afe00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c004d00 0x7fe09c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0afe00640 1 -- 172.21.15.73:0/3736047806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c003520 con 0x7fe09c003f20 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0b7400640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b80cf3e0 0x7fe09c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.006+0000 7fe0b7e00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c004d00 0x7fe09c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.007+0000 7fe0afe00640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c003f20 0x7fe09c0047c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009ac0 tx=0x7fe09c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb7f469c342bc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.007+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a002c6b0 con 0x7fe09c003f20 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.007+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a002c810 con 0x7fe09c003f20 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.007+0000 7fe0b6a00640 1 -- 172.21.15.73:0/3736047806 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a002cba0 con 0x7fe09c003f20 2024-04-05T22:29:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 -- 172.21.15.73:0/3736047806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c003f20 msgr2=0x7fe09c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 --2- 172.21.15.73:0/3736047806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c003f20 0x7fe09c0047c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009ac0 tx=0x7fe09c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0b7e00640 1 -- 172.21.15.73:0/3736047806 reap_dead start 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 -- 172.21.15.73:0/3736047806 shutdown_connections 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 -- 172.21.15.73:0/3736047806 >> 172.21.15.73:0/3736047806 conn(0x7fe0b806a0d0 msgr2=0x7fe0b806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 -- 172.21.15.73:0/3736047806 shutdown_connections 2024-04-05T22:29:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.008+0000 7fe0be600640 1 -- 172.21.15.73:0/3736047806 wait complete. 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.010+0000 7fe0be600640 1 Processor -- start 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.010+0000 7fe0be600640 1 -- start start 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c003f20 0x7fe0b814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b8070080 0x7fe0b80c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0b80c4250 0x7fe0b80c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80c4c10 con 0x7fe0b80c4250 2024-04-05T22:29:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80711f0 con 0x7fe09c003f20 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0be600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8071680 con 0x7fe0b8070080 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0b80c4250 0x7fe0b80c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0b80c4250 0x7fe0b80c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60178/0 (socket says 172.21.15.73:60178) 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.011+0000 7fe0b7e00640 1 -- 172.21.15.73:0/1979681488 learned_addr learned my addr 172.21.15.73:0/1979681488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7400640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c003f20 0x7fe0b814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0afe00640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b8070080 0x7fe0b80c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 -- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b8070080 msgr2=0x7fe0b80c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b8070080 0x7fe0b80c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 -- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c003f20 msgr2=0x7fe0b814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c003f20 0x7fe0b814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 -- 172.21.15.73:0/1979681488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80705e0 con 0x7fe0b80c4250 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7400640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe09c003f20 0x7fe0b814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0afe00640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0b8070080 0x7fe0b80c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0b7e00640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0b80c4250 0x7fe0b80c46d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8071a60 tx=0x7fe0b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0006d50 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0003970 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0003d60 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.012+0000 7fe0be600640 1 -- 172.21.15.73:0/1979681488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b80708c0 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.013+0000 7fe0be600640 1 -- 172.21.15.73:0/1979681488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b8070e80 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.014+0000 7fe0be600640 1 -- 172.21.15.73:0/1979681488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b8068160 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.015+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fe0b0005070 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.015+0000 7fe0af400640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09406b1d0 0x7fe09406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.016+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b0089bb0 con 0x7fe0b80c4250 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.016+0000 7fe0b7400640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09406b1d0 0x7fe09406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.017+0000 7fe0b7400640 1 --2- 172.21.15.73:0/1979681488 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09406b1d0 0x7fe09406d690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80017c0 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.020+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe0b0054460 con 0x7fe0b80c4250 2024-04-05T22:29:50.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.252+0000 7fe0be600640 1 -- 172.21.15.73:0/1979681488 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0b80cb610 con 0x7fe09406b1d0 2024-04-05T22:29:50.255 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:50.256+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:50.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.256+0000 7fe0af400640 1 -- 172.21.15.73:0/1979681488 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fe0a800d040 con 0x7fe09406b1d0 2024-04-05T22:29:50.256 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:50.274 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:50.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-04-05T22:29:50.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.488+0000 7f0c72000640 1 Processor -- start 2024-04-05T22:29:50.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.488+0000 7f0c72000640 1 -- start start 2024-04-05T22:29:50.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.489+0000 7f0c72000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0c6c0cecd0 0x7f0c6c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.489+0000 7f0c72000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 0x7f0c6c164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.489+0000 7f0c72000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c1620e0 con 0x7f0c6c0b3ab0 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.489+0000 7f0c72000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c0ae370 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.489+0000 7f0c72000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c0ae090 con 0x7f0c6c0ad720 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0c6c0cecd0 0x7f0c6c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0c6c0b3ab0 0x7f0c6c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51210/0 (socket says 172.21.15.73:51210) 2024-04-05T22:29:50.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6aa00640 1 -- 172.21.15.73:0/2915405305 learned_addr learned my addr 172.21.15.73:0/2915405305 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6be00640 1 --2- 172.21.15.73:0/2915405305 >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 0x7f0c6c164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6b400640 1 -- 172.21.15.73:0/2915405305 >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 msgr2=0x7f0c6c164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6b400640 1 --2- 172.21.15.73:0/2915405305 >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 0x7f0c6c164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6b400640 1 -- 172.21.15.73:0/2915405305 >> v1:172.21.15.73:6789/0 conn(0x7f0c6c0b3ab0 legacy=0x7f0c6c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6b400640 1 -- 172.21.15.73:0/2915405305 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c161a90 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6a000640 1 -- 172.21.15.73:0/2915405305 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879338869 0 0) 0x7f0c6c1620e0 con 0x7f0c6c0b3ab0 2024-04-05T22:29:50.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.490+0000 7f0c6be00640 1 --2- 172.21.15.73:0/2915405305 >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 0x7f0c6c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.491+0000 7f0c6b400640 1 --2- 172.21.15.73:0/2915405305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c162570 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0b05e0 tx=0x7f0c5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d76d0da5152b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c6a000640 1 -- 172.21.15.73:0/2915405305 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c031830 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c6a000640 1 -- 172.21.15.73:0/2915405305 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c031990 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c6a000640 1 -- 172.21.15.73:0/2915405305 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c031d20 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 -- 172.21.15.73:0/2915405305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 msgr2=0x7f0c6c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 --2- 172.21.15.73:0/2915405305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c162570 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0b05e0 tx=0x7f0c5c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 -- 172.21.15.73:0/2915405305 shutdown_connections 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 --2- 172.21.15.73:0/2915405305 >> v2:172.21.15.157:3300/0 conn(0x7f0c6c0ad720 0x7f0c6c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 --2- 172.21.15.73:0/2915405305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c162570 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 -- 172.21.15.73:0/2915405305 >> 172.21.15.73:0/2915405305 conn(0x7f0c6c06a0d0 msgr2=0x7f0c6c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.492+0000 7f0c72000640 1 -- 172.21.15.73:0/2915405305 shutdown_connections 2024-04-05T22:29:50.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.493+0000 7f0c72000640 1 -- 172.21.15.73:0/2915405305 wait complete. 2024-04-05T22:29:50.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.494+0000 7f0c72000640 1 Processor -- start 2024-04-05T22:29:50.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.494+0000 7f0c72000640 1 -- start start 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0ad720 0x7f0c6c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c6c0b3ab0 0x7f0c6c1653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c155d70 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c156050 con 0x7f0c6c0ad720 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c72000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c156330 con 0x7f0c6c0b3ab0 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c6be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.495+0000 7f0c6be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c06fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60196/0 (socket says 172.21.15.73:60196) 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 -- 172.21.15.73:0/1029834986 learned_addr learned my addr 172.21.15.73:0/1029834986 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6aa00640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c6c0b3ab0 0x7f0c6c1653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 -- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c6c0b3ab0 msgr2=0x7f0c6c1653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6b400640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0ad720 0x7f0c6c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c6c0b3ab0 0x7f0c6c1653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 -- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0ad720 msgr2=0x7f0c6c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c6c0ad720 0x7f0c6c153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 -- 172.21.15.73:0/1029834986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009400 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c6be00640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0cecd0 0x7f0c6c06fd20 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400cd50 tx=0x7f0c6400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400f680 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400f800 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.496+0000 7f0c72000640 1 -- 172.21.15.73:0/1029834986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c156610 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.497+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400fbb0 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.497+0000 7f0c72000640 1 -- 172.21.15.73:0/1029834986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c154420 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.498+0000 7f0c72000640 1 -- 172.21.15.73:0/1029834986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c06d660 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.499+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f0c6400b040 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.500+0000 7f0c63400640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0c4c06b1a0 0x7f0c4c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.500+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c6408a8e0 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.500+0000 7f0c6b400640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0c4c06b1a0 0x7f0c4c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.501+0000 7f0c6b400640 1 --2- 172.21.15.73:0/1029834986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0c4c06b1a0 0x7f0c4c06d660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c036000 tx=0x7f0c5c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.505+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0c64055140 con 0x7f0c6c0cecd0 2024-04-05T22:29:50.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.727+0000 7f0c72000640 1 -- 172.21.15.73:0/1029834986 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c6c15bcf0 con 0x7f0c4c06b1a0 2024-04-05T22:29:50.730 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:50.731+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:50.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.731+0000 7f0c63400640 1 -- 172.21.15.73:0/1029834986 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f0c5c031b80 con 0x7f0c4c06b1a0 2024-04-05T22:29:50.731 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:50.740 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:50.741 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-04-05T22:29:50.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.949+0000 7f891ec00640 1 Processor -- start 2024-04-05T22:29:50.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.949+0000 7f891ec00640 1 -- start start 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.950+0000 7f891ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f891806c9a0 0x7f891806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.950+0000 7f891ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f891806ba70 0x7f8918164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.950+0000 7f891ec00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891806bea0 con 0x7f891815abe0 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.950+0000 7f891ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891815afc0 con 0x7f891806c9a0 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f891ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891806f5b0 con 0x7f891806ba70 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f891806c9a0 0x7f891806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f891806c9a0 0x7f891806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60198/0 (socket says 172.21.15.73:60198) 2024-04-05T22:29:50.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 -- 172.21.15.73:0/1584442712 learned_addr learned my addr 172.21.15.73:0/1584442712 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:50.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f891cc00640 1 --2- 172.21.15.73:0/1584442712 >> v2:172.21.15.157:3300/0 conn(0x7f891806ba70 0x7f8918164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 -- 172.21.15.73:0/1584442712 >> v2:172.21.15.157:3300/0 conn(0x7f891806ba70 msgr2=0x7f8918164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 --2- 172.21.15.73:0/1584442712 >> v2:172.21.15.157:3300/0 conn(0x7f891806ba70 0x7f8918164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 -- 172.21.15.73:0/1584442712 >> v1:172.21.15.157:6789/0 conn(0x7f891815abe0 legacy=0x7f89180712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.951+0000 7f890fe00640 1 -- 172.21.15.73:0/1584442712 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891806f890 con 0x7f891806c9a0 2024-04-05T22:29:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.952+0000 7f890fe00640 1 --2- 172.21.15.73:0/1584442712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806c9a0 0x7f891806b530 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f891806ce30 tx=0x7f890002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a709d1ca4f33baeb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.953+0000 7f8917400640 1 -- 172.21.15.73:0/1584442712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900031750 con 0x7f891806c9a0 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.953+0000 7f8917400640 1 -- 172.21.15.73:0/1584442712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89000318b0 con 0x7f891806c9a0 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.953+0000 7f8917400640 1 -- 172.21.15.73:0/1584442712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900031c40 con 0x7f891806c9a0 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.953+0000 7f891ec00640 1 -- 172.21.15.73:0/1584442712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806c9a0 msgr2=0x7f891806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.953+0000 7f891ec00640 1 --2- 172.21.15.73:0/1584442712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806c9a0 0x7f891806b530 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f891806ce30 tx=0x7f890002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 -- 172.21.15.73:0/1584442712 shutdown_connections 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 --2- 172.21.15.73:0/1584442712 >> v2:172.21.15.157:3300/0 conn(0x7f891806ba70 0x7f8918164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 --2- 172.21.15.73:0/1584442712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806c9a0 0x7f891806b530 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 -- 172.21.15.73:0/1584442712 >> 172.21.15.73:0/1584442712 conn(0x7f8918069cb0 msgr2=0x7f891806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:50.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 -- 172.21.15.73:0/1584442712 shutdown_connections 2024-04-05T22:29:50.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.954+0000 7f891ec00640 1 -- 172.21.15.73:0/1584442712 wait complete. 2024-04-05T22:29:50.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.955+0000 7f891ec00640 1 Processor -- start 2024-04-05T22:29:50.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.955+0000 7f891ec00640 1 -- start start 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806ba70 0x7f89181f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891806c9a0 0x7f891815d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f891815abe0 0x7f89181589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181f4a60 con 0x7f891806ba70 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891815dec0 con 0x7f891806c9a0 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f891ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89181f0590 con 0x7f891815abe0 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.956+0000 7f8917e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806ba70 0x7f89181f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806ba70 0x7f89181f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60200/0 (socket says 172.21.15.73:60200) 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 -- 172.21.15.73:0/3507949498 learned_addr learned my addr 172.21.15.73:0/3507949498 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 -- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f891815abe0 msgr2=0x7f89181589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f891815abe0 0x7f89181589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 -- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891806c9a0 msgr2=0x7f891815d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891806c9a0 0x7f891815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 -- 172.21.15.73:0/3507949498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000094d0 con 0x7f891806ba70 2024-04-05T22:29:50.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8917e00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891806ba70 0x7f89181f4520 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f890800b3e0 tx=0x7f890800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.957+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908011830 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.958+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89080119b0 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.958+0000 7f891ec00640 1 -- 172.21.15.73:0/3507949498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89181f0870 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.958+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908011d60 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.958+0000 7f891ec00640 1 -- 172.21.15.73:0/3507949498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89181f0e30 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.959+0000 7f891ec00640 1 -- 172.21.15.73:0/3507949498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89180700f0 con 0x7f891806ba70 2024-04-05T22:29:50.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.960+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f890801a400 con 0x7f891806ba70 2024-04-05T22:29:50.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.960+0000 7f8914c00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88f406b1a0 0x7f88f406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:50.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.961+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8908089e90 con 0x7f891806ba70 2024-04-05T22:29:50.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.961+0000 7f890fe00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88f406b1a0 0x7f88f406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:50.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.962+0000 7f890fe00640 1 --2- 172.21.15.73:0/3507949498 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f88f406b1a0 0x7f88f406d660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8900035000 tx=0x7f8900030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:50.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:50.965+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8908054890 con 0x7f891806ba70 2024-04-05T22:29:51.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.195+0000 7f891ec00640 1 -- 172.21.15.73:0/3507949498 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f891806fe50 con 0x7f88f406b1a0 2024-04-05T22:29:51.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.209+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:51.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.209+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:51.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.209+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:51.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.209+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:51.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.209+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:29:51.217 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.218+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:51.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.217+0000 7f8914c00640 1 -- 172.21.15.73:0/3507949498 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f8900031b50 con 0x7f88f406b1a0 2024-04-05T22:29:51.218 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:51.228 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:51.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-04-05T22:29:51.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.433+0000 7fa19ce00640 1 Processor -- start 2024-04-05T22:29:51.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.433+0000 7fa19ce00640 1 -- start start 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa19ce00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa198157410 0x7fa198160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa19ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa1980cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa19ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19806d800 con 0x7fa198157030 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa19ce00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980cbaa0 con 0x7fa198157410 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa19ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b3090 con 0x7fa1980cb0b0 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa197400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa198157030 0x7fa1980ad950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51224/0 (socket says 172.21.15.73:51224) 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.435+0000 7fa197400640 1 -- 172.21.15.73:0/2830024671 learned_addr learned my addr 172.21.15.73:0/2830024671 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196a00640 1 --2- 172.21.15.73:0/2830024671 >> v2:172.21.15.157:3300/0 conn(0x7fa198157410 0x7fa198160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300912137 0 0) 0x7fa19806d800 con 0x7fa198157030 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa17c003520 con 0x7fa198157030 2024-04-05T22:29:51.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa197e00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa1980cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102021415 0 0) 0x7fa17c003520 con 0x7fa198157030 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 msgr2=0x7fa1980cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa1980cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 >> v2:172.21.15.157:3300/0 conn(0x7fa198157410 msgr2=0x7fa198160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 --2- 172.21.15.73:0/2830024671 >> v2:172.21.15.157:3300/0 conn(0x7fa198157410 0x7fa198160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980b3370 con 0x7fa198157030 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa1880019f0 con 0x7fa198157030 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 >> v1:172.21.15.73:6789/0 conn(0x7fa198157030 legacy=0x7fa1980ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa17c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa17c003ef0 0x7fa17c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c004d00 0x7fa17c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980cbaa0 con 0x7fa17c003ef0 2024-04-05T22:29:51.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b3090 con 0x7fa17c004d00 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.436+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c00b9d0 con 0x7fa1980cb0b0 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa17c003ef0 0x7fa17c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa197400640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa17c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 -- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 msgr2=0x7fa17c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa17c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 -- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c004d00 msgr2=0x7fa17c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c004d00 0x7fa17c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa196a00640 1 -- 172.21.15.73:0/2830024671 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c003520 con 0x7fa17c003ef0 2024-04-05T22:29:51.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.437+0000 7fa197400640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1980cb0b0 0x7fa17c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.438+0000 7fa196a00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa17c003ef0 0x7fa17c0047c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa180009a90 tx=0x7fa17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff0d610992c1a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.438+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18002c6b0 con 0x7fa17c003ef0 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.438+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18002c810 con 0x7fa17c003ef0 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.438+0000 7fa196000640 1 -- 172.21.15.73:0/2830024671 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18002cbc0 con 0x7fa17c003ef0 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 -- 172.21.15.73:0/2830024671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa17c003ef0 msgr2=0x7fa17c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 --2- 172.21.15.73:0/2830024671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa17c003ef0 0x7fa17c0047c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa180009a90 tx=0x7fa17c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa197e00640 1 -- 172.21.15.73:0/2830024671 reap_dead start 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 -- 172.21.15.73:0/2830024671 shutdown_connections 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 -- 172.21.15.73:0/2830024671 >> 172.21.15.73:0/2830024671 conn(0x7fa198069cb0 msgr2=0x7fa19806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 -- 172.21.15.73:0/2830024671 shutdown_connections 2024-04-05T22:29:51.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.439+0000 7fa19ce00640 1 -- 172.21.15.73:0/2830024671 wait complete. 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.441+0000 7fa19ce00640 1 Processor -- start 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.441+0000 7fa19ce00640 1 -- start start 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa17c003ef0 0x7fa198150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1981520b0 0x7fa198154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1980ce640 0x7fa1980ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980cf000 con 0x7fa1980ce640 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198156110 con 0x7fa1981520b0 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa19ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198155070 con 0x7fa17c003ef0 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa197e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1980ce640 0x7fa1980ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa197e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1980ce640 0x7fa1980ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60218/0 (socket says 172.21.15.73:60218) 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa197e00640 1 -- 172.21.15.73:0/36009174 learned_addr learned my addr 172.21.15.73:0/36009174 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.442+0000 7fa197400640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa17c003ef0 0x7fa198150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa196a00640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1981520b0 0x7fa198154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 -- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa17c003ef0 msgr2=0x7fa198150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa17c003ef0 0x7fa198150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 -- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1981520b0 msgr2=0x7fa198154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1981520b0 0x7fa198154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 -- 172.21.15.73:0/36009174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198155350 con 0x7fa1980ce640 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa196a00640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1981520b0 0x7fa198154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa197e00640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1980ce640 0x7fa1980ceac0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa19815d1a0 tx=0x7fa19000d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190004030 con 0x7fa1980ce640 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1900041b0 con 0x7fa1980ce640 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa19ce00640 1 -- 172.21.15.73:0/36009174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198155630 con 0x7fa1980ce640 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190003610 con 0x7fa1980ce640 2024-04-05T22:29:51.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.443+0000 7fa19ce00640 1 -- 172.21.15.73:0/36009174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa198155bf0 con 0x7fa1980ce640 2024-04-05T22:29:51.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.445+0000 7fa19ce00640 1 -- 172.21.15.73:0/36009174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1980ad7c0 con 0x7fa1980ce640 2024-04-05T22:29:51.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.445+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7fa190003770 con 0x7fa1980ce640 2024-04-05T22:29:51.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.446+0000 7fa18f400640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa17406b1d0 0x7fa17406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.446+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1900894a0 con 0x7fa1980ce640 2024-04-05T22:29:51.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.447+0000 7fa197400640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa17406b1d0 0x7fa17406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.448+0000 7fa197400640 1 --2- 172.21.15.73:0/36009174 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa17406b1d0 0x7fa17406d690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa188007c40 tx=0x7fa188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.451+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa190053e10 con 0x7fa1980ce640 2024-04-05T22:29:51.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.670+0000 7fa19ce00640 1 -- 172.21.15.73:0/36009174 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1980b3930 con 0x7fa17406b1d0 2024-04-05T22:29:51.674 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:51.674+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.674+0000 7fa18f400640 1 -- 172.21.15.73:0/36009174 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fa18800d050 con 0x7fa17406b1d0 2024-04-05T22:29:51.675 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:51.690 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:51.690 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-04-05T22:29:51.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.906+0000 7f56e3e00640 1 Processor -- start 2024-04-05T22:29:51.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.906+0000 7f56e3e00640 1 -- start start 2024-04-05T22:29:51.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.908+0000 7f56e3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e415cf90 0x7f56e40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.908+0000 7f56e3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e415d370 0x7f56e40b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.908+0000 7f56e3e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40d1bf0 con 0x7f56e40cf1f0 2024-04-05T22:29:51.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.908+0000 7f56e3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40cd6d0 con 0x7f56e415cf90 2024-04-05T22:29:51.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.908+0000 7f56e3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b3a70 con 0x7f56e415d370 2024-04-05T22:29:51.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e415cf90 0x7f56e40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e415d370 0x7f56e40b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e415cf90 0x7f56e40b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:56330/0 (socket says 172.21.15.73:56330) 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 -- 172.21.15.73:0/2152662503 learned_addr learned my addr 172.21.15.73:0/2152662503 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e3400640 1 --1- 172.21.15.73:0/2152662503 >> v1:172.21.15.157:6789/0 conn(0x7f56e40cf1f0 0x7f56e40b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:43608/0 (socket says 172.21.15.73:43608) 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 -- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e415d370 msgr2=0x7f56e40b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 --2- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e415d370 0x7f56e40b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.909+0000 7f56e2a00640 1 -- 172.21.15.73:0/2152662503 >> v1:172.21.15.157:6789/0 conn(0x7f56e40cf1f0 legacy=0x7f56e40b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.910+0000 7f56e2a00640 1 -- 172.21.15.73:0/2152662503 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40d1350 con 0x7f56e415cf90 2024-04-05T22:29:51.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.910+0000 7f56e1600640 1 -- 172.21.15.73:0/2152662503 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200765641 0 0) 0x7f56e40d1bf0 con 0x7f56e40cf1f0 2024-04-05T22:29:51.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.911+0000 7f56e2a00640 1 --2- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415cf90 0x7f56e40b3c10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009e50 tx=0x7f56d402bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0a253fb801f4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.911+0000 7f56e1600640 1 -- 172.21.15.73:0/2152662503 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402cb90 con 0x7f56e415cf90 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.911+0000 7f56e1600640 1 -- 172.21.15.73:0/2152662503 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d402ccf0 con 0x7f56e415cf90 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.911+0000 7f56e1600640 1 -- 172.21.15.73:0/2152662503 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4034610 con 0x7f56e415cf90 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 -- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415cf90 msgr2=0x7f56e40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 --2- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415cf90 0x7f56e40b3c10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009e50 tx=0x7f56d402bff0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 -- 172.21.15.73:0/2152662503 shutdown_connections 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 --2- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e415d370 0x7f56e40b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 --2- 172.21.15.73:0/2152662503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415cf90 0x7f56e40b3c10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 -- 172.21.15.73:0/2152662503 >> 172.21.15.73:0/2152662503 conn(0x7f56e40d2ac0 msgr2=0x7f56e40d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 -- 172.21.15.73:0/2152662503 shutdown_connections 2024-04-05T22:29:51.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.912+0000 7f56e3e00640 1 -- 172.21.15.73:0/2152662503 wait complete. 2024-04-05T22:29:51.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.914+0000 7f56e3e00640 1 Processor -- start 2024-04-05T22:29:51.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.914+0000 7f56e3e00640 1 -- start start 2024-04-05T22:29:51.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e40cf1f0 0x7f56e40b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e415cf90 0x7f56e40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415d370 0x7f56e40b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40c9890 con 0x7f56e415cf90 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415d750 con 0x7f56e415d370 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.915+0000 7f56e3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415da30 con 0x7f56e40cf1f0 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e415cf90 0x7f56e40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e415cf90 0x7f56e40b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60220/0 (socket says 172.21.15.73:60220) 2024-04-05T22:29:51.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 -- 172.21.15.73:0/2307910936 learned_addr learned my addr 172.21.15.73:0/2307910936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:51.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e3400640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415d370 0x7f56e40b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2a00640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e40cf1f0 0x7f56e40b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 -- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e40cf1f0 msgr2=0x7f56e40b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56e40cf1f0 0x7f56e40b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 -- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415d370 msgr2=0x7f56e40b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e415d370 0x7f56e40b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 -- 172.21.15.73:0/2307910936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4009b30 con 0x7f56e415cf90 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.916+0000 7f56e2000640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e415cf90 0x7f56e40b7400 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e870 tx=0x7f56d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.917+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00094f0 con 0x7f56e415cf90 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.917+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d000c940 con 0x7f56e415cf90 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.917+0000 7f56e3e00640 1 -- 172.21.15.73:0/2307910936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e40b6100 con 0x7f56e415cf90 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.917+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000ccd0 con 0x7f56e415cf90 2024-04-05T22:29:51.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.917+0000 7f56e3e00640 1 -- 172.21.15.73:0/2307910936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e40b66c0 con 0x7f56e415cf90 2024-04-05T22:29:51.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.918+0000 7f56e3e00640 1 -- 172.21.15.73:0/2307910936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4067250 con 0x7f56e415cf90 2024-04-05T22:29:51.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.919+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f56d0013470 con 0x7f56e415cf90 2024-04-05T22:29:51.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.920+0000 7f56daa00640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56c406b1d0 0x7f56c406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:51.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.920+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56d008a220 con 0x7f56e415cf90 2024-04-05T22:29:51.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.920+0000 7f56e2a00640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56c406b1d0 0x7f56c406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:51.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.922+0000 7f56e2a00640 1 --2- 172.21.15.73:0/2307910936 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56c406b1d0 0x7f56c406d690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f56d4035000 tx=0x7f56d4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:51.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:51.924+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f56d0054b50 con 0x7f56e415cf90 2024-04-05T22:29:52.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.144+0000 7f56e3e00640 1 -- 172.21.15.73:0/2307910936 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f56e40b42f0 con 0x7f56c406b1d0 2024-04-05T22:29:52.148 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:52.149+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:52.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.149+0000 7f56daa00640 1 -- 172.21.15.73:0/2307910936 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f56d402cb60 con 0x7f56c406b1d0 2024-04-05T22:29:52.149 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:52.159 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:52.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-04-05T22:29:52.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.371+0000 7f5a19400640 1 Processor -- start 2024-04-05T22:29:52.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.371+0000 7f5a19400640 1 -- start start 2024-04-05T22:29:52.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.372+0000 7f5a19400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a141599f0 0x7f5a14070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.372+0000 7f5a19400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a1406cd50 0x7f5a14159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.372+0000 7f5a19400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140ae190 con 0x7f5a140add00 2024-04-05T22:29:52.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.372+0000 7f5a19400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140b3890 con 0x7f5a141599f0 2024-04-05T22:29:52.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.372+0000 7f5a19400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140b27f0 con 0x7f5a1406cd50 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a141599f0 0x7f5a14070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a141599f0 0x7f5a14070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60236/0 (socket says 172.21.15.73:60236) 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 -- 172.21.15.73:0/3283662979 learned_addr learned my addr 172.21.15.73:0/3283662979 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 -- 172.21.15.73:0/3283662979 >> v2:172.21.15.157:3300/0 conn(0x7f5a1406cd50 msgr2=0x7f5a14159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 --2- 172.21.15.73:0/3283662979 >> v2:172.21.15.157:3300/0 conn(0x7f5a1406cd50 0x7f5a14159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 -- 172.21.15.73:0/3283662979 >> v1:172.21.15.157:6789/0 conn(0x7f5a140add00 legacy=0x7f5a1406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.373+0000 7f5a12800640 1 -- 172.21.15.73:0/3283662979 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140b2ad0 con 0x7f5a141599f0 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.374+0000 7f5a12800640 1 --2- 172.21.15.73:0/3283662979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a14070a40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5a140adbe0 tx=0x7f5a0402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0473cede8d1a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a11400640 1 -- 172.21.15.73:0/3283662979 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402f8c0 con 0x7f5a141599f0 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a11400640 1 -- 172.21.15.73:0/3283662979 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0402fa20 con 0x7f5a141599f0 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a11400640 1 -- 172.21.15.73:0/3283662979 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402fdb0 con 0x7f5a141599f0 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 -- 172.21.15.73:0/3283662979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 msgr2=0x7f5a14070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 --2- 172.21.15.73:0/3283662979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a14070a40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5a140adbe0 tx=0x7f5a0402db00 comp rx=0 tx=0).stop 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 -- 172.21.15.73:0/3283662979 shutdown_connections 2024-04-05T22:29:52.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 --2- 172.21.15.73:0/3283662979 >> v2:172.21.15.157:3300/0 conn(0x7f5a1406cd50 0x7f5a14159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 --2- 172.21.15.73:0/3283662979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a14070a40 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.375+0000 7f5a19400640 1 -- 172.21.15.73:0/3283662979 >> 172.21.15.73:0/3283662979 conn(0x7f5a14069cb0 msgr2=0x7f5a1406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:52.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.376+0000 7f5a19400640 1 -- 172.21.15.73:0/3283662979 shutdown_connections 2024-04-05T22:29:52.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.376+0000 7f5a19400640 1 -- 172.21.15.73:0/3283662979 wait complete. 2024-04-05T22:29:52.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.377+0000 7f5a19400640 1 Processor -- start 2024-04-05T22:29:52.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.377+0000 7f5a19400640 1 -- start start 2024-04-05T22:29:52.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a141e5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a140add00 0x7f5a14158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a1415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140ad0b0 con 0x7f5a141599f0 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141e48b0 con 0x7f5a140add00 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.378+0000 7f5a19400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141e4b90 con 0x7f5a1406cd50 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a1415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a1415c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60252/0 (socket says 172.21.15.73:60252) 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 -- 172.21.15.73:0/1489420993 learned_addr learned my addr 172.21.15.73:0/1489420993 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a11e00640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a140add00 0x7f5a14158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a12800640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a141e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 -- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 msgr2=0x7f5a141e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a1406cd50 0x7f5a141e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 -- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a140add00 msgr2=0x7f5a14158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a140add00 0x7f5a14158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.379+0000 7f5a13200640 1 -- 172.21.15.73:0/1489420993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040094d0 con 0x7f5a141599f0 2024-04-05T22:29:52.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a13200640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a141599f0 0x7f5a1415c930 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00b9e0 tx=0x7f5a0c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00cb20 con 0x7f5a141599f0 2024-04-05T22:29:52.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c00cc80 con 0x7f5a141599f0 2024-04-05T22:29:52.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a19400640 1 -- 172.21.15.73:0/1489420993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a141e4e70 con 0x7f5a141599f0 2024-04-05T22:29:52.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c014440 con 0x7f5a141599f0 2024-04-05T22:29:52.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.380+0000 7f5a19400640 1 -- 172.21.15.73:0/1489420993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141e5430 con 0x7f5a141599f0 2024-04-05T22:29:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.381+0000 7f5a19400640 1 -- 172.21.15.73:0/1489420993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a141e5620 con 0x7f5a141599f0 2024-04-05T22:29:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.382+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f5a0c01d3b0 con 0x7f5a141599f0 2024-04-05T22:29:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.383+0000 7f5a0aa00640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59f406b1f0 0x7f59f406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.383+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a0c089b90 con 0x7f5a141599f0 2024-04-05T22:29:52.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.384+0000 7f5a12800640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59f406b1f0 0x7f59f406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.385+0000 7f5a12800640 1 --2- 172.21.15.73:0/1489420993 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f59f406b1f0 0x7f59f406d6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a04038000 tx=0x7f5a04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.387+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5a0c054690 con 0x7f5a141599f0 2024-04-05T22:29:52.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.607+0000 7f5a19400640 1 -- 172.21.15.73:0/1489420993 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a140b3370 con 0x7f59f406b1f0 2024-04-05T22:29:52.611 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:52.612+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:52.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.612+0000 7f5a0aa00640 1 -- 172.21.15.73:0/1489420993 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f5a0402fb90 con 0x7f59f406b1f0 2024-04-05T22:29:52.612 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:52.626 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:52.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-04-05T22:29:52.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.841+0000 7ff9ac600640 1 Processor -- start 2024-04-05T22:29:52.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.841+0000 7ff9ac600640 1 -- start start 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9ac600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff9a40712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9ac600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9a406bc40 0x7ff9a4164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9ac600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a415afc0 con 0x7ff9a406c9a0 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9ac600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a406f5b0 con 0x7ff9a406bc40 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9ac600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406f890 con 0x7ff9a415abe0 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9a9200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff9a406c9a0 0x7ff9a406b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51238/0 (socket says 172.21.15.73:51238) 2024-04-05T22:29:52.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9a9200640 1 -- 172.21.15.73:0/1414457947 learned_addr learned my addr 172.21.15.73:0/1414457947 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff9a40712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892140478 0 0) 0x7ff9a415afc0 con 0x7ff9a406c9a0 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9aa600640 1 --2- 172.21.15.73:0/1414457947 >> v2:172.21.15.157:3300/0 conn(0x7ff9a406bc40 0x7ff9a4164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.843+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c003520 con 0x7ff9a406c9a0 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 981666587 0 0) 0x7ff98c003520 con 0x7ff9a406c9a0 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 msgr2=0x7ff9a40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff9a40712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 >> v2:172.21.15.157:3300/0 conn(0x7ff9a406bc40 msgr2=0x7ff9a4164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 --2- 172.21.15.73:0/1414457947 >> v2:172.21.15.157:3300/0 conn(0x7ff9a406bc40 0x7ff9a4164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a406fb70 con 0x7ff9a406c9a0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff990001670 con 0x7ff9a406c9a0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 >> v1:172.21.15.73:6789/0 conn(0x7ff9a406c9a0 legacy=0x7ff9a406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff98c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406f5b0 con 0x7ff98c0047c0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406f890 con 0x7ff98c003970 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00bab0 con 0x7ff9a415abe0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.844+0000 7ff9a9200640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9aa600640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff98c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9200640 1 -- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 msgr2=0x7ff98c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9200640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a415abe0 0x7ff98c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9200640 1 -- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c003970 msgr2=0x7ff98c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9200640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9200640 1 -- 172.21.15.73:0/1414457947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c003520 con 0x7ff98c0047c0 2024-04-05T22:29:52.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.845+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c003970 0x7ff98c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.846+0000 7ff9a9200640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff990007fd0 tx=0x7ff98c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50da873b325fa196 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.846+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99002cb00 con 0x7ff98c0047c0 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.846+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99002cc80 con 0x7ff98c0047c0 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.846+0000 7ff9a3e00640 1 -- 172.21.15.73:0/1414457947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900058d0 con 0x7ff98c0047c0 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 -- 172.21.15.73:0/1414457947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff98c0047c0 msgr2=0x7ff98c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 --2- 172.21.15.73:0/1414457947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff98c0047c0 0x7ff98c004c40 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff990007fd0 tx=0x7ff98c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9aa600640 1 -- 172.21.15.73:0/1414457947 reap_dead start 2024-04-05T22:29:52.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 -- 172.21.15.73:0/1414457947 shutdown_connections 2024-04-05T22:29:52.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 -- 172.21.15.73:0/1414457947 >> 172.21.15.73:0/1414457947 conn(0x7ff9a4069cb0 msgr2=0x7ff9a406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:52.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 -- 172.21.15.73:0/1414457947 shutdown_connections 2024-04-05T22:29:52.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.847+0000 7ff9ac600640 1 -- 172.21.15.73:0/1414457947 wait complete. 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.849+0000 7ff9ac600640 1 Processor -- start 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.849+0000 7ff9ac600640 1 -- start start 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0047c0 0x7ff9a415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9a413f3e0 0x7ff9a415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a414ce70 0x7ff9a414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a414d830 con 0x7ff9a413f3e0 2024-04-05T22:29:52.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415cb90 con 0x7ff9a414ce70 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9ac600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415c0c0 con 0x7ff98c0047c0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9a413f3e0 0x7ff9a415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9a413f3e0 0x7ff9a415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60276/0 (socket says 172.21.15.73:60276) 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 -- 172.21.15.73:0/2567930289 learned_addr learned my addr 172.21.15.73:0/2567930289 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0047c0 0x7ff9a415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9aa600640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a414ce70 0x7ff9a414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 -- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0047c0 msgr2=0x7ff9a415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0047c0 0x7ff9a415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 -- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a414ce70 msgr2=0x7ff9a414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a414ce70 0x7ff9a414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9200640 1 -- 172.21.15.73:0/2567930289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a414f260 con 0x7ff9a413f3e0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0047c0 0x7ff9a415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.850+0000 7ff9aa600640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a414ce70 0x7ff9a414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9a9200640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9a413f3e0 0x7ff9a415bb80 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff990000f60 tx=0x7ff990000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99003c040 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff990001630 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990007270 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9ac600640 1 -- 172.21.15.73:0/2567930289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a414f500 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.851+0000 7ff9ac600640 1 -- 172.21.15.73:0/2567930289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a415c2f0 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.852+0000 7ff9ac600640 1 -- 172.21.15.73:0/2567930289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4068160 con 0x7ff9a413f3e0 2024-04-05T22:29:52.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.853+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7ff990032070 con 0x7ff9a413f3e0 2024-04-05T22:29:52.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.854+0000 7ff9a1600640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff98406b130 0x7ff98406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:52.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.854+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9900ab220 con 0x7ff9a413f3e0 2024-04-05T22:29:52.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.855+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff98406b130 0x7ff98406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:52.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.856+0000 7ff9a9c00640 1 --2- 172.21.15.73:0/2567930289 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff98406b130 0x7ff98406d5f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff998007440 tx=0x7ff99800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:52.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:52.858+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff990075c20 con 0x7ff9a413f3e0 2024-04-05T22:29:53.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.080+0000 7ff9ac600640 1 -- 172.21.15.73:0/2567930289 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a40cdcc0 con 0x7ff98406b130 2024-04-05T22:29:53.083 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:53.084+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:53.084 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:53.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.084+0000 7ff9a1600640 1 -- 172.21.15.73:0/2567930289 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7ff998011040 con 0x7ff98406b130 2024-04-05T22:29:53.103 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:53.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-04-05T22:29:53.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.317+0000 7f95b8000640 1 Processor -- start 2024-04-05T22:29:53.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.317+0000 7f95b8000640 1 -- start start 2024-04-05T22:29:53.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.318+0000 7f95b8000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95b0154580 0x7f95b0164c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.318+0000 7f95b8000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b015d900 con 0x7f95b00cf3e0 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.318+0000 7f95b8000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b015e1a0 con 0x7f95b01541a0 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.318+0000 7f95b8000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00b3490 con 0x7f95b0154580 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b6000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95b0154580 0x7f95b0164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b4c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f95b00cf3e0 0x7f95b00b2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51250/0 (socket says 172.21.15.73:51250) 2024-04-05T22:29:53.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b4c00640 1 -- 172.21.15.73:0/2459255718 learned_addr learned my addr 172.21.15.73:0/2459255718 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:53.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b6000640 1 -- 172.21.15.73:0/2459255718 >> v1:172.21.15.157:6789/0 conn(0x7f95b01541a0 legacy=0x7f95b00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:53.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b6000640 1 -- 172.21.15.73:0/2459255718 >> v1:172.21.15.73:6789/0 conn(0x7f95b00cf3e0 legacy=0x7f95b00b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.319+0000 7f95b6000640 1 -- 172.21.15.73:0/2459255718 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b015db50 con 0x7f95b0154580 2024-04-05T22:29:53.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.320+0000 7f95b6000640 1 --2- 172.21.15.73:0/2459255718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154580 0x7f95b0164c90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009d90 tx=0x7f95a802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba4700282dc0301 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.320+0000 7f95afe00640 1 -- 172.21.15.73:0/2459255718 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a802ee70 con 0x7f95b0154580 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.320+0000 7f95afe00640 1 -- 172.21.15.73:0/2459255718 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8036440 con 0x7f95b0154580 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.320+0000 7f95afe00640 1 -- 172.21.15.73:0/2459255718 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80367d0 con 0x7f95b0154580 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 -- 172.21.15.73:0/2459255718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154580 msgr2=0x7f95b0164c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 --2- 172.21.15.73:0/2459255718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154580 0x7f95b0164c90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009d90 tx=0x7f95a802e0c0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 -- 172.21.15.73:0/2459255718 shutdown_connections 2024-04-05T22:29:53.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 --2- 172.21.15.73:0/2459255718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154580 0x7f95b0164c90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 -- 172.21.15.73:0/2459255718 >> 172.21.15.73:0/2459255718 conn(0x7f95b006a0d0 msgr2=0x7f95b006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:53.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 -- 172.21.15.73:0/2459255718 shutdown_connections 2024-04-05T22:29:53.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.321+0000 7f95b8000640 1 -- 172.21.15.73:0/2459255718 wait complete. 2024-04-05T22:29:53.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.323+0000 7f95b8000640 1 Processor -- start 2024-04-05T22:29:53.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.323+0000 7f95b8000640 1 -- start start 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00cf3e0 0x7f95b014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b01541a0 0x7f95b00afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0154580 0x7f95b00cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b02b0 con 0x7f95b01541a0 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00aecd0 con 0x7f95b0154580 2024-04-05T22:29:53.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b8000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00aefb0 con 0x7f95b00cf3e0 2024-04-05T22:29:53.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b01541a0 0x7f95b00afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b01541a0 0x7f95b00afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60300/0 (socket says 172.21.15.73:60300) 2024-04-05T22:29:53.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.324+0000 7f95b4c00640 1 -- 172.21.15.73:0/2345275287 learned_addr learned my addr 172.21.15.73:0/2345275287 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b6000640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0154580 0x7f95b00cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b5600640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00cf3e0 0x7f95b014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 -- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00cf3e0 msgr2=0x7f95b014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00cf3e0 0x7f95b014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 -- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0154580 msgr2=0x7f95b00cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b0154580 0x7f95b00cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 -- 172.21.15.73:0/2345275287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009a70 con 0x7f95b01541a0 2024-04-05T22:29:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b5600640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b00cf3e0 0x7f95b014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b4c00640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b01541a0 0x7f95b00afd70 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c4a0 tx=0x7f959c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c005ce0 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c005e40 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c007270 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b8000640 1 -- 172.21.15.73:0/2345275287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00af290 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.325+0000 7f95b8000640 1 -- 172.21.15.73:0/2345275287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00af850 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.327+0000 7f95b8000640 1 -- 172.21.15.73:0/2345275287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b006fcd0 con 0x7f95b01541a0 2024-04-05T22:29:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.328+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 89655+0+0 (secure 0 0 0) 0x7f959c0073d0 con 0x7f95b01541a0 2024-04-05T22:29:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.328+0000 7f95ad600640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f959006b1d0 0x7f959006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.328+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f959c089130 con 0x7f95b01541a0 2024-04-05T22:29:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.329+0000 7f95b5600640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f959006b1d0 0x7f959006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.331+0000 7f95b5600640 1 --2- 172.21.15.73:0/2345275287 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f959006b1d0 0x7f959006d690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95a4003eb0 tx=0x7f95a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.333+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f959c053be0 con 0x7f95b01541a0 2024-04-05T22:29:53.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.551+0000 7f95b8000640 1 -- 172.21.15.73:0/2345275287 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b00b2f50 con 0x7f959006b1d0 2024-04-05T22:29:53.554 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:53.555+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.555+0000 7f95ad600640 1 -- 172.21.15.73:0/2345275287 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f95a4005030 con 0x7f959006b1d0 2024-04-05T22:29:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:53.575 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:53.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-04-05T22:29:53.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.785+0000 7fe722a00640 1 Processor -- start 2024-04-05T22:29:53.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.785+0000 7fe722a00640 1 -- start start 2024-04-05T22:29:53.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe722a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe71c1572b0 0x7fe71c070d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe722a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe71c157690 0x7fe71c0712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe722a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c06c970 con 0x7fe71c0b1b80 2024-04-05T22:29:53.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe722a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c15e440 con 0x7fe71c1572b0 2024-04-05T22:29:53.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe722a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0ac970 con 0x7fe71c157690 2024-04-05T22:29:53.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe720a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe71c0b1b80 0x7fe71c1609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51264/0 (socket says 172.21.15.73:51264) 2024-04-05T22:29:53.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe720a00640 1 -- 172.21.15.73:0/3057176894 learned_addr learned my addr 172.21.15.73:0/3057176894 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:53.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe71be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe71c1572b0 0x7fe71c070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.787+0000 7fe71b400640 1 --2- 172.21.15.73:0/3057176894 >> v2:172.21.15.157:3300/0 conn(0x7fe71c157690 0x7fe71c0712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.788+0000 7fe71be00640 1 -- 172.21.15.73:0/3057176894 >> v2:172.21.15.157:3300/0 conn(0x7fe71c157690 msgr2=0x7fe71c0712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:53.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.788+0000 7fe71be00640 1 --2- 172.21.15.73:0/3057176894 >> v2:172.21.15.157:3300/0 conn(0x7fe71c157690 0x7fe71c0712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.788+0000 7fe71be00640 1 -- 172.21.15.73:0/3057176894 >> v1:172.21.15.73:6789/0 conn(0x7fe71c0b1b80 legacy=0x7fe71c1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.788+0000 7fe71be00640 1 -- 172.21.15.73:0/3057176894 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0acbe0 con 0x7fe71c1572b0 2024-04-05T22:29:53.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.788+0000 7fe71aa00640 1 -- 172.21.15.73:0/3057176894 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252954404 0 0) 0x7fe71c06c970 con 0x7fe71c0b1b80 2024-04-05T22:29:53.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.789+0000 7fe71be00640 1 --2- 172.21.15.73:0/3057176894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1572b0 0x7fe71c070d90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097a0 tx=0x7fe70c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6350a030fb981 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.790+0000 7fe71aa00640 1 -- 172.21.15.73:0/3057176894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c031830 con 0x7fe71c1572b0 2024-04-05T22:29:53.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.790+0000 7fe71aa00640 1 -- 172.21.15.73:0/3057176894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c031990 con 0x7fe71c1572b0 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.790+0000 7fe71aa00640 1 -- 172.21.15.73:0/3057176894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c031d20 con 0x7fe71c1572b0 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.790+0000 7fe722a00640 1 -- 172.21.15.73:0/3057176894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1572b0 msgr2=0x7fe71c070d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.790+0000 7fe722a00640 1 --2- 172.21.15.73:0/3057176894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1572b0 0x7fe71c070d90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097a0 tx=0x7fe70c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 -- 172.21.15.73:0/3057176894 shutdown_connections 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 --2- 172.21.15.73:0/3057176894 >> v2:172.21.15.157:3300/0 conn(0x7fe71c157690 0x7fe71c0712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 --2- 172.21.15.73:0/3057176894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1572b0 0x7fe71c070d90 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 -- 172.21.15.73:0/3057176894 >> 172.21.15.73:0/3057176894 conn(0x7fe71c069e90 msgr2=0x7fe71c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:53.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 -- 172.21.15.73:0/3057176894 shutdown_connections 2024-04-05T22:29:53.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.791+0000 7fe722a00640 1 -- 172.21.15.73:0/3057176894 wait complete. 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.793+0000 7fe722a00640 1 Processor -- start 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.793+0000 7fe722a00640 1 -- start start 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b1b80 0x7fe71c1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c1572b0 0x7fe71c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c157690 0x7fe71c14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c15c9a0 con 0x7fe71c157690 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c158570 con 0x7fe71c1572b0 2024-04-05T22:29:53.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe722a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c150a80 con 0x7fe71c0b1b80 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe720a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c157690 0x7fe71c14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe720a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c157690 0x7fe71c14f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60312/0 (socket says 172.21.15.73:60312) 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.794+0000 7fe720a00640 1 -- 172.21.15.73:0/1998914466 learned_addr learned my addr 172.21.15.73:0/1998914466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe71b400640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c1572b0 0x7fe71c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe71be00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b1b80 0x7fe71c1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 -- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b1b80 msgr2=0x7fe71c1501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe71c0b1b80 0x7fe71c1501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 -- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c1572b0 msgr2=0x7fe71c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c1572b0 0x7fe71c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 -- 172.21.15.73:0/1998914466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009400 con 0x7fe71c157690 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe720a00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c157690 0x7fe71c14f9b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c7e0 tx=0x7fe71400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400a640 con 0x7fe71c157690 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71400a7c0 con 0x7fe71c157690 2024-04-05T22:29:53.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe722a00640 1 -- 172.21.15.73:0/1998914466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c0ad840 con 0x7fe71c157690 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400ab70 con 0x7fe71c157690 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.795+0000 7fe722a00640 1 -- 172.21.15.73:0/1998914466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c157f20 con 0x7fe71c157690 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.797+0000 7fe722a00640 1 -- 172.21.15.73:0/1998914466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c067690 con 0x7fe71c157690 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.797+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fe71400acd0 con 0x7fe71c157690 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.798+0000 7fe713e00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6fc06b230 0x7fe6fc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:53.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.798+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe714089630 con 0x7fe71c157690 2024-04-05T22:29:53.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.799+0000 7fe71be00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6fc06b230 0x7fe6fc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:53.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.800+0000 7fe71be00640 1 --2- 172.21.15.73:0/1998914466 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe6fc06b230 0x7fe6fc06d6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe70c036000 tx=0x7fe70c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:53.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:53.803+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe714053fd0 con 0x7fe71c157690 2024-04-05T22:29:54.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.021+0000 7fe722a00640 1 -- 172.21.15.73:0/1998914466 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7fe71c0ad1a0 con 0x7fe6fc06b230 2024-04-05T22:29:54.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:54.026+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.026+0000 7fe713e00640 1 -- 172.21.15.73:0/1998914466 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fe70c031b00 con 0x7fe6fc06b230 2024-04-05T22:29:54.026 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.042 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:54.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-04-05T22:29:54.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.260+0000 7f1bfe200640 1 Processor -- start 2024-04-05T22:29:54.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.261+0000 7f1bfe200640 1 -- start start 2024-04-05T22:29:54.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bfe200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bf80b3ab0 0x7f1bf8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bfe200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bf8164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bfe200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf81620e0 con 0x7f1bf80cecd0 2024-04-05T22:29:54.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bfe200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf80ae370 con 0x7f1bf80b3ab0 2024-04-05T22:29:54.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bfe200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf80ae090 con 0x7f1bf80ad720 2024-04-05T22:29:54.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bf7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1bf80cecd0 0x7f1bf80cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51276/0 (socket says 172.21.15.73:51276) 2024-04-05T22:29:54.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.262+0000 7f1bf7400640 1 -- 172.21.15.73:0/2509350786 learned_addr learned my addr 172.21.15.73:0/2509350786 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bf8164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6a00640 1 --2- 172.21.15.73:0/2509350786 >> v2:172.21.15.157:3300/0 conn(0x7f1bf80b3ab0 0x7f1bf8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352904122 0 0) 0x7f1bf81620e0 con 0x7f1bf80cecd0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bdc003520 con 0x7f1bf80cecd0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1755881424 0 0) 0x7f1bdc003520 con 0x7f1bf80cecd0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 msgr2=0x7f1bf8164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bf8164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 >> v2:172.21.15.157:3300/0 conn(0x7f1bf80b3ab0 msgr2=0x7f1bf8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 --2- 172.21.15.73:0/2509350786 >> v2:172.21.15.157:3300/0 conn(0x7f1bf80b3ab0 0x7f1bf8162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8161a90 con 0x7f1bf80cecd0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f1be0001670 con 0x7f1bf80cecd0 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 >> v1:172.21.15.73:6789/0 conn(0x7f1bf80cecd0 legacy=0x7f1bf80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bdc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc003ef0 0x7f1bdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bdc004d00 0x7f1bdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf80ae370 con 0x7f1bdc004d00 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf80ae090 con 0x7f1bdc003ef0 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.263+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc00b980 con 0x7f1bf80ad720 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bdc004d00 0x7f1bdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7400640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bdc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf6a00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc003ef0 0x7f1bdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 -- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 msgr2=0x7f1bdc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf80ad720 0x7f1bdc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 -- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc003ef0 msgr2=0x7f1bdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc003ef0 0x7f1bdc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.264+0000 7f1bf7e00640 1 -- 172.21.15.73:0/2509350786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc003520 con 0x7f1bdc004d00 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.265+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bdc004d00 0x7f1bdc005180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ac0 tx=0x7f1bdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce491a9b09c265f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.265+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf002c6b0 con 0x7f1bdc004d00 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf002c810 con 0x7f1bdc004d00 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bf6000640 1 -- 172.21.15.73:0/2509350786 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf002cba0 con 0x7f1bdc004d00 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 -- 172.21.15.73:0/2509350786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bdc004d00 msgr2=0x7f1bdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 --2- 172.21.15.73:0/2509350786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bdc004d00 0x7f1bdc005180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ac0 tx=0x7f1bdc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bf7e00640 1 -- 172.21.15.73:0/2509350786 reap_dead start 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 -- 172.21.15.73:0/2509350786 shutdown_connections 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 -- 172.21.15.73:0/2509350786 >> 172.21.15.73:0/2509350786 conn(0x7f1bf806a0d0 msgr2=0x7f1bf806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 -- 172.21.15.73:0/2509350786 shutdown_connections 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.266+0000 7f1bfe200640 1 -- 172.21.15.73:0/2509350786 wait complete. 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.268+0000 7f1bfe200640 1 Processor -- start 2024-04-05T22:29:54.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.268+0000 7f1bfe200640 1 -- start start 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc004d00 0x7f1bf81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf8164400 0x7f1bf81589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bf8156e30 0x7f1bf8162570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf8153740 con 0x7f1bf8156e30 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf815cd90 con 0x7f1bdc004d00 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bfe200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf815c180 con 0x7f1bf8164400 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bf8156e30 0x7f1bf8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bf8156e30 0x7f1bf8162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60330/0 (socket says 172.21.15.73:60330) 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.269+0000 7f1bf7e00640 1 -- 172.21.15.73:0/1384438109 learned_addr learned my addr 172.21.15.73:0/1384438109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7400640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc004d00 0x7f1bf81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf6a00640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf8164400 0x7f1bf81589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 -- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf8164400 msgr2=0x7f1bf81589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bf8164400 0x7f1bf81589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 -- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc004d00 msgr2=0x7f1bf81584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bdc004d00 0x7f1bf81584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 -- 172.21.15.73:0/1384438109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf815c460 con 0x7f1bf8156e30 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bf7e00640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bf8156e30 0x7f1bf8162570 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ac0 tx=0x7f1bf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf002c6b0 con 0x7f1bf8156e30 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.270+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0003830 con 0x7f1bf8156e30 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.271+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0003be0 con 0x7f1bf8156e30 2024-04-05T22:29:54.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.271+0000 7f1bfe200640 1 -- 172.21.15.73:0/1384438109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf815c740 con 0x7f1bf8156e30 2024-04-05T22:29:54.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.271+0000 7f1bfe200640 1 -- 172.21.15.73:0/1384438109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf815be70 con 0x7f1bf8156e30 2024-04-05T22:29:54.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.272+0000 7f1bfe200640 1 -- 172.21.15.73:0/1384438109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf8068160 con 0x7f1bf8156e30 2024-04-05T22:29:54.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.272+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f1bf0003d40 con 0x7f1bf8156e30 2024-04-05T22:29:54.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.273+0000 7f1bef400640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1bd406b260 0x7f1bd406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.273+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bf00aba20 con 0x7f1bf8156e30 2024-04-05T22:29:54.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.274+0000 7f1bf7400640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1bd406b260 0x7f1bd406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.275+0000 7f1bf7400640 1 --2- 172.21.15.73:0/1384438109 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1bd406b260 0x7f1bd406d720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80ad6f0 tx=0x7f1be0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.278+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1bf00763a0 con 0x7f1bf8156e30 2024-04-05T22:29:54.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.496+0000 7f1bfe200640 1 -- 172.21.15.73:0/1384438109 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bf815bcf0 con 0x7f1bd406b260 2024-04-05T22:29:54.500 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:54.500+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.500+0000 7f1bef400640 1 -- 172.21.15.73:0/1384438109 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f1be00108d0 con 0x7f1bd406b260 2024-04-05T22:29:54.500 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.514 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:54.515 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-04-05T22:29:54.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.724+0000 7f0d07e00640 1 Processor -- start 2024-04-05T22:29:54.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.725+0000 7f0d07e00640 1 -- start start 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.726+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.726+0000 7f0d07e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d0806cc00 0x7f0d08160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.726+0000 7f0d07e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d080ad1f0 con 0x7f0d0806d060 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.726+0000 7f0d07e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0806c530 con 0x7f0d0806cc00 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.726+0000 7f0d07e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0815d420 con 0x7f0d080ac8d0 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0cffe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d0806cc00 0x7f0d08160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0d07400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d0806d060 0x7f0d08160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51280/0 (socket says 172.21.15.73:51280) 2024-04-05T22:29:54.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0d07400640 1 -- 172.21.15.73:0/898520670 learned_addr learned my addr 172.21.15.73:0/898520670 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:54.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0d06a00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0cffe00640 1 -- 172.21.15.73:0/898520670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 msgr2=0x7f0d080accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0cffe00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0cffe00640 1 -- 172.21.15.73:0/898520670 >> v1:172.21.15.73:6789/0 conn(0x7f0d0806d060 legacy=0x7f0d08160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0cffe00640 1 -- 172.21.15.73:0/898520670 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0815d700 con 0x7f0d0806cc00 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0d06000640 1 -- 172.21.15.73:0/898520670 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961598150 0 0) 0x7f0d080ad1f0 con 0x7f0d0806d060 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.727+0000 7f0d06a00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.728+0000 7f0cffe00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08160880 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9299485bd0586083 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d06000640 1 -- 172.21.15.73:0/898520670 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf802d040 con 0x7f0d0806cc00 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d06000640 1 -- 172.21.15.73:0/898520670 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8031810 con 0x7f0d0806cc00 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d06000640 1 -- 172.21.15.73:0/898520670 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8031ba0 con 0x7f0d0806cc00 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 -- 172.21.15.73:0/898520670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 msgr2=0x7f0d08160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08160880 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b8b0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 -- 172.21.15.73:0/898520670 shutdown_connections 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08160880 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 --2- 172.21.15.73:0/898520670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 -- 172.21.15.73:0/898520670 >> 172.21.15.73:0/898520670 conn(0x7f0d080b0360 msgr2=0x7f0d080b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 -- 172.21.15.73:0/898520670 shutdown_connections 2024-04-05T22:29:54.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.729+0000 7f0d07e00640 1 -- 172.21.15.73:0/898520670 wait complete. 2024-04-05T22:29:54.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.731+0000 7f0d07e00640 1 Processor -- start 2024-04-05T22:29:54.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.731+0000 7f0d07e00640 1 -- start start 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0806d060 0x7f0d080ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08154320 con 0x7f0d0806cc00 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081514e0 con 0x7f0d0806d060 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d07e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081536a0 con 0x7f0d080ac8d0 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60350/0 (socket says 172.21.15.73:60350) 2024-04-05T22:29:54.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.732+0000 7f0d06a00640 1 -- 172.21.15.73:0/3113411419 learned_addr learned my addr 172.21.15.73:0/3113411419 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d07400640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0cffe00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0806d060 0x7f0d080ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 -- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 msgr2=0x7f0d080aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080ac8d0 0x7f0d080aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 -- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0806d060 msgr2=0x7f0d080ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0806d060 0x7f0d080ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 -- 172.21.15.73:0/3113411419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009400 con 0x7f0d0806cc00 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0d06a00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d0806cc00 0x7f0d08140ba0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c7f0 tx=0x7f0cf000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.733+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0007820 con 0x7f0d0806cc00 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.734+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0007980 con 0x7f0d0806cc00 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.734+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0007cf0 con 0x7f0d0806cc00 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.734+0000 7f0d07e00640 1 -- 172.21.15.73:0/3113411419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d08153980 con 0x7f0d0806cc00 2024-04-05T22:29:54.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.734+0000 7f0d07e00640 1 -- 172.21.15.73:0/3113411419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d08153f40 con 0x7f0d0806cc00 2024-04-05T22:29:54.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.735+0000 7f0d07e00640 1 -- 172.21.15.73:0/3113411419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0815d030 con 0x7f0d0806cc00 2024-04-05T22:29:54.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.736+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f0cf00113f0 con 0x7f0d0806cc00 2024-04-05T22:29:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.737+0000 7f0cfea00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0ce406b230 0x7f0ce406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.737+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cf008a260 con 0x7f0d0806cc00 2024-04-05T22:29:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.737+0000 7f0cffe00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0ce406b230 0x7f0ce406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.738+0000 7f0cffe00640 1 --2- 172.21.15.73:0/3113411419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0ce406b230 0x7f0ce406d6f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8036000 tx=0x7f0cf803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.742+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0cf0054a80 con 0x7f0d0806cc00 2024-04-05T22:29:54.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.965+0000 7f0d07e00640 1 -- 172.21.15.73:0/3113411419 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d080672b0 con 0x7f0ce406b230 2024-04-05T22:29:54.969 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:54.969+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:54.969+0000 7f0cfea00640 1 -- 172.21.15.73:0/3113411419 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f0d080672b0 con 0x7f0ce406b230 2024-04-05T22:29:54.969 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:54.982 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:54.983 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-04-05T22:29:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.196+0000 7f8fdfe00640 1 Processor -- start 2024-04-05T22:29:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.196+0000 7f8fdfe00640 1 -- start start 2024-04-05T22:29:55.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 0x7f8fe00ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdfe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fe00d1410 0x7f8fe0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdfe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe00b68c0 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdfe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe00ced00 con 0x7f8fe00d1410 2024-04-05T22:29:55.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00cefe0 con 0x7f8fe015cd00 2024-04-05T22:29:55.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fde000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8fe00d0fd0 0x7f8fe0160780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51294/0 (socket says 172.21.15.73:51294) 2024-04-05T22:29:55.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fde000640 1 -- 172.21.15.73:0/1655734460 learned_addr learned my addr 172.21.15.73:0/1655734460 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:55.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.198+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 0x7f8fe00ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdf400640 1 --2- 172.21.15.73:0/1655734460 >> v2:172.21.15.157:3300/0 conn(0x7f8fe00d1410 0x7f8fe0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364757026 0 0) 0x7f8fe00b68c0 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc4003520 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2094409597 0 0) 0x7f8fc4003520 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 msgr2=0x7f8fe00ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 0x7f8fe00ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 >> v2:172.21.15.157:3300/0 conn(0x7f8fe00d1410 msgr2=0x7f8fe0160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 --2- 172.21.15.73:0/1655734460 >> v2:172.21.15.157:3300/0 conn(0x7f8fe00d1410 0x7f8fe0160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe00cf2c0 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f8fd00019f0 con 0x7f8fe00d0fd0 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 >> v1:172.21.15.73:6789/0 conn(0x7f8fe00d0fd0 legacy=0x7f8fe0160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc4003970 0x7f8fc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 0x7f8fc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00ced00 con 0x7f8fc40047c0 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00cefe0 con 0x7f8fc4003970 2024-04-05T22:29:55.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.199+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc400b980 con 0x7f8fe015cd00 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc4003970 0x7f8fc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 -- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 msgr2=0x7f8fc4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe015cd00 0x7f8fc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 -- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc4003970 msgr2=0x7f8fc4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc4003970 0x7f8fc4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fde000640 1 -- 172.21.15.73:0/1655734460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc4003520 con 0x7f8fc40047c0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.200+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc4003970 0x7f8fc4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.201+0000 7f8fde000640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000a860 tx=0x7f8fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1d12d68d0be400 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.201+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0034630 con 0x7f8fc40047c0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.201+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd002f070 con 0x7f8fc40047c0 2024-04-05T22:29:55.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.201+0000 7f8fdd600640 1 -- 172.21.15.73:0/1655734460 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00349f0 con 0x7f8fc40047c0 2024-04-05T22:29:55.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1655734460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 msgr2=0x7f8fc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:55.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 --2- 172.21.15.73:0/1655734460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000a860 tx=0x7f8fc40057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdf400640 1 -- 172.21.15.73:0/1655734460 reap_dead start 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1655734460 shutdown_connections 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1655734460 >> 172.21.15.73:0/1655734460 conn(0x7f8fe009bdd0 msgr2=0x7f8fe00d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1655734460 shutdown_connections 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.202+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1655734460 wait complete. 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.204+0000 7f8fdfe00640 1 Processor -- start 2024-04-05T22:29:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.204+0000 7f8fdfe00640 1 -- start start 2024-04-05T22:29:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fe00a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe00b3a80 0x7f8fe01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe00b5640 0x7f8fe00b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b6000 con 0x7f8fc40047c0 2024-04-05T22:29:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe009c7e0 con 0x7f8fe00b5640 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe009cac0 con 0x7f8fe00b3a80 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fe00a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fe00a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60364/0 (socket says 172.21.15.73:60364) 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.205+0000 7f8fdea00640 1 -- 172.21.15.73:0/1254318190 learned_addr learned my addr 172.21.15.73:0/1254318190 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 -- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe00b3a80 msgr2=0x7f8fe01598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fde000640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe00b3a80 0x7f8fe01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdf400640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe00b5640 0x7f8fe00b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe00b3a80 0x7f8fe01598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 -- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe00b5640 msgr2=0x7f8fe00b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe00b5640 0x7f8fe00b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 -- 172.21.15.73:0/1254318190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe009cda0 con 0x7f8fc40047c0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdf400640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe00b5640 0x7f8fe00b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdea00640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc40047c0 0x7f8fe00a03b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80108b0 tx=0x7f8fc8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800eb60 con 0x7f8fc40047c0 2024-04-05T22:29:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc800ecc0 con 0x7f8fc40047c0 2024-04-05T22:29:55.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1254318190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe009d080 con 0x7f8fc40047c0 2024-04-05T22:29:55.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.206+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc801c400 con 0x7f8fc40047c0 2024-04-05T22:29:55.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.207+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1254318190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe0159df0 con 0x7f8fc40047c0 2024-04-05T22:29:55.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.208+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1254318190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe004e160 con 0x7f8fc40047c0 2024-04-05T22:29:55.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.209+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f8fc801c560 con 0x7f8fc40047c0 2024-04-05T22:29:55.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.209+0000 7f8fd6a00640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8fbc06b230 0x7f8fbc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:55.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.209+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fc808a400 con 0x7f8fc40047c0 2024-04-05T22:29:55.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.210+0000 7f8fde000640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8fbc06b230 0x7f8fbc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.211+0000 7f8fde000640 1 --2- 172.21.15.73:0/1254318190 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8fbc06b230 0x7f8fbc06d6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0034000 tx=0x7f8fd0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.214+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8fc8054d50 con 0x7f8fc40047c0 2024-04-05T22:29:55.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.434+0000 7f8fdfe00640 1 -- 172.21.15.73:0/1254318190 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fe00d18a0 con 0x7f8fbc06b230 2024-04-05T22:29:55.438 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:29:55.438+0000 7f1d85400640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-05T22:29:55.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:55.438+0000 7f8fd6a00640 1 -- 172.21.15.73:0/1254318190 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f8fd0001fd0 con 0x7f8fbc06b230 2024-04-05T22:29:55.438 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-05T22:29:55.450 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-05T22:29:55.450 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:29:55.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:55.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:29:55.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:29:55.681 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:55.681 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:55.740 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:29:55.740 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:55.791 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:56.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.001+0000 7f4611e00640 1 Processor -- start 2024-04-05T22:29:56.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.002+0000 7f4611e00640 1 -- start start 2024-04-05T22:29:56.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f4611e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f4611e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f460c0d1930 0x7f460c0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f4611e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c0ceee0 con 0x7f460c0d0bd0 2024-04-05T22:29:56.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f4611e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0ce5b0 con 0x7f460c0d1930 2024-04-05T22:29:56.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f4611e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0ce890 con 0x7f460c15cf70 2024-04-05T22:29:56.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.003+0000 7f460b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f460c0d1930 0x7f460c0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f460c0d1930 0x7f460c0b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33820/0 (socket says 172.21.15.73:33820) 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 -- 172.21.15.73:0/2142055213 learned_addr learned my addr 172.21.15.73:0/2142055213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460aa00640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 -- 172.21.15.73:0/2142055213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 msgr2=0x7f460c0d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 -- 172.21.15.73:0/2142055213 >> v1:172.21.15.157:6789/0 conn(0x7f460c0d0bd0 legacy=0x7f460c0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.004+0000 7f460b400640 1 -- 172.21.15.73:0/2142055213 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0ceb70 con 0x7f460c0d1930 2024-04-05T22:29:56.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.005+0000 7f460b400640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b2900 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4604009870 tx=0x7f460402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd5121433db39d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.005+0000 7f4609600640 1 -- 172.21.15.73:0/2142055213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040317c0 con 0x7f460c0d1930 2024-04-05T22:29:56.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.005+0000 7f4609600640 1 -- 172.21.15.73:0/2142055213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604031920 con 0x7f460c0d1930 2024-04-05T22:29:56.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.005+0000 7f4609600640 1 -- 172.21.15.73:0/2142055213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604031cb0 con 0x7f460c0d1930 2024-04-05T22:29:56.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 -- 172.21.15.73:0/2142055213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 msgr2=0x7f460c0b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b2900 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4604009870 tx=0x7f460402b860 comp rx=0 tx=0).stop 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 -- 172.21.15.73:0/2142055213 shutdown_connections 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b2900 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 --2- 172.21.15.73:0/2142055213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 -- 172.21.15.73:0/2142055213 >> 172.21.15.73:0/2142055213 conn(0x7f460c0d2df0 msgr2=0x7f460c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 -- 172.21.15.73:0/2142055213 shutdown_connections 2024-04-05T22:29:56.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.006+0000 7f4611e00640 1 -- 172.21.15.73:0/2142055213 wait complete. 2024-04-05T22:29:56.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.008+0000 7f4611e00640 1 Processor -- start 2024-04-05T22:29:56.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.008+0000 7f4611e00640 1 -- start start 2024-04-05T22:29:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 0x7f460c156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c156540 con 0x7f460c0d1930 2024-04-05T22:29:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0b74e0 con 0x7f460c0d0bd0 2024-04-05T22:29:56.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f4611e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0b77c0 con 0x7f460c15cf70 2024-04-05T22:29:56.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f460a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f460a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33836/0 (socket says 172.21.15.73:33836) 2024-04-05T22:29:56.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.009+0000 7f460a000640 1 -- 172.21.15.73:0/1904116492 learned_addr learned my addr 172.21.15.73:0/1904116492 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460aa00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 0x7f460c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460b400640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 -- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 msgr2=0x7f460c0b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 -- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 msgr2=0x7f460c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 0x7f460c156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 -- 172.21.15.73:0/1904116492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040094d0 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460aa00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 0x7f460c156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f460a000640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c430 tx=0x7f45f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4011790 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f40118f0 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4011ce0 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c0b7aa0 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.010+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c0b4050 con 0x7f460c0d1930 2024-04-05T22:29:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.012+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c068160 con 0x7f460c0d1930 2024-04-05T22:29:56.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.013+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f45f401c460 con 0x7f460c0d1930 2024-04-05T22:29:56.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.013+0000 7f4603400640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 0x7f45e806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.014+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f4089e30 con 0x7f460c0d1930 2024-04-05T22:29:56.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.014+0000 7f460aa00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 0x7f45e806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.015+0000 7f460aa00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 0x7f45e806d6f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009f50 tx=0x7f45fc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.018+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f45f408f020 con 0x7f460c0d1930 2024-04-05T22:29:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.215+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f460c06b5a0 con 0x7f460c0d1930 2024-04-05T22:29:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.216+0000 7f4603400640 1 -- 172.21.15.73:0/1904116492 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f45f4054720 con 0x7f460c0d1930 2024-04-05T22:29:56.217 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:56.217 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:56.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 msgr2=0x7f45e806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 0x7f45e806d6f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009f50 tx=0x7f45fc00b040 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 msgr2=0x7f460c0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c430 tx=0x7f45f400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 shutdown_connections 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f45e806b230 0x7f45e806d6f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460c15cf70 0x7f460c0b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d1930 0x7f460c0b6990 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 --2- 172.21.15.73:0/1904116492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0d0bd0 0x7f460c156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 >> 172.21.15.73:0/1904116492 conn(0x7f460c0d2df0 msgr2=0x7f460c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.219+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 shutdown_connections 2024-04-05T22:29:56.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.220+0000 7f4611e00640 1 -- 172.21.15.73:0/1904116492 wait complete. 2024-04-05T22:29:56.240 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:29:25.591734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:29:26.571955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:56.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:56.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.451+0000 7f1582600640 1 Processor -- start 2024-04-05T22:29:56.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.452+0000 7f1582600640 1 -- start start 2024-04-05T22:29:56.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.453+0000 7f1582600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f157c157030 0x7f157c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.453+0000 7f1582600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.453+0000 7f1582600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c06d800 con 0x7f157c0cb0b0 2024-04-05T22:29:56.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.453+0000 7f1582600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0b3090 con 0x7f157c157030 2024-04-05T22:29:56.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.453+0000 7f1582600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0b3370 con 0x7f157c157410 2024-04-05T22:29:56.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f1581c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f157c0cb0b0 0x7f157c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52832/0 (socket says 172.21.15.73:52832) 2024-04-05T22:29:56.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f1581c00640 1 -- 172.21.15.73:0/3832362351 learned_addr learned my addr 172.21.15.73:0/3832362351 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f1573e00640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f1581200640 1 --2- 172.21.15.73:0/3832362351 >> v2:172.21.15.157:3300/0 conn(0x7f157c157030 0x7f157c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f157be00640 1 -- 172.21.15.73:0/3832362351 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968768540 0 0) 0x7f157c06d800 con 0x7f157c0cb0b0 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.454+0000 7f157be00640 1 -- 172.21.15.73:0/3832362351 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1560003520 con 0x7f157c0cb0b0 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.455+0000 7f1581200640 1 -- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 msgr2=0x7f157c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.455+0000 7f1581200640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.455+0000 7f1581200640 1 -- 172.21.15.73:0/3832362351 >> v1:172.21.15.73:6789/0 conn(0x7f157c0cb0b0 legacy=0x7f157c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.455+0000 7f1581200640 1 -- 172.21.15.73:0/3832362351 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0b3650 con 0x7f157c157030 2024-04-05T22:29:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.456+0000 7f1581200640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c0ad950 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f156c0097a0 tx=0x7f156c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d27390a3f879e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.456+0000 7f157be00640 1 -- 172.21.15.73:0/3832362351 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c02d040 con 0x7f157c157030 2024-04-05T22:29:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.456+0000 7f157be00640 1 -- 172.21.15.73:0/3832362351 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c031810 con 0x7f157c157030 2024-04-05T22:29:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.456+0000 7f157be00640 1 -- 172.21.15.73:0/3832362351 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c031ba0 con 0x7f157c157030 2024-04-05T22:29:56.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 -- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 msgr2=0x7f157c0ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c0ad950 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f156c0097a0 tx=0x7f156c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 -- 172.21.15.73:0/3832362351 shutdown_connections 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 --2- 172.21.15.73:0/3832362351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c0ad950 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 -- 172.21.15.73:0/3832362351 >> 172.21.15.73:0/3832362351 conn(0x7f157c069cb0 msgr2=0x7f157c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 -- 172.21.15.73:0/3832362351 shutdown_connections 2024-04-05T22:29:56.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.457+0000 7f1582600640 1 -- 172.21.15.73:0/3832362351 wait complete. 2024-04-05T22:29:56.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.459+0000 7f1582600640 1 Processor -- start 2024-04-05T22:29:56.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.459+0000 7f1582600640 1 -- start start 2024-04-05T22:29:56.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c071880 con 0x7f157c0cb0b0 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1548f0 con 0x7f157c157030 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1582600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c153c40 con 0x7f157c157410 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1581200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1581200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33838/0 (socket says 172.21.15.73:33838) 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.460+0000 7f1581200640 1 -- 172.21.15.73:0/3615957619 learned_addr learned my addr 172.21.15.73:0/3615957619 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1573e00640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 -- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 msgr2=0x7f157c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 -- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 msgr2=0x7f157c153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 -- 172.21.15.73:0/3615957619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c009400 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1581200640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098c0 tx=0x7f156c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c03ad60 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c044400 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0445b0 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.461+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c153fc0 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.462+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c154580 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.463+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c04e160 con 0x7f157c0cb0b0 2024-04-05T22:29:56.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.464+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f156c04d440 con 0x7f157c0cb0b0 2024-04-05T22:29:56.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.464+0000 7f1579600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 0x7f155806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.465+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f156c0ac100 con 0x7f157c0cb0b0 2024-04-05T22:29:56.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.465+0000 7f1573e00640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 0x7f155806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.466+0000 7f1573e00640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 0x7f155806d720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f157c06a100 tx=0x7f1564008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.469+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f156c0df650 con 0x7f157c0cb0b0 2024-04-05T22:29:56.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.661+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f157c0ad220 con 0x7f157c0cb0b0 2024-04-05T22:29:56.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.662+0000 7f1579600640 1 -- 172.21.15.73:0/3615957619 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f156c076920 con 0x7f157c0cb0b0 2024-04-05T22:29:56.663 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 msgr2=0x7f155806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 0x7f155806d720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f157c06a100 tx=0x7f1564008040 comp rx=0 tx=0).stop 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 msgr2=0x7f157c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098c0 tx=0x7f156c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 shutdown_connections 2024-04-05T22:29:56.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f155806b260 0x7f155806d720 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157c157410 0x7f157c0af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f157c157030 0x7f157c153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 --2- 172.21.15.73:0/3615957619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f157c0cb0b0 0x7f157c0712a0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 >> 172.21.15.73:0/3615957619 conn(0x7f157c069cb0 msgr2=0x7f157c070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 shutdown_connections 2024-04-05T22:29:56.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.665+0000 7f1582600640 1 -- 172.21.15.73:0/3615957619 wait complete. 2024-04-05T22:29:56.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:29:56.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.889+0000 7fa65a200640 1 Processor -- start 2024-04-05T22:29:56.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.890+0000 7fa65a200640 1 -- start start 2024-04-05T22:29:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.891+0000 7fa65a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa65415abe0 0x7fa6540712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.891+0000 7fa65a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa65406c9a0 0x7fa65406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.891+0000 7fa65a200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65406bea0 con 0x7fa65406ba70 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.891+0000 7fa65a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65415afc0 con 0x7fa65415abe0 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.891+0000 7fa65a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65406f5b0 con 0x7fa65406c9a0 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa65415abe0 0x7fa6540712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa65415abe0 0x7fa6540712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33846/0 (socket says 172.21.15.73:33846) 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 -- 172.21.15.73:0/3740094341 learned_addr learned my addr 172.21.15.73:0/3740094341 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 -- 172.21.15.73:0/3740094341 >> v2:172.21.15.157:3300/0 conn(0x7fa65406c9a0 msgr2=0x7fa65406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 --2- 172.21.15.73:0/3740094341 >> v2:172.21.15.157:3300/0 conn(0x7fa65406c9a0 0x7fa65406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 -- 172.21.15.73:0/3740094341 >> v1:172.21.15.157:6789/0 conn(0x7fa65406ba70 legacy=0x7fa6540cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:56.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.892+0000 7fa653400640 1 -- 172.21.15.73:0/3740094341 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65406f890 con 0x7fa65415abe0 2024-04-05T22:29:56.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.893+0000 7fa653400640 1 --2- 172.21.15.73:0/3740094341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65415abe0 0x7fa6540712e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa65406ce30 tx=0x7fa64402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47f500332c15fca server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.893+0000 7fa652000640 1 -- 172.21.15.73:0/3740094341 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644031750 con 0x7fa65415abe0 2024-04-05T22:29:56.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa652000640 1 -- 172.21.15.73:0/3740094341 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440318b0 con 0x7fa65415abe0 2024-04-05T22:29:56.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa652000640 1 -- 172.21.15.73:0/3740094341 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644031c40 con 0x7fa65415abe0 2024-04-05T22:29:56.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 -- 172.21.15.73:0/3740094341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65415abe0 msgr2=0x7fa6540712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 --2- 172.21.15.73:0/3740094341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65415abe0 0x7fa6540712e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa65406ce30 tx=0x7fa64402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 -- 172.21.15.73:0/3740094341 shutdown_connections 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 --2- 172.21.15.73:0/3740094341 >> v2:172.21.15.157:3300/0 conn(0x7fa65406c9a0 0x7fa65406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 --2- 172.21.15.73:0/3740094341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65415abe0 0x7fa6540712e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 -- 172.21.15.73:0/3740094341 >> 172.21.15.73:0/3740094341 conn(0x7fa654069cb0 msgr2=0x7fa65406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 -- 172.21.15.73:0/3740094341 shutdown_connections 2024-04-05T22:29:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.894+0000 7fa65a200640 1 -- 172.21.15.73:0/3740094341 wait complete. 2024-04-05T22:29:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.896+0000 7fa65a200640 1 Processor -- start 2024-04-05T22:29:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.896+0000 7fa65a200640 1 -- start start 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa65406c9a0 0x7fa65414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65415abe0 0x7fa65413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65413ae00 con 0x7fa65406ba70 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65414ccb0 con 0x7fa65415abe0 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa65a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65414cf60 con 0x7fa65406c9a0 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa653400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa653400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33854/0 (socket says 172.21.15.73:33854) 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.897+0000 7fa653400640 1 -- 172.21.15.73:0/2117605551 learned_addr learned my addr 172.21.15.73:0/2117605551 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 -- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa65406c9a0 msgr2=0x7fa65414c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa65406c9a0 0x7fa65414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 -- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65415abe0 msgr2=0x7fa65413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65415abe0 0x7fa65413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:56.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 -- 172.21.15.73:0/2117605551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440094d0 con 0x7fa65406ba70 2024-04-05T22:29:56.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa653400640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa644000c00 tx=0x7fa6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644033c60 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644033de0 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65414d240 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.898+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64403d6b0 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.899+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65414d800 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.900+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa654068160 con 0x7fa65406ba70 2024-04-05T22:29:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.901+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fa64403a040 con 0x7fa65406ba70 2024-04-05T22:29:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.902+0000 7fa64b400640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 0x7fa63406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.902+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6440abf40 con 0x7fa65406ba70 2024-04-05T22:29:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.902+0000 7fa652a00640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 0x7fa63406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.903+0000 7fa652a00640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 0x7fa63406d6f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6541520f0 tx=0x7fa640006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:56.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:56.906+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa6440767e0 con 0x7fa65406ba70 2024-04-05T22:29:57.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.098+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa65415e370 con 0x7fa65406ba70 2024-04-05T22:29:57.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.098+0000 7fa64b400640 1 -- 172.21.15.73:0/2117605551 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa644079e00 con 0x7fa65406ba70 2024-04-05T22:29:57.099 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:29:57.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 msgr2=0x7fa63406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 0x7fa63406d6f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6541520f0 tx=0x7fa640006c90 comp rx=0 tx=0).stop 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 msgr2=0x7fa65414c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa644000c00 tx=0x7fa6440046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 shutdown_connections 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa63406b230 0x7fa63406d6f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65415abe0 0x7fa65413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa65406c9a0 0x7fa65414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 --2- 172.21.15.73:0/2117605551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa65406ba70 0x7fa65414c230 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 >> 172.21.15.73:0/2117605551 conn(0x7fa654069cb0 msgr2=0x7fa65415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:57.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 shutdown_connections 2024-04-05T22:29:57.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.101+0000 7fa65a200640 1 -- 172.21.15.73:0/2117605551 wait complete. 2024-04-05T22:29:57.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:29:57.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.332+0000 7f468e000640 1 Processor -- start 2024-04-05T22:29:57.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.332+0000 7f468e000640 1 -- start start 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.333+0000 7f468e000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f468806c630 0x7f468806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f468e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46880cbe00 0x7f46880ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f468e000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880b1b30 con 0x7f46880b22a0 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f468e000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880b1e10 con 0x7f46880cbe00 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f468e000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468806cf50 con 0x7f468806c630 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f4687e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46880b22a0 0x7f46880b2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52846/0 (socket says 172.21.15.73:52846) 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f4687e00640 1 -- 172.21.15.73:0/2824179489 learned_addr learned my addr 172.21.15.73:0/2824179489 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f467ea00640 1 --2- 172.21.15.73:0/2824179489 >> v2:172.21.15.73:3300/0 conn(0x7f46880cbe00 0x7f46880ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.334+0000 7f4687400640 1 --2- 172.21.15.73:0/2824179489 >> v2:172.21.15.157:3300/0 conn(0x7f468806c630 0x7f468806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f4686a00640 1 -- 172.21.15.73:0/2824179489 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492917542 0 0) 0x7f46880b1b30 con 0x7f46880b22a0 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f4686a00640 1 -- 172.21.15.73:0/2824179489 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f466c003520 con 0x7f46880b22a0 2024-04-05T22:29:57.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f467ea00640 1 -- 172.21.15.73:0/2824179489 >> v2:172.21.15.157:3300/0 conn(0x7f468806c630 msgr2=0x7f468806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:57.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f467ea00640 1 --2- 172.21.15.73:0/2824179489 >> v2:172.21.15.157:3300/0 conn(0x7f468806c630 0x7f468806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f467ea00640 1 -- 172.21.15.73:0/2824179489 >> v1:172.21.15.73:6789/0 conn(0x7f46880b22a0 legacy=0x7f46880b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.335+0000 7f467ea00640 1 -- 172.21.15.73:0/2824179489 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880cc300 con 0x7f46880cbe00 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.336+0000 7f467ea00640 1 --2- 172.21.15.73:0/2824179489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880cbe00 0x7f46880ce0a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f468806f360 tx=0x7f467002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e13f208a6d3e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.336+0000 7f4686a00640 1 -- 172.21.15.73:0/2824179489 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670031830 con 0x7f46880cbe00 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.336+0000 7f4686a00640 1 -- 172.21.15.73:0/2824179489 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670031990 con 0x7f46880cbe00 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.336+0000 7f4686a00640 1 -- 172.21.15.73:0/2824179489 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670031d20 con 0x7f46880cbe00 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 -- 172.21.15.73:0/2824179489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880cbe00 msgr2=0x7f46880ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 --2- 172.21.15.73:0/2824179489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880cbe00 0x7f46880ce0a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f468806f360 tx=0x7f467002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 -- 172.21.15.73:0/2824179489 shutdown_connections 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 --2- 172.21.15.73:0/2824179489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880cbe00 0x7f46880ce0a0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 --2- 172.21.15.73:0/2824179489 >> v2:172.21.15.157:3300/0 conn(0x7f468806c630 0x7f468806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 -- 172.21.15.73:0/2824179489 >> 172.21.15.73:0/2824179489 conn(0x7f46880b0830 msgr2=0x7f46880b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:57.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 -- 172.21.15.73:0/2824179489 shutdown_connections 2024-04-05T22:29:57.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.337+0000 7f468e000640 1 -- 172.21.15.73:0/2824179489 wait complete. 2024-04-05T22:29:57.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.339+0000 7f468e000640 1 Processor -- start 2024-04-05T22:29:57.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.339+0000 7f468e000640 1 -- start start 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468806c630 0x7f46881f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880cbe00 0x7f4688140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881f3520 con 0x7f46880b22a0 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468813fa90 con 0x7f468806c630 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f468e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468813fd70 con 0x7f46880cbe00 2024-04-05T22:29:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f467ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f467ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33868/0 (socket says 172.21.15.73:33868) 2024-04-05T22:29:57.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f467ea00640 1 -- 172.21.15.73:0/2552965856 learned_addr learned my addr 172.21.15.73:0/2552965856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f4687e00640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880cbe00 0x7f4688140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f4687400640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468806c630 0x7f46881f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.340+0000 7f467ea00640 1 -- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880cbe00 msgr2=0x7f4688140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467ea00640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880cbe00 0x7f4688140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467ea00640 1 -- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468806c630 msgr2=0x7f46881f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467ea00640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468806c630 0x7f46881f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467ea00640 1 -- 172.21.15.73:0/2552965856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670009400 con 0x7f46880b22a0 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467ea00640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4670000c00 tx=0x7f46700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467003a8f0 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467003aa70 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467003ae20 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4688140050 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.341+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688140610 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.343+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688071bf0 con 0x7f46880b22a0 2024-04-05T22:29:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.343+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f467004d450 con 0x7f46880b22a0 2024-04-05T22:29:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.344+0000 7f467fe00640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 0x7f466406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.344+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46700ac0c0 con 0x7f46880b22a0 2024-04-05T22:29:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.345+0000 7f4687400640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 0x7f466406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.346+0000 7f4687400640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 0x7f466406d6f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4678006fd0 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.349+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4670076810 con 0x7f46880b22a0 2024-04-05T22:29:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.543+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4688068160 con 0x7f46880b22a0 2024-04-05T22:29:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.544+0000 7f467fe00640 1 -- 172.21.15.73:0/2552965856 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4670035070 con 0x7f46880b22a0 2024-04-05T22:29:57.544 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:29:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 msgr2=0x7f466406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 0x7f466406d6f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4678006fd0 tx=0x7f4678008040 comp rx=0 tx=0).stop 2024-04-05T22:29:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 msgr2=0x7f468813c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4670000c00 tx=0x7f46700046c0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 shutdown_connections 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f466406b230 0x7f466406d6f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880cbe00 0x7f4688140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46880b22a0 0x7f468813c560 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 --2- 172.21.15.73:0/2552965856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468806c630 0x7f46881f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 >> 172.21.15.73:0/2552965856 conn(0x7f46880b0830 msgr2=0x7f46880afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 shutdown_connections 2024-04-05T22:29:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.547+0000 7f468e000640 1 -- 172.21.15.73:0/2552965856 wait complete. 2024-04-05T22:29:57.566 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:29:57.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.782+0000 7f8a0de00640 1 Processor -- start 2024-04-05T22:29:57.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.782+0000 7f8a0de00640 1 -- start start 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f8a0806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a0de00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a080b3c40 con 0x7f8a08070f50 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a0de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0806c490 con 0x7f8a08146490 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a0de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806c6f0 con 0x7f8a080b3640 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a07e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8a08070f50 0x7f8a080b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52848/0 (socket says 172.21.15.73:52848) 2024-04-05T22:29:57.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a07e00640 1 -- 172.21.15.73:0/932952723 learned_addr learned my addr 172.21.15.73:0/932952723 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:57.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.784+0000 7f8a0d400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f8a0806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664986888 0 0) 0x7f89f4001110 con 0x7f8a08070f50 2024-04-05T22:29:57.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f0003520 con 0x7f8a08070f50 2024-04-05T22:29:57.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110836064 0 0) 0x7f89f0003520 con 0x7f8a08070f50 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 msgr2=0x7f8a0806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f8a0806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 >> v1:172.21.15.157:6789/0 conn(0x7f8a08146490 legacy=0x7f8a08070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0806c9d0 con 0x7f8a08070f50 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f89f4001800 con 0x7f8a08070f50 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 >> v1:172.21.15.73:6789/0 conn(0x7f8a08070f50 legacy=0x7f8a080b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f89f00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f0003f20 0x7f89f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806c6f0 con 0x7f89f0003f20 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000ba60 con 0x7f89f0004d00 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.785+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000bc20 con 0x7f8a080b3640 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f0003f20 0x7f89f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a0ca00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f89f00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a0d400640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 -- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 msgr2=0x7f89f00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b3640 0x7f89f00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 -- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89f0004d00 msgr2=0x7f89f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.786+0000 7f8a07e00640 1 -- 172.21.15.73:0/932952723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0003520 con 0x7f89f0003f20 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.787+0000 7f8a07e00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f0003f20 0x7f89f00047c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c240 tx=0x7f89f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ecd278882bd8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.787+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4005ad0 con 0x7f89f0003f20 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.787+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4005c50 con 0x7f89f0003f20 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.787+0000 7f8a07400640 1 -- 172.21.15.73:0/932952723 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f402c8b0 con 0x7f89f0003f20 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 -- 172.21.15.73:0/932952723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f0003f20 msgr2=0x7f89f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 --2- 172.21.15.73:0/932952723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f0003f20 0x7f89f00047c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c240 tx=0x7f89f00057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0d400640 1 -- 172.21.15.73:0/932952723 reap_dead start 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 -- 172.21.15.73:0/932952723 shutdown_connections 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 -- 172.21.15.73:0/932952723 >> 172.21.15.73:0/932952723 conn(0x7f8a080b0830 msgr2=0x7f8a080b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 -- 172.21.15.73:0/932952723 shutdown_connections 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.788+0000 7f8a0de00640 1 -- 172.21.15.73:0/932952723 wait complete. 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.790+0000 7f8a0de00640 1 Processor -- start 2024-04-05T22:29:57.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.790+0000 7f8a0de00640 1 -- start start 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89f0003f20 0x7f8a080c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a08140550 0x7f8a0815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0815cdd0 con 0x7f8a081409d0 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080ae5f0 con 0x7f8a08140550 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080ae8d0 con 0x7f89f0003f20 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33884/0 (socket says 172.21.15.73:33884) 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.791+0000 7f8a0d400640 1 -- 172.21.15.73:0/3309698311 learned_addr learned my addr 172.21.15.73:0/3309698311 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0ca00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89f0003f20 0x7f8a080c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a07e00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a08140550 0x7f8a0815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 -- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89f0003f20 msgr2=0x7f8a080c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89f0003f20 0x7f8a080c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 -- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a08140550 msgr2=0x7f8a0815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:57.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a08140550 0x7f8a0815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:57.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 -- 172.21.15.73:0/3309698311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080aebb0 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a0d400640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f8a08071720 tx=0x7f8a0000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.792+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00017400 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.793+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00011040 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.793+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000177c0 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.793+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a080aee90 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.793+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a080c4680 con 0x7f8a081409d0 2024-04-05T22:29:57.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.794+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a080af080 con 0x7f8a081409d0 2024-04-05T22:29:57.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.795+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f8a00017920 con 0x7f8a081409d0 2024-04-05T22:29:57.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.795+0000 7f8a04c00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 0x7f89e806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:57.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.796+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a0008a430 con 0x7f8a081409d0 2024-04-05T22:29:57.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.796+0000 7f8a0ca00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 0x7f89e806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:57.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.797+0000 7f8a0ca00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 0x7f89e806d720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a0813dc70 tx=0x7f89fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:57.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:57.800+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8a00054c50 con 0x7f8a081409d0 2024-04-05T22:29:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.005+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a0804e160 con 0x7f8a081409d0 2024-04-05T22:29:58.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.006+0000 7f8a04c00640 1 -- 172.21.15.73:0/3309698311 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a00058270 con 0x7f8a081409d0 2024-04-05T22:29:58.006 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 msgr2=0x7f89e806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 0x7f89e806d720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a0813dc70 tx=0x7f89fc009040 comp rx=0 tx=0).stop 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 msgr2=0x7f8a0815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f8a08071720 tx=0x7f8a0000f4a0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 shutdown_connections 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f89e806b260 0x7f89e806d720 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a081409d0 0x7f8a0815c890 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a08140550 0x7f8a0815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 --2- 172.21.15.73:0/3309698311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89f0003f20 0x7f8a080c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 >> 172.21.15.73:0/3309698311 conn(0x7f8a080b0830 msgr2=0x7f8a0815e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.008+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 shutdown_connections 2024-04-05T22:29:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.009+0000 7f8a0de00640 1 -- 172.21.15.73:0/3309698311 wait complete. 2024-04-05T22:29:58.027 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:29:58.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:58.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:29:58.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:29:58.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:29:58.133 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:29:58.182 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:58.182 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:58.239 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:29:58.240 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:29:58.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:29:58.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.505+0000 7f0d55400640 1 Processor -- start 2024-04-05T22:29:58.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.505+0000 7f0d55400640 1 -- start start 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d55400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d5006cb80 0x7f0d5006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d55400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 0x7f0d5015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d55400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5006f5b0 con 0x7f0d501587d0 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d55400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5006d500 con 0x7f0d5006cb80 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d55400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5015f220 con 0x7f0d5015e850 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d5006cb80 0x7f0d5006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4e800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d501587d0 0x7f0d500cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52850/0 (socket says 172.21.15.73:52850) 2024-04-05T22:29:58.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4e800640 1 -- 172.21.15.73:0/908040426 learned_addr learned my addr 172.21.15.73:0/908040426 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:58.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4f200640 1 --2- 172.21.15.73:0/908040426 >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 0x7f0d5015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4de00640 1 -- 172.21.15.73:0/908040426 >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 msgr2=0x7f0d5015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:58.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4de00640 1 --2- 172.21.15.73:0/908040426 >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 0x7f0d5015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4de00640 1 -- 172.21.15.73:0/908040426 >> v1:172.21.15.73:6789/0 conn(0x7f0d501587d0 legacy=0x7f0d500cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.508+0000 7f0d4de00640 1 -- 172.21.15.73:0/908040426 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50070410 con 0x7f0d5006cb80 2024-04-05T22:29:58.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.507+0000 7f0d4d400640 1 -- 172.21.15.73:0/908040426 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775863853 0 0) 0x7f0d5006f5b0 con 0x7f0d501587d0 2024-04-05T22:29:58.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.508+0000 7f0d4f200640 1 --2- 172.21.15.73:0/908040426 >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 0x7f0d5015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:58.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.509+0000 7f0d4de00640 1 --2- 172.21.15.73:0/908040426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5006cb80 0x7f0d5006cfc0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0d5015c270 tx=0x7f0d3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee659d5de8ea5dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.509+0000 7f0d4d400640 1 -- 172.21.15.73:0/908040426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c031830 con 0x7f0d5006cb80 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.509+0000 7f0d4d400640 1 -- 172.21.15.73:0/908040426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c031990 con 0x7f0d5006cb80 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.509+0000 7f0d4d400640 1 -- 172.21.15.73:0/908040426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c031d20 con 0x7f0d5006cb80 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.509+0000 7f0d55400640 1 -- 172.21.15.73:0/908040426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5006cb80 msgr2=0x7f0d5006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 --2- 172.21.15.73:0/908040426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5006cb80 0x7f0d5006cfc0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0d5015c270 tx=0x7f0d3c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 -- 172.21.15.73:0/908040426 shutdown_connections 2024-04-05T22:29:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 --2- 172.21.15.73:0/908040426 >> v2:172.21.15.157:3300/0 conn(0x7f0d5015e850 0x7f0d5015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 --2- 172.21.15.73:0/908040426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5006cb80 0x7f0d5006cfc0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 -- 172.21.15.73:0/908040426 >> 172.21.15.73:0/908040426 conn(0x7f0d500ce8e0 msgr2=0x7f0d500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 -- 172.21.15.73:0/908040426 shutdown_connections 2024-04-05T22:29:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.510+0000 7f0d55400640 1 -- 172.21.15.73:0/908040426 wait complete. 2024-04-05T22:29:58.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.511+0000 7f0d55400640 1 Processor -- start 2024-04-05T22:29:58.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.512+0000 7f0d55400640 1 -- start start 2024-04-05T22:29:58.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 0x7f0d50154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d501587d0 0x7f0d5014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5014ca20 con 0x7f0d5015e850 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b39f0 con 0x7f0d5006cb80 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d55400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b2bb0 con 0x7f0d501587d0 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33904/0 (socket says 172.21.15.73:33904) 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 -- 172.21.15.73:0/3779244707 learned_addr learned my addr 172.21.15.73:0/3779244707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4e800640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 0x7f0d50154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 -- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d501587d0 msgr2=0x7f0d5014c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4de00640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d501587d0 0x7f0d5014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d501587d0 0x7f0d5014c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 -- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 msgr2=0x7f0d50154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 0x7f0d50154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4f200640 1 -- 172.21.15.73:0/3779244707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c009400 con 0x7f0d5015e850 2024-04-05T22:29:58.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.513+0000 7f0d4e800640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 0x7f0d50154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:58.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d4f200640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800cd50 tx=0x7f0d4800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800f680 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4800f800 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800fbb0 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d500b1b10 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.514+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d500b20d0 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.515+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d50066af0 con 0x7f0d5015e850 2024-04-05T22:29:58.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.516+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f0d4800b040 con 0x7f0d5015e850 2024-04-05T22:29:58.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.517+0000 7f0d46a00640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 0x7f0d3006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.518+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d4808aa30 con 0x7f0d5015e850 2024-04-05T22:29:58.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.518+0000 7f0d4e800640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 0x7f0d3006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.519+0000 7f0d4e800640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 0x7f0d3006d6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000f60 tx=0x7f0d40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.522+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0d480551c0 con 0x7f0d5015e850 2024-04-05T22:29:58.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.717+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d50066f20 con 0x7f0d5015e850 2024-04-05T22:29:58.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.719+0000 7f0d46a00640 1 -- 172.21.15.73:0/3779244707 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f0d4801d090 con 0x7f0d5015e850 2024-04-05T22:29:58.720 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:29:58.720 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:29:58.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 msgr2=0x7f0d3006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 0x7f0d3006d6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000f60 tx=0x7f0d40009040 comp rx=0 tx=0).stop 2024-04-05T22:29:58.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 msgr2=0x7f0d500b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800cd50 tx=0x7f0d4800a650 comp rx=0 tx=0).stop 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 shutdown_connections 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d3006b230 0x7f0d3006d6f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d5015e850 0x7f0d500b34b0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d501587d0 0x7f0d5014c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 --2- 172.21.15.73:0/3779244707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5006cb80 0x7f0d50154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 >> 172.21.15.73:0/3779244707 conn(0x7f0d500ce8e0 msgr2=0x7f0d5015a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 shutdown_connections 2024-04-05T22:29:58.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.722+0000 7f0d55400640 1 -- 172.21.15.73:0/3779244707 wait complete. 2024-04-05T22:29:58.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:29:25.591734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:29:26.571955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:29:58.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:29:58.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.952+0000 7fe1d5a00640 1 Processor -- start 2024-04-05T22:29:58.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.952+0000 7fe1d5a00640 1 -- start start 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.953+0000 7fe1d5a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1d0070d10 0x7fe1d00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1d5a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00b3d50 con 0x7fe1d01599a0 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1d5a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00b2a90 con 0x7fe1d015a120 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1d5a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d015ae30 con 0x7fe1d0070d10 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1cfe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe1d01599a0 0x7fe1d0070600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52858/0 (socket says 172.21.15.73:52858) 2024-04-05T22:29:58.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1cfe00640 1 -- 172.21.15.73:0/3312703535 learned_addr learned my addr 172.21.15.73:0/3312703535 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035580056 0 0) 0x7fe1c0001110 con 0x7fe1d01599a0 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003520 con 0x7fe1d01599a0 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.954+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> v2:172.21.15.157:3300/0 conn(0x7fe1d0070d10 0x7fe1d00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568529249 0 0) 0x7fe1b4003520 con 0x7fe1d01599a0 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 >> v2:172.21.15.157:3300/0 conn(0x7fe1d0070d10 msgr2=0x7fe1d00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:58.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 --2- 172.21.15.73:0/3312703535 >> v2:172.21.15.157:3300/0 conn(0x7fe1d0070d10 0x7fe1d00b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 >> v1:172.21.15.157:6789/0 conn(0x7fe1d015a120 legacy=0x7fe1d015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00b2d30 con 0x7fe1d01599a0 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe1c0001800 con 0x7fe1d01599a0 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> v2:172.21.15.157:3300/0 conn(0x7fe1d0070d10 0x7fe1d00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 >> v1:172.21.15.73:6789/0 conn(0x7fe1d01599a0 legacy=0x7fe1d0070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1b4003970 0x7fe1b4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b4005180 0x7fe1b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d015ae30 con 0x7fe1b40047c0 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400c0b0 con 0x7fe1b4005180 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400e480 con 0x7fe1b4003970 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.955+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1cfe00640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1b4003970 0x7fe1b4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1c7400640 1 -- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1b4003970 msgr2=0x7fe1b4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1b4003970 0x7fe1b4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1c7400640 1 -- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b4005180 msgr2=0x7fe1b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1d5000640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b4005180 0x7fe1b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b4005180 0x7fe1b400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1d5000640 1 -- 172.21.15.73:0/3312703535 reap_dead start 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1c7400640 1 -- 172.21.15.73:0/3312703535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4003520 con 0x7fe1b40047c0 2024-04-05T22:29:58.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.956+0000 7fe1cfe00640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1b4003970 0x7fe1b4003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.957+0000 7fe1c7400640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d07bd5f7a265430 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.957+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8004030 con 0x7fe1b40047c0 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.957+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8004190 con 0x7fe1b40047c0 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.957+0000 7fe1cf400640 1 -- 172.21.15.73:0/3312703535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8003610 con 0x7fe1b40047c0 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 -- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 msgr2=0x7fe1b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b40057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 -- 172.21.15.73:0/3312703535 shutdown_connections 2024-04-05T22:29:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/3312703535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 -- 172.21.15.73:0/3312703535 >> 172.21.15.73:0/3312703535 conn(0x7fe1d006b580 msgr2=0x7fe1d00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 -- 172.21.15.73:0/3312703535 shutdown_connections 2024-04-05T22:29:58.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.958+0000 7fe1d5a00640 1 -- 172.21.15.73:0/3312703535 wait complete. 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.960+0000 7fe1d5a00640 1 Processor -- start 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.960+0000 7fe1d5a00640 1 -- start start 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b40047c0 0x7fe1d00cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1d01510b0 0x7fe1d0151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00ce590 con 0x7fe1d0152f90 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0151aa0 con 0x7fe1b40047c0 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00ce240 con 0x7fe1d01510b0 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33928/0 (socket says 172.21.15.73:33928) 2024-04-05T22:29:58.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 -- 172.21.15.73:0/1215949696 learned_addr learned my addr 172.21.15.73:0/1215949696 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 -- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1d01510b0 msgr2=0x7fe1d0151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1cfe00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b40047c0 0x7fe1d00cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1d01510b0 0x7fe1d0151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 -- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b40047c0 msgr2=0x7fe1d00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b40047c0 0x7fe1d00cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.961+0000 7fe1c7400640 1 -- 172.21.15.73:0/1215949696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0153470 con 0x7fe1d0152f90 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1c7400640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8030610 con 0x7fe1d0152f90 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b803e040 con 0x7fe1d0152f90 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80309b0 con 0x7fe1d0152f90 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d0153750 con 0x7fe1d0152f90 2024-04-05T22:29:58.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.962+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d014fa60 con 0x7fe1d0152f90 2024-04-05T22:29:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.963+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0153c70 con 0x7fe1d0152f90 2024-04-05T22:29:58.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.964+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fe1b8030b10 con 0x7fe1d0152f90 2024-04-05T22:29:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.965+0000 7fe1ccc00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 0x7fe1ac06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:58.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.965+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1b80ac440 con 0x7fe1d0152f90 2024-04-05T22:29:58.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.966+0000 7fe1cfe00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 0x7fe1ac06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.967+0000 7fe1cfe00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 0x7fe1ac06d720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0001270 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:58.969+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe1b8076a70 con 0x7fe1d0152f90 2024-04-05T22:29:59.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.164+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1d004e160 con 0x7fe1d0152f90 2024-04-05T22:29:59.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.165+0000 7fe1ccc00640 1 -- 172.21.15.73:0/1215949696 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1b802d030 con 0x7fe1d0152f90 2024-04-05T22:29:59.166 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 msgr2=0x7fe1ac06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 0x7fe1ac06d720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0001270 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 msgr2=0x7fe1d0142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8004730 comp rx=0 tx=0).stop 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 shutdown_connections 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe1ac06b260 0x7fe1ac06d720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1d01510b0 0x7fe1d0151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0152f90 0x7fe1d0142100 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 --2- 172.21.15.73:0/1215949696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b40047c0 0x7fe1d00cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 >> 172.21.15.73:0/1215949696 conn(0x7fe1d006b580 msgr2=0x7fe1d0156c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:59.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 shutdown_connections 2024-04-05T22:29:59.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.168+0000 7fe1d5a00640 1 -- 172.21.15.73:0/1215949696 wait complete. 2024-04-05T22:29:59.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:29:59.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.399+0000 7f934f200640 1 Processor -- start 2024-04-05T22:29:59.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.399+0000 7f934f200640 1 -- start start 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.400+0000 7f934f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f9350165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.400+0000 7f934f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500706a0 con 0x7f9350144420 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.400+0000 7f934f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9350070980 con 0x7f93500cdb40 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.400+0000 7f934f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9350070c60 con 0x7f935006cad0 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.401+0000 7f934de00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9350144420 0x7f935006fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52868/0 (socket says 172.21.15.73:52868) 2024-04-05T22:29:59.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.401+0000 7f934de00640 1 -- 172.21.15.73:0/1538448945 learned_addr learned my addr 172.21.15.73:0/1538448945 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.401+0000 7f934e800640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f9350165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.401+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731980784 0 0) 0x7f93500706a0 con 0x7f9350144420 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.401+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9334003520 con 0x7f9350144420 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1992484902 0 0) 0x7f9334003520 con 0x7f9350144420 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 msgr2=0x7f9350165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f9350165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 >> v1:172.21.15.157:6789/0 conn(0x7f93500cdb40 legacy=0x7f93500cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935006b5a0 con 0x7f9350144420 2024-04-05T22:29:59.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9340001670 con 0x7f9350144420 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 >> v1:172.21.15.73:6789/0 conn(0x7f9350144420 legacy=0x7f935006fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f93340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334003f20 0x7f93340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350070c60 con 0x7f9334003f20 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933400bb10 con 0x7f9334004d00 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933400bcd0 con 0x7f935006cad0 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934d400640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334003f20 0x7f93340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.402+0000 7f934de00640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f93340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934e800640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934d400640 1 -- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 msgr2=0x7f93340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934d400640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935006cad0 0x7f93340039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934d400640 1 -- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334004d00 msgr2=0x7f9334005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:59.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934d400640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934d400640 1 -- 172.21.15.73:0/1538448945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334003520 con 0x7f9334003f20 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.403+0000 7f934e800640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.404+0000 7f934d400640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334003f20 0x7f93340047c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f933c001750 tx=0x7f93340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57abc82241af6c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.404+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0052d0 con 0x7f9334003f20 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.404+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c02e050 con 0x7f9334003f20 2024-04-05T22:29:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.404+0000 7f934ca00640 1 -- 172.21.15.73:0/1538448945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0035f0 con 0x7f9334003f20 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 -- 172.21.15.73:0/1538448945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334003f20 msgr2=0x7f93340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 --2- 172.21.15.73:0/1538448945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334003f20 0x7f93340047c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f933c001750 tx=0x7f93340057f0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934e800640 1 -- 172.21.15.73:0/1538448945 reap_dead start 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 -- 172.21.15.73:0/1538448945 shutdown_connections 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 -- 172.21.15.73:0/1538448945 >> 172.21.15.73:0/1538448945 conn(0x7f93500b07c0 msgr2=0x7f93500b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 -- 172.21.15.73:0/1538448945 shutdown_connections 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.405+0000 7f934f200640 1 -- 172.21.15.73:0/1538448945 wait complete. 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.406+0000 7f934f200640 1 Processor -- start 2024-04-05T22:29:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 -- start start 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334003f20 0x7f93501403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935013fea0 0x7f93501e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93501e49b0 con 0x7f935013f770 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93501e5380 con 0x7f9334003f20 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.407+0000 7f934f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93501e43a0 con 0x7f935013fea0 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33942/0 (socket says 172.21.15.73:33942) 2024-04-05T22:29:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 -- 172.21.15.73:0/2868625219 learned_addr learned my addr 172.21.15.73:0/2868625219 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:59.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 -- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935013fea0 msgr2=0x7f93501e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935013fea0 0x7f93501e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 -- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334003f20 msgr2=0x7f93501403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334003f20 0x7f93501403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 -- 172.21.15.73:0/2868625219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350070470 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.408+0000 7f934e800640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f93500cb680 tx=0x7f934800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.409+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348017400 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.409+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348011040 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.409+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480177c0 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.409+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935015d7c0 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.409+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93501e4640 con 0x7f935013f770 2024-04-05T22:29:59.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.410+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935004e160 con 0x7f935013f770 2024-04-05T22:29:59.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.411+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f93480203b0 con 0x7f935013f770 2024-04-05T22:29:59.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.412+0000 7f9346000640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 0x7f933006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.412+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f934808a270 con 0x7f935013f770 2024-04-05T22:29:59.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.412+0000 7f934de00640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 0x7f933006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.414+0000 7f934de00640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 0x7f933006d720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93500ad550 tx=0x7f93400011c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.416+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f93480549c0 con 0x7f935013f770 2024-04-05T22:29:59.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.614+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93501e4af0 con 0x7f935013f770 2024-04-05T22:29:59.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.615+0000 7f9346000640 1 -- 172.21.15.73:0/2868625219 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9348057fe0 con 0x7f935013f770 2024-04-05T22:29:59.615 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:29:59.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 msgr2=0x7f933006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 0x7f933006d720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93500ad550 tx=0x7f93400011c0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 msgr2=0x7f93500ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f93500cb680 tx=0x7f934800f4a0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 shutdown_connections 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933006b260 0x7f933006d720 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935013f770 0x7f93500ae6d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935013fea0 0x7f93501e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 --2- 172.21.15.73:0/2868625219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9334003f20 0x7f93501403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 >> 172.21.15.73:0/2868625219 conn(0x7f93500b07c0 msgr2=0x7f93500b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 shutdown_connections 2024-04-05T22:29:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.618+0000 7f934f200640 1 -- 172.21.15.73:0/2868625219 wait complete. 2024-04-05T22:29:59.635 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:29:59.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.845+0000 7f9953200640 1 Processor -- start 2024-04-05T22:29:59.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.845+0000 7f9953200640 1 -- start start 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9953200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f994c06c630 0x7f994c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9953200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f994c070c50 0x7f994c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9953200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c146ac0 con 0x7f994c15dd90 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9953200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994c070720 con 0x7f994c06c630 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9953200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994c0acf60 con 0x7f994c070c50 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f994be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f994c06c630 0x7f994c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9943400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f994c15dd90 0x7f994c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52878/0 (socket says 172.21.15.73:52878) 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.847+0000 7f9943400640 1 -- 172.21.15.73:0/3100507033 learned_addr learned my addr 172.21.15.73:0/3100507033 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.848+0000 7f9951200640 1 --2- 172.21.15.73:0/3100507033 >> v2:172.21.15.157:3300/0 conn(0x7f994c070c50 0x7f994c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.848+0000 7f994be00640 1 -- 172.21.15.73:0/3100507033 >> v2:172.21.15.157:3300/0 conn(0x7f994c070c50 msgr2=0x7f994c0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.848+0000 7f994be00640 1 --2- 172.21.15.73:0/3100507033 >> v2:172.21.15.157:3300/0 conn(0x7f994c070c50 0x7f994c0710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.848+0000 7f994be00640 1 -- 172.21.15.73:0/3100507033 >> v1:172.21.15.73:6789/0 conn(0x7f994c15dd90 legacy=0x7f994c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.848+0000 7f994be00640 1 -- 172.21.15.73:0/3100507033 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0ad240 con 0x7f994c06c630 2024-04-05T22:29:59.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.849+0000 7f994be00640 1 --2- 172.21.15.73:0/3100507033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c06c630 0x7f994c0ac8d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ae750 tx=0x7f993c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f391da300c69e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.849+0000 7f994b400640 1 -- 172.21.15.73:0/3100507033 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c031830 con 0x7f994c06c630 2024-04-05T22:29:59.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.849+0000 7f994b400640 1 -- 172.21.15.73:0/3100507033 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c031990 con 0x7f994c06c630 2024-04-05T22:29:59.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.849+0000 7f994b400640 1 -- 172.21.15.73:0/3100507033 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c031d20 con 0x7f994c06c630 2024-04-05T22:29:59.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 -- 172.21.15.73:0/3100507033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c06c630 msgr2=0x7f994c0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 --2- 172.21.15.73:0/3100507033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c06c630 0x7f994c0ac8d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ae750 tx=0x7f993c02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 -- 172.21.15.73:0/3100507033 shutdown_connections 2024-04-05T22:29:59.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 --2- 172.21.15.73:0/3100507033 >> v2:172.21.15.157:3300/0 conn(0x7f994c070c50 0x7f994c0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 --2- 172.21.15.73:0/3100507033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c06c630 0x7f994c0ac8d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 -- 172.21.15.73:0/3100507033 >> 172.21.15.73:0/3100507033 conn(0x7f994c0b0830 msgr2=0x7f994c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:29:59.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 -- 172.21.15.73:0/3100507033 shutdown_connections 2024-04-05T22:29:59.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.850+0000 7f9953200640 1 -- 172.21.15.73:0/3100507033 wait complete. 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.852+0000 7f9953200640 1 Processor -- start 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.852+0000 7f9953200640 1 -- start start 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f994c06c630 0x7f994c1f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f994c070c50 0x7f994c1424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1429e0 con 0x7f994c15dd90 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1f36a0 con 0x7f994c070c50 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9953200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0c3aa0 con 0x7f994c06c630 2024-04-05T22:29:59.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33956/0 (socket says 172.21.15.73:33956) 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 -- 172.21.15.73:0/748815230 learned_addr learned my addr 172.21.15.73:0/748815230 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 -- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f994c06c630 msgr2=0x7f994c1f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f994c06c630 0x7f994c1f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 -- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f994c070c50 msgr2=0x7f994c1424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f994c070c50 0x7f994c1424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.853+0000 7f9951200640 1 -- 172.21.15.73:0/748815230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c009400 con 0x7f994c15dd90 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9951200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f9944007e60 tx=0x7f994400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944014420 con 0x7f994c15dd90 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99440145a0 con 0x7f994c15dd90 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c0c3d80 con 0x7f994c15dd90 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944014950 con 0x7f994c15dd90 2024-04-05T22:29:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.854+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c0c4340 con 0x7f994c15dd90 2024-04-05T22:29:59.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.855+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c068160 con 0x7f994c15dd90 2024-04-05T22:29:59.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.856+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f9944014ab0 con 0x7f994c15dd90 2024-04-05T22:29:59.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.857+0000 7f9948c00640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 0x7f992806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:29:59.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.857+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f994408a6a0 con 0x7f994c15dd90 2024-04-05T22:29:59.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.857+0000 7f994be00640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 0x7f992806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:29:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.859+0000 7f994be00640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 0x7f992806d6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f993c036000 tx=0x7f993c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:29:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:29:59.861+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9944055040 con 0x7f994c15dd90 2024-04-05T22:30:00.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.056+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f994c0adb90 con 0x7f994c15dd90 2024-04-05T22:30:00.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.057+0000 7f9948c00640 1 -- 172.21.15.73:0/748815230 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9944058660 con 0x7f994c15dd90 2024-04-05T22:30:00.057 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 msgr2=0x7f992806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 0x7f992806d6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f993c036000 tx=0x7f993c030000 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 msgr2=0x7f994c1f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f9944007e60 tx=0x7f994400e4b0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 shutdown_connections 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f992806b230 0x7f992806d6f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994c15dd90 0x7f994c1f3160 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f994c070c50 0x7f994c1424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 --2- 172.21.15.73:0/748815230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f994c06c630 0x7f994c1f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 >> 172.21.15.73:0/748815230 conn(0x7f994c0b0830 msgr2=0x7f994c145d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 shutdown_connections 2024-04-05T22:30:00.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.060+0000 7f9953200640 1 -- 172.21.15.73:0/748815230 wait complete. 2024-04-05T22:30:00.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:30:00.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.294+0000 7f5505c00640 1 Processor -- start 2024-04-05T22:30:00.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.294+0000 7f5505c00640 1 -- start start 2024-04-05T22:30:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f5505c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55001587d0 0x7f55000cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f5505c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f5505c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550006f5b0 con 0x7f55000ccd20 2024-04-05T22:30:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f5505c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550015f240 con 0x7f55001587d0 2024-04-05T22:30:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f5505c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5500070410 con 0x7f550015e850 2024-04-05T22:30:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.296+0000 7f54ff400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.297+0000 7f54fea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55001587d0 0x7f55000cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.297+0000 7f54ff400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36046/0 (socket says 172.21.15.73:36046) 2024-04-05T22:30:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.297+0000 7f54ff400640 1 -- 172.21.15.73:0/1322230809 learned_addr learned my addr 172.21.15.73:0/1322230809 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.297+0000 7f54fea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55001587d0 0x7f55000cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57430/0 (socket says 172.21.15.73:57430) 2024-04-05T22:30:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.298+0000 7f54fea00640 1 -- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 msgr2=0x7f550015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.298+0000 7f54fea00640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.298+0000 7f54fea00640 1 -- 172.21.15.73:0/1322230809 >> v1:172.21.15.157:6789/0 conn(0x7f55000ccd20 legacy=0x7f550006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.298+0000 7f54fea00640 1 -- 172.21.15.73:0/1322230809 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500158bb0 con 0x7f55001587d0 2024-04-05T22:30:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.298+0000 7f54ff400640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:30:00.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.299+0000 7f54fea00640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55001587d0 0x7f55000cc7e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3ee29a42250d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:00.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.299+0000 7f54fd600640 1 -- 172.21.15.73:0/1322230809 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f002d040 con 0x7f55001587d0 2024-04-05T22:30:00.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.299+0000 7f54fd600640 1 -- 172.21.15.73:0/1322230809 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0031810 con 0x7f55001587d0 2024-04-05T22:30:00.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.299+0000 7f54fd600640 1 -- 172.21.15.73:0/1322230809 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0031ba0 con 0x7f55001587d0 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 -- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55001587d0 msgr2=0x7f55000cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55001587d0 0x7f55000cc7e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 -- 172.21.15.73:0/1322230809 shutdown_connections 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015e850 0x7f550015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 --2- 172.21.15.73:0/1322230809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55001587d0 0x7f55000cc7e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 -- 172.21.15.73:0/1322230809 >> 172.21.15.73:0/1322230809 conn(0x7f55000ce8e0 msgr2=0x7f55000cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 -- 172.21.15.73:0/1322230809 shutdown_connections 2024-04-05T22:30:00.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.300+0000 7f5505c00640 1 -- 172.21.15.73:0/1322230809 wait complete. 2024-04-05T22:30:00.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.302+0000 7f5505c00640 1 Processor -- start 2024-04-05T22:30:00.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.302+0000 7f5505c00640 1 -- start start 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 0x7f550015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 0x7f55000c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550015dda0 con 0x7f55001587d0 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550014bc10 con 0x7f550015e850 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.303+0000 7f5505c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55000c47b0 con 0x7f550015d920 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33960/0 (socket says 172.21.15.73:33960) 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 -- 172.21.15.73:0/298469953 learned_addr learned my addr 172.21.15.73:0/298469953 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54ff400640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 0x7f55000c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fe000640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 0x7f550015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 -- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 msgr2=0x7f55000c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 0x7f55000c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 -- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 msgr2=0x7f550015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 0x7f550015d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 -- 172.21.15.73:0/298469953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8009b30 con 0x7f55001587d0 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fe000640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 0x7f550015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54ff400640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 0x7f55000c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:00.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.304+0000 7f54fea00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:00.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.305+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0006e10 con 0x7f55001587d0 2024-04-05T22:30:00.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.305+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f003ad60 con 0x7f55001587d0 2024-04-05T22:30:00.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.305+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0044620 con 0x7f55001587d0 2024-04-05T22:30:00.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.305+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f0009400 con 0x7f55001587d0 2024-04-05T22:30:00.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.305+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55000c4d70 con 0x7f55001587d0 2024-04-05T22:30:00.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.306+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5500066af0 con 0x7f55001587d0 2024-04-05T22:30:00.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.307+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f54f004d440 con 0x7f55001587d0 2024-04-05T22:30:00.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.308+0000 7f54f6a00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 0x7f54e006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:00.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.308+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54f00ac230 con 0x7f55001587d0 2024-04-05T22:30:00.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.308+0000 7f54fe000640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 0x7f54e006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:00.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.310+0000 7f54fe000640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 0x7f54e006d6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f55000af270 tx=0x7f54ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:00.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.312+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f54f0076a10 con 0x7f55001587d0 2024-04-05T22:30:00.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.520+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5500068160 con 0x7f55001587d0 2024-04-05T22:30:00.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.521+0000 7f54f6a00640 1 -- 172.21.15.73:0/298469953 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7f54f007a030 con 0x7f55001587d0 2024-04-05T22:30:00.521 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 msgr2=0x7f54e006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 0x7f54e006d6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f55000af270 tx=0x7f54ec009040 comp rx=0 tx=0).stop 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 msgr2=0x7f550014b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f00046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 shutdown_connections 2024-04-05T22:30:00.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f54e006b230 0x7f54e006d6f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015d920 0x7f55000c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f550015e850 0x7f550015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 --2- 172.21.15.73:0/298469953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55001587d0 0x7f550014b6d0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 >> 172.21.15.73:0/298469953 conn(0x7f55000ce8e0 msgr2=0x7f55000cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 shutdown_connections 2024-04-05T22:30:00.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:00.524+0000 7f5505c00640 1 -- 172.21.15.73:0/298469953 wait complete. 2024-04-05T22:30:00.543 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:30:00.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:00.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:00.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:30:00.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:00.654 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:30:00.702 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:00.702 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:00.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:00.760 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:00.760 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:00.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:00.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.028+0000 7fd2fbe00640 1 Processor -- start 2024-04-05T22:30:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.029+0000 7fd2fbe00640 1 -- start start 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2f41587d0 0x7fd2f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f40ccd20 0x7fd2f406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 0x7fd2f415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f406f5b0 con 0x7fd2f41587d0 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f4070410 con 0x7fd2f406d0c0 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.030+0000 7fd2fbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4158bb0 con 0x7fd2f40ccd20 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2f41587d0 0x7fd2f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2f41587d0 0x7fd2f40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33964/0 (socket says 172.21.15.73:33964) 2024-04-05T22:30:01.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 -- 172.21.15.73:0/2412588499 learned_addr learned my addr 172.21.15.73:0/2412588499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9e00640 1 --2- 172.21.15.73:0/2412588499 >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 0x7fd2f415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 -- 172.21.15.73:0/2412588499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f40ccd20 msgr2=0x7fd2f406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 --2- 172.21.15.73:0/2412588499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f40ccd20 0x7fd2f406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 -- 172.21.15.73:0/2412588499 >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 msgr2=0x7fd2f415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 --2- 172.21.15.73:0/2412588499 >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 0x7fd2f415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9400640 1 -- 172.21.15.73:0/2412588499 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f406f800 con 0x7fd2f41587d0 2024-04-05T22:30:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.031+0000 7fd2f9e00640 1 --2- 172.21.15.73:0/2412588499 >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 0x7fd2f415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:01.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.032+0000 7fd2f9400640 1 --2- 172.21.15.73:0/2412588499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41587d0 0x7fd2f40cc7e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40ae4b0 tx=0x7fd2e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df0928822885073 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2f3e00640 1 -- 172.21.15.73:0/2412588499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8031870 con 0x7fd2f41587d0 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2f3e00640 1 -- 172.21.15.73:0/2412588499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80319d0 con 0x7fd2f41587d0 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2f3e00640 1 -- 172.21.15.73:0/2412588499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8031d60 con 0x7fd2f41587d0 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 -- 172.21.15.73:0/2412588499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41587d0 msgr2=0x7fd2f40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/2412588499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41587d0 0x7fd2f40cc7e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40ae4b0 tx=0x7fd2e802b910 comp rx=0 tx=0).stop 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 -- 172.21.15.73:0/2412588499 shutdown_connections 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/2412588499 >> v2:172.21.15.157:3300/0 conn(0x7fd2f406d0c0 0x7fd2f415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/2412588499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f40ccd20 0x7fd2f406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/2412588499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41587d0 0x7fd2f40cc7e0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 -- 172.21.15.73:0/2412588499 >> 172.21.15.73:0/2412588499 conn(0x7fd2f40ce8e0 msgr2=0x7fd2f40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 -- 172.21.15.73:0/2412588499 shutdown_connections 2024-04-05T22:30:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.033+0000 7fd2fbe00640 1 -- 172.21.15.73:0/2412588499 wait complete. 2024-04-05T22:30:01.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.035+0000 7fd2fbe00640 1 Processor -- start 2024-04-05T22:30:01.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.035+0000 7fd2fbe00640 1 -- start start 2024-04-05T22:30:01.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.036+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f406d0c0 0x7fd2f4157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.036+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f41587d0 0x7fd2f41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.036+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2fbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f41593b0 con 0x7fd2f41f2890 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2fbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4159d10 con 0x7fd2f41587d0 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2fbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4158d80 con 0x7fd2f406d0c0 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33976/0 (socket says 172.21.15.73:33976) 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 -- 172.21.15.73:0/3788509854 learned_addr learned my addr 172.21.15.73:0/3788509854 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f8a00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f41587d0 0x7fd2f41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9400640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f406d0c0 0x7fd2f4157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 -- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f406d0c0 msgr2=0x7fd2f4157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f406d0c0 0x7fd2f4157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 -- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f41587d0 msgr2=0x7fd2f41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f41587d0 0x7fd2f41597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.037+0000 7fd2f9e00640 1 -- 172.21.15.73:0/3788509854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80094d0 con 0x7fd2f41f2890 2024-04-05T22:30:01.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2f9e00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00cf20 tx=0x7fd2ec00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00d840 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec00d9a0 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00dd50 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f41f3350 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.038+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f4158fd0 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.039+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4066af0 con 0x7fd2f41f2890 2024-04-05T22:30:01.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.040+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fd2ec01e3b0 con 0x7fd2f41f2890 2024-04-05T22:30:01.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.041+0000 7fd2f1600640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 0x7fd2d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.041+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2ec08a920 con 0x7fd2f41f2890 2024-04-05T22:30:01.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.042+0000 7fd2f9400640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 0x7fd2d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.043+0000 7fd2f9400640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 0x7fd2d406d6f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8036000 tx=0x7fd2e803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.047+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd2ec055210 con 0x7fd2f41f2890 2024-04-05T22:30:01.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.278+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2f41594f0 con 0x7fd2f41f2890 2024-04-05T22:30:01.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.279+0000 7fd2f1600640 1 -- 172.21.15.73:0/3788509854 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fd2f41594f0 con 0x7fd2f41f2890 2024-04-05T22:30:01.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:01.280 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:30:01.280 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 83 2024-04-05T22:30:01.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 msgr2=0x7fd2d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 0x7fd2d406d6f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8036000 tx=0x7fd2e803c040 comp rx=0 tx=0).stop 2024-04-05T22:30:01.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 msgr2=0x7fd2f41f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00cf20 tx=0x7fd2ec00a7d0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 shutdown_connections 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2d406b230 0x7fd2d406d6f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2f41f2890 0x7fd2f41f2d40 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2f41587d0 0x7fd2f41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 --2- 172.21.15.73:0/3788509854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2f406d0c0 0x7fd2f4157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 >> 172.21.15.73:0/3788509854 conn(0x7fd2f40ce8e0 msgr2=0x7fd2f40ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 shutdown_connections 2024-04-05T22:30:01.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.282+0000 7fd2fbe00640 1 -- 172.21.15.73:0/3788509854 wait complete. 2024-04-05T22:30:01.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:01.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.515+0000 7f86f7000640 1 Processor -- start 2024-04-05T22:30:01.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.515+0000 7f86f7000640 1 -- start start 2024-04-05T22:30:01.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.516+0000 7f86f7000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86f013fa60 0x7f86f0142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.516+0000 7f86f7000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 0x7f86f01429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.516+0000 7f86f7000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0143330 con 0x7f86f013fe40 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.516+0000 7f86f7000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f006d3a0 con 0x7f86f013fa60 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.516+0000 7f86f7000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f0143050 con 0x7f86f006fe90 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86f013fa60 0x7f86f0142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f86f013fe40 0x7f86f006f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52886/0 (socket says 172.21.15.73:52886) 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5200640 1 -- 172.21.15.73:0/4099480562 learned_addr learned my addr 172.21.15.73:0/4099480562 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f6600640 1 --2- 172.21.15.73:0/4099480562 >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 0x7f86f01429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5c00640 1 -- 172.21.15.73:0/4099480562 >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 msgr2=0x7f86f01429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5c00640 1 --2- 172.21.15.73:0/4099480562 >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 0x7f86f01429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5c00640 1 -- 172.21.15.73:0/4099480562 >> v1:172.21.15.73:6789/0 conn(0x7f86f013fe40 legacy=0x7f86f006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f5c00640 1 -- 172.21.15.73:0/4099480562 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00ac970 con 0x7f86f013fa60 2024-04-05T22:30:01.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86f6600640 1 --2- 172.21.15.73:0/4099480562 >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 0x7f86f01429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:01.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.517+0000 7f86efe00640 1 -- 172.21.15.73:0/4099480562 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247469172 0 0) 0x7f86f0143330 con 0x7f86f013fe40 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.518+0000 7f86f5c00640 1 --2- 172.21.15.73:0/4099480562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f0142490 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f86f00677a0 tx=0x7f86e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac833c3405c0c12d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.518+0000 7f86efe00640 1 -- 172.21.15.73:0/4099480562 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4031830 con 0x7f86f013fa60 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86efe00640 1 -- 172.21.15.73:0/4099480562 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4031990 con 0x7f86f013fa60 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86efe00640 1 -- 172.21.15.73:0/4099480562 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4031d20 con 0x7f86f013fa60 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 -- 172.21.15.73:0/4099480562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 msgr2=0x7f86f0142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 --2- 172.21.15.73:0/4099480562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f0142490 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f86f00677a0 tx=0x7f86e402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 -- 172.21.15.73:0/4099480562 shutdown_connections 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 --2- 172.21.15.73:0/4099480562 >> v2:172.21.15.157:3300/0 conn(0x7f86f006fe90 0x7f86f01429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 --2- 172.21.15.73:0/4099480562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f0142490 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 -- 172.21.15.73:0/4099480562 >> 172.21.15.73:0/4099480562 conn(0x7f86f00b06c0 msgr2=0x7f86f00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 -- 172.21.15.73:0/4099480562 shutdown_connections 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.519+0000 7f86f7000640 1 -- 172.21.15.73:0/4099480562 wait complete. 2024-04-05T22:30:01.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.521+0000 7f86f7000640 1 Processor -- start 2024-04-05T22:30:01.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.521+0000 7f86f7000640 1 -- start start 2024-04-05T22:30:01.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f006fe90 0x7f86f015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f013fe40 0x7f86f013e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f6bd0 con 0x7f86f013fa60 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f6610 con 0x7f86f013fe40 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f68f0 con 0x7f86f006fe90 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33992/0 (socket says 172.21.15.73:33992) 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.522+0000 7f86f5200640 1 -- 172.21.15.73:0/3945879468 learned_addr learned my addr 172.21.15.73:0/3945879468 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:01.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5c00640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f006fe90 0x7f86f015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f6600640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f013fe40 0x7f86f013e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 -- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f006fe90 msgr2=0x7f86f015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f006fe90 0x7f86f015e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 -- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f013fe40 msgr2=0x7f86f013e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f013fe40 0x7f86f013e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 -- 172.21.15.73:0/3945879468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009400 con 0x7f86f013fa60 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f5200640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c410 tx=0x7f86e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00074e0 con 0x7f86f013fa60 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0007660 con 0x7f86f013fa60 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0004a20 con 0x7f86f013fa60 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.523+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f00cb470 con 0x7f86f013fa60 2024-04-05T22:30:01.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.524+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f015e840 con 0x7f86f013fa60 2024-04-05T22:30:01.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.525+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f0070bc0 con 0x7f86f013fa60 2024-04-05T22:30:01.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.526+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f86e0004b80 con 0x7f86f013fa60 2024-04-05T22:30:01.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.527+0000 7f86ed600640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 0x7f86d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.527+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86e0089c20 con 0x7f86f013fa60 2024-04-05T22:30:01.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.528+0000 7f86f5c00640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 0x7f86d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.529+0000 7f86f5c00640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 0x7f86d406d6f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f86e4036000 tx=0x7f86e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.532+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f86e0057fe0 con 0x7f86f013fa60 2024-04-05T22:30:01.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.730+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86f00cb990 con 0x7f86f013fa60 2024-04-05T22:30:01.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.733+0000 7f86ed600640 1 -- 172.21.15.73:0/3945879468 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f86f00cb990 con 0x7f86f013fa60 2024-04-05T22:30:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:01.734 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 msgr2=0x7f86d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 0x7f86d406d6f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f86e4036000 tx=0x7f86e4008030 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 msgr2=0x7f86f01f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c410 tx=0x7f86e000c8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 shutdown_connections 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f86d406b230 0x7f86d406d6f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f013fe40 0x7f86f013e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f013fa60 0x7f86f01f5c30 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 --2- 172.21.15.73:0/3945879468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f006fe90 0x7f86f015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 >> 172.21.15.73:0/3945879468 conn(0x7f86f00b06c0 msgr2=0x7f86f01607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.736+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 shutdown_connections 2024-04-05T22:30:01.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.737+0000 7f86f7000640 1 -- 172.21.15.73:0/3945879468 wait complete. 2024-04-05T22:30:01.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:29:25.591734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:29:26.571955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:01.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:01.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.963+0000 7fbfdb400640 1 Processor -- start 2024-04-05T22:30:01.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.963+0000 7fbfdb400640 1 -- start start 2024-04-05T22:30:01.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfd40adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfdb400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfd40ae0b0 0x7fbfd4162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfdb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40b2ff0 con 0x7fbfd40b2190 2024-04-05T22:30:01.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfdb400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd41597b0 con 0x7fbfd40ae0b0 2024-04-05T22:30:01.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfdb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41599a0 con 0x7fbfd406b530 2024-04-05T22:30:01.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfd8a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbfd40b2190 0x7fbfd40b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52900/0 (socket says 172.21.15.73:52900) 2024-04-05T22:30:01.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.965+0000 7fbfd8a00640 1 -- 172.21.15.73:0/3316538322 learned_addr learned my addr 172.21.15.73:0/3316538322 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053260656 0 0) 0x7fbfd40b2ff0 con 0x7fbfd40b2190 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfd40adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb4003520 con 0x7fbfd40b2190 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd9400640 1 --2- 172.21.15.73:0/3316538322 >> v2:172.21.15.157:3300/0 conn(0x7fbfd40ae0b0 0x7fbfd4162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061195261 0 0) 0x7fbfb4003520 con 0x7fbfd40b2190 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 msgr2=0x7fbfd40adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfd40adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 >> v2:172.21.15.157:3300/0 conn(0x7fbfd40ae0b0 msgr2=0x7fbfd4162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 --2- 172.21.15.73:0/3316538322 >> v2:172.21.15.157:3300/0 conn(0x7fbfd40ae0b0 0x7fbfd4162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4159c80 con 0x7fbfd40b2190 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbfc4001670 con 0x7fbfd40b2190 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 >> v1:172.21.15.73:6789/0 conn(0x7fbfd40b2190 legacy=0x7fbfd40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfb4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003fa0 0x7fbfb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4004d00 0x7fbfb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41599a0 con 0x7fbfb4003fa0 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb400bac0 con 0x7fbfb4004d00 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.966+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb400bc80 con 0x7fbfd406b530 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003fa0 0x7fbfb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd9400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4004d00 0x7fbfb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd8a00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfb4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 -- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 msgr2=0x7fbfb4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd406b530 0x7fbfb4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 -- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4004d00 msgr2=0x7fbfb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4004d00 0x7fbfb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.967+0000 7fbfd3e00640 1 -- 172.21.15.73:0/3316538322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4003520 con 0x7fbfb4003fa0 2024-04-05T22:30:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.968+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003fa0 0x7fbfb40047c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007330 tx=0x7fbfb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169d0594e346a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc02cca0 con 0x7fbfb4003fa0 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc02ce00 con 0x7fbfb4003fa0 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfd3400640 1 -- 172.21.15.73:0/3316538322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc034660 con 0x7fbfb4003fa0 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 -- 172.21.15.73:0/3316538322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003fa0 msgr2=0x7fbfb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 --2- 172.21.15.73:0/3316538322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003fa0 0x7fbfb40047c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007330 tx=0x7fbfb40057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfd9400640 1 -- 172.21.15.73:0/3316538322 reap_dead start 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 -- 172.21.15.73:0/3316538322 shutdown_connections 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 -- 172.21.15.73:0/3316538322 >> 172.21.15.73:0/3316538322 conn(0x7fbfd4069cb0 msgr2=0x7fbfd406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 -- 172.21.15.73:0/3316538322 shutdown_connections 2024-04-05T22:30:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.969+0000 7fbfdb400640 1 -- 172.21.15.73:0/3316538322 wait complete. 2024-04-05T22:30:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.971+0000 7fbfdb400640 1 Processor -- start 2024-04-05T22:30:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.971+0000 7fbfdb400640 1 -- start start 2024-04-05T22:30:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4003fa0 0x7fbfd413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd4144b90 0x7fbfd4144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4143870 con 0x7fbfd4143c90 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ad690 con 0x7fbfb4003fa0 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfdb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd413fe40 con 0x7fbfd4144b90 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfd9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfd9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34016/0 (socket says 172.21.15.73:34016) 2024-04-05T22:30:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.972+0000 7fbfd9400640 1 -- 172.21.15.73:0/2454077493 learned_addr learned my addr 172.21.15.73:0/2454077493 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:01.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd8a00640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4003fa0 0x7fbfd413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 -- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd4144b90 msgr2=0x7fbfd4144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd3e00640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd4144b90 0x7fbfd4144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd4144b90 0x7fbfd4144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 -- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4003fa0 msgr2=0x7fbfd413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4003fa0 0x7fbfd413f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 -- 172.21.15.73:0/2454077493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40cde70 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd9400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004440 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc003610 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40ce150 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0039c0 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.973+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4143490 con 0x7fbfd4143c90 2024-04-05T22:30:01.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.975+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4143650 con 0x7fbfd4143c90 2024-04-05T22:30:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.975+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fbfcc003b20 con 0x7fbfd4143c90 2024-04-05T22:30:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.976+0000 7fbfd0c00640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 0x7fbfb006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.976+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfcc089ca0 con 0x7fbfd4143c90 2024-04-05T22:30:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.977+0000 7fbfd8a00640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 0x7fbfb006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.978+0000 7fbfd8a00640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 0x7fbfb006d6f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001050 tx=0x7fbfc4001d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:01.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:01.981+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbfcc054470 con 0x7fbfd4143c90 2024-04-05T22:30:02.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.172+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfd41439b0 con 0x7fbfd4143c90 2024-04-05T22:30:02.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.173+0000 7fbfd0c00640 1 -- 172.21.15.73:0/2454077493 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfd41439b0 con 0x7fbfd4143c90 2024-04-05T22:30:02.173 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.175+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 msgr2=0x7fbfb006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.175+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 0x7fbfb006d6f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001050 tx=0x7fbfc4001d70 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.175+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 msgr2=0x7fbfd4144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.175+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc00f8a0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 shutdown_connections 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfb006b230 0x7fbfb006d6f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd4143c90 0x7fbfd4144110 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd4144b90 0x7fbfd4144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 --2- 172.21.15.73:0/2454077493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb4003fa0 0x7fbfd413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 >> 172.21.15.73:0/2454077493 conn(0x7fbfd4069cb0 msgr2=0x7fbfd40af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 shutdown_connections 2024-04-05T22:30:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.176+0000 7fbfdb400640 1 -- 172.21.15.73:0/2454077493 wait complete. 2024-04-05T22:30:02.194 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:30:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.403+0000 7fac12600640 1 Processor -- start 2024-04-05T22:30:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.403+0000 7fac12600640 1 -- start start 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.404+0000 7fac12600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fac0c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.404+0000 7fac12600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac0c06bc40 0x7fac0c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac12600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c15afc0 con 0x7fac0c06c9a0 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac12600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0c06f5b0 con 0x7fac0c06bc40 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac12600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06f890 con 0x7fac0c15abe0 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fac0c06c9a0 0x7fac0c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52908/0 (socket says 172.21.15.73:52908) 2024-04-05T22:30:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0aa00640 1 -- 172.21.15.73:0/724279784 learned_addr learned my addr 172.21.15.73:0/724279784 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0b400640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fac0c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634797840 0 0) 0x7fac0c15afc0 con 0x7fac0c06c9a0 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf0003520 con 0x7fac0c06c9a0 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.405+0000 7fac0be00640 1 --2- 172.21.15.73:0/724279784 >> v2:172.21.15.157:3300/0 conn(0x7fac0c06bc40 0x7fac0c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840456667 0 0) 0x7fabf0003520 con 0x7fac0c06c9a0 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 msgr2=0x7fac0c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fac0c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 >> v2:172.21.15.157:3300/0 conn(0x7fac0c06bc40 msgr2=0x7fac0c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 --2- 172.21.15.73:0/724279784 >> v2:172.21.15.157:3300/0 conn(0x7fac0c06bc40 0x7fac0c164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c06fb70 con 0x7fac0c06c9a0 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fabf8001670 con 0x7fac0c06c9a0 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 >> v1:172.21.15.73:6789/0 conn(0x7fac0c06c9a0 legacy=0x7fac0c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fabf0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fabf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf0004d00 0x7fabf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06f5b0 con 0x7fabf0003ef0 2024-04-05T22:30:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06f890 con 0x7fabf0004d00 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf000bb10 con 0x7fac0c15abe0 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0be00640 1 --2- 172.21.15.73:0/724279784 >> v2:172.21.15.157:3300/0 conn(0x7fac0c06bc40 0x7fac0c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0aa00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fabf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.406+0000 7fac0b400640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fabf0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0aa00640 1 -- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 msgr2=0x7fabf0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0aa00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c15abe0 0x7fabf0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0aa00640 1 -- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf0004d00 msgr2=0x7fabf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0be00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf0004d00 0x7fabf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0aa00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf0004d00 0x7fabf0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0aa00640 1 -- 172.21.15.73:0/724279784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0003520 con 0x7fabf0003ef0 2024-04-05T22:30:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.407+0000 7fac0be00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf0004d00 0x7fabf0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac0aa00640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fabf00047c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fabf800c210 tx=0x7fabf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a0f05847868f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8005ad0 con 0x7fabf0003ef0 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf8005c30 con 0x7fabf0003ef0 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac0a000640 1 -- 172.21.15.73:0/724279784 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf802c8d0 con 0x7fabf0003ef0 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac12600640 1 -- 172.21.15.73:0/724279784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 msgr2=0x7fabf00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac12600640 1 --2- 172.21.15.73:0/724279784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fabf00047c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fabf800c210 tx=0x7fabf00057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac0be00640 1 -- 172.21.15.73:0/724279784 reap_dead start 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac12600640 1 -- 172.21.15.73:0/724279784 shutdown_connections 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.408+0000 7fac12600640 1 -- 172.21.15.73:0/724279784 >> 172.21.15.73:0/724279784 conn(0x7fac0c069cb0 msgr2=0x7fac0c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.409+0000 7fac12600640 1 -- 172.21.15.73:0/724279784 shutdown_connections 2024-04-05T22:30:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.409+0000 7fac12600640 1 -- 172.21.15.73:0/724279784 wait complete. 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.410+0000 7fac12600640 1 Processor -- start 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.411+0000 7fac12600640 1 -- start start 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c14e710 0x7fac0c1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c1f28e0 0x7fac0c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1510f0 con 0x7fabf0003ef0 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c14eb90 con 0x7fac0c14e710 2024-04-05T22:30:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac12600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1f2d60 con 0x7fac0c1f28e0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34024/0 (socket says 172.21.15.73:34024) 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 -- 172.21.15.73:0/80441366 learned_addr learned my addr 172.21.15.73:0/80441366 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 -- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c1f28e0 msgr2=0x7fac0c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c1f28e0 0x7fac0c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 -- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c14e710 msgr2=0x7fac0c1f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c14e710 0x7fac0c1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 -- 172.21.15.73:0/80441366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c1f3040 con 0x7fabf0003ef0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.412+0000 7fac0b400640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fabfc010860 tx=0x7fabfc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.413+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc009610 con 0x7fabf0003ef0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.413+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00eae0 con 0x7fabf0003ef0 2024-04-05T22:30:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.413+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c1f3320 con 0x7fabf0003ef0 2024-04-05T22:30:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.413+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00ee70 con 0x7fabf0003ef0 2024-04-05T22:30:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.413+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c0ce380 con 0x7fabf0003ef0 2024-04-05T22:30:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.414+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0c068160 con 0x7fabf0003ef0 2024-04-05T22:30:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.415+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fabfc01c400 con 0x7fabf0003ef0 2024-04-05T22:30:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.416+0000 7fac03400640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 0x7fabec06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.416+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabfc08a320 con 0x7fabf0003ef0 2024-04-05T22:30:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.416+0000 7fac0aa00640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 0x7fabec06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.417+0000 7fac0aa00640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 0x7fabec06d720 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fabf8034000 tx=0x7fabf802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.420+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fabfc054c70 con 0x7fabf0003ef0 2024-04-05T22:30:02.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.601+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac0c0b0240 con 0x7fabf0003ef0 2024-04-05T22:30:02.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.602+0000 7fac03400640 1 -- 172.21.15.73:0/80441366 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabfc058290 con 0x7fabf0003ef0 2024-04-05T22:30:02.602 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 msgr2=0x7fabec06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 0x7fabec06d720 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fabf8034000 tx=0x7fabf802e000 comp rx=0 tx=0).stop 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 msgr2=0x7fac0c153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fabfc010860 tx=0x7fabfc010d20 comp rx=0 tx=0).stop 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 shutdown_connections 2024-04-05T22:30:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fabec06b260 0x7fabec06d720 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac0c1f28e0 0x7fac0c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c14e710 0x7fac0c1f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 --2- 172.21.15.73:0/80441366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf0003ef0 0x7fac0c153ec0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 >> 172.21.15.73:0/80441366 conn(0x7fac0c069cb0 msgr2=0x7fac0c0aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 shutdown_connections 2024-04-05T22:30:02.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.605+0000 7fac12600640 1 -- 172.21.15.73:0/80441366 wait complete. 2024-04-05T22:30:02.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:30:02.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.828+0000 7f8a67a00640 1 Processor -- start 2024-04-05T22:30:02.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.828+0000 7f8a67a00640 1 -- start start 2024-04-05T22:30:02.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.829+0000 7f8a67a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a600cbe00 0x7f8a600ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.829+0000 7f8a67a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a601647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.829+0000 7f8a67a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600b1b30 con 0x7f8a6006c630 2024-04-05T22:30:02.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.829+0000 7f8a67a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a600b1e10 con 0x7f8a600cbe00 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.829+0000 7f8a67a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600cc300 con 0x7f8a600b32b0 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a600cbe00 0x7f8a600ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a600cbe00 0x7f8a600ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34038/0 (socket says 172.21.15.73:34038) 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 -- 172.21.15.73:0/2699180334 learned_addr learned my addr 172.21.15.73:0/2699180334 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a65a00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a601647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 -- 172.21.15.73:0/2699180334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 msgr2=0x7f8a601647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a601647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 -- 172.21.15.73:0/2699180334 >> v1:172.21.15.157:6789/0 conn(0x7f8a6006c630 legacy=0x7f8a6006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:02.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.830+0000 7f8a57e00640 1 -- 172.21.15.73:0/2699180334 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600cc5e0 con 0x7f8a600cbe00 2024-04-05T22:30:02.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.831+0000 7f8a57e00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a600ce0a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009870 tx=0x7f8a4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34284f233426c390 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.831+0000 7f8a5fe00640 1 -- 172.21.15.73:0/2699180334 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48031640 con 0x7f8a600cbe00 2024-04-05T22:30:02.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.831+0000 7f8a5fe00640 1 -- 172.21.15.73:0/2699180334 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a480317a0 con 0x7f8a600cbe00 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.831+0000 7f8a5fe00640 1 -- 172.21.15.73:0/2699180334 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48031b30 con 0x7f8a600cbe00 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 -- 172.21.15.73:0/2699180334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 msgr2=0x7f8a600ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a600ce0a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009870 tx=0x7f8a4802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 -- 172.21.15.73:0/2699180334 shutdown_connections 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a601647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 --2- 172.21.15.73:0/2699180334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a600ce0a0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 -- 172.21.15.73:0/2699180334 >> 172.21.15.73:0/2699180334 conn(0x7f8a600b0830 msgr2=0x7f8a600b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:02.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 -- 172.21.15.73:0/2699180334 shutdown_connections 2024-04-05T22:30:02.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.832+0000 7f8a67a00640 1 -- 172.21.15.73:0/2699180334 wait complete. 2024-04-05T22:30:02.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.834+0000 7f8a67a00640 1 Processor -- start 2024-04-05T22:30:02.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.835+0000 7f8a67a00640 1 -- start start 2024-04-05T22:30:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a6006c630 0x7f8a6015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a60070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6015dc10 con 0x7f8a600cbe00 2024-04-05T22:30:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6015aa90 con 0x7f8a6006c630 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a67a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60071680 con 0x7f8a600b32b0 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34050/0 (socket says 172.21.15.73:34050) 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 -- 172.21.15.73:0/1337686585 learned_addr learned my addr 172.21.15.73:0/1337686585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a57e00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a60070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65000640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a6006c630 0x7f8a6015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 -- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 msgr2=0x7f8a60070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a60070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 -- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a6006c630 msgr2=0x7f8a6015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a6006c630 0x7f8a6015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a65a00640 1 -- 172.21.15.73:0/1337686585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480094d0 con 0x7f8a600cbe00 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.836+0000 7f8a57e00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a60070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a65a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007e30 tx=0x7f8a5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58014620 con 0x7f8a600cbe00 2024-04-05T22:30:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5800ee90 con 0x7f8a600cbe00 2024-04-05T22:30:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580149c0 con 0x7f8a600cbe00 2024-04-05T22:30:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6015d2e0 con 0x7f8a600cbe00 2024-04-05T22:30:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.837+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6015d8a0 con 0x7f8a600cbe00 2024-04-05T22:30:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.838+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a60068160 con 0x7f8a600cbe00 2024-04-05T22:30:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.839+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f8a58014b20 con 0x7f8a600cbe00 2024-04-05T22:30:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.840+0000 7f8a5d600640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 0x7f8a3c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.840+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a5808a8b0 con 0x7f8a600cbe00 2024-04-05T22:30:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.840+0000 7f8a65000640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 0x7f8a3c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:02.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.842+0000 7f8a65000640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 0x7f8a3c06d6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a60071960 tx=0x7f8a50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:02.844+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8a58055150 con 0x7f8a600cbe00 2024-04-05T22:30:03.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.035+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a600cad20 con 0x7f8a600cbe00 2024-04-05T22:30:03.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.036+0000 7f8a5d600640 1 -- 172.21.15.73:0/1337686585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a58058770 con 0x7f8a600cbe00 2024-04-05T22:30:03.036 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.038+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 msgr2=0x7f8a3c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.038+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 0x7f8a3c06d6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a60071960 tx=0x7f8a50009040 comp rx=0 tx=0).stop 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.038+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 msgr2=0x7f8a60159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.038+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007e30 tx=0x7f8a5800e4b0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 shutdown_connections 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8a3c06b230 0x7f8a3c06d6f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a600cbe00 0x7f8a60159af0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a600b32b0 0x7f8a60070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 --2- 172.21.15.73:0/1337686585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a6006c630 0x7f8a6015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 >> 172.21.15.73:0/1337686585 conn(0x7f8a600b0830 msgr2=0x7f8a600b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:03.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 shutdown_connections 2024-04-05T22:30:03.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.039+0000 7f8a67a00640 1 -- 172.21.15.73:0/1337686585 wait complete. 2024-04-05T22:30:03.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:03.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.256+0000 7fb229600640 1 Processor -- start 2024-04-05T22:30:03.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.256+0000 7fb229600640 1 -- start start 2024-04-05T22:30:03.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.257+0000 7fb229600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 0x7fb22406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.257+0000 7fb229600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 0x7fb22415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.258+0000 7fb229600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22406f5b0 con 0x7fb2241587d0 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.258+0000 7fb229600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22406d500 con 0x7fb22406cb80 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.258+0000 7fb229600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22415f220 con 0x7fb22415e850 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.258+0000 7fb21a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 0x7fb22406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.258+0000 7fb223400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 0x7fb22415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb21a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 0x7fb22406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57580/0 (socket says 172.21.15.73:57580) 2024-04-05T22:30:03.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb21a000640 1 -- 172.21.15.73:0/1182566213 learned_addr learned my addr 172.21.15.73:0/1182566213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:03.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb223400640 1 -- 172.21.15.73:0/1182566213 >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 msgr2=0x7fb22406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:03.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb223400640 1 --2- 172.21.15.73:0/1182566213 >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 0x7fb22406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb223400640 1 -- 172.21.15.73:0/1182566213 >> v1:172.21.15.157:6789/0 conn(0x7fb2241587d0 legacy=0x7fb2240cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.259+0000 7fb223400640 1 -- 172.21.15.73:0/1182566213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224070410 con 0x7fb22415e850 2024-04-05T22:30:03.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.261+0000 7fb223400640 1 --2- 172.21.15.73:0/1182566213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 0x7fb22415ece0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009d90 tx=0x7fb21c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02597de4797d5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.261+0000 7fb222000640 1 -- 172.21.15.73:0/1182566213 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c02cc50 con 0x7fb22415e850 2024-04-05T22:30:03.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb222000640 1 -- 172.21.15.73:0/1182566213 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c02cdb0 con 0x7fb22415e850 2024-04-05T22:30:03.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb222000640 1 -- 172.21.15.73:0/1182566213 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c034660 con 0x7fb22415e850 2024-04-05T22:30:03.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 -- 172.21.15.73:0/1182566213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 msgr2=0x7fb22415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 --2- 172.21.15.73:0/1182566213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 0x7fb22415ece0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009d90 tx=0x7fb21c02bea0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 -- 172.21.15.73:0/1182566213 shutdown_connections 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 --2- 172.21.15.73:0/1182566213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb22415e850 0x7fb22415ece0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 --2- 172.21.15.73:0/1182566213 >> v2:172.21.15.157:3300/0 conn(0x7fb22406cb80 0x7fb22406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 -- 172.21.15.73:0/1182566213 >> 172.21.15.73:0/1182566213 conn(0x7fb2240ce8e0 msgr2=0x7fb2240cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 -- 172.21.15.73:0/1182566213 shutdown_connections 2024-04-05T22:30:03.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.262+0000 7fb229600640 1 -- 172.21.15.73:0/1182566213 wait complete. 2024-04-05T22:30:03.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.264+0000 7fb229600640 1 Processor -- start 2024-04-05T22:30:03.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.264+0000 7fb229600640 1 -- start start 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22406cb80 0x7fb224143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2241587d0 0x7fb2240c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c4240 con 0x7fb22415e850 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c4520 con 0x7fb22406cb80 2024-04-05T22:30:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.265+0000 7fb229600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c4800 con 0x7fb2241587d0 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34054/0 (socket says 172.21.15.73:34054) 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 -- 172.21.15.73:0/2365642303 learned_addr learned my addr 172.21.15.73:0/2365642303 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb222a00640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22406cb80 0x7fb224143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb21a000640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2241587d0 0x7fb2240c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 -- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2241587d0 msgr2=0x7fb2240c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2241587d0 0x7fb2240c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 -- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22406cb80 msgr2=0x7fb224143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22406cb80 0x7fb224143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 -- 172.21.15.73:0/2365642303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009a70 con 0x7fb22415e850 2024-04-05T22:30:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.266+0000 7fb223400640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb21c02c4b0 tx=0x7fb21c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.267+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c03e610 con 0x7fb22415e850 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.267+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c03e770 con 0x7fb22415e850 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.267+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c03eb20 con 0x7fb22415e850 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.267+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2240c4ae0 con 0x7fb22415e850 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.267+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240cbc90 con 0x7fb22415e850 2024-04-05T22:30:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.268+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224066af0 con 0x7fb22415e850 2024-04-05T22:30:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.269+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fb21c03b050 con 0x7fb22415e850 2024-04-05T22:30:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.270+0000 7fb21b400640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 0x7fb20006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.270+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb21c0ac800 con 0x7fb22415e850 2024-04-05T22:30:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.270+0000 7fb222a00640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 0x7fb20006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.272+0000 7fb222a00640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 0x7fb20006d6f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb214001640 tx=0x7fb214009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.275+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb21c077180 con 0x7fb22415e850 2024-04-05T22:30:03.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.508+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb224068160 con 0x7fb22415e850 2024-04-05T22:30:03.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.509+0000 7fb21b400640 1 -- 172.21.15.73:0/2365642303 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fb21c07a7a0 con 0x7fb22415e850 2024-04-05T22:30:03.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:03.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:30:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 83 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.511+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 msgr2=0x7fb20006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.511+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 0x7fb20006d6f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb214001640 tx=0x7fb214009040 comp rx=0 tx=0).stop 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.511+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 msgr2=0x7fb2240c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.511+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb21c02c4b0 tx=0x7fb21c004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 shutdown_connections 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb20006b230 0x7fb20006d6f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415e850 0x7fb2240c37e0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2241587d0 0x7fb2240c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 --2- 172.21.15.73:0/2365642303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22406cb80 0x7fb224143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 >> 172.21.15.73:0/2365642303 conn(0x7fb2240ce8e0 msgr2=0x7fb22415a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:03.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 shutdown_connections 2024-04-05T22:30:03.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.512+0000 7fb229600640 1 -- 172.21.15.73:0/2365642303 wait complete. 2024-04-05T22:30:03.530 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:03.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.735+0000 7f330dc00640 1 Processor -- start 2024-04-05T22:30:03.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.735+0000 7f330dc00640 1 -- start start 2024-04-05T22:30:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.737+0000 7f330dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33080ccdc0 0x7f33080c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.737+0000 7f330dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33080c3700 0x7f33080c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.737+0000 7f330dc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33080ace90 con 0x7f330815cee0 2024-04-05T22:30:03.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.737+0000 7f330dc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330815d2c0 con 0x7f33080ccdc0 2024-04-05T22:30:03.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.737+0000 7f330dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33080ad170 con 0x7f33080c3700 2024-04-05T22:30:03.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33080ccdc0 0x7f33080c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33080ccdc0 0x7f33080c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34064/0 (socket says 172.21.15.73:34064) 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 -- 172.21.15.73:0/2727240234 learned_addr learned my addr 172.21.15.73:0/2727240234 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3307400640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33080c3700 0x7f33080c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 -- 172.21.15.73:0/2727240234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33080c3700 msgr2=0x7f33080c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33080c3700 0x7f33080c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 -- 172.21.15.73:0/2727240234 >> v1:172.21.15.157:6789/0 conn(0x7f330815cee0 legacy=0x7f33080cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.738+0000 7f3306000640 1 -- 172.21.15.73:0/2727240234 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080ad450 con 0x7f33080ccdc0 2024-04-05T22:30:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.739+0000 7f3306000640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f33080c9460 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdbed0197ddb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.740+0000 7f3305600640 1 -- 172.21.15.73:0/2727240234 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4030840 con 0x7f33080ccdc0 2024-04-05T22:30:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.740+0000 7f3305600640 1 -- 172.21.15.73:0/2727240234 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f40309a0 con 0x7f33080ccdc0 2024-04-05T22:30:03.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.740+0000 7f3305600640 1 -- 172.21.15.73:0/2727240234 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4030d30 con 0x7f33080ccdc0 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.740+0000 7f330dc00640 1 -- 172.21.15.73:0/2727240234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 msgr2=0x7f33080c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.740+0000 7f330dc00640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f33080c9460 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b980 comp rx=0 tx=0).stop 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 -- 172.21.15.73:0/2727240234 shutdown_connections 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33080c3700 0x7f33080c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 --2- 172.21.15.73:0/2727240234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f33080c9460 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 -- 172.21.15.73:0/2727240234 >> 172.21.15.73:0/2727240234 conn(0x7f33080b0490 msgr2=0x7f33080b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 -- 172.21.15.73:0/2727240234 shutdown_connections 2024-04-05T22:30:03.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.741+0000 7f330dc00640 1 -- 172.21.15.73:0/2727240234 wait complete. 2024-04-05T22:30:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.742+0000 7f330dc00640 1 Processor -- start 2024-04-05T22:30:03.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.742+0000 7f330dc00640 1 -- start start 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33080c3700 0x7f3308151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f330815cee0 0x7f33081450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308153320 con 0x7f33080ccdc0 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330814fcb0 con 0x7f33080c3700 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.743+0000 7f330dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33081517e0 con 0x7f330815cee0 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34076/0 (socket says 172.21.15.73:34076) 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 -- 172.21.15.73:0/3061432423 learned_addr learned my addr 172.21.15.73:0/3061432423 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:03.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3307400640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f330815cee0 0x7f33081450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306a00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33080c3700 0x7f3308151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 -- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f330815cee0 msgr2=0x7f33081450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f330815cee0 0x7f33081450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 -- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33080c3700 msgr2=0x7f3308151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33080c3700 0x7f3308151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.744+0000 7f3306000640 1 -- 172.21.15.73:0/3061432423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f40094d0 con 0x7f33080ccdc0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f3306000640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f32f4000c00 tx=0x7f32f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4039c90 con 0x7f33080ccdc0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4039df0 con 0x7f33080ccdc0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4043640 con 0x7f33080ccdc0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33081514d0 con 0x7f33080ccdc0 2024-04-05T22:30:03.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.745+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330806fda0 con 0x7f33080ccdc0 2024-04-05T22:30:03.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.746+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33080ca880 con 0x7f33080ccdc0 2024-04-05T22:30:03.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.747+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f32f40437a0 con 0x7f33080ccdc0 2024-04-05T22:30:03.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.748+0000 7f32fea00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 0x7f32e806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:03.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.748+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32f403e070 con 0x7f33080ccdc0 2024-04-05T22:30:03.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.749+0000 7f3306a00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 0x7f32e806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:03.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.750+0000 7f3306a00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 0x7f32e806d6f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007fd0 tx=0x7f32f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.752+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f32f4077720 con 0x7f33080ccdc0 2024-04-05T22:30:03.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.936+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3308068160 con 0x7f33080ccdc0 2024-04-05T22:30:03.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.939+0000 7f32fea00640 1 -- 172.21.15.73:0/3061432423 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f32f407ad40 con 0x7f33080ccdc0 2024-04-05T22:30:03.940 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:03.940 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.941+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 msgr2=0x7f32e806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.941+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 0x7f32e806d6f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007fd0 tx=0x7f32f8009040 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.941+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 msgr2=0x7f3308152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.941+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f32f4000c00 tx=0x7f32f4004730 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 shutdown_connections 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f32e806b230 0x7f32e806d6f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f330815cee0 0x7f33081450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33080ccdc0 0x7f3308152de0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 --2- 172.21.15.73:0/3061432423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33080c3700 0x7f3308151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 >> 172.21.15.73:0/3061432423 conn(0x7f33080b0490 msgr2=0x7f3308160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 shutdown_connections 2024-04-05T22:30:03.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:03.942+0000 7f330dc00640 1 -- 172.21.15.73:0/3061432423 wait complete. 2024-04-05T22:30:03.961 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:30:03.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.166+0000 7fda01000640 1 Processor -- start 2024-04-05T22:30:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.166+0000 7fda01000640 1 -- start start 2024-04-05T22:30:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fda01000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9fc158860 0x7fd9fc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fda01000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc06bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fda01000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc06b500 con 0x7fd9fc0b1f00 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fda01000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc0ccaf0 con 0x7fd9fc158860 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fda01000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0b23a0 con 0x7fd9fc15ed30 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fd9fb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9fc158860 0x7fd9fc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fd9f3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd9fc0b1f00 0x7fd9fc15e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52920/0 (socket says 172.21.15.73:52920) 2024-04-05T22:30:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.167+0000 7fd9f3e00640 1 -- 172.21.15.73:0/2507433309 learned_addr learned my addr 172.21.15.73:0/2507433309 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:04.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.168+0000 7fd9fb400640 1 -- 172.21.15.73:0/2507433309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 msgr2=0x7fd9fc06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:04.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.168+0000 7fd9fb400640 1 --2- 172.21.15.73:0/2507433309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc06bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.168+0000 7fd9fb400640 1 -- 172.21.15.73:0/2507433309 >> v1:172.21.15.73:6789/0 conn(0x7fd9fc0b1f00 legacy=0x7fd9fc15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.168+0000 7fd9faa00640 1 -- 172.21.15.73:0/2507433309 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148028727 0 0) 0x7fd9fc06b500 con 0x7fd9fc0b1f00 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.168+0000 7fd9fb400640 1 -- 172.21.15.73:0/2507433309 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc0b2680 con 0x7fd9fc158860 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.169+0000 7fd9fb400640 1 --2- 172.21.15.73:0/2507433309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc158860 0x7fd9fc0b19c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6ca5308df2f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.169+0000 7fd9faa00640 1 -- 172.21.15.73:0/2507433309 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec02e070 con 0x7fd9fc158860 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.169+0000 7fd9faa00640 1 -- 172.21.15.73:0/2507433309 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0315a0 con 0x7fd9fc158860 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.169+0000 7fd9faa00640 1 -- 172.21.15.73:0/2507433309 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec031930 con 0x7fd9fc158860 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 -- 172.21.15.73:0/2507433309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc158860 msgr2=0x7fd9fc0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 --2- 172.21.15.73:0/2507433309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc158860 0x7fd9fc0b19c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b840 comp rx=0 tx=0).stop 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 -- 172.21.15.73:0/2507433309 shutdown_connections 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 --2- 172.21.15.73:0/2507433309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc06bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 --2- 172.21.15.73:0/2507433309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc158860 0x7fd9fc0b19c0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 -- 172.21.15.73:0/2507433309 >> 172.21.15.73:0/2507433309 conn(0x7fd9fc0b06c0 msgr2=0x7fd9fc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 -- 172.21.15.73:0/2507433309 shutdown_connections 2024-04-05T22:30:04.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.170+0000 7fda01000640 1 -- 172.21.15.73:0/2507433309 wait complete. 2024-04-05T22:30:04.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.172+0000 7fda01000640 1 Processor -- start 2024-04-05T22:30:04.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.172+0000 7fda01000640 1 -- start start 2024-04-05T22:30:04.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 0x7fd9fc1efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0adec0 con 0x7fd9fc1f2680 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1eff60 con 0x7fd9fc0b1f00 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fda01000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0ad2b0 con 0x7fd9fc15ed30 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34090/0 (socket says 172.21.15.73:34090) 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9fbe00640 1 -- 172.21.15.73:0/3808455062 learned_addr learned my addr 172.21.15.73:0/3808455062 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9f3e00640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9fb400640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 0x7fd9fc1efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.173+0000 7fd9fbe00640 1 -- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 msgr2=0x7fd9fc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fbe00640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fbe00640 1 -- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 msgr2=0x7fd9fc1efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fbe00640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 0x7fd9fc1efa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fbe00640 1 -- 172.21.15.73:0/3808455062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec009400 con 0x7fd9fc1f2680 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9f3e00640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fb400640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 0x7fd9fc1efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9fbe00640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b9a0 tx=0x7fd9f400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400cba0 con 0x7fd9fc1f2680 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400cd20 con 0x7fd9fc1f2680 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4014680 con 0x7fd9fc1f2680 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0ad590 con 0x7fd9fc1f2680 2024-04-05T22:30:04.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.174+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc0adb50 con 0x7fd9fc1f2680 2024-04-05T22:30:04.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.175+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc06d910 con 0x7fd9fc1f2680 2024-04-05T22:30:04.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.176+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fd9f401d3b0 con 0x7fd9fc1f2680 2024-04-05T22:30:04.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.177+0000 7fd9f3400640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 0x7fd9d806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.177+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9f4089e40 con 0x7fd9fc1f2680 2024-04-05T22:30:04.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.178+0000 7fd9fb400640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 0x7fd9d806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.179+0000 7fd9fb400640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 0x7fd9d806d6f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec036000 tx=0x7fd9ec03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.182+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd9f40546a0 con 0x7fd9fc1f2680 2024-04-05T22:30:04.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.414+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9fc04e160 con 0x7fd9fc1f2680 2024-04-05T22:30:04.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.415+0000 7fd9f3400640 1 -- 172.21.15.73:0/3808455062 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fd9f4019020 con 0x7fd9fc1f2680 2024-04-05T22:30:04.415 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:04.415 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:30:04.416 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 83 2024-04-05T22:30:04.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 msgr2=0x7fd9d806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 0x7fd9d806d6f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec036000 tx=0x7fd9ec03d040 comp rx=0 tx=0).stop 2024-04-05T22:30:04.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 msgr2=0x7fd9fc152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b9a0 tx=0x7fd9f400be60 comp rx=0 tx=0).stop 2024-04-05T22:30:04.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 shutdown_connections 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd9d806b230 0x7fd9d806d6f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9fc1f2680 0x7fd9fc152f10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9fc15ed30 0x7fd9fc0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 --2- 172.21.15.73:0/3808455062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0b1f00 0x7fd9fc1efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 >> 172.21.15.73:0/3808455062 conn(0x7fd9fc0b06c0 msgr2=0x7fd9fc072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 shutdown_connections 2024-04-05T22:30:04.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.418+0000 7fda01000640 1 -- 172.21.15.73:0/3808455062 wait complete. 2024-04-05T22:30:04.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.647+0000 7f7b07e00640 1 Processor -- start 2024-04-05T22:30:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.647+0000 7f7b07e00640 1 -- start start 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.648+0000 7f7b07e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b001541a0 0x7f7b000b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b07e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0015aa60 con 0x7f7b0006d710 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b07e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b000b3b60 con 0x7f7b000ce000 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b07e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000701c0 con 0x7f7b001541a0 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b04a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7b0006d710 0x7f7b000cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52924/0 (socket says 172.21.15.73:52924) 2024-04-05T22:30:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b04a00640 1 -- 172.21.15.73:0/1414772958 learned_addr learned my addr 172.21.15.73:0/1414772958 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:04.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148235609 0 0) 0x7f7b0015aa60 con 0x7f7b0006d710 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003520 con 0x7f7b0006d710 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.649+0000 7f7b05400640 1 --2- 172.21.15.73:0/1414772958 >> v2:172.21.15.157:3300/0 conn(0x7f7b001541a0 0x7f7b000b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3832850002 0 0) 0x7f7ae8003520 con 0x7f7b0006d710 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 >> v2:172.21.15.157:3300/0 conn(0x7f7b001541a0 msgr2=0x7f7b000b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 --2- 172.21.15.73:0/1414772958 >> v2:172.21.15.157:3300/0 conn(0x7f7b001541a0 0x7f7b000b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 >> v1:172.21.15.157:6789/0 conn(0x7f7b000ce000 legacy=0x7f7b0015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000b3880 con 0x7f7b0006d710 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7aec001670 con 0x7f7b0006d710 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 >> v1:172.21.15.73:6789/0 conn(0x7f7b0006d710 legacy=0x7f7b000cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 0x7f7ae8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae8005180 0x7f7ae800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000701c0 con 0x7f7ae80047c0 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800c120 con 0x7f7ae8003970 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800c2e0 con 0x7f7ae8005180 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.650+0000 7f7b04a00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 0x7f7ae8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b05e00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae8005180 0x7f7ae800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b04a00640 1 -- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae8005180 msgr2=0x7f7ae800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b04a00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae8005180 0x7f7ae800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b04a00640 1 -- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003970 msgr2=0x7f7ae8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b04a00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b04a00640 1 -- 172.21.15.73:0/1414772958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8003520 con 0x7f7ae80047c0 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.651+0000 7f7b05e00640 1 -- 172.21.15.73:0/1414772958 reap_dead start 2024-04-05T22:30:04.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.652+0000 7f7b04a00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 0x7f7ae8004c40 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c190 tx=0x7f7ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ed0b3aa34d3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.652+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005ad0 con 0x7f7ae80047c0 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.652+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec005c50 con 0x7f7ae80047c0 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.652+0000 7f7affe00640 1 -- 172.21.15.73:0/1414772958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02c8a0 con 0x7f7ae80047c0 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 -- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 msgr2=0x7f7ae8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 0x7f7ae8004c40 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c190 tx=0x7f7ae80057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 -- 172.21.15.73:0/1414772958 shutdown_connections 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 --2- 172.21.15.73:0/1414772958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae80047c0 0x7f7ae8004c40 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 -- 172.21.15.73:0/1414772958 >> 172.21.15.73:0/1414772958 conn(0x7f7b000ce8e0 msgr2=0x7f7b000cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 -- 172.21.15.73:0/1414772958 shutdown_connections 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.653+0000 7f7b07e00640 1 -- 172.21.15.73:0/1414772958 wait complete. 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.655+0000 7f7b07e00640 1 Processor -- start 2024-04-05T22:30:04.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.655+0000 7f7b07e00640 1 -- start start 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae80047c0 0x7f7b00151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3350 0x7f7b000af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00152050 con 0x7f7b001f2760 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00152ac0 con 0x7f7ae80047c0 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b07e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00156be0 con 0x7f7b000b3350 2024-04-05T22:30:04.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34110/0 (socket says 172.21.15.73:34110) 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 -- 172.21.15.73:0/491297613 learned_addr learned my addr 172.21.15.73:0/491297613 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 -- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3350 msgr2=0x7f7b000af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3350 0x7f7b000af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 -- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae80047c0 msgr2=0x7f7b00151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b05400640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae80047c0 0x7f7b00151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae80047c0 0x7f7b00151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.656+0000 7f7b04a00640 1 -- 172.21.15.73:0/491297613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00156ec0 con 0x7f7b001f2760 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7b04a00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c190 tx=0x7f7aec02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec03c040 con 0x7f7b001f2760 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec02cd70 con 0x7f7b001f2760 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec007290 con 0x7f7b001f2760 2024-04-05T22:30:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b001571a0 con 0x7f7b001f2760 2024-04-05T22:30:04.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.657+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b00157760 con 0x7f7b001f2760 2024-04-05T22:30:04.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.658+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00066af0 con 0x7f7b001f2760 2024-04-05T22:30:04.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.659+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7aec032070 con 0x7f7b001f2760 2024-04-05T22:30:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.660+0000 7f7afd600640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 0x7f7ae006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.660+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aec0ab340 con 0x7f7b001f2760 2024-04-05T22:30:04.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.660+0000 7f7b05400640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 0x7f7ae006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:04.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.662+0000 7f7b05400640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 0x7f7ae006d6f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b00159b00 tx=0x7f7af4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:04.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.664+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7aec075c70 con 0x7f7b001f2760 2024-04-05T22:30:04.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.897+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b00068160 con 0x7f7b001f2760 2024-04-05T22:30:04.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.898+0000 7f7afd600640 1 -- 172.21.15.73:0/491297613 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f7aec079290 con 0x7f7b001f2760 2024-04-05T22:30:04.898 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:04.898 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:29:27.554050+0000","modified":"2024-04-05T22:29:30.583853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8685},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8685":{"gid":8685,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/1637066621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1637066621},{"type":"v1","addr":"172.21.15.73:6835","nonce":1637066621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-05T22:30:04.898 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 83 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.900+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 msgr2=0x7f7ae006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.900+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 0x7f7ae006d6f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b00159b00 tx=0x7f7af4008040 comp rx=0 tx=0).stop 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.900+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 msgr2=0x7f7b000b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.900+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c190 tx=0x7f7aec02e070 comp rx=0 tx=0).stop 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 shutdown_connections 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b230 0x7f7ae006d6f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3350 0x7f7b000af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b001f2760 0x7f7b000b2e10 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 --2- 172.21.15.73:0/491297613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae80047c0 0x7f7b00151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 >> 172.21.15.73:0/491297613 conn(0x7f7b000ce8e0 msgr2=0x7f7b000b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 shutdown_connections 2024-04-05T22:30:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:04.901+0000 7f7b07e00640 1 -- 172.21.15.73:0/491297613 wait complete. 2024-04-05T22:30:04.920 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.130+0000 7fe343e00640 1 Processor -- start 2024-04-05T22:30:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.130+0000 7fe343e00640 1 -- start start 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.131+0000 7fe343e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3440b20a0 0x7fe344070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.131+0000 7fe343e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b2b90 0x7fe34406cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.131+0000 7fe343e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34406d3e0 con 0x7fe3440b1b50 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.131+0000 7fe343e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34406d610 con 0x7fe3440b20a0 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.131+0000 7fe343e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe344070230 con 0x7fe3440b2b90 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3440b20a0 0x7fe344070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe33a000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe3440b1b50 0x7fe3440b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52926/0 (socket says 172.21.15.73:52926) 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe33a000640 1 -- 172.21.15.73:0/2837776750 learned_addr learned my addr 172.21.15.73:0/2837776750 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342a00640 1 -- 172.21.15.73:0/2837776750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b2b90 msgr2=0x7fe34406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342a00640 1 --2- 172.21.15.73:0/2837776750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b2b90 0x7fe34406cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342a00640 1 -- 172.21.15.73:0/2837776750 >> v1:172.21.15.73:6789/0 conn(0x7fe3440b1b50 legacy=0x7fe3440b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342a00640 1 -- 172.21.15.73:0/2837776750 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe344070510 con 0x7fe3440b20a0 2024-04-05T22:30:05.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.132+0000 7fe342000640 1 -- 172.21.15.73:0/2837776750 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925160834 0 0) 0x7fe34406d3e0 con 0x7fe3440b1b50 2024-04-05T22:30:05.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.133+0000 7fe342a00640 1 --2- 172.21.15.73:0/2837776750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe344070ff0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe3340097a0 tx=0x7fe33402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a259568ca76705 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe342000640 1 -- 172.21.15.73:0/2837776750 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33402f050 con 0x7fe3440b20a0 2024-04-05T22:30:05.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe342000640 1 -- 172.21.15.73:0/2837776750 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334031700 con 0x7fe3440b20a0 2024-04-05T22:30:05.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe342000640 1 -- 172.21.15.73:0/2837776750 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334031a90 con 0x7fe3440b20a0 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 -- 172.21.15.73:0/2837776750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 msgr2=0x7fe344070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 --2- 172.21.15.73:0/2837776750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe344070ff0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe3340097a0 tx=0x7fe33402b7a0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 -- 172.21.15.73:0/2837776750 shutdown_connections 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 --2- 172.21.15.73:0/2837776750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b2b90 0x7fe34406cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 --2- 172.21.15.73:0/2837776750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe344070ff0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 -- 172.21.15.73:0/2837776750 >> 172.21.15.73:0/2837776750 conn(0x7fe3440b06c0 msgr2=0x7fe3440b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 -- 172.21.15.73:0/2837776750 shutdown_connections 2024-04-05T22:30:05.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.134+0000 7fe343e00640 1 -- 172.21.15.73:0/2837776750 wait complete. 2024-04-05T22:30:05.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.136+0000 7fe343e00640 1 Processor -- start 2024-04-05T22:30:05.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.136+0000 7fe343e00640 1 -- start start 2024-04-05T22:30:05.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 0x7fe344147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 0x7fe344146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe344159400 con 0x7fe3440b20a0 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34414d7a0 con 0x7fe3440b2b90 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.137+0000 7fe343e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe344147b60 con 0x7fe3440b1b50 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34126/0 (socket says 172.21.15.73:34126) 2024-04-05T22:30:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 -- 172.21.15.73:0/2651086370 learned_addr learned my addr 172.21.15.73:0/2651086370 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe342a00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 0x7fe344147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe343400640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 0x7fe344146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 -- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 msgr2=0x7fe344147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 0x7fe344147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 -- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 msgr2=0x7fe344146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 0x7fe344146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 -- 172.21.15.73:0/2651086370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334009400 con 0x7fe3440b20a0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe342a00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 0x7fe344147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe343400640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 0x7fe344146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.138+0000 7fe33a000640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c460 tx=0x7fe32c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.139+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0074e0 con 0x7fe3440b20a0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.139+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c007660 con 0x7fe3440b20a0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.139+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c004a20 con 0x7fe3440b20a0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.139+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe344156830 con 0x7fe3440b20a0 2024-04-05T22:30:05.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.139+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe344158e40 con 0x7fe3440b20a0 2024-04-05T22:30:05.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.140+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3440ad370 con 0x7fe3440b20a0 2024-04-05T22:30:05.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.141+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fe32c004b80 con 0x7fe3440b20a0 2024-04-05T22:30:05.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.142+0000 7fe33be00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 0x7fe32006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.142+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe32c089050 con 0x7fe3440b20a0 2024-04-05T22:30:05.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.143+0000 7fe342a00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 0x7fe32006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.144+0000 7fe342a00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 0x7fe32006d6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe334035000 tx=0x7fe334008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.147+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe3440ad370 con 0x7fe3440b20a0 2024-04-05T22:30:05.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.348+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3440ad370 con 0x7fe3440b20a0 2024-04-05T22:30:05.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.350+0000 7fe33be00640 1 -- 172.21.15.73:0/2651086370 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16111 (secure 0 0 0) 0x7fe3440ad370 con 0x7fe3440b20a0 2024-04-05T22:30:05.350 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:05.350 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:29:28.557506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:29:25.591734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-05T22:29:26.571955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 msgr2=0x7fe32006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 0x7fe32006d6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe334035000 tx=0x7fe334008030 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 msgr2=0x7fe3441562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c460 tx=0x7fe32c00c920 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 shutdown_connections 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe32006b230 0x7fe32006d6f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3440b2b90 0x7fe344146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3440b20a0 0x7fe3441562f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.352+0000 7fe343e00640 1 --2- 172.21.15.73:0/2651086370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3440b1b50 0x7fe344147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.353+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 >> 172.21.15.73:0/2651086370 conn(0x7fe3440b06c0 msgr2=0x7fe3440cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.353+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 shutdown_connections 2024-04-05T22:30:05.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.353+0000 7fe343e00640 1 -- 172.21.15.73:0/2651086370 wait complete. 2024-04-05T22:30:05.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:30:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.585+0000 7f0531600640 1 Processor -- start 2024-04-05T22:30:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.585+0000 7f0531600640 1 -- start start 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f0531600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f052c06ede0 0x7f052c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f0531600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f052c06e920 0x7f052c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f0531600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c06bbc0 con 0x7f052c15d120 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f0531600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c15d500 con 0x7f052c06e920 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f0531600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c06b8e0 con 0x7f052c06ede0 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f052aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f052c15d120 0x7f052c0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52930/0 (socket says 172.21.15.73:52930) 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f052aa00640 1 -- 172.21.15.73:0/3702655012 learned_addr learned my addr 172.21.15.73:0/3702655012 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.587+0000 7f052b400640 1 --2- 172.21.15.73:0/3702655012 >> v2:172.21.15.73:3300/0 conn(0x7f052c06e920 0x7f052c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f052a000640 1 --2- 172.21.15.73:0/3702655012 >> v2:172.21.15.157:3300/0 conn(0x7f052c06ede0 0x7f052c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f0529600640 1 -- 172.21.15.73:0/3702655012 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766180476 0 0) 0x7f052c06bbc0 con 0x7f052c15d120 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f0529600640 1 -- 172.21.15.73:0/3702655012 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0510003520 con 0x7f052c15d120 2024-04-05T22:30:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f052b400640 1 -- 172.21.15.73:0/3702655012 >> v2:172.21.15.157:3300/0 conn(0x7f052c06ede0 msgr2=0x7f052c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f052b400640 1 --2- 172.21.15.73:0/3702655012 >> v2:172.21.15.157:3300/0 conn(0x7f052c06ede0 0x7f052c071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f052b400640 1 -- 172.21.15.73:0/3702655012 >> v1:172.21.15.73:6789/0 conn(0x7f052c15d120 legacy=0x7f052c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.588+0000 7f052b400640 1 -- 172.21.15.73:0/3702655012 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c070760 con 0x7f052c06e920 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.589+0000 7f052b400640 1 --2- 172.21.15.73:0/3702655012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c06e920 0x7f052c0cd900 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f052c066ac0 tx=0x7f052402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fed515bab752aa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.589+0000 7f0529600640 1 -- 172.21.15.73:0/3702655012 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052402cc80 con 0x7f052c06e920 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.589+0000 7f0529600640 1 -- 172.21.15.73:0/3702655012 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052402cde0 con 0x7f052c06e920 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.589+0000 7f0529600640 1 -- 172.21.15.73:0/3702655012 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524034660 con 0x7f052c06e920 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 -- 172.21.15.73:0/3702655012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c06e920 msgr2=0x7f052c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 --2- 172.21.15.73:0/3702655012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c06e920 0x7f052c0cd900 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f052c066ac0 tx=0x7f052402bed0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 -- 172.21.15.73:0/3702655012 shutdown_connections 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 --2- 172.21.15.73:0/3702655012 >> v2:172.21.15.157:3300/0 conn(0x7f052c06ede0 0x7f052c071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 --2- 172.21.15.73:0/3702655012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c06e920 0x7f052c0cd900 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 -- 172.21.15.73:0/3702655012 >> 172.21.15.73:0/3702655012 conn(0x7f052c06adc0 msgr2=0x7f052c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 -- 172.21.15.73:0/3702655012 shutdown_connections 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.590+0000 7f0531600640 1 -- 172.21.15.73:0/3702655012 wait complete. 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.592+0000 7f0531600640 1 Processor -- start 2024-04-05T22:30:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.592+0000 7f0531600640 1 -- start start 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.592+0000 7f0531600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052c06e920 0x7f052c1f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f0531600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f0531600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052c158e40 0x7f052c15fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f0531600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c15d900 con 0x7f052c15d120 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f0531600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c15dbb0 con 0x7f052c158e40 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f0531600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c15de90 con 0x7f052c06e920 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34144/0 (socket says 172.21.15.73:34144) 2024-04-05T22:30:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 -- 172.21.15.73:0/2811671307 learned_addr learned my addr 172.21.15.73:0/2811671307 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052b400640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052c158e40 0x7f052c15fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 -- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052c06e920 msgr2=0x7f052c1f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052c06e920 0x7f052c1f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 -- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052c158e40 msgr2=0x7f052c15fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052c158e40 0x7f052c15fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.593+0000 7f052a000640 1 -- 172.21.15.73:0/2811671307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009a70 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f052a000640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f051800d800 tx=0x7f051800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800b6c0 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051800b820 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800bbb0 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c15e170 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.594+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c15e730 con 0x7f052c15d120 2024-04-05T22:30:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.595+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c066af0 con 0x7f052c15d120 2024-04-05T22:30:05.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.597+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f051800bd10 con 0x7f052c15d120 2024-04-05T22:30:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.598+0000 7f0522a00640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 0x7f050c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.598+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f051808a450 con 0x7f052c15d120 2024-04-05T22:30:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.598+0000 7f052aa00640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 0x7f050c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.599+0000 7f052aa00640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 0x7f050c06d6f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f051c001610 tx=0x7f051c007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:05.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.602+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0518054cb0 con 0x7f052c15d120 2024-04-05T22:30:05.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.839+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f052c15e8c0 con 0x7f052c15d120 2024-04-05T22:30:05.840 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:30:05.840+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:30:05.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.856+0000 7f0522a00640 1 -- 172.21.15.73:0/2811671307 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f05180582d0 con 0x7f052c15d120 2024-04-05T22:30:05.857 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:30:05.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 msgr2=0x7f050c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 0x7f050c06d6f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f051c001610 tx=0x7f051c007930 comp rx=0 tx=0).stop 2024-04-05T22:30:05.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 msgr2=0x7f052c154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f051800d800 tx=0x7f051800dcc0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 shutdown_connections 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f050c06b230 0x7f050c06d6f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052c158e40 0x7f052c15fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052c15d120 0x7f052c154670 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 --2- 172.21.15.73:0/2811671307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f052c06e920 0x7f052c1f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 >> 172.21.15.73:0/2811671307 conn(0x7f052c06adc0 msgr2=0x7f052c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 shutdown_connections 2024-04-05T22:30:05.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:05.859+0000 7f0531600640 1 -- 172.21.15.73:0/2811671307 wait complete. 2024-04-05T22:30:05.878 INFO:tasks.ceph.mds.a.smithi073.stderr: -3113> 2024-04-05T22:29:33.058+0000 7f45ca600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:30:05.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:30:05.943 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:30:06.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.071+0000 7f0c10400640 1 Processor -- start 2024-04-05T22:30:06.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.072+0000 7f0c10400640 1 -- start start 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.073+0000 7f0c10400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c0806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.073+0000 7f0c10400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0c080b32b0 0x7f0c081647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.073+0000 7f0c10400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c080b1b30 con 0x7f0c080cbe00 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.073+0000 7f0c10400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c080b1e10 con 0x7f0c080b32b0 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.073+0000 7f0c10400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080ce7b0 con 0x7f0c0806c630 2024-04-05T22:30:06.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0d000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0c080cbe00 0x7f0c080ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48242/0 (socket says 172.21.15.73:48242) 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0d000640 1 -- 172.21.15.73:0/2120945993 learned_addr learned my addr 172.21.15.73:0/2120945993 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0e400640 1 --2- 172.21.15.73:0/2120945993 >> v2:172.21.15.73:3300/0 conn(0x7f0c080b32b0 0x7f0c081647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0da00640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c0806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c07e00640 1 -- 172.21.15.73:0/2120945993 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782211238 0 0) 0x7f0c080b1b30 con 0x7f0c080cbe00 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c07e00640 1 -- 172.21.15.73:0/2120945993 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf0003520 con 0x7f0c080cbe00 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0e400640 1 -- 172.21.15.73:0/2120945993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 msgr2=0x7f0c0806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0e400640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c0806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0e400640 1 -- 172.21.15.73:0/2120945993 >> v1:172.21.15.73:6789/0 conn(0x7f0c080cbe00 legacy=0x7f0c080ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.074+0000 7f0c0e400640 1 -- 172.21.15.73:0/2120945993 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0806cf50 con 0x7f0c080b32b0 2024-04-05T22:30:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.075+0000 7f0c0e400640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080b32b0 0x7f0c081647c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c0002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bba62298420671 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c07e00640 1 -- 172.21.15.73:0/2120945993 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0002cc50 con 0x7f0c080b32b0 2024-04-05T22:30:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c07e00640 1 -- 172.21.15.73:0/2120945993 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0002cdb0 con 0x7f0c080b32b0 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c07e00640 1 -- 172.21.15.73:0/2120945993 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00034660 con 0x7f0c080b32b0 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 -- 172.21.15.73:0/2120945993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080b32b0 msgr2=0x7f0c081647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080b32b0 0x7f0c081647c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c0002bea0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 -- 172.21.15.73:0/2120945993 shutdown_connections 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080b32b0 0x7f0c081647c0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 --2- 172.21.15.73:0/2120945993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c0806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 -- 172.21.15.73:0/2120945993 >> 172.21.15.73:0/2120945993 conn(0x7f0c080b0830 msgr2=0x7f0c080b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 -- 172.21.15.73:0/2120945993 shutdown_connections 2024-04-05T22:30:06.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.076+0000 7f0c10400640 1 -- 172.21.15.73:0/2120945993 wait complete. 2024-04-05T22:30:06.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.078+0000 7f0c10400640 1 Processor -- start 2024-04-05T22:30:06.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.078+0000 7f0c10400640 1 -- start start 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c081f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c080b32b0 0x7f0c08142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08141a10 con 0x7f0c080cbe00 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08141cf0 con 0x7f0c080b32b0 2024-04-05T22:30:06.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c10400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08141fd0 con 0x7f0c0806c630 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.079+0000 7f0c0e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55564/0 (socket says 172.21.15.73:55564) 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 -- 172.21.15.73:0/1858427540 learned_addr learned my addr 172.21.15.73:0/1858427540 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0d000640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c080b32b0 0x7f0c08142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 -- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 msgr2=0x7f0c081f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0da00640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c081f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c081f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 -- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c080b32b0 msgr2=0x7f0c08142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c080b32b0 0x7f0c08142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 -- 172.21.15.73:0/1858427540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00009a70 con 0x7f0c080cbe00 2024-04-05T22:30:06.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.080+0000 7f0c0e400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:06.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.081+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00040650 con 0x7f0c080cbe00 2024-04-05T22:30:06.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.081+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000407b0 con 0x7f0c080cbe00 2024-04-05T22:30:06.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.081+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c081422b0 con 0x7f0c080cbe00 2024-04-05T22:30:06.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.081+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00040b60 con 0x7f0c080cbe00 2024-04-05T22:30:06.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.081+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c081f21a0 con 0x7f0c080cbe00 2024-04-05T22:30:06.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.082+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c081427d0 con 0x7f0c080cbe00 2024-04-05T22:30:06.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.083+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f0c00040cc0 con 0x7f0c080cbe00 2024-04-05T22:30:06.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.084+0000 7f0c05600640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 0x7f0be806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:06.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.084+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0c000acd00 con 0x7f0c080cbe00 2024-04-05T22:30:06.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.084+0000 7f0c0da00640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 0x7f0be806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:06.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.085+0000 7f0c0da00640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 0x7f0be806d6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc006fd0 tx=0x7f0bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:06.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.088+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0c00077500 con 0x7f0c080cbe00 2024-04-05T22:30:06.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.305+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0c08068160 con 0x7f0c080cbe00 2024-04-05T22:30:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.857+0000 7f0c05600640 1 -- 172.21.15.73:0/1858427540 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c00009450 con 0x7f0c080cbe00 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 msgr2=0x7f0be806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 0x7f0be806d6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc006fd0 tx=0x7f0bfc008040 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 msgr2=0x7f0c080c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c00004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 shutdown_connections 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0be806b230 0x7f0be806d6f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c080cbe00 0x7f0c080c4ea0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c080b32b0 0x7f0c08142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 --2- 172.21.15.73:0/1858427540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c0806c630 0x7f0c081f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 >> 172.21.15.73:0/1858427540 conn(0x7f0c080b0830 msgr2=0x7f0c080b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:06.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.861+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 shutdown_connections 2024-04-05T22:30:06.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:06.862+0000 7f0c10400640 1 -- 172.21.15.73:0/1858427540 wait complete. 2024-04-05T22:30:06.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:30:07.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.088+0000 7f9f78c00640 1 Processor -- start 2024-04-05T22:30:07.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.088+0000 7f9f78c00640 1 -- start start 2024-04-05T22:30:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f78c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f740cecd0 0x7f9f74162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f78c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f740ad720 0x7f9f74164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f78c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f741620e0 con 0x7f9f740b3ab0 2024-04-05T22:30:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f78c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740ae370 con 0x7f9f740cecd0 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f78c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740ae090 con 0x7f9f740ad720 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f71e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f740cecd0 0x7f9f74162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f71e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f740cecd0 0x7f9f74162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55566/0 (socket says 172.21.15.73:55566) 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.090+0000 7f9f71e00640 1 -- 172.21.15.73:0/3300485275 learned_addr learned my addr 172.21.15.73:0/3300485275 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.091+0000 7f9f71e00640 1 -- 172.21.15.73:0/3300485275 >> v2:172.21.15.157:3300/0 conn(0x7f9f740ad720 msgr2=0x7f9f74164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.091+0000 7f9f71e00640 1 --2- 172.21.15.73:0/3300485275 >> v2:172.21.15.157:3300/0 conn(0x7f9f740ad720 0x7f9f74164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.091+0000 7f9f71e00640 1 -- 172.21.15.73:0/3300485275 >> v1:172.21.15.157:6789/0 conn(0x7f9f740b3ab0 legacy=0x7f9f740cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:07.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.091+0000 7f9f71e00640 1 -- 172.21.15.73:0/3300485275 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74161a90 con 0x7f9f740cecd0 2024-04-05T22:30:07.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.092+0000 7f9f71e00640 1 --2- 172.21.15.73:0/3300485275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740cecd0 0x7f9f74162570 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9f740b05e0 tx=0x7f9f6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d32014d6727099 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:07.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.092+0000 7f9f70a00640 1 -- 172.21.15.73:0/3300485275 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64031870 con 0x7f9f740cecd0 2024-04-05T22:30:07.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.092+0000 7f9f70a00640 1 -- 172.21.15.73:0/3300485275 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f640319d0 con 0x7f9f740cecd0 2024-04-05T22:30:07.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.092+0000 7f9f70a00640 1 -- 172.21.15.73:0/3300485275 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64031d60 con 0x7f9f740cecd0 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 -- 172.21.15.73:0/3300485275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740cecd0 msgr2=0x7f9f74162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3300485275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740cecd0 0x7f9f74162570 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9f740b05e0 tx=0x7f9f6402b910 comp rx=0 tx=0).stop 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 -- 172.21.15.73:0/3300485275 shutdown_connections 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3300485275 >> v2:172.21.15.157:3300/0 conn(0x7f9f740ad720 0x7f9f74164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3300485275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740cecd0 0x7f9f74162570 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 -- 172.21.15.73:0/3300485275 >> 172.21.15.73:0/3300485275 conn(0x7f9f7406a0d0 msgr2=0x7f9f7406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 -- 172.21.15.73:0/3300485275 shutdown_connections 2024-04-05T22:30:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.093+0000 7f9f78c00640 1 -- 172.21.15.73:0/3300485275 wait complete. 2024-04-05T22:30:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.095+0000 7f9f78c00640 1 Processor -- start 2024-04-05T22:30:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.095+0000 7f9f78c00640 1 -- start start 2024-04-05T22:30:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f740ad720 0x7f9f74164250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f740cecd0 0x7f9f741629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74162f10 con 0x7f9f740b3ab0 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f741631b0 con 0x7f9f740ad720 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f78c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74163490 con 0x7f9f740cecd0 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55570/0 (socket says 172.21.15.73:55570) 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 -- 172.21.15.73:0/3278461697 learned_addr learned my addr 172.21.15.73:0/3278461697 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 -- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f740cecd0 msgr2=0x7f9f741629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f740cecd0 0x7f9f741629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 -- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f740ad720 msgr2=0x7f9f74164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71e00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f740ad720 0x7f9f74164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f740ad720 0x7f9f74164250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.096+0000 7f9f71400640 1 -- 172.21.15.73:0/3278461697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640094d0 con 0x7f9f740b3ab0 2024-04-05T22:30:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f71400640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c400 tx=0x7f9f6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600035f0 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60003750 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f74163770 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003b40 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.097+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f74163d30 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.098+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f74163f20 con 0x7f9f740b3ab0 2024-04-05T22:30:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.099+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f9f60003ca0 con 0x7f9f740b3ab0 2024-04-05T22:30:07.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.101+0000 7f9f6a000640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 0x7f9f5406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:07.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.101+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9f60089430 con 0x7f9f740b3ab0 2024-04-05T22:30:07.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.102+0000 7f9f71e00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 0x7f9f5406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:07.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.103+0000 7f9f71e00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 0x7f9f5406d6f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f64036000 tx=0x7f9f6403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:07.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.106+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9f60053d70 con 0x7f9f740b3ab0 2024-04-05T22:30:07.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.298+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f74068160 con 0x7f9f740b3ab0 2024-04-05T22:30:07.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.875+0000 7f9f6a000640 1 -- 172.21.15.73:0/3278461697 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v138) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9f60057390 con 0x7f9f740b3ab0 2024-04-05T22:30:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:30:07.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.879+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 msgr2=0x7f9f5406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:07.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.879+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 0x7f9f5406d6f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f64036000 tx=0x7f9f6403c040 comp rx=0 tx=0).stop 2024-04-05T22:30:07.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.879+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 msgr2=0x7f9f7415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:07.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.879+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c400 tx=0x7f9f6000c8c0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 shutdown_connections 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9f5406b230 0x7f9f5406d6f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f740cecd0 0x7f9f741629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f740b3ab0 0x7f9f7415e750 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 --2- 172.21.15.73:0/3278461697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f740ad720 0x7f9f74164250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:07.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 >> 172.21.15.73:0/3278461697 conn(0x7f9f7406a0d0 msgr2=0x7f9f74160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:07.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 shutdown_connections 2024-04-05T22:30:07.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:07.880+0000 7f9f78c00640 1 -- 172.21.15.73:0/3278461697 wait complete. 2024-04-05T22:30:07.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:30:08.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.109+0000 7f7425e00640 1 Processor -- start 2024-04-05T22:30:08.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.110+0000 7f7425e00640 1 -- start start 2024-04-05T22:30:08.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f742015cd00 0x7f74200ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74200d0fd0 0x7f74200d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74200d1950 con 0x7f7420160780 2024-04-05T22:30:08.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74200b68c0 con 0x7f742015cd00 2024-04-05T22:30:08.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74200ced00 con 0x7f74200d0fd0 2024-04-05T22:30:08.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7424a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f742015cd00 0x7f74200ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7420160780 0x7f7420160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48248/0 (socket says 172.21.15.73:48248) 2024-04-05T22:30:08.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.111+0000 7f7425400640 1 -- 172.21.15.73:0/276353670 learned_addr learned my addr 172.21.15.73:0/276353670 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f7417e00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74200d0fd0 0x7f74200d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f7424a00640 1 -- 172.21.15.73:0/276353670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74200d0fd0 msgr2=0x7f74200d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f7424a00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74200d0fd0 0x7f74200d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f7424a00640 1 -- 172.21.15.73:0/276353670 >> v1:172.21.15.73:6789/0 conn(0x7f7420160780 legacy=0x7f7420160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f7424a00640 1 -- 172.21.15.73:0/276353670 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200cefe0 con 0x7f742015cd00 2024-04-05T22:30:08.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.112+0000 7f741fe00640 1 -- 172.21.15.73:0/276353670 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578130185 0 0) 0x7f74200d1950 con 0x7f7420160780 2024-04-05T22:30:08.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.113+0000 7f7424a00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f742015cd00 0x7f74200ce360 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f74100097a0 tx=0x7f741002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae57cc9955d13d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:08.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.113+0000 7f741fe00640 1 -- 172.21.15.73:0/276353670 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741002d040 con 0x7f742015cd00 2024-04-05T22:30:08.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.113+0000 7f741fe00640 1 -- 172.21.15.73:0/276353670 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7410031810 con 0x7f742015cd00 2024-04-05T22:30:08.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.113+0000 7f741fe00640 1 -- 172.21.15.73:0/276353670 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410031ba0 con 0x7f742015cd00 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 -- 172.21.15.73:0/276353670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f742015cd00 msgr2=0x7f74200ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f742015cd00 0x7f74200ce360 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f74100097a0 tx=0x7f741002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 -- 172.21.15.73:0/276353670 shutdown_connections 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74200d0fd0 0x7f74200d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 --2- 172.21.15.73:0/276353670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f742015cd00 0x7f74200ce360 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 -- 172.21.15.73:0/276353670 >> 172.21.15.73:0/276353670 conn(0x7f742009bdd0 msgr2=0x7f74200d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 -- 172.21.15.73:0/276353670 shutdown_connections 2024-04-05T22:30:08.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.114+0000 7f7425e00640 1 -- 172.21.15.73:0/276353670 wait complete. 2024-04-05T22:30:08.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.116+0000 7f7425e00640 1 Processor -- start 2024-04-05T22:30:08.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.116+0000 7f7425e00640 1 -- start start 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 0x7f742015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 0x7f74200cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200ca540 con 0x7f74200d0fd0 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742015b8c0 con 0x7f7420160780 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7425e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74200ca790 con 0x7f742015cd00 2024-04-05T22:30:08.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7424a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7424a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55592/0 (socket says 172.21.15.73:55592) 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.117+0000 7f7424a00640 1 -- 172.21.15.73:0/534894786 learned_addr learned my addr 172.21.15.73:0/534894786 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7417e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 0x7f742015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 -- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 msgr2=0x7f742015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 0x7f742015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 -- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 msgr2=0x7f74200cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7425400640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 0x7f74200cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 0x7f74200cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 -- 172.21.15.73:0/534894786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410009400 con 0x7f74200d0fd0 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7417e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 0x7f742015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7425400640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 0x7f74200cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:08.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7424a00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f74100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741003aa50 con 0x7f74200d0fd0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741003abb0 con 0x7f74200d0fd0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.118+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74200caa70 con 0x7f74200d0fd0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.119+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410004030 con 0x7f74200d0fd0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.119+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74200b7b80 con 0x7f74200d0fd0 2024-04-05T22:30:08.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.120+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742015bb60 con 0x7f74200d0fd0 2024-04-05T22:30:08.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.121+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7410042070 con 0x7f74200d0fd0 2024-04-05T22:30:08.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.121+0000 7f741d600640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 0x7f73fc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:08.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.122+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f74100abc60 con 0x7f74200d0fd0 2024-04-05T22:30:08.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.122+0000 7f7417e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 0x7f73fc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:08.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.123+0000 7f7417e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 0x7f73fc06d720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7408006fd0 tx=0x7f7408008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:08.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.126+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7410076510 con 0x7f74200d0fd0 2024-04-05T22:30:08.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.321+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f74200caf90 con 0x7f74200d0fd0 2024-04-05T22:30:08.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.880+0000 7f741d600640 1 -- 172.21.15.73:0/534894786 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v139) v1 ==== 169+0+0 (secure 0 0 0) 0x7f74100350d0 con 0x7f74200d0fd0 2024-04-05T22:30:08.883 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:30:08.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.886+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 msgr2=0x7f73fc06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.886+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 0x7f73fc06d720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7408006fd0 tx=0x7f7408008040 comp rx=0 tx=0).stop 2024-04-05T22:30:08.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.886+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 msgr2=0x7f7420151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.886+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f74100046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 shutdown_connections 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f73fc06b260 0x7f73fc06d720 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7420160780 0x7f74200cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015cd00 0x7f742015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 --2- 172.21.15.73:0/534894786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74200d0fd0 0x7f7420151290 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:08.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.887+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 >> 172.21.15.73:0/534894786 conn(0x7f742009bdd0 msgr2=0x7f74200cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:08.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.888+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 shutdown_connections 2024-04-05T22:30:08.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:08.888+0000 7f7425e00640 1 -- 172.21.15.73:0/534894786 wait complete. 2024-04-05T22:30:08.912 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:30:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.105+0000 7fa9f7200640 1 Processor -- start 2024-04-05T22:30:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.105+0000 7fa9f7200640 1 -- start start 2024-04-05T22:30:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.106+0000 7fa9f7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9f0160720 0x7fa9f00cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.106+0000 7fa9f7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9f00cfb10 0x7fa9f00b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.106+0000 7fa9f7200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00cff50 con 0x7fa9f00b2ad0 2024-04-05T22:30:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.106+0000 7fa9f7200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00b44e0 con 0x7fa9f0160720 2024-04-05T22:30:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.106+0000 7fa9f7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00d11a0 con 0x7fa9f00cfb10 2024-04-05T22:30:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9f0160720 0x7fa9f00cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9f0160720 0x7fa9f00cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55596/0 (socket says 172.21.15.73:55596) 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 -- 172.21.15.73:0/1697540879 learned_addr learned my addr 172.21.15.73:0/1697540879 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 -- 172.21.15.73:0/1697540879 >> v2:172.21.15.157:3300/0 conn(0x7fa9f00cfb10 msgr2=0x7fa9f00b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 --2- 172.21.15.73:0/1697540879 >> v2:172.21.15.157:3300/0 conn(0x7fa9f00cfb10 0x7fa9f00b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 -- 172.21.15.73:0/1697540879 >> v1:172.21.15.157:6789/0 conn(0x7fa9f00b2ad0 legacy=0x7fa9f00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.107+0000 7fa9efe00640 1 -- 172.21.15.73:0/1697540879 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00b3610 con 0x7fa9f0160720 2024-04-05T22:30:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.108+0000 7fa9efe00640 1 --2- 172.21.15.73:0/1697540879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cf5d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa9f008bbc0 tx=0x7fa9e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56172b74bdf0cdba server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.109+0000 7fa9eea00640 1 -- 172.21.15.73:0/1697540879 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00317e0 con 0x7fa9f0160720 2024-04-05T22:30:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.109+0000 7fa9eea00640 1 -- 172.21.15.73:0/1697540879 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0031940 con 0x7fa9f0160720 2024-04-05T22:30:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.109+0000 7fa9eea00640 1 -- 172.21.15.73:0/1697540879 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0031cd0 con 0x7fa9f0160720 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.109+0000 7fa9f7200640 1 -- 172.21.15.73:0/1697540879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 msgr2=0x7fa9f00cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.109+0000 7fa9f7200640 1 --2- 172.21.15.73:0/1697540879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cf5d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa9f008bbc0 tx=0x7fa9e002b880 comp rx=0 tx=0).stop 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 -- 172.21.15.73:0/1697540879 shutdown_connections 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 --2- 172.21.15.73:0/1697540879 >> v2:172.21.15.157:3300/0 conn(0x7fa9f00cfb10 0x7fa9f00b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 --2- 172.21.15.73:0/1697540879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cf5d0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 -- 172.21.15.73:0/1697540879 >> 172.21.15.73:0/1697540879 conn(0x7fa9f009c000 msgr2=0x7fa9f00d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 -- 172.21.15.73:0/1697540879 shutdown_connections 2024-04-05T22:30:09.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.110+0000 7fa9f7200640 1 -- 172.21.15.73:0/1697540879 wait complete. 2024-04-05T22:30:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.111+0000 7fa9f7200640 1 Processor -- start 2024-04-05T22:30:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 -- start start 2024-04-05T22:30:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f00b2ad0 0x7fa9f01f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cfb10 0x7fa9f00b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01f25d0 con 0x7fa9f0160720 2024-04-05T22:30:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00b6ab0 con 0x7fa9f00b2ad0 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.112+0000 7fa9f7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00b6d90 con 0x7fa9f00cfb10 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55602/0 (socket says 172.21.15.73:55602) 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 -- 172.21.15.73:0/606455788 learned_addr learned my addr 172.21.15.73:0/606455788 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 -- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cfb10 msgr2=0x7fa9f00b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cfb10 0x7fa9f00b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 -- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f00b2ad0 msgr2=0x7fa9f01f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f00b2ad0 0x7fa9f01f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.113+0000 7fa9f5200640 1 -- 172.21.15.73:0/606455788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00094d0 con 0x7fa9f0160720 2024-04-05T22:30:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9f5200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b990 tx=0x7fa9e800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800cba0 con 0x7fa9f0160720 2024-04-05T22:30:09.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800cd00 con 0x7fa9f0160720 2024-04-05T22:30:09.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8014680 con 0x7fa9f0160720 2024-04-05T22:30:09.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00b7070 con 0x7fa9f0160720 2024-04-05T22:30:09.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.114+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f00b7630 con 0x7fa9f0160720 2024-04-05T22:30:09.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.115+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f0068160 con 0x7fa9f0160720 2024-04-05T22:30:09.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.116+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fa9e801d3b0 con 0x7fa9f0160720 2024-04-05T22:30:09.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.117+0000 7fa9e7e00640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 0x7fa9cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.117+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9e8089ab0 con 0x7fa9f0160720 2024-04-05T22:30:09.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.118+0000 7fa9efe00640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 0x7fa9cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.119+0000 7fa9efe00640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 0x7fa9cc06d6f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0036000 tx=0x7fa9e0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.121+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9e80546e0 con 0x7fa9f0160720 2024-04-05T22:30:09.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.318+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa9f00b08a0 con 0x7fa9f0160720 2024-04-05T22:30:09.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.327+0000 7fa9e7e00640 1 -- 172.21.15.73:0/606455788 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa9e80190a0 con 0x7fa9f0160720 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 msgr2=0x7fa9cc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 0x7fa9cc06d6f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0036000 tx=0x7fa9e0030000 comp rx=0 tx=0).stop 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 msgr2=0x7fa9f00cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b990 tx=0x7fa9e800be50 comp rx=0 tx=0).stop 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 shutdown_connections 2024-04-05T22:30:09.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9cc06b230 0x7fa9cc06d6f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f0160720 0x7fa9f00cc9f0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cfb10 0x7fa9f00b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 --2- 172.21.15.73:0/606455788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f00b2ad0 0x7fa9f01f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 >> 172.21.15.73:0/606455788 conn(0x7fa9f009c000 msgr2=0x7fa9f00d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 shutdown_connections 2024-04-05T22:30:09.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.330+0000 7fa9f7200640 1 -- 172.21.15.73:0/606455788 wait complete. 2024-04-05T22:30:09.348 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-05T22:30:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.553+0000 7fbe2e400640 1 Processor -- start 2024-04-05T22:30:09.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.554+0000 7fbe2e400640 1 -- start start 2024-04-05T22:30:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.555+0000 7fbe2e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 0x7fbe280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.555+0000 7fbe2e400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe28070c00 0x7fbe2815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.555+0000 7fbe2e400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2806cf30 con 0x7fbe2815de40 2024-04-05T22:30:09.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.555+0000 7fbe2e400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe28071080 con 0x7fbe28070c00 2024-04-05T22:30:09.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.555+0000 7fbe2e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280cc550 con 0x7fbe280cc170 2024-04-05T22:30:09.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe27e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbe2815de40 0x7fbe280b3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48250/0 (socket says 172.21.15.73:48250) 2024-04-05T22:30:09.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe27e00640 1 -- 172.21.15.73:0/3740045914 learned_addr learned my addr 172.21.15.73:0/3740045914 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe27400640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 0x7fbe280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3740045914 >> v2:172.21.15.157:3300/0 conn(0x7fbe28070c00 0x7fbe2815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879988494 0 0) 0x7fbe2806cf30 con 0x7fbe2815de40 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe0c003520 con 0x7fbe2815de40 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.556+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1066992676 0 0) 0x7fbe0c003520 con 0x7fbe2815de40 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 msgr2=0x7fbe280706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 0x7fbe280706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 >> v2:172.21.15.157:3300/0 conn(0x7fbe28070c00 msgr2=0x7fbe2815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 --2- 172.21.15.73:0/3740045914 >> v2:172.21.15.157:3300/0 conn(0x7fbe28070c00 0x7fbe2815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2806c600 con 0x7fbe2815de40 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbe20001670 con 0x7fbe2815de40 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 >> v1:172.21.15.73:6789/0 conn(0x7fbe2815de40 legacy=0x7fbe280b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 0x7fbe0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28071080 con 0x7fbe0c0047c0 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280cc550 con 0x7fbe0c003970 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c00db90 con 0x7fbe280cc170 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe27400640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 -- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 msgr2=0x7fbe0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280cc170 0x7fbe0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 -- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c003970 msgr2=0x7fbe0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.557+0000 7fbe26a00640 1 -- 172.21.15.73:0/3740045914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c003520 con 0x7fbe0c0047c0 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.558+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009fc0 tx=0x7fbe0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbc1d015b388716 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1802cc60 con 0x7fbe0c0047c0 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1800f040 con 0x7fbe0c0047c0 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe26000640 1 -- 172.21.15.73:0/3740045914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18034440 con 0x7fbe0c0047c0 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe2e400640 1 -- 172.21.15.73:0/3740045914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0c0047c0 msgr2=0x7fbe0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3740045914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009fc0 tx=0x7fbe0c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe27e00640 1 -- 172.21.15.73:0/3740045914 reap_dead start 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe2e400640 1 -- 172.21.15.73:0/3740045914 shutdown_connections 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.559+0000 7fbe2e400640 1 -- 172.21.15.73:0/3740045914 >> 172.21.15.73:0/3740045914 conn(0x7fbe280b16d0 msgr2=0x7fbe280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:09.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.560+0000 7fbe2e400640 1 -- 172.21.15.73:0/3740045914 shutdown_connections 2024-04-05T22:30:09.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.560+0000 7fbe2e400640 1 -- 172.21.15.73:0/3740045914 wait complete. 2024-04-05T22:30:09.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.561+0000 7fbe2e400640 1 Processor -- start 2024-04-05T22:30:09.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.562+0000 7fbe2e400640 1 -- start start 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.562+0000 7fbe2e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 0x7fbe281f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.562+0000 7fbe2e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 0x7fbe281590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe2e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe2e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28159630 con 0x7fbe28142260 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe2e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281f2340 con 0x7fbe0c0047c0 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe2e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28142f90 con 0x7fbe28158c70 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55622/0 (socket says 172.21.15.73:55622) 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 -- 172.21.15.73:0/3303061144 learned_addr learned my addr 172.21.15.73:0/3303061144 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 0x7fbe281f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 0x7fbe281590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 -- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 msgr2=0x7fbe281590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 0x7fbe281590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 -- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 msgr2=0x7fbe281f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 0x7fbe281f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27e00640 1 -- 172.21.15.73:0/3303061144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28143270 con 0x7fbe28142260 2024-04-05T22:30:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe26a00640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 0x7fbe281590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.563+0000 7fbe27400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 0x7fbe281f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe27e00640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f60 tx=0x7fbe2000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20007230 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000ced0 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe200075d0 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe28143550 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.564+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe28143b10 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.565+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe28143d00 con 0x7fbe28142260 2024-04-05T22:30:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.566+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fbe200047e0 con 0x7fbe28142260 2024-04-05T22:30:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.567+0000 7fbe1f400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 0x7fbe0406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.567+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe20089080 con 0x7fbe28142260 2024-04-05T22:30:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.568+0000 7fbe27400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 0x7fbe0406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:09.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.569+0000 7fbe27400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 0x7fbe0406d720 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009910 tx=0x7fbe10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.572+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbe20053c20 con 0x7fbe28142260 2024-04-05T22:30:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.836+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7fbe28068160 con 0x7fbe28142260 2024-04-05T22:30:09.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.895+0000 7fbe1f400640 1 -- 172.21.15.73:0/3303061144 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v479) v1 ==== 151+0+0 (secure 0 0 0) 0x7fbe2001a020 con 0x7fbe28142260 2024-04-05T22:30:09.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.897+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 msgr2=0x7fbe0406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 0x7fbe0406d720 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009910 tx=0x7fbe10008040 comp rx=0 tx=0).stop 2024-04-05T22:30:09.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 msgr2=0x7fbe281426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:09.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f60 tx=0x7fbe2000cb90 comp rx=0 tx=0).stop 2024-04-05T22:30:09.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 shutdown_connections 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbe0406b260 0x7fbe0406d720 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe28142260 0x7fbe281426e0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28158c70 0x7fbe281590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 --2- 172.21.15.73:0/3303061144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe0c0047c0 0x7fbe281f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 >> 172.21.15.73:0/3303061144 conn(0x7fbe280b16d0 msgr2=0x7fbe2815aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 shutdown_connections 2024-04-05T22:30:09.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:09.898+0000 7fbe2e400640 1 -- 172.21.15.73:0/3303061144 wait complete. 2024-04-05T22:30:09.916 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-05T22:30:09.917 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-04-05T22:30:09.917 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:30:09.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:09.963 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:30:09.970 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:30:10.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.113+0000 7f274c000640 1 Processor -- start 2024-04-05T22:30:10.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.113+0000 7f274c000640 1 -- start start 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274c000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2744160620 0x7f27440caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274c000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440ad880 con 0x7f27440cb2f0 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274c000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440ce5d0 con 0x7f27440b1870 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274c000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440ce010 con 0x7f2744160620 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f2749600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2744160620 0x7f27440caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274a000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f27440cb2f0 0x7f27440cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48256/0 (socket says 172.21.15.73:48256) 2024-04-05T22:30:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.115+0000 7f274a000640 1 -- 172.21.15.73:0/2068934585 learned_addr learned my addr 172.21.15.73:0/2068934585 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.116+0000 7f2749600640 1 -- 172.21.15.73:0/2068934585 >> v1:172.21.15.157:6789/0 conn(0x7f27440b1870 legacy=0x7f27440ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.116+0000 7f2749600640 1 -- 172.21.15.73:0/2068934585 >> v1:172.21.15.73:6789/0 conn(0x7f27440cb2f0 legacy=0x7f27440cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.116+0000 7f2749600640 1 -- 172.21.15.73:0/2068934585 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440ce2f0 con 0x7f2744160620 2024-04-05T22:30:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.116+0000 7f2743e00640 1 -- 172.21.15.73:0/2068934585 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438527796 0 0) 0x7f27440ad880 con 0x7f27440cb2f0 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.117+0000 7f2749600640 1 --2- 172.21.15.73:0/2068934585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f27440caa60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d6133723ba74e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.117+0000 7f2743e00640 1 -- 172.21.15.73:0/2068934585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738031830 con 0x7f2744160620 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.117+0000 7f2743e00640 1 -- 172.21.15.73:0/2068934585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738031990 con 0x7f2744160620 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.117+0000 7f2743e00640 1 -- 172.21.15.73:0/2068934585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738031d20 con 0x7f2744160620 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 -- 172.21.15.73:0/2068934585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 msgr2=0x7f27440caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 --2- 172.21.15.73:0/2068934585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f27440caa60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 -- 172.21.15.73:0/2068934585 shutdown_connections 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 --2- 172.21.15.73:0/2068934585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f27440caa60 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 -- 172.21.15.73:0/2068934585 >> 172.21.15.73:0/2068934585 conn(0x7f27440ce8e0 msgr2=0x7f27440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 -- 172.21.15.73:0/2068934585 shutdown_connections 2024-04-05T22:30:10.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.118+0000 7f274c000640 1 -- 172.21.15.73:0/2068934585 wait complete. 2024-04-05T22:30:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.120+0000 7f274c000640 1 Processor -- start 2024-04-05T22:30:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.120+0000 7f274c000640 1 -- start start 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27440b1870 0x7f27440aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440cb2f0 0x7f27440ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2744155270 con 0x7f2744160620 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440ae920 con 0x7f27440b1870 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2744159340 con 0x7f27440cb2f0 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55646/0 (socket says 172.21.15.73:55646) 2024-04-05T22:30:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274a000640 1 -- 172.21.15.73:0/307775505 learned_addr learned my addr 172.21.15.73:0/307775505 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274a000640 1 -- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440cb2f0 msgr2=0x7f27440ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.121+0000 7f274a000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440cb2f0 0x7f27440ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274a000640 1 -- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27440b1870 msgr2=0x7f27440aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274a000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27440b1870 0x7f27440aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274a000640 1 -- 172.21.15.73:0/307775505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009400 con 0x7f2744160620 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274a000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c7e0 tx=0x7f273c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00a640 con 0x7f2744160620 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c00a7c0 con 0x7f2744160620 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27441468c0 con 0x7f2744160620 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00ab70 con 0x7f2744160620 2024-04-05T22:30:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.122+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27440aeb30 con 0x7f2744160620 2024-04-05T22:30:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.123+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2744066af0 con 0x7f2744160620 2024-04-05T22:30:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.124+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f273c00acd0 con 0x7f2744160620 2024-04-05T22:30:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.125+0000 7f2741600640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 0x7f272406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.126+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f273c0893a0 con 0x7f2744160620 2024-04-05T22:30:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.126+0000 7f2749600640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 0x7f272406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.127+0000 7f2749600640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 0x7f272406d6f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f2738030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.130+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f273c053fd0 con 0x7f2744160620 2024-04-05T22:30:10.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.322+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27440b1fc0 con 0x7f2744160620 2024-04-05T22:30:10.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.324+0000 7f2741600640 1 -- 172.21.15.73:0/307775505 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+12528 (secure 0 0 0) 0x7f273c018090 con 0x7f2744160620 2024-04-05T22:30:10.324 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:10.324 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":139,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:08.866692+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6835/1637066621":"2024-04-06T22:30:05.840867+0000","172.21.15.73:6833/1637066621":"2024-04-06T22:30:05.840867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.326+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 msgr2=0x7f272406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.326+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 0x7f272406d6f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f2738030000 comp rx=0 tx=0).stop 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.326+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 msgr2=0x7f2744146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.326+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c7e0 tx=0x7f273c00cca0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 shutdown_connections 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f272406b230 0x7f272406d6f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2744160620 0x7f2744146380 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440cb2f0 0x7f27440ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 --2- 172.21.15.73:0/307775505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27440b1870 0x7f27440aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 >> 172.21.15.73:0/307775505 conn(0x7f27440ce8e0 msgr2=0x7f274415f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:10.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 shutdown_connections 2024-04-05T22:30:10.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.327+0000 7f274c000640 1 -- 172.21.15.73:0/307775505 wait complete. 2024-04-05T22:30:10.345 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:10.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.599+0000 7f2349600640 1 Processor -- start 2024-04-05T22:30:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.599+0000 7f2349600640 1 -- start start 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2349600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f2344070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2349600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23440b27f0 con 0x7f234406cd50 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2349600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23440b2ad0 con 0x7f23440add00 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2349600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23440b3890 con 0x7f23441599f0 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2343400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f234406cd50 0x7f2344159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48268/0 (socket says 172.21.15.73:48268) 2024-04-05T22:30:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2343400640 1 -- 172.21.15.73:0/683228691 learned_addr learned my addr 172.21.15.73:0/683228691 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.601+0000 7f2342a00640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f2344070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145212964 0 0) 0x7f23440b27f0 con 0x7f234406cd50 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2328003520 con 0x7f234406cd50 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3836066895 0 0) 0x7f2328003520 con 0x7f234406cd50 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 msgr2=0x7f2344070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f2344070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 >> v1:172.21.15.157:6789/0 conn(0x7f23440add00 legacy=0x7f234406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440b2db0 con 0x7f234406cd50 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f233c0019f0 con 0x7f234406cd50 2024-04-05T22:30:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 >> v1:172.21.15.73:6789/0 conn(0x7f234406cd50 legacy=0x7f2344159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280039a0 0x7f2328003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f23280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2328004d00 0x7f2328005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23440b3890 con 0x7f2328004d00 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232800dc40 con 0x7f23280039a0 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.602+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232800de00 con 0x7f23441599f0 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2328004d00 0x7f2328005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2342a00640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280039a0 0x7f2328003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2342000640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f23280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 -- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 msgr2=0x7f23280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f23280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 -- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280039a0 msgr2=0x7f2328003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:10.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280039a0 0x7f2328003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2343400640 1 -- 172.21.15.73:0/683228691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328003520 con 0x7f2328004d00 2024-04-05T22:30:10.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.603+0000 7f2342000640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23441599f0 0x7f23280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.604+0000 7f2343400640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2328004d00 0x7f2328005180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f23440ae570 tx=0x7f23280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5e506af46cc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.604+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0072d0 con 0x7f2328004d00 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.604+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c02f070 con 0x7f2328004d00 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.604+0000 7f2341600640 1 -- 172.21.15.73:0/683228691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0076b0 con 0x7f2328004d00 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 -- 172.21.15.73:0/683228691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2328004d00 msgr2=0x7f2328005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 --2- 172.21.15.73:0/683228691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2328004d00 0x7f2328005180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f23440ae570 tx=0x7f23280057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2343400640 1 -- 172.21.15.73:0/683228691 reap_dead start 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 -- 172.21.15.73:0/683228691 shutdown_connections 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 -- 172.21.15.73:0/683228691 >> 172.21.15.73:0/683228691 conn(0x7f2344069cb0 msgr2=0x7f234406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 -- 172.21.15.73:0/683228691 shutdown_connections 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.605+0000 7f2349600640 1 -- 172.21.15.73:0/683228691 wait complete. 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.607+0000 7f2349600640 1 Processor -- start 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.607+0000 7f2349600640 1 -- start start 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 0x7f23440ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 0x7f23440af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23440cc570 con 0x7f23440cc990 2024-04-05T22:30:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23440cbc60 con 0x7f2328004d00 2024-04-05T22:30:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2349600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23440afc40 con 0x7f23440aeed0 2024-04-05T22:30:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2343400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2343400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55660/0 (socket says 172.21.15.73:55660) 2024-04-05T22:30:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.608+0000 7f2343400640 1 -- 172.21.15.73:0/801175920 learned_addr learned my addr 172.21.15.73:0/801175920 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2342000640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 0x7f23440af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2342a00640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 0x7f23440ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 -- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 msgr2=0x7f23440af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 0x7f23440af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 -- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 msgr2=0x7f23440ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 0x7f23440ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 -- 172.21.15.73:0/801175920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440cbeb0 con 0x7f23440cc990 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2342000640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 0x7f23440af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2342a00640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 0x7f23440ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2343400640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f233c00a8f0 tx=0x7f233c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c004d60 con 0x7f23440cc990 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c001250 con 0x7f23440cc990 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0072d0 con 0x7f23440cc990 2024-04-05T22:30:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23440cc190 con 0x7f23440cc990 2024-04-05T22:30:10.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.609+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23440ced80 con 0x7f23440cc990 2024-04-05T22:30:10.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.610+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23440cc6b0 con 0x7f23440cc990 2024-04-05T22:30:10.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.611+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f233c031030 con 0x7f23440cc990 2024-04-05T22:30:10.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.612+0000 7f233aa00640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 0x7f232406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:10.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.612+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f233c0aab60 con 0x7f23440cc990 2024-04-05T22:30:10.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.613+0000 7f2342a00640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 0x7f232406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:10.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.614+0000 7f2342a00640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 0x7f232406d720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2334009a00 tx=0x7f2334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:10.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.617+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f23440cc6b0 con 0x7f23440cc990 2024-04-05T22:30:10.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.795+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23440cc6b0 con 0x7f23440cc990 2024-04-05T22:30:10.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.796+0000 7f233aa00640 1 -- 172.21.15.73:0/801175920 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23440cc6b0 con 0x7f23440cc990 2024-04-05T22:30:10.796 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 msgr2=0x7f232406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 0x7f232406d720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2334009a00 tx=0x7f2334008040 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 msgr2=0x7f23440cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f233c00a8f0 tx=0x7f233c001d80 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 shutdown_connections 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f232406b260 0x7f232406d720 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23440cc990 0x7f23440cce10 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23440aeed0 0x7f23440af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 --2- 172.21.15.73:0/801175920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328004d00 0x7f23440ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 >> 172.21.15.73:0/801175920 conn(0x7f2344069cb0 msgr2=0x7f2344153080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 shutdown_connections 2024-04-05T22:30:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:10.798+0000 7f2349600640 1 -- 172.21.15.73:0/801175920 wait complete. 2024-04-05T22:30:10.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-05T22:30:11.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.023+0000 7f7057e00640 1 Processor -- start 2024-04-05T22:30:11.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.023+0000 7f7057e00640 1 -- start start 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058158860 0x7f70580b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705806b500 con 0x7f705815ed30 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70580b23a0 con 0x7f70580b1f00 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70580ccaf0 con 0x7f7058158860 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f705815ed30 0x7f705806bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48282/0 (socket says 172.21.15.73:48282) 2024-04-05T22:30:11.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7057400640 1 -- 172.21.15.73:0/368311155 learned_addr learned my addr 172.21.15.73:0/368311155 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:11.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.025+0000 7f7056a00640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058158860 0x7f70580b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750134844 0 0) 0x7f705806b500 con 0x7f705815ed30 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c003520 con 0x7f705815ed30 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3764864094 0 0) 0x7f703c003520 con 0x7f705815ed30 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058158860 msgr2=0x7f70580b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058158860 0x7f70580b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 >> v1:172.21.15.157:6789/0 conn(0x7f70580b1f00 legacy=0x7f705815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70580b2680 con 0x7f705815ed30 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f70500019f0 con 0x7f705815ed30 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 >> v1:172.21.15.73:6789/0 conn(0x7f705815ed30 legacy=0x7f705806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0039a0 0x7f703c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c0047c0 0x7f703c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f703c005140 0x7f703c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580ccaf0 con 0x7f703c005140 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c00bec0 con 0x7f703c0039a0 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c00c050 con 0x7f703c0047c0 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.026+0000 7f7057400640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f703c005140 0x7f703c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7056a00640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0039a0 0x7f703c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7057400640 1 -- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c0047c0 msgr2=0x7f703c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7056000640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c0047c0 0x7f703c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7057400640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c0047c0 0x7f703c004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7057400640 1 -- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0039a0 msgr2=0x7f703c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7057400640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0039a0 0x7f703c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.027+0000 7f7057400640 1 -- 172.21.15.73:0/368311155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c003520 con 0x7f703c005140 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.028+0000 7f7057400640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f703c005140 0x7f703c00b980 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f705000a800 tx=0x7f703c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04c7858f7b2c4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.028+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500072d0 con 0x7f703c005140 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.028+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705002f070 con 0x7f703c005140 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.028+0000 7f7055600640 1 -- 172.21.15.73:0/368311155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500076b0 con 0x7f703c005140 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 -- 172.21.15.73:0/368311155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f703c005140 msgr2=0x7f703c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 --2- 172.21.15.73:0/368311155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f703c005140 0x7f703c00b980 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f705000a800 tx=0x7f703c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057400640 1 -- 172.21.15.73:0/368311155 reap_dead start 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 -- 172.21.15.73:0/368311155 shutdown_connections 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 -- 172.21.15.73:0/368311155 >> 172.21.15.73:0/368311155 conn(0x7f70580b06c0 msgr2=0x7f70580b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 -- 172.21.15.73:0/368311155 shutdown_connections 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.029+0000 7f7057e00640 1 -- 172.21.15.73:0/368311155 wait complete. 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.031+0000 7f7057e00640 1 Processor -- start 2024-04-05T22:30:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.031+0000 7f7057e00640 1 -- start start 2024-04-05T22:30:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c005140 0x7f70581403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70580afb50 0x7f70580affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058140920 con 0x7f705815c330 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580b0510 con 0x7f70580afb50 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705815dcb0 con 0x7f703c005140 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55680/0 (socket says 172.21.15.73:55680) 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 -- 172.21.15.73:0/2975868180 learned_addr learned my addr 172.21.15.73:0/2975868180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 -- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c005140 msgr2=0x7f70581403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c005140 0x7f70581403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7057400640 1 -- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70580afb50 msgr2=0x7f70580affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.032+0000 7f7056000640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70580afb50 0x7f70580affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f7057400640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70580afb50 0x7f70580affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f7057400640 1 -- 172.21.15.73:0/2975868180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705815d0a0 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f7057400640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f705002cb50 tx=0x7f7050001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500017b0 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705002ce90 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500070d0 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705815d380 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.033+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705815d940 con 0x7f705815c330 2024-04-05T22:30:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.034+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705804e160 con 0x7f705815c330 2024-04-05T22:30:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.035+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7050031030 con 0x7f705815c330 2024-04-05T22:30:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.036+0000 7f704f400640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 0x7f703406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.036+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70500ab850 con 0x7f705815c330 2024-04-05T22:30:11.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.037+0000 7f7056a00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 0x7f703406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.038+0000 7f7056a00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 0x7f703406d720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f70581f1380 tx=0x7f7040008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:11.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.040+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f70500762a0 con 0x7f705815c330 2024-04-05T22:30:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.302+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7f705804e160 con 0x7f705815c330 2024-04-05T22:30:11.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.906+0000 7f704f400640 1 -- 172.21.15.73:0/2975868180 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v481) v1 ==== 152+0+0 (secure 0 0 0) 0x7f70500798c0 con 0x7f705815c330 2024-04-05T22:30:11.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 msgr2=0x7f703406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 0x7f703406d720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f70581f1380 tx=0x7f7040008040 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 msgr2=0x7f705815c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f705002cb50 tx=0x7f7050001d80 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 shutdown_connections 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f703406b260 0x7f703406d720 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f705815c330 0x7f705815c7b0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70580afb50 0x7f70580affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 --2- 172.21.15.73:0/2975868180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c005140 0x7f70581403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 >> 172.21.15.73:0/2975868180 conn(0x7f70580b06c0 msgr2=0x7f70580af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.910+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 shutdown_connections 2024-04-05T22:30:11.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:11.911+0000 7f7057e00640 1 -- 172.21.15.73:0/2975868180 wait complete. 2024-04-05T22:30:11.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:30:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.148+0000 7ff86f200640 1 Processor -- start 2024-04-05T22:30:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.148+0000 7ff86f200640 1 -- start start 2024-04-05T22:30:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.149+0000 7ff86f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.149+0000 7ff86f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff87015e580 0x7ff87006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.149+0000 7ff86f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87006bc50 con 0x7ff87015e9a0 2024-04-05T22:30:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.149+0000 7ff86f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87015f530 con 0x7ff87015e580 2024-04-05T22:30:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.149+0000 7ff86f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87015ef70 con 0x7ff87015a850 2024-04-05T22:30:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.150+0000 7ff86de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.150+0000 7ff86de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:37832/0 (socket says 172.21.15.73:37832) 2024-04-05T22:30:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.150+0000 7ff86de00640 1 -- 172.21.15.73:0/4012098355 learned_addr learned my addr 172.21.15.73:0/4012098355 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.150+0000 7ff867e00640 1 --2- 172.21.15.73:0/4012098355 >> v2:172.21.15.157:3300/0 conn(0x7ff87015e580 0x7ff87006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.151+0000 7ff86de00640 1 -- 172.21.15.73:0/4012098355 >> v2:172.21.15.157:3300/0 conn(0x7ff87015e580 msgr2=0x7ff87006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:12.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.151+0000 7ff86de00640 1 --2- 172.21.15.73:0/4012098355 >> v2:172.21.15.157:3300/0 conn(0x7ff87015e580 0x7ff87006ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.151+0000 7ff86de00640 1 -- 172.21.15.73:0/4012098355 >> v1:172.21.15.157:6789/0 conn(0x7ff87015e9a0 legacy=0x7ff8700b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.151+0000 7ff86de00640 1 -- 172.21.15.73:0/4012098355 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87015f250 con 0x7ff87015a850 2024-04-05T22:30:12.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.153+0000 7ff86de00640 1 --2- 172.21.15.73:0/4012098355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff86002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89678ca134e76206 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86d400640 1 -- 172.21.15.73:0/4012098355 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860030840 con 0x7ff87015a850 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86d400640 1 -- 172.21.15.73:0/4012098355 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8600309a0 con 0x7ff87015a850 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86d400640 1 -- 172.21.15.73:0/4012098355 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860030d30 con 0x7ff87015a850 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 -- 172.21.15.73:0/4012098355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 msgr2=0x7ff87015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 --2- 172.21.15.73:0/4012098355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff86002b980 comp rx=0 tx=0).stop 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 -- 172.21.15.73:0/4012098355 shutdown_connections 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 --2- 172.21.15.73:0/4012098355 >> v2:172.21.15.157:3300/0 conn(0x7ff87015e580 0x7ff87006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 --2- 172.21.15.73:0/4012098355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015a850 0x7ff87015ac30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 -- 172.21.15.73:0/4012098355 >> 172.21.15.73:0/4012098355 conn(0x7ff870069cb0 msgr2=0x7ff87006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 -- 172.21.15.73:0/4012098355 shutdown_connections 2024-04-05T22:30:12.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.154+0000 7ff86f200640 1 -- 172.21.15.73:0/4012098355 wait complete. 2024-04-05T22:30:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.156+0000 7ff86f200640 1 Processor -- start 2024-04-05T22:30:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.156+0000 7ff86f200640 1 -- start start 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff87015a850 0x7ff870140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015e580 0x7ff870156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870140950 con 0x7ff87015e9a0 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870142c60 con 0x7ff87015a850 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff870143ac0 con 0x7ff87015e580 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55682/0 (socket says 172.21.15.73:55682) 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.157+0000 7ff86e800640 1 -- 172.21.15.73:0/2021043757 learned_addr learned my addr 172.21.15.73:0/2021043757 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff867e00640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015e580 0x7ff870156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86de00640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff87015a850 0x7ff870140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 -- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015e580 msgr2=0x7ff870156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015e580 0x7ff870156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 -- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff87015a850 msgr2=0x7ff870140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff87015a850 0x7ff870140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 -- 172.21.15.73:0/2021043757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600094d0 con 0x7ff87015e9a0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff86e800640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c6e0 tx=0x7ff86800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.158+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868003b70 con 0x7ff87015e9a0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.159+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868003cf0 con 0x7ff87015e9a0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.159+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868005310 con 0x7ff87015e9a0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.159+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff870142eb0 con 0x7ff87015e9a0 2024-04-05T22:30:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.159+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff870143470 con 0x7ff87015e9a0 2024-04-05T22:30:12.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.160+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87004e160 con 0x7ff87015e9a0 2024-04-05T22:30:12.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.161+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7ff868005470 con 0x7ff87015e9a0 2024-04-05T22:30:12.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.161+0000 7ff866000640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 0x7ff84c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:12.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.162+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff868089910 con 0x7ff87015e9a0 2024-04-05T22:30:12.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.162+0000 7ff86de00640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 0x7ff84c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:12.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.163+0000 7ff86de00640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 0x7ff84c06d6f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff860036000 tx=0x7ff860008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:12.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.166+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff868054470 con 0x7ff87015e9a0 2024-04-05T22:30:12.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.357+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8700b1f80 con 0x7ff87015e9a0 2024-04-05T22:30:12.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.366+0000 7ff866000640 1 -- 172.21.15.73:0/2021043757 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff868003e60 con 0x7ff87015e9a0 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 msgr2=0x7ff84c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 0x7ff84c06d6f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff860036000 tx=0x7ff860008030 comp rx=0 tx=0).stop 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 msgr2=0x7ff87015c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c6e0 tx=0x7ff86800cba0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 shutdown_connections 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff84c06b230 0x7ff84c06d6f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff87015e9a0 0x7ff87015c930 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff87015e580 0x7ff870156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 --2- 172.21.15.73:0/2021043757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff87015a850 0x7ff870140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:12.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 >> 172.21.15.73:0/2021043757 conn(0x7ff870069cb0 msgr2=0x7ff8700ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:12.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 shutdown_connections 2024-04-05T22:30:12.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:12.369+0000 7ff86f200640 1 -- 172.21.15.73:0/2021043757 wait complete. 2024-04-05T22:30:12.386 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:30:12.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:12.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:12.608 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:30:12.608 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:12.608 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:12.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:12.666 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:12.666 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:12.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:12.724 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:12.724 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:12.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:12.780 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:12.780 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:12.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:12.838 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:30:12.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:13.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.056+0000 7f25fd800640 1 Processor -- start 2024-04-05T22:30:13.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.056+0000 7f25fd800640 1 -- start start 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.057+0000 7f25fd800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25f806cb80 0x7f25f806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25fd800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f806f5b0 con 0x7f25f815e850 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25fd800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f8070410 con 0x7f25f81587d0 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25fd800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f806d500 con 0x7f25f806cb80 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25f7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25f815e850 0x7f25f80708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48298/0 (socket says 172.21.15.73:48298) 2024-04-05T22:30:13.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25f7400640 1 -- 172.21.15.73:0/2084892071 learned_addr learned my addr 172.21.15.73:0/2084892071 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:13.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25ee000640 1 --2- 172.21.15.73:0/2084892071 >> v2:172.21.15.157:3300/0 conn(0x7f25f806cb80 0x7f25f806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455863579 0 0) 0x7f25f806f5b0 con 0x7f25f815e850 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.058+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25dc003520 con 0x7f25f815e850 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145694696 0 0) 0x7f25dc003520 con 0x7f25f815e850 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 >> v2:172.21.15.157:3300/0 conn(0x7f25f806cb80 msgr2=0x7f25f806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 --2- 172.21.15.73:0/2084892071 >> v2:172.21.15.157:3300/0 conn(0x7f25f806cb80 0x7f25f806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 >> v1:172.21.15.157:6789/0 conn(0x7f25f81587d0 legacy=0x7f25f80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80ccef0 con 0x7f25f815e850 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f25f00019f0 con 0x7f25f815e850 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 >> v1:172.21.15.73:6789/0 conn(0x7f25f815e850 legacy=0x7f25f80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 0x7f25dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc0047c0 0x7f25dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25dc005180 0x7f25dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f806d500 con 0x7f25dc003970 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc00c020 con 0x7f25dc0047c0 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc00c1e0 con 0x7f25dc005180 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.059+0000 7f25f6a00640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 0x7f25dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f6a00640 1 -- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25dc005180 msgr2=0x7f25dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f6a00640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25dc005180 0x7f25dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f6a00640 1 -- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc0047c0 msgr2=0x7f25dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f6a00640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc0047c0 0x7f25dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f6a00640 1 -- 172.21.15.73:0/2084892071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc003520 con 0x7f25dc003970 2024-04-05T22:30:13.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.060+0000 7f25f7400640 1 -- 172.21.15.73:0/2084892071 reap_dead start 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.061+0000 7f25f6a00640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 0x7f25dc003d90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f25e8001750 tx=0x7f25dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e195cf1e5e4df233 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.061+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e80052d0 con 0x7f25dc003970 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.061+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e802e050 con 0x7f25dc003970 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.061+0000 7f25f6000640 1 -- 172.21.15.73:0/2084892071 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8003830 con 0x7f25dc003970 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 -- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 msgr2=0x7f25dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 0x7f25dc003d90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f25e8001750 tx=0x7f25dc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 -- 172.21.15.73:0/2084892071 shutdown_connections 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 --2- 172.21.15.73:0/2084892071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc003970 0x7f25dc003d90 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 -- 172.21.15.73:0/2084892071 >> 172.21.15.73:0/2084892071 conn(0x7f25f80ce8e0 msgr2=0x7f25f80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 -- 172.21.15.73:0/2084892071 shutdown_connections 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.062+0000 7f25fd800640 1 -- 172.21.15.73:0/2084892071 wait complete. 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.064+0000 7f25fd800640 1 Processor -- start 2024-04-05T22:30:13.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.064+0000 7f25fd800640 1 -- start start 2024-04-05T22:30:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc003970 0x7f25f815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f815d080 0x7f25f80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f815c6b0 con 0x7f25f815d7f0 2024-04-05T22:30:13.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81f3090 con 0x7f25dc003970 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25fd800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81f3370 con 0x7f25f815d080 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25ee000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25ee000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55698/0 (socket says 172.21.15.73:55698) 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25ee000640 1 -- 172.21.15.73:0/3304149374 learned_addr learned my addr 172.21.15.73:0/3304149374 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25f6a00640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc003970 0x7f25f815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.065+0000 7f25f7400640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f815d080 0x7f25f80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 -- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f815d080 msgr2=0x7f25f80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f815d080 0x7f25f80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 -- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc003970 msgr2=0x7f25f815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc003970 0x7f25f815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 -- 172.21.15.73:0/3304149374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f81f3650 con 0x7f25f815d7f0 2024-04-05T22:30:13.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ee000640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e770 tx=0x7f25e000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000c730 con 0x7f25f815d7f0 2024-04-05T22:30:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e000c890 con 0x7f25f815d7f0 2024-04-05T22:30:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cc40 con 0x7f25f815d7f0 2024-04-05T22:30:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.066+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f815c340 con 0x7f25f815d7f0 2024-04-05T22:30:13.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.067+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f8159ff0 con 0x7f25f815d7f0 2024-04-05T22:30:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.068+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f8066af0 con 0x7f25f815d7f0 2024-04-05T22:30:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.069+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f25e0013470 con 0x7f25f815d7f0 2024-04-05T22:30:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.069+0000 7f25ef400640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 0x7f25d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.070+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25e0089db0 con 0x7f25f815d7f0 2024-04-05T22:30:13.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.070+0000 7f25f6a00640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 0x7f25d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.072+0000 7f25f6a00640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 0x7f25d406d6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25e8033000 tx=0x7f25e8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.074+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f25e0054910 con 0x7f25f815d7f0 2024-04-05T22:30:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.305+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25f8068160 con 0x7f25f815d7f0 2024-04-05T22:30:13.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.306+0000 7f25ef400640 1 -- 172.21.15.73:0/3304149374 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f25e0021de0 con 0x7f25f815d7f0 2024-04-05T22:30:13.306 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:13.306 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5675,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/955642581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":955642581},{"type":"v1","addr":"172.21.15.157:6835","nonce":955642581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3373379007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3373379007},{"type":"v1","addr":"172.21.15.157:6834","nonce":3373379007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/1904760581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1904760581},{"type":"v1","addr":"172.21.15.73:6834","nonce":1904760581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3810711073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3810711073},{"type":"v1","addr":"172.21.15.73:6835","nonce":3810711073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[]} 2024-04-05T22:30:13.306 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 85 2024-04-05T22:30:13.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 msgr2=0x7f25d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 0x7f25d406d6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25e8033000 tx=0x7f25e8009030 comp rx=0 tx=0).stop 2024-04-05T22:30:13.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 msgr2=0x7f25f80b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e770 tx=0x7f25e000ec30 comp rx=0 tx=0).stop 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 shutdown_connections 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25d406b230 0x7f25d406d6f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25f815d080 0x7f25f80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25f815d7f0 0x7f25f80b1600 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 --2- 172.21.15.73:0/3304149374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc003970 0x7f25f815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 >> 172.21.15.73:0/3304149374 conn(0x7f25f80ce8e0 msgr2=0x7f25f806c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 shutdown_connections 2024-04-05T22:30:13.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.309+0000 7f25fd800640 1 -- 172.21.15.73:0/3304149374 wait complete. 2024-04-05T22:30:13.328 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:30:13.328 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:30:13.328 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:30:13.328 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:13.328 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:30:13.328 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:30:13.329 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:30:13.329 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:13.329 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:30:13.329 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:30:13.329 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:30:13.329 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:13.329 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:30:13.330 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:30:13.330 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:30:13.330 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:13.346 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:30:13.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:30:13.347 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:30:13.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:30:13.350 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:30:13.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:30:13.356 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:30:13.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:30:13.393 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:30:13.394 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:13.399 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:30:13.399 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:13.402 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:30:13.402 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:13.409 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:30:13.409 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:13.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:30:13.548 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:30:13.548 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:30:13.563 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:30:13.578 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:30:13.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.594+0000 7fdd7f000640 1 Processor -- start 2024-04-05T22:30:13.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.595+0000 7fdd7f000640 1 -- start start 2024-04-05T22:30:13.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.596+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.596+0000 7fdd7f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd780cac60 0x7fdd780abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.596+0000 7fdd7f000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd780cb220 con 0x7fdd780ac500 2024-04-05T22:30:13.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.596+0000 7fdd7f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cd220 con 0x7fdd780cac60 2024-04-05T22:30:13.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.596+0000 7fdd7f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780caa60 con 0x7fdd780ad040 2024-04-05T22:30:13.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd780cac60 0x7fdd780abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:37904/0 (socket says 172.21.15.73:37904) 2024-04-05T22:30:13.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 -- 172.21.15.73:0/195059302 learned_addr learned my addr 172.21.15.73:0/195059302 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:13.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 -- 172.21.15.73:0/195059302 >> v2:172.21.15.157:3300/0 conn(0x7fdd780cac60 msgr2=0x7fdd780abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:13.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 --2- 172.21.15.73:0/195059302 >> v2:172.21.15.157:3300/0 conn(0x7fdd780cac60 0x7fdd780abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 -- 172.21.15.73:0/195059302 >> v1:172.21.15.157:6789/0 conn(0x7fdd780ac500 legacy=0x7fdd7806a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.597+0000 7fdd77e00640 1 -- 172.21.15.73:0/195059302 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780cb500 con 0x7fdd780ad040 2024-04-05T22:30:13.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.598+0000 7fdd77e00640 1 --2- 172.21.15.73:0/195059302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b920 tx=0x7fdd6802bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fadf351cbe629 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.598+0000 7fdd76a00640 1 -- 172.21.15.73:0/195059302 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6802ccf0 con 0x7fdd780ad040 2024-04-05T22:30:13.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.598+0000 7fdd76a00640 1 -- 172.21.15.73:0/195059302 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6802ce50 con 0x7fdd780ad040 2024-04-05T22:30:13.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.598+0000 7fdd76a00640 1 -- 172.21.15.73:0/195059302 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68034660 con 0x7fdd780ad040 2024-04-05T22:30:13.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 -- 172.21.15.73:0/195059302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 msgr2=0x7fdd780ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 --2- 172.21.15.73:0/195059302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b920 tx=0x7fdd6802bf40 comp rx=0 tx=0).stop 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 -- 172.21.15.73:0/195059302 shutdown_connections 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 --2- 172.21.15.73:0/195059302 >> v2:172.21.15.157:3300/0 conn(0x7fdd780cac60 0x7fdd780abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 --2- 172.21.15.73:0/195059302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780ca520 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 -- 172.21.15.73:0/195059302 >> 172.21.15.73:0/195059302 conn(0x7fdd780cf190 msgr2=0x7fdd780cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 -- 172.21.15.73:0/195059302 shutdown_connections 2024-04-05T22:30:13.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.599+0000 7fdd7f000640 1 -- 172.21.15.73:0/195059302 wait complete. 2024-04-05T22:30:13.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.601+0000 7fdd7f000640 1 Processor -- start 2024-04-05T22:30:13.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.601+0000 7fdd7f000640 1 -- start start 2024-04-05T22:30:13.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd780cac60 0x7fdd78152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7809c370 0x7fdd780ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c8530 con 0x7fdd780cac60 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c87d0 con 0x7fdd7809c370 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd7f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780b23e0 con 0x7fdd780ad040 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd77400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd780cac60 0x7fdd78152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:37908/0 (socket says 172.21.15.73:37908) 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.602+0000 7fdd77e00640 1 -- 172.21.15.73:0/4202242569 learned_addr learned my addr 172.21.15.73:0/4202242569 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:13.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd7d000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7809c370 0x7fdd780ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 -- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7809c370 msgr2=0x7fdd780ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7809c370 0x7fdd780ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 -- 172.21.15.73:0/4202242569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd780cac60 msgr2=0x7fdd78152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:13.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd780cac60 0x7fdd78152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:13.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 -- 172.21.15.73:0/4202242569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6800b580 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.603+0000 7fdd77e00640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b2c0 tx=0x7fdd680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.604+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6803d600 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.604+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6803d780 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.604+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6803db30 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.604+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd780b26c0 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.604+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78152dc0 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.605+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd780b2240 con 0x7fdd780ad040 2024-04-05T22:30:13.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.606+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fdd6803b040 con 0x7fdd780ad040 2024-04-05T22:30:13.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.606+0000 7fdd6fe00640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 0x7fdd5806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:13.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.607+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd680abec0 con 0x7fdd780ad040 2024-04-05T22:30:13.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.607+0000 7fdd77400640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 0x7fdd5806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:13.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.607+0000 7fdd77400640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 0x7fdd5806d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdd780c25c0 tx=0x7fdd6400f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:13.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.612+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdd780b2240 con 0x7fdd780ad040 2024-04-05T22:30:13.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:13.811+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdd780b2240 con 0x7fdd780ad040 2024-04-05T22:30:14.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.602+0000 7fdd6fe00640 1 -- 172.21.15.73:0/4202242569 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdd780b2240 con 0x7fdd780ad040 2024-04-05T22:30:14.603 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 msgr2=0x7fdd5806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 0x7fdd5806d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdd780c25c0 tx=0x7fdd6400f040 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 msgr2=0x7fdd780c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b2c0 tx=0x7fdd680046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 shutdown_connections 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdd5806b230 0x7fdd5806d6f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7809c370 0x7fdd780ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd780cac60 0x7fdd78152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 --2- 172.21.15.73:0/4202242569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd780ad040 0x7fdd780c7ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.606+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 >> 172.21.15.73:0/4202242569 conn(0x7fdd780cf190 msgr2=0x7fdd7815c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.607+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 shutdown_connections 2024-04-05T22:30:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.607+0000 7fdd7f000640 1 -- 172.21.15.73:0/4202242569 wait complete. 2024-04-05T22:30:14.627 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:30:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.832+0000 7f608f200640 1 Processor -- start 2024-04-05T22:30:14.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.832+0000 7f608f200640 1 -- start start 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f6090070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60900b2b90 0x7f609006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609006d3e0 con 0x7f60900b1b50 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609006d610 con 0x7f60900b2b90 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6090070230 con 0x7f60900b20a0 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60900b2b90 0x7f609006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608e800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60900b2b90 0x7f609006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55734/0 (socket says 172.21.15.73:55734) 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.834+0000 7f608e800640 1 -- 172.21.15.73:0/4080697058 learned_addr learned my addr 172.21.15.73:0/4080697058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.835+0000 7f608e800640 1 -- 172.21.15.73:0/4080697058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 msgr2=0x7f6090070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.835+0000 7f608e800640 1 --2- 172.21.15.73:0/4080697058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f6090070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.835+0000 7f608e800640 1 -- 172.21.15.73:0/4080697058 >> v1:172.21.15.157:6789/0 conn(0x7f60900b1b50 legacy=0x7f60900b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.835+0000 7f608e800640 1 -- 172.21.15.73:0/4080697058 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090070510 con 0x7f60900b2b90 2024-04-05T22:30:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.836+0000 7f608e800640 1 --2- 172.21.15.73:0/4080697058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b2b90 0x7f609006cea0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6088009e50 tx=0x7f608802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a5ca486070407a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.836+0000 7f608ca00640 1 -- 172.21.15.73:0/4080697058 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608802c950 con 0x7f60900b2b90 2024-04-05T22:30:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.836+0000 7f608ca00640 1 -- 172.21.15.73:0/4080697058 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608802cab0 con 0x7f60900b2b90 2024-04-05T22:30:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.836+0000 7f608ca00640 1 -- 172.21.15.73:0/4080697058 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608802ce40 con 0x7f60900b2b90 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 -- 172.21.15.73:0/4080697058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b2b90 msgr2=0x7f609006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 --2- 172.21.15.73:0/4080697058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b2b90 0x7f609006cea0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6088009e50 tx=0x7f608802bdb0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 -- 172.21.15.73:0/4080697058 shutdown_connections 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 --2- 172.21.15.73:0/4080697058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b2b90 0x7f609006cea0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 --2- 172.21.15.73:0/4080697058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f6090070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 -- 172.21.15.73:0/4080697058 >> 172.21.15.73:0/4080697058 conn(0x7f60900b06c0 msgr2=0x7f60900b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 -- 172.21.15.73:0/4080697058 shutdown_connections 2024-04-05T22:30:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.837+0000 7f608f200640 1 -- 172.21.15.73:0/4080697058 wait complete. 2024-04-05T22:30:14.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.839+0000 7f608f200640 1 Processor -- start 2024-04-05T22:30:14.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.839+0000 7f608f200640 1 -- start start 2024-04-05T22:30:14.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f60900cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60900b2b90 0x7f60900cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60901597f0 con 0x7f60900b1b50 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6090147c80 con 0x7f60900b2b90 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6090158f50 con 0x7f60900b20a0 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55748/0 (socket says 172.21.15.73:55748) 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 -- 172.21.15.73:0/2101861760 learned_addr learned my addr 172.21.15.73:0/2101861760 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608d400640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f60900cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 -- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 msgr2=0x7f60900cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f60900cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 -- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60900b2b90 msgr2=0x7f60900cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:14.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608e800640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60900b2b90 0x7f60900cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60900b2b90 0x7f60900cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.840+0000 7f608de00640 1 -- 172.21.15.73:0/2101861760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088009b30 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f608de00640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f608000b4a0 tx=0x7f608000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080010610 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080010770 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080010b20 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6090159230 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.841+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60901546a0 con 0x7f60900b1b50 2024-04-05T22:30:14.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.842+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6090154500 con 0x7f60900b1b50 2024-04-05T22:30:14.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.843+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f6080010c80 con 0x7f60900b1b50 2024-04-05T22:30:14.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.844+0000 7f6086000640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 0x7f607006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:14.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.845+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6080089820 con 0x7f60900b1b50 2024-04-05T22:30:14.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.845+0000 7f608d400640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 0x7f607006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:14.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.846+0000 7f608d400640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 0x7f607006d6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f60900af890 tx=0x7f607c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:14.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:14.849+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6090154500 con 0x7f60900b1b50 2024-04-05T22:30:15.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.111+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6090154500 con 0x7f60900b1b50 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.112+0000 7f6086000640 1 -- 172.21.15.73:0/2101861760 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6090154500 con 0x7f60900b1b50 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:30:15.112 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:15.113 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.114 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:30:15.117 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:30:15.120 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:15.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:30:15.125 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:30:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 msgr2=0x7f607006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 0x7f607006d6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f60900af890 tx=0x7f607c008040 comp rx=0 tx=0).stop 2024-04-05T22:30:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 msgr2=0x7f6090147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f608000b4a0 tx=0x7f608000b960 comp rx=0 tx=0).stop 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 shutdown_connections 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f607006b230 0x7f607006d6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60900b2b90 0x7f60900cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60900b20a0 0x7f60900cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 --2- 172.21.15.73:0/2101861760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60900b1b50 0x7f6090147740 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 >> 172.21.15.73:0/2101861760 conn(0x7f60900b06c0 msgr2=0x7f60900ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 shutdown_connections 2024-04-05T22:30:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.115+0000 7f608f200640 1 -- 172.21.15.73:0/2101861760 wait complete. 2024-04-05T22:30:15.133 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:15.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.337+0000 7fe0c2200640 1 Processor -- start 2024-04-05T22:30:15.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.337+0000 7fe0c2200640 1 -- start start 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0c2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0bc1541a0 0x7fe0bc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc06d710 0x7fe0bc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0c2200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc15aa60 con 0x7fe0bc15aef0 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0c2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc0b3b60 con 0x7fe0bc1541a0 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0c2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0701c0 con 0x7fe0bc06d710 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0bb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0bc1541a0 0x7fe0bc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0bb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0bc1541a0 0x7fe0bc0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55762/0 (socket says 172.21.15.73:55762) 2024-04-05T22:30:15.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0bb400640 1 -- 172.21.15.73:0/227754463 learned_addr learned my addr 172.21.15.73:0/227754463 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:15.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.339+0000 7fe0baa00640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc06d710 0x7fe0bc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.340+0000 7fe0bb400640 1 -- 172.21.15.73:0/227754463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc06d710 msgr2=0x7fe0bc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:15.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.340+0000 7fe0bb400640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc06d710 0x7fe0bc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.340+0000 7fe0bb400640 1 -- 172.21.15.73:0/227754463 >> v1:172.21.15.157:6789/0 conn(0x7fe0bc15aef0 legacy=0x7fe0bc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:15.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.340+0000 7fe0bb400640 1 -- 172.21.15.73:0/227754463 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0b3880 con 0x7fe0bc1541a0 2024-04-05T22:30:15.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.341+0000 7fe0bb400640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc1541a0 0x7fe0bc0b3050 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00d150 tx=0x7fe0ac02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1f3f9908945ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.341+0000 7fe0ba000640 1 -- 172.21.15.73:0/227754463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac02ea80 con 0x7fe0bc1541a0 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.341+0000 7fe0ba000640 1 -- 172.21.15.73:0/227754463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac02ebe0 con 0x7fe0bc1541a0 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.341+0000 7fe0ba000640 1 -- 172.21.15.73:0/227754463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac009600 con 0x7fe0bc1541a0 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 -- 172.21.15.73:0/227754463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc1541a0 msgr2=0x7fe0bc0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc1541a0 0x7fe0bc0b3050 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00d150 tx=0x7fe0ac02e0e0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 -- 172.21.15.73:0/227754463 shutdown_connections 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc06d710 0x7fe0bc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 --2- 172.21.15.73:0/227754463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc1541a0 0x7fe0bc0b3050 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 -- 172.21.15.73:0/227754463 >> 172.21.15.73:0/227754463 conn(0x7fe0bc0ce8e0 msgr2=0x7fe0bc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 -- 172.21.15.73:0/227754463 shutdown_connections 2024-04-05T22:30:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.342+0000 7fe0c2200640 1 -- 172.21.15.73:0/227754463 wait complete. 2024-04-05T22:30:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.343+0000 7fe0c2200640 1 Processor -- start 2024-04-05T22:30:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.344+0000 7fe0c2200640 1 -- start start 2024-04-05T22:30:15.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.344+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 0x7fe0bc13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.344+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc1541a0 0x7fe0bc140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.344+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0c2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc13d650 con 0x7fe0bc15aef0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0c2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc13d930 con 0x7fe0bc06d710 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0c2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc13dc10 con 0x7fe0bc1541a0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55776/0 (socket says 172.21.15.73:55776) 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 -- 172.21.15.73:0/1756635756 learned_addr learned my addr 172.21.15.73:0/1756635756 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bb400640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 0x7fe0bc13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0baa00640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc1541a0 0x7fe0bc140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 -- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc1541a0 msgr2=0x7fe0bc140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc1541a0 0x7fe0bc140b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 -- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 msgr2=0x7fe0bc13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 0x7fe0bc13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bbe00640 1 -- 172.21.15.73:0/1756635756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac00bd40 con 0x7fe0bc15aef0 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.345+0000 7fe0bb400640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 0x7fe0bc13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:15.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0bbe00640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c410 tx=0x7fe0b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011790 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b40118f0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011ce0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc13def0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.346+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc13e4b0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.347+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc066af0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.348+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fe0b401c460 con 0x7fe0bc15aef0 2024-04-05T22:30:15.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.349+0000 7fe0b3400640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 0x7fe09806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.349+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0b4089930 con 0x7fe0bc15aef0 2024-04-05T22:30:15.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.350+0000 7fe0bb400640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 0x7fe09806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.351+0000 7fe0bb400640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 0x7fe09806d6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac037000 tx=0x7fe0ac032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.353+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe0b4054450 con 0x7fe0bc15aef0 2024-04-05T22:30:15.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.546+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0bc13e6d0 con 0x7fe0bc15aef0 2024-04-05T22:30:15.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.547+0000 7fe0b3400640 1 -- 172.21.15.73:0/1756635756 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+12399 (secure 0 0 0) 0x7fe0b4057a70 con 0x7fe0bc15aef0 2024-04-05T22:30:15.547 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:15.547 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":144,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:14.593986+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.549+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 msgr2=0x7fe09806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.549+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 0x7fe09806d6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac037000 tx=0x7fe0ac032000 comp rx=0 tx=0).stop 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.549+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 msgr2=0x7fe0bc0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.549+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c410 tx=0x7fe0b400c8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 shutdown_connections 2024-04-05T22:30:15.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe09806b230 0x7fe09806d6f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0bc15aef0 0x7fe0bc0cd860 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0bc1541a0 0x7fe0bc140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 --2- 172.21.15.73:0/1756635756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0bc06d710 0x7fe0bc13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 >> 172.21.15.73:0/1756635756 conn(0x7fe0bc0ce8e0 msgr2=0x7fe0bc15b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 shutdown_connections 2024-04-05T22:30:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.550+0000 7fe0c2200640 1 -- 172.21.15.73:0/1756635756 wait complete. 2024-04-05T22:30:15.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:15.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.782+0000 7fd29de00640 1 Processor -- start 2024-04-05T22:30:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.782+0000 7fd29de00640 1 -- start start 2024-04-05T22:30:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd29de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 0x7fd29815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd29de00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29815ae50 con 0x7fd29815dbc0 2024-04-05T22:30:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd29de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29815e3b0 con 0x7fd29815aa70 2024-04-05T22:30:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd29de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980ac9e0 con 0x7fd298071670 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd297400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd29815dbc0 0x7fd29806ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48316/0 (socket says 172.21.15.73:48316) 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd297400640 1 -- 172.21.15.73:0/187557348 learned_addr learned my addr 172.21.15.73:0/187557348 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.784+0000 7fd296000640 1 --2- 172.21.15.73:0/187557348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 0x7fd29815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.785+0000 7fd295600640 1 -- 172.21.15.73:0/187557348 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642074579 0 0) 0x7fd29815ae50 con 0x7fd29815dbc0 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.785+0000 7fd295600640 1 -- 172.21.15.73:0/187557348 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c003520 con 0x7fd29815dbc0 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.785+0000 7fd296000640 1 -- 172.21.15.73:0/187557348 >> v1:172.21.15.157:6789/0 conn(0x7fd29815aa70 legacy=0x7fd298070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.785+0000 7fd296000640 1 -- 172.21.15.73:0/187557348 >> v1:172.21.15.73:6789/0 conn(0x7fd29815dbc0 legacy=0x7fd29806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.785+0000 7fd296000640 1 -- 172.21.15.73:0/187557348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980accc0 con 0x7fd298071670 2024-04-05T22:30:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.786+0000 7fd296000640 1 --2- 172.21.15.73:0/187557348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 0x7fd29815d680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2840097d0 tx=0x7fd28402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d044458723539424 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.787+0000 7fd295600640 1 -- 172.21.15.73:0/187557348 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284031890 con 0x7fd298071670 2024-04-05T22:30:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.787+0000 7fd295600640 1 -- 172.21.15.73:0/187557348 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2840319f0 con 0x7fd298071670 2024-04-05T22:30:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.787+0000 7fd295600640 1 -- 172.21.15.73:0/187557348 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284031d60 con 0x7fd298071670 2024-04-05T22:30:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 -- 172.21.15.73:0/187557348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 msgr2=0x7fd29815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 --2- 172.21.15.73:0/187557348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 0x7fd29815d680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2840097d0 tx=0x7fd28402b930 comp rx=0 tx=0).stop 2024-04-05T22:30:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 -- 172.21.15.73:0/187557348 shutdown_connections 2024-04-05T22:30:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 --2- 172.21.15.73:0/187557348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd298071670 0x7fd29815d680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 -- 172.21.15.73:0/187557348 >> 172.21.15.73:0/187557348 conn(0x7fd298069cb0 msgr2=0x7fd29806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 -- 172.21.15.73:0/187557348 shutdown_connections 2024-04-05T22:30:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.788+0000 7fd29de00640 1 -- 172.21.15.73:0/187557348 wait complete. 2024-04-05T22:30:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.790+0000 7fd29de00640 1 Processor -- start 2024-04-05T22:30:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.791+0000 7fd29de00640 1 -- start start 2024-04-05T22:30:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815aa70 0x7fd29815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815dbc0 0x7fd29815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980b0200 con 0x7fd298071670 2024-04-05T22:30:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981478b0 con 0x7fd29815dbc0 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd29de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd298153a70 con 0x7fd29815aa70 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd296a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd296a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55784/0 (socket says 172.21.15.73:55784) 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd296a00640 1 -- 172.21.15.73:0/1261774131 learned_addr learned my addr 172.21.15.73:0/1261774131 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd296000640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815aa70 0x7fd29815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.792+0000 7fd297400640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815dbc0 0x7fd29815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 -- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815aa70 msgr2=0x7fd29815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815aa70 0x7fd29815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 -- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815dbc0 msgr2=0x7fd29815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815dbc0 0x7fd29815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 -- 172.21.15.73:0/1261774131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284009400 con 0x7fd298071670 2024-04-05T22:30:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd296a00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c430 tx=0x7fd28800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880052d0 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288005450 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288007260 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.793+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298153140 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.794+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd298153700 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.795+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298068160 con 0x7fd298071670 2024-04-05T22:30:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.796+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fd2880073c0 con 0x7fd298071670 2024-04-05T22:30:15.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.797+0000 7fd28f400640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 0x7fd27806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:15.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.797+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd288088fa0 con 0x7fd298071670 2024-04-05T22:30:15.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.798+0000 7fd296000640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 0x7fd27806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.799+0000 7fd296000640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 0x7fd27806d6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd284036000 tx=0x7fd284008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:15.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.802+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd288053b90 con 0x7fd298071670 2024-04-05T22:30:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.998+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd29815d7b0 con 0x7fd298071670 2024-04-05T22:30:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:15.999+0000 7fd28f400640 1 -- 172.21.15.73:0/1261774131 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2880571b0 con 0x7fd298071670 2024-04-05T22:30:15.999 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:16.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 msgr2=0x7fd27806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 0x7fd27806d6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd284036000 tx=0x7fd284008030 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 msgr2=0x7fd2980afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c430 tx=0x7fd28800c8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 shutdown_connections 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd27806b230 0x7fd27806d6f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815dbc0 0x7fd29815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815aa70 0x7fd29815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 --2- 172.21.15.73:0/1261774131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd298071670 0x7fd2980afcc0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 >> 172.21.15.73:0/1261774131 conn(0x7fd298069cb0 msgr2=0x7fd2981553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 shutdown_connections 2024-04-05T22:30:16.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.002+0000 7fd29de00640 1 -- 172.21.15.73:0/1261774131 wait complete. 2024-04-05T22:30:16.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:30:16.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.226+0000 7f196f000640 1 Processor -- start 2024-04-05T22:30:16.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.226+0000 7f196f000640 1 -- start start 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f196806d060 0x7f19680b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 0x7f19680b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680b3ea0 con 0x7f196815aaa0 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196815e2c0 con 0x7f196806d060 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196815d500 con 0x7f1968157560 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f196806d060 0x7f19680b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196d200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f196815aaa0 0x7f1968156e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45148/0 (socket says 172.21.15.73:45148) 2024-04-05T22:30:16.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.228+0000 7f196d200640 1 -- 172.21.15.73:0/2126665316 learned_addr learned my addr 172.21.15.73:0/2126665316 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:16.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196e600640 1 --2- 172.21.15.73:0/2126665316 >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 0x7f19680b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196dc00640 1 -- 172.21.15.73:0/2126665316 >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 msgr2=0x7f19680b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196dc00640 1 --2- 172.21.15.73:0/2126665316 >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 0x7f19680b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196dc00640 1 -- 172.21.15.73:0/2126665316 >> v1:172.21.15.73:6789/0 conn(0x7f196815aaa0 legacy=0x7f1968156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196dc00640 1 -- 172.21.15.73:0/2126665316 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815d7e0 con 0x7f196806d060 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f1967e00640 1 -- 172.21.15.73:0/2126665316 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472418706 0 0) 0x7f19680b3ea0 con 0x7f196815aaa0 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.229+0000 7f196e600640 1 --2- 172.21.15.73:0/2126665316 >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 0x7f19680b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:16.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.230+0000 7f196dc00640 1 --2- 172.21.15.73:0/2126665316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196806d060 0x7f19680b3530 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f196806b4f0 tx=0x7f195802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a2e7dbb1c00746 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.230+0000 7f1967e00640 1 -- 172.21.15.73:0/2126665316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195802cc80 con 0x7f196806d060 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.230+0000 7f1967e00640 1 -- 172.21.15.73:0/2126665316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195802cde0 con 0x7f196806d060 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.230+0000 7f1967e00640 1 -- 172.21.15.73:0/2126665316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958034660 con 0x7f196806d060 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 -- 172.21.15.73:0/2126665316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196806d060 msgr2=0x7f19680b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 --2- 172.21.15.73:0/2126665316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196806d060 0x7f19680b3530 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f196806b4f0 tx=0x7f195802bed0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 -- 172.21.15.73:0/2126665316 shutdown_connections 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 --2- 172.21.15.73:0/2126665316 >> v2:172.21.15.157:3300/0 conn(0x7f1968157560 0x7f19680b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 --2- 172.21.15.73:0/2126665316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196806d060 0x7f19680b3530 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 -- 172.21.15.73:0/2126665316 >> 172.21.15.73:0/2126665316 conn(0x7f19680b0830 msgr2=0x7f19680b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:16.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 -- 172.21.15.73:0/2126665316 shutdown_connections 2024-04-05T22:30:16.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.231+0000 7f196f000640 1 -- 172.21.15.73:0/2126665316 wait complete. 2024-04-05T22:30:16.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.233+0000 7f196f000640 1 Processor -- start 2024-04-05T22:30:16.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.233+0000 7f196f000640 1 -- start start 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f196806d060 0x7f19681f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f196815aaa0 0x7f196815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f2eb0 con 0x7f1968157560 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f5920 con 0x7f196815aaa0 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.234+0000 7f196f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f5c00 con 0x7f196806d060 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42270/0 (socket says 172.21.15.73:42270) 2024-04-05T22:30:16.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 -- 172.21.15.73:0/17061545 learned_addr learned my addr 172.21.15.73:0/17061545 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196dc00640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f196806d060 0x7f19681f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196e600640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f196815aaa0 0x7f196815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 -- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f196806d060 msgr2=0x7f19681f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f196806d060 0x7f19681f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 -- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f196815aaa0 msgr2=0x7f196815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f196815aaa0 0x7f196815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.235+0000 7f196d200640 1 -- 172.21.15.73:0/17061545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009a70 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f196d200640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f195400c430 tx=0x7f195400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19540074e0 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954007660 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954004a20 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19681f5ee0 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.236+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681f3190 con 0x7f1968157560 2024-04-05T22:30:16.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.237+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196804e160 con 0x7f1968157560 2024-04-05T22:30:16.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.238+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f1954004b80 con 0x7f1968157560 2024-04-05T22:30:16.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.239+0000 7f1965600640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 0x7f194806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.240+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19540899a0 con 0x7f1968157560 2024-04-05T22:30:16.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.240+0000 7f196dc00640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 0x7f194806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.241+0000 7f196dc00640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 0x7f194806d720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.244+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1954054550 con 0x7f1968157560 2024-04-05T22:30:16.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.439+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f196806cbd0 con 0x7f1968157560 2024-04-05T22:30:16.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.440+0000 7f1965600640 1 -- 172.21.15.73:0/17061545 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1954057b70 con 0x7f1968157560 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:16.440 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 msgr2=0x7f194806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 0x7f194806d720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 msgr2=0x7f1968152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f195400c430 tx=0x7f195400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 shutdown_connections 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f194806b260 0x7f194806d720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f196815aaa0 0x7f196815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968157560 0x7f1968152450 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 --2- 172.21.15.73:0/17061545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f196806d060 0x7f19681f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 >> 172.21.15.73:0/17061545 conn(0x7f19680b0830 msgr2=0x7f19680cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 shutdown_connections 2024-04-05T22:30:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.442+0000 7f196f000640 1 -- 172.21.15.73:0/17061545 wait complete. 2024-04-05T22:30:16.460 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:30:16.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:30:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.671+0000 7f74c1800640 1 Processor -- start 2024-04-05T22:30:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.672+0000 7f74c1800640 1 -- start start 2024-04-05T22:30:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74bc15ee20 0x7f74bc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 0x7f74bc070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c1800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc06c8c0 con 0x7f74bc164c50 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c1800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc071780 con 0x7f74bc15ee20 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc06bfd0 con 0x7f74bc06cd50 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c0e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f74bc164c50 0x7f74bc162820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45162/0 (socket says 172.21.15.73:45162) 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74c0e00640 1 -- 172.21.15.73:0/1626565850 learned_addr learned my addr 172.21.15.73:0/1626565850 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.673+0000 7f74bbe00640 1 --2- 172.21.15.73:0/1626565850 >> v2:172.21.15.73:3300/0 conn(0x7f74bc15ee20 0x7f74bc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bb400640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 0x7f74bc070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74baa00640 1 -- 172.21.15.73:0/1626565850 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711928956 0 0) 0x7f74bc06c8c0 con 0x7f74bc164c50 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74baa00640 1 -- 172.21.15.73:0/1626565850 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a0003520 con 0x7f74bc164c50 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bbe00640 1 -- 172.21.15.73:0/1626565850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 msgr2=0x7f74bc070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bbe00640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 0x7f74bc070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bbe00640 1 -- 172.21.15.73:0/1626565850 >> v1:172.21.15.73:6789/0 conn(0x7f74bc164c50 legacy=0x7f74bc162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bbe00640 1 -- 172.21.15.73:0/1626565850 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0714c0 con 0x7f74bc15ee20 2024-04-05T22:30:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.674+0000 7f74bb400640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 0x7f74bc070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.675+0000 7f74bbe00640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc15ee20 0x7f74bc06f5e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0097a0 tx=0x7f74ac02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f0e6d15fc25fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.675+0000 7f74baa00640 1 -- 172.21.15.73:0/1626565850 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac031780 con 0x7f74bc15ee20 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.675+0000 7f74baa00640 1 -- 172.21.15.73:0/1626565850 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac0318e0 con 0x7f74bc15ee20 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.675+0000 7f74baa00640 1 -- 172.21.15.73:0/1626565850 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac031c70 con 0x7f74bc15ee20 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 -- 172.21.15.73:0/1626565850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc15ee20 msgr2=0x7f74bc06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc15ee20 0x7f74bc06f5e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0097a0 tx=0x7f74ac02b820 comp rx=0 tx=0).stop 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 -- 172.21.15.73:0/1626565850 shutdown_connections 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc06cd50 0x7f74bc070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 --2- 172.21.15.73:0/1626565850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc15ee20 0x7f74bc06f5e0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 -- 172.21.15.73:0/1626565850 >> 172.21.15.73:0/1626565850 conn(0x7f74bc0b0360 msgr2=0x7f74bc0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 -- 172.21.15.73:0/1626565850 shutdown_connections 2024-04-05T22:30:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.676+0000 7f74c1800640 1 -- 172.21.15.73:0/1626565850 wait complete. 2024-04-05T22:30:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.678+0000 7f74c1800640 1 Processor -- start 2024-04-05T22:30:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.678+0000 7f74c1800640 1 -- start start 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc06cd50 0x7f74bc1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc15ee20 0x7f74bc0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1f6c40 con 0x7f74bc164c50 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc15ae00 con 0x7f74bc06cd50 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc15b980 con 0x7f74bc15ee20 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.679+0000 7f74c0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42290/0 (socket says 172.21.15.73:42290) 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 -- 172.21.15.73:0/4067059412 learned_addr learned my addr 172.21.15.73:0/4067059412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74bbe00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc06cd50 0x7f74bc1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74bb400640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc15ee20 0x7f74bc0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 -- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc15ee20 msgr2=0x7f74bc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc15ee20 0x7f74bc0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 -- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc06cd50 msgr2=0x7f74bc1f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc06cd50 0x7f74bc1f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 -- 172.21.15.73:0/4067059412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009400 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.680+0000 7f74c0e00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f74b400caa0 tx=0x7f74b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.681+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400ac00 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.681+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400ad60 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.681+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4012620 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.681+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc15b050 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.681+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc15b610 con 0x7f74bc164c50 2024-04-05T22:30:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.682+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc15baf0 con 0x7f74bc164c50 2024-04-05T22:30:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.683+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f74b4007070 con 0x7f74bc164c50 2024-04-05T22:30:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.683+0000 7f74b3e00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 0x7f749c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.684+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f74b4089a70 con 0x7f74bc164c50 2024-04-05T22:30:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.684+0000 7f74bbe00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 0x7f749c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.685+0000 7f74bbe00640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 0x7f749c06d6f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74ac036000 tx=0x7f74ac03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.688+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f74b40546f0 con 0x7f74bc164c50 2024-04-05T22:30:16.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.882+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f74bc04e160 con 0x7f74bc164c50 2024-04-05T22:30:16.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.991+0000 7f74b3e00640 1 -- 172.21.15.73:0/4067059412 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v145) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74b4017020 con 0x7f74bc164c50 2024-04-05T22:30:16.993 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.995+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 msgr2=0x7f749c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.995+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 0x7f749c06d6f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74ac036000 tx=0x7f74ac03d040 comp rx=0 tx=0).stop 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.995+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 msgr2=0x7f74bc0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.995+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f74b400caa0 tx=0x7f74b400cf60 comp rx=0 tx=0).stop 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 shutdown_connections 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f749c06b230 0x7f749c06d6f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74bc164c50 0x7f74bc0ae290 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74bc15ee20 0x7f74bc0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 --2- 172.21.15.73:0/4067059412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc06cd50 0x7f74bc1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:16.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 >> 172.21.15.73:0/4067059412 conn(0x7f74bc0b0360 msgr2=0x7f74bc15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:16.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 shutdown_connections 2024-04-05T22:30:16.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:16.996+0000 7f74c1800640 1 -- 172.21.15.73:0/4067059412 wait complete. 2024-04-05T22:30:17.018 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:30:17.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.226+0000 7f65a1200640 1 Processor -- start 2024-04-05T22:30:17.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.226+0000 7f65a1200640 1 -- start start 2024-04-05T22:30:17.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f65a1200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f659c06cc00 0x7f659c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f65a1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f659c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f65a1200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c1436b0 con 0x7f659c1432d0 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f65a1200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c0b38c0 con 0x7f659c06cc00 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f65a1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c160840 con 0x7f659c0b3140 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f659b400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f659c1432d0 0x7f659c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45176/0 (socket says 172.21.15.73:45176) 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f659b400640 1 -- 172.21.15.73:0/77617192 learned_addr learned my addr 172.21.15.73:0/77617192 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f659be00640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f659c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f659aa00640 1 --2- 172.21.15.73:0/77617192 >> v2:172.21.15.157:3300/0 conn(0x7f659c06cc00 0x7f659c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.228+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739607630 0 0) 0x7f659c1436b0 con 0x7f659c1432d0 2024-04-05T22:30:17.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6580003520 con 0x7f659c1432d0 2024-04-05T22:30:17.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037171154 0 0) 0x7f6580003520 con 0x7f659c1432d0 2024-04-05T22:30:17.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 msgr2=0x7f659c0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:17.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f659c0705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 >> v2:172.21.15.157:3300/0 conn(0x7f659c06cc00 msgr2=0x7f659c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:17.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 --2- 172.21.15.73:0/77617192 >> v2:172.21.15.157:3300/0 conn(0x7f659c06cc00 0x7f659c0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c160b20 con 0x7f659c1432d0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f658c001690 con 0x7f659c1432d0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 >> v1:172.21.15.73:6789/0 conn(0x7f659c1432d0 legacy=0x7f659c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6580003970 0x7f6580003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f6580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b38c0 con 0x7f6580003970 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c160840 con 0x7f65800047c0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658000b980 con 0x7f659c0b3140 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.229+0000 7f659b400640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6580003970 0x7f6580003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659be00640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f6580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659aa00640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659b400640 1 -- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 msgr2=0x7f6580005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:17.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659b400640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c0b3140 0x7f6580005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659b400640 1 -- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65800047c0 msgr2=0x7f6580004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659b400640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.230+0000 7f659b400640 1 -- 172.21.15.73:0/77617192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580003520 con 0x7f6580003970 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.231+0000 7f659b400640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6580003970 0x7f6580003d90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c210 tx=0x7f65800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c709863be874ddff server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.231+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c005ad0 con 0x7f6580003970 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.231+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c005c30 con 0x7f6580003970 2024-04-05T22:30:17.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.231+0000 7f659a000640 1 -- 172.21.15.73:0/77617192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c02c8b0 con 0x7f6580003970 2024-04-05T22:30:17.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 -- 172.21.15.73:0/77617192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6580003970 msgr2=0x7f6580003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:17.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 --2- 172.21.15.73:0/77617192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6580003970 0x7f6580003d90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c210 tx=0x7f65800057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f659be00640 1 -- 172.21.15.73:0/77617192 reap_dead start 2024-04-05T22:30:17.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 -- 172.21.15.73:0/77617192 shutdown_connections 2024-04-05T22:30:17.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 -- 172.21.15.73:0/77617192 >> 172.21.15.73:0/77617192 conn(0x7f659c069cb0 msgr2=0x7f659c06a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:17.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 -- 172.21.15.73:0/77617192 shutdown_connections 2024-04-05T22:30:17.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.232+0000 7f65a1200640 1 -- 172.21.15.73:0/77617192 wait complete. 2024-04-05T22:30:17.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.234+0000 7f65a1200640 1 Processor -- start 2024-04-05T22:30:17.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.234+0000 7f65a1200640 1 -- start start 2024-04-05T22:30:17.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 0x7f659c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c138f10 0x7f659c139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c136e00 con 0x7f659c1f5d60 2024-04-05T22:30:17.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c1398d0 con 0x7f6580003970 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f65a1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c139b80 con 0x7f659c138f10 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f659be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f659be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42320/0 (socket says 172.21.15.73:42320) 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.235+0000 7f659be00640 1 -- 172.21.15.73:0/3845196712 learned_addr learned my addr 172.21.15.73:0/3845196712 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659b400640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 0x7f659c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 -- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c138f10 msgr2=0x7f659c139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c138f10 0x7f659c139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 -- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 msgr2=0x7f659c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 0x7f659c15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 -- 172.21.15.73:0/3845196712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c1f38f0 con 0x7f659c1f5d60 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659b400640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 0x7f659c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f659be00640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f659c0672a0 tx=0x7f65940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.236+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400e070 con 0x7f659c1f5d60 2024-04-05T22:30:17.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.237+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594011ed0 con 0x7f659c1f5d60 2024-04-05T22:30:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.237+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c1f3bd0 con 0x7f659c1f5d60 2024-04-05T22:30:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.237+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594003990 con 0x7f659c1f5d60 2024-04-05T22:30:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.237+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c1f4190 con 0x7f659c1f5d60 2024-04-05T22:30:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.238+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c1f4320 con 0x7f659c1f5d60 2024-04-05T22:30:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.239+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f6594003b10 con 0x7f659c1f5d60 2024-04-05T22:30:17.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.240+0000 7f6593400640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 0x7f657c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:17.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.240+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f659408b4c0 con 0x7f659c1f5d60 2024-04-05T22:30:17.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.240+0000 7f659b400640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 0x7f657c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:17.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.242+0000 7f659b400640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 0x7f657c06d6f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f658c034000 tx=0x7f658c037040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:17.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.244+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6594055eb0 con 0x7f659c1f5d60 2024-04-05T22:30:17.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:17.438+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f659c13f040 con 0x7f659c1f5d60 2024-04-05T22:30:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.004+0000 7f6593400640 1 -- 172.21.15.73:0/3845196712 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v146) v1 ==== 170+0+0 (secure 0 0 0) 0x7f65940594d0 con 0x7f659c1f5d60 2024-04-05T22:30:18.005 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:30:18.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.013+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 msgr2=0x7f657c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.013+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 0x7f657c06d6f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f658c034000 tx=0x7f658c037040 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.013+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 msgr2=0x7f659c1f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.013+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f659c0672a0 tx=0x7f65940115a0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 shutdown_connections 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f657c06b230 0x7f657c06d6f0 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f658c034000 tx=0x7f658c037040 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c1f5d60 0x7f659c1f61e0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f659c138f10 0x7f659c139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 --2- 172.21.15.73:0/3845196712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6580003970 0x7f659c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.014+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 >> 172.21.15.73:0/3845196712 conn(0x7f659c069cb0 msgr2=0x7f659c140c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:18.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.017+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 shutdown_connections 2024-04-05T22:30:18.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.019+0000 7f65a1200640 1 -- 172.21.15.73:0/3845196712 wait complete. 2024-04-05T22:30:18.047 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:30:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.230+0000 7fbb51400640 1 Processor -- start 2024-04-05T22:30:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.230+0000 7fbb51400640 1 -- start start 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.231+0000 7fbb51400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb4c15abe0 0x7fbb4c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.231+0000 7fbb51400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06bc40 0x7fbb4c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.231+0000 7fbb51400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c15afc0 con 0x7fbb4c06c9a0 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.231+0000 7fbb51400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c06f5b0 con 0x7fbb4c15abe0 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.231+0000 7fbb51400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c06f890 con 0x7fbb4c06bc40 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb41e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbb4c06c9a0 0x7fbb4c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45188/0 (socket says 172.21.15.73:45188) 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb41e00640 1 -- 172.21.15.73:0/558937220 learned_addr learned my addr 172.21.15.73:0/558937220 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4a800640 1 --2- 172.21.15.73:0/558937220 >> v2:172.21.15.73:3300/0 conn(0x7fbb4c15abe0 0x7fbb4c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4b200640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06bc40 0x7fbb4c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4a800640 1 -- 172.21.15.73:0/558937220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06bc40 msgr2=0x7fbb4c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4a800640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06bc40 0x7fbb4c164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb49e00640 1 -- 172.21.15.73:0/558937220 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911852944 0 0) 0x7fbb4c15afc0 con 0x7fbb4c06c9a0 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4a800640 1 -- 172.21.15.73:0/558937220 >> v1:172.21.15.73:6789/0 conn(0x7fbb4c06c9a0 legacy=0x7fbb4c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.232+0000 7fbb4a800640 1 -- 172.21.15.73:0/558937220 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c06fb70 con 0x7fbb4c15abe0 2024-04-05T22:30:18.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.234+0000 7fbb4a800640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c0712e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0097a0 tx=0x7fbb3c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817f485b10d3cbae server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.234+0000 7fbb49e00640 1 -- 172.21.15.73:0/558937220 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c02f050 con 0x7fbb4c15abe0 2024-04-05T22:30:18.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.234+0000 7fbb49e00640 1 -- 172.21.15.73:0/558937220 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c031700 con 0x7fbb4c15abe0 2024-04-05T22:30:18.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.234+0000 7fbb49e00640 1 -- 172.21.15.73:0/558937220 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c031a90 con 0x7fbb4c15abe0 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 -- 172.21.15.73:0/558937220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c15abe0 msgr2=0x7fbb4c0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c0712e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0097a0 tx=0x7fbb3c02b7a0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 -- 172.21.15.73:0/558937220 shutdown_connections 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06bc40 0x7fbb4c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 --2- 172.21.15.73:0/558937220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c0712e0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 -- 172.21.15.73:0/558937220 >> 172.21.15.73:0/558937220 conn(0x7fbb4c069cb0 msgr2=0x7fbb4c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 -- 172.21.15.73:0/558937220 shutdown_connections 2024-04-05T22:30:18.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.235+0000 7fbb51400640 1 -- 172.21.15.73:0/558937220 wait complete. 2024-04-05T22:30:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.237+0000 7fbb51400640 1 Processor -- start 2024-04-05T22:30:18.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.237+0000 7fbb51400640 1 -- start start 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06c9a0 0x7fbb4c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c14bf80 con 0x7fbb4c06bc40 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c15b6d0 con 0x7fbb4c15abe0 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb51400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c14d8f0 con 0x7fbb4c06c9a0 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42340/0 (socket says 172.21.15.73:42340) 2024-04-05T22:30:18.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 -- 172.21.15.73:0/1441528982 learned_addr learned my addr 172.21.15.73:0/1441528982 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:18.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb41e00640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06c9a0 0x7fbb4c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 -- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06c9a0 msgr2=0x7fbb4c15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06c9a0 0x7fbb4c15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4b200640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 -- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4c15abe0 msgr2=0x7fbb4c14c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.238+0000 7fbb4a800640 1 -- 172.21.15.73:0/1441528982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c009400 con 0x7fbb4c06bc40 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb4a800640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004030 con 0x7fbb4c06bc40 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c004190 con 0x7fbb4c06bc40 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004520 con 0x7fbb4c06bc40 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c14dbd0 con 0x7fbb4c06bc40 2024-04-05T22:30:18.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.239+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c14ba60 con 0x7fbb4c06bc40 2024-04-05T22:30:18.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.240+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c14bc50 con 0x7fbb4c06bc40 2024-04-05T22:30:18.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.242+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fbb3c033930 con 0x7fbb4c06bc40 2024-04-05T22:30:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.242+0000 7fbb43400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 0x7fbb2806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.243+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbb3c0ac4b0 con 0x7fbb4c06bc40 2024-04-05T22:30:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.243+0000 7fbb41e00640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 0x7fbb2806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.244+0000 7fbb41e00640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 0x7fbb2806d720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbb34007fd0 tx=0x7fbb34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.246+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbb3c076e30 con 0x7fbb4c06bc40 2024-04-05T22:30:18.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.479+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbb4c068160 con 0x7fbb4c06bc40 2024-04-05T22:30:18.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.493+0000 7fbb43400640 1 -- 172.21.15.73:0/1441528982 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbb3c038070 con 0x7fbb4c06bc40 2024-04-05T22:30:18.494 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 22 and data pool 23 2024-04-05T22:30:18.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.499+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 msgr2=0x7fbb2806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.499+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 0x7fbb2806d720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbb34007fd0 tx=0x7fbb34009040 comp rx=0 tx=0).stop 2024-04-05T22:30:18.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.499+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 msgr2=0x7fbb4c141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.499+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 shutdown_connections 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb2806b260 0x7fbb2806d720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4c15abe0 0x7fbb4c14c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb4c06c9a0 0x7fbb4c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 --2- 172.21.15.73:0/1441528982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb4c06bc40 0x7fbb4c141470 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.502+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 >> 172.21.15.73:0/1441528982 conn(0x7fbb4c069cb0 msgr2=0x7fbb4c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:18.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.503+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 shutdown_connections 2024-04-05T22:30:18.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.503+0000 7fbb51400640 1 -- 172.21.15.73:0/1441528982 wait complete. 2024-04-05T22:30:18.531 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.708+0000 7fce02600640 1 Processor -- start 2024-04-05T22:30:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.708+0000 7fce02600640 1 -- start start 2024-04-05T22:30:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.709+0000 7fce02600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcdfc1587d0 0x7fcdfc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.709+0000 7fce02600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 0x7fcdfc15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.709+0000 7fce02600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc06f5b0 con 0x7fcdfc0ccd20 2024-04-05T22:30:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.709+0000 7fce02600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc15f240 con 0x7fcdfc1587d0 2024-04-05T22:30:18.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.709+0000 7fce02600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc070410 con 0x7fcdfc15e850 2024-04-05T22:30:18.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.710+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 0x7fcdfc15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.710+0000 7fcdf2a00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcdfc0ccd20 0x7fcdfc06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:46672/0 (socket says 172.21.15.73:46672) 2024-04-05T22:30:18.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.710+0000 7fcdf2a00640 1 -- 172.21.15.73:0/2876490822 learned_addr learned my addr 172.21.15.73:0/2876490822 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.710+0000 7fcdfb400640 1 --2- 172.21.15.73:0/2876490822 >> v2:172.21.15.157:3300/0 conn(0x7fcdfc1587d0 0x7fcdfc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.711+0000 7fcdfbe00640 1 -- 172.21.15.73:0/2876490822 >> v2:172.21.15.157:3300/0 conn(0x7fcdfc1587d0 msgr2=0x7fcdfc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.711+0000 7fcdfbe00640 1 --2- 172.21.15.73:0/2876490822 >> v2:172.21.15.157:3300/0 conn(0x7fcdfc1587d0 0x7fcdfc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.711+0000 7fcdfbe00640 1 -- 172.21.15.73:0/2876490822 >> v1:172.21.15.157:6789/0 conn(0x7fcdfc0ccd20 legacy=0x7fcdfc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.711+0000 7fcdfbe00640 1 -- 172.21.15.73:0/2876490822 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc158bb0 con 0x7fcdfc15e850 2024-04-05T22:30:18.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.713+0000 7fcdfbe00640 1 --2- 172.21.15.73:0/2876490822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 0x7fcdfc15ed00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009d90 tx=0x7fcdf402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae38965dd1505112 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.713+0000 7fcdfaa00640 1 -- 172.21.15.73:0/2876490822 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf402cc50 con 0x7fcdfc15e850 2024-04-05T22:30:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.713+0000 7fcdfaa00640 1 -- 172.21.15.73:0/2876490822 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf402cdb0 con 0x7fcdfc15e850 2024-04-05T22:30:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.713+0000 7fcdfaa00640 1 -- 172.21.15.73:0/2876490822 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4034660 con 0x7fcdfc15e850 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 -- 172.21.15.73:0/2876490822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 msgr2=0x7fcdfc15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 --2- 172.21.15.73:0/2876490822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 0x7fcdfc15ed00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009d90 tx=0x7fcdf402bea0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 -- 172.21.15.73:0/2876490822 shutdown_connections 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 --2- 172.21.15.73:0/2876490822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc15e850 0x7fcdfc15ed00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 --2- 172.21.15.73:0/2876490822 >> v2:172.21.15.157:3300/0 conn(0x7fcdfc1587d0 0x7fcdfc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 -- 172.21.15.73:0/2876490822 >> 172.21.15.73:0/2876490822 conn(0x7fcdfc0ce8e0 msgr2=0x7fcdfc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 -- 172.21.15.73:0/2876490822 shutdown_connections 2024-04-05T22:30:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.714+0000 7fce02600640 1 -- 172.21.15.73:0/2876490822 wait complete. 2024-04-05T22:30:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.716+0000 7fce02600640 1 Processor -- start 2024-04-05T22:30:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.716+0000 7fce02600640 1 -- start start 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdfc1587d0 0x7fcdfc06a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc0cbab0 0x7fcdfc0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc06ab20 con 0x7fcdfc15e850 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc142980 con 0x7fcdfc1587d0 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fce02600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0b39f0 con 0x7fcdfc0cbab0 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fcdf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fcdf2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42352/0 (socket says 172.21.15.73:42352) 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.717+0000 7fcdf2a00640 1 -- 172.21.15.73:0/2619196631 learned_addr learned my addr 172.21.15.73:0/2619196631 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdfb400640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdfc1587d0 0x7fcdfc06a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdfbe00640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc0cbab0 0x7fcdfc0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 -- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc0cbab0 msgr2=0x7fcdfc0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc0cbab0 0x7fcdfc0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 -- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdfc1587d0 msgr2=0x7fcdfc06a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdfc1587d0 0x7fcdfc06a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 -- 172.21.15.73:0/2619196631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4009a70 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf2a00640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c4d0 tx=0x7fcde400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4005ce0 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4005e40 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.718+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc0c42f0 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.719+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4007270 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.719+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc069de0 con 0x7fcdfc15e850 2024-04-05T22:30:18.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.720+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc066af0 con 0x7fcdfc15e850 2024-04-05T22:30:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.721+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fcde40073d0 con 0x7fcdfc15e850 2024-04-05T22:30:18.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.722+0000 7fcdf3e00640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 0x7fcdd806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.722+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde4089240 con 0x7fcdfc15e850 2024-04-05T22:30:18.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.722+0000 7fcdfb400640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 0x7fcdd806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:18.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.724+0000 7fcdfb400640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 0x7fcdd806d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009ad0 tx=0x7fcdec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:18.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.726+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcde4053c20 con 0x7fcdfc15e850 2024-04-05T22:30:18.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.919+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdfc068160 con 0x7fcdfc15e850 2024-04-05T22:30:18.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.921+0000 7fcdf3e00640 1 -- 172.21.15.73:0/2619196631 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fcde4057240 con 0x7fcdfc15e850 2024-04-05T22:30:18.922 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:18.922 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":147,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:18.482049+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 msgr2=0x7fcdd806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 0x7fcdd806d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009ad0 tx=0x7fcdec008040 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 msgr2=0x7fcdfc142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c4d0 tx=0x7fcde400c990 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 shutdown_connections 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcdd806b230 0x7fcdd806d6f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdfc0cbab0 0x7fcdfc0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdfc15e850 0x7fcdfc142440 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 --2- 172.21.15.73:0/2619196631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdfc1587d0 0x7fcdfc06a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 >> 172.21.15.73:0/2619196631 conn(0x7fcdfc0ce8e0 msgr2=0x7fcdfc1434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 shutdown_connections 2024-04-05T22:30:18.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:18.924+0000 7fce02600640 1 -- 172.21.15.73:0/2619196631 wait complete. 2024-04-05T22:30:18.944 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:19.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.156+0000 7f0ff6600640 1 Processor -- start 2024-04-05T22:30:19.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.156+0000 7f0ff6600640 1 -- start start 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 0x7f0ff0142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff013fe40 0x7f0ff01429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff006f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff006d3a0 con 0x7f0ff013fe40 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff0143330 con 0x7f0ff013fa60 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0ff6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0143050 con 0x7f0ff006b500 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0fefe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff013fe40 0x7f0ff01429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0fefe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff013fe40 0x7f0ff01429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42360/0 (socket says 172.21.15.73:42360) 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.157+0000 7f0fefe00640 1 -- 172.21.15.73:0/1278650391 learned_addr learned my addr 172.21.15.73:0/1278650391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0fefe00640 1 -- 172.21.15.73:0/1278650391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 msgr2=0x7f0ff006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0fefe00640 1 --2- 172.21.15.73:0/1278650391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff006f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0fefe00640 1 -- 172.21.15.73:0/1278650391 >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 msgr2=0x7f0ff0142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0ff5200640 1 --2- 172.21.15.73:0/1278650391 >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 0x7f0ff0142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0fefe00640 1 --2- 172.21.15.73:0/1278650391 >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 0x7f0ff0142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0fefe00640 1 -- 172.21.15.73:0/1278650391 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00ac970 con 0x7f0ff013fe40 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.158+0000 7f0ff5200640 1 --2- 172.21.15.73:0/1278650391 >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 0x7f0ff0142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.159+0000 7f0fefe00640 1 --2- 172.21.15.73:0/1278650391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01429d0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0ff006f2f0 tx=0x7f0fdc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a071776274525a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.159+0000 7f0fef400640 1 -- 172.21.15.73:0/1278650391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031860 con 0x7f0ff013fe40 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.159+0000 7f0fef400640 1 -- 172.21.15.73:0/1278650391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0319c0 con 0x7f0ff013fe40 2024-04-05T22:30:19.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.159+0000 7f0fef400640 1 -- 172.21.15.73:0/1278650391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031d50 con 0x7f0ff013fe40 2024-04-05T22:30:19.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 -- 172.21.15.73:0/1278650391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 msgr2=0x7f0ff01429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 --2- 172.21.15.73:0/1278650391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01429d0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0ff006f2f0 tx=0x7f0fdc02b900 comp rx=0 tx=0).stop 2024-04-05T22:30:19.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 -- 172.21.15.73:0/1278650391 shutdown_connections 2024-04-05T22:30:19.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 --2- 172.21.15.73:0/1278650391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01429d0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 --2- 172.21.15.73:0/1278650391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 --2- 172.21.15.73:0/1278650391 >> v2:172.21.15.157:3300/0 conn(0x7f0ff013fa60 0x7f0ff0142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 -- 172.21.15.73:0/1278650391 >> 172.21.15.73:0/1278650391 conn(0x7f0ff00b06c0 msgr2=0x7f0ff00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:19.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 -- 172.21.15.73:0/1278650391 shutdown_connections 2024-04-05T22:30:19.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.160+0000 7f0ff6600640 1 -- 172.21.15.73:0/1278650391 wait complete. 2024-04-05T22:30:19.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.162+0000 7f0ff6600640 1 Processor -- start 2024-04-05T22:30:19.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.162+0000 7f0ff6600640 1 -- start start 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff01f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 0x7f0ff013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff01f3860 con 0x7f0ff013fe40 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00b1f20 con 0x7f0ff013fa60 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00b2200 con 0x7f0ff006b500 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42364/0 (socket says 172.21.15.73:42364) 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.163+0000 7f0ff5c00640 1 -- 172.21.15.73:0/3227852431 learned_addr learned my addr 172.21.15.73:0/3227852431 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:19.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5200640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff01f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0fefe00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 0x7f0ff013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 -- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 msgr2=0x7f0ff01f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff01f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 -- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 msgr2=0x7f0ff013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 0x7f0ff013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 -- 172.21.15.73:0/3227852431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0094d0 con 0x7f0ff013fe40 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0fefe00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 0x7f0ff013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5200640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff01f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:19.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff5c00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800b970 tx=0x7f0fe800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800cb40 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800ccc0 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff00b24e0 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8014680 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.164+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff013d3e0 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.166+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff013d630 con 0x7f0ff013fe40 2024-04-05T22:30:19.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.166+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f0fe801d3b0 con 0x7f0ff013fe40 2024-04-05T22:30:19.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.167+0000 7f0fecc00640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 0x7f0fd006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.167+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fe8089ea0 con 0x7f0ff013fe40 2024-04-05T22:30:19.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.168+0000 7f0ff5200640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 0x7f0fd006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.169+0000 7f0ff5200640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 0x7f0fd006d6f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4006fd0 tx=0x7f0fe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.171+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0fe8054700 con 0x7f0ff013fe40 2024-04-05T22:30:19.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.366+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff00b2a00 con 0x7f0ff013fe40 2024-04-05T22:30:19.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.367+0000 7f0fecc00640 1 -- 172.21.15.73:0/3227852431 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f0fe8019020 con 0x7f0ff013fe40 2024-04-05T22:30:19.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:19.368 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":147,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:18.482049+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:19.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 msgr2=0x7f0fd006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 0x7f0fd006d6f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4006fd0 tx=0x7f0fe4008040 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 msgr2=0x7f0ff01ef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800b970 tx=0x7f0fe800be30 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 shutdown_connections 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fd006b230 0x7f0fd006d6f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff013fe40 0x7f0ff01ef900 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ff013fa60 0x7f0ff013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 --2- 172.21.15.73:0/3227852431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ff006b500 0x7f0ff01f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 >> 172.21.15.73:0/3227852431 conn(0x7f0ff00b06c0 msgr2=0x7f0ff00704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.370+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 shutdown_connections 2024-04-05T22:30:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.371+0000 7f0ff6600640 1 -- 172.21.15.73:0/3227852431 wait complete. 2024-04-05T22:30:19.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:30:19.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.603+0000 7f7b02400640 1 Processor -- start 2024-04-05T22:30:19.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.603+0000 7f7b02400640 1 -- start start 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7b02400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7afc15d1f0 0x7f7afc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7b02400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7b02400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc15ddd0 con 0x7f7afc15ce10 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7b02400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0ce370 con 0x7f7afc15d1f0 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7b02400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15dbb0 con 0x7f7afc0b0990 2024-04-05T22:30:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.605+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:43148/0 (socket says 172.21.15.73:43148) 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 -- 172.21.15.73:0/342751154 learned_addr learned my addr 172.21.15.73:0/342751154 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afaa00640 1 --2- 172.21.15.73:0/342751154 >> v2:172.21.15.157:3300/0 conn(0x7f7afc15d1f0 0x7f7afc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 -- 172.21.15.73:0/342751154 >> v2:172.21.15.157:3300/0 conn(0x7f7afc15d1f0 msgr2=0x7f7afc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afa000640 1 -- 172.21.15.73:0/342751154 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594831998 0 0) 0x7f7afc15ddd0 con 0x7f7afc15ce10 2024-04-05T22:30:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 --2- 172.21.15.73:0/342751154 >> v2:172.21.15.157:3300/0 conn(0x7f7afc15d1f0 0x7f7afc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 -- 172.21.15.73:0/342751154 >> v1:172.21.15.157:6789/0 conn(0x7f7afc15ce10 legacy=0x7f7afc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.606+0000 7f7afbe00640 1 -- 172.21.15.73:0/342751154 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc15e0b0 con 0x7f7afc0b0990 2024-04-05T22:30:19.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.608+0000 7f7afbe00640 1 --2- 172.21.15.73:0/342751154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009e50 tx=0x7f7af402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0de795a386b88c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.609+0000 7f7afa000640 1 -- 172.21.15.73:0/342751154 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402cb00 con 0x7f7afc0b0990 2024-04-05T22:30:19.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.609+0000 7f7afa000640 1 -- 172.21.15.73:0/342751154 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af402cc60 con 0x7f7afc0b0990 2024-04-05T22:30:19.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.609+0000 7f7afa000640 1 -- 172.21.15.73:0/342751154 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40343f0 con 0x7f7afc0b0990 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 -- 172.21.15.73:0/342751154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 msgr2=0x7f7afc0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 --2- 172.21.15.73:0/342751154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009e50 tx=0x7f7af402bf60 comp rx=0 tx=0).stop 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 -- 172.21.15.73:0/342751154 shutdown_connections 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 --2- 172.21.15.73:0/342751154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc0b0e20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 --2- 172.21.15.73:0/342751154 >> v2:172.21.15.157:3300/0 conn(0x7f7afc15d1f0 0x7f7afc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 -- 172.21.15.73:0/342751154 >> 172.21.15.73:0/342751154 conn(0x7f7afc099a60 msgr2=0x7f7afc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 -- 172.21.15.73:0/342751154 shutdown_connections 2024-04-05T22:30:19.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.610+0000 7f7b02400640 1 -- 172.21.15.73:0/342751154 wait complete. 2024-04-05T22:30:19.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.611+0000 7f7b02400640 1 Processor -- start 2024-04-05T22:30:19.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.612+0000 7f7b02400640 1 -- start start 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.612+0000 7f7b02400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc1f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7b02400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7b02400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc15d1f0 0x7f7afc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7b02400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1f7530 con 0x7f7afc15ce10 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7b02400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0c62b0 con 0x7f7afc15d1f0 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7b02400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0b3ad0 con 0x7f7afc0b0990 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42366/0 (socket says 172.21.15.73:42366) 2024-04-05T22:30:19.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 -- 172.21.15.73:0/3473034076 learned_addr learned my addr 172.21.15.73:0/3473034076 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afb400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc1f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afbe00640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc15d1f0 0x7f7afc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 -- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 msgr2=0x7f7afc1f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc1f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 -- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc15d1f0 msgr2=0x7f7afc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.613+0000 7f7afaa00640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc15d1f0 0x7f7afc09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7afaa00640 1 -- 172.21.15.73:0/3473034076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009b30 con 0x7f7afc15ce10 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7afaa00640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800e9b0 tx=0x7f7ae800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8009670 con 0x7f7afc15ce10 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae800c9a0 con 0x7f7afc15ce10 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0b3db0 con 0x7f7afc15ce10 2024-04-05T22:30:19.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800cd50 con 0x7f7afc15ce10 2024-04-05T22:30:19.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.614+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc0b4370 con 0x7f7afc15ce10 2024-04-05T22:30:19.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.615+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc067480 con 0x7f7afc15ce10 2024-04-05T22:30:19.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.616+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7ae8013470 con 0x7f7afc15ce10 2024-04-05T22:30:19.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.617+0000 7f7af3400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 0x7f7adc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:19.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.617+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ae808a100 con 0x7f7afc15ce10 2024-04-05T22:30:19.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.618+0000 7f7afb400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 0x7f7adc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:19.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.619+0000 7f7afb400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 0x7f7adc06d720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7aec000f60 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:19.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.623+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7ae80549f0 con 0x7f7afc15ce10 2024-04-05T22:30:19.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:19.858+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7afc0d1bb0 con 0x7f7afc15ce10 2024-04-05T22:30:20.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.513+0000 7f7af3400640 1 -- 172.21.15.73:0/3473034076 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7ae8058010 con 0x7f7afc15ce10 2024-04-05T22:30:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 msgr2=0x7f7adc06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 0x7f7adc06d720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7aec000f60 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2024-04-05T22:30:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 msgr2=0x7f7afc0cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800e9b0 tx=0x7f7ae800ee70 comp rx=0 tx=0).stop 2024-04-05T22:30:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 shutdown_connections 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7adc06b260 0x7f7adc06d720 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc15d1f0 0x7f7afc09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15ce10 0x7f7afc0cfc70 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 --2- 172.21.15.73:0/3473034076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0b0990 0x7f7afc1f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 >> 172.21.15.73:0/3473034076 conn(0x7f7afc099a60 msgr2=0x7f7afc160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 shutdown_connections 2024-04-05T22:30:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.517+0000 7f7b02400640 1 -- 172.21.15.73:0/3473034076 wait complete. 2024-04-05T22:30:20.536 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:30:20.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:30:20.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.745+0000 7f1eb6e00640 1 Processor -- start 2024-04-05T22:30:20.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.745+0000 7f1eb6e00640 1 -- start start 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eb6e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb015abe0 0x7f1eb00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eb6e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1eb006bc40 0x7f1eb0164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eb6e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb015afc0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eb6e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb006f5b0 con 0x7f1eb015abe0 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eb6e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb006f890 con 0x7f1eb006bc40 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb015abe0 0x7f1eb00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eb015abe0 0x7f1eb00712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42376/0 (socket says 172.21.15.73:42376) 2024-04-05T22:30:20.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 -- 172.21.15.73:0/1548360950 learned_addr learned my addr 172.21.15.73:0/1548360950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:20.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 -- 172.21.15.73:0/1548360950 >> v2:172.21.15.157:3300/0 conn(0x7f1eb006bc40 msgr2=0x7f1eb0164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:20.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 --2- 172.21.15.73:0/1548360950 >> v2:172.21.15.157:3300/0 conn(0x7f1eb006bc40 0x7f1eb0164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 -- 172.21.15.73:0/1548360950 >> v1:172.21.15.157:6789/0 conn(0x7f1eb006c9a0 legacy=0x7f1eb006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:20.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.747+0000 7f1eafe00640 1 -- 172.21.15.73:0/1548360950 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb006fb70 con 0x7f1eb015abe0 2024-04-05T22:30:20.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.749+0000 7f1eafe00640 1 --2- 172.21.15.73:0/1548360950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb015abe0 0x7f1eb00712e0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1eb006c900 tx=0x7f1ea002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a79c9bd598d4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:20.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.749+0000 7f1eaea00640 1 -- 172.21.15.73:0/1548360950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002c960 con 0x7f1eb015abe0 2024-04-05T22:30:20.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.749+0000 7f1eaea00640 1 -- 172.21.15.73:0/1548360950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea002cac0 con 0x7f1eb015abe0 2024-04-05T22:30:20.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.749+0000 7f1eaea00640 1 -- 172.21.15.73:0/1548360950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002ce50 con 0x7f1eb015abe0 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 -- 172.21.15.73:0/1548360950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb015abe0 msgr2=0x7f1eb00712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/1548360950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb015abe0 0x7f1eb00712e0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1eb006c900 tx=0x7f1ea002bdc0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 -- 172.21.15.73:0/1548360950 shutdown_connections 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/1548360950 >> v2:172.21.15.157:3300/0 conn(0x7f1eb006bc40 0x7f1eb0164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/1548360950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb015abe0 0x7f1eb00712e0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 -- 172.21.15.73:0/1548360950 >> 172.21.15.73:0/1548360950 conn(0x7f1eb0069cb0 msgr2=0x7f1eb006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 -- 172.21.15.73:0/1548360950 shutdown_connections 2024-04-05T22:30:20.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.750+0000 7f1eb6e00640 1 -- 172.21.15.73:0/1548360950 wait complete. 2024-04-05T22:30:20.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.752+0000 7f1eb6e00640 1 Processor -- start 2024-04-05T22:30:20.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.752+0000 7f1eb6e00640 1 -- start start 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb006bc40 0x7f1eb0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb015abe0 0x7f1eb00b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0158530 con 0x7f1eb006c9a0 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b2120 con 0x7f1eb006bc40 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.753+0000 7f1eb6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b2400 con 0x7f1eb015abe0 2024-04-05T22:30:20.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42388/0 (socket says 172.21.15.73:42388) 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 -- 172.21.15.73:0/402221951 learned_addr learned my addr 172.21.15.73:0/402221951 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eafe00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb006bc40 0x7f1eb0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 -- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb015abe0 msgr2=0x7f1eb00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb015abe0 0x7f1eb00b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 -- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb006bc40 msgr2=0x7f1eb0158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb006bc40 0x7f1eb0158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 -- 172.21.15.73:0/402221951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0009b30 con 0x7f1eb006c9a0 2024-04-05T22:30:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.754+0000 7f1eaf400640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c410 tx=0x7f1e9800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.755+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980035f0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.755+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98003750 con 0x7f1eb006c9a0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.755+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98003b40 con 0x7f1eb006c9a0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.755+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb0158780 con 0x7f1eb006c9a0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.755+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb01594c0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.756+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb0158ca0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.757+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f1e98003ca0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.758+0000 7f1ea7e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 0x7f1e8c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.758+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e980890a0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.759+0000 7f1eafe00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 0x7f1e8c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:20.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.760+0000 7f1eafe00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 0x7f1e8c06d6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0035000 tx=0x7f1ea0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:20.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.763+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1eb0158ca0 con 0x7f1eb006c9a0 2024-04-05T22:30:20.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:20.994+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1eb0158ca0 con 0x7f1eb006c9a0 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.964+0000 7f1ea7e00640 1 -- 172.21.15.73:0/402221951 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1eb0158ca0 con 0x7f1eb006c9a0 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 msgr2=0x7f1e8c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 0x7f1e8c06d6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0035000 tx=0x7f1ea0030000 comp rx=0 tx=0).stop 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 msgr2=0x7f1eb0145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c410 tx=0x7f1e9800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 shutdown_connections 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1e8c06b230 0x7f1e8c06d6f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb015abe0 0x7f1eb00b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eb006c9a0 0x7f1eb0145af0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:21.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 --2- 172.21.15.73:0/402221951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb006bc40 0x7f1eb0158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 >> 172.21.15.73:0/402221951 conn(0x7f1eb0069cb0 msgr2=0x7f1eb0155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 shutdown_connections 2024-04-05T22:30:21.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:21.967+0000 7f1eb6e00640 1 -- 172.21.15.73:0/402221951 wait complete. 2024-04-05T22:30:21.986 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:30:21.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:22.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.199+0000 7f5dff200640 1 Processor -- start 2024-04-05T22:30:22.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.200+0000 7f5dff200640 1 -- start start 2024-04-05T22:30:22.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.201+0000 7f5dff200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5df81587d0 0x7f5df80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.201+0000 7f5dff200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5df815e850 0x7f5df815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.201+0000 7f5dff200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df806f5b0 con 0x7f5df80ccd20 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.201+0000 7f5dff200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df815f240 con 0x7f5df81587d0 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.201+0000 7f5dff200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df8070410 con 0x7f5df815e850 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5def400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5df80ccd20 0x7f5df806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45202/0 (socket says 172.21.15.73:45202) 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5def400640 1 -- 172.21.15.73:0/600517795 learned_addr learned my addr 172.21.15.73:0/600517795 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:22.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7e00640 1 --2- 172.21.15.73:0/600517795 >> v2:172.21.15.73:3300/0 conn(0x7f5df81587d0 0x7f5df80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5dfd200640 1 --2- 172.21.15.73:0/600517795 >> v2:172.21.15.157:3300/0 conn(0x7f5df815e850 0x7f5df815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7400640 1 -- 172.21.15.73:0/600517795 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268745905 0 0) 0x7f5df806f5b0 con 0x7f5df80ccd20 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7400640 1 -- 172.21.15.73:0/600517795 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ddc003520 con 0x7f5df80ccd20 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7e00640 1 -- 172.21.15.73:0/600517795 >> v2:172.21.15.157:3300/0 conn(0x7f5df815e850 msgr2=0x7f5df815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7e00640 1 --2- 172.21.15.73:0/600517795 >> v2:172.21.15.157:3300/0 conn(0x7f5df815e850 0x7f5df815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7e00640 1 -- 172.21.15.73:0/600517795 >> v1:172.21.15.73:6789/0 conn(0x7f5df80ccd20 legacy=0x7f5df806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.202+0000 7f5df7e00640 1 -- 172.21.15.73:0/600517795 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8158bb0 con 0x7f5df81587d0 2024-04-05T22:30:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.204+0000 7f5df7e00640 1 --2- 172.21.15.73:0/600517795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df80cc7e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a6736087297837 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.204+0000 7f5df7400640 1 -- 172.21.15.73:0/600517795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8031830 con 0x7f5df81587d0 2024-04-05T22:30:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.204+0000 7f5df7400640 1 -- 172.21.15.73:0/600517795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8031990 con 0x7f5df81587d0 2024-04-05T22:30:22.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.204+0000 7f5df7400640 1 -- 172.21.15.73:0/600517795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8031d00 con 0x7f5df81587d0 2024-04-05T22:30:22.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 -- 172.21.15.73:0/600517795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 msgr2=0x7f5df80cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 --2- 172.21.15.73:0/600517795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df80cc7e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 -- 172.21.15.73:0/600517795 shutdown_connections 2024-04-05T22:30:22.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 --2- 172.21.15.73:0/600517795 >> v2:172.21.15.157:3300/0 conn(0x7f5df815e850 0x7f5df815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 --2- 172.21.15.73:0/600517795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df80cc7e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 -- 172.21.15.73:0/600517795 >> 172.21.15.73:0/600517795 conn(0x7f5df80ce8e0 msgr2=0x7f5df80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 -- 172.21.15.73:0/600517795 shutdown_connections 2024-04-05T22:30:22.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.205+0000 7f5dff200640 1 -- 172.21.15.73:0/600517795 wait complete. 2024-04-05T22:30:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.207+0000 7f5dff200640 1 Processor -- start 2024-04-05T22:30:22.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.207+0000 7f5dff200640 1 -- start start 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df815e850 0x7f5df806a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8140ca0 0x7f5df8141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8141690 con 0x7f5df81587d0 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80afff0 con 0x7f5df8140ca0 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5dff200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80af3e0 con 0x7f5df815e850 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42396/0 (socket says 172.21.15.73:42396) 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.208+0000 7f5df7e00640 1 -- 172.21.15.73:0/3143089866 learned_addr learned my addr 172.21.15.73:0/3143089866 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5def400640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df815e850 0x7f5df806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 -- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df815e850 msgr2=0x7f5df806a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5dfd200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8140ca0 0x7f5df8141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df815e850 0x7f5df806a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 -- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8140ca0 msgr2=0x7f5df8141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8140ca0 0x7f5df8141150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 -- 172.21.15.73:0/3143089866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8009400 con 0x7f5df81587d0 2024-04-05T22:30:22.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df7e00640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5de80098c0 tx=0x7f5de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de803a6f0 con 0x7f5df81587d0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de803a850 con 0x7f5df81587d0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80af6c0 con 0x7f5df81587d0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de803ac00 con 0x7f5df81587d0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.209+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80afc80 con 0x7f5df81587d0 2024-04-05T22:30:22.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.210+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df8066af0 con 0x7f5df81587d0 2024-04-05T22:30:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.212+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f5de804c3b0 con 0x7f5df81587d0 2024-04-05T22:30:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.212+0000 7f5df4c00640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 0x7f5dd406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.212+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de80abb90 con 0x7f5df81587d0 2024-04-05T22:30:22.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.213+0000 7f5def400640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 0x7f5dd406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.214+0000 7f5def400640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 0x7f5dd406d720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5df8142ad0 tx=0x7f5de0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.217+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5de8076490 con 0x7f5df81587d0 2024-04-05T22:30:22.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.447+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5df80afe10 con 0x7f5df81587d0 2024-04-05T22:30:22.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.448+0000 7f5df4c00640 1 -- 172.21.15.73:0/3143089866 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f5de8079ab0 con 0x7f5df81587d0 2024-04-05T22:30:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:22.447 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:22.448 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 91 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 msgr2=0x7f5dd406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 0x7f5dd406d720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5df8142ad0 tx=0x7f5de0009040 comp rx=0 tx=0).stop 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 msgr2=0x7f5df813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5de80098c0 tx=0x7f5de80046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 shutdown_connections 2024-04-05T22:30:22.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5dd406b260 0x7f5dd406d720 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df8140ca0 0x7f5df8141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5df815e850 0x7f5df806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 --2- 172.21.15.73:0/3143089866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5df81587d0 0x7f5df813f5a0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 >> 172.21.15.73:0/3143089866 conn(0x7f5df80ce8e0 msgr2=0x7f5df8158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 shutdown_connections 2024-04-05T22:30:22.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.451+0000 7f5dff200640 1 -- 172.21.15.73:0/3143089866 wait complete. 2024-04-05T22:30:22.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:22.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.675+0000 7fa855e00640 1 Processor -- start 2024-04-05T22:30:22.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.676+0000 7fa855e00640 1 -- start start 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa855e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8500ce1d0 0x7fa8500b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa855e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8501436f0 con 0x7fa850143310 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa855e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa850160c60 con 0x7fa850146660 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa855e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa850161800 con 0x7fa8500ce1d0 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa854a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa850143310 0x7fa8501610f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45206/0 (socket says 172.21.15.73:45206) 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.677+0000 7fa854a00640 1 -- 172.21.15.73:0/848470660 learned_addr learned my addr 172.21.15.73:0/848470660 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661267366 0 0) 0x7fa8501436f0 con 0x7fa850143310 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa834003520 con 0x7fa850143310 2024-04-05T22:30:22.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa855400640 1 --2- 172.21.15.73:0/848470660 >> v2:172.21.15.157:3300/0 conn(0x7fa8500ce1d0 0x7fa8500b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3225498298 0 0) 0x7fa834003520 con 0x7fa850143310 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 >> v2:172.21.15.157:3300/0 conn(0x7fa8500ce1d0 msgr2=0x7fa8500b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 --2- 172.21.15.73:0/848470660 >> v2:172.21.15.157:3300/0 conn(0x7fa8500ce1d0 0x7fa8500b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 >> v1:172.21.15.157:6789/0 conn(0x7fa850146660 legacy=0x7fa8500cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850160930 con 0x7fa850143310 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.678+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa840001670 con 0x7fa850143310 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 >> v1:172.21.15.73:6789/0 conn(0x7fa850143310 legacy=0x7fa8501610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa834003970 0x7fa834003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340047c0 0x7fa834004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 0x7fa83400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850161800 con 0x7fa8340052a0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83400c170 con 0x7fa834003970 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83400c330 con 0x7fa8340047c0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 0x7fa83400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa854a00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa834003970 0x7fa834003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa847e00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340047c0 0x7fa834004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 -- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340047c0 msgr2=0x7fa834004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340047c0 0x7fa834004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 -- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa834003970 msgr2=0x7fa834003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa834003970 0x7fa834003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa855400640 1 -- 172.21.15.73:0/848470660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834003520 con 0x7fa8340052a0 2024-04-05T22:30:22.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.679+0000 7fa854a00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa834003970 0x7fa834003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.680+0000 7fa855400640 1 -- 172.21.15.73:0/848470660 reap_dead start 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.680+0000 7fa855400640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 0x7fa83400b980 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fc0 tx=0x7fa8340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f86e02821e73e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84802ccb0 con 0x7fa8340052a0 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84802ce30 con 0x7fa8340052a0 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa84fe00640 1 -- 172.21.15.73:0/848470660 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848034680 con 0x7fa8340052a0 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 -- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 msgr2=0x7fa83400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 0x7fa83400b980 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fc0 tx=0x7fa8340057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 -- 172.21.15.73:0/848470660 shutdown_connections 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 --2- 172.21.15.73:0/848470660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8340052a0 0x7fa83400b980 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 -- 172.21.15.73:0/848470660 >> 172.21.15.73:0/848470660 conn(0x7fa850069cb0 msgr2=0x7fa85006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 -- 172.21.15.73:0/848470660 shutdown_connections 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.681+0000 7fa855e00640 1 -- 172.21.15.73:0/848470660 wait complete. 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.683+0000 7fa855e00640 1 Processor -- start 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.683+0000 7fa855e00640 1 -- start start 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340052a0 0x7fa85015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8501448f0 0x7fa850144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850145290 con 0x7fa8500b0150 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85015d410 con 0x7fa8501448f0 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.684+0000 7fa855e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500b14e0 con 0x7fa8340052a0 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42412/0 (socket says 172.21.15.73:42412) 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 -- 172.21.15.73:0/1287337646 learned_addr learned my addr 172.21.15.73:0/1287337646 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa847e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8501448f0 0x7fa850144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa854a00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340052a0 0x7fa85015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 -- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340052a0 msgr2=0x7fa85015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340052a0 0x7fa85015c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 -- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8501448f0 msgr2=0x7fa850144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8501448f0 0x7fa850144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.685+0000 7fa855400640 1 -- 172.21.15.73:0/1287337646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500b0f20 con 0x7fa8500b0150 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa855400640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa84802c720 tx=0x7fa848004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84802ccb0 con 0x7fa8500b0150 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84803e410 con 0x7fa8500b0150 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84803e780 con 0x7fa8500b0150 2024-04-05T22:30:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8500b1200 con 0x7fa8500b0150 2024-04-05T22:30:22.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.686+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85015c870 con 0x7fa8500b0150 2024-04-05T22:30:22.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.687+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85015d8f0 con 0x7fa8500b0150 2024-04-05T22:30:22.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.688+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fa84803b050 con 0x7fa8500b0150 2024-04-05T22:30:22.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.689+0000 7fa84d600640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 0x7fa82c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:22.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.690+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8480abcf0 con 0x7fa8500b0150 2024-04-05T22:30:22.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.690+0000 7fa854a00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 0x7fa82c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:22.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.691+0000 7fa854a00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 0x7fa82c06d720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa840000fb0 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:22.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.694+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa85015d8f0 con 0x7fa8500b0150 2024-04-05T22:30:22.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.886+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa850145f70 con 0x7fa8500b0150 2024-04-05T22:30:22.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.888+0000 7fa84d600640 1 -- 172.21.15.73:0/1287337646 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7fa848039090 con 0x7fa8500b0150 2024-04-05T22:30:22.887 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:22.888 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:22.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 msgr2=0x7fa82c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 0x7fa82c06d720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa840000fb0 tx=0x7fa840009040 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 msgr2=0x7fa8500b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa84802c720 tx=0x7fa848004ab0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 shutdown_connections 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa82c06b260 0x7fa82c06d720 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8500b0150 0x7fa8500b0600 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8501448f0 0x7fa850144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 --2- 172.21.15.73:0/1287337646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8340052a0 0x7fa85015c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 >> 172.21.15.73:0/1287337646 conn(0x7fa850069cb0 msgr2=0x7fa85006f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 shutdown_connections 2024-04-05T22:30:22.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:22.891+0000 7fa855e00640 1 -- 172.21.15.73:0/1287337646 wait complete. 2024-04-05T22:30:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:30:22.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:30:23.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.114+0000 7f077ee00640 1 Processor -- start 2024-04-05T22:30:23.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.114+0000 7f077ee00640 1 -- start start 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f077ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f0778162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f077ee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0778165cf0 con 0x7f0778164e50 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f077ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0778165730 con 0x7f07780b2ba0 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f077ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0778158790 con 0x7f0778161f90 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f076f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0778164e50 0x7f07780b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45222/0 (socket says 172.21.15.73:45222) 2024-04-05T22:30:23.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f076f400640 1 -- 172.21.15.73:0/3215648707 learned_addr learned my addr 172.21.15.73:0/3215648707 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:23.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f0777e00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f0778162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472012373 0 0) 0x7f0778165cf0 con 0x7f0778164e50 2024-04-05T22:30:23.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.116+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f075c003520 con 0x7f0778164e50 2024-04-05T22:30:23.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400617091 0 0) 0x7f075c003520 con 0x7f0778164e50 2024-04-05T22:30:23.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 msgr2=0x7f0778162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f0778162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 >> v1:172.21.15.157:6789/0 conn(0x7f07780b2ba0 legacy=0x7f0778070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778165a10 con 0x7f0778164e50 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0760001670 con 0x7f0778164e50 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 >> v1:172.21.15.73:6789/0 conn(0x7f0778164e50 legacy=0x7f07780b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0039a0 0x7f075c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075c0047c0 0x7f075c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f075c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778158790 con 0x7f075c0047c0 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c00bab0 con 0x7f075c0039a0 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c00bc70 con 0x7f0778161f90 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.117+0000 7f076f400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075c0047c0 0x7f075c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f0777e00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0039a0 0x7f075c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f077ce00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f075c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f076f400640 1 -- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 msgr2=0x7f075c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f076f400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f075c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f076f400640 1 -- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0039a0 msgr2=0x7f075c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f076f400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0039a0 0x7f075c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f076f400640 1 -- 172.21.15.73:0/3215648707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c003520 con 0x7f075c0047c0 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f077ce00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0778161f90 0x7f075c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.118+0000 7f0777e00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0039a0 0x7f075c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:23.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.119+0000 7f076f400640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075c0047c0 0x7f075c004c40 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f076000c190 tx=0x7f075c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686a9679b9d15ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.119+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760005ad0 con 0x7f075c0047c0 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.119+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760005c30 con 0x7f075c0047c0 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.119+0000 7f0777400640 1 -- 172.21.15.73:0/3215648707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076002c8b0 con 0x7f075c0047c0 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 -- 172.21.15.73:0/3215648707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075c0047c0 msgr2=0x7f075c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 --2- 172.21.15.73:0/3215648707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075c0047c0 0x7f075c004c40 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f076000c190 tx=0x7f075c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ce00640 1 -- 172.21.15.73:0/3215648707 reap_dead start 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 -- 172.21.15.73:0/3215648707 shutdown_connections 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 -- 172.21.15.73:0/3215648707 >> 172.21.15.73:0/3215648707 conn(0x7f07780ce8e0 msgr2=0x7f07780cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 -- 172.21.15.73:0/3215648707 shutdown_connections 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.120+0000 7f077ee00640 1 -- 172.21.15.73:0/3215648707 wait complete. 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.121+0000 7f077ee00640 1 Processor -- start 2024-04-05T22:30:23.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.122+0000 7f077ee00640 1 -- start start 2024-04-05T22:30:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.122+0000 7f077ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 0x7f077806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f077814f670 0x7f077814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778151770 con 0x7f0778150470 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077806c700 con 0x7f075c0047c0 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077806b870 con 0x7f077814f670 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42430/0 (socket says 172.21.15.73:42430) 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 -- 172.21.15.73:0/2735753352 learned_addr learned my addr 172.21.15.73:0/2735753352 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f0777e00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 0x7f077806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f077ce00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f077814f670 0x7f077814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 -- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f077814f670 msgr2=0x7f077814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f077814f670 0x7f077814faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 -- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 msgr2=0x7f077806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 0x7f077806b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f076f400640 1 -- 172.21.15.73:0/2735753352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778150030 con 0x7f0778150470 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.123+0000 7f0777e00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 0x7f077806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f076f400640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f076002c450 tx=0x7f077815be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760039070 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076002ccc0 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077806baf0 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076003c040 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.124+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077806c0b0 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.125+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0778066af0 con 0x7f0778150470 2024-04-05T22:30:23.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.126+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f0760032070 con 0x7f0778150470 2024-04-05T22:30:23.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.127+0000 7f0774c00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 0x7f075406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.127+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07600ab1b0 con 0x7f0778150470 2024-04-05T22:30:23.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.128+0000 7f0777e00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 0x7f075406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.129+0000 7f0777e00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 0x7f075406d720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0768006fd0 tx=0x7f0768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.131+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0760075a10 con 0x7f0778150470 2024-04-05T22:30:23.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.322+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07780b1bf0 con 0x7f075406b260 2024-04-05T22:30:23.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.343+0000 7f0774c00640 1 -- 172.21.15.73:0/2735753352 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276978 (secure 0 0 0) 0x7f0768009690 con 0x7f075406b260 2024-04-05T22:30:23.348 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:23.350 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:30:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 msgr2=0x7f075406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 0x7f075406d720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0768006fd0 tx=0x7f0768008040 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 msgr2=0x7f0778152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f076002c450 tx=0x7f077815be20 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 shutdown_connections 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f075406b260 0x7f075406d720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f077814f670 0x7f077814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0778150470 0x7f0778152f10 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 --2- 172.21.15.73:0/2735753352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c0047c0 0x7f077806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.350+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 >> 172.21.15.73:0/2735753352 conn(0x7f07780ce8e0 msgr2=0x7f07780cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.351+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 shutdown_connections 2024-04-05T22:30:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.351+0000 7f077ee00640 1 -- 172.21.15.73:0/2735753352 wait complete. 2024-04-05T22:30:23.370 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-05T22:30:21.581743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280672,"kb_used_data":15308,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754694048,"statfs":{"total":773094113280,"available":772806705152,"internally_reserved":0,"allocated":15675392,"data_stored":7550763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.591783"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.002321+0000","last_change":"2024-04-05T22:30:17.002321+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.002321+0000","last_undegraded":"2024-04-05T22:30:17.002321+0000","last_fullsized":"2024-04-05T22:30:17.002321+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:54:04.341583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.998054+0000","last_change":"2024-04-05T22:30:16.998054+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.998054+0000","last_undegraded":"2024-04-05T22:30:16.998054+0000","last_fullsized":"2024-04-05T22:30:16.998054+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:52:26.158238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993808+0000","last_change":"2024-04-05T22:30:16.993808+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993808+0000","last_undegraded":"2024-04-05T22:30:16.993808+0000","last_fullsized":"2024-04-05T22:30:16.993808+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:13:20.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993329+0000","last_change":"2024-04-05T22:30:16.993329+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993329+0000","last_undegraded":"2024-04-05T22:30:16.993329+0000","last_fullsized":"2024-04-05T22:30:16.993329+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:48.409069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.008094+0000","last_change":"2024-04-05T22:30:17.008094+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.008094+0000","last_undegraded":"2024-04-05T22:30:17.008094+0000","last_fullsized":"2024-04-05T22:30:17.008094+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:53:54.034825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.994130+0000","last_change":"2024-04-05T22:30:16.994130+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.994130+0000","last_undegraded":"2024-04-05T22:30:16.994130+0000","last_fullsized":"2024-04-05T22:30:16.994130+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:52:26.158238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.006004+0000","last_change":"2024-04-05T22:30:17.006004+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.006004+0000","last_undegraded":"2024-04-05T22:30:17.006004+0000","last_fullsized":"2024-04-05T22:30:17.006004+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:29:18.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.999428+0000","last_change":"2024-04-05T22:30:16.999428+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.999428+0000","last_undegraded":"2024-04-05T22:30:16.999428+0000","last_fullsized":"2024-04-05T22:30:16.999428+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:17.146868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.994229+0000","last_change":"2024-04-05T22:30:16.994229+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.994229+0000","last_undegraded":"2024-04-05T22:30:16.994229+0000","last_fullsized":"2024-04-05T22:30:16.994229+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:23:26.886393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993996+0000","last_change":"2024-04-05T22:30:16.993996+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993996+0000","last_undegraded":"2024-04-05T22:30:16.993996+0000","last_fullsized":"2024-04-05T22:30:16.993996+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:07:51.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.995841+0000","last_change":"2024-04-05T22:30:16.995841+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.995841+0000","last_undegraded":"2024-04-05T22:30:16.995841+0000","last_fullsized":"2024-04-05T22:30:16.995841+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:53:44.140224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993916+0000","last_change":"2024-04-05T22:30:16.993916+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993916+0000","last_undegraded":"2024-04-05T22:30:16.993916+0000","last_fullsized":"2024-04-05T22:30:16.993916+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:07:53.876275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.002309+0000","last_change":"2024-04-05T22:30:17.002309+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.002309+0000","last_undegraded":"2024-04-05T22:30:17.002309+0000","last_fullsized":"2024-04-05T22:30:17.002309+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:31:16.737923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.995001+0000","last_change":"2024-04-05T22:30:16.995001+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.995001+0000","last_undegraded":"2024-04-05T22:30:16.995001+0000","last_fullsized":"2024-04-05T22:30:16.995001+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:45:23.186558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.997420+0000","last_change":"2024-04-05T22:30:16.997420+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.997420+0000","last_undegraded":"2024-04-05T22:30:16.997420+0000","last_fullsized":"2024-04-05T22:30:16.997420+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:33:00.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.009248+0000","last_change":"2024-04-05T22:30:17.009248+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.009248+0000","last_undegraded":"2024-04-05T22:30:17.009248+0000","last_fullsized":"2024-04-05T22:30:17.009248+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:14:59.935171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.008208+0000","last_change":"2024-04-05T22:30:17.008208+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.008208+0000","last_undegraded":"2024-04-05T22:30:17.008208+0000","last_fullsized":"2024-04-05T22:30:17.008208+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:08:09.848590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.002154+0000","last_change":"2024-04-05T22:30:17.002154+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.002154+0000","last_undegraded":"2024-04-05T22:30:17.002154+0000","last_fullsized":"2024-04-05T22:30:17.002154+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:57:26.074862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.001379+0000","last_change":"2024-04-05T22:30:17.001379+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.001379+0000","last_undegraded":"2024-04-05T22:30:17.001379+0000","last_fullsized":"2024-04-05T22:30:17.001379+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:14:16.551667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993307+0000","last_change":"2024-04-05T22:30:16.993307+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993307+0000","last_undegraded":"2024-04-05T22:30:16.993307+0000","last_fullsized":"2024-04-05T22:30:16.993307+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:38:27.092211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.001500+0000","last_change":"2024-04-05T22:30:17.001500+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.001500+0000","last_undegraded":"2024-04-05T22:30:17.001500+0000","last_fullsized":"2024-04-05T22:30:17.001500+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:06.554098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993670+0000","last_change":"2024-04-05T22:30:16.993670+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993670+0000","last_undegraded":"2024-04-05T22:30:16.993670+0000","last_fullsized":"2024-04-05T22:30:16.993670+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:54:03.325735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.001447+0000","last_change":"2024-04-05T22:30:17.001447+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.001447+0000","last_undegraded":"2024-04-05T22:30:17.001447+0000","last_fullsized":"2024-04-05T22:30:17.001447+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:05.823650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.002103+0000","last_change":"2024-04-05T22:30:17.002103+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.002103+0000","last_undegraded":"2024-04-05T22:30:17.002103+0000","last_fullsized":"2024-04-05T22:30:17.002103+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:33.403791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.006882+0000","last_change":"2024-04-05T22:30:17.006882+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.006882+0000","last_undegraded":"2024-04-05T22:30:17.006882+0000","last_fullsized":"2024-04-05T22:30:17.006882+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:37:14.159704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.001118+0000","last_change":"2024-04-05T22:30:17.001118+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.001118+0000","last_undegraded":"2024-04-05T22:30:17.001118+0000","last_fullsized":"2024-04-05T22:30:17.001118+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:58.024153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.002018+0000","last_change":"2024-04-05T22:30:17.002018+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.002018+0000","last_undegraded":"2024-04-05T22:30:17.002018+0000","last_fullsized":"2024-04-05T22:30:17.002018+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:24:19.495037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.995482+0000","last_change":"2024-04-05T22:30:16.995482+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.995482+0000","last_undegraded":"2024-04-05T22:30:16.995482+0000","last_fullsized":"2024-04-05T22:30:16.995482+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:28:49.106597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.993375+0000","last_change":"2024-04-05T22:30:16.993375+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.993375+0000","last_undegraded":"2024-04-05T22:30:16.993375+0000","last_fullsized":"2024-04-05T22:30:16.993375+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:33:00.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:17.000702+0000","last_change":"2024-04-05T22:30:17.000702+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.000702+0000","last_undegraded":"2024-04-05T22:30:17.000702+0000","last_fullsized":"2024-04-05T22:30:17.000702+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:59:48.868830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.996244+0000","last_change":"2024-04-05T22:30:16.996244+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.996244+0000","last_undegraded":"2024-04-05T22:30:16.996244+0000","last_fullsized":"2024-04-05T22:30:16.996244+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:26:03.524566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:17.988190+0000","last_change":"2024-04-05T22:30:17.988190+0000","last_active":"2024-04-05T22:30:17.988190+0000","last_peered":"2024-04-05T22:30:17.988190+0000","last_clean":"2024-04-05T22:30:17.988190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:17.988190+0000","last_undegraded":"2024-04-05T22:30:17.988190+0000","last_fullsized":"2024-04-05T22:30:17.988190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-05T22:30:16.995558+0000","last_change":"2024-04-05T22:30:16.995558+0000","last_active":"2024-04-05T22:30:16.983164+0000","last_peered":"2024-04-05T22:30:16.983164+0000","last_clean":"2024-04-05T22:30:16.983164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:16.995558+0000","last_undegraded":"2024-04-05T22:30:16.995558+0000","last_fullsized":"2024-04-05T22:30:16.995558+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:51.274819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":308,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-05T22:30:15.151020+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:30:15.151020+0000","last_peered":"2024-04-05T22:30:15.151020+0000","last_clean":"2024-04-05T22:30:15.151020+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:30:15.151020+0000","last_undegraded":"2024-04-05T22:30:15.151020+0000","last_fullsized":"2024-04-05T22:30:15.151020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542257,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37572,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334268,"statfs":{"total":96636764160,"available":96598290432,"internally_reserved":0,"allocated":2412544,"data_stored":1390257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.391},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.425}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.107},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.6180000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.155},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3540000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.216},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4490000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542258,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36988,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334852,"statfs":{"total":96636764160,"available":96598888448,"internally_reserved":0,"allocated":1818624,"data_stored":799889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":0.871}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.204}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.155}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.0009999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1806336,"data_stored":794072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.1799999999999999,"15min":1.0580000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":7.2329999999999997,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.458,"5min":1.2,"15min":1.1220000000000001},"min":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2330000000000001,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.2770000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.4039999999999999,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":17.381,"5min":17.381,"15min":17.381},"last":1.5169999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.4399999999999999,"15min":1.2529999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":24.52,"5min":24.52,"15min":24.52},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.3109999999999999,"15min":1.2729999999999999},"min":{"1min":0.99199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":7.5179999999999998,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.171}]},{"osd":3,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.121},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.329,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.4000000000000004,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.008}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.429,"15min":1.321},"min":{"1min":0.96399999999999997,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":14.933,"5min":14.933,"15min":14.933},"last":1.401},{"interface":"front","average":{"1min":1.504,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.85499999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.413,"15min":1.355},"min":{"1min":1.0289999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.1959999999999997,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.4079999999999999,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":3.782,"5min":3.782,"15min":3.782},"last":1.492}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":1.5389999999999999,"15min":1.4790000000000001},"min":{"1min":1.2509999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.5580000000000001,"15min":1.6040000000000001},"min":{"1min":1.278,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":5.1429999999999998,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.587}]}]},{"osd":4,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1806336,"data_stored":794072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1020000000000001,"15min":1.0289999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.59399999999999997},"max":{"1min":1.407,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0880000000000001},"min":{"1min":0.752,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5309999999999999,"5min":1.859,"15min":1.859},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1459999999999999,"15min":1.131},"min":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.728,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.286,"5min":1.2150000000000001,"15min":1.181},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.645,"5min":2.577,"15min":2.577},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6100000000000001,"5min":1.6579999999999999,"15min":1.974},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2490000000000001,"15min":1.2050000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.67400000000000004},"max":{"1min":1.7869999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.325,"15min":1.3620000000000001},"min":{"1min":0.82899999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7689999999999999,"5min":1.843,"15min":2.1840000000000002},"last":1.024},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4079999999999999,"15min":1.4039999999999999},"min":{"1min":1.018,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.8899999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3029999999999999,"15min":1.3},"min":{"1min":0.78800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1120000000000001,"5min":2.665,"15min":2.665},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4139999999999999,"15min":1.399},"min":{"1min":0.90200000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7490000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4550000000000001,"15min":1.458},"min":{"1min":0.85099999999999998,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":2.0800000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.472,"15min":1.4870000000000001},"min":{"1min":0.93700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.3540000000000001,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.5680000000000001,"15min":1.5449999999999999},"min":{"1min":1.0660000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.651,"15min":1.599},"min":{"1min":1.296,"5min":0.93500000000000005,"15min":0.72399999999999998},"max":{"1min":2.2530000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.82399999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1818624,"data_stored":799889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.83899999999999997},"min":{"1min":0.53600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.302,"5min":1.4910000000000001,"15min":1.744},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5369999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4279999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.008,"5min":0.95899999999999996,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3720000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.028,"15min":1.0209999999999999},"min":{"1min":0.59699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4510000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.069,"15min":1.0800000000000001},"min":{"1min":0.66700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.956,"5min":1.956,"15min":2.2549999999999999},"last":1.0209999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1719999999999999,"15min":1.222},"min":{"1min":0.76300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5509999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.248,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7649999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.2989999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2809999999999999,"15min":1.343},"min":{"1min":0.84299999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.71,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.468},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3029999999999999,"15min":1.294},"min":{"1min":0.98199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.895,"5min":1.958,"15min":2.3239999999999998},"last":1.278}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.79200000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.248},{"interface":"front","average":{"1min":1.351,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.91400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7,"5min":2.298,"15min":2.298},"last":1.4099999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.341,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.899,"5min":1.9410000000000001,"15min":2.468},"last":1.383},{"interface":"front","average":{"1min":1.343,"5min":1.3620000000000001,"15min":1.4079999999999999},"min":{"1min":0.92100000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8160000000000001,"5min":2.508,"15min":2.798},"last":1.4399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1806336,"data_stored":794072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.669}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.056},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.999}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.4730000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.8360000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.127}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.556}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.524}]}]},{"osd":1,"up_from":14,"seq":60129542258,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1806336,"data_stored":794072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3680000000000001,"5min":1.379,"15min":1.6000000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86499999999999999,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1779999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97999999999999998,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4570000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.054,"15min":1.095},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.75},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0149999999999999,"15min":0.97599999999999998},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.4530000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.171,"15min":1.141},"min":{"1min":0.84199999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.716,"5min":1.8500000000000001,"15min":1.855},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.26,"5min":1.28,"15min":1.2809999999999999},"min":{"1min":0.94499999999999995,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.575,"5min":1.806,"15min":2.2789999999999999},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.8200000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.2649999999999999,"15min":1.2410000000000001},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.173},"last":1.056}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.367,"15min":1.341},"min":{"1min":0.88900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.675,"5min":1.9059999999999999,"15min":1.917},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.357,"15min":1.3560000000000001},"min":{"1min":0.86899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.405,"15min":1.369},"min":{"1min":0.95899999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0259999999999998,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":0.98899999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2400256,"data_stored":1384440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.4339999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.90900000000000003,"15min":0.88600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.8680000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.54,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95499999999999996,"15min":0.96399999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.6639999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.244,"5min":1.976,"15min":1.976},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0580000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.581,"5min":2.157,"15min":2.157},"last":1.0940000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1919999999999999,"15min":1.1719999999999999},"min":{"1min":0.80400000000000005,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.5429999999999999,"5min":1.931,"15min":1.931},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.288,"5min":1.298,"15min":1.3},"min":{"1min":0.75900000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7569999999999999,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.1839999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2450000000000001,"15min":1.2609999999999999},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0739999999999998,"5min":2.105,"15min":2.3879999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2130000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.7010000000000001,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.3740000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.7470000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.3640000000000001},"min":{"1min":0.90100000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8720000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.0780000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.3080000000000001},"min":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.9810000000000001},"last":1.129}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:30:23.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:30:23.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.623+0000 7f990e600640 1 Processor -- start 2024-04-05T22:30:23.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.623+0000 7f990e600640 1 -- start start 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.624+0000 7f990e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f990806cb80 0x7f990806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.624+0000 7f990e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990806f5b0 con 0x7f990815e850 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.624+0000 7f990e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9908070410 con 0x7f99081587d0 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.624+0000 7f990e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990806d500 con 0x7f990806cb80 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f98fea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f990806cb80 0x7f990806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f9907e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f990815e850 0x7f99080708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45226/0 (socket says 172.21.15.73:45226) 2024-04-05T22:30:23.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f9907e00640 1 -- 172.21.15.73:0/2408597476 learned_addr learned my addr 172.21.15.73:0/2408597476 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:23.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f98fea00640 1 -- 172.21.15.73:0/2408597476 >> v1:172.21.15.157:6789/0 conn(0x7f99081587d0 legacy=0x7f99080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f98fea00640 1 -- 172.21.15.73:0/2408597476 >> v1:172.21.15.73:6789/0 conn(0x7f990815e850 legacy=0x7f99080708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f98fea00640 1 -- 172.21.15.73:0/2408597476 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080ccef0 con 0x7f990806cb80 2024-04-05T22:30:23.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.625+0000 7f9906a00640 1 -- 172.21.15.73:0/2408597476 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046612405 0 0) 0x7f990806f5b0 con 0x7f990815e850 2024-04-05T22:30:23.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.626+0000 7f98fea00640 1 --2- 172.21.15.73:0/2408597476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f990806cb80 0x7f990806cfc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097a0 tx=0x7f98f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a183acfd13ea99 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f9906a00640 1 -- 172.21.15.73:0/2408597476 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0030610 con 0x7f990806cb80 2024-04-05T22:30:23.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f9906a00640 1 -- 172.21.15.73:0/2408597476 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0030770 con 0x7f990806cb80 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f9906a00640 1 -- 172.21.15.73:0/2408597476 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0030b00 con 0x7f990806cb80 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 -- 172.21.15.73:0/2408597476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f990806cb80 msgr2=0x7f990806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 --2- 172.21.15.73:0/2408597476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f990806cb80 0x7f990806cfc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097a0 tx=0x7f98f002b960 comp rx=0 tx=0).stop 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 -- 172.21.15.73:0/2408597476 shutdown_connections 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 --2- 172.21.15.73:0/2408597476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f990806cb80 0x7f990806cfc0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 -- 172.21.15.73:0/2408597476 >> 172.21.15.73:0/2408597476 conn(0x7f99080ce8e0 msgr2=0x7f99080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 -- 172.21.15.73:0/2408597476 shutdown_connections 2024-04-05T22:30:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.627+0000 7f990e600640 1 -- 172.21.15.73:0/2408597476 wait complete. 2024-04-05T22:30:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.629+0000 7f990e600640 1 Processor -- start 2024-04-05T22:30:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.629+0000 7f990e600640 1 -- start start 2024-04-05T22:30:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990806cb80 0x7f99080acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f990815e850 0x7f9908143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080c4220 con 0x7f99081587d0 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080b3670 con 0x7f990815e850 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f990e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99080c3880 con 0x7f990806cb80 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f98fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f98fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42448/0 (socket says 172.21.15.73:42448) 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.630+0000 7f98fea00640 1 -- 172.21.15.73:0/3391875846 learned_addr learned my addr 172.21.15.73:0/3391875846 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f9907e00640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f990815e850 0x7f9908143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f9907400640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990806cb80 0x7f99080acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 -- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990806cb80 msgr2=0x7f99080acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990806cb80 0x7f99080acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 -- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f990815e850 msgr2=0x7f9908143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f990815e850 0x7f9908143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 -- 172.21.15.73:0/3391875846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009400 con 0x7f99081587d0 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98fea00640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.631+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0039c90 con 0x7f99081587d0 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.632+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f003f040 con 0x7f99081587d0 2024-04-05T22:30:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.632+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0004030 con 0x7f99081587d0 2024-04-05T22:30:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.632+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99080c3b60 con 0x7f99081587d0 2024-04-05T22:30:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.632+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99080ad080 con 0x7f99081587d0 2024-04-05T22:30:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.633+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9908066af0 con 0x7f99081587d0 2024-04-05T22:30:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.634+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f98f0004190 con 0x7f99081587d0 2024-04-05T22:30:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.635+0000 7f98ffe00640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 0x7f98e406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.635+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98f00abd00 con 0x7f99081587d0 2024-04-05T22:30:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.635+0000 7f9907400640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 0x7f98e406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:23.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.636+0000 7f9907400640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 0x7f98e406d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000f60 tx=0x7f98f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:23.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.639+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f98f0076520 con 0x7f99081587d0 2024-04-05T22:30:23.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.831+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f990806fae0 con 0x7f98e406b230 2024-04-05T22:30:23.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.853+0000 7f98ffe00640 1 -- 172.21.15.73:0/3391875846 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287764 (secure 0 0 0) 0x7f98f8001c70 con 0x7f98e406b230 2024-04-05T22:30:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:23.857 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 msgr2=0x7f98e406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 0x7f98e406d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000f60 tx=0x7f98f8009040 comp rx=0 tx=0).stop 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 msgr2=0x7f9908143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f00046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 shutdown_connections 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f98e406b230 0x7f98e406d6f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f990815e850 0x7f9908143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99081587d0 0x7f9908143050 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 --2- 172.21.15.73:0/3391875846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f990806cb80 0x7f99080acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 >> 172.21.15.73:0/3391875846 conn(0x7f99080ce8e0 msgr2=0x7f99080afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 shutdown_connections 2024-04-05T22:30:23.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:23.858+0000 7f990e600640 1 -- 172.21.15.73:0/3391875846 wait complete. 2024-04-05T22:30:23.876 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-04-05T22:30:23.582390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281056,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754693664,"statfs":{"total":773094113280,"available":772806311936,"internally_reserved":0,"allocated":16097280,"data_stored":7739372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002059"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.700727+0000","last_change":"2024-04-05T22:30:18.533329+0000","last_active":"2024-04-05T22:30:19.700727+0000","last_peered":"2024-04-05T22:30:19.700727+0000","last_clean":"2024-04-05T22:30:19.700727+0000","last_became_active":"2024-04-05T22:30:18.532882+0000","last_became_peered":"2024-04-05T22:30:18.532882+0000","last_unstale":"2024-04-05T22:30:19.700727+0000","last_undegraded":"2024-04-05T22:30:19.700727+0000","last_fullsized":"2024-04-05T22:30:19.700727+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:18.151893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699598+0000","last_change":"2024-04-05T22:30:18.534070+0000","last_active":"2024-04-05T22:30:19.699598+0000","last_peered":"2024-04-05T22:30:19.699598+0000","last_clean":"2024-04-05T22:30:19.699598+0000","last_became_active":"2024-04-05T22:30:18.533723+0000","last_became_peered":"2024-04-05T22:30:18.533723+0000","last_unstale":"2024-04-05T22:30:19.699598+0000","last_undegraded":"2024-04-05T22:30:19.699598+0000","last_fullsized":"2024-04-05T22:30:19.699598+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:12:43.382332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504625+0000","last_change":"2024-04-05T22:30:18.537048+0000","last_active":"2024-04-05T22:30:19.504625+0000","last_peered":"2024-04-05T22:30:19.504625+0000","last_clean":"2024-04-05T22:30:19.504625+0000","last_became_active":"2024-04-05T22:30:18.536744+0000","last_became_peered":"2024-04-05T22:30:18.536744+0000","last_unstale":"2024-04-05T22:30:19.504625+0000","last_undegraded":"2024-04-05T22:30:19.504625+0000","last_fullsized":"2024-04-05T22:30:19.504625+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:19:45.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501764+0000","last_change":"2024-04-05T22:30:18.509141+0000","last_active":"2024-04-05T22:30:19.501764+0000","last_peered":"2024-04-05T22:30:19.501764+0000","last_clean":"2024-04-05T22:30:19.501764+0000","last_became_active":"2024-04-05T22:30:18.508849+0000","last_became_peered":"2024-04-05T22:30:18.508849+0000","last_unstale":"2024-04-05T22:30:19.501764+0000","last_undegraded":"2024-04-05T22:30:19.501764+0000","last_fullsized":"2024-04-05T22:30:19.501764+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:34.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698041+0000","last_change":"2024-04-05T22:30:18.529483+0000","last_active":"2024-04-05T22:30:19.698041+0000","last_peered":"2024-04-05T22:30:19.698041+0000","last_clean":"2024-04-05T22:30:19.698041+0000","last_became_active":"2024-04-05T22:30:18.528652+0000","last_became_peered":"2024-04-05T22:30:18.528652+0000","last_unstale":"2024-04-05T22:30:19.698041+0000","last_undegraded":"2024-04-05T22:30:19.698041+0000","last_fullsized":"2024-04-05T22:30:19.698041+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:07.584633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500207+0000","last_change":"2024-04-05T22:30:18.506344+0000","last_active":"2024-04-05T22:30:19.500207+0000","last_peered":"2024-04-05T22:30:19.500207+0000","last_clean":"2024-04-05T22:30:19.500207+0000","last_became_active":"2024-04-05T22:30:18.506089+0000","last_became_peered":"2024-04-05T22:30:18.506089+0000","last_unstale":"2024-04-05T22:30:19.500207+0000","last_undegraded":"2024-04-05T22:30:19.500207+0000","last_fullsized":"2024-04-05T22:30:19.500207+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:18.397155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697665+0000","last_change":"2024-04-05T22:30:18.520047+0000","last_active":"2024-04-05T22:30:19.697665+0000","last_peered":"2024-04-05T22:30:19.697665+0000","last_clean":"2024-04-05T22:30:19.697665+0000","last_became_active":"2024-04-05T22:30:18.519278+0000","last_became_peered":"2024-04-05T22:30:18.519278+0000","last_unstale":"2024-04-05T22:30:19.697665+0000","last_undegraded":"2024-04-05T22:30:19.697665+0000","last_fullsized":"2024-04-05T22:30:19.697665+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:59:53.117713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697100+0000","last_change":"2024-04-05T22:30:18.528675+0000","last_active":"2024-04-05T22:30:19.697100+0000","last_peered":"2024-04-05T22:30:19.697100+0000","last_clean":"2024-04-05T22:30:19.697100+0000","last_became_active":"2024-04-05T22:30:18.528251+0000","last_became_peered":"2024-04-05T22:30:18.528251+0000","last_unstale":"2024-04-05T22:30:19.697100+0000","last_undegraded":"2024-04-05T22:30:19.697100+0000","last_fullsized":"2024-04-05T22:30:19.697100+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:04:53.805921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.700886+0000","last_change":"2024-04-05T22:30:18.531386+0000","last_active":"2024-04-05T22:30:19.700886+0000","last_peered":"2024-04-05T22:30:19.700886+0000","last_clean":"2024-04-05T22:30:19.700886+0000","last_became_active":"2024-04-05T22:30:18.531109+0000","last_became_peered":"2024-04-05T22:30:18.531109+0000","last_unstale":"2024-04-05T22:30:19.700886+0000","last_undegraded":"2024-04-05T22:30:19.700886+0000","last_fullsized":"2024-04-05T22:30:19.700886+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:54.674688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699852+0000","last_change":"2024-04-05T22:30:18.518960+0000","last_active":"2024-04-05T22:30:19.699852+0000","last_peered":"2024-04-05T22:30:19.699852+0000","last_clean":"2024-04-05T22:30:19.699852+0000","last_became_active":"2024-04-05T22:30:18.518633+0000","last_became_peered":"2024-04-05T22:30:18.518633+0000","last_unstale":"2024-04-05T22:30:19.699852+0000","last_undegraded":"2024-04-05T22:30:19.699852+0000","last_fullsized":"2024-04-05T22:30:19.699852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:40:23.338747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699023+0000","last_change":"2024-04-05T22:30:18.532284+0000","last_active":"2024-04-05T22:30:19.699023+0000","last_peered":"2024-04-05T22:30:19.699023+0000","last_clean":"2024-04-05T22:30:19.699023+0000","last_became_active":"2024-04-05T22:30:18.532040+0000","last_became_peered":"2024-04-05T22:30:18.532040+0000","last_unstale":"2024-04-05T22:30:19.699023+0000","last_undegraded":"2024-04-05T22:30:19.699023+0000","last_fullsized":"2024-04-05T22:30:19.699023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:26:21.658561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699210+0000","last_change":"2024-04-05T22:30:18.533742+0000","last_active":"2024-04-05T22:30:19.699210+0000","last_peered":"2024-04-05T22:30:19.699210+0000","last_clean":"2024-04-05T22:30:19.699210+0000","last_became_active":"2024-04-05T22:30:18.533493+0000","last_became_peered":"2024-04-05T22:30:18.533493+0000","last_unstale":"2024-04-05T22:30:19.699210+0000","last_undegraded":"2024-04-05T22:30:19.699210+0000","last_fullsized":"2024-04-05T22:30:19.699210+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:44:30.169843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504251+0000","last_change":"2024-04-05T22:30:18.523615+0000","last_active":"2024-04-05T22:30:19.504251+0000","last_peered":"2024-04-05T22:30:19.504251+0000","last_clean":"2024-04-05T22:30:19.504251+0000","last_became_active":"2024-04-05T22:30:18.520421+0000","last_became_peered":"2024-04-05T22:30:18.520421+0000","last_unstale":"2024-04-05T22:30:19.504251+0000","last_undegraded":"2024-04-05T22:30:19.504251+0000","last_fullsized":"2024-04-05T22:30:19.504251+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:22:18.397155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504093+0000","last_change":"2024-04-05T22:30:18.536542+0000","last_active":"2024-04-05T22:30:19.504093+0000","last_peered":"2024-04-05T22:30:19.504093+0000","last_clean":"2024-04-05T22:30:19.504093+0000","last_became_active":"2024-04-05T22:30:18.536206+0000","last_became_peered":"2024-04-05T22:30:18.536206+0000","last_unstale":"2024-04-05T22:30:19.504093+0000","last_undegraded":"2024-04-05T22:30:19.504093+0000","last_fullsized":"2024-04-05T22:30:19.504093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:41.005502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699038+0000","last_change":"2024-04-05T22:30:18.528509+0000","last_active":"2024-04-05T22:30:19.699038+0000","last_peered":"2024-04-05T22:30:19.699038+0000","last_clean":"2024-04-05T22:30:19.699038+0000","last_became_active":"2024-04-05T22:30:18.528187+0000","last_became_peered":"2024-04-05T22:30:18.528187+0000","last_unstale":"2024-04-05T22:30:19.699038+0000","last_undegraded":"2024-04-05T22:30:19.699038+0000","last_fullsized":"2024-04-05T22:30:19.699038+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:34.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.696151+0000","last_change":"2024-04-05T22:30:18.529905+0000","last_active":"2024-04-05T22:30:19.696151+0000","last_peered":"2024-04-05T22:30:19.696151+0000","last_clean":"2024-04-05T22:30:19.696151+0000","last_became_active":"2024-04-05T22:30:18.529595+0000","last_became_peered":"2024-04-05T22:30:18.529595+0000","last_unstale":"2024-04-05T22:30:19.696151+0000","last_undegraded":"2024-04-05T22:30:19.696151+0000","last_fullsized":"2024-04-05T22:30:19.696151+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:18:06.336141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500214+0000","last_change":"2024-04-05T22:30:18.523594+0000","last_active":"2024-04-05T22:30:19.500214+0000","last_peered":"2024-04-05T22:30:19.500214+0000","last_clean":"2024-04-05T22:30:19.500214+0000","last_became_active":"2024-04-05T22:30:18.523345+0000","last_became_peered":"2024-04-05T22:30:18.523345+0000","last_unstale":"2024-04-05T22:30:19.500214+0000","last_undegraded":"2024-04-05T22:30:19.500214+0000","last_fullsized":"2024-04-05T22:30:19.500214+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:52.279844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697697+0000","last_change":"2024-04-05T22:30:18.501586+0000","last_active":"2024-04-05T22:30:19.697697+0000","last_peered":"2024-04-05T22:30:19.697697+0000","last_clean":"2024-04-05T22:30:19.697697+0000","last_became_active":"2024-04-05T22:30:18.501346+0000","last_became_peered":"2024-04-05T22:30:18.501346+0000","last_unstale":"2024-04-05T22:30:19.697697+0000","last_undegraded":"2024-04-05T22:30:19.697697+0000","last_fullsized":"2024-04-05T22:30:19.697697+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:15:35.236542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500260+0000","last_change":"2024-04-05T22:30:18.519739+0000","last_active":"2024-04-05T22:30:19.500260+0000","last_peered":"2024-04-05T22:30:19.500260+0000","last_clean":"2024-04-05T22:30:19.500260+0000","last_became_active":"2024-04-05T22:30:18.519286+0000","last_became_peered":"2024-04-05T22:30:18.519286+0000","last_unstale":"2024-04-05T22:30:19.500260+0000","last_undegraded":"2024-04-05T22:30:19.500260+0000","last_fullsized":"2024-04-05T22:30:19.500260+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:08.689637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504531+0000","last_change":"2024-04-05T22:30:18.537615+0000","last_active":"2024-04-05T22:30:19.504531+0000","last_peered":"2024-04-05T22:30:19.504531+0000","last_clean":"2024-04-05T22:30:19.504531+0000","last_became_active":"2024-04-05T22:30:18.537333+0000","last_became_peered":"2024-04-05T22:30:18.537333+0000","last_unstale":"2024-04-05T22:30:19.504531+0000","last_undegraded":"2024-04-05T22:30:19.504531+0000","last_fullsized":"2024-04-05T22:30:19.504531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:30:08.689637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500053+0000","last_change":"2024-04-05T22:30:18.523192+0000","last_active":"2024-04-05T22:30:19.500053+0000","last_peered":"2024-04-05T22:30:19.500053+0000","last_clean":"2024-04-05T22:30:19.500053+0000","last_became_active":"2024-04-05T22:30:18.522913+0000","last_became_peered":"2024-04-05T22:30:18.522913+0000","last_unstale":"2024-04-05T22:30:19.500053+0000","last_undegraded":"2024-04-05T22:30:19.500053+0000","last_fullsized":"2024-04-05T22:30:19.500053+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:26:04.529592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697886+0000","last_change":"2024-04-05T22:30:18.524305+0000","last_active":"2024-04-05T22:30:19.697886+0000","last_peered":"2024-04-05T22:30:19.697886+0000","last_clean":"2024-04-05T22:30:19.697886+0000","last_became_active":"2024-04-05T22:30:18.524057+0000","last_became_peered":"2024-04-05T22:30:18.524057+0000","last_unstale":"2024-04-05T22:30:19.697886+0000","last_undegraded":"2024-04-05T22:30:19.697886+0000","last_fullsized":"2024-04-05T22:30:19.697886+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:46.188245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.499653+0000","last_change":"2024-04-05T22:30:18.518441+0000","last_active":"2024-04-05T22:30:19.499653+0000","last_peered":"2024-04-05T22:30:19.499653+0000","last_clean":"2024-04-05T22:30:19.499653+0000","last_became_active":"2024-04-05T22:30:18.517949+0000","last_became_peered":"2024-04-05T22:30:18.517949+0000","last_unstale":"2024-04-05T22:30:19.499653+0000","last_undegraded":"2024-04-05T22:30:19.499653+0000","last_fullsized":"2024-04-05T22:30:19.499653+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:28:41.005502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697289+0000","last_change":"2024-04-05T22:30:18.527976+0000","last_active":"2024-04-05T22:30:19.697289+0000","last_peered":"2024-04-05T22:30:19.697289+0000","last_clean":"2024-04-05T22:30:19.697289+0000","last_became_active":"2024-04-05T22:30:18.526677+0000","last_became_peered":"2024-04-05T22:30:18.526677+0000","last_unstale":"2024-04-05T22:30:19.697289+0000","last_undegraded":"2024-04-05T22:30:19.697289+0000","last_fullsized":"2024-04-05T22:30:19.697289+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:30:01.312853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504942+0000","last_change":"2024-04-05T22:30:18.531891+0000","last_active":"2024-04-05T22:30:19.504942+0000","last_peered":"2024-04-05T22:30:19.504942+0000","last_clean":"2024-04-05T22:30:19.504942+0000","last_became_active":"2024-04-05T22:30:18.531601+0000","last_became_peered":"2024-04-05T22:30:18.531601+0000","last_unstale":"2024-04-05T22:30:19.504942+0000","last_undegraded":"2024-04-05T22:30:19.504942+0000","last_fullsized":"2024-04-05T22:30:19.504942+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:10:02.716276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698321+0000","last_change":"2024-04-05T22:30:18.510495+0000","last_active":"2024-04-05T22:30:19.698321+0000","last_peered":"2024-04-05T22:30:19.698321+0000","last_clean":"2024-04-05T22:30:19.698321+0000","last_became_active":"2024-04-05T22:30:18.510032+0000","last_became_peered":"2024-04-05T22:30:18.510032+0000","last_unstale":"2024-04-05T22:30:19.698321+0000","last_undegraded":"2024-04-05T22:30:19.698321+0000","last_fullsized":"2024-04-05T22:30:19.698321+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:15:35.236542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504589+0000","last_change":"2024-04-05T22:30:18.528764+0000","last_active":"2024-04-05T22:30:19.504589+0000","last_peered":"2024-04-05T22:30:19.504589+0000","last_clean":"2024-04-05T22:30:19.504589+0000","last_became_active":"2024-04-05T22:30:18.528471+0000","last_became_peered":"2024-04-05T22:30:18.528471+0000","last_unstale":"2024-04-05T22:30:19.504589+0000","last_undegraded":"2024-04-05T22:30:19.504589+0000","last_fullsized":"2024-04-05T22:30:19.504589+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:22:16.295359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501895+0000","last_change":"2024-04-05T22:30:18.525453+0000","last_active":"2024-04-05T22:30:19.501895+0000","last_peered":"2024-04-05T22:30:19.501895+0000","last_clean":"2024-04-05T22:30:19.501895+0000","last_became_active":"2024-04-05T22:30:18.525177+0000","last_became_peered":"2024-04-05T22:30:18.525177+0000","last_unstale":"2024-04-05T22:30:19.501895+0000","last_undegraded":"2024-04-05T22:30:19.501895+0000","last_fullsized":"2024-04-05T22:30:19.501895+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:18.151893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504466+0000","last_change":"2024-04-05T22:30:18.528715+0000","last_active":"2024-04-05T22:30:19.504466+0000","last_peered":"2024-04-05T22:30:19.504466+0000","last_clean":"2024-04-05T22:30:19.504466+0000","last_became_active":"2024-04-05T22:30:18.528431+0000","last_became_peered":"2024-04-05T22:30:18.528431+0000","last_unstale":"2024-04-05T22:30:19.504466+0000","last_undegraded":"2024-04-05T22:30:19.504466+0000","last_fullsized":"2024-04-05T22:30:19.504466+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:25:33.352429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698564+0000","last_change":"2024-04-05T22:30:18.532742+0000","last_active":"2024-04-05T22:30:19.698564+0000","last_peered":"2024-04-05T22:30:19.698564+0000","last_clean":"2024-04-05T22:30:19.698564+0000","last_became_active":"2024-04-05T22:30:18.530974+0000","last_became_peered":"2024-04-05T22:30:18.530974+0000","last_unstale":"2024-04-05T22:30:19.698564+0000","last_undegraded":"2024-04-05T22:30:19.698564+0000","last_fullsized":"2024-04-05T22:30:19.698564+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:07.559123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699999+0000","last_change":"2024-04-05T22:30:18.502477+0000","last_active":"2024-04-05T22:30:19.699999+0000","last_peered":"2024-04-05T22:30:19.699999+0000","last_clean":"2024-04-05T22:30:19.699999+0000","last_became_active":"2024-04-05T22:30:18.501659+0000","last_became_peered":"2024-04-05T22:30:18.501659+0000","last_unstale":"2024-04-05T22:30:19.699999+0000","last_undegraded":"2024-04-05T22:30:19.699999+0000","last_fullsized":"2024-04-05T22:30:19.699999+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:14.233969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.696390+0000","last_change":"2024-04-05T22:30:18.499683+0000","last_active":"2024-04-05T22:30:19.696390+0000","last_peered":"2024-04-05T22:30:19.696390+0000","last_clean":"2024-04-05T22:30:19.696390+0000","last_became_active":"2024-04-05T22:30:18.499217+0000","last_became_peered":"2024-04-05T22:30:18.499217+0000","last_unstale":"2024-04-05T22:30:19.696390+0000","last_undegraded":"2024-04-05T22:30:19.696390+0000","last_fullsized":"2024-04-05T22:30:19.696390+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:03.810617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698548+0000","last_change":"2024-04-05T22:30:18.510317+0000","last_active":"2024-04-05T22:30:19.698548+0000","last_peered":"2024-04-05T22:30:19.698548+0000","last_clean":"2024-04-05T22:30:19.698548+0000","last_became_active":"2024-04-05T22:30:18.509700+0000","last_became_peered":"2024-04-05T22:30:18.509700+0000","last_unstale":"2024-04-05T22:30:19.698548+0000","last_undegraded":"2024-04-05T22:30:19.698548+0000","last_fullsized":"2024-04-05T22:30:19.698548+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:16:49.306647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"147'4","reported_seq":15,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.538220+0000","last_change":"2024-04-05T22:30:18.040654+0000","last_active":"2024-04-05T22:30:18.538220+0000","last_peered":"2024-04-05T22:30:18.538220+0000","last_clean":"2024-04-05T22:30:18.538220+0000","last_became_active":"2024-04-05T22:30:18.040260+0000","last_became_peered":"2024-04-05T22:30:18.040260+0000","last_unstale":"2024-04-05T22:30:18.538220+0000","last_undegraded":"2024-04-05T22:30:18.538220+0000","last_fullsized":"2024-04-05T22:30:18.538220+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:54:04.341583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.699533+0000","last_change":"2024-04-05T22:30:18.508596+0000","last_active":"2024-04-05T22:30:19.699533+0000","last_peered":"2024-04-05T22:30:19.699533+0000","last_clean":"2024-04-05T22:30:19.699533+0000","last_became_active":"2024-04-05T22:30:18.507848+0000","last_became_peered":"2024-04-05T22:30:18.507848+0000","last_unstale":"2024-04-05T22:30:19.699533+0000","last_undegraded":"2024-04-05T22:30:19.699533+0000","last_fullsized":"2024-04-05T22:30:19.699533+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:42.884416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"147'1","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.524788+0000","last_change":"2024-04-05T22:30:18.033898+0000","last_active":"2024-04-05T22:30:18.524788+0000","last_peered":"2024-04-05T22:30:18.524788+0000","last_clean":"2024-04-05T22:30:18.524788+0000","last_became_active":"2024-04-05T22:30:18.030369+0000","last_became_peered":"2024-04-05T22:30:18.030369+0000","last_unstale":"2024-04-05T22:30:18.524788+0000","last_undegraded":"2024-04-05T22:30:18.524788+0000","last_fullsized":"2024-04-05T22:30:18.524788+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:52:26.158238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500033+0000","last_change":"2024-04-05T22:30:18.527468+0000","last_active":"2024-04-05T22:30:19.500033+0000","last_peered":"2024-04-05T22:30:19.500033+0000","last_clean":"2024-04-05T22:30:19.500033+0000","last_became_active":"2024-04-05T22:30:18.527185+0000","last_became_peered":"2024-04-05T22:30:18.527185+0000","last_unstale":"2024-04-05T22:30:19.500033+0000","last_undegraded":"2024-04-05T22:30:19.500033+0000","last_fullsized":"2024-04-05T22:30:19.500033+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:23:29.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.519038+0000","last_change":"2024-04-05T22:30:18.030122+0000","last_active":"2024-04-05T22:30:18.519038+0000","last_peered":"2024-04-05T22:30:18.519038+0000","last_clean":"2024-04-05T22:30:18.519038+0000","last_became_active":"2024-04-05T22:30:18.029029+0000","last_became_peered":"2024-04-05T22:30:18.029029+0000","last_unstale":"2024-04-05T22:30:18.519038+0000","last_undegraded":"2024-04-05T22:30:18.519038+0000","last_fullsized":"2024-04-05T22:30:18.519038+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:13:20.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504861+0000","last_change":"2024-04-05T22:30:18.532661+0000","last_active":"2024-04-05T22:30:19.504861+0000","last_peered":"2024-04-05T22:30:19.504861+0000","last_clean":"2024-04-05T22:30:19.504861+0000","last_became_active":"2024-04-05T22:30:18.530814+0000","last_became_peered":"2024-04-05T22:30:18.530814+0000","last_unstale":"2024-04-05T22:30:19.504861+0000","last_undegraded":"2024-04-05T22:30:19.504861+0000","last_fullsized":"2024-04-05T22:30:19.504861+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:07:30.384530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.534318+0000","last_change":"2024-04-05T22:30:18.032011+0000","last_active":"2024-04-05T22:30:18.534318+0000","last_peered":"2024-04-05T22:30:18.534318+0000","last_clean":"2024-04-05T22:30:18.534318+0000","last_became_active":"2024-04-05T22:30:18.031746+0000","last_became_peered":"2024-04-05T22:30:18.031746+0000","last_unstale":"2024-04-05T22:30:18.534318+0000","last_undegraded":"2024-04-05T22:30:18.534318+0000","last_fullsized":"2024-04-05T22:30:18.534318+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:07:48.409069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697022+0000","last_change":"2024-04-05T22:30:18.505800+0000","last_active":"2024-04-05T22:30:19.697022+0000","last_peered":"2024-04-05T22:30:19.697022+0000","last_clean":"2024-04-05T22:30:19.697022+0000","last_became_active":"2024-04-05T22:30:18.505421+0000","last_became_peered":"2024-04-05T22:30:18.505421+0000","last_unstale":"2024-04-05T22:30:19.697022+0000","last_undegraded":"2024-04-05T22:30:19.697022+0000","last_fullsized":"2024-04-05T22:30:19.697022+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:03.810617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"147'1","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.534783+0000","last_change":"2024-04-05T22:30:18.035026+0000","last_active":"2024-04-05T22:30:18.534783+0000","last_peered":"2024-04-05T22:30:18.534783+0000","last_clean":"2024-04-05T22:30:18.534783+0000","last_became_active":"2024-04-05T22:30:18.034703+0000","last_became_peered":"2024-04-05T22:30:18.034703+0000","last_unstale":"2024-04-05T22:30:18.534783+0000","last_undegraded":"2024-04-05T22:30:18.534783+0000","last_fullsized":"2024-04-05T22:30:18.534783+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:53:54.034825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501514+0000","last_change":"2024-04-05T22:30:18.508810+0000","last_active":"2024-04-05T22:30:19.501514+0000","last_peered":"2024-04-05T22:30:19.501514+0000","last_clean":"2024-04-05T22:30:19.501514+0000","last_became_active":"2024-04-05T22:30:18.508543+0000","last_became_peered":"2024-04-05T22:30:18.508543+0000","last_unstale":"2024-04-05T22:30:19.501514+0000","last_undegraded":"2024-04-05T22:30:19.501514+0000","last_fullsized":"2024-04-05T22:30:19.501514+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:24:20.500063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"147'1","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.508412+0000","last_change":"2024-04-05T22:30:18.026632+0000","last_active":"2024-04-05T22:30:18.508412+0000","last_peered":"2024-04-05T22:30:18.508412+0000","last_clean":"2024-04-05T22:30:18.508412+0000","last_became_active":"2024-04-05T22:30:18.026361+0000","last_became_peered":"2024-04-05T22:30:18.026361+0000","last_unstale":"2024-04-05T22:30:18.508412+0000","last_undegraded":"2024-04-05T22:30:18.508412+0000","last_fullsized":"2024-04-05T22:30:18.508412+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:52:26.158238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501284+0000","last_change":"2024-04-05T22:30:18.526824+0000","last_active":"2024-04-05T22:30:19.501284+0000","last_peered":"2024-04-05T22:30:19.501284+0000","last_clean":"2024-04-05T22:30:19.501284+0000","last_became_active":"2024-04-05T22:30:18.526556+0000","last_became_peered":"2024-04-05T22:30:18.526556+0000","last_unstale":"2024-04-05T22:30:19.501284+0000","last_undegraded":"2024-04-05T22:30:19.501284+0000","last_fullsized":"2024-04-05T22:30:19.501284+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:19.136902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.513001+0000","last_change":"2024-04-05T22:30:18.029791+0000","last_active":"2024-04-05T22:30:18.513001+0000","last_peered":"2024-04-05T22:30:18.513001+0000","last_clean":"2024-04-05T22:30:18.513001+0000","last_became_active":"2024-04-05T22:30:18.029224+0000","last_became_peered":"2024-04-05T22:30:18.029224+0000","last_unstale":"2024-04-05T22:30:18.513001+0000","last_undegraded":"2024-04-05T22:30:18.513001+0000","last_fullsized":"2024-04-05T22:30:18.513001+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:29:18.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.499813+0000","last_change":"2024-04-05T22:30:18.505475+0000","last_active":"2024-04-05T22:30:19.499813+0000","last_peered":"2024-04-05T22:30:19.499813+0000","last_clean":"2024-04-05T22:30:19.499813+0000","last_became_active":"2024-04-05T22:30:18.505218+0000","last_became_peered":"2024-04-05T22:30:18.505218+0000","last_unstale":"2024-04-05T22:30:19.499813+0000","last_undegraded":"2024-04-05T22:30:19.499813+0000","last_fullsized":"2024-04-05T22:30:19.499813+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:10:02.716276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.491701+0000","last_change":"2024-04-05T22:30:18.023340+0000","last_active":"2024-04-05T22:30:18.491701+0000","last_peered":"2024-04-05T22:30:18.491701+0000","last_clean":"2024-04-05T22:30:18.491701+0000","last_became_active":"2024-04-05T22:30:18.022963+0000","last_became_peered":"2024-04-05T22:30:18.022963+0000","last_unstale":"2024-04-05T22:30:18.491701+0000","last_undegraded":"2024-04-05T22:30:18.491701+0000","last_fullsized":"2024-04-05T22:30:18.491701+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:17.146868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697730+0000","last_change":"2024-04-05T22:30:18.509938+0000","last_active":"2024-04-05T22:30:19.697730+0000","last_peered":"2024-04-05T22:30:19.697730+0000","last_clean":"2024-04-05T22:30:19.697730+0000","last_became_active":"2024-04-05T22:30:18.509675+0000","last_became_peered":"2024-04-05T22:30:18.509675+0000","last_unstale":"2024-04-05T22:30:19.697730+0000","last_undegraded":"2024-04-05T22:30:19.697730+0000","last_fullsized":"2024-04-05T22:30:19.697730+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:16:49.306647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.493562+0000","last_change":"2024-04-05T22:30:18.034214+0000","last_active":"2024-04-05T22:30:18.493562+0000","last_peered":"2024-04-05T22:30:18.493562+0000","last_clean":"2024-04-05T22:30:18.493562+0000","last_became_active":"2024-04-05T22:30:18.033890+0000","last_became_peered":"2024-04-05T22:30:18.033890+0000","last_unstale":"2024-04-05T22:30:18.493562+0000","last_undegraded":"2024-04-05T22:30:18.493562+0000","last_fullsized":"2024-04-05T22:30:18.493562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:23:26.886393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697192+0000","last_change":"2024-04-05T22:30:18.511339+0000","last_active":"2024-04-05T22:30:19.697192+0000","last_peered":"2024-04-05T22:30:19.697192+0000","last_clean":"2024-04-05T22:30:19.697192+0000","last_became_active":"2024-04-05T22:30:18.510961+0000","last_became_peered":"2024-04-05T22:30:18.510961+0000","last_unstale":"2024-04-05T22:30:19.697192+0000","last_undegraded":"2024-04-05T22:30:19.697192+0000","last_fullsized":"2024-04-05T22:30:19.697192+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:53:43.222210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"147'3","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.527701+0000","last_change":"2024-04-05T22:30:18.032215+0000","last_active":"2024-04-05T22:30:18.527701+0000","last_peered":"2024-04-05T22:30:18.527701+0000","last_clean":"2024-04-05T22:30:18.527701+0000","last_became_active":"2024-04-05T22:30:18.031922+0000","last_became_peered":"2024-04-05T22:30:18.031922+0000","last_unstale":"2024-04-05T22:30:18.527701+0000","last_undegraded":"2024-04-05T22:30:18.527701+0000","last_fullsized":"2024-04-05T22:30:18.527701+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:07:51.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.700785+0000","last_change":"2024-04-05T22:30:18.530152+0000","last_active":"2024-04-05T22:30:19.700785+0000","last_peered":"2024-04-05T22:30:19.700785+0000","last_clean":"2024-04-05T22:30:19.700785+0000","last_became_active":"2024-04-05T22:30:18.529880+0000","last_became_peered":"2024-04-05T22:30:18.529880+0000","last_unstale":"2024-04-05T22:30:19.700785+0000","last_undegraded":"2024-04-05T22:30:19.700785+0000","last_fullsized":"2024-04-05T22:30:19.700785+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:14:17.556693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"147'6","reported_seq":17,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.540108+0000","last_change":"2024-04-05T22:30:18.033430+0000","last_active":"2024-04-05T22:30:18.540108+0000","last_peered":"2024-04-05T22:30:18.540108+0000","last_clean":"2024-04-05T22:30:18.540108+0000","last_became_active":"2024-04-05T22:30:18.033179+0000","last_became_peered":"2024-04-05T22:30:18.033179+0000","last_unstale":"2024-04-05T22:30:18.540108+0000","last_undegraded":"2024-04-05T22:30:18.540108+0000","last_fullsized":"2024-04-05T22:30:18.540108+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:53:44.140224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697055+0000","last_change":"2024-04-05T22:30:18.523966+0000","last_active":"2024-04-05T22:30:19.697055+0000","last_peered":"2024-04-05T22:30:19.697055+0000","last_clean":"2024-04-05T22:30:19.697055+0000","last_became_active":"2024-04-05T22:30:18.523691+0000","last_became_peered":"2024-04-05T22:30:18.523691+0000","last_unstale":"2024-04-05T22:30:19.697055+0000","last_undegraded":"2024-04-05T22:30:19.697055+0000","last_fullsized":"2024-04-05T22:30:19.697055+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:41.439935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.515053+0000","last_change":"2024-04-05T22:30:18.025023+0000","last_active":"2024-04-05T22:30:18.515053+0000","last_peered":"2024-04-05T22:30:18.515053+0000","last_clean":"2024-04-05T22:30:18.515053+0000","last_became_active":"2024-04-05T22:30:18.024766+0000","last_became_peered":"2024-04-05T22:30:18.024766+0000","last_unstale":"2024-04-05T22:30:18.515053+0000","last_undegraded":"2024-04-05T22:30:18.515053+0000","last_fullsized":"2024-04-05T22:30:18.515053+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:07:53.876275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697731+0000","last_change":"2024-04-05T22:30:18.527450+0000","last_active":"2024-04-05T22:30:19.697731+0000","last_peered":"2024-04-05T22:30:19.697731+0000","last_clean":"2024-04-05T22:30:19.697731+0000","last_became_active":"2024-04-05T22:30:18.527117+0000","last_became_peered":"2024-04-05T22:30:18.527117+0000","last_unstale":"2024-04-05T22:30:19.697731+0000","last_undegraded":"2024-04-05T22:30:19.697731+0000","last_fullsized":"2024-04-05T22:30:19.697731+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:10:39.467989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.492714+0000","last_change":"2024-04-05T22:30:18.025226+0000","last_active":"2024-04-05T22:30:18.492714+0000","last_peered":"2024-04-05T22:30:18.492714+0000","last_clean":"2024-04-05T22:30:18.492714+0000","last_became_active":"2024-04-05T22:30:18.023614+0000","last_became_peered":"2024-04-05T22:30:18.023614+0000","last_unstale":"2024-04-05T22:30:18.492714+0000","last_undegraded":"2024-04-05T22:30:18.492714+0000","last_fullsized":"2024-04-05T22:30:18.492714+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:31:16.737923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501359+0000","last_change":"2024-04-05T22:30:18.523523+0000","last_active":"2024-04-05T22:30:19.501359+0000","last_peered":"2024-04-05T22:30:19.501359+0000","last_clean":"2024-04-05T22:30:19.501359+0000","last_became_active":"2024-04-05T22:30:18.523140+0000","last_became_peered":"2024-04-05T22:30:18.523140+0000","last_unstale":"2024-04-05T22:30:19.501359+0000","last_undegraded":"2024-04-05T22:30:19.501359+0000","last_fullsized":"2024-04-05T22:30:19.501359+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:05.135844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.500905+0000","last_change":"2024-04-05T22:30:18.028894+0000","last_active":"2024-04-05T22:30:18.500905+0000","last_peered":"2024-04-05T22:30:18.500905+0000","last_clean":"2024-04-05T22:30:18.500905+0000","last_became_active":"2024-04-05T22:30:18.028616+0000","last_became_peered":"2024-04-05T22:30:18.028616+0000","last_unstale":"2024-04-05T22:30:18.500905+0000","last_undegraded":"2024-04-05T22:30:18.500905+0000","last_fullsized":"2024-04-05T22:30:18.500905+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:45:23.186558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.503562+0000","last_change":"2024-04-05T22:30:18.521601+0000","last_active":"2024-04-05T22:30:19.503562+0000","last_peered":"2024-04-05T22:30:19.503562+0000","last_clean":"2024-04-05T22:30:19.503562+0000","last_became_active":"2024-04-05T22:30:18.521348+0000","last_became_peered":"2024-04-05T22:30:18.521348+0000","last_unstale":"2024-04-05T22:30:19.503562+0000","last_undegraded":"2024-04-05T22:30:19.503562+0000","last_fullsized":"2024-04-05T22:30:19.503562+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:53:07.260172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"147'8","reported_seq":19,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.536584+0000","last_change":"2024-04-05T22:30:18.034879+0000","last_active":"2024-04-05T22:30:18.536584+0000","last_peered":"2024-04-05T22:30:18.536584+0000","last_clean":"2024-04-05T22:30:18.536584+0000","last_became_active":"2024-04-05T22:30:18.033576+0000","last_became_peered":"2024-04-05T22:30:18.033576+0000","last_unstale":"2024-04-05T22:30:18.536584+0000","last_undegraded":"2024-04-05T22:30:18.536584+0000","last_fullsized":"2024-04-05T22:30:18.536584+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:33:00.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501921+0000","last_change":"2024-04-05T22:30:18.512559+0000","last_active":"2024-04-05T22:30:19.501921+0000","last_peered":"2024-04-05T22:30:19.501921+0000","last_clean":"2024-04-05T22:30:19.501921+0000","last_became_active":"2024-04-05T22:30:18.511895+0000","last_became_peered":"2024-04-05T22:30:18.511895+0000","last_unstale":"2024-04-05T22:30:19.501921+0000","last_undegraded":"2024-04-05T22:30:19.501921+0000","last_fullsized":"2024-04-05T22:30:19.501921+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:57:27.079887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"147'1","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.531765+0000","last_change":"2024-04-05T22:30:18.034870+0000","last_active":"2024-04-05T22:30:18.531765+0000","last_peered":"2024-04-05T22:30:18.531765+0000","last_clean":"2024-04-05T22:30:18.531765+0000","last_became_active":"2024-04-05T22:30:18.034589+0000","last_became_peered":"2024-04-05T22:30:18.034589+0000","last_unstale":"2024-04-05T22:30:18.531765+0000","last_undegraded":"2024-04-05T22:30:18.531765+0000","last_fullsized":"2024-04-05T22:30:18.531765+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:14:59.935171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697938+0000","last_change":"2024-04-05T22:30:18.514657+0000","last_active":"2024-04-05T22:30:19.697938+0000","last_peered":"2024-04-05T22:30:19.697938+0000","last_clean":"2024-04-05T22:30:19.697938+0000","last_became_active":"2024-04-05T22:30:18.514257+0000","last_became_peered":"2024-04-05T22:30:18.514257+0000","last_unstale":"2024-04-05T22:30:19.697938+0000","last_undegraded":"2024-04-05T22:30:19.697938+0000","last_fullsized":"2024-04-05T22:30:19.697938+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:10:39.467989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.525193+0000","last_change":"2024-04-05T22:30:18.033546+0000","last_active":"2024-04-05T22:30:18.525193+0000","last_peered":"2024-04-05T22:30:18.525193+0000","last_clean":"2024-04-05T22:30:18.525193+0000","last_became_active":"2024-04-05T22:30:18.032970+0000","last_became_peered":"2024-04-05T22:30:18.032970+0000","last_unstale":"2024-04-05T22:30:18.525193+0000","last_undegraded":"2024-04-05T22:30:18.525193+0000","last_fullsized":"2024-04-05T22:30:18.525193+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:08:09.848590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.501994+0000","last_change":"2024-04-05T22:30:18.509093+0000","last_active":"2024-04-05T22:30:19.501994+0000","last_peered":"2024-04-05T22:30:19.501994+0000","last_clean":"2024-04-05T22:30:19.501994+0000","last_became_active":"2024-04-05T22:30:18.508825+0000","last_became_peered":"2024-04-05T22:30:18.508825+0000","last_unstale":"2024-04-05T22:30:19.501994+0000","last_undegraded":"2024-04-05T22:30:19.501994+0000","last_fullsized":"2024-04-05T22:30:19.501994+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:31:17.742949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"147'1","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.533468+0000","last_change":"2024-04-05T22:30:18.040476+0000","last_active":"2024-04-05T22:30:18.533468+0000","last_peered":"2024-04-05T22:30:18.533468+0000","last_clean":"2024-04-05T22:30:18.533468+0000","last_became_active":"2024-04-05T22:30:18.040180+0000","last_became_peered":"2024-04-05T22:30:18.040180+0000","last_unstale":"2024-04-05T22:30:18.533468+0000","last_undegraded":"2024-04-05T22:30:18.533468+0000","last_fullsized":"2024-04-05T22:30:18.533468+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:57:26.074862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698232+0000","last_change":"2024-04-05T22:30:18.525791+0000","last_active":"2024-04-05T22:30:19.698232+0000","last_peered":"2024-04-05T22:30:19.698232+0000","last_clean":"2024-04-05T22:30:19.698232+0000","last_became_active":"2024-04-05T22:30:18.525532+0000","last_became_peered":"2024-04-05T22:30:18.525532+0000","last_unstale":"2024-04-05T22:30:19.698232+0000","last_undegraded":"2024-04-05T22:30:19.698232+0000","last_fullsized":"2024-04-05T22:30:19.698232+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:53:43.222210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.521378+0000","last_change":"2024-04-05T22:30:18.039099+0000","last_active":"2024-04-05T22:30:18.521378+0000","last_peered":"2024-04-05T22:30:18.521378+0000","last_clean":"2024-04-05T22:30:18.521378+0000","last_became_active":"2024-04-05T22:30:18.038656+0000","last_became_peered":"2024-04-05T22:30:18.038656+0000","last_unstale":"2024-04-05T22:30:18.521378+0000","last_undegraded":"2024-04-05T22:30:18.521378+0000","last_fullsized":"2024-04-05T22:30:18.521378+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:14:16.551667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.504803+0000","last_change":"2024-04-05T22:30:18.519702+0000","last_active":"2024-04-05T22:30:19.504803+0000","last_peered":"2024-04-05T22:30:19.504803+0000","last_clean":"2024-04-05T22:30:19.504803+0000","last_became_active":"2024-04-05T22:30:18.519286+0000","last_became_peered":"2024-04-05T22:30:18.519286+0000","last_unstale":"2024-04-05T22:30:19.504803+0000","last_undegraded":"2024-04-05T22:30:19.504803+0000","last_fullsized":"2024-04-05T22:30:19.504803+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:38:54.971655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.525350+0000","last_change":"2024-04-05T22:30:18.029409+0000","last_active":"2024-04-05T22:30:18.525350+0000","last_peered":"2024-04-05T22:30:18.525350+0000","last_clean":"2024-04-05T22:30:18.525350+0000","last_became_active":"2024-04-05T22:30:18.026874+0000","last_became_peered":"2024-04-05T22:30:18.026874+0000","last_unstale":"2024-04-05T22:30:18.525350+0000","last_undegraded":"2024-04-05T22:30:18.525350+0000","last_fullsized":"2024-04-05T22:30:18.525350+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:38:27.092211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698392+0000","last_change":"2024-04-05T22:30:18.524047+0000","last_active":"2024-04-05T22:30:19.698392+0000","last_peered":"2024-04-05T22:30:19.698392+0000","last_clean":"2024-04-05T22:30:19.698392+0000","last_became_active":"2024-04-05T22:30:18.523792+0000","last_became_peered":"2024-04-05T22:30:18.523792+0000","last_unstale":"2024-04-05T22:30:19.698392+0000","last_undegraded":"2024-04-05T22:30:19.698392+0000","last_fullsized":"2024-04-05T22:30:19.698392+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:54:39.036492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.515254+0000","last_change":"2024-04-05T22:30:18.039136+0000","last_active":"2024-04-05T22:30:18.515254+0000","last_peered":"2024-04-05T22:30:18.515254+0000","last_clean":"2024-04-05T22:30:18.515254+0000","last_became_active":"2024-04-05T22:30:18.038844+0000","last_became_peered":"2024-04-05T22:30:18.038844+0000","last_unstale":"2024-04-05T22:30:18.515254+0000","last_undegraded":"2024-04-05T22:30:18.515254+0000","last_fullsized":"2024-04-05T22:30:18.515254+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:52:06.554098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698242+0000","last_change":"2024-04-05T22:30:18.530518+0000","last_active":"2024-04-05T22:30:19.698242+0000","last_peered":"2024-04-05T22:30:19.698242+0000","last_clean":"2024-04-05T22:30:19.698242+0000","last_became_active":"2024-04-05T22:30:18.530225+0000","last_became_peered":"2024-04-05T22:30:18.530225+0000","last_unstale":"2024-04-05T22:30:19.698242+0000","last_undegraded":"2024-04-05T22:30:19.698242+0000","last_fullsized":"2024-04-05T22:30:19.698242+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:42:19.136902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.500994+0000","last_change":"2024-04-05T22:30:18.034227+0000","last_active":"2024-04-05T22:30:18.500994+0000","last_peered":"2024-04-05T22:30:18.500994+0000","last_clean":"2024-04-05T22:30:18.500994+0000","last_became_active":"2024-04-05T22:30:18.033920+0000","last_became_peered":"2024-04-05T22:30:18.033920+0000","last_unstale":"2024-04-05T22:30:18.500994+0000","last_undegraded":"2024-04-05T22:30:18.500994+0000","last_fullsized":"2024-04-05T22:30:18.500994+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:54:03.325735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.499815+0000","last_change":"2024-04-05T22:30:18.523959+0000","last_active":"2024-04-05T22:30:19.499815+0000","last_peered":"2024-04-05T22:30:19.499815+0000","last_clean":"2024-04-05T22:30:19.499815+0000","last_became_active":"2024-04-05T22:30:18.523690+0000","last_became_peered":"2024-04-05T22:30:18.523690+0000","last_unstale":"2024-04-05T22:30:19.499815+0000","last_undegraded":"2024-04-05T22:30:19.499815+0000","last_fullsized":"2024-04-05T22:30:19.499815+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:25:33.352429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.514810+0000","last_change":"2024-04-05T22:30:18.039005+0000","last_active":"2024-04-05T22:30:18.514810+0000","last_peered":"2024-04-05T22:30:18.514810+0000","last_clean":"2024-04-05T22:30:18.514810+0000","last_became_active":"2024-04-05T22:30:18.038359+0000","last_became_peered":"2024-04-05T22:30:18.038359+0000","last_unstale":"2024-04-05T22:30:18.514810+0000","last_undegraded":"2024-04-05T22:30:18.514810+0000","last_fullsized":"2024-04-05T22:30:18.514810+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:37:05.823650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698123+0000","last_change":"2024-04-05T22:30:18.509955+0000","last_active":"2024-04-05T22:30:19.698123+0000","last_peered":"2024-04-05T22:30:19.698123+0000","last_clean":"2024-04-05T22:30:19.698123+0000","last_became_active":"2024-04-05T22:30:18.509695+0000","last_became_peered":"2024-04-05T22:30:18.509695+0000","last_unstale":"2024-04-05T22:30:19.698123+0000","last_undegraded":"2024-04-05T22:30:19.698123+0000","last_fullsized":"2024-04-05T22:30:19.698123+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:59:53.117713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"147'2","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.519239+0000","last_change":"2024-04-05T22:30:18.031824+0000","last_active":"2024-04-05T22:30:18.519239+0000","last_peered":"2024-04-05T22:30:18.519239+0000","last_clean":"2024-04-05T22:30:18.519239+0000","last_became_active":"2024-04-05T22:30:18.028803+0000","last_became_peered":"2024-04-05T22:30:18.028803+0000","last_unstale":"2024-04-05T22:30:18.519239+0000","last_undegraded":"2024-04-05T22:30:18.519239+0000","last_fullsized":"2024-04-05T22:30:18.519239+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:33.403791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.698414+0000","last_change":"2024-04-05T22:30:18.518556+0000","last_active":"2024-04-05T22:30:19.698414+0000","last_peered":"2024-04-05T22:30:19.698414+0000","last_clean":"2024-04-05T22:30:19.698414+0000","last_became_active":"2024-04-05T22:30:18.518203+0000","last_became_peered":"2024-04-05T22:30:18.518203+0000","last_unstale":"2024-04-05T22:30:19.698414+0000","last_undegraded":"2024-04-05T22:30:19.698414+0000","last_fullsized":"2024-04-05T22:30:19.698414+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:30:41.439935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.507392+0000","last_change":"2024-04-05T22:30:18.033720+0000","last_active":"2024-04-05T22:30:18.507392+0000","last_peered":"2024-04-05T22:30:18.507392+0000","last_clean":"2024-04-05T22:30:18.507392+0000","last_became_active":"2024-04-05T22:30:18.033291+0000","last_became_peered":"2024-04-05T22:30:18.033291+0000","last_unstale":"2024-04-05T22:30:18.507392+0000","last_undegraded":"2024-04-05T22:30:18.507392+0000","last_fullsized":"2024-04-05T22:30:18.507392+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:37:14.159704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.500123+0000","last_change":"2024-04-05T22:30:18.509891+0000","last_active":"2024-04-05T22:30:19.500123+0000","last_peered":"2024-04-05T22:30:19.500123+0000","last_clean":"2024-04-05T22:30:19.500123+0000","last_became_active":"2024-04-05T22:30:18.509613+0000","last_became_peered":"2024-04-05T22:30:18.509613+0000","last_unstale":"2024-04-05T22:30:19.500123+0000","last_undegraded":"2024-04-05T22:30:19.500123+0000","last_fullsized":"2024-04-05T22:30:19.500123+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:19:45.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.510111+0000","last_change":"2024-04-05T22:30:18.033902+0000","last_active":"2024-04-05T22:30:18.510111+0000","last_peered":"2024-04-05T22:30:18.510111+0000","last_clean":"2024-04-05T22:30:18.510111+0000","last_became_active":"2024-04-05T22:30:18.033649+0000","last_became_peered":"2024-04-05T22:30:18.033649+0000","last_unstale":"2024-04-05T22:30:18.510111+0000","last_undegraded":"2024-04-05T22:30:18.510111+0000","last_fullsized":"2024-04-05T22:30:18.510111+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:24:58.024153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697093+0000","last_change":"2024-04-05T22:30:18.524645+0000","last_active":"2024-04-05T22:30:19.697093+0000","last_peered":"2024-04-05T22:30:19.697093+0000","last_clean":"2024-04-05T22:30:19.697093+0000","last_became_active":"2024-04-05T22:30:18.524347+0000","last_became_peered":"2024-04-05T22:30:18.524347+0000","last_unstale":"2024-04-05T22:30:19.697093+0000","last_undegraded":"2024-04-05T22:30:19.697093+0000","last_fullsized":"2024-04-05T22:30:19.697093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:18:06.336141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.494604+0000","last_change":"2024-04-05T22:30:18.034688+0000","last_active":"2024-04-05T22:30:18.494604+0000","last_peered":"2024-04-05T22:30:18.494604+0000","last_clean":"2024-04-05T22:30:18.494604+0000","last_became_active":"2024-04-05T22:30:18.034422+0000","last_became_peered":"2024-04-05T22:30:18.034422+0000","last_unstale":"2024-04-05T22:30:18.494604+0000","last_undegraded":"2024-04-05T22:30:18.494604+0000","last_fullsized":"2024-04-05T22:30:18.494604+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:24:19.495037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697196+0000","last_change":"2024-04-05T22:30:18.501968+0000","last_active":"2024-04-05T22:30:19.697196+0000","last_peered":"2024-04-05T22:30:19.697196+0000","last_clean":"2024-04-05T22:30:19.697196+0000","last_became_active":"2024-04-05T22:30:18.501687+0000","last_became_peered":"2024-04-05T22:30:18.501687+0000","last_unstale":"2024-04-05T22:30:19.697196+0000","last_undegraded":"2024-04-05T22:30:19.697196+0000","last_fullsized":"2024-04-05T22:30:19.697196+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:29:56.879639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.504258+0000","last_change":"2024-04-05T22:30:18.027242+0000","last_active":"2024-04-05T22:30:18.504258+0000","last_peered":"2024-04-05T22:30:18.504258+0000","last_clean":"2024-04-05T22:30:18.504258+0000","last_became_active":"2024-04-05T22:30:18.026965+0000","last_became_peered":"2024-04-05T22:30:18.026965+0000","last_unstale":"2024-04-05T22:30:18.504258+0000","last_undegraded":"2024-04-05T22:30:18.504258+0000","last_fullsized":"2024-04-05T22:30:18.504258+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:28:49.106597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.700853+0000","last_change":"2024-04-05T22:30:18.516726+0000","last_active":"2024-04-05T22:30:19.700853+0000","last_peered":"2024-04-05T22:30:19.700853+0000","last_clean":"2024-04-05T22:30:19.700853+0000","last_became_active":"2024-04-05T22:30:18.514948+0000","last_became_peered":"2024-04-05T22:30:18.514948+0000","last_unstale":"2024-04-05T22:30:19.700853+0000","last_undegraded":"2024-04-05T22:30:19.700853+0000","last_fullsized":"2024-04-05T22:30:19.700853+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:40:22.993353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.501976+0000","last_change":"2024-04-05T22:30:18.027530+0000","last_active":"2024-04-05T22:30:18.501976+0000","last_peered":"2024-04-05T22:30:18.501976+0000","last_clean":"2024-04-05T22:30:18.501976+0000","last_became_active":"2024-04-05T22:30:18.026342+0000","last_became_peered":"2024-04-05T22:30:18.026342+0000","last_unstale":"2024-04-05T22:30:18.501976+0000","last_undegraded":"2024-04-05T22:30:18.501976+0000","last_fullsized":"2024-04-05T22:30:18.501976+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:33:00.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.696238+0000","last_change":"2024-04-05T22:30:18.529900+0000","last_active":"2024-04-05T22:30:19.696238+0000","last_peered":"2024-04-05T22:30:19.696238+0000","last_clean":"2024-04-05T22:30:19.696238+0000","last_became_active":"2024-04-05T22:30:18.529590+0000","last_became_peered":"2024-04-05T22:30:18.529590+0000","last_unstale":"2024-04-05T22:30:19.696238+0000","last_undegraded":"2024-04-05T22:30:19.696238+0000","last_fullsized":"2024-04-05T22:30:19.696238+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:58.122855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.513158+0000","last_change":"2024-04-05T22:30:18.025476+0000","last_active":"2024-04-05T22:30:18.513158+0000","last_peered":"2024-04-05T22:30:18.513158+0000","last_clean":"2024-04-05T22:30:18.513158+0000","last_became_active":"2024-04-05T22:30:18.025059+0000","last_became_peered":"2024-04-05T22:30:18.025059+0000","last_unstale":"2024-04-05T22:30:18.513158+0000","last_undegraded":"2024-04-05T22:30:18.513158+0000","last_fullsized":"2024-04-05T22:30:18.513158+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:59:48.868830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.503988+0000","last_change":"2024-04-05T22:30:18.531662+0000","last_active":"2024-04-05T22:30:19.503988+0000","last_peered":"2024-04-05T22:30:19.503988+0000","last_clean":"2024-04-05T22:30:19.503988+0000","last_became_active":"2024-04-05T22:30:18.531353+0000","last_became_peered":"2024-04-05T22:30:18.531353+0000","last_unstale":"2024-04-05T22:30:19.503988+0000","last_undegraded":"2024-04-05T22:30:19.503988+0000","last_fullsized":"2024-04-05T22:30:19.503988+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:26:48.333674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.504524+0000","last_change":"2024-04-05T22:30:18.026493+0000","last_active":"2024-04-05T22:30:18.504524+0000","last_peered":"2024-04-05T22:30:18.504524+0000","last_clean":"2024-04-05T22:30:18.504524+0000","last_became_active":"2024-04-05T22:30:18.026129+0000","last_became_peered":"2024-04-05T22:30:18.026129+0000","last_unstale":"2024-04-05T22:30:18.504524+0000","last_undegraded":"2024-04-05T22:30:18.504524+0000","last_fullsized":"2024-04-05T22:30:18.504524+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:26:03.524566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-05T22:30:19.697914+0000","last_change":"2024-04-05T22:30:18.511319+0000","last_active":"2024-04-05T22:30:19.697914+0000","last_peered":"2024-04-05T22:30:19.697914+0000","last_clean":"2024-04-05T22:30:19.697914+0000","last_became_active":"2024-04-05T22:30:18.510776+0000","last_became_peered":"2024-04-05T22:30:18.510776+0000","last_unstale":"2024-04-05T22:30:19.697914+0000","last_undegraded":"2024-04-05T22:30:19.697914+0000","last_fullsized":"2024-04-05T22:30:19.697914+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:17.988190+0000","last_clean_scrub_stamp":"2024-04-05T22:30:17.988190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:09:58.122855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.504867+0000","last_change":"2024-04-05T22:30:18.030259+0000","last_active":"2024-04-05T22:30:18.504867+0000","last_peered":"2024-04-05T22:30:18.504867+0000","last_clean":"2024-04-05T22:30:18.504867+0000","last_became_active":"2024-04-05T22:30:18.029978+0000","last_became_peered":"2024-04-05T22:30:18.029978+0000","last_unstale":"2024-04-05T22:30:18.504867+0000","last_undegraded":"2024-04-05T22:30:18.504867+0000","last_fullsized":"2024-04-05T22:30:18.504867+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:16.983164+0000","last_clean_scrub_stamp":"2024-04-05T22:30:16.983164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:51.274819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":311,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-05T22:30:18.514902+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:30:18.514902+0000","last_peered":"2024-04-05T22:30:18.514902+0000","last_clean":"2024-04-05T22:30:18.514902+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:30:18.514902+0000","last_undegraded":"2024-04-05T22:30:18.514902+0000","last_fullsized":"2024-04-05T22:30:18.514902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542258,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37636,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334204,"statfs":{"total":96636764160,"available":96598224896,"internally_reserved":0,"allocated":2473984,"data_stored":1411913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.071,"15min":1.0449999999999999},"min":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.099,"15min":1.036},"min":{"1min":0.81699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1699999999999999,"15min":1.0860000000000001},"min":{"1min":0.76100000000000001,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.391},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.071},"min":{"1min":0.85299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.796,"5min":1.819,"15min":1.819},"last":1.425}]},{"osd":2,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.1850000000000001,"15min":1.1819999999999999},"min":{"1min":1.036,"5min":0.71699999999999997,"15min":0.66900000000000004},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.107},{"interface":"front","average":{"1min":1.448,"5min":1.2789999999999999,"15min":1.242},"min":{"1min":1.2609999999999999,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.6180000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.21,"15min":1.1559999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.6359999999999999,"5min":1.667,"15min":1.712},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.2410000000000001,"15min":1.2270000000000001},"min":{"1min":0.95899999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.95699999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.155},{"interface":"front","average":{"1min":1.5,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3540000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.341,"15min":1.272},"min":{"1min":0.88200000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.216},{"interface":"front","average":{"1min":1.556,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.1599999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.544,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.90600000000000003,"15min":0.83799999999999997},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.5089999999999999,"15min":1.5049999999999999},"min":{"1min":1.22,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4490000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37028,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334812,"statfs":{"total":96636764160,"available":96598847488,"internally_reserved":0,"allocated":1859584,"data_stored":819261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0529999999999999},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.45500000000000002},"max":{"1min":1.462,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.0780000000000001,"15min":1.0880000000000001},"min":{"1min":0.71099999999999997,"5min":0.66800000000000004,"15min":0.439},"max":{"1min":1.589,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":1.4470000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.139},"min":{"1min":0.72399999999999998,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.542,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.115,"15min":1.141},"min":{"1min":0.746,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.369}]},{"osd":2,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.095,"15min":1.1719999999999999},"min":{"1min":0.83999999999999997,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.3839999999999999,"5min":1.6220000000000001,"15min":2.3450000000000002},"last":1.276},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.153,"15min":1.196},"min":{"1min":0.82699999999999996,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.506,"5min":1.706,"15min":2.2250000000000001},"last":1.054}]},{"osd":3,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1830000000000001,"15min":1.2889999999999999},"min":{"1min":0.77100000000000002,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":1.96,"15min":1.96},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2609999999999999,"15min":1.3640000000000001},"min":{"1min":0.97299999999999998,"5min":0.77500000000000002,"15min":0.68100000000000005},"max":{"1min":1.466,"5min":1.8140000000000001,"15min":3.1720000000000002},"last":1.7549999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.302,"15min":1.3260000000000001},"min":{"1min":1.0429999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8340000000000001,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.5229999999999999},"min":{"1min":1.1080000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.639,"5min":1.9259999999999999,"15min":3.27},"last":1.6619999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3069999999999999,"15min":1.409},"min":{"1min":0.98799999999999999,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.776,"5min":1.927,"15min":2.3650000000000002},"last":1.421},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.4770000000000001,"15min":1.5369999999999999},"min":{"1min":1.0669999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9330000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.702}]},{"osd":7,"last update":"Fri Apr 5 22:29:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.393,"15min":1.506},"min":{"1min":1.258,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8200000000000001,"5min":1.99,"15min":3.3540000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.4199999999999999,"15min":1.524},"min":{"1min":1.2629999999999999,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.877,"5min":1.899,"15min":3.0920000000000001},"last":1.482}]}]},{"osd":5,"up_from":14,"seq":60129542258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37032,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334808,"statfs":{"total":96636764160,"available":96598843392,"internally_reserved":0,"allocated":1863680,"data_stored":819843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.1799999999999999,"15min":1.0580000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":7.2329999999999997,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.216},{"interface":"front","average":{"1min":1.458,"5min":1.2,"15min":1.1220000000000001},"min":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2330000000000001,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.2770000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.427},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.4039999999999999,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":17.381,"5min":17.381,"15min":17.381},"last":1.0429999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.4399999999999999,"15min":1.2529999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":24.52,"5min":24.52,"15min":24.52},"last":0.753},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.3109999999999999,"15min":1.2729999999999999},"min":{"1min":0.99199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":7.5179999999999998,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.266},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.329,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.4000000000000004,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.1359999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.429,"15min":1.321},"min":{"1min":0.96399999999999997,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":14.933,"5min":14.933,"15min":14.933},"last":1.375},{"interface":"front","average":{"1min":1.504,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.85499999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.347}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.413,"15min":1.355},"min":{"1min":1.0289999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.1959999999999997,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.196},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.4079999999999999,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":3.782,"5min":3.782,"15min":3.782},"last":1.4810000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":1.5389999999999999,"15min":1.4790000000000001},"min":{"1min":1.2509999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.5580000000000001,"15min":1.6040000000000001},"min":{"1min":1.278,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":5.1429999999999998,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.1579999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542257,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1871872,"data_stored":820873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1020000000000001,"15min":1.0289999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.59399999999999997},"max":{"1min":1.407,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0880000000000001},"min":{"1min":0.752,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5309999999999999,"5min":1.859,"15min":1.859},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1459999999999999,"15min":1.131},"min":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.60399999999999998},"max":{"1min":1.728,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.286,"5min":1.2150000000000001,"15min":1.181},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.645,"5min":2.577,"15min":2.577},"last":1.0840000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6100000000000001,"5min":1.6579999999999999,"15min":1.974},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2490000000000001,"15min":1.2050000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.67400000000000004},"max":{"1min":1.7869999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.325,"15min":1.3620000000000001},"min":{"1min":0.82899999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7689999999999999,"5min":1.843,"15min":2.1840000000000002},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4079999999999999,"15min":1.4039999999999999},"min":{"1min":1.018,"5min":0.91200000000000003,"15min":0.874},"max":{"1min":1.8899999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.3440000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3029999999999999,"15min":1.3},"min":{"1min":0.78800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.1120000000000001,"5min":2.665,"15min":2.665},"last":1.22},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4139999999999999,"15min":1.399},"min":{"1min":0.90200000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7490000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.383}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4550000000000001,"15min":1.458},"min":{"1min":0.85099999999999998,"5min":0.64900000000000002,"15min":0.53500000000000003},"max":{"1min":2.0800000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.246},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.472,"15min":1.4870000000000001},"min":{"1min":0.93700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.3540000000000001,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.2809999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.5680000000000001,"15min":1.5449999999999999},"min":{"1min":1.0660000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.651,"15min":1.599},"min":{"1min":1.296,"5min":0.93500000000000005,"15min":0.72399999999999998},"max":{"1min":2.2530000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.466}]}]},{"osd":3,"up_from":14,"seq":60129542258,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1859584,"data_stored":819273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.83899999999999997},"min":{"1min":0.53600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.302,"5min":1.4910000000000001,"15min":1.744},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5369999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.0309999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4279999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.008,"5min":0.95899999999999996,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3720000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.87}]},{"osd":2,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.028,"15min":1.0209999999999999},"min":{"1min":0.59699999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4510000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.069,"15min":1.0800000000000001},"min":{"1min":0.66700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.956,"5min":1.956,"15min":2.2549999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1719999999999999,"15min":1.222},"min":{"1min":0.76300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5509999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.248,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7649999999999999,"5min":1.7909999999999999,"15min":2.1619999999999999},"last":1.0660000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2809999999999999,"15min":1.343},"min":{"1min":0.84299999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.71,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3029999999999999,"15min":1.294},"min":{"1min":0.98199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.895,"5min":1.958,"15min":2.3239999999999998},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.79200000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.391},{"interface":"front","average":{"1min":1.351,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.91400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7,"5min":2.298,"15min":2.298},"last":1.2370000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:29:29 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.341,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.899,"5min":1.9410000000000001,"15min":2.468},"last":1.421},{"interface":"front","average":{"1min":1.343,"5min":1.3620000000000001,"15min":1.4079999999999999},"min":{"1min":0.92100000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8160000000000001,"5min":2.508,"15min":2.798},"last":1.3}]}]},{"osd":2,"up_from":14,"seq":60129542257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1859584,"data_stored":819329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85699999999999998,"15min":0.83599999999999997},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3360000000000001,"5min":2.879,"15min":2.879},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.036,"5min":0.99399999999999999,"15min":0.93700000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.258}]},{"osd":1,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83199999999999996,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.252,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91900000000000004,"15min":0.89400000000000002},"min":{"1min":0.56299999999999994,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.538,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.93799999999999994}]},{"osd":3,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.52400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6060000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.53200000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.675,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3169999999999999,"15min":1.2569999999999999},"min":{"1min":0.93400000000000005,"5min":0.81399999999999995,"15min":0.57799999999999996},"max":{"1min":2.1989999999999998,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.121,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.4079999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7849999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.359},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.248,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.157,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.1160000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"max":{"1min":2.218,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.282},"min":{"1min":0.80100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8979999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.591}]},{"osd":7,"last update":"Fri Apr 5 22:29:22 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.339,"15min":1.3560000000000001},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0680000000000001,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.401,"15min":1.397},"min":{"1min":0.98999999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.286,"5min":2.597,"15min":2.597},"last":1.1910000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542259,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1859584,"data_stored":819273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3680000000000001,"5min":1.379,"15min":1.6000000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0640000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86499999999999999,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1779999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97999999999999998,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.4570000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.1279999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.054,"15min":1.095},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.75},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.0149999999999999,"15min":0.97599999999999998},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.4530000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.167}]},{"osd":4,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.171,"15min":1.141},"min":{"1min":0.84199999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.716,"5min":1.8500000000000001,"15min":1.855},"last":1.256},{"interface":"front","average":{"1min":1.26,"5min":1.28,"15min":1.2809999999999999},"min":{"1min":0.94499999999999995,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.575,"5min":1.806,"15min":2.2789999999999999},"last":1.367}]},{"osd":5,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.8200000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.319},{"interface":"front","average":{"1min":1.323,"5min":1.2649999999999999,"15min":1.2410000000000001},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.173},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.367,"15min":1.341},"min":{"1min":0.88900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.675,"5min":1.9059999999999999,"15min":1.917},"last":1.516},{"interface":"front","average":{"1min":1.391,"5min":1.357,"15min":1.3560000000000001},"min":{"1min":0.86899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":1.4039999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:29:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.405,"15min":1.369},"min":{"1min":0.95899999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0259999999999998,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.8660000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.751},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.468}]}]},{"osd":0,"up_from":14,"seq":60129542259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33516,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338324,"statfs":{"total":96636764160,"available":96602443776,"internally_reserved":0,"allocated":2449408,"data_stored":1409607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"max":{"1min":1.4339999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.90900000000000003,"15min":0.88600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.8680000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":1.143}]},{"osd":2,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.54,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.747},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95499999999999996,"15min":0.96399999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.318},"max":{"1min":1.6639999999999999,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.3600000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.244,"5min":1.976,"15min":1.976},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0580000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.581,"5min":2.157,"15min":2.157},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1919999999999999,"15min":1.1719999999999999},"min":{"1min":0.80400000000000005,"5min":0.69999999999999996,"15min":0.66700000000000004},"max":{"1min":1.5429999999999999,"5min":1.931,"15min":1.931},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.298,"15min":1.3},"min":{"1min":0.75900000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7569999999999999,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2450000000000001,"15min":1.2609999999999999},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0739999999999998,"5min":2.105,"15min":2.3879999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2130000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.7010000000000001,"5min":1.7110000000000001,"15min":1.9550000000000001},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.3740000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.7470000000000001,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":0.873},{"interface":"front","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.3640000000000001},"min":{"1min":0.90100000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8720000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.234}]},{"osd":7,"last update":"Fri Apr 5 22:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.3080000000000001},"min":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.9810000000000001},"last":1.413}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:30:23.881 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:30:23.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:30:23.924 INFO:tasks.ceph.mds.a.smithi073.stderr:2024-04-05T22:30:23.925+0000 7f7356800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:30:24.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.143+0000 7f8be0400640 1 Processor -- start 2024-04-05T22:30:24.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.143+0000 7f8be0400640 1 -- start start 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd806c630 0x7f8bd80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bd815dd90 0x7f8bd806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8bd8160930 0x7f8bd8160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd8146ac0 con 0x7f8bd815dd90 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd806cf50 con 0x7f8bd8160930 2024-04-05T22:30:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8be0400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8070720 con 0x7f8bd806c630 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8bdd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bd815dd90 0x7f8bd806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8bdd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8bd815dd90 0x7f8bd806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42450/0 (socket says 172.21.15.73:42450) 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8bdd000640 1 -- 172.21.15.73:0/2168003577 learned_addr learned my addr 172.21.15.73:0/2168003577 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8bdda00640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd806c630 0x7f8bd80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.145+0000 7f8bde400640 1 --2- 172.21.15.73:0/2168003577 >> v2:172.21.15.157:3300/0 conn(0x7f8bd8160930 0x7f8bd8160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.146+0000 7f8bdd000640 1 -- 172.21.15.73:0/2168003577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd806c630 msgr2=0x7f8bd80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:24.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.146+0000 7f8bdd000640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd806c630 0x7f8bd80ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.146+0000 7f8bdd000640 1 -- 172.21.15.73:0/2168003577 >> v2:172.21.15.157:3300/0 conn(0x7f8bd8160930 msgr2=0x7f8bd8160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:24.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.146+0000 7f8bdd000640 1 --2- 172.21.15.73:0/2168003577 >> v2:172.21.15.157:3300/0 conn(0x7f8bd8160930 0x7f8bd8160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.146+0000 7f8bdd000640 1 -- 172.21.15.73:0/2168003577 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80acf60 con 0x7f8bd815dd90 2024-04-05T22:30:24.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.147+0000 7f8bdd000640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd815dd90 0x7f8bd806ca10 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8066a70 tx=0x7f8bc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969b8c05219f6e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.147+0000 7f8bd7e00640 1 -- 172.21.15.73:0/2168003577 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4031870 con 0x7f8bd815dd90 2024-04-05T22:30:24.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.147+0000 7f8bd7e00640 1 -- 172.21.15.73:0/2168003577 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc40319d0 con 0x7f8bd815dd90 2024-04-05T22:30:24.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.147+0000 7f8bd7e00640 1 -- 172.21.15.73:0/2168003577 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4031d60 con 0x7f8bd815dd90 2024-04-05T22:30:24.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 -- 172.21.15.73:0/2168003577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd815dd90 msgr2=0x7f8bd806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd815dd90 0x7f8bd806ca10 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8066a70 tx=0x7f8bc402b910 comp rx=0 tx=0).stop 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 -- 172.21.15.73:0/2168003577 shutdown_connections 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd815dd90 0x7f8bd806ca10 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 --2- 172.21.15.73:0/2168003577 >> v2:172.21.15.157:3300/0 conn(0x7f8bd8160930 0x7f8bd8160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 --2- 172.21.15.73:0/2168003577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd806c630 0x7f8bd80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 -- 172.21.15.73:0/2168003577 >> 172.21.15.73:0/2168003577 conn(0x7f8bd80b0830 msgr2=0x7f8bd80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 -- 172.21.15.73:0/2168003577 shutdown_connections 2024-04-05T22:30:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.148+0000 7f8be0400640 1 -- 172.21.15.73:0/2168003577 wait complete. 2024-04-05T22:30:24.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.151+0000 7f8be0400640 1 Processor -- start 2024-04-05T22:30:24.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.151+0000 7f8be0400640 1 -- start start 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815dd90 0x7f8bd80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8160930 0x7f8bd80b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8134f30 con 0x7f8bd806c630 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8137e80 con 0x7f8bd8160930 2024-04-05T22:30:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8be0400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80ccf40 con 0x7f8bd815dd90 2024-04-05T22:30:24.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8bdda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8bdda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42452/0 (socket says 172.21.15.73:42452) 2024-04-05T22:30:24.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.152+0000 7f8bdda00640 1 -- 172.21.15.73:0/915864267 learned_addr learned my addr 172.21.15.73:0/915864267 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdd000640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815dd90 0x7f8bd80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bde400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8160930 0x7f8bd80b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 -- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815dd90 msgr2=0x7f8bd80cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815dd90 0x7f8bd80cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 -- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8160930 msgr2=0x7f8bd80b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8160930 0x7f8bd80b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 -- 172.21.15.73:0/915864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40094d0 con 0x7f8bd806c630 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bdda00640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d800 tx=0x7f8bcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0096a0 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc009800 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.153+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc009990 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.154+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd80cd190 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.154+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd80cd750 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.155+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd8068160 con 0x7f8bd806c630 2024-04-05T22:30:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.156+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f8bcc009af0 con 0x7f8bd806c630 2024-04-05T22:30:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.157+0000 7f8bd5600640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 0x7f8bb806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.157+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bcc08a3e0 con 0x7f8bd806c630 2024-04-05T22:30:24.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.157+0000 7f8bdd000640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 0x7f8bb806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.159+0000 7f8bdd000640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 0x7f8bb806d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4036000 tx=0x7f8bc403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.161+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8bcc054c40 con 0x7f8bd806c630 2024-04-05T22:30:24.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.421+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8bd80adb90 con 0x7f8bd806c630 2024-04-05T22:30:24.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.426+0000 7f8bd5600640 1 -- 172.21.15.73:0/915864267 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8bcc00b3a0 con 0x7f8bd806c630 2024-04-05T22:30:24.425 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:30:24.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 msgr2=0x7f8bb806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 0x7f8bb806d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4036000 tx=0x7f8bc403c040 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 msgr2=0x7f8bd8137940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d800 tx=0x7f8bcc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 shutdown_connections 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8bb806b230 0x7f8bb806d6f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8160930 0x7f8bd80b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815dd90 0x7f8bd80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 --2- 172.21.15.73:0/915864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bd806c630 0x7f8bd8137940 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 >> 172.21.15.73:0/915864267 conn(0x7f8bd80b0830 msgr2=0x7f8bd815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 shutdown_connections 2024-04-05T22:30:24.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.429+0000 7f8be0400640 1 -- 172.21.15.73:0/915864267 wait complete. 2024-04-05T22:30:24.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:24.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.649+0000 7fee97200640 1 Processor -- start 2024-04-05T22:30:24.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.649+0000 7fee97200640 1 -- start start 2024-04-05T22:30:24.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.650+0000 7fee97200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee980d1600 0x7fee9815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee97200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee980d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee97200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee980b2420 con 0x7fee9815ccd0 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee97200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee980b2770 con 0x7fee980d1600 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee97200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980ce380 con 0x7fee980d35e0 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee980d1600 0x7fee9815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee980d1600 0x7fee9815d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42456/0 (socket says 172.21.15.73:42456) 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 -- 172.21.15.73:0/66775412 learned_addr learned my addr 172.21.15.73:0/66775412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 -- 172.21.15.73:0/66775412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 msgr2=0x7fee980d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 --2- 172.21.15.73:0/66775412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee980d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 -- 172.21.15.73:0/66775412 >> v1:172.21.15.157:6789/0 conn(0x7fee9815ccd0 legacy=0x7fee980d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:24.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.651+0000 7fee8d400640 1 -- 172.21.15.73:0/66775412 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980ce660 con 0x7fee980d1600 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.652+0000 7fee8d400640 1 --2- 172.21.15.73:0/66775412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee9815d0b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fee80009870 tx=0x7fee8002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6befd7f2eed8c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee95400640 1 -- 172.21.15.73:0/66775412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80031740 con 0x7fee980d1600 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee95400640 1 -- 172.21.15.73:0/66775412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee800318a0 con 0x7fee980d1600 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee95400640 1 -- 172.21.15.73:0/66775412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80031c30 con 0x7fee980d1600 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 -- 172.21.15.73:0/66775412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 msgr2=0x7fee9815d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 --2- 172.21.15.73:0/66775412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee9815d0b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fee80009870 tx=0x7fee8002b7e0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 -- 172.21.15.73:0/66775412 shutdown_connections 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 --2- 172.21.15.73:0/66775412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee980d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 --2- 172.21.15.73:0/66775412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee9815d0b0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.653+0000 7fee97200640 1 -- 172.21.15.73:0/66775412 >> 172.21.15.73:0/66775412 conn(0x7fee980d2ac0 msgr2=0x7fee980d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:24.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.654+0000 7fee97200640 1 -- 172.21.15.73:0/66775412 shutdown_connections 2024-04-05T22:30:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.654+0000 7fee97200640 1 -- 172.21.15.73:0/66775412 wait complete. 2024-04-05T22:30:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.655+0000 7fee97200640 1 Processor -- start 2024-04-05T22:30:24.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.655+0000 7fee97200640 1 -- start start 2024-04-05T22:30:24.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee9815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9815ccd0 0x7fee9815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980b6d90 con 0x7fee980d1600 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980caed0 con 0x7fee9815ccd0 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.656+0000 7fee97200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980b5c70 con 0x7fee980d35e0 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42462/0 (socket says 172.21.15.73:42462) 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 -- 172.21.15.73:0/2454386704 learned_addr learned my addr 172.21.15.73:0/2454386704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:24.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 -- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 msgr2=0x7fee9815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee8d400640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee9815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee9815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 -- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9815ccd0 msgr2=0x7fee9815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9815ccd0 0x7fee9815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 -- 172.21.15.73:0/2454386704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee800094d0 con 0x7fee980d1600 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.657+0000 7fee95e00640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b420 tx=0x7fee8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.658+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011830 con 0x7fee980d1600 2024-04-05T22:30:24.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.658+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88011990 con 0x7fee980d1600 2024-04-05T22:30:24.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.658+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011d40 con 0x7fee980d1600 2024-04-05T22:30:24.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.658+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980b5f50 con 0x7fee980d1600 2024-04-05T22:30:24.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.658+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980b6510 con 0x7fee980d1600 2024-04-05T22:30:24.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.659+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee98067250 con 0x7fee980d1600 2024-04-05T22:30:24.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.660+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fee8801b460 con 0x7fee980d1600 2024-04-05T22:30:24.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.661+0000 7fee8ea00640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 0x7fee7406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:24.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.661+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee88089af0 con 0x7fee980d1600 2024-04-05T22:30:24.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.662+0000 7fee8d400640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 0x7fee7406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:24.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.663+0000 7fee8d400640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 0x7fee7406d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee80035000 tx=0x7fee80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:24.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.666+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fee88054390 con 0x7fee980d1600 2024-04-05T22:30:24.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.895+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee9804e160 con 0x7fee980d1600 2024-04-05T22:30:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.896+0000 7fee8ea00640 1 -- 172.21.15.73:0/2454386704 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fee880579b0 con 0x7fee980d1600 2024-04-05T22:30:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:24.895 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:24.895 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 91 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 msgr2=0x7fee7406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 0x7fee7406d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee80035000 tx=0x7fee80008030 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 msgr2=0x7fee980cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b420 tx=0x7fee8800b8e0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 shutdown_connections 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fee7406b230 0x7fee7406d6f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9815ccd0 0x7fee9815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee980d35e0 0x7fee9815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 --2- 172.21.15.73:0/2454386704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee980d1600 0x7fee980cb2a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.898+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 >> 172.21.15.73:0/2454386704 conn(0x7fee980d2ac0 msgr2=0x7fee980b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.899+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 shutdown_connections 2024-04-05T22:30:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:24.899+0000 7fee97200640 1 -- 172.21.15.73:0/2454386704 wait complete. 2024-04-05T22:30:25.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:30:18.482729+0000', 'modified': '2024-04-05T22:30:21.949075+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9270': {'gid': 9270, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6835/2701059182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6833', 'nonce': 2701059182}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2701059182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:30:25.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:30:25.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-05T22:30:25.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.539+0000 7f5cfb800640 1 Processor -- start 2024-04-05T22:30:25.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.539+0000 7f5cfb800640 1 -- start start 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cfb800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5cf406cb80 0x7f5cf406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cfb800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf406f5b0 con 0x7f5cf415e850 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cfb800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf4070410 con 0x7f5cf41587d0 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cfb800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf406d500 con 0x7f5cf406cb80 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cf9800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5cf415e850 0x7f5cf40708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45238/0 (socket says 172.21.15.73:45238) 2024-04-05T22:30:25.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.541+0000 7f5cf9800640 1 -- 172.21.15.73:0/3271900455 learned_addr learned my addr 172.21.15.73:0/3271900455 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255171917 0 0) 0x7f5cf406f5b0 con 0x7f5cf415e850 2024-04-05T22:30:25.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cdc003520 con 0x7f5cf415e850 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3271900455 >> v2:172.21.15.157:3300/0 conn(0x7f5cf406cb80 0x7f5cf406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496509701 0 0) 0x7f5cdc003520 con 0x7f5cf415e850 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 >> v2:172.21.15.157:3300/0 conn(0x7f5cf406cb80 msgr2=0x7f5cf406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 --2- 172.21.15.73:0/3271900455 >> v2:172.21.15.157:3300/0 conn(0x7f5cf406cb80 0x7f5cf406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 >> v1:172.21.15.157:6789/0 conn(0x7f5cf41587d0 legacy=0x7f5cf40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40ccef0 con 0x7f5cf415e850 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5cec0019f0 con 0x7f5cf415e850 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 >> v1:172.21.15.73:6789/0 conn(0x7f5cf415e850 legacy=0x7f5cf40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc003970 0x7f5cdc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc0047c0 0x7f5cdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 0x7f5cdc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf406d500 con 0x7f5cdc005180 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc00c0b0 con 0x7f5cdc0047c0 2024-04-05T22:30:25.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.542+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc00c270 con 0x7f5cdc003970 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 0x7f5cdc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc003970 0x7f5cdc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc0047c0 0x7f5cdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 -- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc003970 msgr2=0x7f5cdc003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc003970 0x7f5cdc003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 -- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc0047c0 msgr2=0x7f5cdc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc0047c0 0x7f5cdc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf9800640 1 -- 172.21.15.73:0/3271900455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc003520 con 0x7f5cdc005180 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc0047c0 0x7f5cdc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:25.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.543+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc003970 0x7f5cdc003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:25.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.544+0000 7f5cf9800640 1 -- 172.21.15.73:0/3271900455 reap_dead start 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.544+0000 7f5cf9800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 0x7f5cdc00b980 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5cec002230 tx=0x7f5cdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f28e75ed055050 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.544+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec034630 con 0x7f5cdc005180 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.544+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec02f070 con 0x7f5cdc005180 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.544+0000 7f5cf3400640 1 -- 172.21.15.73:0/3271900455 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0349f0 con 0x7f5cdc005180 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 -- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 msgr2=0x7f5cdc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 0x7f5cdc00b980 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5cec002230 tx=0x7f5cdc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 -- 172.21.15.73:0/3271900455 shutdown_connections 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 --2- 172.21.15.73:0/3271900455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cdc005180 0x7f5cdc00b980 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 -- 172.21.15.73:0/3271900455 >> 172.21.15.73:0/3271900455 conn(0x7f5cf40ce8e0 msgr2=0x7f5cf40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 -- 172.21.15.73:0/3271900455 shutdown_connections 2024-04-05T22:30:25.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.545+0000 7f5cfb800640 1 -- 172.21.15.73:0/3271900455 wait complete. 2024-04-05T22:30:25.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.547+0000 7f5cfb800640 1 Processor -- start 2024-04-05T22:30:25.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.548+0000 7f5cfb800640 1 -- start start 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.548+0000 7f5cfb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc005180 0x7f5cf4142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cfb800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c4900 0x7f5cf40c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cfb800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cfb800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c6130 con 0x7f5cf40708a0 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cfb800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4071260 con 0x7f5cf40c4900 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cfb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41f3830 con 0x7f5cdc005180 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42486/0 (socket says 172.21.15.73:42486) 2024-04-05T22:30:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 -- 172.21.15.73:0/480037433 learned_addr learned my addr 172.21.15.73:0/480037433 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc005180 0x7f5cf4142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 -- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc005180 msgr2=0x7f5cf4142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc005180 0x7f5cf4142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c4900 0x7f5cf40c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 -- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c4900 msgr2=0x7f5cf40c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c4900 0x7f5cf40c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.549+0000 7f5cf9800640 1 -- 172.21.15.73:0/480037433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40b2b70 con 0x7f5cf40708a0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cf9800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000f80 tx=0x7f5cec001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cf0c00640 1 -- 172.21.15.73:0/480037433 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec004d60 con 0x7f5cf40708a0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cf0c00640 1 -- 172.21.15.73:0/480037433 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec03f470 con 0x7f5cf40708a0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cf0c00640 1 -- 172.21.15.73:0/480037433 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec03f800 con 0x7f5cf40708a0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40b2e50 con 0x7f5cf40708a0 2024-04-05T22:30:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.550+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40b3410 con 0x7f5cf40708a0 2024-04-05T22:30:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.552+0000 7f5cf0c00640 1 -- 172.21.15.73:0/480037433 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f5cec031030 con 0x7f5cf40708a0 2024-04-05T22:30:25.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.554+0000 7f5cf0c00640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 0x7f5cd406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.554+0000 7f5cf0c00640 1 -- 172.21.15.73:0/480037433 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cec0ab5a0 con 0x7f5cf40708a0 2024-04-05T22:30:25.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.554+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 0x7f5cd406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.554+0000 7f5cfe5b3740 1 Processor -- start 2024-04-05T22:30:25.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.555+0000 7f5cfe5b3740 1 -- start start 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.555+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 0x7f5cd406d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.555+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.555+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca16c6d0 0x563aca19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca0dc8c0 con 0x563aca1991f0 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca0da050 con 0x563aca1995d0 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aca0d5940 con 0x563aca16c6d0 2024-04-05T22:30:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42498/0 (socket says 172.21.15.73:42498) 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 -- 172.21.15.73:0/2728739927 learned_addr learned my addr 172.21.15.73:0/2728739927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 -- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca16c6d0 msgr2=0x563aca19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca16c6d0 0x563aca19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 -- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 msgr2=0x563aca19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 -- 172.21.15.73:0/2728739927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aca0a09e0 con 0x563aca1991f0 2024-04-05T22:30:25.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.556+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800e960 tx=0x7f5ce8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e970bcac2dd0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5ce5600640 1 -- 172.21.15.73:0/2728739927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800c050 con 0x563aca1991f0 2024-04-05T22:30:25.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5ce5600640 1 -- 172.21.15.73:0/2728739927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8018070 con 0x563aca1991f0 2024-04-05T22:30:25.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5ce5600640 1 -- 172.21.15.73:0/2728739927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8013b30 con 0x563aca1991f0 2024-04-05T22:30:25.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/2728739927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 msgr2=0x563aca16c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800e960 tx=0x7f5ce8011000 comp rx=0 tx=0).stop 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/2728739927 shutdown_connections 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca16c6d0 0x563aca19c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/2728739927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16c190 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.557+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/2728739927 >> 172.21.15.73:0/2728739927 conn(0x563aca16e400 msgr2=0x563aca1b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.558+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/2728739927 shutdown_connections 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.558+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/2728739927 wait complete. 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.558+0000 7f5cfe5b3740 1 -- start start 2024-04-05T22:30:25.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.558+0000 7f5cfe5b3740 1 Processor -- start 2024-04-05T22:30:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1991f0 0x563aca1af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca1ba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca077070 con 0x563aca1b8a40 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca076d10 con 0x563aca1995d0 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ac9e3cf20 con 0x563aca1991f0 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.559+0000 7f5cfe5b3740 10 client.0 _ll_register_callbacks cb 0x563aca0cefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42512/0 (socket says 172.21.15.73:42512) 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 -- 172.21.15.73:0/988852838 learned_addr learned my addr 172.21.15.73:0/988852838 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 -- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1991f0 msgr2=0x563aca1af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1991f0 0x563aca1af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 -- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 msgr2=0x563aca1ba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca1ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 -- 172.21.15.73:0/988852838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce800e610 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cf9800640 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5cec075eb0 tx=0x7f5cec079870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f050a32abad1b26c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cda000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0799b0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aca0ffae0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.560+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cec07d7e0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cec081da0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 10 client.9411 fetch_fsmap learned FSMap version 91 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563aca0d3fc0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563aca0ffae0 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5cec081910 con 0x563aca1b8a40 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 10 client.9411 fetch_fsmap finished waiting for FSMap version 91 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 10 client.9411 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 20 client.9411 populate_metadata read hostname 'smithi073' 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1ca5c0 0x563aca1ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 4 client.9411 mds_command: new command op to 9270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:30:25.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 --> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ac9dc93c0 con 0x563aca1ca5c0 2024-04-05T22:30:25.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.561+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1ca5c0 0x563aca1ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.562+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1ca5c0 0x563aca1ca9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06d390fa3c8d343 server_cookie=77d6c4d58acfe962 in_seq=0 out_seq=0 2024-04-05T22:30:25.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.562+0000 7f5cda000640 10 client.9411 ms_handle_connect on v2:172.21.15.73:6833/2701059182 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.562+0000 7f5cda000640 1 -- 172.21.15.73:0/988852838 <== mds.0 v2:172.21.15.73:6833/2701059182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ac9dc93c0 con 0x563aca1ca5c0 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.562+0000 7f5cda000640 10 client.9411 handle_command_reply: tid=0 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 client.9411 shutdown 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 msgr2=0x563aca1c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5cec075eb0 tx=0x7f5cec079870 comp rx=0 tx=0).stop 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 shutdown_connections 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1ca5c0 0x563aca1ca9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1b8a40 0x563aca1c6980 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1995d0 0x563aca1ba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/988852838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1991f0 0x563aca1af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cda000640 0 client.9411 ms_handle_reset on v2:172.21.15.73:6833/2701059182 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 >> 172.21.15.73:0/988852838 conn(0x563aca16c6d0 msgr2=0x563aca1700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 shutdown_connections 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/988852838 wait complete. 2024-04-05T22:30:25.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.563+0000 7f5cfe5b3740 20 client.9411 trim_cache size 0 max 16384 2024-04-05T22:30:25.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.579+0000 7f5cfe5b3740 1 Processor -- start 2024-04-05T22:30:25.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.579+0000 7f5cfe5b3740 1 -- start start 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca16b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca16b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca16bc30 con 0x563aca1991f0 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca1984b0 con 0x563aca1b8a40 2024-04-05T22:30:25.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aca0db990 con 0x563aca1995d0 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42524/0 (socket says 172.21.15.73:42524) 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.580+0000 7f5cf8e00640 1 -- 172.21.15.73:0/1961808298 learned_addr learned my addr 172.21.15.73:0/1961808298 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca16b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 -- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 msgr2=0x563aca16b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca16b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf9800640 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca16b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 -- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 msgr2=0x563aca16b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca16b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 -- 172.21.15.73:0/1961808298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aca108f80 con 0x563aca1991f0 2024-04-05T22:30:25.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.581+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009010 tx=0x7f5ce801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633530dc65626199 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5ce6000640 1 -- 172.21.15.73:0/1961808298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800c040 con 0x563aca1991f0 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5ce6000640 1 -- 172.21.15.73:0/1961808298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8030040 con 0x563aca1991f0 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5ce6000640 1 -- 172.21.15.73:0/1961808298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032510 con 0x563aca1991f0 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/1961808298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 msgr2=0x563aca219050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009010 tx=0x7f5ce801e000 comp rx=0 tx=0).stop 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/1961808298 shutdown_connections 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca16b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca16b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/1961808298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca219050 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/1961808298 >> 172.21.15.73:0/1961808298 conn(0x563aca16c6d0 msgr2=0x563aca1d8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/1961808298 shutdown_connections 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.582+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/1961808298 wait complete. 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.583+0000 7f5cfe5b3740 1 -- start start 2024-04-05T22:30:25.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.583+0000 7f5cfe5b3740 1 Processor -- start 2024-04-05T22:30:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca217800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca1b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca1b5270 con 0x563aca1991f0 2024-04-05T22:30:25.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aca1727d0 con 0x563aca1b8a40 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aca217e20 con 0x563aca1995d0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cfe5b3740 10 client.0 _ll_register_callbacks cb 0x7f5cec02cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42530/0 (socket says 172.21.15.73:42530) 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.584+0000 7f5cf8e00640 1 -- 172.21.15.73:0/3000657456 learned_addr learned my addr 172.21.15.73:0/3000657456 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca217800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 -- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 msgr2=0x563aca217800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf9800640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca1b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca217800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 -- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 msgr2=0x563aca1b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca1b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 -- 172.21.15.73:0/3000657456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563aca0a09e0 con 0x563aca1991f0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cf8e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009010 tx=0x7f5ce8003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b9991d3918155 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cda000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800c030 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ce8032820 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.585+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ce8018070 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ce80329e0 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 10 client.9417 fetch_fsmap learned FSMap version 91 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563aca076280 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8018070 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5ce8044030 con 0x563aca1991f0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 10 client.9417 fetch_fsmap finished waiting for FSMap version 91 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 10 client.9417 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 20 client.9417 populate_metadata read hostname 'smithi073' 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1c54d0 0x563aca1c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 4 client.9417 mds_command: new command op to 9270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 --> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ac9dda6c0 con 0x563aca1c54d0 2024-04-05T22:30:25.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.586+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1c54d0 0x563aca1c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cf3e00640 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1c54d0 0x563aca1c58b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e88d007f625992 server_cookie=72430598533e6508 in_seq=0 out_seq=0 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cda000640 10 client.9417 ms_handle_connect on v2:172.21.15.73:6833/2701059182 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cda000640 1 -- 172.21.15.73:0/3000657456 <== mds.0 v2:172.21.15.73:6833/2701059182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563ac9dda6c0 con 0x563aca1c54d0 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cda000640 10 client.9417 handle_command_reply: tid=0 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 client.9417 shutdown 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 msgr2=0x563aca16b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009010 tx=0x7f5ce8003760 comp rx=0 tx=0).stop 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 shutdown_connections 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:6833/2701059182,v1:172.21.15.73:6835/2701059182] conn(0x563aca1c54d0 0x563aca1c58b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aca1b8a40 0x563aca1b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563aca1995d0 0x563aca217800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cfe5b3740 1 --2- 172.21.15.73:0/3000657456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563aca1991f0 0x563aca16b6a0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.587+0000 7f5cda000640 0 client.9417 ms_handle_reset on v2:172.21.15.73:6833/2701059182 2024-04-05T22:30:25.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 >> 172.21.15.73:0/3000657456 conn(0x563aca16c6d0 msgr2=0x563aca1731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 shutdown_connections 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfe5b3740 1 -- 172.21.15.73:0/3000657456 wait complete. 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfe5b3740 20 client.9417 trim_cache size 0 max 16384 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 9270, 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:30:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 7.1005517510000002, 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 91, 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 148, 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 148, 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 12.033833818 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:30:25.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 msgr2=0x7f5cd406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 0x7f5cd406d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8009040 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 msgr2=0x7f5cf4070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000f80 tx=0x7f5cec001d80 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 shutdown_connections 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5cd406b230 0x7f5cd406d6f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cf40708a0 0x7f5cf4070d20 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c4900 0x7f5cf40c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 --2- 172.21.15.73:0/480037433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc005180 0x7f5cf4142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.588+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 >> 172.21.15.73:0/480037433 conn(0x7f5cf40ce8e0 msgr2=0x7f5cf40ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.589+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 shutdown_connections 2024-04-05T22:30:25.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.589+0000 7f5cfb800640 1 -- 172.21.15.73:0/480037433 wait complete. 2024-04-05T22:30:25.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:30:25.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.797+0000 7fcb97800640 1 Processor -- start 2024-04-05T22:30:25.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.798+0000 7fcb97800640 1 -- start start 2024-04-05T22:30:25.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb97800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb901587d0 0x7fcb900cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb97800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9015e850 0x7fcb9015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb97800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9006f5b0 con 0x7fcb900ccd20 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb97800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9015f240 con 0x7fcb901587d0 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb97800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90070410 con 0x7fcb9015e850 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb94e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb901587d0 0x7fcb900cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb94e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb901587d0 0x7fcb900cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42544/0 (socket says 172.21.15.73:42544) 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.799+0000 7fcb94e00640 1 -- 172.21.15.73:0/160265400 learned_addr learned my addr 172.21.15.73:0/160265400 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.800+0000 7fcb94e00640 1 -- 172.21.15.73:0/160265400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9015e850 msgr2=0x7fcb9015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.800+0000 7fcb94e00640 1 --2- 172.21.15.73:0/160265400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9015e850 0x7fcb9015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.800+0000 7fcb94e00640 1 -- 172.21.15.73:0/160265400 >> v1:172.21.15.157:6789/0 conn(0x7fcb900ccd20 legacy=0x7fcb9006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:25.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.800+0000 7fcb94e00640 1 -- 172.21.15.73:0/160265400 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90158bb0 con 0x7fcb901587d0 2024-04-05T22:30:25.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.801+0000 7fcb94e00640 1 --2- 172.21.15.73:0/160265400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb901587d0 0x7fcb900cc7e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009870 tx=0x7fcb7802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99e59940fa0628d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.801+0000 7fcb8fe00640 1 -- 172.21.15.73:0/160265400 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78031640 con 0x7fcb901587d0 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.801+0000 7fcb8fe00640 1 -- 172.21.15.73:0/160265400 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb780317a0 con 0x7fcb901587d0 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.801+0000 7fcb8fe00640 1 -- 172.21.15.73:0/160265400 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78031b30 con 0x7fcb901587d0 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 -- 172.21.15.73:0/160265400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb901587d0 msgr2=0x7fcb900cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 --2- 172.21.15.73:0/160265400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb901587d0 0x7fcb900cc7e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009870 tx=0x7fcb7802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 -- 172.21.15.73:0/160265400 shutdown_connections 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 --2- 172.21.15.73:0/160265400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9015e850 0x7fcb9015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 --2- 172.21.15.73:0/160265400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb901587d0 0x7fcb900cc7e0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 -- 172.21.15.73:0/160265400 >> 172.21.15.73:0/160265400 conn(0x7fcb900ce8e0 msgr2=0x7fcb900cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 -- 172.21.15.73:0/160265400 shutdown_connections 2024-04-05T22:30:25.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.802+0000 7fcb97800640 1 -- 172.21.15.73:0/160265400 wait complete. 2024-04-05T22:30:25.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.804+0000 7fcb97800640 1 Processor -- start 2024-04-05T22:30:25.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.804+0000 7fcb97800640 1 -- start start 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb901587d0 0x7fcb901efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb901467c0 0x7fcb9006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901f03c0 con 0x7fcb9015e850 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90143c00 con 0x7fcb901467c0 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb97800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901f4320 con 0x7fcb901587d0 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42554/0 (socket says 172.21.15.73:42554) 2024-04-05T22:30:25.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 -- 172.21.15.73:0/2552104981 learned_addr learned my addr 172.21.15.73:0/2552104981 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:25.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb94e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb901587d0 0x7fcb901efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 -- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb901587d0 msgr2=0x7fcb901efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb901587d0 0x7fcb901efe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 -- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb901467c0 msgr2=0x7fcb9006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb901467c0 0x7fcb9006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.805+0000 7fcb87e00640 1 -- 172.21.15.73:0/2552104981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780094d0 con 0x7fcb9015e850 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb87e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c430 tx=0x7fcb8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb80011790 con 0x7fcb9015e850 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb800118f0 con 0x7fcb9015e850 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb901f4600 con 0x7fcb9015e850 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb80011ce0 con 0x7fcb9015e850 2024-04-05T22:30:25.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.806+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb90144190 con 0x7fcb9015e850 2024-04-05T22:30:25.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.807+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb90066af0 con 0x7fcb9015e850 2024-04-05T22:30:25.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.808+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fcb8001c460 con 0x7fcb9015e850 2024-04-05T22:30:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.809+0000 7fcb8d600640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 0x7fcb6c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:25.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.809+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb80089e60 con 0x7fcb9015e850 2024-04-05T22:30:25.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.810+0000 7fcb94e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 0x7fcb6c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:25.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.811+0000 7fcb94e00640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 0x7fcb6c06d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb78036000 tx=0x7fcb7803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:25.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:25.814+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcb80054700 con 0x7fcb9015e850 2024-04-05T22:30:26.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.050+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb90068160 con 0x7fcb9015e850 2024-04-05T22:30:26.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.051+0000 7fcb8d600640 1 -- 172.21.15.73:0/2552104981 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb80057d20 con 0x7fcb9015e850 2024-04-05T22:30:26.115 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:30:26.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.053+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 msgr2=0x7fcb6c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:26.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.053+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 0x7fcb6c06d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb78036000 tx=0x7fcb7803c040 comp rx=0 tx=0).stop 2024-04-05T22:30:26.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.053+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 msgr2=0x7fcb901f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.053+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c430 tx=0x7fcb8000c8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 shutdown_connections 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb6c06b230 0x7fcb6c06d6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb901467c0 0x7fcb9006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9015e850 0x7fcb901f3a40 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 --2- 172.21.15.73:0/2552104981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb901587d0 0x7fcb901efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 >> 172.21.15.73:0/2552104981 conn(0x7fcb900ce8e0 msgr2=0x7fcb900ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 shutdown_connections 2024-04-05T22:30:26.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:26.054+0000 7fcb97800640 1 -- 172.21.15.73:0/2552104981 wait complete. 2024-04-05T22:30:26.117 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:30:26.117 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:30:26.117 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:30:26.117 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:30:26.118 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:30:26.118 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:30:26.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:30:26.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:30:26.118 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:30:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.118 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:26.174 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42037sec preferred_lft 42037sec 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:30:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:30:26.176 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:26.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.177 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:30:26.267 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:30:26.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.271 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:30:26.326 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:30:26.326 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:30:26.326 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:30:26.326 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:30:26.326 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:30:26.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:30:26.327 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:30:26.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.508 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:30:26.559 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:30:26.559 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:30:26.559 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:30:26.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:30:26.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:30:26.646 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:30:26.698 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:30:26.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:30:26.699 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:30:26.751 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:30:26.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:26.882 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:26.882 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:30:26.916 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:26.917 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:30:26.917 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:30:26.917 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:30:26.917 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:30:26.919 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:30:26.920 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:30:26.920 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:30:26.920 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:30:26.920 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:30:26.924 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:30:26.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:26.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:30:27.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:27.011 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:30:27.080 INFO:teuthology.orchestra.run.smithi073.stdout:client6487 2024-04-05T22:30:27.081 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:27.081 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:27.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:30:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.345+0000 7f9a0d600640 1 Processor -- start 2024-04-05T22:30:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.346+0000 7f9a0d600640 1 -- start start 2024-04-05T22:30:27.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.347+0000 7f9a0d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 0x7f9a080cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.347+0000 7f9a0d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0806f5b0 con 0x7f9a080ccd20 2024-04-05T22:30:27.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.347+0000 7f9a0d600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a08070410 con 0x7f9a0815e850 2024-04-05T22:30:27.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.347+0000 7f9a0d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08158bb0 con 0x7f9a081587d0 2024-04-05T22:30:27.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a06000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9a080ccd20 0x7f9a0806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59976/0 (socket says 172.21.15.73:59976) 2024-04-05T22:30:27.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a06000640 1 -- 172.21.15.73:0/3843185324 learned_addr learned my addr 172.21.15.73:0/3843185324 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392434288 0 0) 0x7f9a0806f5b0 con 0x7f9a080ccd20 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 0x7f9a080cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ec003520 con 0x7f9a080ccd20 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371745629 0 0) 0x7f99ec003520 con 0x7f9a080ccd20 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 msgr2=0x7f9a080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 0x7f9a080cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 >> v1:172.21.15.157:6789/0 conn(0x7f9a0815e850 legacy=0x7f9a080708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0806f800 con 0x7f9a080ccd20 2024-04-05T22:30:27.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f99f4001670 con 0x7f9a080ccd20 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.348+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 >> v1:172.21.15.73:6789/0 conn(0x7f9a080ccd20 legacy=0x7f9a0806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0039a0 0x7f99ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ec0047c0 0x7f99ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 0x7f99ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08158bb0 con 0x7f99ec0047c0 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec00bab0 con 0x7f99ec0039a0 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec00bc70 con 0x7f9a081587d0 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ec0047c0 0x7f99ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0039a0 0x7f99ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 -- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 msgr2=0x7f99ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a081587d0 0x7f99ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 -- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0039a0 msgr2=0x7f99ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0039a0 0x7f99ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06000640 1 -- 172.21.15.73:0/3843185324 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec003520 con 0x7f99ec0047c0 2024-04-05T22:30:27.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.349+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0039a0 0x7f99ec003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.350+0000 7f9a06000640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ec0047c0 0x7f99ec004c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c190 tx=0x7f99ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d6d192e95a99bf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4005ad0 con 0x7f99ec0047c0 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4005c30 con 0x7f99ec0047c0 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a05600640 1 -- 172.21.15.73:0/3843185324 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f402c8a0 con 0x7f99ec0047c0 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 -- 172.21.15.73:0/3843185324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ec0047c0 msgr2=0x7f99ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3843185324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ec0047c0 0x7f99ec004c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c190 tx=0x7f99ec0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a07400640 1 -- 172.21.15.73:0/3843185324 reap_dead start 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 -- 172.21.15.73:0/3843185324 shutdown_connections 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 -- 172.21.15.73:0/3843185324 >> 172.21.15.73:0/3843185324 conn(0x7f9a080ce8e0 msgr2=0x7f9a080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 -- 172.21.15.73:0/3843185324 shutdown_connections 2024-04-05T22:30:27.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.351+0000 7f9a0d600640 1 -- 172.21.15.73:0/3843185324 wait complete. 2024-04-05T22:30:27.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.353+0000 7f9a0d600640 1 Processor -- start 2024-04-05T22:30:27.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.353+0000 7f9a0d600640 1 -- start start 2024-04-05T22:30:27.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0047c0 0x7f9a080ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 0x7f9a081f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080aef30 con 0x7f9a080c4490 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080af140 con 0x7f99ec0047c0 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.354+0000 7f9a0d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080af420 con 0x7f9a08157880 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38064/0 (socket says 172.21.15.73:38064) 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 -- 172.21.15.73:0/3665767278 learned_addr learned my addr 172.21.15.73:0/3665767278 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0047c0 0x7f9a080ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a07400640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 0x7f9a081f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 -- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 msgr2=0x7f9a081f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 0x7f9a081f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 -- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0047c0 msgr2=0x7f9a080ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0047c0 0x7f9a080ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a06000640 1 -- 172.21.15.73:0/3665767278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080af700 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.355+0000 7f9a07400640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 0x7f9a081f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f9a06000640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f99f402c440 tx=0x7f9a08070380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4039070 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f402ccb0 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a080af9e0 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f403c040 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.356+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a080c4910 con 0x7f9a080c4490 2024-04-05T22:30:27.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.357+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a08066af0 con 0x7f9a080c4490 2024-04-05T22:30:27.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.358+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f99f4032070 con 0x7f9a080c4490 2024-04-05T22:30:27.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.359+0000 7f99fea00640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 0x7f99e806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:27.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.359+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99f40ab240 con 0x7f9a080c4490 2024-04-05T22:30:27.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.360+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 0x7f99e806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:27.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.361+0000 7f9a06a00640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 0x7f99e806d720 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99f8006fd0 tx=0x7f99f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:27.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.363+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f99f4075aa0 con 0x7f9a080c4490 2024-04-05T22:30:27.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.598+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a080af560 con 0x7f9a080c4490 2024-04-05T22:30:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.599+0000 7f99fea00640 1 -- 172.21.15.73:0/3665767278 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9a080af560 con 0x7f9a080c4490 2024-04-05T22:30:27.598 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:30:27.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 msgr2=0x7f99e806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 0x7f99e806d720 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99f8006fd0 tx=0x7f99f8008040 comp rx=0 tx=0).stop 2024-04-05T22:30:27.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 msgr2=0x7f9a08157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:27.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f99f402c440 tx=0x7f9a08070380 comp rx=0 tx=0).stop 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 shutdown_connections 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f99e806b260 0x7f99e806d720 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a08157880 0x7f9a081f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a080c4490 0x7f9a08157340 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 --2- 172.21.15.73:0/3665767278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99ec0047c0 0x7f9a080ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 >> 172.21.15.73:0/3665767278 conn(0x7f9a080ce8e0 msgr2=0x7f9a080ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 shutdown_connections 2024-04-05T22:30:27.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:27.602+0000 7f9a0d600640 1 -- 172.21.15.73:0/3665767278 wait complete. 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:30:27.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:30:27.620 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:30:27.620 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:30:27.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:27.620 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:30:27.678 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:30:27.678 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:30:27.678 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:30:27.678 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:30:27.679 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42059sec preferred_lft 42059sec 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:30:27.680 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:30:27.681 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:30:27.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:27.681 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:30:27.769 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:30:27.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:27.774 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:30:27.826 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:30:27.827 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:30:27.827 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:30:27.827 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:30:27.827 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:30:27.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:27.827 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:30:27.827 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:30:27.827 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:30:27.827 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:30:27.827 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:30:27.828 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:30:28.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:28.013 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:30:28.063 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:30:28.064 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:30:28.064 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:30:28.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:28.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:30:28.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:30:28.157 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:28.211 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:30:28.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:30:28.211 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:28.262 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:30:28.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:28.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:28.399 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:28.400 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:30:28.433 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:30:28.434 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:30:28.434 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:30:28.434 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:30:28.434 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:30:28.436 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:30:28.436 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:30:28.436 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:30:28.436 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:30:28.436 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:30:28.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:30:28.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:28.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:28.521 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:30:28.577 INFO:teuthology.orchestra.run.smithi157.stdout:client5945 2024-04-05T22:30:28.577 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:28.577 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:28.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:30:28.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.850+0000 7f25f0e00640 1 Processor -- start 2024-04-05T22:30:28.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.850+0000 7f25f0e00640 1 -- start start 2024-04-05T22:30:28.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25f0e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25ec0ac8d0 0x7f25ec0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:28.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25f0e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec1616a0 con 0x7f25ec06d060 2024-04-05T22:30:28.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25f0e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec0ad1f0 con 0x7f25ec06cc00 2024-04-05T22:30:28.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25f0e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec06c530 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25ebe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25ec06d060 0x7f25ec160f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59986/0 (socket says 172.21.15.73:59986) 2024-04-05T22:30:28.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25ebe00640 1 -- 172.21.15.73:0/2532166183 learned_addr learned my addr 172.21.15.73:0/2532166183 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:28.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.852+0000 7f25eb400640 1 --2- 172.21.15.73:0/2532166183 >> v2:172.21.15.157:3300/0 conn(0x7f25ec0ac8d0 0x7f25ec0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:28.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441347067 0 0) 0x7f25ec1616a0 con 0x7f25ec06d060 2024-04-05T22:30:28.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d0003520 con 0x7f25ec06d060 2024-04-05T22:30:28.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25eb400640 1 -- 172.21.15.73:0/2532166183 >> v1:172.21.15.157:6789/0 conn(0x7f25ec06cc00 legacy=0x7f25ec160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:28.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341897347 0 0) 0x7f25d0003520 con 0x7f25ec06d060 2024-04-05T22:30:28.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25eb400640 1 -- 172.21.15.73:0/2532166183 >> v1:172.21.15.73:6789/0 conn(0x7f25ec06d060 legacy=0x7f25ec160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:28.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.853+0000 7f25eb400640 1 -- 172.21.15.73:0/2532166183 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec15d420 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.855+0000 7f25eb400640 1 --2- 172.21.15.73:0/2532166183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec0accb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7ba64c8dd9f3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:28.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.855+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc02f050 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.855+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc031500 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.855+0000 7f25ea000640 1 -- 172.21.15.73:0/2532166183 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40019f0 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 -- 172.21.15.73:0/2532166183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 msgr2=0x7f25ec0accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:28.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2532166183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec0accb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc02b820 comp rx=0 tx=0).stop 2024-04-05T22:30:28.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 -- 172.21.15.73:0/2532166183 shutdown_connections 2024-04-05T22:30:28.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2532166183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec0accb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:28.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 -- 172.21.15.73:0/2532166183 >> 172.21.15.73:0/2532166183 conn(0x7f25ec0b0360 msgr2=0x7f25ec0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:28.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 -- 172.21.15.73:0/2532166183 shutdown_connections 2024-04-05T22:30:28.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.856+0000 7f25f0e00640 1 -- 172.21.15.73:0/2532166183 wait complete. 2024-04-05T22:30:28.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.858+0000 7f25f0e00640 1 Processor -- start 2024-04-05T22:30:28.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.858+0000 7f25f0e00640 1 -- start start 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25ec06cc00 0x7f25ec154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec154610 con 0x7f25ec06d060 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec155480 con 0x7f25ec0ac8d0 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25f0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec159840 con 0x7f25ec06cc00 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38072/0 (socket says 172.21.15.73:38072) 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.859+0000 7f25eaa00640 1 -- 172.21.15.73:0/2210295 learned_addr learned my addr 172.21.15.73:0/2210295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eb400640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25ec06cc00 0x7f25ec154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:28.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25ebe00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 -- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25ec06cc00 msgr2=0x7f25ec154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25ec06cc00 0x7f25ec154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 -- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 msgr2=0x7f25ec152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 -- 172.21.15.73:0/2210295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc009400 con 0x7f25ec06d060 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25eaa00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c400 tx=0x7f25d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80052d0 con 0x7f25ec06d060 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8005430 con 0x7f25ec06d060 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec159b20 con 0x7f25ec06d060 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.860+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8003830 con 0x7f25ec06d060 2024-04-05T22:30:28.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.861+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec15a0e0 con 0x7f25ec06d060 2024-04-05T22:30:28.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.862+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec04e160 con 0x7f25ec06d060 2024-04-05T22:30:28.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.863+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f25d8003990 con 0x7f25ec06d060 2024-04-05T22:30:28.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.864+0000 7f25e3400640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 0x7f25cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:28.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.864+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25d8089450 con 0x7f25ec06d060 2024-04-05T22:30:28.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.865+0000 7f25eb400640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 0x7f25cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.866+0000 7f25eb400640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 0x7f25cc06d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f25dc035000 tx=0x7f25dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:28.869+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f25d8053cf0 con 0x7f25ec06d060 2024-04-05T22:30:29.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.102+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f25ec0b1030 con 0x7f25ec06d060 2024-04-05T22:30:29.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.161+0000 7f25e3400640 1 -- 172.21.15.73:0/2210295 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 136+0+0 (secure 0 0 0) 0x7f25d8057310 con 0x7f25ec06d060 2024-04-05T22:30:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 msgr2=0x7f25cc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 0x7f25cc06d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f25dc035000 tx=0x7f25dc008030 comp rx=0 tx=0).stop 2024-04-05T22:30:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 msgr2=0x7f25ec152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c400 tx=0x7f25d800c8c0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 shutdown_connections 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f25cc06b230 0x7f25cc06d6f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ec0ac8d0 0x7f25ec152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ec06d060 0x7f25ec152130 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 --2- 172.21.15.73:0/2210295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25ec06cc00 0x7f25ec154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 >> 172.21.15.73:0/2210295 conn(0x7f25ec0b0360 msgr2=0x7f25ec06b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 shutdown_connections 2024-04-05T22:30:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.164+0000 7f25f0e00640 1 -- 172.21.15.73:0/2210295 wait complete. 2024-04-05T22:30:29.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:30:29.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.394+0000 7f813c600640 1 Processor -- start 2024-04-05T22:30:29.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.394+0000 7f813c600640 1 -- start start 2024-04-05T22:30:29.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 0x7f81340d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813c600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81340d0bd0 0x7f81340d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813c600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340ceee0 con 0x7f813415d350 2024-04-05T22:30:29.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813c600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81340ce5b0 con 0x7f81340d0bd0 2024-04-05T22:30:29.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81340ce890 con 0x7f813415cf70 2024-04-05T22:30:29.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f8139200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81340d0bd0 0x7f81340d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813a600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f813415d350 0x7f81340cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59998/0 (socket says 172.21.15.73:59998) 2024-04-05T22:30:29.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.395+0000 7f813a600640 1 -- 172.21.15.73:0/2463741635 learned_addr learned my addr 172.21.15.73:0/2463741635 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139c00640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 0x7f81340d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139200640 1 -- 172.21.15.73:0/2463741635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 msgr2=0x7f81340d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139200640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 0x7f81340d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139200640 1 -- 172.21.15.73:0/2463741635 >> v1:172.21.15.73:6789/0 conn(0x7f813415d350 legacy=0x7f81340cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139200640 1 -- 172.21.15.73:0/2463741635 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81340ceb70 con 0x7f81340d0bd0 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8133e00640 1 -- 172.21.15.73:0/2463741635 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934114555 0 0) 0x7f81340ceee0 con 0x7f813415d350 2024-04-05T22:30:29.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.396+0000 7f8139c00640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 0x7f81340d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:29.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.397+0000 7f8139200640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81340d1220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f81200097a0 tx=0x7f812002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799b8e4640c6f8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.397+0000 7f8133e00640 1 -- 172.21.15.73:0/2463741635 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120031780 con 0x7f81340d0bd0 2024-04-05T22:30:29.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.397+0000 7f8133e00640 1 -- 172.21.15.73:0/2463741635 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81200318e0 con 0x7f81340d0bd0 2024-04-05T22:30:29.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.397+0000 7f8133e00640 1 -- 172.21.15.73:0/2463741635 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120031c70 con 0x7f81340d0bd0 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 -- 172.21.15.73:0/2463741635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 msgr2=0x7f81340d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81340d1220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f81200097a0 tx=0x7f812002b820 comp rx=0 tx=0).stop 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 -- 172.21.15.73:0/2463741635 shutdown_connections 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81340d1220 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 --2- 172.21.15.73:0/2463741635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415cf70 0x7f81340d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 -- 172.21.15.73:0/2463741635 >> 172.21.15.73:0/2463741635 conn(0x7f81340d2df0 msgr2=0x7f81340d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 -- 172.21.15.73:0/2463741635 shutdown_connections 2024-04-05T22:30:29.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.398+0000 7f813c600640 1 -- 172.21.15.73:0/2463741635 wait complete. 2024-04-05T22:30:29.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.400+0000 7f813c600640 1 Processor -- start 2024-04-05T22:30:29.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.400+0000 7f813c600640 1 -- start start 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f813415cf70 0x7f8134157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415d350 0x7f81340b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134149630 con 0x7f81340d0bd0 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134157bc0 con 0x7f813415cf70 2024-04-05T22:30:29.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f813c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8134157ea0 con 0x7f813415d350 2024-04-05T22:30:29.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f8139c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f8139c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38092/0 (socket says 172.21.15.73:38092) 2024-04-05T22:30:29.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.401+0000 7f8139c00640 1 -- 172.21.15.73:0/1401613154 learned_addr learned my addr 172.21.15.73:0/1401613154 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139200640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f813415cf70 0x7f8134157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 -- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415d350 msgr2=0x7f81340b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415d350 0x7f81340b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 -- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f813415cf70 msgr2=0x7f8134157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f813415cf70 0x7f8134157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 -- 172.21.15.73:0/1401613154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120009400 con 0x7f81340d0bd0 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8139c00640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f812400c7f0 tx=0x7f812400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124007a30 con 0x7f81340d0bd0 2024-04-05T22:30:29.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124007b90 con 0x7f81340d0bd0 2024-04-05T22:30:29.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400ce40 con 0x7f81340d0bd0 2024-04-05T22:30:29.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8134158180 con 0x7f81340d0bd0 2024-04-05T22:30:29.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.402+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81341577b0 con 0x7f81340d0bd0 2024-04-05T22:30:29.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.403+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134158ab0 con 0x7f81340d0bd0 2024-04-05T22:30:29.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.405+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f81240113f0 con 0x7f81340d0bd0 2024-04-05T22:30:29.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.405+0000 7f8131600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 0x7f811406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.406+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f812408a100 con 0x7f81340d0bd0 2024-04-05T22:30:29.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.406+0000 7f8139200640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 0x7f811406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.407+0000 7f8139200640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 0x7f811406d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8120036000 tx=0x7f812003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.410+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f81240bd650 con 0x7f81340d0bd0 2024-04-05T22:30:29.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.627+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81341579a0 con 0x7f811406b230 2024-04-05T22:30:29.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.631+0000 7f8131600640 1 -- 172.21.15.73:0/1401613154 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8120031ad0 con 0x7f811406b230 2024-04-05T22:30:29.630 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:30:29.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:29.631 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:30:29.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:29.631 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 msgr2=0x7f811406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 0x7f811406d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8120036000 tx=0x7f812003d040 comp rx=0 tx=0).stop 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 msgr2=0x7f81341490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f812400c7f0 tx=0x7f812400cbf0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 shutdown_connections 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811406b230 0x7f811406d6f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f813415d350 0x7f81340b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f813415cf70 0x7f8134157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 --2- 172.21.15.73:0/1401613154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81340d0bd0 0x7f81341490f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 >> 172.21.15.73:0/1401613154 conn(0x7f81340d2df0 msgr2=0x7f813406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:29.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 shutdown_connections 2024-04-05T22:30:29.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.634+0000 7f813c600640 1 -- 172.21.15.73:0/1401613154 wait complete. 2024-04-05T22:30:29.652 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:30:29.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.857+0000 7fb37f600640 1 Processor -- start 2024-04-05T22:30:29.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.857+0000 7fb37f600640 1 -- start start 2024-04-05T22:30:29.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb37815aa70 0x7fb378070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3780714a0 0x7fb37815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37f600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb378071900 con 0x7fb37815dbc0 2024-04-05T22:30:29.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37815ae50 con 0x7fb3780714a0 2024-04-05T22:30:29.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37815e3b0 con 0x7fb37815aa70 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb377e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3780714a0 0x7fb37815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37d600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb37815dbc0 0x7fb3781609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60010/0 (socket says 172.21.15.73:60010) 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.859+0000 7fb37d600640 1 -- 172.21.15.73:0/860900269 learned_addr learned my addr 172.21.15.73:0/860900269 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb37cc00640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb37815aa70 0x7fb378070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb377e00640 1 -- 172.21.15.73:0/860900269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb37815aa70 msgr2=0x7fb378070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb377e00640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb37815aa70 0x7fb378070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb377e00640 1 -- 172.21.15.73:0/860900269 >> v1:172.21.15.73:6789/0 conn(0x7fb37815dbc0 legacy=0x7fb3781609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb377e00640 1 -- 172.21.15.73:0/860900269 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37806ca50 con 0x7fb3780714a0 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.860+0000 7fb377400640 1 -- 172.21.15.73:0/860900269 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987341679 0 0) 0x7fb378071900 con 0x7fb37815dbc0 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.861+0000 7fb377e00640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780714a0 0x7fb37815d680 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097a0 tx=0x7fb36402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f400145693df1d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.861+0000 7fb377400640 1 -- 172.21.15.73:0/860900269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36402d040 con 0x7fb3780714a0 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.861+0000 7fb377400640 1 -- 172.21.15.73:0/860900269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364031810 con 0x7fb3780714a0 2024-04-05T22:30:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.861+0000 7fb377400640 1 -- 172.21.15.73:0/860900269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364031ba0 con 0x7fb3780714a0 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 -- 172.21.15.73:0/860900269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780714a0 msgr2=0x7fb37815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780714a0 0x7fb37815d680 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097a0 tx=0x7fb36402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 -- 172.21.15.73:0/860900269 shutdown_connections 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780714a0 0x7fb37815d680 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 --2- 172.21.15.73:0/860900269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb37815aa70 0x7fb378070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 -- 172.21.15.73:0/860900269 >> 172.21.15.73:0/860900269 conn(0x7fb378069cb0 msgr2=0x7fb37806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 -- 172.21.15.73:0/860900269 shutdown_connections 2024-04-05T22:30:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.862+0000 7fb37f600640 1 -- 172.21.15.73:0/860900269 wait complete. 2024-04-05T22:30:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.864+0000 7fb37f600640 1 Processor -- start 2024-04-05T22:30:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.864+0000 7fb37f600640 1 -- start start 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3780714a0 0x7fb37806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37815dbc0 0x7fb3781607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3781f6ce0 con 0x7fb37815aa70 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378146c90 con 0x7fb37815dbc0 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.865+0000 7fb37f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3781f5d80 con 0x7fb3780714a0 2024-04-05T22:30:29.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38116/0 (socket says 172.21.15.73:38116) 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 -- 172.21.15.73:0/4159651932 learned_addr learned my addr 172.21.15.73:0/4159651932 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb37cc00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3780714a0 0x7fb37806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 -- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3780714a0 msgr2=0x7fb37806fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3780714a0 0x7fb37806fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 -- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37815dbc0 msgr2=0x7fb3781607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37815dbc0 0x7fb3781607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 -- 172.21.15.73:0/4159651932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009400 con 0x7fb37815aa70 2024-04-05T22:30:29.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.866+0000 7fb377e00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fb3640098c0 tx=0x7fb3640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.867+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36403ad60 con 0x7fb37815aa70 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.867+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364044400 con 0x7fb37815aa70 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.867+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3640447b0 con 0x7fb37815aa70 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.867+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3781f6060 con 0x7fb37815aa70 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.867+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3781f6620 con 0x7fb37815aa70 2024-04-05T22:30:29.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.868+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37804ea00 con 0x7fb37815aa70 2024-04-05T22:30:29.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.869+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7fb36404d440 con 0x7fb37815aa70 2024-04-05T22:30:29.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.870+0000 7fb374c00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 0x7fb35806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:29.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.870+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3640ac400 con 0x7fb37815aa70 2024-04-05T22:30:29.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.871+0000 7fb37cc00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 0x7fb35806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.872+0000 7fb37cc00640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 0x7fb35806d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb36c006fd0 tx=0x7fb36c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:29.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:29.875+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb364076b90 con 0x7fb37815aa70 2024-04-05T22:30:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.068+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb378068160 con 0x7fb37815aa70 2024-04-05T22:30:30.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.069+0000 7fb374c00640 1 -- 172.21.15.73:0/4159651932 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb36407a1b0 con 0x7fb37815aa70 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 msgr2=0x7fb35806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 0x7fb35806d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb36c006fd0 tx=0x7fb36c008040 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 msgr2=0x7fb3781f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fb3640098c0 tx=0x7fb3640046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 shutdown_connections 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb35806b230 0x7fb35806d6f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37815dbc0 0x7fb3781607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37815aa70 0x7fb3781f21b0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 --2- 172.21.15.73:0/4159651932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3780714a0 0x7fb37806fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 >> 172.21.15.73:0/4159651932 conn(0x7fb378069cb0 msgr2=0x7fb37815fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.071+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 shutdown_connections 2024-04-05T22:30:30.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.072+0000 7fb37f600640 1 -- 172.21.15.73:0/4159651932 wait complete. 2024-04-05T22:30:30.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:30.090 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol50260) 2024-04-05T22:30:30.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:30.189 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp68880/subvol57341) 2024-04-05T22:30:30.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol50260 2024-04-05T22:30:30.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.489+0000 7f7b18a00640 1 Processor -- start 2024-04-05T22:30:30.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.489+0000 7f7b18a00640 1 -- start start 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.491+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 0x7f7b140712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.491+0000 7f7b18a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b1406bc40 0x7f7b14164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.491+0000 7f7b18a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1415afc0 con 0x7f7b1406c9a0 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.491+0000 7f7b18a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1406f5b0 con 0x7f7b1406bc40 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.491+0000 7f7b18a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1406f890 con 0x7f7b1415abe0 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b1406bc40 0x7f7b14164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b1406bc40 0x7f7b14164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38118/0 (socket says 172.21.15.73:38118) 2024-04-05T22:30:30.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 -- 172.21.15.73:0/1712375140 learned_addr learned my addr 172.21.15.73:0/1712375140 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b11e00640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 0x7f7b140712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 -- 172.21.15.73:0/1712375140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 msgr2=0x7f7b140712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 0x7f7b140712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 -- 172.21.15.73:0/1712375140 >> v1:172.21.15.157:6789/0 conn(0x7f7b1406c9a0 legacy=0x7f7b1406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b12800640 1 -- 172.21.15.73:0/1712375140 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1406fb70 con 0x7f7b1406bc40 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.492+0000 7f7b11e00640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 0x7f7b140712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.493+0000 7f7b12800640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1406bc40 0x7f7b14164740 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009870 tx=0x7f7b0c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f28f05035aa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:30.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b10a00640 1 -- 172.21.15.73:0/1712375140 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031740 con 0x7f7b1406bc40 2024-04-05T22:30:30.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b10a00640 1 -- 172.21.15.73:0/1712375140 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c0318a0 con 0x7f7b1406bc40 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b10a00640 1 -- 172.21.15.73:0/1712375140 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031c30 con 0x7f7b1406bc40 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 -- 172.21.15.73:0/1712375140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1406bc40 msgr2=0x7f7b14164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1406bc40 0x7f7b14164740 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009870 tx=0x7f7b0c02b7e0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 -- 172.21.15.73:0/1712375140 shutdown_connections 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1406bc40 0x7f7b14164740 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 --2- 172.21.15.73:0/1712375140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1415abe0 0x7f7b140712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 -- 172.21.15.73:0/1712375140 >> 172.21.15.73:0/1712375140 conn(0x7f7b14069cb0 msgr2=0x7f7b1406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 -- 172.21.15.73:0/1712375140 shutdown_connections 2024-04-05T22:30:30.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.494+0000 7f7b18a00640 1 -- 172.21.15.73:0/1712375140 wait complete. 2024-04-05T22:30:30.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.496+0000 7f7b18a00640 1 Processor -- start 2024-04-05T22:30:30.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.496+0000 7f7b18a00640 1 -- start start 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 0x7f7b1414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1406c9a0 0x7f7b1414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1414c7a0 con 0x7f7b1415abe0 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14158660 con 0x7f7b1406bc40 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.497+0000 7f7b18a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14158940 con 0x7f7b1406c9a0 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38124/0 (socket says 172.21.15.73:38124) 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 -- 172.21.15.73:0/143466059 learned_addr learned my addr 172.21.15.73:0/143466059 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b11400640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1406c9a0 0x7f7b1414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b11e00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 0x7f7b1414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 -- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1406c9a0 msgr2=0x7f7b1414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1406c9a0 0x7f7b1414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 -- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 msgr2=0x7f7b1414a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 0x7f7b1414a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 -- 172.21.15.73:0/143466059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0094d0 con 0x7f7b1415abe0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b11e00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 0x7f7b1414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.498+0000 7f7b12800640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02b820 tx=0x7f7b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.499+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c033c70 con 0x7f7b1415abe0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.499+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c033df0 con 0x7f7b1415abe0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.499+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b14158c20 con 0x7f7b1415abe0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.499+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c03c640 con 0x7f7b1415abe0 2024-04-05T22:30:30.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.499+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1414cf40 con 0x7f7b1415abe0 2024-04-05T22:30:30.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.500+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b14158a80 con 0x7f7b1415abe0 2024-04-05T22:30:30.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.501+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7b0c03a040 con 0x7f7b1415abe0 2024-04-05T22:30:30.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.502+0000 7f7b0a000640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 0x7f7af006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:30.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.502+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b0c0abb70 con 0x7f7b1415abe0 2024-04-05T22:30:30.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.502+0000 7f7b11e00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 0x7f7af006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:30.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.503+0000 7f7b11e00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 0x7f7af006d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006fd0 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:30.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.507+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7b14158a80 con 0x7f7b1415abe0 2024-04-05T22:30:30.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.732+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol50260", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b14071820 con 0x7f7af006b230 2024-04-05T22:30:30.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:30.745+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:30.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:30.745+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:30.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:30.745+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:30.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:30.745+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:30.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:30.745+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:30.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.799+0000 7f7b0a000640 1 -- 172.21.15.73:0/143466059 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f7afc00c040 con 0x7f7af006b230 2024-04-05T22:30:30.798 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol50260 2024-04-05T22:30:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 msgr2=0x7f7af006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 0x7f7af006d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006fd0 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 msgr2=0x7f7b141561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02b820 tx=0x7f7b0c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 shutdown_connections 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7af006b230 0x7f7af006d6f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b1415abe0 0x7f7b141561b0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b1406c9a0 0x7f7b1414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 --2- 172.21.15.73:0/143466059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1406bc40 0x7f7b1414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 >> 172.21.15.73:0/143466059 conn(0x7f7b14069cb0 msgr2=0x7f7b141579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 shutdown_connections 2024-04-05T22:30:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:30.802+0000 7f7b18a00640 1 -- 172.21.15.73:0/143466059 wait complete. 2024-04-05T22:30:30.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57341 subvol_grp68880 2024-04-05T22:30:31.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.036+0000 7f7b7a000640 1 Processor -- start 2024-04-05T22:30:31.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.036+0000 7f7b7a000640 1 -- start start 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b7a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b740cf3e0 0x7f7b740b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b7a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b740cce90 0x7f7b740b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b7a000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7415b230 con 0x7f7b7415ae50 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b7a000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b740ce2f0 con 0x7f7b740cce90 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b7a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b740cd2f0 con 0x7f7b740cf3e0 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b73e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b740cce90 0x7f7b740b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b73e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b740cce90 0x7f7b740b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38130/0 (socket says 172.21.15.73:38130) 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.038+0000 7f7b73e00640 1 -- 172.21.15.73:0/22241945 learned_addr learned my addr 172.21.15.73:0/22241945 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.039+0000 7f7b6aa00640 1 --2- 172.21.15.73:0/22241945 >> v2:172.21.15.157:3300/0 conn(0x7f7b740cf3e0 0x7f7b740b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.039+0000 7f7b73e00640 1 -- 172.21.15.73:0/22241945 >> v2:172.21.15.157:3300/0 conn(0x7f7b740cf3e0 msgr2=0x7f7b740b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.039+0000 7f7b73e00640 1 --2- 172.21.15.73:0/22241945 >> v2:172.21.15.157:3300/0 conn(0x7f7b740cf3e0 0x7f7b740b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.039+0000 7f7b73e00640 1 -- 172.21.15.73:0/22241945 >> v1:172.21.15.157:6789/0 conn(0x7f7b7415ae50 legacy=0x7f7b740cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:31.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.039+0000 7f7b73e00640 1 -- 172.21.15.73:0/22241945 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7415f4a0 con 0x7f7b740cce90 2024-04-05T22:30:31.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.040+0000 7f7b73e00640 1 --2- 172.21.15.73:0/22241945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cce90 0x7f7b740b2e00 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7b740cf960 tx=0x7f7b6c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb26b74f690178a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.040+0000 7f7b72a00640 1 -- 172.21.15.73:0/22241945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c02c9f0 con 0x7f7b740cce90 2024-04-05T22:30:31.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.040+0000 7f7b72a00640 1 -- 172.21.15.73:0/22241945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c02cb50 con 0x7f7b740cce90 2024-04-05T22:30:31.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.040+0000 7f7b72a00640 1 -- 172.21.15.73:0/22241945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0343f0 con 0x7f7b740cce90 2024-04-05T22:30:31.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 -- 172.21.15.73:0/22241945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cce90 msgr2=0x7f7b740b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 --2- 172.21.15.73:0/22241945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cce90 0x7f7b740b2e00 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7b740cf960 tx=0x7f7b6c02be50 comp rx=0 tx=0).stop 2024-04-05T22:30:31.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 -- 172.21.15.73:0/22241945 shutdown_connections 2024-04-05T22:30:31.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 --2- 172.21.15.73:0/22241945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cce90 0x7f7b740b2e00 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 --2- 172.21.15.73:0/22241945 >> v2:172.21.15.157:3300/0 conn(0x7f7b740cf3e0 0x7f7b740b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 -- 172.21.15.73:0/22241945 >> 172.21.15.73:0/22241945 conn(0x7f7b7406a0d0 msgr2=0x7f7b7406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:31.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 -- 172.21.15.73:0/22241945 shutdown_connections 2024-04-05T22:30:31.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.041+0000 7f7b7a000640 1 -- 172.21.15.73:0/22241945 wait complete. 2024-04-05T22:30:31.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.043+0000 7f7b7a000640 1 Processor -- start 2024-04-05T22:30:31.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.043+0000 7f7b7a000640 1 -- start start 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b740cce90 0x7f7b74148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b7415ae50 0x7f7b740ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741e4c50 con 0x7f7b740cf3e0 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741e4f30 con 0x7f7b740cce90 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b7a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741e5210 con 0x7f7b7415ae50 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38132/0 (socket says 172.21.15.73:38132) 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.044+0000 7f7b6aa00640 1 -- 172.21.15.73:0/2566457760 learned_addr learned my addr 172.21.15.73:0/2566457760 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 -- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b7415ae50 msgr2=0x7f7b740ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b7415ae50 0x7f7b740ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 -- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b740cce90 msgr2=0x7f7b74148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b740cce90 0x7f7b74148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 -- 172.21.15.73:0/2566457760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009b30 con 0x7f7b740cf3e0 2024-04-05T22:30:31.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6aa00640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7f0 tx=0x7f7b5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c004220 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c004380 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007840 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b741e54f0 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.045+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b74148970 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.047+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b741e5a10 con 0x7f7b740cf3e0 2024-04-05T22:30:31.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.047+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 89706+0+0 (secure 0 0 0) 0x7f7b5c007bb0 con 0x7f7b740cf3e0 2024-04-05T22:30:31.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.048+0000 7f7b6be00640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 0x7f7b5006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.048+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b5c089f40 con 0x7f7b740cf3e0 2024-04-05T22:30:31.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.049+0000 7f7b73400640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 0x7f7b5006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.050+0000 7f7b73400640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 0x7f7b5006d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009650 tx=0x7f7b64006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.053+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7b741e5a10 con 0x7f7b740cf3e0 2024-04-05T22:30:31.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.191+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 24) v1 ==== 89757+0+0 (secure 0 0 0) 0x7f7b5c08f020 con 0x7f7b740cf3e0 2024-04-05T22:30:31.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.278+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57341", "group_name": "subvol_grp68880", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b740cb610 con 0x7f7b5006b230 2024-04-05T22:30:31.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.322+0000 7f7b6be00640 1 -- 172.21.15.73:0/2566457760 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7b64013430 con 0x7f7b5006b230 2024-04-05T22:30:31.322 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp68880/subvol57341 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 msgr2=0x7f7b5006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 0x7f7b5006d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009650 tx=0x7f7b64006ba0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 msgr2=0x7f7b741e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7f0 tx=0x7f7b5c00ccb0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 shutdown_connections 2024-04-05T22:30:31.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7b5006b230 0x7f7b5006d6f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b7415ae50 0x7f7b740ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b740cf3e0 0x7f7b741e8f80 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 --2- 172.21.15.73:0/2566457760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b740cce90 0x7f7b74148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 >> 172.21.15.73:0/2566457760 conn(0x7f7b7406a0d0 msgr2=0x7f7b7406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 shutdown_connections 2024-04-05T22:30:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.325+0000 7f7b7a000640 1 -- 172.21.15.73:0/2566457760 wait complete. 2024-04-05T22:30:31.343 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol50260 2024-04-05T22:30:31.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.541+0000 7fc7b9800640 1 Processor -- start 2024-04-05T22:30:31.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.542+0000 7fc7b9800640 1 -- start start 2024-04-05T22:30:31.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b9800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc7b406c9a0 0x7fc7b406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b9800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b406bea0 con 0x7fc7b415abe0 2024-04-05T22:30:31.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b9800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b415afc0 con 0x7fc7b406ba70 2024-04-05T22:30:31.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b9800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b406f5b0 con 0x7fc7b406c9a0 2024-04-05T22:30:31.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b2a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc7b415abe0 0x7fc7b40712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60012/0 (socket says 172.21.15.73:60012) 2024-04-05T22:30:31.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7b2a00640 1 -- 172.21.15.73:0/2489796010 learned_addr learned my addr 172.21.15.73:0/2489796010 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:31.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.543+0000 7fc7aa000640 1 --2- 172.21.15.73:0/2489796010 >> v2:172.21.15.73:3300/0 conn(0x7fc7b406c9a0 0x7fc7b406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.544+0000 7fc7b2000640 1 -- 172.21.15.73:0/2489796010 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674679931 0 0) 0x7fc7b406bea0 con 0x7fc7b415abe0 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.544+0000 7fc7b2000640 1 -- 172.21.15.73:0/2489796010 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc798003520 con 0x7fc7b415abe0 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.544+0000 7fc7aa000640 1 -- 172.21.15.73:0/2489796010 >> v1:172.21.15.157:6789/0 conn(0x7fc7b406ba70 legacy=0x7fc7b4164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.544+0000 7fc7aa000640 1 -- 172.21.15.73:0/2489796010 >> v1:172.21.15.73:6789/0 conn(0x7fc7b415abe0 legacy=0x7fc7b40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.544+0000 7fc7aa000640 1 -- 172.21.15.73:0/2489796010 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b406f890 con 0x7fc7b406c9a0 2024-04-05T22:30:31.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.545+0000 7fc7aa000640 1 --2- 172.21.15.73:0/2489796010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b406b530 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d521e6fff52894a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.545+0000 7fc7b2000640 1 -- 172.21.15.73:0/2489796010 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02f050 con 0x7fc7b406c9a0 2024-04-05T22:30:31.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.545+0000 7fc7b2000640 1 -- 172.21.15.73:0/2489796010 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c031700 con 0x7fc7b406c9a0 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.545+0000 7fc7b2000640 1 -- 172.21.15.73:0/2489796010 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c031a70 con 0x7fc7b406c9a0 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 -- 172.21.15.73:0/2489796010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 msgr2=0x7fc7b406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 --2- 172.21.15.73:0/2489796010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b406b530 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02b7a0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 -- 172.21.15.73:0/2489796010 shutdown_connections 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 --2- 172.21.15.73:0/2489796010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b406b530 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 -- 172.21.15.73:0/2489796010 >> 172.21.15.73:0/2489796010 conn(0x7fc7b4069cb0 msgr2=0x7fc7b406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 -- 172.21.15.73:0/2489796010 shutdown_connections 2024-04-05T22:30:31.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.546+0000 7fc7b9800640 1 -- 172.21.15.73:0/2489796010 wait complete. 2024-04-05T22:30:31.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.548+0000 7fc7b9800640 1 Processor -- start 2024-04-05T22:30:31.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.548+0000 7fc7b9800640 1 -- start start 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b406ba70 0x7fc7b4136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b415abe0 0x7fc7b4154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4136ea0 con 0x7fc7b406c9a0 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b414f1a0 con 0x7fc7b415abe0 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.549+0000 7fc7b9800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4157df0 con 0x7fc7b406ba70 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38156/0 (socket says 172.21.15.73:38156) 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 -- 172.21.15.73:0/52183065 learned_addr learned my addr 172.21.15.73:0/52183065 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:31.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 -- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b406ba70 msgr2=0x7fc7b4136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b406ba70 0x7fc7b4136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 -- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b415abe0 msgr2=0x7fc7b4154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b415abe0 0x7fc7b4154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 -- 172.21.15.73:0/52183065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009400 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7aa000640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02b7e0 tx=0x7fc79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.550+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0339b0 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.551+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c033b10 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.551+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c03e480 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.551+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b414f460 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.551+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b414fa20 con 0x7fc7b406c9a0 2024-04-05T22:30:31.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.552+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b414fc10 con 0x7fc7b406c9a0 2024-04-05T22:30:31.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.553+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 89757+0+0 (secure 0 0 0) 0x7fc79c03a040 con 0x7fc7b406c9a0 2024-04-05T22:30:31.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.553+0000 7fc7ab400640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 0x7fc79006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:31.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.553+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79c0abff0 con 0x7fc7b406c9a0 2024-04-05T22:30:31.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.554+0000 7fc7b2a00640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 0x7fc79006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:31.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.555+0000 7fc7b2a00640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 0x7fc79006d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40012e0 tx=0x7fc7a40015d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:31.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.558+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc79c076880 con 0x7fc7b406c9a0 2024-04-05T22:30:31.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.780+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol50260", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7b4070130 con 0x7fc79006b2c0 2024-04-05T22:30:31.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.791+0000 7fc7ab400640 1 -- 172.21.15.73:0/52183065 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fc7a40020e0 con 0x7fc79006b2c0 2024-04-05T22:30:31.791 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol50260 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 msgr2=0x7fc79006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 0x7fc79006d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40012e0 tx=0x7fc7a40015d0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 msgr2=0x7fc7b4133e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02b7e0 tx=0x7fc79c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 shutdown_connections 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc79006b2c0 0x7fc79006d780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b415abe0 0x7fc7b4154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b406c9a0 0x7fc7b4133e20 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 --2- 172.21.15.73:0/52183065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b406ba70 0x7fc7b4136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 >> 172.21.15.73:0/52183065 conn(0x7fc7b4069cb0 msgr2=0x7fc7b40ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 shutdown_connections 2024-04-05T22:30:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:31.793+0000 7fc7b9800640 1 -- 172.21.15.73:0/52183065 wait complete. 2024-04-05T22:30:31.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:31.813 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/fe085c81697e1eeb78f75cc8321c7b54.meta) 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol50260 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:30:33.571 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:33.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57341 subvol_grp68880 2024-04-05T22:30:33.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.801+0000 7f3c3ee00640 1 Processor -- start 2024-04-05T22:30:33.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.801+0000 7f3c3ee00640 1 -- start start 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.802+0000 7f3c3ee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c3806b230 0x7f3c38167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.802+0000 7f3c3ee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3815e3d0 con 0x7f3c3815d230 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.802+0000 7f3c3ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3815d7c0 con 0x7f3c3815ce50 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.802+0000 7f3c3ee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3815daa0 con 0x7f3c3806b230 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c37400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3c3815d230 0x7f3c380b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60028/0 (socket says 172.21.15.73:60028) 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c3ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c3806b230 0x7f3c38167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:33.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c37400640 1 -- 172.21.15.73:0/2658256560 learned_addr learned my addr 172.21.15.73:0/2658256560 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:33.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c3ce00640 1 -- 172.21.15.73:0/2658256560 >> v1:172.21.15.157:6789/0 conn(0x7f3c3815ce50 legacy=0x7f3c381614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:33.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c3ce00640 1 -- 172.21.15.73:0/2658256560 >> v1:172.21.15.73:6789/0 conn(0x7f3c3815d230 legacy=0x7f3c380b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:33.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.803+0000 7f3c3ce00640 1 -- 172.21.15.73:0/2658256560 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3815dd80 con 0x7f3c3806b230 2024-04-05T22:30:33.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.804+0000 7f3c3ce00640 1 --2- 172.21.15.73:0/2658256560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3806b230 0x7f3c38167e30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f3c300097a0 tx=0x7f3c3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11ffd9d321e80e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:33.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.805+0000 7f3c36a00640 1 -- 172.21.15.73:0/2658256560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30031830 con 0x7f3c3806b230 2024-04-05T22:30:33.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.805+0000 7f3c36a00640 1 -- 172.21.15.73:0/2658256560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c30031990 con 0x7f3c3806b230 2024-04-05T22:30:33.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.805+0000 7f3c36a00640 1 -- 172.21.15.73:0/2658256560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30031d20 con 0x7f3c3806b230 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.805+0000 7f3c3ee00640 1 -- 172.21.15.73:0/2658256560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3806b230 msgr2=0x7f3c38167e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.805+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/2658256560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3806b230 0x7f3c38167e30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f3c300097a0 tx=0x7f3c3002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.806+0000 7f3c3ee00640 1 -- 172.21.15.73:0/2658256560 shutdown_connections 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.806+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/2658256560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3806b230 0x7f3c38167e30 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.806+0000 7f3c3ee00640 1 -- 172.21.15.73:0/2658256560 >> 172.21.15.73:0/2658256560 conn(0x7f3c380b3ed0 msgr2=0x7f3c380b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.806+0000 7f3c3ee00640 1 -- 172.21.15.73:0/2658256560 shutdown_connections 2024-04-05T22:30:33.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.806+0000 7f3c3ee00640 1 -- 172.21.15.73:0/2658256560 wait complete. 2024-04-05T22:30:33.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.807+0000 7f3c3ee00640 1 Processor -- start 2024-04-05T22:30:33.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.808+0000 7f3c3ee00640 1 -- start start 2024-04-05T22:30:33.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.808+0000 7f3c3ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 0x7f3c38098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:33.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.808+0000 7f3c3ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3815ce50 0x7f3c381553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:33.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:33.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38155e20 con 0x7f3c3815d230 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381570b0 con 0x7f3c3815ce50 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381501b0 con 0x7f3c3806b230 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38172/0 (socket says 172.21.15.73:38172) 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 -- 172.21.15.73:0/395571071 learned_addr learned my addr 172.21.15.73:0/395571071 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c37400640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3815ce50 0x7f3c381553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c37e00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 0x7f3c38098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 -- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 msgr2=0x7f3c38098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 0x7f3c38098880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 -- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3815ce50 msgr2=0x7f3c381553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3815ce50 0x7f3c381553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c3ce00640 1 -- 172.21.15.73:0/395571071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c30009400 con 0x7f3c3815d230 2024-04-05T22:30:33.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.809+0000 7f3c37e00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 0x7f3c38098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c3ce00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3c3002bee0 tx=0x7f3c300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003b9f0 con 0x7f3c3815d230 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3003bb50 con 0x7f3c3815d230 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c38156010 con 0x7f3c3815d230 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c300433f0 con 0x7f3c3815d230 2024-04-05T22:30:33.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.810+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c381565d0 con 0x7f3c3815d230 2024-04-05T22:30:33.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.811+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c38068160 con 0x7f3c3815d230 2024-04-05T22:30:33.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.812+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 89757+0+0 (secure 0 0 0) 0x7f3c3004c3b0 con 0x7f3c3815d230 2024-04-05T22:30:33.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.813+0000 7f3c2fe00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:33.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.814+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c300abf70 con 0x7f3c3815d230 2024-04-05T22:30:33.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.814+0000 7f3c37e00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:33.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.815+0000 7f3c37e00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 0x7f3c1406d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c28003eb0 tx=0x7f3c28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:33.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:33.818+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3c30076890 con 0x7f3c3815d230 2024-04-05T22:30:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.046+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57341", "group_name": "subvol_grp68880", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c380b07f0 con 0x7f3c1406b2c0 2024-04-05T22:30:34.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.059+0000 7f3c2fe00640 1 -- 172.21.15.73:0/395571071 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3c28005030 con 0x7f3c1406b2c0 2024-04-05T22:30:34.060 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp68880/subvol57341 2024-04-05T22:30:34.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 msgr2=0x7f3c1406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 0x7f3c1406d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c28003eb0 tx=0x7f3c28009040 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 msgr2=0x7f3c381558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3c3002bee0 tx=0x7f3c300046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 shutdown_connections 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3815d230 0x7f3c381558e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3815ce50 0x7f3c381553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 --2- 172.21.15.73:0/395571071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c3806b230 0x7f3c38098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 >> 172.21.15.73:0/395571071 conn(0x7f3c380b3ed0 msgr2=0x7f3c380b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 shutdown_connections 2024-04-05T22:30:34.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.062+0000 7f3c3ee00640 1 -- 172.21.15.73:0/395571071 wait complete. 2024-04-05T22:30:34.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:34.082 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/424910c0e40a799caa28fef56ab78e69.meta) 2024-04-05T22:30:34.181 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:30:34.182 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:30:34.182 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:30:34.182 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/subvol_grp68880/subvol57341 2024-04-05T22:30:34.182 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:30:34.182 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:34.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol50260 2024-04-05T22:30:34.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.400+0000 7f8c91800640 1 Processor -- start 2024-04-05T22:30:34.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.400+0000 7f8c91800640 1 -- start start 2024-04-05T22:30:34.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c91800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8c8c15abe0 0x7f8c8c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c91800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c8c06bc40 0x7f8c8c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c91800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c15afc0 con 0x7f8c8c06c9a0 2024-04-05T22:30:34.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c91800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c06f5b0 con 0x7f8c8c15abe0 2024-04-05T22:30:34.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c91800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c06f890 con 0x7f8c8c06bc40 2024-04-05T22:30:34.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c83e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8c8c06c9a0 0x7f8c8c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60040/0 (socket says 172.21.15.73:60040) 2024-04-05T22:30:34.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c83e00640 1 -- 172.21.15.73:0/1839551972 learned_addr learned my addr 172.21.15.73:0/1839551972 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:34.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.402+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1839551972 >> v2:172.21.15.73:3300/0 conn(0x7f8c8c15abe0 0x7f8c8c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8b400640 1 --2- 172.21.15.73:0/1839551972 >> v2:172.21.15.157:3300/0 conn(0x7f8c8c06bc40 0x7f8c8c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8a000640 1 -- 172.21.15.73:0/1839551972 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449119864 0 0) 0x7f8c8c15afc0 con 0x7f8c8c06c9a0 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8a000640 1 -- 172.21.15.73:0/1839551972 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c70003520 con 0x7f8c8c06c9a0 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1839551972 >> v2:172.21.15.157:3300/0 conn(0x7f8c8c06bc40 msgr2=0x7f8c8c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1839551972 >> v2:172.21.15.157:3300/0 conn(0x7f8c8c06bc40 0x7f8c8c164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1839551972 >> v1:172.21.15.73:6789/0 conn(0x7f8c8c06c9a0 legacy=0x7f8c8c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.403+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1839551972 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c06fb70 con 0x7f8c8c15abe0 2024-04-05T22:30:34.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.405+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1839551972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0712e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c06c900 tx=0x7f8c7c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626cd3d5a3dd296b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.405+0000 7f8c8a000640 1 -- 172.21.15.73:0/1839551972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031710 con 0x7f8c8c15abe0 2024-04-05T22:30:34.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.405+0000 7f8c8a000640 1 -- 172.21.15.73:0/1839551972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c02f050 con 0x7f8c8c15abe0 2024-04-05T22:30:34.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.405+0000 7f8c8a000640 1 -- 172.21.15.73:0/1839551972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031a90 con 0x7f8c8c15abe0 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 -- 172.21.15.73:0/1839551972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c15abe0 msgr2=0x7f8c8c0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 --2- 172.21.15.73:0/1839551972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0712e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c06c900 tx=0x7f8c7c02b7b0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 -- 172.21.15.73:0/1839551972 shutdown_connections 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 --2- 172.21.15.73:0/1839551972 >> v2:172.21.15.157:3300/0 conn(0x7f8c8c06bc40 0x7f8c8c164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 --2- 172.21.15.73:0/1839551972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0712e0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 -- 172.21.15.73:0/1839551972 >> 172.21.15.73:0/1839551972 conn(0x7f8c8c069cb0 msgr2=0x7f8c8c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:34.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 -- 172.21.15.73:0/1839551972 shutdown_connections 2024-04-05T22:30:34.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.406+0000 7f8c91800640 1 -- 172.21.15.73:0/1839551972 wait complete. 2024-04-05T22:30:34.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.408+0000 7f8c91800640 1 Processor -- start 2024-04-05T22:30:34.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.408+0000 7f8c91800640 1 -- start start 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 0x7f8c8c15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1f1890 con 0x7f8c8c06bc40 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1ef020 con 0x7f8c8c15abe0 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c91800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1f1aa0 con 0x7f8c8c06c9a0 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38198/0 (socket says 172.21.15.73:38198) 2024-04-05T22:30:34.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.409+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1973739837 learned_addr learned my addr 172.21.15.73:0/1973739837 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c83e00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 0x7f8c8c15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 msgr2=0x7f8c8c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8b400640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 0x7f8c8c15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8c15abe0 msgr2=0x7f8c8c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 -- 172.21.15.73:0/1973739837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c009400 con 0x7f8c8c06bc40 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c83e00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 0x7f8c8c15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:34.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c8aa00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c000c00 tx=0x7f8c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c03c8d0 con 0x7f8c8c06bc40 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c03ca30 con 0x7f8c8c06bc40 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c03cde0 con 0x7f8c8c06bc40 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.410+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c1f1d80 con 0x7f8c8c06bc40 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.411+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c1f2580 con 0x7f8c8c06bc40 2024-04-05T22:30:34.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.412+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c1f22a0 con 0x7f8c8c06bc40 2024-04-05T22:30:34.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.412+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 89757+0+0 (secure 0 0 0) 0x7f8c7c03a040 con 0x7f8c8c06bc40 2024-04-05T22:30:34.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.413+0000 7f8c82a00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 0x7f8c6806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.413+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c7c0ab420 con 0x7f8c8c06bc40 2024-04-05T22:30:34.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.414+0000 7f8c83e00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 0x7f8c6806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.415+0000 7f8c83e00640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 0x7f8c6806d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c14e620 tx=0x7f8c74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.418+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8c8c1f22a0 con 0x7f8c8c06bc40 2024-04-05T22:30:34.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.646+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol50260", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c8c1f27d0 con 0x7f8c6806b2c0 2024-04-05T22:30:34.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.684+0000 7f8c82a00640 1 -- 172.21.15.73:0/1973739837 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8c74001d20 con 0x7f8c6806b2c0 2024-04-05T22:30:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 msgr2=0x7f8c6806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 0x7f8c6806d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c14e620 tx=0x7f8c74009040 comp rx=0 tx=0).stop 2024-04-05T22:30:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 msgr2=0x7f8c8c15d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c000c00 tx=0x7f8c7c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 shutdown_connections 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8c6806b2c0 0x7f8c6806d780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8c15abe0 0x7f8c8c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c8c06c9a0 0x7f8c8c15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 --2- 172.21.15.73:0/1973739837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c8c06bc40 0x7f8c8c15d1a0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 >> 172.21.15.73:0/1973739837 conn(0x7f8c8c069cb0 msgr2=0x7f8c8c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 shutdown_connections 2024-04-05T22:30:34.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.687+0000 7f8c91800640 1 -- 172.21.15.73:0/1973739837 wait complete. 2024-04-05T22:30:34.691 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.692+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.691 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.692+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.691 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.692+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.691 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.692+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.691 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.692+0000 7f1d89000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol57341 subvol_grp68880 2024-04-05T22:30:34.719 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.720+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.720 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.720+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.720 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.720+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.720 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.720+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.720 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:30:34.720+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:30:34.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.910+0000 7fd953c00640 1 Processor -- start 2024-04-05T22:30:34.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.910+0000 7fd953c00640 1 -- start start 2024-04-05T22:30:34.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.912+0000 7fd953c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 0x7fd94c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.912+0000 7fd953c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd94c0cce90 0x7fd94c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.912+0000 7fd953c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94c15b230 con 0x7fd94c0cecd0 2024-04-05T22:30:34.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.912+0000 7fd953c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94c0ce2f0 con 0x7fd94c0cce90 2024-04-05T22:30:34.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.912+0000 7fd953c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c0cd2f0 con 0x7fd94c15ae50 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd951c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd94c0cecd0 0x7fd94c0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60050/0 (socket says 172.21.15.73:60050) 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd943e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd94c0cce90 0x7fd94c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd951c00640 1 -- 172.21.15.73:0/3047350842 learned_addr learned my addr 172.21.15.73:0/3047350842 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd951200640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 0x7fd94c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537180726 0 0) 0x7fd94c15b230 con 0x7fd94c0cecd0 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd930003520 con 0x7fd94c0cecd0 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd943e00640 1 -- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 msgr2=0x7fd94c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:34.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd943e00640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 0x7fd94c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd943e00640 1 -- 172.21.15.73:0/3047350842 >> v1:172.21.15.73:6789/0 conn(0x7fd94c0cecd0 legacy=0x7fd94c0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400521612 0 0) 0x7fd930003520 con 0x7fd94c0cecd0 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.913+0000 7fd943e00640 1 -- 172.21.15.73:0/3047350842 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c15f4a0 con 0x7fd94c0cce90 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.914+0000 7fd951200640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 0x7fd94c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.915+0000 7fd943e00640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0cce90 0x7fd94c0b2e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0ce430 tx=0x7fd93402b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20edc9c9333a52a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.915+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9440019f0 con 0x7fd94c0cce90 2024-04-05T22:30:34.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93402e050 con 0x7fd94c0cce90 2024-04-05T22:30:34.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd94be00640 1 -- 172.21.15.73:0/3047350842 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934030710 con 0x7fd94c0cce90 2024-04-05T22:30:34.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 -- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0cce90 msgr2=0x7fd94c0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0cce90 0x7fd94c0b2e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0ce430 tx=0x7fd93402b6f0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 -- 172.21.15.73:0/3047350842 shutdown_connections 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0cce90 0x7fd94c0b2e00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 --2- 172.21.15.73:0/3047350842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15ae50 0x7fd94c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 -- 172.21.15.73:0/3047350842 >> 172.21.15.73:0/3047350842 conn(0x7fd94c06a0d0 msgr2=0x7fd94c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 -- 172.21.15.73:0/3047350842 shutdown_connections 2024-04-05T22:30:34.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.916+0000 7fd953c00640 1 -- 172.21.15.73:0/3047350842 wait complete. 2024-04-05T22:30:34.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.918+0000 7fd953c00640 1 Processor -- start 2024-04-05T22:30:34.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.918+0000 7fd953c00640 1 -- start start 2024-04-05T22:30:34.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.919+0000 7fd953c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0cce90 0x7fd94c1f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.919+0000 7fd953c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd953c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c15ae50 0x7fd94c1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd953c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c14c7e0 con 0x7fd94c0cecd0 2024-04-05T22:30:34.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd953c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c14cac0 con 0x7fd94c15ae50 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd953c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c14cda0 con 0x7fd94c0cce90 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38204/0 (socket says 172.21.15.73:38204) 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 -- 172.21.15.73:0/1023567935 learned_addr learned my addr 172.21.15.73:0/1023567935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd951200640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0cce90 0x7fd94c1f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 -- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0cce90 msgr2=0x7fd94c1f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0cce90 0x7fd94c1f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 -- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c15ae50 msgr2=0x7fd94c1f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c15ae50 0x7fd94c1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.920+0000 7fd943e00640 1 -- 172.21.15.73:0/1023567935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009400 con 0x7fd94c0cecd0 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd943e00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd93402b730 tx=0x7fd9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9340044c0 con 0x7fd94c0cecd0 2024-04-05T22:30:34.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934003e20 con 0x7fd94c0cecd0 2024-04-05T22:30:34.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934030750 con 0x7fd94c0cecd0 2024-04-05T22:30:34.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd934009400 con 0x7fd94c0cecd0 2024-04-05T22:30:34.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.921+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c14a280 con 0x7fd94c0cecd0 2024-04-05T22:30:34.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.922+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c14a410 con 0x7fd94c0cecd0 2024-04-05T22:30:34.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.923+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 89757+0+0 (secure 0 0 0) 0x7fd934039070 con 0x7fd94c0cecd0 2024-04-05T22:30:34.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.924+0000 7fd949600640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 0x7fd92806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:34.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.924+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9340ab5a0 con 0x7fd94c0cecd0 2024-04-05T22:30:34.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.925+0000 7fd951200640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 0x7fd92806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:34.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.926+0000 7fd951200640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 0x7fd92806d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009b80 tx=0x7fd93c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:34.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:34.929+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd94c14a410 con 0x7fd94c0cecd0 2024-04-05T22:30:35.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.151+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol57341", "group_name": "subvol_grp68880", "target": ["mon-mgr", ""]}) v1 -- 0x7fd94c14cc30 con 0x7fd92806b2c0 2024-04-05T22:30:35.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.188+0000 7fd949600640 1 -- 172.21.15.73:0/1023567935 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd94c14cc30 con 0x7fd92806b2c0 2024-04-05T22:30:35.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 msgr2=0x7fd92806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 0x7fd92806d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009b80 tx=0x7fd93c008040 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 msgr2=0x7fd94c149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd93402b730 tx=0x7fd9340046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 shutdown_connections 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92806b2c0 0x7fd92806d780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c15ae50 0x7fd94c1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd94c0cecd0 0x7fd94c149d40 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 --2- 172.21.15.73:0/1023567935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0cce90 0x7fd94c1f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 >> 172.21.15.73:0/1023567935 conn(0x7fd94c06a0d0 msgr2=0x7fd94c1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 shutdown_connections 2024-04-05T22:30:35.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.191+0000 7fd953c00640 1 -- 172.21.15.73:0/1023567935 wait complete. 2024-04-05T22:30:35.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:35.215 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:30:35.269 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:30:35.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp68880 2024-04-05T22:30:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.476+0000 7faf7e200640 1 Processor -- start 2024-04-05T22:30:35.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.476+0000 7faf7e200640 1 -- start start 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.477+0000 7faf7e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf7806bc40 0x7faf78165750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.477+0000 7faf7e200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7815afc0 con 0x7faf7806c9a0 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.477+0000 7faf7e200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7806f5b0 con 0x7faf7815abe0 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.477+0000 7faf7e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7806f890 con 0x7faf7806bc40 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf6ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faf7806c9a0 0x7faf7806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60064/0 (socket says 172.21.15.73:60064) 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf6ea00640 1 -- 172.21.15.73:0/4082922203 learned_addr learned my addr 172.21.15.73:0/4082922203 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:35.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf77e00640 1 --2- 172.21.15.73:0/4082922203 >> v2:172.21.15.73:3300/0 conn(0x7faf7806bc40 0x7faf78165750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:35.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf77e00640 1 -- 172.21.15.73:0/4082922203 >> v1:172.21.15.157:6789/0 conn(0x7faf7815abe0 legacy=0x7faf780712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:35.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf77e00640 1 -- 172.21.15.73:0/4082922203 >> v1:172.21.15.73:6789/0 conn(0x7faf7806c9a0 legacy=0x7faf7806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.478+0000 7faf77e00640 1 -- 172.21.15.73:0/4082922203 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7806fb70 con 0x7faf7806bc40 2024-04-05T22:30:35.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.479+0000 7faf77e00640 1 --2- 172.21.15.73:0/4082922203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78165750 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7faf70009d90 tx=0x7faf7002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243fbf3799475506 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf76a00640 1 -- 172.21.15.73:0/4082922203 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7002cb40 con 0x7faf7806bc40 2024-04-05T22:30:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf76a00640 1 -- 172.21.15.73:0/4082922203 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7002cca0 con 0x7faf7806bc40 2024-04-05T22:30:35.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf76a00640 1 -- 172.21.15.73:0/4082922203 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70034660 con 0x7faf7806bc40 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf7e200640 1 -- 172.21.15.73:0/4082922203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 msgr2=0x7faf78165750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf7e200640 1 --2- 172.21.15.73:0/4082922203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78165750 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7faf70009d90 tx=0x7faf7002bd90 comp rx=0 tx=0).stop 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf7e200640 1 -- 172.21.15.73:0/4082922203 shutdown_connections 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.480+0000 7faf7e200640 1 --2- 172.21.15.73:0/4082922203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78165750 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.481+0000 7faf7e200640 1 -- 172.21.15.73:0/4082922203 >> 172.21.15.73:0/4082922203 conn(0x7faf78069cb0 msgr2=0x7faf7806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.481+0000 7faf7e200640 1 -- 172.21.15.73:0/4082922203 shutdown_connections 2024-04-05T22:30:35.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.481+0000 7faf7e200640 1 -- 172.21.15.73:0/4082922203 wait complete. 2024-04-05T22:30:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.482+0000 7faf7e200640 1 Processor -- start 2024-04-05T22:30:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.483+0000 7faf7e200640 1 -- start start 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.483+0000 7faf7e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.483+0000 7faf7e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7806c9a0 0x7faf7815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.483+0000 7faf7e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7815abe0 0x7faf781504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf7e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7815b990 con 0x7faf7806bc40 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf7e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78150a10 con 0x7faf7806c9a0 2024-04-05T22:30:35.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf7e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf78156090 con 0x7faf7815abe0 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38230/0 (socket says 172.21.15.73:38230) 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 -- 172.21.15.73:0/1437975271 learned_addr learned my addr 172.21.15.73:0/1437975271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 -- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7815abe0 msgr2=0x7faf781504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7815abe0 0x7faf781504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 -- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7806c9a0 msgr2=0x7faf7815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7806c9a0 0x7faf7815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.484+0000 7faf77400640 1 -- 172.21.15.73:0/1437975271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf70009a70 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf77400640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7faf6800b420 tx=0x7faf6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68011830 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf68011990 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7814e8d0 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68011d40 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.485+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7814e280 con 0x7faf7806bc40 2024-04-05T22:30:35.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.486+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf78068160 con 0x7faf7806bc40 2024-04-05T22:30:35.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.487+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7faf6801b460 con 0x7faf7806bc40 2024-04-05T22:30:35.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.488+0000 7faf6fe00640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 0x7faf5406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:35.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.488+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf68087540 con 0x7faf7806bc40 2024-04-05T22:30:35.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.488+0000 7faf6ea00640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 0x7faf5406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:35.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.490+0000 7faf6ea00640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 0x7faf5406d8d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faf780b3bb0 tx=0x7faf60007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.492+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7faf6808e070 con 0x7faf7806bc40 2024-04-05T22:30:35.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.709+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp68880", "target": ["mon-mgr", ""]}) v1 -- 0x7faf780cdcc0 con 0x7faf5406b410 2024-04-05T22:30:35.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.720+0000 7faf6fe00640 1 -- 172.21.15.73:0/1437975271 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faf600012b0 con 0x7faf5406b410 2024-04-05T22:30:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 msgr2=0x7faf5406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 0x7faf5406d8d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faf780b3bb0 tx=0x7faf60007a40 comp rx=0 tx=0).stop 2024-04-05T22:30:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 msgr2=0x7faf78147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7faf6800b420 tx=0x7faf6800b8e0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 shutdown_connections 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7faf5406b410 0x7faf5406d8d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7815abe0 0x7faf781504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7806c9a0 0x7faf7815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 --2- 172.21.15.73:0/1437975271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf7806bc40 0x7faf78147050 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 >> 172.21.15.73:0/1437975271 conn(0x7faf78069cb0 msgr2=0x7faf780af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 shutdown_connections 2024-04-05T22:30:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:35.723+0000 7faf7e200640 1 -- 172.21.15.73:0/1437975271 wait complete. 2024-04-05T22:30:35.744 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:30:35.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:35.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:35.971 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:30:35.972 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:35.972 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:36.029 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:36.029 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:36.081 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:36.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.294+0000 7f8136000640 1 Processor -- start 2024-04-05T22:30:36.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.294+0000 7f8136000640 1 -- start start 2024-04-05T22:30:36.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.295+0000 7f8136000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130158860 0x7f81300b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.295+0000 7f8136000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81300b1f00 0x7f813015e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.295+0000 7f8136000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813006b500 con 0x7f813015eb60 2024-04-05T22:30:36.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.295+0000 7f8136000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813015eff0 con 0x7f81300b1f00 2024-04-05T22:30:36.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.295+0000 7f8136000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81300ccaf0 con 0x7f8130158860 2024-04-05T22:30:36.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f8135600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f813015eb60 0x7f813015afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59516/0 (socket says 172.21.15.73:59516) 2024-04-05T22:30:36.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f8135600640 1 -- 172.21.15.73:0/1877334214 learned_addr learned my addr 172.21.15.73:0/1877334214 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812fe00640 1 --2- 172.21.15.73:0/1877334214 >> v2:172.21.15.73:3300/0 conn(0x7f81300b1f00 0x7f813015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f8134c00640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130158860 0x7f81300b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812fe00640 1 -- 172.21.15.73:0/1877334214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130158860 msgr2=0x7f81300b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812fe00640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130158860 0x7f81300b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812fe00640 1 -- 172.21.15.73:0/1877334214 >> v1:172.21.15.73:6789/0 conn(0x7f813015eb60 legacy=0x7f813015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812fe00640 1 -- 172.21.15.73:0/1877334214 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300b23a0 con 0x7f81300b1f00 2024-04-05T22:30:36.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.296+0000 7f812f400640 1 -- 172.21.15.73:0/1877334214 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719301698 0 0) 0x7f813006b500 con 0x7f813015eb60 2024-04-05T22:30:36.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f812fe00640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81300b1f00 0x7f813015e620 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097a0 tx=0x7f811c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b59cba4bd6677 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f812f400640 1 -- 172.21.15.73:0/1877334214 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c02e070 con 0x7f81300b1f00 2024-04-05T22:30:36.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f812f400640 1 -- 172.21.15.73:0/1877334214 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c0315a0 con 0x7f81300b1f00 2024-04-05T22:30:36.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f812f400640 1 -- 172.21.15.73:0/1877334214 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c031930 con 0x7f81300b1f00 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 -- 172.21.15.73:0/1877334214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81300b1f00 msgr2=0x7f813015e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81300b1f00 0x7f813015e620 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097a0 tx=0x7f811c02b840 comp rx=0 tx=0).stop 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 -- 172.21.15.73:0/1877334214 shutdown_connections 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81300b1f00 0x7f813015e620 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 --2- 172.21.15.73:0/1877334214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130158860 0x7f81300b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.298+0000 7f8136000640 1 -- 172.21.15.73:0/1877334214 >> 172.21.15.73:0/1877334214 conn(0x7f81300b06c0 msgr2=0x7f81300b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.299+0000 7f8136000640 1 -- 172.21.15.73:0/1877334214 shutdown_connections 2024-04-05T22:30:36.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.299+0000 7f8136000640 1 -- 172.21.15.73:0/1877334214 wait complete. 2024-04-05T22:30:36.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.300+0000 7f8136000640 1 Processor -- start 2024-04-05T22:30:36.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.300+0000 7f8136000640 1 -- start start 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81300b1f00 0x7f813013fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130151f80 0x7f81300c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130140830 con 0x7f813015eb60 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130152490 con 0x7f81300b1f00 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.301+0000 7f8136000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8130152770 con 0x7f8130151f80 2024-04-05T22:30:36.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39156/0 (socket says 172.21.15.73:39156) 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 -- 172.21.15.73:0/174324641 learned_addr learned my addr 172.21.15.73:0/174324641 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 -- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130151f80 msgr2=0x7f81300c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130151f80 0x7f81300c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 -- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81300b1f00 msgr2=0x7f813013fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81300b1f00 0x7f813013fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 -- 172.21.15.73:0/174324641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009400 con 0x7f813015eb60 2024-04-05T22:30:36.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.302+0000 7f812fe00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097a0 tx=0x7f811c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.303+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c004230 con 0x7f813015eb60 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.303+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c031f00 con 0x7f813015eb60 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.303+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c032640 con 0x7f813015eb60 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.303+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81301404c0 con 0x7f813015eb60 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.303+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130150590 con 0x7f813015eb60 2024-04-05T22:30:36.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.304+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8130140300 con 0x7f813015eb60 2024-04-05T22:30:36.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.305+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f811c0327a0 con 0x7f813015eb60 2024-04-05T22:30:36.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.306+0000 7f812cc00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 0x7f811006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.306+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f811c0a9a70 con 0x7f813015eb60 2024-04-05T22:30:36.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.307+0000 7f8134c00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 0x7f811006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.308+0000 7f8134c00640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 0x7f811006d8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f81240098f0 tx=0x7f8124008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.311+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f811c0df650 con 0x7f813015eb60 2024-04-05T22:30:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.503+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8130140970 con 0x7f813015eb60 2024-04-05T22:30:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.506+0000 7f812cc00640 1 -- 172.21.15.73:0/174324641 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f8130140970 con 0x7f813015eb60 2024-04-05T22:30:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:36.507 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 msgr2=0x7f811006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 0x7f811006d8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f81240098f0 tx=0x7f8124008040 comp rx=0 tx=0).stop 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 msgr2=0x7f8130151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097a0 tx=0x7f811c0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 shutdown_connections 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f811006b3e0 0x7f811006d8a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8130151f80 0x7f81300c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813015eb60 0x7f8130151a40 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 --2- 172.21.15.73:0/174324641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81300b1f00 0x7f813013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.508+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 >> 172.21.15.73:0/174324641 conn(0x7f81300b06c0 msgr2=0x7f8130072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:36.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.509+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 shutdown_connections 2024-04-05T22:30:36.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.509+0000 7f8136000640 1 -- 172.21.15.73:0/174324641 wait complete. 2024-04-05T22:30:36.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:16.932891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:17.493378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:36.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:36.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.741+0000 7fd114c00640 1 Processor -- start 2024-04-05T22:30:36.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.741+0000 7fd114c00640 1 -- start start 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.742+0000 7fd114c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd11015abe0 0x7fd1100712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.742+0000 7fd114c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd11006c9a0 0x7fd11006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.742+0000 7fd114c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11006bea0 con 0x7fd11006ba70 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd114c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11015afc0 con 0x7fd11015abe0 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd114c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11006f5b0 con 0x7fd11006c9a0 2024-04-05T22:30:36.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd11015abe0 0x7fd1100712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd11015abe0 0x7fd1100712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39164/0 (socket says 172.21.15.73:39164) 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 -- 172.21.15.73:0/3282212283 learned_addr learned my addr 172.21.15.73:0/3282212283 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd105400640 1 --2- 172.21.15.73:0/3282212283 >> v2:172.21.15.157:3300/0 conn(0x7fd11006c9a0 0x7fd11006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 -- 172.21.15.73:0/3282212283 >> v2:172.21.15.157:3300/0 conn(0x7fd11006c9a0 msgr2=0x7fd11006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 --2- 172.21.15.73:0/3282212283 >> v2:172.21.15.157:3300/0 conn(0x7fd11006c9a0 0x7fd11006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 -- 172.21.15.73:0/3282212283 >> v1:172.21.15.157:6789/0 conn(0x7fd11006ba70 legacy=0x7fd1100cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:36.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.743+0000 7fd10de00640 1 -- 172.21.15.73:0/3282212283 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11006f890 con 0x7fd11015abe0 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.744+0000 7fd10de00640 1 --2- 172.21.15.73:0/3282212283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11015abe0 0x7fd1100712e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd11006ce30 tx=0x7fd10002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c4569dd717517c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.745+0000 7fd10d400640 1 -- 172.21.15.73:0/3282212283 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100031750 con 0x7fd11015abe0 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.745+0000 7fd10d400640 1 -- 172.21.15.73:0/3282212283 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1000318b0 con 0x7fd11015abe0 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.745+0000 7fd10d400640 1 -- 172.21.15.73:0/3282212283 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100031c40 con 0x7fd11015abe0 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.745+0000 7fd114c00640 1 -- 172.21.15.73:0/3282212283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11015abe0 msgr2=0x7fd1100712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.745+0000 7fd114c00640 1 --2- 172.21.15.73:0/3282212283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11015abe0 0x7fd1100712e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd11006ce30 tx=0x7fd10002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 -- 172.21.15.73:0/3282212283 shutdown_connections 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 --2- 172.21.15.73:0/3282212283 >> v2:172.21.15.157:3300/0 conn(0x7fd11006c9a0 0x7fd11006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 --2- 172.21.15.73:0/3282212283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11015abe0 0x7fd1100712e0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 -- 172.21.15.73:0/3282212283 >> 172.21.15.73:0/3282212283 conn(0x7fd110069cb0 msgr2=0x7fd11006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:36.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 -- 172.21.15.73:0/3282212283 shutdown_connections 2024-04-05T22:30:36.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.746+0000 7fd114c00640 1 -- 172.21.15.73:0/3282212283 wait complete. 2024-04-05T22:30:36.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.747+0000 7fd114c00640 1 Processor -- start 2024-04-05T22:30:36.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.747+0000 7fd114c00640 1 -- start start 2024-04-05T22:30:36.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd11006c9a0 0x7fd11014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd11015abe0 0x7fd110155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11013a5a0 con 0x7fd11006ba70 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110158be0 con 0x7fd11006c9a0 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.748+0000 7fd114c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd110158ec0 con 0x7fd11015abe0 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39174/0 (socket says 172.21.15.73:39174) 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 -- 172.21.15.73:0/2181622711 learned_addr learned my addr 172.21.15.73:0/2181622711 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 -- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd11015abe0 msgr2=0x7fd110155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd11015abe0 0x7fd110155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 -- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd11006c9a0 msgr2=0x7fd11014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd11006c9a0 0x7fd11014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 -- 172.21.15.73:0/2181622711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000094d0 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.749+0000 7fd10de00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd100000c00 tx=0x7fd1000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.750+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100033c80 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.750+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100033de0 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.750+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11014dab0 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.750+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10003c640 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.750+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11014e070 con 0x7fd11006ba70 2024-04-05T22:30:36.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.751+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11014d910 con 0x7fd11006ba70 2024-04-05T22:30:36.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.752+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fd10003a040 con 0x7fd11006ba70 2024-04-05T22:30:36.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.753+0000 7fd106a00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 0x7fd0ec06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:36.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.753+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1000a9860 con 0x7fd11006ba70 2024-04-05T22:30:36.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.754+0000 7fd105400640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 0x7fd0ec06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:36.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.755+0000 7fd105400640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 0x7fd0ec06d8a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8006fd0 tx=0x7fd0f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:36.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.758+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd11014d910 con 0x7fd11006ba70 2024-04-05T22:30:36.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.952+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd11014d910 con 0x7fd11006ba70 2024-04-05T22:30:36.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.953+0000 7fd106a00640 1 -- 172.21.15.73:0/2181622711 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd11014d910 con 0x7fd11006ba70 2024-04-05T22:30:36.953 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 msgr2=0x7fd0ec06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 0x7fd0ec06d8a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8006fd0 tx=0x7fd0f8008040 comp rx=0 tx=0).stop 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 msgr2=0x7fd11013a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd100000c00 tx=0x7fd1000046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 shutdown_connections 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd0ec06b3e0 0x7fd0ec06d8a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd11015abe0 0x7fd110155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd11006c9a0 0x7fd11014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 --2- 172.21.15.73:0/2181622711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd11006ba70 0x7fd11013a060 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:36.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.955+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 >> 172.21.15.73:0/2181622711 conn(0x7fd110069cb0 msgr2=0x7fd11015b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:36.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.956+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 shutdown_connections 2024-04-05T22:30:36.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:36.956+0000 7fd114c00640 1 -- 172.21.15.73:0/2181622711 wait complete. 2024-04-05T22:30:36.974 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:30:37.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.186+0000 7fbdbe600640 1 Processor -- start 2024-04-05T22:30:37.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.186+0000 7fbdbe600640 1 -- start start 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 0x7fbdb80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 0x7fbdb806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbdb806ba70 0x7fbdb80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb806bec0 con 0x7fbdb806ba70 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb815afc0 con 0x7fbdb815abe0 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.187+0000 7fbdbe600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb806f5b0 con 0x7fbdb806c9a0 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbdb806ba70 0x7fbdb80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbdb806ba70 0x7fbdb80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39176/0 (socket says 172.21.15.73:39176) 2024-04-05T22:30:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 -- 172.21.15.73:0/557355778 learned_addr learned my addr 172.21.15.73:0/557355778 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:37.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7400640 1 --2- 172.21.15.73:0/557355778 >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 0x7fbdb80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdaea00640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 0x7fbdb806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 -- 172.21.15.73:0/557355778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 msgr2=0x7fbdb806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:37.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 0x7fbdb806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 -- 172.21.15.73:0/557355778 >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 msgr2=0x7fbdb80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/557355778 >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 0x7fbdb80712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7e00640 1 -- 172.21.15.73:0/557355778 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb806f890 con 0x7fbdb806ba70 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdb7400640 1 --2- 172.21.15.73:0/557355778 >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 0x7fbdb80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.188+0000 7fbdaea00640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 0x7fbdb806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.189+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb806ba70 0x7fbdb80cccb0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbdb806d300 tx=0x7fbdb002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bf53f4714729cb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.190+0000 7fbdb6a00640 1 -- 172.21.15.73:0/557355778 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0031740 con 0x7fbdb806ba70 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.190+0000 7fbdb6a00640 1 -- 172.21.15.73:0/557355778 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb00318a0 con 0x7fbdb806ba70 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.190+0000 7fbdb6a00640 1 -- 172.21.15.73:0/557355778 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0031c30 con 0x7fbdb806ba70 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.190+0000 7fbdbe600640 1 -- 172.21.15.73:0/557355778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb806ba70 msgr2=0x7fbdb80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.190+0000 7fbdbe600640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb806ba70 0x7fbdb80cccb0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbdb806d300 tx=0x7fbdb002b7e0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 -- 172.21.15.73:0/557355778 shutdown_connections 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb806ba70 0x7fbdb80cccb0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 --2- 172.21.15.73:0/557355778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806c9a0 0x7fbdb806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 --2- 172.21.15.73:0/557355778 >> v2:172.21.15.157:3300/0 conn(0x7fbdb815abe0 0x7fbdb80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 -- 172.21.15.73:0/557355778 >> 172.21.15.73:0/557355778 conn(0x7fbdb8069cb0 msgr2=0x7fbdb806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 -- 172.21.15.73:0/557355778 shutdown_connections 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.191+0000 7fbdbe600640 1 -- 172.21.15.73:0/557355778 wait complete. 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.192+0000 7fbdbe600640 1 Processor -- start 2024-04-05T22:30:37.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 -- start start 2024-04-05T22:30:37.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806ba70 0x7fbdb8142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb806c9a0 0x7fbdb80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8149ce0 con 0x7fbdb815abe0 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb814a580 con 0x7fbdb806c9a0 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.193+0000 7fbdbe600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8155910 con 0x7fbdb806ba70 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39190/0 (socket says 172.21.15.73:39190) 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 -- 172.21.15.73:0/3606315417 learned_addr learned my addr 172.21.15.73:0/3606315417 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 -- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806ba70 msgr2=0x7fbdb8142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806ba70 0x7fbdb8142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 -- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb806c9a0 msgr2=0x7fbdb80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb806c9a0 0x7fbdb80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 -- 172.21.15.73:0/3606315417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00094d0 con 0x7fbdb815abe0 2024-04-05T22:30:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.194+0000 7fbdb7e00640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fbdb002b820 tx=0x7fbdb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.195+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0033d60 con 0x7fbdb815abe0 2024-04-05T22:30:37.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.195+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb003c400 con 0x7fbdb815abe0 2024-04-05T22:30:37.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.195+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb003c7b0 con 0x7fbdb815abe0 2024-04-05T22:30:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.195+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb8149f30 con 0x7fbdb815abe0 2024-04-05T22:30:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.195+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb81427b0 con 0x7fbdb815abe0 2024-04-05T22:30:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.196+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb8068160 con 0x7fbdb815abe0 2024-04-05T22:30:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.197+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fbdb003a040 con 0x7fbdb815abe0 2024-04-05T22:30:37.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.198+0000 7fbdafe00640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 0x7fbd9406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.198+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdb00a95e0 con 0x7fbdb815abe0 2024-04-05T22:30:37.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.198+0000 7fbdb7400640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 0x7fbd9406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.200+0000 7fbdb7400640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 0x7fbd9406d8d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007130 tx=0x7fbda8007300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.202+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbdb00764e0 con 0x7fbdb815abe0 2024-04-05T22:30:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.400+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbdb814a710 con 0x7fbdb815abe0 2024-04-05T22:30:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.401+0000 7fbdafe00640 1 -- 172.21.15.73:0/3606315417 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbdb0038020 con 0x7fbdb815abe0 2024-04-05T22:30:37.402 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 msgr2=0x7fbd9406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 0x7fbd9406d8d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007130 tx=0x7fbda8007300 comp rx=0 tx=0).stop 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 msgr2=0x7fbdb80af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fbdb002b820 tx=0x7fbdb00046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 shutdown_connections 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd9406b410 0x7fbd9406d8d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbdb815abe0 0x7fbdb80af0a0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb806c9a0 0x7fbdb80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 --2- 172.21.15.73:0/3606315417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdb806ba70 0x7fbdb8142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 >> 172.21.15.73:0/3606315417 conn(0x7fbdb8069cb0 msgr2=0x7fbdb80cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 shutdown_connections 2024-04-05T22:30:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.406+0000 7fbdbe600640 1 -- 172.21.15.73:0/3606315417 wait complete. 2024-04-05T22:30:37.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:30:37.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.634+0000 7f2eaee00640 1 Processor -- start 2024-04-05T22:30:37.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.634+0000 7f2eaee00640 1 -- start start 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eaee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ea80cbe00 0x7f2ea80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eaee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80b1b30 con 0x7f2ea80b32b0 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eaee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80b1e10 con 0x7f2ea806c630 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eaee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80cc300 con 0x7f2ea80cbe00 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eace00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ea80b32b0 0x7f2ea81647c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59526/0 (socket says 172.21.15.73:59526) 2024-04-05T22:30:37.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2eace00640 1 -- 172.21.15.73:0/974667450 learned_addr learned my addr 172.21.15.73:0/974667450 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:37.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/974667450 >> v2:172.21.15.157:3300/0 conn(0x7f2ea80cbe00 0x7f2ea80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.636+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114705807 0 0) 0x7f2ea80b1b30 con 0x7f2ea80b32b0 2024-04-05T22:30:37.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e8c003520 con 0x7f2ea80b32b0 2024-04-05T22:30:37.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146299316 0 0) 0x7f2e8c003520 con 0x7f2ea80b32b0 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 >> v2:172.21.15.157:3300/0 conn(0x7f2ea80cbe00 msgr2=0x7f2ea80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 --2- 172.21.15.73:0/974667450 >> v2:172.21.15.157:3300/0 conn(0x7f2ea80cbe00 0x7f2ea80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 >> v1:172.21.15.157:6789/0 conn(0x7f2ea806c630 legacy=0x7f2ea806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80cc5e0 con 0x7f2ea80b32b0 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2ea00019f0 con 0x7f2ea80b32b0 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 >> v1:172.21.15.73:6789/0 conn(0x7f2ea80b32b0 legacy=0x7f2ea81647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e8c003970 0x7f2e8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 0x7f2e8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c005180 0x7f2e8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cc300 con 0x7f2e8c0047c0 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c00c020 con 0x7f2e8c005180 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c00c1e0 con 0x7f2e8c003970 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.637+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 0x7f2e8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2ea7e00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e8c003970 0x7f2e8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2eace00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c005180 0x7f2e8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2e9fe00640 1 -- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e8c003970 msgr2=0x7f2e8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e8c003970 0x7f2e8c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2e9fe00640 1 -- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c005180 msgr2=0x7f2e8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c005180 0x7f2e8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2e9fe00640 1 -- 172.21.15.73:0/974667450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c003520 con 0x7f2e8c0047c0 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2eace00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c005180 0x7f2e8c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:37.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.638+0000 7f2eace00640 1 -- 172.21.15.73:0/974667450 reap_dead start 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.639+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 0x7f2e8c004c40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009ac0 tx=0x7f2e8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d82889254f5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.639+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9002c6b0 con 0x7f2e8c0047c0 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.639+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9002c810 con 0x7f2e8c0047c0 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.639+0000 7f2ea7400640 1 -- 172.21.15.73:0/974667450 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9002cba0 con 0x7f2e8c0047c0 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 -- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 msgr2=0x7f2e8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 0x7f2e8c004c40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009ac0 tx=0x7f2e8c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 -- 172.21.15.73:0/974667450 shutdown_connections 2024-04-05T22:30:37.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 --2- 172.21.15.73:0/974667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e8c0047c0 0x7f2e8c004c40 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 -- 172.21.15.73:0/974667450 >> 172.21.15.73:0/974667450 conn(0x7f2ea80b0830 msgr2=0x7f2ea80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:37.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 -- 172.21.15.73:0/974667450 shutdown_connections 2024-04-05T22:30:37.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.640+0000 7f2eaee00640 1 -- 172.21.15.73:0/974667450 wait complete. 2024-04-05T22:30:37.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.642+0000 7f2eaee00640 1 Processor -- start 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.642+0000 7f2eaee00640 1 -- start start 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c0047c0 0x7f2ea8138270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 0x7f2ea8140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8140980 con 0x7f2ea8142950 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8143310 con 0x7f2e8c0047c0 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eaee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80c4e00 con 0x7f2ea813ffe0 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39206/0 (socket says 172.21.15.73:39206) 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 -- 172.21.15.73:0/550145818 learned_addr learned my addr 172.21.15.73:0/550145818 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2ea7e00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c0047c0 0x7f2ea8138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 0x7f2ea8140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 -- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 msgr2=0x7f2ea8140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 0x7f2ea8140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 -- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c0047c0 msgr2=0x7f2ea8138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c0047c0 0x7f2ea8138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2eace00640 1 -- 172.21.15.73:0/550145818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea81387b0 con 0x7f2ea8142950 2024-04-05T22:30:37.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.643+0000 7f2e9fe00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 0x7f2ea8140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:37.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2eace00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00012e0 tx=0x7f2ea000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0004d60 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000bb20 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000bed0 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea8141790 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.644+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea8140e60 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.645+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8141920 con 0x7f2ea8142950 2024-04-05T22:30:37.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.646+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f2ea0007110 con 0x7f2ea8142950 2024-04-05T22:30:37.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.647+0000 7f2ea4c00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 0x7f2e8406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:37.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.647+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ea0087d80 con 0x7f2ea8142950 2024-04-05T22:30:37.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.648+0000 7f2ea7e00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 0x7f2e8406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:37.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.649+0000 7f2ea7e00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 0x7f2e8406d8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e98007fd0 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:37.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.652+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2ea0054b60 con 0x7f2ea8142950 2024-04-05T22:30:37.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.844+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ea8068160 con 0x7f2ea8142950 2024-04-05T22:30:37.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.845+0000 7f2ea4c00640 1 -- 172.21.15.73:0/550145818 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ea0058180 con 0x7f2ea8142950 2024-04-05T22:30:37.845 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:30:37.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 msgr2=0x7f2e8406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 0x7f2e8406d8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e98007fd0 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 msgr2=0x7f2ea8142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00012e0 tx=0x7f2ea000b4a0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 shutdown_connections 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e8406b410 0x7f2e8406d8d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ea8142950 0x7f2ea8142dd0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ea813ffe0 0x7f2ea8140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 --2- 172.21.15.73:0/550145818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8c0047c0 0x7f2ea8138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 >> 172.21.15.73:0/550145818 conn(0x7f2ea80b0830 msgr2=0x7f2ea80aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 shutdown_connections 2024-04-05T22:30:37.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:37.848+0000 7f2eaee00640 1 -- 172.21.15.73:0/550145818 wait complete. 2024-04-05T22:30:37.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:30:38.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.081+0000 7f56d0600640 1 Processor -- start 2024-04-05T22:30:38.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.081+0000 7f56d0600640 1 -- start start 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 0x7f56c80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56c80c4530 0x7f56c80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c815f5f0 con 0x7f56c80c4530 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c815f350 con 0x7f56c815acc0 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.082+0000 7f56d0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c3830 con 0x7f56c815a8e0 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56c80c4530 0x7f56c80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56c80c4530 0x7f56c80ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39208/0 (socket says 172.21.15.73:39208) 2024-04-05T22:30:38.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 -- 172.21.15.73:0/3472143817 learned_addr learned my addr 172.21.15.73:0/3472143817 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56cd200640 1 --2- 172.21.15.73:0/3472143817 >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 0x7f56c80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 -- 172.21.15.73:0/3472143817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 msgr2=0x7f56c80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56cdc00640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 -- 172.21.15.73:0/3472143817 >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 msgr2=0x7f56c80c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 --2- 172.21.15.73:0/3472143817 >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 0x7f56c80c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56ce600640 1 -- 172.21.15.73:0/3472143817 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80c3ad0 con 0x7f56c80c4530 2024-04-05T22:30:38.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.083+0000 7f56cd200640 1 --2- 172.21.15.73:0/3472143817 >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 0x7f56c80c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:38.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.084+0000 7f56ce600640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c80ca9f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f56c815f520 tx=0x7f56c002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a6137e3c292d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.085+0000 7f56c7e00640 1 -- 172.21.15.73:0/3472143817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c002c950 con 0x7f56c80c4530 2024-04-05T22:30:38.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.085+0000 7f56c7e00640 1 -- 172.21.15.73:0/3472143817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c002cab0 con 0x7f56c80c4530 2024-04-05T22:30:38.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.085+0000 7f56c7e00640 1 -- 172.21.15.73:0/3472143817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c002ce40 con 0x7f56c80c4530 2024-04-05T22:30:38.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.085+0000 7f56d0600640 1 -- 172.21.15.73:0/3472143817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 msgr2=0x7f56c80ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.085+0000 7f56d0600640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c80ca9f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f56c815f520 tx=0x7f56c002bdb0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 -- 172.21.15.73:0/3472143817 shutdown_connections 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c80ca9f0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 --2- 172.21.15.73:0/3472143817 >> v2:172.21.15.157:3300/0 conn(0x7f56c815acc0 0x7f56c80c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 --2- 172.21.15.73:0/3472143817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 -- 172.21.15.73:0/3472143817 >> 172.21.15.73:0/3472143817 conn(0x7f56c80b0490 msgr2=0x7f56c80b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 -- 172.21.15.73:0/3472143817 shutdown_connections 2024-04-05T22:30:38.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.086+0000 7f56d0600640 1 -- 172.21.15.73:0/3472143817 wait complete. 2024-04-05T22:30:38.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.087+0000 7f56d0600640 1 Processor -- start 2024-04-05T22:30:38.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.087+0000 7f56d0600640 1 -- start start 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c8148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c815acc0 0x7f56c8146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8148450 con 0x7f56c80c4530 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8148db0 con 0x7f56c815acc0 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.088+0000 7f56d0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8147520 con 0x7f56c815a8e0 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39218/0 (socket says 172.21.15.73:39218) 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 -- 172.21.15.73:0/4237635881 learned_addr learned my addr 172.21.15.73:0/4237635881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 -- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 msgr2=0x7f56c8148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c8148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 -- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c815acc0 msgr2=0x7f56c8146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c815acc0 0x7f56c8146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 -- 172.21.15.73:0/4237635881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0009b30 con 0x7f56c80c4530 2024-04-05T22:30:38.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.089+0000 7f56cdc00640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c7f0 tx=0x7f56bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.090+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004220 con 0x7f56c80c4530 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.090+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc004380 con 0x7f56c80c4530 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.090+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007840 con 0x7f56c80c4530 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.090+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c81477d0 con 0x7f56c80c4530 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.090+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c8147d90 con 0x7f56c80c4530 2024-04-05T22:30:38.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.091+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c8068160 con 0x7f56c80c4530 2024-04-05T22:30:38.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.092+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f56bc007bb0 con 0x7f56c80c4530 2024-04-05T22:30:38.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.093+0000 7f56c5600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 0x7f56a806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.093+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56bc0879c0 con 0x7f56c80c4530 2024-04-05T22:30:38.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.094+0000 7f56cd200640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 0x7f56a806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.095+0000 7f56cd200640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 0x7f56a806d8a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56c8157100 tx=0x7f56b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.097+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f56bc0547a0 con 0x7f56c80c4530 2024-04-05T22:30:38.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.306+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56c80c4cb0 con 0x7f56c80c4530 2024-04-05T22:30:38.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.307+0000 7f56c5600640 1 -- 172.21.15.73:0/4237635881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56bc01e020 con 0x7f56c80c4530 2024-04-05T22:30:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:30:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 msgr2=0x7f56a806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 0x7f56a806d8a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56c8157100 tx=0x7f56b4008040 comp rx=0 tx=0).stop 2024-04-05T22:30:38.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 msgr2=0x7f56c8149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c7f0 tx=0x7f56bc00ccb0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 shutdown_connections 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f56a806b3e0 0x7f56a806d8a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c815acc0 0x7f56c8146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56c815a8e0 0x7f56c8148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 --2- 172.21.15.73:0/4237635881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c80c4530 0x7f56c8149c00 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 >> 172.21.15.73:0/4237635881 conn(0x7f56c80b0490 msgr2=0x7f56c806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 shutdown_connections 2024-04-05T22:30:38.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.309+0000 7f56d0600640 1 -- 172.21.15.73:0/4237635881 wait complete. 2024-04-05T22:30:38.330 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:30:38.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:38.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:30:38.440 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:30:38.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:38.440 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:30:38.491 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:38.491 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:38.549 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:38.549 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:38.597 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:38.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.816+0000 7f4104a00640 1 Processor -- start 2024-04-05T22:30:38.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.817+0000 7f4104a00640 1 -- start start 2024-04-05T22:30:38.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f4104a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4100160620 0x7f41000caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f4104a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 0x7f41000cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f4104a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41000ad880 con 0x7f41000b1870 2024-04-05T22:30:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f4104a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41000cbcc0 con 0x7f4100160620 2024-04-05T22:30:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f4104a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41000ce5d0 con 0x7f41000cb2f0 2024-04-05T22:30:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.818+0000 7f40f7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f41000b1870 0x7f41000ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59538/0 (socket says 172.21.15.73:59538) 2024-04-05T22:30:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40f7e00640 1 -- 172.21.15.73:0/3733138079 learned_addr learned my addr 172.21.15.73:0/3733138079 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154611649 0 0) 0x7f41000ad880 con 0x7f41000b1870 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e0003520 con 0x7f41000b1870 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fde00640 1 --2- 172.21.15.73:0/3733138079 >> v2:172.21.15.157:3300/0 conn(0x7f4100160620 0x7f41000caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fe800640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 0x7f41000cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961048556 0 0) 0x7f40e0003520 con 0x7f41000b1870 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 msgr2=0x7f41000cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 0x7f41000cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 >> v2:172.21.15.157:3300/0 conn(0x7f4100160620 msgr2=0x7f41000caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 --2- 172.21.15.73:0/3733138079 >> v2:172.21.15.157:3300/0 conn(0x7f4100160620 0x7f41000caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000ce010 con 0x7f41000b1870 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f40e8001690 con 0x7f41000b1870 2024-04-05T22:30:38.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 >> v1:172.21.15.73:6789/0 conn(0x7f41000b1870 legacy=0x7f41000ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e0003970 0x7f40e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 0x7f40e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e0004d00 0x7f40e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000cbcc0 con 0x7f40e0004d00 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000ce5d0 con 0x7f40e0003970 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.819+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e000bb10 con 0x7f41000cb2f0 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e0004d00 0x7f40e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fde00640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e0003970 0x7f40e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 -- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 msgr2=0x7f40e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41000cb2f0 0x7f40e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 -- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e0003970 msgr2=0x7f40e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e0003970 0x7f40e0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.820+0000 7f40fe800640 1 -- 172.21.15.73:0/3733138079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0003520 con 0x7f40e0004d00 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.821+0000 7f40fe800640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e0004d00 0x7f40e0005180 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f40f8007360 tx=0x7f40e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41367ef5bcb99cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.821+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f802ccf0 con 0x7f40e0004d00 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.821+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f802ce50 con 0x7f40e0004d00 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.821+0000 7f40fd400640 1 -- 172.21.15.73:0/3733138079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8034660 con 0x7f40e0004d00 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 -- 172.21.15.73:0/3733138079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e0004d00 msgr2=0x7f40e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 --2- 172.21.15.73:0/3733138079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40e0004d00 0x7f40e0005180 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f40f8007360 tx=0x7f40e00057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f40fe800640 1 -- 172.21.15.73:0/3733138079 reap_dead start 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 -- 172.21.15.73:0/3733138079 shutdown_connections 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 -- 172.21.15.73:0/3733138079 >> 172.21.15.73:0/3733138079 conn(0x7f41000ce8e0 msgr2=0x7f41000cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 -- 172.21.15.73:0/3733138079 shutdown_connections 2024-04-05T22:30:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.822+0000 7f4104a00640 1 -- 172.21.15.73:0/3733138079 wait complete. 2024-04-05T22:30:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.824+0000 7f4104a00640 1 Processor -- start 2024-04-05T22:30:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.824+0000 7f4104a00640 1 -- start start 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 0x7f410014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410006fc00 0x7f410014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000b00b0 con 0x7f4100153b60 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4100150020 con 0x7f410006f820 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f4104a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41000af430 con 0x7f410006fc00 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f40fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f40fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39244/0 (socket says 172.21.15.73:39244) 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.825+0000 7f40fe800640 1 -- 172.21.15.73:0/2059267418 learned_addr learned my addr 172.21.15.73:0/2059267418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fde00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 0x7f410014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40f7e00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410006fc00 0x7f410014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 -- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410006fc00 msgr2=0x7f410014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410006fc00 0x7f410014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 -- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 msgr2=0x7f410014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 0x7f410014e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 -- 172.21.15.73:0/2059267418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000af710 con 0x7f4100153b60 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fde00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 0x7f410014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:38.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40fe800640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f40f8037040 tx=0x7f40f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.826+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f803e600 con 0x7f4100153b60 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.827+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f803e760 con 0x7f4100153b60 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.827+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41000af9f0 con 0x7f4100153b60 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.827+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f803eaf0 con 0x7f4100153b60 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.827+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4100154550 con 0x7f4100153b60 2024-04-05T22:30:38.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.828+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4100066af0 con 0x7f4100153b60 2024-04-05T22:30:38.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.829+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f40f803ec50 con 0x7f4100153b60 2024-04-05T22:30:38.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.830+0000 7f40f6000640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 0x7f40dc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:38.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.830+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40f80a9f00 con 0x7f4100153b60 2024-04-05T22:30:38.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.830+0000 7f40fde00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 0x7f40dc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:38.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.832+0000 7f40fde00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 0x7f40dc06d8d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009900 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:38.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:38.834+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f40f8076e00 con 0x7f4100153b60 2024-04-05T22:30:39.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.027+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41000b01f0 con 0x7f4100153b60 2024-04-05T22:30:39.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.029+0000 7f40f6000640 1 -- 172.21.15.73:0/2059267418 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f41000b01f0 con 0x7f4100153b60 2024-04-05T22:30:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:39.030 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 msgr2=0x7f40dc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 0x7f40dc06d8d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009900 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 msgr2=0x7f4100154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f40f8037040 tx=0x7f40f8004ab0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 shutdown_connections 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f40dc06b410 0x7f40dc06d8d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4100153b60 0x7f4100154010 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410006fc00 0x7f410014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 --2- 172.21.15.73:0/2059267418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410006f820 0x7f410014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 >> 172.21.15.73:0/2059267418 conn(0x7f41000ce8e0 msgr2=0x7f41000705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:39.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 shutdown_connections 2024-04-05T22:30:39.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.032+0000 7f4104a00640 1 -- 172.21.15.73:0/2059267418 wait complete. 2024-04-05T22:30:39.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:16.932891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:17.493378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:39.051 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:39.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.261+0000 7f3ee1000640 1 Processor -- start 2024-04-05T22:30:39.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.261+0000 7f3ee1000640 1 -- start start 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3ee1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3edc15cf30 0x7f3edc0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3ee1000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc0cecd0 con 0x7f3edc0d1810 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3ee1000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc0cdc30 con 0x7f3edc15d310 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3ee1000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc0ce8b0 con 0x7f3edc15cf30 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3ed9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3edc15cf30 0x7f3edc0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3eda800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3edc0d1810 0x7f3edc0b7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59540/0 (socket says 172.21.15.73:59540) 2024-04-05T22:30:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.263+0000 7f3eda800640 1 -- 172.21.15.73:0/1091795483 learned_addr learned my addr 172.21.15.73:0/1091795483 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.264+0000 7f3ed8a00640 1 -- 172.21.15.73:0/1091795483 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843618201 0 0) 0x7f3edc0cecd0 con 0x7f3edc0d1810 2024-04-05T22:30:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.264+0000 7f3ed8a00640 1 -- 172.21.15.73:0/1091795483 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec0003520 con 0x7f3edc0d1810 2024-04-05T22:30:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.264+0000 7f3ed9e00640 1 -- 172.21.15.73:0/1091795483 >> v1:172.21.15.157:6789/0 conn(0x7f3edc15d310 legacy=0x7f3edc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.264+0000 7f3ed9e00640 1 -- 172.21.15.73:0/1091795483 >> v1:172.21.15.73:6789/0 conn(0x7f3edc0d1810 legacy=0x7f3edc0b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.264+0000 7f3ed9e00640 1 -- 172.21.15.73:0/1091795483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc0cdf10 con 0x7f3edc15cf30 2024-04-05T22:30:39.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.265+0000 7f3ed9e00640 1 --2- 172.21.15.73:0/1091795483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc15cf30 0x7f3edc0d0a00 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffd2a576bd9b851 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.265+0000 7f3ed8a00640 1 -- 172.21.15.73:0/1091795483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc02e070 con 0x7f3edc15cf30 2024-04-05T22:30:39.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.265+0000 7f3ed8a00640 1 -- 172.21.15.73:0/1091795483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0315a0 con 0x7f3edc15cf30 2024-04-05T22:30:39.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.265+0000 7f3ed8a00640 1 -- 172.21.15.73:0/1091795483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc031930 con 0x7f3edc15cf30 2024-04-05T22:30:39.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 -- 172.21.15.73:0/1091795483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc15cf30 msgr2=0x7f3edc0d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 --2- 172.21.15.73:0/1091795483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc15cf30 0x7f3edc0d0a00 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b840 comp rx=0 tx=0).stop 2024-04-05T22:30:39.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 -- 172.21.15.73:0/1091795483 shutdown_connections 2024-04-05T22:30:39.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 --2- 172.21.15.73:0/1091795483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc15cf30 0x7f3edc0d0a00 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 -- 172.21.15.73:0/1091795483 >> 172.21.15.73:0/1091795483 conn(0x7f3edc0d2c20 msgr2=0x7f3edc0d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:39.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 -- 172.21.15.73:0/1091795483 shutdown_connections 2024-04-05T22:30:39.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.266+0000 7f3ee1000640 1 -- 172.21.15.73:0/1091795483 wait complete. 2024-04-05T22:30:39.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.268+0000 7f3ee1000640 1 Processor -- start 2024-04-05T22:30:39.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.268+0000 7f3ee1000640 1 -- start start 2024-04-05T22:30:39.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc15cf30 0x7f3edc0b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15d310 0x7f3edc09d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc09c900 con 0x7f3edc0d1810 2024-04-05T22:30:39.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1f3450 con 0x7f3edc15cf30 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ee1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1f3730 con 0x7f3edc15d310 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39250/0 (socket says 172.21.15.73:39250) 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 -- 172.21.15.73:0/2511836447 learned_addr learned my addr 172.21.15.73:0/2511836447 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 -- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15d310 msgr2=0x7f3edc09d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15d310 0x7f3edc09d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 -- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc15cf30 msgr2=0x7f3edc0b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc15cf30 0x7f3edc0b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.269+0000 7f3ed9e00640 1 -- 172.21.15.73:0/2511836447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009400 con 0x7f3edc0d1810 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ed9e00640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034040 tx=0x7f3ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc032840 con 0x7f3edc0d1810 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0329c0 con 0x7f3edc0d1810 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc032d70 con 0x7f3edc0d1810 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc09cb50 con 0x7f3edc0d1810 2024-04-05T22:30:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.270+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc1f4050 con 0x7f3edc0d1810 2024-04-05T22:30:39.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.272+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc09d070 con 0x7f3edc0d1810 2024-04-05T22:30:39.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.272+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f3ecc03b3f0 con 0x7f3edc0d1810 2024-04-05T22:30:39.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.273+0000 7f3ed2000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 0x7f3ebc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.273+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ecc0a9930 con 0x7f3edc0d1810 2024-04-05T22:30:39.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.274+0000 7f3ed9400640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 0x7f3ebc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.275+0000 7f3ed9400640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 0x7f3ebc06d8a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3edc09e6e0 tx=0x7f3ec80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.278+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3edc09d070 con 0x7f3edc0d1810 2024-04-05T22:30:39.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.471+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3edc09d070 con 0x7f3edc0d1810 2024-04-05T22:30:39.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.472+0000 7f3ed2000640 1 -- 172.21.15.73:0/2511836447 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3edc09d070 con 0x7f3edc0d1810 2024-04-05T22:30:39.472 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 msgr2=0x7f3ebc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 0x7f3ebc06d8a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3edc09e6e0 tx=0x7f3ec80079b0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 msgr2=0x7f3edc1f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034040 tx=0x7f3ecc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 shutdown_connections 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3ebc06b3e0 0x7f3ebc06d8a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15d310 0x7f3edc09d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc15cf30 0x7f3edc0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 --2- 172.21.15.73:0/2511836447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3edc0d1810 0x7f3edc1f3b10 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 >> 172.21.15.73:0/2511836447 conn(0x7f3edc0d2c20 msgr2=0x7f3edc06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:39.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 shutdown_connections 2024-04-05T22:30:39.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.475+0000 7f3ee1000640 1 -- 172.21.15.73:0/2511836447 wait complete. 2024-04-05T22:30:39.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:30:39.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.705+0000 7fb4fce00640 1 Processor -- start 2024-04-05T22:30:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.705+0000 7fb4fce00640 1 -- start start 2024-04-05T22:30:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4fce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f815d1a0 0x7fb4f80d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4fce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4f80d15b0 0x7fb4f80b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4fce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80cd630 con 0x7fb4f815cdc0 2024-04-05T22:30:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4fce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f80d1a40 con 0x7fb4f80d15b0 2024-04-05T22:30:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4fce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80ceab0 con 0x7fb4f815d1a0 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4f7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4f80d15b0 0x7fb4f80b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4f7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4f80d15b0 0x7fb4f80b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39262/0 (socket says 172.21.15.73:39262) 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.707+0000 7fb4f7e00640 1 -- 172.21.15.73:0/998969377 learned_addr learned my addr 172.21.15.73:0/998969377 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.708+0000 7fb4f6a00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f815d1a0 0x7fb4f80d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.708+0000 7fb4f7e00640 1 -- 172.21.15.73:0/998969377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f815d1a0 msgr2=0x7fb4f80d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.708+0000 7fb4f7e00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f815d1a0 0x7fb4f80d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.708+0000 7fb4f7e00640 1 -- 172.21.15.73:0/998969377 >> v1:172.21.15.157:6789/0 conn(0x7fb4f815cdc0 legacy=0x7fb4f80d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.708+0000 7fb4f7e00640 1 -- 172.21.15.73:0/998969377 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f815e280 con 0x7fb4f80d15b0 2024-04-05T22:30:39.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.709+0000 7fb4f7e00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f80d15b0 0x7fb4f80b37c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6141f294dc33c86e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.709+0000 7fb4f6000640 1 -- 172.21.15.73:0/998969377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f002cb00 con 0x7fb4f80d15b0 2024-04-05T22:30:39.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.709+0000 7fb4f6000640 1 -- 172.21.15.73:0/998969377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f002cc60 con 0x7fb4f80d15b0 2024-04-05T22:30:39.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.709+0000 7fb4f6000640 1 -- 172.21.15.73:0/998969377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00343f0 con 0x7fb4f80d15b0 2024-04-05T22:30:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 -- 172.21.15.73:0/998969377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f80d15b0 msgr2=0x7fb4f80b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f80d15b0 0x7fb4f80b37c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f002bf60 comp rx=0 tx=0).stop 2024-04-05T22:30:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 -- 172.21.15.73:0/998969377 shutdown_connections 2024-04-05T22:30:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f80d15b0 0x7fb4f80b37c0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 --2- 172.21.15.73:0/998969377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f815d1a0 0x7fb4f80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 -- 172.21.15.73:0/998969377 >> 172.21.15.73:0/998969377 conn(0x7fb4f809be20 msgr2=0x7fb4f80d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 -- 172.21.15.73:0/998969377 shutdown_connections 2024-04-05T22:30:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.710+0000 7fb4fce00640 1 -- 172.21.15.73:0/998969377 wait complete. 2024-04-05T22:30:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.712+0000 7fb4fce00640 1 Processor -- start 2024-04-05T22:30:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.712+0000 7fb4fce00640 1 -- start start 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 0x7fb4f8147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 0x7fb4f815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8159d20 con 0x7fb4f815d1a0 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8149ca0 con 0x7fb4f815cdc0 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4fce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8147760 con 0x7fb4f80d15b0 2024-04-05T22:30:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39266/0 (socket says 172.21.15.73:39266) 2024-04-05T22:30:39.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.713+0000 7fb4f7e00640 1 -- 172.21.15.73:0/318230091 learned_addr learned my addr 172.21.15.73:0/318230091 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f6a00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 0x7fb4f815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7400640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 0x7fb4f8147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 -- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 msgr2=0x7fb4f8147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 0x7fb4f8147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 -- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 msgr2=0x7fb4f815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 0x7fb4f815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 -- 172.21.15.73:0/318230091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009b30 con 0x7fb4f815d1a0 2024-04-05T22:30:39.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f6a00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 0x7fb4f815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7400640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 0x7fb4f8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4f7e00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0009510 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f003e450 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f81f6b50 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f003e800 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.714+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f814a240 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.716+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f815fb40 con 0x7fb4f815d1a0 2024-04-05T22:30:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.717+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fb4f003e960 con 0x7fb4f815d1a0 2024-04-05T22:30:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.717+0000 7fb4ef400640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 0x7fb4d406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.718+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4f00aa7b0 con 0x7fb4f815d1a0 2024-04-05T22:30:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.718+0000 7fb4f7400640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 0x7fb4d406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.719+0000 7fb4f7400640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 0x7fb4d406d8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8000f60 tx=0x7fb4e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:39.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.722+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb4f0077590 con 0x7fb4f815d1a0 2024-04-05T22:30:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.914+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4f804e160 con 0x7fb4f815d1a0 2024-04-05T22:30:39.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.915+0000 7fb4ef400640 1 -- 172.21.15.73:0/318230091 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4f007abb0 con 0x7fb4f815d1a0 2024-04-05T22:30:39.915 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.917+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 msgr2=0x7fb4d406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.917+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 0x7fb4d406d8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8000f60 tx=0x7fb4e8009040 comp rx=0 tx=0).stop 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.917+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 msgr2=0x7fb4f815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.917+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f0004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 shutdown_connections 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4d406b3e0 0x7fb4d406d8a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4f815d1a0 0x7fb4f815f0c0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f815cdc0 0x7fb4f815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 --2- 172.21.15.73:0/318230091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4f80d15b0 0x7fb4f8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 >> 172.21.15.73:0/318230091 conn(0x7fb4f809be20 msgr2=0x7fb4f806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 shutdown_connections 2024-04-05T22:30:39.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:39.918+0000 7fb4fce00640 1 -- 172.21.15.73:0/318230091 wait complete. 2024-04-05T22:30:39.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:30:40.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:30:40.101 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:30:40.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.138+0000 7f59b8400640 1 Processor -- start 2024-04-05T22:30:40.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.138+0000 7f59b8400640 1 -- start start 2024-04-05T22:30:40.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 0x7f59b006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b8400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 0x7f59b00ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b8400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b00b1b30 con 0x7f59b00b22a0 2024-04-05T22:30:40.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b8400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00b1e10 con 0x7f59b00cbe00 2024-04-05T22:30:40.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b006cf50 con 0x7f59b006c630 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b5a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 0x7f59b006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b6400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f59b00b22a0 0x7f59b00b2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:46044/0 (socket says 172.21.15.73:46044) 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b6400640 1 -- 172.21.15.73:0/589678363 learned_addr learned my addr 172.21.15.73:0/589678363 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.140+0000 7f59b5000640 1 --2- 172.21.15.73:0/589678363 >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 0x7f59b00ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59b5a00640 1 -- 172.21.15.73:0/589678363 >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 msgr2=0x7f59b00ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59b5a00640 1 --2- 172.21.15.73:0/589678363 >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 0x7f59b00ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59b5a00640 1 -- 172.21.15.73:0/589678363 >> v1:172.21.15.157:6789/0 conn(0x7f59b00b22a0 legacy=0x7f59b00b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59b5a00640 1 -- 172.21.15.73:0/589678363 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00cc300 con 0x7f59b006c630 2024-04-05T22:30:40.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59b5000640 1 --2- 172.21.15.73:0/589678363 >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 0x7f59b00ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:30:40.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.141+0000 7f59afe00640 1 -- 172.21.15.73:0/589678363 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141197923 0 0) 0x7f59b00b1b30 con 0x7f59b00b22a0 2024-04-05T22:30:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.142+0000 7f59b5a00640 1 --2- 172.21.15.73:0/589678363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 0x7f59b006ca10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59a40097a0 tx=0x7f59a402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106402aeea48a60d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.142+0000 7f59afe00640 1 -- 172.21.15.73:0/589678363 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a402d040 con 0x7f59b006c630 2024-04-05T22:30:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59afe00640 1 -- 172.21.15.73:0/589678363 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4031810 con 0x7f59b006c630 2024-04-05T22:30:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59afe00640 1 -- 172.21.15.73:0/589678363 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4031ba0 con 0x7f59b006c630 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 -- 172.21.15.73:0/589678363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 msgr2=0x7f59b006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 --2- 172.21.15.73:0/589678363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 0x7f59b006ca10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59a40097a0 tx=0x7f59a402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 -- 172.21.15.73:0/589678363 shutdown_connections 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 --2- 172.21.15.73:0/589678363 >> v2:172.21.15.157:3300/0 conn(0x7f59b00cbe00 0x7f59b00ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 --2- 172.21.15.73:0/589678363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b006c630 0x7f59b006ca10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 -- 172.21.15.73:0/589678363 >> 172.21.15.73:0/589678363 conn(0x7f59b00b0830 msgr2=0x7f59b00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 -- 172.21.15.73:0/589678363 shutdown_connections 2024-04-05T22:30:40.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.143+0000 7f59b8400640 1 -- 172.21.15.73:0/589678363 wait complete. 2024-04-05T22:30:40.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.145+0000 7f59b8400640 1 Processor -- start 2024-04-05T22:30:40.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.145+0000 7f59b8400640 1 -- start start 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b00b22a0 0x7f59b00ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b00cbe00 0x7f59b013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0140a30 con 0x7f59b006c630 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0140c60 con 0x7f59b00b22a0 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.146+0000 7f59b8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0140f40 con 0x7f59b00cbe00 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39276/0 (socket says 172.21.15.73:39276) 2024-04-05T22:30:40.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 -- 172.21.15.73:0/3528714156 learned_addr learned my addr 172.21.15.73:0/3528714156 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5000640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b00b22a0 0x7f59b00ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b6400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b00cbe00 0x7f59b013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 -- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b00cbe00 msgr2=0x7f59b013e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b00cbe00 0x7f59b013e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 -- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b00b22a0 msgr2=0x7f59b00ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b00b22a0 0x7f59b00ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.147+0000 7f59b5a00640 1 -- 172.21.15.73:0/3528714156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4009400 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59b5a00640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a403ad60 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a40443f0 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b0141220 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40447a0 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.148+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01417e0 con 0x7f59b006c630 2024-04-05T22:30:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.149+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b0068160 con 0x7f59b006c630 2024-04-05T22:30:40.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.150+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f59a404d450 con 0x7f59b006c630 2024-04-05T22:30:40.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.151+0000 7f59ad600640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 0x7f599006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.151+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59a40a9c50 con 0x7f59b006c630 2024-04-05T22:30:40.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.151+0000 7f59b5000640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 0x7f599006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.152+0000 7f59b5000640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 0x7f599006d8a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f59b006ce90 tx=0x7f599c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.159+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f59a4076c00 con 0x7f59b006c630 2024-04-05T22:30:40.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.356+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59b00b0e20 con 0x7f59b006c630 2024-04-05T22:30:40.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.357+0000 7f59ad600640 1 -- 172.21.15.73:0/3528714156 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59a407a220 con 0x7f59b006c630 2024-04-05T22:30:40.357 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:30:40.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.359+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 msgr2=0x7f599006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.359+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 0x7f599006d8a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f59b006ce90 tx=0x7f599c008040 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 msgr2=0x7f59b00710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a40046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 shutdown_connections 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f599006b3e0 0x7f599006d8a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b00cbe00 0x7f59b013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b00b22a0 0x7f59b00ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 --2- 172.21.15.73:0/3528714156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b006c630 0x7f59b00710a0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 >> 172.21.15.73:0/3528714156 conn(0x7f59b00b0830 msgr2=0x7f59b015b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 shutdown_connections 2024-04-05T22:30:40.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.360+0000 7f59b8400640 1 -- 172.21.15.73:0/3528714156 wait complete. 2024-04-05T22:30:40.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:30:40.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.585+0000 7f52f2400640 1 Processor -- start 2024-04-05T22:30:40.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.585+0000 7f52f2400640 1 -- start start 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52f2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52ec15ef60 0x7f52ec06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52f2400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec06f5b0 con 0x7f52ec1587d0 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52f2400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec070410 con 0x7f52ec06cb80 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52f2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec06d500 con 0x7f52ec15ef60 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52eb400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f52ec1587d0 0x7f52ec0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59554/0 (socket says 172.21.15.73:59554) 2024-04-05T22:30:40.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52eb400640 1 -- 172.21.15.73:0/2334594776 learned_addr learned my addr 172.21.15.73:0/2334594776 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:40.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805001847 0 0) 0x7f52ec06f5b0 con 0x7f52ec1587d0 2024-04-05T22:30:40.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d0003520 con 0x7f52ec1587d0 2024-04-05T22:30:40.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.587+0000 7f52ebe00640 1 --2- 172.21.15.73:0/2334594776 >> v2:172.21.15.157:3300/0 conn(0x7f52ec15ef60 0x7f52ec06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903860337 0 0) 0x7f52d0003520 con 0x7f52ec1587d0 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 >> v2:172.21.15.157:3300/0 conn(0x7f52ec15ef60 msgr2=0x7f52ec06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 --2- 172.21.15.73:0/2334594776 >> v2:172.21.15.157:3300/0 conn(0x7f52ec15ef60 0x7f52ec06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 >> v1:172.21.15.157:6789/0 conn(0x7f52ec06cb80 legacy=0x7f52ec15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0ccef0 con 0x7f52ec1587d0 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f52dc001670 con 0x7f52ec1587d0 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 >> v1:172.21.15.73:6789/0 conn(0x7f52ec1587d0 legacy=0x7f52ec0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d00047c0 0x7f52d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d0005180 0x7f52d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec06d500 con 0x7f52d0003970 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d000c0d0 con 0x7f52d0005180 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d000c290 con 0x7f52d00047c0 2024-04-05T22:30:40.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52eb400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.588+0000 7f52e2a00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d00047c0 0x7f52d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52ebe00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d0005180 0x7f52d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52eb400640 1 -- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d00047c0 msgr2=0x7f52d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52eb400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d00047c0 0x7f52d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52eb400640 1 -- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d0005180 msgr2=0x7f52d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52eb400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d0005180 0x7f52d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52ebe00640 1 -- 172.21.15.73:0/2334594776 reap_dead start 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52eb400640 1 -- 172.21.15.73:0/2334594776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0003520 con 0x7f52d0003970 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.589+0000 7f52e2a00640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d00047c0 0x7f52d0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:40.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52eb400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52d0003d90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c240 tx=0x7f52d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae55ff92e6a6d982 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc005ad0 con 0x7f52d0003970 2024-04-05T22:30:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc005c30 con 0x7f52d0003970 2024-04-05T22:30:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52eaa00640 1 -- 172.21.15.73:0/2334594776 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02caf0 con 0x7f52d0003970 2024-04-05T22:30:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52f2400640 1 -- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 msgr2=0x7f52d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.590+0000 7f52f2400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52d0003d90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c240 tx=0x7f52d00057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.591+0000 7f52f2400640 1 -- 172.21.15.73:0/2334594776 shutdown_connections 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.591+0000 7f52f2400640 1 --2- 172.21.15.73:0/2334594776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52d0003d90 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.591+0000 7f52f2400640 1 -- 172.21.15.73:0/2334594776 >> 172.21.15.73:0/2334594776 conn(0x7f52ec0ce8e0 msgr2=0x7f52ec0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.591+0000 7f52f2400640 1 -- 172.21.15.73:0/2334594776 shutdown_connections 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.591+0000 7f52f2400640 1 -- 172.21.15.73:0/2334594776 wait complete. 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.592+0000 7f52f2400640 1 Processor -- start 2024-04-05T22:30:40.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 -- start start 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0a84c0 0x7f52ec0a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52ec0cda20 0x7f52ec0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0ce3e0 con 0x7f52d0003970 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0aaf00 con 0x7f52ec0a84c0 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.593+0000 7f52f2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec148740 con 0x7f52ec0cda20 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39284/0 (socket says 172.21.15.73:39284) 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 -- 172.21.15.73:0/1390507394 learned_addr learned my addr 172.21.15.73:0/1390507394 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52e2a00640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0a84c0 0x7f52ec0a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52ebe00640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52ec0cda20 0x7f52ec0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 -- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52ec0cda20 msgr2=0x7f52ec0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52ec0cda20 0x7f52ec0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 -- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0a84c0 msgr2=0x7f52ec0a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0a84c0 0x7f52ec0a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.594+0000 7f52eb400640 1 -- 172.21.15.73:0/1390507394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0a8f40 con 0x7f52d0003970 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52eb400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f52dc02c450 tx=0x7f52dc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc03c040 con 0x7f52d0003970 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc02ccc0 con 0x7f52d0003970 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc007290 con 0x7f52d0003970 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec0a9220 con 0x7f52d0003970 2024-04-05T22:30:40.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.595+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec0a97e0 con 0x7f52d0003970 2024-04-05T22:30:40.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.596+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec066af0 con 0x7f52d0003970 2024-04-05T22:30:40.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.597+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f52dc032070 con 0x7f52d0003970 2024-04-05T22:30:40.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.598+0000 7f52e3e00640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 0x7f52c806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:40.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.598+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52dc0a8c20 con 0x7f52d0003970 2024-04-05T22:30:40.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.599+0000 7f52e2a00640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 0x7f52c806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:40.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.600+0000 7f52e2a00640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 0x7f52c806d8a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.602+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f52dc075ab0 con 0x7f52d0003970 2024-04-05T22:30:40.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.810+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52ec068160 con 0x7f52d0003970 2024-04-05T22:30:40.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.810+0000 7f52e3e00640 1 -- 172.21.15.73:0/1390507394 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52dc046070 con 0x7f52d0003970 2024-04-05T22:30:40.810 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:30:40.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 msgr2=0x7f52c806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 0x7f52c806d8a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4009040 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 msgr2=0x7f52ec148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f52dc02c450 tx=0x7f52dc02e070 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 shutdown_connections 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f52c806b3e0 0x7f52c806d8a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52ec0cda20 0x7f52ec0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0a84c0 0x7f52ec0a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 --2- 172.21.15.73:0/1390507394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d0003970 0x7f52ec148200 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 >> 172.21.15.73:0/1390507394 conn(0x7f52ec0ce8e0 msgr2=0x7f52ec0ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 shutdown_connections 2024-04-05T22:30:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:40.813+0000 7f52f2400640 1 -- 172.21.15.73:0/1390507394 wait complete. 2024-04-05T22:30:40.834 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:30:40.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:40.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:30:40.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:30:40.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:30:40.946 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:30:40.998 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:40.998 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:41.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:41.056 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:41.056 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:41.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:41.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:41.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.330+0000 7fd6b4200640 1 Processor -- start 2024-04-05T22:30:41.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.330+0000 7fd6b4200640 1 -- start start 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac06cb80 0x7fd6ac06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b4200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd6ac15e850 0x7fd6ac15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b4200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac06f5b0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b4200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac06d500 con 0x7fd6ac15e850 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac15f220 con 0x7fd6ac06cb80 2024-04-05T22:30:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b1800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd6ac1587d0 0x7fd6ac0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59570/0 (socket says 172.21.15.73:59570) 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b1800640 1 -- 172.21.15.73:0/1883702555 learned_addr learned my addr 172.21.15.73:0/1883702555 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b2200640 1 --2- 172.21.15.73:0/1883702555 >> v2:172.21.15.73:3300/0 conn(0x7fd6ac15e850 0x7fd6ac15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6b0e00640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac06cb80 0x7fd6ac06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.332+0000 7fd6abe00640 1 -- 172.21.15.73:0/1883702555 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445218749 0 0) 0x7fd6ac06f5b0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.333+0000 7fd6abe00640 1 -- 172.21.15.73:0/1883702555 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd690003520 con 0x7fd6ac1587d0 2024-04-05T22:30:41.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.333+0000 7fd6b2200640 1 -- 172.21.15.73:0/1883702555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac06cb80 msgr2=0x7fd6ac06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.333+0000 7fd6b2200640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac06cb80 0x7fd6ac06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.333+0000 7fd6b2200640 1 -- 172.21.15.73:0/1883702555 >> v1:172.21.15.73:6789/0 conn(0x7fd6ac1587d0 legacy=0x7fd6ac0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.333+0000 7fd6b2200640 1 -- 172.21.15.73:0/1883702555 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac070410 con 0x7fd6ac15e850 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.334+0000 7fd6b2200640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac15e850 0x7fd6ac15ece0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009dc0 tx=0x7fd6a402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d63c56f2cdf90f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.334+0000 7fd6abe00640 1 -- 172.21.15.73:0/1883702555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a402cc40 con 0x7fd6ac15e850 2024-04-05T22:30:41.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.334+0000 7fd6abe00640 1 -- 172.21.15.73:0/1883702555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a402cda0 con 0x7fd6ac15e850 2024-04-05T22:30:41.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.334+0000 7fd6abe00640 1 -- 172.21.15.73:0/1883702555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4034660 con 0x7fd6ac15e850 2024-04-05T22:30:41.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 -- 172.21.15.73:0/1883702555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac15e850 msgr2=0x7fd6ac15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac15e850 0x7fd6ac15ece0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009dc0 tx=0x7fd6a402be90 comp rx=0 tx=0).stop 2024-04-05T22:30:41.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 -- 172.21.15.73:0/1883702555 shutdown_connections 2024-04-05T22:30:41.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac06cb80 0x7fd6ac06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 --2- 172.21.15.73:0/1883702555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac15e850 0x7fd6ac15ece0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 -- 172.21.15.73:0/1883702555 >> 172.21.15.73:0/1883702555 conn(0x7fd6ac0ce8e0 msgr2=0x7fd6ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 -- 172.21.15.73:0/1883702555 shutdown_connections 2024-04-05T22:30:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.335+0000 7fd6b4200640 1 -- 172.21.15.73:0/1883702555 wait complete. 2024-04-05T22:30:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.337+0000 7fd6b4200640 1 Processor -- start 2024-04-05T22:30:41.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.337+0000 7fd6b4200640 1 -- start start 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6ac06cb80 0x7fd6ac133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac15e850 0x7fd6ac15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac135120 con 0x7fd6ac1587d0 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a84c0 con 0x7fd6ac06cb80 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac134270 con 0x7fd6ac15e850 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39292/0 (socket says 172.21.15.73:39292) 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 -- 172.21.15.73:0/3445045008 learned_addr learned my addr 172.21.15.73:0/3445045008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b2200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac15e850 0x7fd6ac15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b1800640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6ac06cb80 0x7fd6ac133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 -- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac15e850 msgr2=0x7fd6ac15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac15e850 0x7fd6ac15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 -- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6ac06cb80 msgr2=0x7fd6ac133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6ac06cb80 0x7fd6ac133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.338+0000 7fd6b0e00640 1 -- 172.21.15.73:0/3445045008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4009a70 con 0x7fd6ac1587d0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6b0e00640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fd69800d830 tx=0x7fd69800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69800b6c0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69800b820 con 0x7fd6ac1587d0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69800bbb0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac1344a0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.339+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac134a60 con 0x7fd6ac1587d0 2024-04-05T22:30:41.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.340+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac066af0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.341+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fd69801e460 con 0x7fd6ac1587d0 2024-04-05T22:30:41.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.343+0000 7fd6a9600640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 0x7fd68c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.343+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6980881e0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.343+0000 7fd6b1800640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 0x7fd68c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.345+0000 7fd6b1800640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 0x7fd68c06d8a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0011e0 tx=0x7fd69c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.347+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd698054fc0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.576+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6ac068160 con 0x7fd6ac1587d0 2024-04-05T22:30:41.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.578+0000 7fd6a9600640 1 -- 172.21.15.73:0/3445045008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fd6980585e0 con 0x7fd6ac1587d0 2024-04-05T22:30:41.578 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:41.578 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:41.578 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 92 2024-04-05T22:30:41.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 msgr2=0x7fd68c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 0x7fd68c06d8a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0011e0 tx=0x7fd69c009040 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 msgr2=0x7fd6ac156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fd69800d830 tx=0x7fd69800dcf0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 shutdown_connections 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd68c06b3e0 0x7fd68c06d8a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6ac15e850 0x7fd6ac15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6ac1587d0 0x7fd6ac156fa0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 --2- 172.21.15.73:0/3445045008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6ac06cb80 0x7fd6ac133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.580+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 >> 172.21.15.73:0/3445045008 conn(0x7fd6ac0ce8e0 msgr2=0x7fd6ac1423f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.581+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 shutdown_connections 2024-04-05T22:30:41.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.581+0000 7fd6b4200640 1 -- 172.21.15.73:0/3445045008 wait complete. 2024-04-05T22:30:41.599 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:41.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.811+0000 7f5989a00640 1 Processor -- start 2024-04-05T22:30:41.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.812+0000 7f5989a00640 1 -- start start 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.813+0000 7f5989a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598406cb80 0x7f598406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.813+0000 7f5989a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f598415e850 0x7f598415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.813+0000 7f5989a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598406f5b0 con 0x7f59841587d0 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.813+0000 7f5989a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598406d500 con 0x7f598415e850 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.813+0000 7f5989a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598415f220 con 0x7f598406cb80 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f598415e850 0x7f598415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f598415e850 0x7f598415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39294/0 (socket says 172.21.15.73:39294) 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 -- 172.21.15.73:0/3282917659 learned_addr learned my addr 172.21.15.73:0/3282917659 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f597be00640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598406cb80 0x7f598406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 -- 172.21.15.73:0/3282917659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598406cb80 msgr2=0x7f598406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598406cb80 0x7f598406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 -- 172.21.15.73:0/3282917659 >> v1:172.21.15.157:6789/0 conn(0x7f59841587d0 legacy=0x7f59840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.814+0000 7f5983400640 1 -- 172.21.15.73:0/3282917659 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984070410 con 0x7f598415e850 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.815+0000 7f5983400640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415ece0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f597c009e80 tx=0x7f597c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fdcdd487b49674 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.815+0000 7f5982000640 1 -- 172.21.15.73:0/3282917659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c02ca50 con 0x7f598415e850 2024-04-05T22:30:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5982000640 1 -- 172.21.15.73:0/3282917659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c02cbb0 con 0x7f598415e850 2024-04-05T22:30:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5982000640 1 -- 172.21.15.73:0/3282917659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0343f0 con 0x7f598415e850 2024-04-05T22:30:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 -- 172.21.15.73:0/3282917659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 msgr2=0x7f598415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415ece0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f597c009e80 tx=0x7f597c02beb0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 -- 172.21.15.73:0/3282917659 shutdown_connections 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415ece0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 --2- 172.21.15.73:0/3282917659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598406cb80 0x7f598406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 -- 172.21.15.73:0/3282917659 >> 172.21.15.73:0/3282917659 conn(0x7f59840ce8e0 msgr2=0x7f59840cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 -- 172.21.15.73:0/3282917659 shutdown_connections 2024-04-05T22:30:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.816+0000 7f5989a00640 1 -- 172.21.15.73:0/3282917659 wait complete. 2024-04-05T22:30:41.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.818+0000 7f5989a00640 1 Processor -- start 2024-04-05T22:30:41.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.818+0000 7f5989a00640 1 -- start start 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f598406cb80 0x7f59840c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59841587d0 0x7f59840c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598415e0d0 con 0x7f598415e850 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598415ddf0 con 0x7f598406cb80 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5989a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59840c3980 con 0x7f59841587d0 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5983400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5983400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39300/0 (socket says 172.21.15.73:39300) 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.819+0000 7f5983400640 1 -- 172.21.15.73:0/2960894776 learned_addr learned my addr 172.21.15.73:0/2960894776 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f597be00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59841587d0 0x7f59840c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5982a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f598406cb80 0x7f59840c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 -- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59841587d0 msgr2=0x7f59840c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59841587d0 0x7f59840c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 -- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f598406cb80 msgr2=0x7f59840c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f598406cb80 0x7f59840c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 -- 172.21.15.73:0/2960894776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c009b30 con 0x7f598415e850 2024-04-05T22:30:41.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5983400640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f597c036040 tx=0x7f597c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c040670 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c0407f0 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c040ba0 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59840c3c60 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.820+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59840c57b0 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.822+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5984066af0 con 0x7f598415e850 2024-04-05T22:30:41.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.822+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f597c040d00 con 0x7f598415e850 2024-04-05T22:30:41.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.823+0000 7f597aa00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 0x7f596006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:41.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.823+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f597c0aab30 con 0x7f598415e850 2024-04-05T22:30:41.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.824+0000 7f5982a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 0x7f596006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:41.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.825+0000 7f5982a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 0x7f596006d8a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5974003d90 tx=0x7f5974009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:41.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:41.827+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f597c077a30 con 0x7f598415e850 2024-04-05T22:30:42.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.020+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59840c37e0 con 0x7f598415e850 2024-04-05T22:30:42.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.023+0000 7f597aa00640 1 -- 172.21.15.73:0/2960894776 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f59840c37e0 con 0x7f598415e850 2024-04-05T22:30:42.024 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:42.024 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 msgr2=0x7f596006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 0x7f596006d8a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5974003d90 tx=0x7f5974009040 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 msgr2=0x7f598415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f597c036040 tx=0x7f597c004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 shutdown_connections 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f596006b3e0 0x7f596006d8a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598415e850 0x7f598415b060 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59841587d0 0x7f59840c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 --2- 172.21.15.73:0/2960894776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f598406cb80 0x7f59840c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.026+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 >> 172.21.15.73:0/2960894776 conn(0x7f59840ce8e0 msgr2=0x7f5984158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.027+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 shutdown_connections 2024-04-05T22:30:42.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.027+0000 7f5989a00640 1 -- 172.21.15.73:0/2960894776 wait complete. 2024-04-05T22:30:42.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:16.932891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:17.493378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:42.047 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:42.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.252+0000 7f0a95600640 1 Processor -- start 2024-04-05T22:30:42.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.252+0000 7f0a95600640 1 -- start start 2024-04-05T22:30:42.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.254+0000 7f0a95600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a9015aa70 0x7f0a90070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.254+0000 7f0a95600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.254+0000 7f0a95600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a90071900 con 0x7f0a9015dbc0 2024-04-05T22:30:42.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.254+0000 7f0a95600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9015ae50 con 0x7f0a9015aa70 2024-04-05T22:30:42.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.254+0000 7f0a95600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9015e3b0 con 0x7f0a900714a0 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:57212/0 (socket says 172.21.15.73:57212) 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8e000640 1 -- 172.21.15.73:0/122963924 learned_addr learned my addr 172.21.15.73:0/122963924 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8f400640 1 --1- 172.21.15.73:0/122963924 >> v1:172.21.15.157:6789/0 conn(0x7f0a9015dbc0 0x7f0a901609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:46070/0 (socket says 172.21.15.73:46070) 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8ea00640 1 --2- 172.21.15.73:0/122963924 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015aa70 0x7f0a90070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.255+0000 7f0a8d600640 1 -- 172.21.15.73:0/122963924 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826468973 0 0) 0x7f0a90071900 con 0x7f0a9015dbc0 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.256+0000 7f0a8d600640 1 -- 172.21.15.73:0/122963924 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a74003520 con 0x7f0a9015dbc0 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.256+0000 7f0a8e000640 1 -- 172.21.15.73:0/122963924 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015aa70 msgr2=0x7f0a90070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.256+0000 7f0a8e000640 1 --2- 172.21.15.73:0/122963924 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015aa70 0x7f0a90070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.256+0000 7f0a8e000640 1 -- 172.21.15.73:0/122963924 >> v1:172.21.15.157:6789/0 conn(0x7f0a9015dbc0 legacy=0x7f0a901609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.256+0000 7f0a8e000640 1 -- 172.21.15.73:0/122963924 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9006ca50 con 0x7f0a900714a0 2024-04-05T22:30:42.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.257+0000 7f0a8e000640 1 --2- 172.21.15.73:0/122963924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009e50 tx=0x7f0a8002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf7bcbdc5333f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.257+0000 7f0a8d600640 1 -- 172.21.15.73:0/122963924 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002cb00 con 0x7f0a900714a0 2024-04-05T22:30:42.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.257+0000 7f0a8d600640 1 -- 172.21.15.73:0/122963924 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8002cc60 con 0x7f0a900714a0 2024-04-05T22:30:42.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.257+0000 7f0a8d600640 1 -- 172.21.15.73:0/122963924 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800343f0 con 0x7f0a900714a0 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 -- 172.21.15.73:0/122963924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 msgr2=0x7f0a9015d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 --2- 172.21.15.73:0/122963924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009e50 tx=0x7f0a8002bf60 comp rx=0 tx=0).stop 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 -- 172.21.15.73:0/122963924 shutdown_connections 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 --2- 172.21.15.73:0/122963924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a900714a0 0x7f0a9015d680 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 --2- 172.21.15.73:0/122963924 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015aa70 0x7f0a90070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 -- 172.21.15.73:0/122963924 >> 172.21.15.73:0/122963924 conn(0x7f0a90069cb0 msgr2=0x7f0a9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 -- 172.21.15.73:0/122963924 shutdown_connections 2024-04-05T22:30:42.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.258+0000 7f0a95600640 1 -- 172.21.15.73:0/122963924 wait complete. 2024-04-05T22:30:42.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.260+0000 7f0a95600640 1 Processor -- start 2024-04-05T22:30:42.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.260+0000 7f0a95600640 1 -- start start 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015aa70 0x7f0a900cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015dbc0 0x7f0a900ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900cc6d0 con 0x7f0a900714a0 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900cc9b0 con 0x7f0a9015aa70 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a95600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900ccc90 con 0x7f0a9015dbc0 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a8ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a8ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39304/0 (socket says 172.21.15.73:39304) 2024-04-05T22:30:42.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.261+0000 7f0a8ea00640 1 -- 172.21.15.73:0/1314126164 learned_addr learned my addr 172.21.15.73:0/1314126164 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8e000640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015aa70 0x7f0a900cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8f400640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015dbc0 0x7f0a900ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 -- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015dbc0 msgr2=0x7f0a900ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015dbc0 0x7f0a900ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 -- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015aa70 msgr2=0x7f0a900cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015aa70 0x7f0a900cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 -- 172.21.15.73:0/1314126164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009b30 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.262+0000 7f0a8ea00640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e9b0 tx=0x7f0a7800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.263+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78009670 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.263+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7800c9a0 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.263+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800cd50 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.263+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900ccf70 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.263+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a900cc2a0 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.264+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90068160 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.265+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f0a78013470 con 0x7f0a900714a0 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.266+0000 7f0a86a00640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 0x7f0a6c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.266+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a78087c10 con 0x7f0a900714a0 2024-04-05T22:30:42.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.266+0000 7f0a8e000640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 0x7f0a6c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.268+0000 7f0a8e000640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 0x7f0a6c06d8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0a80035000 tx=0x7f0a80030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.270+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0a78054aa0 con 0x7f0a900714a0 2024-04-05T22:30:42.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.467+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a90158290 con 0x7f0a900714a0 2024-04-05T22:30:42.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.468+0000 7f0a86a00640 1 -- 172.21.15.73:0/1314126164 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a780170a0 con 0x7f0a900714a0 2024-04-05T22:30:42.469 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.470+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 msgr2=0x7f0a6c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.470+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 0x7f0a6c06d8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0a80035000 tx=0x7f0a80030000 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.470+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 msgr2=0x7f0a901f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.470+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e9b0 tx=0x7f0a7800ee70 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 shutdown_connections 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0a6c06b3e0 0x7f0a6c06d8a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015dbc0 0x7f0a900ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015aa70 0x7f0a900cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 --2- 172.21.15.73:0/1314126164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a900714a0 0x7f0a901f1d60 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 >> 172.21.15.73:0/1314126164 conn(0x7f0a90069cb0 msgr2=0x7f0a901568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:42.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 shutdown_connections 2024-04-05T22:30:42.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.471+0000 7f0a95600640 1 -- 172.21.15.73:0/1314126164 wait complete. 2024-04-05T22:30:42.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:30:42.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.706+0000 7f5143200640 1 Processor -- start 2024-04-05T22:30:42.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.706+0000 7f5143200640 1 -- start start 2024-04-05T22:30:42.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.707+0000 7f5143200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f513c06c9a0 0x7f513c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.707+0000 7f5143200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c06bea0 con 0x7f513c15abe0 2024-04-05T22:30:42.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5143200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c15afc0 con 0x7f513c06ba70 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5143200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c06f5b0 con 0x7f513c06c9a0 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f513be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f513c15abe0 0x7f513c0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59578/0 (socket says 172.21.15.73:59578) 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f513be00640 1 -- 172.21.15.73:0/2710000747 learned_addr learned my addr 172.21.15.73:0/2710000747 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5133400640 1 --2- 172.21.15.73:0/2710000747 >> v2:172.21.15.73:3300/0 conn(0x7f513c06c9a0 0x7f513c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f513b400640 1 -- 172.21.15.73:0/2710000747 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017326648 0 0) 0x7f513c06bea0 con 0x7f513c15abe0 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f513b400640 1 -- 172.21.15.73:0/2710000747 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5120003520 con 0x7f513c15abe0 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5133400640 1 -- 172.21.15.73:0/2710000747 >> v1:172.21.15.157:6789/0 conn(0x7f513c06ba70 legacy=0x7f513c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5133400640 1 -- 172.21.15.73:0/2710000747 >> v1:172.21.15.73:6789/0 conn(0x7f513c15abe0 legacy=0x7f513c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.708+0000 7f5133400640 1 -- 172.21.15.73:0/2710000747 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c06f890 con 0x7f513c06c9a0 2024-04-05T22:30:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5133400640 1 --2- 172.21.15.73:0/2710000747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c06c9a0 0x7f513c06b530 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f51240097a0 tx=0x7f512402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bf14fd46fa20b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f513b400640 1 -- 172.21.15.73:0/2710000747 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512402f050 con 0x7f513c06c9a0 2024-04-05T22:30:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f513b400640 1 -- 172.21.15.73:0/2710000747 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124031700 con 0x7f513c06c9a0 2024-04-05T22:30:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f513b400640 1 -- 172.21.15.73:0/2710000747 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124031a70 con 0x7f513c06c9a0 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5143200640 1 -- 172.21.15.73:0/2710000747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c06c9a0 msgr2=0x7f513c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5143200640 1 --2- 172.21.15.73:0/2710000747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c06c9a0 0x7f513c06b530 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f51240097a0 tx=0x7f512402b7a0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5143200640 1 -- 172.21.15.73:0/2710000747 shutdown_connections 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5143200640 1 --2- 172.21.15.73:0/2710000747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c06c9a0 0x7f513c06b530 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.710+0000 7f5143200640 1 -- 172.21.15.73:0/2710000747 >> 172.21.15.73:0/2710000747 conn(0x7f513c069cb0 msgr2=0x7f513c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.711+0000 7f5143200640 1 -- 172.21.15.73:0/2710000747 shutdown_connections 2024-04-05T22:30:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.711+0000 7f5143200640 1 -- 172.21.15.73:0/2710000747 wait complete. 2024-04-05T22:30:42.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.712+0000 7f5143200640 1 Processor -- start 2024-04-05T22:30:42.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.712+0000 7f5143200640 1 -- start start 2024-04-05T22:30:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513c06ba70 0x7f513c148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f513c06c9a0 0x7f513c141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b19f0 con 0x7f513c15abe0 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b1e60 con 0x7f513c06c9a0 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.713+0000 7f5143200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b2140 con 0x7f513c06ba70 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39334/0 (socket says 172.21.15.73:39334) 2024-04-05T22:30:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 -- 172.21.15.73:0/2736016973 learned_addr learned my addr 172.21.15.73:0/2736016973 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 -- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513c06ba70 msgr2=0x7f513c148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513c06ba70 0x7f513c148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 -- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f513c06c9a0 msgr2=0x7f513c141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f513c06c9a0 0x7f513c141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 -- 172.21.15.73:0/2736016973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009400 con 0x7f513c15abe0 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5141200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f513400c780 tx=0x7f513400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.714+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134004b60 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.715+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400ce90 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.715+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400a850 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.715+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0b2420 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.715+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c0b29e0 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.716+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c0b2bd0 con 0x7f513c15abe0 2024-04-05T22:30:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.717+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f513400a9b0 con 0x7f513c15abe0 2024-04-05T22:30:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.718+0000 7f5138c00640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 0x7f511806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:42.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.718+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5134087ce0 con 0x7f513c15abe0 2024-04-05T22:30:42.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.718+0000 7f513be00640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 0x7f511806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:42.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.719+0000 7f513be00640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 0x7f511806d8a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f512c000f60 tx=0x7f512c0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:42.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.722+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5134054a30 con 0x7f513c15abe0 2024-04-05T22:30:42.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.914+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f513c0cdc40 con 0x7f513c15abe0 2024-04-05T22:30:42.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.915+0000 7f5138c00640 1 -- 172.21.15.73:0/2736016973 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5134058050 con 0x7f513c15abe0 2024-04-05T22:30:42.915 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.917+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 msgr2=0x7f511806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.917+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 0x7f511806d8a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f512c000f60 tx=0x7f512c0078f0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.917+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 msgr2=0x7f513c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.917+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f513400c780 tx=0x7f513400cc40 comp rx=0 tx=0).stop 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 shutdown_connections 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f511806b3e0 0x7f511806d8a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c15abe0 0x7f513c0b2f00 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f513c06c9a0 0x7f513c141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 --2- 172.21.15.73:0/2736016973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f513c06ba70 0x7f513c148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:42.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 >> 172.21.15.73:0/2736016973 conn(0x7f513c069cb0 msgr2=0x7f513c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:42.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 shutdown_connections 2024-04-05T22:30:42.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:42.918+0000 7f5143200640 1 -- 172.21.15.73:0/2736016973 wait complete. 2024-04-05T22:30:42.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:30:43.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.150+0000 7ff432000640 1 Processor -- start 2024-04-05T22:30:43.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.150+0000 7ff432000640 1 -- start start 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff432000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff42c1657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff432000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c0b1b30 con 0x7ff42c0cbe00 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff432000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c0b1e10 con 0x7ff42c06c630 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff432000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0ce7b0 con 0x7ff42c0b32b0 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff42aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff42c0cbe00 0x7ff42c0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59582/0 (socket says 172.21.15.73:59582) 2024-04-05T22:30:43.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff42aa00640 1 -- 172.21.15.73:0/2316007388 learned_addr learned my addr 172.21.15.73:0/2316007388 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:43.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff42be00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff42c1657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.152+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189468572 0 0) 0x7ff42c0b1b30 con 0x7ff42c0cbe00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40c003520 con 0x7ff42c0cbe00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504068076 0 0) 0x7ff40c003520 con 0x7ff42c0cbe00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 msgr2=0x7ff42c1657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff42c1657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 >> v1:172.21.15.157:6789/0 conn(0x7ff42c06c630 legacy=0x7ff42c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0cc300 con 0x7ff42c0cbe00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff414001670 con 0x7ff42c0cbe00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 >> v1:172.21.15.73:6789/0 conn(0x7ff42c0cbe00 legacy=0x7ff42c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0039a0 0x7ff40c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff40c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40c004d00 0x7ff40c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0ce7b0 con 0x7ff40c004d00 2024-04-05T22:30:43.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c00ba60 con 0x7ff40c0039a0 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c00bc20 con 0x7ff42c0b32b0 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.153+0000 7ff42be00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40c004d00 0x7ff40c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42b400640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0039a0 0x7ff40c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42aa00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff40c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42be00640 1 -- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 msgr2=0x7ff40c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42be00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c0b32b0 0x7ff40c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42be00640 1 -- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0039a0 msgr2=0x7ff40c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42be00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0039a0 0x7ff40c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.154+0000 7ff42be00640 1 -- 172.21.15.73:0/2316007388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c003520 con 0x7ff40c004d00 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.155+0000 7ff42be00640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40c004d00 0x7ff40c005180 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0b3740 tx=0x7ff40c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b706d70f69afba server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.155+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42402cb90 con 0x7ff40c004d00 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.155+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400f040 con 0x7ff40c004d00 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.155+0000 7ff42a000640 1 -- 172.21.15.73:0/2316007388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424034440 con 0x7ff40c004d00 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 -- 172.21.15.73:0/2316007388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40c004d00 msgr2=0x7ff40c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 --2- 172.21.15.73:0/2316007388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40c004d00 0x7ff40c005180 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0b3740 tx=0x7ff40c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff42be00640 1 -- 172.21.15.73:0/2316007388 reap_dead start 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 -- 172.21.15.73:0/2316007388 shutdown_connections 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 -- 172.21.15.73:0/2316007388 >> 172.21.15.73:0/2316007388 conn(0x7ff42c0b0830 msgr2=0x7ff42c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 -- 172.21.15.73:0/2316007388 shutdown_connections 2024-04-05T22:30:43.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.156+0000 7ff432000640 1 -- 172.21.15.73:0/2316007388 wait complete. 2024-04-05T22:30:43.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.158+0000 7ff432000640 1 Processor -- start 2024-04-05T22:30:43.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.158+0000 7ff432000640 1 -- start start 2024-04-05T22:30:43.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c004d00 0x7ff42c0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c13c9b0 0x7ff42c13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0ae550 con 0x7ff42c13d530 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0c5780 con 0x7ff40c004d00 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff432000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c13d370 con 0x7ff42c13c9b0 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39352/0 (socket says 172.21.15.73:39352) 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 -- 172.21.15.73:0/2580337453 learned_addr learned my addr 172.21.15.73:0/2580337453 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42b400640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c004d00 0x7ff42c0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 -- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c13c9b0 msgr2=0x7ff42c13ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c13c9b0 0x7ff42c13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 -- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c004d00 msgr2=0x7ff42c0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c004d00 0x7ff42c0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.159+0000 7ff42aa00640 1 -- 172.21.15.73:0/2580337453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c1433b0 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff42aa00640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e820 tx=0x7ff41400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400eed0 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41400c7f0 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400cb80 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c0adf90 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.160+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c070310 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.161+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c068160 con 0x7ff42c13d530 2024-04-05T22:30:43.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.162+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7ff41400cce0 con 0x7ff42c13d530 2024-04-05T22:30:43.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.163+0000 7ff423400640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 0x7ff40806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.163+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff414087930 con 0x7ff42c13d530 2024-04-05T22:30:43.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.164+0000 7ff42b400640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 0x7ff40806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.165+0000 7ff42b400640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 0x7ff40806d8d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.168+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff414054710 con 0x7ff42c13d530 2024-04-05T22:30:43.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.361+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff42c15d870 con 0x7ff42c13d530 2024-04-05T22:30:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.362+0000 7ff423400640 1 -- 172.21.15.73:0/2580337453 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff414057d30 con 0x7ff42c13d530 2024-04-05T22:30:43.362 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 msgr2=0x7ff40806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 0x7ff40806d8d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c009040 comp rx=0 tx=0).stop 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 msgr2=0x7ff42c0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e820 tx=0x7ff41400ece0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 shutdown_connections 2024-04-05T22:30:43.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff40806b410 0x7ff40806d8d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff42c13c9b0 0x7ff42c13ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff42c13d530 0x7ff42c0b29b0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 --2- 172.21.15.73:0/2580337453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c004d00 0x7ff42c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 >> 172.21.15.73:0/2580337453 conn(0x7ff42c0b0830 msgr2=0x7ff42c15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 shutdown_connections 2024-04-05T22:30:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.365+0000 7ff432000640 1 -- 172.21.15.73:0/2580337453 wait complete. 2024-04-05T22:30:43.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:43.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.590+0000 7f0e66c00640 1 Processor -- start 2024-04-05T22:30:43.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.591+0000 7f0e66c00640 1 -- start start 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e66c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e601587d0 0x7f0e600cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e66c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600ccd20 0x7f0e6006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e66c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6006f5b0 con 0x7f0e6006d0c0 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e66c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e60070410 con 0x7f0e601587d0 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e66c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60158bb0 con 0x7f0e600ccd20 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e5fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e601587d0 0x7f0e600cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e5fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e601587d0 0x7f0e600cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39368/0 (socket says 172.21.15.73:39368) 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.592+0000 7f0e5fe00640 1 -- 172.21.15.73:0/1190108046 learned_addr learned my addr 172.21.15.73:0/1190108046 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.593+0000 7f0e5f400640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600ccd20 0x7f0e6006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.593+0000 7f0e5fe00640 1 -- 172.21.15.73:0/1190108046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600ccd20 msgr2=0x7f0e6006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.593+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600ccd20 0x7f0e6006cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.593+0000 7f0e5fe00640 1 -- 172.21.15.73:0/1190108046 >> v1:172.21.15.157:6789/0 conn(0x7f0e6006d0c0 legacy=0x7f0e6015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:43.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.593+0000 7f0e5fe00640 1 -- 172.21.15.73:0/1190108046 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6006f800 con 0x7f0e601587d0 2024-04-05T22:30:43.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.594+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600cc7e0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2b113e9a4695b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.594+0000 7f0e5ea00640 1 -- 172.21.15.73:0/1190108046 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031640 con 0x7f0e601587d0 2024-04-05T22:30:43.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.594+0000 7f0e5ea00640 1 -- 172.21.15.73:0/1190108046 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500317a0 con 0x7f0e601587d0 2024-04-05T22:30:43.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.594+0000 7f0e5ea00640 1 -- 172.21.15.73:0/1190108046 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031b30 con 0x7f0e601587d0 2024-04-05T22:30:43.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 -- 172.21.15.73:0/1190108046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 msgr2=0x7f0e600cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600cc7e0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 -- 172.21.15.73:0/1190108046 shutdown_connections 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600ccd20 0x7f0e6006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 --2- 172.21.15.73:0/1190108046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600cc7e0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 -- 172.21.15.73:0/1190108046 >> 172.21.15.73:0/1190108046 conn(0x7f0e600ce8e0 msgr2=0x7f0e600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 -- 172.21.15.73:0/1190108046 shutdown_connections 2024-04-05T22:30:43.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.595+0000 7f0e66c00640 1 -- 172.21.15.73:0/1190108046 wait complete. 2024-04-05T22:30:43.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.597+0000 7f0e66c00640 1 Processor -- start 2024-04-05T22:30:43.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.597+0000 7f0e66c00640 1 -- start start 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e6006d0c0 0x7f0e601f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600c4a20 0x7f0e600c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600c3980 con 0x7f0e601587d0 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600c3c60 con 0x7f0e6006d0c0 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e66c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600c3f40 con 0x7f0e600c4a20 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39378/0 (socket says 172.21.15.73:39378) 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.598+0000 7f0e5f400640 1 -- 172.21.15.73:0/924864267 learned_addr learned my addr 172.21.15.73:0/924864267 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e6006d0c0 0x7f0e601f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 -- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600c4a20 msgr2=0x7f0e600c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600c4a20 0x7f0e600c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 -- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e6006d0c0 msgr2=0x7f0e601f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e6006d0c0 0x7f0e601f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 -- 172.21.15.73:0/924864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500094d0 con 0x7f0e601587d0 2024-04-05T22:30:43.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e5f400640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c7f0 tx=0x7f0e4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c004220 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c0043a0 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c007860 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600c4220 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.599+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600ad080 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.600+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e60066af0 con 0x7f0e601587d0 2024-04-05T22:30:43.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.601+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f0e4c007bd0 con 0x7f0e601587d0 2024-04-05T22:30:43.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.603+0000 7f0e57e00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 0x7f0e4006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:43.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.603+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e4c087a50 con 0x7f0e601587d0 2024-04-05T22:30:43.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.603+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 0x7f0e4006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:43.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.605+0000 7f0e5fe00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 0x7f0e4006d8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0e50036000 tx=0x7f0e5003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:43.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.607+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0e4c054830 con 0x7f0e601587d0 2024-04-05T22:30:43.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.838+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e600c4740 con 0x7f0e601587d0 2024-04-05T22:30:43.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.839+0000 7f0e57e00640 1 -- 172.21.15.73:0/924864267 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f0e600c4740 con 0x7f0e601587d0 2024-04-05T22:30:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:43.840 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:43.840 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 92 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 msgr2=0x7f0e4006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 0x7f0e4006d8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0e50036000 tx=0x7f0e5003c040 comp rx=0 tx=0).stop 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 msgr2=0x7f0e600acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c7f0 tx=0x7f0e4c00ccb0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 shutdown_connections 2024-04-05T22:30:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0e4006b3e0 0x7f0e4006d8a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600c4a20 0x7f0e600c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e601587d0 0x7f0e600acb40 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 --2- 172.21.15.73:0/924864267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e6006d0c0 0x7f0e601f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 >> 172.21.15.73:0/924864267 conn(0x7f0e600ce8e0 msgr2=0x7f0e600ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 shutdown_connections 2024-04-05T22:30:43.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:43.842+0000 7f0e66c00640 1 -- 172.21.15.73:0/924864267 wait complete. 2024-04-05T22:30:43.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:44.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.067+0000 7f76a6e00640 1 Processor -- start 2024-04-05T22:30:44.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.067+0000 7f76a6e00640 1 -- start start 2024-04-05T22:30:44.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.068+0000 7f76a6e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76a01541a0 0x7f76a00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.068+0000 7f76a6e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76a006d710 0x7f76a00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.068+0000 7f76a6e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a015aa60 con 0x7f76a015aef0 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f76a6e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00b3b60 con 0x7f76a01541a0 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f76a6e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00701c0 con 0x7f76a006d710 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f76a4e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f76a015aef0 0x7f76a015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59592/0 (socket says 172.21.15.73:59592) 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f76a4e00640 1 -- 172.21.15.73:0/1890912265 learned_addr learned my addr 172.21.15.73:0/1890912265 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769fe00640 1 --2- 172.21.15.73:0/1890912265 >> v2:172.21.15.73:3300/0 conn(0x7f76a01541a0 0x7f76a00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769f400640 1 --2- 172.21.15.73:0/1890912265 >> v2:172.21.15.157:3300/0 conn(0x7f76a006d710 0x7f76a00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769ea00640 1 -- 172.21.15.73:0/1890912265 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367743019 0 0) 0x7f76a015aa60 con 0x7f76a015aef0 2024-04-05T22:30:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769fe00640 1 -- 172.21.15.73:0/1890912265 >> v2:172.21.15.157:3300/0 conn(0x7f76a006d710 msgr2=0x7f76a00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769fe00640 1 --2- 172.21.15.73:0/1890912265 >> v2:172.21.15.157:3300/0 conn(0x7f76a006d710 0x7f76a00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769fe00640 1 -- 172.21.15.73:0/1890912265 >> v1:172.21.15.73:6789/0 conn(0x7f76a015aef0 legacy=0x7f76a015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.069+0000 7f769fe00640 1 -- 172.21.15.73:0/1890912265 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00b3880 con 0x7f76a01541a0 2024-04-05T22:30:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.070+0000 7f769fe00640 1 --2- 172.21.15.73:0/1890912265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a00b3050 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f76a015e980 tx=0x7f769002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f071ed95ce236b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.071+0000 7f769ea00640 1 -- 172.21.15.73:0/1890912265 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690031830 con 0x7f76a01541a0 2024-04-05T22:30:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.071+0000 7f769ea00640 1 -- 172.21.15.73:0/1890912265 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690031990 con 0x7f76a01541a0 2024-04-05T22:30:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.071+0000 7f769ea00640 1 -- 172.21.15.73:0/1890912265 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690031d20 con 0x7f76a01541a0 2024-04-05T22:30:44.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.071+0000 7f76a6e00640 1 -- 172.21.15.73:0/1890912265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 msgr2=0x7f76a00b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.071+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1890912265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a00b3050 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f76a015e980 tx=0x7f769002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 -- 172.21.15.73:0/1890912265 shutdown_connections 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1890912265 >> v2:172.21.15.157:3300/0 conn(0x7f76a006d710 0x7f76a00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1890912265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a00b3050 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 -- 172.21.15.73:0/1890912265 >> 172.21.15.73:0/1890912265 conn(0x7f76a00ce8e0 msgr2=0x7f76a00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 -- 172.21.15.73:0/1890912265 shutdown_connections 2024-04-05T22:30:44.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.072+0000 7f76a6e00640 1 -- 172.21.15.73:0/1890912265 wait complete. 2024-04-05T22:30:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.074+0000 7f76a6e00640 1 Processor -- start 2024-04-05T22:30:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.074+0000 7f76a6e00640 1 -- start start 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a006d710 0x7f76a0144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a015aef0 0x7f76a0158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0144980 con 0x7f76a01541a0 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0144670 con 0x7f76a006d710 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f76a6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0147260 con 0x7f76a015aef0 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f769f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f769f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39400/0 (socket says 172.21.15.73:39400) 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.075+0000 7f769f400640 1 -- 172.21.15.73:0/1512329115 learned_addr learned my addr 172.21.15.73:0/1512329115 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f76a4e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a015aef0 0x7f76a0158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769fe00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a006d710 0x7f76a0144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 -- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a015aef0 msgr2=0x7f76a0158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a015aef0 0x7f76a0158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 -- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a006d710 msgr2=0x7f76a0144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a006d710 0x7f76a0144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 -- 172.21.15.73:0/1512329115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009400 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f769f400640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f768800d800 tx=0x7f768800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800b6c0 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768800b820 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a0147540 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.076+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800bbb0 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.077+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a0144c20 con 0x7f76a01541a0 2024-04-05T22:30:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.078+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a0144de0 con 0x7f76a01541a0 2024-04-05T22:30:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.079+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f768800bd10 con 0x7f76a01541a0 2024-04-05T22:30:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.080+0000 7f7697e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 0x7f767c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.080+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7688087fb0 con 0x7f76a01541a0 2024-04-05T22:30:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.080+0000 7f769fe00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 0x7f767c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.082+0000 7f769fe00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 0x7f767c06d8a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7690036000 tx=0x7f7690008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.085+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7688054d90 con 0x7f76a01541a0 2024-04-05T22:30:44.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.279+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76a0066af0 con 0x7f76a01541a0 2024-04-05T22:30:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.281+0000 7f7697e00640 1 -- 172.21.15.73:0/1512329115 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f76880583b0 con 0x7f76a01541a0 2024-04-05T22:30:44.282 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:44.282 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:44.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 msgr2=0x7f767c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 0x7f767c06d8a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7690036000 tx=0x7f7690008030 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 msgr2=0x7f76a01469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f768800d800 tx=0x7f768800dcc0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 shutdown_connections 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f767c06b3e0 0x7f767c06d8a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a015aef0 0x7f76a0158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76a01541a0 0x7f76a01469d0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 --2- 172.21.15.73:0/1512329115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a006d710 0x7f76a0144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 >> 172.21.15.73:0/1512329115 conn(0x7f76a00ce8e0 msgr2=0x7f76a00cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 shutdown_connections 2024-04-05T22:30:44.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.284+0000 7f76a6e00640 1 -- 172.21.15.73:0/1512329115 wait complete. 2024-04-05T22:30:44.306 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:30:44.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:44.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.514+0000 7f3bd7200640 1 Processor -- start 2024-04-05T22:30:44.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.514+0000 7f3bd7200640 1 -- start start 2024-04-05T22:30:44.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.515+0000 7f3bd7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bd013fa60 0x7f3bd0142490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.515+0000 7f3bd7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd006fe90 0x7f3bd01429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.515+0000 7f3bd7200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0143330 con 0x7f3bd013fe40 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.515+0000 7f3bd7200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd006d3a0 con 0x7f3bd006fe90 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.515+0000 7f3bd7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd0143050 con 0x7f3bd013fa60 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd006fe90 0x7f3bd01429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bd006fe90 0x7f3bd01429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39408/0 (socket says 172.21.15.73:39408) 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 -- 172.21.15.73:0/3910692940 learned_addr learned my addr 172.21.15.73:0/3910692940 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd5e00640 1 --2- 172.21.15.73:0/3910692940 >> v2:172.21.15.157:3300/0 conn(0x7f3bd013fa60 0x7f3bd0142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 -- 172.21.15.73:0/3910692940 >> v2:172.21.15.157:3300/0 conn(0x7f3bd013fa60 msgr2=0x7f3bd0142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 --2- 172.21.15.73:0/3910692940 >> v2:172.21.15.157:3300/0 conn(0x7f3bd013fa60 0x7f3bd0142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 -- 172.21.15.73:0/3910692940 >> v1:172.21.15.157:6789/0 conn(0x7f3bd013fe40 legacy=0x7f3bd006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.516+0000 7f3bd6800640 1 -- 172.21.15.73:0/3910692940 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00ac970 con 0x7f3bd006fe90 2024-04-05T22:30:44.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.518+0000 7f3bd6800640 1 --2- 172.21.15.73:0/3910692940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd01429d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f2f0 tx=0x7f3bcc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ca3f59d2e47bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.518+0000 7f3bd4a00640 1 -- 172.21.15.73:0/3910692940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc02ca80 con 0x7f3bd006fe90 2024-04-05T22:30:44.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.518+0000 7f3bd4a00640 1 -- 172.21.15.73:0/3910692940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc02cbe0 con 0x7f3bd006fe90 2024-04-05T22:30:44.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.518+0000 7f3bd4a00640 1 -- 172.21.15.73:0/3910692940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0343f0 con 0x7f3bd006fe90 2024-04-05T22:30:44.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 -- 172.21.15.73:0/3910692940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 msgr2=0x7f3bd01429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 --2- 172.21.15.73:0/3910692940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd01429d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f2f0 tx=0x7f3bcc02bee0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 -- 172.21.15.73:0/3910692940 shutdown_connections 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 --2- 172.21.15.73:0/3910692940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd01429d0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 --2- 172.21.15.73:0/3910692940 >> v2:172.21.15.157:3300/0 conn(0x7f3bd013fa60 0x7f3bd0142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 -- 172.21.15.73:0/3910692940 >> 172.21.15.73:0/3910692940 conn(0x7f3bd00b06c0 msgr2=0x7f3bd00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 -- 172.21.15.73:0/3910692940 shutdown_connections 2024-04-05T22:30:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.519+0000 7f3bd7200640 1 -- 172.21.15.73:0/3910692940 wait complete. 2024-04-05T22:30:44.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.521+0000 7f3bd7200640 1 Processor -- start 2024-04-05T22:30:44.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.521+0000 7f3bd7200640 1 -- start start 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd013fa60 0x7f3bd01f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd013fe40 0x7f3bd0158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01f6cb0 con 0x7f3bd006fe90 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01f6220 con 0x7f3bd013fa60 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01f6410 con 0x7f3bd013fe40 2024-04-05T22:30:44.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39422/0 (socket says 172.21.15.73:39422) 2024-04-05T22:30:44.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 -- 172.21.15.73:0/1814315717 learned_addr learned my addr 172.21.15.73:0/1814315717 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd6800640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd013fe40 0x7f3bd0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 -- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd013fe40 msgr2=0x7f3bd0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd013fe40 0x7f3bd0158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 -- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd013fa60 msgr2=0x7f3bd01f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd013fa60 0x7f3bd01f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.522+0000 7f3bd5e00640 1 -- 172.21.15.73:0/1814315717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009b30 con 0x7f3bd006fe90 2024-04-05T22:30:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bd5e00640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c7f0 tx=0x7f3bbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004220 con 0x7f3bd006fe90 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc004380 con 0x7f3bd006fe90 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc007840 con 0x7f3bd006fe90 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd01f66f0 con 0x7f3bd006fe90 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.523+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd015d1d0 con 0x7f3bd006fe90 2024-04-05T22:30:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.524+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd015d030 con 0x7f3bd006fe90 2024-04-05T22:30:44.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.525+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f3bbc007bb0 con 0x7f3bd006fe90 2024-04-05T22:30:44.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.526+0000 7f3bca000640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 0x7f3bb006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.526+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bbc087960 con 0x7f3bd006fe90 2024-04-05T22:30:44.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.527+0000 7f3bd5400640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 0x7f3bb006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.528+0000 7f3bd5400640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 0x7f3bb006d8a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00b1600 tx=0x7f3bc40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:44.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.531+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3bd015d030 con 0x7f3bd006fe90 2024-04-05T22:30:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.763+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bd015d030 con 0x7f3bd006fe90 2024-04-05T22:30:44.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.765+0000 7f3bca000640 1 -- 172.21.15.73:0/1814315717 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f3bd015d030 con 0x7f3bd006fe90 2024-04-05T22:30:44.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:44.765 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:44.765 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 92 2024-04-05T22:30:44.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 msgr2=0x7f3bb006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 0x7f3bb006d8a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00b1600 tx=0x7f3bc40079b0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 msgr2=0x7f3bd00cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c7f0 tx=0x7f3bbc00ccb0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 shutdown_connections 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3bb006b3e0 0x7f3bb006d8a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bd013fe40 0x7f3bd0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bd013fa60 0x7f3bd01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 --2- 172.21.15.73:0/1814315717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bd006fe90 0x7f3bd00cce10 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.767+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 >> 172.21.15.73:0/1814315717 conn(0x7f3bd00b06c0 msgr2=0x7f3bd00af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.768+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 shutdown_connections 2024-04-05T22:30:44.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.768+0000 7f3bd7200640 1 -- 172.21.15.73:0/1814315717 wait complete. 2024-04-05T22:30:44.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:44.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.993+0000 7fa4d0e00640 1 Processor -- start 2024-04-05T22:30:44.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.993+0000 7fa4d0e00640 1 -- start start 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4d0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4cc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4d0e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4cc0ad6b0 0x7fa4cc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4d0e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc0ce5d0 con 0x7fa4cc160620 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4d0e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ce010 con 0x7fa4cc0ad6b0 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4d0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ce2f0 con 0x7fa4cc0b1870 2024-04-05T22:30:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4c9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa4cc160620 0x7fa4cc0caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59600/0 (socket says 172.21.15.73:59600) 2024-04-05T22:30:44.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.995+0000 7fa4c9e00640 1 -- 172.21.15.73:0/331495627 learned_addr learned my addr 172.21.15.73:0/331495627 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4ca800640 1 --2- 172.21.15.73:0/331495627 >> v2:172.21.15.157:3300/0 conn(0x7fa4cc0ad6b0 0x7fa4cc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358506045 0 0) 0x7fa4cc0ce5d0 con 0x7fa4cc160620 2024-04-05T22:30:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b0003520 con 0x7fa4cc160620 2024-04-05T22:30:44.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c1400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4cc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789277644 0 0) 0x7fa4b0003520 con 0x7fa4cc160620 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 msgr2=0x7fa4cc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4cc0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 >> v2:172.21.15.157:3300/0 conn(0x7fa4cc0ad6b0 msgr2=0x7fa4cc0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 --2- 172.21.15.73:0/331495627 >> v2:172.21.15.157:3300/0 conn(0x7fa4cc0ad6b0 0x7fa4cc0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0ac970 con 0x7fa4cc160620 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa4bc001670 con 0x7fa4cc160620 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 >> v1:172.21.15.73:6789/0 conn(0x7fa4cc160620 legacy=0x7fa4cc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4b0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b0003fa0 0x7fa4b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0004d00 0x7fa4b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ce010 con 0x7fa4b0003fa0 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ce2f0 con 0x7fa4b0004d00 2024-04-05T22:30:44.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000ba80 con 0x7fa4cc0b1870 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.996+0000 7fa4c1400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b0003fa0 0x7fa4b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c9e00640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4b0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4ca800640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0004d00 0x7fa4b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c1400640 1 -- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 msgr2=0x7fa4b0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c1400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0b1870 0x7fa4b0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c1400640 1 -- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0004d00 msgr2=0x7fa4b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c1400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0004d00 0x7fa4b0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:44.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.997+0000 7fa4c1400640 1 -- 172.21.15.73:0/331495627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0003520 con 0x7fa4b0003fa0 2024-04-05T22:30:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.998+0000 7fa4c1400640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b0003fa0 0x7fa4b00047c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400ba30 tx=0x7fa4b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f035f3d90c8032 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.998+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032640 con 0x7fa4b0003fa0 2024-04-05T22:30:45.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.998+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b40327a0 con 0x7fa4b0003fa0 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.998+0000 7fa4c9400640 1 -- 172.21.15.73:0/331495627 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032b30 con 0x7fa4b0003fa0 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 -- 172.21.15.73:0/331495627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b0003fa0 msgr2=0x7fa4b00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/331495627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b0003fa0 0x7fa4b00047c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400ba30 tx=0x7fa4b00057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4ca800640 1 -- 172.21.15.73:0/331495627 reap_dead start 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 -- 172.21.15.73:0/331495627 shutdown_connections 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 -- 172.21.15.73:0/331495627 >> 172.21.15.73:0/331495627 conn(0x7fa4cc0ce8e0 msgr2=0x7fa4cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 -- 172.21.15.73:0/331495627 shutdown_connections 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:44.999+0000 7fa4d0e00640 1 -- 172.21.15.73:0/331495627 wait complete. 2024-04-05T22:30:45.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.001+0000 7fa4d0e00640 1 Processor -- start 2024-04-05T22:30:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.001+0000 7fa4d0e00640 1 -- start start 2024-04-05T22:30:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0003fa0 0x7fa4cc1f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0cb9e0 0x7fa4cc1f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1f2720 con 0x7fa4cc0cb560 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc14b100 con 0x7fa4b0003fa0 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4d0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1f64e0 con 0x7fa4cc0cb9e0 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4c1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4c1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39428/0 (socket says 172.21.15.73:39428) 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4c1400640 1 -- 172.21.15.73:0/478266881 learned_addr learned my addr 172.21.15.73:0/478266881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4c9e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0003fa0 0x7fa4cc1f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.002+0000 7fa4ca800640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0cb9e0 0x7fa4cc1f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 -- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0cb9e0 msgr2=0x7fa4cc1f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0cb9e0 0x7fa4cc1f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 -- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0003fa0 msgr2=0x7fa4cc1f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0003fa0 0x7fa4cc1f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 -- 172.21.15.73:0/478266881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc147950 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c1400640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bb50 tx=0x7fa4b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4004220 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4004380 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc147c30 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b403caa0 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.003+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc1f6170 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.004+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc066af0 con 0x7fa4cc0cb560 2024-04-05T22:30:45.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.005+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fa4b403cc00 con 0x7fa4cc0cb560 2024-04-05T22:30:45.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.007+0000 7fa4c2a00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 0x7fa4a806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.007+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4b40aa320 con 0x7fa4cc0cb560 2024-04-05T22:30:45.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.007+0000 7fa4c9e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 0x7fa4a806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.008+0000 7fa4c9e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 0x7fa4a806d8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc001050 tx=0x7fa4bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.011+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa4b4077220 con 0x7fa4cc0cb560 2024-04-05T22:30:45.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.249+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4cc066af0 con 0x7fa4cc0cb560 2024-04-05T22:30:45.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.250+0000 7fa4c2a00640 1 -- 172.21.15.73:0/478266881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fa4b4006c30 con 0x7fa4cc0cb560 2024-04-05T22:30:45.250 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:45.250 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:18.482729+0000","modified":"2024-04-05T22:30:21.949075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9270":{"gid":9270,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6835/2701059182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":2701059182},{"type":"v1","addr":"172.21.15.73:6835","nonce":2701059182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-05T22:30:45.251 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 92 2024-04-05T22:30:45.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 msgr2=0x7fa4a806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 0x7fa4a806d8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc001050 tx=0x7fa4bc009040 comp rx=0 tx=0).stop 2024-04-05T22:30:45.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 msgr2=0x7fa4cc1f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bb50 tx=0x7fa4b4004730 comp rx=0 tx=0).stop 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 shutdown_connections 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa4a806b410 0x7fa4a806d8d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4cc0cb9e0 0x7fa4cc1f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4cc0cb560 0x7fa4cc1f1ca0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 --2- 172.21.15.73:0/478266881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b0003fa0 0x7fa4cc1f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 >> 172.21.15.73:0/478266881 conn(0x7fa4cc0ce8e0 msgr2=0x7fa4cc072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.253+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 shutdown_connections 2024-04-05T22:30:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.254+0000 7fa4d0e00640 1 -- 172.21.15.73:0/478266881 wait complete. 2024-04-05T22:30:45.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:45.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.474+0000 7f2e2e200640 1 Processor -- start 2024-04-05T22:30:45.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.474+0000 7f2e2e200640 1 -- start start 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e2e200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e281541a0 0x7f2e280b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e2e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e2806d710 0x7f2e280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e2e200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2815aa60 con 0x7f2e2815aef0 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e2e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e280b3b60 con 0x7f2e2806d710 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e2e200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e280701c0 con 0x7f2e281541a0 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e26a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e2806d710 0x7f2e280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e27e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2e2815aef0 0x7f2e2815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59608/0 (socket says 172.21.15.73:59608) 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e27e00640 1 -- 172.21.15.73:0/1883509701 learned_addr learned my addr 172.21.15.73:0/1883509701 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.476+0000 7f2e27400640 1 --2- 172.21.15.73:0/1883509701 >> v2:172.21.15.157:3300/0 conn(0x7f2e281541a0 0x7f2e280b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.477+0000 7f2e26a00640 1 -- 172.21.15.73:0/1883509701 >> v2:172.21.15.157:3300/0 conn(0x7f2e281541a0 msgr2=0x7f2e280b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.477+0000 7f2e26a00640 1 --2- 172.21.15.73:0/1883509701 >> v2:172.21.15.157:3300/0 conn(0x7f2e281541a0 0x7f2e280b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.477+0000 7f2e26a00640 1 -- 172.21.15.73:0/1883509701 >> v1:172.21.15.73:6789/0 conn(0x7f2e2815aef0 legacy=0x7f2e2815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.477+0000 7f2e26a00640 1 -- 172.21.15.73:0/1883509701 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e280b3880 con 0x7f2e2806d710 2024-04-05T22:30:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.477+0000 7f2e26000640 1 -- 172.21.15.73:0/1883509701 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365230142 0 0) 0x7f2e2815aa60 con 0x7f2e2815aef0 2024-04-05T22:30:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.478+0000 7f2e26a00640 1 --2- 172.21.15.73:0/1883509701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e280cd8f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f2e28066850 tx=0x7f2e1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d60f8d01d80f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.478+0000 7f2e26000640 1 -- 172.21.15.73:0/1883509701 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14031830 con 0x7f2e2806d710 2024-04-05T22:30:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.478+0000 7f2e26000640 1 -- 172.21.15.73:0/1883509701 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e14031990 con 0x7f2e2806d710 2024-04-05T22:30:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.478+0000 7f2e26000640 1 -- 172.21.15.73:0/1883509701 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14031d20 con 0x7f2e2806d710 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 -- 172.21.15.73:0/1883509701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 msgr2=0x7f2e280cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 --2- 172.21.15.73:0/1883509701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e280cd8f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f2e28066850 tx=0x7f2e1402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 -- 172.21.15.73:0/1883509701 shutdown_connections 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 --2- 172.21.15.73:0/1883509701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e280cd8f0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 --2- 172.21.15.73:0/1883509701 >> v2:172.21.15.157:3300/0 conn(0x7f2e281541a0 0x7f2e280b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 -- 172.21.15.73:0/1883509701 >> 172.21.15.73:0/1883509701 conn(0x7f2e280ce8e0 msgr2=0x7f2e280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 -- 172.21.15.73:0/1883509701 shutdown_connections 2024-04-05T22:30:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.479+0000 7f2e2e200640 1 -- 172.21.15.73:0/1883509701 wait complete. 2024-04-05T22:30:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.481+0000 7f2e2e200640 1 Processor -- start 2024-04-05T22:30:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.481+0000 7f2e2e200640 1 -- start start 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e281541a0 0x7f2e2814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e2815aef0 0x7f2e2814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2814cb20 con 0x7f2e2806d710 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280ab020 con 0x7f2e281541a0 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e2e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28151b70 con 0x7f2e2815aef0 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39446/0 (socket says 172.21.15.73:39446) 2024-04-05T22:30:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 -- 172.21.15.73:0/461193023 learned_addr learned my addr 172.21.15.73:0/461193023 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 -- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e2815aef0 msgr2=0x7f2e2814f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e2815aef0 0x7f2e2814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 -- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e281541a0 msgr2=0x7f2e2814efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e281541a0 0x7f2e2814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.482+0000 7f2e27400640 1 -- 172.21.15.73:0/461193023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14009400 con 0x7f2e2806d710 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e27400640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800d800 tx=0x7f2e1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800b6c0 con 0x7f2e2806d710 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800b820 con 0x7f2e2806d710 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e28151e50 con 0x7f2e2806d710 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800bb90 con 0x7f2e2806d710 2024-04-05T22:30:45.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.483+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e28152410 con 0x7f2e2806d710 2024-04-05T22:30:45.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.484+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28066af0 con 0x7f2e2806d710 2024-04-05T22:30:45.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.485+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f2e1800bcf0 con 0x7f2e2806d710 2024-04-05T22:30:45.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.486+0000 7f2e1f400640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 0x7f2e0806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.487+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e18087fe0 con 0x7f2e2806d710 2024-04-05T22:30:45.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.487+0000 7f2e26a00640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 0x7f2e0806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.488+0000 7f2e26a00640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 0x7f2e0806d8a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e14036000 tx=0x7f2e14030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.491+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2e18054dc0 con 0x7f2e2806d710 2024-04-05T22:30:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.689+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e28068160 con 0x7f2e2806d710 2024-04-05T22:30:45.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.692+0000 7f2e1f400640 1 -- 172.21.15.73:0/461193023 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f2e180583e0 con 0x7f2e2806d710 2024-04-05T22:30:45.692 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:45.692 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":148,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:19.486506+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:16.932891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:17.493378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 msgr2=0x7f2e0806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 0x7f2e0806d8a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e14036000 tx=0x7f2e14030000 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 msgr2=0x7f2e2814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800d800 tx=0x7f2e1800dcc0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 shutdown_connections 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2e0806b3e0 0x7f2e0806d8a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e2815aef0 0x7f2e2814f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e281541a0 0x7f2e2814efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 --2- 172.21.15.73:0/461193023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e2806d710 0x7f2e2814c5e0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.694+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 >> 172.21.15.73:0/461193023 conn(0x7f2e280ce8e0 msgr2=0x7f2e280b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.695+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 shutdown_connections 2024-04-05T22:30:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.695+0000 7f2e2e200640 1 -- 172.21.15.73:0/461193023 wait complete. 2024-04-05T22:30:45.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:30:45.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.924+0000 7f464be00640 1 Processor -- start 2024-04-05T22:30:45.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.924+0000 7f464be00640 1 -- start start 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f464c15aaa0 0x7f464c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f464c06d440 0x7f464c157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0b3c40 con 0x7f464c06d060 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c0b3ea0 con 0x7f464c15aaa0 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c15e2c0 con 0x7f464c06d440 2024-04-05T22:30:45.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f464c15aaa0 0x7f464c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f464c15aaa0 0x7f464c156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48744/0 (socket says 172.21.15.73:48744) 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464a000640 1 -- 172.21.15.73:0/1280801772 learned_addr learned my addr 172.21.15.73:0/1280801772 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.926+0000 7f464b400640 1 --2- 172.21.15.73:0/1280801772 >> v2:172.21.15.157:3300/0 conn(0x7f464c06d440 0x7f464c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.927+0000 7f464a000640 1 -- 172.21.15.73:0/1280801772 >> v2:172.21.15.157:3300/0 conn(0x7f464c06d440 msgr2=0x7f464c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.927+0000 7f464a000640 1 --2- 172.21.15.73:0/1280801772 >> v2:172.21.15.157:3300/0 conn(0x7f464c06d440 0x7f464c157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.927+0000 7f464a000640 1 -- 172.21.15.73:0/1280801772 >> v1:172.21.15.157:6789/0 conn(0x7f464c06d060 legacy=0x7f464c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:45.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.927+0000 7f464a000640 1 -- 172.21.15.73:0/1280801772 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c15d500 con 0x7f464c15aaa0 2024-04-05T22:30:45.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.928+0000 7f464a000640 1 --2- 172.21.15.73:0/1280801772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c15aaa0 0x7f464c156e50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f464c071590 tx=0x7f463802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf927a33f4e35470 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.928+0000 7f4649600640 1 -- 172.21.15.73:0/1280801772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638031640 con 0x7f464c15aaa0 2024-04-05T22:30:45.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.928+0000 7f4649600640 1 -- 172.21.15.73:0/1280801772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380317a0 con 0x7f464c15aaa0 2024-04-05T22:30:45.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.928+0000 7f4649600640 1 -- 172.21.15.73:0/1280801772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638031b30 con 0x7f464c15aaa0 2024-04-05T22:30:45.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 -- 172.21.15.73:0/1280801772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c15aaa0 msgr2=0x7f464c156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 --2- 172.21.15.73:0/1280801772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c15aaa0 0x7f464c156e50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f464c071590 tx=0x7f463802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 -- 172.21.15.73:0/1280801772 shutdown_connections 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 --2- 172.21.15.73:0/1280801772 >> v2:172.21.15.157:3300/0 conn(0x7f464c06d440 0x7f464c157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 --2- 172.21.15.73:0/1280801772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c15aaa0 0x7f464c156e50 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 -- 172.21.15.73:0/1280801772 >> 172.21.15.73:0/1280801772 conn(0x7f464c0b0830 msgr2=0x7f464c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 -- 172.21.15.73:0/1280801772 shutdown_connections 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.929+0000 7f464be00640 1 -- 172.21.15.73:0/1280801772 wait complete. 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.931+0000 7f464be00640 1 Processor -- start 2024-04-05T22:30:45.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.931+0000 7f464be00640 1 -- start start 2024-04-05T22:30:45.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c06d440 0x7f464c165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f464c15aaa0 0x7f464c156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0c3df0 con 0x7f464c06d060 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0c40d0 con 0x7f464c06d440 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0c43b0 con 0x7f464c15aaa0 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48760/0 (socket says 172.21.15.73:48760) 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 -- 172.21.15.73:0/2814904890 learned_addr learned my addr 172.21.15.73:0/2814904890 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464a000640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c06d440 0x7f464c165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 -- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f464c15aaa0 msgr2=0x7f464c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f464c15aaa0 0x7f464c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 -- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c06d440 msgr2=0x7f464c165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c06d440 0x7f464c165c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:45.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.932+0000 7f464aa00640 1 -- 172.21.15.73:0/2814904890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380094d0 con 0x7f464c06d060 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f464aa00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c430 tx=0x7f463c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c011790 con 0x7f464c06d060 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c0118f0 con 0x7f464c06d060 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c0c4690 con 0x7f464c06d060 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c011ce0 con 0x7f464c06d060 2024-04-05T22:30:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.933+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c0ca830 con 0x7f464c06d060 2024-04-05T22:30:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.934+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c0c4bb0 con 0x7f464c06d060 2024-04-05T22:30:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.935+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f463c01c460 con 0x7f464c06d060 2024-04-05T22:30:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.936+0000 7f4642a00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 0x7f462c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.936+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f463c0879e0 con 0x7f464c06d060 2024-04-05T22:30:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.937+0000 7f464a000640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 0x7f462c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:45.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.938+0000 7f464a000640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 0x7f462c06d8a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4638036000 tx=0x7f463803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:45.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:45.940+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f464c0c4bb0 con 0x7f464c06d060 2024-04-05T22:30:46.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.170+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f464c0c4bb0 con 0x7f464c06d060 2024-04-05T22:30:46.172 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:30:46.172+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:30:46.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.186+0000 7f4642a00640 1 -- 172.21.15.73:0/2814904890 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 161+0+0 (secure 0 0 0) 0x7f464c0c4bb0 con 0x7f464c06d060 2024-04-05T22:30:46.186 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:30:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 msgr2=0x7f462c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 0x7f462c06d8a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4638036000 tx=0x7f463803c040 comp rx=0 tx=0).stop 2024-04-05T22:30:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 msgr2=0x7f464c0c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:46.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c430 tx=0x7f463c00c8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 shutdown_connections 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f462c06b3e0 0x7f462c06d8a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f464c15aaa0 0x7f464c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c06d440 0x7f464c165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 --2- 172.21.15.73:0/2814904890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f464c06d060 0x7f464c0c4e90 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 >> 172.21.15.73:0/2814904890 conn(0x7f464c0b0830 msgr2=0x7f464c162ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 shutdown_connections 2024-04-05T22:30:46.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.189+0000 7f464be00640 1 -- 172.21.15.73:0/2814904890 wait complete. 2024-04-05T22:30:46.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:30:46.312 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:30:46.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.406+0000 7f2a92800640 1 Processor -- start 2024-04-05T22:30:46.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.406+0000 7f2a92800640 1 -- start start 2024-04-05T22:30:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a92800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a8c158a30 0x7f2a8c0b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a92800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a92800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0b35b0 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a92800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0b1b10 con 0x7f2a8c158a30 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a92800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cf310 con 0x7f2a8c0ce940 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.407+0000 7f2a91400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a8c158a30 0x7f2a8c0b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a8c158a30 0x7f2a8c0b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48774/0 (socket says 172.21.15.73:48774) 2024-04-05T22:30:46.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 -- 172.21.15.73:0/856287371 learned_addr learned my addr 172.21.15.73:0/856287371 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91e00640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 -- 172.21.15.73:0/856287371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 msgr2=0x7f2a8c0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c0cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 -- 172.21.15.73:0/856287371 >> v1:172.21.15.157:6789/0 conn(0x7f2a8c0b30a0 legacy=0x7f2a8c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.408+0000 7f2a91400640 1 -- 172.21.15.73:0/856287371 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0b3890 con 0x7f2a8c158a30 2024-04-05T22:30:46.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.409+0000 7f2a91400640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c158a30 0x7f2a8c0b2b60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a8002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1ac8173e245a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:46.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.409+0000 7f2a8be00640 1 -- 172.21.15.73:0/856287371 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80031640 con 0x7f2a8c158a30 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.409+0000 7f2a8be00640 1 -- 172.21.15.73:0/856287371 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a800317a0 con 0x7f2a8c158a30 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.409+0000 7f2a8be00640 1 -- 172.21.15.73:0/856287371 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80031b30 con 0x7f2a8c158a30 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 -- 172.21.15.73:0/856287371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c158a30 msgr2=0x7f2a8c0b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c158a30 0x7f2a8c0b2b60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a8002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 -- 172.21.15.73:0/856287371 shutdown_connections 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 --2- 172.21.15.73:0/856287371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c158a30 0x7f2a8c0b2b60 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 -- 172.21.15.73:0/856287371 >> 172.21.15.73:0/856287371 conn(0x7f2a8c069e90 msgr2=0x7f2a8c06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 -- 172.21.15.73:0/856287371 shutdown_connections 2024-04-05T22:30:46.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.410+0000 7f2a92800640 1 -- 172.21.15.73:0/856287371 wait complete. 2024-04-05T22:30:46.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.412+0000 7f2a92800640 1 Processor -- start 2024-04-05T22:30:46.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.412+0000 7f2a92800640 1 -- start start 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a8c158a30 0x7f2a8c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c14f7b0 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c06b4f0 con 0x7f2a8c158a30 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a92800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c06b7d0 con 0x7f2a8c0ce940 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a91400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:46.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a91400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48780/0 (socket says 172.21.15.73:48780) 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.413+0000 7f2a91400640 1 -- 172.21.15.73:0/2175539996 learned_addr learned my addr 172.21.15.73:0/2175539996 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a90a00640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 -- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 msgr2=0x7f2a8c148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 -- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a8c158a30 msgr2=0x7f2a8c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a8c158a30 0x7f2a8c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 -- 172.21.15.73:0/2175539996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800094d0 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a90a00640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:46.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a91400640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:46.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8003ac60 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8003ade0 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80044630 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c0b27c0 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.414+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c06bd90 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.415+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c04e160 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.416+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f2a8002f050 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.417+0000 7f2a89600640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 0x7f2a6c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:46.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.417+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2a800a9b40 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.418+0000 7f2a90a00640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 0x7f2a6c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:46.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.419+0000 7f2a90a00640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 0x7f2a6c06d8a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c153240 tx=0x7f2a78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:46.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.421+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2a80076920 con 0x7f2a8c0b30a0 2024-04-05T22:30:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:46.655+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2a8c15e450 con 0x7f2a8c0b30a0 2024-04-05T22:30:47.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.314+0000 7f2a89600640 1 -- 172.21.15.73:0/2175539996 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a8004e020 con 0x7f2a8c0b30a0 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 msgr2=0x7f2a6c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 0x7f2a6c06d8a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c153240 tx=0x7f2a78009040 comp rx=0 tx=0).stop 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 msgr2=0x7f2a8c14f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a800046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 shutdown_connections 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2a6c06b3e0 0x7f2a6c06d8a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a8c158a30 0x7f2a8c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8c0ce940 0x7f2a8c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 --2- 172.21.15.73:0/2175539996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a8c0b30a0 0x7f2a8c14f270 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 >> 172.21.15.73:0/2175539996 conn(0x7f2a8c069e90 msgr2=0x7f2a8c15c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:47.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 shutdown_connections 2024-04-05T22:30:47.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.318+0000 7f2a92800640 1 -- 172.21.15.73:0/2175539996 wait complete. 2024-04-05T22:30:47.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:30:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.555+0000 7f952a600640 1 Processor -- start 2024-04-05T22:30:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.555+0000 7f952a600640 1 -- start start 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f952a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95240b3ab0 0x7f9524162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f952a600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95241620e0 con 0x7f95240cecd0 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f952a600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95240ae370 con 0x7f95240ad720 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f952a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95240ae090 con 0x7f95240b3ab0 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f9522a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95240b3ab0 0x7f9524162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f9523400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f95240cecd0 0x7f95240cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60070/0 (socket says 172.21.15.73:60070) 2024-04-05T22:30:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.557+0000 7f9523400640 1 -- 172.21.15.73:0/889371102 learned_addr learned my addr 172.21.15.73:0/889371102 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.558+0000 7f9522a00640 1 -- 172.21.15.73:0/889371102 >> v1:172.21.15.157:6789/0 conn(0x7f95240ad720 legacy=0x7f9524164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.558+0000 7f9522a00640 1 -- 172.21.15.73:0/889371102 >> v1:172.21.15.73:6789/0 conn(0x7f95240cecd0 legacy=0x7f95240cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.558+0000 7f9522a00640 1 -- 172.21.15.73:0/889371102 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524161a90 con 0x7f95240b3ab0 2024-04-05T22:30:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.558+0000 7f9522000640 1 -- 172.21.15.73:0/889371102 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887424814 0 0) 0x7f95241620e0 con 0x7f95240cecd0 2024-04-05T22:30:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.559+0000 7f9522a00640 1 --2- 172.21.15.73:0/889371102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240b3ab0 0x7f9524162570 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f95100097a0 tx=0x7f951002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d52ca72f7881008 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.559+0000 7f9522000640 1 -- 172.21.15.73:0/889371102 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510031830 con 0x7f95240b3ab0 2024-04-05T22:30:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.559+0000 7f9522000640 1 -- 172.21.15.73:0/889371102 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510031990 con 0x7f95240b3ab0 2024-04-05T22:30:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.559+0000 7f9522000640 1 -- 172.21.15.73:0/889371102 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510031d20 con 0x7f95240b3ab0 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 -- 172.21.15.73:0/889371102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240b3ab0 msgr2=0x7f9524162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 --2- 172.21.15.73:0/889371102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240b3ab0 0x7f9524162570 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f95100097a0 tx=0x7f951002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 -- 172.21.15.73:0/889371102 shutdown_connections 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 --2- 172.21.15.73:0/889371102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240b3ab0 0x7f9524162570 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 -- 172.21.15.73:0/889371102 >> 172.21.15.73:0/889371102 conn(0x7f952406a0d0 msgr2=0x7f952406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 -- 172.21.15.73:0/889371102 shutdown_connections 2024-04-05T22:30:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.560+0000 7f952a600640 1 -- 172.21.15.73:0/889371102 wait complete. 2024-04-05T22:30:47.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.562+0000 7f952a600640 1 Processor -- start 2024-04-05T22:30:47.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.562+0000 7f952a600640 1 -- start start 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 0x7f952415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 0x7f9524159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95241577e0 con 0x7f95240cecd0 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524153de0 con 0x7f95240b3ab0 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.563+0000 7f952a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952415c6c0 con 0x7f95240ad720 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48810/0 (socket says 172.21.15.73:48810) 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 -- 172.21.15.73:0/3363928158 learned_addr learned my addr 172.21.15.73:0/3363928158 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523400640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 0x7f952415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9522a00640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 0x7f9524159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 -- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 msgr2=0x7f952415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 0x7f952415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 -- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 msgr2=0x7f9524159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 0x7f9524159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 -- 172.21.15.73:0/3363928158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510009400 con 0x7f95240cecd0 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523400640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 0x7f952415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9522a00640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 0x7f9524159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.564+0000 7f9523e00640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c7b0 tx=0x7f951c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.565+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c004b60 con 0x7f95240cecd0 2024-04-05T22:30:47.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.565+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c004cc0 con 0x7f95240cecd0 2024-04-05T22:30:47.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.565+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00aa90 con 0x7f95240cecd0 2024-04-05T22:30:47.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.565+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952415c9a0 con 0x7f95240cecd0 2024-04-05T22:30:47.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.565+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952415c3b0 con 0x7f95240cecd0 2024-04-05T22:30:47.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.566+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9524157920 con 0x7f95240cecd0 2024-04-05T22:30:47.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.567+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f951c00abf0 con 0x7f95240cecd0 2024-04-05T22:30:47.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.568+0000 7f951b400640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 0x7f950406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:47.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.568+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f951c087c00 con 0x7f95240cecd0 2024-04-05T22:30:47.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.568+0000 7f9523400640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 0x7f950406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:47.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.570+0000 7f9523400640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 0x7f950406d8a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9514000f60 tx=0x7f9514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:47.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.572+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f951c0549e0 con 0x7f95240cecd0 2024-04-05T22:30:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:47.767+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f952406cb70 con 0x7f95240cecd0 2024-04-05T22:30:48.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.338+0000 7f951b400640 1 -- 172.21.15.73:0/3363928158 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 181+0+0 (secure 0 0 0) 0x7f951c058000 con 0x7f95240cecd0 2024-04-05T22:30:48.339 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:30:48.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.343+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 msgr2=0x7f950406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.343+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 0x7f950406d8a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9514000f60 tx=0x7f9514009040 comp rx=0 tx=0).stop 2024-04-05T22:30:48.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.343+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 msgr2=0x7f9524148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.343+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c7b0 tx=0x7f951c00cc70 comp rx=0 tx=0).stop 2024-04-05T22:30:48.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 shutdown_connections 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f950406b3e0 0x7f950406d8a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95240cecd0 0x7f9524148450 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b3ab0 0x7f9524159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 --2- 172.21.15.73:0/3363928158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95240ad720 0x7f952415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 >> 172.21.15.73:0/3363928158 conn(0x7f952406a0d0 msgr2=0x7f95240afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.346+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 shutdown_connections 2024-04-05T22:30:48.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.347+0000 7f952a600640 1 -- 172.21.15.73:0/3363928158 wait complete. 2024-04-05T22:30:48.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:30:48.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.568+0000 7f9540400640 1 Processor -- start 2024-04-05T22:30:48.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.568+0000 7f9540400640 1 -- start start 2024-04-05T22:30:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.569+0000 7f9540400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.569+0000 7f9540400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95380cce90 0x7f95380b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.569+0000 7f9540400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95380cd2e0 con 0x7f953815ae50 2024-04-05T22:30:48.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.569+0000 7f9540400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95380ce2f0 con 0x7f95380cce90 2024-04-05T22:30:48.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.569+0000 7f9540400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953815b230 con 0x7f95380cf3e0 2024-04-05T22:30:48.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.570+0000 7f953d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.570+0000 7f953d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:53290/0 (socket says 172.21.15.73:53290) 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.570+0000 7f953d000640 1 -- 172.21.15.73:0/3562582318 learned_addr learned my addr 172.21.15.73:0/3562582318 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.570+0000 7f953e400640 1 --2- 172.21.15.73:0/3562582318 >> v2:172.21.15.157:3300/0 conn(0x7f95380cce90 0x7f95380b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.571+0000 7f953d000640 1 -- 172.21.15.73:0/3562582318 >> v2:172.21.15.157:3300/0 conn(0x7f95380cce90 msgr2=0x7f95380b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.571+0000 7f953d000640 1 --2- 172.21.15.73:0/3562582318 >> v2:172.21.15.157:3300/0 conn(0x7f95380cce90 0x7f95380b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.571+0000 7f953d000640 1 -- 172.21.15.73:0/3562582318 >> v1:172.21.15.157:6789/0 conn(0x7f953815ae50 legacy=0x7f95380cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.571+0000 7f953d000640 1 -- 172.21.15.73:0/3562582318 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953815f4a0 con 0x7f95380cf3e0 2024-04-05T22:30:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.572+0000 7f953d000640 1 --2- 172.21.15.73:0/3562582318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9524009870 tx=0x7f952402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd01fcb246e5843a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9537e00640 1 -- 172.21.15.73:0/3562582318 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524031640 con 0x7f95380cf3e0 2024-04-05T22:30:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9537e00640 1 -- 172.21.15.73:0/3562582318 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95240317a0 con 0x7f95380cf3e0 2024-04-05T22:30:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9537e00640 1 -- 172.21.15.73:0/3562582318 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524031b30 con 0x7f95380cf3e0 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 -- 172.21.15.73:0/3562582318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 msgr2=0x7f95380b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 --2- 172.21.15.73:0/3562582318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9524009870 tx=0x7f952402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 -- 172.21.15.73:0/3562582318 shutdown_connections 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 --2- 172.21.15.73:0/3562582318 >> v2:172.21.15.157:3300/0 conn(0x7f95380cce90 0x7f95380b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 --2- 172.21.15.73:0/3562582318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95380cf3e0 0x7f95380b3aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 -- 172.21.15.73:0/3562582318 >> 172.21.15.73:0/3562582318 conn(0x7f953806a0d0 msgr2=0x7f953806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 -- 172.21.15.73:0/3562582318 shutdown_connections 2024-04-05T22:30:48.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.573+0000 7f9540400640 1 -- 172.21.15.73:0/3562582318 wait complete. 2024-04-05T22:30:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.575+0000 7f9540400640 1 Processor -- start 2024-04-05T22:30:48.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.575+0000 7f9540400640 1 -- start start 2024-04-05T22:30:48.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95380cce90 0x7f9538144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f953815ae50 0x7f9538144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538144790 con 0x7f95380cf3e0 2024-04-05T22:30:48.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381483c0 con 0x7f95380cce90 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f9540400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95381486a0 con 0x7f953815ae50 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f953d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f953d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48816/0 (socket says 172.21.15.73:48816) 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.576+0000 7f953d000640 1 -- 172.21.15.73:0/3709794504 learned_addr learned my addr 172.21.15.73:0/3709794504 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953da00640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95380cce90 0x7f9538144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953e400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f953815ae50 0x7f9538144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 -- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f953815ae50 msgr2=0x7f9538144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f953815ae50 0x7f9538144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 -- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95380cce90 msgr2=0x7f9538144f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95380cce90 0x7f9538144f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 -- 172.21.15.73:0/3709794504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240094d0 con 0x7f95380cf3e0 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.577+0000 7f953d000640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9524000c00 tx=0x7f95240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:48.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.578+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952403a930 con 0x7f95380cf3e0 2024-04-05T22:30:48.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.578+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9538148980 con 0x7f95380cf3e0 2024-04-05T22:30:48.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.578+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952403aa90 con 0x7f95380cf3e0 2024-04-05T22:30:48.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.578+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952403ae40 con 0x7f95380cf3e0 2024-04-05T22:30:48.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.578+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9538148f40 con 0x7f95380cf3e0 2024-04-05T22:30:48.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.579+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9538068160 con 0x7f95380cf3e0 2024-04-05T22:30:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.580+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f952402f050 con 0x7f95380cf3e0 2024-04-05T22:30:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.581+0000 7f9535600640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 0x7f951806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.581+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f95240a98e0 con 0x7f95380cf3e0 2024-04-05T22:30:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.581+0000 7f953da00640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 0x7f951806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:48.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.582+0000 7f953da00640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 0x7f951806d8a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f60 tx=0x7f952c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.585+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9524076640 con 0x7f95380cf3e0 2024-04-05T22:30:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:48.783+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9538153eb0 con 0x7f95380cf3e0 2024-04-05T22:30:49.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.341+0000 7f9535600640 1 -- 172.21.15.73:0/3709794504 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9524079c60 con 0x7f95380cf3e0 2024-04-05T22:30:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:30:49.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.351+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 msgr2=0x7f951806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.351+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 0x7f951806d8a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f60 tx=0x7f952c009040 comp rx=0 tx=0).stop 2024-04-05T22:30:49.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.351+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 msgr2=0x7f9538143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.351+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9524000c00 tx=0x7f95240046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.352+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 shutdown_connections 2024-04-05T22:30:49.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.352+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f951806b3e0 0x7f951806d8a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.352+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f953815ae50 0x7f9538144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.352+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95380cf3e0 0x7f9538143d10 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.352+0000 7f9540400640 1 --2- 172.21.15.73:0/3709794504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95380cce90 0x7f9538144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.353+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 >> 172.21.15.73:0/3709794504 conn(0x7f953806a0d0 msgr2=0x7f95380afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.353+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 shutdown_connections 2024-04-05T22:30:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.353+0000 7f9540400640 1 -- 172.21.15.73:0/3709794504 wait complete. 2024-04-05T22:30:49.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:30:49.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.570+0000 7f07f3800640 1 Processor -- start 2024-04-05T22:30:49.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.570+0000 7f07f3800640 1 -- start start 2024-04-05T22:30:49.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07f3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 0x7f07ec0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07f3800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec146ac0 con 0x7f07ec15dd90 2024-04-05T22:30:49.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07f3800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec070720 con 0x7f07ec070c50 2024-04-05T22:30:49.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07f3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0acf60 con 0x7f07ec06c630 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07e3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f07ec15dd90 0x7f07ec06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60084/0 (socket says 172.21.15.73:60084) 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.572+0000 7f07e3e00640 1 -- 172.21.15.73:0/2447315419 learned_addr learned my addr 172.21.15.73:0/2447315419 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07f0e00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 0x7f07ec0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079099622 0 0) 0x7f07ec146ac0 con 0x7f07ec15dd90 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07cc003520 con 0x7f07ec15dd90 2024-04-05T22:30:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862808699 0 0) 0x7f07cc003520 con 0x7f07ec15dd90 2024-04-05T22:30:49.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 msgr2=0x7f07ec0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:49.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 0x7f07ec0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 >> v1:172.21.15.157:6789/0 conn(0x7f07ec070c50 legacy=0x7f07ec0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0ad240 con 0x7f07ec15dd90 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f07d4001670 con 0x7f07ec15dd90 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 >> v1:172.21.15.73:6789/0 conn(0x7f07ec15dd90 legacy=0x7f07ec06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.573+0000 7f07ebe00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 0x7f07cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07ebe00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07cc004d00 0x7f07cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0acf60 con 0x7f07cc0039a0 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00ba30 con 0x7f07cc004d00 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00bbf0 con 0x7f07ec06c630 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 -- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 msgr2=0x7f07cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:49.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec06c630 0x7f07cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 -- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07cc004d00 msgr2=0x7f07cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:49.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07cc004d00 0x7f07cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.574+0000 7f07f0e00640 1 -- 172.21.15.73:0/2447315419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc003520 con 0x7f07cc0039a0 2024-04-05T22:30:49.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.575+0000 7f07f0e00640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009a90 tx=0x7f07cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76535284d751ab10 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:49.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc02c6b0 con 0x7f07cc0039a0 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc02c810 con 0x7f07cc0039a0 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07ebe00640 1 -- 172.21.15.73:0/2447315419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc02cba0 con 0x7f07cc0039a0 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 -- 172.21.15.73:0/2447315419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 msgr2=0x7f07cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 --2- 172.21.15.73:0/2447315419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07cc003dc0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009a90 tx=0x7f07cc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f1800640 1 -- 172.21.15.73:0/2447315419 reap_dead start 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 -- 172.21.15.73:0/2447315419 shutdown_connections 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 -- 172.21.15.73:0/2447315419 >> 172.21.15.73:0/2447315419 conn(0x7f07ec0b0830 msgr2=0x7f07ec0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 -- 172.21.15.73:0/2447315419 shutdown_connections 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.576+0000 7f07f3800640 1 -- 172.21.15.73:0/2447315419 wait complete. 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.578+0000 7f07f3800640 1 Processor -- start 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.578+0000 7f07f3800640 1 -- start start 2024-04-05T22:30:49.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec0ca2d0 0x7f07ec0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ec137f60 0x7f07ec1383e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1f6ef0 con 0x7f07cc0039a0 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec13aed0 con 0x7f07ec137f60 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.579+0000 7f07f3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0c9550 con 0x7f07ec0ca2d0 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48832/0 (socket says 172.21.15.73:48832) 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 -- 172.21.15.73:0/3000314881 learned_addr learned my addr 172.21.15.73:0/3000314881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 -- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec0ca2d0 msgr2=0x7f07ec0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec0ca2d0 0x7f07ec0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 -- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ec137f60 msgr2=0x7f07ec1383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ec137f60 0x7f07ec1383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 -- 172.21.15.73:0/3000314881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec1f6c10 con 0x7f07cc0039a0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.580+0000 7f07f0e00640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f07dc037040 tx=0x7f07dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.581+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003810 con 0x7f07cc0039a0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.581+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc003970 con 0x7f07cc0039a0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.581+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003d00 con 0x7f07cc0039a0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.581+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec0c97f0 con 0x7f07cc0039a0 2024-04-05T22:30:49.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.581+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec0c9db0 con 0x7f07cc0039a0 2024-04-05T22:30:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.582+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec068160 con 0x7f07cc0039a0 2024-04-05T22:30:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.583+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f07dc047450 con 0x7f07cc0039a0 2024-04-05T22:30:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.584+0000 7f07e9600640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 0x7f07c806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.584+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07dc0a9c00 con 0x7f07cc0039a0 2024-04-05T22:30:49.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.584+0000 7f07e3e00640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 0x7f07c806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:49.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.586+0000 7f07e3e00640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 0x7f07c806d8a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000f60 tx=0x7f07d4001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:49.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.588+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f07dc076bb0 con 0x7f07cc0039a0 2024-04-05T22:30:49.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.781+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f07ec0adb90 con 0x7f07cc0039a0 2024-04-05T22:30:49.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.787+0000 7f07e9600640 1 -- 172.21.15.73:0/3000314881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07dc0044e0 con 0x7f07cc0039a0 2024-04-05T22:30:49.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 msgr2=0x7f07c806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 0x7f07c806d8a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000f60 tx=0x7f07d4001520 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 msgr2=0x7f07ec071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f07dc037040 tx=0x7f07dc004730 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 shutdown_connections 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f07c806b3e0 0x7f07c806d8a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ec137f60 0x7f07ec1383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07ec0ca2d0 0x7f07ec0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 --2- 172.21.15.73:0/3000314881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07cc0039a0 0x7f07ec071350 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 >> 172.21.15.73:0/3000314881 conn(0x7f07ec0b0830 msgr2=0x7f07ec0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 shutdown_connections 2024-04-05T22:30:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:49.790+0000 7f07f3800640 1 -- 172.21.15.73:0/3000314881 wait complete. 2024-04-05T22:30:49.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-05T22:30:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.000+0000 7f5a6fe00640 1 Processor -- start 2024-04-05T22:30:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.000+0000 7f5a6fe00640 1 -- start start 2024-04-05T22:30:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a7006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a7006c890 0x7f5a700ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a700adcd0 0x7f5a7015e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700ae160 con 0x7f5a7006c890 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7006ccf0 con 0x7f5a700adcd0 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.001+0000 7f5a6fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7015bb60 con 0x7f5a70158540 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a7006c890 0x7f5a700ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a7006c890 0x7f5a700ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48848/0 (socket says 172.21.15.73:48848) 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 -- 172.21.15.73:0/4023203058 learned_addr learned my addr 172.21.15.73:0/4023203058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 -- 172.21.15.73:0/4023203058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 msgr2=0x7f5a7006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 --2- 172.21.15.73:0/4023203058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a7006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 -- 172.21.15.73:0/4023203058 >> v2:172.21.15.157:3300/0 conn(0x7f5a700adcd0 msgr2=0x7f5a7015e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 --2- 172.21.15.73:0/4023203058 >> v2:172.21.15.157:3300/0 conn(0x7f5a700adcd0 0x7f5a7015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.002+0000 7f5a66000640 1 -- 172.21.15.73:0/4023203058 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7006b950 con 0x7f5a7006c890 2024-04-05T22:30:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.003+0000 7f5a66000640 1 --2- 172.21.15.73:0/4023203058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7006c890 0x7f5a700ad790 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5a700ce6f0 tx=0x7f5a5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953dd801c96d238 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.003+0000 7f5a6e000640 1 -- 172.21.15.73:0/4023203058 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58031870 con 0x7f5a7006c890 2024-04-05T22:30:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.003+0000 7f5a6e000640 1 -- 172.21.15.73:0/4023203058 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a580319d0 con 0x7f5a7006c890 2024-04-05T22:30:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.003+0000 7f5a6e000640 1 -- 172.21.15.73:0/4023203058 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58031d60 con 0x7f5a7006c890 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4023203058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7006c890 msgr2=0x7f5a700ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4023203058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7006c890 0x7f5a700ad790 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5a700ce6f0 tx=0x7f5a5802b910 comp rx=0 tx=0).stop 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4023203058 shutdown_connections 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4023203058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7006c890 0x7f5a700ad790 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4023203058 >> v2:172.21.15.157:3300/0 conn(0x7f5a700adcd0 0x7f5a7015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4023203058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a7006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4023203058 >> 172.21.15.73:0/4023203058 conn(0x7f5a70069cb0 msgr2=0x7f5a7006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4023203058 shutdown_connections 2024-04-05T22:30:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.004+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4023203058 wait complete. 2024-04-05T22:30:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.006+0000 7f5a6fe00640 1 Processor -- start 2024-04-05T22:30:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.006+0000 7f5a6fe00640 1 -- start start 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 0x7f5a701f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a700ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a70141ea0 con 0x7f5a700adcd0 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a701f5760 con 0x7f5a7006c890 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a6fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a70141290 con 0x7f5a70158540 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a66000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a66000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48850/0 (socket says 172.21.15.73:48850) 2024-04-05T22:30:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.007+0000 7f5a66000640 1 -- 172.21.15.73:0/4076807881 learned_addr learned my addr 172.21.15.73:0/4076807881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a6ea00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 0x7f5a701f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 -- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 msgr2=0x7f5a700ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a700ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 -- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 msgr2=0x7f5a701f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 0x7f5a701f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 -- 172.21.15.73:0/4076807881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580094d0 con 0x7f5a700adcd0 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a6ea00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 0x7f5a701f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a66000640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009870 tx=0x7f5a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5803ad30 con 0x7f5a700adcd0 2024-04-05T22:30:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5803ae90 con 0x7f5a700adcd0 2024-04-05T22:30:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58044630 con 0x7f5a700adcd0 2024-04-05T22:30:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.008+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a70141570 con 0x7f5a700adcd0 2024-04-05T22:30:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.009+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a70141b30 con 0x7f5a700adcd0 2024-04-05T22:30:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.010+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a70142050 con 0x7f5a700adcd0 2024-04-05T22:30:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.011+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f5a5804d450 con 0x7f5a700adcd0 2024-04-05T22:30:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.012+0000 7f5a67400640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 0x7f5a4c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.012+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a5802d040 con 0x7f5a700adcd0 2024-04-05T22:30:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.013+0000 7f5a6ea00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 0x7f5a4c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.014+0000 7f5a6ea00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 0x7f5a4c06d8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a870 tx=0x7f5a60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.016+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5a70142050 con 0x7f5a700adcd0 2024-04-05T22:30:50.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.279+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7f5a7004e160 con 0x7f5a700adcd0 2024-04-05T22:30:50.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.367+0000 7f5a67400640 1 -- 172.21.15.73:0/4076807881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v518) v1 ==== 149+0+0 (secure 0 0 0) 0x7f5a58076f10 con 0x7f5a700adcd0 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.369+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 msgr2=0x7f5a4c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.369+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 0x7f5a4c06d8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a870 tx=0x7f5a60008040 comp rx=0 tx=0).stop 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.369+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 msgr2=0x7f5a70142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.369+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009870 tx=0x7f5a580046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 shutdown_connections 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a4c06b3e0 0x7f5a4c06d8a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a70158540 0x7f5a700ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a700adcd0 0x7f5a70142330 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 --2- 172.21.15.73:0/4076807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7006c890 0x7f5a701f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 >> 172.21.15.73:0/4076807881 conn(0x7f5a70069cb0 msgr2=0x7f5a700b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:50.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 shutdown_connections 2024-04-05T22:30:50.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.370+0000 7f5a6fe00640 1 -- 172.21.15.73:0/4076807881 wait complete. 2024-04-05T22:30:50.389 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-04-05T22:30:50.389 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-05T22:30:50.389 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:30:50.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.591+0000 7fd94b000640 1 Processor -- start 2024-04-05T22:30:50.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.592+0000 7fd94b000640 1 -- start start 2024-04-05T22:30:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd94b000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9441587d0 0x7fd9440cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd94b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9440ccd20 0x7fd94406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd94b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94406f5b0 con 0x7fd94406d0c0 2024-04-05T22:30:50.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd94b000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd944070410 con 0x7fd9441587d0 2024-04-05T22:30:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd94b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd944158bb0 con 0x7fd9440ccd20 2024-04-05T22:30:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd949000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd94406d0c0 0x7fd94415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60100/0 (socket says 172.21.15.73:60100) 2024-04-05T22:30:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.593+0000 7fd949000640 1 -- 172.21.15.73:0/1883213518 learned_addr learned my addr 172.21.15.73:0/1883213518 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd943e00640 1 --2- 172.21.15.73:0/1883213518 >> v2:172.21.15.157:3300/0 conn(0x7fd9441587d0 0x7fd9440cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd943400640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9440ccd20 0x7fd94406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750759308 0 0) 0x7fd94406f5b0 con 0x7fd94406d0c0 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd928003520 con 0x7fd94406d0c0 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168215129 0 0) 0x7fd928003520 con 0x7fd94406d0c0 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9440ccd20 msgr2=0x7fd94406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9440ccd20 0x7fd94406cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 >> v2:172.21.15.157:3300/0 conn(0x7fd9441587d0 msgr2=0x7fd9440cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 --2- 172.21.15.73:0/1883213518 >> v2:172.21.15.157:3300/0 conn(0x7fd9441587d0 0x7fd9440cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94406f800 con 0x7fd94406d0c0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fd93c0019f0 con 0x7fd94406d0c0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd943400640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9440ccd20 0x7fd94406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 >> v1:172.21.15.73:6789/0 conn(0x7fd94406d0c0 legacy=0x7fd94415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd928003970 0x7fd928003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280047c0 0x7fd928004bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd92800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd944070410 con 0x7fd928005100 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd944158bb0 con 0x7fd928003970 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.594+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280045c0 con 0x7fd9280047c0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd92800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd943400640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280047c0 0x7fd928004bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd943e00640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd928003970 0x7fd928003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 -- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280047c0 msgr2=0x7fd928004bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280047c0 0x7fd928004bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 -- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd928003970 msgr2=0x7fd928003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:50.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd928003970 0x7fd928003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.595+0000 7fd949000640 1 -- 172.21.15.73:0/1883213518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928003520 con 0x7fd928005100 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.596+0000 7fd949000640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd92800b980 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00a830 tx=0x7fd9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eedd4974489adbb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c034630 con 0x7fd928005100 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c02f070 con 0x7fd928005100 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd942a00640 1 -- 172.21.15.73:0/1883213518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0349f0 con 0x7fd928005100 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 -- 172.21.15.73:0/1883213518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 msgr2=0x7fd92800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 --2- 172.21.15.73:0/1883213518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd92800b980 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00a830 tx=0x7fd9280057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd949000640 1 -- 172.21.15.73:0/1883213518 reap_dead start 2024-04-05T22:30:50.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 -- 172.21.15.73:0/1883213518 shutdown_connections 2024-04-05T22:30:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 -- 172.21.15.73:0/1883213518 >> 172.21.15.73:0/1883213518 conn(0x7fd9440ce8e0 msgr2=0x7fd9440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 -- 172.21.15.73:0/1883213518 shutdown_connections 2024-04-05T22:30:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.597+0000 7fd94b000640 1 -- 172.21.15.73:0/1883213518 wait complete. 2024-04-05T22:30:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.599+0000 7fd94b000640 1 Processor -- start 2024-04-05T22:30:50.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.599+0000 7fd94b000640 1 -- start start 2024-04-05T22:30:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9441f5d30 0x7fd9440b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94413d940 0x7fd94413ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441f6540 con 0x7fd928005100 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441f6820 con 0x7fd94413d940 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd94b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441f6b00 con 0x7fd9441f5d30 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd943e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd943e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48864/0 (socket says 172.21.15.73:48864) 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.600+0000 7fd943e00640 1 -- 172.21.15.73:0/1591703617 learned_addr learned my addr 172.21.15.73:0/1591703617 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 -- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9441f5d30 msgr2=0x7fd9440b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9441f5d30 0x7fd9440b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 -- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94413d940 msgr2=0x7fd94413ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94413d940 0x7fd94413ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 -- 172.21.15.73:0/1591703617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94415b060 con 0x7fd928005100 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd943e00640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd934011600 tx=0x7fd934011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934015870 con 0x7fd928005100 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9340159d0 con 0x7fd928005100 2024-04-05T22:30:50.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934015d40 con 0x7fd928005100 2024-04-05T22:30:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94415b340 con 0x7fd928005100 2024-04-05T22:30:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.601+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9440b3020 con 0x7fd928005100 2024-04-05T22:30:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.602+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd944066af0 con 0x7fd928005100 2024-04-05T22:30:50.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.603+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fd93400f490 con 0x7fd928005100 2024-04-05T22:30:50.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.604+0000 7fd93be00640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 0x7fd92006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:50.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.604+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd934087ee0 con 0x7fd928005100 2024-04-05T22:30:50.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.605+0000 7fd943400640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 0x7fd92006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:50.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.606+0000 7fd943400640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 0x7fd92006d8d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd92c006f80 tx=0x7fd92c006e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:50.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.609+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd934054cc0 con 0x7fd928005100 2024-04-05T22:30:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.787+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd944068160 con 0x7fd928005100 2024-04-05T22:30:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.789+0000 7fd93be00640 1 -- 172.21.15.73:0/1591703617 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+12528 (secure 0 0 0) 0x7fd9340582e0 con 0x7fd928005100 2024-04-05T22:30:50.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:50.789 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":151,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:49.332115+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6835/2701059182":"2024-04-06T22:30:46.172791+0000","172.21.15.73:6833/2701059182":"2024-04-06T22:30:46.172791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 msgr2=0x7fd92006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 0x7fd92006d8d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd92c006f80 tx=0x7fd92c006e30 comp rx=0 tx=0).stop 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 msgr2=0x7fd9440afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd934011600 tx=0x7fd934011ac0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 shutdown_connections 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd92006b410 0x7fd92006d8d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94413d940 0x7fd94413ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9441f5d30 0x7fd9440b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 --2- 172.21.15.73:0/1591703617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928005100 0x7fd9440afbe0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 >> 172.21.15.73:0/1591703617 conn(0x7fd9440ce8e0 msgr2=0x7fd944158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 shutdown_connections 2024-04-05T22:30:50.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:50.792+0000 7fd94b000640 1 -- 172.21.15.73:0/1591703617 wait complete. 2024-04-05T22:30:50.809 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:50.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:51.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.016+0000 7f6d95e00640 1 Processor -- start 2024-04-05T22:30:51.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.016+0000 7f6d95e00640 1 -- start start 2024-04-05T22:30:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d90164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d95e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d90165310 0x7f6d9006b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d95e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9015f260 con 0x7f6d9015ee80 2024-04-05T22:30:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d95e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d901657c0 con 0x7f6d90165310 2024-04-05T22:30:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d95e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90070d20 con 0x7f6d900b1950 2024-04-05T22:30:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d8ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d9015ee80 0x7f6d900cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60104/0 (socket says 172.21.15.73:60104) 2024-04-05T22:30:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.018+0000 7f6d8f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d90165310 0x7f6d9006b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8ea00640 1 -- 172.21.15.73:0/178506996 learned_addr learned my addr 172.21.15.73:0/178506996 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d86000640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d90164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8e000640 1 -- 172.21.15.73:0/178506996 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891153545 0 0) 0x7f6d80001110 con 0x7f6d9015ee80 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8f400640 1 -- 172.21.15.73:0/178506996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 msgr2=0x7f6d90164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8f400640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d90164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8f400640 1 -- 172.21.15.73:0/178506996 >> v1:172.21.15.73:6789/0 conn(0x7f6d9015ee80 legacy=0x7f6d900cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.019+0000 7f6d8f400640 1 -- 172.21.15.73:0/178506996 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9006f5b0 con 0x7f6d90165310 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.020+0000 7f6d8f400640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d9006b670 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009d90 tx=0x7f6d8802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57878b83a9c683b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.020+0000 7f6d8e000640 1 -- 172.21.15.73:0/178506996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8802cc60 con 0x7f6d90165310 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.020+0000 7f6d8e000640 1 -- 172.21.15.73:0/178506996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8802cdc0 con 0x7f6d90165310 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.020+0000 7f6d8e000640 1 -- 172.21.15.73:0/178506996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88034660 con 0x7f6d90165310 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 -- 172.21.15.73:0/178506996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 msgr2=0x7f6d9006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d9006b670 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009d90 tx=0x7f6d8802beb0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 -- 172.21.15.73:0/178506996 shutdown_connections 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d90164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 --2- 172.21.15.73:0/178506996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d9006b670 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 -- 172.21.15.73:0/178506996 >> 172.21.15.73:0/178506996 conn(0x7f6d90069cb0 msgr2=0x7f6d9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 -- 172.21.15.73:0/178506996 shutdown_connections 2024-04-05T22:30:51.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.021+0000 7f6d95e00640 1 -- 172.21.15.73:0/178506996 wait complete. 2024-04-05T22:30:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.023+0000 7f6d95e00640 1 Processor -- start 2024-04-05T22:30:51.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.023+0000 7f6d95e00640 1 -- start start 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d9014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 0x7f6d900ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c4810 con 0x7f6d90165310 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c4a40 con 0x7f6d9015ee80 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d95e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c93b0 con 0x7f6d900b1950 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48882/0 (socket says 172.21.15.73:48882) 2024-04-05T22:30:51.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d8f400640 1 -- 172.21.15.73:0/3062740338 learned_addr learned my addr 172.21.15.73:0/3062740338 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d8ea00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d9014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d86000640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 0x7f6d900ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.024+0000 7f6d8f400640 1 -- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 msgr2=0x7f6d9014a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8f400640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d9014a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8f400640 1 -- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 msgr2=0x7f6d900ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8f400640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 0x7f6d900ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8f400640 1 -- 172.21.15.73:0/3062740338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009a70 con 0x7f6d90165310 2024-04-05T22:30:51.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8ea00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d9014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d86000640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 0x7f6d900ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d8f400640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009d90 tx=0x7f6d88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803e610 con 0x7f6d90165310 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8803e790 con 0x7f6d90165310 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900c9690 con 0x7f6d90165310 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803eb40 con 0x7f6d90165310 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.025+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900c9060 con 0x7f6d90165310 2024-04-05T22:30:51.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.026+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900c9bb0 con 0x7f6d90165310 2024-04-05T22:30:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.027+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f6d8803b050 con 0x7f6d90165310 2024-04-05T22:30:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.028+0000 7f6d87e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 0x7f6d6c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.028+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d880aa380 con 0x7f6d90165310 2024-04-05T22:30:51.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.029+0000 7f6d8ea00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 0x7f6d6c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.030+0000 7f6d8ea00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 0x7f6d6c06d8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d80000f60 tx=0x7f6d80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.033+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6d900c9bb0 con 0x7f6d90165310 2024-04-05T22:30:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.225+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d900c9bb0 con 0x7f6d90165310 2024-04-05T22:30:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.226+0000 7f6d87e00640 1 -- 172.21.15.73:0/3062740338 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d900c9bb0 con 0x7f6d90165310 2024-04-05T22:30:51.226 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 msgr2=0x7f6d6c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 0x7f6d6c06d8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d80000f60 tx=0x7f6d80009040 comp rx=0 tx=0).stop 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 msgr2=0x7f6d900c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009d90 tx=0x7f6d88004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 shutdown_connections 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6d6c06b3e0 0x7f6d6c06d8a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d90165310 0x7f6d900c42d0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9015ee80 0x7f6d900ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 --2- 172.21.15.73:0/3062740338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d900b1950 0x7f6d9014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.228+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 >> 172.21.15.73:0/3062740338 conn(0x7f6d90069cb0 msgr2=0x7f6d9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.229+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 shutdown_connections 2024-04-05T22:30:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.229+0000 7f6d95e00640 1 -- 172.21.15.73:0/3062740338 wait complete. 2024-04-05T22:30:51.248 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-05T22:30:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.461+0000 7f7248800640 1 Processor -- start 2024-04-05T22:30:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.461+0000 7f7248800640 1 -- start start 2024-04-05T22:30:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7248800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f724006ede0 0x7f7240071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7248800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f724006e920 0x7f72400cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7248800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724006bbc0 con 0x7f724015d120 2024-04-05T22:30:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7248800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724015d500 con 0x7f724006e920 2024-04-05T22:30:51.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7248800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724006b8e0 con 0x7f724006ede0 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7246800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f724006e920 0x7f72400cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7246800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f724006e920 0x7f72400cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48894/0 (socket says 172.21.15.73:48894) 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.463+0000 7f7246800640 1 -- 172.21.15.73:0/4148924983 learned_addr learned my addr 172.21.15.73:0/4148924983 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.464+0000 7f7246800640 1 -- 172.21.15.73:0/4148924983 >> v2:172.21.15.157:3300/0 conn(0x7f724006ede0 msgr2=0x7f7240071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.464+0000 7f7246800640 1 --2- 172.21.15.73:0/4148924983 >> v2:172.21.15.157:3300/0 conn(0x7f724006ede0 0x7f7240071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.464+0000 7f7246800640 1 -- 172.21.15.73:0/4148924983 >> v1:172.21.15.157:6789/0 conn(0x7f724015d120 legacy=0x7f72400b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.464+0000 7f7246800640 1 -- 172.21.15.73:0/4148924983 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240070760 con 0x7f724006e920 2024-04-05T22:30:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.465+0000 7f7246800640 1 --2- 172.21.15.73:0/4148924983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724006e920 0x7f72400cd900 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7240066ac0 tx=0x7f723c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e072b8af95c7049 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.465+0000 7f7244a00640 1 -- 172.21.15.73:0/4148924983 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c02ca80 con 0x7f724006e920 2024-04-05T22:30:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.465+0000 7f7244a00640 1 -- 172.21.15.73:0/4148924983 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c02cbe0 con 0x7f724006e920 2024-04-05T22:30:51.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.465+0000 7f7244a00640 1 -- 172.21.15.73:0/4148924983 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0343f0 con 0x7f724006e920 2024-04-05T22:30:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 -- 172.21.15.73:0/4148924983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724006e920 msgr2=0x7f72400cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 --2- 172.21.15.73:0/4148924983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724006e920 0x7f72400cd900 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7240066ac0 tx=0x7f723c02bee0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 -- 172.21.15.73:0/4148924983 shutdown_connections 2024-04-05T22:30:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 --2- 172.21.15.73:0/4148924983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724006e920 0x7f72400cd900 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 --2- 172.21.15.73:0/4148924983 >> v2:172.21.15.157:3300/0 conn(0x7f724006ede0 0x7f7240071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 -- 172.21.15.73:0/4148924983 >> 172.21.15.73:0/4148924983 conn(0x7f724006adc0 msgr2=0x7f724006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 -- 172.21.15.73:0/4148924983 shutdown_connections 2024-04-05T22:30:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.466+0000 7f7248800640 1 -- 172.21.15.73:0/4148924983 wait complete. 2024-04-05T22:30:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.468+0000 7f7248800640 1 Processor -- start 2024-04-05T22:30:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.468+0000 7f7248800640 1 -- start start 2024-04-05T22:30:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 0x7f724006d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 0x7f724006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724015e9a0 con 0x7f724015d640 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72400ad4d0 con 0x7f724015d120 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.469+0000 7f7248800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72400ada90 con 0x7f724006e920 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48902/0 (socket says 172.21.15.73:48902) 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 -- 172.21.15.73:0/2493450252 learned_addr learned my addr 172.21.15.73:0/2493450252 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7245e00640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 0x7f724006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7245400640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 0x7f724006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 -- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 msgr2=0x7f724006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 0x7f724006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 -- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 msgr2=0x7f724006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 0x7f724006d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 -- 172.21.15.73:0/2493450252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c009b30 con 0x7f724015d640 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7245e00640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 0x7f724006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:51.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7245400640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 0x7f724006d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:51.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.470+0000 7f7246800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f723c036040 tx=0x7f723c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.471+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c03e450 con 0x7f724015d640 2024-04-05T22:30:51.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.471+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c03e5d0 con 0x7f724015d640 2024-04-05T22:30:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.471+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c03e980 con 0x7f724015d640 2024-04-05T22:30:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.471+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7240145f60 con 0x7f724015d640 2024-04-05T22:30:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.471+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724006e150 con 0x7f724015d640 2024-04-05T22:30:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.472+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7240066af0 con 0x7f724015d640 2024-04-05T22:30:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.473+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f723c02cd50 con 0x7f724015d640 2024-04-05T22:30:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.474+0000 7f723a000640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 0x7f722406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.474+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f723c0aa690 con 0x7f724015d640 2024-04-05T22:30:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.474+0000 7f7245e00640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 0x7f722406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.476+0000 7f7245e00640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 0x7f722406d8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f723400a830 tx=0x7f7234008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:51.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.478+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f723c077590 con 0x7f724015d640 2024-04-05T22:30:51.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:51.747+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f72400adc00 con 0x7f724015d640 2024-04-05T22:30:52.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.379+0000 7f723a000640 1 -- 172.21.15.73:0/2493450252 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v520) v1 ==== 148+0+0 (secure 0 0 0) 0x7f723c0430d0 con 0x7f724015d640 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 msgr2=0x7f722406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 0x7f722406d8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f723400a830 tx=0x7f7234008040 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 msgr2=0x7f724015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f723c036040 tx=0x7f723c004a40 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 shutdown_connections 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f722406b3e0 0x7f722406d8a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f724015d640 0x7f724015dad0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724015d120 0x7f724006d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 --2- 172.21.15.73:0/2493450252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f724006e920 0x7f724006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 >> 172.21.15.73:0/2493450252 conn(0x7f724006adc0 msgr2=0x7f72400b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:52.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.383+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 shutdown_connections 2024-04-05T22:30:52.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.384+0000 7f7248800640 1 -- 172.21.15.73:0/2493450252 wait complete. 2024-04-05T22:30:52.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:30:52.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.622+0000 7f0025200640 1 Processor -- start 2024-04-05T22:30:52.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.623+0000 7f0025200640 1 -- start start 2024-04-05T22:30:52.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f002015abe0 0x7f00200712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 0x7f002006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006ba70 0x7f00200cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002006bea0 con 0x7f002015abe0 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002015afc0 con 0x7f002006c9a0 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.624+0000 7f0025200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002006f5b0 con 0x7f002006ba70 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f002015abe0 0x7f00200712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f002015abe0 0x7f00200712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48914/0 (socket says 172.21.15.73:48914) 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 -- 172.21.15.73:0/1597882738 learned_addr learned my addr 172.21.15.73:0/1597882738 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001e800640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006ba70 0x7f00200cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f0017e00640 1 --2- 172.21.15.73:0/1597882738 >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 0x7f002006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 -- 172.21.15.73:0/1597882738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006ba70 msgr2=0x7f00200cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006ba70 0x7f00200cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 -- 172.21.15.73:0/1597882738 >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 msgr2=0x7f002006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:52.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 --2- 172.21.15.73:0/1597882738 >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 0x7f002006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f001de00640 1 -- 172.21.15.73:0/1597882738 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002006f890 con 0x7f002015abe0 2024-04-05T22:30:52.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.625+0000 7f0017e00640 1 --2- 172.21.15.73:0/1597882738 >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 0x7f002006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.626+0000 7f001de00640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002015abe0 0x7f00200712e0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f002006ce30 tx=0x7f001002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae946982690dc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.627+0000 7f001d400640 1 -- 172.21.15.73:0/1597882738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010031750 con 0x7f002015abe0 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.627+0000 7f001d400640 1 -- 172.21.15.73:0/1597882738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00100318b0 con 0x7f002015abe0 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.627+0000 7f001d400640 1 -- 172.21.15.73:0/1597882738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010031c40 con 0x7f002015abe0 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.627+0000 7f0025200640 1 -- 172.21.15.73:0/1597882738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002015abe0 msgr2=0x7f00200712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.627+0000 7f0025200640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002015abe0 0x7f00200712e0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f002006ce30 tx=0x7f001002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 -- 172.21.15.73:0/1597882738 shutdown_connections 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006ba70 0x7f00200cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 --2- 172.21.15.73:0/1597882738 >> v2:172.21.15.157:3300/0 conn(0x7f002006c9a0 0x7f002006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 --2- 172.21.15.73:0/1597882738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002015abe0 0x7f00200712e0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 -- 172.21.15.73:0/1597882738 >> 172.21.15.73:0/1597882738 conn(0x7f0020069cb0 msgr2=0x7f002006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:52.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 -- 172.21.15.73:0/1597882738 shutdown_connections 2024-04-05T22:30:52.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.628+0000 7f0025200640 1 -- 172.21.15.73:0/1597882738 wait complete. 2024-04-05T22:30:52.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.629+0000 7f0025200640 1 Processor -- start 2024-04-05T22:30:52.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.630+0000 7f0025200640 1 -- start start 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.630+0000 7f0025200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.630+0000 7f0025200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 0x7f00201f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0025200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f002015abe0 0x7f002014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0025200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201428e0 con 0x7f002006ba70 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0025200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020149f80 con 0x7f002015abe0 2024-04-05T22:30:52.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0025200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002014a260 con 0x7f002006c9a0 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48930/0 (socket says 172.21.15.73:48930) 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 -- 172.21.15.73:0/2691876503 learned_addr learned my addr 172.21.15.73:0/2691876503 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0017e00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 0x7f00201f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 -- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 msgr2=0x7f00201f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001e800640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f002015abe0 0x7f002014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 0x7f00201f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 -- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f002015abe0 msgr2=0x7f002014f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f002015abe0 0x7f002014f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f001de00640 1 -- 172.21.15.73:0/2691876503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100094d0 con 0x7f002006ba70 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.631+0000 7f0017e00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 0x7f00201f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f001de00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0010000c00 tx=0x7f00100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010033c80 con 0x7f002006ba70 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010033e00 con 0x7f002006ba70 2024-04-05T22:30:52.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001003c640 con 0x7f002006ba70 2024-04-05T22:30:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002014a540 con 0x7f002006ba70 2024-04-05T22:30:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.632+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002014ab00 con 0x7f002006ba70 2024-04-05T22:30:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.633+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0020068160 con 0x7f002006ba70 2024-04-05T22:30:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.634+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f001003a040 con 0x7f002006ba70 2024-04-05T22:30:52.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.635+0000 7f0016000640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 0x7efffc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:52.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.635+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f00100a95d0 con 0x7f002006ba70 2024-04-05T22:30:52.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.636+0000 7f0017e00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 0x7efffc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:52.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.637+0000 7f0017e00640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 0x7efffc06d8a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0020150900 tx=0x7f0008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:52.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.639+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00100763b0 con 0x7f002006ba70 2024-04-05T22:30:52.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.833+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f00200b0110 con 0x7f002006ba70 2024-04-05T22:30:52.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.842+0000 7f0016000640 1 -- 172.21.15.73:0/2691876503 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00100380c0 con 0x7f002006ba70 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 msgr2=0x7efffc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 0x7efffc06d8a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0020150900 tx=0x7f0008008040 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 msgr2=0x7f00201423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0010000c00 tx=0x7f00100046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 shutdown_connections 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efffc06b3e0 0x7efffc06d8a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f002015abe0 0x7f002014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f002006c9a0 0x7f00201f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 --2- 172.21.15.73:0/2691876503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f002006ba70 0x7f00201423a0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 >> 172.21.15.73:0/2691876503 conn(0x7f0020069cb0 msgr2=0x7f0020155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 shutdown_connections 2024-04-05T22:30:52.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:52.844+0000 7f0025200640 1 -- 172.21.15.73:0/2691876503 wait complete. 2024-04-05T22:30:52.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:30:52.864 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:52.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:30:53.087 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:30:53.087 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:53.087 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:53.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:53.144 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:53.144 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:53.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:53.200 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:30:53.200 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:53.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:30:53.257 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:30:53.257 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:30:53.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:30:53.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:30:53.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:30:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.526+0000 7fd8f5600640 1 Processor -- start 2024-04-05T22:30:53.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.527+0000 7fd8f5600640 1 -- start start 2024-04-05T22:30:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8f5600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8f0144610 0x7fd8f00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8f5600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 0x7fd8f015e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8f5600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f015c260 con 0x7fd8f015bdd0 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8f5600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f00b29d0 con 0x7fd8f0144610 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8f5600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f015ee60 con 0x7fd8f006a3c0 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8eea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8f0144610 0x7fd8f00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8ee000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd8f015bdd0 0x7fd8f0069cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60114/0 (socket says 172.21.15.73:60114) 2024-04-05T22:30:53.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.528+0000 7fd8ee000640 1 -- 172.21.15.73:0/1731641649 learned_addr learned my addr 172.21.15.73:0/1731641649 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8eea00640 1 -- 172.21.15.73:0/1731641649 >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 msgr2=0x7fd8f015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8ef400640 1 --2- 172.21.15.73:0/1731641649 >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 0x7fd8f015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8ed600640 1 -- 172.21.15.73:0/1731641649 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242056544 0 0) 0x7fd8f015c260 con 0x7fd8f015bdd0 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8eea00640 1 --2- 172.21.15.73:0/1731641649 >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 0x7fd8f015e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8eea00640 1 -- 172.21.15.73:0/1731641649 >> v1:172.21.15.73:6789/0 conn(0x7fd8f015bdd0 legacy=0x7fd8f0069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8eea00640 1 -- 172.21.15.73:0/1731641649 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f01449f0 con 0x7fd8f0144610 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.529+0000 7fd8ef400640 1 --2- 172.21.15.73:0/1731641649 >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 0x7fd8f015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.530+0000 7fd8eea00640 1 --2- 172.21.15.73:0/1731641649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f0144610 0x7fd8f00cdeb0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00b4250 tx=0x7fd8e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bfa6cfb1c41611 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.530+0000 7fd8ed600640 1 -- 172.21.15.73:0/1731641649 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0031830 con 0x7fd8f0144610 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.530+0000 7fd8ed600640 1 -- 172.21.15.73:0/1731641649 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0031990 con 0x7fd8f0144610 2024-04-05T22:30:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.530+0000 7fd8ed600640 1 -- 172.21.15.73:0/1731641649 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0031d20 con 0x7fd8f0144610 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 -- 172.21.15.73:0/1731641649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f0144610 msgr2=0x7fd8f00cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 --2- 172.21.15.73:0/1731641649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f0144610 0x7fd8f00cdeb0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00b4250 tx=0x7fd8e002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 -- 172.21.15.73:0/1731641649 shutdown_connections 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 --2- 172.21.15.73:0/1731641649 >> v2:172.21.15.157:3300/0 conn(0x7fd8f006a3c0 0x7fd8f015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 --2- 172.21.15.73:0/1731641649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f0144610 0x7fd8f00cdeb0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 -- 172.21.15.73:0/1731641649 >> 172.21.15.73:0/1731641649 conn(0x7fd8f00b16d0 msgr2=0x7fd8f00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:53.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 -- 172.21.15.73:0/1731641649 shutdown_connections 2024-04-05T22:30:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.531+0000 7fd8f5600640 1 -- 172.21.15.73:0/1731641649 wait complete. 2024-04-05T22:30:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.533+0000 7fd8f5600640 1 Processor -- start 2024-04-05T22:30:53.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.533+0000 7fd8f5600640 1 -- start start 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 0x7fd8f006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f015bdd0 0x7fd8f013bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01f24a0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006e480 con 0x7fd8f015bdd0 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8f5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006de70 con 0x7fd8f0144610 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48940/0 (socket says 172.21.15.73:48940) 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.534+0000 7fd8eea00640 1 -- 172.21.15.73:0/3881201085 learned_addr learned my addr 172.21.15.73:0/3881201085 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8ef400640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f015bdd0 0x7fd8f013bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8ee000640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 0x7fd8f006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 -- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 msgr2=0x7fd8f006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 0x7fd8f006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 -- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f015bdd0 msgr2=0x7fd8f013bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f015bdd0 0x7fd8f013bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 -- 172.21.15.73:0/3881201085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0009400 con 0x7fd8f006a3c0 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8ee000640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 0x7fd8f006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8eea00640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002bee0 tx=0x7fd8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e003ba70 con 0x7fd8f006a3c0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e003bbf0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00433f0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.535+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f006d4b0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.536+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f00cc630 con 0x7fd8f006a3c0 2024-04-05T22:30:53.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.537+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f00ccc30 con 0x7fd8f006a3c0 2024-04-05T22:30:53.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.538+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fd8e0043550 con 0x7fd8f006a3c0 2024-04-05T22:30:53.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.538+0000 7fd8e6a00640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 0x7fd8d006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:53.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.539+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8e00a98a0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.539+0000 7fd8ee000640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 0x7fd8d006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:53.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.540+0000 7fd8ee000640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 0x7fd8d006d8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8f01589b0 tx=0x7fd8dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:53.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.543+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd8e00767a0 con 0x7fd8f006a3c0 2024-04-05T22:30:53.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.772+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8f0068160 con 0x7fd8f006a3c0 2024-04-05T22:30:53.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.773+0000 7fd8e6a00640 1 -- 172.21.15.73:0/3881201085 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fd8e0035020 con 0x7fd8f006a3c0 2024-04-05T22:30:53.773 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:53.773 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2961837054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2961837054},{"type":"v1","addr":"172.21.15.157:6833","nonce":2961837054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1576950992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1576950992},{"type":"v1","addr":"172.21.15.157:6835","nonce":1576950992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/2643902087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2643902087},{"type":"v1","addr":"172.21.15.73:6834","nonce":2643902087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3130694513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":3130694513},{"type":"v1","addr":"172.21.15.73:6835","nonce":3130694513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[]} 2024-04-05T22:30:53.774 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 94 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.775+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 msgr2=0x7fd8d006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.775+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 0x7fd8d006d8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8f01589b0 tx=0x7fd8dc009040 comp rx=0 tx=0).stop 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.775+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 msgr2=0x7fd8f01f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.775+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002bee0 tx=0x7fd8e00046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 shutdown_connections 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd8d006b3e0 0x7fd8d006d8a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f015bdd0 0x7fd8f013bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f0144610 0x7fd8f006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 --2- 172.21.15.73:0/3881201085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f006a3c0 0x7fd8f01f1f60 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 >> 172.21.15.73:0/3881201085 conn(0x7fd8f00b16d0 msgr2=0x7fd8f015b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 shutdown_connections 2024-04-05T22:30:53.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:53.776+0000 7fd8f5600640 1 -- 172.21.15.73:0/3881201085 wait complete. 2024-04-05T22:30:53.793 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:30:53.793 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:30:53.794 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:30:53.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:53.794 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:30:53.794 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:30:53.794 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:30:53.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:53.794 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:30:53.795 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:30:53.795 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:30:53.795 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:53.795 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:30:53.795 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:30:53.795 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:30:53.795 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:30:53.811 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:30:53.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:30:53.813 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:30:53.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:30:53.824 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:30:53.824 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:30:53.825 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:30:53.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:30:53.861 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:30:53.861 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:53.867 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:30:53.868 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:53.871 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:30:53.871 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:53.877 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:30:53.877 DEBUG:teuthology.parallel:result is None 2024-04-05T22:30:53.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:30:54.009 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:30:54.009 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:30:54.035 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:30:54.044 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:30:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.055+0000 7f482d600640 1 Processor -- start 2024-04-05T22:30:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.056+0000 7f482d600640 1 -- start start 2024-04-05T22:30:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f482d600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f48281541a0 0x7f48280b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f482d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482815aa60 con 0x7f482806d710 2024-04-05T22:30:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f482d600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48280b3b60 con 0x7f48280ce000 2024-04-05T22:30:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f482d600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48280701c0 con 0x7f48281541a0 2024-04-05T22:30:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f4826000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f482806d710 0x7f48280cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60130/0 (socket says 172.21.15.73:60130) 2024-04-05T22:30:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f4826000640 1 -- 172.21.15.73:0/288497929 learned_addr learned my addr 172.21.15.73:0/288497929 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.057+0000 7f4826a00640 1 --2- 172.21.15.73:0/288497929 >> v2:172.21.15.157:3300/0 conn(0x7f48281541a0 0x7f48280b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.058+0000 7f4825600640 1 -- 172.21.15.73:0/288497929 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695439362 0 0) 0x7f482815aa60 con 0x7f482806d710 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.058+0000 7f4825600640 1 -- 172.21.15.73:0/288497929 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c005730 con 0x7f482806d710 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.058+0000 7f4826a00640 1 -- 172.21.15.73:0/288497929 >> v1:172.21.15.157:6789/0 conn(0x7f48280ce000 legacy=0x7f482815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.058+0000 7f4826a00640 1 -- 172.21.15.73:0/288497929 >> v1:172.21.15.73:6789/0 conn(0x7f482806d710 legacy=0x7f48280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.058+0000 7f4826a00640 1 -- 172.21.15.73:0/288497929 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280b3880 con 0x7f48281541a0 2024-04-05T22:30:54.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.059+0000 7f4826a00640 1 --2- 172.21.15.73:0/288497929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48281541a0 0x7f48280b3050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f481802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4ee1d24c5eccb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.059+0000 7f4825600640 1 -- 172.21.15.73:0/288497929 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818031830 con 0x7f48281541a0 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.059+0000 7f4825600640 1 -- 172.21.15.73:0/288497929 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4818031990 con 0x7f48281541a0 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.059+0000 7f4825600640 1 -- 172.21.15.73:0/288497929 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818031d00 con 0x7f48281541a0 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 -- 172.21.15.73:0/288497929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48281541a0 msgr2=0x7f48280b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 --2- 172.21.15.73:0/288497929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48281541a0 0x7f48280b3050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f481802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:54.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 -- 172.21.15.73:0/288497929 shutdown_connections 2024-04-05T22:30:54.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 --2- 172.21.15.73:0/288497929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48281541a0 0x7f48280b3050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:54.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 -- 172.21.15.73:0/288497929 >> 172.21.15.73:0/288497929 conn(0x7f48280ce8e0 msgr2=0x7f48280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:54.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 -- 172.21.15.73:0/288497929 shutdown_connections 2024-04-05T22:30:54.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.060+0000 7f482d600640 1 -- 172.21.15.73:0/288497929 wait complete. 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.062+0000 7f482d600640 1 Processor -- start 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.062+0000 7f482d600640 1 -- start start 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48280ce000 0x7f4828155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48281541a0 0x7f48280ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828145800 con 0x7f482806d710 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48281454d0 con 0x7f48280ce000 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f482d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4828145a50 con 0x7f48281541a0 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f4826a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f4826a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48978/0 (socket says 172.21.15.73:48978) 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.063+0000 7f4826a00640 1 -- 172.21.15.73:0/2278507830 learned_addr learned my addr 172.21.15.73:0/2278507830 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4827400640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48281541a0 0x7f48280ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826000640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48280ce000 0x7f4828155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 -- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48281541a0 msgr2=0x7f48280ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48281541a0 0x7f48280ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 -- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48280ce000 msgr2=0x7f4828155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48280ce000 0x7f4828155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 -- 172.21.15.73:0/2278507830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009400 con 0x7f482806d710 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.064+0000 7f4826a00640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f48180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.065+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818004230 con 0x7f482806d710 2024-04-05T22:30:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.065+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48180043b0 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.065+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818004520 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.065+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4828145d30 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.065+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48281462f0 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.066+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4828066af0 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.067+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f481803a870 con 0x7f482806d710 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.068+0000 7f481ea00640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 0x7f480806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.068+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f48180a99a0 con 0x7f482806d710 2024-04-05T22:30:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.068+0000 7f4826000640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 0x7f480806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.069+0000 7f4826000640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 0x7f480806d8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4828159930 tx=0x7f4814007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:54.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.072+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4818076830 con 0x7f482806d710 2024-04-05T22:30:54.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:54.264+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4828068160 con 0x7f482806d710 2024-04-05T22:30:55.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.068+0000 7f481ea00640 1 -- 172.21.15.73:0/2278507830 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v156) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4818035020 con 0x7f482806d710 2024-04-05T22:30:55.069 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:30:55.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 msgr2=0x7f480806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 0x7f480806d8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4828159930 tx=0x7f4814007d30 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 msgr2=0x7f4828144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f48180046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 shutdown_connections 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f480806b410 0x7f480806d8d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48281541a0 0x7f48280ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48280ce000 0x7f4828155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 --2- 172.21.15.73:0/2278507830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f482806d710 0x7f4828144f90 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 >> 172.21.15.73:0/2278507830 conn(0x7f48280ce8e0 msgr2=0x7f48280aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 shutdown_connections 2024-04-05T22:30:55.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.072+0000 7f482d600640 1 -- 172.21.15.73:0/2278507830 wait complete. 2024-04-05T22:30:55.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:30:55.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.296+0000 7f3567400640 1 Processor -- start 2024-04-05T22:30:55.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.297+0000 7f3567400640 1 -- start start 2024-04-05T22:30:55.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3567400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f356006d710 0x7f35600cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3567400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f35600cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3567400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356015aa60 con 0x7f35601541a0 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3567400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600b3b60 con 0x7f356006d710 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3567400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600701c0 con 0x7f356015aef0 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3564a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f35601541a0 0x7f35600b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60142/0 (socket says 172.21.15.73:60142) 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.298+0000 7f3564a00640 1 -- 172.21.15.73:0/991804097 learned_addr learned my addr 172.21.15.73:0/991804097 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399719004 0 0) 0x7f356015aa60 con 0x7f35601541a0 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3544003520 con 0x7f35601541a0 2024-04-05T22:30:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f3557e00640 1 --2- 172.21.15.73:0/991804097 >> v2:172.21.15.157:3300/0 conn(0x7f356006d710 0x7f35600cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f3565400640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f35600cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099442573 0 0) 0x7f3544003520 con 0x7f35601541a0 2024-04-05T22:30:55.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 msgr2=0x7f35600cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f35600cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 >> v2:172.21.15.157:3300/0 conn(0x7f356006d710 msgr2=0x7f35600cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 --2- 172.21.15.73:0/991804097 >> v2:172.21.15.157:3300/0 conn(0x7f356006d710 0x7f35600cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600b3880 con 0x7f35601541a0 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3550001670 con 0x7f35601541a0 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 >> v1:172.21.15.73:6789/0 conn(0x7f35601541a0 legacy=0x7f35600b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f3544003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3544003fa0 0x7f35440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004d00 0x7f3544005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600b3b60 con 0x7f3544003fa0 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600701c0 con 0x7f3544004d00 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.299+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354400ba80 con 0x7f356015aef0 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3544003fa0 0x7f35440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3565400640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004d00 0x7f3544005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3564a00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f3544003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 -- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 msgr2=0x7f3544003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f356015aef0 0x7f3544003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 -- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004d00 msgr2=0x7f3544005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004d00 0x7f3544005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3557e00640 1 -- 172.21.15.73:0/991804097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544003520 con 0x7f3544003fa0 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.300+0000 7f3565400640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004d00 0x7f3544005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.301+0000 7f3557e00640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3544003fa0 0x7f35440047c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f354800ba30 tx=0x7f35440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9407be98c0f1935 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.301+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548032680 con 0x7f3544003fa0 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.301+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480327e0 con 0x7f3544003fa0 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.301+0000 7f355fe00640 1 -- 172.21.15.73:0/991804097 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548032b70 con 0x7f3544003fa0 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 -- 172.21.15.73:0/991804097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3544003fa0 msgr2=0x7f35440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 --2- 172.21.15.73:0/991804097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3544003fa0 0x7f35440047c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f354800ba30 tx=0x7f35440057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3565400640 1 -- 172.21.15.73:0/991804097 reap_dead start 2024-04-05T22:30:55.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 -- 172.21.15.73:0/991804097 shutdown_connections 2024-04-05T22:30:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 -- 172.21.15.73:0/991804097 >> 172.21.15.73:0/991804097 conn(0x7f35600ce8e0 msgr2=0x7f35600cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 -- 172.21.15.73:0/991804097 shutdown_connections 2024-04-05T22:30:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.302+0000 7f3567400640 1 -- 172.21.15.73:0/991804097 wait complete. 2024-04-05T22:30:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.304+0000 7f3567400640 1 Processor -- start 2024-04-05T22:30:55.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.304+0000 7f3567400640 1 -- start start 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 0x7f3560155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 0x7f35600acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600ad290 con 0x7f356014a120 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3560156f60 con 0x7f35600ac8d0 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3567400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3560156350 con 0x7f3544003fa0 2024-04-05T22:30:55.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3557e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49008/0 (socket says 172.21.15.73:49008) 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3557e00640 1 -- 172.21.15.73:0/3102459628 learned_addr learned my addr 172.21.15.73:0/3102459628 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3564a00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 0x7f3560155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.305+0000 7f3565400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 0x7f35600acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 -- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 msgr2=0x7f3560155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 0x7f3560155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 -- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 msgr2=0x7f35600acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 0x7f35600acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 -- 172.21.15.73:0/3102459628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3560156630 con 0x7f356014a120 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3564a00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 0x7f3560155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:55.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3565400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 0x7f35600acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3557e00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f354802bf00 tx=0x7f3548004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548004220 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480043a0 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3560156910 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548030aa0 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.306+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3560166ce0 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.307+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3560066af0 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.308+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f3548030c00 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.309+0000 7f355d600640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 0x7f353c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.309+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f35480aa4d0 con 0x7f356014a120 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.310+0000 7f3564a00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 0x7f353c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.311+0000 7f3564a00640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 0x7f353c06d8a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3550001820 tx=0x7f3550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.313+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f35480772b0 con 0x7f356014a120 2024-04-05T22:30:55.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.577+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3560066af0 con 0x7f356014a120 2024-04-05T22:30:55.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.579+0000 7f355d600640 1 -- 172.21.15.73:0/3102459628 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3548030eb0 con 0x7f356014a120 2024-04-05T22:30:55.579 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:55.579 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:30:55.580 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:55.581 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.583 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:30:55.584 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:30:55.587 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:30:55.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:30:55.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:55.592 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:30:55.592 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:30:55.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 msgr2=0x7f353c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 0x7f353c06d8a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3550001820 tx=0x7f3550009040 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 msgr2=0x7f35601667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f354802bf00 tx=0x7f3548004730 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 shutdown_connections 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f353c06b3e0 0x7f353c06d8a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35600ac8d0 0x7f35600acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f356014a120 0x7f35601667a0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.581+0000 7f3567400640 1 --2- 172.21.15.73:0/3102459628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3544003fa0 0x7f3560155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.582+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 >> 172.21.15.73:0/3102459628 conn(0x7f35600ce8e0 msgr2=0x7f356006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.582+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 shutdown_connections 2024-04-05T22:30:55.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.582+0000 7f3567400640 1 -- 172.21.15.73:0/3102459628 wait complete. 2024-04-05T22:30:55.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:55.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.799+0000 7f0fd4c00640 1 Processor -- start 2024-04-05T22:30:55.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.799+0000 7f0fd4c00640 1 -- start start 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.800+0000 7f0fd4c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0fd01587d0 0x7f0fd00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fd4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ccd20 0x7f0fd006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fd4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd006f5b0 con 0x7f0fd006d0c0 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fd4c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd0070410 con 0x7f0fd01587d0 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fd4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0158bb0 con 0x7f0fd00ccd20 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fce800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0fd006d0c0 0x7f0fd015e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60152/0 (socket says 172.21.15.73:60152) 2024-04-05T22:30:55.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fce800640 1 -- 172.21.15.73:0/3223534911 learned_addr learned my addr 172.21.15.73:0/3223534911 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fcd400640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ccd20 0x7f0fd006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080067998 0 0) 0x7f0fd006f5b0 con 0x7f0fd006d0c0 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb4003520 con 0x7f0fd006d0c0 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.801+0000 7f0fcde00640 1 --2- 172.21.15.73:0/3223534911 >> v2:172.21.15.157:3300/0 conn(0x7f0fd01587d0 0x7f0fd00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1610707892 0 0) 0x7f0fb4003520 con 0x7f0fd006d0c0 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ccd20 msgr2=0x7f0fd006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ccd20 0x7f0fd006cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 >> v2:172.21.15.157:3300/0 conn(0x7f0fd01587d0 msgr2=0x7f0fd00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 --2- 172.21.15.73:0/3223534911 >> v2:172.21.15.157:3300/0 conn(0x7f0fd01587d0 0x7f0fd00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd006f800 con 0x7f0fd006d0c0 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f0fc80019f0 con 0x7f0fd006d0c0 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcde00640 1 --2- 172.21.15.73:0/3223534911 >> v2:172.21.15.157:3300/0 conn(0x7f0fd01587d0 0x7f0fd00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 >> v1:172.21.15.73:6789/0 conn(0x7f0fd006d0c0 legacy=0x7f0fd015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb4003970 0x7f0fb4003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004380 0x7f0fb4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb4004f80 0x7f0fb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0070410 con 0x7f0fb4004f80 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0158bb0 con 0x7f0fb4004380 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb400bec0 con 0x7f0fb4003970 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.802+0000 7f0fce800640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb4004f80 0x7f0fb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fcde00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb4003970 0x7f0fb4003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fce800640 1 -- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb4003970 msgr2=0x7f0fb4003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fcd400640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004380 0x7f0fb4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fce800640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb4003970 0x7f0fb4003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fce800640 1 -- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004380 msgr2=0x7f0fb4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fce800640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004380 0x7f0fb4004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fce800640 1 -- 172.21.15.73:0/3223534911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4003520 con 0x7f0fb4004f80 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.803+0000 7f0fcde00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fb4003970 0x7f0fb4003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.804+0000 7f0fce800640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb4004f80 0x7f0fb400b980 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800a830 tx=0x7f0fb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d946901fea2c092 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.804+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8034630 con 0x7f0fb4004f80 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.804+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc802f070 con 0x7f0fb4004f80 2024-04-05T22:30:55.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.804+0000 7f0fcca00640 1 -- 172.21.15.73:0/3223534911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8034a30 con 0x7f0fb4004f80 2024-04-05T22:30:55.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 -- 172.21.15.73:0/3223534911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb4004f80 msgr2=0x7f0fb400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/3223534911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb4004f80 0x7f0fb400b980 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800a830 tx=0x7f0fb40057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fce800640 1 -- 172.21.15.73:0/3223534911 reap_dead start 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 -- 172.21.15.73:0/3223534911 shutdown_connections 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 -- 172.21.15.73:0/3223534911 >> 172.21.15.73:0/3223534911 conn(0x7f0fd00ce8e0 msgr2=0x7f0fd00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 -- 172.21.15.73:0/3223534911 shutdown_connections 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.805+0000 7f0fd4c00640 1 -- 172.21.15.73:0/3223534911 wait complete. 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.807+0000 7f0fd4c00640 1 Processor -- start 2024-04-05T22:30:55.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.807+0000 7f0fd4c00640 1 -- start start 2024-04-05T22:30:55.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004f80 0x7f0fd015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ac8d0 0x7f0fd00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00ad270 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00b3960 con 0x7f0fb4004f80 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.808+0000 7f0fd4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01f5c80 con 0x7f0fd00ac8d0 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49016/0 (socket says 172.21.15.73:49016) 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 -- 172.21.15.73:0/2670705630 learned_addr learned my addr 172.21.15.73:0/2670705630 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fcd400640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ac8d0 0x7f0fd00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fcde00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004f80 0x7f0fd015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 -- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ac8d0 msgr2=0x7f0fd00acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ac8d0 0x7f0fd00acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 -- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004f80 msgr2=0x7f0fd015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004f80 0x7f0fd015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:55.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.809+0000 7f0fce800640 1 -- 172.21.15.73:0/2670705630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd01f5f60 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fce800640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802cb80 tx=0x7f0fd0070380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80019e0 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8001fc0 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8034630 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd01f6240 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.810+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd01f6800 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.811+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0066af0 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.812+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f0fc8031030 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.814+0000 7f0fc6000640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 0x7f0fb006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:55.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.814+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fc80a8f00 con 0x7f0fd00b2fa0 2024-04-05T22:30:55.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.814+0000 7f0fcde00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 0x7f0fb006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.815+0000 7f0fcde00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 0x7f0fb006d8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:55.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:55.819+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0fc8079770 con 0x7f0fd00b2fa0 2024-04-05T22:30:56.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.019+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fd0068160 con 0x7f0fd00b2fa0 2024-04-05T22:30:56.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.021+0000 7f0fc6000640 1 -- 172.21.15.73:0/2670705630 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+12399 (secure 0 0 0) 0x7f0fc8045020 con 0x7f0fd00b2fa0 2024-04-05T22:30:56.021 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:56.021 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":156,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:55.061436+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.023+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 msgr2=0x7f0fb006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.023+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 0x7f0fb006d8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.023+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 msgr2=0x7f0fd00b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.023+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802cb80 tx=0x7f0fd0070380 comp rx=0 tx=0).stop 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 shutdown_connections 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0fb006b3e0 0x7f0fb006d8a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fd00b2fa0 0x7f0fd00b3420 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd00ac8d0 0x7f0fd00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 --2- 172.21.15.73:0/2670705630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb4004f80 0x7f0fd015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 >> 172.21.15.73:0/2670705630 conn(0x7f0fd00ce8e0 msgr2=0x7f0fd0069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:56.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 shutdown_connections 2024-04-05T22:30:56.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.024+0000 7f0fd4c00640 1 -- 172.21.15.73:0/2670705630 wait complete. 2024-04-05T22:30:56.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:30:56.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:30:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.247+0000 7f6f4b200640 1 Processor -- start 2024-04-05T22:30:56.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.248+0000 7f6f4b200640 1 -- start start 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f4c065e70 0x7f6f4c06bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0cd070 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0cb340 con 0x7f6f4c160310 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c0cd330 con 0x7f6f4c065e70 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4a800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6f4c0ae5e0 0x7f6f4c06acc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56584/0 (socket says 172.21.15.73:56584) 2024-04-05T22:30:56.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f4a800640 1 -- 172.21.15.73:0/1229686948 learned_addr learned my addr 172.21.15.73:0/1229686948 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600065961 0 0) 0x7f6f4c0cd070 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f2c003520 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.249+0000 7f6f49400640 1 --2- 172.21.15.73:0/1229686948 >> v2:172.21.15.157:3300/0 conn(0x7f6f4c065e70 0x7f6f4c06bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3646712138 0 0) 0x7f6f2c003520 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 >> v2:172.21.15.157:3300/0 conn(0x7f6f4c065e70 msgr2=0x7f6f4c06bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 --2- 172.21.15.73:0/1229686948 >> v2:172.21.15.157:3300/0 conn(0x7f6f4c065e70 0x7f6f4c06bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 >> v1:172.21.15.157:6789/0 conn(0x7f6f4c160310 legacy=0x7f6f4c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0ca150 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6f44001670 con 0x7f6f4c0ae5e0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 >> v1:172.21.15.73:6789/0 conn(0x7f6f4c0ae5e0 legacy=0x7f6f4c06acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c003970 0x7f6f2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 0x7f6f2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0052a0 0x7f6f2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0cd330 con 0x7f6f2c0047c0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c00c170 con 0x7f6f2c003970 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c00c330 con 0x7f6f2c0052a0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.250+0000 7f6f49400640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 0x7f6f2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f4a800640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0052a0 0x7f6f2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49400640 1 -- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0052a0 msgr2=0x7f6f2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49400640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0052a0 0x7f6f2c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49e00640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c003970 0x7f6f2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49400640 1 -- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c003970 msgr2=0x7f6f2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49400640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c003970 0x7f6f2c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f4a800640 1 -- 172.21.15.73:0/1229686948 reap_dead start 2024-04-05T22:30:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49400640 1 -- 172.21.15.73:0/1229686948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c003520 con 0x7f6f2c0047c0 2024-04-05T22:30:56.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.251+0000 7f6f49e00640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c003970 0x7f6f2c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f49400640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 0x7f6f2c004c40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0662b0 tx=0x7f6f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e127a0b45de532 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402cc40 con 0x7f6f2c0047c0 2024-04-05T22:30:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3402cda0 con 0x7f6f2c0047c0 2024-04-05T22:30:56.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f48a00640 1 -- 172.21.15.73:0/1229686948 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34034680 con 0x7f6f2c0047c0 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f4b200640 1 -- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 msgr2=0x7f6f2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.252+0000 7f6f4b200640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 0x7f6f2c004c40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0662b0 tx=0x7f6f2c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.253+0000 7f6f4b200640 1 -- 172.21.15.73:0/1229686948 shutdown_connections 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.253+0000 7f6f4b200640 1 --2- 172.21.15.73:0/1229686948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f2c0047c0 0x7f6f2c004c40 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.253+0000 7f6f4b200640 1 -- 172.21.15.73:0/1229686948 >> 172.21.15.73:0/1229686948 conn(0x7f6f4c0cba40 msgr2=0x7f6f4c0cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.253+0000 7f6f4b200640 1 -- 172.21.15.73:0/1229686948 shutdown_connections 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.253+0000 7f6f4b200640 1 -- 172.21.15.73:0/1229686948 wait complete. 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.254+0000 7f6f4b200640 1 Processor -- start 2024-04-05T22:30:56.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 -- start start 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0047c0 0x7f6f4c09c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f4c0b2020 0x7f6f4c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c09ca40 con 0x7f6f4c0b4140 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0b2bd0 con 0x7f6f4c0b2020 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.255+0000 7f6f4b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c09ef90 con 0x7f6f2c0047c0 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54282/0 (socket says 172.21.15.73:54282) 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 -- 172.21.15.73:0/3370389861 learned_addr learned my addr 172.21.15.73:0/3370389861 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 -- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0047c0 msgr2=0x7f6f4c09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0047c0 0x7f6f4c09c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 -- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f4c0b2020 msgr2=0x7f6f4c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f4c0b2020 0x7f6f4c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 -- 172.21.15.73:0/3370389861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c09f270 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f4a800640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c067660 tx=0x7f6f4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.256+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44007230 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.257+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44007390 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.257+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440047e0 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.257+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c09f550 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.257+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c09fb10 con 0x7f6f4c0b4140 2024-04-05T22:30:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.258+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c06f8a0 con 0x7f6f4c0b4140 2024-04-05T22:30:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.259+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f6f44004940 con 0x7f6f4c0b4140 2024-04-05T22:30:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.259+0000 7f6f42000640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 0x7f6f2806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.260+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f44087770 con 0x7f6f4c0b4140 2024-04-05T22:30:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.260+0000 7f6f49e00640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 0x7f6f2806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.261+0000 7f6f49e00640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 0x7f6f2806d8a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c007fd0 tx=0x7f6f3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.264+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6f440544c0 con 0x7f6f4c0b4140 2024-04-05T22:30:56.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.456+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f4c067480 con 0x7f6f4c0b4140 2024-04-05T22:30:56.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.457+0000 7f6f42000640 1 -- 172.21.15.73:0/3370389861 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f44057ae0 con 0x7f6f4c0b4140 2024-04-05T22:30:56.457 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 msgr2=0x7f6f2806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 0x7f6f2806d8a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c007fd0 tx=0x7f6f3c009040 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 msgr2=0x7f6f4c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c067660 tx=0x7f6f4400dc00 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 shutdown_connections 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2806b3e0 0x7f6f2806d8a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4c0b4140 0x7f6f4c0b45c0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f4c0b2020 0x7f6f4c0b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 --2- 172.21.15.73:0/3370389861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f2c0047c0 0x7f6f4c09c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 >> 172.21.15.73:0/3370389861 conn(0x7f6f4c0cba40 msgr2=0x7f6f4c06cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:56.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 shutdown_connections 2024-04-05T22:30:56.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.459+0000 7f6f4b200640 1 -- 172.21.15.73:0/3370389861 wait complete. 2024-04-05T22:30:56.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:30:56.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.684+0000 7f6dd2400640 1 Processor -- start 2024-04-05T22:30:56.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.684+0000 7f6dd2400640 1 -- start start 2024-04-05T22:30:56.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dd2400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dd2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dcc15d200 0x7f6dcc0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dd2400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc06aa40 con 0x7f6dcc0ca050 2024-04-05T22:30:56.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dd2400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc09d110 con 0x7f6dcc15ce20 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dd2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc06ad20 con 0x7f6dcc15d200 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dcb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dcb400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54292/0 (socket says 172.21.15.73:54292) 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.686+0000 7f6dcb400640 1 -- 172.21.15.73:0/1720605109 learned_addr learned my addr 172.21.15.73:0/1720605109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.687+0000 7f6dcb400640 1 -- 172.21.15.73:0/1720605109 >> v2:172.21.15.157:3300/0 conn(0x7f6dcc15d200 msgr2=0x7f6dcc0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.687+0000 7f6dcb400640 1 --2- 172.21.15.73:0/1720605109 >> v2:172.21.15.157:3300/0 conn(0x7f6dcc15d200 0x7f6dcc0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.687+0000 7f6dcb400640 1 -- 172.21.15.73:0/1720605109 >> v1:172.21.15.157:6789/0 conn(0x7f6dcc0ca050 legacy=0x7f6dcc160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:56.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.687+0000 7f6dcb400640 1 -- 172.21.15.73:0/1720605109 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0cde30 con 0x7f6dcc15ce20 2024-04-05T22:30:56.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.688+0000 7f6dcb400640 1 --2- 172.21.15.73:0/1720605109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc160ab0 tx=0x7f6dbc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122b7fa882e6747f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.688+0000 7f6dcaa00640 1 -- 172.21.15.73:0/1720605109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc02ca80 con 0x7f6dcc15ce20 2024-04-05T22:30:56.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.688+0000 7f6dcaa00640 1 -- 172.21.15.73:0/1720605109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc02cbe0 con 0x7f6dcc15ce20 2024-04-05T22:30:56.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.688+0000 7f6dcaa00640 1 -- 172.21.15.73:0/1720605109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0343f0 con 0x7f6dcc15ce20 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 -- 172.21.15.73:0/1720605109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 msgr2=0x7f6dcc0c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 --2- 172.21.15.73:0/1720605109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc160ab0 tx=0x7f6dbc02bee0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 -- 172.21.15.73:0/1720605109 shutdown_connections 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 --2- 172.21.15.73:0/1720605109 >> v2:172.21.15.157:3300/0 conn(0x7f6dcc15d200 0x7f6dcc0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 --2- 172.21.15.73:0/1720605109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc0c9b10 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 -- 172.21.15.73:0/1720605109 >> 172.21.15.73:0/1720605109 conn(0x7f6dcc0cf190 msgr2=0x7f6dcc0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 -- 172.21.15.73:0/1720605109 shutdown_connections 2024-04-05T22:30:56.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.689+0000 7f6dd2400640 1 -- 172.21.15.73:0/1720605109 wait complete. 2024-04-05T22:30:56.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.691+0000 7f6dd2400640 1 Processor -- start 2024-04-05T22:30:56.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.691+0000 7f6dd2400640 1 -- start start 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.692+0000 7f6dd2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0ca050 0x7f6dcc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dd2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dd2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 0x7f6dcc0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dd2400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc156c30 con 0x7f6dcc15ce20 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dd2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c6a90 con 0x7f6dcc15d200 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dd2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c6d70 con 0x7f6dcc0ca050 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54300/0 (socket says 172.21.15.73:54300) 2024-04-05T22:30:56.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 -- 172.21.15.73:0/3054334715 learned_addr learned my addr 172.21.15.73:0/3054334715 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dcb400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0ca050 0x7f6dcc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 -- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0ca050 msgr2=0x7f6dcc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dcbe00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 0x7f6dcc0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0ca050 0x7f6dcc15b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 -- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 msgr2=0x7f6dcc0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 0x7f6dcc0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dc2a00640 1 -- 172.21.15.73:0/3054334715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009b30 con 0x7f6dcc15ce20 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.693+0000 7f6dcbe00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 0x7f6dcc0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dc2a00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b410 tx=0x7f6db400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4011830 con 0x7f6dcc15ce20 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db40119b0 con 0x7f6dcc15ce20 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4011d60 con 0x7f6dcc15ce20 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc0c7050 con 0x7f6dcc15ce20 2024-04-05T22:30:56.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.694+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0c7610 con 0x7f6dcc15ce20 2024-04-05T22:30:56.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.695+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc0c7800 con 0x7f6dcc15ce20 2024-04-05T22:30:56.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.696+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f6db401a400 con 0x7f6dcc15ce20 2024-04-05T22:30:56.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.697+0000 7f6dc3e00640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 0x7f6da806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:56.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.697+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6db4087b80 con 0x7f6dcc15ce20 2024-04-05T22:30:56.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.698+0000 7f6dcb400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 0x7f6da806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:56.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.699+0000 7f6dcb400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 0x7f6da806d8d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc035000 tx=0x7f6dbc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:56.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.701+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6db4054960 con 0x7f6dcc15ce20 2024-04-05T22:30:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.898+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6dcc068160 con 0x7f6dcc15ce20 2024-04-05T22:30:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.898+0000 7f6dc3e00640 1 -- 172.21.15.73:0/3054334715 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v156) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6db4015060 con 0x7f6dcc15ce20 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:30:56.899 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:30:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 msgr2=0x7f6da806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 0x7f6da806d8d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc035000 tx=0x7f6dbc030000 comp rx=0 tx=0).stop 2024-04-05T22:30:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 msgr2=0x7f6dcc1566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b410 tx=0x7f6db400b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 shutdown_connections 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6da806b410 0x7f6da806d8d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc15d200 0x7f6dcc0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dcc15ce20 0x7f6dcc1566f0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 --2- 172.21.15.73:0/3054334715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0ca050 0x7f6dcc15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 >> 172.21.15.73:0/3054334715 conn(0x7f6dcc0cf190 msgr2=0x7f6dcc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 shutdown_connections 2024-04-05T22:30:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:56.901+0000 7f6dd2400640 1 -- 172.21.15.73:0/3054334715 wait complete. 2024-04-05T22:30:56.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:30:56.921 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:30:57.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.130+0000 7fd312600640 1 Processor -- start 2024-04-05T22:30:57.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.131+0000 7fd312600640 1 -- start start 2024-04-05T22:30:57.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd312600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd30c1605d0 0x7fd30c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd312600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd30c0d1800 0x7fd30c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd312600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0af6d0 con 0x7fd30c0d09d0 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd312600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0b22b0 con 0x7fd30c1605d0 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd312600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0b2590 con 0x7fd30c0d1800 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd30be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd30c0d09d0 0x7fd30c0d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56588/0 (socket says 172.21.15.73:56588) 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd30be00640 1 -- 172.21.15.73:0/4065509070 learned_addr learned my addr 172.21.15.73:0/4065509070 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.132+0000 7fd311200640 1 --2- 172.21.15.73:0/4065509070 >> v2:172.21.15.73:3300/0 conn(0x7fd30c1605d0 0x7fd30c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.133+0000 7fd311c00640 1 --2- 172.21.15.73:0/4065509070 >> v2:172.21.15.157:3300/0 conn(0x7fd30c0d1800 0x7fd30c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.133+0000 7fd311200640 1 -- 172.21.15.73:0/4065509070 >> v2:172.21.15.157:3300/0 conn(0x7fd30c0d1800 msgr2=0x7fd30c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:57.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.133+0000 7fd311200640 1 --2- 172.21.15.73:0/4065509070 >> v2:172.21.15.157:3300/0 conn(0x7fd30c0d1800 0x7fd30c0b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.133+0000 7fd311200640 1 -- 172.21.15.73:0/4065509070 >> v1:172.21.15.73:6789/0 conn(0x7fd30c0d09d0 legacy=0x7fd30c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.133+0000 7fd311200640 1 -- 172.21.15.73:0/4065509070 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0b2870 con 0x7fd30c1605d0 2024-04-05T22:30:57.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.134+0000 7fd311200640 1 --2- 172.21.15.73:0/4065509070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c0d3630 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0cd110 tx=0x7fd2fc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e17d8d36711f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd30b400640 1 -- 172.21.15.73:0/4065509070 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc02cd00 con 0x7fd30c1605d0 2024-04-05T22:30:57.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd30b400640 1 -- 172.21.15.73:0/4065509070 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc02ce60 con 0x7fd30c1605d0 2024-04-05T22:30:57.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd30b400640 1 -- 172.21.15.73:0/4065509070 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc034660 con 0x7fd30c1605d0 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 -- 172.21.15.73:0/4065509070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 msgr2=0x7fd30c0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 --2- 172.21.15.73:0/4065509070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c0d3630 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0cd110 tx=0x7fd2fc02bf50 comp rx=0 tx=0).stop 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 -- 172.21.15.73:0/4065509070 shutdown_connections 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 --2- 172.21.15.73:0/4065509070 >> v2:172.21.15.157:3300/0 conn(0x7fd30c0d1800 0x7fd30c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 --2- 172.21.15.73:0/4065509070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c0d3630 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.135+0000 7fd312600640 1 -- 172.21.15.73:0/4065509070 >> 172.21.15.73:0/4065509070 conn(0x7fd30c0d2b10 msgr2=0x7fd30c0d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.136+0000 7fd312600640 1 -- 172.21.15.73:0/4065509070 shutdown_connections 2024-04-05T22:30:57.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.136+0000 7fd312600640 1 -- 172.21.15.73:0/4065509070 wait complete. 2024-04-05T22:30:57.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.137+0000 7fd312600640 1 Processor -- start 2024-04-05T22:30:57.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.137+0000 7fd312600640 1 -- start start 2024-04-05T22:30:57.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd30c0d09d0 0x7fd30c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd30c0d1800 0x7fd30c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c8c20 con 0x7fd30c1605d0 2024-04-05T22:30:57.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c9710 con 0x7fd30c0d09d0 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.138+0000 7fd312600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c8e70 con 0x7fd30c0d1800 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54314/0 (socket says 172.21.15.73:54314) 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 -- 172.21.15.73:0/982181569 learned_addr learned my addr 172.21.15.73:0/982181569 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 -- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd30c0d1800 msgr2=0x7fd30c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd30c0d1800 0x7fd30c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 -- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd30c0d09d0 msgr2=0x7fd30c14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd30c0d09d0 0x7fd30c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 -- 172.21.15.73:0/982181569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc009a70 con 0x7fd30c1605d0 2024-04-05T22:30:57.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.139+0000 7fd311c00640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c5c0 tx=0x7fd30400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.140+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304004220 con 0x7fd30c1605d0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.140+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3040043a0 con 0x7fd30c1605d0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.140+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304003850 con 0x7fd30c1605d0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.140+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0c9150 con 0x7fd30c1605d0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.140+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0cad60 con 0x7fd30c1605d0 2024-04-05T22:30:57.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.141+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0caf80 con 0x7fd30c1605d0 2024-04-05T22:30:57.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.142+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fd3040039b0 con 0x7fd30c1605d0 2024-04-05T22:30:57.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.143+0000 7fd308c00640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 0x7fd2e806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.143+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd304087820 con 0x7fd30c1605d0 2024-04-05T22:30:57.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.144+0000 7fd311200640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 0x7fd2e806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.145+0000 7fd311200640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 0x7fd2e806d8a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc036000 tx=0x7fd2fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.147+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd304054580 con 0x7fd30c1605d0 2024-04-05T22:30:57.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.327+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd30c04e160 con 0x7fd30c1605d0 2024-04-05T22:30:57.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.425+0000 7fd308c00640 1 -- 172.21.15.73:0/982181569 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v157) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd30401e0b0 con 0x7fd30c1605d0 2024-04-05T22:30:57.425 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:30:57.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 msgr2=0x7fd2e806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 0x7fd2e806d8a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc036000 tx=0x7fd2fc02e000 comp rx=0 tx=0).stop 2024-04-05T22:30:57.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 msgr2=0x7fd30c14f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c5c0 tx=0x7fd30400ca80 comp rx=0 tx=0).stop 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 shutdown_connections 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd2e806b3e0 0x7fd2e806d8a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd30c1605d0 0x7fd30c14f650 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.429+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd30c0d1800 0x7fd30c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.430+0000 7fd312600640 1 --2- 172.21.15.73:0/982181569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd30c0d09d0 0x7fd30c14cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.430+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 >> 172.21.15.73:0/982181569 conn(0x7fd30c0d2b10 msgr2=0x7fd30c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.430+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 shutdown_connections 2024-04-05T22:30:57.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.430+0000 7fd312600640 1 -- 172.21.15.73:0/982181569 wait complete. 2024-04-05T22:30:57.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:30:57.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.650+0000 7f6837000640 1 Processor -- start 2024-04-05T22:30:57.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.650+0000 7f6837000640 1 -- start start 2024-04-05T22:30:57.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f6837000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f683015dd90 0x7f6830167d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f6837000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683015e170 con 0x7f683015aa70 2024-04-05T22:30:57.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f6837000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683015ae50 con 0x7f6830071670 2024-04-05T22:30:57.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f6837000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683015e3b0 con 0x7f683015dd90 2024-04-05T22:30:57.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f682fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f683015aa70 0x7f6830070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56592/0 (socket says 172.21.15.73:56592) 2024-04-05T22:30:57.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f682fe00640 1 -- 172.21.15.73:0/3373973356 learned_addr learned my addr 172.21.15.73:0/3373973356 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:57.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.652+0000 7f6835000640 1 --2- 172.21.15.73:0/3373973356 >> v2:172.21.15.73:3300/0 conn(0x7f683015dd90 0x7f6830167d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.653+0000 7f682ea00640 1 -- 172.21.15.73:0/3373973356 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642321737 0 0) 0x7f6820001490 con 0x7f683015aa70 2024-04-05T22:30:57.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.653+0000 7f682ea00640 1 -- 172.21.15.73:0/3373973356 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6814003520 con 0x7f683015aa70 2024-04-05T22:30:57.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.653+0000 7f6835000640 1 -- 172.21.15.73:0/3373973356 >> v1:172.21.15.157:6789/0 conn(0x7f6830071670 legacy=0x7f683015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.653+0000 7f6835000640 1 -- 172.21.15.73:0/3373973356 >> v1:172.21.15.73:6789/0 conn(0x7f683015aa70 legacy=0x7f6830070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.653+0000 7f6835000640 1 -- 172.21.15.73:0/3373973356 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683006ca50 con 0x7f683015dd90 2024-04-05T22:30:57.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.654+0000 7f6835000640 1 --2- 172.21.15.73:0/3373973356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830167d60 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f68280097a0 tx=0x7f682802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114de46f5c696a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.654+0000 7f682ea00640 1 -- 172.21.15.73:0/3373973356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828030610 con 0x7f683015dd90 2024-04-05T22:30:57.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.654+0000 7f682ea00640 1 -- 172.21.15.73:0/3373973356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828030770 con 0x7f683015dd90 2024-04-05T22:30:57.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.654+0000 7f682ea00640 1 -- 172.21.15.73:0/3373973356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828030b00 con 0x7f683015dd90 2024-04-05T22:30:57.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 -- 172.21.15.73:0/3373973356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 msgr2=0x7f6830167d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 --2- 172.21.15.73:0/3373973356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830167d60 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f68280097a0 tx=0x7f682802b960 comp rx=0 tx=0).stop 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 -- 172.21.15.73:0/3373973356 shutdown_connections 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 --2- 172.21.15.73:0/3373973356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830167d60 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 -- 172.21.15.73:0/3373973356 >> 172.21.15.73:0/3373973356 conn(0x7f6830069cb0 msgr2=0x7f683006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 -- 172.21.15.73:0/3373973356 shutdown_connections 2024-04-05T22:30:57.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.655+0000 7f6837000640 1 -- 172.21.15.73:0/3373973356 wait complete. 2024-04-05T22:30:57.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.657+0000 7f6837000640 1 Processor -- start 2024-04-05T22:30:57.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.657+0000 7f6837000640 1 -- start start 2024-04-05T22:30:57.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.658+0000 7f6837000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830071670 0x7f68301511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6837000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683015aa70 0x7f683014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6837000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6837000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830159a20 con 0x7f683015dd90 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6837000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683014cb10 con 0x7f683015aa70 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6837000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830158e10 con 0x7f6830071670 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54332/0 (socket says 172.21.15.73:54332) 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 -- 172.21.15.73:0/2537862513 learned_addr learned my addr 172.21.15.73:0/2537862513 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f682fe00640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830071670 0x7f68301511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f682f400640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683015aa70 0x7f683014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 -- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830071670 msgr2=0x7f68301511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830071670 0x7f68301511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 -- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683015aa70 msgr2=0x7f683014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:57.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683015aa70 0x7f683014c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:57.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.659+0000 7f6835000640 1 -- 172.21.15.73:0/2537862513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009400 con 0x7f683015dd90 2024-04-05T22:30:57.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6835000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f68280098c0 tx=0x7f68280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682803e070 con 0x7f683015dd90 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828030e10 con 0x7f683015dd90 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828039bd0 con 0x7f683015dd90 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301590f0 con 0x7f683015dd90 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.660+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301596b0 con 0x7f683015dd90 2024-04-05T22:30:57.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.661+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830068160 con 0x7f683015dd90 2024-04-05T22:30:57.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.663+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f6828039d30 con 0x7f683015dd90 2024-04-05T22:30:57.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.663+0000 7f6827e00640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 0x7f680c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:57.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.663+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f68280aa830 con 0x7f683015dd90 2024-04-05T22:30:57.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.664+0000 7f682fe00640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 0x7f680c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:57.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.665+0000 7f682fe00640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 0x7f680c06d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6820005c00 tx=0x7f6820009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:57.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.667+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6828077610 con 0x7f683015dd90 2024-04-05T22:30:57.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:57.865+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f683015f740 con 0x7f683015dd90 2024-04-05T22:30:58.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.446+0000 7f6827e00640 1 -- 172.21.15.73:0/2537862513 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v158) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6828034070 con 0x7f683015dd90 2024-04-05T22:30:58.449 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:30:58.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 msgr2=0x7f680c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 0x7f680c06d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6820005c00 tx=0x7f6820009040 comp rx=0 tx=0).stop 2024-04-05T22:30:58.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 msgr2=0x7f6830158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f68280098c0 tx=0x7f68280046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 shutdown_connections 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f680c06b3e0 0x7f680c06d8a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683015dd90 0x7f6830158520 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683015aa70 0x7f683014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 --2- 172.21.15.73:0/2537862513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830071670 0x7f68301511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.454+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 >> 172.21.15.73:0/2537862513 conn(0x7f6830069cb0 msgr2=0x7f68300b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:58.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.456+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 shutdown_connections 2024-04-05T22:30:58.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.456+0000 7f6837000640 1 -- 172.21.15.73:0/2537862513 wait complete. 2024-04-05T22:30:58.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:30:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.667+0000 7f3aab000640 1 Processor -- start 2024-04-05T22:30:58.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.667+0000 7f3aab000640 1 -- start start 2024-04-05T22:30:58.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aab000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3aa415abe0 0x7f3aa40712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aab000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 0x7f3aa406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aab000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa406bec0 con 0x7f3aa406ba70 2024-04-05T22:30:58.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aab000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa415afc0 con 0x7f3aa415abe0 2024-04-05T22:30:58.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aab000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa406f5b0 con 0x7f3aa406c9a0 2024-04-05T22:30:58.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.669+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 0x7f3aa406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa9000640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3aa406ba70 0x7f3aa40cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:41996/0 (socket says 172.21.15.73:41996) 2024-04-05T22:30:58.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa9000640 1 -- 172.21.15.73:0/2432075749 learned_addr learned my addr 172.21.15.73:0/2432075749 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:58.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa3e00640 1 --2- 172.21.15.73:0/2432075749 >> v2:172.21.15.157:3300/0 conn(0x7f3aa415abe0 0x7f3aa40712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa3400640 1 -- 172.21.15.73:0/2432075749 >> v2:172.21.15.157:3300/0 conn(0x7f3aa415abe0 msgr2=0x7f3aa40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa3400640 1 --2- 172.21.15.73:0/2432075749 >> v2:172.21.15.157:3300/0 conn(0x7f3aa415abe0 0x7f3aa40712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa3400640 1 -- 172.21.15.73:0/2432075749 >> v1:172.21.15.157:6789/0 conn(0x7f3aa406ba70 legacy=0x7f3aa40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.670+0000 7f3aa3400640 1 -- 172.21.15.73:0/2432075749 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa406f890 con 0x7f3aa406c9a0 2024-04-05T22:30:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.672+0000 7f3aa3400640 1 --2- 172.21.15.73:0/2432075749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 0x7f3aa406b530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a9402bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25fe71f8300a15a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.672+0000 7f3aa2a00640 1 -- 172.21.15.73:0/2432075749 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402cb40 con 0x7f3aa406c9a0 2024-04-05T22:30:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.672+0000 7f3aa2a00640 1 -- 172.21.15.73:0/2432075749 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9402cca0 con 0x7f3aa406c9a0 2024-04-05T22:30:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.672+0000 7f3aa2a00640 1 -- 172.21.15.73:0/2432075749 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94034660 con 0x7f3aa406c9a0 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 -- 172.21.15.73:0/2432075749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 msgr2=0x7f3aa406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 --2- 172.21.15.73:0/2432075749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 0x7f3aa406b530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a9402bd90 comp rx=0 tx=0).stop 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 -- 172.21.15.73:0/2432075749 shutdown_connections 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 --2- 172.21.15.73:0/2432075749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa406c9a0 0x7f3aa406b530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 --2- 172.21.15.73:0/2432075749 >> v2:172.21.15.157:3300/0 conn(0x7f3aa415abe0 0x7f3aa40712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 -- 172.21.15.73:0/2432075749 >> 172.21.15.73:0/2432075749 conn(0x7f3aa4069cb0 msgr2=0x7f3aa406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 -- 172.21.15.73:0/2432075749 shutdown_connections 2024-04-05T22:30:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.673+0000 7f3aab000640 1 -- 172.21.15.73:0/2432075749 wait complete. 2024-04-05T22:30:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.675+0000 7f3aab000640 1 Processor -- start 2024-04-05T22:30:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.675+0000 7f3aab000640 1 -- start start 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aa406c9a0 0x7f3aa4147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 0x7f3aa40cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41f6ab0 con 0x7f3aa406ba70 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4148820 con 0x7f3aa406c9a0 2024-04-05T22:30:58.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aab000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4156b30 con 0x7f3aa415abe0 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aa3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aa3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54344/0 (socket says 172.21.15.73:54344) 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aa3e00640 1 -- 172.21.15.73:0/3514645837 learned_addr learned my addr 172.21.15.73:0/3514645837 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aa3400640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aa406c9a0 0x7f3aa4147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.676+0000 7f3aa9000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 0x7f3aa40cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 -- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 msgr2=0x7f3aa40cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 0x7f3aa40cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 -- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aa406c9a0 msgr2=0x7f3aa4147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aa406c9a0 0x7f3aa4147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 -- 172.21.15.73:0/3514645837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009a70 con 0x7f3aa406ba70 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa9000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 0x7f3aa40cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aa3e00640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e870 tx=0x7f3a8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0094f0 con 0x7f3aa406ba70 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c00c940 con 0x7f3aa406ba70 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa4156850 con 0x7f3aa406ba70 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00ccf0 con 0x7f3aa406ba70 2024-04-05T22:30:58.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.677+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa4151ae0 con 0x7f3aa406ba70 2024-04-05T22:30:58.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.679+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa40adf50 con 0x7f3aa406ba70 2024-04-05T22:30:58.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.680+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f3a8c013470 con 0x7f3aa406ba70 2024-04-05T22:30:58.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.681+0000 7f3a9be00640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 0x7f3a8006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:58.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.681+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3a8c087b10 con 0x7f3aa406ba70 2024-04-05T22:30:58.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.682+0000 7f3aa3400640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 0x7f3a8006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:58.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.683+0000 7f3aa3400640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 0x7f3a8006d8a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a94036000 tx=0x7f3a94030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:58.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.686+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3a8c0548f0 con 0x7f3aa406ba70 2024-04-05T22:30:58.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.912+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3aa4068160 con 0x7f3aa406ba70 2024-04-05T22:30:58.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.925+0000 7f3a9be00640 1 -- 172.21.15.73:0/3514645837 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3a8c0170c0 con 0x7f3aa406ba70 2024-04-05T22:30:58.926 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 24 and data pool 25 2024-04-05T22:30:58.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.930+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 msgr2=0x7f3a8006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.930+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 0x7f3a8006d8a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a94036000 tx=0x7f3a94030000 comp rx=0 tx=0).stop 2024-04-05T22:30:58.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.930+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 msgr2=0x7f3aa41f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:58.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.930+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e870 tx=0x7f3a8c00ed30 comp rx=0 tx=0).stop 2024-04-05T22:30:58.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 shutdown_connections 2024-04-05T22:30:58.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3a8006b3e0 0x7f3a8006d8a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aa415abe0 0x7f3aa40cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aa406c9a0 0x7f3aa4147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 --2- 172.21.15.73:0/3514645837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa406ba70 0x7f3aa41f6570 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:58.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.933+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 >> 172.21.15.73:0/3514645837 conn(0x7f3aa4069cb0 msgr2=0x7f3aa40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:58.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.937+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 shutdown_connections 2024-04-05T22:30:58.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:58.938+0000 7f3aab000640 1 -- 172.21.15.73:0/3514645837 wait complete. 2024-04-05T22:30:58.968 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.138+0000 7ff3b2600640 1 Processor -- start 2024-04-05T22:30:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.138+0000 7ff3b2600640 1 -- start start 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.139+0000 7ff3b2600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.139+0000 7ff3b2600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06f5b0 con 0x7ff3ac15e850 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.139+0000 7ff3b2600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac070410 con 0x7ff3ac1587d0 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.139+0000 7ff3b2600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac06d500 con 0x7ff3ac06cb80 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54352/0 (socket says 172.21.15.73:54352) 2024-04-05T22:30:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 -- 172.21.15.73:0/14406661 learned_addr learned my addr 172.21.15.73:0/14406661 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:59.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 -- 172.21.15.73:0/14406661 >> v1:172.21.15.157:6789/0 conn(0x7ff3ac1587d0 legacy=0x7ff3ac0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:59.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 -- 172.21.15.73:0/14406661 >> v1:172.21.15.73:6789/0 conn(0x7ff3ac15e850 legacy=0x7ff3ac0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3a2a00640 1 -- 172.21.15.73:0/14406661 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0ccef0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.140+0000 7ff3aaa00640 1 -- 172.21.15.73:0/14406661 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501974594 0 0) 0x7ff3ac06f5b0 con 0x7ff3ac15e850 2024-04-05T22:30:59.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.141+0000 7ff3a2a00640 1 --2- 172.21.15.73:0/14406661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff394009870 tx=0x7ff39402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81da4a40bd1ac8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.141+0000 7ff3aaa00640 1 -- 172.21.15.73:0/14406661 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394030860 con 0x7ff3ac06cb80 2024-04-05T22:30:59.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3aaa00640 1 -- 172.21.15.73:0/14406661 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3940309c0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3aaa00640 1 -- 172.21.15.73:0/14406661 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394030d50 con 0x7ff3ac06cb80 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 -- 172.21.15.73:0/14406661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 msgr2=0x7ff3ac06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 --2- 172.21.15.73:0/14406661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff394009870 tx=0x7ff39402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 -- 172.21.15.73:0/14406661 shutdown_connections 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 --2- 172.21.15.73:0/14406661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac06cfc0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 -- 172.21.15.73:0/14406661 >> 172.21.15.73:0/14406661 conn(0x7ff3ac0ce8e0 msgr2=0x7ff3ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 -- 172.21.15.73:0/14406661 shutdown_connections 2024-04-05T22:30:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.142+0000 7ff3b2600640 1 -- 172.21.15.73:0/14406661 wait complete. 2024-04-05T22:30:59.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.144+0000 7ff3b2600640 1 Processor -- start 2024-04-05T22:30:59.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.144+0000 7ff3b2600640 1 -- start start 2024-04-05T22:30:59.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac1587d0 0x7ff3ac0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15e850 0x7ff3ac15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1f2570 con 0x7ff3ac06cb80 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0ae770 con 0x7ff3ac15e850 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3b2600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0aea50 con 0x7ff3ac1587d0 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54358/0 (socket says 172.21.15.73:54358) 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 -- 172.21.15.73:0/1989024529 learned_addr learned my addr 172.21.15.73:0/1989024529 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 -- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac1587d0 msgr2=0x7ff3ac0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac1587d0 0x7ff3ac0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 -- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15e850 msgr2=0x7ff3ac15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15e850 0x7ff3ac15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.145+0000 7ff3ab400640 1 -- 172.21.15.73:0/1989024529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940094d0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3ab400640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c430 tx=0x7ff39c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c011790 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c0118f0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac0aed30 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c011ce0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.146+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac0af7a0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.147+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac066af0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.148+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7ff39c01c460 con 0x7ff3ac06cb80 2024-04-05T22:30:59.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.149+0000 7ff3a3e00640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 0x7ff38806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.149+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff39c0879c0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.150+0000 7ff3a2a00640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 0x7ff38806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.151+0000 7ff3a2a00640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 0x7ff38806d8a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff394036000 tx=0x7ff39403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.153+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff39c0547a0 con 0x7ff3ac06cb80 2024-04-05T22:30:59.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.346+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3ac0af250 con 0x7ff3ac06cb80 2024-04-05T22:30:59.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.349+0000 7ff3a3e00640 1 -- 172.21.15.73:0/1989024529 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16124 (secure 0 0 0) 0x7ff3ac0af250 con 0x7ff3ac06cb80 2024-04-05T22:30:59.349 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:59.349 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":159,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:58.914551+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.351+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 msgr2=0x7ff38806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.351+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 0x7ff38806d8a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff394036000 tx=0x7ff39403b040 comp rx=0 tx=0).stop 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.351+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 msgr2=0x7ff3ac1f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.351+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c430 tx=0x7ff39c00c8f0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 shutdown_connections 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff38806b3e0 0x7ff38806d8a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac15e850 0x7ff3ac15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3ac1587d0 0x7ff3ac0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 --2- 172.21.15.73:0/1989024529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3ac06cb80 0x7ff3ac1f2030 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 >> 172.21.15.73:0/1989024529 conn(0x7ff3ac0ce8e0 msgr2=0x7ff3ac0ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:59.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 shutdown_connections 2024-04-05T22:30:59.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.352+0000 7ff3b2600640 1 -- 172.21.15.73:0/1989024529 wait complete. 2024-04-05T22:30:59.369 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:30:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.566+0000 7f53dc400640 1 Processor -- start 2024-04-05T22:30:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.567+0000 7f53dc400640 1 -- start start 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53dc400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 0x7f53d40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53dc400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d40ccd20 0x7f53d406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53dc400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d406f5b0 con 0x7f53d406d0c0 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53dc400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d4070410 con 0x7f53d40ccd20 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53dc400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4158bb0 con 0x7f53d41587d0 2024-04-05T22:30:59.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53d9000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d40ccd20 0x7f53d406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53da400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f53d406d0c0 0x7f53d415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56614/0 (socket says 172.21.15.73:56614) 2024-04-05T22:30:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.568+0000 7f53da400640 1 -- 172.21.15.73:0/3541135680 learned_addr learned my addr 172.21.15.73:0/3541135680 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9000640 1 -- 172.21.15.73:0/3541135680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 msgr2=0x7f53d40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9a00640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 0x7f53d40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9000640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 0x7f53d40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9000640 1 -- 172.21.15.73:0/3541135680 >> v1:172.21.15.73:6789/0 conn(0x7f53d406d0c0 legacy=0x7f53d415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d3e00640 1 -- 172.21.15.73:0/3541135680 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033674733 0 0) 0x7f53d406f5b0 con 0x7f53d406d0c0 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9000640 1 -- 172.21.15.73:0/3541135680 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d406f800 con 0x7f53d40ccd20 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.569+0000 7f53d9a00640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 0x7f53d40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53d9000640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40ccd20 0x7f53d406cb80 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f53c00097a0 tx=0x7f53c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4396132ed533a2dd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53d3e00640 1 -- 172.21.15.73:0/3541135680 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0031830 con 0x7f53d40ccd20 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53d3e00640 1 -- 172.21.15.73:0/3541135680 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0031990 con 0x7f53d40ccd20 2024-04-05T22:30:59.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53d3e00640 1 -- 172.21.15.73:0/3541135680 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0031d20 con 0x7f53d40ccd20 2024-04-05T22:30:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53dc400640 1 -- 172.21.15.73:0/3541135680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40ccd20 msgr2=0x7f53d406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.570+0000 7f53dc400640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40ccd20 0x7f53d406cb80 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f53c00097a0 tx=0x7f53c002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 -- 172.21.15.73:0/3541135680 shutdown_connections 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40ccd20 0x7f53d406cb80 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 --2- 172.21.15.73:0/3541135680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d41587d0 0x7f53d40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 -- 172.21.15.73:0/3541135680 >> 172.21.15.73:0/3541135680 conn(0x7f53d40ce8e0 msgr2=0x7f53d40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 -- 172.21.15.73:0/3541135680 shutdown_connections 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.571+0000 7f53dc400640 1 -- 172.21.15.73:0/3541135680 wait complete. 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.572+0000 7f53dc400640 1 Processor -- start 2024-04-05T22:30:59.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.572+0000 7f53dc400640 1 -- start start 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d406d0c0 0x7f53d41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d40ad230 0x7f53d414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41f6ba0 con 0x7f53d41587d0 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41f6020 con 0x7f53d40ad230 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.573+0000 7f53dc400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41f6300 con 0x7f53d406d0c0 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54372/0 (socket says 172.21.15.73:54372) 2024-04-05T22:30:59.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 -- 172.21.15.73:0/4056803093 learned_addr learned my addr 172.21.15.73:0/4056803093 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 -- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d406d0c0 msgr2=0x7f53d41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d406d0c0 0x7f53d41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 -- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d40ad230 msgr2=0x7f53d414c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d40ad230 0x7f53d414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 -- 172.21.15.73:0/4056803093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0009400 con 0x7f53d41587d0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.574+0000 7f53d9000640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f53c002bee0 tx=0x7f53c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.575+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c003ba70 con 0x7f53d41587d0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.575+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c003bbf0 con 0x7f53d41587d0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.575+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00453f0 con 0x7f53d41587d0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.575+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d41f65e0 con 0x7f53d41587d0 2024-04-05T22:30:59.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.575+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d41f2c60 con 0x7f53d41587d0 2024-04-05T22:30:59.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.576+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4066af0 con 0x7f53d41587d0 2024-04-05T22:30:59.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.577+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f53c004e450 con 0x7f53d41587d0 2024-04-05T22:30:59.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.578+0000 7f53d1600640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 0x7f53b406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:30:59.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.578+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53c00a9cc0 con 0x7f53d41587d0 2024-04-05T22:30:59.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.578+0000 7f53d9a00640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 0x7f53b406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:30:59.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.579+0000 7f53d9a00640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 0x7f53b406d8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006fd0 tx=0x7f53c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:30:59.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.582+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f53c0076aa0 con 0x7f53d41587d0 2024-04-05T22:30:59.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.761+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53d40b1a00 con 0x7f53d41587d0 2024-04-05T22:30:59.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.762+0000 7f53d1600640 1 -- 172.21.15.73:0/4056803093 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16124 (secure 0 0 0) 0x7f53d40b1a00 con 0x7f53d41587d0 2024-04-05T22:30:59.763 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:30:59.763 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":159,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:58.914551+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 msgr2=0x7f53b406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 0x7f53b406d8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006fd0 tx=0x7f53c8008040 comp rx=0 tx=0).stop 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 msgr2=0x7f53d41f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f53c002bee0 tx=0x7f53c00046c0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 shutdown_connections 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f53b406b3e0 0x7f53b406d8a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d40ad230 0x7f53d414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d41587d0 0x7f53d41f2720 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 --2- 172.21.15.73:0/4056803093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53d406d0c0 0x7f53d41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:30:59.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 >> 172.21.15.73:0/4056803093 conn(0x7f53d40ce8e0 msgr2=0x7f53d40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:30:59.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 shutdown_connections 2024-04-05T22:30:59.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:30:59.765+0000 7f53dc400640 1 -- 172.21.15.73:0/4056803093 wait complete. 2024-04-05T22:30:59.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:31:00.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.001+0000 7ff283c00640 1 Processor -- start 2024-04-05T22:31:00.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.002+0000 7ff283c00640 1 -- start start 2024-04-05T22:31:00.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff283c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff27c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff283c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff27c06ba70 0x7ff27c164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff283c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c06bec0 con 0x7ff27c15abe0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff283c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c15afc0 con 0x7ff27c06ba70 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff283c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c06f5b0 con 0x7ff27c06c9a0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff281200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff27c15abe0 0x7ff27c0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56624/0 (socket says 172.21.15.73:56624) 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.003+0000 7ff281200640 1 -- 172.21.15.73:0/342814175 learned_addr learned my addr 172.21.15.73:0/342814175 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310891549 0 0) 0x7ff26c001110 con 0x7ff27c15abe0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff260003520 con 0x7ff27c15abe0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff273e00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff27c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff281c00640 1 --2- 172.21.15.73:0/342814175 >> v2:172.21.15.157:3300/0 conn(0x7ff27c06ba70 0x7ff27c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2319569801 0 0) 0x7ff260003520 con 0x7ff27c15abe0 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 msgr2=0x7ff27c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff27c06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 >> v2:172.21.15.157:3300/0 conn(0x7ff27c06ba70 msgr2=0x7ff27c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 --2- 172.21.15.73:0/342814175 >> v2:172.21.15.157:3300/0 conn(0x7ff27c06ba70 0x7ff27c164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c06f890 con 0x7ff27c15abe0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff26c001800 con 0x7ff27c15abe0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 >> v1:172.21.15.73:6789/0 conn(0x7ff27c15abe0 legacy=0x7ff27c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff260003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260003fa0 0x7ff2600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff260004d00 0x7ff260005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c15afc0 con 0x7ff260004d00 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c06f5b0 con 0x7ff260003fa0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.004+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26000bbf0 con 0x7ff27c06c9a0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff260004d00 0x7ff260005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff273e00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260003fa0 0x7ff2600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281200640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff260003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 -- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 msgr2=0x7ff260003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff260003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 -- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260003fa0 msgr2=0x7ff2600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:00.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff260003fa0 0x7ff2600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281c00640 1 -- 172.21.15.73:0/342814175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260003520 con 0x7ff260004d00 2024-04-05T22:31:00.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.005+0000 7ff281200640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff27c06c9a0 0x7ff260003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.006+0000 7ff281c00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff260004d00 0x7ff260005180 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff274007360 tx=0x7ff2600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffaf5beaa4d2f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27402ccf0 con 0x7ff260004d00 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27402ce50 con 0x7ff260004d00 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff27be00640 1 -- 172.21.15.73:0/342814175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274034660 con 0x7ff260004d00 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 -- 172.21.15.73:0/342814175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff260004d00 msgr2=0x7ff260005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 --2- 172.21.15.73:0/342814175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff260004d00 0x7ff260005180 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff274007360 tx=0x7ff2600057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff281c00640 1 -- 172.21.15.73:0/342814175 reap_dead start 2024-04-05T22:31:00.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 -- 172.21.15.73:0/342814175 shutdown_connections 2024-04-05T22:31:00.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 -- 172.21.15.73:0/342814175 >> 172.21.15.73:0/342814175 conn(0x7ff27c069cb0 msgr2=0x7ff27c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:00.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 -- 172.21.15.73:0/342814175 shutdown_connections 2024-04-05T22:31:00.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.007+0000 7ff283c00640 1 -- 172.21.15.73:0/342814175 wait complete. 2024-04-05T22:31:00.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.009+0000 7ff283c00640 1 Processor -- start 2024-04-05T22:31:00.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.009+0000 7ff283c00640 1 -- start start 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff260004d00 0x7ff27c0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff27c150eb0 0x7ff27c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c150540 con 0x7ff27c154130 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c151ba0 con 0x7ff27c150eb0 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff283c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c151870 con 0x7ff260004d00 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff273e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff273e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54386/0 (socket says 172.21.15.73:54386) 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.010+0000 7ff273e00640 1 -- 172.21.15.73:0/3431348778 learned_addr learned my addr 172.21.15.73:0/3431348778 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff281200640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff260004d00 0x7ff27c0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff281c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff27c150eb0 0x7ff27c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 -- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff260004d00 msgr2=0x7ff27c0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff260004d00 0x7ff27c0b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 -- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff27c150eb0 msgr2=0x7ff27c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff27c150eb0 0x7ff27c153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 -- 172.21.15.73:0/3431348778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c14ff80 con 0x7ff27c154130 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff273e00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff26400f580 tx=0x7ff26400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.011+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264013810 con 0x7ff27c154130 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.012+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264013970 con 0x7ff27c154130 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.012+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264013ce0 con 0x7ff27c154130 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.012+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c150260 con 0x7ff27c154130 2024-04-05T22:31:00.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.012+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c143860 con 0x7ff27c154130 2024-04-05T22:31:00.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.013+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c06a610 con 0x7ff27c154130 2024-04-05T22:31:00.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.014+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7ff26401f460 con 0x7ff27c154130 2024-04-05T22:31:00.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.015+0000 7ff279600640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 0x7ff25806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:00.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.015+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff264087df0 con 0x7ff27c154130 2024-04-05T22:31:00.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.015+0000 7ff281200640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 0x7ff25806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:00.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.016+0000 7ff281200640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 0x7ff25806d8d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000f60 tx=0x7ff26c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:00.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.020+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff264054b50 con 0x7ff27c154130 2024-04-05T22:31:00.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.253+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff27c1516b0 con 0x7ff27c154130 2024-04-05T22:31:00.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.936+0000 7ff279600640 1 -- 172.21.15.73:0/3431348778 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff264075e60 con 0x7ff27c154130 2024-04-05T22:31:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 msgr2=0x7ff25806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 0x7ff25806d8d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000f60 tx=0x7ff26c009040 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 msgr2=0x7ff27c143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff26400f580 tx=0x7ff26400fa40 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 shutdown_connections 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff25806b410 0x7ff25806d8d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff27c150eb0 0x7ff27c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c154130 0x7ff27c143320 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 --2- 172.21.15.73:0/3431348778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff260004d00 0x7ff27c0b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 >> 172.21.15.73:0/3431348778 conn(0x7ff27c069cb0 msgr2=0x7ff27c070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.939+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 shutdown_connections 2024-04-05T22:31:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:00.940+0000 7ff283c00640 1 -- 172.21.15.73:0/3431348778 wait complete. 2024-04-05T22:31:00.959 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:31:00.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:31:01.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.173+0000 7ff6a5600640 1 Processor -- start 2024-04-05T22:31:01.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.173+0000 7ff6a5600640 1 -- start start 2024-04-05T22:31:01.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff6a5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6a0070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff6a5600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a0071900 con 0x7ff6a015dd90 2024-04-05T22:31:01.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff6a5600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a015ae50 con 0x7ff6a00714a0 2024-04-05T22:31:01.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff6a5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a015e3b0 con 0x7ff6a015aa70 2024-04-05T22:31:01.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff69f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff6a015dd90 0x7ff6a006ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56628/0 (socket says 172.21.15.73:56628) 2024-04-05T22:31:01.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.175+0000 7ff69f400640 1 -- 172.21.15.73:0/3269577843 learned_addr learned my addr 172.21.15.73:0/3269577843 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69ea00640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6a0070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687537384 0 0) 0x7ff6a0071900 con 0x7ff6a015dd90 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff684003520 con 0x7ff6a015dd90 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297631343 0 0) 0x7ff684003520 con 0x7ff6a015dd90 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 msgr2=0x7ff6a0070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6a0070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 >> v1:172.21.15.157:6789/0 conn(0x7ff6a00714a0 legacy=0x7ff6a015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00ac9e0 con 0x7ff6a015dd90 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.176+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff698001670 con 0x7ff6a015dd90 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 >> v1:172.21.15.73:6789/0 conn(0x7ff6a015dd90 legacy=0x7ff6a006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6840039a0 0x7ff684003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff684004d00 0x7ff684005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a015e3b0 con 0x7ff684004d00 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400ba30 con 0x7ff6840039a0 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400bbf0 con 0x7ff6a015aa70 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff684004d00 0x7ff684005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69ea00640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6840039a0 0x7ff684003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69e000640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 -- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 msgr2=0x7ff6840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 -- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6840039a0 msgr2=0x7ff684003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:01.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6840039a0 0x7ff684003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.177+0000 7ff69f400640 1 -- 172.21.15.73:0/3269577843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684003520 con 0x7ff684004d00 2024-04-05T22:31:01.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.178+0000 7ff69ea00640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6840039a0 0x7ff684003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:01.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.178+0000 7ff69e000640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a015aa70 0x7ff6840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff69f400640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff684004d00 0x7ff684005180 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c210 tx=0x7ff6840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc06d92c5b1f5a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698005ad0 con 0x7ff684004d00 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698005c50 con 0x7ff684004d00 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff69d600640 1 -- 172.21.15.73:0/3269577843 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69802c880 con 0x7ff684004d00 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff6a5600640 1 -- 172.21.15.73:0/3269577843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff684004d00 msgr2=0x7ff684005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff6a5600640 1 --2- 172.21.15.73:0/3269577843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff684004d00 0x7ff684005180 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c210 tx=0x7ff6840057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.179+0000 7ff69f400640 1 -- 172.21.15.73:0/3269577843 reap_dead start 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.180+0000 7ff6a5600640 1 -- 172.21.15.73:0/3269577843 shutdown_connections 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.180+0000 7ff6a5600640 1 -- 172.21.15.73:0/3269577843 >> 172.21.15.73:0/3269577843 conn(0x7ff6a0069cb0 msgr2=0x7ff6a006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.180+0000 7ff6a5600640 1 -- 172.21.15.73:0/3269577843 shutdown_connections 2024-04-05T22:31:01.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.180+0000 7ff6a5600640 1 -- 172.21.15.73:0/3269577843 wait complete. 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.181+0000 7ff6a5600640 1 Processor -- start 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 -- start start 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff684004d00 0x7ff6a00c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a01f5a80 0x7ff6a00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00ca490 con 0x7ff6a00b0470 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00b0e30 con 0x7ff6a01f5a80 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.182+0000 7ff6a5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00cb350 con 0x7ff684004d00 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54400/0 (socket says 172.21.15.73:54400) 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 -- 172.21.15.73:0/456382171 learned_addr learned my addr 172.21.15.73:0/456382171 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69ea00640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff684004d00 0x7ff6a00c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69e000640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a01f5a80 0x7ff6a00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 -- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff684004d00 msgr2=0x7ff6a00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff684004d00 0x7ff6a00c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 -- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a01f5a80 msgr2=0x7ff6a00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a01f5a80 0x7ff6a00cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.183+0000 7ff69f400640 1 -- 172.21.15.73:0/456382171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0155fb0 con 0x7ff6a00b0470 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff69f400640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c300 tx=0x7ff69802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69803c040 con 0x7ff6a00b0470 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69802cc90 con 0x7ff6a00b0470 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698007070 con 0x7ff6a00b0470 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a01f5fb0 con 0x7ff6a00b0470 2024-04-05T22:31:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.184+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a01f6570 con 0x7ff6a00b0470 2024-04-05T22:31:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.185+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a0068160 con 0x7ff6a00b0470 2024-04-05T22:31:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.186+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7ff698032070 con 0x7ff6a00b0470 2024-04-05T22:31:01.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.187+0000 7ff696a00640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 0x7ff67c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:01.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.187+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6980a81a0 con 0x7ff6a00b0470 2024-04-05T22:31:01.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.188+0000 7ff69ea00640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 0x7ff67c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.189+0000 7ff69ea00640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 0x7ff67c06d8a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff690009fd0 tx=0x7ff69000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:01.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.193+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff698074f80 con 0x7ff6a00b0470 2024-04-05T22:31:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.424+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff6a00b2490 con 0x7ff6a00b0470 2024-04-05T22:31:01.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.941+0000 7ff696a00640 1 -- 172.21.15.73:0/456382171 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6980785a0 con 0x7ff6a00b0470 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.944+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 msgr2=0x7ff67c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.944+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 0x7ff67c06d8a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff690009fd0 tx=0x7ff69000b040 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.944+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 msgr2=0x7ff6a00b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.944+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c300 tx=0x7ff69802e070 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 shutdown_connections 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff67c06b3e0 0x7ff67c06d8a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a00b0470 0x7ff6a00b08f0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a01f5a80 0x7ff6a00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 --2- 172.21.15.73:0/456382171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff684004d00 0x7ff6a00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 >> 172.21.15.73:0/456382171 conn(0x7ff6a0069cb0 msgr2=0x7ff6a015e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 shutdown_connections 2024-04-05T22:31:01.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:01.945+0000 7ff6a5600640 1 -- 172.21.15.73:0/456382171 wait complete. 2024-04-05T22:31:01.963 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:31:01.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:02.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.173+0000 7f1ef0a00640 1 Processor -- start 2024-04-05T22:31:02.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.173+0000 7f1ef0a00640 1 -- start start 2024-04-05T22:31:02.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.175+0000 7f1ef0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1eec0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.175+0000 7f1ef0a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec15aa60 con 0x7f1eec0ce000 2024-04-05T22:31:02.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.175+0000 7f1ef0a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0b3b60 con 0x7f1eec06d710 2024-04-05T22:31:02.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.175+0000 7f1ef0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0701c0 con 0x7f1eec1541a0 2024-04-05T22:31:02.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1eea800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1eec0ce000 0x7f1eec15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56644/0 (socket says 172.21.15.73:56644) 2024-04-05T22:31:02.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1eea800640 1 -- 172.21.15.73:0/1452635253 learned_addr learned my addr 172.21.15.73:0/1452635253 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9e00640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1eec0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847419854 0 0) 0x7f1eec15aa60 con 0x7f1eec0ce000 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed0005730 con 0x7f1eec0ce000 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 966008590 0 0) 0x7f1ed0005730 con 0x7f1eec0ce000 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 msgr2=0x7f1eec0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1eec0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 >> v1:172.21.15.157:6789/0 conn(0x7f1eec06d710 legacy=0x7f1eec0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0b3880 con 0x7f1eec0ce000 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f1ee40019f0 con 0x7f1eec0ce000 2024-04-05T22:31:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 >> v1:172.21.15.73:6789/0 conn(0x7f1eec0ce000 legacy=0x7f1eec15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.176+0000 7f1ee9400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed0005bb0 0x7f1ed0005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ed00069d0 0x7f1ed0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1ed0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0701c0 con 0x7f1ed00069d0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed000dc40 con 0x7f1ed0005bb0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed000de00 con 0x7f1eec1541a0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ed00069d0 0x7f1ed0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1eea800640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1ed0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee9e00640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed0005bb0 0x7f1ed0005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 -- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 msgr2=0x7f1ed0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1ed0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 -- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed0005bb0 msgr2=0x7f1ed0005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed0005bb0 0x7f1ed0005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1ee1400640 1 -- 172.21.15.73:0/1452635253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0005730 con 0x7f1ed00069d0 2024-04-05T22:31:02.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.177+0000 7f1eea800640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1541a0 0x7f1ed0007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:02.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.178+0000 7f1ee1400640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ed00069d0 0x7f1ed0006e50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4001720 tx=0x7f1ed0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2be46fae9ffaf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40052d0 con 0x7f1ed00069d0 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed402e050 con 0x7f1ed00069d0 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ee9400640 1 -- 172.21.15.73:0/1452635253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40035f0 con 0x7f1ed00069d0 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 -- 172.21.15.73:0/1452635253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ed00069d0 msgr2=0x7f1ed0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/1452635253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ed00069d0 0x7f1ed0006e50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4001720 tx=0x7f1ed0007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1eea800640 1 -- 172.21.15.73:0/1452635253 reap_dead start 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 -- 172.21.15.73:0/1452635253 shutdown_connections 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 -- 172.21.15.73:0/1452635253 >> 172.21.15.73:0/1452635253 conn(0x7f1eec0ce8e0 msgr2=0x7f1eec0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 -- 172.21.15.73:0/1452635253 shutdown_connections 2024-04-05T22:31:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.179+0000 7f1ef0a00640 1 -- 172.21.15.73:0/1452635253 wait complete. 2024-04-05T22:31:02.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.181+0000 7f1ef0a00640 1 Processor -- start 2024-04-05T22:31:02.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.181+0000 7f1ef0a00640 1 -- start start 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed00069d0 0x7f1eec0a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1484c0 0x7f1eec0a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec14ae50 con 0x7f1eec144a40 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec145170 con 0x7f1ed00069d0 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1ef0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec145450 con 0x7f1eec1484c0 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.182+0000 7f1eea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54420/0 (socket says 172.21.15.73:54420) 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 -- 172.21.15.73:0/3857855798 learned_addr learned my addr 172.21.15.73:0/3857855798 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1ee9e00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed00069d0 0x7f1eec0a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1ee1400640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1484c0 0x7f1eec0a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 -- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1484c0 msgr2=0x7f1eec0a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1484c0 0x7f1eec0a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 -- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed00069d0 msgr2=0x7f1eec0a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed00069d0 0x7f1eec0a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 -- 172.21.15.73:0/3857855798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec145730 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1eea800640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f1eec066850 tx=0x7f1ee400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4004d60 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.183+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400bb80 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.184+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec145a10 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.184+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4007070 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.184+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec159460 con 0x7f1eec144a40 2024-04-05T22:31:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.185+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec066af0 con 0x7f1eec144a40 2024-04-05T22:31:02.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.186+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f1ee40071d0 con 0x7f1eec144a40 2024-04-05T22:31:02.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.187+0000 7f1ee2a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 0x7f1ec806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.187+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ee4087bd0 con 0x7f1eec144a40 2024-04-05T22:31:02.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.187+0000 7f1ee9e00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 0x7f1ec806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.188+0000 7f1ee9e00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 0x7f1ec806d8d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009a10 tx=0x7f1edc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.194+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1ee40549b0 con 0x7f1eec144a40 2024-04-05T22:31:02.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.426+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eec159710 con 0x7f1eec144a40 2024-04-05T22:31:02.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.428+0000 7f1ee2a00640 1 -- 172.21.15.73:0/3857855798 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4746 (secure 0 0 0) 0x7f1ee4057fd0 con 0x7f1eec144a40 2024-04-05T22:31:02.427 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:02.427 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:02.427 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 100 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 msgr2=0x7f1ec806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 0x7f1ec806d8d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009a10 tx=0x7f1edc008040 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 msgr2=0x7f1eec158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f1eec066850 tx=0x7f1ee400b4a0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 shutdown_connections 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.430+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1ec806b410 0x7f1ec806d8d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec144a40 0x7f1eec158d40 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eec1484c0 0x7f1eec0a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 --2- 172.21.15.73:0/3857855798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed00069d0 0x7f1eec0a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 >> 172.21.15.73:0/3857855798 conn(0x7f1eec0ce8e0 msgr2=0x7f1eec0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 shutdown_connections 2024-04-05T22:31:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.431+0000 7f1ef0a00640 1 -- 172.21.15.73:0/3857855798 wait complete. 2024-04-05T22:31:02.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.651+0000 7f2f36c00640 1 Processor -- start 2024-04-05T22:31:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.651+0000 7f2f36c00640 1 -- start start 2024-04-05T22:31:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.653+0000 7f2f36c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 0x7f2f300ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.653+0000 7f2f36c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.653+0000 7f2f36c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300ce5d0 con 0x7f2f30160620 2024-04-05T22:31:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.653+0000 7f2f36c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f300ce010 con 0x7f2f300b1870 2024-04-05T22:31:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.653+0000 7f2f36c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300ce2f0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:47588/0 (socket says 172.21.15.73:47588) 2024-04-05T22:31:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 -- 172.21.15.73:0/3203380301 learned_addr learned my addr 172.21.15.73:0/3203380301 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f2f400640 1 --2- 172.21.15.73:0/3203380301 >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 0x7f2f300ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 -- 172.21.15.73:0/3203380301 >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 msgr2=0x7f2f300ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 --2- 172.21.15.73:0/3203380301 >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 0x7f2f300ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.654+0000 7f2f34c00640 1 -- 172.21.15.73:0/3203380301 >> v1:172.21.15.157:6789/0 conn(0x7f2f30160620 legacy=0x7f2f300caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.655+0000 7f2f34c00640 1 -- 172.21.15.73:0/3203380301 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300ac970 con 0x7f2f300ad6b0 2024-04-05T22:31:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.655+0000 7f2f2f400640 1 --2- 172.21.15.73:0/3203380301 >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 0x7f2f300ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:31:02.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.656+0000 7f2f34c00640 1 --2- 172.21.15.73:0/3203380301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009870 tx=0x7f2f2802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfc3b502945c251 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.657+0000 7f2f2ea00640 1 -- 172.21.15.73:0/3203380301 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28030840 con 0x7f2f300ad6b0 2024-04-05T22:31:02.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.657+0000 7f2f2ea00640 1 -- 172.21.15.73:0/3203380301 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280309a0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.657+0000 7f2f2ea00640 1 -- 172.21.15.73:0/3203380301 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28030d30 con 0x7f2f300ad6b0 2024-04-05T22:31:02.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.657+0000 7f2f36c00640 1 -- 172.21.15.73:0/3203380301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 msgr2=0x7f2f300cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.657+0000 7f2f36c00640 1 --2- 172.21.15.73:0/3203380301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009870 tx=0x7f2f2802b980 comp rx=0 tx=0).stop 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 -- 172.21.15.73:0/3203380301 shutdown_connections 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 --2- 172.21.15.73:0/3203380301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f300ad6b0 0x7f2f300cb2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 --2- 172.21.15.73:0/3203380301 >> v2:172.21.15.157:3300/0 conn(0x7f2f300b1870 0x7f2f300ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 -- 172.21.15.73:0/3203380301 >> 172.21.15.73:0/3203380301 conn(0x7f2f300ce8e0 msgr2=0x7f2f300cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 -- 172.21.15.73:0/3203380301 shutdown_connections 2024-04-05T22:31:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.658+0000 7f2f36c00640 1 -- 172.21.15.73:0/3203380301 wait complete. 2024-04-05T22:31:02.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.660+0000 7f2f36c00640 1 Processor -- start 2024-04-05T22:31:02.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.660+0000 7f2f36c00640 1 -- start start 2024-04-05T22:31:02.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f300b1870 0x7f2f3014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f30160620 0x7f2f300af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300c3b30 con 0x7f2f300ad6b0 2024-04-05T22:31:02.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301507a0 con 0x7f2f300b1870 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.661+0000 7f2f36c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30153780 con 0x7f2f30160620 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54422/0 (socket says 172.21.15.73:54422) 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 -- 172.21.15.73:0/2016738148 learned_addr learned my addr 172.21.15.73:0/2016738148 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2f400640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f300b1870 0x7f2f3014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f34c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f30160620 0x7f2f300af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 -- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f30160620 msgr2=0x7f2f300af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f30160620 0x7f2f300af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 -- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f300b1870 msgr2=0x7f2f3014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f300b1870 0x7f2f3014fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.662+0000 7f2f2fe00640 1 -- 172.21.15.73:0/2016738148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c009a70 con 0x7f2f300ad6b0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f2fe00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c390 tx=0x7f2f2000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f200052d0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20005430 con 0x7f2f300ad6b0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f280094d0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003830 con 0x7f2f300ad6b0 2024-04-05T22:31:02.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.663+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f301f73f0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.664+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f30066af0 con 0x7f2f300ad6b0 2024-04-05T22:31:02.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.665+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f2f20003990 con 0x7f2f300ad6b0 2024-04-05T22:31:02.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.666+0000 7f2f27e00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 0x7f2f1006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:02.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.666+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f20086f70 con 0x7f2f300ad6b0 2024-04-05T22:31:02.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.666+0000 7f2f2f400640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 0x7f2f1006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:02.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.668+0000 7f2f2f400640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 0x7f2f1006d8a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f301f47d0 tx=0x7f2f1c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:02.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.671+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2f20053d50 con 0x7f2f300ad6b0 2024-04-05T22:31:02.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.863+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f30068160 con 0x7f2f300ad6b0 2024-04-05T22:31:02.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.865+0000 7f2f27e00640 1 -- 172.21.15.73:0/2016738148 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f2f20017090 con 0x7f2f300ad6b0 2024-04-05T22:31:02.864 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:02.864 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 msgr2=0x7f2f1006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 0x7f2f1006d8a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f301f47d0 tx=0x7f2f1c00b040 comp rx=0 tx=0).stop 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 msgr2=0x7f2f30153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c390 tx=0x7f2f2000c850 comp rx=0 tx=0).stop 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 shutdown_connections 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2f1006b3e0 0x7f2f1006d8a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f30160620 0x7f2f300af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f300b1870 0x7f2f3014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.867+0000 7f2f36c00640 1 --2- 172.21.15.73:0/2016738148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f300ad6b0 0x7f2f30153240 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:02.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.868+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 >> 172.21.15.73:0/2016738148 conn(0x7f2f300ce8e0 msgr2=0x7f2f300ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:02.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.868+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 shutdown_connections 2024-04-05T22:31:02.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:02.868+0000 7f2f36c00640 1 -- 172.21.15.73:0/2016738148 wait complete. 2024-04-05T22:31:02.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:31:02.888 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:31:03.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.097+0000 7f5456800640 1 Processor -- start 2024-04-05T22:31:03.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.097+0000 7f5456800640 1 -- start start 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5456800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54500b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5456800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54501627f0 con 0x7f5450162370 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5456800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5450165cf0 con 0x7f5450161f90 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5456800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5450158790 con 0x7f54500b2ba0 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5446a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5450162370 0x7f54500b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56652/0 (socket says 172.21.15.73:56652) 2024-04-05T22:31:03.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f5446a00640 1 -- 172.21.15.73:0/2953181942 learned_addr learned my addr 172.21.15.73:0/2953181942 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:03.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f544fe00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54500b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.099+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313425163 0 0) 0x7f5438001110 con 0x7f5450162370 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5430003520 con 0x7f5450162370 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75575086 0 0) 0x7f5430003520 con 0x7f5450162370 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 msgr2=0x7f54500b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54500b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 >> v1:172.21.15.157:6789/0 conn(0x7f5450161f90 legacy=0x7f5450164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450165730 con 0x7f5450162370 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f5438001800 con 0x7f5450162370 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 >> v1:172.21.15.73:6789/0 conn(0x7f5450162370 legacy=0x7f54500b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54300039a0 0x7f5430003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5430004d00 0x7f5430005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450158790 con 0x7f54300039a0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543000bb10 con 0x7f5430004d00 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543000bca0 con 0x7f54500b2ba0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.100+0000 7f544f400640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54300039a0 0x7f5430003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544fe00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5430004d00 0x7f5430005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544f400640 1 -- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 msgr2=0x7f54300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f5446a00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544f400640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500b2ba0 0x7f54300047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544f400640 1 -- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5430004d00 msgr2=0x7f5430005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544f400640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5430004d00 0x7f5430005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544f400640 1 -- 172.21.15.73:0/2953181942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430003520 con 0x7f54300039a0 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.101+0000 7f544fe00640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5430004d00 0x7f5430005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:03.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f544f400640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54300039a0 0x7f5430003dc0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f5440001780 tx=0x7f54300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d136e47fcf2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54400052d0 con 0x7f54300039a0 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544002e050 con 0x7f54300039a0 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f544ea00640 1 -- 172.21.15.73:0/2953181942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54400035f0 con 0x7f54300039a0 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f5456800640 1 -- 172.21.15.73:0/2953181942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54300039a0 msgr2=0x7f5430003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.102+0000 7f5456800640 1 --2- 172.21.15.73:0/2953181942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54300039a0 0x7f5430003dc0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f5440001780 tx=0x7f54300057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.103+0000 7f544fe00640 1 -- 172.21.15.73:0/2953181942 reap_dead start 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.103+0000 7f5456800640 1 -- 172.21.15.73:0/2953181942 shutdown_connections 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.103+0000 7f5456800640 1 -- 172.21.15.73:0/2953181942 >> 172.21.15.73:0/2953181942 conn(0x7f54500ce8e0 msgr2=0x7f54500cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.103+0000 7f5456800640 1 -- 172.21.15.73:0/2953181942 shutdown_connections 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.103+0000 7f5456800640 1 -- 172.21.15.73:0/2953181942 wait complete. 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.104+0000 7f5456800640 1 Processor -- start 2024-04-05T22:31:03.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.105+0000 7f5456800640 1 -- start start 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 0x7f545014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54501592e0 0x7f5450159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450159cd0 con 0x7f54501ed490 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501ec3f0 con 0x7f54300039a0 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5456800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54501ec6d0 con 0x7f54501592e0 2024-04-05T22:31:03.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54448/0 (socket says 172.21.15.73:54448) 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 -- 172.21.15.73:0/1113788024 learned_addr learned my addr 172.21.15.73:0/1113788024 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f544f400640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 0x7f545014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 -- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54501592e0 msgr2=0x7f5450159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54501592e0 0x7f5450159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 -- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 msgr2=0x7f545014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 0x7f545014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f5446a00640 1 -- 172.21.15.73:0/1113788024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54501ec9b0 con 0x7f54501ed490 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.106+0000 7f544f400640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 0x7f545014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5446a00640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f543800d7e0 tx=0x7f543800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380074e0 con 0x7f54501ed490 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438007640 con 0x7f54501ed490 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380049e0 con 0x7f54501ed490 2024-04-05T22:31:03.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54501ecc90 con 0x7f54501ed490 2024-04-05T22:31:03.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.107+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545014c7f0 con 0x7f54501ed490 2024-04-05T22:31:03.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.108+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5450066af0 con 0x7f54501ed490 2024-04-05T22:31:03.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.109+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f5438004b40 con 0x7f54501ed490 2024-04-05T22:31:03.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.110+0000 7f5447e00640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 0x7f542c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.110+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54380871d0 con 0x7f54501ed490 2024-04-05T22:31:03.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.111+0000 7f544f400640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 0x7f542c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.112+0000 7f544f400640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 0x7f542c06d8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5440033000 tx=0x7f5440009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.114+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5438053fb0 con 0x7f54501ed490 2024-04-05T22:31:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.303+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54500b1bf0 con 0x7f542c06b3e0 2024-04-05T22:31:03.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.325+0000 7f5447e00640 1 -- 172.21.15.73:0/1113788024 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276471 (secure 0 0 0) 0x7f54400054a0 con 0x7f542c06b3e0 2024-04-05T22:31:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:03.329 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:31:03.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 msgr2=0x7f542c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 0x7f542c06d8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5440033000 tx=0x7f5440009030 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 msgr2=0x7f545014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f543800d7e0 tx=0x7f543800dca0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 shutdown_connections 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b3e0 0x7f542c06d8a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54501592e0 0x7f5450159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54501ed490 0x7f545014c2b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 --2- 172.21.15.73:0/1113788024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54300039a0 0x7f545014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.331+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 >> 172.21.15.73:0/1113788024 conn(0x7f54500ce8e0 msgr2=0x7f5450163280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.332+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 shutdown_connections 2024-04-05T22:31:03.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.332+0000 7f5456800640 1 -- 172.21.15.73:0/1113788024 wait complete. 2024-04-05T22:31:03.350 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-05T22:31:01.596636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281868,"kb_used_data":16488,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754692852,"statfs":{"total":773094113280,"available":772805480448,"internally_reserved":0,"allocated":16883712,"data_stored":8081432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.172516"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.438718+0000","last_change":"2024-04-05T22:30:57.438718+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.438718+0000","last_undegraded":"2024-04-05T22:30:57.438718+0000","last_fullsized":"2024-04-05T22:30:57.438718+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:43.346196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.442735+0000","last_change":"2024-04-05T22:30:57.442735+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.442735+0000","last_undegraded":"2024-04-05T22:30:57.442735+0000","last_fullsized":"2024-04-05T22:30:57.442735+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:41.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.432026+0000","last_change":"2024-04-05T22:30:57.432026+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.432026+0000","last_undegraded":"2024-04-05T22:30:57.432026+0000","last_fullsized":"2024-04-05T22:30:57.432026+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:17.958611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434991+0000","last_change":"2024-04-05T22:30:57.434991+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434991+0000","last_undegraded":"2024-04-05T22:30:57.434991+0000","last_fullsized":"2024-04-05T22:30:57.434991+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:04.611779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.440747+0000","last_change":"2024-04-05T22:30:57.440747+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.440747+0000","last_undegraded":"2024-04-05T22:30:57.440747+0000","last_fullsized":"2024-04-05T22:30:57.440747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:27.787632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.430159+0000","last_change":"2024-04-05T22:30:57.430159+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.430159+0000","last_undegraded":"2024-04-05T22:30:57.430159+0000","last_fullsized":"2024-04-05T22:30:57.430159+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:12.716550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.432342+0000","last_change":"2024-04-05T22:30:57.432342+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.432342+0000","last_undegraded":"2024-04-05T22:30:57.432342+0000","last_fullsized":"2024-04-05T22:30:57.432342+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:27.097967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.436044+0000","last_change":"2024-04-05T22:30:57.436044+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.436044+0000","last_undegraded":"2024-04-05T22:30:57.436044+0000","last_fullsized":"2024-04-05T22:30:57.436044+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:34.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.439520+0000","last_change":"2024-04-05T22:30:57.439520+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.439520+0000","last_undegraded":"2024-04-05T22:30:57.439520+0000","last_fullsized":"2024-04-05T22:30:57.439520+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:12.128598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.440199+0000","last_change":"2024-04-05T22:30:57.440199+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.440199+0000","last_undegraded":"2024-04-05T22:30:57.440199+0000","last_fullsized":"2024-04-05T22:30:57.440199+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:31.337633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.432976+0000","last_change":"2024-04-05T22:30:57.432976+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.432976+0000","last_undegraded":"2024-04-05T22:30:57.432976+0000","last_fullsized":"2024-04-05T22:30:57.432976+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:34.941070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429190+0000","last_change":"2024-04-05T22:30:57.429190+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429190+0000","last_undegraded":"2024-04-05T22:30:57.429190+0000","last_fullsized":"2024-04-05T22:30:57.429190+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:27.162529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434551+0000","last_change":"2024-04-05T22:30:57.434551+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434551+0000","last_undegraded":"2024-04-05T22:30:57.434551+0000","last_fullsized":"2024-04-05T22:30:57.434551+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:15.253748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.428465+0000","last_change":"2024-04-05T22:30:57.428465+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.428465+0000","last_undegraded":"2024-04-05T22:30:57.428465+0000","last_fullsized":"2024-04-05T22:30:57.428465+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:51:14.094034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429451+0000","last_change":"2024-04-05T22:30:57.429451+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429451+0000","last_undegraded":"2024-04-05T22:30:57.429451+0000","last_fullsized":"2024-04-05T22:30:57.429451+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:17.958611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.437817+0000","last_change":"2024-04-05T22:30:57.437817+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.437817+0000","last_undegraded":"2024-04-05T22:30:57.437817+0000","last_fullsized":"2024-04-05T22:30:57.437817+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:41.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429101+0000","last_change":"2024-04-05T22:30:57.429101+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429101+0000","last_undegraded":"2024-04-05T22:30:57.429101+0000","last_fullsized":"2024-04-05T22:30:57.429101+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:55.270524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429327+0000","last_change":"2024-04-05T22:30:57.429327+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429327+0000","last_undegraded":"2024-04-05T22:30:57.429327+0000","last_fullsized":"2024-04-05T22:30:57.429327+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:05.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434903+0000","last_change":"2024-04-05T22:30:57.434903+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434903+0000","last_undegraded":"2024-04-05T22:30:57.434903+0000","last_fullsized":"2024-04-05T22:30:57.434903+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:17:59.106090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.427870+0000","last_change":"2024-04-05T22:30:57.427870+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.427870+0000","last_undegraded":"2024-04-05T22:30:57.427870+0000","last_fullsized":"2024-04-05T22:30:57.427870+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:28.182098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.437810+0000","last_change":"2024-04-05T22:30:57.437810+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.437810+0000","last_undegraded":"2024-04-05T22:30:57.437810+0000","last_fullsized":"2024-04-05T22:30:57.437810+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:16:22.654879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429010+0000","last_change":"2024-04-05T22:30:57.429010+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429010+0000","last_undegraded":"2024-04-05T22:30:57.429010+0000","last_fullsized":"2024-04-05T22:30:57.429010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:05.255712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.441866+0000","last_change":"2024-04-05T22:30:57.441866+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.441866+0000","last_undegraded":"2024-04-05T22:30:57.441866+0000","last_fullsized":"2024-04-05T22:30:57.441866+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:48:50.166340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429218+0000","last_change":"2024-04-05T22:30:57.429218+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429218+0000","last_undegraded":"2024-04-05T22:30:57.429218+0000","last_fullsized":"2024-04-05T22:30:57.429218+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:34.941070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434426+0000","last_change":"2024-04-05T22:30:57.434426+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434426+0000","last_undegraded":"2024-04-05T22:30:57.434426+0000","last_fullsized":"2024-04-05T22:30:57.434426+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:55.270524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434465+0000","last_change":"2024-04-05T22:30:57.434465+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434465+0000","last_undegraded":"2024-04-05T22:30:57.434465+0000","last_fullsized":"2024-04-05T22:30:57.434465+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:12.716550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.428726+0000","last_change":"2024-04-05T22:30:57.428726+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.428726+0000","last_undegraded":"2024-04-05T22:30:57.428726+0000","last_fullsized":"2024-04-05T22:30:57.428726+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:15.253748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.441871+0000","last_change":"2024-04-05T22:30:57.441871+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.441871+0000","last_undegraded":"2024-04-05T22:30:57.441871+0000","last_fullsized":"2024-04-05T22:30:57.441871+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:14.387165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.442611+0000","last_change":"2024-04-05T22:30:57.442611+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.442611+0000","last_undegraded":"2024-04-05T22:30:57.442611+0000","last_fullsized":"2024-04-05T22:30:57.442611+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:05:58.488855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.434827+0000","last_change":"2024-04-05T22:30:57.434827+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.434827+0000","last_undegraded":"2024-04-05T22:30:57.434827+0000","last_fullsized":"2024-04-05T22:30:57.434827+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:13:01.300453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.432036+0000","last_change":"2024-04-05T22:30:57.432036+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.432036+0000","last_undegraded":"2024-04-05T22:30:57.432036+0000","last_fullsized":"2024-04-05T22:30:57.432036+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:27.097967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-05T22:30:58.422723+0000","last_change":"2024-04-05T22:30:58.422723+0000","last_active":"2024-04-05T22:30:58.422723+0000","last_peered":"2024-04-05T22:30:58.422723+0000","last_clean":"2024-04-05T22:30:58.422723+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:58.422723+0000","last_undegraded":"2024-04-05T22:30:58.422723+0000","last_fullsized":"2024-04-05T22:30:58.422723+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-05T22:30:57.429275+0000","last_change":"2024-04-05T22:30:57.429275+0000","last_active":"2024-04-05T22:30:57.404705+0000","last_peered":"2024-04-05T22:30:57.404705+0000","last_clean":"2024-04-05T22:30:57.404705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-05T22:30:57.429275+0000","last_undegraded":"2024-04-05T22:30:57.429275+0000","last_fullsized":"2024-04-05T22:30:57.429275+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:05.255712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":318,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-05T22:30:55.696347+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:30:55.696347+0000","last_peered":"2024-04-05T22:30:55.696347+0000","last_clean":"2024-04-05T22:30:55.696347+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:30:55.696347+0000","last_undegraded":"2024-04-05T22:30:55.696347+0000","last_fullsized":"2024-04-05T22:30:55.696347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542265,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37716,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334124,"statfs":{"total":96636764160,"available":96598142976,"internally_reserved":0,"allocated":2555904,"data_stored":1452955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.048,"15min":1.0409999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4850000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0900000000000001,"15min":1.0409999999999999},"min":{"1min":0.67900000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5109999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.575}]},{"osd":1,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.167,"15min":1.0960000000000001},"min":{"1min":0.88,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":1.4750000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.161},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1060000000000001,"15min":1.077},"min":{"1min":0.78500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4299999999999999,"5min":1.819,"15min":1.819},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.177},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.597,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.2909999999999999,"15min":1.246},"min":{"1min":0.995,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.6339999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2709999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1879999999999999,"15min":1.1559999999999999},"min":{"1min":0.79800000000000004,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.516,"5min":1.667,"15min":1.712},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.2290000000000001,"15min":1.224},"min":{"1min":0.879,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.6970000000000001,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3779999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.226},"min":{"1min":0.91500000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.581,"5min":2.71,"15min":2.71},"last":1.889},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.7290000000000001,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.623}]},{"osd":5,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.335,"15min":1.274},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5549999999999999,"5min":3.492,"15min":3.492},"last":1.8460000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.3899999999999999,"15min":1.363},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.72099999999999997},"max":{"1min":1.6870000000000001,"5min":2.992,"15min":2.992},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.5369999999999999,"15min":1.4870000000000001},"min":{"1min":0.99299999999999999,"5min":0.95799999999999996,"15min":0.83799999999999997},"max":{"1min":1.9379999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.508,"15min":1.502},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.8089999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.002,"15min":1.0449999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.577,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":1.774},{"interface":"front","average":{"1min":1.034,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":1.544,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.91200000000000003}]},{"osd":1,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0529999999999999,"15min":1.133},"min":{"1min":0.73999999999999999,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.629,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.73199999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.4890000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0960000000000001,"15min":1.165},"min":{"1min":0.81799999999999995,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.167},{"interface":"front","average":{"1min":1.135,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.80500000000000005,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.51,"5min":1.51,"15min":2.2250000000000001},"last":1.397}]},{"osd":3,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.278},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.657,"5min":1.96,"15min":1.96},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.248,"15min":1.3560000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.95,"5min":1.95,"15min":3.1720000000000002},"last":1.2689999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.327,"15min":1.325},"min":{"1min":0.83899999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8939999999999999,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.897},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.413,"15min":1.51},"min":{"1min":1.153,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.4570000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3009999999999999,"15min":1.3939999999999999},"min":{"1min":0.67300000000000004,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.972,"5min":1.972,"15min":2.3650000000000002},"last":1.613},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.48,"15min":1.526},"min":{"1min":0.83499999999999996,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8759999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.845}]},{"osd":7,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.417,"15min":1.494},"min":{"1min":1.1040000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":0.85399999999999998,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.909,"5min":1.909,"15min":3.0920000000000001},"last":1.744}]}]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.163,"15min":1.0660000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.625,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.179,"15min":1.1220000000000001},"min":{"1min":0.65800000000000003,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.6990000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.23,"15min":1.1879999999999999},"min":{"1min":0.872,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.472,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.397,"15min":1.2410000000000001},"min":{"1min":0.83899999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6579999999999999,"5min":17.381,"15min":17.381},"last":1.0229999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.4219999999999999,"15min":1.2470000000000001},"min":{"1min":0.73199999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.694,"5min":24.52,"15min":24.52},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.298,"15min":1.264},"min":{"1min":0.73499999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5049999999999999,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.081}]},{"osd":3,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.24,"15min":1.2030000000000001},"min":{"1min":1.032,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6759999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3500000000000001},"min":{"1min":1.008,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.748,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.466,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.8160000000000001,"5min":14.933,"15min":14.933},"last":1.22},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.88400000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.7390000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.2889999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4490000000000001,"15min":1.3600000000000001},"min":{"1min":0.99399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.585,"15min":1.518},"min":{"1min":1.3420000000000001,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.893,"5min":3.782,"15min":3.782},"last":1.375}]},{"osd":7,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.5669999999999999,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9430000000000001,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.579,"15min":1.593},"min":{"1min":1.1579999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":1.7609999999999999,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.4890000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0580000000000001,"15min":1.0269999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.51,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.306},{"interface":"front","average":{"1min":1.03,"5min":1.085,"15min":1.085},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.536,"5min":1.859,"15min":1.859},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0960000000000001,"15min":1.121},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.51,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.196,"15min":1.1759999999999999},"min":{"1min":0.78900000000000003,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.502,"5min":2.577,"15min":2.577},"last":1.008}]},{"osd":2,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.127,"15min":1.1870000000000001},"min":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.476,"5min":1.6579999999999999,"15min":1.974},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.218,"15min":1.198},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67400000000000004},"max":{"1min":1.5429999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.532}]},{"osd":3,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3,"15min":1.349},"min":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.593,"5min":1.835,"15min":2.1840000000000002},"last":1.468},{"interface":"front","average":{"1min":1.222,"5min":1.375,"15min":1.3899999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.627,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.6240000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.2969999999999999},"min":{"1min":0.83399999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":2.665,"15min":2.665},"last":1.395},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.405,"15min":1.389},"min":{"1min":0.89400000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.643,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.581}]},{"osd":6,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4430000000000001,"15min":1.4419999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.53500000000000003},"max":{"1min":1.8799999999999999,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.444,"15min":1.4690000000000001},"min":{"1min":0.74099999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.647,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.4399999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5489999999999999,"15min":1.5349999999999999},"min":{"1min":1.0589999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6419999999999999,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.663},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.6220000000000001,"15min":1.5880000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.72399999999999998},"max":{"1min":2.0609999999999999,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.782}]}]},{"osd":3,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.83199999999999996},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2150000000000001,"5min":1.486,"15min":1.744},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.33900000000000002},"max":{"1min":1.157,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3700000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.014},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.40000000000000002},"max":{"1min":1.546,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.0549999999999999,"15min":1.075},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.524,"5min":1.956,"15min":2.2549999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1579999999999999,"15min":1.2090000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.734,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.746},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.258,"15min":1.2829999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.52000000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.266,"15min":1.3280000000000001},"min":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.7050000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.216},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2729999999999999,"15min":1.286},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.6850000000000001,"5min":1.958,"15min":2.3239999999999998},"last":1.109}]},{"osd":6,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.262,"15min":1.3260000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.8400000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3779999999999999},"min":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.59099999999999997},"max":{"1min":1.6559999999999999,"5min":2.298,"15min":2.298},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3200000000000001,"15min":1.395},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.7789999999999999,"5min":1.899,"15min":2.468},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.3120000000000001,"15min":1.389},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.601,"5min":2.508,"15min":2.798},"last":0.83799999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.82899999999999996,"15min":0.83099999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.7,"5min":2.879,"15min":2.879},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.99199999999999999,"15min":0.93999999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.669,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83099999999999996,"15min":0.83999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3300000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.89800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.8089999999999999,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0369999999999999,"15min":1.044},"min":{"1min":0.54300000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.927,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.2480000000000002,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.3480000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.325,"15min":1.2629999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.8839999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.194},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2829999999999999,"15min":1.2529999999999999},"min":{"1min":0.69199999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8360000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.4379999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2509999999999999,"15min":1.25},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.081,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.383},{"interface":"front","average":{"1min":1.226,"5min":1.252,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1419999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.095}]},{"osd":6,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2649999999999999,"15min":1.2669999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.855,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.296,"15min":1.2849999999999999},"min":{"1min":0.72499999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7969999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.5449999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3500000000000001,"15min":1.3540000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.1789999999999998,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.395},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.9750000000000001,"5min":2.597,"15min":2.597},"last":1.5369999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1961984,"data_stored":862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.38500000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.202,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.86199999999999999},"min":{"1min":0.53900000000000003,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5960000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.3520000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4139999999999999,"5min":1.488,"15min":1.5389999999999999},"last":1.153},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.99399999999999999},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.54}]},{"osd":3,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0289999999999999,"15min":1.087},"min":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98599999999999999,"15min":0.97299999999999998},"min":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.5249999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2869999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.135},"min":{"1min":0.67700000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.278,"5min":1.272,"15min":1.2809999999999999},"min":{"1min":0.85399999999999998,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.238,"15min":1.258},"min":{"1min":0.84499999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.4910000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.70699999999999996,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.619,"5min":2.0680000000000001,"15min":2.173},"last":1.385}]},{"osd":6,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3480000000000001,"15min":1.343},"min":{"1min":0.83799999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8540000000000001,"5min":1.857,"15min":1.917},"last":1.329},{"interface":"front","average":{"1min":1.24,"5min":1.3220000000000001,"15min":1.3460000000000001},"min":{"1min":0.83399999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.8400000000000001,"5min":2.157,"15min":2.157},"last":1.472}]},{"osd":7,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8660000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.421},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5720000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.2609999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542266,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2555904,"data_stored":1452955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2769999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88400000000000001,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.4490000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":1.157}]},{"osd":2,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.875,"15min":0.91500000000000004},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.177,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94299999999999995,"15min":0.96099999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.405,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.92800000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.89000000000000001},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.155,"5min":1.976,"15min":1.976},"last":0.745},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0149999999999999,"15min":1.077},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5089999999999999,"5min":2.157,"15min":2.157},"last":1.0820000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.165,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.575,"5min":1.931,"15min":1.931},"last":1.024},{"interface":"front","average":{"1min":1.236,"5min":1.268,"15min":1.296},"min":{"1min":0.84499999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.534,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.2070000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2290000000000001,"15min":1.258},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5089999999999999,"5min":2.105,"15min":2.3879999999999999},"last":1.52},{"interface":"front","average":{"1min":1.145,"5min":1.1950000000000001,"15min":1.2010000000000001},"min":{"1min":0.86899999999999999,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5269999999999999,"5min":1.7010000000000001,"15min":1.9550000000000001},"last":1.403}]},{"osd":6,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3069999999999999,"15min":1.363},"min":{"1min":0.85199999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.589,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.292,"15min":1.355},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5760000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.6830000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3,"15min":1.333},"min":{"1min":0.79600000000000004,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.637,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.24,"15min":1.3},"min":{"1min":0.94799999999999995,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.639,"5min":1.839,"15min":1.9810000000000001},"last":1.492}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:31:03.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:31:03.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.608+0000 7fc099800640 1 Processor -- start 2024-04-05T22:31:03.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.609+0000 7fc099800640 1 -- start start 2024-04-05T22:31:03.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.610+0000 7fc099800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc094160620 0x7fc0940caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.610+0000 7fc099800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940cb2f0 0x7fc0940cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.610+0000 7fc099800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940ad880 con 0x7fc0940b1870 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.610+0000 7fc099800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0940cbcc0 con 0x7fc094160620 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.610+0000 7fc099800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940ce5d0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc094160620 0x7fc0940caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc08be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc0940b1870 0x7fc0940ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56666/0 (socket says 172.21.15.73:56666) 2024-04-05T22:31:03.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc08be00640 1 -- 172.21.15.73:0/3873838503 learned_addr learned my addr 172.21.15.73:0/3873838503 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc093400640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940cb2f0 0x7fc0940cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092a00640 1 -- 172.21.15.73:0/3873838503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940cb2f0 msgr2=0x7fc0940cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092a00640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940cb2f0 0x7fc0940cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092a00640 1 -- 172.21.15.73:0/3873838503 >> v1:172.21.15.73:6789/0 conn(0x7fc0940b1870 legacy=0x7fc0940ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092a00640 1 -- 172.21.15.73:0/3873838503 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940ce010 con 0x7fc094160620 2024-04-05T22:31:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.611+0000 7fc092000640 1 -- 172.21.15.73:0/3873838503 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216006030 0 0) 0x7fc0940ad880 con 0x7fc0940b1870 2024-04-05T22:31:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.612+0000 7fc092a00640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc094160620 0x7fc0940caa60 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fc0840097a0 tx=0x7fc08402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1b461ef4c67d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc092000640 1 -- 172.21.15.73:0/3873838503 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084031830 con 0x7fc094160620 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc092000640 1 -- 172.21.15.73:0/3873838503 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084031990 con 0x7fc094160620 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc092000640 1 -- 172.21.15.73:0/3873838503 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084031d20 con 0x7fc094160620 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 -- 172.21.15.73:0/3873838503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc094160620 msgr2=0x7fc0940caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc094160620 0x7fc0940caa60 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fc0840097a0 tx=0x7fc08402b8d0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 -- 172.21.15.73:0/3873838503 shutdown_connections 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940cb2f0 0x7fc0940cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 --2- 172.21.15.73:0/3873838503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc094160620 0x7fc0940caa60 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 -- 172.21.15.73:0/3873838503 >> 172.21.15.73:0/3873838503 conn(0x7fc0940ce8e0 msgr2=0x7fc0940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 -- 172.21.15.73:0/3873838503 shutdown_connections 2024-04-05T22:31:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.613+0000 7fc099800640 1 -- 172.21.15.73:0/3873838503 wait complete. 2024-04-05T22:31:03.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.615+0000 7fc099800640 1 Processor -- start 2024-04-05T22:31:03.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.615+0000 7fc099800640 1 -- start start 2024-04-05T22:31:03.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940b1870 0x7fc094153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 0x7fc0940cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094158850 con 0x7fc0940cb2f0 2024-04-05T22:31:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094157c40 con 0x7fc094160620 2024-04-05T22:31:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc099800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc094157f20 con 0x7fc0940b1870 2024-04-05T22:31:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc08be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc08be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54478/0 (socket says 172.21.15.73:54478) 2024-04-05T22:31:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.616+0000 7fc08be00640 1 -- 172.21.15.73:0/3558827647 learned_addr learned my addr 172.21.15.73:0/3558827647 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 -- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940b1870 msgr2=0x7fc094153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940b1870 0x7fc094153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 -- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 msgr2=0x7fc0940cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc093400640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 0x7fc0940cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 0x7fc0940cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 -- 172.21.15.73:0/3558827647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009400 con 0x7fc0940cb2f0 2024-04-05T22:31:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc093400640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 0x7fc0940cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08be00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c4a0 tx=0x7fc07c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c005ce0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c005e40 con 0x7fc0940cb2f0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0072b0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.617+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc094158200 con 0x7fc0940cb2f0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.618+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941578d0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.619+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094066af0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.620+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fc07c007410 con 0x7fc0940cb2f0 2024-04-05T22:31:03.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.621+0000 7fc08aa00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 0x7fc07006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:03.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.621+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc07c087000 con 0x7fc0940cb2f0 2024-04-05T22:31:03.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.621+0000 7fc092a00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 0x7fc07006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:03.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.623+0000 7fc092a00640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 0x7fc07006d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc084036000 tx=0x7fc084030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:03.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.625+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc07c053de0 con 0x7fc0940cb2f0 2024-04-05T22:31:03.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.809+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0940acc50 con 0x7fc07006b3e0 2024-04-05T22:31:03.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.833+0000 7fc08aa00640 1 -- 172.21.15.73:0/3558827647 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287421 (secure 0 0 0) 0x7fc084031b00 con 0x7fc07006b3e0 2024-04-05T22:31:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:03.838 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 msgr2=0x7fc07006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 0x7fc07006d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc084036000 tx=0x7fc084030000 comp rx=0 tx=0).stop 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 msgr2=0x7fc09415c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c4a0 tx=0x7fc07c00c960 comp rx=0 tx=0).stop 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 shutdown_connections 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc07006b3e0 0x7fc07006d8a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc094160620 0x7fc0940cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0940cb2f0 0x7fc09415c8e0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 --2- 172.21.15.73:0/3558827647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0940b1870 0x7fc094153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 >> 172.21.15.73:0/3558827647 conn(0x7fc0940ce8e0 msgr2=0x7fc09406f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 shutdown_connections 2024-04-05T22:31:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:03.839+0000 7fc099800640 1 -- 172.21.15.73:0/3558827647 wait complete. 2024-04-05T22:31:03.857 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-04-05T22:31:03.597318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282276,"kb_used_data":16936,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754692444,"statfs":{"total":773094113280,"available":772805062656,"internally_reserved":0,"allocated":17342464,"data_stored":8287492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":271568443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002091"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.100614+0000","last_change":"2024-04-05T22:30:58.956290+0000","last_active":"2024-04-05T22:31:00.100614+0000","last_peered":"2024-04-05T22:31:00.100614+0000","last_clean":"2024-04-05T22:31:00.100614+0000","last_became_active":"2024-04-05T22:30:58.953873+0000","last_became_peered":"2024-04-05T22:30:58.953873+0000","last_unstale":"2024-04-05T22:31:00.100614+0000","last_undegraded":"2024-04-05T22:31:00.100614+0000","last_fullsized":"2024-04-05T22:31:00.100614+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:43:30.407850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099097+0000","last_change":"2024-04-05T22:30:58.947590+0000","last_active":"2024-04-05T22:31:00.099097+0000","last_peered":"2024-04-05T22:31:00.099097+0000","last_clean":"2024-04-05T22:31:00.099097+0000","last_became_active":"2024-04-05T22:30:58.947330+0000","last_became_peered":"2024-04-05T22:30:58.947330+0000","last_unstale":"2024-04-05T22:31:00.099097+0000","last_undegraded":"2024-04-05T22:31:00.099097+0000","last_fullsized":"2024-04-05T22:31:00.099097+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:01:29.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935981+0000","last_change":"2024-04-05T22:30:58.940600+0000","last_active":"2024-04-05T22:30:59.935981+0000","last_peered":"2024-04-05T22:30:59.935981+0000","last_clean":"2024-04-05T22:30:59.935981+0000","last_became_active":"2024-04-05T22:30:58.940327+0000","last_became_peered":"2024-04-05T22:30:58.940327+0000","last_unstale":"2024-04-05T22:30:59.935981+0000","last_undegraded":"2024-04-05T22:30:59.935981+0000","last_fullsized":"2024-04-05T22:30:59.935981+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:47:39.323009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099081+0000","last_change":"2024-04-05T22:30:58.942157+0000","last_active":"2024-04-05T22:31:00.099081+0000","last_peered":"2024-04-05T22:31:00.099081+0000","last_clean":"2024-04-05T22:31:00.099081+0000","last_became_active":"2024-04-05T22:30:58.941801+0000","last_became_peered":"2024-04-05T22:30:58.941801+0000","last_unstale":"2024-04-05T22:31:00.099081+0000","last_undegraded":"2024-04-05T22:31:00.099081+0000","last_fullsized":"2024-04-05T22:31:00.099081+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:31:14.759412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.095828+0000","last_change":"2024-04-05T22:30:58.945068+0000","last_active":"2024-04-05T22:31:00.095828+0000","last_peered":"2024-04-05T22:31:00.095828+0000","last_clean":"2024-04-05T22:31:00.095828+0000","last_became_active":"2024-04-05T22:30:58.942544+0000","last_became_peered":"2024-04-05T22:30:58.942544+0000","last_unstale":"2024-04-05T22:31:00.095828+0000","last_undegraded":"2024-04-05T22:31:00.095828+0000","last_fullsized":"2024-04-05T22:31:00.095828+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:51:09.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.934857+0000","last_change":"2024-04-05T22:30:58.947338+0000","last_active":"2024-04-05T22:30:59.934857+0000","last_peered":"2024-04-05T22:30:59.934857+0000","last_clean":"2024-04-05T22:30:59.934857+0000","last_became_active":"2024-04-05T22:30:58.947036+0000","last_became_peered":"2024-04-05T22:30:58.947036+0000","last_unstale":"2024-04-05T22:30:59.934857+0000","last_undegraded":"2024-04-05T22:30:59.934857+0000","last_fullsized":"2024-04-05T22:30:59.934857+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:40:27.012950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936183+0000","last_change":"2024-04-05T22:30:58.952845+0000","last_active":"2024-04-05T22:30:59.936183+0000","last_peered":"2024-04-05T22:30:59.936183+0000","last_clean":"2024-04-05T22:30:59.936183+0000","last_became_active":"2024-04-05T22:30:58.952517+0000","last_became_peered":"2024-04-05T22:30:58.952517+0000","last_unstale":"2024-04-05T22:30:59.936183+0000","last_undegraded":"2024-04-05T22:30:59.936183+0000","last_fullsized":"2024-04-05T22:30:59.936183+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:02.157872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097687+0000","last_change":"2024-04-05T22:30:58.955204+0000","last_active":"2024-04-05T22:31:00.097687+0000","last_peered":"2024-04-05T22:31:00.097687+0000","last_clean":"2024-04-05T22:31:00.097687+0000","last_became_active":"2024-04-05T22:30:58.954956+0000","last_became_peered":"2024-04-05T22:30:58.954956+0000","last_unstale":"2024-04-05T22:31:00.097687+0000","last_undegraded":"2024-04-05T22:31:00.097687+0000","last_fullsized":"2024-04-05T22:31:00.097687+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.144625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099389+0000","last_change":"2024-04-05T22:30:58.944307+0000","last_active":"2024-04-05T22:31:00.099389+0000","last_peered":"2024-04-05T22:31:00.099389+0000","last_clean":"2024-04-05T22:31:00.099389+0000","last_became_active":"2024-04-05T22:30:58.943950+0000","last_became_peered":"2024-04-05T22:30:58.943950+0000","last_unstale":"2024-04-05T22:31:00.099389+0000","last_undegraded":"2024-04-05T22:31:00.099389+0000","last_fullsized":"2024-04-05T22:31:00.099389+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:55:38.751905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937206+0000","last_change":"2024-04-05T22:30:58.948746+0000","last_active":"2024-04-05T22:30:59.937206+0000","last_peered":"2024-04-05T22:30:59.937206+0000","last_clean":"2024-04-05T22:30:59.937206+0000","last_became_active":"2024-04-05T22:30:58.948503+0000","last_became_peered":"2024-04-05T22:30:58.948503+0000","last_unstale":"2024-04-05T22:30:59.937206+0000","last_undegraded":"2024-04-05T22:30:59.937206+0000","last_fullsized":"2024-04-05T22:30:59.937206+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:43.144625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936534+0000","last_change":"2024-04-05T22:30:58.954032+0000","last_active":"2024-04-05T22:30:59.936534+0000","last_peered":"2024-04-05T22:30:59.936534+0000","last_clean":"2024-04-05T22:30:59.936534+0000","last_became_active":"2024-04-05T22:30:58.953731+0000","last_became_peered":"2024-04-05T22:30:58.953731+0000","last_unstale":"2024-04-05T22:30:59.936534+0000","last_undegraded":"2024-04-05T22:30:59.936534+0000","last_fullsized":"2024-04-05T22:30:59.936534+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:06.273730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099325+0000","last_change":"2024-04-05T22:30:58.947630+0000","last_active":"2024-04-05T22:31:00.099325+0000","last_peered":"2024-04-05T22:31:00.099325+0000","last_clean":"2024-04-05T22:31:00.099325+0000","last_became_active":"2024-04-05T22:30:58.947316+0000","last_became_peered":"2024-04-05T22:30:58.947316+0000","last_unstale":"2024-04-05T22:31:00.099325+0000","last_undegraded":"2024-04-05T22:31:00.099325+0000","last_fullsized":"2024-04-05T22:31:00.099325+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:40:57.338592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097971+0000","last_change":"2024-04-05T22:30:58.941477+0000","last_active":"2024-04-05T22:31:00.097971+0000","last_peered":"2024-04-05T22:31:00.097971+0000","last_clean":"2024-04-05T22:31:00.097971+0000","last_became_active":"2024-04-05T22:30:58.940317+0000","last_became_peered":"2024-04-05T22:30:58.940317+0000","last_unstale":"2024-04-05T22:31:00.097971+0000","last_undegraded":"2024-04-05T22:31:00.097971+0000","last_fullsized":"2024-04-05T22:31:00.097971+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:10:37.527452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097285+0000","last_change":"2024-04-05T22:30:58.951494+0000","last_active":"2024-04-05T22:31:00.097285+0000","last_peered":"2024-04-05T22:31:00.097285+0000","last_clean":"2024-04-05T22:31:00.097285+0000","last_became_active":"2024-04-05T22:30:58.951263+0000","last_became_peered":"2024-04-05T22:30:58.951263+0000","last_unstale":"2024-04-05T22:31:00.097285+0000","last_undegraded":"2024-04-05T22:31:00.097285+0000","last_fullsized":"2024-04-05T22:31:00.097285+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:47:35.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098315+0000","last_change":"2024-04-05T22:30:58.948076+0000","last_active":"2024-04-05T22:31:00.098315+0000","last_peered":"2024-04-05T22:31:00.098315+0000","last_clean":"2024-04-05T22:31:00.098315+0000","last_became_active":"2024-04-05T22:30:58.946060+0000","last_became_peered":"2024-04-05T22:30:58.946060+0000","last_unstale":"2024-04-05T22:31:00.098315+0000","last_undegraded":"2024-04-05T22:31:00.098315+0000","last_fullsized":"2024-04-05T22:31:00.098315+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:17:14.652120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097875+0000","last_change":"2024-04-05T22:30:58.943701+0000","last_active":"2024-04-05T22:31:00.097875+0000","last_peered":"2024-04-05T22:31:00.097875+0000","last_clean":"2024-04-05T22:31:00.097875+0000","last_became_active":"2024-04-05T22:30:58.943449+0000","last_became_peered":"2024-04-05T22:30:58.943449+0000","last_unstale":"2024-04-05T22:31:00.097875+0000","last_undegraded":"2024-04-05T22:31:00.097875+0000","last_fullsized":"2024-04-05T22:31:00.097875+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:28.770826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.096639+0000","last_change":"2024-04-05T22:30:58.935994+0000","last_active":"2024-04-05T22:31:00.096639+0000","last_peered":"2024-04-05T22:31:00.096639+0000","last_clean":"2024-04-05T22:31:00.096639+0000","last_became_active":"2024-04-05T22:30:58.934363+0000","last_became_peered":"2024-04-05T22:30:58.934363+0000","last_unstale":"2024-04-05T22:31:00.096639+0000","last_undegraded":"2024-04-05T22:31:00.096639+0000","last_fullsized":"2024-04-05T22:31:00.096639+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:28.115985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.100487+0000","last_change":"2024-04-05T22:30:58.957058+0000","last_active":"2024-04-05T22:31:00.100487+0000","last_peered":"2024-04-05T22:31:00.100487+0000","last_clean":"2024-04-05T22:31:00.100487+0000","last_became_active":"2024-04-05T22:30:58.955806+0000","last_became_peered":"2024-04-05T22:30:58.955806+0000","last_unstale":"2024-04-05T22:31:00.100487+0000","last_undegraded":"2024-04-05T22:31:00.100487+0000","last_fullsized":"2024-04-05T22:31:00.100487+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:32:20.940660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936211+0000","last_change":"2024-04-05T22:30:58.951293+0000","last_active":"2024-04-05T22:30:59.936211+0000","last_peered":"2024-04-05T22:30:59.936211+0000","last_clean":"2024-04-05T22:30:59.936211+0000","last_became_active":"2024-04-05T22:30:58.951013+0000","last_became_peered":"2024-04-05T22:30:58.951013+0000","last_unstale":"2024-04-05T22:30:59.936211+0000","last_undegraded":"2024-04-05T22:30:59.936211+0000","last_fullsized":"2024-04-05T22:30:59.936211+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:18:04.460508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.096102+0000","last_change":"2024-04-05T22:30:58.932225+0000","last_active":"2024-04-05T22:31:00.096102+0000","last_peered":"2024-04-05T22:31:00.096102+0000","last_clean":"2024-04-05T22:31:00.096102+0000","last_became_active":"2024-04-05T22:30:58.931988+0000","last_became_peered":"2024-04-05T22:30:58.931988+0000","last_unstale":"2024-04-05T22:31:00.096102+0000","last_undegraded":"2024-04-05T22:31:00.096102+0000","last_fullsized":"2024-04-05T22:31:00.096102+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:18.976628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.096081+0000","last_change":"2024-04-05T22:30:58.943471+0000","last_active":"2024-04-05T22:31:00.096081+0000","last_peered":"2024-04-05T22:31:00.096081+0000","last_clean":"2024-04-05T22:31:00.096081+0000","last_became_active":"2024-04-05T22:30:58.941671+0000","last_became_peered":"2024-04-05T22:30:58.941671+0000","last_unstale":"2024-04-05T22:31:00.096081+0000","last_undegraded":"2024-04-05T22:31:00.096081+0000","last_fullsized":"2024-04-05T22:31:00.096081+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:17:44.665776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935250+0000","last_change":"2024-04-05T22:30:58.957807+0000","last_active":"2024-04-05T22:30:59.935250+0000","last_peered":"2024-04-05T22:30:59.935250+0000","last_clean":"2024-04-05T22:30:59.935250+0000","last_became_active":"2024-04-05T22:30:58.957566+0000","last_became_peered":"2024-04-05T22:30:58.957566+0000","last_unstale":"2024-04-05T22:30:59.935250+0000","last_undegraded":"2024-04-05T22:30:59.935250+0000","last_fullsized":"2024-04-05T22:30:59.935250+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:53:14.431630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935097+0000","last_change":"2024-04-05T22:30:58.947994+0000","last_active":"2024-04-05T22:30:59.935097+0000","last_peered":"2024-04-05T22:30:59.935097+0000","last_clean":"2024-04-05T22:30:59.935097+0000","last_became_active":"2024-04-05T22:30:58.947750+0000","last_became_peered":"2024-04-05T22:30:58.947750+0000","last_unstale":"2024-04-05T22:30:59.935097+0000","last_undegraded":"2024-04-05T22:30:59.935097+0000","last_fullsized":"2024-04-05T22:30:59.935097+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:56:50.009243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097934+0000","last_change":"2024-04-05T22:30:58.947980+0000","last_active":"2024-04-05T22:31:00.097934+0000","last_peered":"2024-04-05T22:31:00.097934+0000","last_clean":"2024-04-05T22:31:00.097934+0000","last_became_active":"2024-04-05T22:30:58.947712+0000","last_became_peered":"2024-04-05T22:30:58.947712+0000","last_unstale":"2024-04-05T22:31:00.097934+0000","last_undegraded":"2024-04-05T22:31:00.097934+0000","last_fullsized":"2024-04-05T22:31:00.097934+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:56:32.743279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099387+0000","last_change":"2024-04-05T22:30:58.951536+0000","last_active":"2024-04-05T22:31:00.099387+0000","last_peered":"2024-04-05T22:31:00.099387+0000","last_clean":"2024-04-05T22:31:00.099387+0000","last_became_active":"2024-04-05T22:30:58.949414+0000","last_became_peered":"2024-04-05T22:30:58.949414+0000","last_unstale":"2024-04-05T22:31:00.099387+0000","last_undegraded":"2024-04-05T22:31:00.099387+0000","last_fullsized":"2024-04-05T22:31:00.099387+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:57.383728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936663+0000","last_change":"2024-04-05T22:30:58.953779+0000","last_active":"2024-04-05T22:30:59.936663+0000","last_peered":"2024-04-05T22:30:59.936663+0000","last_clean":"2024-04-05T22:30:59.936663+0000","last_became_active":"2024-04-05T22:30:58.953350+0000","last_became_peered":"2024-04-05T22:30:58.953350+0000","last_unstale":"2024-04-05T22:30:59.936663+0000","last_undegraded":"2024-04-05T22:30:59.936663+0000","last_fullsized":"2024-04-05T22:30:59.936663+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:28.180547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936211+0000","last_change":"2024-04-05T22:30:58.947524+0000","last_active":"2024-04-05T22:30:59.936211+0000","last_peered":"2024-04-05T22:30:59.936211+0000","last_clean":"2024-04-05T22:30:59.936211+0000","last_became_active":"2024-04-05T22:30:58.947178+0000","last_became_peered":"2024-04-05T22:30:58.947178+0000","last_unstale":"2024-04-05T22:30:59.936211+0000","last_undegraded":"2024-04-05T22:30:59.936211+0000","last_fullsized":"2024-04-05T22:30:59.936211+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:37:33.405386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.096054+0000","last_change":"2024-04-05T22:30:58.945134+0000","last_active":"2024-04-05T22:31:00.096054+0000","last_peered":"2024-04-05T22:31:00.096054+0000","last_clean":"2024-04-05T22:31:00.096054+0000","last_became_active":"2024-04-05T22:30:58.942476+0000","last_became_peered":"2024-04-05T22:30:58.942476+0000","last_unstale":"2024-04-05T22:31:00.096054+0000","last_undegraded":"2024-04-05T22:31:00.096054+0000","last_fullsized":"2024-04-05T22:31:00.096054+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:28:45.093854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098224+0000","last_change":"2024-04-05T22:30:58.950520+0000","last_active":"2024-04-05T22:31:00.098224+0000","last_peered":"2024-04-05T22:31:00.098224+0000","last_clean":"2024-04-05T22:31:00.098224+0000","last_became_active":"2024-04-05T22:30:58.950220+0000","last_became_peered":"2024-04-05T22:30:58.950220+0000","last_unstale":"2024-04-05T22:31:00.098224+0000","last_undegraded":"2024-04-05T22:31:00.098224+0000","last_fullsized":"2024-04-05T22:31:00.098224+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:11:33.346384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098574+0000","last_change":"2024-04-05T22:30:58.939416+0000","last_active":"2024-04-05T22:31:00.098574+0000","last_peered":"2024-04-05T22:31:00.098574+0000","last_clean":"2024-04-05T22:31:00.098574+0000","last_became_active":"2024-04-05T22:30:58.939098+0000","last_became_peered":"2024-04-05T22:30:58.939098+0000","last_unstale":"2024-04-05T22:31:00.098574+0000","last_undegraded":"2024-04-05T22:31:00.098574+0000","last_fullsized":"2024-04-05T22:31:00.098574+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:18:00.124107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935347+0000","last_change":"2024-04-05T22:30:58.935023+0000","last_active":"2024-04-05T22:30:59.935347+0000","last_peered":"2024-04-05T22:30:59.935347+0000","last_clean":"2024-04-05T22:30:59.935347+0000","last_became_active":"2024-04-05T22:30:58.934671+0000","last_became_peered":"2024-04-05T22:30:58.934671+0000","last_unstale":"2024-04-05T22:30:59.935347+0000","last_undegraded":"2024-04-05T22:30:59.935347+0000","last_fullsized":"2024-04-05T22:30:59.935347+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:31:02.492142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.095502+0000","last_change":"2024-04-05T22:30:58.939036+0000","last_active":"2024-04-05T22:31:00.095502+0000","last_peered":"2024-04-05T22:31:00.095502+0000","last_clean":"2024-04-05T22:31:00.095502+0000","last_became_active":"2024-04-05T22:30:58.938780+0000","last_became_peered":"2024-04-05T22:30:58.938780+0000","last_unstale":"2024-04-05T22:31:00.095502+0000","last_undegraded":"2024-04-05T22:31:00.095502+0000","last_fullsized":"2024-04-05T22:31:00.095502+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:15:17.272105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098671+0000","last_change":"2024-04-05T22:30:58.959217+0000","last_active":"2024-04-05T22:31:00.098671+0000","last_peered":"2024-04-05T22:31:00.098671+0000","last_clean":"2024-04-05T22:31:00.098671+0000","last_became_active":"2024-04-05T22:30:58.958947+0000","last_became_peered":"2024-04-05T22:30:58.958947+0000","last_unstale":"2024-04-05T22:31:00.098671+0000","last_undegraded":"2024-04-05T22:31:00.098671+0000","last_fullsized":"2024-04-05T22:31:00.098671+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:53:14.431630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"159'4","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.957336+0000","last_change":"2024-04-05T22:30:58.476562+0000","last_active":"2024-04-05T22:30:58.957336+0000","last_peered":"2024-04-05T22:30:58.957336+0000","last_clean":"2024-04-05T22:30:58.957336+0000","last_became_active":"2024-04-05T22:30:58.476294+0000","last_became_peered":"2024-04-05T22:30:58.476294+0000","last_unstale":"2024-04-05T22:30:58.957336+0000","last_undegraded":"2024-04-05T22:30:58.957336+0000","last_fullsized":"2024-04-05T22:30:58.957336+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:00:43.346196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937515+0000","last_change":"2024-04-05T22:30:58.954935+0000","last_active":"2024-04-05T22:30:59.937515+0000","last_peered":"2024-04-05T22:30:59.937515+0000","last_clean":"2024-04-05T22:30:59.937515+0000","last_became_active":"2024-04-05T22:30:58.954438+0000","last_became_peered":"2024-04-05T22:30:58.954438+0000","last_unstale":"2024-04-05T22:30:59.937515+0000","last_undegraded":"2024-04-05T22:30:59.937515+0000","last_fullsized":"2024-04-05T22:30:59.937515+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:16:53.082868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"159'1","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.950092+0000","last_change":"2024-04-05T22:30:58.479463+0000","last_active":"2024-04-05T22:30:58.950092+0000","last_peered":"2024-04-05T22:30:58.950092+0000","last_clean":"2024-04-05T22:30:58.950092+0000","last_became_active":"2024-04-05T22:30:58.479105+0000","last_became_peered":"2024-04-05T22:30:58.479105+0000","last_unstale":"2024-04-05T22:30:58.950092+0000","last_undegraded":"2024-04-05T22:30:58.950092+0000","last_fullsized":"2024-04-05T22:30:58.950092+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:41.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936367+0000","last_change":"2024-04-05T22:30:58.938252+0000","last_active":"2024-04-05T22:30:59.936367+0000","last_peered":"2024-04-05T22:30:59.936367+0000","last_clean":"2024-04-05T22:30:59.936367+0000","last_became_active":"2024-04-05T22:30:58.937972+0000","last_became_peered":"2024-04-05T22:30:58.937972+0000","last_unstale":"2024-04-05T22:30:59.936367+0000","last_undegraded":"2024-04-05T22:30:59.936367+0000","last_fullsized":"2024-04-05T22:30:59.936367+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:18.976628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.949601+0000","last_change":"2024-04-05T22:30:58.466904+0000","last_active":"2024-04-05T22:30:58.949601+0000","last_peered":"2024-04-05T22:30:58.949601+0000","last_clean":"2024-04-05T22:30:58.949601+0000","last_became_active":"2024-04-05T22:30:58.465685+0000","last_became_peered":"2024-04-05T22:30:58.465685+0000","last_unstale":"2024-04-05T22:30:58.949601+0000","last_undegraded":"2024-04-05T22:30:58.949601+0000","last_fullsized":"2024-04-05T22:30:58.949601+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:17.958611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.100219+0000","last_change":"2024-04-05T22:30:58.955552+0000","last_active":"2024-04-05T22:31:00.100219+0000","last_peered":"2024-04-05T22:31:00.100219+0000","last_clean":"2024-04-05T22:31:00.100219+0000","last_became_active":"2024-04-05T22:30:58.955238+0000","last_became_peered":"2024-04-05T22:30:58.955238+0000","last_unstale":"2024-04-05T22:31:00.100219+0000","last_undegraded":"2024-04-05T22:31:00.100219+0000","last_fullsized":"2024-04-05T22:31:00.100219+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:35.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.951041+0000","last_change":"2024-04-05T22:30:58.476862+0000","last_active":"2024-04-05T22:30:58.951041+0000","last_peered":"2024-04-05T22:30:58.951041+0000","last_clean":"2024-04-05T22:30:58.951041+0000","last_became_active":"2024-04-05T22:30:58.476563+0000","last_became_peered":"2024-04-05T22:30:58.476563+0000","last_unstale":"2024-04-05T22:30:58.951041+0000","last_undegraded":"2024-04-05T22:30:58.951041+0000","last_fullsized":"2024-04-05T22:30:58.951041+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:15:04.611779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935368+0000","last_change":"2024-04-05T22:30:58.958787+0000","last_active":"2024-04-05T22:30:59.935368+0000","last_peered":"2024-04-05T22:30:59.935368+0000","last_clean":"2024-04-05T22:30:59.935368+0000","last_became_active":"2024-04-05T22:30:58.957836+0000","last_became_peered":"2024-04-05T22:30:58.957836+0000","last_unstale":"2024-04-05T22:30:59.935368+0000","last_undegraded":"2024-04-05T22:30:59.935368+0000","last_fullsized":"2024-04-05T22:30:59.935368+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:24:41.042843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"159'1","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.952049+0000","last_change":"2024-04-05T22:30:58.474771+0000","last_active":"2024-04-05T22:30:58.952049+0000","last_peered":"2024-04-05T22:30:58.952049+0000","last_clean":"2024-04-05T22:30:58.952049+0000","last_became_active":"2024-04-05T22:30:58.474505+0000","last_became_peered":"2024-04-05T22:30:58.474505+0000","last_unstale":"2024-04-05T22:30:58.952049+0000","last_undegraded":"2024-04-05T22:30:58.952049+0000","last_fullsized":"2024-04-05T22:30:58.952049+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:31:27.787632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098026+0000","last_change":"2024-04-05T22:30:58.945318+0000","last_active":"2024-04-05T22:31:00.098026+0000","last_peered":"2024-04-05T22:31:00.098026+0000","last_clean":"2024-04-05T22:31:00.098026+0000","last_became_active":"2024-04-05T22:30:58.945042+0000","last_became_peered":"2024-04-05T22:30:58.945042+0000","last_unstale":"2024-04-05T22:31:00.098026+0000","last_undegraded":"2024-04-05T22:31:00.098026+0000","last_fullsized":"2024-04-05T22:31:00.098026+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:24:41.042843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"159'1","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.958193+0000","last_change":"2024-04-05T22:30:58.460562+0000","last_active":"2024-04-05T22:30:58.958193+0000","last_peered":"2024-04-05T22:30:58.958193+0000","last_clean":"2024-04-05T22:30:58.958193+0000","last_became_active":"2024-04-05T22:30:58.460030+0000","last_became_peered":"2024-04-05T22:30:58.460030+0000","last_unstale":"2024-04-05T22:30:58.958193+0000","last_undegraded":"2024-04-05T22:30:58.958193+0000","last_fullsized":"2024-04-05T22:30:58.958193+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:12.716550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936258+0000","last_change":"2024-04-05T22:30:58.956428+0000","last_active":"2024-04-05T22:30:59.936258+0000","last_peered":"2024-04-05T22:30:59.936258+0000","last_clean":"2024-04-05T22:30:59.936258+0000","last_became_active":"2024-04-05T22:30:58.955415+0000","last_became_peered":"2024-04-05T22:30:58.955415+0000","last_unstale":"2024-04-05T22:30:59.936258+0000","last_undegraded":"2024-04-05T22:30:59.936258+0000","last_fullsized":"2024-04-05T22:30:59.936258+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:28.115985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.939908+0000","last_change":"2024-04-05T22:30:58.477560+0000","last_active":"2024-04-05T22:30:58.939908+0000","last_peered":"2024-04-05T22:30:58.939908+0000","last_clean":"2024-04-05T22:30:58.939908+0000","last_became_active":"2024-04-05T22:30:58.477198+0000","last_became_peered":"2024-04-05T22:30:58.477198+0000","last_unstale":"2024-04-05T22:30:58.939908+0000","last_undegraded":"2024-04-05T22:30:58.939908+0000","last_fullsized":"2024-04-05T22:30:58.939908+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:27.097967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937656+0000","last_change":"2024-04-05T22:30:58.946492+0000","last_active":"2024-04-05T22:30:59.937656+0000","last_peered":"2024-04-05T22:30:59.937656+0000","last_clean":"2024-04-05T22:30:59.937656+0000","last_became_active":"2024-04-05T22:30:58.946234+0000","last_became_peered":"2024-04-05T22:30:58.946234+0000","last_unstale":"2024-04-05T22:30:59.937656+0000","last_undegraded":"2024-04-05T22:30:59.937656+0000","last_fullsized":"2024-04-05T22:30:59.937656+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:44:00.790777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.934684+0000","last_change":"2024-04-05T22:30:58.474127+0000","last_active":"2024-04-05T22:30:58.934684+0000","last_peered":"2024-04-05T22:30:58.934684+0000","last_clean":"2024-04-05T22:30:58.934684+0000","last_became_active":"2024-04-05T22:30:58.473104+0000","last_became_peered":"2024-04-05T22:30:58.473104+0000","last_unstale":"2024-04-05T22:30:58.934684+0000","last_undegraded":"2024-04-05T22:30:58.934684+0000","last_fullsized":"2024-04-05T22:30:58.934684+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:34.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098674+0000","last_change":"2024-04-05T22:30:58.953748+0000","last_active":"2024-04-05T22:31:00.098674+0000","last_peered":"2024-04-05T22:31:00.098674+0000","last_clean":"2024-04-05T22:31:00.098674+0000","last_became_active":"2024-04-05T22:30:58.953464+0000","last_became_peered":"2024-04-05T22:30:58.953464+0000","last_unstale":"2024-04-05T22:31:00.098674+0000","last_undegraded":"2024-04-05T22:31:00.098674+0000","last_fullsized":"2024-04-05T22:31:00.098674+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:06.637606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.924924+0000","last_change":"2024-04-05T22:30:58.478175+0000","last_active":"2024-04-05T22:30:58.924924+0000","last_peered":"2024-04-05T22:30:58.924924+0000","last_clean":"2024-04-05T22:30:58.924924+0000","last_became_active":"2024-04-05T22:30:58.477911+0000","last_became_peered":"2024-04-05T22:30:58.477911+0000","last_unstale":"2024-04-05T22:30:58.924924+0000","last_undegraded":"2024-04-05T22:30:58.924924+0000","last_fullsized":"2024-04-05T22:30:58.924924+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:52:12.128598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936407+0000","last_change":"2024-04-05T22:30:58.951961+0000","last_active":"2024-04-05T22:30:59.936407+0000","last_peered":"2024-04-05T22:30:59.936407+0000","last_clean":"2024-04-05T22:30:59.936407+0000","last_became_active":"2024-04-05T22:30:58.951684+0000","last_became_peered":"2024-04-05T22:30:58.951684+0000","last_unstale":"2024-04-05T22:30:59.936407+0000","last_undegraded":"2024-04-05T22:30:59.936407+0000","last_fullsized":"2024-04-05T22:30:59.936407+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:35.959088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"159'3","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.957012+0000","last_change":"2024-04-05T22:30:58.475747+0000","last_active":"2024-04-05T22:30:58.957012+0000","last_peered":"2024-04-05T22:30:58.957012+0000","last_clean":"2024-04-05T22:30:58.957012+0000","last_became_active":"2024-04-05T22:30:58.475439+0000","last_became_peered":"2024-04-05T22:30:58.475439+0000","last_unstale":"2024-04-05T22:30:58.957012+0000","last_undegraded":"2024-04-05T22:30:58.957012+0000","last_fullsized":"2024-04-05T22:30:58.957012+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:31.337633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936312+0000","last_change":"2024-04-05T22:30:58.953824+0000","last_active":"2024-04-05T22:30:59.936312+0000","last_peered":"2024-04-05T22:30:59.936312+0000","last_clean":"2024-04-05T22:30:59.936312+0000","last_became_active":"2024-04-05T22:30:58.953498+0000","last_became_peered":"2024-04-05T22:30:58.953498+0000","last_unstale":"2024-04-05T22:30:59.936312+0000","last_undegraded":"2024-04-05T22:30:59.936312+0000","last_fullsized":"2024-04-05T22:30:59.936312+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:52:12.873988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"159'6","reported_seq":17,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.960832+0000","last_change":"2024-04-05T22:30:58.462560+0000","last_active":"2024-04-05T22:30:58.960832+0000","last_peered":"2024-04-05T22:30:58.960832+0000","last_clean":"2024-04-05T22:30:58.960832+0000","last_became_active":"2024-04-05T22:30:58.462247+0000","last_became_peered":"2024-04-05T22:30:58.462247+0000","last_unstale":"2024-04-05T22:30:58.960832+0000","last_undegraded":"2024-04-05T22:30:58.960832+0000","last_fullsized":"2024-04-05T22:30:58.960832+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:34.941070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.934962+0000","last_change":"2024-04-05T22:30:58.950536+0000","last_active":"2024-04-05T22:30:59.934962+0000","last_peered":"2024-04-05T22:30:59.934962+0000","last_clean":"2024-04-05T22:30:59.934962+0000","last_became_active":"2024-04-05T22:30:58.950232+0000","last_became_peered":"2024-04-05T22:30:58.950232+0000","last_unstale":"2024-04-05T22:30:59.934962+0000","last_undegraded":"2024-04-05T22:30:59.934962+0000","last_fullsized":"2024-04-05T22:30:59.934962+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:14.556023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.959186+0000","last_change":"2024-04-05T22:30:58.477081+0000","last_active":"2024-04-05T22:30:58.959186+0000","last_peered":"2024-04-05T22:30:58.959186+0000","last_clean":"2024-04-05T22:30:58.959186+0000","last_became_active":"2024-04-05T22:30:58.476807+0000","last_became_peered":"2024-04-05T22:30:58.476807+0000","last_unstale":"2024-04-05T22:30:58.959186+0000","last_undegraded":"2024-04-05T22:30:58.959186+0000","last_fullsized":"2024-04-05T22:30:58.959186+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:27.162529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937112+0000","last_change":"2024-04-05T22:30:58.952862+0000","last_active":"2024-04-05T22:30:59.937112+0000","last_peered":"2024-04-05T22:30:59.937112+0000","last_clean":"2024-04-05T22:30:59.937112+0000","last_became_active":"2024-04-05T22:30:58.952500+0000","last_became_peered":"2024-04-05T22:30:58.952500+0000","last_unstale":"2024-04-05T22:30:59.937112+0000","last_undegraded":"2024-04-05T22:30:59.937112+0000","last_fullsized":"2024-04-05T22:30:59.937112+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:06.725415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.945508+0000","last_change":"2024-04-05T22:30:58.474587+0000","last_active":"2024-04-05T22:30:58.945508+0000","last_peered":"2024-04-05T22:30:58.945508+0000","last_clean":"2024-04-05T22:30:58.945508+0000","last_became_active":"2024-04-05T22:30:58.473779+0000","last_became_peered":"2024-04-05T22:30:58.473779+0000","last_unstale":"2024-04-05T22:30:58.945508+0000","last_undegraded":"2024-04-05T22:30:58.945508+0000","last_fullsized":"2024-04-05T22:30:58.945508+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:15.253748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097190+0000","last_change":"2024-04-05T22:30:58.953599+0000","last_active":"2024-04-05T22:31:00.097190+0000","last_peered":"2024-04-05T22:31:00.097190+0000","last_clean":"2024-04-05T22:31:00.097190+0000","last_became_active":"2024-04-05T22:30:58.953206+0000","last_became_peered":"2024-04-05T22:30:58.953206+0000","last_unstale":"2024-04-05T22:31:00.097190+0000","last_undegraded":"2024-04-05T22:31:00.097190+0000","last_fullsized":"2024-04-05T22:31:00.097190+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:22:34.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.934073+0000","last_change":"2024-04-05T22:30:58.470126+0000","last_active":"2024-04-05T22:30:58.934073+0000","last_peered":"2024-04-05T22:30:58.934073+0000","last_clean":"2024-04-05T22:30:58.934073+0000","last_became_active":"2024-04-05T22:30:58.469859+0000","last_became_peered":"2024-04-05T22:30:58.469859+0000","last_unstale":"2024-04-05T22:30:58.934073+0000","last_undegraded":"2024-04-05T22:30:58.934073+0000","last_fullsized":"2024-04-05T22:30:58.934073+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:51:14.094034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935456+0000","last_change":"2024-04-05T22:30:58.950063+0000","last_active":"2024-04-05T22:30:59.935456+0000","last_peered":"2024-04-05T22:30:59.935456+0000","last_clean":"2024-04-05T22:30:59.935456+0000","last_became_active":"2024-04-05T22:30:58.949743+0000","last_became_peered":"2024-04-05T22:30:58.949743+0000","last_unstale":"2024-04-05T22:30:59.935456+0000","last_undegraded":"2024-04-05T22:30:59.935456+0000","last_fullsized":"2024-04-05T22:30:59.935456+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:13:06.637606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"159'8","reported_seq":19,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.960527+0000","last_change":"2024-04-05T22:30:58.475420+0000","last_active":"2024-04-05T22:30:58.960527+0000","last_peered":"2024-04-05T22:30:58.960527+0000","last_clean":"2024-04-05T22:30:58.960527+0000","last_became_active":"2024-04-05T22:30:58.475127+0000","last_became_peered":"2024-04-05T22:30:58.475127+0000","last_unstale":"2024-04-05T22:30:58.960527+0000","last_undegraded":"2024-04-05T22:30:58.960527+0000","last_fullsized":"2024-04-05T22:30:58.960527+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:20:17.958611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937454+0000","last_change":"2024-04-05T22:30:58.956123+0000","last_active":"2024-04-05T22:30:59.937454+0000","last_peered":"2024-04-05T22:30:59.937454+0000","last_clean":"2024-04-05T22:30:59.937454+0000","last_became_active":"2024-04-05T22:30:58.955858+0000","last_became_peered":"2024-04-05T22:30:58.955858+0000","last_unstale":"2024-04-05T22:30:59.937454+0000","last_undegraded":"2024-04-05T22:30:59.937454+0000","last_fullsized":"2024-04-05T22:30:59.937454+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:17:28.770826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"159'1","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.952554+0000","last_change":"2024-04-05T22:30:58.474144+0000","last_active":"2024-04-05T22:30:58.952554+0000","last_peered":"2024-04-05T22:30:58.952554+0000","last_clean":"2024-04-05T22:30:58.952554+0000","last_became_active":"2024-04-05T22:30:58.473807+0000","last_became_peered":"2024-04-05T22:30:58.473807+0000","last_unstale":"2024-04-05T22:30:58.952554+0000","last_undegraded":"2024-04-05T22:30:58.952554+0000","last_fullsized":"2024-04-05T22:30:58.952554+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:32:41.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099621+0000","last_change":"2024-04-05T22:30:58.951275+0000","last_active":"2024-04-05T22:31:00.099621+0000","last_peered":"2024-04-05T22:31:00.099621+0000","last_clean":"2024-04-05T22:31:00.099621+0000","last_became_active":"2024-04-05T22:30:58.951033+0000","last_became_peered":"2024-04-05T22:30:58.951033+0000","last_unstale":"2024-04-05T22:31:00.099621+0000","last_undegraded":"2024-04-05T22:31:00.099621+0000","last_fullsized":"2024-04-05T22:31:00.099621+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:15:17.272105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.958739+0000","last_change":"2024-04-05T22:30:58.454624+0000","last_active":"2024-04-05T22:30:58.958739+0000","last_peered":"2024-04-05T22:30:58.958739+0000","last_clean":"2024-04-05T22:30:58.958739+0000","last_became_active":"2024-04-05T22:30:58.454311+0000","last_became_peered":"2024-04-05T22:30:58.454311+0000","last_unstale":"2024-04-05T22:30:58.958739+0000","last_undegraded":"2024-04-05T22:30:58.958739+0000","last_fullsized":"2024-04-05T22:30:58.958739+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:55.270524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.099294+0000","last_change":"2024-04-05T22:30:58.950526+0000","last_active":"2024-04-05T22:31:00.099294+0000","last_peered":"2024-04-05T22:31:00.099294+0000","last_clean":"2024-04-05T22:31:00.099294+0000","last_became_active":"2024-04-05T22:30:58.949910+0000","last_became_peered":"2024-04-05T22:30:58.949910+0000","last_unstale":"2024-04-05T22:31:00.099294+0000","last_undegraded":"2024-04-05T22:31:00.099294+0000","last_fullsized":"2024-04-05T22:31:00.099294+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:14:58.878155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"159'1","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.958525+0000","last_change":"2024-04-05T22:30:58.476545+0000","last_active":"2024-04-05T22:30:58.958525+0000","last_peered":"2024-04-05T22:30:58.958525+0000","last_clean":"2024-04-05T22:30:58.958525+0000","last_became_active":"2024-04-05T22:30:58.476294+0000","last_became_peered":"2024-04-05T22:30:58.476294+0000","last_unstale":"2024-04-05T22:30:58.958525+0000","last_undegraded":"2024-04-05T22:30:58.958525+0000","last_fullsized":"2024-04-05T22:30:58.958525+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:05.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098065+0000","last_change":"2024-04-05T22:30:58.947510+0000","last_active":"2024-04-05T22:31:00.098065+0000","last_peered":"2024-04-05T22:31:00.098065+0000","last_clean":"2024-04-05T22:31:00.098065+0000","last_became_active":"2024-04-05T22:30:58.947232+0000","last_became_peered":"2024-04-05T22:30:58.947232+0000","last_unstale":"2024-04-05T22:31:00.098065+0000","last_undegraded":"2024-04-05T22:31:00.098065+0000","last_fullsized":"2024-04-05T22:31:00.098065+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:37:32.428193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.936305+0000","last_change":"2024-04-05T22:30:58.464428+0000","last_active":"2024-04-05T22:30:58.936305+0000","last_peered":"2024-04-05T22:30:58.936305+0000","last_clean":"2024-04-05T22:30:58.936305+0000","last_became_active":"2024-04-05T22:30:58.463551+0000","last_became_peered":"2024-04-05T22:30:58.463551+0000","last_unstale":"2024-04-05T22:30:58.936305+0000","last_undegraded":"2024-04-05T22:30:58.936305+0000","last_fullsized":"2024-04-05T22:30:58.936305+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:17:59.106090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.935465+0000","last_change":"2024-04-05T22:30:58.953904+0000","last_active":"2024-04-05T22:30:59.935465+0000","last_peered":"2024-04-05T22:30:59.935465+0000","last_clean":"2024-04-05T22:30:59.935465+0000","last_became_active":"2024-04-05T22:30:58.953573+0000","last_became_peered":"2024-04-05T22:30:58.953573+0000","last_unstale":"2024-04-05T22:30:59.935465+0000","last_undegraded":"2024-04-05T22:30:59.935465+0000","last_fullsized":"2024-04-05T22:30:59.935465+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:43:03.123987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.953727+0000","last_change":"2024-04-05T22:30:58.474847+0000","last_active":"2024-04-05T22:30:58.953727+0000","last_peered":"2024-04-05T22:30:58.953727+0000","last_clean":"2024-04-05T22:30:58.953727+0000","last_became_active":"2024-04-05T22:30:58.474558+0000","last_became_peered":"2024-04-05T22:30:58.474558+0000","last_unstale":"2024-04-05T22:30:58.953727+0000","last_undegraded":"2024-04-05T22:30:58.953727+0000","last_fullsized":"2024-04-05T22:30:58.953727+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:06:28.182098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937035+0000","last_change":"2024-04-05T22:30:58.952312+0000","last_active":"2024-04-05T22:30:59.937035+0000","last_peered":"2024-04-05T22:30:59.937035+0000","last_clean":"2024-04-05T22:30:59.937035+0000","last_became_active":"2024-04-05T22:30:58.949171+0000","last_became_peered":"2024-04-05T22:30:58.949171+0000","last_unstale":"2024-04-05T22:30:59.937035+0000","last_undegraded":"2024-04-05T22:30:59.937035+0000","last_fullsized":"2024-04-05T22:30:59.937035+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:51:09.688994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.923903+0000","last_change":"2024-04-05T22:30:58.473760+0000","last_active":"2024-04-05T22:30:58.923903+0000","last_peered":"2024-04-05T22:30:58.923903+0000","last_clean":"2024-04-05T22:30:58.923903+0000","last_became_active":"2024-04-05T22:30:58.473321+0000","last_became_peered":"2024-04-05T22:30:58.473321+0000","last_unstale":"2024-04-05T22:30:58.923903+0000","last_undegraded":"2024-04-05T22:30:58.923903+0000","last_fullsized":"2024-04-05T22:30:58.923903+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:16:22.654879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097410+0000","last_change":"2024-04-05T22:30:58.947726+0000","last_active":"2024-04-05T22:31:00.097410+0000","last_peered":"2024-04-05T22:31:00.097410+0000","last_clean":"2024-04-05T22:31:00.097410+0000","last_became_active":"2024-04-05T22:30:58.947167+0000","last_became_peered":"2024-04-05T22:30:58.947167+0000","last_unstale":"2024-04-05T22:31:00.097410+0000","last_undegraded":"2024-04-05T22:31:00.097410+0000","last_fullsized":"2024-04-05T22:31:00.097410+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:11:54.866360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.931958+0000","last_change":"2024-04-05T22:30:58.473725+0000","last_active":"2024-04-05T22:30:58.931958+0000","last_peered":"2024-04-05T22:30:58.931958+0000","last_clean":"2024-04-05T22:30:58.931958+0000","last_became_active":"2024-04-05T22:30:58.473181+0000","last_became_peered":"2024-04-05T22:30:58.473181+0000","last_unstale":"2024-04-05T22:30:58.931958+0000","last_undegraded":"2024-04-05T22:30:58.931958+0000","last_fullsized":"2024-04-05T22:30:58.931958+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:05.255712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.095528+0000","last_change":"2024-04-05T22:30:58.936406+0000","last_active":"2024-04-05T22:31:00.095528+0000","last_peered":"2024-04-05T22:31:00.095528+0000","last_clean":"2024-04-05T22:31:00.095528+0000","last_became_active":"2024-04-05T22:30:58.936141+0000","last_became_peered":"2024-04-05T22:30:58.936141+0000","last_unstale":"2024-04-05T22:31:00.095528+0000","last_undegraded":"2024-04-05T22:31:00.095528+0000","last_fullsized":"2024-04-05T22:31:00.095528+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:02.157872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.931081+0000","last_change":"2024-04-05T22:30:58.469986+0000","last_active":"2024-04-05T22:30:58.931081+0000","last_peered":"2024-04-05T22:30:58.931081+0000","last_clean":"2024-04-05T22:30:58.931081+0000","last_became_active":"2024-04-05T22:30:58.469555+0000","last_became_peered":"2024-04-05T22:30:58.469555+0000","last_unstale":"2024-04-05T22:30:58.931081+0000","last_undegraded":"2024-04-05T22:30:58.931081+0000","last_fullsized":"2024-04-05T22:30:58.931081+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:48:50.166340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.095804+0000","last_change":"2024-04-05T22:30:58.943345+0000","last_active":"2024-04-05T22:31:00.095804+0000","last_peered":"2024-04-05T22:31:00.095804+0000","last_clean":"2024-04-05T22:31:00.095804+0000","last_became_active":"2024-04-05T22:30:58.942240+0000","last_became_peered":"2024-04-05T22:30:58.942240+0000","last_unstale":"2024-04-05T22:31:00.095804+0000","last_undegraded":"2024-04-05T22:31:00.095804+0000","last_fullsized":"2024-04-05T22:31:00.095804+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:08:28.180547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"159'2","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.955230+0000","last_change":"2024-04-05T22:30:58.476024+0000","last_active":"2024-04-05T22:30:58.955230+0000","last_peered":"2024-04-05T22:30:58.955230+0000","last_clean":"2024-04-05T22:30:58.955230+0000","last_became_active":"2024-04-05T22:30:58.475746+0000","last_became_peered":"2024-04-05T22:30:58.475746+0000","last_unstale":"2024-04-05T22:30:58.955230+0000","last_undegraded":"2024-04-05T22:30:58.955230+0000","last_fullsized":"2024-04-05T22:30:58.955230+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:39:34.941070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.937747+0000","last_change":"2024-04-05T22:30:58.950214+0000","last_active":"2024-04-05T22:30:59.937747+0000","last_peered":"2024-04-05T22:30:59.937747+0000","last_clean":"2024-04-05T22:30:59.937747+0000","last_became_active":"2024-04-05T22:30:58.949948+0000","last_became_peered":"2024-04-05T22:30:58.949948+0000","last_unstale":"2024-04-05T22:30:59.937747+0000","last_undegraded":"2024-04-05T22:30:59.937747+0000","last_fullsized":"2024-04-05T22:30:59.937747+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:30:00.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.945492+0000","last_change":"2024-04-05T22:30:58.466022+0000","last_active":"2024-04-05T22:30:58.945492+0000","last_peered":"2024-04-05T22:30:58.945492+0000","last_clean":"2024-04-05T22:30:58.945492+0000","last_became_active":"2024-04-05T22:30:58.465657+0000","last_became_peered":"2024-04-05T22:30:58.465657+0000","last_unstale":"2024-04-05T22:30:58.945492+0000","last_undegraded":"2024-04-05T22:30:58.945492+0000","last_fullsized":"2024-04-05T22:30:58.945492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:43:55.270524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098046+0000","last_change":"2024-04-05T22:30:58.946041+0000","last_active":"2024-04-05T22:31:00.098046+0000","last_peered":"2024-04-05T22:31:00.098046+0000","last_clean":"2024-04-05T22:31:00.098046+0000","last_became_active":"2024-04-05T22:30:58.945791+0000","last_became_peered":"2024-04-05T22:30:58.945791+0000","last_unstale":"2024-04-05T22:31:00.098046+0000","last_undegraded":"2024-04-05T22:31:00.098046+0000","last_fullsized":"2024-04-05T22:31:00.098046+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:44.785053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.945580+0000","last_change":"2024-04-05T22:30:58.466196+0000","last_active":"2024-04-05T22:30:58.945580+0000","last_peered":"2024-04-05T22:30:58.945580+0000","last_clean":"2024-04-05T22:30:58.945580+0000","last_became_active":"2024-04-05T22:30:58.465895+0000","last_became_peered":"2024-04-05T22:30:58.465895+0000","last_unstale":"2024-04-05T22:30:58.945580+0000","last_undegraded":"2024-04-05T22:30:58.945580+0000","last_fullsized":"2024-04-05T22:30:58.945580+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:02:12.716550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.098797+0000","last_change":"2024-04-05T22:30:58.957808+0000","last_active":"2024-04-05T22:31:00.098797+0000","last_peered":"2024-04-05T22:31:00.098797+0000","last_clean":"2024-04-05T22:31:00.098797+0000","last_became_active":"2024-04-05T22:30:58.957558+0000","last_became_peered":"2024-04-05T22:30:58.957558+0000","last_unstale":"2024-04-05T22:31:00.098797+0000","last_undegraded":"2024-04-05T22:31:00.098797+0000","last_fullsized":"2024-04-05T22:31:00.098797+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:30:00.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.941317+0000","last_change":"2024-04-05T22:30:58.456714+0000","last_active":"2024-04-05T22:30:58.941317+0000","last_peered":"2024-04-05T22:30:58.941317+0000","last_clean":"2024-04-05T22:30:58.941317+0000","last_became_active":"2024-04-05T22:30:58.456445+0000","last_became_peered":"2024-04-05T22:30:58.456445+0000","last_unstale":"2024-04-05T22:30:58.941317+0000","last_undegraded":"2024-04-05T22:30:58.941317+0000","last_fullsized":"2024-04-05T22:30:58.941317+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:56:15.253748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.100292+0000","last_change":"2024-04-05T22:30:58.957475+0000","last_active":"2024-04-05T22:31:00.100292+0000","last_peered":"2024-04-05T22:31:00.100292+0000","last_clean":"2024-04-05T22:31:00.100292+0000","last_became_active":"2024-04-05T22:30:58.957136+0000","last_became_peered":"2024-04-05T22:30:58.957136+0000","last_unstale":"2024-04-05T22:31:00.100292+0000","last_undegraded":"2024-04-05T22:31:00.100292+0000","last_fullsized":"2024-04-05T22:31:00.100292+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:11:06.216767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.941096+0000","last_change":"2024-04-05T22:30:58.471234+0000","last_active":"2024-04-05T22:30:58.941096+0000","last_peered":"2024-04-05T22:30:58.941096+0000","last_clean":"2024-04-05T22:30:58.941096+0000","last_became_active":"2024-04-05T22:30:58.470516+0000","last_became_peered":"2024-04-05T22:30:58.470516+0000","last_unstale":"2024-04-05T22:30:58.941096+0000","last_undegraded":"2024-04-05T22:30:58.941096+0000","last_fullsized":"2024-04-05T22:30:58.941096+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:14.387165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.100140+0000","last_change":"2024-04-05T22:30:58.958067+0000","last_active":"2024-04-05T22:31:00.100140+0000","last_peered":"2024-04-05T22:31:00.100140+0000","last_clean":"2024-04-05T22:31:00.100140+0000","last_became_active":"2024-04-05T22:30:58.957802+0000","last_became_peered":"2024-04-05T22:30:58.957802+0000","last_unstale":"2024-04-05T22:31:00.100140+0000","last_undegraded":"2024-04-05T22:31:00.100140+0000","last_fullsized":"2024-04-05T22:31:00.100140+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:58:30.253888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.930118+0000","last_change":"2024-04-05T22:30:58.476142+0000","last_active":"2024-04-05T22:30:58.930118+0000","last_peered":"2024-04-05T22:30:58.930118+0000","last_clean":"2024-04-05T22:30:58.930118+0000","last_became_active":"2024-04-05T22:30:58.475884+0000","last_became_peered":"2024-04-05T22:30:58.475884+0000","last_unstale":"2024-04-05T22:30:58.930118+0000","last_undegraded":"2024-04-05T22:30:58.930118+0000","last_fullsized":"2024-04-05T22:30:58.930118+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:05:58.488855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.097470+0000","last_change":"2024-04-05T22:30:58.935644+0000","last_active":"2024-04-05T22:31:00.097470+0000","last_peered":"2024-04-05T22:31:00.097470+0000","last_clean":"2024-04-05T22:31:00.097470+0000","last_became_active":"2024-04-05T22:30:58.934982+0000","last_became_peered":"2024-04-05T22:30:58.934982+0000","last_unstale":"2024-04-05T22:31:00.097470+0000","last_undegraded":"2024-04-05T22:31:00.097470+0000","last_fullsized":"2024-04-05T22:31:00.097470+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:39:56.159154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.926085+0000","last_change":"2024-04-05T22:30:58.476153+0000","last_active":"2024-04-05T22:30:58.926085+0000","last_peered":"2024-04-05T22:30:58.926085+0000","last_clean":"2024-04-05T22:30:58.926085+0000","last_became_active":"2024-04-05T22:30:58.475824+0000","last_became_peered":"2024-04-05T22:30:58.475824+0000","last_unstale":"2024-04-05T22:30:58.926085+0000","last_undegraded":"2024-04-05T22:30:58.926085+0000","last_fullsized":"2024-04-05T22:30:58.926085+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:13:01.300453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:31:00.095665+0000","last_change":"2024-04-05T22:30:58.932985+0000","last_active":"2024-04-05T22:31:00.095665+0000","last_peered":"2024-04-05T22:31:00.095665+0000","last_clean":"2024-04-05T22:31:00.095665+0000","last_became_active":"2024-04-05T22:30:58.932510+0000","last_became_peered":"2024-04-05T22:30:58.932510+0000","last_unstale":"2024-04-05T22:31:00.095665+0000","last_undegraded":"2024-04-05T22:31:00.095665+0000","last_fullsized":"2024-04-05T22:31:00.095665+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:06.725415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.931093+0000","last_change":"2024-04-05T22:30:58.469153+0000","last_active":"2024-04-05T22:30:58.931093+0000","last_peered":"2024-04-05T22:30:58.931093+0000","last_clean":"2024-04-05T22:30:58.931093+0000","last_became_active":"2024-04-05T22:30:58.468900+0000","last_became_peered":"2024-04-05T22:30:58.468900+0000","last_unstale":"2024-04-05T22:30:58.931093+0000","last_undegraded":"2024-04-05T22:30:58.931093+0000","last_fullsized":"2024-04-05T22:30:58.931093+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:31:27.097967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-05T22:30:59.936053+0000","last_change":"2024-04-05T22:30:58.950958+0000","last_active":"2024-04-05T22:30:59.936053+0000","last_peered":"2024-04-05T22:30:59.936053+0000","last_clean":"2024-04-05T22:30:59.936053+0000","last_became_active":"2024-04-05T22:30:58.945186+0000","last_became_peered":"2024-04-05T22:30:58.945186+0000","last_unstale":"2024-04-05T22:30:59.936053+0000","last_undegraded":"2024-04-05T22:30:59.936053+0000","last_fullsized":"2024-04-05T22:30:59.936053+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:58.422723+0000","last_clean_scrub_stamp":"2024-04-05T22:30:58.422723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:54:36.949568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.940813+0000","last_change":"2024-04-05T22:30:58.478577+0000","last_active":"2024-04-05T22:30:58.940813+0000","last_peered":"2024-04-05T22:30:58.940813+0000","last_clean":"2024-04-05T22:30:58.940813+0000","last_became_active":"2024-04-05T22:30:58.478043+0000","last_became_peered":"2024-04-05T22:30:58.478043+0000","last_unstale":"2024-04-05T22:30:58.940813+0000","last_undegraded":"2024-04-05T22:30:58.940813+0000","last_fullsized":"2024-04-05T22:30:58.940813+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:30:57.404705+0000","last_clean_scrub_stamp":"2024-04-05T22:30:57.404705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:45:05.255712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":321,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-05T22:30:58.940132+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:30:58.940132+0000","last_peered":"2024-04-05T22:30:58.940132+0000","last_clean":"2024-04-05T22:30:58.940132+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:30:58.940132+0000","last_undegraded":"2024-04-05T22:30:58.940132+0000","last_fullsized":"2024-04-05T22:30:58.940132+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542266,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37764,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334076,"statfs":{"total":96636764160,"available":96598093824,"internally_reserved":0,"allocated":2613248,"data_stored":1478226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.048,"15min":1.0409999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4850000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0900000000000001,"15min":1.0409999999999999},"min":{"1min":0.67900000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5109999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.97399999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.167,"15min":1.0960000000000001},"min":{"1min":0.88,"5min":0.69399999999999995,"15min":0.58099999999999996},"max":{"1min":1.4750000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1060000000000001,"15min":1.077},"min":{"1min":0.78500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4299999999999999,"5min":1.819,"15min":1.819},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.177},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.597,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.2909999999999999,"15min":1.246},"min":{"1min":0.995,"5min":0.84899999999999998,"15min":0.75},"max":{"1min":1.6339999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1879999999999999,"15min":1.1559999999999999},"min":{"1min":0.79800000000000004,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.516,"5min":1.667,"15min":1.712},"last":1.052},{"interface":"front","average":{"1min":1.171,"5min":1.2290000000000001,"15min":1.224},"min":{"1min":0.879,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.6970000000000001,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.0129999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.226},"min":{"1min":0.91500000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.581,"5min":2.71,"15min":2.71},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.7290000000000001,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.161}]},{"osd":5,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.335,"15min":1.274},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5549999999999999,"5min":3.492,"15min":3.492},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.3899999999999999,"15min":1.363},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.72099999999999997},"max":{"1min":1.6870000000000001,"5min":2.992,"15min":2.992},"last":1.2849999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.5369999999999999,"15min":1.4870000000000001},"min":{"1min":0.99299999999999999,"5min":0.95799999999999996,"15min":0.83799999999999997},"max":{"1min":1.9379999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.508,"15min":1.502},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.2609999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":2011136,"data_stored":887754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.002,"15min":1.0449999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.577,"5min":1.6319999999999999,"15min":2.2509999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.034,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":1.544,"5min":1.6559999999999999,"15min":2.0110000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0529999999999999,"15min":1.133},"min":{"1min":0.73999999999999999,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":1.629,"5min":1.7430000000000001,"15min":1.7769999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.73199999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.4770000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0960000000000001,"15min":1.165},"min":{"1min":0.81799999999999995,"5min":0.623,"15min":0.57099999999999995},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.80500000000000005,"5min":0.63800000000000001,"15min":0.43099999999999999},"max":{"1min":1.51,"5min":1.51,"15min":2.2250000000000001},"last":1.089}]},{"osd":3,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.278},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.65500000000000003},"max":{"1min":1.657,"5min":1.96,"15min":1.96},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.248,"15min":1.3560000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.95,"5min":1.95,"15min":3.1720000000000002},"last":1.4059999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.327,"15min":1.325},"min":{"1min":0.83899999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8939999999999999,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.72},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.413,"15min":1.51},"min":{"1min":1.153,"5min":0.66200000000000003,"15min":0.64000000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.5660000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3009999999999999,"15min":1.3939999999999999},"min":{"1min":0.67300000000000004,"5min":0.53500000000000003,"15min":0.50600000000000001},"max":{"1min":1.972,"5min":1.972,"15min":2.3650000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.48,"15min":1.526},"min":{"1min":0.83499999999999996,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8759999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.7789999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:30:27 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.417,"15min":1.494},"min":{"1min":1.1040000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":0.85399999999999998,"5min":0.71599999999999997,"15min":0.66400000000000003},"max":{"1min":1.909,"5min":1.909,"15min":3.0920000000000001},"last":1.355}]}]},{"osd":5,"up_from":14,"seq":60129542266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":2011136,"data_stored":887754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.163,"15min":1.0660000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.625,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.179,"15min":1.1220000000000001},"min":{"1min":0.65800000000000003,"5min":0.65200000000000002,"15min":0.58199999999999996},"max":{"1min":1.6990000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.052}]},{"osd":1,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.23,"15min":1.1879999999999999},"min":{"1min":0.872,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.472,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.105},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.397,"15min":1.2410000000000001},"min":{"1min":0.83899999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6579999999999999,"5min":17.381,"15min":17.381},"last":1.383}]},{"osd":2,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.4219999999999999,"15min":1.2470000000000001},"min":{"1min":0.73199999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.694,"5min":24.52,"15min":24.52},"last":1.583},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.298,"15min":1.264},"min":{"1min":0.73499999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5049999999999999,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":0.95699999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.24,"15min":1.2030000000000001},"min":{"1min":1.032,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6759999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.429},{"interface":"front","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3500000000000001},"min":{"1min":1.008,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.748,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.302}]},{"osd":4,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.466,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.8160000000000001,"5min":14.933,"15min":14.933},"last":1.206},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.88400000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.7390000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.238}]},{"osd":6,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4490000000000001,"15min":1.3600000000000001},"min":{"1min":0.99399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.504},{"interface":"front","average":{"1min":1.546,"5min":1.585,"15min":1.518},"min":{"1min":1.3420000000000001,"5min":0.72999999999999998,"15min":0.55200000000000005},"max":{"1min":1.893,"5min":3.782,"15min":3.782},"last":1.7170000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.5669999999999999,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9430000000000001,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.579,"15min":1.593},"min":{"1min":1.1579999999999999,"5min":0.50900000000000001,"15min":0.47399999999999998},"max":{"1min":1.7609999999999999,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.6619999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542265,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":2027520,"data_stored":889900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0580000000000001,"15min":1.0269999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.51,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.03,"5min":1.085,"15min":1.085},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.536,"5min":1.859,"15min":1.859},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0960000000000001,"15min":1.121},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.51,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.196,"15min":1.1759999999999999},"min":{"1min":0.78900000000000003,"5min":0.68100000000000005,"15min":0.60299999999999998},"max":{"1min":1.502,"5min":2.577,"15min":2.577},"last":1.026}]},{"osd":2,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.127,"15min":1.1870000000000001},"min":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.476,"5min":1.6579999999999999,"15min":1.974},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.218,"15min":1.198},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67400000000000004},"max":{"1min":1.5429999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3,"15min":1.349},"min":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.593,"5min":1.835,"15min":2.1840000000000002},"last":1.252},{"interface":"front","average":{"1min":1.222,"5min":1.375,"15min":1.3899999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.627,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1919999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.2969999999999999},"min":{"1min":0.83399999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":2.665,"15min":2.665},"last":1.123},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.405,"15min":1.389},"min":{"1min":0.89400000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.643,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.3540000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4430000000000001,"15min":1.4419999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.53500000000000003},"max":{"1min":1.8799999999999999,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.381},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.444,"15min":1.4690000000000001},"min":{"1min":0.74099999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.647,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.226}]},{"osd":7,"last update":"Fri Apr 5 22:30:29 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5489999999999999,"15min":1.5349999999999999},"min":{"1min":1.0589999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6419999999999999,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.611},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.6220000000000001,"15min":1.5880000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.72399999999999998},"max":{"1min":2.0609999999999999,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.522}]}]},{"osd":3,"up_from":14,"seq":60129542266,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":2023424,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.83199999999999996},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2150000000000001,"5min":1.486,"15min":1.744},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.33900000000000002},"max":{"1min":1.157,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3700000000000001,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.91800000000000004}]},{"osd":2,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.014},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.40000000000000002},"max":{"1min":1.546,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.018,"5min":1.0549999999999999,"15min":1.075},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.524,"5min":1.956,"15min":2.2549999999999999},"last":1.2410000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1579999999999999,"15min":1.2090000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.734,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.258,"15min":1.2829999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.52000000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":1.1830000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.266,"15min":1.3280000000000001},"min":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.7050000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2729999999999999,"15min":1.286},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.6850000000000001,"5min":1.958,"15min":2.3239999999999998},"last":1.143}]},{"osd":6,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.262,"15min":1.3260000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.8400000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.228,"5min":1.3480000000000001,"15min":1.3779999999999999},"min":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.59099999999999997},"max":{"1min":1.6559999999999999,"5min":2.298,"15min":2.298},"last":1.3080000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3200000000000001,"15min":1.395},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.7789999999999999,"5min":1.899,"15min":2.468},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.3120000000000001,"15min":1.389},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.601,"5min":2.508,"15min":2.798},"last":1.206}]}]},{"osd":2,"up_from":14,"seq":60129542265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":2023424,"data_stored":888448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.82899999999999996,"15min":0.83099999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.7,"5min":2.879,"15min":2.879},"last":0.879},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.99199999999999999,"15min":0.93999999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.669,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.3169999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83099999999999996,"15min":0.83999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3300000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.89800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.8089999999999999,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.012}]},{"osd":3,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0369999999999999,"15min":1.044},"min":{"1min":0.54300000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.927,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.2480000000000002,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.286}]},{"osd":4,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.325,"15min":1.2629999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.8839999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.204},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2829999999999999,"15min":1.2529999999999999},"min":{"1min":0.69199999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8360000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2509999999999999,"15min":1.25},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.081,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.47},{"interface":"front","average":{"1min":1.226,"5min":1.252,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1419999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.3959999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2649999999999999,"15min":1.2669999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.855,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.26},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.296,"15min":1.2849999999999999},"min":{"1min":0.72499999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7969999999999999,"5min":2.242,"15min":2.7890000000000001},"last":1.4330000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:30:23 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3500000000000001,"15min":1.3540000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.1789999999999998,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.536},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.395},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.9750000000000001,"5min":2.597,"15min":2.597},"last":1.4990000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542267,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":2023424,"data_stored":889318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.38500000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.202,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88,"15min":0.86199999999999999},"min":{"1min":0.53900000000000003,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.5960000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4139999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.99399999999999999},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0289999999999999,"15min":1.087},"min":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.038},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98599999999999999,"15min":0.97299999999999998},"min":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.42299999999999999},"max":{"1min":1.5249999999999999,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.109}]},{"osd":4,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.135},"min":{"1min":0.67700000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4330000000000001,"5min":1.8500000000000001,"15min":1.855},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.272,"15min":1.2809999999999999},"min":{"1min":0.85399999999999998,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.1879999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.238,"15min":1.258},"min":{"1min":0.84499999999999997,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.4910000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.532},{"interface":"front","average":{"1min":1.141,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.70699999999999996,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.619,"5min":2.0680000000000001,"15min":2.173},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3480000000000001,"15min":1.343},"min":{"1min":0.83799999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8540000000000001,"5min":1.857,"15min":1.917},"last":1.26},{"interface":"front","average":{"1min":1.24,"5min":1.3220000000000001,"15min":1.3460000000000001},"min":{"1min":0.83399999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.8400000000000001,"5min":2.157,"15min":2.157},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:30:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8660000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.278},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5720000000000001,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.1559999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542267,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33664,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338176,"statfs":{"total":96636764160,"available":96602292224,"internally_reserved":0,"allocated":2609152,"data_stored":1478144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2769999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.329},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88400000000000001,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.4490000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.875,"15min":0.91500000000000004},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.177,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94299999999999995,"15min":0.96099999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.405,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.159}]},{"osd":3,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.89000000000000001},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.155,"5min":1.976,"15min":1.976},"last":1.218},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0149999999999999,"15min":1.077},"min":{"1min":0.52600000000000002,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5089999999999999,"5min":2.157,"15min":2.157},"last":1.5640000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.165,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.575,"5min":1.931,"15min":1.931},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.268,"15min":1.296},"min":{"1min":0.84499999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.534,"5min":1.9950000000000001,"15min":2.0790000000000002},"last":1.641}]},{"osd":5,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2290000000000001,"15min":1.258},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5089999999999999,"5min":2.105,"15min":2.3879999999999999},"last":1.286},{"interface":"front","average":{"1min":1.145,"5min":1.1950000000000001,"15min":1.2010000000000001},"min":{"1min":0.86899999999999999,"5min":0.80100000000000005,"15min":0.64200000000000002},"max":{"1min":1.5269999999999999,"5min":1.7010000000000001,"15min":1.9550000000000001},"last":1.514}]},{"osd":6,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3069999999999999,"15min":1.363},"min":{"1min":0.85199999999999998,"5min":0.72499999999999998,"15min":0.65500000000000003},"max":{"1min":1.589,"5min":2.0430000000000001,"15min":2.1589999999999998},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.292,"15min":1.355},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5760000000000001,"5min":2.0630000000000002,"15min":2.4430000000000001},"last":1.468}]},{"osd":7,"last update":"Fri Apr 5 22:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3,"15min":1.333},"min":{"1min":0.79600000000000004,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":1.637,"5min":2.1400000000000001,"15min":2.2839999999999998},"last":1.607},{"interface":"front","average":{"1min":1.22,"5min":1.24,"15min":1.3},"min":{"1min":0.94799999999999995,"5min":0.77000000000000002,"15min":0.68999999999999995},"max":{"1min":1.639,"5min":1.839,"15min":1.9810000000000001},"last":1.6579999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:31:03.862 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:31:03.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:31:04.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.114+0000 7ff61c000640 1 Processor -- start 2024-04-05T22:31:04.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.114+0000 7ff61c000640 1 -- start start 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61c000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff61406c630 0x7ff61406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61c000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6140b32b0 0x7ff6141647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61c000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6140b1b30 con 0x7ff6140cbe00 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61c000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140b1e10 con 0x7ff6140b32b0 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61c000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140ce7b0 con 0x7ff61406c630 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6140b32b0 0x7ff6141647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6140b32b0 0x7ff6141647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54494/0 (socket says 172.21.15.73:54494) 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.116+0000 7ff61a000640 1 -- 172.21.15.73:0/927038825 learned_addr learned my addr 172.21.15.73:0/927038825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.117+0000 7ff619600640 1 --2- 172.21.15.73:0/927038825 >> v2:172.21.15.157:3300/0 conn(0x7ff61406c630 0x7ff61406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.117+0000 7ff61a000640 1 -- 172.21.15.73:0/927038825 >> v2:172.21.15.157:3300/0 conn(0x7ff61406c630 msgr2=0x7ff61406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.117+0000 7ff61a000640 1 --2- 172.21.15.73:0/927038825 >> v2:172.21.15.157:3300/0 conn(0x7ff61406c630 0x7ff61406ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.117+0000 7ff61a000640 1 -- 172.21.15.73:0/927038825 >> v1:172.21.15.73:6789/0 conn(0x7ff6140cbe00 legacy=0x7ff6140ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.117+0000 7ff61a000640 1 -- 172.21.15.73:0/927038825 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61406cf50 con 0x7ff6140b32b0 2024-04-05T22:31:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.118+0000 7ff61a000640 1 --2- 172.21.15.73:0/927038825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6140b32b0 0x7ff6141647c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff61406f360 tx=0x7ff60c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9907318eea72e8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.118+0000 7ff613e00640 1 -- 172.21.15.73:0/927038825 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c02cb10 con 0x7ff6140b32b0 2024-04-05T22:31:04.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.118+0000 7ff613e00640 1 -- 172.21.15.73:0/927038825 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c02cc70 con 0x7ff6140b32b0 2024-04-05T22:31:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.118+0000 7ff613e00640 1 -- 172.21.15.73:0/927038825 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0343f0 con 0x7ff6140b32b0 2024-04-05T22:31:04.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 -- 172.21.15.73:0/927038825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6140b32b0 msgr2=0x7ff6141647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 --2- 172.21.15.73:0/927038825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6140b32b0 0x7ff6141647c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff61406f360 tx=0x7ff60c02bf70 comp rx=0 tx=0).stop 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 -- 172.21.15.73:0/927038825 shutdown_connections 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 --2- 172.21.15.73:0/927038825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6140b32b0 0x7ff6141647c0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 --2- 172.21.15.73:0/927038825 >> v2:172.21.15.157:3300/0 conn(0x7ff61406c630 0x7ff61406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 -- 172.21.15.73:0/927038825 >> 172.21.15.73:0/927038825 conn(0x7ff6140b0830 msgr2=0x7ff6140b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 -- 172.21.15.73:0/927038825 shutdown_connections 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.119+0000 7ff61c000640 1 -- 172.21.15.73:0/927038825 wait complete. 2024-04-05T22:31:04.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.121+0000 7ff61c000640 1 Processor -- start 2024-04-05T22:31:04.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.121+0000 7ff61c000640 1 -- start start 2024-04-05T22:31:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6140b32b0 0x7ff61415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6140cbe00 0x7ff614141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415db50 con 0x7ff61406c630 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415e520 con 0x7ff6140b32b0 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff61c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415cf40 con 0x7ff6140cbe00 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54506/0 (socket says 172.21.15.73:54506) 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 -- 172.21.15.73:0/1015664233 learned_addr learned my addr 172.21.15.73:0/1015664233 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 -- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6140cbe00 msgr2=0x7ff614141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6140cbe00 0x7ff614141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 -- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6140b32b0 msgr2=0x7ff61415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6140b32b0 0x7ff61415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.122+0000 7ff619600640 1 -- 172.21.15.73:0/1015664233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c009b30 con 0x7ff61406c630 2024-04-05T22:31:04.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff619600640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d5f0 tx=0x7ff60800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080125f0 con 0x7ff61406c630 2024-04-05T22:31:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608012750 con 0x7ff61406c630 2024-04-05T22:31:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608012b00 con 0x7ff61406c630 2024-04-05T22:31:04.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61415d220 con 0x7ff61406c630 2024-04-05T22:31:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.123+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61415d7e0 con 0x7ff61406c630 2024-04-05T22:31:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.124+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61415d9a0 con 0x7ff61406c630 2024-04-05T22:31:04.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.125+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7ff608012c60 con 0x7ff61406c630 2024-04-05T22:31:04.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.126+0000 7ff611600640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 0x7ff5f406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.126+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff608087eb0 con 0x7ff61406c630 2024-04-05T22:31:04.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.127+0000 7ff618c00640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 0x7ff5f406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.128+0000 7ff618c00640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 0x7ff5f406d8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff600000f60 tx=0x7ff6000079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.130+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff608054c90 con 0x7ff61406c630 2024-04-05T22:31:04.396 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-04-05T22:31:04.396+0000 7f506fa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:31:04.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.397+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff6140cb380 con 0x7ff61406c630 2024-04-05T22:31:04.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.403+0000 7ff611600640 1 -- 172.21.15.73:0/1015664233 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff6080582b0 con 0x7ff61406c630 2024-04-05T22:31:04.403 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.406+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 msgr2=0x7ff5f406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.406+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 0x7ff5f406d8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff600000f60 tx=0x7ff6000079b0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.406+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 msgr2=0x7ff61415dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.406+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d5f0 tx=0x7ff60800dab0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 shutdown_connections 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff5f406b410 0x7ff5f406d8d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6140cbe00 0x7ff614141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6140b32b0 0x7ff61415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 --2- 172.21.15.73:0/1015664233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61406c630 0x7ff61415dfe0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 >> 172.21.15.73:0/1015664233 conn(0x7ff6140b0830 msgr2=0x7ff61415b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:04.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 shutdown_connections 2024-04-05T22:31:04.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.407+0000 7ff61c000640 1 -- 172.21.15.73:0/1015664233 wait complete. 2024-04-05T22:31:04.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.640+0000 7f9899200640 1 Processor -- start 2024-04-05T22:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.640+0000 7f9899200640 1 -- start start 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 0x7f989415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98940d1670 0x7f98941602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98941607e0 con 0x7f98940d1670 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98940cd670 con 0x7f98940d0b70 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9899200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940cf460 con 0x7f98940d1290 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9893e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98940d1670 0x7f98941602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9893e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98940d1670 0x7f98941602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54516/0 (socket says 172.21.15.73:54516) 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.642+0000 7f9893e00640 1 -- 172.21.15.73:0/2700389746 learned_addr learned my addr 172.21.15.73:0/2700389746 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f988aa00640 1 --2- 172.21.15.73:0/2700389746 >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 0x7f989415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f9893e00640 1 -- 172.21.15.73:0/2700389746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 msgr2=0x7f98940d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f9893e00640 1 --2- 172.21.15.73:0/2700389746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f9893e00640 1 -- 172.21.15.73:0/2700389746 >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 msgr2=0x7f989415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f9893e00640 1 --2- 172.21.15.73:0/2700389746 >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 0x7f989415fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f9893e00640 1 -- 172.21.15.73:0/2700389746 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940cf740 con 0x7f98940d1670 2024-04-05T22:31:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.643+0000 7f988aa00640 1 --2- 172.21.15.73:0/2700389746 >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 0x7f989415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.644+0000 7f9893e00640 1 --2- 172.21.15.73:0/2700389746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98941602a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f989408c440 tx=0x7f988c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae59ad0e49b767f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.644+0000 7f9892a00640 1 -- 172.21.15.73:0/2700389746 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c02c9e0 con 0x7f98940d1670 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.644+0000 7f9892a00640 1 -- 172.21.15.73:0/2700389746 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c02cb40 con 0x7f98940d1670 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.644+0000 7f9892a00640 1 -- 172.21.15.73:0/2700389746 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0343f0 con 0x7f98940d1670 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 -- 172.21.15.73:0/2700389746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 msgr2=0x7f98941602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 --2- 172.21.15.73:0/2700389746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98941602a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f989408c440 tx=0x7f988c02be40 comp rx=0 tx=0).stop 2024-04-05T22:31:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 -- 172.21.15.73:0/2700389746 shutdown_connections 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 --2- 172.21.15.73:0/2700389746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98941602a0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 --2- 172.21.15.73:0/2700389746 >> v2:172.21.15.157:3300/0 conn(0x7f98940d0b70 0x7f989415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 --2- 172.21.15.73:0/2700389746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 -- 172.21.15.73:0/2700389746 >> 172.21.15.73:0/2700389746 conn(0x7f989409bdd0 msgr2=0x7f98940d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 -- 172.21.15.73:0/2700389746 shutdown_connections 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.645+0000 7f9899200640 1 -- 172.21.15.73:0/2700389746 wait complete. 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.646+0000 7f9899200640 1 Processor -- start 2024-04-05T22:31:04.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 -- start start 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d0b70 0x7f98940b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9894152ae0 con 0x7f98940d1670 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940b23b0 con 0x7f98940d0b70 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.647+0000 7f9899200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940b17a0 con 0x7f98940d1290 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54520/0 (socket says 172.21.15.73:54520) 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 -- 172.21.15.73:0/1598617122 learned_addr learned my addr 172.21.15.73:0/1598617122 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f988aa00640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 -- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 msgr2=0x7f98940c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 -- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d0b70 msgr2=0x7f98940b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d0b70 0x7f98940b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 -- 172.21.15.73:0/1598617122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009b30 con 0x7f98940d1670 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.648+0000 7f9893e00640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f988c009f40 tx=0x7f988c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.649+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c040670 con 0x7f98940d1670 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.649+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c0407d0 con 0x7f98940d1670 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.649+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c040b80 con 0x7f98940d1670 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.649+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98940b3c30 con 0x7f98940d1670 2024-04-05T22:31:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.649+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940b1d60 con 0x7f98940d1670 2024-04-05T22:31:04.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.650+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989406c120 con 0x7f98940d1670 2024-04-05T22:31:04.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.651+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f988c040ce0 con 0x7f98940d1670 2024-04-05T22:31:04.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.651+0000 7f988be00640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 0x7f987006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:04.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.652+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f988c0aae90 con 0x7f98940d1670 2024-04-05T22:31:04.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.652+0000 7f9893400640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 0x7f987006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:04.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.653+0000 7f9893400640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 0x7f987006d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9884009710 tx=0x7f9884006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:04.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.657+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f988c077d90 con 0x7f98940d1670 2024-04-05T22:31:04.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.874+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f989404e160 con 0x7f98940d1670 2024-04-05T22:31:04.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.875+0000 7f988be00640 1 -- 172.21.15.73:0/1598617122 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4746 (secure 0 0 0) 0x7f988c009510 con 0x7f98940d1670 2024-04-05T22:31:04.874 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:04.875 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:04.875 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 100 2024-04-05T22:31:04.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 msgr2=0x7f987006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 0x7f987006d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9884009710 tx=0x7f9884006c90 comp rx=0 tx=0).stop 2024-04-05T22:31:04.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 msgr2=0x7f98940af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:04.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f988c009f40 tx=0x7f988c004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:04.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 shutdown_connections 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f987006b3e0 0x7f987006d8a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98940d1670 0x7f98940af6e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98940d1290 0x7f98940c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 --2- 172.21.15.73:0/1598617122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d0b70 0x7f98940b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 >> 172.21.15.73:0/1598617122 conn(0x7f989409bdd0 msgr2=0x7f98940b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 shutdown_connections 2024-04-05T22:31:04.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:04.878+0000 7f9899200640 1 -- 172.21.15.73:0/1598617122 wait complete. 2024-04-05T22:31:05.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:30:58.914891+0000', 'modified': '2024-04-05T22:31:01.932703+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9756}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9756': {'gid': 9756, 'name': 'd', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/3806798161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3806798161}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3806798161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:31:05.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:31:05.297 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-05T22:31:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.523+0000 7f4c06e00640 1 Processor -- start 2024-04-05T22:31:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.523+0000 7f4c06e00640 1 -- start start 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.524+0000 7f4c06e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c000c80a0 0x7f4c000c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c06e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c0006cb70 0x7f4c0006d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c06e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0006c6e0 con 0x7f4c0006fa70 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c06e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c000c89c0 con 0x7f4c0006cb70 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c06e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0006bad0 con 0x7f4c000c80a0 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c0006cb70 0x7f4c0006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c0006cb70 0x7f4c0006d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54522/0 (socket says 172.21.15.73:54522) 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 -- 172.21.15.73:0/1900570843 learned_addr learned my addr 172.21.15.73:0/1900570843 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4bff400640 1 --1- 172.21.15.73:0/1900570843 >> v1:172.21.15.73:6789/0 conn(0x7f4c0006fa70 0x7f4c000cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56684/0 (socket says 172.21.15.73:56684) 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4bffe00640 1 --2- 172.21.15.73:0/1900570843 >> v2:172.21.15.157:3300/0 conn(0x7f4c000c80a0 0x7f4c000c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 -- 172.21.15.73:0/1900570843 >> v2:172.21.15.157:3300/0 conn(0x7f4c000c80a0 msgr2=0x7f4c000c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 --2- 172.21.15.73:0/1900570843 >> v2:172.21.15.157:3300/0 conn(0x7f4c000c80a0 0x7f4c000c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 -- 172.21.15.73:0/1900570843 >> v1:172.21.15.73:6789/0 conn(0x7f4c0006fa70 legacy=0x7f4c000cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4c04e00640 1 -- 172.21.15.73:0/1900570843 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0006bdb0 con 0x7f4c0006cb70 2024-04-05T22:31:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.525+0000 7f4bfea00640 1 -- 172.21.15.73:0/1900570843 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13741125 0 0) 0x7f4c0006c6e0 con 0x7f4c0006fa70 2024-04-05T22:31:05.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4c04e00640 1 --2- 172.21.15.73:0/1900570843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c0006d000 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4c0004e2e0 tx=0x7f4bf802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e13cc5f5363183 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4bfea00640 1 -- 172.21.15.73:0/1900570843 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf802cb20 con 0x7f4c0006cb70 2024-04-05T22:31:05.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4bfea00640 1 -- 172.21.15.73:0/1900570843 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf802cc80 con 0x7f4c0006cb70 2024-04-05T22:31:05.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4bfea00640 1 -- 172.21.15.73:0/1900570843 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80343f0 con 0x7f4c0006cb70 2024-04-05T22:31:05.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4c06e00640 1 -- 172.21.15.73:0/1900570843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 msgr2=0x7f4c0006d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.527+0000 7f4c06e00640 1 --2- 172.21.15.73:0/1900570843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c0006d000 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4c0004e2e0 tx=0x7f4bf802bf80 comp rx=0 tx=0).stop 2024-04-05T22:31:05.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 -- 172.21.15.73:0/1900570843 shutdown_connections 2024-04-05T22:31:05.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 --2- 172.21.15.73:0/1900570843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c0006d000 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 --2- 172.21.15.73:0/1900570843 >> v2:172.21.15.157:3300/0 conn(0x7f4c000c80a0 0x7f4c000c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 -- 172.21.15.73:0/1900570843 >> 172.21.15.73:0/1900570843 conn(0x7f4c000cb7d0 msgr2=0x7f4c000cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 -- 172.21.15.73:0/1900570843 shutdown_connections 2024-04-05T22:31:05.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.528+0000 7f4c06e00640 1 -- 172.21.15.73:0/1900570843 wait complete. 2024-04-05T22:31:05.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.529+0000 7f4c06e00640 1 Processor -- start 2024-04-05T22:31:05.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.530+0000 7f4c06e00640 1 -- start start 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.530+0000 7f4c06e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.530+0000 7f4c06e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0006fa70 0x7f4c0014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4c06e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c000c80a0 0x7f4c0015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4c06e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0009a1d0 con 0x7f4c0006cb70 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4c06e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0015eda0 con 0x7f4c0006fa70 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4c06e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0015e470 con 0x7f4c000c80a0 2024-04-05T22:31:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54524/0 (socket says 172.21.15.73:54524) 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 -- 172.21.15.73:0/3438178021 learned_addr learned my addr 172.21.15.73:0/3438178021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 -- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c000c80a0 msgr2=0x7f4c0015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c000c80a0 0x7f4c0015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 -- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0006fa70 msgr2=0x7f4c0014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0006fa70 0x7f4c0014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.531+0000 7f4bffe00640 1 -- 172.21.15.73:0/3438178021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009b30 con 0x7f4c0006cb70 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d5f0 tx=0x7f4bf000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4bf7e00640 1 -- 172.21.15.73:0/3438178021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00125f0 con 0x7f4c0006cb70 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4bf7e00640 1 -- 172.21.15.73:0/3438178021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0012750 con 0x7f4c0006cb70 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0015e750 con 0x7f4c0006cb70 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4bf7e00640 1 -- 172.21.15.73:0/3438178021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0012b00 con 0x7f4c0006cb70 2024-04-05T22:31:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.532+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0015a010 con 0x7f4c0006cb70 2024-04-05T22:31:05.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.534+0000 7f4bf7e00640 1 -- 172.21.15.73:0/3438178021 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f4bf0012c60 con 0x7f4c0006cb70 2024-04-05T22:31:05.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.535+0000 7f4bf7e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 0x7f4bdc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.535+0000 7f4bf7e00640 1 -- 172.21.15.73:0/3438178021 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bf0087ec0 con 0x7f4c0006cb70 2024-04-05T22:31:05.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.536+0000 7f4bff400640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 0x7f4bdc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.536+0000 7f4c09c03740 1 Processor -- start 2024-04-05T22:31:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.536+0000 7f4c09c03740 1 -- start start 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4bff400640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 0x7f4bdc06d8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4be8007fd0 tx=0x7f4be8007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37eb26d0 0x55bf37ee24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37e228c0 con 0x55bf37edf1f0 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37e20050 con 0x55bf37edf5d0 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37e1b940 con 0x55bf37eb26d0 2024-04-05T22:31:05.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54538/0 (socket says 172.21.15.73:54538) 2024-04-05T22:31:05.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4bffe00640 1 -- 172.21.15.73:0/1805159354 learned_addr learned my addr 172.21.15.73:0/1805159354 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.537+0000 7f4bff400640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4c04e00640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37eb26d0 0x55bf37ee24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 -- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37eb26d0 msgr2=0x55bf37ee24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37eb26d0 0x55bf37ee24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 -- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 msgr2=0x55bf37ee1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 -- 172.21.15.73:0/1805159354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf37de69e0 con 0x55bf37edf1f0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bff400640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bffe00640 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0058660 tx=0x7f4bf00585d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dbac91eecf4463 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bf4c00640 1 -- 172.21.15.73:0/1805159354 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf005c790 con 0x55bf37edf1f0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bf4c00640 1 -- 172.21.15.73:0/1805159354 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0060d50 con 0x55bf37edf1f0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.538+0000 7f4bf4c00640 1 -- 172.21.15.73:0/1805159354 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0055070 con 0x55bf37edf1f0 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 -- 172.21.15.73:0/1805159354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 msgr2=0x55bf37eb2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0058660 tx=0x7f4bf00585d0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 -- 172.21.15.73:0/1805159354 shutdown_connections 2024-04-05T22:31:05.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37eb26d0 0x55bf37ee24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 --2- 172.21.15.73:0/1805159354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf1f0 0x55bf37eb2190 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 -- 172.21.15.73:0/1805159354 >> 172.21.15.73:0/1805159354 conn(0x55bf37eb4400 msgr2=0x55bf37efcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 -- 172.21.15.73:0/1805159354 shutdown_connections 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.539+0000 7f4c09c03740 1 -- 172.21.15.73:0/1805159354 wait complete. 2024-04-05T22:31:05.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.540+0000 7f4c09c03740 1 -- start start 2024-04-05T22:31:05.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.540+0000 7f4c09c03740 1 Processor -- start 2024-04-05T22:31:05.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 0x55bf37ef3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edda80 0x55bf37eddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37dbd070 con 0x55bf37edf5d0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37dbcd10 con 0x55bf37edf1f0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37b82f20 con 0x55bf37edda80 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4c09c03740 10 client.0 _ll_register_callbacks cb 0x55bf37e14fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4bff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4bff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54552/0 (socket says 172.21.15.73:54552) 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.541+0000 7f4bff400640 1 -- 172.21.15.73:0/2154593522 learned_addr learned my addr 172.21.15.73:0/2154593522 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bffe00640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 0x55bf37ef3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4c04e00640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edda80 0x55bf37eddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 -- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edda80 msgr2=0x55bf37eddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edda80 0x55bf37eddf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 -- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 msgr2=0x55bf37ef3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 0x55bf37ef3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 -- 172.21.15.73:0/2154593522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0015a010 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bffe00640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 0x55bf37ef3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bff400640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005b20 tx=0x7f4be8001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f192172e961b48de server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bed600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8014e80 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4be8022ad0 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.542+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf37e45ae0 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4be8022c90 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 10 client.9891 fetch_fsmap learned FSMap version 100 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55bf37e19fc0 con 0x55bf37edf5d0 2024-04-05T22:31:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf37e45ae0 con 0x55bf37edf5d0 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4be8012440 con 0x55bf37edf5d0 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 10 client.9891 fetch_fsmap finished waiting for FSMap version 100 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 10 client.9891 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 20 client.9891 populate_metadata read hostname 'smithi073' 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37ee3fc0 0x55bf37ee43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 4 client.9891 mds_command: new command op to 9756 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.543+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 --> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf37b0f3c0 con 0x55bf37ee3fc0 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.544+0000 7f4bffe00640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37ee3fc0 0x55bf37ee43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.545+0000 7f4bffe00640 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37ee3fc0 0x55bf37ee43a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5affeb83443dc8 server_cookie=791500e0bf89776 in_seq=0 out_seq=0 2024-04-05T22:31:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.545+0000 7f4bed600640 10 client.9891 ms_handle_connect on v2:172.21.15.157:6834/3806798161 2024-04-05T22:31:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.546+0000 7f4bed600640 1 -- 172.21.15.73:0/2154593522 <== mds.0 v2:172.21.15.157:6834/3806798161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bf37b0f3c0 con 0x55bf37ee3fc0 2024-04-05T22:31:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4bed600640 10 client.9891 handle_command_reply: tid=0 2024-04-05T22:31:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 client.9891 shutdown 2024-04-05T22:31:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 msgr2=0x55bf37ef6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005b20 tx=0x7f4be8001080 comp rx=0 tx=0).stop 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 shutdown_connections 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37ee3fc0 0x55bf37ee43a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edda80 0x55bf37eddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edf5d0 0x55bf37ef6940 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4bed600640 0 client.9891 ms_handle_reset on v2:172.21.15.157:6834/3806798161 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 --2- 172.21.15.73:0/2154593522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf1f0 0x55bf37ef3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 >> 172.21.15.73:0/2154593522 conn(0x55bf37eb26d0 msgr2=0x55bf37ee1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 shutdown_connections 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 1 -- 172.21.15.73:0/2154593522 wait complete. 2024-04-05T22:31:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.547+0000 7f4c09c03740 20 client.9891 trim_cache size 0 max 16384 2024-04-05T22:31:05.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.564+0000 7f4c09c03740 1 Processor -- start 2024-04-05T22:31:05.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.564+0000 7f4c09c03740 1 -- start start 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.564+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.564+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37f0bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37f16840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4c09c03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37eb7f40 con 0x55bf37edda80 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37f0c4a0 con 0x55bf37edf5d0 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37ee29b0 con 0x55bf37edf1f0 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54554/0 (socket says 172.21.15.73:54554) 2024-04-05T22:31:05.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 -- 172.21.15.73:0/7203785 learned_addr learned my addr 172.21.15.73:0/7203785 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 -- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 msgr2=0x55bf37f0bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37f0bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 -- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 msgr2=0x55bf37f16840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37f16840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 -- 172.21.15.73:0/7203785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf37e4ef80 con 0x55bf37edda80 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.565+0000 7f4bffe00640 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0056cb0 tx=0x7f4bf006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc97928a93f1e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4bf5600640 1 -- 172.21.15.73:0/7203785 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0077040 con 0x55bf37edda80 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4bf5600640 1 -- 172.21.15.73:0/7203785 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0070210 con 0x55bf37edda80 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4bf5600640 1 -- 172.21.15.73:0/7203785 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0070380 con 0x55bf37edda80 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 -- 172.21.15.73:0/7203785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 msgr2=0x55bf37ee5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0056cb0 tx=0x7f4bf006d000 comp rx=0 tx=0).stop 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 -- 172.21.15.73:0/7203785 shutdown_connections 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37f16840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37f0bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 --2- 172.21.15.73:0/7203785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37ee5f50 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 -- 172.21.15.73:0/7203785 >> 172.21.15.73:0/7203785 conn(0x55bf37eb26d0 msgr2=0x55bf37eb91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 -- 172.21.15.73:0/7203785 shutdown_connections 2024-04-05T22:31:05.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.566+0000 7f4c09c03740 1 -- 172.21.15.73:0/7203785 wait complete. 2024-04-05T22:31:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.567+0000 7f4c09c03740 1 -- start start 2024-04-05T22:31:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.567+0000 7f4c09c03740 1 Processor -- start 2024-04-05T22:31:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37ee43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37ee32e0 con 0x55bf37edda80 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37edd2b0 con 0x55bf37edf5d0 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37ee5100 con 0x55bf37edf1f0 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4c09c03740 10 client.0 _ll_register_callbacks cb 0x7f4be8001310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4bffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54562/0 (socket says 172.21.15.73:54562) 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.568+0000 7f4bffe00640 1 -- 172.21.15.73:0/3811221058 learned_addr learned my addr 172.21.15.73:0/3811221058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 -- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 msgr2=0x55bf37ee43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37ee43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 -- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 msgr2=0x55bf37ee48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 -- 172.21.15.73:0/3811221058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf37de69e0 con 0x55bf37edda80 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bffe00640 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00588f0 tx=0x7f4bf0020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4013afd998ae69b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bed600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0077040 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bf0078d60 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bf0060d50 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf00771a0 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4c09c03740 10 client.9897 fetch_fsmap learned FSMap version 100 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.569+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55bf37dbc280 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0060d50 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4bf0057e50 con 0x55bf37edda80 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 10 client.9897 fetch_fsmap finished waiting for FSMap version 100 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 10 client.9897 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 20 client.9897 populate_metadata read hostname 'smithi073' 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37f650c0 0x55bf37f654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 4 client.9897 mds_command: new command op to 9756 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:31:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 --> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf37b206c0 con 0x55bf37f650c0 2024-04-05T22:31:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.570+0000 7f4bff400640 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37f650c0 0x55bf37f654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.571+0000 7f4bff400640 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37f650c0 0x55bf37f654a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3920d18dceb8faf2 server_cookie=bc0bf2487871f089 in_seq=0 out_seq=0 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.571+0000 7f4bed600640 10 client.9897 ms_handle_connect on v2:172.21.15.157:6834/3806798161 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4bed600640 1 -- 172.21.15.73:0/3811221058 <== mds.0 v2:172.21.15.157:6834/3806798161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55bf37b206c0 con 0x55bf37f650c0 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4bed600640 10 client.9897 handle_command_reply: tid=0 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 client.9897 shutdown 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 msgr2=0x55bf37f62310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00588f0 tx=0x7f4bf0020040 comp rx=0 tx=0).stop 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 shutdown_connections 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:6834/3806798161,v1:172.21.15.157:6835/3806798161] conn(0x55bf37f650c0 0x55bf37f654a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf37edf5d0 0x55bf37ee48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bf37edf1f0 0x55bf37ee43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 --2- 172.21.15.73:0/3811221058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf37edda80 0x55bf37f62310 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 >> 172.21.15.73:0/3811221058 conn(0x55bf37eb26d0 msgr2=0x55bf37ead8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4bed600640 0 client.9897 ms_handle_reset on v2:172.21.15.157:6834/3806798161 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.572+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 shutdown_connections 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c09c03740 1 -- 172.21.15.73:0/3811221058 wait complete. 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c09c03740 20 client.9897 trim_cache size 0 max 16384 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:31:05.574 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 9756, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.6519797220000001, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 100, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 160, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 160, 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 11.522537288000001 2024-04-05T22:31:05.575 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 msgr2=0x7f4bdc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 0x7f4bdc06d8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4be8007fd0 tx=0x7f4be8007920 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 msgr2=0x7f4c00099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d5f0 tx=0x7f4bf000dab0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 shutdown_connections 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4bdc06b410 0x7f4bdc06d8d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c000c80a0 0x7f4c0015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0006fa70 0x7f4c0014a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 --2- 172.21.15.73:0/3438178021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c0006cb70 0x7f4c00099bf0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 >> 172.21.15.73:0/3438178021 conn(0x7f4c000cb7d0 msgr2=0x7f4c000707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.573+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 shutdown_connections 2024-04-05T22:31:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.574+0000 7f4c06e00640 1 -- 172.21.15.73:0/3438178021 wait complete. 2024-04-05T22:31:05.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:31:05.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.784+0000 7f6605600640 1 Processor -- start 2024-04-05T22:31:05.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.785+0000 7f6605600640 1 -- start start 2024-04-05T22:31:05.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.786+0000 7f6605600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66001587d0 0x7f66000cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.786+0000 7f6605600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660006f5b0 con 0x7f660015e850 2024-04-05T22:31:05.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.786+0000 7f6605600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6600070410 con 0x7f66000ccd20 2024-04-05T22:31:05.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.786+0000 7f6605600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6600158bb0 con 0x7f66001587d0 2024-04-05T22:31:05.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65ff400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f660015e850 0x7f66000708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56686/0 (socket says 172.21.15.73:56686) 2024-04-05T22:31:05.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65ff400640 1 -- 172.21.15.73:0/2699147327 learned_addr learned my addr 172.21.15.73:0/2699147327 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:05.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fea00640 1 --2- 172.21.15.73:0/2699147327 >> v2:172.21.15.157:3300/0 conn(0x7f66001587d0 0x7f66000cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485277789 0 0) 0x7f660006f5b0 con 0x7f660015e850 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003520 con 0x7f660015e850 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2936769632 0 0) 0x7f65e4003520 con 0x7f660015e850 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 >> v2:172.21.15.157:3300/0 conn(0x7f66001587d0 msgr2=0x7f66000cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 --2- 172.21.15.73:0/2699147327 >> v2:172.21.15.157:3300/0 conn(0x7f66001587d0 0x7f66000cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.787+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 >> v1:172.21.15.157:6789/0 conn(0x7f66000ccd20 legacy=0x7f660006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660006f800 con 0x7f660015e850 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f65f80019f0 con 0x7f660015e850 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 >> v1:172.21.15.73:6789/0 conn(0x7f660015e850 legacy=0x7f66000708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600158bb0 con 0x7f65e40047c0 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c020 con 0x7f65e4005180 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c1e0 con 0x7f65e4003970 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65fea00640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65ff400640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 -- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003970 msgr2=0x7f65e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 -- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4005180 msgr2=0x7f65e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:05.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65f7e00640 1 -- 172.21.15.73:0/2699147327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003520 con 0x7f65e40047c0 2024-04-05T22:31:05.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65ff400640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:05.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.788+0000 7f65ff400640 1 -- 172.21.15.73:0/2699147327 reap_dead start 2024-04-05T22:31:05.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f65f7e00640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001750 tx=0x7f65e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92337e469cbd6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e80052d0 con 0x7f65e40047c0 2024-04-05T22:31:05.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802e050 con 0x7f65e40047c0 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f65fe000640 1 -- 172.21.15.73:0/2699147327 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8003830 con 0x7f65e40047c0 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f6605600640 1 -- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 msgr2=0x7f65e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f6605600640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001750 tx=0x7f65e40057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f6605600640 1 -- 172.21.15.73:0/2699147327 shutdown_connections 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f6605600640 1 --2- 172.21.15.73:0/2699147327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.790+0000 7f6605600640 1 -- 172.21.15.73:0/2699147327 >> 172.21.15.73:0/2699147327 conn(0x7f66000ce8e0 msgr2=0x7f66000cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.791+0000 7f6605600640 1 -- 172.21.15.73:0/2699147327 shutdown_connections 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.791+0000 7f6605600640 1 -- 172.21.15.73:0/2699147327 wait complete. 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.792+0000 7f6605600640 1 Processor -- start 2024-04-05T22:31:05.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.793+0000 7f6605600640 1 -- start start 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.793+0000 7f6605600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f6605600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000aa330 0x7f66000ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f6605600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ad8a0 0x7f66000ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f6605600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600137740 con 0x7f65e40047c0 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f6605600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000a9290 con 0x7f66000ad8a0 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f6605600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000a9570 con 0x7f66000aa330 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54580/0 (socket says 172.21.15.73:54580) 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 -- 172.21.15.73:0/195900329 learned_addr learned my addr 172.21.15.73:0/195900329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:05.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 -- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000aa330 msgr2=0x7f66000ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65f7e00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000aa330 0x7f66000ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65ff400640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ad8a0 0x7f66000ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000aa330 0x7f66000ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 -- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ad8a0 msgr2=0x7f66000ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ad8a0 0x7f66000ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.794+0000 7f65fea00640 1 -- 172.21.15.73:0/195900329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000a9850 con 0x7f65e40047c0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f65fea00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f65f0010790 tx=0x7f65f0010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0009630 con 0x7f65e40047c0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f000eb60 con 0x7f65e40047c0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000a9b30 con 0x7f65e40047c0 2024-04-05T22:31:05.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0010ea0 con 0x7f65e40047c0 2024-04-05T22:31:05.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.795+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000aa7b0 con 0x7f65e40047c0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.796+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6600066af0 con 0x7f65e40047c0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.797+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f65f001c400 con 0x7f65e40047c0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.798+0000 7f65f6a00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 0x7f65dc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.799+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f0087f60 con 0x7f65e40047c0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.799+0000 7f65f7e00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 0x7f65dc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.800+0000 7f65f7e00640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 0x7f65dc06d8d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f65e8033000 tx=0x7f65e8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:05.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:05.803+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f65f0054d40 con 0x7f65e40047c0 2024-04-05T22:31:06.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.038+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6600068160 con 0x7f65e40047c0 2024-04-05T22:31:06.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.038+0000 7f65f6a00640 1 -- 172.21.15.73:0/195900329 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7f65f0058360 con 0x7f65e40047c0 2024-04-05T22:31:06.038 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 msgr2=0x7f65dc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 0x7f65dc06d8d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f65e8033000 tx=0x7f65e8009030 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 msgr2=0x7f6600137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f65f0010790 tx=0x7f65f0010c50 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 shutdown_connections 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65dc06b410 0x7f65dc06d8d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ad8a0 0x7f66000ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000aa330 0x7f66000ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 --2- 172.21.15.73:0/195900329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e40047c0 0x7f6600137200 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.041+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 >> 172.21.15.73:0/195900329 conn(0x7f66000ce8e0 msgr2=0x7f66000ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:06.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.042+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 shutdown_connections 2024-04-05T22:31:06.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:06.042+0000 7f6605600640 1 -- 172.21.15.73:0/195900329 wait complete. 2024-04-05T22:31:06.059 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:31:06.059 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:31:06.059 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:31:06.059 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:31:06.059 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:06.060 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:31:06.060 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:31:06.060 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:31:06.060 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:06.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.060 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:31:06.107 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:31:06.107 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:31:06.107 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:31:06.107 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.107 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-05T22:31:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:31:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:06.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.111 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:31:06.187 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:31:06.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.192 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:31:06.244 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:31:06.244 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:31:06.244 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:31:06.245 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:31:06.245 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:31:06.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.245 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:31:06.245 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:31:06.245 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:31:06.245 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:31:06.245 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:31:06.246 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:31:06.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.424 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:31:06.477 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:31:06.478 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:31:06.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:06.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:31:06.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:06.571 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:06.623 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:06.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:06.623 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:06.675 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:31:06.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:06.808 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:06.808 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:31:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:31:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:31:06.846 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:31:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:31:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:31:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:31:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:31:06.852 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:31:06.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:06.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:06.931 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:31:07.001 INFO:teuthology.orchestra.run.smithi073.stdout:client6703 2024-04-05T22:31:07.001 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:07.001 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:07.053 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:31:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.260+0000 7fae62000640 1 Processor -- start 2024-04-05T22:31:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.260+0000 7fae62000640 1 -- start start 2024-04-05T22:31:07.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae62000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 0x7fae5c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae62000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0ce5d0 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae62000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0ce010 con 0x7fae5c160620 2024-04-05T22:31:07.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae62000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0ce2f0 con 0x7fae5c0b1870 2024-04-05T22:31:07.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae5be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fae5c0ad6b0 0x7fae5c0cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58554/0 (socket says 172.21.15.73:58554) 2024-04-05T22:31:07.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.262+0000 7fae5be00640 1 -- 172.21.15.73:0/4160336596 learned_addr learned my addr 172.21.15.73:0/4160336596 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 0x7fae5c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787675565 0 0) 0x7fae5c0ce5d0 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae40003520 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261297780 0 0) 0x7fae40003520 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 msgr2=0x7fae5c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 0x7fae5c0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 >> v1:172.21.15.157:6789/0 conn(0x7fae5c160620 legacy=0x7fae5c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0ac970 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fae54001670 con 0x7fae5c0ad6b0 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 >> v1:172.21.15.73:6789/0 conn(0x7fae5c0ad6b0 legacy=0x7fae5c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400039a0 0x7fae40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae400047c0 0x7fae40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 0x7fae40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0ce2f0 con 0x7fae400047c0 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4000ba30 con 0x7fae400039a0 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.263+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4000bbc0 con 0x7fae5c0b1870 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae400047c0 0x7fae40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 -- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 msgr2=0x7fae40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c0b1870 0x7fae40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 -- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400039a0 msgr2=0x7fae40003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5b400640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400039a0 0x7fae40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400039a0 0x7fae40003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5aa00640 1 -- 172.21.15.73:0/4160336596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40003520 con 0x7fae400047c0 2024-04-05T22:31:07.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.264+0000 7fae5b400640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400039a0 0x7fae40003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:07.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.265+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae400047c0 0x7fae40004c40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009fc0 tx=0x7fae400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1de6f6c1d532c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:07.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.265+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c02cbf0 con 0x7fae400047c0 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c00f040 con 0x7fae400047c0 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae5a000640 1 -- 172.21.15.73:0/4160336596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c034440 con 0x7fae400047c0 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 -- 172.21.15.73:0/4160336596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae400047c0 msgr2=0x7fae40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 --2- 172.21.15.73:0/4160336596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae400047c0 0x7fae40004c40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009fc0 tx=0x7fae400057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae5be00640 1 -- 172.21.15.73:0/4160336596 reap_dead start 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 -- 172.21.15.73:0/4160336596 shutdown_connections 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 -- 172.21.15.73:0/4160336596 >> 172.21.15.73:0/4160336596 conn(0x7fae5c0ce8e0 msgr2=0x7fae5c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 -- 172.21.15.73:0/4160336596 shutdown_connections 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.266+0000 7fae62000640 1 -- 172.21.15.73:0/4160336596 wait complete. 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.268+0000 7fae62000640 1 Processor -- start 2024-04-05T22:31:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.268+0000 7fae62000640 1 -- start start 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400047c0 0x7fae5c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c157310 0x7fae5c069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0af3b0 con 0x7fae5c1559b0 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0ae920 con 0x7fae400047c0 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae62000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c1521e0 con 0x7fae5c157310 2024-04-05T22:31:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.269+0000 7fae5aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47920/0 (socket says 172.21.15.73:47920) 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 -- 172.21.15.73:0/4280158082 learned_addr learned my addr 172.21.15.73:0/4280158082 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 -- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c157310 msgr2=0x7fae5c069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c157310 0x7fae5c069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 -- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400047c0 msgr2=0x7fae5c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5b400640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400047c0 0x7fae5c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400047c0 0x7fae5c0ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 -- 172.21.15.73:0/4280158082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0aeb10 con 0x7fae5c1559b0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae5aa00640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e230 tx=0x7fae4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.270+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c034d50 con 0x7fae5c1559b0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.271+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c040450 con 0x7fae5c1559b0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.271+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0407e0 con 0x7fae5c1559b0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.271+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0aedf0 con 0x7fae5c1559b0 2024-04-05T22:31:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.271+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c159060 con 0x7fae5c1559b0 2024-04-05T22:31:07.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.272+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c066af0 con 0x7fae5c1559b0 2024-04-05T22:31:07.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.273+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fae4c040940 con 0x7fae5c1559b0 2024-04-05T22:31:07.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.274+0000 7fae53400640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 0x7fae3806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:07.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.274+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae4c0aa2b0 con 0x7fae5c1559b0 2024-04-05T22:31:07.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.274+0000 7fae5b400640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 0x7fae3806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:07.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.275+0000 7fae5b400640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 0x7fae3806d8d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fae44007fd0 tx=0x7fae44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:07.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.278+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fae4c0aee60 con 0x7fae5c1559b0 2024-04-05T22:31:07.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.510+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae5c158ec0 con 0x7fae5c1559b0 2024-04-05T22:31:07.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.511+0000 7fae53400640 1 -- 172.21.15.73:0/4280158082 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7fae5c158ec0 con 0x7fae5c1559b0 2024-04-05T22:31:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:31:07.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 msgr2=0x7fae3806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 0x7fae3806d8d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fae44007fd0 tx=0x7fae44009040 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 msgr2=0x7fae5c151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e230 tx=0x7fae4c009750 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 shutdown_connections 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fae3806b410 0x7fae3806d8d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae5c157310 0x7fae5c069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae5c1559b0 0x7fae5c151ca0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 --2- 172.21.15.73:0/4280158082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae400047c0 0x7fae5c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 >> 172.21.15.73:0/4280158082 conn(0x7fae5c0ce8e0 msgr2=0x7fae5c071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.513+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 shutdown_connections 2024-04-05T22:31:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:07.514+0000 7fae62000640 1 -- 172.21.15.73:0/4280158082 wait complete. 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:31:07.531 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:07.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.532 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:31:07.588 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:31:07.588 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:31:07.588 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:31:07.588 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.588 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:31:07.589 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42019sec preferred_lft 42019sec 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:31:07.590 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.591 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:07.591 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:31:07.591 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:31:07.591 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:07.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.592 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:31:07.679 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:31:07.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.684 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:31:07.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:31:07.738 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:31:07.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.921 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:31:07.972 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:31:07.972 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:31:07.973 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:07.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:07.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:31:08.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:08.069 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:08.121 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:08.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:08.122 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:08.175 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:31:08.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:08.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:08.311 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:08.312 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:08.345 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:08.345 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:08.345 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:31:08.345 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:31:08.345 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:31:08.348 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:31:08.348 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:31:08.348 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:31:08.348 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:31:08.349 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:31:08.352 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:31:08.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:08.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:08.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:08.433 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:31:08.485 INFO:teuthology.orchestra.run.smithi157.stdout:client6706 2024-04-05T22:31:08.486 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:08.486 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:08.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:31:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.764+0000 7f7bc9400640 1 Processor -- start 2024-04-05T22:31:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.764+0000 7f7bc9400640 1 -- start start 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.765+0000 7f7bc9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc415ae50 0x7f7bc40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.765+0000 7f7bc9400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7bc40cce90 0x7f7bc40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.765+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc40cecd0 0x7f7bc40cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc9400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40cd320 con 0x7f7bc415ae50 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc9400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40ce2f0 con 0x7f7bc40cce90 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40cf660 con 0x7f7bc40cecd0 2024-04-05T22:31:08.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc415ae50 0x7f7bc40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7bc415ae50 0x7f7bc40b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47930/0 (socket says 172.21.15.73:47930) 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 -- 172.21.15.73:0/2889730637 learned_addr learned my addr 172.21.15.73:0/2889730637 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 -- 172.21.15.73:0/2889730637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc40cecd0 msgr2=0x7f7bc40cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2889730637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc40cecd0 0x7f7bc40cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc1e00640 1 --2- 172.21.15.73:0/2889730637 >> v2:172.21.15.157:3300/0 conn(0x7f7bc40cce90 0x7f7bc40b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 -- 172.21.15.73:0/2889730637 >> v2:172.21.15.157:3300/0 conn(0x7f7bc40cce90 msgr2=0x7f7bc40b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2889730637 >> v2:172.21.15.157:3300/0 conn(0x7f7bc40cce90 0x7f7bc40b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.766+0000 7f7bc2800640 1 -- 172.21.15.73:0/2889730637 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc415b230 con 0x7f7bc415ae50 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2889730637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc415ae50 0x7f7bc40b3aa0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7bc406d1e0 tx=0x7f7bb402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e08fb6c6742ebfa server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc1400640 1 -- 172.21.15.73:0/2889730637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4031750 con 0x7f7bc415ae50 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc1400640 1 -- 172.21.15.73:0/2889730637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb40318b0 con 0x7f7bc415ae50 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc1400640 1 -- 172.21.15.73:0/2889730637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4031c40 con 0x7f7bc415ae50 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc9400640 1 -- 172.21.15.73:0/2889730637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc415ae50 msgr2=0x7f7bc40b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.768+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2889730637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc415ae50 0x7f7bc40b3aa0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7bc406d1e0 tx=0x7f7bb402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 -- 172.21.15.73:0/2889730637 shutdown_connections 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2889730637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc40cecd0 0x7f7bc40cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2889730637 >> v2:172.21.15.157:3300/0 conn(0x7f7bc40cce90 0x7f7bc40b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2889730637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc415ae50 0x7f7bc40b3aa0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 -- 172.21.15.73:0/2889730637 >> 172.21.15.73:0/2889730637 conn(0x7f7bc406a0d0 msgr2=0x7f7bc406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:08.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 -- 172.21.15.73:0/2889730637 shutdown_connections 2024-04-05T22:31:08.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.769+0000 7f7bc9400640 1 -- 172.21.15.73:0/2889730637 wait complete. 2024-04-05T22:31:08.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.770+0000 7f7bc9400640 1 Processor -- start 2024-04-05T22:31:08.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.770+0000 7f7bc9400640 1 -- start start 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc40cecd0 0x7f7bc40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc415ae50 0x7f7bc40ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415d6a0 con 0x7f7bc40cce90 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415e390 con 0x7f7bc40cecd0 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.771+0000 7f7bc9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415d210 con 0x7f7bc415ae50 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47946/0 (socket says 172.21.15.73:47946) 2024-04-05T22:31:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 -- 172.21.15.73:0/2061540798 learned_addr learned my addr 172.21.15.73:0/2061540798 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc3200640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc415ae50 0x7f7bc40ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 -- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc415ae50 msgr2=0x7f7bc40ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc1e00640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc40cecd0 0x7f7bc40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc415ae50 0x7f7bc40ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 -- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc40cecd0 msgr2=0x7f7bc40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc40cecd0 0x7f7bc40ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.772+0000 7f7bc2800640 1 -- 172.21.15.73:0/2061540798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40094d0 con 0x7f7bc40cce90 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bc2800640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4000c00 tx=0x7f7bb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4033c80 con 0x7f7bc40cce90 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb4033de0 con 0x7f7bc40cce90 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc414f660 con 0x7f7bc40cce90 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb403c620 con 0x7f7bc40cce90 2024-04-05T22:31:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.773+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc415c8e0 con 0x7f7bc40cce90 2024-04-05T22:31:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.774+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4068160 con 0x7f7bc40cce90 2024-04-05T22:31:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.775+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f7bb403a040 con 0x7f7bc40cce90 2024-04-05T22:31:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.776+0000 7f7bbaa00640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 0x7f7ba406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.776+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bb40a9800 con 0x7f7bc40cce90 2024-04-05T22:31:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.776+0000 7f7bc1e00640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 0x7f7ba406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:08.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.778+0000 7f7bc1e00640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 0x7f7ba406d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4158820 tx=0x7f7bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:08.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:08.780+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7bb40765e0 con 0x7f7bc40cce90 2024-04-05T22:31:09.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.012+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7bc4073770 con 0x7f7bc40cce90 2024-04-05T22:31:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.590+0000 7f7bbaa00640 1 -- 172.21.15.73:0/2061540798 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7bb4038020 con 0x7f7bc40cce90 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 msgr2=0x7f7ba406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 0x7f7ba406d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4158820 tx=0x7f7bb0008040 comp rx=0 tx=0).stop 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 msgr2=0x7f7bc415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4000c00 tx=0x7f7bb40046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 shutdown_connections 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ba406b410 0x7f7ba406d8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bc415ae50 0x7f7bc40ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bc40cecd0 0x7f7bc40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 --2- 172.21.15.73:0/2061540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc40cce90 0x7f7bc415de50 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 >> 172.21.15.73:0/2061540798 conn(0x7f7bc406a0d0 msgr2=0x7f7bc406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 shutdown_connections 2024-04-05T22:31:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.594+0000 7f7bc9400640 1 -- 172.21.15.73:0/2061540798 wait complete. 2024-04-05T22:31:09.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:31:09.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.821+0000 7f6cb4800640 1 Processor -- start 2024-04-05T22:31:09.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.821+0000 7f6cb4800640 1 -- start start 2024-04-05T22:31:09.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb4800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cac1587d0 0x7f6cac0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb4800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cac0ccd20 0x7f6cac06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb4800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac06f5b0 con 0x7f6cac06d0c0 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb4800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac070410 con 0x7f6cac0ccd20 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb4800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac158bb0 con 0x7f6cac1587d0 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cac0ccd20 0x7f6cac06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6cac0ccd20 0x7f6cac06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47960/0 (socket says 172.21.15.73:47960) 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb1400640 1 -- 172.21.15.73:0/3660814637 learned_addr learned my addr 172.21.15.73:0/3660814637 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.823+0000 7f6cb2800640 1 --1- 172.21.15.73:0/3660814637 >> v1:172.21.15.73:6789/0 conn(0x7f6cac06d0c0 0x7f6cac15e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58556/0 (socket says 172.21.15.73:58556) 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.824+0000 7f6cb1e00640 1 --2- 172.21.15.73:0/3660814637 >> v2:172.21.15.157:3300/0 conn(0x7f6cac1587d0 0x7f6cac0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.824+0000 7f6cb1400640 1 -- 172.21.15.73:0/3660814637 >> v2:172.21.15.157:3300/0 conn(0x7f6cac1587d0 msgr2=0x7f6cac0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.824+0000 7f6cb1400640 1 --2- 172.21.15.73:0/3660814637 >> v2:172.21.15.157:3300/0 conn(0x7f6cac1587d0 0x7f6cac0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.824+0000 7f6cb1400640 1 -- 172.21.15.73:0/3660814637 >> v1:172.21.15.73:6789/0 conn(0x7f6cac06d0c0 legacy=0x7f6cac15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.824+0000 7f6cb1400640 1 -- 172.21.15.73:0/3660814637 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac06f800 con 0x7f6cac0ccd20 2024-04-05T22:31:09.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.825+0000 7f6cb1400640 1 --2- 172.21.15.73:0/3660814637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac0ccd20 0x7f6cac06cb80 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff03efbd3ad51cff server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:09.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.825+0000 7f6cb0a00640 1 -- 172.21.15.73:0/3660814637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c030860 con 0x7f6cac0ccd20 2024-04-05T22:31:09.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.825+0000 7f6cb0a00640 1 -- 172.21.15.73:0/3660814637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c0309c0 con 0x7f6cac0ccd20 2024-04-05T22:31:09.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.825+0000 7f6cb0a00640 1 -- 172.21.15.73:0/3660814637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c030d50 con 0x7f6cac0ccd20 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 -- 172.21.15.73:0/3660814637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac0ccd20 msgr2=0x7f6cac06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3660814637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac0ccd20 0x7f6cac06cb80 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c02b9a0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 -- 172.21.15.73:0/3660814637 shutdown_connections 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3660814637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac0ccd20 0x7f6cac06cb80 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3660814637 >> v2:172.21.15.157:3300/0 conn(0x7f6cac1587d0 0x7f6cac0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 -- 172.21.15.73:0/3660814637 >> 172.21.15.73:0/3660814637 conn(0x7f6cac0ce8e0 msgr2=0x7f6cac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 -- 172.21.15.73:0/3660814637 shutdown_connections 2024-04-05T22:31:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.826+0000 7f6cb4800640 1 -- 172.21.15.73:0/3660814637 wait complete. 2024-04-05T22:31:09.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.828+0000 7f6cb4800640 1 Processor -- start 2024-04-05T22:31:09.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.828+0000 7f6cb4800640 1 -- start start 2024-04-05T22:31:09.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac06d0c0 0x7f6cac0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 0x7f6cac0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0b3900 con 0x7f6cac1587d0 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac149880 con 0x7f6cac06d0c0 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1f2670 con 0x7f6cac151f30 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47964/0 (socket says 172.21.15.73:47964) 2024-04-05T22:31:09.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb1400640 1 -- 172.21.15.73:0/3716489725 learned_addr learned my addr 172.21.15.73:0/3716489725 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:09.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.829+0000 7f6cb1e00640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac06d0c0 0x7f6cac0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb2800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 0x7f6cac0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 -- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 msgr2=0x7f6cac0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 0x7f6cac0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 -- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac06d0c0 msgr2=0x7f6cac0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:09.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac06d0c0 0x7f6cac0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:09.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 -- 172.21.15.73:0/3716489725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0094d0 con 0x7f6cac1587d0 2024-04-05T22:31:09.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb2800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 0x7f6cac0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb1400640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c000c00 tx=0x7f6c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c039c90 con 0x7f6cac1587d0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c039df0 con 0x7f6cac1587d0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac0b2000 con 0x7f6cac1587d0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c043640 con 0x7f6cac1587d0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.830+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0b25c0 con 0x7f6cac1587d0 2024-04-05T22:31:09.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.832+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac066af0 con 0x7f6cac1587d0 2024-04-05T22:31:09.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.833+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f6c9c0437a0 con 0x7f6cac1587d0 2024-04-05T22:31:09.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.834+0000 7f6ca6000640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 0x7f6c9006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:09.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.834+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c9c03e070 con 0x7f6cac1587d0 2024-04-05T22:31:09.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.835+0000 7f6cb1e00640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 0x7f6c9006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:09.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.836+0000 7f6cb1e00640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 0x7f6c9006d8a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006fd0 tx=0x7f6ca0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:09.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:09.838+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6c9c077a20 con 0x7f6cac1587d0 2024-04-05T22:31:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.056+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cac0700a0 con 0x7f6c9006b3e0 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.060+0000 7f6ca6000640 1 -- 172.21.15.73:0/3716489725 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6ca000f860 con 0x7f6c9006b3e0 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:10.059 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 msgr2=0x7f6c9006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 0x7f6c9006d8a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006fd0 tx=0x7f6ca0008040 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 msgr2=0x7f6cac149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c000c00 tx=0x7f6c9c004730 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 shutdown_connections 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6c9006b3e0 0x7f6c9006d8a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cac151f30 0x7f6cac0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cac1587d0 0x7f6cac149340 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 --2- 172.21.15.73:0/3716489725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac06d0c0 0x7f6cac0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.062+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 >> 172.21.15.73:0/3716489725 conn(0x7f6cac0ce8e0 msgr2=0x7f6cac159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.063+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 shutdown_connections 2024-04-05T22:31:10.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.063+0000 7f6cb4800640 1 -- 172.21.15.73:0/3716489725 wait complete. 2024-04-05T22:31:10.081 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:31:10.233 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:31:10.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:31:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.281+0000 7f85b5800640 1 Processor -- start 2024-04-05T22:31:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.282+0000 7f85b5800640 1 -- start start 2024-04-05T22:31:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85b5800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85b015aa70 0x7f85b0070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85b5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85b5800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b015ae50 con 0x7f85b00714a0 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85b5800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b0071930 con 0x7f85b015aa70 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85b5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b015e3b0 con 0x7f85b015dd90 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85a7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f85b00714a0 0x7f85b015d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58560/0 (socket says 172.21.15.73:58560) 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85a7e00640 1 -- 172.21.15.73:0/3234003971 learned_addr learned my addr 172.21.15.73:0/3234003971 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85aea00640 1 --2- 172.21.15.73:0/3234003971 >> v2:172.21.15.157:3300/0 conn(0x7f85b015aa70 0x7f85b0070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.283+0000 7f85af400640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85ae000640 1 -- 172.21.15.73:0/3234003971 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783208622 0 0) 0x7f85b015ae50 con 0x7f85b00714a0 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85ae000640 1 -- 172.21.15.73:0/3234003971 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8594003520 con 0x7f85b00714a0 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85aea00640 1 -- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 msgr2=0x7f85b006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85aea00640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85aea00640 1 -- 172.21.15.73:0/3234003971 >> v1:172.21.15.73:6789/0 conn(0x7f85b00714a0 legacy=0x7f85b015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.284+0000 7f85aea00640 1 -- 172.21.15.73:0/3234003971 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00ac9e0 con 0x7f85b015aa70 2024-04-05T22:31:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.285+0000 7f85aea00640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0070f60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86b06b7d36387ab server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:10.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.286+0000 7f85ae000640 1 -- 172.21.15.73:0/3234003971 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a002d040 con 0x7f85b015aa70 2024-04-05T22:31:10.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.286+0000 7f85ae000640 1 -- 172.21.15.73:0/3234003971 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0031810 con 0x7f85b015aa70 2024-04-05T22:31:10.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.286+0000 7f85ae000640 1 -- 172.21.15.73:0/3234003971 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031b80 con 0x7f85b015aa70 2024-04-05T22:31:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 -- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 msgr2=0x7f85b0070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0070f60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002b8b0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 -- 172.21.15.73:0/3234003971 shutdown_connections 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 --2- 172.21.15.73:0/3234003971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0070f60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 -- 172.21.15.73:0/3234003971 >> 172.21.15.73:0/3234003971 conn(0x7f85b0069cb0 msgr2=0x7f85b006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 -- 172.21.15.73:0/3234003971 shutdown_connections 2024-04-05T22:31:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.287+0000 7f85b5800640 1 -- 172.21.15.73:0/3234003971 wait complete. 2024-04-05T22:31:10.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.290+0000 7f85b5800640 1 Processor -- start 2024-04-05T22:31:10.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.290+0000 7f85b5800640 1 -- start start 2024-04-05T22:31:10.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b01512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01f2500 con 0x7f85b00714a0 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b015b000 con 0x7f85b015aa70 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.291+0000 7f85b5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b015b860 con 0x7f85b015dd90 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47968/0 (socket says 172.21.15.73:47968) 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 -- 172.21.15.73:0/4179325037 learned_addr learned my addr 172.21.15.73:0/4179325037 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:10.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85a7e00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85af400640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b01512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 -- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 msgr2=0x7f85b01512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b01512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 -- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 msgr2=0x7f85b0150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 -- 172.21.15.73:0/4179325037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009400 con 0x7f85b00714a0 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85a7e00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.292+0000 7f85aea00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000c00 tx=0x7f85a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:10.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.293+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0003820 con 0x7f85b00714a0 2024-04-05T22:31:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.293+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a003f070 con 0x7f85b00714a0 2024-04-05T22:31:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.293+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0003bc0 con 0x7f85b00714a0 2024-04-05T22:31:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.293+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b014c300 con 0x7f85b00714a0 2024-04-05T22:31:10.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.293+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b015b210 con 0x7f85b00714a0 2024-04-05T22:31:10.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.294+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0068160 con 0x7f85b00714a0 2024-04-05T22:31:10.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.295+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f85a0003d20 con 0x7f85b00714a0 2024-04-05T22:31:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.296+0000 7f85a6a00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 0x7f858c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.296+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85a0041070 con 0x7f85b00714a0 2024-04-05T22:31:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.296+0000 7f85a7e00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 0x7f858c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.297+0000 7f85a7e00640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 0x7f858c06d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85b01505e0 tx=0x7f8598009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:10.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.300+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f85a0076da0 con 0x7f85b00714a0 2024-04-05T22:31:10.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.495+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f85b00ae1c0 con 0x7f85b00714a0 2024-04-05T22:31:10.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.496+0000 7f85a6a00640 1 -- 172.21.15.73:0/4179325037 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85a00350c0 con 0x7f85b00714a0 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 msgr2=0x7f858c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 0x7f858c06d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85b01505e0 tx=0x7f8598009040 comp rx=0 tx=0).stop 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 msgr2=0x7f85b014bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000c00 tx=0x7f85a00046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 shutdown_connections 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f858c06b3e0 0x7f858c06d8a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85b015dd90 0x7f85b01512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85b015aa70 0x7f85b0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 --2- 172.21.15.73:0/4179325037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b00714a0 0x7f85b014bdc0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:10.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.498+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 >> 172.21.15.73:0/4179325037 conn(0x7f85b0069cb0 msgr2=0x7f85b006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:10.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.499+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 shutdown_connections 2024-04-05T22:31:10.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:10.499+0000 7f85b5800640 1 -- 172.21.15.73:0/4179325037 wait complete. 2024-04-05T22:31:10.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:10.516 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol94430/c483c139-b244-46a6-8dfe-2cfd4539088a) 2024-04-05T22:31:10.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:10.620 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:31:10.673 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:31:10.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:10.673 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol94430/c483c139-b244-46a6-8dfe-2cfd4539088a) 2024-04-05T22:31:10.770 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:10.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol94430/.meta 2024-04-05T22:31:10.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:10.875 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp11670/subvol93001/76189e2d-ae32-4420-820d-44fb97bb3d17) 2024-04-05T22:31:11.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:11.023 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:31:11.079 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:31:11.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:11.080 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp11670/subvol93001/76189e2d-ae32-4420-820d-44fb97bb3d17) 2024-04-05T22:31:11.179 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:11.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp11670/subvol93001/.meta 2024-04-05T22:31:11.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol94430 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.529+0000 7fa9aac00640 1 Processor -- start 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.529+0000 7fa9aac00640 1 -- start start 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9aac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9a415a850 0x7fa9a415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9aac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a406bc50 con 0x7fa9a4070640 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9aac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a415ef70 con 0x7fa9a415e580 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9aac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a415f530 con 0x7fa9a415a850 2024-04-05T22:31:11.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9a9800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9a415a850 0x7fa9a415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9a9800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9a415a850 0x7fa9a415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47978/0 (socket says 172.21.15.73:47978) 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9a9800640 1 -- 172.21.15.73:0/827344552 learned_addr learned my addr 172.21.15.73:0/827344552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9aa200640 1 --1- 172.21.15.73:0/827344552 >> v1:172.21.15.73:6789/0 conn(0x7fa9a4070640 0x7fa9a406c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58570/0 (socket says 172.21.15.73:58570) 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.531+0000 7fa9a9800640 1 -- 172.21.15.73:0/827344552 >> v1:172.21.15.157:6789/0 conn(0x7fa9a415e580 legacy=0x7fa9a406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.532+0000 7fa9a9800640 1 -- 172.21.15.73:0/827344552 >> v1:172.21.15.73:6789/0 conn(0x7fa9a4070640 legacy=0x7fa9a406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.532+0000 7fa9a9800640 1 -- 172.21.15.73:0/827344552 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a415f250 con 0x7fa9a415a850 2024-04-05T22:31:11.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.533+0000 7fa9a9800640 1 --2- 172.21.15.73:0/827344552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415ac30 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa998009870 tx=0x7fa99802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a33c1426edd3e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:11.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.533+0000 7fa9a3e00640 1 -- 172.21.15.73:0/827344552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998031870 con 0x7fa9a415a850 2024-04-05T22:31:11.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.533+0000 7fa9a3e00640 1 -- 172.21.15.73:0/827344552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9980319d0 con 0x7fa9a415a850 2024-04-05T22:31:11.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.533+0000 7fa9a3e00640 1 -- 172.21.15.73:0/827344552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998031d60 con 0x7fa9a415a850 2024-04-05T22:31:11.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 -- 172.21.15.73:0/827344552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 msgr2=0x7fa9a415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 --2- 172.21.15.73:0/827344552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415ac30 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa998009870 tx=0x7fa99802b910 comp rx=0 tx=0).stop 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 -- 172.21.15.73:0/827344552 shutdown_connections 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 --2- 172.21.15.73:0/827344552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415ac30 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 -- 172.21.15.73:0/827344552 >> 172.21.15.73:0/827344552 conn(0x7fa9a4069cb0 msgr2=0x7fa9a406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 -- 172.21.15.73:0/827344552 shutdown_connections 2024-04-05T22:31:11.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.534+0000 7fa9aac00640 1 -- 172.21.15.73:0/827344552 wait complete. 2024-04-05T22:31:11.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.536+0000 7fa9aac00640 1 Processor -- start 2024-04-05T22:31:11.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.536+0000 7fa9aac00640 1 -- start start 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9a4070640 0x7fa9a413a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9a415e580 0x7fa9a415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41f04b0 con 0x7fa9a415a850 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41f0730 con 0x7fa9a4070640 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9aac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41f0a10 con 0x7fa9a415e580 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47988/0 (socket says 172.21.15.73:47988) 2024-04-05T22:31:11.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 -- 172.21.15.73:0/1207194301 learned_addr learned my addr 172.21.15.73:0/1207194301 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:11.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 -- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9a415e580 msgr2=0x7fa9a415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9a415e580 0x7fa9a415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 -- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9a4070640 msgr2=0x7fa9a413a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9a4070640 0x7fa9a413a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.537+0000 7fa9a8e00640 1 -- 172.21.15.73:0/1207194301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9980094d0 con 0x7fa9a415a850 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9a8e00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b3f0 tx=0x7fa99000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990011830 con 0x7fa9a415a850 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990011990 con 0x7fa9a415a850 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990011d40 con 0x7fa9a415a850 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a41f0cf0 con 0x7fa9a415a850 2024-04-05T22:31:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.538+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a413a860 con 0x7fa9a415a850 2024-04-05T22:31:11.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.539+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a41f1210 con 0x7fa9a415a850 2024-04-05T22:31:11.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.540+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fa99001a400 con 0x7fa9a415a850 2024-04-05T22:31:11.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.541+0000 7fa9a1600640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 0x7fa98406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:11.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.541+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9900878f0 con 0x7fa9a415a850 2024-04-05T22:31:11.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.542+0000 7fa9a9800640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 0x7fa98406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:11.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.543+0000 7fa9a9800640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 0x7fa98406d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa998036000 tx=0x7fa998030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:11.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.545+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9a41f1210 con 0x7fa9a415a850 2024-04-05T22:31:11.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.767+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol94430", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9a406c1d0 con 0x7fa98406b3e0 2024-04-05T22:31:11.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:11.781+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:11.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:11.782+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:11.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:11.782+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:11.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:11.782+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:11.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:11.782+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:11.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.899+0000 7fa9a1600640 1 -- 172.21.15.73:0/1207194301 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7fa998031c40 con 0x7fa98406b3e0 2024-04-05T22:31:11.898 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/_nogroup/subvol94430/c483c139-b244-46a6-8dfe-2cfd4539088a 2024-04-05T22:31:11.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.902+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 msgr2=0x7fa98406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.902+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 0x7fa98406d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa998036000 tx=0x7fa998030000 comp rx=0 tx=0).stop 2024-04-05T22:31:11.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.902+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 msgr2=0x7fa9a415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:11.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.902+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b3f0 tx=0x7fa99000b8b0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 shutdown_connections 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa98406b3e0 0x7fa98406d8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9a415e580 0x7fa9a415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9a415a850 0x7fa9a415c6f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 --2- 172.21.15.73:0/1207194301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9a4070640 0x7fa9a413a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 >> 172.21.15.73:0/1207194301 conn(0x7fa9a4069cb0 msgr2=0x7fa9a40af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 shutdown_connections 2024-04-05T22:31:11.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:11.903+0000 7fa9aac00640 1 -- 172.21.15.73:0/1207194301 wait complete. 2024-04-05T22:31:11.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol93001 subvol_grp11670 2024-04-05T22:31:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.145+0000 7f8ea2c00640 1 Processor -- start 2024-04-05T22:31:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.145+0000 7f8ea2c00640 1 -- start start 2024-04-05T22:31:12.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea2c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e9c0b3ab0 0x7f8e9c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea2c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9c1620e0 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea2c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c0ae370 con 0x7f8e9c0b3ab0 2024-04-05T22:31:12.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0ae090 con 0x7f8e9c0ad720 2024-04-05T22:31:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.147+0000 7f8ea0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.148+0000 7f8e9b400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e9c0b3ab0 0x7f8e9c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.148+0000 7f8ea0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:32796/0 (socket says 172.21.15.73:32796) 2024-04-05T22:31:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.148+0000 7f8ea0c00640 1 -- 172.21.15.73:0/3959242111 learned_addr learned my addr 172.21.15.73:0/3959242111 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.149+0000 7f8ea0c00640 1 -- 172.21.15.73:0/3959242111 >> v2:172.21.15.157:3300/0 conn(0x7f8e9c0b3ab0 msgr2=0x7f8e9c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.149+0000 7f8ea0c00640 1 --2- 172.21.15.73:0/3959242111 >> v2:172.21.15.157:3300/0 conn(0x7f8e9c0b3ab0 0x7f8e9c162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.149+0000 7f8ea0c00640 1 -- 172.21.15.73:0/3959242111 >> v1:172.21.15.157:6789/0 conn(0x7f8e9c0cecd0 legacy=0x7f8e9c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.149+0000 7f8ea0c00640 1 -- 172.21.15.73:0/3959242111 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c161a90 con 0x7f8e9c0ad720 2024-04-05T22:31:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.150+0000 7f8ea0c00640 1 --2- 172.21.15.73:0/3959242111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e9402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdcd1c76d2def5df server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.151+0000 7f8e9aa00640 1 -- 172.21.15.73:0/3959242111 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9402cb00 con 0x7f8e9c0ad720 2024-04-05T22:31:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.151+0000 7f8e9aa00640 1 -- 172.21.15.73:0/3959242111 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9402cc60 con 0x7f8e9c0ad720 2024-04-05T22:31:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.151+0000 7f8e9aa00640 1 -- 172.21.15.73:0/3959242111 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940343f0 con 0x7f8e9c0ad720 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.151+0000 7f8ea2c00640 1 -- 172.21.15.73:0/3959242111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 msgr2=0x7f8e9c164f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.151+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/3959242111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e9402bf60 comp rx=0 tx=0).stop 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 -- 172.21.15.73:0/3959242111 shutdown_connections 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/3959242111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c164f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/3959242111 >> v2:172.21.15.157:3300/0 conn(0x7f8e9c0b3ab0 0x7f8e9c162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 -- 172.21.15.73:0/3959242111 >> 172.21.15.73:0/3959242111 conn(0x7f8e9c06a0d0 msgr2=0x7f8e9c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 -- 172.21.15.73:0/3959242111 shutdown_connections 2024-04-05T22:31:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.152+0000 7f8ea2c00640 1 -- 172.21.15.73:0/3959242111 wait complete. 2024-04-05T22:31:12.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.154+0000 7f8ea2c00640 1 Processor -- start 2024-04-05T22:31:12.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.154+0000 7f8ea2c00640 1 -- start start 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c1507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 0x7f8e9c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c150d30 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1f7f60 con 0x7f8e9c0b3ab0 2024-04-05T22:31:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c155db0 con 0x7f8e9c0ad720 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47996/0 (socket says 172.21.15.73:47996) 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8ea0c00640 1 -- 172.21.15.73:0/1949651493 learned_addr learned my addr 172.21.15.73:0/1949651493 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8e9b400640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 0x7f8e9c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.155+0000 7f8e9be00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 -- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 msgr2=0x7f8e9c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 -- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 msgr2=0x7f8e9c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 0x7f8e9c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 -- 172.21.15.73:0/1949651493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94009b30 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8e9b400640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 0x7f8e9c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8e9be00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c1507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea0c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94009510 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9403e450 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9403e800 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.156+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c156090 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.157+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1f8490 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.158+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c0b1550 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.159+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7f8e9403e960 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.159+0000 7f8e93e00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 0x7f8e7c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.159+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e940aa710 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.160+0000 7f8e9be00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 0x7f8e7c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.161+0000 7f8e9be00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 0x7f8e7c06d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000f60 tx=0x7f8e8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.164+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8e940774f0 con 0x7f8e9c0cecd0 2024-04-05T22:31:12.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.389+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol93001", "group_name": "subvol_grp11670", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e9c15bcf0 con 0x7f8e7c06b3e0 2024-04-05T22:31:12.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.510+0000 7f8e93e00640 1 -- 172.21.15.73:0/1949651493 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f8e8c001340 con 0x7f8e7c06b3e0 2024-04-05T22:31:12.510 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp11670/subvol93001/76189e2d-ae32-4420-820d-44fb97bb3d17 2024-04-05T22:31:12.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 msgr2=0x7f8e7c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 0x7f8e7c06d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000f60 tx=0x7f8e8c009040 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 msgr2=0x7f8e9c0706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e94004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 shutdown_connections 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8e7c06b3e0 0x7f8e7c06d8a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c0cecd0 0x7f8e9c0706f0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e9c0b3ab0 0x7f8e9c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 --2- 172.21.15.73:0/1949651493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e9c0ad720 0x7f8e9c1507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 >> 172.21.15.73:0/1949651493 conn(0x7f8e9c06a0d0 msgr2=0x7f8e9c1640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 shutdown_connections 2024-04-05T22:31:12.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.514+0000 7f8ea2c00640 1 -- 172.21.15.73:0/1949651493 wait complete. 2024-04-05T22:31:12.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:12.534 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol94430/.meta) 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout:version = 2 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/_nogroup/subvol94430/c483c139-b244-46a6-8dfe-2cfd4539088a 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:31:12.623 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:12.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:12.627 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp11670/subvol93001/.meta) 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout:version = 2 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/subvol_grp11670/subvol93001/76189e2d-ae32-4420-820d-44fb97bb3d17 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:31:12.712 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:12.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol94430 2024-04-05T22:31:12.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.936+0000 7fc4e6c00640 1 Processor -- start 2024-04-05T22:31:12.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.936+0000 7fc4e6c00640 1 -- start start 2024-04-05T22:31:12.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e6c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4e0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e6c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e00add00 0x7fc4e006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e6c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e00b27f0 con 0x7fc4e006cb80 2024-04-05T22:31:12.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e6c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e006d010 con 0x7fc4e00add00 2024-04-05T22:31:12.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e6c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00b3890 con 0x7fc4e01599f0 2024-04-05T22:31:12.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e4c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc4e006cb80 0x7fc4e015d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58572/0 (socket says 172.21.15.73:58572) 2024-04-05T22:31:12.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.938+0000 7fc4e4c00640 1 -- 172.21.15.73:0/3193423259 learned_addr learned my addr 172.21.15.73:0/3193423259 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:12.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4d7400640 1 --2- 172.21.15.73:0/3193423259 >> v2:172.21.15.157:3300/0 conn(0x7fc4e00add00 0x7fc4e006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4e0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703658208 0 0) 0x7fc4e00b27f0 con 0x7fc4e006cb80 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c4003520 con 0x7fc4e006cb80 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3607065830 0 0) 0x7fc4c4003520 con 0x7fc4e006cb80 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 msgr2=0x7fc4e0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4e0070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 >> v2:172.21.15.157:3300/0 conn(0x7fc4e00add00 msgr2=0x7fc4e006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 --2- 172.21.15.73:0/3193423259 >> v2:172.21.15.157:3300/0 conn(0x7fc4e00add00 0x7fc4e006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00b2ad0 con 0x7fc4e006cb80 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc4d80019f0 con 0x7fc4e006cb80 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 >> v1:172.21.15.73:6789/0 conn(0x7fc4e006cb80 legacy=0x7fc4e015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004d00 0x7fc4c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e006d010 con 0x7fc4c4003970 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.939+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00b3890 con 0x7fc4c4004d00 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c400b980 con 0x7fc4e01599f0 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4d7400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4e4c00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004d00 0x7fc4c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 -- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 msgr2=0x7fc4c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4c40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 -- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004d00 msgr2=0x7fc4c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004d00 0x7fc4c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4dfe00640 1 -- 172.21.15.73:0/3193423259 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4003520 con 0x7fc4c4003970 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4d7400640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01599f0 0x7fc4c40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:12.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.940+0000 7fc4e4c00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004d00 0x7fc4c4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:12.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.941+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4c4003d90 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009ac0 tx=0x7fc4c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb195922adf3cc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.941+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d002c6b0 con 0x7fc4c4003970 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.941+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d002c810 con 0x7fc4c4003970 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.941+0000 7fc4df400640 1 -- 172.21.15.73:0/3193423259 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d002cba0 con 0x7fc4c4003970 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 -- 172.21.15.73:0/3193423259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 msgr2=0x7fc4c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/3193423259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4c4003d90 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009ac0 tx=0x7fc4c40057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e4c00640 1 -- 172.21.15.73:0/3193423259 reap_dead start 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 -- 172.21.15.73:0/3193423259 shutdown_connections 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 -- 172.21.15.73:0/3193423259 >> 172.21.15.73:0/3193423259 conn(0x7fc4e0069cb0 msgr2=0x7fc4e006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 -- 172.21.15.73:0/3193423259 shutdown_connections 2024-04-05T22:31:12.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.942+0000 7fc4e6c00640 1 -- 172.21.15.73:0/3193423259 wait complete. 2024-04-05T22:31:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.944+0000 7fc4e6c00640 1 Processor -- start 2024-04-05T22:31:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.944+0000 7fc4e6c00640 1 -- start start 2024-04-05T22:31:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0152480 0x7fc4e01f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01f6250 0x7fc4e00aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00a8a40 con 0x7fc4c4003970 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e01f6cb0 con 0x7fc4e0152480 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4e6c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e01f66f0 con 0x7fc4e01f6250 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48026/0 (socket says 172.21.15.73:48026) 2024-04-05T22:31:12.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.945+0000 7fc4dfe00640 1 -- 172.21.15.73:0/2646925615 learned_addr learned my addr 172.21.15.73:0/2646925615 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 -- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01f6250 msgr2=0x7fc4e00aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01f6250 0x7fc4e00aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 -- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0152480 msgr2=0x7fc4e01f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0152480 0x7fc4e01f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 -- 172.21.15.73:0/2646925615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e01f69d0 con 0x7fc4c4003970 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dfe00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0003810 con 0x7fc4c4003970 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0003970 con 0x7fc4c4003970 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e00a8660 con 0x7fc4c4003970 2024-04-05T22:31:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.946+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0003d20 con 0x7fc4c4003970 2024-04-05T22:31:12.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.947+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e0152900 con 0x7fc4c4003970 2024-04-05T22:31:12.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.948+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0152a90 con 0x7fc4c4003970 2024-04-05T22:31:12.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.948+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fc4d0034440 con 0x7fc4c4003970 2024-04-05T22:31:12.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.949+0000 7fc4dcc00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 0x7fc4bc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:12.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.949+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4d00a9e40 con 0x7fc4c4003970 2024-04-05T22:31:12.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.950+0000 7fc4d7400640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 0x7fc4bc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:12.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.951+0000 7fc4d7400640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 0x7fc4bc06d8a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc4e01572f0 tx=0x7fc4c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:12.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:12.954+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc4e0152a90 con 0x7fc4c4003970 2024-04-05T22:31:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.174+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol94430", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e00b3370 con 0x7fc4bc06b3e0 2024-04-05T22:31:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.218+0000 7fc4dcc00640 1 -- 172.21.15.73:0/2646925615 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc4c8006bd0 con 0x7fc4bc06b3e0 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.220+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 msgr2=0x7fc4bc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 0x7fc4bc06d8a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc4e01572f0 tx=0x7fc4c8008040 comp rx=0 tx=0).stop 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 msgr2=0x7fc4e00ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0004730 comp rx=0 tx=0).stop 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 shutdown_connections 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc4bc06b3e0 0x7fc4bc06d8a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e01f6250 0x7fc4e00aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0152480 0x7fc4e01f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 --2- 172.21.15.73:0/2646925615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4003970 0x7fc4e00ccc70 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 >> 172.21.15.73:0/2646925615 conn(0x7fc4e0069cb0 msgr2=0x7fc4e00aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 shutdown_connections 2024-04-05T22:31:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.221+0000 7fc4e6c00640 1 -- 172.21.15.73:0/2646925615 wait complete. 2024-04-05T22:31:13.230 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.230+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.230 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.230+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.230 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.230+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.230 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.230+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.230 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.230+0000 7f1d88600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol93001 subvol_grp11670 2024-04-05T22:31:13.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.251+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.251+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.251+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.251+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:13.251+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:13.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.449+0000 7fdf99200640 1 Processor -- start 2024-04-05T22:31:13.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.450+0000 7fdf99200640 1 -- start start 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf9415ae50 0x7fdf940b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 0x7fdf940b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf940cecd0 0x7fdf940cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940ce2f0 con 0x7fdf940cecd0 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940cd2f0 con 0x7fdf940cce90 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.451+0000 7fdf99200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9415b230 con 0x7fdf9415ae50 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf940cecd0 0x7fdf940cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf940cecd0 0x7fdf940cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48040/0 (socket says 172.21.15.73:48040) 2024-04-05T22:31:13.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 -- 172.21.15.73:0/502059762 learned_addr learned my addr 172.21.15.73:0/502059762 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf91e00640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf9415ae50 0x7fdf940b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf8be00640 1 --2- 172.21.15.73:0/502059762 >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 0x7fdf940b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 -- 172.21.15.73:0/502059762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf9415ae50 msgr2=0x7fdf940b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf9415ae50 0x7fdf940b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 -- 172.21.15.73:0/502059762 >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 msgr2=0x7fdf940b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 --2- 172.21.15.73:0/502059762 >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 0x7fdf940b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf92800640 1 -- 172.21.15.73:0/502059762 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940cf620 con 0x7fdf940cecd0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.452+0000 7fdf8be00640 1 --2- 172.21.15.73:0/502059762 >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 0x7fdf940b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.453+0000 7fdf92800640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cecd0 0x7fdf940cf0e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fdf94071ae0 tx=0x7fdf8c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abf28abed141f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf91400640 1 -- 172.21.15.73:0/502059762 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c02c9f0 con 0x7fdf940cecd0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf91400640 1 -- 172.21.15.73:0/502059762 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c02cb50 con 0x7fdf940cecd0 2024-04-05T22:31:13.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf91400640 1 -- 172.21.15.73:0/502059762 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0343f0 con 0x7fdf940cecd0 2024-04-05T22:31:13.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 -- 172.21.15.73:0/502059762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cecd0 msgr2=0x7fdf940cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:13.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cecd0 0x7fdf940cf0e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fdf94071ae0 tx=0x7fdf8c02be50 comp rx=0 tx=0).stop 2024-04-05T22:31:13.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 -- 172.21.15.73:0/502059762 shutdown_connections 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cecd0 0x7fdf940cf0e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 --2- 172.21.15.73:0/502059762 >> v2:172.21.15.157:3300/0 conn(0x7fdf940cce90 0x7fdf940b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 --2- 172.21.15.73:0/502059762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf9415ae50 0x7fdf940b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.454+0000 7fdf99200640 1 -- 172.21.15.73:0/502059762 >> 172.21.15.73:0/502059762 conn(0x7fdf9406a0d0 msgr2=0x7fdf9406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.455+0000 7fdf99200640 1 -- 172.21.15.73:0/502059762 shutdown_connections 2024-04-05T22:31:13.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.455+0000 7fdf99200640 1 -- 172.21.15.73:0/502059762 wait complete. 2024-04-05T22:31:13.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.457+0000 7fdf99200640 1 Processor -- start 2024-04-05T22:31:13.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.457+0000 7fdf99200640 1 -- start start 2024-04-05T22:31:13.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 0x7fdf9415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9415ae50 0x7fdf94071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9415db70 con 0x7fdf940cce90 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9415de50 con 0x7fdf9415ae50 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf99200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9415e130 con 0x7fdf940cecd0 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.458+0000 7fdf91e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48052/0 (socket says 172.21.15.73:48052) 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 -- 172.21.15.73:0/1012681094 learned_addr learned my addr 172.21.15.73:0/1012681094 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf8be00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 0x7fdf9415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 -- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 msgr2=0x7fdf9415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 0x7fdf9415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 -- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9415ae50 msgr2=0x7fdf94071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:13.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9415ae50 0x7fdf94071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 -- 172.21.15.73:0/1012681094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c009b30 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf8be00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 0x7fdf9415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.459+0000 7fdf91e00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400d800 tx=0x7fdf8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.460+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400b6c0 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.460+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8400b820 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.460+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf94070750 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.460+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400bbb0 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.460+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf94070d10 con 0x7fdf940cce90 2024-04-05T22:31:13.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.461+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf94068160 con 0x7fdf940cce90 2024-04-05T22:31:13.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.462+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 89859+0+0 (secure 0 0 0) 0x7fdf8400bd10 con 0x7fdf940cce90 2024-04-05T22:31:13.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.463+0000 7fdf8a000640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 0x7fdf7006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:13.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.463+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf84088010 con 0x7fdf940cce90 2024-04-05T22:31:13.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.463+0000 7fdf8be00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 0x7fdf7006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:13.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.464+0000 7fdf8be00640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 0x7fdf7006d8d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdf941f5a30 tx=0x7fdf7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:13.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.467+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdf84054d70 con 0x7fdf940cce90 2024-04-05T22:31:13.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.632+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 26) v1 ==== 89910+0+0 (secure 0 0 0) 0x7fdf84055140 con 0x7fdf940cce90 2024-04-05T22:31:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:13.694+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol93001", "group_name": "subvol_grp11670", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf940cb330 con 0x7fdf7006b410 2024-04-05T22:31:15.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:15.644+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fdf84060990 con 0x7fdf940cce90 2024-04-05T22:31:19.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.088+0000 7fdf8a000640 1 -- 172.21.15.73:0/1012681094 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdf7c00c040 con 0x7fdf7006b410 2024-04-05T22:31:19.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 msgr2=0x7fdf7006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 0x7fdf7006d8d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdf941f5a30 tx=0x7fdf7c008040 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 msgr2=0x7fdf941559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400d800 tx=0x7fdf8400dcc0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 shutdown_connections 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdf7006b410 0x7fdf7006d8d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9415ae50 0x7fdf94071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf940cecd0 0x7fdf9415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 --2- 172.21.15.73:0/1012681094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf940cce90 0x7fdf941559a0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 >> 172.21.15.73:0/1012681094 conn(0x7fdf9406a0d0 msgr2=0x7fdf9415a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 shutdown_connections 2024-04-05T22:31:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.092+0000 7fdf99200640 1 -- 172.21.15.73:0/1012681094 wait complete. 2024-04-05T22:31:19.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:19.115 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:31:19.174 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:31:19.175 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:31:19.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:19.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:19.415 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:31:19.415 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:19.415 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:19.473 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:19.473 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:19.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:19.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.737+0000 7f47ad600640 1 Processor -- start 2024-04-05T22:31:19.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.737+0000 7f47ad600640 1 -- start start 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47a81587d0 0x7f47a80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a80ccd20 0x7f47a806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47a806d0c0 0x7f47a815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a806f5b0 con 0x7f47a806d0c0 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8070410 con 0x7f47a81587d0 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47ad600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8158bb0 con 0x7f47a80ccd20 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47a7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47a806d0c0 0x7f47a815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47a7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f47a806d0c0 0x7f47a815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41552/0 (socket says 172.21.15.73:41552) 2024-04-05T22:31:19.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.739+0000 7f47a7400640 1 -- 172.21.15.73:0/3161362500 learned_addr learned my addr 172.21.15.73:0/3161362500 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a6a00640 1 --2- 172.21.15.73:0/3161362500 >> v2:172.21.15.157:3300/0 conn(0x7f47a81587d0 0x7f47a80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f479fe00640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a80ccd20 0x7f47a806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a7400640 1 -- 172.21.15.73:0/3161362500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a80ccd20 msgr2=0x7f47a806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a7400640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a80ccd20 0x7f47a806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a7400640 1 -- 172.21.15.73:0/3161362500 >> v2:172.21.15.157:3300/0 conn(0x7f47a81587d0 msgr2=0x7f47a80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a7400640 1 --2- 172.21.15.73:0/3161362500 >> v2:172.21.15.157:3300/0 conn(0x7f47a81587d0 0x7f47a80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.740+0000 7f47a7400640 1 -- 172.21.15.73:0/3161362500 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a806f800 con 0x7f47a806d0c0 2024-04-05T22:31:19.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.741+0000 7f47a7400640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a815e850 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f47a80667b0 tx=0x7f47a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db39d15cf4dc8d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:19.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.741+0000 7f47a6000640 1 -- 172.21.15.73:0/3161362500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a002ca70 con 0x7f47a806d0c0 2024-04-05T22:31:19.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47a6000640 1 -- 172.21.15.73:0/3161362500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a002cbd0 con 0x7f47a806d0c0 2024-04-05T22:31:19.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47a6000640 1 -- 172.21.15.73:0/3161362500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00343f0 con 0x7f47a806d0c0 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 -- 172.21.15.73:0/3161362500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 msgr2=0x7f47a815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a815e850 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f47a80667b0 tx=0x7f47a002bed0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 -- 172.21.15.73:0/3161362500 shutdown_connections 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a815e850 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 --2- 172.21.15.73:0/3161362500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a80ccd20 0x7f47a806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 --2- 172.21.15.73:0/3161362500 >> v2:172.21.15.157:3300/0 conn(0x7f47a81587d0 0x7f47a80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.742+0000 7f47ad600640 1 -- 172.21.15.73:0/3161362500 >> 172.21.15.73:0/3161362500 conn(0x7f47a80ce8e0 msgr2=0x7f47a80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:19.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.743+0000 7f47ad600640 1 -- 172.21.15.73:0/3161362500 shutdown_connections 2024-04-05T22:31:19.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.743+0000 7f47ad600640 1 -- 172.21.15.73:0/3161362500 wait complete. 2024-04-05T22:31:19.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.744+0000 7f47ad600640 1 Processor -- start 2024-04-05T22:31:19.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.745+0000 7f47ad600640 1 -- start start 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.745+0000 7f47ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.745+0000 7f47ad600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a81587d0 0x7f47a81f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47ad600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a81f60f0 0x7f47a815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47ad600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81f25c0 con 0x7f47a806d0c0 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47ad600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81f6570 con 0x7f47a81587d0 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47ad600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80c41b0 con 0x7f47a81f60f0 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41562/0 (socket says 172.21.15.73:41562) 2024-04-05T22:31:19.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 -- 172.21.15.73:0/2310987012 learned_addr learned my addr 172.21.15.73:0/2310987012 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 -- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a81f60f0 msgr2=0x7f47a815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a81f60f0 0x7f47a815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 -- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a81587d0 msgr2=0x7f47a81f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a81587d0 0x7f47a81f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 -- 172.21.15.73:0/2310987012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0009b30 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.746+0000 7f47a6a00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f479800d5f0 tx=0x7f479800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.747+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980125f0 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.747+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798012750 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.747+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798012ae0 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.747+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a814e2c0 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.747+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a80c3880 con 0x7f47a806d0c0 2024-04-05T22:31:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.748+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8066af0 con 0x7f47a806d0c0 2024-04-05T22:31:19.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.749+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f4798012c40 con 0x7f47a806d0c0 2024-04-05T22:31:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.750+0000 7f479ea00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 0x7f478406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.750+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47980882b0 con 0x7f47a806d0c0 2024-04-05T22:31:19.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.750+0000 7f479fe00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 0x7f478406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:19.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.751+0000 7f479fe00640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 0x7f478406da50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4790006fd0 tx=0x7f4790008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:19.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.755+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4798054d40 con 0x7f47a806d0c0 2024-04-05T22:31:19.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.951+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47a8068160 con 0x7f47a806d0c0 2024-04-05T22:31:19.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.953+0000 7f479ea00640 1 -- 172.21.15.73:0/2310987012 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f4798058360 con 0x7f47a806d0c0 2024-04-05T22:31:19.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:19.954 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:19.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 msgr2=0x7f478406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 0x7f478406da50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4790006fd0 tx=0x7f4790008040 comp rx=0 tx=0).stop 2024-04-05T22:31:19.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 msgr2=0x7f47a814dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f479800d5f0 tx=0x7f479800dab0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 shutdown_connections 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f478406b590 0x7f478406da50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a81f60f0 0x7f47a815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a81587d0 0x7f47a81f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 --2- 172.21.15.73:0/2310987012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806d0c0 0x7f47a814dd80 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:19.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 >> 172.21.15.73:0/2310987012 conn(0x7f47a80ce8e0 msgr2=0x7f47a815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:19.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 shutdown_connections 2024-04-05T22:31:19.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:19.956+0000 7f47ad600640 1 -- 172.21.15.73:0/2310987012 wait complete. 2024-04-05T22:31:19.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:57.382314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:57.918101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:19.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.191+0000 7fdf20c00640 1 Processor -- start 2024-04-05T22:31:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.191+0000 7fdf20c00640 1 -- start start 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf20c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 0x7fdf1c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf20c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf1c15dbc0 0x7fdf1c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf20c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c15ae50 con 0x7fdf1c15aa70 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf20c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c15e3b0 con 0x7fdf1c15dbc0 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf20c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0ac9e0 con 0x7fdf1c071670 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf19e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdf1c15aa70 0x7fdf1c070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39110/0 (socket says 172.21.15.73:39110) 2024-04-05T22:31:20.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.193+0000 7fdf19e00640 1 -- 172.21.15.73:0/1240982009 learned_addr learned my addr 172.21.15.73:0/1240982009 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf1a800640 1 --2- 172.21.15.73:0/1240982009 >> v2:172.21.15.157:3300/0 conn(0x7fdf1c15dbc0 0x7fdf1c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf19400640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 0x7fdf1c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921935599 0 0) 0x7fdf1c15ae50 con 0x7fdf1c15aa70 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf00003520 con 0x7fdf1c15aa70 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649994921 0 0) 0x7fdf00003520 con 0x7fdf1c15aa70 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 msgr2=0x7fdf1c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 0x7fdf1c15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 >> v2:172.21.15.157:3300/0 conn(0x7fdf1c15dbc0 msgr2=0x7fdf1c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 --2- 172.21.15.73:0/1240982009 >> v2:172.21.15.157:3300/0 conn(0x7fdf1c15dbc0 0x7fdf1c06ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0accc0 con 0x7fdf1c15aa70 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fdf0c001670 con 0x7fdf1c15aa70 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 >> v1:172.21.15.73:6789/0 conn(0x7fdf1c15aa70 legacy=0x7fdf1c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 0x7fdf00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00003ef0 0x7fdf000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c15e3b0 con 0x7fdf00004d00 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0ac9e0 con 0x7fdf00003ef0 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.194+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0000b980 con 0x7fdf1c071670 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf19400640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00003ef0 0x7fdf000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 -- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 msgr2=0x7fdf00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c071670 0x7fdf00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 -- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00003ef0 msgr2=0x7fdf000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00003ef0 0x7fdf000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.195+0000 7fdf1a800640 1 -- 172.21.15.73:0/1240982009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf00003520 con 0x7fdf00004d00 2024-04-05T22:31:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.196+0000 7fdf1a800640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf00005180 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009fc0 tx=0x7fdf000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86295f7873d2977e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.196+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1402cc40 con 0x7fdf00004d00 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1400f040 con 0x7fdf00004d00 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf18a00640 1 -- 172.21.15.73:0/1240982009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14034440 con 0x7fdf00004d00 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 -- 172.21.15.73:0/1240982009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 msgr2=0x7fdf00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 --2- 172.21.15.73:0/1240982009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf00005180 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009fc0 tx=0x7fdf000057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf1a800640 1 -- 172.21.15.73:0/1240982009 reap_dead start 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 -- 172.21.15.73:0/1240982009 shutdown_connections 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 -- 172.21.15.73:0/1240982009 >> 172.21.15.73:0/1240982009 conn(0x7fdf1c069cb0 msgr2=0x7fdf1c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 -- 172.21.15.73:0/1240982009 shutdown_connections 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.197+0000 7fdf20c00640 1 -- 172.21.15.73:0/1240982009 wait complete. 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.199+0000 7fdf20c00640 1 Processor -- start 2024-04-05T22:31:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.199+0000 7fdf20c00640 1 -- start start 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 0x7fdf1c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c15f070 0x7fdf1c15f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c15fa60 con 0x7fdf00004d00 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c143a40 con 0x7fdf1c1f5ae0 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf20c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0cb680 con 0x7fdf1c15f070 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41588/0 (socket says 172.21.15.73:41588) 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.200+0000 7fdf19e00640 1 -- 172.21.15.73:0/2513007085 learned_addr learned my addr 172.21.15.73:0/2513007085 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19400640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 0x7fdf1c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 -- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c15f070 msgr2=0x7fdf1c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c15f070 0x7fdf1c15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 -- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 msgr2=0x7fdf1c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 0x7fdf1c1431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 -- 172.21.15.73:0/2513007085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c146a20 con 0x7fdf00004d00 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19400640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 0x7fdf1c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf19e00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c840 tx=0x7fdf0c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0074e0 con 0x7fdf00004d00 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c007640 con 0x7fdf00004d00 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c004a00 con 0x7fdf00004d00 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c143730 con 0x7fdf00004d00 2024-04-05T22:31:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.201+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c1f6280 con 0x7fdf00004d00 2024-04-05T22:31:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.202+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c143bf0 con 0x7fdf00004d00 2024-04-05T22:31:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.204+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fdf0c004b60 con 0x7fdf00004d00 2024-04-05T22:31:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.204+0000 7fdf12000640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 0x7fdefc06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.205+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf0c0873e0 con 0x7fdf00004d00 2024-04-05T22:31:20.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.205+0000 7fdf19400640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 0x7fdefc06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.206+0000 7fdf19400640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 0x7fdefc06da50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c1f5a40 tx=0x7fdf08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.209+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdf0c053e10 con 0x7fdf00004d00 2024-04-05T22:31:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.402+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf1c068160 con 0x7fdf00004d00 2024-04-05T22:31:20.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.403+0000 7fdf12000640 1 -- 172.21.15.73:0/2513007085 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf0c018070 con 0x7fdf00004d00 2024-04-05T22:31:20.403 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:20.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 msgr2=0x7fdefc06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 0x7fdefc06da50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c1f5a40 tx=0x7fdf08008040 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 msgr2=0x7fdf1c1464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c840 tx=0x7fdf0c00cd00 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 shutdown_connections 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdefc06b590 0x7fdefc06da50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf1c15f070 0x7fdf1c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf1c1f5ae0 0x7fdf1c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 --2- 172.21.15.73:0/2513007085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf00004d00 0x7fdf1c1464e0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 >> 172.21.15.73:0/2513007085 conn(0x7fdf1c069cb0 msgr2=0x7fdf1c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 shutdown_connections 2024-04-05T22:31:20.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.406+0000 7fdf20c00640 1 -- 172.21.15.73:0/2513007085 wait complete. 2024-04-05T22:31:20.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:31:20.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.638+0000 7f9b95400640 1 Processor -- start 2024-04-05T22:31:20.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.638+0000 7f9b95400640 1 -- start start 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b95400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b9015aa70 0x7f9b90070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b95400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b9015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b95400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b90071900 con 0x7f9b9015dbc0 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b95400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9015ae50 con 0x7f9b9015aa70 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b95400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9015e3b0 con 0x7f9b900714a0 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b8f200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9b9015dbc0 0x7f9b901609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39126/0 (socket says 172.21.15.73:39126) 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.640+0000 7f9b8f200640 1 -- 172.21.15.73:0/3179568278 learned_addr learned my addr 172.21.15.73:0/3179568278 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3179568278 >> v2:172.21.15.157:3300/0 conn(0x7f9b9015aa70 0x7f9b90070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8de00640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b9015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650701154 0 0) 0x7f9b90071900 con 0x7f9b9015dbc0 2024-04-05T22:31:20.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b74003520 con 0x7f9b9015dbc0 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2815789154 0 0) 0x7f9b74003520 con 0x7f9b9015dbc0 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 msgr2=0x7f9b9015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b9015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 >> v2:172.21.15.157:3300/0 conn(0x7f9b9015aa70 msgr2=0x7f9b90070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 --2- 172.21.15.73:0/3179568278 >> v2:172.21.15.157:3300/0 conn(0x7f9b9015aa70 0x7f9b90070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9006ca50 con 0x7f9b9015dbc0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9b880019f0 con 0x7f9b9015dbc0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 >> v1:172.21.15.73:6789/0 conn(0x7f9b9015dbc0 legacy=0x7f9b901609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b74003970 0x7f9b74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74004d00 0x7f9b74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9015ae50 con 0x7f9b74003970 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9015e3b0 con 0x7f9b74004d00 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.641+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7400b980 con 0x7f9b900714a0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b74003970 0x7f9b74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8f200640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74004d00 0x7f9b74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8de00640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 -- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 msgr2=0x7f9b740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900714a0 0x7f9b740047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 -- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74004d00 msgr2=0x7f9b74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74004d00 0x7f9b74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8e800640 1 -- 172.21.15.73:0/3179568278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b74003520 con 0x7f9b74003970 2024-04-05T22:31:20.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.642+0000 7f9b8f200640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74004d00 0x7f9b74005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:20.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.643+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b74003970 0x7f9b74003d90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009af0 tx=0x7f9b740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f3d74a53c793 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.643+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8002c710 con 0x7f9b74003970 2024-04-05T22:31:20.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.643+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8002c870 con 0x7f9b74003970 2024-04-05T22:31:20.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.643+0000 7f9b8d400640 1 -- 172.21.15.73:0/3179568278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8002cc00 con 0x7f9b74003970 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 -- 172.21.15.73:0/3179568278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b74003970 msgr2=0x7f9b74003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 --2- 172.21.15.73:0/3179568278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b74003970 0x7f9b74003d90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009af0 tx=0x7f9b740057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b8f200640 1 -- 172.21.15.73:0/3179568278 reap_dead start 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 -- 172.21.15.73:0/3179568278 shutdown_connections 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 -- 172.21.15.73:0/3179568278 >> 172.21.15.73:0/3179568278 conn(0x7f9b90069cb0 msgr2=0x7f9b9006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 -- 172.21.15.73:0/3179568278 shutdown_connections 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.644+0000 7f9b95400640 1 -- 172.21.15.73:0/3179568278 wait complete. 2024-04-05T22:31:20.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.646+0000 7f9b95400640 1 Processor -- start 2024-04-05T22:31:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.646+0000 7f9b95400640 1 -- start start 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74003970 0x7f9b90158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900aec30 0x7f9b90153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b90158130 con 0x7f9b900af0b0 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b90158a90 con 0x7f9b74003970 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.647+0000 7f9b95400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b901574b0 con 0x7f9b900aec30 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41608/0 (socket says 172.21.15.73:41608) 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 -- 172.21.15.73:0/3924823665 learned_addr learned my addr 172.21.15.73:0/3924823665 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74003970 0x7f9b90158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8de00640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900aec30 0x7f9b90153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 -- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900aec30 msgr2=0x7f9b90153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900aec30 0x7f9b90153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 -- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74003970 msgr2=0x7f9b90158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74003970 0x7f9b90158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.648+0000 7f9b8f200640 1 -- 172.21.15.73:0/3924823665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90157790 con 0x7f9b900af0b0 2024-04-05T22:31:20.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.649+0000 7f9b8f200640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f9b9015dfa0 tx=0x7f9b8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.651+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800bb80 con 0x7f9b900af0b0 2024-04-05T22:31:20.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.651+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800bce0 con 0x7f9b900af0b0 2024-04-05T22:31:20.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.651+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88007350 con 0x7f9b900af0b0 2024-04-05T22:31:20.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.651+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b90157a70 con 0x7f9b900af0b0 2024-04-05T22:31:20.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.651+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b90157100 con 0x7f9b900af0b0 2024-04-05T22:31:20.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.652+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b90158270 con 0x7f9b900af0b0 2024-04-05T22:31:20.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.653+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f9b880074b0 con 0x7f9b900af0b0 2024-04-05T22:31:20.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.655+0000 7f9b86a00640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 0x7f9b6c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:20.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.655+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b880882f0 con 0x7f9b900af0b0 2024-04-05T22:31:20.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.655+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 0x7f9b6c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:20.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.656+0000 7f9b8e800640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 0x7f9b6c06da50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b80030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:20.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.660+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9b90158270 con 0x7f9b900af0b0 2024-04-05T22:31:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.854+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b90158270 con 0x7f9b900af0b0 2024-04-05T22:31:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.855+0000 7f9b86a00640 1 -- 172.21.15.73:0/3924823665 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b90158270 con 0x7f9b900af0b0 2024-04-05T22:31:20.855 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 msgr2=0x7f9b6c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 0x7f9b6c06da50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b80030000 comp rx=0 tx=0).stop 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 msgr2=0x7f9b90152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f9b9015dfa0 tx=0x7f9b8800b4a0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 shutdown_connections 2024-04-05T22:31:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9b6c06b590 0x7f9b6c06da50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b900af0b0 0x7f9b90152000 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b900aec30 0x7f9b90153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 --2- 172.21.15.73:0/3924823665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b74003970 0x7f9b90158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 >> 172.21.15.73:0/3924823665 conn(0x7f9b90069cb0 msgr2=0x7f9b901607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 shutdown_connections 2024-04-05T22:31:20.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:20.858+0000 7f9b95400640 1 -- 172.21.15.73:0/3924823665 wait complete. 2024-04-05T22:31:20.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:31:21.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.086+0000 7f0102600640 1 Processor -- start 2024-04-05T22:31:21.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.086+0000 7f0102600640 1 -- start start 2024-04-05T22:31:21.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.087+0000 7f0102600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00fc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.087+0000 7f0102600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0b1b30 con 0x7f00fc0b32b0 2024-04-05T22:31:21.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.087+0000 7f0102600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0b1e10 con 0x7f00fc0cbe00 2024-04-05T22:31:21.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.087+0000 7f0102600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ce7b0 con 0x7f00fc06c630 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00fbe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f00fc0b32b0 0x7f00fc1647c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39142/0 (socket says 172.21.15.73:39142) 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00fbe00640 1 -- 172.21.15.73:0/1023443392 learned_addr learned my addr 172.21.15.73:0/1023443392 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00fc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961529878 0 0) 0x7f00fc0b1b30 con 0x7f00fc0b32b0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e0003520 con 0x7f00fc0b32b0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361802813 0 0) 0x7f00e0003520 con 0x7f00fc0b32b0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 msgr2=0x7f00fc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00fc06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 >> v1:172.21.15.157:6789/0 conn(0x7f00fc0cbe00 legacy=0x7f00fc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc06cf50 con 0x7f00fc0b32b0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f00f4001670 con 0x7f00fc0b32b0 2024-04-05T22:31:21.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00fc06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.088+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 >> v1:172.21.15.73:6789/0 conn(0x7f00fc0b32b0 legacy=0x7f00fc1647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e00039a0 0x7f00e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ce7b0 con 0x7f00e00039a0 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e000ba90 con 0x7f00e0004d00 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e000bc50 con 0x7f00fc06c630 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e00039a0 0x7f00e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fbe00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00f2a00640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 -- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 msgr2=0x7f00e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:21.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc06c630 0x7f00e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 -- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e0004d00 msgr2=0x7f00e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:21.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.089+0000 7f00fb400640 1 -- 172.21.15.73:0/1023443392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0003520 con 0x7f00e00039a0 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.091+0000 7f00fb400640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e00039a0 0x7f00e0003dc0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006fd0 tx=0x7f00e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcda5f97160d9b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.091+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec004230 con 0x7f00e00039a0 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.091+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec004390 con 0x7f00e00039a0 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.091+0000 7f00faa00640 1 -- 172.21.15.73:0/1023443392 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec003830 con 0x7f00e00039a0 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 -- 172.21.15.73:0/1023443392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e00039a0 msgr2=0x7f00e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 --2- 172.21.15.73:0/1023443392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e00039a0 0x7f00e0003dc0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006fd0 tx=0x7f00e00057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f00fbe00640 1 -- 172.21.15.73:0/1023443392 reap_dead start 2024-04-05T22:31:21.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 -- 172.21.15.73:0/1023443392 shutdown_connections 2024-04-05T22:31:21.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 -- 172.21.15.73:0/1023443392 >> 172.21.15.73:0/1023443392 conn(0x7f00fc0b0830 msgr2=0x7f00fc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:21.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 -- 172.21.15.73:0/1023443392 shutdown_connections 2024-04-05T22:31:21.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.092+0000 7f0102600640 1 -- 172.21.15.73:0/1023443392 wait complete. 2024-04-05T22:31:21.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.094+0000 7f0102600640 1 Processor -- start 2024-04-05T22:31:21.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.094+0000 7f0102600640 1 -- start start 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.094+0000 7f0102600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e00039a0 0x7f00fc12f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f0102600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc134fc0 0x7f00fc0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f0102600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f0102600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0cb310 con 0x7f00fc070d60 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f0102600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0717e0 con 0x7f00e00039a0 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f0102600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc135440 con 0x7f00fc134fc0 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41630/0 (socket says 172.21.15.73:41630) 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 -- 172.21.15.73:0/916773142 learned_addr learned my addr 172.21.15.73:0/916773142 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fb400640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e00039a0 0x7f00fc12f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00f2a00640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc134fc0 0x7f00fc0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 -- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc134fc0 msgr2=0x7f00fc0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc134fc0 0x7f00fc0cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 -- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e00039a0 msgr2=0x7f00fc12f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e00039a0 0x7f00fc12f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.095+0000 7f00fbe00640 1 -- 172.21.15.73:0/916773142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc071500 con 0x7f00fc070d60 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f00fbe00640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b3740 tx=0x7f00f400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4010510 con 0x7f00fc070d60 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4010670 con 0x7f00fc070d60 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4010a20 con 0x7f00fc070d60 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc0aca00 con 0x7f00fc070d60 2024-04-05T22:31:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.096+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0acfc0 con 0x7f00fc070d60 2024-04-05T22:31:21.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.097+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0ad1e0 con 0x7f00fc070d60 2024-04-05T22:31:21.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.098+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f00f4010b80 con 0x7f00fc070d60 2024-04-05T22:31:21.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.099+0000 7f00f3e00640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 0x7f00d806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.099+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00f40874a0 con 0x7f00fc070d60 2024-04-05T22:31:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.100+0000 7f00fb400640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 0x7f00d806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.101+0000 7f00fb400640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 0x7f00d806da50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f00ec035000 tx=0x7f00ec031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.104+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00f4053ff0 con 0x7f00fc070d60 2024-04-05T22:31:21.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.296+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00fc068160 con 0x7f00fc070d60 2024-04-05T22:31:21.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.297+0000 7f00f3e00640 1 -- 172.21.15.73:0/916773142 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00f4019070 con 0x7f00fc070d60 2024-04-05T22:31:21.297 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 msgr2=0x7f00d806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 0x7f00d806da50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f00ec035000 tx=0x7f00ec031000 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 msgr2=0x7f00fc1310c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b3740 tx=0x7f00f400e960 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 shutdown_connections 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f00d806b590 0x7f00d806da50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc070d60 0x7f00fc1310c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00fc134fc0 0x7f00fc0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 --2- 172.21.15.73:0/916773142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00e00039a0 0x7f00fc12f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.299+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 >> 172.21.15.73:0/916773142 conn(0x7f00fc0b0830 msgr2=0x7f00fc0b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.300+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 shutdown_connections 2024-04-05T22:31:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.300+0000 7f0102600640 1 -- 172.21.15.73:0/916773142 wait complete. 2024-04-05T22:31:21.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:31:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.530+0000 7f10d7c00640 1 Processor -- start 2024-04-05T22:31:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.531+0000 7f10d7c00640 1 -- start start 2024-04-05T22:31:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.532+0000 7f10d7c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d015cf70 0x7f10d00d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.532+0000 7f10d7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 0x7f10d00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.532+0000 7f10d7c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d00ceee0 con 0x7f10d00d0bd0 2024-04-05T22:31:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.532+0000 7f10d7c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d00ce5b0 con 0x7f10d015cf70 2024-04-05T22:31:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.532+0000 7f10d7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00ce890 con 0x7f10d00d1930 2024-04-05T22:31:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.533+0000 7f10d5200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d015cf70 0x7f10d00d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.533+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 0x7f10d00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.533+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 0x7f10d00b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36380/0 (socket says 172.21.15.73:36380) 2024-04-05T22:31:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.533+0000 7f10d5c00640 1 -- 172.21.15.73:0/1323237636 learned_addr learned my addr 172.21.15.73:0/1323237636 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.533+0000 7f10d5200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d015cf70 0x7f10d00d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:49682/0 (socket says 172.21.15.73:49682) 2024-04-05T22:31:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.534+0000 7f10d5200640 1 -- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 msgr2=0x7f10d00b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.534+0000 7f10d5200640 1 --2- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 0x7f10d00b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.534+0000 7f10d5200640 1 -- 172.21.15.73:0/1323237636 >> v1:172.21.15.157:6789/0 conn(0x7f10d00d0bd0 legacy=0x7f10d00d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.534+0000 7f10d5200640 1 -- 172.21.15.73:0/1323237636 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d00ceb70 con 0x7f10d015cf70 2024-04-05T22:31:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.535+0000 7f10d5200640 1 --2- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d015cf70 0x7f10d00d3910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c16b7c09e98e0e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10cf400640 1 -- 172.21.15.73:0/1323237636 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0031780 con 0x7f10d015cf70 2024-04-05T22:31:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10cf400640 1 -- 172.21.15.73:0/1323237636 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c00318e0 con 0x7f10d015cf70 2024-04-05T22:31:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10cf400640 1 -- 172.21.15.73:0/1323237636 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0031c70 con 0x7f10d015cf70 2024-04-05T22:31:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 -- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d015cf70 msgr2=0x7f10d00d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d015cf70 0x7f10d00d3910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002b820 comp rx=0 tx=0).stop 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 -- 172.21.15.73:0/1323237636 shutdown_connections 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d00d1930 0x7f10d00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1323237636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d015cf70 0x7f10d00d3910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 -- 172.21.15.73:0/1323237636 >> 172.21.15.73:0/1323237636 conn(0x7f10d00d2df0 msgr2=0x7f10d00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 -- 172.21.15.73:0/1323237636 shutdown_connections 2024-04-05T22:31:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.536+0000 7f10d7c00640 1 -- 172.21.15.73:0/1323237636 wait complete. 2024-04-05T22:31:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.538+0000 7f10d7c00640 1 Processor -- start 2024-04-05T22:31:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.538+0000 7f10d7c00640 1 -- start start 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d00d0bd0 0x7f10d0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 0x7f10d00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00cb5d0 con 0x7f10d00d1930 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00caa00 con 0x7f10d00d0bd0 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10d7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00caca0 con 0x7f10d015cf70 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41636/0 (socket says 172.21.15.73:41636) 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.539+0000 7f10cfe00640 1 -- 172.21.15.73:0/1625091435 learned_addr learned my addr 172.21.15.73:0/1625091435 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10d5200640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d00d0bd0 0x7f10d0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10d5c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 0x7f10d00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 -- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 msgr2=0x7f10d00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 0x7f10d00b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 -- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d00d0bd0 msgr2=0x7f10d0151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d00d0bd0 0x7f10d0151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 -- 172.21.15.73:0/1625091435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0009400 con 0x7f10d00d1930 2024-04-05T22:31:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10d5c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 0x7f10d00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10cfe00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c870 tx=0x7f10b800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.540+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8003b70 con 0x7f10d00d1930 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.541+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b8003cf0 con 0x7f10d00d1930 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.541+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b800aa90 con 0x7f10d00d1930 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.541+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d00caf80 con 0x7f10d00d1930 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.541+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00cc8a0 con 0x7f10d00d1930 2024-04-05T22:31:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.542+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d00cb740 con 0x7f10d00d1930 2024-04-05T22:31:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.543+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f10b800abf0 con 0x7f10d00d1930 2024-04-05T22:31:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.543+0000 7f10ccc00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 0x7f10ac06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.544+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10b808a6a0 con 0x7f10d00d1930 2024-04-05T22:31:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.544+0000 7f10d5200640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 0x7f10ac06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.545+0000 7f10d5200640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 0x7f10ac06da80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f10c0036000 tx=0x7f10c003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:21.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.548+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f10b8054720 con 0x7f10d00d1930 2024-04-05T22:31:21.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.755+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10d0068160 con 0x7f10d00d1930 2024-04-05T22:31:21.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.756+0000 7f10ccc00640 1 -- 172.21.15.73:0/1625091435 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v160) v1 ==== 80+0+0 (secure 0 0 0) 0x7f10b8003e60 con 0x7f10d00d1930 2024-04-05T22:31:21.756 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 msgr2=0x7f10ac06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 0x7f10ac06da80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f10c0036000 tx=0x7f10c003d040 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 msgr2=0x7f10d00cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c870 tx=0x7f10b800cd30 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 shutdown_connections 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f10ac06b5c0 0x7f10ac06da80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d015cf70 0x7f10d00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d00d1930 0x7f10d00cc360 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.758+0000 7f10d7c00640 1 --2- 172.21.15.73:0/1625091435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d00d0bd0 0x7f10d0151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:21.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.759+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 >> 172.21.15.73:0/1625091435 conn(0x7f10d00d2df0 msgr2=0x7f10d00d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:21.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.759+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 shutdown_connections 2024-04-05T22:31:21.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:21.759+0000 7f10d7c00640 1 -- 172.21.15.73:0/1625091435 wait complete. 2024-04-05T22:31:21.776 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:31:21.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:21.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:21.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:31:21.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:21.892 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:31:21.937 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:21.937 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:21.976 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:21.976 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:22.062 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:22.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.260+0000 7f7b0a400640 1 Processor -- start 2024-04-05T22:31:22.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.260+0000 7f7b0a400640 1 -- start start 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.261+0000 7f7b0a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7b040b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.261+0000 7f7b0a400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0415aa60 con 0x7f7b040ce000 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.261+0000 7f7b0a400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040b3b60 con 0x7f7b0406d710 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.261+0000 7f7b0a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040701c0 con 0x7f7b041541a0 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.262+0000 7f7b03e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7b040ce000 0x7f7b0415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39148/0 (socket says 172.21.15.73:39148) 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.262+0000 7f7b03e00640 1 -- 172.21.15.73:0/3226676656 learned_addr learned my addr 172.21.15.73:0/3226676656 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.262+0000 7f7b03400640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7b040b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.262+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195671243 0 0) 0x7f7b0415aa60 con 0x7f7b040ce000 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.262+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003520 con 0x7f7b040ce000 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191066233 0 0) 0x7f7ae8003520 con 0x7f7b040ce000 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 msgr2=0x7f7b040b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7b040b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 >> v1:172.21.15.157:6789/0 conn(0x7f7b0406d710 legacy=0x7f7b040cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040b3880 con 0x7f7b040ce000 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7afc0019f0 con 0x7f7b040ce000 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 >> v1:172.21.15.73:6789/0 conn(0x7f7b040ce000 legacy=0x7f7b0415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7ae80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003f20 0x7f7ae80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040701c0 con 0x7f7ae8004d00 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800ba60 con 0x7f7ae8003f20 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800de30 con 0x7f7b041541a0 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03400640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7ae80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7afaa00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003f20 0x7f7ae80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 -- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 msgr2=0x7f7ae80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7ae80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 -- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003f20 msgr2=0x7f7ae80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae8003f20 0x7f7ae80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.263+0000 7f7b03e00640 1 -- 172.21.15.73:0/3226676656 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8003520 con 0x7f7ae8004d00 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.264+0000 7f7b03400640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b041541a0 0x7f7ae80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b03e00640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a830 tx=0x7f7ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f43c607b7edb809 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0072d0 con 0x7f7ae8004d00 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc02f070 con 0x7f7ae8004d00 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b02a00640 1 -- 172.21.15.73:0/3226676656 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc007690 con 0x7f7ae8004d00 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b0a400640 1 -- 172.21.15.73:0/3226676656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 msgr2=0x7f7ae8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b0a400640 1 --2- 172.21.15.73:0/3226676656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a830 tx=0x7f7ae80057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b03e00640 1 -- 172.21.15.73:0/3226676656 reap_dead start 2024-04-05T22:31:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b0a400640 1 -- 172.21.15.73:0/3226676656 shutdown_connections 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b0a400640 1 -- 172.21.15.73:0/3226676656 >> 172.21.15.73:0/3226676656 conn(0x7f7b040ce8e0 msgr2=0x7f7b040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.265+0000 7f7b0a400640 1 -- 172.21.15.73:0/3226676656 shutdown_connections 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.266+0000 7f7b0a400640 1 -- 172.21.15.73:0/3226676656 wait complete. 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.267+0000 7f7b0a400640 1 Processor -- start 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.267+0000 7f7b0a400640 1 -- start start 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b040b2a40 0x7f7b040b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b040a8680 0x7f7b040a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040b3400 con 0x7f7ae8004d00 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040a93d0 con 0x7f7b040b2a40 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b0a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04145670 con 0x7f7b040a8680 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41654/0 (socket says 172.21.15.73:41654) 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.268+0000 7f7b03400640 1 -- 172.21.15.73:0/400263555 learned_addr learned my addr 172.21.15.73:0/400263555 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 -- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b040a8680 msgr2=0x7f7b040a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b040a8680 0x7f7b040a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 -- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b040b2a40 msgr2=0x7f7b040b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b040b2a40 0x7f7b040b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 -- 172.21.15.73:0/400263555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04145950 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b03400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7af40108b0 tx=0x7f7af4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400eae0 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af400ec40 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af401c400 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.269+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b04145c30 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.270+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b04145230 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.270+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04066af0 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.271+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f7af401c560 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.272+0000 7f7afbe00640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 0x7f7ae006da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.272+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af4088380 con 0x7f7ae8004d00 2024-04-05T22:31:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.273+0000 7f7afaa00640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 0x7f7ae006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.274+0000 7f7afaa00640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 0x7f7ae006da80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7b040ac360 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.276+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7af4054e60 con 0x7f7ae8004d00 2024-04-05T22:31:22.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.473+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b04068160 con 0x7f7ae8004d00 2024-04-05T22:31:22.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.475+0000 7f7afbe00640 1 -- 172.21.15.73:0/400263555 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f7af4058480 con 0x7f7ae8004d00 2024-04-05T22:31:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:22.475 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:22.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.477+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 msgr2=0x7f7ae006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.477+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 0x7f7ae006da80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7b040ac360 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.477+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 msgr2=0x7f7b04144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.477+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7af40108b0 tx=0x7f7af4010d70 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 shutdown_connections 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ae006b5c0 0x7f7ae006da80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b040a8680 0x7f7b040a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b040b2a40 0x7f7b040b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 --2- 172.21.15.73:0/400263555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae8004d00 0x7f7b04144cf0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 >> 172.21.15.73:0/400263555 conn(0x7f7b040ce8e0 msgr2=0x7f7b0406f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 shutdown_connections 2024-04-05T22:31:22.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.478+0000 7f7b0a400640 1 -- 172.21.15.73:0/400263555 wait complete. 2024-04-05T22:31:22.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:57.382314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:57.918101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:22.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:22.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.694+0000 7ff7c8a00640 1 Processor -- start 2024-04-05T22:31:22.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.695+0000 7ff7c8a00640 1 -- start start 2024-04-05T22:31:22.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.696+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 0x7ff7c4070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.696+0000 7ff7c8a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7c406cd50 0x7ff7c4159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.696+0000 7ff7c8a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c40b27f0 con 0x7ff7c40add00 2024-04-05T22:31:22.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.696+0000 7ff7c8a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c40b3890 con 0x7ff7c406cd50 2024-04-05T22:31:22.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.696+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b2ad0 con 0x7ff7c41599f0 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c1400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff7c40add00 0x7ff7c406c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39160/0 (socket says 172.21.15.73:39160) 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c1400640 1 -- 172.21.15.73:0/336616814 learned_addr learned my addr 172.21.15.73:0/336616814 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 0x7ff7c4070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c2800640 1 --2- 172.21.15.73:0/336616814 >> v2:172.21.15.157:3300/0 conn(0x7ff7c406cd50 0x7ff7c4159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871845031 0 0) 0x7ff7c40b27f0 con 0x7ff7c40add00 2024-04-05T22:31:22.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.697+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a4005730 con 0x7ff7c40add00 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 240383389 0 0) 0x7ff7a4005730 con 0x7ff7c40add00 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 msgr2=0x7ff7c4070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 0x7ff7c4070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 >> v2:172.21.15.157:3300/0 conn(0x7ff7c406cd50 msgr2=0x7ff7c4159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 --2- 172.21.15.73:0/336616814 >> v2:172.21.15.157:3300/0 conn(0x7ff7c406cd50 0x7ff7c4159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40b2db0 con 0x7ff7c40add00 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7ff7ac0019f0 con 0x7ff7c40add00 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 >> v1:172.21.15.73:6789/0 conn(0x7ff7c40add00 legacy=0x7ff7c406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7a4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 0x7ff7a40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7a4006f10 0x7ff7a4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b3890 con 0x7ff7a4005b80 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b2ad0 con 0x7ff7a4006f10 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a400db90 con 0x7ff7c41599f0 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7a4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 -- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 msgr2=0x7ff7a40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c41599f0 0x7ff7a40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 -- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7a4006f10 msgr2=0x7ff7a4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7a4006f10 0x7ff7a4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.698+0000 7ff7c1e00640 1 -- 172.21.15.73:0/336616814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4005730 con 0x7ff7a4005b80 2024-04-05T22:31:22.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.699+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7a4005fa0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009ac0 tx=0x7ff7a4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f1399930380563 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402c710 con 0x7ff7a4005b80 2024-04-05T22:31:22.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b402c870 con 0x7ff7a4005b80 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c0a00640 1 -- 172.21.15.73:0/336616814 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402cc00 con 0x7ff7a4005b80 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 -- 172.21.15.73:0/336616814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 msgr2=0x7ff7a4005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/336616814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7a4005fa0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009ac0 tx=0x7ff7a4007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c2800640 1 -- 172.21.15.73:0/336616814 reap_dead start 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 -- 172.21.15.73:0/336616814 shutdown_connections 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 -- 172.21.15.73:0/336616814 >> 172.21.15.73:0/336616814 conn(0x7ff7c4069cb0 msgr2=0x7ff7c406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 -- 172.21.15.73:0/336616814 shutdown_connections 2024-04-05T22:31:22.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.700+0000 7ff7c8a00640 1 -- 172.21.15.73:0/336616814 wait complete. 2024-04-05T22:31:22.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.702+0000 7ff7c8a00640 1 Processor -- start 2024-04-05T22:31:22.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.702+0000 7ff7c8a00640 1 -- start start 2024-04-05T22:31:22.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41427c0 0x7ff7c4140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c4156600 0x7ff7c4140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4142330 con 0x7ff7a4005b80 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4141720 con 0x7ff7c41427c0 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4141a00 con 0x7ff7c4156600 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.703+0000 7ff7c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41672/0 (socket says 172.21.15.73:41672) 2024-04-05T22:31:22.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 -- 172.21.15.73:0/3527633495 learned_addr learned my addr 172.21.15.73:0/3527633495 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 -- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c4156600 msgr2=0x7ff7c4140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c4156600 0x7ff7c4140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 -- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41427c0 msgr2=0x7ff7c4140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41427c0 0x7ff7c4140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 -- 172.21.15.73:0/3527633495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4141ce0 con 0x7ff7a4005b80 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.704+0000 7ff7c1e00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009ac0 tx=0x7ff7b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.705+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003810 con 0x7ff7a4005b80 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.705+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b402de90 con 0x7ff7a4005b80 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.705+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c4141fc0 con 0x7ff7a4005b80 2024-04-05T22:31:22.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.705+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003bb0 con 0x7ff7a4005b80 2024-04-05T22:31:22.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.705+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c4142be0 con 0x7ff7a4005b80 2024-04-05T22:31:22.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.706+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4068160 con 0x7ff7a4005b80 2024-04-05T22:31:22.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.707+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7ff7b4003d10 con 0x7ff7a4005b80 2024-04-05T22:31:22.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.708+0000 7ff7ba000640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 0x7ff7a006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:22.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.708+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7b40a9e90 con 0x7ff7a4005b80 2024-04-05T22:31:22.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.708+0000 7ff7c1400640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 0x7ff7a006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:22.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.709+0000 7ff7c1400640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 0x7ff7a006da50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac001370 tx=0x7ff7ac0017c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:22.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.712+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff7b4076830 con 0x7ff7a4005b80 2024-04-05T22:31:22.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.905+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7c40b2310 con 0x7ff7a4005b80 2024-04-05T22:31:22.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.906+0000 7ff7ba000640 1 -- 172.21.15.73:0/3527633495 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7b4079e50 con 0x7ff7a4005b80 2024-04-05T22:31:22.906 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:22.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 msgr2=0x7ff7a006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 0x7ff7a006da50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac001370 tx=0x7ff7ac0017c0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 msgr2=0x7ff7c40a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009ac0 tx=0x7ff7b4004730 comp rx=0 tx=0).stop 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 shutdown_connections 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff7a006b590 0x7ff7a006da50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c4156600 0x7ff7c4140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c41427c0 0x7ff7c4140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 --2- 172.21.15.73:0/3527633495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7a4005b80 0x7ff7c40a91e0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:22.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 >> 172.21.15.73:0/3527633495 conn(0x7ff7c4069cb0 msgr2=0x7ff7c4070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:22.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.908+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 shutdown_connections 2024-04-05T22:31:22.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:22.909+0000 7ff7c8a00640 1 -- 172.21.15.73:0/3527633495 wait complete. 2024-04-05T22:31:22.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:31:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.130+0000 7f3c2b200640 1 Processor -- start 2024-04-05T22:31:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.130+0000 7f3c2b200640 1 -- start start 2024-04-05T22:31:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.132+0000 7f3c2b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c2406cd50 0x7f3c24070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.132+0000 7f3c2b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c240b27f0 con 0x7f3c240add00 2024-04-05T22:31:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.132+0000 7f3c2b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c240b2ad0 con 0x7f3c241599f0 2024-04-05T22:31:23.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.132+0000 7f3c2b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c240b3890 con 0x7f3c2406cd50 2024-04-05T22:31:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.132+0000 7f3c29200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3c240add00 0x7f3c24159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39176/0 (socket says 172.21.15.73:39176) 2024-04-05T22:31:23.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c29200640 1 -- 172.21.15.73:0/2141091758 learned_addr learned my addr 172.21.15.73:0/2141091758 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:23.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c23400640 1 --2- 172.21.15.73:0/2141091758 >> v2:172.21.15.157:3300/0 conn(0x7f3c2406cd50 0x7f3c24070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947059382 0 0) 0x7f3c240b27f0 con 0x7f3c240add00 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c08003520 con 0x7f3c240add00 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795395267 0 0) 0x7f3c08003520 con 0x7f3c240add00 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 >> v2:172.21.15.157:3300/0 conn(0x7f3c2406cd50 msgr2=0x7f3c24070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 --2- 172.21.15.73:0/2141091758 >> v2:172.21.15.157:3300/0 conn(0x7f3c2406cd50 0x7f3c24070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 >> v1:172.21.15.157:6789/0 conn(0x7f3c241599f0 legacy=0x7f3c2406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240b2db0 con 0x7f3c240add00 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.133+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f3c1c0019f0 con 0x7f3c240add00 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 >> v1:172.21.15.73:6789/0 conn(0x7f3c240add00 legacy=0x7f3c24159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08003970 0x7f3c08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c080047c0 0x7f3c08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 0x7f3c0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240b3890 con 0x7f3c08005180 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0800c020 con 0x7f3c080047c0 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0800c1e0 con 0x7f3c08003970 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 0x7f3c0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c23e00640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08003970 0x7f3c08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 -- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08003970 msgr2=0x7f3c08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c23400640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c080047c0 0x7f3c08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08003970 0x7f3c08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 -- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c080047c0 msgr2=0x7f3c08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c080047c0 0x7f3c08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c29200640 1 -- 172.21.15.73:0/2141091758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08003520 con 0x7f3c08005180 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c23e00640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08003970 0x7f3c08003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.134+0000 7f3c23400640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c080047c0 0x7f3c08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:23.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.135+0000 7f3c29200640 1 -- 172.21.15.73:0/2141091758 reap_dead start 2024-04-05T22:31:23.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.135+0000 7f3c29200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 0x7f3c0800b980 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a830 tx=0x7f3c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c32bbdd7fa645 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0072d0 con 0x7f3c08005180 2024-04-05T22:31:23.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c02f070 con 0x7f3c08005180 2024-04-05T22:31:23.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c22a00640 1 -- 172.21.15.73:0/2141091758 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0076b0 con 0x7f3c08005180 2024-04-05T22:31:23.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 -- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 msgr2=0x7f3c0800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 0x7f3c0800b980 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a830 tx=0x7f3c080057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 -- 172.21.15.73:0/2141091758 shutdown_connections 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 --2- 172.21.15.73:0/2141091758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c08005180 0x7f3c0800b980 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 -- 172.21.15.73:0/2141091758 >> 172.21.15.73:0/2141091758 conn(0x7f3c24069cb0 msgr2=0x7f3c2406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 -- 172.21.15.73:0/2141091758 shutdown_connections 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.136+0000 7f3c2b200640 1 -- 172.21.15.73:0/2141091758 wait complete. 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.138+0000 7f3c2b200640 1 Processor -- start 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.138+0000 7f3c2b200640 1 -- start start 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 0x7f3c2414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c241f1f50 0x7f3c241f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2414cba0 con 0x7f3c241f60f0 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241f6a90 con 0x7f3c241f1f50 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c2b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2414a0f0 con 0x7f3c08005180 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.139+0000 7f3c23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41704/0 (socket says 172.21.15.73:41704) 2024-04-05T22:31:23.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 -- 172.21.15.73:0/707467369 learned_addr learned my addr 172.21.15.73:0/707467369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23e00640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 0x7f3c2414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 -- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 msgr2=0x7f3c2414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 0x7f3c2414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 -- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c241f1f50 msgr2=0x7f3c241f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c241f1f50 0x7f3c241f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 -- 172.21.15.73:0/707467369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c241f2940 con 0x7f3c241f60f0 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23e00640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 0x7f3c2414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c23400640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0108b0 tx=0x7f3c0c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00eb60 con 0x7f3c241f60f0 2024-04-05T22:31:23.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c00ecc0 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01c400 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.140+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2414bf20 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.141+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2414c4e0 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.142+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c24068160 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.143+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f3c0c01c560 con 0x7f3c241f60f0 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.143+0000 7f3c1be00640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 0x7f3c0006da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.144+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c0c088460 con 0x7f3c241f60f0 2024-04-05T22:31:23.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.144+0000 7f3c23e00640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 0x7f3c0006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.145+0000 7f3c23e00640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 0x7f3c0006da80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3c14007fd0 tx=0x7f3c14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.147+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3c0c054f40 con 0x7f3c241f60f0 2024-04-05T22:31:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.334+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c240b00a0 con 0x7f3c241f60f0 2024-04-05T22:31:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.334+0000 7f3c1be00640 1 -- 172.21.15.73:0/707467369 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c0c058560 con 0x7f3c241f60f0 2024-04-05T22:31:23.335 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:31:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 msgr2=0x7f3c0006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 0x7f3c0006da80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3c14007fd0 tx=0x7f3c14009040 comp rx=0 tx=0).stop 2024-04-05T22:31:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 msgr2=0x7f3c241f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0108b0 tx=0x7f3c0c010d70 comp rx=0 tx=0).stop 2024-04-05T22:31:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 shutdown_connections 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f3c0006b5c0 0x7f3c0006da80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c241f1f50 0x7f3c241f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c241f60f0 0x7f3c241f6550 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 --2- 172.21.15.73:0/707467369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c08005180 0x7f3c2414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 >> 172.21.15.73:0/707467369 conn(0x7f3c24069cb0 msgr2=0x7f3c241553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 shutdown_connections 2024-04-05T22:31:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.337+0000 7f3c2b200640 1 -- 172.21.15.73:0/707467369 wait complete. 2024-04-05T22:31:23.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:31:23.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.562+0000 7efe53800640 1 Processor -- start 2024-04-05T22:31:23.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.562+0000 7efe53800640 1 -- start start 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe53800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c143660 0x7efe4c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe53800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe4c06cd40 0x7efe4c071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe53800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c071b80 con 0x7efe4c070660 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe53800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c0cb8c0 con 0x7efe4c06cd40 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe53800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c070e70 con 0x7efe4c143660 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe51800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe4c06cd40 0x7efe4c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe51800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe4c06cd40 0x7efe4c071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41718/0 (socket says 172.21.15.73:41718) 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe4be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe4c070660 0x7efe4c06c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39180/0 (socket says 172.21.15.73:39180) 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.564+0000 7efe51800640 1 -- 172.21.15.73:0/146689931 learned_addr learned my addr 172.21.15.73:0/146689931 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:23.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe50e00640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c143660 0x7efe4c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe51800640 1 -- 172.21.15.73:0/146689931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c143660 msgr2=0x7efe4c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:23.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe51800640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c143660 0x7efe4c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe51800640 1 -- 172.21.15.73:0/146689931 >> v1:172.21.15.73:6789/0 conn(0x7efe4c070660 legacy=0x7efe4c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe51800640 1 -- 172.21.15.73:0/146689931 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0cceb0 con 0x7efe4c06cd40 2024-04-05T22:31:23.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.565+0000 7efe4b400640 1 -- 172.21.15.73:0/146689931 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202955850 0 0) 0x7efe4c071b80 con 0x7efe4c070660 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.566+0000 7efe51800640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c06cd40 0x7efe4c071640 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e50 tx=0x7efe4402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef7995fc7102fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.566+0000 7efe4b400640 1 -- 172.21.15.73:0/146689931 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4402ca70 con 0x7efe4c06cd40 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.566+0000 7efe4b400640 1 -- 172.21.15.73:0/146689931 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4402cbd0 con 0x7efe4c06cd40 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.566+0000 7efe4b400640 1 -- 172.21.15.73:0/146689931 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe440343f0 con 0x7efe4c06cd40 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 -- 172.21.15.73:0/146689931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c06cd40 msgr2=0x7efe4c071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c06cd40 0x7efe4c071640 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e50 tx=0x7efe4402bed0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 -- 172.21.15.73:0/146689931 shutdown_connections 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c06cd40 0x7efe4c071640 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 --2- 172.21.15.73:0/146689931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c143660 0x7efe4c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 -- 172.21.15.73:0/146689931 >> 172.21.15.73:0/146689931 conn(0x7efe4c0b0830 msgr2=0x7efe4c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 -- 172.21.15.73:0/146689931 shutdown_connections 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.567+0000 7efe53800640 1 -- 172.21.15.73:0/146689931 wait complete. 2024-04-05T22:31:23.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.569+0000 7efe53800640 1 Processor -- start 2024-04-05T22:31:23.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.569+0000 7efe53800640 1 -- start start 2024-04-05T22:31:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 0x7efe4c14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 0x7efe4c1387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c138d20 con 0x7efe4c143660 2024-04-05T22:31:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c148ef0 con 0x7efe4c06cd40 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe53800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c149190 con 0x7efe4c070660 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe51800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe51800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41728/0 (socket says 172.21.15.73:41728) 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe51800640 1 -- 172.21.15.73:0/2224746437 learned_addr learned my addr 172.21.15.73:0/2224746437 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe50e00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 0x7efe4c14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.570+0000 7efe4be00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 0x7efe4c1387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 -- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 msgr2=0x7efe4c1387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 0x7efe4c1387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 -- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 msgr2=0x7efe4c14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 0x7efe4c14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 -- 172.21.15.73:0/2224746437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44009b30 con 0x7efe4c143660 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe50e00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 0x7efe4c14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe4be00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 0x7efe4c1387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe51800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e50 tx=0x7efe44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44009510 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4403e450 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4403e800 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c149470 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.571+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c149a30 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.572+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c068160 con 0x7efe4c143660 2024-04-05T22:31:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.573+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7efe4403e960 con 0x7efe4c143660 2024-04-05T22:31:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.574+0000 7efe48c00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 0x7efe2c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.574+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe440aabf0 con 0x7efe4c143660 2024-04-05T22:31:23.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.575+0000 7efe50e00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 0x7efe2c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:23.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.576+0000 7efe50e00640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 0x7efe2c06da50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe3c006fd0 tx=0x7efe3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.578+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7efe44077620 con 0x7efe4c143660 2024-04-05T22:31:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.774+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe4c06d620 con 0x7efe4c143660 2024-04-05T22:31:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.775+0000 7efe48c00640 1 -- 172.21.15.73:0/2224746437 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe4407ac40 con 0x7efe4c143660 2024-04-05T22:31:23.776 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:31:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.777+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 msgr2=0x7efe2c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.777+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 0x7efe2c06da50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe3c006fd0 tx=0x7efe3c008040 comp rx=0 tx=0).stop 2024-04-05T22:31:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.777+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 msgr2=0x7efe4c14eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.777+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7efe44009e50 tx=0x7efe44004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 shutdown_connections 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efe2c06b590 0x7efe2c06da50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe4c143660 0x7efe4c14eb30 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe4c070660 0x7efe4c1387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 --2- 172.21.15.73:0/2224746437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4c06cd40 0x7efe4c14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 >> 172.21.15.73:0/2224746437 conn(0x7efe4c0b0830 msgr2=0x7efe4c140530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 shutdown_connections 2024-04-05T22:31:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:23.778+0000 7efe53800640 1 -- 172.21.15.73:0/2224746437 wait complete. 2024-04-05T22:31:23.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:31:24.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.006+0000 7ffa48800640 1 Processor -- start 2024-04-05T22:31:24.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.006+0000 7ffa48800640 1 -- start start 2024-04-05T22:31:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa48800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa48800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa400b1870 0x7ffa400ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa48800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400ce5d0 con 0x7ffa400ad6b0 2024-04-05T22:31:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa48800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa400ce010 con 0x7ffa400b1870 2024-04-05T22:31:24.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa48800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400ce2f0 con 0x7ffa40160620 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:36498/0 (socket says 172.21.15.73:36498) 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa45400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa400b1870 0x7ffa400ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.008+0000 7ffa45e00640 1 -- 172.21.15.73:0/723183803 learned_addr learned my addr 172.21.15.73:0/723183803 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.009+0000 7ffa45400640 1 -- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 msgr2=0x7ffa400caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:24.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.009+0000 7ffa45400640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.009+0000 7ffa45400640 1 -- 172.21.15.73:0/723183803 >> v1:172.21.15.157:6789/0 conn(0x7ffa400ad6b0 legacy=0x7ffa400cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.009+0000 7ffa45400640 1 -- 172.21.15.73:0/723183803 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400ac970 con 0x7ffa400b1870 2024-04-05T22:31:24.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.009+0000 7ffa45e00640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:31:24.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.011+0000 7ffa45400640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa400ad170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097a0 tx=0x7ffa3402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c15532267b70ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.011+0000 7ffa44a00640 1 -- 172.21.15.73:0/723183803 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3402d040 con 0x7ffa400b1870 2024-04-05T22:31:24.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.011+0000 7ffa44a00640 1 -- 172.21.15.73:0/723183803 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa34031810 con 0x7ffa400b1870 2024-04-05T22:31:24.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.011+0000 7ffa44a00640 1 -- 172.21.15.73:0/723183803 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34031ba0 con 0x7ffa400b1870 2024-04-05T22:31:24.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 -- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 msgr2=0x7ffa400ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa400ad170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097a0 tx=0x7ffa3402b8b0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 -- 172.21.15.73:0/723183803 shutdown_connections 2024-04-05T22:31:24.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa400ad170 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 --2- 172.21.15.73:0/723183803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa400caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 -- 172.21.15.73:0/723183803 >> 172.21.15.73:0/723183803 conn(0x7ffa400ce8e0 msgr2=0x7ffa400cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:24.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 -- 172.21.15.73:0/723183803 shutdown_connections 2024-04-05T22:31:24.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.012+0000 7ffa48800640 1 -- 172.21.15.73:0/723183803 wait complete. 2024-04-05T22:31:24.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.014+0000 7ffa48800640 1 Processor -- start 2024-04-05T22:31:24.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.014+0000 7ffa48800640 1 -- start start 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa4006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa4006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4006a880 con 0x7ffa400ad6b0 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40069cb0 con 0x7ffa400b1870 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa48800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40069f50 con 0x7ffa40160620 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41744/0 (socket says 172.21.15.73:41744) 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.015+0000 7ffa45e00640 1 -- 172.21.15.73:0/1490391939 learned_addr learned my addr 172.21.15.73:0/1490391939 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45400640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa4006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa46800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa4006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 -- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 msgr2=0x7ffa4006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa4006fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 -- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 msgr2=0x7ffa4006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa4006f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 -- 172.21.15.73:0/1490391939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0094d0 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa45e00640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e9b0 tx=0x7ffa2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00c9a0 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c00cb00 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00ce70 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.016+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa34009400 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.017+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4006a510 con 0x7ffa400ad6b0 2024-04-05T22:31:24.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.018+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa40066af0 con 0x7ffa400ad6b0 2024-04-05T22:31:24.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.019+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7ffa2c013470 con 0x7ffa400ad6b0 2024-04-05T22:31:24.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.019+0000 7ffa3a000640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 0x7ffa2006da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.020+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa2c0883d0 con 0x7ffa400ad6b0 2024-04-05T22:31:24.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.020+0000 7ffa45400640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 0x7ffa2006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.021+0000 7ffa45400640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 0x7ffa2006da80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036000 tx=0x7ffa3403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.024+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ffa2c054e00 con 0x7ffa400ad6b0 2024-04-05T22:31:24.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.214+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa40068160 con 0x7ffa400ad6b0 2024-04-05T22:31:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.215+0000 7ffa3a000640 1 -- 172.21.15.73:0/1490391939 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v160) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa2c017080 con 0x7ffa400ad6b0 2024-04-05T22:31:24.215 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:31:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.217+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 msgr2=0x7ffa2006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.217+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 0x7ffa2006da80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036000 tx=0x7ffa3403c040 comp rx=0 tx=0).stop 2024-04-05T22:31:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.217+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 msgr2=0x7ffa400ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.217+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e9b0 tx=0x7ffa2c00ee70 comp rx=0 tx=0).stop 2024-04-05T22:31:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.217+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 shutdown_connections 2024-04-05T22:31:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa2006b5c0 0x7ffa2006da80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa40160620 0x7ffa4006fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa400b1870 0x7ffa4006f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 --2- 172.21.15.73:0/1490391939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa400ad6b0 0x7ffa400ca450 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 >> 172.21.15.73:0/1490391939 conn(0x7ffa400ce8e0 msgr2=0x7ffa400cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 shutdown_connections 2024-04-05T22:31:24.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.218+0000 7ffa48800640 1 -- 172.21.15.73:0/1490391939 wait complete. 2024-04-05T22:31:24.235 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:31:24.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:24.236 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:24.350 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:31:24.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:24.351 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:31:24.400 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:24.400 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:24.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:31:24.453 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:24.453 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:24.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:31:24.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:24.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.726+0000 7fe3a7600640 1 Processor -- start 2024-04-05T22:31:24.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.726+0000 7fe3a7600640 1 -- start start 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.727+0000 7fe3a7600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3a01587d0 0x7fe3a00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe3a7600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a006f5b0 con 0x7fe3a00ccd20 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe3a7600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a0070410 con 0x7fe3a015e850 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe3a7600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0158bb0 con 0x7fe3a01587d0 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe397e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe3a00ccd20 0x7fe3a006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39194/0 (socket says 172.21.15.73:39194) 2024-04-05T22:31:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe397e00640 1 -- 172.21.15.73:0/3929453655 learned_addr learned my addr 172.21.15.73:0/3929453655 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3929453655 >> v2:172.21.15.157:3300/0 conn(0x7fe3a01587d0 0x7fe3a00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784864903 0 0) 0x7fe3a006f5b0 con 0x7fe3a00ccd20 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.728+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe380003520 con 0x7fe3a00ccd20 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134466212 0 0) 0x7fe380003520 con 0x7fe3a00ccd20 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 >> v2:172.21.15.157:3300/0 conn(0x7fe3a01587d0 msgr2=0x7fe3a00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 --2- 172.21.15.73:0/3929453655 >> v2:172.21.15.157:3300/0 conn(0x7fe3a01587d0 0x7fe3a00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 >> v1:172.21.15.157:6789/0 conn(0x7fe3a015e850 legacy=0x7fe3a00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a006f800 con 0x7fe3a00ccd20 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe390001670 con 0x7fe3a00ccd20 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 >> v1:172.21.15.73:6789/0 conn(0x7fe3a00ccd20 legacy=0x7fe3a006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe380003970 0x7fe380003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 0x7fe380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe380005180 0x7fe38000fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0158bb0 con 0x7fe3800047c0 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380010430 con 0x7fe380005180 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800105f0 con 0x7fe380003970 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe397e00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 0x7fe380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.729+0000 7fe3a5600640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe380005180 0x7fe38000fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe380003970 0x7fe380003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe397e00640 1 -- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe380003970 msgr2=0x7fe380003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe397e00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe380003970 0x7fe380003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe397e00640 1 -- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe380005180 msgr2=0x7fe38000fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe397e00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe380005180 0x7fe38000fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe397e00640 1 -- 172.21.15.73:0/3929453655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380003520 con 0x7fe3800047c0 2024-04-05T22:31:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.730+0000 7fe3a5600640 1 -- 172.21.15.73:0/3929453655 reap_dead start 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe397e00640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 0x7fe380004c40 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c190 tx=0x7fe3800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df59138624c0edf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390005ad0 con 0x7fe3800047c0 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390005c30 con 0x7fe3800047c0 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe39fe00640 1 -- 172.21.15.73:0/3929453655 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39002cae0 con 0x7fe3800047c0 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe3a7600640 1 -- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 msgr2=0x7fe380004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.731+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 0x7fe380004c40 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c190 tx=0x7fe3800057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.732+0000 7fe3a7600640 1 -- 172.21.15.73:0/3929453655 shutdown_connections 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.732+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3929453655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3800047c0 0x7fe380004c40 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.732+0000 7fe3a7600640 1 -- 172.21.15.73:0/3929453655 >> 172.21.15.73:0/3929453655 conn(0x7fe3a00ce8e0 msgr2=0x7fe3a00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:24.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.732+0000 7fe3a7600640 1 -- 172.21.15.73:0/3929453655 shutdown_connections 2024-04-05T22:31:24.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.732+0000 7fe3a7600640 1 -- 172.21.15.73:0/3929453655 wait complete. 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.733+0000 7fe3a7600640 1 Processor -- start 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 -- start start 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 0x7fe3a015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a013d5f0 0x7fe3a015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0159ca0 con 0x7fe3a015a9e0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0159090 con 0x7fe3800047c0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.734+0000 7fe3a7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0159370 con 0x7fe3a013d5f0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41756/0 (socket says 172.21.15.73:41756) 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 -- 172.21.15.73:0/3227869801 learned_addr learned my addr 172.21.15.73:0/3227869801 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 0x7fe3a015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 -- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a013d5f0 msgr2=0x7fe3a015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe3a5600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a013d5f0 0x7fe3a015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a013d5f0 0x7fe3a015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 -- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 msgr2=0x7fe3a015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 0x7fe3a015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe397e00640 1 -- 172.21.15.73:0/3227869801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0159650 con 0x7fe3a015a9e0 2024-04-05T22:31:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.735+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 0x7fe3a015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:24.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe397e00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fe390001930 tx=0x7fe39002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39003c040 con 0x7fe3a015a9e0 2024-04-05T22:31:24.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39002ccb0 con 0x7fe3a015a9e0 2024-04-05T22:31:24.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a0159930 con 0x7fe3a015a9e0 2024-04-05T22:31:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390007290 con 0x7fe3a015a9e0 2024-04-05T22:31:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.736+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0158cf0 con 0x7fe3a015a9e0 2024-04-05T22:31:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.737+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0066af0 con 0x7fe3a015a9e0 2024-04-05T22:31:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.738+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fe390032070 con 0x7fe3a015a9e0 2024-04-05T22:31:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.739+0000 7fe39d600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 0x7fe37c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.739+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3900a9230 con 0x7fe3a015a9e0 2024-04-05T22:31:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.740+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 0x7fe37c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:24.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.741+0000 7fe3a4c00640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 0x7fe37c06da80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0142010 tx=0x7fe388008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:24.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.743+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe390075cc0 con 0x7fe3a015a9e0 2024-04-05T22:31:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.975+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3a013dd10 con 0x7fe3a015a9e0 2024-04-05T22:31:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.976+0000 7fe39d600640 1 -- 172.21.15.73:0/3227869801 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4746 (secure 0 0 0) 0x7fe3a013dd10 con 0x7fe3a015a9e0 2024-04-05T22:31:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:24.976 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:24.977 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 101 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.978+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 msgr2=0x7fe37c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.978+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 0x7fe37c06da80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0142010 tx=0x7fe388008040 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.978+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 msgr2=0x7fe3a0142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.978+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fe390001930 tx=0x7fe39002e070 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 shutdown_connections 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe37c06b5c0 0x7fe37c06da80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a013d5f0 0x7fe3a015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3a015a9e0 0x7fe3a0142220 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 --2- 172.21.15.73:0/3227869801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3800047c0 0x7fe3a015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 >> 172.21.15.73:0/3227869801 conn(0x7fe3a00ce8e0 msgr2=0x7fe3a00ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 shutdown_connections 2024-04-05T22:31:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:24.979+0000 7fe3a7600640 1 -- 172.21.15.73:0/3227869801 wait complete. 2024-04-05T22:31:24.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:25.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.199+0000 7f17e8400640 1 Processor -- start 2024-04-05T22:31:25.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.200+0000 7f17e8400640 1 -- start start 2024-04-05T22:31:25.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.201+0000 7f17e8400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17e006d710 0x7f17e00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.201+0000 7f17e8400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e015aa60 con 0x7f17e01541a0 2024-04-05T22:31:25.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.201+0000 7f17e8400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e00b3b60 con 0x7f17e015aef0 2024-04-05T22:31:25.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.201+0000 7f17e8400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00701c0 con 0x7f17e006d710 2024-04-05T22:31:25.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17e5a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f17e01541a0 0x7f17e00b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39204/0 (socket says 172.21.15.73:39204) 2024-04-05T22:31:25.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17e5a00640 1 -- 172.21.15.73:0/952366624 learned_addr learned my addr 172.21.15.73:0/952366624 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17e5000640 1 --2- 172.21.15.73:0/952366624 >> v2:172.21.15.157:3300/0 conn(0x7f17e006d710 0x7f17e00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086923444 0 0) 0x7f17e015aa60 con 0x7f17e01541a0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c8005730 con 0x7f17e01541a0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959708421 0 0) 0x7f17c8005730 con 0x7f17e01541a0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 >> v2:172.21.15.157:3300/0 conn(0x7f17e006d710 msgr2=0x7f17e00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 --2- 172.21.15.73:0/952366624 >> v2:172.21.15.157:3300/0 conn(0x7f17e006d710 0x7f17e00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 >> v1:172.21.15.157:6789/0 conn(0x7f17e015aef0 legacy=0x7f17e015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.202+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00b3880 con 0x7f17e01541a0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f17d4001670 con 0x7f17e01541a0 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 >> v1:172.21.15.73:6789/0 conn(0x7f17e01541a0 legacy=0x7f17e00b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17c8005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c80069d0 0x7f17c8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17c8007390 0x7f17c800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00701c0 con 0x7f17c8005b80 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c800e210 con 0x7f17c80069d0 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c800e3d0 con 0x7f17c8007390 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17c8005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 -- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17c8007390 msgr2=0x7f17c800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17c8007390 0x7f17c800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 -- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c80069d0 msgr2=0x7f17c8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c80069d0 0x7f17c8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e5a00640 1 -- 172.21.15.73:0/952366624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8005730 con 0x7f17c8005b80 2024-04-05T22:31:25.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.203+0000 7f17e6400640 1 -- 172.21.15.73:0/952366624 reap_dead start 2024-04-05T22:31:25.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.204+0000 7f17e5a00640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17c8005f80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c240 tx=0x7f17c8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871bc26ec2871592 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4005ad0 con 0x7f17c8005b80 2024-04-05T22:31:25.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d4005c30 con 0x7f17c8005b80 2024-04-05T22:31:25.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17dfe00640 1 -- 172.21.15.73:0/952366624 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d402c8d0 con 0x7f17c8005b80 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 -- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 msgr2=0x7f17c8005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17c8005f80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c240 tx=0x7f17c8007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 -- 172.21.15.73:0/952366624 shutdown_connections 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 --2- 172.21.15.73:0/952366624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17c8005f80 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 -- 172.21.15.73:0/952366624 >> 172.21.15.73:0/952366624 conn(0x7f17e00ce8e0 msgr2=0x7f17e00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 -- 172.21.15.73:0/952366624 shutdown_connections 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.205+0000 7f17e8400640 1 -- 172.21.15.73:0/952366624 wait complete. 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.207+0000 7f17e8400640 1 Processor -- start 2024-04-05T22:31:25.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.207+0000 7f17e8400640 1 -- start start 2024-04-05T22:31:25.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17e0151e80 0x7f17e01522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 0x7f17e00aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0149980 con 0x7f17c8005b80 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0152820 con 0x7f17e00a9d30 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.208+0000 7f17e8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00a8e90 con 0x7f17e0151e80 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41782/0 (socket says 172.21.15.73:41782) 2024-04-05T22:31:25.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 -- 172.21.15.73:0/1043846307 learned_addr learned my addr 172.21.15.73:0/1043846307 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:25.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e6400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 0x7f17e00aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5000640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17e0151e80 0x7f17e01522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 -- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17e0151e80 msgr2=0x7f17e01522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17e0151e80 0x7f17e01522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 -- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 msgr2=0x7f17e00aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 0x7f17e00aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 -- 172.21.15.73:0/1043846307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00aaa80 con 0x7f17c8005b80 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e6400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 0x7f17e00aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.209+0000 7f17e5a00640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c240 tx=0x7f17d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.210+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d403c040 con 0x7f17c8005b80 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.210+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d402cda0 con 0x7f17c8005b80 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.210+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4007290 con 0x7f17c8005b80 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.210+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e00a8560 con 0x7f17c8005b80 2024-04-05T22:31:25.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.210+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e00a8b20 con 0x7f17c8005b80 2024-04-05T22:31:25.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.211+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0066af0 con 0x7f17c8005b80 2024-04-05T22:31:25.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.212+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f17d4032070 con 0x7f17c8005b80 2024-04-05T22:31:25.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.213+0000 7f17dd600640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 0x7f17c006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.213+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17d40a9240 con 0x7f17c8005b80 2024-04-05T22:31:25.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.214+0000 7f17e5000640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 0x7f17c006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.215+0000 7f17e5000640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 0x7f17c006da50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f17e01529d0 tx=0x7f17cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.218+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f17d4075cd0 con 0x7f17c8005b80 2024-04-05T22:31:25.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.416+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17e00a8cb0 con 0x7f17c8005b80 2024-04-05T22:31:25.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.419+0000 7f17dd600640 1 -- 172.21.15.73:0/1043846307 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f17d40abe60 con 0x7f17c8005b80 2024-04-05T22:31:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:25.419 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 msgr2=0x7f17c006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 0x7f17c006da50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f17e01529d0 tx=0x7f17cc008040 comp rx=0 tx=0).stop 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 msgr2=0x7f17e0149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c240 tx=0x7f17d402e070 comp rx=0 tx=0).stop 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 shutdown_connections 2024-04-05T22:31:25.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f17c006b590 0x7f17c006da50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17e00a9d30 0x7f17e00aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17e0151e80 0x7f17e01522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 --2- 172.21.15.73:0/1043846307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17c8005b80 0x7f17e0149440 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 >> 172.21.15.73:0/1043846307 conn(0x7f17e00ce8e0 msgr2=0x7f17e0159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 shutdown_connections 2024-04-05T22:31:25.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.422+0000 7f17e8400640 1 -- 172.21.15.73:0/1043846307 wait complete. 2024-04-05T22:31:25.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:30:57.382314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:30:57.918101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:25.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:25.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.652+0000 7fb4e0e00640 1 Processor -- start 2024-04-05T22:31:25.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.652+0000 7fb4e0e00640 1 -- start start 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4e0e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4dc06c9a0 0x7fb4dc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4e0e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc06bea0 con 0x7fb4dc15abe0 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4e0e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc15afc0 con 0x7fb4dc06ba70 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4e0e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc06f5b0 con 0x7fb4dc06c9a0 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4d9e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb4dc15abe0 0x7fb4dc0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39208/0 (socket says 172.21.15.73:39208) 2024-04-05T22:31:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.654+0000 7fb4d9e00640 1 -- 172.21.15.73:0/3897199952 learned_addr learned my addr 172.21.15.73:0/3897199952 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d9400640 1 --2- 172.21.15.73:0/3897199952 >> v2:172.21.15.157:3300/0 conn(0x7fb4dc06c9a0 0x7fb4dc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098296986 0 0) 0x7fb4dc06bea0 con 0x7fb4dc15abe0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c0003520 con 0x7fb4dc15abe0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3287272227 0 0) 0x7fb4c0003520 con 0x7fb4dc15abe0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 >> v2:172.21.15.157:3300/0 conn(0x7fb4dc06c9a0 msgr2=0x7fb4dc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 --2- 172.21.15.73:0/3897199952 >> v2:172.21.15.157:3300/0 conn(0x7fb4dc06c9a0 0x7fb4dc06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 >> v1:172.21.15.157:6789/0 conn(0x7fb4dc06ba70 legacy=0x7fb4dc164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc06f890 con 0x7fb4dc15abe0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb4cc001690 con 0x7fb4dc15abe0 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 >> v1:172.21.15.73:6789/0 conn(0x7fb4dc15abe0 legacy=0x7fb4dc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0003970 0x7fb4c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00052a0 0x7fb4c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc06f5b0 con 0x7fb4c00047c0 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c000c170 con 0x7fb4c0003970 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.655+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c000c330 con 0x7fb4c00052a0 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9e00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0003970 0x7fb4c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 -- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00052a0 msgr2=0x7fb4c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00052a0 0x7fb4c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 -- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0003970 msgr2=0x7fb4c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0003970 0x7fb4c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4d9400640 1 -- 172.21.15.73:0/3897199952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0003520 con 0x7fb4c00047c0 2024-04-05T22:31:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.656+0000 7fb4da800640 1 -- 172.21.15.73:0/3897199952 reap_dead start 2024-04-05T22:31:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.657+0000 7fb4d9400640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4c0004c40 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009a90 tx=0x7fb4c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8e26a7bce141b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.657+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c802c680 con 0x7fb4c00047c0 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.657+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c802c7e0 con 0x7fb4c00047c0 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.657+0000 7fb4d8a00640 1 -- 172.21.15.73:0/3897199952 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c802cb70 con 0x7fb4c00047c0 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 -- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 msgr2=0x7fb4c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4c0004c40 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009a90 tx=0x7fb4c00057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 -- 172.21.15.73:0/3897199952 shutdown_connections 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/3897199952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4c0004c40 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 -- 172.21.15.73:0/3897199952 >> 172.21.15.73:0/3897199952 conn(0x7fb4dc069cb0 msgr2=0x7fb4dc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 -- 172.21.15.73:0/3897199952 shutdown_connections 2024-04-05T22:31:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.658+0000 7fb4e0e00640 1 -- 172.21.15.73:0/3897199952 wait complete. 2024-04-05T22:31:25.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.660+0000 7fb4e0e00640 1 Processor -- start 2024-04-05T22:31:25.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.660+0000 7fb4e0e00640 1 -- start start 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.661+0000 7fb4e0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.661+0000 7fb4e0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4dc13a480 0x7fb4dc13c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.661+0000 7fb4e0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4dc13ffa0 0x7fb4dc155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4e0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc13a240 con 0x7fb4c00047c0 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4e0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc13d500 con 0x7fb4dc13a480 2024-04-05T22:31:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4e0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc13ca50 con 0x7fb4dc13ffa0 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41790/0 (socket says 172.21.15.73:41790) 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 -- 172.21.15.73:0/215158058 learned_addr learned my addr 172.21.15.73:0/215158058 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 -- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4dc13ffa0 msgr2=0x7fb4dc155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4dc13ffa0 0x7fb4dc155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 -- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4dc13a480 msgr2=0x7fb4dc13c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4dc13a480 0x7fb4dc13c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 -- 172.21.15.73:0/215158058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc14d730 con 0x7fb4c00047c0 2024-04-05T22:31:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.662+0000 7fb4d9e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e7d0 tx=0x7fb4cc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.663+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00eee0 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.663+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc00c8d0 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.663+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00cc80 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.663+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc13cc60 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.663+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc13d220 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.664+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc068160 con 0x7fb4c00047c0 2024-04-05T22:31:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.665+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fb4cc01d460 con 0x7fb4c00047c0 2024-04-05T22:31:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.666+0000 7fb4d2000640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 0x7fb4bc06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.666+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4cc087e40 con 0x7fb4c00047c0 2024-04-05T22:31:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.667+0000 7fb4d9400640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 0x7fb4bc06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:25.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.668+0000 7fb4d9400640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 0x7fb4bc06da50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8036000 tx=0x7fb4c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:25.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.671+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb4cc058360 con 0x7fb4c00047c0 2024-04-05T22:31:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.866+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4dc0714d0 con 0x7fb4c00047c0 2024-04-05T22:31:25.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.867+0000 7fb4d2000640 1 -- 172.21.15.73:0/215158058 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4cc057ef0 con 0x7fb4c00047c0 2024-04-05T22:31:25.867 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:25.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.869+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 msgr2=0x7fb4bc06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.869+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 0x7fb4bc06da50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8036000 tx=0x7fb4c802e000 comp rx=0 tx=0).stop 2024-04-05T22:31:25.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.869+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 msgr2=0x7fb4dc157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:25.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.869+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e7d0 tx=0x7fb4cc00ec90 comp rx=0 tx=0).stop 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 shutdown_connections 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb4bc06b590 0x7fb4bc06da50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4dc13ffa0 0x7fb4dc155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4dc13a480 0x7fb4dc13c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 --2- 172.21.15.73:0/215158058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4c00047c0 0x7fb4dc157b50 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 >> 172.21.15.73:0/215158058 conn(0x7fb4dc069cb0 msgr2=0x7fb4dc15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 shutdown_connections 2024-04-05T22:31:25.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:25.870+0000 7fb4e0e00640 1 -- 172.21.15.73:0/215158058 wait complete. 2024-04-05T22:31:25.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:31:26.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.102+0000 7f6acbc00640 1 Processor -- start 2024-04-05T22:31:26.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.102+0000 7f6acbc00640 1 -- start start 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.103+0000 7f6acbc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ac406c9a0 0x7f6ac406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6acbc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac406bea0 con 0x7f6ac406ba70 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6acbc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac415afc0 con 0x7f6ac415abe0 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6acbc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac406f5b0 con 0x7f6ac406c9a0 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ac406c9a0 0x7f6ac406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ac406c9a0 0x7f6ac406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51030/0 (socket says 172.21.15.73:51030) 2024-04-05T22:31:26.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 -- 172.21.15.73:0/2764979615 learned_addr learned my addr 172.21.15.73:0/2764979615 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:26.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 -- 172.21.15.73:0/2764979615 >> v1:172.21.15.157:6789/0 conn(0x7f6ac415abe0 legacy=0x7f6ac40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 -- 172.21.15.73:0/2764979615 >> v1:172.21.15.73:6789/0 conn(0x7f6ac406ba70 legacy=0x7f6ac4164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.104+0000 7f6ac3e00640 1 -- 172.21.15.73:0/2764979615 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac406f890 con 0x7f6ac406c9a0 2024-04-05T22:31:26.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.106+0000 7f6ac3e00640 1 --2- 172.21.15.73:0/2764979615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac406c9a0 0x7f6ac406b530 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe7eeb24833bcd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.106+0000 7f6ac3400640 1 -- 172.21.15.73:0/2764979615 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00317e0 con 0x7f6ac406c9a0 2024-04-05T22:31:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.106+0000 7f6ac3400640 1 -- 172.21.15.73:0/2764979615 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0031940 con 0x7f6ac406c9a0 2024-04-05T22:31:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.106+0000 7f6ac3400640 1 -- 172.21.15.73:0/2764979615 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0031cd0 con 0x7f6ac406c9a0 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.106+0000 7f6acbc00640 1 -- 172.21.15.73:0/2764979615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac406c9a0 msgr2=0x7f6ac406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 --2- 172.21.15.73:0/2764979615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac406c9a0 0x7f6ac406b530 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab002b880 comp rx=0 tx=0).stop 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 -- 172.21.15.73:0/2764979615 shutdown_connections 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 --2- 172.21.15.73:0/2764979615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac406c9a0 0x7f6ac406b530 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 -- 172.21.15.73:0/2764979615 >> 172.21.15.73:0/2764979615 conn(0x7f6ac4069cb0 msgr2=0x7f6ac406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 -- 172.21.15.73:0/2764979615 shutdown_connections 2024-04-05T22:31:26.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.107+0000 7f6acbc00640 1 -- 172.21.15.73:0/2764979615 wait complete. 2024-04-05T22:31:26.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.109+0000 7f6acbc00640 1 Processor -- start 2024-04-05T22:31:26.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.109+0000 7f6acbc00640 1 -- start start 2024-04-05T22:31:26.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ac406ba70 0x7f6ac4154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ac406c9a0 0x7f6ac41f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41f37a0 con 0x7f6ac415abe0 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41f31e0 con 0x7f6ac406c9a0 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6acbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41f34c0 con 0x7f6ac406ba70 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51034/0 (socket says 172.21.15.73:51034) 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 -- 172.21.15.73:0/3886797983 learned_addr learned my addr 172.21.15.73:0/3886797983 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac3e00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ac406c9a0 0x7f6ac41f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9200640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ac406ba70 0x7f6ac4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 -- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ac406ba70 msgr2=0x7f6ac4154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ac406ba70 0x7f6ac4154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 -- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ac406c9a0 msgr2=0x7f6ac41f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.110+0000 7f6ac9c00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ac406c9a0 0x7f6ac41f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6ac9c00640 1 -- 172.21.15.73:0/3886797983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00094d0 con 0x7f6ac415abe0 2024-04-05T22:31:26.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6ac9c00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c730 tx=0x7f6abc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc003b90 con 0x7f6ac415abe0 2024-04-05T22:31:26.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc003cf0 con 0x7f6ac415abe0 2024-04-05T22:31:26.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00aa90 con 0x7f6ac415abe0 2024-04-05T22:31:26.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac415bc80 con 0x7f6ac415abe0 2024-04-05T22:31:26.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.111+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac414b1b0 con 0x7f6ac415abe0 2024-04-05T22:31:26.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.112+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac40cdc40 con 0x7f6ac415abe0 2024-04-05T22:31:26.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.113+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f6abc00abf0 con 0x7f6ac415abe0 2024-04-05T22:31:26.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.114+0000 7f6ac0c00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 0x7f6aa406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.115+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6abc087ce0 con 0x7f6ac415abe0 2024-04-05T22:31:26.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.115+0000 7f6ac9200640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 0x7f6aa406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.116+0000 7f6ac9200640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 0x7f6aa406da50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800a750 tx=0x7f6ab8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.119+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6abc0546d0 con 0x7f6ac415abe0 2024-04-05T22:31:26.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.311+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ac4068160 con 0x7f6ac415abe0 2024-04-05T22:31:26.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.312+0000 7f6ac0c00640 1 -- 172.21.15.73:0/3886797983 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6abc057cf0 con 0x7f6ac415abe0 2024-04-05T22:31:26.312 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 msgr2=0x7f6aa406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 0x7f6aa406da50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800a750 tx=0x7f6ab8008040 comp rx=0 tx=0).stop 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 msgr2=0x7f6ac41516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c730 tx=0x7f6abc00cbf0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 shutdown_connections 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6aa406b590 0x7f6aa406da50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ac415abe0 0x7f6ac41516d0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ac406c9a0 0x7f6ac41f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 --2- 172.21.15.73:0/3886797983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ac406ba70 0x7f6ac4154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 >> 172.21.15.73:0/3886797983 conn(0x7f6ac4069cb0 msgr2=0x7f6ac40af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 shutdown_connections 2024-04-05T22:31:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.315+0000 7f6acbc00640 1 -- 172.21.15.73:0/3886797983 wait complete. 2024-04-05T22:31:26.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:31:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.549+0000 7fb3f7400640 1 Processor -- start 2024-04-05T22:31:26.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.549+0000 7fb3f7400640 1 -- start start 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f7400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3f00cfd70 0x7fb3f0167fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f7400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00ce460 con 0x7fb3f0160490 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f7400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00ce740 con 0x7fb3f00b3c70 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f7400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00cea20 con 0x7fb3f00cfd70 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f4a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb3f0160490 0x7fb3f00cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51774/0 (socket says 172.21.15.73:51774) 2024-04-05T22:31:26.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.550+0000 7fb3f4a00640 1 -- 172.21.15.73:0/306656613 learned_addr learned my addr 172.21.15.73:0/306656613 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377035181 0 0) 0x7fb3f00ce460 con 0x7fb3f0160490 2024-04-05T22:31:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3f5400640 1 --2- 172.21.15.73:0/306656613 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00cfd70 0x7fb3f0167fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d0003520 con 0x7fb3f0160490 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640534464 0 0) 0x7fb3d0003520 con 0x7fb3f0160490 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00cfd70 msgr2=0x7fb3f0167fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 --2- 172.21.15.73:0/306656613 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00cfd70 0x7fb3f0167fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 >> v1:172.21.15.157:6789/0 conn(0x7fb3f00b3c70 legacy=0x7fb3f00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00ced00 con 0x7fb3f0160490 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fb3e0001670 con 0x7fb3f0160490 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 >> v1:172.21.15.73:6789/0 conn(0x7fb3f0160490 legacy=0x7fb3f00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 0x7fb3d0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d00047c0 0x7fb3d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0005180 0x7fb3d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00cea20 con 0x7fb3d0003970 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d000c0b0 con 0x7fb3d00047c0 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d000c270 con 0x7fb3d0005180 2024-04-05T22:31:26.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.551+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 0x7fb3d0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f4a00640 1 -- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0005180 msgr2=0x7fb3d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0005180 0x7fb3d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f4a00640 1 -- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d00047c0 msgr2=0x7fb3d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d00047c0 0x7fb3d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f4a00640 1 -- 172.21.15.73:0/306656613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0003520 con 0x7fb3d0003970 2024-04-05T22:31:26.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.552+0000 7fb3f5400640 1 -- 172.21.15.73:0/306656613 reap_dead start 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.553+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 0x7fb3d0003d70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0001eb0 tx=0x7fb3d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9225bd958fe561 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.553+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0005ad0 con 0x7fb3d0003970 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.553+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e0005c50 con 0x7fb3d0003970 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.553+0000 7fb3ef400640 1 -- 172.21.15.73:0/306656613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e002c890 con 0x7fb3d0003970 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 -- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 msgr2=0x7fb3d0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 0x7fb3d0003d70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0001eb0 tx=0x7fb3d00057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 -- 172.21.15.73:0/306656613 shutdown_connections 2024-04-05T22:31:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 --2- 172.21.15.73:0/306656613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d0003970 0x7fb3d0003d70 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 -- 172.21.15.73:0/306656613 >> 172.21.15.73:0/306656613 conn(0x7fb3f009c020 msgr2=0x7fb3f00d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:26.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 -- 172.21.15.73:0/306656613 shutdown_connections 2024-04-05T22:31:26.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.554+0000 7fb3f7400640 1 -- 172.21.15.73:0/306656613 wait complete. 2024-04-05T22:31:26.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.555+0000 7fb3f7400640 1 Processor -- start 2024-04-05T22:31:26.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.556+0000 7fb3f7400640 1 -- start start 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.556+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0003970 0x7fb3f0148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f00cbc60 0x7fb3f014c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00ccd80 con 0x7fb3f00d2d90 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f7400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00d3760 con 0x7fb3f00cbc60 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01493a0 con 0x7fb3d0003970 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51054/0 (socket says 172.21.15.73:51054) 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 -- 172.21.15.73:0/2391217242 learned_addr learned my addr 172.21.15.73:0/2391217242 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0003970 0x7fb3f0148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 -- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0003970 msgr2=0x7fb3f0148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0003970 0x7fb3f0148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 -- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f00cbc60 msgr2=0x7fb3f014c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3efe00640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f00cbc60 0x7fb3f014c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f00cbc60 0x7fb3f014c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.557+0000 7fb3f5400640 1 -- 172.21.15.73:0/2391217242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00cd1a0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3f5400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cd470 tx=0x7fb3e800d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8017070 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8004020 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80041d0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f014c5d0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.558+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f00cc3e0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.559+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f00ccec0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.560+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fb3e8004330 con 0x7fb3f00d2d90 2024-04-05T22:31:26.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.561+0000 7fb3ecc00640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 0x7fb3cc06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.561+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3e8087590 con 0x7fb3f00d2d90 2024-04-05T22:31:26.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.562+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 0x7fb3cc06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.563+0000 7fb3f4a00640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 0x7fb3cc06da50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0034000 tx=0x7fb3e0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.566+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb3f00ccec0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.751+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3f00ccec0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.752+0000 7fb3ecc00640 1 -- 172.21.15.73:0/2391217242 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3f00ccec0 con 0x7fb3f00d2d90 2024-04-05T22:31:26.753 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.755+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 msgr2=0x7fb3cc06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.755+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 0x7fb3cc06da50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0034000 tx=0x7fb3e0009030 comp rx=0 tx=0).stop 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.755+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 msgr2=0x7fb3f00d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.755+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cd470 tx=0x7fb3e800d7e0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 shutdown_connections 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3cc06b590 0x7fb3cc06da50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f00d2d90 0x7fb3f00d3220 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f00cbc60 0x7fb3f014c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 --2- 172.21.15.73:0/2391217242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d0003970 0x7fb3f0148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 >> 172.21.15.73:0/2391217242 conn(0x7fb3f009c020 msgr2=0x7fb3f015d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 shutdown_connections 2024-04-05T22:31:26.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.756+0000 7fb3f7400640 1 -- 172.21.15.73:0/2391217242 wait complete. 2024-04-05T22:31:26.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.961+0000 7f043fa00640 1 Processor -- start 2024-04-05T22:31:26.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.961+0000 7f043fa00640 1 -- start start 2024-04-05T22:31:26.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043fa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f043806d710 0x7f04380cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f043815aef0 0x7f04380cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043fa00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043815aa60 con 0x7f04381541a0 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380b3b60 con 0x7f043815aef0 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043fa00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380701c0 con 0x7f043806d710 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f043815aef0 0x7f04380cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f043815aef0 0x7f04380cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51068/0 (socket says 172.21.15.73:51068) 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 -- 172.21.15.73:0/3053889638 learned_addr learned my addr 172.21.15.73:0/3053889638 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f0437e00640 1 --2- 172.21.15.73:0/3053889638 >> v2:172.21.15.157:3300/0 conn(0x7f043806d710 0x7f04380cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 -- 172.21.15.73:0/3053889638 >> v2:172.21.15.157:3300/0 conn(0x7f043806d710 msgr2=0x7f04380cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 --2- 172.21.15.73:0/3053889638 >> v2:172.21.15.157:3300/0 conn(0x7f043806d710 0x7f04380cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 -- 172.21.15.73:0/3053889638 >> v1:172.21.15.157:6789/0 conn(0x7f04381541a0 legacy=0x7f04380b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.963+0000 7f043da00640 1 -- 172.21.15.73:0/3053889638 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380b3880 con 0x7f043815aef0 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.964+0000 7f043da00640 1 --2- 172.21.15.73:0/3053889638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f043815aef0 0x7f04380cde30 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f043815e980 tx=0x7f043002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c84414a0ba46e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f0437400640 1 -- 172.21.15.73:0/3053889638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043002ca80 con 0x7f043815aef0 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f0437400640 1 -- 172.21.15.73:0/3053889638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043002cbe0 con 0x7f043815aef0 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f0437400640 1 -- 172.21.15.73:0/3053889638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300343f0 con 0x7f043815aef0 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 -- 172.21.15.73:0/3053889638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f043815aef0 msgr2=0x7f04380cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 --2- 172.21.15.73:0/3053889638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f043815aef0 0x7f04380cde30 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f043815e980 tx=0x7f043002bee0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 -- 172.21.15.73:0/3053889638 shutdown_connections 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 --2- 172.21.15.73:0/3053889638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f043815aef0 0x7f04380cde30 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 --2- 172.21.15.73:0/3053889638 >> v2:172.21.15.157:3300/0 conn(0x7f043806d710 0x7f04380cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.965+0000 7f043fa00640 1 -- 172.21.15.73:0/3053889638 >> 172.21.15.73:0/3053889638 conn(0x7f04380ce8e0 msgr2=0x7f04380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.966+0000 7f043fa00640 1 -- 172.21.15.73:0/3053889638 shutdown_connections 2024-04-05T22:31:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.966+0000 7f043fa00640 1 -- 172.21.15.73:0/3053889638 wait complete. 2024-04-05T22:31:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.967+0000 7f043fa00640 1 Processor -- start 2024-04-05T22:31:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.967+0000 7f043fa00640 1 -- start start 2024-04-05T22:31:26.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f043806d710 0x7f0438150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043815aef0 0x7f04380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04381f3630 con 0x7f04381541a0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04381509b0 con 0x7f043815aef0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f043fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04381f29b0 con 0x7f043806d710 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f0437e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f0437e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51078/0 (socket says 172.21.15.73:51078) 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.968+0000 7f0437e00640 1 -- 172.21.15.73:0/1245167139 learned_addr learned my addr 172.21.15.73:0/1245167139 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 -- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f043806d710 msgr2=0x7f0438150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f043806d710 0x7f0438150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 -- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043815aef0 msgr2=0x7f04380cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043815aef0 0x7f04380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 -- 172.21.15.73:0/1245167139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430009b30 con 0x7f04381541a0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0437e00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f042000c7f0 tx=0x7f042000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420004220 con 0x7f04381541a0 2024-04-05T22:31:26.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200043a0 con 0x7f04381541a0 2024-04-05T22:31:26.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420007860 con 0x7f04381541a0 2024-04-05T22:31:26.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.969+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04381f2c90 con 0x7f04381541a0 2024-04-05T22:31:26.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.970+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381f3250 con 0x7f04381541a0 2024-04-05T22:31:26.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.971+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438066af0 con 0x7f04381541a0 2024-04-05T22:31:26.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.972+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f0420007bd0 con 0x7f04381541a0 2024-04-05T22:31:26.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.972+0000 7f0434c00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 0x7f041406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:26.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.973+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0420087df0 con 0x7f04381541a0 2024-04-05T22:31:26.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.973+0000 7f043d000640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 0x7f041406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:26.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.974+0000 7f043d000640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 0x7f041406da50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0428001560 tx=0x7f0428007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:26.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:26.977+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0420054880 con 0x7f04381541a0 2024-04-05T22:31:27.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.212+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0438068160 con 0x7f04381541a0 2024-04-05T22:31:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.213+0000 7f0434c00640 1 -- 172.21.15.73:0/1245167139 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4746 (secure 0 0 0) 0x7f042001e090 con 0x7f04381541a0 2024-04-05T22:31:27.213 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:27.214 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 101 2024-04-05T22:31:27.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 msgr2=0x7f041406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 0x7f041406da50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0428001560 tx=0x7f0428007990 comp rx=0 tx=0).stop 2024-04-05T22:31:27.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 msgr2=0x7f04380aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f042000c7f0 tx=0x7f042000ccb0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 shutdown_connections 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f041406b590 0x7f041406da50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043815aef0 0x7f04380cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04381541a0 0x7f04380aa3d0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 --2- 172.21.15.73:0/1245167139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f043806d710 0x7f0438150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 >> 172.21.15.73:0/1245167139 conn(0x7f04380ce8e0 msgr2=0x7f04380b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 shutdown_connections 2024-04-05T22:31:27.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.216+0000 7f043fa00640 1 -- 172.21.15.73:0/1245167139 wait complete. 2024-04-05T22:31:27.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:27.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.439+0000 7f942e600640 1 Processor -- start 2024-04-05T22:31:27.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.440+0000 7f942e600640 1 -- start start 2024-04-05T22:31:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f942e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 0x7f942815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f942e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942815aa60 con 0x7f942806d710 2024-04-05T22:31:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f942e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94280b3b60 con 0x7f94281541a0 2024-04-05T22:31:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f942e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94280701c0 con 0x7f94280ce000 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f941fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f942806d710 0x7f94280cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51782/0 (socket says 172.21.15.73:51782) 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.441+0000 7f941fe00640 1 -- 172.21.15.73:0/1784100416 learned_addr learned my addr 172.21.15.73:0/1784100416 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262232766 0 0) 0x7f942815aa60 con 0x7f942806d710 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f940c005730 con 0x7f942806d710 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9427e00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 0x7f942815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2666573351 0 0) 0x7f940c005730 con 0x7f942806d710 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 msgr2=0x7f942815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 0x7f942815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 >> v1:172.21.15.157:6789/0 conn(0x7f94281541a0 legacy=0x7f94280b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280b3880 con 0x7f942806d710 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9410001670 con 0x7f942806d710 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 >> v1:172.21.15.73:6789/0 conn(0x7f942806d710 legacy=0x7f94280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c005bb0 0x7f940c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 0x7f940c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f940c006f10 0x7f940c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94280701c0 con 0x7f940c006f10 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c00dd20 con 0x7f940c005bb0 2024-04-05T22:31:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.442+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c00deb0 con 0x7f94280ce000 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f940c006f10 0x7f940c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 -- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 msgr2=0x7f940c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94280ce000 0x7f940c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 -- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c005bb0 msgr2=0x7f940c005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427400640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c005bb0 0x7f940c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c005bb0 0x7f940c005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.443+0000 7f9427e00640 1 -- 172.21.15.73:0/1784100416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c005730 con 0x7f940c006f10 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.444+0000 7f9427e00640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f940c006f10 0x7f940c007390 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f9420007390 tx=0x7f940c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba6f2d98c8b433a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.444+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942002ccf0 con 0x7f940c006f10 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.444+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942002ce50 con 0x7f940c006f10 2024-04-05T22:31:27.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.444+0000 7f9426a00640 1 -- 172.21.15.73:0/1784100416 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420034660 con 0x7f940c006f10 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 -- 172.21.15.73:0/1784100416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f940c006f10 msgr2=0x7f940c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 --2- 172.21.15.73:0/1784100416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f940c006f10 0x7f940c007390 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f9420007390 tx=0x7f940c007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f9427e00640 1 -- 172.21.15.73:0/1784100416 reap_dead start 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 -- 172.21.15.73:0/1784100416 shutdown_connections 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 -- 172.21.15.73:0/1784100416 >> 172.21.15.73:0/1784100416 conn(0x7f94280ce8e0 msgr2=0x7f94280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 -- 172.21.15.73:0/1784100416 shutdown_connections 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.445+0000 7f942e600640 1 -- 172.21.15.73:0/1784100416 wait complete. 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.447+0000 7f942e600640 1 Processor -- start 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.447+0000 7f942e600640 1 -- start start 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c006f10 0x7f942814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94281ef120 0x7f94280a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94280a9970 con 0x7f94280ac850 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942814fa50 con 0x7f940c006f10 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f942e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94280a8cf0 con 0x7f94281ef120 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51086/0 (socket says 172.21.15.73:51086) 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 -- 172.21.15.73:0/2932291508 learned_addr learned my addr 172.21.15.73:0/2932291508 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 -- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94281ef120 msgr2=0x7f94280a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94281ef120 0x7f94280a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 -- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c006f10 msgr2=0x7f942814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c006f10 0x7f942814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.448+0000 7f9427e00640 1 -- 172.21.15.73:0/2932291508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280a8fd0 con 0x7f94280ac850 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f9427e00640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f9420037040 tx=0x7f9420004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942003d600 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942003d760 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942003db10 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94280a92b0 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.449+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94280a8a00 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.450+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9428066af0 con 0x7f94280ac850 2024-04-05T22:31:27.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.451+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f942003b040 con 0x7f94280ac850 2024-04-05T22:31:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.452+0000 7f941f400640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 0x7f940406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.452+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94200a9ca0 con 0x7f94280ac850 2024-04-05T22:31:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.453+0000 7f9427400640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 0x7f940406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.454+0000 7f9427400640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 0x7f940406da50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f94280cd9d0 tx=0x7f9418009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.456+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9420076690 con 0x7f94280ac850 2024-04-05T22:31:27.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.649+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9428068160 con 0x7f94280ac850 2024-04-05T22:31:27.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.652+0000 7f941f400640 1 -- 172.21.15.73:0/2932291508 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f9420079cb0 con 0x7f94280ac850 2024-04-05T22:31:27.652 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:27.653 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:27.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 msgr2=0x7f940406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 0x7f940406da50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f94280cd9d0 tx=0x7f9418009040 comp rx=0 tx=0).stop 2024-04-05T22:31:27.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 msgr2=0x7f94280accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f9420037040 tx=0x7f9420004ab0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 shutdown_connections 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f940406b590 0x7f940406da50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94280ac850 0x7f94280accd0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94281ef120 0x7f94280a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 --2- 172.21.15.73:0/2932291508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f940c006f10 0x7f942814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 >> 172.21.15.73:0/2932291508 conn(0x7f94280ce8e0 msgr2=0x7f942806f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 shutdown_connections 2024-04-05T22:31:27.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.655+0000 7f942e600640 1 -- 172.21.15.73:0/2932291508 wait complete. 2024-04-05T22:31:27.675 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:31:27.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:27.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.884+0000 7febd0800640 1 Processor -- start 2024-04-05T22:31:27.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.884+0000 7febd0800640 1 -- start start 2024-04-05T22:31:27.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.885+0000 7febd0800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febc81587d0 0x7febc80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febd0800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc806f5b0 con 0x7febc815e850 2024-04-05T22:31:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febd0800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc8070410 con 0x7febc80ccd20 2024-04-05T22:31:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febd0800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc8158bb0 con 0x7febc81587d0 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febce800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7febc815e850 0x7febc80708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51792/0 (socket says 172.21.15.73:51792) 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febce800640 1 -- 172.21.15.73:0/3265843736 learned_addr learned my addr 172.21.15.73:0/3265843736 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febcde00640 1 --2- 172.21.15.73:0/3265843736 >> v2:172.21.15.157:3300/0 conn(0x7febc81587d0 0x7febc80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.886+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721601743 0 0) 0x7febc806f5b0 con 0x7febc815e850 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb0003520 con 0x7febc815e850 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562412689 0 0) 0x7febb0003520 con 0x7febc815e850 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 >> v2:172.21.15.157:3300/0 conn(0x7febc81587d0 msgr2=0x7febc80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 --2- 172.21.15.73:0/3265843736 >> v2:172.21.15.157:3300/0 conn(0x7febc81587d0 0x7febc80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 >> v1:172.21.15.157:6789/0 conn(0x7febc80ccd20 legacy=0x7febc806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc806f800 con 0x7febc815e850 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7febc40019f0 con 0x7febc815e850 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 >> v1:172.21.15.73:6789/0 conn(0x7febc815e850 legacy=0x7febc80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 0x7febb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb00047c0 0x7febb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0005180 0x7febb000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8158bb0 con 0x7febb0003970 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb000c0d0 con 0x7febb0005180 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb000c290 con 0x7febb00047c0 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.887+0000 7febcde00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 0x7febb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febcde00640 1 -- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb00047c0 msgr2=0x7febb0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febcde00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb00047c0 0x7febb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febcde00640 1 -- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0005180 msgr2=0x7febb000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febcde00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0005180 0x7febb000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febcde00640 1 -- 172.21.15.73:0/3265843736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0003520 con 0x7febb0003970 2024-04-05T22:31:27.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.888+0000 7febce800640 1 -- 172.21.15.73:0/3265843736 reap_dead start 2024-04-05T22:31:27.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.889+0000 7febcde00640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 0x7febb0003d90 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ba30 tx=0x7febb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e3fc7c261e217b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.889+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc032660 con 0x7febb0003970 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.889+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc0327c0 con 0x7febb0003970 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.889+0000 7febcca00640 1 -- 172.21.15.73:0/3265843736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc032b50 con 0x7febb0003970 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 -- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 msgr2=0x7febb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 0x7febb0003d90 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ba30 tx=0x7febb00057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 -- 172.21.15.73:0/3265843736 shutdown_connections 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 --2- 172.21.15.73:0/3265843736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febb0003970 0x7febb0003d90 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 -- 172.21.15.73:0/3265843736 >> 172.21.15.73:0/3265843736 conn(0x7febc80ce8e0 msgr2=0x7febc80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 -- 172.21.15.73:0/3265843736 shutdown_connections 2024-04-05T22:31:27.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.890+0000 7febd0800640 1 -- 172.21.15.73:0/3265843736 wait complete. 2024-04-05T22:31:27.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.892+0000 7febd0800640 1 Processor -- start 2024-04-05T22:31:27.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.892+0000 7febd0800640 1 -- start start 2024-04-05T22:31:27.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0003970 0x7febc8152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febc81f3b80 0x7febc80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81f3520 con 0x7febc81498f0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81f68b0 con 0x7febb0003970 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febd0800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc81f6b90 con 0x7febc81f3b80 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51098/0 (socket says 172.21.15.73:51098) 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 -- 172.21.15.73:0/3695573106 learned_addr learned my addr 172.21.15.73:0/3695573106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcde00640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0003970 0x7febc8152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febce800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febc81f3b80 0x7febc80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 -- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febc81f3b80 msgr2=0x7febc80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febc81f3b80 0x7febc80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 -- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0003970 msgr2=0x7febc8152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0003970 0x7febc8152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.893+0000 7febcd400640 1 -- 172.21.15.73:0/3695573106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc81f6e70 con 0x7febc81498f0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febcd400640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7febb800c390 tx=0x7febb800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb80052d0 con 0x7febc81498f0 2024-04-05T22:31:27.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8005450 con 0x7febc81498f0 2024-04-05T22:31:27.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8007260 con 0x7febc81498f0 2024-04-05T22:31:27.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc81f7150 con 0x7febc81498f0 2024-04-05T22:31:27.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.894+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81f6580 con 0x7febc81498f0 2024-04-05T22:31:27.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.895+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8066af0 con 0x7febc81498f0 2024-04-05T22:31:27.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.896+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7febb80073c0 con 0x7febc81498f0 2024-04-05T22:31:27.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.897+0000 7febc2000640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 0x7febac06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:27.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.897+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febb8087220 con 0x7febc81498f0 2024-04-05T22:31:27.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.898+0000 7febcde00640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 0x7febac06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:27.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.899+0000 7febcde00640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 0x7febac06da50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ba00 tx=0x7febbc03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:27.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:27.901+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7febb8053cb0 con 0x7febc81498f0 2024-04-05T22:31:28.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.130+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc8068160 con 0x7febc81498f0 2024-04-05T22:31:28.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.132+0000 7febc2000640 1 -- 172.21.15.73:0/3695573106 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4746 (secure 0 0 0) 0x7febb80572d0 con 0x7febc81498f0 2024-04-05T22:31:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:28.132 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:28.132 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 101 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 msgr2=0x7febac06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 0x7febac06da50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ba00 tx=0x7febbc03d040 comp rx=0 tx=0).stop 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 msgr2=0x7febc81f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7febb800c390 tx=0x7febb800c850 comp rx=0 tx=0).stop 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 shutdown_connections 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7febac06b590 0x7febac06da50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febc81f3b80 0x7febc80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc81498f0 0x7febc81f2fe0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 --2- 172.21.15.73:0/3695573106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb0003970 0x7febc8152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 >> 172.21.15.73:0/3695573106 conn(0x7febc80ce8e0 msgr2=0x7febc806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:28.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 shutdown_connections 2024-04-05T22:31:28.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.135+0000 7febd0800640 1 -- 172.21.15.73:0/3695573106 wait complete. 2024-04-05T22:31:28.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:28.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.363+0000 7f8834200640 1 Processor -- start 2024-04-05T22:31:28.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.363+0000 7f8834200640 1 -- start start 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c159440 0x7f882c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f882c06d3a0 0x7f882c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 0x7f882c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c0b2280 con 0x7f882c06d3a0 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c0b3a20 con 0x7f882c071340 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.364+0000 7f8834200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c0b2520 con 0x7f882c159440 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f882c06d3a0 0x7f882c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f882c06d3a0 0x7f882c070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51106/0 (socket says 172.21.15.73:51106) 2024-04-05T22:31:28.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 -- 172.21.15.73:0/94558422 learned_addr learned my addr 172.21.15.73:0/94558422 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:28.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 -- 172.21.15.73:0/94558422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c159440 msgr2=0x7f882c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 --2- 172.21.15.73:0/94558422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c159440 0x7f882c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 -- 172.21.15.73:0/94558422 >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 msgr2=0x7f882c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8832200640 1 --2- 172.21.15.73:0/94558422 >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 0x7f882c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 --2- 172.21.15.73:0/94558422 >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 0x7f882c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8830e00640 1 -- 172.21.15.73:0/94558422 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c0b2800 con 0x7f882c06d3a0 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.365+0000 7f8832200640 1 --2- 172.21.15.73:0/94558422 >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 0x7f882c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.366+0000 7f8830e00640 1 --2- 172.21.15.73:0/94558422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c06d3a0 0x7f882c070e00 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f882c0b0500 tx=0x7f881802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c495ca057d1313 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f882be00640 1 -- 172.21.15.73:0/94558422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881802ee90 con 0x7f882c06d3a0 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f882be00640 1 -- 172.21.15.73:0/94558422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818036440 con 0x7f882c06d3a0 2024-04-05T22:31:28.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f882be00640 1 -- 172.21.15.73:0/94558422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88180367d0 con 0x7f882c06d3a0 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 -- 172.21.15.73:0/94558422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c06d3a0 msgr2=0x7f882c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 --2- 172.21.15.73:0/94558422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c06d3a0 0x7f882c070e00 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f882c0b0500 tx=0x7f881802e0e0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 -- 172.21.15.73:0/94558422 shutdown_connections 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 --2- 172.21.15.73:0/94558422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c06d3a0 0x7f882c070e00 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 --2- 172.21.15.73:0/94558422 >> v2:172.21.15.157:3300/0 conn(0x7f882c071340 0x7f882c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 --2- 172.21.15.73:0/94558422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c159440 0x7f882c0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 -- 172.21.15.73:0/94558422 >> 172.21.15.73:0/94558422 conn(0x7f882c0ce8e0 msgr2=0x7f882c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:28.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 -- 172.21.15.73:0/94558422 shutdown_connections 2024-04-05T22:31:28.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.367+0000 7f8834200640 1 -- 172.21.15.73:0/94558422 wait complete. 2024-04-05T22:31:28.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.369+0000 7f8834200640 1 Processor -- start 2024-04-05T22:31:28.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.369+0000 7f8834200640 1 -- start start 2024-04-05T22:31:28.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c06d3a0 0x7f882c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c071340 0x7f882c15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c150b10 con 0x7f882c159440 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c1e3ba0 con 0x7f882c071340 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8834200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c151600 con 0x7f882c06d3a0 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8832200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.370+0000 7f8832200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51108/0 (socket says 172.21.15.73:51108) 2024-04-05T22:31:28.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 -- 172.21.15.73:0/3874603308 learned_addr learned my addr 172.21.15.73:0/3874603308 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8830e00640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c071340 0x7f882c15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 -- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c06d3a0 msgr2=0x7f882c151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8831800640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c06d3a0 0x7f882c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c06d3a0 0x7f882c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 -- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c071340 msgr2=0x7f882c15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c071340 0x7f882c15c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 -- 172.21.15.73:0/3874603308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881800b770 con 0x7f882c159440 2024-04-05T22:31:28.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.371+0000 7f8832200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f882400cef0 tx=0x7f882400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.372+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400d840 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.372+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882400d9c0 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.372+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400dd70 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.372+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c1e3e40 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.372+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c152c40 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.373+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c066af0 con 0x7f882c159440 2024-04-05T22:31:28.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.374+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f882401e3b0 con 0x7f882c159440 2024-04-05T22:31:28.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.375+0000 7f8829600640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 0x7f880c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.375+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8824088940 con 0x7f882c159440 2024-04-05T22:31:28.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.375+0000 7f8831800640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 0x7f880c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.377+0000 7f8831800640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 0x7f880c06da50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f882000a770 tx=0x7f8820008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.379+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8824055430 con 0x7f882c159440 2024-04-05T22:31:28.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.608+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f882c152aa0 con 0x7f882c159440 2024-04-05T22:31:28.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.609+0000 7f8829600640 1 -- 172.21.15.73:0/3874603308 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4746 (secure 0 0 0) 0x7f882c152aa0 con 0x7f882c159440 2024-04-05T22:31:28.609 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:28.610 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:30:58.914891+0000","modified":"2024-04-05T22:31:01.932703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9756},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9756":{"gid":9756,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3806798161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3806798161},{"type":"v1","addr":"172.21.15.157:6835","nonce":3806798161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-05T22:31:28.610 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 101 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.611+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 msgr2=0x7f880c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.611+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 0x7f880c06da50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f882000a770 tx=0x7f8820008040 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.611+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 msgr2=0x7f882c1401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.611+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f882400cef0 tx=0x7f882400a710 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 shutdown_connections 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f880c06b590 0x7f880c06da50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f882c159440 0x7f882c1401f0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c071340 0x7f882c15c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 --2- 172.21.15.73:0/3874603308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c06d3a0 0x7f882c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 >> 172.21.15.73:0/3874603308 conn(0x7f882c0ce8e0 msgr2=0x7f882c06cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 shutdown_connections 2024-04-05T22:31:28.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.612+0000 7f8834200640 1 -- 172.21.15.73:0/3874603308 wait complete. 2024-04-05T22:31:28.629 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.833+0000 7f713ee00640 1 Processor -- start 2024-04-05T22:31:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.833+0000 7f713ee00640 1 -- start start 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.834+0000 7f713ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 0x7f71380caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.834+0000 7f713ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71380cb2f0 0x7f71380cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.834+0000 7f713ee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380ad880 con 0x7f71380b1870 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.834+0000 7f713ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380cbcc0 con 0x7f71380cb2f0 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.834+0000 7f713ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380ce5d0 con 0x7f7138160620 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f712f400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f71380b1870 0x7f71380ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51794/0 (socket says 172.21.15.73:51794) 2024-04-05T22:31:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f712f400640 1 -- 172.21.15.73:0/3040383877 learned_addr learned my addr 172.21.15.73:0/3040383877 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744494455 0 0) 0x7f71380ad880 con 0x7f71380b1870 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c003520 con 0x7f71380b1870 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f713ce00640 1 --2- 172.21.15.73:0/3040383877 >> v2:172.21.15.157:3300/0 conn(0x7f71380cb2f0 0x7f71380cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137e00640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 0x7f71380caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 318580485 0 0) 0x7f711c003520 con 0x7f71380b1870 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 msgr2=0x7f71380caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.835+0000 7f7137400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 0x7f71380caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 >> v2:172.21.15.157:3300/0 conn(0x7f71380cb2f0 msgr2=0x7f71380cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 --2- 172.21.15.73:0/3040383877 >> v2:172.21.15.157:3300/0 conn(0x7f71380cb2f0 0x7f71380cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380ce010 con 0x7f71380b1870 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7120001670 con 0x7f71380b1870 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 >> v1:172.21.15.73:6789/0 conn(0x7f71380b1870 legacy=0x7f71380ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 0x7f711c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c003fa0 0x7f711c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380cbcc0 con 0x7f711c003fa0 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380ce5d0 con 0x7f711c004d00 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c00ba80 con 0x7f7138160620 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c003fa0 0x7f711c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 -- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 msgr2=0x7f711c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7138160620 0x7f711c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 -- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d00 msgr2=0x7f711c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.836+0000 7f712f400640 1 -- 172.21.15.73:0/3040383877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c003520 con 0x7f711c003fa0 2024-04-05T22:31:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.837+0000 7f712f400640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c003fa0 0x7f711c0047c0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f712000c240 tx=0x7f711c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3f55a8bd2658a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712002cb00 con 0x7f711c003fa0 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712002cc60 con 0x7f711c003fa0 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f7137400640 1 -- 172.21.15.73:0/3040383877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120005af0 con 0x7f711c003fa0 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 -- 172.21.15.73:0/3040383877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c003fa0 msgr2=0x7f711c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 --2- 172.21.15.73:0/3040383877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c003fa0 0x7f711c0047c0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f712000c240 tx=0x7f711c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ce00640 1 -- 172.21.15.73:0/3040383877 reap_dead start 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 -- 172.21.15.73:0/3040383877 shutdown_connections 2024-04-05T22:31:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 -- 172.21.15.73:0/3040383877 >> 172.21.15.73:0/3040383877 conn(0x7f71380ce8e0 msgr2=0x7f71380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 -- 172.21.15.73:0/3040383877 shutdown_connections 2024-04-05T22:31:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.838+0000 7f713ee00640 1 -- 172.21.15.73:0/3040383877 wait complete. 2024-04-05T22:31:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.840+0000 7f713ee00640 1 Processor -- start 2024-04-05T22:31:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.840+0000 7f713ee00640 1 -- start start 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c003fa0 0x7f71380b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71380ae920 0x7f71380aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138159b00 con 0x7f71380b3740 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b3bc0 con 0x7f711c003fa0 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f713ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7138146f60 con 0x7f71380ae920 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f712f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f712f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51126/0 (socket says 172.21.15.73:51126) 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.841+0000 7f712f400640 1 -- 172.21.15.73:0/3099972846 learned_addr learned my addr 172.21.15.73:0/3099972846 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f7137e00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c003fa0 0x7f71380b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 -- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71380ae920 msgr2=0x7f71380aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71380ae920 0x7f71380aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 -- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c003fa0 msgr2=0x7f71380b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c003fa0 0x7f71380b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 -- 172.21.15.73:0/3099972846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138146c80 con 0x7f71380b3740 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f712f400640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f712002c4a0 tx=0x7f7120001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712003c040 con 0x7f71380b3740 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120005d00 con 0x7f71380b3740 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200072b0 con 0x7f71380b3740 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7138159d10 con 0x7f71380b3740 2024-04-05T22:31:28.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.842+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713815a2d0 con 0x7f71380b3740 2024-04-05T22:31:28.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.844+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7138066af0 con 0x7f71380b3740 2024-04-05T22:31:28.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.844+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f7120032070 con 0x7f71380b3740 2024-04-05T22:31:28.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.845+0000 7f7134c00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 0x7f711406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:28.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.845+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71200a9300 con 0x7f71380b3740 2024-04-05T22:31:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.846+0000 7f7137e00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 0x7f711406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:28.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.847+0000 7f7137e00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 0x7f711406da50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7128006a90 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:28.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:28.849+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7120075d30 con 0x7f71380b3740 2024-04-05T22:31:29.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.044+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71380caa30 con 0x7f71380b3740 2024-04-05T22:31:29.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.046+0000 7f7134c00640 1 -- 172.21.15.73:0/3099972846 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f7120046020 con 0x7f71380b3740 2024-04-05T22:31:29.046 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:29.046 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":160,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:30:59.918940+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:30:57.382314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-05T22:30:57.918101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 msgr2=0x7f711406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 0x7f711406da50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7128006a90 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 msgr2=0x7f71380ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f712002c4a0 tx=0x7f7120001a00 comp rx=0 tx=0).stop 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 shutdown_connections 2024-04-05T22:31:29.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f711406b590 0x7f711406da50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71380ae920 0x7f71380aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71380b3740 0x7f71380ae3e0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 --2- 172.21.15.73:0/3099972846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c003fa0 0x7f71380b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 >> 172.21.15.73:0/3099972846 conn(0x7f71380ce8e0 msgr2=0x7f71380ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 shutdown_connections 2024-04-05T22:31:29.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.049+0000 7f713ee00640 1 -- 172.21.15.73:0/3099972846 wait complete. 2024-04-05T22:31:29.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:31:29.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.275+0000 7f8f4e600640 1 Processor -- start 2024-04-05T22:31:29.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.275+0000 7f8f4e600640 1 -- start start 2024-04-05T22:31:29.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f4e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f480b2c00 0x7f8f4806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f4e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480cc760 con 0x7f8f480b3070 2024-04-05T22:31:29.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f4e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480b1b60 con 0x7f8f481441e0 2024-04-05T22:31:29.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f4e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480b1950 con 0x7f8f480b2c00 2024-04-05T22:31:29.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f47e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f480b2c00 0x7f8f4806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f47e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f480b2c00 0x7f8f4806bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51142/0 (socket says 172.21.15.73:51142) 2024-04-05T22:31:29.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f47e00640 1 -- 172.21.15.73:0/3818103736 learned_addr learned my addr 172.21.15.73:0/3818103736 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:29.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.277+0000 7f8f4dc00640 1 --1- 172.21.15.73:0/3818103736 >> v1:172.21.15.73:6789/0 conn(0x7f8f480b3070 0x7f8f480cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51800/0 (socket says 172.21.15.73:51800) 2024-04-05T22:31:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.278+0000 7f8f47e00640 1 -- 172.21.15.73:0/3818103736 >> v1:172.21.15.157:6789/0 conn(0x7f8f481441e0 legacy=0x7f8f480cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.278+0000 7f8f47e00640 1 -- 172.21.15.73:0/3818103736 >> v1:172.21.15.73:6789/0 conn(0x7f8f480b3070 legacy=0x7f8f480cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.278+0000 7f8f47e00640 1 -- 172.21.15.73:0/3818103736 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480b1e40 con 0x7f8f480b2c00 2024-04-05T22:31:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.278+0000 7f8f47400640 1 -- 172.21.15.73:0/3818103736 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008044153 0 0) 0x7f8f480cc760 con 0x7f8f480b3070 2024-04-05T22:31:29.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.279+0000 7f8f47e00640 1 --2- 172.21.15.73:0/3818103736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f480b2c00 0x7f8f4806bbc0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be58332e27c5a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.279+0000 7f8f47400640 1 -- 172.21.15.73:0/3818103736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30030850 con 0x7f8f480b2c00 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.279+0000 7f8f47400640 1 -- 172.21.15.73:0/3818103736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f300309b0 con 0x7f8f480b2c00 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.279+0000 7f8f47400640 1 -- 172.21.15.73:0/3818103736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30030d40 con 0x7f8f480b2c00 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 -- 172.21.15.73:0/3818103736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f480b2c00 msgr2=0x7f8f4806bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 --2- 172.21.15.73:0/3818103736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f480b2c00 0x7f8f4806bbc0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002b990 comp rx=0 tx=0).stop 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 -- 172.21.15.73:0/3818103736 shutdown_connections 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 --2- 172.21.15.73:0/3818103736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f480b2c00 0x7f8f4806bbc0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 -- 172.21.15.73:0/3818103736 >> 172.21.15.73:0/3818103736 conn(0x7f8f480b0830 msgr2=0x7f8f480b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:29.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 -- 172.21.15.73:0/3818103736 shutdown_connections 2024-04-05T22:31:29.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.280+0000 7f8f4e600640 1 -- 172.21.15.73:0/3818103736 wait complete. 2024-04-05T22:31:29.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.282+0000 7f8f4e600640 1 Processor -- start 2024-04-05T22:31:29.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.282+0000 7f8f4e600640 1 -- start start 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f480b2c00 0x7f8f48140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f480b3070 0x7f8f481f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481549a0 con 0x7f8f481441e0 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48155660 con 0x7f8f480b3070 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4806f950 con 0x7f8f480b2c00 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51158/0 (socket says 172.21.15.73:51158) 2024-04-05T22:31:29.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 -- 172.21.15.73:0/125170097 learned_addr learned my addr 172.21.15.73:0/125170097 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4d200640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f480b2c00 0x7f8f48140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f47e00640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f480b3070 0x7f8f481f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 -- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f480b2c00 msgr2=0x7f8f48140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f480b2c00 0x7f8f48140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 -- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f480b3070 msgr2=0x7f8f481f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f480b3070 0x7f8f481f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.283+0000 7f8f4dc00640 1 -- 172.21.15.73:0/125170097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300094d0 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f4dc00640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c7a0 tx=0x7f8f4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000a640 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000a7a0 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000abd0 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4806fc30 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.284+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480701f0 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.285+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4804e160 con 0x7f8f481441e0 2024-04-05T22:31:29.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.287+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f8f4000ad30 con 0x7f8f481441e0 2024-04-05T22:31:29.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.288+0000 7f8f44c00640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 0x7f8f2406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.288+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f40087390 con 0x7f8f481441e0 2024-04-05T22:31:29.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.288+0000 7f8f4d200640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 0x7f8f2406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.289+0000 7f8f4d200640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 0x7f8f2406da50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006fd0 tx=0x7f8f38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.292+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8f40053e20 con 0x7f8f481441e0 2024-04-05T22:31:29.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.525+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f480b03c0 con 0x7f8f481441e0 2024-04-05T22:31:29.527 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:31:29.527+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:31:29.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.543+0000 7f8f44c00640 1 -- 172.21.15.73:0/125170097 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f40018070 con 0x7f8f481441e0 2024-04-05T22:31:29.544 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:31:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 msgr2=0x7f8f2406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 0x7f8f2406da50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006fd0 tx=0x7f8f38008040 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 msgr2=0x7f8f480ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c7a0 tx=0x7f8f4000cc60 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 shutdown_connections 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8f2406b590 0x7f8f2406da50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f481441e0 0x7f8f480ac8d0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f480b3070 0x7f8f481f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 --2- 172.21.15.73:0/125170097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f480b2c00 0x7f8f48140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 >> 172.21.15.73:0/125170097 conn(0x7f8f480b0830 msgr2=0x7f8f4806bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 shutdown_connections 2024-04-05T22:31:29.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.547+0000 7f8f4e600640 1 -- 172.21.15.73:0/125170097 wait complete. 2024-04-05T22:31:29.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:31:29.669 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:31:29.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.771+0000 7f6fc9c00640 1 Processor -- start 2024-04-05T22:31:29.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.771+0000 7f6fc9c00640 1 -- start start 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fc40ac8d0 0x7f6fc415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc9c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc4146ac0 con 0x7f6fc4070c50 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc9c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40acd30 con 0x7f6fc406c630 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc9c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc415e2d0 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc3400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6fc4070c50 0x7f6fc40710e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51814/0 (socket says 172.21.15.73:51814) 2024-04-05T22:31:29.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc3400640 1 -- 172.21.15.73:0/2084898208 learned_addr learned my addr 172.21.15.73:0/2084898208 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:29.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.773+0000 7f6fc2000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fc40ac8d0 0x7f6fc415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.774+0000 7f6fc2000640 1 -- 172.21.15.73:0/2084898208 >> v1:172.21.15.157:6789/0 conn(0x7f6fc406c630 legacy=0x7f6fc406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:29.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.774+0000 7f6fc2000640 1 -- 172.21.15.73:0/2084898208 >> v1:172.21.15.73:6789/0 conn(0x7f6fc4070c50 legacy=0x7f6fc40710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.774+0000 7f6fc2000640 1 -- 172.21.15.73:0/2084898208 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc4070720 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.775+0000 7f6fc2000640 1 --2- 172.21.15.73:0/2084898208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc415dd90 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c371c52fb76950 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.775+0000 7f6fc1600640 1 -- 172.21.15.73:0/2084898208 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac031830 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.775+0000 7f6fc1600640 1 -- 172.21.15.73:0/2084898208 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac031990 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.775+0000 7f6fc1600640 1 -- 172.21.15.73:0/2084898208 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac031d20 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 -- 172.21.15.73:0/2084898208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 msgr2=0x7f6fc415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/2084898208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc415dd90 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02b8d0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 -- 172.21.15.73:0/2084898208 shutdown_connections 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/2084898208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc415dd90 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 -- 172.21.15.73:0/2084898208 >> 172.21.15.73:0/2084898208 conn(0x7f6fc40b0830 msgr2=0x7f6fc40b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 -- 172.21.15.73:0/2084898208 shutdown_connections 2024-04-05T22:31:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.776+0000 7f6fc9c00640 1 -- 172.21.15.73:0/2084898208 wait complete. 2024-04-05T22:31:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.778+0000 7f6fc9c00640 1 Processor -- start 2024-04-05T22:31:29.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.778+0000 7f6fc9c00640 1 -- start start 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc406c630 0x7f6fc40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 0x7f6fc414ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40cce10 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40cd0b0 con 0x7f6fc406c630 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40cd390 con 0x7f6fc4070c50 2024-04-05T22:31:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51186/0 (socket says 172.21.15.73:51186) 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 -- 172.21.15.73:0/3653519420 learned_addr learned my addr 172.21.15.73:0/3653519420 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc2a00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc406c630 0x7f6fc40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc2000640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 0x7f6fc414ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 -- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 msgr2=0x7f6fc414ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 0x7f6fc414ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 -- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc406c630 msgr2=0x7f6fc40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc406c630 0x7f6fc40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc3400640 1 -- 172.21.15.73:0/3653519420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac009400 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.779+0000 7f6fc2000640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 0x7f6fc414ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fc3400640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ca80 tx=0x7f6fbc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00ac30 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc00ad90 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc40cd670 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc012600 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.780+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc40cdc30 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.781+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc40cde20 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.782+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f6fbc007070 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.783+0000 7f6fbaa00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 0x7f6fa006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.784+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6fbc08a690 con 0x7f6fc40ac8d0 2024-04-05T22:31:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.784+0000 7f6fc2a00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 0x7f6fa006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:29.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.785+0000 7f6fc2a00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 0x7f6fa006da50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006fd0 tx=0x7f6fb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:29.788+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6fbc054770 con 0x7f6fc40ac8d0 2024-04-05T22:31:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.005+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6fc4068160 con 0x7f6fc40ac8d0 2024-04-05T22:31:30.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.554+0000 7f6fbaa00640 1 -- 172.21.15.73:0/3653519420 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6fbc0170a0 con 0x7f6fc40ac8d0 2024-04-05T22:31:30.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 msgr2=0x7f6fa006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:30.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 0x7f6fa006da50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006fd0 tx=0x7f6fb4008040 comp rx=0 tx=0).stop 2024-04-05T22:31:30.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 msgr2=0x7f6fc414f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ca80 tx=0x7f6fbc00cf40 comp rx=0 tx=0).stop 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 shutdown_connections 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6fa006b590 0x7f6fa006da50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fc40ac8d0 0x7f6fc414f200 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fc4070c50 0x7f6fc414ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 --2- 172.21.15.73:0/3653519420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc406c630 0x7f6fc40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 >> 172.21.15.73:0/3653519420 conn(0x7f6fc40b0830 msgr2=0x7f6fc406ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 shutdown_connections 2024-04-05T22:31:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.558+0000 7f6fc9c00640 1 -- 172.21.15.73:0/3653519420 wait complete. 2024-04-05T22:31:30.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:31:30.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.797+0000 7f319b000640 1 Processor -- start 2024-04-05T22:31:30.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.797+0000 7f319b000640 1 -- start start 2024-04-05T22:31:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.798+0000 7f319b000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f319415aa80 0x7f319406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.798+0000 7f319b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31940714b0 0x7f319406b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.798+0000 7f319b000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940ccc50 con 0x7f319406b9d0 2024-04-05T22:31:30.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.798+0000 7f319b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940cd5f0 con 0x7f31940714b0 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.798+0000 7f319b000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940ccf30 con 0x7f319415aa80 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31940714b0 0x7f319406b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31940714b0 0x7f319406b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51190/0 (socket says 172.21.15.73:51190) 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 -- 172.21.15.73:0/515024106 learned_addr learned my addr 172.21.15.73:0/515024106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 -- 172.21.15.73:0/515024106 >> v2:172.21.15.157:3300/0 conn(0x7f319415aa80 msgr2=0x7f319406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 --2- 172.21.15.73:0/515024106 >> v2:172.21.15.157:3300/0 conn(0x7f319415aa80 0x7f319406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 -- 172.21.15.73:0/515024106 >> v1:172.21.15.157:6789/0 conn(0x7f319406b9d0 legacy=0x7f31940cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:30.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.799+0000 7f3199200640 1 -- 172.21.15.73:0/515024106 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940cd210 con 0x7f31940714b0 2024-04-05T22:31:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.800+0000 7f3199200640 1 --2- 172.21.15.73:0/515024106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f319406b490 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f319406b2b0 tx=0x7f318002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7470e506edd688b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.800+0000 7f3193e00640 1 -- 172.21.15.73:0/515024106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180031750 con 0x7f31940714b0 2024-04-05T22:31:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f3193e00640 1 -- 172.21.15.73:0/515024106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31800318b0 con 0x7f31940714b0 2024-04-05T22:31:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f3193e00640 1 -- 172.21.15.73:0/515024106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180031c40 con 0x7f31940714b0 2024-04-05T22:31:30.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 -- 172.21.15.73:0/515024106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 msgr2=0x7f319406b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 --2- 172.21.15.73:0/515024106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f319406b490 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f319406b2b0 tx=0x7f318002b7f0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 -- 172.21.15.73:0/515024106 shutdown_connections 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 --2- 172.21.15.73:0/515024106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f319406b490 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 --2- 172.21.15.73:0/515024106 >> v2:172.21.15.157:3300/0 conn(0x7f319415aa80 0x7f319406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 -- 172.21.15.73:0/515024106 >> 172.21.15.73:0/515024106 conn(0x7f31940b0360 msgr2=0x7f31940b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 -- 172.21.15.73:0/515024106 shutdown_connections 2024-04-05T22:31:30.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.801+0000 7f319b000640 1 -- 172.21.15.73:0/515024106 wait complete. 2024-04-05T22:31:30.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.803+0000 7f319b000640 1 Processor -- start 2024-04-05T22:31:30.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.803+0000 7f319b000640 1 -- start start 2024-04-05T22:31:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 0x7f3194146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 0x7f31941566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c4090 con 0x7f31940714b0 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c4370 con 0x7f319415aa80 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f319b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c4650 con 0x7f319406b9d0 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.804+0000 7f3199200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51198/0 (socket says 172.21.15.73:51198) 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 -- 172.21.15.73:0/88322368 learned_addr learned my addr 172.21.15.73:0/88322368 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:30.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199c00640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 0x7f3194146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f319a600640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 0x7f31941566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 -- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 msgr2=0x7f3194146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 0x7f3194146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 -- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 msgr2=0x7f31941566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 0x7f31941566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:30.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 -- 172.21.15.73:0/88322368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31800094d0 con 0x7f31940714b0 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199c00640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 0x7f3194146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f319a600640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 0x7f31941566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3199200640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f3180000c00 tx=0x7f31800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.805+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180033c60 con 0x7f31940714b0 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.806+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180033de0 con 0x7f31940714b0 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.806+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318003d6b0 con 0x7f31940714b0 2024-04-05T22:31:30.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.806+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940c4930 con 0x7f31940714b0 2024-04-05T22:31:30.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.806+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31941465c0 con 0x7f31940714b0 2024-04-05T22:31:30.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.807+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319404e160 con 0x7f31940714b0 2024-04-05T22:31:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.808+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f318003a040 con 0x7f31940714b0 2024-04-05T22:31:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.808+0000 7f3191600640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 0x7f317406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.809+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31800a9f30 con 0x7f31940714b0 2024-04-05T22:31:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.809+0000 7f3199c00640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 0x7f317406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:30.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.810+0000 7f3199c00640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 0x7f317406da50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3188006fd0 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:30.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:30.813+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3180076960 con 0x7f31940714b0 2024-04-05T22:31:31.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.008+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31940cae30 con 0x7f31940714b0 2024-04-05T22:31:31.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.728+0000 7f3191600640 1 -- 172.21.15.73:0/88322368 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v162) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3180079f80 con 0x7f31940714b0 2024-04-05T22:31:31.729 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:31:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.733+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 msgr2=0x7f317406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.733+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 0x7f317406da50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3188006fd0 tx=0x7f3188008040 comp rx=0 tx=0).stop 2024-04-05T22:31:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.733+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 msgr2=0x7f31940c35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.733+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f3180000c00 tx=0x7f31800046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 shutdown_connections 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f317406b590 0x7f317406da50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319415aa80 0x7f31941566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31940714b0 0x7f31940c35d0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 --2- 172.21.15.73:0/88322368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f319406b9d0 0x7f3194146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.734+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 >> 172.21.15.73:0/88322368 conn(0x7f31940b0360 msgr2=0x7f319406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.735+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 shutdown_connections 2024-04-05T22:31:31.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.735+0000 7f319b000640 1 -- 172.21.15.73:0/88322368 wait complete. 2024-04-05T22:31:31.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:31:31.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.956+0000 7f87d4200640 1 Processor -- start 2024-04-05T22:31:31.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.956+0000 7f87d4200640 1 -- start start 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d4200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87cc15ae50 0x7f87cc0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d4200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d4200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc15b230 con 0x7f87cc0cce90 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d4200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0cd320 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d4200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0ce2f0 con 0x7f87cc15ae50 2024-04-05T22:31:31.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51208/0 (socket says 172.21.15.73:51208) 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d2200640 1 -- 172.21.15.73:0/166138106 learned_addr learned my addr 172.21.15.73:0/166138106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.958+0000 7f87d1800640 1 --2- 172.21.15.73:0/166138106 >> v2:172.21.15.157:3300/0 conn(0x7f87cc15ae50 0x7f87cc0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.959+0000 7f87d2200640 1 -- 172.21.15.73:0/166138106 >> v2:172.21.15.157:3300/0 conn(0x7f87cc15ae50 msgr2=0x7f87cc0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.959+0000 7f87d2200640 1 --2- 172.21.15.73:0/166138106 >> v2:172.21.15.157:3300/0 conn(0x7f87cc15ae50 0x7f87cc0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.959+0000 7f87d2200640 1 -- 172.21.15.73:0/166138106 >> v1:172.21.15.157:6789/0 conn(0x7f87cc0cce90 legacy=0x7f87cc0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.959+0000 7f87d2200640 1 -- 172.21.15.73:0/166138106 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc15f4a0 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.960+0000 7f87d2200640 1 --2- 172.21.15.73:0/166138106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f87cc06dbb0 tx=0x7f87c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e295dd592f9a4148 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:31.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.960+0000 7f87cbe00640 1 -- 172.21.15.73:0/166138106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c402ca80 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.960+0000 7f87cbe00640 1 -- 172.21.15.73:0/166138106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c402cbe0 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.960+0000 7f87cbe00640 1 -- 172.21.15.73:0/166138106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c40343f0 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 -- 172.21.15.73:0/166138106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cf3e0 msgr2=0x7f87cc0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 --2- 172.21.15.73:0/166138106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f87cc06dbb0 tx=0x7f87c402bee0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 -- 172.21.15.73:0/166138106 shutdown_connections 2024-04-05T22:31:31.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 --2- 172.21.15.73:0/166138106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b2e00 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 --2- 172.21.15.73:0/166138106 >> v2:172.21.15.157:3300/0 conn(0x7f87cc15ae50 0x7f87cc0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 -- 172.21.15.73:0/166138106 >> 172.21.15.73:0/166138106 conn(0x7f87cc06a0d0 msgr2=0x7f87cc06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:31.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 -- 172.21.15.73:0/166138106 shutdown_connections 2024-04-05T22:31:31.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.961+0000 7f87d4200640 1 -- 172.21.15.73:0/166138106 wait complete. 2024-04-05T22:31:31.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.963+0000 7f87d4200640 1 Processor -- start 2024-04-05T22:31:31.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.964+0000 7f87d4200640 1 -- start start 2024-04-05T22:31:31.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87cc15ae50 0x7f87cc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1f31a0 con 0x7f87cc0cce90 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1f2520 con 0x7f87cc0cf3e0 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1f2800 con 0x7f87cc15ae50 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51220/0 (socket says 172.21.15.73:51220) 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 -- 172.21.15.73:0/1196039648 learned_addr learned my addr 172.21.15.73:0/1196039648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 -- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87cc15ae50 msgr2=0x7f87cc0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87cc15ae50 0x7f87cc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 -- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87cc0cf3e0 msgr2=0x7f87cc0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.965+0000 7f87d1800640 1 -- 172.21.15.73:0/1196039648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4009b30 con 0x7f87cc0cce90 2024-04-05T22:31:31.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87d1800640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f87c000d5f0 tx=0x7f87c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:31.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00125f0 con 0x7f87cc0cce90 2024-04-05T22:31:31.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0012750 con 0x7f87cc0cce90 2024-04-05T22:31:31.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc1f2ae0 con 0x7f87cc0cce90 2024-04-05T22:31:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0012b00 con 0x7f87cc0cce90 2024-04-05T22:31:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.966+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc14b8a0 con 0x7f87cc0cce90 2024-04-05T22:31:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.967+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc068160 con 0x7f87cc0cce90 2024-04-05T22:31:31.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.968+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f87c0012c60 con 0x7f87cc0cce90 2024-04-05T22:31:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.969+0000 7f87c9600640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 0x7f87ac06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.969+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f87c0088350 con 0x7f87cc0cce90 2024-04-05T22:31:31.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.970+0000 7f87d0e00640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 0x7f87ac06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:31.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.971+0000 7f87d0e00640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 0x7f87ac06da50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000f60 tx=0x7f87b80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:31.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:31.974+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f87c0054d80 con 0x7f87cc0cce90 2024-04-05T22:31:32.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.167+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87cc06ce80 con 0x7f87cc0cce90 2024-04-05T22:31:32.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.735+0000 7f87c9600640 1 -- 172.21.15.73:0/1196039648 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v163) v1 ==== 169+0+0 (secure 0 0 0) 0x7f87c00583a0 con 0x7f87cc0cce90 2024-04-05T22:31:32.736 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.741+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 msgr2=0x7f87ac06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.741+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 0x7f87ac06da50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000f60 tx=0x7f87b80079b0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.741+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 msgr2=0x7f87cc14b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.741+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f87c000d5f0 tx=0x7f87c000dab0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 shutdown_connections 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f87ac06b590 0x7f87ac06da50 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000f60 tx=0x7f87b80079b0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87cc15ae50 0x7f87cc0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87cc0cf3e0 0x7f87cc0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 --2- 172.21.15.73:0/1196039648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87cc0cce90 0x7f87cc14b360 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.742+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 >> 172.21.15.73:0/1196039648 conn(0x7f87cc06a0d0 msgr2=0x7f87cc06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:32.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.744+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 shutdown_connections 2024-04-05T22:31:32.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.745+0000 7f87d4200640 1 -- 172.21.15.73:0/1196039648 wait complete. 2024-04-05T22:31:32.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:31:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.954+0000 7fe5f3600640 1 Processor -- start 2024-04-05T22:31:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.955+0000 7fe5f3600640 1 -- start start 2024-04-05T22:31:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5f3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec06c630 0x7fe5ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5f3600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5f3600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec0b1b30 con 0x7fe5ec0b22a0 2024-04-05T22:31:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5f3600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec0b1e10 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5f3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec06cf50 con 0x7fe5ec06c630 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51228/0 (socket says 172.21.15.73:51228) 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.956+0000 7fe5e3e00640 1 -- 172.21.15.73:0/4132989223 learned_addr learned my addr 172.21.15.73:0/4132989223 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.957+0000 7fe5e3e00640 1 -- 172.21.15.73:0/4132989223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec06c630 msgr2=0x7fe5ec06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.957+0000 7fe5e3e00640 1 --2- 172.21.15.73:0/4132989223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec06c630 0x7fe5ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.957+0000 7fe5e3e00640 1 -- 172.21.15.73:0/4132989223 >> v1:172.21.15.157:6789/0 conn(0x7fe5ec0b22a0 legacy=0x7fe5ec0b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:32.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.957+0000 7fe5e3e00640 1 -- 172.21.15.73:0/4132989223 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec0cc300 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.958+0000 7fe5e3e00640 1 --2- 172.21.15.73:0/4132989223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4ba983fb9fe695 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:32.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.958+0000 7fe5ebe00640 1 -- 172.21.15.73:0/4132989223 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4031640 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.958+0000 7fe5ebe00640 1 -- 172.21.15.73:0/4132989223 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d40317a0 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.958+0000 7fe5ebe00640 1 -- 172.21.15.73:0/4132989223 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4031b30 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 -- 172.21.15.73:0/4132989223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec0cbe00 msgr2=0x7fe5ec0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:32.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 --2- 172.21.15.73:0/4132989223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 -- 172.21.15.73:0/4132989223 shutdown_connections 2024-04-05T22:31:32.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 --2- 172.21.15.73:0/4132989223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec0ce0a0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 --2- 172.21.15.73:0/4132989223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec06c630 0x7fe5ec06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 -- 172.21.15.73:0/4132989223 >> 172.21.15.73:0/4132989223 conn(0x7fe5ec0b0830 msgr2=0x7fe5ec0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:32.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 -- 172.21.15.73:0/4132989223 shutdown_connections 2024-04-05T22:31:32.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.959+0000 7fe5f3600640 1 -- 172.21.15.73:0/4132989223 wait complete. 2024-04-05T22:31:32.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.961+0000 7fe5f3600640 1 Processor -- start 2024-04-05T22:31:32.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.961+0000 7fe5f3600640 1 -- start start 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec0b22a0 0x7fe5ec1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec132ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1419a0 con 0x7fe5ec06c630 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1403c0 con 0x7fe5ec0cbe00 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1406a0 con 0x7fe5ec0b22a0 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51244/0 (socket says 172.21.15.73:51244) 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.962+0000 7fe5f0c00640 1 -- 172.21.15.73:0/872417744 learned_addr learned my addr 172.21.15.73:0/872417744 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f1600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec132ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5e3e00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec0b22a0 0x7fe5ec1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 -- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec0b22a0 msgr2=0x7fe5ec1426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:32.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec0b22a0 0x7fe5ec1426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 -- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 msgr2=0x7fe5ec132ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec132ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 -- 172.21.15.73:0/872417744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40094d0 con 0x7fe5ec06c630 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f1600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f0c00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00ba50 tx=0x7fe5dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00cbe0 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00cd60 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc014680 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec140980 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.963+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec140f40 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.964+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec0b19a0 con 0x7fe5ec06c630 2024-04-05T22:31:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.965+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fe5dc01d3b0 con 0x7fe5ec06c630 2024-04-05T22:31:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.966+0000 7fe5e9600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 0x7fe5c806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.966+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe5dc087dd0 con 0x7fe5ec06c630 2024-04-05T22:31:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.967+0000 7fe5e3e00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 0x7fe5c806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:32.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.968+0000 7fe5e3e00640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 0x7fe5c806da50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:32.970+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe5dc054860 con 0x7fe5ec06c630 2024-04-05T22:31:33.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.164+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe5ec068160 con 0x7fe5ec06c630 2024-04-05T22:31:33.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.170+0000 7fe5e9600640 1 -- 172.21.15.73:0/872417744 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5dc057e80 con 0x7fe5ec06c630 2024-04-05T22:31:33.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.172+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 msgr2=0x7fe5c806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.172+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 0x7fe5c806da50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d403c040 comp rx=0 tx=0).stop 2024-04-05T22:31:33.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.172+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 msgr2=0x7fe5ec141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.172+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00ba50 tx=0x7fe5dc00bf10 comp rx=0 tx=0).stop 2024-04-05T22:31:33.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 shutdown_connections 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe5c806b590 0x7fe5c806da50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5ec0cbe00 0x7fe5ec132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5ec0b22a0 0x7fe5ec1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 --2- 172.21.15.73:0/872417744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5ec06c630 0x7fe5ec141460 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 >> 172.21.15.73:0/872417744 conn(0x7fe5ec0b0830 msgr2=0x7fe5ec071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 shutdown_connections 2024-04-05T22:31:33.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.173+0000 7fe5f3600640 1 -- 172.21.15.73:0/872417744 wait complete. 2024-04-05T22:31:33.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-05T22:31:33.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.398+0000 7fa9fb800640 1 Processor -- start 2024-04-05T22:31:33.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.399+0000 7fa9fb800640 1 -- start start 2024-04-05T22:31:33.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.400+0000 7fa9fb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9f40c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.400+0000 7fa9fb800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f406aa40 con 0x7fa9f415d200 2024-04-05T22:31:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.400+0000 7fa9fb800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f406ad20 con 0x7fa9f40ca050 2024-04-05T22:31:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.400+0000 7fa9fb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40ad6e0 con 0x7fa9f415ce20 2024-04-05T22:31:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.401+0000 7fa9f3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa9f415d200 0x7fa9f40ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51816/0 (socket says 172.21.15.73:51816) 2024-04-05T22:31:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.401+0000 7fa9f3e00640 1 -- 172.21.15.73:0/4128320242 learned_addr learned my addr 172.21.15.73:0/4128320242 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:33.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.401+0000 7fa9f8e00640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9f40c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.402+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841141652 0 0) 0x7fa9f406aa40 con 0x7fa9f415d200 2024-04-05T22:31:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d8003520 con 0x7fa9f415d200 2024-04-05T22:31:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263123011 0 0) 0x7fa9f406ad20 con 0x7fa9f40ca050 2024-04-05T22:31:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f406aa40 con 0x7fa9f40ca050 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613022034 0 0) 0x7fa9d8003520 con 0x7fa9f415d200 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 msgr2=0x7fa9f40c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9f40c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 >> v1:172.21.15.157:6789/0 conn(0x7fa9f40ca050 legacy=0x7fa9f40acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40cde30 con 0x7fa9f415d200 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fa9dc001670 con 0x7fa9f415d200 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 >> v1:172.21.15.73:6789/0 conn(0x7fa9f415d200 legacy=0x7fa9f40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d8004b90 0x7fa9d8004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9d8003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9d8004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.403+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40ad6e0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.404+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800ba30 con 0x7fa9d8004b90 2024-04-05T22:31:33.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.404+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800bb70 con 0x7fa9f415ce20 2024-04-05T22:31:33.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.404+0000 7fa9f9800640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9d8004120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.404+0000 7fa9f3e00640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9d8003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.405+0000 7fa9f9800640 1 -- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 msgr2=0x7fa9d8003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.405+0000 7fa9f9800640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f415ce20 0x7fa9d8003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.405+0000 7fa9f9800640 1 -- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d8004b90 msgr2=0x7fa9d8004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.405+0000 7fa9f9800640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d8004b90 0x7fa9d8004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.405+0000 7fa9f9800640 1 -- 172.21.15.73:0/4128320242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40cde30 con 0x7fa9d8003ca0 2024-04-05T22:31:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.406+0000 7fa9f9800640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9d8004120 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec001fd0 tx=0x7fa9d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4a452038db72d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.406+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003b70 con 0x7fa9d8003ca0 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.406+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec003cd0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.406+0000 7fa9f3400640 1 -- 172.21.15.73:0/4128320242 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0052f0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.407+0000 7fa9fb800640 1 -- 172.21.15.73:0/4128320242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 msgr2=0x7fa9d8004120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.407+0000 7fa9fb800640 1 --2- 172.21.15.73:0/4128320242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9d8004120 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec001fd0 tx=0x7fa9d80057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.407+0000 7fa9f9800640 1 -- 172.21.15.73:0/4128320242 reap_dead start 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.407+0000 7fa9fb800640 1 -- 172.21.15.73:0/4128320242 shutdown_connections 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.407+0000 7fa9fb800640 1 -- 172.21.15.73:0/4128320242 >> 172.21.15.73:0/4128320242 conn(0x7fa9f40cf190 msgr2=0x7fa9f40cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.408+0000 7fa9fb800640 1 -- 172.21.15.73:0/4128320242 shutdown_connections 2024-04-05T22:31:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.408+0000 7fa9fb800640 1 -- 172.21.15.73:0/4128320242 wait complete. 2024-04-05T22:31:33.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.409+0000 7fa9fb800640 1 Processor -- start 2024-04-05T22:31:33.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.410+0000 7fa9fb800640 1 -- start start 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f41445c0 0x7fa9f40b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f406ff50 0x7fa9f41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41f2840 con 0x7fa9d8003ca0 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c8d00 con 0x7fa9f406ff50 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9fb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40c8fe0 con 0x7fa9f41445c0 2024-04-05T22:31:33.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9f8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9f8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51272/0 (socket says 172.21.15.73:51272) 2024-04-05T22:31:33.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.411+0000 7fa9f8e00640 1 -- 172.21.15.73:0/2753981872 learned_addr learned my addr 172.21.15.73:0/2753981872 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 -- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f41445c0 msgr2=0x7fa9f40b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f41445c0 0x7fa9f40b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 -- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f406ff50 msgr2=0x7fa9f41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f406ff50 0x7fa9f41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 -- 172.21.15.73:0/2753981872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40c92c0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f8e00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e7c0 tx=0x7fa9e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400eed0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400c8c0 con 0x7fa9d8003ca0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400cc70 con 0x7fa9d8003ca0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f4070920 con 0x7fa9d8003ca0 2024-04-05T22:31:33.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.412+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f40b4d60 con 0x7fa9d8003ca0 2024-04-05T22:31:33.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.414+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4068160 con 0x7fa9d8003ca0 2024-04-05T22:31:33.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.415+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fa9e4013470 con 0x7fa9d8003ca0 2024-04-05T22:31:33.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.415+0000 7fa9f0c00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 0x7fa9d006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.416+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9e4087f20 con 0x7fa9d8003ca0 2024-04-05T22:31:33.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.416+0000 7fa9f3e00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 0x7fa9d006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.418+0000 7fa9f3e00640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 0x7fa9d006da50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc001eb0 tx=0x7fa9dc001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:33.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.420+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa9e4054950 con 0x7fa9d8003ca0 2024-04-05T22:31:33.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.685+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7fa9f40c5a10 con 0x7fa9d8003ca0 2024-04-05T22:31:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.755+0000 7fa9f0c00640 1 -- 172.21.15.73:0/2753981872 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v558) v1 ==== 145+0+0 (secure 0 0 0) 0x7fa9e4057f70 con 0x7fa9d8003ca0 2024-04-05T22:31:33.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 msgr2=0x7fa9d006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 0x7fa9d006da50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc001eb0 tx=0x7fa9dc001170 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 msgr2=0x7fa9f41471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e7c0 tx=0x7fa9e400ec80 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 shutdown_connections 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa9d006b590 0x7fa9d006da50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f406ff50 0x7fa9f41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f41445c0 0x7fa9f40b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 --2- 172.21.15.73:0/2753981872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d8003ca0 0x7fa9f41471f0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 >> 172.21.15.73:0/2753981872 conn(0x7fa9f40cf190 msgr2=0x7fa9f40af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 shutdown_connections 2024-04-05T22:31:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.758+0000 7fa9fb800640 1 -- 172.21.15.73:0/2753981872 wait complete. 2024-04-05T22:31:33.778 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-05T22:31:33.779 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-04-05T22:31:33.779 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-05T22:31:33.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:33.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.985+0000 7f57dbe00640 1 Processor -- start 2024-04-05T22:31:33.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.985+0000 7f57dbe00640 1 -- start start 2024-04-05T22:31:33.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.987+0000 7f57dbe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57dc158860 0x7f57dc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.987+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.987+0000 7f57dbe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc06b500 con 0x7f57dc15eb60 2024-04-05T22:31:33.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.987+0000 7f57dbe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc15eff0 con 0x7f57dc158860 2024-04-05T22:31:33.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.987+0000 7f57dbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0ccaf0 con 0x7f57dc0b1f00 2024-04-05T22:31:33.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.988+0000 7f57daa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57dc158860 0x7f57dc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.988+0000 7f57da000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.988+0000 7f57daa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57dc158860 0x7f57dc0b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57394/0 (socket says 172.21.15.73:57394) 2024-04-05T22:31:33.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.988+0000 7f57da000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:53000/0 (socket says 172.21.15.73:53000) 2024-04-05T22:31:33.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.988+0000 7f57daa00640 1 -- 172.21.15.73:0/3542169526 learned_addr learned my addr 172.21.15.73:0/3542169526 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:33.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.989+0000 7f57daa00640 1 -- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 msgr2=0x7f57dc15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:33.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.989+0000 7f57daa00640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.989+0000 7f57daa00640 1 -- 172.21.15.73:0/3542169526 >> v1:172.21.15.157:6789/0 conn(0x7f57dc15eb60 legacy=0x7f57dc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.989+0000 7f57daa00640 1 -- 172.21.15.73:0/3542169526 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0b23a0 con 0x7f57dc158860 2024-04-05T22:31:33.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.989+0000 7f57da000640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:31:33.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.990+0000 7f57daa00640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc158860 0x7f57dc0b19c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009870 tx=0x7f57cc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c6ccc086401652 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:33.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.991+0000 7f57d9600640 1 -- 172.21.15.73:0/3542169526 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc031640 con 0x7f57dc158860 2024-04-05T22:31:33.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.991+0000 7f57d9600640 1 -- 172.21.15.73:0/3542169526 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0317a0 con 0x7f57dc158860 2024-04-05T22:31:33.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.991+0000 7f57d9600640 1 -- 172.21.15.73:0/3542169526 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc031b30 con 0x7f57dc158860 2024-04-05T22:31:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.991+0000 7f57dbe00640 1 -- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc158860 msgr2=0x7f57dc0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.991+0000 7f57dbe00640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc158860 0x7f57dc0b19c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009870 tx=0x7f57cc02b8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 -- 172.21.15.73:0/3542169526 shutdown_connections 2024-04-05T22:31:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc0b1f00 0x7f57dc15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 --2- 172.21.15.73:0/3542169526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc158860 0x7f57dc0b19c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 -- 172.21.15.73:0/3542169526 >> 172.21.15.73:0/3542169526 conn(0x7f57dc0b06c0 msgr2=0x7f57dc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:33.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 -- 172.21.15.73:0/3542169526 shutdown_connections 2024-04-05T22:31:33.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.992+0000 7f57dbe00640 1 -- 172.21.15.73:0/3542169526 wait complete. 2024-04-05T22:31:33.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.994+0000 7f57dbe00640 1 Processor -- start 2024-04-05T22:31:33.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.994+0000 7f57dbe00640 1 -- start start 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc15eb60 0x7f57dc070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc0713f0 0x7f57dc14a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0ca570 con 0x7f57dc0b1f00 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0c91f0 con 0x7f57dc0713f0 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57dbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0c94d0 con 0x7f57dc15eb60 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57daa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57daa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51278/0 (socket says 172.21.15.73:51278) 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.995+0000 7f57daa00640 1 -- 172.21.15.73:0/1398010958 learned_addr learned my addr 172.21.15.73:0/1398010958 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:33.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57db400640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc0713f0 0x7f57dc14a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57da000640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc15eb60 0x7f57dc070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 -- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc15eb60 msgr2=0x7f57dc070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc15eb60 0x7f57dc070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 -- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc0713f0 msgr2=0x7f57dc14a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc0713f0 0x7f57dc14a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:33.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 -- 172.21.15.73:0/1398010958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c8009400 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.996+0000 7f57daa00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.997+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc03a930 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.997+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc03aa90 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.997+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc03ae40 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.997+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc0094d0 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.997+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc0c9a90 con 0x7f57dc0b1f00 2024-04-05T22:31:33.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.998+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc04e160 con 0x7f57dc0b1f00 2024-04-05T22:31:33.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:33.999+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f57cc02f050 con 0x7f57dc0b1f00 2024-04-05T22:31:34.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.000+0000 7f57d2a00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 0x7f57bc06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.000+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f57cc0a9cd0 con 0x7f57dc0b1f00 2024-04-05T22:31:34.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.001+0000 7f57da000640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 0x7f57bc06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.002+0000 7f57da000640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 0x7f57bc06da50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f57dc071960 tx=0x7f57c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.004+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f57cc076700 con 0x7f57dc0b1f00 2024-04-05T22:31:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.199+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57dc0b13f0 con 0x7f57dc0b1f00 2024-04-05T22:31:34.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.201+0000 7f57d2a00640 1 -- 172.21.15.73:0/1398010958 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+12530 (secure 0 0 0) 0x7f57cc079d20 con 0x7f57dc0b1f00 2024-04-05T22:31:34.201 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:34.201 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":163,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:32.722449+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/3806798161":"2024-04-06T22:31:29.527584+0000","172.21.15.157:6834/3806798161":"2024-04-06T22:31:29.527584+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:34.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.203+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 msgr2=0x7f57bc06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.203+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 0x7f57bc06da50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f57dc071960 tx=0x7f57c8008040 comp rx=0 tx=0).stop 2024-04-05T22:31:34.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.203+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 msgr2=0x7f57dc15d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.203+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 shutdown_connections 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f57bc06b590 0x7f57bc06da50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57dc0713f0 0x7f57dc14a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57dc15eb60 0x7f57dc070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 --2- 172.21.15.73:0/1398010958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57dc0b1f00 0x7f57dc15d580 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 >> 172.21.15.73:0/1398010958 conn(0x7f57dc0b06c0 msgr2=0x7f57dc0ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 shutdown_connections 2024-04-05T22:31:34.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.204+0000 7f57dbe00640 1 -- 172.21.15.73:0/1398010958 wait complete. 2024-04-05T22:31:34.223 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:34.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:34.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.421+0000 7f2225200640 1 Processor -- start 2024-04-05T22:31:34.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.421+0000 7f2225200640 1 -- start start 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f2225200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f222015a850 0x7f222015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f2225200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222006bc50 con 0x7f2220070640 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f2225200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222015ef70 con 0x7f222015e580 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f2225200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222015f530 con 0x7f222015a850 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f221f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f222015a850 0x7f222015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f221fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2220070640 0x7f222006c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51822/0 (socket says 172.21.15.73:51822) 2024-04-05T22:31:34.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f221fe00640 1 -- 172.21.15.73:0/2703501491 learned_addr learned my addr 172.21.15.73:0/2703501491 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:34.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.423+0000 7f221f400640 1 -- 172.21.15.73:0/2703501491 >> v1:172.21.15.157:6789/0 conn(0x7f222015e580 legacy=0x7f222006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:34.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.424+0000 7f221f400640 1 -- 172.21.15.73:0/2703501491 >> v1:172.21.15.73:6789/0 conn(0x7f2220070640 legacy=0x7f222006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.424+0000 7f221f400640 1 -- 172.21.15.73:0/2703501491 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222015f250 con 0x7f222015a850 2024-04-05T22:31:34.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f221f400640 1 --2- 172.21.15.73:0/2703501491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f222015a850 0x7f222015ac30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f22100097a0 tx=0x7f221002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b98eef67c89816 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f221e000640 1 -- 172.21.15.73:0/2703501491 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221002e070 con 0x7f222015a850 2024-04-05T22:31:34.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f221e000640 1 -- 172.21.15.73:0/2703501491 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22100315a0 con 0x7f222015a850 2024-04-05T22:31:34.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f221e000640 1 -- 172.21.15.73:0/2703501491 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210031930 con 0x7f222015a850 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f2225200640 1 -- 172.21.15.73:0/2703501491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f222015a850 msgr2=0x7f222015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.425+0000 7f2225200640 1 --2- 172.21.15.73:0/2703501491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f222015a850 0x7f222015ac30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f22100097a0 tx=0x7f221002b840 comp rx=0 tx=0).stop 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.426+0000 7f2225200640 1 -- 172.21.15.73:0/2703501491 shutdown_connections 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.426+0000 7f2225200640 1 --2- 172.21.15.73:0/2703501491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f222015a850 0x7f222015ac30 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.426+0000 7f2225200640 1 -- 172.21.15.73:0/2703501491 >> 172.21.15.73:0/2703501491 conn(0x7f2220069cb0 msgr2=0x7f222006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.426+0000 7f2225200640 1 -- 172.21.15.73:0/2703501491 shutdown_connections 2024-04-05T22:31:34.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.426+0000 7f2225200640 1 -- 172.21.15.73:0/2703501491 wait complete. 2024-04-05T22:31:34.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.427+0000 7f2225200640 1 Processor -- start 2024-04-05T22:31:34.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.427+0000 7f2225200640 1 -- start start 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f222015a850 0x7f222015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f222015e580 0x7f222015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f04b0 con 0x7f2220070640 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f0730 con 0x7f222015a850 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.428+0000 7f2225200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f0a10 con 0x7f222015e580 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51310/0 (socket says 172.21.15.73:51310) 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 -- 172.21.15.73:0/1221234270 learned_addr learned my addr 172.21.15.73:0/1221234270 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:34.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 -- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f222015e580 msgr2=0x7f222015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f222015e580 0x7f222015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 -- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f222015a850 msgr2=0x7f222015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f222015a850 0x7f222015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 -- 172.21.15.73:0/1221234270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2210009400 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.429+0000 7f221f400640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f22100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.430+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210032900 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.430+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2210032a80 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.430+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22201f0cf0 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.430+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210032e30 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.430+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222013a860 con 0x7f2220070640 2024-04-05T22:31:34.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.431+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22201f1210 con 0x7f2220070640 2024-04-05T22:31:34.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.432+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f221004d3f0 con 0x7f2220070640 2024-04-05T22:31:34.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.433+0000 7f2217400640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 0x7f220006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.433+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f22100ac3a0 con 0x7f2220070640 2024-04-05T22:31:34.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.434+0000 7f221ea00640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 0x7f220006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.435+0000 7f221ea00640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 0x7f220006da50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f220c006fd0 tx=0x7f220c006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.437+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f22201f1210 con 0x7f2220070640 2024-04-05T22:31:34.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.626+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22201f1210 con 0x7f2220070640 2024-04-05T22:31:34.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.627+0000 7f2217400640 1 -- 172.21.15.73:0/1221234270 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22201f1210 con 0x7f2220070640 2024-04-05T22:31:34.627 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 msgr2=0x7f220006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 0x7f220006da50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f220c006fd0 tx=0x7f220c006bc0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 msgr2=0x7f222013a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f22100046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 shutdown_connections 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f220006b590 0x7f220006da50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f222015e580 0x7f222015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f222015a850 0x7f222015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 --2- 172.21.15.73:0/1221234270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2220070640 0x7f222013a320 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.629+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 >> 172.21.15.73:0/1221234270 conn(0x7f2220069cb0 msgr2=0x7f22200af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.630+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 shutdown_connections 2024-04-05T22:31:34.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.630+0000 7f2225200640 1 -- 172.21.15.73:0/1221234270 wait complete. 2024-04-05T22:31:34.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:31:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.847+0000 7f299b000640 1 Processor -- start 2024-04-05T22:31:34.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.847+0000 7f299b000640 1 -- start start 2024-04-05T22:31:34.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f299b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29940b1870 0x7f29940ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f299b000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29940ad6b0 0x7f29940cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f299b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29940ce5d0 con 0x7f2994160620 2024-04-05T22:31:34.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f299b000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29940ce010 con 0x7f29940ad6b0 2024-04-05T22:31:34.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f299b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940ce2f0 con 0x7f29940b1870 2024-04-05T22:31:34.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f2993e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2994160620 0x7f29940caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51836/0 (socket says 172.21.15.73:51836) 2024-04-05T22:31:34.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.849+0000 7f2993e00640 1 -- 172.21.15.73:0/2390821921 learned_addr learned my addr 172.21.15.73:0/2390821921 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2999000640 1 --2- 172.21.15.73:0/2390821921 >> v2:172.21.15.157:3300/0 conn(0x7f29940ad6b0 0x7f29940cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2993400640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29940b1870 0x7f29940ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463738172 0 0) 0x7f29940ce5d0 con 0x7f2994160620 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2978003520 con 0x7f2994160620 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434759782 0 0) 0x7f2978003520 con 0x7f2994160620 2024-04-05T22:31:34.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29940b1870 msgr2=0x7f29940ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29940b1870 0x7f29940ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 >> v2:172.21.15.157:3300/0 conn(0x7f29940ad6b0 msgr2=0x7f29940cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 --2- 172.21.15.73:0/2390821921 >> v2:172.21.15.157:3300/0 conn(0x7f29940ad6b0 0x7f29940cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29940ac970 con 0x7f2994160620 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f2984001670 con 0x7f2994160620 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.850+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 >> v1:172.21.15.73:6789/0 conn(0x7f2994160620 legacy=0x7f29940caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2978003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29780047c0 0x7f2978004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2978005180 0x7f297800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940ce010 con 0x7f2978003970 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940ce2f0 con 0x7f2978005180 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297800bef0 con 0x7f29780047c0 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993400640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29940b1870 0x7f29940ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2978003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2999000640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2978005180 0x7f297800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993400640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29780047c0 0x7f2978004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 -- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29780047c0 msgr2=0x7f2978004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29780047c0 0x7f2978004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 -- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2978005180 msgr2=0x7f297800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2978005180 0x7f297800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.851+0000 7f2993e00640 1 -- 172.21.15.73:0/2390821921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978003520 con 0x7f2978003970 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.853+0000 7f2993e00640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2978003d90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f298400c210 tx=0x7f29780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d214ba27d543af8b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.853+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298402cae0 con 0x7f2978003970 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.853+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298402cc40 con 0x7f2978003970 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.853+0000 7f2992a00640 1 -- 172.21.15.73:0/2390821921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29840058d0 con 0x7f2978003970 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 -- 172.21.15.73:0/2390821921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 msgr2=0x7f2978003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 --2- 172.21.15.73:0/2390821921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2978003d90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f298400c210 tx=0x7f29780057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f2999000640 1 -- 172.21.15.73:0/2390821921 reap_dead start 2024-04-05T22:31:34.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 -- 172.21.15.73:0/2390821921 shutdown_connections 2024-04-05T22:31:34.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 -- 172.21.15.73:0/2390821921 >> 172.21.15.73:0/2390821921 conn(0x7f29940ce8e0 msgr2=0x7f29940cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:34.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 -- 172.21.15.73:0/2390821921 shutdown_connections 2024-04-05T22:31:34.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.854+0000 7f299b000640 1 -- 172.21.15.73:0/2390821921 wait complete. 2024-04-05T22:31:34.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.856+0000 7f299b000640 1 Processor -- start 2024-04-05T22:31:34.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.856+0000 7f299b000640 1 -- start start 2024-04-05T22:31:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f299415bb10 0x7f2994157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29941581f0 0x7f2994160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299415b6f0 con 0x7f2978003970 2024-04-05T22:31:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941471b0 con 0x7f29941581f0 2024-04-05T22:31:34.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f299b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299415aa70 con 0x7f299415bb10 2024-04-05T22:31:34.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f2993e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f2993e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51322/0 (socket says 172.21.15.73:51322) 2024-04-05T22:31:34.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.857+0000 7f2993e00640 1 -- 172.21.15.73:0/895377487 learned_addr learned my addr 172.21.15.73:0/895377487 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993400640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f299415bb10 0x7f2994157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2999000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29941581f0 0x7f2994160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 -- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f299415bb10 msgr2=0x7f2994157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f299415bb10 0x7f2994157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 -- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29941581f0 msgr2=0x7f2994160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29941581f0 0x7f2994160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 -- 172.21.15.73:0/895377487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299415ad50 con 0x7f2978003970 2024-04-05T22:31:34.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f2993e00640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f298402c480 tx=0x7f29940cb170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.858+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298403c040 con 0x7f2978003970 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.859+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2984005c80 con 0x7f2978003970 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.859+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984007070 con 0x7f2978003970 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.859+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299415b030 con 0x7f2978003970 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.859+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2994154500 con 0x7f2978003970 2024-04-05T22:31:34.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.860+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2994066af0 con 0x7f2978003970 2024-04-05T22:31:34.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.861+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f2984032070 con 0x7f2978003970 2024-04-05T22:31:34.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.862+0000 7f298be00640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 0x7f297406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:34.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.862+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29840a98e0 con 0x7f2978003970 2024-04-05T22:31:34.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.862+0000 7f2993400640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 0x7f297406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:34.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.863+0000 7f2993400640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 0x7f297406da50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f299406cf30 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:34.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:34.867+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2984076310 con 0x7f2978003970 2024-04-05T22:31:35.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.133+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f29940b1fc0 con 0x7f2978003970 2024-04-05T22:31:35.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.767+0000 7f298be00640 1 -- 172.21.15.73:0/895377487 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v560) v1 ==== 166+0+0 (secure 0 0 0) 0x7f2984079930 con 0x7f2978003970 2024-04-05T22:31:35.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 msgr2=0x7f297406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:35.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 0x7f297406da50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f299406cf30 tx=0x7f2980008040 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 msgr2=0x7f2994146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f298402c480 tx=0x7f29940cb170 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 shutdown_connections 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f297406b590 0x7f297406da50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29941581f0 0x7f2994160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f299415bb10 0x7f2994157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 --2- 172.21.15.73:0/895377487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2978003970 0x7f2994146c70 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 >> 172.21.15.73:0/895377487 conn(0x7f29940ce8e0 msgr2=0x7f299406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 shutdown_connections 2024-04-05T22:31:35.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:35.770+0000 7f299b000640 1 -- 172.21.15.73:0/895377487 wait complete. 2024-04-05T22:31:35.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:31:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.006+0000 7f826b200640 1 Processor -- start 2024-04-05T22:31:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.006+0000 7f826b200640 1 -- start start 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f826c157030 0x7f826c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f826c157410 0x7f826c160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c06d800 con 0x7f826c0cb0b0 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0b3090 con 0x7f826c157410 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0b3370 con 0x7f826c157030 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826a800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f826c0cb0b0 0x7f826c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60524/0 (socket says 172.21.15.73:60524) 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f826a800640 1 -- 172.21.15.73:0/2594376935 learned_addr learned my addr 172.21.15.73:0/2594376935 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f8269400640 1 --2- 172.21.15.73:0/2594376935 >> v2:172.21.15.73:3300/0 conn(0x7f826c157410 0x7f826c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.008+0000 7f8269e00640 1 --2- 172.21.15.73:0/2594376935 >> v2:172.21.15.157:3300/0 conn(0x7f826c157030 0x7f826c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.009+0000 7f8269400640 1 -- 172.21.15.73:0/2594376935 >> v2:172.21.15.157:3300/0 conn(0x7f826c157030 msgr2=0x7f826c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.009+0000 7f8269400640 1 --2- 172.21.15.73:0/2594376935 >> v2:172.21.15.157:3300/0 conn(0x7f826c157030 0x7f826c0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.009+0000 7f8269400640 1 -- 172.21.15.73:0/2594376935 >> v1:172.21.15.73:6789/0 conn(0x7f826c0cb0b0 legacy=0x7f826c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.009+0000 7f8269400640 1 -- 172.21.15.73:0/2594376935 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0b3650 con 0x7f826c157410 2024-04-05T22:31:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.010+0000 7f8269400640 1 --2- 172.21.15.73:0/2594376935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157410 0x7f826c160790 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ab5080e5bac042 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.010+0000 7f8268a00640 1 -- 172.21.15.73:0/2594376935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258030710 con 0x7f826c157410 2024-04-05T22:31:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.010+0000 7f8268a00640 1 -- 172.21.15.73:0/2594376935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825802e050 con 0x7f826c157410 2024-04-05T22:31:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.010+0000 7f8268a00640 1 -- 172.21.15.73:0/2594376935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258030a90 con 0x7f826c157410 2024-04-05T22:31:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 -- 172.21.15.73:0/2594376935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157410 msgr2=0x7f826c160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 --2- 172.21.15.73:0/2594376935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157410 0x7f826c160790 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b680 comp rx=0 tx=0).stop 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 -- 172.21.15.73:0/2594376935 shutdown_connections 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 --2- 172.21.15.73:0/2594376935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157410 0x7f826c160790 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 --2- 172.21.15.73:0/2594376935 >> v2:172.21.15.157:3300/0 conn(0x7f826c157030 0x7f826c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 -- 172.21.15.73:0/2594376935 >> 172.21.15.73:0/2594376935 conn(0x7f826c069cb0 msgr2=0x7f826c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 -- 172.21.15.73:0/2594376935 shutdown_connections 2024-04-05T22:31:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.011+0000 7f826b200640 1 -- 172.21.15.73:0/2594376935 wait complete. 2024-04-05T22:31:36.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.013+0000 7f826b200640 1 Processor -- start 2024-04-05T22:31:36.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.013+0000 7f826b200640 1 -- start start 2024-04-05T22:31:36.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 0x7f826c14f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c157410 0x7f826c14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0cd890 con 0x7f826c157030 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0ce1f0 con 0x7f826c0cb0b0 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f826b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0af920 con 0x7f826c157410 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46866/0 (socket says 172.21.15.73:46866) 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 -- 172.21.15.73:0/1190520350 learned_addr learned my addr 172.21.15.73:0/1190520350 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 -- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c157410 msgr2=0x7f826c14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c157410 0x7f826c14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269e00640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 0x7f826c14f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.014+0000 7f8269400640 1 -- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 msgr2=0x7f826c14f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8269400640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 0x7f826c14f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8269400640 1 -- 172.21.15.73:0/1190520350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009400 con 0x7f826c157030 2024-04-05T22:31:36.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8269e00640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 0x7f826c14f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:36.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8269400640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f82580098c0 tx=0x7f82580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825803b8d0 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825803ba50 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825803be00 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c14e9a0 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.015+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c0ccc10 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.016+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c0cd9d0 con 0x7f826c157030 2024-04-05T22:31:36.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.017+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f8258039040 con 0x7f826c157030 2024-04-05T22:31:36.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.018+0000 7f8262000640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 0x7f824c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.018+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82580a9360 con 0x7f826c157030 2024-04-05T22:31:36.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.019+0000 7f8269e00640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 0x7f824c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.020+0000 7f8269e00640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 0x7f824c06da80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.023+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f826c0cd9d0 con 0x7f826c157030 2024-04-05T22:31:36.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.215+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f826c0cd9d0 con 0x7f826c157030 2024-04-05T22:31:36.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.221+0000 7f8262000640 1 -- 172.21.15.73:0/1190520350 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f826c0cd9d0 con 0x7f826c157030 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 msgr2=0x7f824c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 0x7f824c06da80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c008040 comp rx=0 tx=0).stop 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 msgr2=0x7f826c0cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f82580098c0 tx=0x7f82580046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 shutdown_connections 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f824c06b5c0 0x7f824c06da80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c157410 0x7f826c14e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c157030 0x7f826c0cdcb0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 --2- 172.21.15.73:0/1190520350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826c0cb0b0 0x7f826c14f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 >> 172.21.15.73:0/1190520350 conn(0x7f826c069cb0 msgr2=0x7f826c06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:36.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 shutdown_connections 2024-04-05T22:31:36.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.224+0000 7f826b200640 1 -- 172.21.15.73:0/1190520350 wait complete. 2024-04-05T22:31:36.241 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:31:36.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:36.357 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:36.477 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:31:36.477 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:36.477 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:36.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:31:36.534 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:36.534 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:36.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:31:36.589 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:36.589 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:36.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:31:36.647 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:36.647 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:36.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:31:36.703 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-05T22:31:36.704 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:36.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.916+0000 7f7701200640 1 Processor -- start 2024-04-05T22:31:36.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.916+0000 7f7701200640 1 -- start start 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f7701200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 0x7f76fc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f7701200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc0ce000 0x7f76fc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f7701200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc15aa60 con 0x7f76fc06d710 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f7701200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc0b3b60 con 0x7f76fc0ce000 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f7701200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0701c0 con 0x7f76fc1541a0 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f76fa800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc0ce000 0x7f76fc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f76fa800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76fc0ce000 0x7f76fc15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46876/0 (socket says 172.21.15.73:46876) 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.918+0000 7f76fa800640 1 -- 172.21.15.73:0/3199210069 learned_addr learned my addr 172.21.15.73:0/3199210069 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76f9e00640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 0x7f76fc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76fa800640 1 -- 172.21.15.73:0/3199210069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 msgr2=0x7f76fc0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76fa800640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 0x7f76fc0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76fa800640 1 -- 172.21.15.73:0/3199210069 >> v1:172.21.15.157:6789/0 conn(0x7f76fc06d710 legacy=0x7f76fc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:36.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76fa800640 1 -- 172.21.15.73:0/3199210069 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0b3880 con 0x7f76fc0ce000 2024-04-05T22:31:36.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.919+0000 7f76f9e00640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 0x7f76fc0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.920+0000 7f76fa800640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc0ce000 0x7f76fc15d860 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009e80 tx=0x7f76f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a742cd0d8aef6b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.920+0000 7f76f8a00640 1 -- 172.21.15.73:0/3199210069 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f402ca50 con 0x7f76fc0ce000 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.920+0000 7f76f8a00640 1 -- 172.21.15.73:0/3199210069 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f402cbb0 con 0x7f76fc0ce000 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.920+0000 7f76f8a00640 1 -- 172.21.15.73:0/3199210069 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40343f0 con 0x7f76fc0ce000 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 -- 172.21.15.73:0/3199210069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc0ce000 msgr2=0x7f76fc15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc0ce000 0x7f76fc15d860 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009e80 tx=0x7f76f402beb0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 -- 172.21.15.73:0/3199210069 shutdown_connections 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc0ce000 0x7f76fc15d860 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 --2- 172.21.15.73:0/3199210069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc1541a0 0x7f76fc0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 -- 172.21.15.73:0/3199210069 >> 172.21.15.73:0/3199210069 conn(0x7f76fc0ce8e0 msgr2=0x7f76fc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 -- 172.21.15.73:0/3199210069 shutdown_connections 2024-04-05T22:31:36.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.921+0000 7f7701200640 1 -- 172.21.15.73:0/3199210069 wait complete. 2024-04-05T22:31:36.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.923+0000 7f7701200640 1 Processor -- start 2024-04-05T22:31:36.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.923+0000 7f7701200640 1 -- start start 2024-04-05T22:31:36.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 0x7f76fc0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc0ce000 0x7f76fc0a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0aada0 con 0x7f76fc1541a0 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0acc10 con 0x7f76fc06d710 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f7701200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0aa120 con 0x7f76fc0ce000 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f76fa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f76fa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46880/0 (socket says 172.21.15.73:46880) 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.924+0000 7f76fa800640 1 -- 172.21.15.73:0/955906901 learned_addr learned my addr 172.21.15.73:0/955906901 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f9400640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc0ce000 0x7f76fc0a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f9e00640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 0x7f76fc0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 -- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc0ce000 msgr2=0x7f76fc0a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc0ce000 0x7f76fc0a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 -- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 msgr2=0x7f76fc0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 0x7f76fc0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 -- 172.21.15.73:0/955906901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4009b30 con 0x7f76fc1541a0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f9e00640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 0x7f76fc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76fa800640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f76f402c4c0 tx=0x7f76f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4009510 con 0x7f76fc1541a0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f403e450 con 0x7f76fc1541a0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f403e7e0 con 0x7f76fc1541a0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.925+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc0aa400 con 0x7f76fc1541a0 2024-04-05T22:31:36.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.926+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc0aa9c0 con 0x7f76fc1541a0 2024-04-05T22:31:36.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.927+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc066af0 con 0x7f76fc1541a0 2024-04-05T22:31:36.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.927+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f76f403e940 con 0x7f76fc1541a0 2024-04-05T22:31:36.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.929+0000 7f76f2000640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 0x7f76dc06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:36.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.929+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f76f40aafa0 con 0x7f76fc1541a0 2024-04-05T22:31:36.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.930+0000 7f76f9e00640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 0x7f76dc06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.931+0000 7f76f9e00640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 0x7f76dc06da50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006fd0 tx=0x7f76ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:36.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:36.935+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f76f40779d0 con 0x7f76fc1541a0 2024-04-05T22:31:37.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.148+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76fc0aaee0 con 0x7f76fc1541a0 2024-04-05T22:31:37.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.149+0000 7f76f2000640 1 -- 172.21.15.73:0/955906901 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f76f4043df0 con 0x7f76fc1541a0 2024-04-05T22:31:37.150 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:37.150 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2480178961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2480178961},{"type":"v1","addr":"172.21.15.157:6835","nonce":2480178961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3374798984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3374798984},{"type":"v1","addr":"172.21.15.157:6833","nonce":3374798984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/4049936428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":4049936428},{"type":"v1","addr":"172.21.15.73:6834","nonce":4049936428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":9741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1968345167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":1968345167},{"type":"v1","addr":"172.21.15.73:6835","nonce":1968345167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[]} 2024-04-05T22:31:37.150 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 103 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.152+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 msgr2=0x7f76dc06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.152+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 0x7f76dc06da50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006fd0 tx=0x7f76ec008040 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.152+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 msgr2=0x7f76fc0a9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.152+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f76f402c4c0 tx=0x7f76f4004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 shutdown_connections 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f76dc06b590 0x7f76dc06da50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76fc1541a0 0x7f76fc0a9a50 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76fc0ce000 0x7f76fc0a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 --2- 172.21.15.73:0/955906901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76fc06d710 0x7f76fc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 >> 172.21.15.73:0/955906901 conn(0x7f76fc0ce8e0 msgr2=0x7f76fc14b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 shutdown_connections 2024-04-05T22:31:37.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.153+0000 7f7701200640 1 -- 172.21.15.73:0/955906901 wait complete. 2024-04-05T22:31:37.171 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-05T22:31:37.172 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-05T22:31:37.172 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:31:37.172 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:31:37.172 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-05T22:31:37.172 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-05T22:31:37.173 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:31:37.173 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:31:37.173 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-05T22:31:37.173 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-05T22:31:37.173 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:31:37.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:31:37.174 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-05T22:31:37.174 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-05T22:31:37.174 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:31:37.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:31:37.187 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:31:37.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-05T22:31:37.189 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:31:37.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-05T22:31:37.195 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:31:37.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-05T22:31:37.198 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:31:37.198 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-05T22:31:37.233 INFO:tasks.ceph.mds.a:Started 2024-04-05T22:31:37.233 DEBUG:teuthology.parallel:result is None 2024-04-05T22:31:37.239 INFO:tasks.ceph.mds.c:Started 2024-04-05T22:31:37.240 DEBUG:teuthology.parallel:result is None 2024-04-05T22:31:37.246 INFO:tasks.ceph.mds.d:Started 2024-04-05T22:31:37.246 DEBUG:teuthology.parallel:result is None 2024-04-05T22:31:37.252 INFO:tasks.ceph.mds.b:Started 2024-04-05T22:31:37.252 DEBUG:teuthology.parallel:result is None 2024-04-05T22:31:37.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-05T22:31:37.385 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:31:37.386 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-05T22:31:37.416 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-05T22:31:37.420 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-05T22:31:37.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.440+0000 7f78b9200640 1 Processor -- start 2024-04-05T22:31:37.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.440+0000 7f78b9200640 1 -- start start 2024-04-05T22:31:37.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.441+0000 7f78b9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 0x7f78b40cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:37.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.441+0000 7f78b9200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40b1b60 con 0x7f78b406c2d0 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.441+0000 7f78b9200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40b1e40 con 0x7f78b40b2c00 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.441+0000 7f78b9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40b1950 con 0x7f78b41441e0 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78b406c2d0 0x7f78b40cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60538/0 (socket says 172.21.15.73:60538) 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3e00640 1 -- 172.21.15.73:0/157762816 learned_addr learned my addr 172.21.15.73:0/157762816 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3400640 1 --2- 172.21.15.73:0/157762816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 0x7f78b40cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564429393 0 0) 0x7f78b40b1e40 con 0x7f78b40b2c00 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7898003520 con 0x7f78b40b2c00 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638962855 0 0) 0x7f78b40b1b60 con 0x7f78b406c2d0 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b40b1e40 con 0x7f78b406c2d0 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3400640 1 -- 172.21.15.73:0/157762816 >> v1:172.21.15.157:6789/0 conn(0x7f78b40b2c00 legacy=0x7f78b406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3400640 1 -- 172.21.15.73:0/157762816 >> v1:172.21.15.73:6789/0 conn(0x7f78b406c2d0 legacy=0x7f78b40cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.442+0000 7f78b3400640 1 -- 172.21.15.73:0/157762816 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40b2120 con 0x7f78b41441e0 2024-04-05T22:31:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.443+0000 7f78b3400640 1 --2- 172.21.15.73:0/157762816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 0x7f78b40cc050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78a40097d0 tx=0x7f78a402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=375511cb06b2c1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4031890 con 0x7f78b41441e0 2024-04-05T22:31:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40319f0 con 0x7f78b41441e0 2024-04-05T22:31:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b2000640 1 -- 172.21.15.73:0/157762816 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4031d80 con 0x7f78b41441e0 2024-04-05T22:31:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 -- 172.21.15.73:0/157762816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 msgr2=0x7f78b40cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 --2- 172.21.15.73:0/157762816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 0x7f78b40cc050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78a40097d0 tx=0x7f78a402b930 comp rx=0 tx=0).stop 2024-04-05T22:31:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 -- 172.21.15.73:0/157762816 shutdown_connections 2024-04-05T22:31:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 --2- 172.21.15.73:0/157762816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41441e0 0x7f78b40cc050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 -- 172.21.15.73:0/157762816 >> 172.21.15.73:0/157762816 conn(0x7f78b40b0830 msgr2=0x7f78b40b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:37.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 -- 172.21.15.73:0/157762816 shutdown_connections 2024-04-05T22:31:37.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.444+0000 7f78b9200640 1 -- 172.21.15.73:0/157762816 wait complete. 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.446+0000 7f78b9200640 1 Processor -- start 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.446+0000 7f78b9200640 1 -- start start 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 0x7f78b41433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b40b2c00 0x7f78b40ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406f5b0 con 0x7f78b41441e0 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406f800 con 0x7f78b406c2d0 2024-04-05T22:31:37.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406fae0 con 0x7f78b40b2c00 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46920/0 (socket says 172.21.15.73:46920) 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b3e00640 1 -- 172.21.15.73:0/3465239408 learned_addr learned my addr 172.21.15.73:0/3465239408 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b3400640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 0x7f78b41433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.447+0000 7f78b2a00640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b40b2c00 0x7f78b40ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 -- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b40b2c00 msgr2=0x7f78b40ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b40b2c00 0x7f78b40ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 -- 172.21.15.73:0/3465239408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 msgr2=0x7f78b40aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 -- 172.21.15.73:0/3465239408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009400 con 0x7f78b406c2d0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3e00640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78b3400640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 0x7f78b41433d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f78a4033000 tx=0x7f78a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.448+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a403ac60 con 0x7f78b406c2d0 2024-04-05T22:31:37.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.449+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a403ae00 con 0x7f78b406c2d0 2024-04-05T22:31:37.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.449+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4044630 con 0x7f78b406c2d0 2024-04-05T22:31:37.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.449+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b406fdc0 con 0x7f78b406c2d0 2024-04-05T22:31:37.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.449+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b4070380 con 0x7f78b406c2d0 2024-04-05T22:31:37.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.450+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4070510 con 0x7f78b406c2d0 2024-04-05T22:31:37.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.450+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f78a404d450 con 0x7f78b406c2d0 2024-04-05T22:31:37.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.451+0000 7f78ab400640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 0x7f789406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:37.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.451+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78a40aa220 con 0x7f78b406c2d0 2024-04-05T22:31:37.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.452+0000 7f78b2a00640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 0x7f789406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:37.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.453+0000 7f78b2a00640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 0x7f789406da50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78a0007fd0 tx=0x7f78a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:37.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.458+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f78a4076d00 con 0x7f78b406c2d0 2024-04-05T22:31:37.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:37.665+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f78b40b26c0 con 0x7f78b406c2d0 2024-04-05T22:31:38.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.447+0000 7f78ab400640 1 -- 172.21.15.73:0/3465239408 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v168) v1 ==== 97+0+0 (secure 0 0 0) 0x7f78b40b26c0 con 0x7f78b406c2d0 2024-04-05T22:31:38.448 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-05T22:31:38.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 msgr2=0x7f789406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 0x7f789406da50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78a0007fd0 tx=0x7f78a0009040 comp rx=0 tx=0).stop 2024-04-05T22:31:38.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 msgr2=0x7f78b41433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 0x7f78b41433d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f78a4033000 tx=0x7f78a40046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 shutdown_connections 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f789406b590 0x7f789406da50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78b41441e0 0x7f78b40aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b40b2c00 0x7f78b40ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 --2- 172.21.15.73:0/3465239408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b406c2d0 0x7f78b41433d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 >> 172.21.15.73:0/3465239408 conn(0x7f78b40b0830 msgr2=0x7f78b40cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.451+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 shutdown_connections 2024-04-05T22:31:38.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.452+0000 7f78b9200640 1 -- 172.21.15.73:0/3465239408 wait complete. 2024-04-05T22:31:38.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-05T22:31:38.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.685+0000 7f5929000640 1 Processor -- start 2024-04-05T22:31:38.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.686+0000 7f5929000640 1 -- start start 2024-04-05T22:31:38.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.687+0000 7f5929000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f5924162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.687+0000 7f5929000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5924164e50 0x7f59240b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.687+0000 7f5929000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5924165cf0 con 0x7f59240b2ba0 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f5929000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5924158790 con 0x7f5924164e50 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f5929000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5924165730 con 0x7f5924161f90 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f591be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5924164e50 0x7f59240b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f5921e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f5924162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f5921e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f5924162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:42522/0 (socket says 172.21.15.73:42522) 2024-04-05T22:31:38.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.688+0000 7f5921e00640 1 -- 172.21.15.73:0/2347523938 learned_addr learned my addr 172.21.15.73:0/2347523938 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:38.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.689+0000 7f591be00640 1 -- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 msgr2=0x7f5924162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:38.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.689+0000 7f591be00640 1 --2- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f5924162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.689+0000 7f591be00640 1 -- 172.21.15.73:0/2347523938 >> v1:172.21.15.157:6789/0 conn(0x7f59240b2ba0 legacy=0x7f5924070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.689+0000 7f591be00640 1 -- 172.21.15.73:0/2347523938 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924165a10 con 0x7f5924164e50 2024-04-05T22:31:38.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.691+0000 7f591be00640 1 --2- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59240b2110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f590c0097a0 tx=0x7f590c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3451497db81b7163 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:38.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.691+0000 7f5921400640 1 -- 172.21.15.73:0/2347523938 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c02d040 con 0x7f5924164e50 2024-04-05T22:31:38.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.691+0000 7f5921400640 1 -- 172.21.15.73:0/2347523938 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c031810 con 0x7f5924164e50 2024-04-05T22:31:38.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.691+0000 7f5921400640 1 -- 172.21.15.73:0/2347523938 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c031ba0 con 0x7f5924164e50 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 -- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 msgr2=0x7f59240b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 --2- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59240b2110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f590c0097a0 tx=0x7f590c02b8b0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 -- 172.21.15.73:0/2347523938 shutdown_connections 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 --2- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59240b2110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 --2- 172.21.15.73:0/2347523938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f5924162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 -- 172.21.15.73:0/2347523938 >> 172.21.15.73:0/2347523938 conn(0x7f59240ce8e0 msgr2=0x7f59240cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 -- 172.21.15.73:0/2347523938 shutdown_connections 2024-04-05T22:31:38.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.692+0000 7f5929000640 1 -- 172.21.15.73:0/2347523938 wait complete. 2024-04-05T22:31:38.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.694+0000 7f5929000640 1 Processor -- start 2024-04-05T22:31:38.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.694+0000 7f5929000640 1 -- start start 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f59241524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59241f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59241f6800 con 0x7f59240b2ba0 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924151500 con 0x7f5924164e50 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5929000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592414d880 con 0x7f5924161f90 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46922/0 (socket says 172.21.15.73:46922) 2024-04-05T22:31:38.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 -- 172.21.15.73:0/523569296 learned_addr learned my addr 172.21.15.73:0/523569296 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f591be00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f59241524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5922800640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59241f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 -- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 msgr2=0x7f59241524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f59241524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 -- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 msgr2=0x7f59241f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59241f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.695+0000 7f5921e00640 1 -- 172.21.15.73:0/523569296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009400 con 0x7f59240b2ba0 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f5921e00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f5914009fd0 tx=0x7f591400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:38.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400c940 con 0x7f59240b2ba0 2024-04-05T22:31:38.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400caa0 con 0x7f59240b2ba0 2024-04-05T22:31:38.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400ce30 con 0x7f59240b2ba0 2024-04-05T22:31:38.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592414db60 con 0x7f59240b2ba0 2024-04-05T22:31:38.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.696+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59241517e0 con 0x7f59240b2ba0 2024-04-05T22:31:38.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.697+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5924066af0 con 0x7f59240b2ba0 2024-04-05T22:31:38.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.698+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f5914013470 con 0x7f59240b2ba0 2024-04-05T22:31:38.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.699+0000 7f591a000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 0x7f590006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:38.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.699+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5914087fb0 con 0x7f59240b2ba0 2024-04-05T22:31:38.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.700+0000 7f591be00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 0x7f590006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:38.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.701+0000 7f591be00640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 0x7f590006da50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f590c036000 tx=0x7f590c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:38.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.704+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f59140549e0 con 0x7f59240b2ba0 2024-04-05T22:31:38.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.971+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5924068160 con 0x7f59240b2ba0 2024-04-05T22:31:38.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.972+0000 7f591a000640 1 -- 172.21.15.73:0/523569296 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5914058000 con 0x7f59240b2ba0 2024-04-05T22:31:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:38.972 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:31:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmMOF5FhAAzttH/apBpd3BkNgc+VZulw==", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmZ0lUNxAA6Lod+ihsLueCLf7S9AGJYw==", 2024-04-05T22:31:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmXqw0JhAA9ZuG/OZEYLbplqHFuPwCAA==", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmEslfDBAAdJdEgQp3Is/3Hb4y11RPFg==", 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAgeRBm5gioMhAAAh4tbliS5YXg6QIV9I5VRA==", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAkeRBmpT9GEBAAtR3HAThsQbqOu58LiBIaeA==", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAneRBmn6ieJxAAJl8CXKhu4Z+hdp9qF6tXtA==", 2024-04-05T22:31:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAreRBmDSwmBBAAWvorRj+QozwremjGe6VV5g==", 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAveRBm050aKxAAhgWYbMGvvOsaeJXE1tY3dw==", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAyeRBmXMsnNRAAQDmLomvrN15LCm8RauIJ7g==", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA2eRBmMuW8EhAAQbhzW1r1byGs4E+DVtz1vQ==", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQA5eRBmvo4SKxAAuDso0r7wQslDZs9duHkNeQ==", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmIvMJGhAAp2ComztZb7+a/EHyuE3Xjw==", 2024-04-05T22:31:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmZ/DZLxAABf+tJ82vPG3D/U2pDOYqLw==", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:31:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmKwxUJBAAnynd2jGctWQRGAkwmDl85g==", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAceRBmI21ZOxAAiCBevBt0J/SbI7lb1zw5DA==", 2024-04-05T22:31:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmRAsXCRAAsUmlqVURHEcK5WqbMwd0Ww==", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-05T22:31:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAbeRBmBmPcChAA5NIChI1xnaC8qeTj4PU9vA==", 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQAaeRBmZhOhOhAAOwqwR6cFv0/mni/MtIau+g==", 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-05T22:31:38.986 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:31:38.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.974+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 msgr2=0x7f590006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.974+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 0x7f590006da50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f590c036000 tx=0x7f590c03c040 comp rx=0 tx=0).stop 2024-04-05T22:31:38.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.974+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 msgr2=0x7f592414e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.974+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f5914009fd0 tx=0x7f591400ed40 comp rx=0 tx=0).stop 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 shutdown_connections 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f590006b590 0x7f590006da50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5924164e50 0x7f59241f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5924161f90 0x7f59241524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 --2- 172.21.15.73:0/523569296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59240b2ba0 0x7f592414e360 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 >> 172.21.15.73:0/523569296 conn(0x7f59240ce8e0 msgr2=0x7f592406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 shutdown_connections 2024-04-05T22:31:38.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:38.975+0000 7f5929000640 1 -- 172.21.15.73:0/523569296 wait complete. 2024-04-05T22:31:38.994 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:39.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.189+0000 7fb48d000640 1 Processor -- start 2024-04-05T22:31:39.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.190+0000 7fb48d000640 1 -- start start 2024-04-05T22:31:39.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb48d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb48806cb80 0x7fb48806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb48d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb48d000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48806f5b0 con 0x7fb4881587d0 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb48d000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48806d500 con 0x7fb48806cb80 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb48d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48815f220 con 0x7fb48815e850 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb47fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb48806cb80 0x7fb48806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb485e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb4881587d0 0x7fb4880cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60554/0 (socket says 172.21.15.73:60554) 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.191+0000 7fb485e00640 1 -- 172.21.15.73:0/3020054309 learned_addr learned my addr 172.21.15.73:0/3020054309 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:39.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.192+0000 7fb47fe00640 1 -- 172.21.15.73:0/3020054309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 msgr2=0x7fb48815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:39.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.192+0000 7fb47fe00640 1 --2- 172.21.15.73:0/3020054309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.192+0000 7fb47fe00640 1 -- 172.21.15.73:0/3020054309 >> v1:172.21.15.73:6789/0 conn(0x7fb4881587d0 legacy=0x7fb4880cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.192+0000 7fb47fe00640 1 -- 172.21.15.73:0/3020054309 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb488070410 con 0x7fb48806cb80 2024-04-05T22:31:39.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.192+0000 7fb485400640 1 -- 172.21.15.73:0/3020054309 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125443888 0 0) 0x7fb48806f5b0 con 0x7fb4881587d0 2024-04-05T22:31:39.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.193+0000 7fb47fe00640 1 --2- 172.21.15.73:0/3020054309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb48806cfc0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb4700097d0 tx=0x7fb47002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f992b8123be9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.193+0000 7fb485400640 1 -- 172.21.15.73:0/3020054309 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470031800 con 0x7fb48806cb80 2024-04-05T22:31:39.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.193+0000 7fb485400640 1 -- 172.21.15.73:0/3020054309 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb470031960 con 0x7fb48806cb80 2024-04-05T22:31:39.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.193+0000 7fb485400640 1 -- 172.21.15.73:0/3020054309 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470031cf0 con 0x7fb48806cb80 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 -- 172.21.15.73:0/3020054309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 msgr2=0x7fb48806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 --2- 172.21.15.73:0/3020054309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb48806cfc0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb4700097d0 tx=0x7fb47002b8a0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 -- 172.21.15.73:0/3020054309 shutdown_connections 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 --2- 172.21.15.73:0/3020054309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 --2- 172.21.15.73:0/3020054309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb48806cfc0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 -- 172.21.15.73:0/3020054309 >> 172.21.15.73:0/3020054309 conn(0x7fb4880ce8e0 msgr2=0x7fb4880cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 -- 172.21.15.73:0/3020054309 shutdown_connections 2024-04-05T22:31:39.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.194+0000 7fb48d000640 1 -- 172.21.15.73:0/3020054309 wait complete. 2024-04-05T22:31:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.196+0000 7fb48d000640 1 Processor -- start 2024-04-05T22:31:39.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.196+0000 7fb48d000640 1 -- start start 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4881587d0 0x7fb4881f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488145f50 con 0x7fb48806cb80 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488146230 con 0x7fb4881587d0 2024-04-05T22:31:39.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb48d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881f41e0 con 0x7fb48815e850 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb485e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb485e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46948/0 (socket says 172.21.15.73:46948) 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.197+0000 7fb485e00640 1 -- 172.21.15.73:0/4003690362 learned_addr learned my addr 172.21.15.73:0/4003690362 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 -- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 msgr2=0x7fb48815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb47fe00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4881587d0 0x7fb4881f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 -- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4881587d0 msgr2=0x7fb4881f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4881587d0 0x7fb4881f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 -- 172.21.15.73:0/4003690362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470009400 con 0x7fb48806cb80 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb485e00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fb47800c4d0 tx=0x7fb47800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780074e0 con 0x7fb48806cb80 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478007660 con 0x7fb48806cb80 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478004a20 con 0x7fb48806cb80 2024-04-05T22:31:39.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4881f4440 con 0x7fb48806cb80 2024-04-05T22:31:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.198+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4881f4a00 con 0x7fb48806cb80 2024-04-05T22:31:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.199+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb488066af0 con 0x7fb48806cb80 2024-04-05T22:31:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.200+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fb478004b80 con 0x7fb48806cb80 2024-04-05T22:31:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.201+0000 7fb47e000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 0x7fb46406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.201+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb478087050 con 0x7fb48806cb80 2024-04-05T22:31:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.202+0000 7fb47fe00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 0x7fb46406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.203+0000 7fb47fe00640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 0x7fb46406da50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb470036000 tx=0x7fb470008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.206+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb478053a80 con 0x7fb48806cb80 2024-04-05T22:31:39.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.385+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb488068160 con 0x7fb48806cb80 2024-04-05T22:31:39.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.386+0000 7fb47e000640 1 -- 172.21.15.73:0/4003690362 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+12399 (secure 0 0 0) 0x7fb478018090 con 0x7fb48806cb80 2024-04-05T22:31:39.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:39.387 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":168,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:38.438415+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 msgr2=0x7fb46406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 0x7fb46406da50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb470036000 tx=0x7fb470008030 comp rx=0 tx=0).stop 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 msgr2=0x7fb4880ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fb47800c4d0 tx=0x7fb47800c990 comp rx=0 tx=0).stop 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 shutdown_connections 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb46406b590 0x7fb46406da50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48815e850 0x7fb48815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4881587d0 0x7fb4881f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 --2- 172.21.15.73:0/4003690362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb48806cb80 0x7fb4880ab500 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 >> 172.21.15.73:0/4003690362 conn(0x7fb4880ce8e0 msgr2=0x7fb4880cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:39.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 shutdown_connections 2024-04-05T22:31:39.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.389+0000 7fb48d000640 1 -- 172.21.15.73:0/4003690362 wait complete. 2024-04-05T22:31:39.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:39.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.606+0000 7f77bc200640 1 Processor -- start 2024-04-05T22:31:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.607+0000 7f77bc200640 1 -- start start 2024-04-05T22:31:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.608+0000 7f77bc200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f77b4070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.608+0000 7f77bc200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b40ae190 con 0x7f77b40add00 2024-04-05T22:31:39.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.608+0000 7f77bc200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b40b27f0 con 0x7f77b41599f0 2024-04-05T22:31:39.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.608+0000 7f77bc200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b3890 con 0x7f77b406cd50 2024-04-05T22:31:39.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77ba200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f77b40add00 0x7f77b4159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60568/0 (socket says 172.21.15.73:60568) 2024-04-05T22:31:39.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77ba200640 1 -- 172.21.15.73:0/2372386558 learned_addr learned my addr 172.21.15.73:0/2372386558 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b8e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f77b4070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531399338 0 0) 0x7f77b40ae190 con 0x7f77b40add00 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f779c005730 con 0x7f77b40add00 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 188540829 0 0) 0x7f779c005730 con 0x7f77b40add00 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 msgr2=0x7f77b4070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f77b4070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 >> v1:172.21.15.157:6789/0 conn(0x7f77b41599f0 legacy=0x7f77b406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40b2ad0 con 0x7f77b40add00 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.609+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f77ac0019f0 con 0x7f77b40add00 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 >> v1:172.21.15.73:6789/0 conn(0x7f77b40add00 legacy=0x7f77b4159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f779c005bb0 0x7f779c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f779c0069d0 0x7f779c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f779c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b3890 con 0x7f779c0069d0 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c00dd20 con 0x7f779c005bb0 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c00dee0 con 0x7f77b406cd50 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b8e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f779c0069d0 0x7f779c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b9800640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f779c005bb0 0x7f779c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77ba200640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f779c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b8e00640 1 -- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 msgr2=0x7f779c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:39.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b8e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b406cd50 0x7f779c0073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b8e00640 1 -- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f779c005bb0 msgr2=0x7f779c005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:39.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.610+0000 7f77b8e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f779c005bb0 0x7f779c005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.611+0000 7f77b8e00640 1 -- 172.21.15.73:0/2372386558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c005730 con 0x7f779c0069d0 2024-04-05T22:31:39.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.611+0000 7f77b8e00640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f779c0069d0 0x7f779c006e50 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ba30 tx=0x7f779c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f93036f7b41c0d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032660 con 0x7f779c0069d0 2024-04-05T22:31:39.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a00327c0 con 0x7f779c0069d0 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77b3e00640 1 -- 172.21.15.73:0/2372386558 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032b50 con 0x7f779c0069d0 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 -- 172.21.15.73:0/2372386558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f779c0069d0 msgr2=0x7f779c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 --2- 172.21.15.73:0/2372386558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f779c0069d0 0x7f779c006e50 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ba30 tx=0x7f779c007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77ba200640 1 -- 172.21.15.73:0/2372386558 reap_dead start 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 -- 172.21.15.73:0/2372386558 shutdown_connections 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 -- 172.21.15.73:0/2372386558 >> 172.21.15.73:0/2372386558 conn(0x7f77b4069cb0 msgr2=0x7f77b406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 -- 172.21.15.73:0/2372386558 shutdown_connections 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.612+0000 7f77bc200640 1 -- 172.21.15.73:0/2372386558 wait complete. 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.614+0000 7f77bc200640 1 Processor -- start 2024-04-05T22:31:39.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.614+0000 7f77bc200640 1 -- start start 2024-04-05T22:31:39.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f779c0069d0 0x7f77b4148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b4148d30 0x7f77b40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4148940 con 0x7f77b40a8740 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40a8d00 con 0x7f77b4148d30 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77bc200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40a8fe0 con 0x7f779c0069d0 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77b8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77b8e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46972/0 (socket says 172.21.15.73:46972) 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.615+0000 7f77b8e00640 1 -- 172.21.15.73:0/4188197357 learned_addr learned my addr 172.21.15.73:0/4188197357 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b9800640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f779c0069d0 0x7f77b4148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77ba200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b4148d30 0x7f77b40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 -- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f779c0069d0 msgr2=0x7f77b4148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f779c0069d0 0x7f77b4148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 -- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b4148d30 msgr2=0x7f77b40aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b4148d30 0x7f77b40aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 -- 172.21.15.73:0/4188197357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40a92c0 con 0x7f77b40a8740 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b8e00640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f77a002bee0 tx=0x7f77a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0004220 con 0x7f77b40a8740 2024-04-05T22:31:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0004380 con 0x7f77b40a8740 2024-04-05T22:31:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0030a80 con 0x7f77b40a8740 2024-04-05T22:31:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.616+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b40a95a0 con 0x7f77b40a8740 2024-04-05T22:31:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.617+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40ad580 con 0x7f77b40a8740 2024-04-05T22:31:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.618+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b4068160 con 0x7f77b40a8740 2024-04-05T22:31:39.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.618+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f77a0030be0 con 0x7f77b40a8740 2024-04-05T22:31:39.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.619+0000 7f77b1600640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 0x7f779406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:39.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.619+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77a00aa9f0 con 0x7f77b40a8740 2024-04-05T22:31:39.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.620+0000 7f77b9800640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 0x7f779406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:39.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.621+0000 7f77b9800640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 0x7f779406da50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f77a8007fd0 tx=0x7f77a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:39.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.624+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f77a0077540 con 0x7f77b40a8740 2024-04-05T22:31:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.815+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77b4068160 con 0x7f77b40a8740 2024-04-05T22:31:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.816+0000 7f77b1600640 1 -- 172.21.15.73:0/4188197357 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77a007ab60 con 0x7f77b40a8740 2024-04-05T22:31:39.817 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:39.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 msgr2=0x7f779406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 0x7f779406da50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f77a8007fd0 tx=0x7f77a8009040 comp rx=0 tx=0).stop 2024-04-05T22:31:39.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 msgr2=0x7f77b40ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:39.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f77a002bee0 tx=0x7f77a0004730 comp rx=0 tx=0).stop 2024-04-05T22:31:39.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 shutdown_connections 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b590 0x7f779406da50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b4148d30 0x7f77b40aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b40a8740 0x7f77b40ad040 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 --2- 172.21.15.73:0/4188197357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f779c0069d0 0x7f77b4148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 >> 172.21.15.73:0/4188197357 conn(0x7f77b4069cb0 msgr2=0x7f77b41585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 shutdown_connections 2024-04-05T22:31:39.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:39.819+0000 7f77bc200640 1 -- 172.21.15.73:0/4188197357 wait complete. 2024-04-05T22:31:39.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-05T22:31:40.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.038+0000 7fd08de00640 1 Processor -- start 2024-04-05T22:31:40.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.038+0000 7fd08de00640 1 -- start start 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd08de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd08815cd00 0x7fd0880d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd08de00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0880b5960 con 0x7fd0880d3730 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd08de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0880d1920 con 0x7fd0880cfbf0 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd08de00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0880b5c00 con 0x7fd08815cd00 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd087e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd0880d3730 0x7fd0880cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60576/0 (socket says 172.21.15.73:60576) 2024-04-05T22:31:40.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd087e00640 1 -- 172.21.15.73:0/4118316994 learned_addr learned my addr 172.21.15.73:0/4118316994 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:40.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.040+0000 7fd08ca00640 1 --2- 172.21.15.73:0/4118316994 >> v2:172.21.15.157:3300/0 conn(0x7fd08815cd00 0x7fd0880d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724608386 0 0) 0x7fd0880b5960 con 0x7fd0880d3730 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003520 con 0x7fd0880d3730 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285421566 0 0) 0x7fd06c003520 con 0x7fd0880d3730 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 >> v2:172.21.15.157:3300/0 conn(0x7fd08815cd00 msgr2=0x7fd0880d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 --2- 172.21.15.73:0/4118316994 >> v2:172.21.15.157:3300/0 conn(0x7fd08815cd00 0x7fd0880d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 >> v1:172.21.15.157:6789/0 conn(0x7fd0880cfbf0 legacy=0x7fd0880b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880b5ee0 con 0x7fd0880d3730 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fd074001670 con 0x7fd0880d3730 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 >> v1:172.21.15.73:6789/0 conn(0x7fd0880d3730 legacy=0x7fd0880cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06c005180 0x7fd06c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880b5c00 con 0x7fd06c0047c0 2024-04-05T22:31:40.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00c120 con 0x7fd06c003970 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00c2e0 con 0x7fd06c005180 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.041+0000 7fd087e00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd08d400640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06c005180 0x7fd06c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd08ca00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd087e00640 1 -- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06c005180 msgr2=0x7fd06c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd087e00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06c005180 0x7fd06c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd087e00640 1 -- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06c003970 msgr2=0x7fd06c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd087e00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd087e00640 1 -- 172.21.15.73:0/4118316994 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c003520 con 0x7fd06c0047c0 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd08d400640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06c005180 0x7fd06c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.042+0000 7fd08d400640 1 -- 172.21.15.73:0/4118316994 reap_dead start 2024-04-05T22:31:40.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.043+0000 7fd087e00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c240 tx=0x7fd06c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feaa4464be5ff75 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.043+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074005ad0 con 0x7fd06c0047c0 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.043+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074005c30 con 0x7fd06c0047c0 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.043+0000 7fd087400640 1 -- 172.21.15.73:0/4118316994 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07402caf0 con 0x7fd06c0047c0 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 -- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 msgr2=0x7fd06c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c240 tx=0x7fd06c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 -- 172.21.15.73:0/4118316994 shutdown_connections 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 --2- 172.21.15.73:0/4118316994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 -- 172.21.15.73:0/4118316994 >> 172.21.15.73:0/4118316994 conn(0x7fd0880d2b10 msgr2=0x7fd0880d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 -- 172.21.15.73:0/4118316994 shutdown_connections 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.044+0000 7fd08de00640 1 -- 172.21.15.73:0/4118316994 wait complete. 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.046+0000 7fd08de00640 1 Processor -- start 2024-04-05T22:31:40.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.046+0000 7fd08de00640 1 -- start start 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.046+0000 7fd08de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08814fb40 0x7fd08814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0881f2740 0x7fd0880cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815f3c0 con 0x7fd06c0047c0 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815e7f0 con 0x7fd0881f2740 2024-04-05T22:31:40.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815ea90 con 0x7fd08814fb40 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46996/0 (socket says 172.21.15.73:46996) 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 -- 172.21.15.73:0/2330850537 learned_addr learned my addr 172.21.15.73:0/2330850537 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 -- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08814fb40 msgr2=0x7fd08814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08814fb40 0x7fd08814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 -- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0881f2740 msgr2=0x7fd0880cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0881f2740 0x7fd0880cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 -- 172.21.15.73:0/2330850537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08815ed70 con 0x7fd06c0047c0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.047+0000 7fd08ca00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd07800f3d0 tx=0x7fd07800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.048+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078016620 con 0x7fd06c0047c0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.048+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0780167a0 con 0x7fd06c0047c0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.048+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078016b50 con 0x7fd06c0047c0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.048+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08815f050 con 0x7fd06c0047c0 2024-04-05T22:31:40.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.048+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0881f2bc0 con 0x7fd06c0047c0 2024-04-05T22:31:40.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.049+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08804e160 con 0x7fd06c0047c0 2024-04-05T22:31:40.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.050+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fd078016cb0 con 0x7fd06c0047c0 2024-04-05T22:31:40.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.050+0000 7fd084c00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 0x7fd06806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.051+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd078088dd0 con 0x7fd06c0047c0 2024-04-05T22:31:40.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.051+0000 7fd087e00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 0x7fd06806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.052+0000 7fd087e00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 0x7fd06806da80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd074039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.055+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd078055860 con 0x7fd06c0047c0 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.234+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd08815c3d0 con 0x7fd06c0047c0 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.235+0000 7fd084c00640 1 -- 172.21.15.73:0/2330850537 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v168) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd078058e80 con 0x7fd06c0047c0 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-05T22:31:40.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:40.236 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:31:40.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 msgr2=0x7fd06806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 0x7fd06806da80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd074039040 comp rx=0 tx=0).stop 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 msgr2=0x7fd0880c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd07800f3d0 tx=0x7fd07800f890 comp rx=0 tx=0).stop 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 shutdown_connections 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd06806b5c0 0x7fd06806da80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0881f2740 0x7fd0880cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08814fb40 0x7fd08814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 --2- 172.21.15.73:0/2330850537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd06c0047c0 0x7fd0880c7910 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 >> 172.21.15.73:0/2330850537 conn(0x7fd0880d2b10 msgr2=0x7fd0880ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:40.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 shutdown_connections 2024-04-05T22:31:40.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.237+0000 7fd08de00640 1 -- 172.21.15.73:0/2330850537 wait complete. 2024-04-05T22:31:40.254 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-05T22:31:40.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-05T22:31:40.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:31:40.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:31:40.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.461+0000 7f5455a00640 1 Processor -- start 2024-04-05T22:31:40.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.461+0000 7f5455a00640 1 -- start start 2024-04-05T22:31:40.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5455a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 0x7f54500d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5455a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54500cd7f0 0x7f54500cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5455a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54500d1360 con 0x7f54500ce170 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5455a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545015d3d0 con 0x7f54500cd7f0 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5455a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500d1640 con 0x7f545015cff0 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5446000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54500cd7f0 0x7f54500cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5446000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54500cd7f0 0x7f54500cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47006/0 (socket says 172.21.15.73:47006) 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f5446000640 1 -- 172.21.15.73:0/2159866565 learned_addr learned my addr 172.21.15.73:0/2159866565 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.463+0000 7f544ea00640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 0x7f54500d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.464+0000 7f5446000640 1 -- 172.21.15.73:0/2159866565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 msgr2=0x7f54500d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.464+0000 7f5446000640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 0x7f54500d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.464+0000 7f5446000640 1 -- 172.21.15.73:0/2159866565 >> v1:172.21.15.157:6789/0 conn(0x7f54500ce170 legacy=0x7f54500cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.464+0000 7f5446000640 1 -- 172.21.15.73:0/2159866565 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500d1920 con 0x7f54500cd7f0 2024-04-05T22:31:40.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.464+0000 7f544ea00640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 0x7f54500d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.465+0000 7f5446000640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54500cd7f0 0x7f54500cdc30 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f5438009870 tx=0x7f543802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437ec06b5f1babc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.465+0000 7f544e000640 1 -- 172.21.15.73:0/2159866565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380317c0 con 0x7f54500cd7f0 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.465+0000 7f544e000640 1 -- 172.21.15.73:0/2159866565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438031920 con 0x7f54500cd7f0 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.465+0000 7f544e000640 1 -- 172.21.15.73:0/2159866565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438031cb0 con 0x7f54500cd7f0 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 -- 172.21.15.73:0/2159866565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54500cd7f0 msgr2=0x7f54500cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54500cd7f0 0x7f54500cdc30 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f5438009870 tx=0x7f543802b860 comp rx=0 tx=0).stop 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 -- 172.21.15.73:0/2159866565 shutdown_connections 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54500cd7f0 0x7f54500cdc30 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 --2- 172.21.15.73:0/2159866565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f545015cff0 0x7f54500d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 -- 172.21.15.73:0/2159866565 >> 172.21.15.73:0/2159866565 conn(0x7f54500d2ce0 msgr2=0x7f54500d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 -- 172.21.15.73:0/2159866565 shutdown_connections 2024-04-05T22:31:40.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.466+0000 7f5455a00640 1 -- 172.21.15.73:0/2159866565 wait complete. 2024-04-05T22:31:40.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.467+0000 7f5455a00640 1 Processor -- start 2024-04-05T22:31:40.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 -- start start 2024-04-05T22:31:40.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54500cd7f0 0x7f54501510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500ce170 0x7f545015f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545015df60 con 0x7f545015cff0 2024-04-05T22:31:40.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450151610 con 0x7f54500cd7f0 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.468+0000 7f5455a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545015e220 con 0x7f54500ce170 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47014/0 (socket says 172.21.15.73:47014) 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 -- 172.21.15.73:0/2649767263 learned_addr learned my addr 172.21.15.73:0/2649767263 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544ea00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54500cd7f0 0x7f54501510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f5446000640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500ce170 0x7f545015f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 -- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500ce170 msgr2=0x7f545015f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500ce170 0x7f545015f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 -- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54500cd7f0 msgr2=0x7f54501510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54500cd7f0 0x7f54501510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.469+0000 7f544f400640 1 -- 172.21.15.73:0/2649767263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380094d0 con 0x7f545015cff0 2024-04-05T22:31:40.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f544f400640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f544800b970 tx=0x7f544800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544800cb40 con 0x7f545015cff0 2024-04-05T22:31:40.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544800cca0 con 0x7f545015cff0 2024-04-05T22:31:40.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448014680 con 0x7f545015cff0 2024-04-05T22:31:40.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545015e500 con 0x7f545015cff0 2024-04-05T22:31:40.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.470+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545015eac0 con 0x7f545015cff0 2024-04-05T22:31:40.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.471+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54500d0c20 con 0x7f545015cff0 2024-04-05T22:31:40.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.472+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f544801d3b0 con 0x7f545015cff0 2024-04-05T22:31:40.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.473+0000 7f5447400640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 0x7f542c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:40.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.473+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5448087d00 con 0x7f545015cff0 2024-04-05T22:31:40.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.473+0000 7f544ea00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 0x7f542c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:40.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.474+0000 7f544ea00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 0x7f542c06da50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:40.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.478+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5448054730 con 0x7f545015cff0 2024-04-05T22:31:40.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.673+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5450068160 con 0x7f545015cff0 2024-04-05T22:31:40.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.810+0000 7f5447400640 1 -- 172.21.15.73:0/2649767263 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v169) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5448057d50 con 0x7f545015cff0 2024-04-05T22:31:40.810 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-05T22:31:40.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.814+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 msgr2=0x7f542c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.814+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 0x7f542c06da50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440008040 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.814+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 msgr2=0x7f54501f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.814+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f544800b970 tx=0x7f544800be30 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 shutdown_connections 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f542c06b590 0x7f542c06da50 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440008040 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f545015cff0 0x7f54501f2bf0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54500ce170 0x7f545015f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 --2- 172.21.15.73:0/2649767263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54500cd7f0 0x7f54501510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:40.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 >> 172.21.15.73:0/2649767263 conn(0x7f54500d2ce0 msgr2=0x7f545006b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:40.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.815+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 shutdown_connections 2024-04-05T22:31:40.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:40.816+0000 7f5455a00640 1 -- 172.21.15.73:0/2649767263 wait complete. 2024-04-05T22:31:40.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-05T22:31:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.050+0000 7fcb82200640 1 Processor -- start 2024-04-05T22:31:41.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.050+0000 7fcb82200640 1 -- start start 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.051+0000 7fcb82200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb7c15abe0 0x7fcb7c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.051+0000 7fcb82200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 0x7fcb7c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb82200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c06bec0 con 0x7fcb7c06ba70 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb82200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c15afc0 con 0x7fcb7c15abe0 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb82200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c06f5b0 con 0x7fcb7c06c9a0 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb7be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcb7c06ba70 0x7fcb7c0cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60588/0 (socket says 172.21.15.73:60588) 2024-04-05T22:31:41.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb7be00640 1 -- 172.21.15.73:0/4125734603 learned_addr learned my addr 172.21.15.73:0/4125734603 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb7b400640 1 --2- 172.21.15.73:0/4125734603 >> v2:172.21.15.157:3300/0 conn(0x7fcb7c15abe0 0x7fcb7c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320654108 0 0) 0x7fcb74001490 con 0x7fcb7c06ba70 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.052+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb60003520 con 0x7fcb7c06ba70 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 0x7fcb7c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010998639 0 0) 0x7fcb60003520 con 0x7fcb7c06ba70 2024-04-05T22:31:41.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 msgr2=0x7fcb7c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 0x7fcb7c06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 >> v2:172.21.15.157:3300/0 conn(0x7fcb7c15abe0 msgr2=0x7fcb7c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 --2- 172.21.15.73:0/4125734603 >> v2:172.21.15.157:3300/0 conn(0x7fcb7c15abe0 0x7fcb7c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c06f890 con 0x7fcb7c06ba70 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fcb74001b80 con 0x7fcb7c06ba70 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 >> v1:172.21.15.73:6789/0 conn(0x7fcb7c06ba70 legacy=0x7fcb7c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 0x7fcb60003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb60003ef0 0x7fcb600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60004d00 0x7fcb60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c15afc0 con 0x7fcb60003ef0 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c06f5b0 con 0x7fcb60004d00 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6000bb10 con 0x7fcb7c06c9a0 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.053+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb60003ef0 0x7fcb600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.054+0000 7fcb73e00640 1 -- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 msgr2=0x7fcb60003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:41.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.054+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb7c06c9a0 0x7fcb60003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.054+0000 7fcb73e00640 1 -- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60004d00 msgr2=0x7fcb60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:41.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.054+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60004d00 0x7fcb60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.054+0000 7fcb73e00640 1 -- 172.21.15.73:0/4125734603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60003520 con 0x7fcb60003ef0 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.055+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb60003ef0 0x7fcb600047c0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400ba30 tx=0x7fcb600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7bc2690644bada server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.055+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64032680 con 0x7fcb60003ef0 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.055+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb640327e0 con 0x7fcb60003ef0 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.055+0000 7fcb7aa00640 1 -- 172.21.15.73:0/4125734603 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64032b70 con 0x7fcb60003ef0 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 -- 172.21.15.73:0/4125734603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb60003ef0 msgr2=0x7fcb600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 --2- 172.21.15.73:0/4125734603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb60003ef0 0x7fcb600047c0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400ba30 tx=0x7fcb600057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb7be00640 1 -- 172.21.15.73:0/4125734603 reap_dead start 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 -- 172.21.15.73:0/4125734603 shutdown_connections 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 -- 172.21.15.73:0/4125734603 >> 172.21.15.73:0/4125734603 conn(0x7fcb7c069cb0 msgr2=0x7fcb7c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 -- 172.21.15.73:0/4125734603 shutdown_connections 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.056+0000 7fcb82200640 1 -- 172.21.15.73:0/4125734603 wait complete. 2024-04-05T22:31:41.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.057+0000 7fcb82200640 1 Processor -- start 2024-04-05T22:31:41.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.058+0000 7fcb82200640 1 -- start start 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.058+0000 7fcb82200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb60003ef0 0x7fcb7c147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.058+0000 7fcb82200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb7c159280 0x7fcb7c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.058+0000 7fcb82200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb82200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c147ac0 con 0x7fcb7c159700 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb82200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1581e0 con 0x7fcb7c159280 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb82200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1584c0 con 0x7fcb60003ef0 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47038/0 (socket says 172.21.15.73:47038) 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 -- 172.21.15.73:0/4227517438 learned_addr learned my addr 172.21.15.73:0/4227517438 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb73e00640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb7c159280 0x7fcb7c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7b400640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb60003ef0 0x7fcb7c147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 -- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb60003ef0 msgr2=0x7fcb7c147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb60003ef0 0x7fcb7c147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 -- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb7c159280 msgr2=0x7fcb7c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb7c159280 0x7fcb7c0b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.059+0000 7fcb7be00640 1 -- 172.21.15.73:0/4227517438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c1587a0 con 0x7fcb7c159700 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb7be00640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fcb74002230 tx=0x7fcb7400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ce70 con 0x7fcb7c159700 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7400a610 con 0x7fcb7c159700 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c158a80 con 0x7fcb7c159700 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400a9c0 con 0x7fcb7c159700 2024-04-05T22:31:41.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.060+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c148900 con 0x7fcb7c159700 2024-04-05T22:31:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.061+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c158fa0 con 0x7fcb7c159700 2024-04-05T22:31:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.063+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fcb7400ab20 con 0x7fcb7c159700 2024-04-05T22:31:41.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.064+0000 7fcb73400640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 0x7fcb5806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:41.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.064+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcb74087a20 con 0x7fcb7c159700 2024-04-05T22:31:41.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.064+0000 7fcb7b400640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 0x7fcb5806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:41.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.065+0000 7fcb7b400640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 0x7fcb5806da80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c004770 tx=0x7fcb6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:41.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.069+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fcb7c158fa0 con 0x7fcb7c159700 2024-04-05T22:31:41.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.262+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcb7c06bec0 con 0x7fcb7c159700 2024-04-05T22:31:41.825 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-05T22:31:41.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.824+0000 7fcb73400640 1 -- 172.21.15.73:0/4227517438 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v170) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcb7c06bec0 con 0x7fcb7c159700 2024-04-05T22:31:41.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.831+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 msgr2=0x7fcb5806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.831+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 0x7fcb5806da80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c004770 tx=0x7fcb6c008040 comp rx=0 tx=0).stop 2024-04-05T22:31:41.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.831+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 msgr2=0x7fcb7c0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:41.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.831+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fcb74002230 tx=0x7fcb7400cc20 comp rx=0 tx=0).stop 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 shutdown_connections 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fcb5806b5c0 0x7fcb5806da80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb7c159700 0x7fcb7c0b2ab0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb7c159280 0x7fcb7c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 --2- 172.21.15.73:0/4227517438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb60003ef0 0x7fcb7c147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:41.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.834+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 >> 172.21.15.73:0/4227517438 conn(0x7fcb7c069cb0 msgr2=0x7fcb7c0ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:41.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.836+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 shutdown_connections 2024-04-05T22:31:41.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:41.837+0000 7fcb82200640 1 -- 172.21.15.73:0/4227517438 wait complete. 2024-04-05T22:31:41.865 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-05T22:31:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.049+0000 7ff90fc00640 1 Processor -- start 2024-04-05T22:31:42.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.049+0000 7ff90fc00640 1 -- start start 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.050+0000 7ff90fc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff90806cd50 0x7ff908070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.050+0000 7ff90fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9080add00 0x7ff908159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.050+0000 7ff90fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080b27f0 con 0x7ff9081599f0 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.050+0000 7ff90fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b3890 con 0x7ff9080add00 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.050+0000 7ff90fc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b2ad0 con 0x7ff90806cd50 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9080add00 0x7ff908159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9080add00 0x7ff908159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47054/0 (socket says 172.21.15.73:47054) 2024-04-05T22:31:42.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 -- 172.21.15.73:0/2604108797 learned_addr learned my addr 172.21.15.73:0/2604108797 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 -- 172.21.15.73:0/2604108797 >> v2:172.21.15.157:3300/0 conn(0x7ff90806cd50 msgr2=0x7ff908070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff8ffe00640 1 --2- 172.21.15.73:0/2604108797 >> v2:172.21.15.157:3300/0 conn(0x7ff90806cd50 0x7ff908070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 --2- 172.21.15.73:0/2604108797 >> v2:172.21.15.157:3300/0 conn(0x7ff90806cd50 0x7ff908070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 -- 172.21.15.73:0/2604108797 >> v1:172.21.15.73:6789/0 conn(0x7ff9081599f0 legacy=0x7ff90806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff907e00640 1 -- 172.21.15.73:0/2604108797 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78783538 0 0) 0x7ff9080b27f0 con 0x7ff9081599f0 2024-04-05T22:31:42.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.051+0000 7ff90dc00640 1 -- 172.21.15.73:0/2604108797 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080b2db0 con 0x7ff9080add00 2024-04-05T22:31:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.052+0000 7ff90dc00640 1 --2- 172.21.15.73:0/2604108797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9080add00 0x7ff908159dd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff908071ad0 tx=0x7ff90002e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c162f8a78c0092f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff907e00640 1 -- 172.21.15.73:0/2604108797 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90000b0d0 con 0x7ff9080add00 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff907e00640 1 -- 172.21.15.73:0/2604108797 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000b230 con 0x7ff9080add00 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff907e00640 1 -- 172.21.15.73:0/2604108797 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900009600 con 0x7ff9080add00 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 -- 172.21.15.73:0/2604108797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9080add00 msgr2=0x7ff908159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 --2- 172.21.15.73:0/2604108797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9080add00 0x7ff908159dd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff908071ad0 tx=0x7ff90002e180 comp rx=0 tx=0).stop 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 -- 172.21.15.73:0/2604108797 shutdown_connections 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 --2- 172.21.15.73:0/2604108797 >> v2:172.21.15.157:3300/0 conn(0x7ff90806cd50 0x7ff908070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 --2- 172.21.15.73:0/2604108797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9080add00 0x7ff908159dd0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.053+0000 7ff90fc00640 1 -- 172.21.15.73:0/2604108797 >> 172.21.15.73:0/2604108797 conn(0x7ff908069cb0 msgr2=0x7ff90806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.054+0000 7ff90fc00640 1 -- 172.21.15.73:0/2604108797 shutdown_connections 2024-04-05T22:31:42.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.054+0000 7ff90fc00640 1 -- 172.21.15.73:0/2604108797 wait complete. 2024-04-05T22:31:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.055+0000 7ff90fc00640 1 Processor -- start 2024-04-05T22:31:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.055+0000 7ff90fc00640 1 -- start start 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9080add00 0x7ff908152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9081599f0 0x7ff908153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080af390 con 0x7ff90806cd50 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815a610 con 0x7ff9081599f0 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.056+0000 7ff90fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff908146900 con 0x7ff9080add00 2024-04-05T22:31:42.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47058/0 (socket says 172.21.15.73:47058) 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 -- 172.21.15.73:0/1518075138 learned_addr learned my addr 172.21.15.73:0/1518075138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 -- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9080add00 msgr2=0x7ff908152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9080add00 0x7ff908152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 -- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9081599f0 msgr2=0x7ff908153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9081599f0 0x7ff908153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 -- 172.21.15.73:0/1518075138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90000bd40 con 0x7ff90806cd50 2024-04-05T22:31:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.057+0000 7ff90d200640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c410 tx=0x7ff8f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.058+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00035f0 con 0x7ff90806cd50 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.058+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0003770 con 0x7ff90806cd50 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.058+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9080af5a0 con 0x7ff90806cd50 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.058+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0003b20 con 0x7ff90806cd50 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.058+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90815a820 con 0x7ff90806cd50 2024-04-05T22:31:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.059+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908068160 con 0x7ff90806cd50 2024-04-05T22:31:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.060+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7ff8f0003c80 con 0x7ff90806cd50 2024-04-05T22:31:42.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.061+0000 7ff905600640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 0x7ff8e406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.061+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff8f0087440 con 0x7ff90806cd50 2024-04-05T22:31:42.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.062+0000 7ff8ffe00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 0x7ff8e406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.063+0000 7ff8ffe00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 0x7ff8e406da80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006fd0 tx=0x7ff8f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.068+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff8f0053ed0 con 0x7ff90806cd50 2024-04-05T22:31:42.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.295+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff908156b90 con 0x7ff90806cd50 2024-04-05T22:31:42.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.309+0000 7ff905600640 1 -- 172.21.15.73:0/1518075138 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v105) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff8f0017020 con 0x7ff90806cd50 2024-04-05T22:31:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 26 and data pool 27 2024-04-05T22:31:42.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.313+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 msgr2=0x7ff8e406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.313+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 0x7ff8e406da80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006fd0 tx=0x7ff8f8008040 comp rx=0 tx=0).stop 2024-04-05T22:31:42.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.313+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 msgr2=0x7ff9081463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.313+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c410 tx=0x7ff8f000c8d0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 shutdown_connections 2024-04-05T22:31:42.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff8e406b5c0 0x7ff8e406da80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9081599f0 0x7ff908153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9080add00 0x7ff908152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 --2- 172.21.15.73:0/1518075138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff90806cd50 0x7ff9081463c0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.314+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 >> 172.21.15.73:0/1518075138 conn(0x7ff908069cb0 msgr2=0x7ff90806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.315+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 shutdown_connections 2024-04-05T22:31:42.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.315+0000 7ff90fc00640 1 -- 172.21.15.73:0/1518075138 wait complete. 2024-04-05T22:31:42.343 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.508+0000 7f89af000640 1 Processor -- start 2024-04-05T22:31:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.508+0000 7f89af000640 1 -- start start 2024-04-05T22:31:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.509+0000 7f89af000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a806d710 0x7f89a80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.509+0000 7f89af000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89a815aef0 0x7f89a80cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.509+0000 7f89af000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a815aa60 con 0x7f89a81541a0 2024-04-05T22:31:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.509+0000 7f89af000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a80b3b60 con 0x7f89a815aef0 2024-04-05T22:31:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.509+0000 7f89af000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80701c0 con 0x7f89a806d710 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89a815aef0 0x7f89a80cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89a815aef0 0x7f89a80cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47070/0 (socket says 172.21.15.73:47070) 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 -- 172.21.15.73:0/4141004354 learned_addr learned my addr 172.21.15.73:0/4141004354 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 -- 172.21.15.73:0/4141004354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a806d710 msgr2=0x7f89a80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89a7400640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a806d710 0x7f89a80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a806d710 0x7f89a80cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 -- 172.21.15.73:0/4141004354 >> v1:172.21.15.157:6789/0 conn(0x7f89a81541a0 legacy=0x7f89a80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.510+0000 7f89ad000640 1 -- 172.21.15.73:0/4141004354 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80b3880 con 0x7f89a815aef0 2024-04-05T22:31:42.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.511+0000 7f89ad000640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a815aef0 0x7f89a80cde30 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a784ad675385a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89a6a00640 1 -- 172.21.15.73:0/4141004354 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0031640 con 0x7f89a815aef0 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89a6a00640 1 -- 172.21.15.73:0/4141004354 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a00317a0 con 0x7f89a815aef0 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89a6a00640 1 -- 172.21.15.73:0/4141004354 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0031b30 con 0x7f89a815aef0 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 -- 172.21.15.73:0/4141004354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a815aef0 msgr2=0x7f89a80cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a815aef0 0x7f89a80cde30 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 -- 172.21.15.73:0/4141004354 shutdown_connections 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a815aef0 0x7f89a80cde30 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 --2- 172.21.15.73:0/4141004354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a806d710 0x7f89a80cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 -- 172.21.15.73:0/4141004354 >> 172.21.15.73:0/4141004354 conn(0x7f89a80ce8e0 msgr2=0x7f89a80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 -- 172.21.15.73:0/4141004354 shutdown_connections 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.512+0000 7f89af000640 1 -- 172.21.15.73:0/4141004354 wait complete. 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.514+0000 7f89af000640 1 Processor -- start 2024-04-05T22:31:42.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.514+0000 7f89af000640 1 -- start start 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a81541a0 0x7f89a81f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a815aef0 0x7f89a8144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8145290 con 0x7f89a806d710 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81f2d10 con 0x7f89a815aef0 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89af000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81f2f70 con 0x7f89a81541a0 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47074/0 (socket says 172.21.15.73:47074) 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.515+0000 7f89a7e00640 1 -- 172.21.15.73:0/3873087453 learned_addr learned my addr 172.21.15.73:0/3873087453 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 -- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a81541a0 msgr2=0x7f89a81f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7400640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a81541a0 0x7f89a81f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89ad000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a815aef0 0x7f89a8144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a81541a0 0x7f89a81f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 -- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a815aef0 msgr2=0x7f89a8144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a815aef0 0x7f89a8144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 -- 172.21.15.73:0/3873087453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00094d0 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89a7e00640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f899800c410 tx=0x7f899800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980035f0 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998003750 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a81f3250 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998003b40 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.516+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a8144ba0 con 0x7f89a806d710 2024-04-05T22:31:42.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.517+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a8066af0 con 0x7f89a806d710 2024-04-05T22:31:42.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.518+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f8998003ca0 con 0x7f89a806d710 2024-04-05T22:31:42.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.519+0000 7f899fe00640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 0x7f898406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.519+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8998087530 con 0x7f89a806d710 2024-04-05T22:31:42.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.520+0000 7f89a7400640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 0x7f898406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.521+0000 7f89a7400640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 0x7f898406da80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f89a80afa60 tx=0x7f8990008440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.523+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8998053f60 con 0x7f89a806d710 2024-04-05T22:31:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.711+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89a81f3770 con 0x7f89a806d710 2024-04-05T22:31:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.713+0000 7f899fe00640 1 -- 172.21.15.73:0/3873087453 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16128 (secure 0 0 0) 0x7f89a81f3770 con 0x7f89a806d710 2024-04-05T22:31:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:42.714 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":171,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:42.297552+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 msgr2=0x7f898406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 0x7f898406da80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f89a80afa60 tx=0x7f8990008440 comp rx=0 tx=0).stop 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 msgr2=0x7f89a8147b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f899800c410 tx=0x7f899800c8d0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 shutdown_connections 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f898406b5c0 0x7f898406da80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a815aef0 0x7f89a8144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a81541a0 0x7f89a81f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 --2- 172.21.15.73:0/3873087453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a806d710 0x7f89a8147b70 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 >> 172.21.15.73:0/3873087453 conn(0x7f89a80ce8e0 msgr2=0x7f89a815b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 shutdown_connections 2024-04-05T22:31:42.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.717+0000 7f89af000640 1 -- 172.21.15.73:0/3873087453 wait complete. 2024-04-05T22:31:42.733 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:42.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.937+0000 7f9bfda00640 1 Processor -- start 2024-04-05T22:31:42.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.938+0000 7f9bfda00640 1 -- start start 2024-04-05T22:31:42.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.939+0000 7f9bfda00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bf80caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.939+0000 7f9bfda00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80ad880 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.939+0000 7f9bfda00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80ce5d0 con 0x7f9bf80b1870 2024-04-05T22:31:42.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.939+0000 7f9bfda00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ce010 con 0x7f9bf8160620 2024-04-05T22:31:42.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9bf80cb2f0 0x7f9bf80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60604/0 (socket says 172.21.15.73:60604) 2024-04-05T22:31:42.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf7400640 1 -- 172.21.15.73:0/1763820312 learned_addr learned my addr 172.21.15.73:0/1763820312 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf6a00640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bf80caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779352627 0 0) 0x7f9bf80ad880 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bdc003520 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.940+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65038580 0 0) 0x7f9bdc003520 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 msgr2=0x7f9bf80caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bf80caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 >> v1:172.21.15.157:6789/0 conn(0x7f9bf80b1870 legacy=0x7f9bf80ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80ce2f0 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9bf00019f0 con 0x7f9bf80cb2f0 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 >> v1:172.21.15.73:6789/0 conn(0x7f9bf80cb2f0 legacy=0x7f9bf80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bdc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bdc003f20 0x7f9bdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ce010 con 0x7f9bdc004d00 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc00ba60 con 0x7f9bdc003f20 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc00bc20 con 0x7f9bf8160620 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf6a00640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bdc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf6000640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bdc003f20 0x7f9bdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 -- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 msgr2=0x7f9bdc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8160620 0x7f9bdc0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 -- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bdc003f20 msgr2=0x7f9bdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:42.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bdc003f20 0x7f9bdc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf7400640 1 -- 172.21.15.73:0/1763820312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc003520 con 0x7f9bdc004d00 2024-04-05T22:31:42.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.941+0000 7f9bf6000640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bdc003f20 0x7f9bdc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.942+0000 7f9bf7400640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bdc005180 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000a830 tx=0x7f9bdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433d2eb6be982fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00072d0 con 0x7f9bdc004d00 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf002f070 con 0x7f9bdc004d00 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bf5600640 1 -- 172.21.15.73:0/1763820312 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00076f0 con 0x7f9bdc004d00 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 -- 172.21.15.73:0/1763820312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 msgr2=0x7f9bdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1763820312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bdc005180 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000a830 tx=0x7f9bdc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bf7400640 1 -- 172.21.15.73:0/1763820312 reap_dead start 2024-04-05T22:31:42.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 -- 172.21.15.73:0/1763820312 shutdown_connections 2024-04-05T22:31:42.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 -- 172.21.15.73:0/1763820312 >> 172.21.15.73:0/1763820312 conn(0x7f9bf80ce8e0 msgr2=0x7f9bf80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:42.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 -- 172.21.15.73:0/1763820312 shutdown_connections 2024-04-05T22:31:42.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.943+0000 7f9bfda00640 1 -- 172.21.15.73:0/1763820312 wait complete. 2024-04-05T22:31:42.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.945+0000 7f9bfda00640 1 Processor -- start 2024-04-05T22:31:42.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.945+0000 7f9bfda00640 1 -- start start 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80cd3b0 0x7f9bf81f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf81f6150 0x7f9bf814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf81f65d0 con 0x7f9bdc004d00 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815d590 con 0x7f9bf81f6150 2024-04-05T22:31:42.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.946+0000 7f9bfda00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815c9a0 con 0x7f9bf80cd3b0 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47084/0 (socket says 172.21.15.73:47084) 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 -- 172.21.15.73:0/1356551221 learned_addr learned my addr 172.21.15.73:0/1356551221 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 -- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80cd3b0 msgr2=0x7f9bf81f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf7400640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf81f6150 0x7f9bf814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80cd3b0 0x7f9bf81f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 -- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf81f6150 msgr2=0x7f9bf814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf81f6150 0x7f9bf814d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 -- 172.21.15.73:0/1356551221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf815cc60 con 0x7f9bdc004d00 2024-04-05T22:31:42.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.947+0000 7f9bf6a00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9be8010860 tx=0x7f9be8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.948+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8009610 con 0x7f9bdc004d00 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.948+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be800eb60 con 0x7f9bdc004d00 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.948+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf815cf40 con 0x7f9bdc004d00 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.948+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be801c400 con 0x7f9bdc004d00 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.948+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf8147860 con 0x7f9bdc004d00 2024-04-05T22:31:42.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.949+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8066af0 con 0x7f9bdc004d00 2024-04-05T22:31:42.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.950+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f9be801c560 con 0x7f9bdc004d00 2024-04-05T22:31:42.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.951+0000 7f9beea00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 0x7f9bd406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:42.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.951+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9be8088380 con 0x7f9bdc004d00 2024-04-05T22:31:42.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.952+0000 7f9bf6000640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 0x7f9bd406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:42.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.953+0000 7f9bf6000640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 0x7f9bd406da80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80ca820 tx=0x7f9be0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:42.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:42.955+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9be8054e60 con 0x7f9bdc004d00 2024-04-05T22:31:43.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.135+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf8068160 con 0x7f9bdc004d00 2024-04-05T22:31:43.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.138+0000 7f9beea00640 1 -- 172.21.15.73:0/1356551221 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16128 (secure 0 0 0) 0x7f9be8058480 con 0x7f9bdc004d00 2024-04-05T22:31:43.138 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:43.138 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":171,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:42.297552+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:43.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 msgr2=0x7f9bd406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:43.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 0x7f9bd406da80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80ca820 tx=0x7f9be0009040 comp rx=0 tx=0).stop 2024-04-05T22:31:43.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 msgr2=0x7f9bf8147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:43.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9be8010860 tx=0x7f9be8010d20 comp rx=0 tx=0).stop 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 shutdown_connections 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9bd406b5c0 0x7f9bd406da80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf81f6150 0x7f9bf814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80cd3b0 0x7f9bf81f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 --2- 172.21.15.73:0/1356551221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bdc004d00 0x7f9bf8147320 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 >> 172.21.15.73:0/1356551221 conn(0x7f9bf80ce8e0 msgr2=0x7f9bf8072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 shutdown_connections 2024-04-05T22:31:43.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.141+0000 7f9bfda00640 1 -- 172.21.15.73:0/1356551221 wait complete. 2024-04-05T22:31:43.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-05T22:31:43.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.365+0000 7fbad4400640 1 Processor -- start 2024-04-05T22:31:43.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.365+0000 7fbad4400640 1 -- start start 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbacc06c9a0 0x7fbacc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbacc06ba70 0x7fbacc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc06bec0 con 0x7fbacc06c9a0 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc15afc0 con 0x7fbacc06ba70 2024-04-05T22:31:43.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc06f5b0 con 0x7fbacc15abe0 2024-04-05T22:31:43.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbacc06c9a0 0x7fbacc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbacc06c9a0 0x7fbacc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47092/0 (socket says 172.21.15.73:47092) 2024-04-05T22:31:43.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.367+0000 7fbad1000640 1 -- 172.21.15.73:0/41983172 learned_addr learned my addr 172.21.15.73:0/41983172 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:43.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad2400640 1 --2- 172.21.15.73:0/41983172 >> v2:172.21.15.157:3300/0 conn(0x7fbacc06ba70 0x7fbacc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1a00640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1000640 1 -- 172.21.15.73:0/41983172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 msgr2=0x7fbacc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1000640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1000640 1 -- 172.21.15.73:0/41983172 >> v2:172.21.15.157:3300/0 conn(0x7fbacc06ba70 msgr2=0x7fbacc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1000640 1 --2- 172.21.15.73:0/41983172 >> v2:172.21.15.157:3300/0 conn(0x7fbacc06ba70 0x7fbacc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1000640 1 -- 172.21.15.73:0/41983172 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc06f890 con 0x7fbacc06c9a0 2024-04-05T22:31:43.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.368+0000 7fbad1a00640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.369+0000 7fbad1000640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc06b530 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fbacc070bc0 tx=0x7fbab802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5a330750326409 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.369+0000 7fbacbe00640 1 -- 172.21.15.73:0/41983172 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8031740 con 0x7fbacc06c9a0 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.369+0000 7fbacbe00640 1 -- 172.21.15.73:0/41983172 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab80318a0 con 0x7fbacc06c9a0 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.369+0000 7fbacbe00640 1 -- 172.21.15.73:0/41983172 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8031c30 con 0x7fbacc06c9a0 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 -- 172.21.15.73:0/41983172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 msgr2=0x7fbacc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:43.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc06b530 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fbacc070bc0 tx=0x7fbab802b7e0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 -- 172.21.15.73:0/41983172 shutdown_connections 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc06b530 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 --2- 172.21.15.73:0/41983172 >> v2:172.21.15.157:3300/0 conn(0x7fbacc06ba70 0x7fbacc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 --2- 172.21.15.73:0/41983172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 -- 172.21.15.73:0/41983172 >> 172.21.15.73:0/41983172 conn(0x7fbacc069cb0 msgr2=0x7fbacc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 -- 172.21.15.73:0/41983172 shutdown_connections 2024-04-05T22:31:43.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.370+0000 7fbad4400640 1 -- 172.21.15.73:0/41983172 wait complete. 2024-04-05T22:31:43.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.372+0000 7fbad4400640 1 Processor -- start 2024-04-05T22:31:43.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.372+0000 7fbad4400640 1 -- start start 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbacc06ba70 0x7fbacc150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc153290 con 0x7fbacc06c9a0 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc150960 con 0x7fbacc06ba70 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc152ef0 con 0x7fbacc15abe0 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47104/0 (socket says 172.21.15.73:47104) 2024-04-05T22:31:43.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 -- 172.21.15.73:0/3591016931 learned_addr learned my addr 172.21.15.73:0/3591016931 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1a00640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbacc06ba70 0x7fbacc150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 -- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 msgr2=0x7fbacc154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 -- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbacc06ba70 msgr2=0x7fbacc150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbacc06ba70 0x7fbacc150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.373+0000 7fbad1000640 1 -- 172.21.15.73:0/3591016931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80094d0 con 0x7fbacc06c9a0 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbad1000640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fbab802b820 tx=0x7fbab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:43.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8033d60 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab803c400 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab803c7b0 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc13cc30 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.374+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc15dc60 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.375+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc152d50 con 0x7fbacc06c9a0 2024-04-05T22:31:43.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.376+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fbab803a040 con 0x7fbacc06c9a0 2024-04-05T22:31:43.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.377+0000 7fbac9600640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 0x7fbaac06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:43.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.377+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbab80a9a70 con 0x7fbacc06c9a0 2024-04-05T22:31:43.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.378+0000 7fbad1a00640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 0x7fbaac06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:43.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.379+0000 7fbad1a00640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 0x7fbaac06da50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0adca0 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:43.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.381+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbacc152d50 con 0x7fbacc06c9a0 2024-04-05T22:31:43.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:43.617+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbacc1533d0 con 0x7fbacc06c9a0 2024-04-05T22:31:44.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.326+0000 7fbac9600640 1 -- 172.21.15.73:0/3591016931 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v108) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbacc1533d0 con 0x7fbacc06c9a0 2024-04-05T22:31:44.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.329+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 msgr2=0x7fbaac06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 0x7fbaac06da50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0adca0 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 msgr2=0x7fbacc13c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fbab802b820 tx=0x7fbab80046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 shutdown_connections 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbaac06b590 0x7fbaac06da50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbacc15abe0 0x7fbacc154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbacc06c9a0 0x7fbacc13c6f0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 --2- 172.21.15.73:0/3591016931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbacc06ba70 0x7fbacc150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 >> 172.21.15.73:0/3591016931 conn(0x7fbacc069cb0 msgr2=0x7fbacc157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 shutdown_connections 2024-04-05T22:31:44.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.330+0000 7fbad4400640 1 -- 172.21.15.73:0/3591016931 wait complete. 2024-04-05T22:31:44.352 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-05T22:31:44.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-05T22:31:44.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.564+0000 7f0d38200640 1 Processor -- start 2024-04-05T22:31:44.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.565+0000 7f0d38200640 1 -- start start 2024-04-05T22:31:44.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d38200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d30071670 0x7f0d3015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d38200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d3015dbc0 0x7f0d3006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d38200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3015ae50 con 0x7f0d3015aa70 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d38200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3015e3b0 con 0x7f0d3015dbc0 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d38200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300ac9e0 con 0x7f0d30071670 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d35800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d3015aa70 0x7f0d30070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60608/0 (socket says 172.21.15.73:60608) 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.566+0000 7f0d35800640 1 -- 172.21.15.73:0/1402707799 learned_addr learned my addr 172.21.15.73:0/1402707799 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d36200640 1 --2- 172.21.15.73:0/1402707799 >> v2:172.21.15.73:3300/0 conn(0x7f0d3015dbc0 0x7f0d3006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d34e00640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d30071670 0x7f0d3015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d2fe00640 1 -- 172.21.15.73:0/1402707799 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472038790 0 0) 0x7f0d3015ae50 con 0x7f0d3015aa70 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d2fe00640 1 -- 172.21.15.73:0/1402707799 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d18003520 con 0x7f0d3015aa70 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d36200640 1 -- 172.21.15.73:0/1402707799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d30071670 msgr2=0x7f0d3015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d36200640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d30071670 0x7f0d3015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d36200640 1 -- 172.21.15.73:0/1402707799 >> v1:172.21.15.73:6789/0 conn(0x7f0d3015aa70 legacy=0x7f0d30070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.567+0000 7f0d36200640 1 -- 172.21.15.73:0/1402707799 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300accc0 con 0x7f0d3015dbc0 2024-04-05T22:31:44.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d36200640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d3015dbc0 0x7f0d3006ba40 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009d90 tx=0x7f0d2802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41efb580b92c82e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:44.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d2fe00640 1 -- 172.21.15.73:0/1402707799 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2802cc50 con 0x7f0d3015dbc0 2024-04-05T22:31:44.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d2fe00640 1 -- 172.21.15.73:0/1402707799 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2802cdb0 con 0x7f0d3015dbc0 2024-04-05T22:31:44.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d2fe00640 1 -- 172.21.15.73:0/1402707799 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28034640 con 0x7f0d3015dbc0 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d38200640 1 -- 172.21.15.73:0/1402707799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d3015dbc0 msgr2=0x7f0d3006ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.569+0000 7f0d38200640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d3015dbc0 0x7f0d3006ba40 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009d90 tx=0x7f0d2802bea0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 -- 172.21.15.73:0/1402707799 shutdown_connections 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d30071670 0x7f0d3015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 --2- 172.21.15.73:0/1402707799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d3015dbc0 0x7f0d3006ba40 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 -- 172.21.15.73:0/1402707799 >> 172.21.15.73:0/1402707799 conn(0x7f0d30069cb0 msgr2=0x7f0d3006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 -- 172.21.15.73:0/1402707799 shutdown_connections 2024-04-05T22:31:44.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.570+0000 7f0d38200640 1 -- 172.21.15.73:0/1402707799 wait complete. 2024-04-05T22:31:44.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.571+0000 7f0d38200640 1 Processor -- start 2024-04-05T22:31:44.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.571+0000 7f0d38200640 1 -- start start 2024-04-05T22:31:44.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3015aa70 0x7f0d301f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015dbc0 0x7f0d301583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300af690 con 0x7f0d30071670 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300aea10 con 0x7f0d3015dbc0 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.572+0000 7f0d38200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300aecf0 con 0x7f0d3015aa70 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47116/0 (socket says 172.21.15.73:47116) 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 -- 172.21.15.73:0/3930767810 learned_addr learned my addr 172.21.15.73:0/3930767810 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d34e00640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3015aa70 0x7f0d301f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d36200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015dbc0 0x7f0d301583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 -- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3015aa70 msgr2=0x7f0d301f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3015aa70 0x7f0d301f5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 -- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015dbc0 msgr2=0x7f0d301583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015dbc0 0x7f0d301583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 -- 172.21.15.73:0/3930767810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009a70 con 0x7f0d30071670 2024-04-05T22:31:44.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.573+0000 7f0d35800640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c430 tx=0x7f0d2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.574+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24005ce0 con 0x7f0d30071670 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.574+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24005e40 con 0x7f0d30071670 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.574+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240072f0 con 0x7f0d30071670 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.574+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300aefd0 con 0x7f0d30071670 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.574+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d301f59d0 con 0x7f0d30071670 2024-04-05T22:31:44.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.575+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30068160 con 0x7f0d30071670 2024-04-05T22:31:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.576+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f0d24007450 con 0x7f0d30071670 2024-04-05T22:31:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.577+0000 7f0d2d600640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 0x7f0d1406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.577+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d24087540 con 0x7f0d30071670 2024-04-05T22:31:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.578+0000 7f0d34e00640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 0x7f0d1406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.579+0000 7f0d34e00640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 0x7f0d1406da50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d3006c330 tx=0x7f0d20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:44.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.582+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0d24053f70 con 0x7f0d30071670 2024-04-05T22:31:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:44.801+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0d300ae1c0 con 0x7f0d30071670 2024-04-05T22:31:45.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.330+0000 7f0d2d600640 1 -- 172.21.15.73:0/3930767810 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v109) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d24017020 con 0x7f0d30071670 2024-04-05T22:31:45.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 msgr2=0x7f0d1406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 0x7f0d1406da50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d3006c330 tx=0x7f0d20008040 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 msgr2=0x7f0d300ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c430 tx=0x7f0d2400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 shutdown_connections 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0d1406b590 0x7f0d1406da50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015dbc0 0x7f0d301583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3015aa70 0x7f0d301f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 --2- 172.21.15.73:0/3930767810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d30071670 0x7f0d300ae3e0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 >> 172.21.15.73:0/3930767810 conn(0x7f0d30069cb0 msgr2=0x7f0d301612e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.333+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 shutdown_connections 2024-04-05T22:31:45.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.334+0000 7f0d38200640 1 -- 172.21.15.73:0/3930767810 wait complete. 2024-04-05T22:31:45.353 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-05T22:31:45.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.568+0000 7fc32ea00640 1 Processor -- start 2024-04-05T22:31:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.568+0000 7fc32ea00640 1 -- start start 2024-04-05T22:31:45.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 0x7fc3280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc32ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc32815aef0 0x7fc3280cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc32ea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32815aa60 con 0x7fc3281541a0 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc32ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3280b3b60 con 0x7fc32815aef0 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc32ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280701c0 con 0x7fc32806d710 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc327e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc3281541a0 0x7fc3280b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60622/0 (socket says 172.21.15.73:60622) 2024-04-05T22:31:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.570+0000 7fc327e00640 1 -- 172.21.15.73:0/3085354207 learned_addr learned my addr 172.21.15.73:0/3085354207 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751225581 0 0) 0x7fc32815aa60 con 0x7fc3281541a0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30c005730 con 0x7fc3281541a0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc32ca00640 1 --2- 172.21.15.73:0/3085354207 >> v2:172.21.15.157:3300/0 conn(0x7fc32815aef0 0x7fc3280cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc327400640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 0x7fc3280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4139634789 0 0) 0x7fc30c005730 con 0x7fc3281541a0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 msgr2=0x7fc3280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 0x7fc3280cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 >> v2:172.21.15.157:3300/0 conn(0x7fc32815aef0 msgr2=0x7fc3280cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 --2- 172.21.15.73:0/3085354207 >> v2:172.21.15.157:3300/0 conn(0x7fc32815aef0 0x7fc3280cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280b3880 con 0x7fc3281541a0 2024-04-05T22:31:45.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fc310001670 con 0x7fc3281541a0 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 >> v1:172.21.15.73:6789/0 conn(0x7fc3281541a0 legacy=0x7fc3280b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 0x7fc30c005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc30c0061b0 0x7fc30c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30c006f10 0x7fc30c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280b3b60 con 0x7fc30c0061b0 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280701c0 con 0x7fc30c006f10 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.571+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c00dc90 con 0x7fc32806d710 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc30c0061b0 0x7fc30c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 -- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 msgr2=0x7fc30c005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc32806d710 0x7fc30c005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 -- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30c006f10 msgr2=0x7fc30c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30c006f10 0x7fc30c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.572+0000 7fc327400640 1 -- 172.21.15.73:0/3085354207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c005730 con 0x7fc30c0061b0 2024-04-05T22:31:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.573+0000 7fc327400640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc30c0061b0 0x7fc30c0069d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc31800ba30 tx=0x7fc30c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac248d1276da4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.573+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318032680 con 0x7fc30c0061b0 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.573+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3180327e0 con 0x7fc30c0061b0 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.573+0000 7fc326a00640 1 -- 172.21.15.73:0/3085354207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318032b70 con 0x7fc30c0061b0 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 -- 172.21.15.73:0/3085354207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc30c0061b0 msgr2=0x7fc30c0069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 --2- 172.21.15.73:0/3085354207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc30c0061b0 0x7fc30c0069d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc31800ba30 tx=0x7fc30c007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ca00640 1 -- 172.21.15.73:0/3085354207 reap_dead start 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 -- 172.21.15.73:0/3085354207 shutdown_connections 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 -- 172.21.15.73:0/3085354207 >> 172.21.15.73:0/3085354207 conn(0x7fc3280ce8e0 msgr2=0x7fc3280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 -- 172.21.15.73:0/3085354207 shutdown_connections 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.574+0000 7fc32ea00640 1 -- 172.21.15.73:0/3085354207 wait complete. 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.576+0000 7fc32ea00640 1 Processor -- start 2024-04-05T22:31:45.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.576+0000 7fc32ea00640 1 -- start start 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc30c0061b0 0x7fc32815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3280cdc50 0x7fc328150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815cf50 con 0x7fc3280cd180 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815c990 con 0x7fc3280cdc50 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815cc70 con 0x7fc30c0061b0 2024-04-05T22:31:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47126/0 (socket says 172.21.15.73:47126) 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 -- 172.21.15.73:0/4097822391 learned_addr learned my addr 172.21.15.73:0/4097822391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 -- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc30c0061b0 msgr2=0x7fc32815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc30c0061b0 0x7fc32815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 -- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3280cdc50 msgr2=0x7fc328150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3280cdc50 0x7fc328150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.577+0000 7fc32ca00640 1 -- 172.21.15.73:0/4097822391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280cc980 con 0x7fc3280cd180 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc32ca00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc32815a9d0 tx=0x7fc32000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000ed90 con 0x7fc3280cd180 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320014420 con 0x7fc3280cd180 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200147d0 con 0x7fc3280cd180 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3280ccc60 con 0x7fc3280cd180 2024-04-05T22:31:45.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.578+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc328150a60 con 0x7fc3280cd180 2024-04-05T22:31:45.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.579+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc328066af0 con 0x7fc3280cd180 2024-04-05T22:31:45.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.580+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fc320014930 con 0x7fc3280cd180 2024-04-05T22:31:45.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.581+0000 7fc31fe00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 0x7fc30406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:45.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.581+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc320088230 con 0x7fc3280cd180 2024-04-05T22:31:45.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.582+0000 7fc327e00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 0x7fc30406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:45.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.583+0000 7fc327e00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 0x7fc30406da50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc32815d380 tx=0x7fc310001540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:45.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.586+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc320054c60 con 0x7fc3280cd180 2024-04-05T22:31:45.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.818+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3280cc7e0 con 0x7fc3280cd180 2024-04-05T22:31:45.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.819+0000 7fc31fe00640 1 -- 172.21.15.73:0/4097822391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4753 (secure 0 0 0) 0x7fc3280cc7e0 con 0x7fc3280cd180 2024-04-05T22:31:45.819 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:45.819 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:31:45.819 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 109 2024-04-05T22:31:45.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 msgr2=0x7fc30406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 0x7fc30406da50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc32815d380 tx=0x7fc310001540 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 msgr2=0x7fc328156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc32815a9d0 tx=0x7fc32000e4b0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 shutdown_connections 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc30406b590 0x7fc30406da50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280cd180 0x7fc328156bc0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3280cdc50 0x7fc328150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 --2- 172.21.15.73:0/4097822391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc30c0061b0 0x7fc32815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 >> 172.21.15.73:0/4097822391 conn(0x7fc3280ce8e0 msgr2=0x7fc3281533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.822+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 shutdown_connections 2024-04-05T22:31:45.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:45.823+0000 7fc32ea00640 1 -- 172.21.15.73:0/4097822391 wait complete. 2024-04-05T22:31:45.840 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:46.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.048+0000 7f65eea00640 1 Processor -- start 2024-04-05T22:31:46.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.048+0000 7f65eea00640 1 -- start start 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.049+0000 7f65eea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 0x7f65e806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.049+0000 7f65eea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65e80d09b0 0x7f65e80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.049+0000 7f65eea00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80cb950 con 0x7f65e80cd090 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.049+0000 7f65eea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80ce990 con 0x7f65e80d09b0 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.049+0000 7f65eea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80d1350 con 0x7f65e815cf50 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65e80d09b0 0x7f65e80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65e80d09b0 0x7f65e80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42952/0 (socket says 172.21.15.73:42952) 2024-04-05T22:31:46.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 -- 172.21.15.73:0/985814464 learned_addr learned my addr 172.21.15.73:0/985814464 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7e00640 1 --2- 172.21.15.73:0/985814464 >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 0x7f65e806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 -- 172.21.15.73:0/985814464 >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 msgr2=0x7f65e806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 --2- 172.21.15.73:0/985814464 >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 0x7f65e806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 -- 172.21.15.73:0/985814464 >> v1:172.21.15.73:6789/0 conn(0x7f65e80cd090 legacy=0x7f65e80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7400640 1 -- 172.21.15.73:0/985814464 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80cdce0 con 0x7f65e80d09b0 2024-04-05T22:31:46.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.050+0000 7f65e7e00640 1 --2- 172.21.15.73:0/985814464 >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 0x7f65e806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:46.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.051+0000 7f65e7400640 1 --2- 172.21.15.73:0/985814464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80d0e10 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f65e80ce590 tx=0x7f65d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbf66536ab3b22c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65e6a00640 1 -- 172.21.15.73:0/985814464 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4030860 con 0x7f65e80d09b0 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65e6a00640 1 -- 172.21.15.73:0/985814464 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d40309c0 con 0x7f65e80d09b0 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65e6a00640 1 -- 172.21.15.73:0/985814464 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4030d50 con 0x7f65e80d09b0 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 -- 172.21.15.73:0/985814464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 msgr2=0x7f65e80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 --2- 172.21.15.73:0/985814464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80d0e10 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f65e80ce590 tx=0x7f65d402b9a0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 -- 172.21.15.73:0/985814464 shutdown_connections 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 --2- 172.21.15.73:0/985814464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80d0e10 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 --2- 172.21.15.73:0/985814464 >> v2:172.21.15.157:3300/0 conn(0x7f65e815cf50 0x7f65e806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.052+0000 7f65eea00640 1 -- 172.21.15.73:0/985814464 >> 172.21.15.73:0/985814464 conn(0x7f65e8099a60 msgr2=0x7f65e8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:46.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.053+0000 7f65eea00640 1 -- 172.21.15.73:0/985814464 shutdown_connections 2024-04-05T22:31:46.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.053+0000 7f65eea00640 1 -- 172.21.15.73:0/985814464 wait complete. 2024-04-05T22:31:46.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.055+0000 7f65eea00640 1 Processor -- start 2024-04-05T22:31:46.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.055+0000 7f65eea00640 1 -- start start 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e80cd090 0x7f65e814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 0x7f65e80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80c9e50 con 0x7f65e80d09b0 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e814d630 con 0x7f65e815cf50 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65eea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80c8f60 con 0x7f65e80cd090 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65e7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65e7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42964/0 (socket says 172.21.15.73:42964) 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.056+0000 7f65e7400640 1 -- 172.21.15.73:0/3019376473 learned_addr learned my addr 172.21.15.73:0/3019376473 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 -- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e80cd090 msgr2=0x7f65e814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65eca00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 0x7f65e80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7e00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e80cd090 0x7f65e814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e80cd090 0x7f65e814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 -- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 msgr2=0x7f65e80ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 0x7f65e80ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 -- 172.21.15.73:0/3019376473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40094d0 con 0x7f65e80d09b0 2024-04-05T22:31:46.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65eca00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 0x7f65e80ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.057+0000 7f65e7400640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000c00 tx=0x7f65d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.058+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4030830 con 0x7f65e80d09b0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.058+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4043400 con 0x7f65e80d09b0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.058+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40437b0 con 0x7f65e80d09b0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.058+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e80c9240 con 0x7f65e80d09b0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.058+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e80c9800 con 0x7f65e80d09b0 2024-04-05T22:31:46.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.059+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e8068160 con 0x7f65e80d09b0 2024-04-05T22:31:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.060+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f65d4043910 con 0x7f65e80d09b0 2024-04-05T22:31:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.061+0000 7f65dfe00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 0x7f65c806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.061+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65d403e070 con 0x7f65e80d09b0 2024-04-05T22:31:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.061+0000 7f65e7e00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 0x7f65c806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.062+0000 7f65e7e00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 0x7f65c806da50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b840 tx=0x7f65d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.065+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f65d4077e50 con 0x7f65e80d09b0 2024-04-05T22:31:46.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.243+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65e806cdb0 con 0x7f65e80d09b0 2024-04-05T22:31:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.246+0000 7f65dfe00640 1 -- 172.21.15.73:0/3019376473 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7f65d4034020 con 0x7f65e80d09b0 2024-04-05T22:31:46.245 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:46.246 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 msgr2=0x7f65c806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 0x7f65c806da50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b840 tx=0x7f65d8008040 comp rx=0 tx=0).stop 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 msgr2=0x7f65e80c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000c00 tx=0x7f65d4004730 comp rx=0 tx=0).stop 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 shutdown_connections 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f65c806b590 0x7f65c806da50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e815cf50 0x7f65e80ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e80d09b0 0x7f65e80c6eb0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 --2- 172.21.15.73:0/3019376473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e80cd090 0x7f65e814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 >> 172.21.15.73:0/3019376473 conn(0x7f65e8099a60 msgr2=0x7f65e8099e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:46.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 shutdown_connections 2024-04-05T22:31:46.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.249+0000 7f65eea00640 1 -- 172.21.15.73:0/3019376473 wait complete. 2024-04-05T22:31:46.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-05T22:31:46.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:31:46.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.466+0000 7f256be00640 1 Processor -- start 2024-04-05T22:31:46.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.466+0000 7f256be00640 1 -- start start 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.467+0000 7f256be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f256c0b2b90 0x7f256c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.467+0000 7f256be00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f256c0b1b50 0x7f256c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f256be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c06d3e0 con 0x7f256c0b20a0 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f256be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c06d610 con 0x7f256c0b2b90 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f256be00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c070230 con 0x7f256c0b1b50 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f256c0b2b90 0x7f256c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f256c0b2b90 0x7f256c070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42968/0 (socket says 172.21.15.73:42968) 2024-04-05T22:31:46.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 -- 172.21.15.73:0/45957184 learned_addr learned my addr 172.21.15.73:0/45957184 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 -- 172.21.15.73:0/45957184 >> v2:172.21.15.157:3300/0 conn(0x7f256c0b1b50 msgr2=0x7f256c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:46.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 --2- 172.21.15.73:0/45957184 >> v2:172.21.15.157:3300/0 conn(0x7f256c0b1b50 0x7f256c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 -- 172.21.15.73:0/45957184 >> v1:172.21.15.157:6789/0 conn(0x7f256c0b20a0 legacy=0x7f256c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:46.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.468+0000 7f2562000640 1 -- 172.21.15.73:0/45957184 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c070510 con 0x7f256c0b2b90 2024-04-05T22:31:46.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.469+0000 7f2562000640 1 --2- 172.21.15.73:0/45957184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b2b90 0x7f256c070ff0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f256c0b4560 tx=0x7f255402dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8492efef83433d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.469+0000 7f256a000640 1 -- 172.21.15.73:0/45957184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402ed80 con 0x7f256c0b2b90 2024-04-05T22:31:46.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256a000640 1 -- 172.21.15.73:0/45957184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2554036440 con 0x7f256c0b2b90 2024-04-05T22:31:46.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256a000640 1 -- 172.21.15.73:0/45957184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25540367d0 con 0x7f256c0b2b90 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 -- 172.21.15.73:0/45957184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b2b90 msgr2=0x7f256c070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 --2- 172.21.15.73:0/45957184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b2b90 0x7f256c070ff0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f256c0b4560 tx=0x7f255402dfd0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 -- 172.21.15.73:0/45957184 shutdown_connections 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 --2- 172.21.15.73:0/45957184 >> v2:172.21.15.157:3300/0 conn(0x7f256c0b1b50 0x7f256c06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 --2- 172.21.15.73:0/45957184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b2b90 0x7f256c070ff0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 -- 172.21.15.73:0/45957184 >> 172.21.15.73:0/45957184 conn(0x7f256c0b06c0 msgr2=0x7f256c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 -- 172.21.15.73:0/45957184 shutdown_connections 2024-04-05T22:31:46.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.470+0000 7f256be00640 1 -- 172.21.15.73:0/45957184 wait complete. 2024-04-05T22:31:46.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.472+0000 7f256be00640 1 Processor -- start 2024-04-05T22:31:46.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.472+0000 7f256be00640 1 -- start start 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256c0b1b50 0x7f256c146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c0b2b90 0x7f256c156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c140e30 con 0x7f256c0b20a0 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1f24d0 con 0x7f256c0b2b90 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f256be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1f18c0 con 0x7f256c0b1b50 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f2562000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f2562000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42974/0 (socket says 172.21.15.73:42974) 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.473+0000 7f2562000640 1 -- 172.21.15.73:0/2818112258 learned_addr learned my addr 172.21.15.73:0/2818112258 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f256aa00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256c0b1b50 0x7f256c146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f256b400640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c0b2b90 0x7f256c156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 -- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256c0b1b50 msgr2=0x7f256c146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256c0b1b50 0x7f256c146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 -- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c0b2b90 msgr2=0x7f256c156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c0b2b90 0x7f256c156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 -- 172.21.15.73:0/2818112258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255400b770 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2562000640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f255402e010 tx=0x7f25540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255403f810 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255403f990 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.474+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255403fd40 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.475+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c1f1ba0 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.475+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1f2160 con 0x7f256c0b20a0 2024-04-05T22:31:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.476+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c04e160 con 0x7f256c0b20a0 2024-04-05T22:31:46.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.477+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f255403d040 con 0x7f256c0b20a0 2024-04-05T22:31:46.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.477+0000 7f2563e00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 0x7f254806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.478+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25540ac650 con 0x7f256c0b20a0 2024-04-05T22:31:46.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.478+0000 7f256aa00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 0x7f254806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.479+0000 7f256aa00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 0x7f254806da50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f255c0015d0 tx=0x7f255c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.482+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2554079080 con 0x7f256c0b20a0 2024-04-05T22:31:46.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.674+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f256c070ad0 con 0x7f254806b590 2024-04-05T22:31:46.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.697+0000 7f2563e00640 1 -- 172.21.15.73:0/2818112258 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287669 (secure 0 0 0) 0x7f255c0050d0 con 0x7f254806b590 2024-04-05T22:31:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:46.705 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:31:46.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.703+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 msgr2=0x7f254806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.703+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 0x7f254806da50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f255c0015d0 tx=0x7f255c009040 comp rx=0 tx=0).stop 2024-04-05T22:31:46.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.703+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 msgr2=0x7f256c15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.703+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f255402e010 tx=0x7f25540046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 shutdown_connections 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f254806b590 0x7f254806da50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f256c0b2b90 0x7f256c156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f256c0b20a0 0x7f256c15c8b0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 --2- 172.21.15.73:0/2818112258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f256c0b1b50 0x7f256c146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 >> 172.21.15.73:0/2818112258 conn(0x7f256c0b06c0 msgr2=0x7f256c0ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:46.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 shutdown_connections 2024-04-05T22:31:46.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.704+0000 7f256be00640 1 -- 172.21.15.73:0/2818112258 wait complete. 2024-04-05T22:31:46.723 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-04-05T22:31:45.612810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287324,"kb_used_data":17900,"kb_used_omap":12,"kb_used_meta":269299,"kb_avail":754687396,"statfs":{"total":773094113280,"available":772799893504,"internally_reserved":0,"allocated":18329600,"data_stored":8728206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":275762747},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.803507"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347292+0000","last_change":"2024-04-05T22:31:42.345705+0000","last_active":"2024-04-05T22:31:42.347292+0000","last_peered":"2024-04-05T22:31:42.347292+0000","last_clean":"2024-04-05T22:31:42.347292+0000","last_became_active":"2024-04-05T22:31:42.345434+0000","last_became_peered":"2024-04-05T22:31:42.345434+0000","last_unstale":"2024-04-05T22:31:42.347292+0000","last_undegraded":"2024-04-05T22:31:42.347292+0000","last_fullsized":"2024-04-05T22:31:42.347292+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:01.012844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345282+0000","last_change":"2024-04-05T22:31:42.342228+0000","last_active":"2024-04-05T22:31:42.345282+0000","last_peered":"2024-04-05T22:31:42.345282+0000","last_clean":"2024-04-05T22:31:42.345282+0000","last_became_active":"2024-04-05T22:31:42.341735+0000","last_became_peered":"2024-04-05T22:31:42.341735+0000","last_unstale":"2024-04-05T22:31:42.345282+0000","last_undegraded":"2024-04-05T22:31:42.345282+0000","last_fullsized":"2024-04-05T22:31:42.345282+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:59.115418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348885+0000","last_change":"2024-04-05T22:31:42.348080+0000","last_active":"2024-04-05T22:31:42.348885+0000","last_peered":"2024-04-05T22:31:42.348885+0000","last_clean":"2024-04-05T22:31:42.348885+0000","last_became_active":"2024-04-05T22:31:42.347754+0000","last_became_peered":"2024-04-05T22:31:42.347754+0000","last_unstale":"2024-04-05T22:31:42.348885+0000","last_undegraded":"2024-04-05T22:31:42.348885+0000","last_fullsized":"2024-04-05T22:31:42.348885+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:45.043543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349491+0000","last_change":"2024-04-05T22:31:42.345817+0000","last_active":"2024-04-05T22:31:42.349491+0000","last_peered":"2024-04-05T22:31:42.349491+0000","last_clean":"2024-04-05T22:31:42.349491+0000","last_became_active":"2024-04-05T22:31:42.345512+0000","last_became_peered":"2024-04-05T22:31:42.345512+0000","last_unstale":"2024-04-05T22:31:42.349491+0000","last_undegraded":"2024-04-05T22:31:42.349491+0000","last_fullsized":"2024-04-05T22:31:42.349491+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:19.428791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330226+0000","last_change":"2024-04-05T22:31:42.326414+0000","last_active":"2024-04-05T22:31:42.330226+0000","last_peered":"2024-04-05T22:31:42.330226+0000","last_clean":"2024-04-05T22:31:42.330226+0000","last_became_active":"2024-04-05T22:31:42.325817+0000","last_became_peered":"2024-04-05T22:31:42.325817+0000","last_unstale":"2024-04-05T22:31:42.330226+0000","last_undegraded":"2024-04-05T22:31:42.330226+0000","last_fullsized":"2024-04-05T22:31:42.330226+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:56:24.844384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.343984+0000","last_change":"2024-04-05T22:31:42.343387+0000","last_active":"2024-04-05T22:31:42.343984+0000","last_peered":"2024-04-05T22:31:42.343984+0000","last_clean":"2024-04-05T22:31:42.343984+0000","last_became_active":"2024-04-05T22:31:42.342966+0000","last_became_peered":"2024-04-05T22:31:42.342966+0000","last_unstale":"2024-04-05T22:31:42.343984+0000","last_undegraded":"2024-04-05T22:31:42.343984+0000","last_fullsized":"2024-04-05T22:31:42.343984+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:43.369766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.337235+0000","last_change":"2024-04-05T22:31:42.336569+0000","last_active":"2024-04-05T22:31:42.337235+0000","last_peered":"2024-04-05T22:31:42.337235+0000","last_clean":"2024-04-05T22:31:42.337235+0000","last_became_active":"2024-04-05T22:31:42.336279+0000","last_became_peered":"2024-04-05T22:31:42.336279+0000","last_unstale":"2024-04-05T22:31:42.337235+0000","last_undegraded":"2024-04-05T22:31:42.337235+0000","last_fullsized":"2024-04-05T22:31:42.337235+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:05:47.621581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321248+0000","last_change":"2024-04-05T22:31:42.319522+0000","last_active":"2024-04-05T22:31:42.321248+0000","last_peered":"2024-04-05T22:31:42.321248+0000","last_clean":"2024-04-05T22:31:42.321248+0000","last_became_active":"2024-04-05T22:31:42.319248+0000","last_became_peered":"2024-04-05T22:31:42.319248+0000","last_unstale":"2024-04-05T22:31:42.321248+0000","last_undegraded":"2024-04-05T22:31:42.321248+0000","last_fullsized":"2024-04-05T22:31:42.321248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:32.520839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348673+0000","last_change":"2024-04-05T22:31:42.345872+0000","last_active":"2024-04-05T22:31:42.348673+0000","last_peered":"2024-04-05T22:31:42.348673+0000","last_clean":"2024-04-05T22:31:42.348673+0000","last_became_active":"2024-04-05T22:31:42.345542+0000","last_became_peered":"2024-04-05T22:31:42.345542+0000","last_unstale":"2024-04-05T22:31:42.348673+0000","last_undegraded":"2024-04-05T22:31:42.348673+0000","last_fullsized":"2024-04-05T22:31:42.348673+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:30:19.604469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330881+0000","last_change":"2024-04-05T22:31:42.325832+0000","last_active":"2024-04-05T22:31:42.330881+0000","last_peered":"2024-04-05T22:31:42.330881+0000","last_clean":"2024-04-05T22:31:42.330881+0000","last_became_active":"2024-04-05T22:31:42.324711+0000","last_became_peered":"2024-04-05T22:31:42.324711+0000","last_unstale":"2024-04-05T22:31:42.330881+0000","last_undegraded":"2024-04-05T22:31:42.330881+0000","last_fullsized":"2024-04-05T22:31:42.330881+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:37:37.102529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339207+0000","last_change":"2024-04-05T22:31:42.335332+0000","last_active":"2024-04-05T22:31:42.339207+0000","last_peered":"2024-04-05T22:31:42.339207+0000","last_clean":"2024-04-05T22:31:42.339207+0000","last_became_active":"2024-04-05T22:31:42.334155+0000","last_became_peered":"2024-04-05T22:31:42.334155+0000","last_unstale":"2024-04-05T22:31:42.339207+0000","last_undegraded":"2024-04-05T22:31:42.339207+0000","last_fullsized":"2024-04-05T22:31:42.339207+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:10.992539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342786+0000","last_change":"2024-04-05T22:31:42.339091+0000","last_active":"2024-04-05T22:31:42.342786+0000","last_peered":"2024-04-05T22:31:42.342786+0000","last_clean":"2024-04-05T22:31:42.342786+0000","last_became_active":"2024-04-05T22:31:42.338706+0000","last_became_peered":"2024-04-05T22:31:42.338706+0000","last_unstale":"2024-04-05T22:31:42.342786+0000","last_undegraded":"2024-04-05T22:31:42.342786+0000","last_fullsized":"2024-04-05T22:31:42.342786+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:54:18.242102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342619+0000","last_change":"2024-04-05T22:31:42.338052+0000","last_active":"2024-04-05T22:31:42.342619+0000","last_peered":"2024-04-05T22:31:42.342619+0000","last_clean":"2024-04-05T22:31:42.342619+0000","last_became_active":"2024-04-05T22:31:42.337748+0000","last_became_peered":"2024-04-05T22:31:42.337748+0000","last_unstale":"2024-04-05T22:31:42.342619+0000","last_undegraded":"2024-04-05T22:31:42.342619+0000","last_fullsized":"2024-04-05T22:31:42.342619+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:57.734750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336679+0000","last_change":"2024-04-05T22:31:42.332970+0000","last_active":"2024-04-05T22:31:42.336679+0000","last_peered":"2024-04-05T22:31:42.336679+0000","last_clean":"2024-04-05T22:31:42.336679+0000","last_became_active":"2024-04-05T22:31:42.332687+0000","last_became_peered":"2024-04-05T22:31:42.332687+0000","last_unstale":"2024-04-05T22:31:42.336679+0000","last_undegraded":"2024-04-05T22:31:42.336679+0000","last_fullsized":"2024-04-05T22:31:42.336679+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:58:29.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321303+0000","last_change":"2024-04-05T22:31:42.318249+0000","last_active":"2024-04-05T22:31:42.321303+0000","last_peered":"2024-04-05T22:31:42.321303+0000","last_clean":"2024-04-05T22:31:42.321303+0000","last_became_active":"2024-04-05T22:31:42.317925+0000","last_became_peered":"2024-04-05T22:31:42.317925+0000","last_unstale":"2024-04-05T22:31:42.321303+0000","last_undegraded":"2024-04-05T22:31:42.321303+0000","last_fullsized":"2024-04-05T22:31:42.321303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:38:02.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336526+0000","last_change":"2024-04-05T22:31:42.333445+0000","last_active":"2024-04-05T22:31:42.336526+0000","last_peered":"2024-04-05T22:31:42.336526+0000","last_clean":"2024-04-05T22:31:42.336526+0000","last_became_active":"2024-04-05T22:31:42.333110+0000","last_became_peered":"2024-04-05T22:31:42.333110+0000","last_unstale":"2024-04-05T22:31:42.336526+0000","last_undegraded":"2024-04-05T22:31:42.336526+0000","last_fullsized":"2024-04-05T22:31:42.336526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:33.673205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347262+0000","last_change":"2024-04-05T22:31:42.345591+0000","last_active":"2024-04-05T22:31:42.347262+0000","last_peered":"2024-04-05T22:31:42.347262+0000","last_clean":"2024-04-05T22:31:42.347262+0000","last_became_active":"2024-04-05T22:31:42.345097+0000","last_became_peered":"2024-04-05T22:31:42.345097+0000","last_unstale":"2024-04-05T22:31:42.347262+0000","last_undegraded":"2024-04-05T22:31:42.347262+0000","last_fullsized":"2024-04-05T22:31:42.347262+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:57:50.061087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330899+0000","last_change":"2024-04-05T22:31:42.327490+0000","last_active":"2024-04-05T22:31:42.330899+0000","last_peered":"2024-04-05T22:31:42.330899+0000","last_clean":"2024-04-05T22:31:42.330899+0000","last_became_active":"2024-04-05T22:31:42.326923+0000","last_became_peered":"2024-04-05T22:31:42.326923+0000","last_unstale":"2024-04-05T22:31:42.330899+0000","last_undegraded":"2024-04-05T22:31:42.330899+0000","last_fullsized":"2024-04-05T22:31:42.330899+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:47.359209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333648+0000","last_change":"2024-04-05T22:31:42.331608+0000","last_active":"2024-04-05T22:31:42.333648+0000","last_peered":"2024-04-05T22:31:42.333648+0000","last_clean":"2024-04-05T22:31:42.333648+0000","last_became_active":"2024-04-05T22:31:42.330407+0000","last_became_peered":"2024-04-05T22:31:42.330407+0000","last_unstale":"2024-04-05T22:31:42.333648+0000","last_undegraded":"2024-04-05T22:31:42.333648+0000","last_fullsized":"2024-04-05T22:31:42.333648+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:57:50.061087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329903+0000","last_change":"2024-04-05T22:31:42.324291+0000","last_active":"2024-04-05T22:31:42.329903+0000","last_peered":"2024-04-05T22:31:42.329903+0000","last_clean":"2024-04-05T22:31:42.329903+0000","last_became_active":"2024-04-05T22:31:42.324006+0000","last_became_peered":"2024-04-05T22:31:42.324006+0000","last_unstale":"2024-04-05T22:31:42.329903+0000","last_undegraded":"2024-04-05T22:31:42.329903+0000","last_fullsized":"2024-04-05T22:31:42.329903+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:19.620848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348416+0000","last_change":"2024-04-05T22:31:42.342104+0000","last_active":"2024-04-05T22:31:42.348416+0000","last_peered":"2024-04-05T22:31:42.348416+0000","last_clean":"2024-04-05T22:31:42.348416+0000","last_became_active":"2024-04-05T22:31:42.341805+0000","last_became_peered":"2024-04-05T22:31:42.341805+0000","last_unstale":"2024-04-05T22:31:42.348416+0000","last_undegraded":"2024-04-05T22:31:42.348416+0000","last_fullsized":"2024-04-05T22:31:42.348416+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:14.569310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.335993+0000","last_change":"2024-04-05T22:31:42.327342+0000","last_active":"2024-04-05T22:31:42.335993+0000","last_peered":"2024-04-05T22:31:42.335993+0000","last_clean":"2024-04-05T22:31:42.335993+0000","last_became_active":"2024-04-05T22:31:42.326993+0000","last_became_peered":"2024-04-05T22:31:42.326993+0000","last_unstale":"2024-04-05T22:31:42.335993+0000","last_undegraded":"2024-04-05T22:31:42.335993+0000","last_fullsized":"2024-04-05T22:31:42.335993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:05.287932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.335021+0000","last_change":"2024-04-05T22:31:42.321704+0000","last_active":"2024-04-05T22:31:42.335021+0000","last_peered":"2024-04-05T22:31:42.335021+0000","last_clean":"2024-04-05T22:31:42.335021+0000","last_became_active":"2024-04-05T22:31:42.321000+0000","last_became_peered":"2024-04-05T22:31:42.321000+0000","last_unstale":"2024-04-05T22:31:42.335021+0000","last_undegraded":"2024-04-05T22:31:42.335021+0000","last_fullsized":"2024-04-05T22:31:42.335021+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:42:54.958862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.350034+0000","last_change":"2024-04-05T22:31:42.348049+0000","last_active":"2024-04-05T22:31:42.350034+0000","last_peered":"2024-04-05T22:31:42.350034+0000","last_clean":"2024-04-05T22:31:42.350034+0000","last_became_active":"2024-04-05T22:31:42.347787+0000","last_became_peered":"2024-04-05T22:31:42.347787+0000","last_unstale":"2024-04-05T22:31:42.350034+0000","last_undegraded":"2024-04-05T22:31:42.350034+0000","last_fullsized":"2024-04-05T22:31:42.350034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:45.043543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347564+0000","last_change":"2024-04-05T22:31:42.343220+0000","last_active":"2024-04-05T22:31:42.347564+0000","last_peered":"2024-04-05T22:31:42.347564+0000","last_clean":"2024-04-05T22:31:42.347564+0000","last_became_active":"2024-04-05T22:31:42.342959+0000","last_became_peered":"2024-04-05T22:31:42.342959+0000","last_unstale":"2024-04-05T22:31:42.347564+0000","last_undegraded":"2024-04-05T22:31:42.347564+0000","last_fullsized":"2024-04-05T22:31:42.347564+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:56.517269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336712+0000","last_change":"2024-04-05T22:31:42.334183+0000","last_active":"2024-04-05T22:31:42.336712+0000","last_peered":"2024-04-05T22:31:42.336712+0000","last_clean":"2024-04-05T22:31:42.336712+0000","last_became_active":"2024-04-05T22:31:42.333885+0000","last_became_peered":"2024-04-05T22:31:42.333885+0000","last_unstale":"2024-04-05T22:31:42.336712+0000","last_undegraded":"2024-04-05T22:31:42.336712+0000","last_fullsized":"2024-04-05T22:31:42.336712+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:31.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342638+0000","last_change":"2024-04-05T22:31:42.337305+0000","last_active":"2024-04-05T22:31:42.342638+0000","last_peered":"2024-04-05T22:31:42.342638+0000","last_clean":"2024-04-05T22:31:42.342638+0000","last_became_active":"2024-04-05T22:31:42.335524+0000","last_became_peered":"2024-04-05T22:31:42.335524+0000","last_unstale":"2024-04-05T22:31:42.342638+0000","last_undegraded":"2024-04-05T22:31:42.342638+0000","last_fullsized":"2024-04-05T22:31:42.342638+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:16.187575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321372+0000","last_change":"2024-04-05T22:31:42.318684+0000","last_active":"2024-04-05T22:31:42.321372+0000","last_peered":"2024-04-05T22:31:42.321372+0000","last_clean":"2024-04-05T22:31:42.321372+0000","last_became_active":"2024-04-05T22:31:42.318434+0000","last_became_peered":"2024-04-05T22:31:42.318434+0000","last_unstale":"2024-04-05T22:31:42.321372+0000","last_undegraded":"2024-04-05T22:31:42.321372+0000","last_fullsized":"2024-04-05T22:31:42.321372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:42:54.958862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342853+0000","last_change":"2024-04-05T22:31:42.339310+0000","last_active":"2024-04-05T22:31:42.342853+0000","last_peered":"2024-04-05T22:31:42.342853+0000","last_clean":"2024-04-05T22:31:42.342853+0000","last_became_active":"2024-04-05T22:31:42.337225+0000","last_became_peered":"2024-04-05T22:31:42.337225+0000","last_unstale":"2024-04-05T22:31:42.342853+0000","last_undegraded":"2024-04-05T22:31:42.342853+0000","last_fullsized":"2024-04-05T22:31:42.342853+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:50.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344737+0000","last_change":"2024-04-05T22:31:42.335684+0000","last_active":"2024-04-05T22:31:42.344737+0000","last_peered":"2024-04-05T22:31:42.344737+0000","last_clean":"2024-04-05T22:31:42.344737+0000","last_became_active":"2024-04-05T22:31:42.335293+0000","last_became_peered":"2024-04-05T22:31:42.335293+0000","last_unstale":"2024-04-05T22:31:42.344737+0000","last_undegraded":"2024-04-05T22:31:42.344737+0000","last_fullsized":"2024-04-05T22:31:42.344737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:10.978043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347128+0000","last_change":"2024-04-05T22:31:42.345820+0000","last_active":"2024-04-05T22:31:42.347128+0000","last_peered":"2024-04-05T22:31:42.347128+0000","last_clean":"2024-04-05T22:31:42.347128+0000","last_became_active":"2024-04-05T22:31:42.344945+0000","last_became_peered":"2024-04-05T22:31:42.344945+0000","last_unstale":"2024-04-05T22:31:42.347128+0000","last_undegraded":"2024-04-05T22:31:42.347128+0000","last_fullsized":"2024-04-05T22:31:42.347128+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:07.327855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321121+0000","last_change":"2024-04-05T22:31:42.318185+0000","last_active":"2024-04-05T22:31:42.321121+0000","last_peered":"2024-04-05T22:31:42.321121+0000","last_clean":"2024-04-05T22:31:42.321121+0000","last_became_active":"2024-04-05T22:31:42.316494+0000","last_became_peered":"2024-04-05T22:31:42.316494+0000","last_unstale":"2024-04-05T22:31:42.321121+0000","last_undegraded":"2024-04-05T22:31:42.321121+0000","last_fullsized":"2024-04-05T22:31:42.321121+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:47.359209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345191+0000","last_change":"2024-04-05T22:31:42.338095+0000","last_active":"2024-04-05T22:31:42.345191+0000","last_peered":"2024-04-05T22:31:42.345191+0000","last_clean":"2024-04-05T22:31:42.345191+0000","last_became_active":"2024-04-05T22:31:42.337787+0000","last_became_peered":"2024-04-05T22:31:42.337787+0000","last_unstale":"2024-04-05T22:31:42.345191+0000","last_undegraded":"2024-04-05T22:31:42.345191+0000","last_fullsized":"2024-04-05T22:31:42.345191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:43:13.566684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"171'4","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346497+0000","last_change":"2024-04-05T22:31:41.857820+0000","last_active":"2024-04-05T22:31:42.346497+0000","last_peered":"2024-04-05T22:31:42.346497+0000","last_clean":"2024-04-05T22:31:42.346497+0000","last_became_active":"2024-04-05T22:31:41.856777+0000","last_became_peered":"2024-04-05T22:31:41.856777+0000","last_unstale":"2024-04-05T22:31:42.346497+0000","last_undegraded":"2024-04-05T22:31:42.346497+0000","last_fullsized":"2024-04-05T22:31:42.346497+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:45.822999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.340720+0000","last_change":"2024-04-05T22:31:42.336675+0000","last_active":"2024-04-05T22:31:42.340720+0000","last_peered":"2024-04-05T22:31:42.340720+0000","last_clean":"2024-04-05T22:31:42.340720+0000","last_became_active":"2024-04-05T22:31:42.336326+0000","last_became_peered":"2024-04-05T22:31:42.336326+0000","last_unstale":"2024-04-05T22:31:42.340720+0000","last_undegraded":"2024-04-05T22:31:42.340720+0000","last_fullsized":"2024-04-05T22:31:42.340720+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:26:54.232462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349410+0000","last_change":"2024-04-05T22:31:41.853865+0000","last_active":"2024-04-05T22:31:42.349410+0000","last_peered":"2024-04-05T22:31:42.349410+0000","last_clean":"2024-04-05T22:31:42.349410+0000","last_became_active":"2024-04-05T22:31:41.853598+0000","last_became_peered":"2024-04-05T22:31:41.853598+0000","last_unstale":"2024-04-05T22:31:42.349410+0000","last_undegraded":"2024-04-05T22:31:42.349410+0000","last_fullsized":"2024-04-05T22:31:42.349410+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:02.313670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.323209+0000","last_change":"2024-04-05T22:31:42.320351+0000","last_active":"2024-04-05T22:31:42.323209+0000","last_peered":"2024-04-05T22:31:42.323209+0000","last_clean":"2024-04-05T22:31:42.323209+0000","last_became_active":"2024-04-05T22:31:42.320082+0000","last_became_peered":"2024-04-05T22:31:42.320082+0000","last_unstale":"2024-04-05T22:31:42.323209+0000","last_undegraded":"2024-04-05T22:31:42.323209+0000","last_fullsized":"2024-04-05T22:31:42.323209+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:14:38.808826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344379+0000","last_change":"2024-04-05T22:31:41.851643+0000","last_active":"2024-04-05T22:31:42.344379+0000","last_peered":"2024-04-05T22:31:42.344379+0000","last_clean":"2024-04-05T22:31:42.344379+0000","last_became_active":"2024-04-05T22:31:41.851399+0000","last_became_peered":"2024-04-05T22:31:41.851399+0000","last_unstale":"2024-04-05T22:31:42.344379+0000","last_undegraded":"2024-04-05T22:31:42.344379+0000","last_fullsized":"2024-04-05T22:31:42.344379+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:20.548841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333666+0000","last_change":"2024-04-05T22:31:42.331888+0000","last_active":"2024-04-05T22:31:42.333666+0000","last_peered":"2024-04-05T22:31:42.333666+0000","last_clean":"2024-04-05T22:31:42.333666+0000","last_became_active":"2024-04-05T22:31:42.331601+0000","last_became_peered":"2024-04-05T22:31:42.331601+0000","last_unstale":"2024-04-05T22:31:42.333666+0000","last_undegraded":"2024-04-05T22:31:42.333666+0000","last_fullsized":"2024-04-05T22:31:42.333666+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:32.458969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349305+0000","last_change":"2024-04-05T22:31:41.854869+0000","last_active":"2024-04-05T22:31:42.349305+0000","last_peered":"2024-04-05T22:31:42.349305+0000","last_clean":"2024-04-05T22:31:42.349305+0000","last_became_active":"2024-04-05T22:31:41.854494+0000","last_became_peered":"2024-04-05T22:31:41.854494+0000","last_unstale":"2024-04-05T22:31:42.349305+0000","last_undegraded":"2024-04-05T22:31:42.349305+0000","last_fullsized":"2024-04-05T22:31:42.349305+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:43:12.551360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348435+0000","last_change":"2024-04-05T22:31:42.342237+0000","last_active":"2024-04-05T22:31:42.348435+0000","last_peered":"2024-04-05T22:31:42.348435+0000","last_clean":"2024-04-05T22:31:42.348435+0000","last_became_active":"2024-04-05T22:31:42.341862+0000","last_became_peered":"2024-04-05T22:31:42.341862+0000","last_unstale":"2024-04-05T22:31:42.348435+0000","last_undegraded":"2024-04-05T22:31:42.348435+0000","last_fullsized":"2024-04-05T22:31:42.348435+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:09.348577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336433+0000","last_change":"2024-04-05T22:31:41.831405+0000","last_active":"2024-04-05T22:31:42.336433+0000","last_peered":"2024-04-05T22:31:42.336433+0000","last_clean":"2024-04-05T22:31:42.336433+0000","last_became_active":"2024-04-05T22:31:41.831007+0000","last_became_peered":"2024-04-05T22:31:41.831007+0000","last_unstale":"2024-04-05T22:31:42.336433+0000","last_undegraded":"2024-04-05T22:31:42.336433+0000","last_fullsized":"2024-04-05T22:31:42.336433+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:44.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.341837+0000","last_change":"2024-04-05T22:31:42.339844+0000","last_active":"2024-04-05T22:31:42.341837+0000","last_peered":"2024-04-05T22:31:42.341837+0000","last_clean":"2024-04-05T22:31:42.341837+0000","last_became_active":"2024-04-05T22:31:42.339600+0000","last_became_peered":"2024-04-05T22:31:42.339600+0000","last_unstale":"2024-04-05T22:31:42.341837+0000","last_undegraded":"2024-04-05T22:31:42.341837+0000","last_fullsized":"2024-04-05T22:31:42.341837+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:28.415076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314546+0000","last_change":"2024-04-05T22:31:41.847754+0000","last_active":"2024-04-05T22:31:42.314546+0000","last_peered":"2024-04-05T22:31:42.314546+0000","last_clean":"2024-04-05T22:31:42.314546+0000","last_became_active":"2024-04-05T22:31:41.844413+0000","last_became_peered":"2024-04-05T22:31:41.844413+0000","last_unstale":"2024-04-05T22:31:42.314546+0000","last_undegraded":"2024-04-05T22:31:42.314546+0000","last_fullsized":"2024-04-05T22:31:42.314546+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:56.890791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349390+0000","last_change":"2024-04-05T22:31:42.344400+0000","last_active":"2024-04-05T22:31:42.349390+0000","last_peered":"2024-04-05T22:31:42.349390+0000","last_clean":"2024-04-05T22:31:42.349390+0000","last_became_active":"2024-04-05T22:31:42.344066+0000","last_became_peered":"2024-04-05T22:31:42.344066+0000","last_unstale":"2024-04-05T22:31:42.349390+0000","last_undegraded":"2024-04-05T22:31:42.349390+0000","last_fullsized":"2024-04-05T22:31:42.349390+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:42.495611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.311718+0000","last_change":"2024-04-05T22:31:41.841678+0000","last_active":"2024-04-05T22:31:42.311718+0000","last_peered":"2024-04-05T22:31:42.311718+0000","last_clean":"2024-04-05T22:31:42.311718+0000","last_became_active":"2024-04-05T22:31:41.841303+0000","last_became_peered":"2024-04-05T22:31:41.841303+0000","last_unstale":"2024-04-05T22:31:42.311718+0000","last_undegraded":"2024-04-05T22:31:42.311718+0000","last_fullsized":"2024-04-05T22:31:42.311718+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:48.467450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.350083+0000","last_change":"2024-04-05T22:31:42.348222+0000","last_active":"2024-04-05T22:31:42.350083+0000","last_peered":"2024-04-05T22:31:42.350083+0000","last_clean":"2024-04-05T22:31:42.350083+0000","last_became_active":"2024-04-05T22:31:42.347935+0000","last_became_peered":"2024-04-05T22:31:42.347935+0000","last_unstale":"2024-04-05T22:31:42.350083+0000","last_undegraded":"2024-04-05T22:31:42.350083+0000","last_fullsized":"2024-04-05T22:31:42.350083+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:31:40.592103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314776+0000","last_change":"2024-04-05T22:31:41.856671+0000","last_active":"2024-04-05T22:31:42.314776+0000","last_peered":"2024-04-05T22:31:42.314776+0000","last_clean":"2024-04-05T22:31:42.314776+0000","last_became_active":"2024-04-05T22:31:41.856314+0000","last_became_peered":"2024-04-05T22:31:41.856314+0000","last_unstale":"2024-04-05T22:31:42.314776+0000","last_undegraded":"2024-04-05T22:31:42.314776+0000","last_fullsized":"2024-04-05T22:31:42.314776+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:41.480287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339214+0000","last_change":"2024-04-05T22:31:42.334931+0000","last_active":"2024-04-05T22:31:42.339214+0000","last_peered":"2024-04-05T22:31:42.339214+0000","last_clean":"2024-04-05T22:31:42.339214+0000","last_became_active":"2024-04-05T22:31:42.334518+0000","last_became_peered":"2024-04-05T22:31:42.334518+0000","last_unstale":"2024-04-05T22:31:42.339214+0000","last_undegraded":"2024-04-05T22:31:42.339214+0000","last_fullsized":"2024-04-05T22:31:42.339214+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:19.428791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.317451+0000","last_change":"2024-04-05T22:31:41.852167+0000","last_active":"2024-04-05T22:31:42.317451+0000","last_peered":"2024-04-05T22:31:42.317451+0000","last_clean":"2024-04-05T22:31:42.317451+0000","last_became_active":"2024-04-05T22:31:41.851854+0000","last_became_peered":"2024-04-05T22:31:41.851854+0000","last_unstale":"2024-04-05T22:31:42.317451+0000","last_undegraded":"2024-04-05T22:31:42.317451+0000","last_fullsized":"2024-04-05T22:31:42.317451+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:35:04.539933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349512+0000","last_change":"2024-04-05T22:31:42.346019+0000","last_active":"2024-04-05T22:31:42.349512+0000","last_peered":"2024-04-05T22:31:42.349512+0000","last_clean":"2024-04-05T22:31:42.349512+0000","last_became_active":"2024-04-05T22:31:42.345629+0000","last_became_peered":"2024-04-05T22:31:42.345629+0000","last_unstale":"2024-04-05T22:31:42.349512+0000","last_undegraded":"2024-04-05T22:31:42.349512+0000","last_fullsized":"2024-04-05T22:31:42.349512+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:10.992539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"171'3","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.331881+0000","last_change":"2024-04-05T22:31:41.847753+0000","last_active":"2024-04-05T22:31:42.331881+0000","last_peered":"2024-04-05T22:31:42.331881+0000","last_clean":"2024-04-05T22:31:42.331881+0000","last_became_active":"2024-04-05T22:31:41.847477+0000","last_became_peered":"2024-04-05T22:31:41.847477+0000","last_unstale":"2024-04-05T22:31:42.331881+0000","last_undegraded":"2024-04-05T22:31:42.331881+0000","last_fullsized":"2024-04-05T22:31:42.331881+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.112289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.328052+0000","last_change":"2024-04-05T22:31:42.324785+0000","last_active":"2024-04-05T22:31:42.328052+0000","last_peered":"2024-04-05T22:31:42.328052+0000","last_clean":"2024-04-05T22:31:42.328052+0000","last_became_active":"2024-04-05T22:31:42.324569+0000","last_became_peered":"2024-04-05T22:31:42.324569+0000","last_unstale":"2024-04-05T22:31:42.328052+0000","last_undegraded":"2024-04-05T22:31:42.328052+0000","last_fullsized":"2024-04-05T22:31:42.328052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:37:37.102529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"171'6","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.351645+0000","last_change":"2024-04-05T22:31:41.849643+0000","last_active":"2024-04-05T22:31:42.351645+0000","last_peered":"2024-04-05T22:31:42.351645+0000","last_clean":"2024-04-05T22:31:42.351645+0000","last_became_active":"2024-04-05T22:31:41.849341+0000","last_became_peered":"2024-04-05T22:31:41.849341+0000","last_unstale":"2024-04-05T22:31:42.351645+0000","last_undegraded":"2024-04-05T22:31:42.351645+0000","last_fullsized":"2024-04-05T22:31:42.351645+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:49:50.800680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.331159+0000","last_change":"2024-04-05T22:31:42.325837+0000","last_active":"2024-04-05T22:31:42.331159+0000","last_peered":"2024-04-05T22:31:42.331159+0000","last_clean":"2024-04-05T22:31:42.331159+0000","last_became_active":"2024-04-05T22:31:42.325601+0000","last_became_peered":"2024-04-05T22:31:42.325601+0000","last_unstale":"2024-04-05T22:31:42.331159+0000","last_undegraded":"2024-04-05T22:31:42.331159+0000","last_fullsized":"2024-04-05T22:31:42.331159+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:11.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336673+0000","last_change":"2024-04-05T22:31:41.837689+0000","last_active":"2024-04-05T22:31:42.336673+0000","last_peered":"2024-04-05T22:31:42.336673+0000","last_clean":"2024-04-05T22:31:42.336673+0000","last_became_active":"2024-04-05T22:31:41.836071+0000","last_became_peered":"2024-04-05T22:31:41.836071+0000","last_unstale":"2024-04-05T22:31:42.336673+0000","last_undegraded":"2024-04-05T22:31:42.336673+0000","last_fullsized":"2024-04-05T22:31:42.336673+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:04:18.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339297+0000","last_change":"2024-04-05T22:31:42.338349+0000","last_active":"2024-04-05T22:31:42.339297+0000","last_peered":"2024-04-05T22:31:42.339297+0000","last_clean":"2024-04-05T22:31:42.339297+0000","last_became_active":"2024-04-05T22:31:42.338075+0000","last_became_peered":"2024-04-05T22:31:42.338075+0000","last_unstale":"2024-04-05T22:31:42.339297+0000","last_undegraded":"2024-04-05T22:31:42.339297+0000","last_fullsized":"2024-04-05T22:31:42.339297+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:00.436586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.311617+0000","last_change":"2024-04-05T22:31:41.848610+0000","last_active":"2024-04-05T22:31:42.311617+0000","last_peered":"2024-04-05T22:31:42.311617+0000","last_clean":"2024-04-05T22:31:42.311617+0000","last_became_active":"2024-04-05T22:31:41.848296+0000","last_became_peered":"2024-04-05T22:31:41.848296+0000","last_unstale":"2024-04-05T22:31:42.311617+0000","last_undegraded":"2024-04-05T22:31:42.311617+0000","last_fullsized":"2024-04-05T22:31:42.311617+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:15.479974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339127+0000","last_change":"2024-04-05T22:31:42.334497+0000","last_active":"2024-04-05T22:31:42.339127+0000","last_peered":"2024-04-05T22:31:42.339127+0000","last_clean":"2024-04-05T22:31:42.339127+0000","last_became_active":"2024-04-05T22:31:42.334246+0000","last_became_peered":"2024-04-05T22:31:42.334246+0000","last_unstale":"2024-04-05T22:31:42.339127+0000","last_undegraded":"2024-04-05T22:31:42.339127+0000","last_fullsized":"2024-04-05T22:31:42.339127+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:42.495611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321389+0000","last_change":"2024-04-05T22:31:41.849675+0000","last_active":"2024-04-05T22:31:42.321389+0000","last_peered":"2024-04-05T22:31:42.321389+0000","last_clean":"2024-04-05T22:31:42.321389+0000","last_became_active":"2024-04-05T22:31:41.849367+0000","last_became_peered":"2024-04-05T22:31:41.849367+0000","last_unstale":"2024-04-05T22:31:42.321389+0000","last_undegraded":"2024-04-05T22:31:42.321389+0000","last_fullsized":"2024-04-05T22:31:42.321389+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:27.399752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347823+0000","last_change":"2024-04-05T22:31:42.347022+0000","last_active":"2024-04-05T22:31:42.347823+0000","last_peered":"2024-04-05T22:31:42.347823+0000","last_clean":"2024-04-05T22:31:42.347823+0000","last_became_active":"2024-04-05T22:31:42.346728+0000","last_became_peered":"2024-04-05T22:31:42.346728+0000","last_unstale":"2024-04-05T22:31:42.347823+0000","last_undegraded":"2024-04-05T22:31:42.347823+0000","last_fullsized":"2024-04-05T22:31:42.347823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:33.673205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"171'8","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348267+0000","last_change":"2024-04-05T22:31:41.848381+0000","last_active":"2024-04-05T22:31:42.348267+0000","last_peered":"2024-04-05T22:31:42.348267+0000","last_clean":"2024-04-05T22:31:42.348267+0000","last_became_active":"2024-04-05T22:31:41.848109+0000","last_became_peered":"2024-04-05T22:31:41.848109+0000","last_unstale":"2024-04-05T22:31:42.348267+0000","last_undegraded":"2024-04-05T22:31:42.348267+0000","last_fullsized":"2024-04-05T22:31:42.348267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:35:04.539933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.351546+0000","last_change":"2024-04-05T22:31:42.350283+0000","last_active":"2024-04-05T22:31:42.351546+0000","last_peered":"2024-04-05T22:31:42.351546+0000","last_clean":"2024-04-05T22:31:42.351546+0000","last_became_active":"2024-04-05T22:31:42.350001+0000","last_became_peered":"2024-04-05T22:31:42.350001+0000","last_unstale":"2024-04-05T22:31:42.351546+0000","last_undegraded":"2024-04-05T22:31:42.351546+0000","last_fullsized":"2024-04-05T22:31:42.351546+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:21.564165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.327734+0000","last_change":"2024-04-05T22:31:41.838712+0000","last_active":"2024-04-05T22:31:42.327734+0000","last_peered":"2024-04-05T22:31:42.327734+0000","last_clean":"2024-04-05T22:31:42.327734+0000","last_became_active":"2024-04-05T22:31:41.838122+0000","last_became_peered":"2024-04-05T22:31:41.838122+0000","last_unstale":"2024-04-05T22:31:42.327734+0000","last_undegraded":"2024-04-05T22:31:42.327734+0000","last_fullsized":"2024-04-05T22:31:42.327734+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:13.553986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345739+0000","last_change":"2024-04-05T22:31:42.342604+0000","last_active":"2024-04-05T22:31:42.345739+0000","last_peered":"2024-04-05T22:31:42.345739+0000","last_clean":"2024-04-05T22:31:42.345739+0000","last_became_active":"2024-04-05T22:31:42.342118+0000","last_became_peered":"2024-04-05T22:31:42.342118+0000","last_unstale":"2024-04-05T22:31:42.345739+0000","last_undegraded":"2024-04-05T22:31:42.345739+0000","last_fullsized":"2024-04-05T22:31:42.345739+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:21:00.091010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.341933+0000","last_change":"2024-04-05T22:31:41.855537+0000","last_active":"2024-04-05T22:31:42.341933+0000","last_peered":"2024-04-05T22:31:42.341933+0000","last_clean":"2024-04-05T22:31:42.341933+0000","last_became_active":"2024-04-05T22:31:41.854914+0000","last_became_peered":"2024-04-05T22:31:41.854914+0000","last_unstale":"2024-04-05T22:31:42.341933+0000","last_undegraded":"2024-04-05T22:31:42.341933+0000","last_fullsized":"2024-04-05T22:31:42.341933+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:13:28.327743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346244+0000","last_change":"2024-04-05T22:31:42.345098+0000","last_active":"2024-04-05T22:31:42.346244+0000","last_peered":"2024-04-05T22:31:42.346244+0000","last_clean":"2024-04-05T22:31:42.346244+0000","last_became_active":"2024-04-05T22:31:42.344285+0000","last_became_peered":"2024-04-05T22:31:42.344285+0000","last_unstale":"2024-04-05T22:31:42.346244+0000","last_undegraded":"2024-04-05T22:31:42.346244+0000","last_fullsized":"2024-04-05T22:31:42.346244+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:14.569310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.340649+0000","last_change":"2024-04-05T22:31:41.831117+0000","last_active":"2024-04-05T22:31:42.340649+0000","last_peered":"2024-04-05T22:31:42.340649+0000","last_clean":"2024-04-05T22:31:42.340649+0000","last_became_active":"2024-04-05T22:31:41.829649+0000","last_became_peered":"2024-04-05T22:31:41.829649+0000","last_unstale":"2024-04-05T22:31:42.340649+0000","last_undegraded":"2024-04-05T22:31:42.340649+0000","last_fullsized":"2024-04-05T22:31:42.340649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:41.515181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346336+0000","last_change":"2024-04-05T22:31:42.345531+0000","last_active":"2024-04-05T22:31:42.346336+0000","last_peered":"2024-04-05T22:31:42.346336+0000","last_clean":"2024-04-05T22:31:42.346336+0000","last_became_active":"2024-04-05T22:31:42.344852+0000","last_became_peered":"2024-04-05T22:31:42.344852+0000","last_unstale":"2024-04-05T22:31:42.346336+0000","last_undegraded":"2024-04-05T22:31:42.346336+0000","last_fullsized":"2024-04-05T22:31:42.346336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:58:50.399735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.305845+0000","last_change":"2024-04-05T22:31:41.846499+0000","last_active":"2024-04-05T22:31:42.305845+0000","last_peered":"2024-04-05T22:31:42.305845+0000","last_clean":"2024-04-05T22:31:42.305845+0000","last_became_active":"2024-04-05T22:31:41.846216+0000","last_became_peered":"2024-04-05T22:31:41.846216+0000","last_unstale":"2024-04-05T22:31:42.305845+0000","last_undegraded":"2024-04-05T22:31:42.305845+0000","last_fullsized":"2024-04-05T22:31:42.305845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:15.479974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.325547+0000","last_change":"2024-04-05T22:31:42.323945+0000","last_active":"2024-04-05T22:31:42.325547+0000","last_peered":"2024-04-05T22:31:42.325547+0000","last_clean":"2024-04-05T22:31:42.325547+0000","last_became_active":"2024-04-05T22:31:42.323704+0000","last_became_peered":"2024-04-05T22:31:42.323704+0000","last_unstale":"2024-04-05T22:31:42.325547+0000","last_undegraded":"2024-04-05T22:31:42.325547+0000","last_fullsized":"2024-04-05T22:31:42.325547+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:44.121984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329575+0000","last_change":"2024-04-05T22:31:41.843680+0000","last_active":"2024-04-05T22:31:42.329575+0000","last_peered":"2024-04-05T22:31:42.329575+0000","last_clean":"2024-04-05T22:31:42.329575+0000","last_became_active":"2024-04-05T22:31:41.843319+0000","last_became_peered":"2024-04-05T22:31:41.843319+0000","last_unstale":"2024-04-05T22:31:42.329575+0000","last_undegraded":"2024-04-05T22:31:42.329575+0000","last_fullsized":"2024-04-05T22:31:42.329575+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:42.354442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321111+0000","last_change":"2024-04-05T22:31:42.317706+0000","last_active":"2024-04-05T22:31:42.321111+0000","last_peered":"2024-04-05T22:31:42.321111+0000","last_clean":"2024-04-05T22:31:42.321111+0000","last_became_active":"2024-04-05T22:31:42.317418+0000","last_became_peered":"2024-04-05T22:31:42.317418+0000","last_unstale":"2024-04-05T22:31:42.321111+0000","last_undegraded":"2024-04-05T22:31:42.321111+0000","last_fullsized":"2024-04-05T22:31:42.321111+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:57.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.310888+0000","last_change":"2024-04-05T22:31:41.852712+0000","last_active":"2024-04-05T22:31:42.310888+0000","last_peered":"2024-04-05T22:31:42.310888+0000","last_clean":"2024-04-05T22:31:42.310888+0000","last_became_active":"2024-04-05T22:31:41.852383+0000","last_became_peered":"2024-04-05T22:31:41.852383+0000","last_unstale":"2024-04-05T22:31:42.310888+0000","last_undegraded":"2024-04-05T22:31:42.310888+0000","last_fullsized":"2024-04-05T22:31:42.310888+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:41.515181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345102+0000","last_change":"2024-04-05T22:31:42.337678+0000","last_active":"2024-04-05T22:31:42.345102+0000","last_peered":"2024-04-05T22:31:42.345102+0000","last_clean":"2024-04-05T22:31:42.345102+0000","last_became_active":"2024-04-05T22:31:42.336826+0000","last_became_peered":"2024-04-05T22:31:42.336826+0000","last_unstale":"2024-04-05T22:31:42.345102+0000","last_undegraded":"2024-04-05T22:31:42.345102+0000","last_fullsized":"2024-04-05T22:31:42.345102+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:16.495298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.309717+0000","last_change":"2024-04-05T22:31:41.851514+0000","last_active":"2024-04-05T22:31:42.309717+0000","last_peered":"2024-04-05T22:31:42.309717+0000","last_clean":"2024-04-05T22:31:42.309717+0000","last_became_active":"2024-04-05T22:31:41.850997+0000","last_became_peered":"2024-04-05T22:31:41.850997+0000","last_unstale":"2024-04-05T22:31:42.309717+0000","last_undegraded":"2024-04-05T22:31:42.309717+0000","last_fullsized":"2024-04-05T22:31:42.309717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:04:59.160117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342471+0000","last_change":"2024-04-05T22:31:42.339215+0000","last_active":"2024-04-05T22:31:42.342471+0000","last_peered":"2024-04-05T22:31:42.342471+0000","last_clean":"2024-04-05T22:31:42.342471+0000","last_became_active":"2024-04-05T22:31:42.338033+0000","last_became_peered":"2024-04-05T22:31:42.338033+0000","last_unstale":"2024-04-05T22:31:42.342471+0000","last_undegraded":"2024-04-05T22:31:42.342471+0000","last_fullsized":"2024-04-05T22:31:42.342471+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:59:24.393944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314447+0000","last_change":"2024-04-05T22:31:41.842721+0000","last_active":"2024-04-05T22:31:42.314447+0000","last_peered":"2024-04-05T22:31:42.314447+0000","last_clean":"2024-04-05T22:31:42.314447+0000","last_became_active":"2024-04-05T22:31:41.839296+0000","last_became_peered":"2024-04-05T22:31:41.839296+0000","last_unstale":"2024-04-05T22:31:42.314447+0000","last_undegraded":"2024-04-05T22:31:42.314447+0000","last_fullsized":"2024-04-05T22:31:42.314447+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:18.413467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342667+0000","last_change":"2024-04-05T22:31:42.333854+0000","last_active":"2024-04-05T22:31:42.342667+0000","last_peered":"2024-04-05T22:31:42.342667+0000","last_clean":"2024-04-05T22:31:42.342667+0000","last_became_active":"2024-04-05T22:31:42.333289+0000","last_became_peered":"2024-04-05T22:31:42.333289+0000","last_unstale":"2024-04-05T22:31:42.342667+0000","last_undegraded":"2024-04-05T22:31:42.342667+0000","last_fullsized":"2024-04-05T22:31:42.342667+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:07.572633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344059+0000","last_change":"2024-04-05T22:31:41.856664+0000","last_active":"2024-04-05T22:31:42.344059+0000","last_peered":"2024-04-05T22:31:42.344059+0000","last_clean":"2024-04-05T22:31:42.344059+0000","last_became_active":"2024-04-05T22:31:41.856324+0000","last_became_peered":"2024-04-05T22:31:41.856324+0000","last_unstale":"2024-04-05T22:31:42.344059+0000","last_undegraded":"2024-04-05T22:31:42.344059+0000","last_fullsized":"2024-04-05T22:31:42.344059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:48.467450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336836+0000","last_change":"2024-04-05T22:31:42.335543+0000","last_active":"2024-04-05T22:31:42.336836+0000","last_peered":"2024-04-05T22:31:42.336836+0000","last_clean":"2024-04-05T22:31:42.336836+0000","last_became_active":"2024-04-05T22:31:42.334702+0000","last_became_peered":"2024-04-05T22:31:42.334702+0000","last_unstale":"2024-04-05T22:31:42.336836+0000","last_undegraded":"2024-04-05T22:31:42.336836+0000","last_fullsized":"2024-04-05T22:31:42.336836+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:12:00.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333093+0000","last_change":"2024-04-05T22:31:41.851650+0000","last_active":"2024-04-05T22:31:42.333093+0000","last_peered":"2024-04-05T22:31:42.333093+0000","last_clean":"2024-04-05T22:31:42.333093+0000","last_became_active":"2024-04-05T22:31:41.851087+0000","last_became_peered":"2024-04-05T22:31:41.851087+0000","last_unstale":"2024-04-05T22:31:42.333093+0000","last_undegraded":"2024-04-05T22:31:42.333093+0000","last_fullsized":"2024-04-05T22:31:42.333093+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:58.100094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345381+0000","last_change":"2024-04-05T22:31:42.341975+0000","last_active":"2024-04-05T22:31:42.345381+0000","last_peered":"2024-04-05T22:31:42.345381+0000","last_clean":"2024-04-05T22:31:42.345381+0000","last_became_active":"2024-04-05T22:31:42.341702+0000","last_became_peered":"2024-04-05T22:31:42.341702+0000","last_unstale":"2024-04-05T22:31:42.345381+0000","last_undegraded":"2024-04-05T22:31:42.345381+0000","last_fullsized":"2024-04-05T22:31:42.345381+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:26:47.664135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314671+0000","last_change":"2024-04-05T22:31:41.846878+0000","last_active":"2024-04-05T22:31:42.314671+0000","last_peered":"2024-04-05T22:31:42.314671+0000","last_clean":"2024-04-05T22:31:42.314671+0000","last_became_active":"2024-04-05T22:31:41.846583+0000","last_became_peered":"2024-04-05T22:31:41.846583+0000","last_unstale":"2024-04-05T22:31:42.314671+0000","last_undegraded":"2024-04-05T22:31:42.314671+0000","last_fullsized":"2024-04-05T22:31:42.314671+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:08.333253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333232+0000","last_change":"2024-04-05T22:31:42.324334+0000","last_active":"2024-04-05T22:31:42.333232+0000","last_peered":"2024-04-05T22:31:42.333232+0000","last_clean":"2024-04-05T22:31:42.333232+0000","last_became_active":"2024-04-05T22:31:42.319871+0000","last_became_peered":"2024-04-05T22:31:42.319871+0000","last_unstale":"2024-04-05T22:31:42.333232+0000","last_undegraded":"2024-04-05T22:31:42.333232+0000","last_fullsized":"2024-04-05T22:31:42.333232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:57.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.305931+0000","last_change":"2024-04-05T22:31:41.852151+0000","last_active":"2024-04-05T22:31:42.305931+0000","last_peered":"2024-04-05T22:31:42.305931+0000","last_clean":"2024-04-05T22:31:42.305931+0000","last_became_active":"2024-04-05T22:31:41.851846+0000","last_became_peered":"2024-04-05T22:31:41.851846+0000","last_unstale":"2024-04-05T22:31:42.305931+0000","last_undegraded":"2024-04-05T22:31:42.305931+0000","last_fullsized":"2024-04-05T22:31:42.305931+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:58.100094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336699+0000","last_change":"2024-04-05T22:31:42.334826+0000","last_active":"2024-04-05T22:31:42.336699+0000","last_peered":"2024-04-05T22:31:42.336699+0000","last_clean":"2024-04-05T22:31:42.336699+0000","last_became_active":"2024-04-05T22:31:42.334599+0000","last_became_peered":"2024-04-05T22:31:42.334599+0000","last_unstale":"2024-04-05T22:31:42.336699+0000","last_undegraded":"2024-04-05T22:31:42.336699+0000","last_fullsized":"2024-04-05T22:31:42.336699+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:22.717040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329041+0000","last_change":"2024-04-05T22:31:41.852462+0000","last_active":"2024-04-05T22:31:42.329041+0000","last_peered":"2024-04-05T22:31:42.329041+0000","last_clean":"2024-04-05T22:31:42.329041+0000","last_became_active":"2024-04-05T22:31:41.852160+0000","last_became_peered":"2024-04-05T22:31:41.852160+0000","last_unstale":"2024-04-05T22:31:42.329041+0000","last_undegraded":"2024-04-05T22:31:42.329041+0000","last_fullsized":"2024-04-05T22:31:42.329041+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:20.548841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342569+0000","last_change":"2024-04-05T22:31:42.337518+0000","last_active":"2024-04-05T22:31:42.342569+0000","last_peered":"2024-04-05T22:31:42.342569+0000","last_clean":"2024-04-05T22:31:42.342569+0000","last_became_active":"2024-04-05T22:31:42.336674+0000","last_became_peered":"2024-04-05T22:31:42.336674+0000","last_unstale":"2024-04-05T22:31:42.342569+0000","last_undegraded":"2024-04-05T22:31:42.342569+0000","last_fullsized":"2024-04-05T22:31:42.342569+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:49.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314491+0000","last_change":"2024-04-05T22:31:41.833839+0000","last_active":"2024-04-05T22:31:42.314491+0000","last_peered":"2024-04-05T22:31:42.314491+0000","last_clean":"2024-04-05T22:31:42.314491+0000","last_became_active":"2024-04-05T22:31:41.833408+0000","last_became_peered":"2024-04-05T22:31:41.833408+0000","last_unstale":"2024-04-05T22:31:42.314491+0000","last_undegraded":"2024-04-05T22:31:42.314491+0000","last_fullsized":"2024-04-05T22:31:42.314491+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:09.977215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339109+0000","last_change":"2024-04-05T22:31:42.334142+0000","last_active":"2024-04-05T22:31:42.339109+0000","last_peered":"2024-04-05T22:31:42.339109+0000","last_clean":"2024-04-05T22:31:42.339109+0000","last_became_active":"2024-04-05T22:31:42.333912+0000","last_became_peered":"2024-04-05T22:31:42.333912+0000","last_unstale":"2024-04-05T22:31:42.339109+0000","last_undegraded":"2024-04-05T22:31:42.339109+0000","last_fullsized":"2024-04-05T22:31:42.339109+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:09.348577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.312304+0000","last_change":"2024-04-05T22:31:41.848102+0000","last_active":"2024-04-05T22:31:42.312304+0000","last_peered":"2024-04-05T22:31:42.312304+0000","last_clean":"2024-04-05T22:31:42.312304+0000","last_became_active":"2024-04-05T22:31:41.847799+0000","last_became_peered":"2024-04-05T22:31:41.847799+0000","last_unstale":"2024-04-05T22:31:42.312304+0000","last_undegraded":"2024-04-05T22:31:42.312304+0000","last_fullsized":"2024-04-05T22:31:42.312304+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:56.325069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347644+0000","last_change":"2024-04-05T22:31:42.344352+0000","last_active":"2024-04-05T22:31:42.347644+0000","last_peered":"2024-04-05T22:31:42.347644+0000","last_clean":"2024-04-05T22:31:42.347644+0000","last_became_active":"2024-04-05T22:31:42.341062+0000","last_became_peered":"2024-04-05T22:31:42.341062+0000","last_unstale":"2024-04-05T22:31:42.347644+0000","last_undegraded":"2024-04-05T22:31:42.347644+0000","last_fullsized":"2024-04-05T22:31:42.347644+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:49.482774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.327234+0000","last_change":"2024-04-05T22:31:41.851856+0000","last_active":"2024-04-05T22:31:42.327234+0000","last_peered":"2024-04-05T22:31:42.327234+0000","last_clean":"2024-04-05T22:31:42.327234+0000","last_became_active":"2024-04-05T22:31:41.851428+0000","last_became_peered":"2024-04-05T22:31:41.851428+0000","last_unstale":"2024-04-05T22:31:42.327234+0000","last_undegraded":"2024-04-05T22:31:42.327234+0000","last_fullsized":"2024-04-05T22:31:42.327234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:25.186825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321830+0000","last_change":"2024-04-05T22:31:42.318977+0000","last_active":"2024-04-05T22:31:42.321830+0000","last_peered":"2024-04-05T22:31:42.321830+0000","last_clean":"2024-04-05T22:31:42.321830+0000","last_became_active":"2024-04-05T22:31:42.318687+0000","last_became_peered":"2024-04-05T22:31:42.318687+0000","last_unstale":"2024-04-05T22:31:42.321830+0000","last_undegraded":"2024-04-05T22:31:42.321830+0000","last_fullsized":"2024-04-05T22:31:42.321830+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:49:55.770780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336935+0000","last_change":"2024-04-05T22:31:41.848177+0000","last_active":"2024-04-05T22:31:42.336935+0000","last_peered":"2024-04-05T22:31:42.336935+0000","last_clean":"2024-04-05T22:31:42.336935+0000","last_became_active":"2024-04-05T22:31:41.846549+0000","last_became_peered":"2024-04-05T22:31:41.846549+0000","last_unstale":"2024-04-05T22:31:42.336935+0000","last_undegraded":"2024-04-05T22:31:42.336935+0000","last_fullsized":"2024-04-05T22:31:42.336935+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:25:34.690546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":331,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-05T22:31:41.841175+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:31:41.841175+0000","last_peered":"2024-04-05T22:31:41.841175+0000","last_clean":"2024-04-05T22:31:41.841175+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:31:41.841175+0000","last_undegraded":"2024-04-05T22:31:41.841175+0000","last_fullsized":"2024-04-05T22:31:41.841175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542274,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37884,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333956,"statfs":{"total":96636764160,"available":96597970944,"internally_reserved":0,"allocated":2736128,"data_stored":1533728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.133},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":0.98399999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.292}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.49},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.246},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.413}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.081},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37304,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334536,"statfs":{"total":96636764160,"available":96598564864,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.181},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.5880000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.262},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.98699999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.7150000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.669},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.754}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.454}]}]},{"osd":5,"up_from":14,"seq":60129542274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.84299999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.145},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.2230000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.079}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.157},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.361}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.4079999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542273,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":0.622}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.95099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.034},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.097}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":1.661}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":1.716},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.534}]}]},{"osd":2,"up_from":14,"seq":60129542273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2129920,"data_stored":937085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.874}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.526},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.107}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.333}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.25},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.3}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.151},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.458},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.393}]}]},{"osd":1,"up_from":14,"seq":60129542275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.083}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2190000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":1.117},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.2430000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.327}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.1779999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542275,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37904,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333936,"statfs":{"total":96636764160,"available":96597950464,"internally_reserved":0,"allocated":2752512,"data_stored":1540593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.88}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.621}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":1.034},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.73}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":1.24}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.593}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.5549999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:31:46.729 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:31:46.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.980+0000 7fb674e00640 1 Processor -- start 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.980+0000 7fb674e00640 1 -- start start 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb674e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb67015cf70 0x7fb6700d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb674e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 0x7fb6700d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb674e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6700ceee0 con 0x7fb67015d350 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb674e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700ce5b0 con 0x7fb67015cf70 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb674e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ce890 con 0x7fb6700d0bd0 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb67015cf70 0x7fb6700d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb67015cf70 0x7fb6700d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42982/0 (socket says 172.21.15.73:42982) 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 -- 172.21.15.73:0/3132478154 learned_addr learned my addr 172.21.15.73:0/3132478154 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 -- 172.21.15.73:0/3132478154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 msgr2=0x7fb6700d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:46.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66d400640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 0x7fb6700d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 0x7fb6700d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 -- 172.21.15.73:0/3132478154 >> v1:172.21.15.157:6789/0 conn(0x7fb67015d350 legacy=0x7fb6700cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:46.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66de00640 1 -- 172.21.15.73:0/3132478154 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700ceb70 con 0x7fb67015cf70 2024-04-05T22:31:46.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.982+0000 7fb66d400640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 0x7fb6700d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:46.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.984+0000 7fb66de00640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015cf70 0x7fb6700d3910 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb660009870 tx=0x7fb66002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285cf4d1bc26a5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.984+0000 7fb66ca00640 1 -- 172.21.15.73:0/3132478154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600317c0 con 0x7fb67015cf70 2024-04-05T22:31:46.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.984+0000 7fb66ca00640 1 -- 172.21.15.73:0/3132478154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660031920 con 0x7fb67015cf70 2024-04-05T22:31:46.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.984+0000 7fb66ca00640 1 -- 172.21.15.73:0/3132478154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660031cb0 con 0x7fb67015cf70 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 -- 172.21.15.73:0/3132478154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015cf70 msgr2=0x7fb6700d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015cf70 0x7fb6700d3910 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb660009870 tx=0x7fb66002b860 comp rx=0 tx=0).stop 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 -- 172.21.15.73:0/3132478154 shutdown_connections 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6700d0bd0 0x7fb6700d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 --2- 172.21.15.73:0/3132478154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015cf70 0x7fb6700d3910 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 -- 172.21.15.73:0/3132478154 >> 172.21.15.73:0/3132478154 conn(0x7fb6700d2df0 msgr2=0x7fb6700d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 -- 172.21.15.73:0/3132478154 shutdown_connections 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.985+0000 7fb674e00640 1 -- 172.21.15.73:0/3132478154 wait complete. 2024-04-05T22:31:46.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.987+0000 7fb674e00640 1 Processor -- start 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.987+0000 7fb674e00640 1 -- start start 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 0x7fb6700b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb67015cf70 0x7fb67015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700c7340 con 0x7fb67015d350 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700c6750 con 0x7fb6700d0bd0 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb674e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700c69a0 con 0x7fb67015cf70 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42998/0 (socket says 172.21.15.73:42998) 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66e800640 1 -- 172.21.15.73:0/3221465918 learned_addr learned my addr 172.21.15.73:0/3221465918 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:46.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66d400640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb67015cf70 0x7fb67015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66de00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 0x7fb6700b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66e800640 1 -- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb67015cf70 msgr2=0x7fb67015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.988+0000 7fb66e800640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb67015cf70 0x7fb67015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb66e800640 1 -- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 msgr2=0x7fb6700b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb66e800640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 0x7fb6700b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb66e800640 1 -- 172.21.15.73:0/3221465918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600094d0 con 0x7fb67015d350 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb66de00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 0x7fb6700b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb66e800640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb66800b990 tx=0x7fb66800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800cba0 con 0x7fb67015d350 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800cd00 con 0x7fb67015d350 2024-04-05T22:31:46.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668014680 con 0x7fb67015d350 2024-04-05T22:31:46.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6700c6c80 con 0x7fb67015d350 2024-04-05T22:31:46.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.989+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6700b0770 con 0x7fb67015d350 2024-04-05T22:31:46.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.990+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700c7480 con 0x7fb67015d350 2024-04-05T22:31:46.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.991+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fb66801d3b0 con 0x7fb67015d350 2024-04-05T22:31:46.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.992+0000 7fb666000640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 0x7fb65006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:46.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.992+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb668087db0 con 0x7fb67015d350 2024-04-05T22:31:46.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.993+0000 7fb66de00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 0x7fb65006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:46.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.994+0000 7fb66de00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 0x7fb65006da50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb660036000 tx=0x7fb66003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:46.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:46.997+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb6700c7480 con 0x7fb67015d350 2024-04-05T22:31:47.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.185+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6700c7480 con 0x7fb65006b590 2024-04-05T22:31:47.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.208+0000 7fb666000640 1 -- 172.21.15.73:0/3221465918 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287669 (secure 0 0 0) 0x7fb6700c7480 con 0x7fb65006b590 2024-04-05T22:31:47.211 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:47.213 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:31:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.214+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 msgr2=0x7fb65006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.214+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 0x7fb65006da50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb660036000 tx=0x7fb66003d040 comp rx=0 tx=0).stop 2024-04-05T22:31:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.214+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 msgr2=0x7fb67015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.214+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb66800b990 tx=0x7fb66800be50 comp rx=0 tx=0).stop 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 shutdown_connections 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb65006b590 0x7fb65006da50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67015d350 0x7fb67015b420 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb67015cf70 0x7fb67015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 --2- 172.21.15.73:0/3221465918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6700d0bd0 0x7fb6700b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 >> 172.21.15.73:0/3221465918 conn(0x7fb6700d2df0 msgr2=0x7fb6700b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 shutdown_connections 2024-04-05T22:31:47.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.215+0000 7fb674e00640 1 -- 172.21.15.73:0/3221465918 wait complete. 2024-04-05T22:31:47.234 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-04-05T22:31:45.612810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287324,"kb_used_data":17900,"kb_used_omap":12,"kb_used_meta":269299,"kb_avail":754687396,"statfs":{"total":773094113280,"available":772799893504,"internally_reserved":0,"allocated":18329600,"data_stored":8728206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":275762747},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.803507"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347292+0000","last_change":"2024-04-05T22:31:42.345705+0000","last_active":"2024-04-05T22:31:42.347292+0000","last_peered":"2024-04-05T22:31:42.347292+0000","last_clean":"2024-04-05T22:31:42.347292+0000","last_became_active":"2024-04-05T22:31:42.345434+0000","last_became_peered":"2024-04-05T22:31:42.345434+0000","last_unstale":"2024-04-05T22:31:42.347292+0000","last_undegraded":"2024-04-05T22:31:42.347292+0000","last_fullsized":"2024-04-05T22:31:42.347292+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:01.012844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345282+0000","last_change":"2024-04-05T22:31:42.342228+0000","last_active":"2024-04-05T22:31:42.345282+0000","last_peered":"2024-04-05T22:31:42.345282+0000","last_clean":"2024-04-05T22:31:42.345282+0000","last_became_active":"2024-04-05T22:31:42.341735+0000","last_became_peered":"2024-04-05T22:31:42.341735+0000","last_unstale":"2024-04-05T22:31:42.345282+0000","last_undegraded":"2024-04-05T22:31:42.345282+0000","last_fullsized":"2024-04-05T22:31:42.345282+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:59.115418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348885+0000","last_change":"2024-04-05T22:31:42.348080+0000","last_active":"2024-04-05T22:31:42.348885+0000","last_peered":"2024-04-05T22:31:42.348885+0000","last_clean":"2024-04-05T22:31:42.348885+0000","last_became_active":"2024-04-05T22:31:42.347754+0000","last_became_peered":"2024-04-05T22:31:42.347754+0000","last_unstale":"2024-04-05T22:31:42.348885+0000","last_undegraded":"2024-04-05T22:31:42.348885+0000","last_fullsized":"2024-04-05T22:31:42.348885+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:45.043543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349491+0000","last_change":"2024-04-05T22:31:42.345817+0000","last_active":"2024-04-05T22:31:42.349491+0000","last_peered":"2024-04-05T22:31:42.349491+0000","last_clean":"2024-04-05T22:31:42.349491+0000","last_became_active":"2024-04-05T22:31:42.345512+0000","last_became_peered":"2024-04-05T22:31:42.345512+0000","last_unstale":"2024-04-05T22:31:42.349491+0000","last_undegraded":"2024-04-05T22:31:42.349491+0000","last_fullsized":"2024-04-05T22:31:42.349491+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:19.428791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330226+0000","last_change":"2024-04-05T22:31:42.326414+0000","last_active":"2024-04-05T22:31:42.330226+0000","last_peered":"2024-04-05T22:31:42.330226+0000","last_clean":"2024-04-05T22:31:42.330226+0000","last_became_active":"2024-04-05T22:31:42.325817+0000","last_became_peered":"2024-04-05T22:31:42.325817+0000","last_unstale":"2024-04-05T22:31:42.330226+0000","last_undegraded":"2024-04-05T22:31:42.330226+0000","last_fullsized":"2024-04-05T22:31:42.330226+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:56:24.844384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.343984+0000","last_change":"2024-04-05T22:31:42.343387+0000","last_active":"2024-04-05T22:31:42.343984+0000","last_peered":"2024-04-05T22:31:42.343984+0000","last_clean":"2024-04-05T22:31:42.343984+0000","last_became_active":"2024-04-05T22:31:42.342966+0000","last_became_peered":"2024-04-05T22:31:42.342966+0000","last_unstale":"2024-04-05T22:31:42.343984+0000","last_undegraded":"2024-04-05T22:31:42.343984+0000","last_fullsized":"2024-04-05T22:31:42.343984+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:43.369766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.337235+0000","last_change":"2024-04-05T22:31:42.336569+0000","last_active":"2024-04-05T22:31:42.337235+0000","last_peered":"2024-04-05T22:31:42.337235+0000","last_clean":"2024-04-05T22:31:42.337235+0000","last_became_active":"2024-04-05T22:31:42.336279+0000","last_became_peered":"2024-04-05T22:31:42.336279+0000","last_unstale":"2024-04-05T22:31:42.337235+0000","last_undegraded":"2024-04-05T22:31:42.337235+0000","last_fullsized":"2024-04-05T22:31:42.337235+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:05:47.621581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321248+0000","last_change":"2024-04-05T22:31:42.319522+0000","last_active":"2024-04-05T22:31:42.321248+0000","last_peered":"2024-04-05T22:31:42.321248+0000","last_clean":"2024-04-05T22:31:42.321248+0000","last_became_active":"2024-04-05T22:31:42.319248+0000","last_became_peered":"2024-04-05T22:31:42.319248+0000","last_unstale":"2024-04-05T22:31:42.321248+0000","last_undegraded":"2024-04-05T22:31:42.321248+0000","last_fullsized":"2024-04-05T22:31:42.321248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:15:32.520839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348673+0000","last_change":"2024-04-05T22:31:42.345872+0000","last_active":"2024-04-05T22:31:42.348673+0000","last_peered":"2024-04-05T22:31:42.348673+0000","last_clean":"2024-04-05T22:31:42.348673+0000","last_became_active":"2024-04-05T22:31:42.345542+0000","last_became_peered":"2024-04-05T22:31:42.345542+0000","last_unstale":"2024-04-05T22:31:42.348673+0000","last_undegraded":"2024-04-05T22:31:42.348673+0000","last_fullsized":"2024-04-05T22:31:42.348673+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:30:19.604469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330881+0000","last_change":"2024-04-05T22:31:42.325832+0000","last_active":"2024-04-05T22:31:42.330881+0000","last_peered":"2024-04-05T22:31:42.330881+0000","last_clean":"2024-04-05T22:31:42.330881+0000","last_became_active":"2024-04-05T22:31:42.324711+0000","last_became_peered":"2024-04-05T22:31:42.324711+0000","last_unstale":"2024-04-05T22:31:42.330881+0000","last_undegraded":"2024-04-05T22:31:42.330881+0000","last_fullsized":"2024-04-05T22:31:42.330881+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:37:37.102529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339207+0000","last_change":"2024-04-05T22:31:42.335332+0000","last_active":"2024-04-05T22:31:42.339207+0000","last_peered":"2024-04-05T22:31:42.339207+0000","last_clean":"2024-04-05T22:31:42.339207+0000","last_became_active":"2024-04-05T22:31:42.334155+0000","last_became_peered":"2024-04-05T22:31:42.334155+0000","last_unstale":"2024-04-05T22:31:42.339207+0000","last_undegraded":"2024-04-05T22:31:42.339207+0000","last_fullsized":"2024-04-05T22:31:42.339207+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:10.992539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342786+0000","last_change":"2024-04-05T22:31:42.339091+0000","last_active":"2024-04-05T22:31:42.342786+0000","last_peered":"2024-04-05T22:31:42.342786+0000","last_clean":"2024-04-05T22:31:42.342786+0000","last_became_active":"2024-04-05T22:31:42.338706+0000","last_became_peered":"2024-04-05T22:31:42.338706+0000","last_unstale":"2024-04-05T22:31:42.342786+0000","last_undegraded":"2024-04-05T22:31:42.342786+0000","last_fullsized":"2024-04-05T22:31:42.342786+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:54:18.242102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342619+0000","last_change":"2024-04-05T22:31:42.338052+0000","last_active":"2024-04-05T22:31:42.342619+0000","last_peered":"2024-04-05T22:31:42.342619+0000","last_clean":"2024-04-05T22:31:42.342619+0000","last_became_active":"2024-04-05T22:31:42.337748+0000","last_became_peered":"2024-04-05T22:31:42.337748+0000","last_unstale":"2024-04-05T22:31:42.342619+0000","last_undegraded":"2024-04-05T22:31:42.342619+0000","last_fullsized":"2024-04-05T22:31:42.342619+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:53:57.734750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336679+0000","last_change":"2024-04-05T22:31:42.332970+0000","last_active":"2024-04-05T22:31:42.336679+0000","last_peered":"2024-04-05T22:31:42.336679+0000","last_clean":"2024-04-05T22:31:42.336679+0000","last_became_active":"2024-04-05T22:31:42.332687+0000","last_became_peered":"2024-04-05T22:31:42.332687+0000","last_unstale":"2024-04-05T22:31:42.336679+0000","last_undegraded":"2024-04-05T22:31:42.336679+0000","last_fullsized":"2024-04-05T22:31:42.336679+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T05:58:29.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321303+0000","last_change":"2024-04-05T22:31:42.318249+0000","last_active":"2024-04-05T22:31:42.321303+0000","last_peered":"2024-04-05T22:31:42.321303+0000","last_clean":"2024-04-05T22:31:42.321303+0000","last_became_active":"2024-04-05T22:31:42.317925+0000","last_became_peered":"2024-04-05T22:31:42.317925+0000","last_unstale":"2024-04-05T22:31:42.321303+0000","last_undegraded":"2024-04-05T22:31:42.321303+0000","last_fullsized":"2024-04-05T22:31:42.321303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:38:02.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336526+0000","last_change":"2024-04-05T22:31:42.333445+0000","last_active":"2024-04-05T22:31:42.336526+0000","last_peered":"2024-04-05T22:31:42.336526+0000","last_clean":"2024-04-05T22:31:42.336526+0000","last_became_active":"2024-04-05T22:31:42.333110+0000","last_became_peered":"2024-04-05T22:31:42.333110+0000","last_unstale":"2024-04-05T22:31:42.336526+0000","last_undegraded":"2024-04-05T22:31:42.336526+0000","last_fullsized":"2024-04-05T22:31:42.336526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:33.673205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347262+0000","last_change":"2024-04-05T22:31:42.345591+0000","last_active":"2024-04-05T22:31:42.347262+0000","last_peered":"2024-04-05T22:31:42.347262+0000","last_clean":"2024-04-05T22:31:42.347262+0000","last_became_active":"2024-04-05T22:31:42.345097+0000","last_became_peered":"2024-04-05T22:31:42.345097+0000","last_unstale":"2024-04-05T22:31:42.347262+0000","last_undegraded":"2024-04-05T22:31:42.347262+0000","last_fullsized":"2024-04-05T22:31:42.347262+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:57:50.061087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.330899+0000","last_change":"2024-04-05T22:31:42.327490+0000","last_active":"2024-04-05T22:31:42.330899+0000","last_peered":"2024-04-05T22:31:42.330899+0000","last_clean":"2024-04-05T22:31:42.330899+0000","last_became_active":"2024-04-05T22:31:42.326923+0000","last_became_peered":"2024-04-05T22:31:42.326923+0000","last_unstale":"2024-04-05T22:31:42.330899+0000","last_undegraded":"2024-04-05T22:31:42.330899+0000","last_fullsized":"2024-04-05T22:31:42.330899+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:47.359209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333648+0000","last_change":"2024-04-05T22:31:42.331608+0000","last_active":"2024-04-05T22:31:42.333648+0000","last_peered":"2024-04-05T22:31:42.333648+0000","last_clean":"2024-04-05T22:31:42.333648+0000","last_became_active":"2024-04-05T22:31:42.330407+0000","last_became_peered":"2024-04-05T22:31:42.330407+0000","last_unstale":"2024-04-05T22:31:42.333648+0000","last_undegraded":"2024-04-05T22:31:42.333648+0000","last_fullsized":"2024-04-05T22:31:42.333648+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:57:50.061087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329903+0000","last_change":"2024-04-05T22:31:42.324291+0000","last_active":"2024-04-05T22:31:42.329903+0000","last_peered":"2024-04-05T22:31:42.329903+0000","last_clean":"2024-04-05T22:31:42.329903+0000","last_became_active":"2024-04-05T22:31:42.324006+0000","last_became_peered":"2024-04-05T22:31:42.324006+0000","last_unstale":"2024-04-05T22:31:42.329903+0000","last_undegraded":"2024-04-05T22:31:42.329903+0000","last_fullsized":"2024-04-05T22:31:42.329903+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:54:19.620848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348416+0000","last_change":"2024-04-05T22:31:42.342104+0000","last_active":"2024-04-05T22:31:42.348416+0000","last_peered":"2024-04-05T22:31:42.348416+0000","last_clean":"2024-04-05T22:31:42.348416+0000","last_became_active":"2024-04-05T22:31:42.341805+0000","last_became_peered":"2024-04-05T22:31:42.341805+0000","last_unstale":"2024-04-05T22:31:42.348416+0000","last_undegraded":"2024-04-05T22:31:42.348416+0000","last_fullsized":"2024-04-05T22:31:42.348416+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:14.569310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.335993+0000","last_change":"2024-04-05T22:31:42.327342+0000","last_active":"2024-04-05T22:31:42.335993+0000","last_peered":"2024-04-05T22:31:42.335993+0000","last_clean":"2024-04-05T22:31:42.335993+0000","last_became_active":"2024-04-05T22:31:42.326993+0000","last_became_peered":"2024-04-05T22:31:42.326993+0000","last_unstale":"2024-04-05T22:31:42.335993+0000","last_undegraded":"2024-04-05T22:31:42.335993+0000","last_fullsized":"2024-04-05T22:31:42.335993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:14:05.287932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.335021+0000","last_change":"2024-04-05T22:31:42.321704+0000","last_active":"2024-04-05T22:31:42.335021+0000","last_peered":"2024-04-05T22:31:42.335021+0000","last_clean":"2024-04-05T22:31:42.335021+0000","last_became_active":"2024-04-05T22:31:42.321000+0000","last_became_peered":"2024-04-05T22:31:42.321000+0000","last_unstale":"2024-04-05T22:31:42.335021+0000","last_undegraded":"2024-04-05T22:31:42.335021+0000","last_fullsized":"2024-04-05T22:31:42.335021+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:42:54.958862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.350034+0000","last_change":"2024-04-05T22:31:42.348049+0000","last_active":"2024-04-05T22:31:42.350034+0000","last_peered":"2024-04-05T22:31:42.350034+0000","last_clean":"2024-04-05T22:31:42.350034+0000","last_became_active":"2024-04-05T22:31:42.347787+0000","last_became_peered":"2024-04-05T22:31:42.347787+0000","last_unstale":"2024-04-05T22:31:42.350034+0000","last_undegraded":"2024-04-05T22:31:42.350034+0000","last_fullsized":"2024-04-05T22:31:42.350034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:45.043543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347564+0000","last_change":"2024-04-05T22:31:42.343220+0000","last_active":"2024-04-05T22:31:42.347564+0000","last_peered":"2024-04-05T22:31:42.347564+0000","last_clean":"2024-04-05T22:31:42.347564+0000","last_became_active":"2024-04-05T22:31:42.342959+0000","last_became_peered":"2024-04-05T22:31:42.342959+0000","last_unstale":"2024-04-05T22:31:42.347564+0000","last_undegraded":"2024-04-05T22:31:42.347564+0000","last_fullsized":"2024-04-05T22:31:42.347564+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:01:56.517269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336712+0000","last_change":"2024-04-05T22:31:42.334183+0000","last_active":"2024-04-05T22:31:42.336712+0000","last_peered":"2024-04-05T22:31:42.336712+0000","last_clean":"2024-04-05T22:31:42.336712+0000","last_became_active":"2024-04-05T22:31:42.333885+0000","last_became_peered":"2024-04-05T22:31:42.333885+0000","last_unstale":"2024-04-05T22:31:42.336712+0000","last_undegraded":"2024-04-05T22:31:42.336712+0000","last_fullsized":"2024-04-05T22:31:42.336712+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:31.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342638+0000","last_change":"2024-04-05T22:31:42.337305+0000","last_active":"2024-04-05T22:31:42.342638+0000","last_peered":"2024-04-05T22:31:42.342638+0000","last_clean":"2024-04-05T22:31:42.342638+0000","last_became_active":"2024-04-05T22:31:42.335524+0000","last_became_peered":"2024-04-05T22:31:42.335524+0000","last_unstale":"2024-04-05T22:31:42.342638+0000","last_undegraded":"2024-04-05T22:31:42.342638+0000","last_fullsized":"2024-04-05T22:31:42.342638+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:16.187575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321372+0000","last_change":"2024-04-05T22:31:42.318684+0000","last_active":"2024-04-05T22:31:42.321372+0000","last_peered":"2024-04-05T22:31:42.321372+0000","last_clean":"2024-04-05T22:31:42.321372+0000","last_became_active":"2024-04-05T22:31:42.318434+0000","last_became_peered":"2024-04-05T22:31:42.318434+0000","last_unstale":"2024-04-05T22:31:42.321372+0000","last_undegraded":"2024-04-05T22:31:42.321372+0000","last_fullsized":"2024-04-05T22:31:42.321372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:42:54.958862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342853+0000","last_change":"2024-04-05T22:31:42.339310+0000","last_active":"2024-04-05T22:31:42.342853+0000","last_peered":"2024-04-05T22:31:42.342853+0000","last_clean":"2024-04-05T22:31:42.342853+0000","last_became_active":"2024-04-05T22:31:42.337225+0000","last_became_peered":"2024-04-05T22:31:42.337225+0000","last_unstale":"2024-04-05T22:31:42.342853+0000","last_undegraded":"2024-04-05T22:31:42.342853+0000","last_fullsized":"2024-04-05T22:31:42.342853+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:54:50.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344737+0000","last_change":"2024-04-05T22:31:42.335684+0000","last_active":"2024-04-05T22:31:42.344737+0000","last_peered":"2024-04-05T22:31:42.344737+0000","last_clean":"2024-04-05T22:31:42.344737+0000","last_became_active":"2024-04-05T22:31:42.335293+0000","last_became_peered":"2024-04-05T22:31:42.335293+0000","last_unstale":"2024-04-05T22:31:42.344737+0000","last_undegraded":"2024-04-05T22:31:42.344737+0000","last_fullsized":"2024-04-05T22:31:42.344737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:21:10.978043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347128+0000","last_change":"2024-04-05T22:31:42.345820+0000","last_active":"2024-04-05T22:31:42.347128+0000","last_peered":"2024-04-05T22:31:42.347128+0000","last_clean":"2024-04-05T22:31:42.347128+0000","last_became_active":"2024-04-05T22:31:42.344945+0000","last_became_peered":"2024-04-05T22:31:42.344945+0000","last_unstale":"2024-04-05T22:31:42.347128+0000","last_undegraded":"2024-04-05T22:31:42.347128+0000","last_fullsized":"2024-04-05T22:31:42.347128+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:36:07.327855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321121+0000","last_change":"2024-04-05T22:31:42.318185+0000","last_active":"2024-04-05T22:31:42.321121+0000","last_peered":"2024-04-05T22:31:42.321121+0000","last_clean":"2024-04-05T22:31:42.321121+0000","last_became_active":"2024-04-05T22:31:42.316494+0000","last_became_peered":"2024-04-05T22:31:42.316494+0000","last_unstale":"2024-04-05T22:31:42.321121+0000","last_undegraded":"2024-04-05T22:31:42.321121+0000","last_fullsized":"2024-04-05T22:31:42.321121+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:12:47.359209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345191+0000","last_change":"2024-04-05T22:31:42.338095+0000","last_active":"2024-04-05T22:31:42.345191+0000","last_peered":"2024-04-05T22:31:42.345191+0000","last_clean":"2024-04-05T22:31:42.345191+0000","last_became_active":"2024-04-05T22:31:42.337787+0000","last_became_peered":"2024-04-05T22:31:42.337787+0000","last_unstale":"2024-04-05T22:31:42.345191+0000","last_undegraded":"2024-04-05T22:31:42.345191+0000","last_fullsized":"2024-04-05T22:31:42.345191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:43:13.566684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"171'4","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346497+0000","last_change":"2024-04-05T22:31:41.857820+0000","last_active":"2024-04-05T22:31:42.346497+0000","last_peered":"2024-04-05T22:31:42.346497+0000","last_clean":"2024-04-05T22:31:42.346497+0000","last_became_active":"2024-04-05T22:31:41.856777+0000","last_became_peered":"2024-04-05T22:31:41.856777+0000","last_unstale":"2024-04-05T22:31:42.346497+0000","last_undegraded":"2024-04-05T22:31:42.346497+0000","last_fullsized":"2024-04-05T22:31:42.346497+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:17:45.822999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.340720+0000","last_change":"2024-04-05T22:31:42.336675+0000","last_active":"2024-04-05T22:31:42.340720+0000","last_peered":"2024-04-05T22:31:42.340720+0000","last_clean":"2024-04-05T22:31:42.340720+0000","last_became_active":"2024-04-05T22:31:42.336326+0000","last_became_peered":"2024-04-05T22:31:42.336326+0000","last_unstale":"2024-04-05T22:31:42.340720+0000","last_undegraded":"2024-04-05T22:31:42.340720+0000","last_fullsized":"2024-04-05T22:31:42.340720+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:26:54.232462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349410+0000","last_change":"2024-04-05T22:31:41.853865+0000","last_active":"2024-04-05T22:31:42.349410+0000","last_peered":"2024-04-05T22:31:42.349410+0000","last_clean":"2024-04-05T22:31:42.349410+0000","last_became_active":"2024-04-05T22:31:41.853598+0000","last_became_peered":"2024-04-05T22:31:41.853598+0000","last_unstale":"2024-04-05T22:31:42.349410+0000","last_undegraded":"2024-04-05T22:31:42.349410+0000","last_fullsized":"2024-04-05T22:31:42.349410+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:47:02.313670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.323209+0000","last_change":"2024-04-05T22:31:42.320351+0000","last_active":"2024-04-05T22:31:42.323209+0000","last_peered":"2024-04-05T22:31:42.323209+0000","last_clean":"2024-04-05T22:31:42.323209+0000","last_became_active":"2024-04-05T22:31:42.320082+0000","last_became_peered":"2024-04-05T22:31:42.320082+0000","last_unstale":"2024-04-05T22:31:42.323209+0000","last_undegraded":"2024-04-05T22:31:42.323209+0000","last_fullsized":"2024-04-05T22:31:42.323209+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:14:38.808826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344379+0000","last_change":"2024-04-05T22:31:41.851643+0000","last_active":"2024-04-05T22:31:42.344379+0000","last_peered":"2024-04-05T22:31:42.344379+0000","last_clean":"2024-04-05T22:31:42.344379+0000","last_became_active":"2024-04-05T22:31:41.851399+0000","last_became_peered":"2024-04-05T22:31:41.851399+0000","last_unstale":"2024-04-05T22:31:42.344379+0000","last_undegraded":"2024-04-05T22:31:42.344379+0000","last_fullsized":"2024-04-05T22:31:42.344379+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:20.548841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333666+0000","last_change":"2024-04-05T22:31:42.331888+0000","last_active":"2024-04-05T22:31:42.333666+0000","last_peered":"2024-04-05T22:31:42.333666+0000","last_clean":"2024-04-05T22:31:42.333666+0000","last_became_active":"2024-04-05T22:31:42.331601+0000","last_became_peered":"2024-04-05T22:31:42.331601+0000","last_unstale":"2024-04-05T22:31:42.333666+0000","last_undegraded":"2024-04-05T22:31:42.333666+0000","last_fullsized":"2024-04-05T22:31:42.333666+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:32.458969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349305+0000","last_change":"2024-04-05T22:31:41.854869+0000","last_active":"2024-04-05T22:31:42.349305+0000","last_peered":"2024-04-05T22:31:42.349305+0000","last_clean":"2024-04-05T22:31:42.349305+0000","last_became_active":"2024-04-05T22:31:41.854494+0000","last_became_peered":"2024-04-05T22:31:41.854494+0000","last_unstale":"2024-04-05T22:31:42.349305+0000","last_undegraded":"2024-04-05T22:31:42.349305+0000","last_fullsized":"2024-04-05T22:31:42.349305+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:43:12.551360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348435+0000","last_change":"2024-04-05T22:31:42.342237+0000","last_active":"2024-04-05T22:31:42.348435+0000","last_peered":"2024-04-05T22:31:42.348435+0000","last_clean":"2024-04-05T22:31:42.348435+0000","last_became_active":"2024-04-05T22:31:42.341862+0000","last_became_peered":"2024-04-05T22:31:42.341862+0000","last_unstale":"2024-04-05T22:31:42.348435+0000","last_undegraded":"2024-04-05T22:31:42.348435+0000","last_fullsized":"2024-04-05T22:31:42.348435+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:09.348577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336433+0000","last_change":"2024-04-05T22:31:41.831405+0000","last_active":"2024-04-05T22:31:42.336433+0000","last_peered":"2024-04-05T22:31:42.336433+0000","last_clean":"2024-04-05T22:31:42.336433+0000","last_became_active":"2024-04-05T22:31:41.831007+0000","last_became_peered":"2024-04-05T22:31:41.831007+0000","last_unstale":"2024-04-05T22:31:42.336433+0000","last_undegraded":"2024-04-05T22:31:42.336433+0000","last_fullsized":"2024-04-05T22:31:42.336433+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:59:44.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.341837+0000","last_change":"2024-04-05T22:31:42.339844+0000","last_active":"2024-04-05T22:31:42.341837+0000","last_peered":"2024-04-05T22:31:42.341837+0000","last_clean":"2024-04-05T22:31:42.341837+0000","last_became_active":"2024-04-05T22:31:42.339600+0000","last_became_peered":"2024-04-05T22:31:42.339600+0000","last_unstale":"2024-04-05T22:31:42.341837+0000","last_undegraded":"2024-04-05T22:31:42.341837+0000","last_fullsized":"2024-04-05T22:31:42.341837+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:28.415076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314546+0000","last_change":"2024-04-05T22:31:41.847754+0000","last_active":"2024-04-05T22:31:42.314546+0000","last_peered":"2024-04-05T22:31:42.314546+0000","last_clean":"2024-04-05T22:31:42.314546+0000","last_became_active":"2024-04-05T22:31:41.844413+0000","last_became_peered":"2024-04-05T22:31:41.844413+0000","last_unstale":"2024-04-05T22:31:42.314546+0000","last_undegraded":"2024-04-05T22:31:42.314546+0000","last_fullsized":"2024-04-05T22:31:42.314546+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:56.890791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349390+0000","last_change":"2024-04-05T22:31:42.344400+0000","last_active":"2024-04-05T22:31:42.349390+0000","last_peered":"2024-04-05T22:31:42.349390+0000","last_clean":"2024-04-05T22:31:42.349390+0000","last_became_active":"2024-04-05T22:31:42.344066+0000","last_became_peered":"2024-04-05T22:31:42.344066+0000","last_unstale":"2024-04-05T22:31:42.349390+0000","last_undegraded":"2024-04-05T22:31:42.349390+0000","last_fullsized":"2024-04-05T22:31:42.349390+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:42.495611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.311718+0000","last_change":"2024-04-05T22:31:41.841678+0000","last_active":"2024-04-05T22:31:42.311718+0000","last_peered":"2024-04-05T22:31:42.311718+0000","last_clean":"2024-04-05T22:31:42.311718+0000","last_became_active":"2024-04-05T22:31:41.841303+0000","last_became_peered":"2024-04-05T22:31:41.841303+0000","last_unstale":"2024-04-05T22:31:42.311718+0000","last_undegraded":"2024-04-05T22:31:42.311718+0000","last_fullsized":"2024-04-05T22:31:42.311718+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:48.467450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.350083+0000","last_change":"2024-04-05T22:31:42.348222+0000","last_active":"2024-04-05T22:31:42.350083+0000","last_peered":"2024-04-05T22:31:42.350083+0000","last_clean":"2024-04-05T22:31:42.350083+0000","last_became_active":"2024-04-05T22:31:42.347935+0000","last_became_peered":"2024-04-05T22:31:42.347935+0000","last_unstale":"2024-04-05T22:31:42.350083+0000","last_undegraded":"2024-04-05T22:31:42.350083+0000","last_fullsized":"2024-04-05T22:31:42.350083+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:31:40.592103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314776+0000","last_change":"2024-04-05T22:31:41.856671+0000","last_active":"2024-04-05T22:31:42.314776+0000","last_peered":"2024-04-05T22:31:42.314776+0000","last_clean":"2024-04-05T22:31:42.314776+0000","last_became_active":"2024-04-05T22:31:41.856314+0000","last_became_peered":"2024-04-05T22:31:41.856314+0000","last_unstale":"2024-04-05T22:31:42.314776+0000","last_undegraded":"2024-04-05T22:31:42.314776+0000","last_fullsized":"2024-04-05T22:31:42.314776+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:41.480287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339214+0000","last_change":"2024-04-05T22:31:42.334931+0000","last_active":"2024-04-05T22:31:42.339214+0000","last_peered":"2024-04-05T22:31:42.339214+0000","last_clean":"2024-04-05T22:31:42.339214+0000","last_became_active":"2024-04-05T22:31:42.334518+0000","last_became_peered":"2024-04-05T22:31:42.334518+0000","last_unstale":"2024-04-05T22:31:42.339214+0000","last_undegraded":"2024-04-05T22:31:42.339214+0000","last_fullsized":"2024-04-05T22:31:42.339214+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:19.428791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.317451+0000","last_change":"2024-04-05T22:31:41.852167+0000","last_active":"2024-04-05T22:31:42.317451+0000","last_peered":"2024-04-05T22:31:42.317451+0000","last_clean":"2024-04-05T22:31:42.317451+0000","last_became_active":"2024-04-05T22:31:41.851854+0000","last_became_peered":"2024-04-05T22:31:41.851854+0000","last_unstale":"2024-04-05T22:31:42.317451+0000","last_undegraded":"2024-04-05T22:31:42.317451+0000","last_fullsized":"2024-04-05T22:31:42.317451+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:35:04.539933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.349512+0000","last_change":"2024-04-05T22:31:42.346019+0000","last_active":"2024-04-05T22:31:42.349512+0000","last_peered":"2024-04-05T22:31:42.349512+0000","last_clean":"2024-04-05T22:31:42.349512+0000","last_became_active":"2024-04-05T22:31:42.345629+0000","last_became_peered":"2024-04-05T22:31:42.345629+0000","last_unstale":"2024-04-05T22:31:42.349512+0000","last_undegraded":"2024-04-05T22:31:42.349512+0000","last_fullsized":"2024-04-05T22:31:42.349512+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:10.992539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"171'3","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.331881+0000","last_change":"2024-04-05T22:31:41.847753+0000","last_active":"2024-04-05T22:31:42.331881+0000","last_peered":"2024-04-05T22:31:42.331881+0000","last_clean":"2024-04-05T22:31:42.331881+0000","last_became_active":"2024-04-05T22:31:41.847477+0000","last_became_peered":"2024-04-05T22:31:41.847477+0000","last_unstale":"2024-04-05T22:31:42.331881+0000","last_undegraded":"2024-04-05T22:31:42.331881+0000","last_fullsized":"2024-04-05T22:31:42.331881+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:23:22.112289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.328052+0000","last_change":"2024-04-05T22:31:42.324785+0000","last_active":"2024-04-05T22:31:42.328052+0000","last_peered":"2024-04-05T22:31:42.328052+0000","last_clean":"2024-04-05T22:31:42.328052+0000","last_became_active":"2024-04-05T22:31:42.324569+0000","last_became_peered":"2024-04-05T22:31:42.324569+0000","last_unstale":"2024-04-05T22:31:42.328052+0000","last_undegraded":"2024-04-05T22:31:42.328052+0000","last_fullsized":"2024-04-05T22:31:42.328052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:37:37.102529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"171'6","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.351645+0000","last_change":"2024-04-05T22:31:41.849643+0000","last_active":"2024-04-05T22:31:42.351645+0000","last_peered":"2024-04-05T22:31:42.351645+0000","last_clean":"2024-04-05T22:31:42.351645+0000","last_became_active":"2024-04-05T22:31:41.849341+0000","last_became_peered":"2024-04-05T22:31:41.849341+0000","last_unstale":"2024-04-05T22:31:42.351645+0000","last_undegraded":"2024-04-05T22:31:42.351645+0000","last_fullsized":"2024-04-05T22:31:42.351645+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:49:50.800680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.331159+0000","last_change":"2024-04-05T22:31:42.325837+0000","last_active":"2024-04-05T22:31:42.331159+0000","last_peered":"2024-04-05T22:31:42.331159+0000","last_clean":"2024-04-05T22:31:42.331159+0000","last_became_active":"2024-04-05T22:31:42.325601+0000","last_became_peered":"2024-04-05T22:31:42.325601+0000","last_unstale":"2024-04-05T22:31:42.331159+0000","last_undegraded":"2024-04-05T22:31:42.331159+0000","last_fullsized":"2024-04-05T22:31:42.331159+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:03:11.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336673+0000","last_change":"2024-04-05T22:31:41.837689+0000","last_active":"2024-04-05T22:31:42.336673+0000","last_peered":"2024-04-05T22:31:42.336673+0000","last_clean":"2024-04-05T22:31:42.336673+0000","last_became_active":"2024-04-05T22:31:41.836071+0000","last_became_peered":"2024-04-05T22:31:41.836071+0000","last_unstale":"2024-04-05T22:31:42.336673+0000","last_undegraded":"2024-04-05T22:31:42.336673+0000","last_fullsized":"2024-04-05T22:31:42.336673+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:04:18.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339297+0000","last_change":"2024-04-05T22:31:42.338349+0000","last_active":"2024-04-05T22:31:42.339297+0000","last_peered":"2024-04-05T22:31:42.339297+0000","last_clean":"2024-04-05T22:31:42.339297+0000","last_became_active":"2024-04-05T22:31:42.338075+0000","last_became_peered":"2024-04-05T22:31:42.338075+0000","last_unstale":"2024-04-05T22:31:42.339297+0000","last_undegraded":"2024-04-05T22:31:42.339297+0000","last_fullsized":"2024-04-05T22:31:42.339297+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T23:48:00.436586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.311617+0000","last_change":"2024-04-05T22:31:41.848610+0000","last_active":"2024-04-05T22:31:42.311617+0000","last_peered":"2024-04-05T22:31:42.311617+0000","last_clean":"2024-04-05T22:31:42.311617+0000","last_became_active":"2024-04-05T22:31:41.848296+0000","last_became_peered":"2024-04-05T22:31:41.848296+0000","last_unstale":"2024-04-05T22:31:42.311617+0000","last_undegraded":"2024-04-05T22:31:42.311617+0000","last_fullsized":"2024-04-05T22:31:42.311617+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:15.479974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339127+0000","last_change":"2024-04-05T22:31:42.334497+0000","last_active":"2024-04-05T22:31:42.339127+0000","last_peered":"2024-04-05T22:31:42.339127+0000","last_clean":"2024-04-05T22:31:42.339127+0000","last_became_active":"2024-04-05T22:31:42.334246+0000","last_became_peered":"2024-04-05T22:31:42.334246+0000","last_unstale":"2024-04-05T22:31:42.339127+0000","last_undegraded":"2024-04-05T22:31:42.339127+0000","last_fullsized":"2024-04-05T22:31:42.339127+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:24:42.495611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321389+0000","last_change":"2024-04-05T22:31:41.849675+0000","last_active":"2024-04-05T22:31:42.321389+0000","last_peered":"2024-04-05T22:31:42.321389+0000","last_clean":"2024-04-05T22:31:42.321389+0000","last_became_active":"2024-04-05T22:31:41.849367+0000","last_became_peered":"2024-04-05T22:31:41.849367+0000","last_unstale":"2024-04-05T22:31:42.321389+0000","last_undegraded":"2024-04-05T22:31:42.321389+0000","last_fullsized":"2024-04-05T22:31:42.321389+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:53:27.399752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347823+0000","last_change":"2024-04-05T22:31:42.347022+0000","last_active":"2024-04-05T22:31:42.347823+0000","last_peered":"2024-04-05T22:31:42.347823+0000","last_clean":"2024-04-05T22:31:42.347823+0000","last_became_active":"2024-04-05T22:31:42.346728+0000","last_became_peered":"2024-04-05T22:31:42.346728+0000","last_unstale":"2024-04-05T22:31:42.347823+0000","last_undegraded":"2024-04-05T22:31:42.347823+0000","last_fullsized":"2024-04-05T22:31:42.347823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:42:33.673205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"171'8","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.348267+0000","last_change":"2024-04-05T22:31:41.848381+0000","last_active":"2024-04-05T22:31:42.348267+0000","last_peered":"2024-04-05T22:31:42.348267+0000","last_clean":"2024-04-05T22:31:42.348267+0000","last_became_active":"2024-04-05T22:31:41.848109+0000","last_became_peered":"2024-04-05T22:31:41.848109+0000","last_unstale":"2024-04-05T22:31:42.348267+0000","last_undegraded":"2024-04-05T22:31:42.348267+0000","last_fullsized":"2024-04-05T22:31:42.348267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:35:04.539933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.351546+0000","last_change":"2024-04-05T22:31:42.350283+0000","last_active":"2024-04-05T22:31:42.351546+0000","last_peered":"2024-04-05T22:31:42.351546+0000","last_clean":"2024-04-05T22:31:42.351546+0000","last_became_active":"2024-04-05T22:31:42.350001+0000","last_became_peered":"2024-04-05T22:31:42.350001+0000","last_unstale":"2024-04-05T22:31:42.351546+0000","last_undegraded":"2024-04-05T22:31:42.351546+0000","last_fullsized":"2024-04-05T22:31:42.351546+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:21.564165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.327734+0000","last_change":"2024-04-05T22:31:41.838712+0000","last_active":"2024-04-05T22:31:42.327734+0000","last_peered":"2024-04-05T22:31:42.327734+0000","last_clean":"2024-04-05T22:31:42.327734+0000","last_became_active":"2024-04-05T22:31:41.838122+0000","last_became_peered":"2024-04-05T22:31:41.838122+0000","last_unstale":"2024-04-05T22:31:42.327734+0000","last_undegraded":"2024-04-05T22:31:42.327734+0000","last_fullsized":"2024-04-05T22:31:42.327734+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:13.553986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345739+0000","last_change":"2024-04-05T22:31:42.342604+0000","last_active":"2024-04-05T22:31:42.345739+0000","last_peered":"2024-04-05T22:31:42.345739+0000","last_clean":"2024-04-05T22:31:42.345739+0000","last_became_active":"2024-04-05T22:31:42.342118+0000","last_became_peered":"2024-04-05T22:31:42.342118+0000","last_unstale":"2024-04-05T22:31:42.345739+0000","last_undegraded":"2024-04-05T22:31:42.345739+0000","last_fullsized":"2024-04-05T22:31:42.345739+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:21:00.091010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.341933+0000","last_change":"2024-04-05T22:31:41.855537+0000","last_active":"2024-04-05T22:31:42.341933+0000","last_peered":"2024-04-05T22:31:42.341933+0000","last_clean":"2024-04-05T22:31:42.341933+0000","last_became_active":"2024-04-05T22:31:41.854914+0000","last_became_peered":"2024-04-05T22:31:41.854914+0000","last_unstale":"2024-04-05T22:31:42.341933+0000","last_undegraded":"2024-04-05T22:31:42.341933+0000","last_fullsized":"2024-04-05T22:31:42.341933+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:13:28.327743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346244+0000","last_change":"2024-04-05T22:31:42.345098+0000","last_active":"2024-04-05T22:31:42.346244+0000","last_peered":"2024-04-05T22:31:42.346244+0000","last_clean":"2024-04-05T22:31:42.346244+0000","last_became_active":"2024-04-05T22:31:42.344285+0000","last_became_peered":"2024-04-05T22:31:42.344285+0000","last_unstale":"2024-04-05T22:31:42.346244+0000","last_undegraded":"2024-04-05T22:31:42.346244+0000","last_fullsized":"2024-04-05T22:31:42.346244+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:17:14.569310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.340649+0000","last_change":"2024-04-05T22:31:41.831117+0000","last_active":"2024-04-05T22:31:42.340649+0000","last_peered":"2024-04-05T22:31:42.340649+0000","last_clean":"2024-04-05T22:31:42.340649+0000","last_became_active":"2024-04-05T22:31:41.829649+0000","last_became_peered":"2024-04-05T22:31:41.829649+0000","last_unstale":"2024-04-05T22:31:42.340649+0000","last_undegraded":"2024-04-05T22:31:42.340649+0000","last_fullsized":"2024-04-05T22:31:42.340649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:41.515181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.346336+0000","last_change":"2024-04-05T22:31:42.345531+0000","last_active":"2024-04-05T22:31:42.346336+0000","last_peered":"2024-04-05T22:31:42.346336+0000","last_clean":"2024-04-05T22:31:42.346336+0000","last_became_active":"2024-04-05T22:31:42.344852+0000","last_became_peered":"2024-04-05T22:31:42.344852+0000","last_unstale":"2024-04-05T22:31:42.346336+0000","last_undegraded":"2024-04-05T22:31:42.346336+0000","last_fullsized":"2024-04-05T22:31:42.346336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:58:50.399735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.305845+0000","last_change":"2024-04-05T22:31:41.846499+0000","last_active":"2024-04-05T22:31:42.305845+0000","last_peered":"2024-04-05T22:31:42.305845+0000","last_clean":"2024-04-05T22:31:42.305845+0000","last_became_active":"2024-04-05T22:31:41.846216+0000","last_became_peered":"2024-04-05T22:31:41.846216+0000","last_unstale":"2024-04-05T22:31:42.305845+0000","last_undegraded":"2024-04-05T22:31:42.305845+0000","last_fullsized":"2024-04-05T22:31:42.305845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:15.479974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.325547+0000","last_change":"2024-04-05T22:31:42.323945+0000","last_active":"2024-04-05T22:31:42.325547+0000","last_peered":"2024-04-05T22:31:42.325547+0000","last_clean":"2024-04-05T22:31:42.325547+0000","last_became_active":"2024-04-05T22:31:42.323704+0000","last_became_peered":"2024-04-05T22:31:42.323704+0000","last_unstale":"2024-04-05T22:31:42.325547+0000","last_undegraded":"2024-04-05T22:31:42.325547+0000","last_fullsized":"2024-04-05T22:31:42.325547+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:44:44.121984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329575+0000","last_change":"2024-04-05T22:31:41.843680+0000","last_active":"2024-04-05T22:31:42.329575+0000","last_peered":"2024-04-05T22:31:42.329575+0000","last_clean":"2024-04-05T22:31:42.329575+0000","last_became_active":"2024-04-05T22:31:41.843319+0000","last_became_peered":"2024-04-05T22:31:41.843319+0000","last_unstale":"2024-04-05T22:31:42.329575+0000","last_undegraded":"2024-04-05T22:31:42.329575+0000","last_fullsized":"2024-04-05T22:31:42.329575+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:57:42.354442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321111+0000","last_change":"2024-04-05T22:31:42.317706+0000","last_active":"2024-04-05T22:31:42.321111+0000","last_peered":"2024-04-05T22:31:42.321111+0000","last_clean":"2024-04-05T22:31:42.321111+0000","last_became_active":"2024-04-05T22:31:42.317418+0000","last_became_peered":"2024-04-05T22:31:42.317418+0000","last_unstale":"2024-04-05T22:31:42.321111+0000","last_undegraded":"2024-04-05T22:31:42.321111+0000","last_fullsized":"2024-04-05T22:31:42.321111+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:57.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.310888+0000","last_change":"2024-04-05T22:31:41.852712+0000","last_active":"2024-04-05T22:31:42.310888+0000","last_peered":"2024-04-05T22:31:42.310888+0000","last_clean":"2024-04-05T22:31:42.310888+0000","last_became_active":"2024-04-05T22:31:41.852383+0000","last_became_peered":"2024-04-05T22:31:41.852383+0000","last_unstale":"2024-04-05T22:31:42.310888+0000","last_undegraded":"2024-04-05T22:31:42.310888+0000","last_fullsized":"2024-04-05T22:31:42.310888+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:33:41.515181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345102+0000","last_change":"2024-04-05T22:31:42.337678+0000","last_active":"2024-04-05T22:31:42.345102+0000","last_peered":"2024-04-05T22:31:42.345102+0000","last_clean":"2024-04-05T22:31:42.345102+0000","last_became_active":"2024-04-05T22:31:42.336826+0000","last_became_peered":"2024-04-05T22:31:42.336826+0000","last_unstale":"2024-04-05T22:31:42.345102+0000","last_undegraded":"2024-04-05T22:31:42.345102+0000","last_fullsized":"2024-04-05T22:31:42.345102+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T00:28:16.495298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.309717+0000","last_change":"2024-04-05T22:31:41.851514+0000","last_active":"2024-04-05T22:31:42.309717+0000","last_peered":"2024-04-05T22:31:42.309717+0000","last_clean":"2024-04-05T22:31:42.309717+0000","last_became_active":"2024-04-05T22:31:41.850997+0000","last_became_peered":"2024-04-05T22:31:41.850997+0000","last_unstale":"2024-04-05T22:31:42.309717+0000","last_undegraded":"2024-04-05T22:31:42.309717+0000","last_fullsized":"2024-04-05T22:31:42.309717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:04:59.160117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342471+0000","last_change":"2024-04-05T22:31:42.339215+0000","last_active":"2024-04-05T22:31:42.342471+0000","last_peered":"2024-04-05T22:31:42.342471+0000","last_clean":"2024-04-05T22:31:42.342471+0000","last_became_active":"2024-04-05T22:31:42.338033+0000","last_became_peered":"2024-04-05T22:31:42.338033+0000","last_unstale":"2024-04-05T22:31:42.342471+0000","last_undegraded":"2024-04-05T22:31:42.342471+0000","last_fullsized":"2024-04-05T22:31:42.342471+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:59:24.393944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314447+0000","last_change":"2024-04-05T22:31:41.842721+0000","last_active":"2024-04-05T22:31:42.314447+0000","last_peered":"2024-04-05T22:31:42.314447+0000","last_clean":"2024-04-05T22:31:42.314447+0000","last_became_active":"2024-04-05T22:31:41.839296+0000","last_became_peered":"2024-04-05T22:31:41.839296+0000","last_unstale":"2024-04-05T22:31:42.314447+0000","last_undegraded":"2024-04-05T22:31:42.314447+0000","last_fullsized":"2024-04-05T22:31:42.314447+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:32:18.413467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342667+0000","last_change":"2024-04-05T22:31:42.333854+0000","last_active":"2024-04-05T22:31:42.342667+0000","last_peered":"2024-04-05T22:31:42.342667+0000","last_clean":"2024-04-05T22:31:42.342667+0000","last_became_active":"2024-04-05T22:31:42.333289+0000","last_became_peered":"2024-04-05T22:31:42.333289+0000","last_unstale":"2024-04-05T22:31:42.342667+0000","last_undegraded":"2024-04-05T22:31:42.342667+0000","last_fullsized":"2024-04-05T22:31:42.342667+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T03:30:07.572633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"171'2","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.344059+0000","last_change":"2024-04-05T22:31:41.856664+0000","last_active":"2024-04-05T22:31:42.344059+0000","last_peered":"2024-04-05T22:31:42.344059+0000","last_clean":"2024-04-05T22:31:42.344059+0000","last_became_active":"2024-04-05T22:31:41.856324+0000","last_became_peered":"2024-04-05T22:31:41.856324+0000","last_unstale":"2024-04-05T22:31:42.344059+0000","last_undegraded":"2024-04-05T22:31:42.344059+0000","last_fullsized":"2024-04-05T22:31:42.344059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:48.467450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336836+0000","last_change":"2024-04-05T22:31:42.335543+0000","last_active":"2024-04-05T22:31:42.336836+0000","last_peered":"2024-04-05T22:31:42.336836+0000","last_clean":"2024-04-05T22:31:42.336836+0000","last_became_active":"2024-04-05T22:31:42.334702+0000","last_became_peered":"2024-04-05T22:31:42.334702+0000","last_unstale":"2024-04-05T22:31:42.336836+0000","last_undegraded":"2024-04-05T22:31:42.336836+0000","last_fullsized":"2024-04-05T22:31:42.336836+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:12:00.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333093+0000","last_change":"2024-04-05T22:31:41.851650+0000","last_active":"2024-04-05T22:31:42.333093+0000","last_peered":"2024-04-05T22:31:42.333093+0000","last_clean":"2024-04-05T22:31:42.333093+0000","last_became_active":"2024-04-05T22:31:41.851087+0000","last_became_peered":"2024-04-05T22:31:41.851087+0000","last_unstale":"2024-04-05T22:31:42.333093+0000","last_undegraded":"2024-04-05T22:31:42.333093+0000","last_fullsized":"2024-04-05T22:31:42.333093+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:58.100094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.345381+0000","last_change":"2024-04-05T22:31:42.341975+0000","last_active":"2024-04-05T22:31:42.345381+0000","last_peered":"2024-04-05T22:31:42.345381+0000","last_clean":"2024-04-05T22:31:42.345381+0000","last_became_active":"2024-04-05T22:31:42.341702+0000","last_became_peered":"2024-04-05T22:31:42.341702+0000","last_unstale":"2024-04-05T22:31:42.345381+0000","last_undegraded":"2024-04-05T22:31:42.345381+0000","last_fullsized":"2024-04-05T22:31:42.345381+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:26:47.664135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314671+0000","last_change":"2024-04-05T22:31:41.846878+0000","last_active":"2024-04-05T22:31:42.314671+0000","last_peered":"2024-04-05T22:31:42.314671+0000","last_clean":"2024-04-05T22:31:42.314671+0000","last_became_active":"2024-04-05T22:31:41.846583+0000","last_became_peered":"2024-04-05T22:31:41.846583+0000","last_unstale":"2024-04-05T22:31:42.314671+0000","last_undegraded":"2024-04-05T22:31:42.314671+0000","last_fullsized":"2024-04-05T22:31:42.314671+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:08.333253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.333232+0000","last_change":"2024-04-05T22:31:42.324334+0000","last_active":"2024-04-05T22:31:42.333232+0000","last_peered":"2024-04-05T22:31:42.333232+0000","last_clean":"2024-04-05T22:31:42.333232+0000","last_became_active":"2024-04-05T22:31:42.319871+0000","last_became_peered":"2024-04-05T22:31:42.319871+0000","last_unstale":"2024-04-05T22:31:42.333232+0000","last_undegraded":"2024-04-05T22:31:42.333232+0000","last_fullsized":"2024-04-05T22:31:42.333232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:16:57.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.305931+0000","last_change":"2024-04-05T22:31:41.852151+0000","last_active":"2024-04-05T22:31:42.305931+0000","last_peered":"2024-04-05T22:31:42.305931+0000","last_clean":"2024-04-05T22:31:42.305931+0000","last_became_active":"2024-04-05T22:31:41.851846+0000","last_became_peered":"2024-04-05T22:31:41.851846+0000","last_unstale":"2024-04-05T22:31:42.305931+0000","last_undegraded":"2024-04-05T22:31:42.305931+0000","last_fullsized":"2024-04-05T22:31:42.305931+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:21:58.100094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336699+0000","last_change":"2024-04-05T22:31:42.334826+0000","last_active":"2024-04-05T22:31:42.336699+0000","last_peered":"2024-04-05T22:31:42.336699+0000","last_clean":"2024-04-05T22:31:42.336699+0000","last_became_active":"2024-04-05T22:31:42.334599+0000","last_became_peered":"2024-04-05T22:31:42.334599+0000","last_unstale":"2024-04-05T22:31:42.336699+0000","last_undegraded":"2024-04-05T22:31:42.336699+0000","last_fullsized":"2024-04-05T22:31:42.336699+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:49:22.717040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.329041+0000","last_change":"2024-04-05T22:31:41.852462+0000","last_active":"2024-04-05T22:31:42.329041+0000","last_peered":"2024-04-05T22:31:42.329041+0000","last_clean":"2024-04-05T22:31:42.329041+0000","last_became_active":"2024-04-05T22:31:41.852160+0000","last_became_peered":"2024-04-05T22:31:41.852160+0000","last_unstale":"2024-04-05T22:31:42.329041+0000","last_undegraded":"2024-04-05T22:31:42.329041+0000","last_fullsized":"2024-04-05T22:31:42.329041+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T02:57:20.548841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.342569+0000","last_change":"2024-04-05T22:31:42.337518+0000","last_active":"2024-04-05T22:31:42.342569+0000","last_peered":"2024-04-05T22:31:42.342569+0000","last_clean":"2024-04-05T22:31:42.342569+0000","last_became_active":"2024-04-05T22:31:42.336674+0000","last_became_peered":"2024-04-05T22:31:42.336674+0000","last_unstale":"2024-04-05T22:31:42.342569+0000","last_undegraded":"2024-04-05T22:31:42.342569+0000","last_fullsized":"2024-04-05T22:31:42.342569+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T01:14:49.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.314491+0000","last_change":"2024-04-05T22:31:41.833839+0000","last_active":"2024-04-05T22:31:42.314491+0000","last_peered":"2024-04-05T22:31:42.314491+0000","last_clean":"2024-04-05T22:31:42.314491+0000","last_became_active":"2024-04-05T22:31:41.833408+0000","last_became_peered":"2024-04-05T22:31:41.833408+0000","last_unstale":"2024-04-05T22:31:42.314491+0000","last_undegraded":"2024-04-05T22:31:42.314491+0000","last_fullsized":"2024-04-05T22:31:42.314491+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:07:09.977215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.339109+0000","last_change":"2024-04-05T22:31:42.334142+0000","last_active":"2024-04-05T22:31:42.339109+0000","last_peered":"2024-04-05T22:31:42.339109+0000","last_clean":"2024-04-05T22:31:42.339109+0000","last_became_active":"2024-04-05T22:31:42.333912+0000","last_became_peered":"2024-04-05T22:31:42.333912+0000","last_unstale":"2024-04-05T22:31:42.339109+0000","last_undegraded":"2024-04-05T22:31:42.339109+0000","last_fullsized":"2024-04-05T22:31:42.339109+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T04:23:09.348577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.312304+0000","last_change":"2024-04-05T22:31:41.848102+0000","last_active":"2024-04-05T22:31:42.312304+0000","last_peered":"2024-04-05T22:31:42.312304+0000","last_clean":"2024-04-05T22:31:42.312304+0000","last_became_active":"2024-04-05T22:31:41.847799+0000","last_became_peered":"2024-04-05T22:31:41.847799+0000","last_unstale":"2024-04-05T22:31:42.312304+0000","last_undegraded":"2024-04-05T22:31:42.312304+0000","last_fullsized":"2024-04-05T22:31:42.312304+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T07:52:56.325069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.347644+0000","last_change":"2024-04-05T22:31:42.344352+0000","last_active":"2024-04-05T22:31:42.347644+0000","last_peered":"2024-04-05T22:31:42.347644+0000","last_clean":"2024-04-05T22:31:42.347644+0000","last_became_active":"2024-04-05T22:31:42.341062+0000","last_became_peered":"2024-04-05T22:31:42.341062+0000","last_unstale":"2024-04-05T22:31:42.347644+0000","last_undegraded":"2024-04-05T22:31:42.347644+0000","last_fullsized":"2024-04-05T22:31:42.347644+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-06T22:50:49.482774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.327234+0000","last_change":"2024-04-05T22:31:41.851856+0000","last_active":"2024-04-05T22:31:42.327234+0000","last_peered":"2024-04-05T22:31:42.327234+0000","last_clean":"2024-04-05T22:31:42.327234+0000","last_became_active":"2024-04-05T22:31:41.851428+0000","last_became_peered":"2024-04-05T22:31:41.851428+0000","last_unstale":"2024-04-05T22:31:42.327234+0000","last_undegraded":"2024-04-05T22:31:42.327234+0000","last_fullsized":"2024-04-05T22:31:42.327234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T08:46:25.186825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.321830+0000","last_change":"2024-04-05T22:31:42.318977+0000","last_active":"2024-04-05T22:31:42.321830+0000","last_peered":"2024-04-05T22:31:42.321830+0000","last_clean":"2024-04-05T22:31:42.321830+0000","last_became_active":"2024-04-05T22:31:42.318687+0000","last_became_peered":"2024-04-05T22:31:42.318687+0000","last_unstale":"2024-04-05T22:31:42.321830+0000","last_undegraded":"2024-04-05T22:31:42.321830+0000","last_fullsized":"2024-04-05T22:31:42.321830+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:41.806319+0000","last_clean_scrub_stamp":"2024-04-05T22:31:41.806319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:49:55.770780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"171'1","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-05T22:31:42.336935+0000","last_change":"2024-04-05T22:31:41.848177+0000","last_active":"2024-04-05T22:31:42.336935+0000","last_peered":"2024-04-05T22:31:42.336935+0000","last_clean":"2024-04-05T22:31:42.336935+0000","last_became_active":"2024-04-05T22:31:41.846549+0000","last_became_peered":"2024-04-05T22:31:41.846549+0000","last_unstale":"2024-04-05T22:31:42.336935+0000","last_undegraded":"2024-04-05T22:31:42.336935+0000","last_fullsized":"2024-04-05T22:31:42.336935+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:31:40.790995+0000","last_clean_scrub_stamp":"2024-04-05T22:31:40.790995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:25:34.690546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":331,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-05T22:31:41.841175+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:31:41.841175+0000","last_peered":"2024-04-05T22:31:41.841175+0000","last_clean":"2024-04-05T22:31:41.841175+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:31:41.841175+0000","last_undegraded":"2024-04-05T22:31:41.841175+0000","last_fullsized":"2024-04-05T22:31:41.841175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542274,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37884,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333956,"statfs":{"total":96636764160,"available":96597970944,"internally_reserved":0,"allocated":2736128,"data_stored":1533728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.133},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":0.98399999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.292}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.49},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.246},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.413}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.081},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37304,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334536,"statfs":{"total":96636764160,"available":96598564864,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.181},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.5880000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.262},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.98699999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.7150000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.669},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.754}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.454}]}]},{"osd":5,"up_from":14,"seq":60129542274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.84299999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.145},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.2230000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.079}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.157},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.361}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.4079999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542273,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":0.622}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.95099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.034},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.097}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":1.661}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":1.716},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.534}]}]},{"osd":2,"up_from":14,"seq":60129542273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2129920,"data_stored":937085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":0.874}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.526},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.107}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.333}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.25},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.3}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.151},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.458},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.393}]}]},{"osd":1,"up_from":14,"seq":60129542275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2142208,"data_stored":943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.083}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2190000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":1.117},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.2430000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.327}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.1779999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542275,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37904,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333936,"statfs":{"total":96636764160,"available":96597950464,"internally_reserved":0,"allocated":2752512,"data_stored":1540593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.88}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.621}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":1.034},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.73}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":1.24}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.593}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.5549999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:31:47.239 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-05T22:31:47.239 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-05T22:31:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.504+0000 7fdbd8600640 1 Processor -- start 2024-04-05T22:31:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.505+0000 7fdbd8600640 1 -- start start 2024-04-05T22:31:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.506+0000 7fdbd8600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 0x7fdbd00b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.506+0000 7fdbd8600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbd00b1750 0x7fdbd0166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.506+0000 7fdbd8600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd006adc0 con 0x7fdbd00b1bf0 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.506+0000 7fdbd8600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00b2460 con 0x7fdbd00b1750 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.506+0000 7fdbd8600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd015d4f0 con 0x7fdbd00b0720 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbd00b1750 0x7fdbd0166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbd00b1750 0x7fdbd0166c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43012/0 (socket says 172.21.15.73:43012) 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 -- 172.21.15.73:0/270914383 learned_addr learned my addr 172.21.15.73:0/270914383 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd5200640 1 --1- 172.21.15.73:0/270914383 >> v1:172.21.15.73:6789/0 conn(0x7fdbd00b1bf0 0x7fdbd00b1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49082/0 (socket says 172.21.15.73:49082) 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd5c00640 1 --2- 172.21.15.73:0/270914383 >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 0x7fdbd00b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 -- 172.21.15.73:0/270914383 >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 msgr2=0x7fdbd00b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 --2- 172.21.15.73:0/270914383 >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 0x7fdbd00b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 -- 172.21.15.73:0/270914383 >> v1:172.21.15.73:6789/0 conn(0x7fdbd00b1bf0 legacy=0x7fdbd00b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd6600640 1 -- 172.21.15.73:0/270914383 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00b51c0 con 0x7fdbd00b1750 2024-04-05T22:31:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbcfe00640 1 -- 172.21.15.73:0/270914383 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551473285 0 0) 0x7fdbd006adc0 con 0x7fdbd00b1bf0 2024-04-05T22:31:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.507+0000 7fdbd5c00640 1 --2- 172.21.15.73:0/270914383 >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 0x7fdbd00b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.508+0000 7fdbd6600640 1 --2- 172.21.15.73:0/270914383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd0166c60 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0071430 tx=0x7fdbc802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd47da591b9a9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbcfe00640 1 -- 172.21.15.73:0/270914383 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc802ca80 con 0x7fdbd00b1750 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbcfe00640 1 -- 172.21.15.73:0/270914383 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc802cbe0 con 0x7fdbd00b1750 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbcfe00640 1 -- 172.21.15.73:0/270914383 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80343f0 con 0x7fdbd00b1750 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 -- 172.21.15.73:0/270914383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 msgr2=0x7fdbd0166c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 --2- 172.21.15.73:0/270914383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd0166c60 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0071430 tx=0x7fdbc802bee0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 -- 172.21.15.73:0/270914383 shutdown_connections 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 --2- 172.21.15.73:0/270914383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd0166c60 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 --2- 172.21.15.73:0/270914383 >> v2:172.21.15.157:3300/0 conn(0x7fdbd00b0720 0x7fdbd00b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.509+0000 7fdbd8600640 1 -- 172.21.15.73:0/270914383 >> 172.21.15.73:0/270914383 conn(0x7fdbd0099a60 msgr2=0x7fdbd0099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.510+0000 7fdbd8600640 1 -- 172.21.15.73:0/270914383 shutdown_connections 2024-04-05T22:31:47.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.510+0000 7fdbd8600640 1 -- 172.21.15.73:0/270914383 wait complete. 2024-04-05T22:31:47.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.511+0000 7fdbd8600640 1 Processor -- start 2024-04-05T22:31:47.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.511+0000 7fdbd8600640 1 -- start start 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbd00b0720 0x7fdbd01f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd00b1bf0 0x7fdbd013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015cd20 con 0x7fdbd00b1750 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0159680 con 0x7fdbd00b1bf0 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.512+0000 7fdbd8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0159960 con 0x7fdbd00b0720 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43018/0 (socket says 172.21.15.73:43018) 2024-04-05T22:31:47.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 -- 172.21.15.73:0/1929252744 learned_addr learned my addr 172.21.15.73:0/1929252744 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5c00640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbd00b0720 0x7fdbd01f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd6600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd00b1bf0 0x7fdbd013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 -- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbd00b0720 msgr2=0x7fdbd01f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbd00b0720 0x7fdbd01f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 -- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd00b1bf0 msgr2=0x7fdbd013ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd00b1bf0 0x7fdbd013ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.513+0000 7fdbd5200640 1 -- 172.21.15.73:0/1929252744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8009b30 con 0x7fdbd00b1750 2024-04-05T22:31:47.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbd5200640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c4d0 tx=0x7fdbbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0074e0 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc007660 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd0159c40 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc004a20 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.514+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd01f2370 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.515+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd006afd0 con 0x7fdbd00b1750 2024-04-05T22:31:47.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.516+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fdbbc004b80 con 0x7fdbd00b1750 2024-04-05T22:31:47.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.517+0000 7fdbcd600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 0x7fdbb006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:47.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.517+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc087360 con 0x7fdbd00b1750 2024-04-05T22:31:47.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.518+0000 7fdbd5c00640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 0x7fdbb006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:47.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.519+0000 7fdbd5c00640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 0x7fdbb006da50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0153ab0 tx=0x7fdbc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:47.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.522+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdbbc053d90 con 0x7fdbd00b1750 2024-04-05T22:31:47.782 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-05T22:31:47.783+0000 7f7373e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-05T22:31:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.786+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdbd0159e30 con 0x7fdbd00b1750 2024-04-05T22:31:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.794+0000 7fdbcd600640 1 -- 172.21.15.73:0/1929252744 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdbbc018020 con 0x7fdbd00b1750 2024-04-05T22:31:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.796+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 msgr2=0x7fdbb006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.796+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 0x7fdbb006da50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0153ab0 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.796+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 msgr2=0x7fdbd01545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.796+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c4d0 tx=0x7fdbbc00c990 comp rx=0 tx=0).stop 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 shutdown_connections 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdbb006b590 0x7fdbb006da50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd00b1bf0 0x7fdbd013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbd00b1750 0x7fdbd01545e0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 --2- 172.21.15.73:0/1929252744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbd00b0720 0x7fdbd01f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 >> 172.21.15.73:0/1929252744 conn(0x7fdbd0099a60 msgr2=0x7fdbd00b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 shutdown_connections 2024-04-05T22:31:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:47.797+0000 7fdbd8600640 1 -- 172.21.15.73:0/1929252744 wait complete. 2024-04-05T22:31:47.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:31:48.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.028+0000 7f823e600640 1 Processor -- start 2024-04-05T22:31:48.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.028+0000 7f823e600640 1 -- start start 2024-04-05T22:31:48.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f823e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82381587d0 0x7f82380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f823e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 0x7f823815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f823e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823806f5b0 con 0x7f82380ccd20 2024-04-05T22:31:48.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f823e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823815f240 con 0x7f82381587d0 2024-04-05T22:31:48.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f823e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8238070410 con 0x7f823815e850 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f8237400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82381587d0 0x7f82380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f8236a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f82380ccd20 0x7f823806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49094/0 (socket says 172.21.15.73:49094) 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.030+0000 7f8236a00640 1 -- 172.21.15.73:0/4054993194 learned_addr learned my addr 172.21.15.73:0/4054993194 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237e00640 1 --2- 172.21.15.73:0/4054993194 >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 0x7f823815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237400640 1 -- 172.21.15.73:0/4054993194 >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 msgr2=0x7f823815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237400640 1 --2- 172.21.15.73:0/4054993194 >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 0x7f823815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8236000640 1 -- 172.21.15.73:0/4054993194 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010474691 0 0) 0x7f823806f5b0 con 0x7f82380ccd20 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237400640 1 -- 172.21.15.73:0/4054993194 >> v1:172.21.15.73:6789/0 conn(0x7f82380ccd20 legacy=0x7f823806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237400640 1 -- 172.21.15.73:0/4054993194 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238158bb0 con 0x7f82381587d0 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.031+0000 7f8237e00640 1 --2- 172.21.15.73:0/4054993194 >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 0x7f823815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.032+0000 7f8237400640 1 --2- 172.21.15.73:0/4054993194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f82380cc7e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f822802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74175a5181f98f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.032+0000 7f8236000640 1 -- 172.21.15.73:0/4054993194 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822802cc60 con 0x7f82381587d0 2024-04-05T22:31:48.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.032+0000 7f8236000640 1 -- 172.21.15.73:0/4054993194 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822802cdc0 con 0x7f82381587d0 2024-04-05T22:31:48.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.032+0000 7f8236000640 1 -- 172.21.15.73:0/4054993194 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228034660 con 0x7f82381587d0 2024-04-05T22:31:48.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 -- 172.21.15.73:0/4054993194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 msgr2=0x7f82380cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 --2- 172.21.15.73:0/4054993194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f82380cc7e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f822802beb0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 -- 172.21.15.73:0/4054993194 shutdown_connections 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 --2- 172.21.15.73:0/4054993194 >> v2:172.21.15.157:3300/0 conn(0x7f823815e850 0x7f823815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 --2- 172.21.15.73:0/4054993194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f82380cc7e0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 -- 172.21.15.73:0/4054993194 >> 172.21.15.73:0/4054993194 conn(0x7f82380ce8e0 msgr2=0x7f82380cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 -- 172.21.15.73:0/4054993194 shutdown_connections 2024-04-05T22:31:48.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.033+0000 7f823e600640 1 -- 172.21.15.73:0/4054993194 wait complete. 2024-04-05T22:31:48.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.035+0000 7f823e600640 1 Processor -- start 2024-04-05T22:31:48.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.035+0000 7f823e600640 1 -- start start 2024-04-05T22:31:48.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f823815e850 0x7f82380af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82380afe30 0x7f82381f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380af4d0 con 0x7f82381587d0 2024-04-05T22:31:48.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380af110 con 0x7f823815e850 2024-04-05T22:31:48.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f823e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381f4430 con 0x7f82380afe30 2024-04-05T22:31:48.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f8237400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f8237400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43040/0 (socket says 172.21.15.73:43040) 2024-04-05T22:31:48.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.036+0000 7f8237400640 1 -- 172.21.15.73:0/2247540295 learned_addr learned my addr 172.21.15.73:0/2247540295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 -- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82380afe30 msgr2=0x7f82381f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82380afe30 0x7f82381f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 -- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f823815e850 msgr2=0x7f82380af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8236a00640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f823815e850 0x7f82380af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f823815e850 0x7f82380af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 -- 172.21.15.73:0/2247540295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228009a70 con 0x7f82381587d0 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f8237400640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f8228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228040650 con 0x7f82381587d0 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82280407b0 con 0x7f82381587d0 2024-04-05T22:31:48.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228040b60 con 0x7f82381587d0 2024-04-05T22:31:48.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.037+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82381f4710 con 0x7f82381587d0 2024-04-05T22:31:48.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.038+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823814a120 con 0x7f82381587d0 2024-04-05T22:31:48.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.039+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8238066af0 con 0x7f82381587d0 2024-04-05T22:31:48.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.039+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f8228040cc0 con 0x7f82381587d0 2024-04-05T22:31:48.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.040+0000 7f822f400640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 0x7f821406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.040+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82280ad690 con 0x7f82381587d0 2024-04-05T22:31:48.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.041+0000 7f8236a00640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 0x7f821406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.042+0000 7f8236a00640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 0x7f821406da80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8220007fd0 tx=0x7f8220009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.045+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f8228077690 con 0x7f82381587d0 2024-04-05T22:31:48.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.279+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8238068160 con 0x7f82381587d0 2024-04-05T22:31:48.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.279+0000 7f822f400640 1 -- 172.21.15.73:0/2247540295 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4753 (secure 0 0 0) 0x7f822807acb0 con 0x7f82381587d0 2024-04-05T22:31:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:48.279 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:31:48.279 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 109 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 msgr2=0x7f821406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 0x7f821406da80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8220007fd0 tx=0x7f8220009040 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 msgr2=0x7f8238143600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f8228004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 shutdown_connections 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f821406b5c0 0x7f821406da80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82380afe30 0x7f82381f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f823815e850 0x7f82380af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 --2- 172.21.15.73:0/2247540295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82381587d0 0x7f8238143600 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 >> 172.21.15.73:0/2247540295 conn(0x7f82380ce8e0 msgr2=0x7f823815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.283+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 shutdown_connections 2024-04-05T22:31:48.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.284+0000 7f823e600640 1 -- 172.21.15.73:0/2247540295 wait complete. 2024-04-05T22:31:48.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-05T22:31:42.297895+0000', 'modified': '2024-04-05T22:31:45.322696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10179': {'gid': 10179, 'name': 'c', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6834/1107789803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1107789803}, {'type': 'v1', 'addr': '172.21.15.73:6834', 'nonce': 1107789803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-05T22:31:48.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-05T22:31:48.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-05T22:31:48.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.922+0000 7feecbc00640 1 Processor -- start 2024-04-05T22:31:48.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.922+0000 7feecbc00640 1 -- start start 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feecbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feec406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feecbc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec406f5b0 con 0x7feec415e850 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feecbc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4070410 con 0x7feec41587d0 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feecbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec406d500 con 0x7feec406cb80 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feec9c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feec415e850 0x7feec40708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49110/0 (socket says 172.21.15.73:49110) 2024-04-05T22:31:48.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.924+0000 7feec9c00640 1 -- 172.21.15.73:0/2560484335 learned_addr learned my addr 172.21.15.73:0/2560484335 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:48.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3e00640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feec406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590620179 0 0) 0x7feec406f5b0 con 0x7feec415e850 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea8003520 con 0x7feec415e850 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199313776 0 0) 0x7feea8003520 con 0x7feec415e850 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 msgr2=0x7feec406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feec406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 >> v1:172.21.15.157:6789/0 conn(0x7feec41587d0 legacy=0x7feec40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40ccef0 con 0x7feec415e850 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7feebc001670 con 0x7feec415e850 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 >> v1:172.21.15.73:6789/0 conn(0x7feec415e850 legacy=0x7feec40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feea8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feea80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feea8004d00 0x7feea8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec406d500 con 0x7feea80039a0 2024-04-05T22:31:48.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea800bb10 con 0x7feea8004d00 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea800bca0 con 0x7feec406cb80 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.925+0000 7feec9200640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feea8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9c00640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feea8004d00 0x7feea8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec3e00640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feea80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9200640 1 -- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 msgr2=0x7feea80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9200640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec406cb80 0x7feea80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9200640 1 -- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feea8004d00 msgr2=0x7feea8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9200640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feea8004d00 0x7feea8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.926+0000 7feec9200640 1 -- 172.21.15.73:0/2560484335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8003520 con 0x7feea80039a0 2024-04-05T22:31:48.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.927+0000 7feec9200640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feea8003dc0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7feeb4001e80 tx=0x7feea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05516536308a53c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.927+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4003b70 con 0x7feea80039a0 2024-04-05T22:31:48.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.927+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4003cd0 con 0x7feea80039a0 2024-04-05T22:31:48.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.927+0000 7feec3400640 1 -- 172.21.15.73:0/2560484335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005310 con 0x7feea80039a0 2024-04-05T22:31:48.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 -- 172.21.15.73:0/2560484335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 msgr2=0x7feea8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 --2- 172.21.15.73:0/2560484335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feea8003dc0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7feeb4001e80 tx=0x7feea80057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feec9c00640 1 -- 172.21.15.73:0/2560484335 reap_dead start 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 -- 172.21.15.73:0/2560484335 shutdown_connections 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 -- 172.21.15.73:0/2560484335 >> 172.21.15.73:0/2560484335 conn(0x7feec40ce8e0 msgr2=0x7feec40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 -- 172.21.15.73:0/2560484335 shutdown_connections 2024-04-05T22:31:48.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.928+0000 7feecbc00640 1 -- 172.21.15.73:0/2560484335 wait complete. 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.930+0000 7feecbc00640 1 Processor -- start 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.930+0000 7feecbc00640 1 -- start start 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec413cc70 0x7feec40aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40aa750 0x7feec40ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40aabd0 con 0x7feea80039a0 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40a9660 con 0x7feec40aa750 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feecbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40ad130 con 0x7feec413cc70 2024-04-05T22:31:48.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feec9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feec9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43054/0 (socket says 172.21.15.73:43054) 2024-04-05T22:31:48.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feec9200640 1 -- 172.21.15.73:0/1409166813 learned_addr learned my addr 172.21.15.73:0/1409166813 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.931+0000 7feec9200640 1 -- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec413cc70 msgr2=0x7feec40aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9c00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40aa750 0x7feec40ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9200640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec413cc70 0x7feec40aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9200640 1 -- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40aa750 msgr2=0x7feec40ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9200640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40aa750 0x7feec40ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9200640 1 -- 172.21.15.73:0/1409166813 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40a8640 con 0x7feea80039a0 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec9200640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005040 tx=0x7feeb40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec0c00640 1 -- 172.21.15.73:0/1409166813 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4006d50 con 0x7feea80039a0 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec0c00640 1 -- 172.21.15.73:0/1409166813 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4007030 con 0x7feea80039a0 2024-04-05T22:31:48.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec40a8920 con 0x7feea80039a0 2024-04-05T22:31:48.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feec0c00640 1 -- 172.21.15.73:0/1409166813 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40073e0 con 0x7feea80039a0 2024-04-05T22:31:48.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.932+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec413d0f0 con 0x7feea80039a0 2024-04-05T22:31:48.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.934+0000 7feec0c00640 1 -- 172.21.15.73:0/1409166813 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7feeb4007540 con 0x7feea80039a0 2024-04-05T22:31:48.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.935+0000 7feec0c00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 0x7feea406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.936+0000 7feec0c00640 1 -- 172.21.15.73:0/1409166813 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeb40a9b90 con 0x7feea80039a0 2024-04-05T22:31:48.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.936+0000 7feec3e00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 0x7feea406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.936+0000 7feecebbf740 1 Processor -- start 2024-04-05T22:31:48.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.936+0000 7feecebbf740 1 -- start start 2024-04-05T22:31:48.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b4082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 0x56011b432f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b3738c0 con 0x56011b450d70 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b371050 con 0x56011b4087e0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011b36c940 con 0x56011b4305b0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feec3e00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 0x7feea406da50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feeb0006f00 tx=0x7feeb0006e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.937+0000 7feec9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43068/0 (socket says 172.21.15.73:43068) 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 -- 172.21.15.73:0/379944547 learned_addr learned my addr 172.21.15.73:0/379944547 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9c00640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 0x56011b432f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec3e00640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b4082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 -- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 msgr2=0x56011b4082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b4082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 -- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 msgr2=0x56011b432f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 0x56011b432f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 -- 172.21.15.73:0/379944547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56011b3379e0 con 0x56011b450d70 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9c00640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 0x56011b432f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feec9200640 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7feeb4082140 tx=0x7feeb4079be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ecfd26640d9587 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feeb9600640 1 -- 172.21.15.73:0/379944547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40ac0c0 con 0x56011b450d70 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feeb9600640 1 -- 172.21.15.73:0/379944547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb407e0b0 con 0x56011b450d70 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.938+0000 7feeb9600640 1 -- 172.21.15.73:0/379944547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4082670 con 0x56011b450d70 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 -- 172.21.15.73:0/379944547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 msgr2=0x56011b44e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7feeb4082140 tx=0x7feeb4079be0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 -- 172.21.15.73:0/379944547 shutdown_connections 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4087e0 0x56011b432f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b4082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 --2- 172.21.15.73:0/379944547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b44e930 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 -- 172.21.15.73:0/379944547 >> 172.21.15.73:0/379944547 conn(0x56011b405400 msgr2=0x56011b44df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 -- 172.21.15.73:0/379944547 shutdown_connections 2024-04-05T22:31:48.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.939+0000 7feecebbf740 1 -- 172.21.15.73:0/379944547 wait complete. 2024-04-05T22:31:48.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.940+0000 7feecebbf740 1 -- start start 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.940+0000 7feecebbf740 1 Processor -- start 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 0x56011b42def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b42c230 0x56011b44f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b30e070 con 0x56011b450d70 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b30dd10 con 0x56011b4305b0 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011b0d3f20 con 0x56011b42c230 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.941+0000 7feecebbf740 10 client.0 _ll_register_callbacks cb 0x56011b365fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43078/0 (socket says 172.21.15.73:43078) 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 -- 172.21.15.73:0/635929444 learned_addr learned my addr 172.21.15.73:0/635929444 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec9200640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 0x56011b42def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec9c00640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b42c230 0x56011b44f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 -- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b42c230 msgr2=0x56011b44f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b42c230 0x56011b44f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 -- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 msgr2=0x56011b42def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 0x56011b42def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 -- 172.21.15.73:0/635929444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec413d0f0 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec9200640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 0x56011b42def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feec3e00640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7feeb0015cb0 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d842ecb8c4b5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.942+0000 7feeae000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0024070 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeb0013b80 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56011b396ae0 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeb0015e60 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 10 client.10329 fetch_fsmap learned FSMap version 109 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56011b36afc0 con 0x56011b450d70 2024-04-05T22:31:48.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56011b396ae0 con 0x56011b450d70 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 109) v1 ==== 3114+0+0 (secure 0 0 0) 0x7feeb0027020 con 0x56011b450d70 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 10 client.10329 fetch_fsmap finished waiting for FSMap version 109 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 10 client.10329 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 20 client.10329 populate_metadata read hostname 'smithi073' 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.943+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b4484d0 0x56011b4488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.944+0000 7feecebbf740 4 client.10329 mds_command: new command op to 10179 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.944+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 --> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56011b0603c0 con 0x56011b4484d0 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.944+0000 7feec9200640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b4484d0 0x56011b4488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.944+0000 7feec9200640 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b4484d0 0x56011b4488b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec52e1753957cf5d server_cookie=c8dac4be01ed7770 in_seq=0 out_seq=0 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.944+0000 7feeae000640 10 client.10329 ms_handle_connect on v2:172.21.15.73:6832/1107789803 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feeae000640 1 -- 172.21.15.73:0/635929444 <== mds.0 v2:172.21.15.73:6832/1107789803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7feeb4091e90 con 0x56011b4484d0 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feeae000640 10 client.10329 handle_command_reply: tid=0 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 client.10329 shutdown 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 msgr2=0x56011b42bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7feeb0015cb0 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 shutdown_connections 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b4484d0 0x56011b4488b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b42c230 0x56011b44f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b42bca0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 --2- 172.21.15.73:0/635929444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b4305b0 0x56011b42def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feeae000640 0 client.10329 ms_handle_reset on v2:172.21.15.73:6832/1107789803 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 >> 172.21.15.73:0/635929444 conn(0x56011b4087e0 msgr2=0x56011b450440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.945+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 shutdown_connections 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.946+0000 7feecebbf740 1 -- 172.21.15.73:0/635929444 wait complete. 2024-04-05T22:31:48.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.946+0000 7feecebbf740 20 client.10329 trim_cache size 0 max 16384 2024-04-05T22:31:48.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.962+0000 7feecebbf740 1 Processor -- start 2024-04-05T22:31:48.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.962+0000 7feecebbf740 1 -- start start 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b42b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b436ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b42e560 con 0x56011b450d70 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b3fea90 con 0x56011b42c230 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011b42bb50 con 0x56011b4305b0 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feec9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feec9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43086/0 (socket says 172.21.15.73:43086) 2024-04-05T22:31:48.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.963+0000 7feec9c00640 1 -- 172.21.15.73:0/1151415320 learned_addr learned my addr 172.21.15.73:0/1151415320 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9200640 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b42b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 -- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 msgr2=0x56011b436ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b436ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 -- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 msgr2=0x56011b42b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b42b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 -- 172.21.15.73:0/1151415320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56011b39ff80 con 0x56011b450d70 2024-04-05T22:31:48.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feec9c00640 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7feec40adc80 tx=0x7feebc006bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ba21a35f3e28e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feeba000640 1 -- 172.21.15.73:0/1151415320 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0062d0 con 0x56011b450d70 2024-04-05T22:31:48.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feeba000640 1 -- 172.21.15.73:0/1151415320 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc006450 con 0x56011b450d70 2024-04-05T22:31:48.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.964+0000 7feeba000640 1 -- 172.21.15.73:0/1151415320 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0047e0 con 0x56011b450d70 2024-04-05T22:31:48.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 -- 172.21.15.73:0/1151415320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 msgr2=0x56011b437000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7feec40adc80 tx=0x7feebc006bd0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 -- 172.21.15.73:0/1151415320 shutdown_connections 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b450d70 0x56011b437000 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b4305b0 0x56011b436ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 --2- 172.21.15.73:0/1151415320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b42b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 -- 172.21.15.73:0/1151415320 >> 172.21.15.73:0/1151415320 conn(0x56011b4087e0 msgr2=0x56011b402f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 -- 172.21.15.73:0/1151415320 shutdown_connections 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.965+0000 7feecebbf740 1 -- 172.21.15.73:0/1151415320 wait complete. 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.966+0000 7feecebbf740 1 -- start start 2024-04-05T22:31:48.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.966+0000 7feecebbf740 1 Processor -- start 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b470e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 0x56011b4488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b3febd0 con 0x56011b4305b0 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011b4713b0 con 0x56011b42c230 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011b471550 con 0x56011b450d70 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feecebbf740 10 client.0 _ll_register_callbacks cb 0x56011b407ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feec3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feec3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43102/0 (socket says 172.21.15.73:43102) 2024-04-05T22:31:48.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.967+0000 7feec3e00640 1 -- 172.21.15.73:0/374209989 learned_addr learned my addr 172.21.15.73:0/374209989 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec9c00640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 0x56011b4488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec9200640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b470e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 -- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 msgr2=0x56011b4488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 0x56011b4488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 -- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 msgr2=0x56011b470e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b470e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 -- 172.21.15.73:0/374209989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec40ccef0 con 0x56011b4305b0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec9c00640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 0x56011b4488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feec3e00640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7feeb0006d20 tx=0x7feeb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18b4e27bb4624ef server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feeae000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0024070 con 0x56011b4305b0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feeb407e0b0 con 0x56011b4305b0 2024-04-05T22:31:48.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.968+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeb0013b80 con 0x56011b4305b0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeb0015800 con 0x56011b4305b0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 10 client.10335 fetch_fsmap learned FSMap version 109 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56011b30d280 con 0x56011b4305b0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb407e0b0 con 0x56011b4305b0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 109) v1 ==== 3114+0+0 (secure 0 0 0) 0x7feeb0027050 con 0x56011b4305b0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 10 client.10335 fetch_fsmap finished waiting for FSMap version 109 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 10 client.10335 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 20 client.10335 populate_metadata read hostname 'smithi073' 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b44c620 0x56011b44ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 4 client.10335 mds_command: new command op to 10179 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.969+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 --> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56011b0716c0 con 0x56011b44c620 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feec9200640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b44c620 0x56011b44ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feec9200640 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b44c620 0x56011b44ca00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e2359136cd4595 server_cookie=dccd18d74702e7c5 in_seq=0 out_seq=0 2024-04-05T22:31:48.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feeae000640 10 client.10335 ms_handle_connect on v2:172.21.15.73:6832/1107789803 2024-04-05T22:31:48.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feeae000640 1 -- 172.21.15.73:0/374209989 <== mds.0 v2:172.21.15.73:6832/1107789803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x56011b0716c0 con 0x56011b44c620 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feeae000640 10 client.10335 handle_command_reply: tid=0 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feecebbf740 1 client.10335 shutdown 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 msgr2=0x56011b4bd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.970+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7feeb0006d20 tx=0x7feeb0004730 comp rx=0 tx=0).stop 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 shutdown_connections 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:6832/1107789803,v1:172.21.15.73:6834/1107789803] conn(0x56011b44c620 0x56011b44ca00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56011b450d70 0x56011b4488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56011b4305b0 0x56011b4bd180 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 --2- 172.21.15.73:0/374209989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56011b42c230 0x56011b470e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feeae000640 0 client.10335 ms_handle_reset on v2:172.21.15.73:6832/1107789803 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 >> 172.21.15.73:0/374209989 conn(0x56011b4087e0 msgr2=0x56011b408cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 shutdown_connections 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 1 -- 172.21.15.73:0/374209989 wait complete. 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecebbf740 20 client.10335 trim_cache size 0 max 16384 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "38861c87-67cf-4180-a71f-97401becb77d", 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 10179, 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 6.6685886669999999, 2024-04-05T22:31:48.974 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 109, 2024-04-05T22:31:48.975 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 172, 2024-04-05T22:31:48.975 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 172, 2024-04-05T22:31:48.975 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 11.580933111 2024-04-05T22:31:48.975 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 msgr2=0x7feea406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 0x7feea406da50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feeb0006f00 tx=0x7feeb0006e10 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 msgr2=0x7feec40a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005040 tx=0x7feeb40014a0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 shutdown_connections 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.971+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feea406b590 0x7feea406da50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40aa750 0x7feec40ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec413cc70 0x7feec40aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 --2- 172.21.15.73:0/1409166813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80039a0 0x7feec40a9120 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 >> 172.21.15.73:0/1409166813 conn(0x7feec40ce8e0 msgr2=0x7feec415ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 shutdown_connections 2024-04-05T22:31:48.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:48.972+0000 7feecbc00640 1 -- 172.21.15.73:0/1409166813 wait complete. 2024-04-05T22:31:48.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:31:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.190+0000 7fb40ce00640 1 Processor -- start 2024-04-05T22:31:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.190+0000 7fb40ce00640 1 -- start start 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb40ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb40815ef60 0x7fb40806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb40ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40806f5b0 con 0x7fb4081587d0 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb40ce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb408070410 con 0x7fb40806cb80 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb40ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40806d500 con 0x7fb40815ef60 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb406800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb40815ef60 0x7fb40806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb406800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb40815ef60 0x7fb40806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43106/0 (socket says 172.21.15.73:43106) 2024-04-05T22:31:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.192+0000 7fb406800640 1 -- 172.21.15.73:0/2469061773 learned_addr learned my addr 172.21.15.73:0/2469061773 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.193+0000 7fb406800640 1 -- 172.21.15.73:0/2469061773 >> v1:172.21.15.157:6789/0 conn(0x7fb40806cb80 legacy=0x7fb40815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.193+0000 7fb406800640 1 -- 172.21.15.73:0/2469061773 >> v1:172.21.15.73:6789/0 conn(0x7fb4081587d0 legacy=0x7fb4080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.193+0000 7fb406800640 1 -- 172.21.15.73:0/2469061773 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4080ccef0 con 0x7fb40815ef60 2024-04-05T22:31:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.193+0000 7fb404a00640 1 -- 172.21.15.73:0/2469061773 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459096732 0 0) 0x7fb40806f5b0 con 0x7fb4081587d0 2024-04-05T22:31:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.194+0000 7fb406800640 1 --2- 172.21.15.73:0/2469061773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40806cfc0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fb400009e50 tx=0x7fb40002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9beb55c3feec8222 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.194+0000 7fb404a00640 1 -- 172.21.15.73:0/2469061773 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40002cbb0 con 0x7fb40815ef60 2024-04-05T22:31:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.194+0000 7fb404a00640 1 -- 172.21.15.73:0/2469061773 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40002cd10 con 0x7fb40815ef60 2024-04-05T22:31:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.194+0000 7fb404a00640 1 -- 172.21.15.73:0/2469061773 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400034610 con 0x7fb40815ef60 2024-04-05T22:31:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 -- 172.21.15.73:0/2469061773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 msgr2=0x7fb40806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2469061773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40806cfc0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fb400009e50 tx=0x7fb40002c010 comp rx=0 tx=0).stop 2024-04-05T22:31:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 -- 172.21.15.73:0/2469061773 shutdown_connections 2024-04-05T22:31:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2469061773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40806cfc0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 -- 172.21.15.73:0/2469061773 >> 172.21.15.73:0/2469061773 conn(0x7fb4080ce8e0 msgr2=0x7fb4080cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:49.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 -- 172.21.15.73:0/2469061773 shutdown_connections 2024-04-05T22:31:49.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.195+0000 7fb40ce00640 1 -- 172.21.15.73:0/2469061773 wait complete. 2024-04-05T22:31:49.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.197+0000 7fb40ce00640 1 Processor -- start 2024-04-05T22:31:49.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.197+0000 7fb40ce00640 1 -- start start 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40806cb80 0x7fb4081f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4081587d0 0x7fb408145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4081f4120 con 0x7fb40815ef60 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080abca0 con 0x7fb40806cb80 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb40ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4081f3b30 con 0x7fb4081587d0 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb406800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb406800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43110/0 (socket says 172.21.15.73:43110) 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.198+0000 7fb406800640 1 -- 172.21.15.73:0/2138666835 learned_addr learned my addr 172.21.15.73:0/2138666835 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:49.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb405e00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40806cb80 0x7fb4081f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb405400640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4081587d0 0x7fb408145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 -- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4081587d0 msgr2=0x7fb408145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4081587d0 0x7fb408145ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 -- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40806cb80 msgr2=0x7fb4081f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40806cb80 0x7fb4081f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 -- 172.21.15.73:0/2138666835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009b30 con 0x7fb40815ef60 2024-04-05T22:31:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb406800640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fb400036040 tx=0x7fb400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400040670 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.199+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000407f0 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.200+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4081f4a80 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.200+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400040ba0 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.200+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb408146c20 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.201+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb408066af0 con 0x7fb40815ef60 2024-04-05T22:31:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.202+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fb400040d00 con 0x7fb40815ef60 2024-04-05T22:31:49.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.202+0000 7fb3fe000640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 0x7fb3e806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:49.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.203+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4000aafc0 con 0x7fb40815ef60 2024-04-05T22:31:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.203+0000 7fb405e00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 0x7fb3e806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.204+0000 7fb405e00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 0x7fb3e806da50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8000f60 tx=0x7fb3f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.207+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fb400077b10 con 0x7fb40815ef60 2024-04-05T22:31:49.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.439+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4081f4260 con 0x7fb40815ef60 2024-04-05T22:31:49.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.439+0000 7fb3fe000640 1 -- 172.21.15.73:0/2138666835 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4081f4260 con 0x7fb40815ef60 2024-04-05T22:31:49.438 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 msgr2=0x7fb3e806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 0x7fb3e806da50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8000f60 tx=0x7fb3f8009040 comp rx=0 tx=0).stop 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 msgr2=0x7fb40815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fb400036040 tx=0x7fb400004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 shutdown_connections 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.442+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb3e806b590 0x7fb3e806da50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb40815ef60 0x7fb40815c930 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4081587d0 0x7fb408145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 --2- 172.21.15.73:0/2138666835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40806cb80 0x7fb4081f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 >> 172.21.15.73:0/2138666835 conn(0x7fb4080ce8e0 msgr2=0x7fb4080cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 shutdown_connections 2024-04-05T22:31:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:49.443+0000 7fb40ce00640 1 -- 172.21.15.73:0/2138666835 wait complete. 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:31:49.460 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:49.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.461 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-05T22:31:49.512 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:31:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:31:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:31:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41954sec preferred_lft 41954sec 2024-04-05T22:31:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c827:25ff:fe06:372f/64 scope link 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ba:25:c3:49:bb:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b825:c3ff:fe49:bb97/64 scope link 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether e6:b8:80:ca:63:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e4b8:80ff:feca:637a/64 scope link 2024-04-05T22:31:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:49.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.516 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:31:49.601 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-05T22:31:49.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.605 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-05T22:31:49.654 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-05T22:31:49.655 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:31:49.655 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:31:49.655 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:31:49.655 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:31:49.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.655 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-04-05T22:31:49.655 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-05T22:31:49.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:31:49.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:31:49.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:31:49.656 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-05T22:31:49.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.831 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:31:49.876 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:31:49.876 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:31:49.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:49.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:49.876 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-05T22:31:50.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:50.006 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:50.061 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-05T22:31:50.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:50.062 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:50.111 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-05T22:31:50.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:50.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:50.250 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:50.251 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:50.285 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:50.286 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:50.286 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:31:50.286 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with new device syntax: 0@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:31:50.286 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:31:50.288 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-05T22:31:50.288 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:31:50.288 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:31:50.288 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:31:50.288 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:31:50.292 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:31:50.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:50.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-05T22:31:50.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:50.380 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:31:50.443 INFO:teuthology.orchestra.run.smithi073.stdout:client10347 2024-04-05T22:31:50.443 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:50.443 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:50.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-05T22:31:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.749+0000 7f05e9000640 1 Processor -- start 2024-04-05T22:31:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.749+0000 7f05e9000640 1 -- start start 2024-04-05T22:31:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.750+0000 7f05e9000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05e4162370 0x7f05e40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e9000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e41627f0 con 0x7f05e4161f90 2024-04-05T22:31:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e9000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e4165cf0 con 0x7f05e40b2ba0 2024-04-05T22:31:50.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e9000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e4158790 con 0x7f05e4162370 2024-04-05T22:31:50.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e1e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f05e4161f90 0x7f05e4164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49140/0 (socket says 172.21.15.73:49140) 2024-04-05T22:31:50.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e1e00640 1 -- 172.21.15.73:0/1245911167 learned_addr learned my addr 172.21.15.73:0/1245911167 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e1400640 1 --2- 172.21.15.73:0/1245911167 >> v2:172.21.15.157:3300/0 conn(0x7f05e4162370 0x7f05e40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814817234 0 0) 0x7f05cc001110 con 0x7f05e4161f90 2024-04-05T22:31:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.751+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c8003520 con 0x7f05e4161f90 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930114028 0 0) 0x7f05c8003520 con 0x7f05e4161f90 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 >> v2:172.21.15.157:3300/0 conn(0x7f05e4162370 msgr2=0x7f05e40b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 --2- 172.21.15.73:0/1245911167 >> v2:172.21.15.157:3300/0 conn(0x7f05e4162370 0x7f05e40b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 >> v1:172.21.15.157:6789/0 conn(0x7f05e40b2ba0 legacy=0x7f05e4070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4165730 con 0x7f05e4161f90 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f05cc001800 con 0x7f05e4161f90 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 >> v1:172.21.15.73:6789/0 conn(0x7f05e4161f90 legacy=0x7f05e4164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c80047c0 0x7f05c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05c8005180 0x7f05c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4158790 con 0x7f05c8003970 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c800c020 con 0x7f05c80047c0 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c800c1e0 con 0x7f05c8005180 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.752+0000 7f05e1e00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1e00640 1 -- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05c8005180 msgr2=0x7f05c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1e00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05c8005180 0x7f05c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1e00640 1 -- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c80047c0 msgr2=0x7f05c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1400640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c80047c0 0x7f05c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1e00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c80047c0 0x7f05c8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e1e00640 1 -- 172.21.15.73:0/1245911167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8003520 con 0x7f05c8003970 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.753+0000 7f05e2800640 1 -- 172.21.15.73:0/1245911167 reap_dead start 2024-04-05T22:31:50.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e1e00640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05c8003d90 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00c190 tx=0x7f05c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d73bb39dc437eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc005ad0 con 0x7f05c8003970 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc005c50 con 0x7f05c8003970 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e0a00640 1 -- 172.21.15.73:0/1245911167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc02c7e0 con 0x7f05c8003970 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e9000640 1 -- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 msgr2=0x7f05c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.754+0000 7f05e9000640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05c8003d90 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00c190 tx=0x7f05c80057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.755+0000 7f05e9000640 1 -- 172.21.15.73:0/1245911167 shutdown_connections 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.755+0000 7f05e9000640 1 --2- 172.21.15.73:0/1245911167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05c8003d90 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.755+0000 7f05e9000640 1 -- 172.21.15.73:0/1245911167 >> 172.21.15.73:0/1245911167 conn(0x7f05e40ce8e0 msgr2=0x7f05e40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:50.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.755+0000 7f05e9000640 1 -- 172.21.15.73:0/1245911167 shutdown_connections 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.755+0000 7f05e9000640 1 -- 172.21.15.73:0/1245911167 wait complete. 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.756+0000 7f05e9000640 1 Processor -- start 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 -- start start 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e414e0d0 0x7f05e41fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e406c890 0x7f05e406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e406d250 con 0x7f05c8003970 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4153ef0 con 0x7f05e414e0d0 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.757+0000 7f05e9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41fa220 con 0x7f05e406c890 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43138/0 (socket says 172.21.15.73:43138) 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 -- 172.21.15.73:0/2553251418 learned_addr learned my addr 172.21.15.73:0/2553251418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 -- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e406c890 msgr2=0x7f05e406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e406c890 0x7f05e406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 -- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e414e0d0 msgr2=0x7f05e41fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:50.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1400640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e414e0d0 0x7f05e41fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e414e0d0 0x7f05e41fa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 -- 172.21.15.73:0/2553251418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e41faf20 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.758+0000 7f05e1e00640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00c190 tx=0x7f05cc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.759+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc03c040 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.759+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc02cbf0 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.759+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e414ec00 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.759+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc007070 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.759+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e414e920 con 0x7f05c8003970 2024-04-05T22:31:50.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.760+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e4066af0 con 0x7f05c8003970 2024-04-05T22:31:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.761+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f05cc032070 con 0x7f05c8003970 2024-04-05T22:31:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.762+0000 7f05da000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 0x7f05c006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.762+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05cc0ab680 con 0x7f05c8003970 2024-04-05T22:31:50.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.762+0000 7f05e1400640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 0x7f05c006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.764+0000 7f05e1400640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 0x7f05c006da50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f05e414fe90 tx=0x7f05d400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:50.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.767+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f05cc075b60 con 0x7f05c8003970 2024-04-05T22:31:50.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.999+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05e41fa080 con 0x7f05c8003970 2024-04-05T22:31:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:50.999+0000 7f05da000640 1 -- 172.21.15.73:0/2553251418 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05e41fa080 con 0x7f05c8003970 2024-04-05T22:31:50.999 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-05T22:31:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 msgr2=0x7f05c006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 0x7f05c006da50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f05e414fe90 tx=0x7f05d400b040 comp rx=0 tx=0).stop 2024-04-05T22:31:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 msgr2=0x7f05e4159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:51.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00c190 tx=0x7f05cc02e070 comp rx=0 tx=0).stop 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 shutdown_connections 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f05c006b590 0x7f05c006da50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e406c890 0x7f05e406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e414e0d0 0x7f05e41fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.002+0000 7f05e9000640 1 --2- 172.21.15.73:0/2553251418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c8003970 0x7f05e4159d60 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.003+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 >> 172.21.15.73:0/2553251418 conn(0x7f05e40ce8e0 msgr2=0x7f05e40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:51.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.003+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 shutdown_connections 2024-04-05T22:31:51.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:51.003+0000 7f05e9000640 1 -- 172.21.15.73:0/2553251418 wait complete. 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:51.020 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-05T22:31:51.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-05T22:31:51.021 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-05T22:31:51.021 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:51.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.021 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-05T22:31:51.081 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:51.082 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41976sec preferred_lft 41976sec 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-05T22:31:51.083 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c8f8:e5ff:fe17:71fb/64 scope link 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 96:d5:6c:f3:36:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::94d5:6cff:fef3:36dc/64 scope link 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 4a:2c:7b:f7:92:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::482c:7bff:fef7:922e/64 scope link 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-05T22:31:51.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.085 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-05T22:31:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-05T22:31:51.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.178 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-05T22:31:51.229 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-05T22:31:51.229 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-05T22:31:51.229 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-05T22:31:51.229 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-05T22:31:51.229 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-05T22:31:51.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.230 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2024-04-05T22:31:51.231 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-05T22:31:51.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-05T22:31:51.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-05T22:31:51.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-05T22:31:51.231 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-05T22:31:51.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.411 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:31:51.463 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:31:51.463 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:31:51.464 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:51.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-05T22:31:51.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:51.559 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:51.612 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-05T22:31:51.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-05T22:31:51.613 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:51.662 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-05T22:31:51.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.662 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:51.795 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:51.795 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-05T22:31:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-05T22:31:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-04-05T22:31:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@38861c87-67cf-4180-a71f-97401becb77d.cephfs=/ 2024-04-05T22:31:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.73:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-04-05T22:31:51.830 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-05T22:31:51.830 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-05T22:31:51.830 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-05T22:31:51.830 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-05T22:31:51.830 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-05T22:31:51.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-05T22:31:51.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-05T22:31:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:51.909 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-05T22:31:51.965 INFO:teuthology.orchestra.run.smithi157.stdout:client10356 2024-04-05T22:31:51.965 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:31:51.965 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:52.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-05T22:31:52.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.229+0000 7fd0c1200640 1 Processor -- start 2024-04-05T22:31:52.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.229+0000 7fd0c1200640 1 -- start start 2024-04-05T22:31:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.231+0000 7fd0c1200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 0x7fd0bc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.231+0000 7fd0c1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.231+0000 7fd0c1200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc0caf80 con 0x7fd0bc15dad0 2024-04-05T22:31:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.231+0000 7fd0c1200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc15df60 con 0x7fd0bc159990 2024-04-05T22:31:52.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.231+0000 7fd0c1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0caa00 con 0x7fd0bc0704a0 2024-04-05T22:31:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.232+0000 7fd0bb400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 0x7fd0bc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.232+0000 7fd0bbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.232+0000 7fd0bbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:51776/0 (socket says 172.21.15.73:51776) 2024-04-05T22:31:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.232+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3299802895 learned_addr learned my addr 172.21.15.73:0/3299802895 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:52.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.232+0000 7fd0bb400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 0x7fd0bc15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.73:57126/0 (socket says 172.21.15.73:57126) 2024-04-05T22:31:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.233+0000 7fd0ba000640 1 -- 172.21.15.73:0/3299802895 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314318474 0 0) 0x7fd0bc0caf80 con 0x7fd0bc15dad0 2024-04-05T22:31:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.233+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3299802895 >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 msgr2=0x7fd0bc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.233+0000 7fd0bbe00640 1 --2- 172.21.15.73:0/3299802895 >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 0x7fd0bc15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.233+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3299802895 >> v1:172.21.15.157:6789/0 conn(0x7fd0bc15dad0 legacy=0x7fd0bc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.233+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3299802895 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc06b610 con 0x7fd0bc0704a0 2024-04-05T22:31:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.234+0000 7fd0bbe00640 1 --2- 172.21.15.73:0/3299802895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400bb10 tx=0x7fd0b402e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497c2402c4510232 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.234+0000 7fd0ba000640 1 -- 172.21.15.73:0/3299802895 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4036440 con 0x7fd0bc0704a0 2024-04-05T22:31:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0ba000640 1 -- 172.21.15.73:0/3299802895 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b40365a0 con 0x7fd0bc0704a0 2024-04-05T22:31:52.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0ba000640 1 -- 172.21.15.73:0/3299802895 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4036930 con 0x7fd0bc0704a0 2024-04-05T22:31:52.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 -- 172.21.15.73:0/3299802895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 msgr2=0x7fd0bc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3299802895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400bb10 tx=0x7fd0b402e150 comp rx=0 tx=0).stop 2024-04-05T22:31:52.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 -- 172.21.15.73:0/3299802895 shutdown_connections 2024-04-05T22:31:52.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3299802895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc0704a0 0x7fd0bc0b2930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3299802895 >> v2:172.21.15.157:3300/0 conn(0x7fd0bc159990 0x7fd0bc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 -- 172.21.15.73:0/3299802895 >> 172.21.15.73:0/3299802895 conn(0x7fd0bc0b0360 msgr2=0x7fd0bc0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:52.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 -- 172.21.15.73:0/3299802895 shutdown_connections 2024-04-05T22:31:52.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.235+0000 7fd0c1200640 1 -- 172.21.15.73:0/3299802895 wait complete. 2024-04-05T22:31:52.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.237+0000 7fd0c1200640 1 Processor -- start 2024-04-05T22:31:52.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.237+0000 7fd0c1200640 1 -- start start 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0bc0704a0 0x7fd0bc15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 0x7fd0bc0ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc148820 con 0x7fd0bc15dad0 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc15b410 con 0x7fd0bc0704a0 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.238+0000 7fd0c1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc15b6f0 con 0x7fd0bc159990 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43140/0 (socket says 172.21.15.73:43140) 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3461176332 learned_addr learned my addr 172.21.15.73:0/3461176332 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0baa00640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 0x7fd0bc0ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bb400640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0bc0704a0 0x7fd0bc15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 msgr2=0x7fd0bc0ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 0x7fd0bc0ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0bc0704a0 msgr2=0x7fd0bc15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0bc0704a0 0x7fd0bc15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 -- 172.21.15.73:0/3461176332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b400b770 con 0x7fd0bc15dad0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0baa00640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 0x7fd0bc0ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.239+0000 7fd0bbe00640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b2c0 tx=0x7fd0b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.240+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4036440 con 0x7fd0bc15dad0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.240+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4034980 con 0x7fd0bc15dad0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.240+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4034d30 con 0x7fd0bc15dad0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.240+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc147ae0 con 0x7fd0bc15dad0 2024-04-05T22:31:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.240+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc147200 con 0x7fd0bc15dad0 2024-04-05T22:31:52.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.241+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc148000 con 0x7fd0bc15dad0 2024-04-05T22:31:52.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.242+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fd0b4049360 con 0x7fd0bc15dad0 2024-04-05T22:31:52.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.243+0000 7fd0b3400640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 0x7fd09806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.243+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0b40aa010 con 0x7fd0bc15dad0 2024-04-05T22:31:52.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.244+0000 7fd0bb400640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 0x7fd09806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.245+0000 7fd0bb400640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 0x7fd09806da50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009a10 tx=0x7fd0a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.247+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd0bc148000 con 0x7fd0bc15dad0 2024-04-05T22:31:52.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.483+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd0bc148000 con 0x7fd0bc15dad0 2024-04-05T22:31:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.520+0000 7fd0b3400640 1 -- 172.21.15.73:0/3461176332 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v110) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd0bc148000 con 0x7fd0bc15dad0 2024-04-05T22:31:52.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 msgr2=0x7fd09806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 0x7fd09806da50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009a10 tx=0x7fd0a4008040 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 msgr2=0x7fd0bc1482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b2c0 tx=0x7fd0b40046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 shutdown_connections 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd09806b590 0x7fd09806da50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0bc15dad0 0x7fd0bc1482e0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0bc159990 0x7fd0bc0ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 --2- 172.21.15.73:0/3461176332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0bc0704a0 0x7fd0bc15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.523+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 >> 172.21.15.73:0/3461176332 conn(0x7fd0bc0b0360 msgr2=0x7fd0bc0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.524+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 shutdown_connections 2024-04-05T22:31:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.524+0000 7fd0c1200640 1 -- 172.21.15.73:0/3461176332 wait complete. 2024-04-05T22:31:52.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-05T22:31:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.758+0000 7fc9d1e00640 1 Processor -- start 2024-04-05T22:31:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.758+0000 7fc9d1e00640 1 -- start start 2024-04-05T22:31:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.759+0000 7fc9d1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc1587d0 0x7fc9cc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.759+0000 7fc9d1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.759+0000 7fc9d1e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc06f5b0 con 0x7fc9cc06d0c0 2024-04-05T22:31:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9d1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc070410 con 0x7fc9cc0ccd20 2024-04-05T22:31:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9d1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc158bb0 con 0x7fc9cc1587d0 2024-04-05T22:31:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43142/0 (socket says 172.21.15.73:43142) 2024-04-05T22:31:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 -- 172.21.15.73:0/1644942246 learned_addr learned my addr 172.21.15.73:0/1644942246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9caa00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc1587d0 0x7fc9cc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 -- 172.21.15.73:0/1644942246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc1587d0 msgr2=0x7fc9cc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc1587d0 0x7fc9cc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 -- 172.21.15.73:0/1644942246 >> v1:172.21.15.157:6789/0 conn(0x7fc9cc06d0c0 legacy=0x7fc9cc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.760+0000 7fc9c3e00640 1 -- 172.21.15.73:0/1644942246 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc06f800 con 0x7fc9cc0ccd20 2024-04-05T22:31:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.761+0000 7fc9c3e00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009870 tx=0x7fc9b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f075d2d7cf1d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.762+0000 7fc9ca000640 1 -- 172.21.15.73:0/1644942246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4031640 con 0x7fc9cc0ccd20 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.762+0000 7fc9ca000640 1 -- 172.21.15.73:0/1644942246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b40317a0 con 0x7fc9cc0ccd20 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.762+0000 7fc9ca000640 1 -- 172.21.15.73:0/1644942246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4031b30 con 0x7fc9cc0ccd20 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.762+0000 7fc9d1e00640 1 -- 172.21.15.73:0/1644942246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc0ccd20 msgr2=0x7fc9cc06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.762+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009870 tx=0x7fc9b402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 -- 172.21.15.73:0/1644942246 shutdown_connections 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc0ccd20 0x7fc9cc06cb80 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/1644942246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc1587d0 0x7fc9cc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 -- 172.21.15.73:0/1644942246 >> 172.21.15.73:0/1644942246 conn(0x7fc9cc0ce8e0 msgr2=0x7fc9cc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 -- 172.21.15.73:0/1644942246 shutdown_connections 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.763+0000 7fc9d1e00640 1 -- 172.21.15.73:0/1644942246 wait complete. 2024-04-05T22:31:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.764+0000 7fc9d1e00640 1 Processor -- start 2024-04-05T22:31:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.765+0000 7fc9d1e00640 1 -- start start 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.765+0000 7fc9d1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc06d0c0 0x7fc9cc140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9d1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc1587d0 0x7fc9cc140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9d1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9d1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15cca0 con 0x7fc9cc13f520 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9d1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15c390 con 0x7fc9cc1587d0 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9d1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15a630 con 0x7fc9cc06d0c0 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43146/0 (socket says 172.21.15.73:43146) 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9cb400640 1 -- 172.21.15.73:0/2105830439 learned_addr learned my addr 172.21.15.73:0/2105830439 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9caa00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc06d0c0 0x7fc9cc140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.766+0000 7fc9c3e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc1587d0 0x7fc9cc140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 -- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc06d0c0 msgr2=0x7fc9cc140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc06d0c0 0x7fc9cc140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 -- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc1587d0 msgr2=0x7fc9cc140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc1587d0 0x7fc9cc140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 -- 172.21.15.73:0/2105830439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b40094d0 con 0x7fc9cc13f520 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9cb400640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c780 tx=0x7fc9c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004b60 con 0x7fc9cc13f520 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c400ce90 con 0x7fc9cc13f520 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc15c650 con 0x7fc9cc13f520 2024-04-05T22:31:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400a850 con 0x7fc9cc13f520 2024-04-05T22:31:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.767+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc1f2b10 con 0x7fc9cc13f520 2024-04-05T22:31:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.768+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc066af0 con 0x7fc9cc13f520 2024-04-05T22:31:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.769+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7fc9c400a9b0 con 0x7fc9cc13f520 2024-04-05T22:31:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.770+0000 7fc9c3400640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 0x7fc9a806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.770+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c40880c0 con 0x7fc9cc13f520 2024-04-05T22:31:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.771+0000 7fc9caa00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 0x7fc9a806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:52.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.772+0000 7fc9caa00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 0x7fc9a806da50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006fd0 tx=0x7fc9bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:52.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.775+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fc9c4054a60 con 0x7fc9cc13f520 2024-04-05T22:31:52.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.990+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9cc0700a0 con 0x7fc9a806b590 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.994+0000 7fc9c3400640 1 -- 172.21.15.73:0/2105830439 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc9bc00f860 con 0x7fc9a806b590 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-05T22:31:52.993 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 msgr2=0x7fc9a806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 0x7fc9a806da50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006fd0 tx=0x7fc9bc008040 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 msgr2=0x7fc9cc15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c780 tx=0x7fc9c400cc40 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 shutdown_connections 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fc9a806b590 0x7fc9a806da50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9cc13f520 0x7fc9cc15d130 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc1587d0 0x7fc9cc140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 --2- 172.21.15.73:0/2105830439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc06d0c0 0x7fc9cc140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 >> 172.21.15.73:0/2105830439 conn(0x7fc9cc0ce8e0 msgr2=0x7fc9cc0cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 shutdown_connections 2024-04-05T22:31:52.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:52.996+0000 7fc9d1e00640 1 -- 172.21.15.73:0/2105830439 wait complete. 2024-04-05T22:31:53.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-05T22:31:53.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.220+0000 7f24fd600640 1 Processor -- start 2024-04-05T22:31:53.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.220+0000 7f24fd600640 1 -- start start 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24fd600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f24f815abe0 0x7f24f80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24fd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806bc40 0x7f24f8164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24fd600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f815afc0 con 0x7f24f806c9a0 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24fd600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f806f5b0 con 0x7f24f815abe0 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24fd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f806f890 con 0x7f24f806bc40 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f24f815abe0 0x7f24f80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f24f815abe0 0x7f24f80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43158/0 (socket says 172.21.15.73:43158) 2024-04-05T22:31:53.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 -- 172.21.15.73:0/915796398 learned_addr learned my addr 172.21.15.73:0/915796398 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:53.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f7400640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806bc40 0x7f24f8164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 -- 172.21.15.73:0/915796398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806bc40 msgr2=0x7f24f8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:53.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806bc40 0x7f24f8164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 -- 172.21.15.73:0/915796398 >> v1:172.21.15.157:6789/0 conn(0x7f24f806c9a0 legacy=0x7f24f806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:53.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.222+0000 7f24f6a00640 1 -- 172.21.15.73:0/915796398 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f806fb70 con 0x7f24f815abe0 2024-04-05T22:31:53.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24f6a00640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f815abe0 0x7f24f80712e0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff781f3c50bbe90 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24f5600640 1 -- 172.21.15.73:0/915796398 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8031740 con 0x7f24f815abe0 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24f5600640 1 -- 172.21.15.73:0/915796398 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e80318a0 con 0x7f24f815abe0 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24f5600640 1 -- 172.21.15.73:0/915796398 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8031c30 con 0x7f24f815abe0 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24fd600640 1 -- 172.21.15.73:0/915796398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f815abe0 msgr2=0x7f24f80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.224+0000 7f24fd600640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f815abe0 0x7f24f80712e0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e802b7e0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 -- 172.21.15.73:0/915796398 shutdown_connections 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806bc40 0x7f24f8164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 --2- 172.21.15.73:0/915796398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f815abe0 0x7f24f80712e0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 -- 172.21.15.73:0/915796398 >> 172.21.15.73:0/915796398 conn(0x7f24f8069cb0 msgr2=0x7f24f806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 -- 172.21.15.73:0/915796398 shutdown_connections 2024-04-05T22:31:53.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.225+0000 7f24fd600640 1 -- 172.21.15.73:0/915796398 wait complete. 2024-04-05T22:31:53.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.226+0000 7f24fd600640 1 Processor -- start 2024-04-05T22:31:53.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.226+0000 7f24fd600640 1 -- start start 2024-04-05T22:31:53.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f806c9a0 0x7f24f8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f815abe0 0x7f24f8147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f815a400 con 0x7f24f806bc40 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f81585b0 con 0x7f24f806c9a0 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24fd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f8158890 con 0x7f24f815abe0 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.227+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43166/0 (socket says 172.21.15.73:43166) 2024-04-05T22:31:53.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 -- 172.21.15.73:0/2381612882 learned_addr learned my addr 172.21.15.73:0/2381612882 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f7400640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f815abe0 0x7f24f8147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 -- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f815abe0 msgr2=0x7f24f8147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f815abe0 0x7f24f8147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6000640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f806c9a0 0x7f24f8159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 -- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f806c9a0 msgr2=0x7f24f8159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:53.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f806c9a0 0x7f24f8159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 -- 172.21.15.73:0/2381612882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e80094d0 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.228+0000 7f24f6a00640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.229+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8033c70 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.229+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8033dd0 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.229+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f8158b70 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.229+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e803c640 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.229+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f81f2650 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.230+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f8068160 con 0x7f24f806bc40 2024-04-05T22:31:53.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.231+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f24e803a040 con 0x7f24f806bc40 2024-04-05T22:31:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.231+0000 7f24eea00640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 0x7f24d806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:53.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.232+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24e80a9a20 con 0x7f24f806bc40 2024-04-05T22:31:53.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.232+0000 7f24f6000640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 0x7f24d806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:53.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.234+0000 7f24f6000640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 0x7f24d806da50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f24f814e5f0 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:53.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.236+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f24e8076450 con 0x7f24f806bc40 2024-04-05T22:31:53.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.429+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f24f80b1160 con 0x7f24f806bc40 2024-04-05T22:31:53.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.430+0000 7f24eea00640 1 -- 172.21.15.73:0/2381612882 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24e807ee60 con 0x7f24f806bc40 2024-04-05T22:31:53.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 msgr2=0x7f24d806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 0x7f24d806da50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f24f814e5f0 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 msgr2=0x7f24f8157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e80046c0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 shutdown_connections 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f24d806b590 0x7f24d806da50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f815abe0 0x7f24f8147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f806c9a0 0x7f24f8159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 --2- 172.21.15.73:0/2381612882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24f806bc40 0x7f24f8157eb0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 >> 172.21.15.73:0/2381612882 conn(0x7f24f8069cb0 msgr2=0x7f24f80b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 shutdown_connections 2024-04-05T22:31:53.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:53.433+0000 7f24fd600640 1 -- 172.21.15.73:0/2381612882 wait complete. 2024-04-05T22:31:53.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:53.450 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp9000/subvol11630) 2024-04-05T22:31:53.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:53.546 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-05T22:31:53.598 INFO:teuthology.orchestra.run.smithi073.stdout:cephfs_data 2024-04-05T22:31:53.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:53.598 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp9000/subvol11630) 2024-04-05T22:31:53.698 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:53.698 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp9000/subvol11630/.meta 2024-04-05T22:31:53.799 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol11630 subvol_grp9000 2024-04-05T22:31:54.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.064+0000 7f7f2c400640 1 Processor -- start 2024-04-05T22:31:54.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.064+0000 7f7f2c400640 1 -- start start 2024-04-05T22:31:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f2c400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f2406d790 0x7f7f240711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f2c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f240cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f2c400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f240b24f0 con 0x7f7f2406b420 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f2c400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240b2a00 con 0x7f7f2406d790 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f2c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240cf750 con 0x7f7f240cedb0 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.066+0000 7f7f29a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f2406d790 0x7f7f240711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f2406d790 0x7f7f240711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43170/0 (socket says 172.21.15.73:43170) 2024-04-05T22:31:54.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 -- 172.21.15.73:0/3745379693 learned_addr learned my addr 172.21.15.73:0/3745379693 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29000640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f240cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 -- 172.21.15.73:0/3745379693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 msgr2=0x7f7f240cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f240cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 -- 172.21.15.73:0/3745379693 >> v1:172.21.15.73:6789/0 conn(0x7f7f2406b420 legacy=0x7f7f2406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.067+0000 7f7f29a00640 1 -- 172.21.15.73:0/3745379693 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240b2720 con 0x7f7f2406d790 2024-04-05T22:31:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.068+0000 7f7f29a00640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f240711d0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffda38439fe2e83c server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.068+0000 7f7f23e00640 1 -- 172.21.15.73:0/3745379693 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18031640 con 0x7f7f2406d790 2024-04-05T22:31:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f23e00640 1 -- 172.21.15.73:0/3745379693 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f180317a0 con 0x7f7f2406d790 2024-04-05T22:31:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f23e00640 1 -- 172.21.15.73:0/3745379693 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18031b30 con 0x7f7f2406d790 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 -- 172.21.15.73:0/3745379693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 msgr2=0x7f7f240711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f240711d0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802b8f0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 -- 172.21.15.73:0/3745379693 shutdown_connections 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f240cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 --2- 172.21.15.73:0/3745379693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f240711d0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 -- 172.21.15.73:0/3745379693 >> 172.21.15.73:0/3745379693 conn(0x7f7f2406a0d0 msgr2=0x7f7f2406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.069+0000 7f7f2c400640 1 -- 172.21.15.73:0/3745379693 shutdown_connections 2024-04-05T22:31:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.070+0000 7f7f2c400640 1 -- 172.21.15.73:0/3745379693 wait complete. 2024-04-05T22:31:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.071+0000 7f7f2c400640 1 Processor -- start 2024-04-05T22:31:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.071+0000 7f7f2c400640 1 -- start start 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 0x7f7f241566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f24155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24146170 con 0x7f7f2406d790 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24144ce0 con 0x7f7f2406b420 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.072+0000 7f7f2c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2414b870 con 0x7f7f240cedb0 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43180/0 (socket says 172.21.15.73:43180) 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 -- 172.21.15.73:0/2921310249 learned_addr learned my addr 172.21.15.73:0/2921310249 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 -- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 msgr2=0x7f7f24155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f2a400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f24155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29a00640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 0x7f7f241566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f24155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 -- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 msgr2=0x7f7f241566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 0x7f7f241566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29000640 1 -- 172.21.15.73:0/2921310249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180094d0 con 0x7f7f2406d790 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f29a00640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 0x7f7f241566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.073+0000 7f7f2a400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f24155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f29000640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c7f0 tx=0x7f7f1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10004220 con 0x7f7f2406d790 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100043a0 con 0x7f7f2406d790 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10007860 con 0x7f7f2406d790 2024-04-05T22:31:54.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241463f0 con 0x7f7f2406d790 2024-04-05T22:31:54.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.074+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2413fb20 con 0x7f7f2406d790 2024-04-05T22:31:54.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.075+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f24068160 con 0x7f7f2406d790 2024-04-05T22:31:54.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.076+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f7f10007bd0 con 0x7f7f2406d790 2024-04-05T22:31:54.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.078+0000 7f7f21600640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 0x7f7f0406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.078+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f10087e60 con 0x7f7f2406d790 2024-04-05T22:31:54.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.078+0000 7f7f29a00640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 0x7f7f0406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.080+0000 7f7f29a00640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 0x7f7f0406da80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f18036000 tx=0x7f7f1803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.082+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7f10054890 con 0x7f7f2406d790 2024-04-05T22:31:54.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.305+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol11630", "group_name": "subvol_grp9000", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f24070cb0 con 0x7f7f0406b5c0 2024-04-05T22:31:54.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:54.319+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:54.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:54.319+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:54.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:54.319+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:54.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:54.319+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:54.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:54.319+0000 7f1d85400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:54.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.407+0000 7f7f21600640 1 -- 172.21.15.73:0/2921310249 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f7f18031ae0 con 0x7f7f0406b5c0 2024-04-05T22:31:54.407 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp9000/subvol11630 2024-04-05T22:31:54.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 msgr2=0x7f7f0406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 0x7f7f0406da80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f18036000 tx=0x7f7f1803c040 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 msgr2=0x7f7f241447a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c7f0 tx=0x7f7f1000ccb0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 shutdown_connections 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7f0406b5c0 0x7f7f0406da80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f240cedb0 0x7f7f24155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2406d790 0x7f7f241447a0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 --2- 172.21.15.73:0/2921310249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f2406b420 0x7f7f241566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 >> 172.21.15.73:0/2921310249 conn(0x7f7f2406a0d0 msgr2=0x7f7f2406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.410+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 shutdown_connections 2024-04-05T22:31:54.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.411+0000 7f7f2c400640 1 -- 172.21.15.73:0/2921310249 wait complete. 2024-04-05T22:31:54.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol11630 subvol_grp9000 2024-04-05T22:31:54.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.642+0000 7f0f5fe00640 1 Processor -- start 2024-04-05T22:31:54.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.642+0000 7f0f5fe00640 1 -- start start 2024-04-05T22:31:54.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f5806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f580b36e0 0x7f0f580cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5815e570 con 0x7f0f580cf210 2024-04-05T22:31:54.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580cf6a0 con 0x7f0f580b36e0 2024-04-05T22:31:54.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815afe0 con 0x7f0f5815ac00 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f580b36e0 0x7f0f580cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5de00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0f580cf210 0x7f0f580ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49160/0 (socket says 172.21.15.73:49160) 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.644+0000 7f0f5de00640 1 -- 172.21.15.73:0/620934207 learned_addr learned my addr 172.21.15.73:0/620934207 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f5d400640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f5806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f5ca00640 1 -- 172.21.15.73:0/620934207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 msgr2=0x7f0f5806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f5ca00640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f5806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f5ca00640 1 -- 172.21.15.73:0/620934207 >> v1:172.21.15.73:6789/0 conn(0x7f0f580cf210 legacy=0x7f0f580ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f5ca00640 1 -- 172.21.15.73:0/620934207 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58161760 con 0x7f0f580b36e0 2024-04-05T22:31:54.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.645+0000 7f0f57e00640 1 -- 172.21.15.73:0/620934207 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138542390 0 0) 0x7f0f5815e570 con 0x7f0f580cf210 2024-04-05T22:31:54.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.646+0000 7f0f5ca00640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cecd0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f0f440097a0 tx=0x7f0f4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b7412bd96afcd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.646+0000 7f0f57e00640 1 -- 172.21.15.73:0/620934207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4402e070 con 0x7f0f580b36e0 2024-04-05T22:31:54.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.646+0000 7f0f57e00640 1 -- 172.21.15.73:0/620934207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f440315a0 con 0x7f0f580b36e0 2024-04-05T22:31:54.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.646+0000 7f0f57e00640 1 -- 172.21.15.73:0/620934207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44031930 con 0x7f0f580b36e0 2024-04-05T22:31:54.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 -- 172.21.15.73:0/620934207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 msgr2=0x7f0f580cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cecd0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f0f440097a0 tx=0x7f0f4402b840 comp rx=0 tx=0).stop 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 -- 172.21.15.73:0/620934207 shutdown_connections 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cecd0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/620934207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f5806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 -- 172.21.15.73:0/620934207 >> 172.21.15.73:0/620934207 conn(0x7f0f5806a0d0 msgr2=0x7f0f5806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 -- 172.21.15.73:0/620934207 shutdown_connections 2024-04-05T22:31:54.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.647+0000 7f0f5fe00640 1 -- 172.21.15.73:0/620934207 wait complete. 2024-04-05T22:31:54.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.649+0000 7f0f5fe00640 1 Processor -- start 2024-04-05T22:31:54.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.649+0000 7f0f5fe00640 1 -- start start 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f580cf210 0x7f0f580cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f581f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58157ca0 con 0x7f0f580b36e0 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580cdc10 con 0x7f0f580cf210 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58157650 con 0x7f0f5815ac00 2024-04-05T22:31:54.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43194/0 (socket says 172.21.15.73:43194) 2024-04-05T22:31:54.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.650+0000 7f0f5d400640 1 -- 172.21.15.73:0/2535090681 learned_addr learned my addr 172.21.15.73:0/2535090681 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 -- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 msgr2=0x7f0f581f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f581f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 -- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f580cf210 msgr2=0x7f0f580cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f580cf210 0x7f0f580cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 -- 172.21.15.73:0/2535090681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44009400 con 0x7f0f580b36e0 2024-04-05T22:31:54.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5d400640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d5f0 tx=0x7f0f4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0125f0 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c012750 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c012b00 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f58157930 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.651+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58160050 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.653+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58068160 con 0x7f0f580b36e0 2024-04-05T22:31:54.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.653+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 90012+0+0 (secure 0 0 0) 0x7f0f4c012c60 con 0x7f0f580b36e0 2024-04-05T22:31:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.654+0000 7f0f55600640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 0x7f0f3806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:54.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.654+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f4c088350 con 0x7f0f580b36e0 2024-04-05T22:31:54.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.655+0000 7f0f5ca00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 0x7f0f3806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:54.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.656+0000 7f0f5ca00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 0x7f0f3806da50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f44036000 tx=0x7f0f44030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:54.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.659+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0f4c054d80 con 0x7f0f580b36e0 2024-04-05T22:31:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.884+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol11630", "group_name": "subvol_grp9000", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f58160e30 con 0x7f0f3806b590 2024-04-05T22:31:54.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.896+0000 7f0f55600640 1 -- 172.21.15.73:0/2535090681 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f0f44004040 con 0x7f0f3806b590 2024-04-05T22:31:54.896 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/subvol_grp9000/subvol11630 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.898+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 msgr2=0x7f0f3806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.898+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 0x7f0f3806da50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f44036000 tx=0x7f0f44030000 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.898+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 msgr2=0x7f0f580cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d5f0 tx=0x7f0f4c00dab0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 shutdown_connections 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0f3806b590 0x7f0f3806da50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5815ac00 0x7f0f581f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f580cf210 0x7f0f580cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 --2- 172.21.15.73:0/2535090681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f580b36e0 0x7f0f580cceb0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:54.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 >> 172.21.15.73:0/2535090681 conn(0x7f0f5806a0d0 msgr2=0x7f0f58155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:54.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 shutdown_connections 2024-04-05T22:31:54.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:54.899+0000 7f0f5fe00640 1 -- 172.21.15.73:0/2535090681 wait complete. 2024-04-05T22:31:54.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:54.919 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/8842af564cd39ace73c2e7f0bd37a924.meta) 2024-04-05T22:31:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:[GLOBAL] 2024-04-05T22:31:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:version = 1 2024-04-05T22:31:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:type = subvolume 2024-04-05T22:31:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:path = /volumes/subvol_grp9000/subvol11630 2024-04-05T22:31:57.403 INFO:teuthology.orchestra.run.smithi073.stdout:state = complete 2024-04-05T22:31:57.404 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:57.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol11630 subvol_grp9000 2024-04-05T22:31:57.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.629+0000 7fbd5a200640 1 Processor -- start 2024-04-05T22:31:57.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.629+0000 7fbd5a200640 1 -- start start 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd5a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbd541541a0 0x7fbd540cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd5a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd540cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd5a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5415d900 con 0x7fbd540b2840 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd5a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5415e1a0 con 0x7fbd541541a0 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd5a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540b3490 con 0x7fbd54154580 2024-04-05T22:31:57.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd53e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbd540b2840 0x7fbd540b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45796/0 (socket says 172.21.15.73:45796) 2024-04-05T22:31:57.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.631+0000 7fbd53e00640 1 -- 172.21.15.73:0/2625266424 learned_addr learned my addr 172.21.15.73:0/2625266424 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd53400640 1 --2- 172.21.15.73:0/2625266424 >> v2:172.21.15.157:3300/0 conn(0x7fbd541541a0 0x7fbd540cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd4aa00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd540cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930244116 0 0) 0x7fbd5415d900 con 0x7fbd540b2840 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd38005730 con 0x7fbd540b2840 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540508973 0 0) 0x7fbd38005730 con 0x7fbd540b2840 2024-04-05T22:31:57.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 msgr2=0x7fbd540cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd540cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 >> v2:172.21.15.157:3300/0 conn(0x7fbd541541a0 msgr2=0x7fbd540cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 --2- 172.21.15.73:0/2625266424 >> v2:172.21.15.157:3300/0 conn(0x7fbd541541a0 0x7fbd540cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5415db50 con 0x7fbd540b2840 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbd4c0019f0 con 0x7fbd540b2840 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 >> v1:172.21.15.73:6789/0 conn(0x7fbd540b2840 legacy=0x7fbd540b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd38005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd38006100 0x7fbd380069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd38006f10 0x7fbd38007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5415e1a0 con 0x7fbd38006f10 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540b3490 con 0x7fbd38006100 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.632+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3800db90 con 0x7fbd54154580 2024-04-05T22:31:57.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd38006f10 0x7fbd38007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd4aa00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd38006100 0x7fbd380069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53400640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd38005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 -- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 msgr2=0x7fbd38005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd54154580 0x7fbd38005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 -- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd38006100 msgr2=0x7fbd380069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd38006100 0x7fbd380069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.633+0000 7fbd53e00640 1 -- 172.21.15.73:0/2625266424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd38005730 con 0x7fbd38006f10 2024-04-05T22:31:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.634+0000 7fbd53e00640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd38006f10 0x7fbd38007390 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00a830 tx=0x7fbd38007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8638188b8105d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:57.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.634+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c034630 con 0x7fbd38006f10 2024-04-05T22:31:57.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.634+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c02f070 con 0x7fbd38006f10 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.634+0000 7fbd52a00640 1 -- 172.21.15.73:0/2625266424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c034a30 con 0x7fbd38006f10 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 -- 172.21.15.73:0/2625266424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd38006f10 msgr2=0x7fbd38007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 --2- 172.21.15.73:0/2625266424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd38006f10 0x7fbd38007390 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00a830 tx=0x7fbd38007a00 comp rx=0 tx=0).stop 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd53e00640 1 -- 172.21.15.73:0/2625266424 reap_dead start 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 -- 172.21.15.73:0/2625266424 shutdown_connections 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 -- 172.21.15.73:0/2625266424 >> 172.21.15.73:0/2625266424 conn(0x7fbd5406a0d0 msgr2=0x7fbd5406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 -- 172.21.15.73:0/2625266424 shutdown_connections 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.635+0000 7fbd5a200640 1 -- 172.21.15.73:0/2625266424 wait complete. 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.637+0000 7fbd5a200640 1 Processor -- start 2024-04-05T22:31:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.637+0000 7fbd5a200640 1 -- start start 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd38006f10 0x7fbd54141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd541402f0 0x7fbd540ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5413efa0 con 0x7fbd5413eb20 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5413f250 con 0x7fbd541402f0 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd5a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5413f530 con 0x7fbd38006f10 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd4aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd4aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33434/0 (socket says 172.21.15.73:33434) 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.638+0000 7fbd4aa00640 1 -- 172.21.15.73:0/609385381 learned_addr learned my addr 172.21.15.73:0/609385381 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd53400640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd38006f10 0x7fbd54141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd53e00640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd541402f0 0x7fbd540ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 -- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd38006f10 msgr2=0x7fbd54141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd38006f10 0x7fbd54141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 -- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd541402f0 msgr2=0x7fbd540ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd541402f0 0x7fbd540ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 -- 172.21.15.73:0/609385381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5413f810 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.639+0000 7fbd4aa00640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c011590 tx=0x7fbd3c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.640+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004220 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.640+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c004380 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.640+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c015930 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.640+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5413faf0 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.640+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540cacd0 con 0x7fbd5413eb20 2024-04-05T22:31:57.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.641+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54068160 con 0x7fbd5413eb20 2024-04-05T22:31:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.642+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 90063+0+0 (secure 0 0 0) 0x7fbd3c015a90 con 0x7fbd5413eb20 2024-04-05T22:31:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.643+0000 7fbd4be00640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 0x7fbd3006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.643+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd3c08a6a0 con 0x7fbd5413eb20 2024-04-05T22:31:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.643+0000 7fbd53400640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 0x7fbd3006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.645+0000 7fbd53400640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 0x7fbd3006dae0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009a90 tx=0x7fbd44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:57.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.648+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbd3c054a50 con 0x7fbd5413eb20 2024-04-05T22:31:57.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.878+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol11630", "group_name": "subvol_grp9000", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd540cf210 con 0x7fbd3006b620 2024-04-05T22:31:57.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.911+0000 7fbd4be00640 1 -- 172.21.15.73:0/609385381 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbd440096e0 con 0x7fbd3006b620 2024-04-05T22:31:57.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 msgr2=0x7fbd3006dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 0x7fbd3006dae0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009a90 tx=0x7fbd44008040 comp rx=0 tx=0).stop 2024-04-05T22:31:57.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 msgr2=0x7fbd540ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:57.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c011590 tx=0x7fbd3c011a50 comp rx=0 tx=0).stop 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 shutdown_connections 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbd3006b620 0x7fbd3006dae0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd541402f0 0x7fbd540ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd5413eb20 0x7fbd540ca790 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 --2- 172.21.15.73:0/609385381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd38006f10 0x7fbd54141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 >> 172.21.15.73:0/609385381 conn(0x7fbd5406a0d0 msgr2=0x7fbd54158910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 shutdown_connections 2024-04-05T22:31:57.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:57.914+0000 7fbd5a200640 1 -- 172.21.15.73:0/609385381 wait complete. 2024-04-05T22:31:57.926 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.926+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.926 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.926+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.927 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.926+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.927 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.926+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.927 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.926+0000 7f1d87c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:31:57.939 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-05T22:31:57.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.955+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.955+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.955+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.955+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-05T22:31:57.955+0000 7f1d87200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-05T22:31:57.986 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-04-05T22:31:57.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp9000 2024-04-05T22:31:58.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.180+0000 7f927f400640 1 Processor -- start 2024-04-05T22:31:58.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.180+0000 7f927f400640 1 -- start start 2024-04-05T22:31:58.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.182+0000 7f927f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f927815abe0 0x7f92780712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.182+0000 7f927f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.182+0000 7f927f400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927806bec0 con 0x7f927806ba70 2024-04-05T22:31:58.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.182+0000 7f927f400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927815afc0 con 0x7f927815abe0 2024-04-05T22:31:58.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.182+0000 7f927f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927806f5b0 con 0x7f927806c9a0 2024-04-05T22:31:58.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.183+0000 7f9277e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.183+0000 7f9277e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:41982/0 (socket says 172.21.15.73:41982) 2024-04-05T22:31:58.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.183+0000 7f9277e00640 1 -- 172.21.15.73:0/3504094380 learned_addr learned my addr 172.21.15.73:0/3504094380 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:58.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.183+0000 7f927ca00640 1 --2- 172.21.15.73:0/3504094380 >> v2:172.21.15.157:3300/0 conn(0x7f927815abe0 0x7f92780712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f9277400640 1 -- 172.21.15.73:0/3504094380 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890622348 0 0) 0x7f9270001110 con 0x7f927806ba70 2024-04-05T22:31:58.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f9277400640 1 -- 172.21.15.73:0/3504094380 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c003520 con 0x7f927806ba70 2024-04-05T22:31:58.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f927ca00640 1 -- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 msgr2=0x7f927806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:58.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f927ca00640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f927ca00640 1 -- 172.21.15.73:0/3504094380 >> v1:172.21.15.157:6789/0 conn(0x7f927806ba70 legacy=0x7f92780cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f927ca00640 1 -- 172.21.15.73:0/3504094380 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927806f890 con 0x7f927815abe0 2024-04-05T22:31:58.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.184+0000 7f9277e00640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:31:58.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.185+0000 7f927ca00640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927815abe0 0x7f92780712e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9268009d90 tx=0x7f926802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7558fcf9f52ecee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:58.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f9277400640 1 -- 172.21.15.73:0/3504094380 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926802cb30 con 0x7f927815abe0 2024-04-05T22:31:58.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f9277400640 1 -- 172.21.15.73:0/3504094380 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926802cc90 con 0x7f927815abe0 2024-04-05T22:31:58.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f9277400640 1 -- 172.21.15.73:0/3504094380 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268034440 con 0x7f927815abe0 2024-04-05T22:31:58.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 -- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927815abe0 msgr2=0x7f92780712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927815abe0 0x7f92780712e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9268009d90 tx=0x7f926802bd80 comp rx=0 tx=0).stop 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 -- 172.21.15.73:0/3504094380 shutdown_connections 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 --2- 172.21.15.73:0/3504094380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927815abe0 0x7f92780712e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.186+0000 7f927f400640 1 -- 172.21.15.73:0/3504094380 >> 172.21.15.73:0/3504094380 conn(0x7f9278069cb0 msgr2=0x7f927806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.187+0000 7f927f400640 1 -- 172.21.15.73:0/3504094380 shutdown_connections 2024-04-05T22:31:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.187+0000 7f927f400640 1 -- 172.21.15.73:0/3504094380 wait complete. 2024-04-05T22:31:58.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.188+0000 7f927f400640 1 Processor -- start 2024-04-05T22:31:58.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.188+0000 7f927f400640 1 -- start start 2024-04-05T22:31:58.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 0x7f927813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927814dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927814e630 con 0x7f927815abe0 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927814e910 con 0x7f927806ba70 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.189+0000 7f927f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927814ebf0 con 0x7f927806c9a0 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33450/0 (socket says 172.21.15.73:33450) 2024-04-05T22:31:58.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 -- 172.21.15.73:0/3104513986 learned_addr learned my addr 172.21.15.73:0/3104513986 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:58.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927ca00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 0x7f927813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f9277e00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927814dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 -- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 msgr2=0x7f927814dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927814dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 -- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 msgr2=0x7f927813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 0x7f927813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 -- 172.21.15.73:0/3104513986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600094d0 con 0x7f927815abe0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927ca00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 0x7f927813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.190+0000 7f927d400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9270007fd0 tx=0x7f927000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.191+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700052d0 con 0x7f927815abe0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.191+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270005450 con 0x7f927815abe0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.191+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270003830 con 0x7f927815abe0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.191+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268009a70 con 0x7f927815abe0 2024-04-05T22:31:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.191+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92780b1d30 con 0x7f927815abe0 2024-04-05T22:31:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.192+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9278068160 con 0x7f927815abe0 2024-04-05T22:31:58.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.193+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 90063+0+0 (secure 0 0 0) 0x7f9270003a50 con 0x7f927815abe0 2024-04-05T22:31:58.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.194+0000 7f9274c00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 0x7f925406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:58.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.194+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9270087310 con 0x7f927815abe0 2024-04-05T22:31:58.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.195+0000 7f927ca00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 0x7f925406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:58.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.196+0000 7f927ca00640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 0x7f925406dae0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f9268030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:58.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.198+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9270053ca0 con 0x7f927815abe0 2024-04-05T22:31:58.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.422+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp9000", "target": ["mon-mgr", ""]}) v1 -- 0x7f9278070130 con 0x7f925406b620 2024-04-05T22:31:58.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.431+0000 7f9274c00640 1 -- 172.21.15.73:0/3104513986 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f92680440c0 con 0x7f925406b620 2024-04-05T22:31:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 msgr2=0x7f925406dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 0x7f925406dae0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f9268030000 comp rx=0 tx=0).stop 2024-04-05T22:31:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 msgr2=0x7f927814e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:58.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9270007fd0 tx=0x7f927000c820 comp rx=0 tx=0).stop 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 shutdown_connections 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f925406b620 0x7f925406dae0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f927815abe0 0x7f927814e020 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f927806c9a0 0x7f927814dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 --2- 172.21.15.73:0/3104513986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f927806ba70 0x7f927813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 >> 172.21.15.73:0/3104513986 conn(0x7f9278069cb0 msgr2=0x7f92780ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 shutdown_connections 2024-04-05T22:31:58.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:58.434+0000 7f927f400640 1 -- 172.21.15.73:0/3104513986 wait complete. 2024-04-05T22:31:58.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-05T22:31:58.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:58.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-05T22:31:58.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-05T22:31:58.682 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:58.682 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:58.740 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:31:58.740 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:31:58.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:31:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.002+0000 7f2dca800640 1 Processor -- start 2024-04-05T22:31:59.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.002+0000 7f2dca800640 1 -- start start 2024-04-05T22:31:59.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.003+0000 7f2dca800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2dc40b2ba0 0x7f2dc40b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.003+0000 7f2dca800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc41627f0 con 0x7f2dc4162370 2024-04-05T22:31:59.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.003+0000 7f2dca800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc40b3510 con 0x7f2dc4161f90 2024-04-05T22:31:59.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.003+0000 7f2dca800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc4158790 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc2a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2dc4162370 0x7f2dc40b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45808/0 (socket says 172.21.15.73:45808) 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc2a00640 1 -- 172.21.15.73:0/2932897686 learned_addr learned my addr 172.21.15.73:0/2932897686 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc3e00640 1 --2- 172.21.15.73:0/2932897686 >> v2:172.21.15.73:3300/0 conn(0x7f2dc40b2ba0 0x7f2dc40b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc2000640 1 -- 172.21.15.73:0/2932897686 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622515951 0 0) 0x7f2dac001110 con 0x7f2dc4162370 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc2000640 1 -- 172.21.15.73:0/2932897686 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da8003520 con 0x7f2dc4162370 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc3e00640 1 -- 172.21.15.73:0/2932897686 >> v1:172.21.15.157:6789/0 conn(0x7f2dc4161f90 legacy=0x7f2dc4164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc3e00640 1 -- 172.21.15.73:0/2932897686 >> v1:172.21.15.73:6789/0 conn(0x7f2dc4162370 legacy=0x7f2dc40b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.004+0000 7f2dc3e00640 1 -- 172.21.15.73:0/2932897686 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4165cf0 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.005+0000 7f2dc3e00640 1 --2- 172.21.15.73:0/2932897686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc40b2ba0 0x7f2dc40b2fd0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009d90 tx=0x7f2dbc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791cb56cb3253eb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dc2000640 1 -- 172.21.15.73:0/2932897686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc02cc60 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dc2000640 1 -- 172.21.15.73:0/2932897686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc02cdc0 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dc2000640 1 -- 172.21.15.73:0/2932897686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc034640 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 -- 172.21.15.73:0/2932897686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc40b2ba0 msgr2=0x7f2dc40b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 --2- 172.21.15.73:0/2932897686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc40b2ba0 0x7f2dc40b2fd0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009d90 tx=0x7f2dbc02beb0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 -- 172.21.15.73:0/2932897686 shutdown_connections 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 --2- 172.21.15.73:0/2932897686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc40b2ba0 0x7f2dc40b2fd0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 -- 172.21.15.73:0/2932897686 >> 172.21.15.73:0/2932897686 conn(0x7f2dc40ce8e0 msgr2=0x7f2dc40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 -- 172.21.15.73:0/2932897686 shutdown_connections 2024-04-05T22:31:59.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.006+0000 7f2dca800640 1 -- 172.21.15.73:0/2932897686 wait complete. 2024-04-05T22:31:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.008+0000 7f2dca800640 1 Processor -- start 2024-04-05T22:31:59.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.008+0000 7f2dca800640 1 -- start start 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc40b2ba0 0x7f2dc4147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc4161f90 0x7f2dc4144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41611f0 con 0x7f2dc4162370 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4145190 con 0x7f2dc4161f90 2024-04-05T22:31:59.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dca800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4145470 con 0x7f2dc40b2ba0 2024-04-05T22:31:59.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dc3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dc3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33456/0 (socket says 172.21.15.73:33456) 2024-04-05T22:31:59.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.009+0000 7f2dc3e00640 1 -- 172.21.15.73:0/1419524403 learned_addr learned my addr 172.21.15.73:0/1419524403 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3400640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc40b2ba0 0x7f2dc4147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc2a00640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc4161f90 0x7f2dc4144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 -- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc40b2ba0 msgr2=0x7f2dc4147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc40b2ba0 0x7f2dc4147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 -- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc4161f90 msgr2=0x7f2dc4144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc4161f90 0x7f2dc4144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 -- 172.21.15.73:0/1419524403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc009a70 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dc3e00640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02bef0 tx=0x7f2dbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.010+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc004250 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.011+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0043b0 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.011+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0047e0 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.011+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4145750 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.011+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc4144c50 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.012+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc4066af0 con 0x7f2dc4162370 2024-04-05T22:31:59.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.013+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 90063+0+0 (secure 0 0 0) 0x7f2dbc03b070 con 0x7f2dc4162370 2024-04-05T22:31:59.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.014+0000 7f2dbb400640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 0x7f2da006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.014+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dbc0aa650 con 0x7f2dc4162370 2024-04-05T22:31:59.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.014+0000 7f2dc3400640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 0x7f2da006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.015+0000 7f2dc3400640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 0x7f2da006dae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2db4004ea0 tx=0x7f2db4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.018+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f2dbc077180 con 0x7f2dc4162370 2024-04-05T22:31:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.210+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dc41452d0 con 0x7f2dc4162370 2024-04-05T22:31:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.213+0000 7f2dbb400640 1 -- 172.21.15.73:0/1419524403 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7f2dc41452d0 con 0x7f2dc4162370 2024-04-05T22:31:59.214 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:31:59.214 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:31:59.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 msgr2=0x7f2da006dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 0x7f2da006dae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2db4004ea0 tx=0x7f2db4009040 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 msgr2=0x7f2dc41608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02bef0 tx=0x7f2dbc004a40 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 shutdown_connections 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f2da006b620 0x7f2da006dae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc4162370 0x7f2dc41608a0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc4161f90 0x7f2dc4144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 --2- 172.21.15.73:0/1419524403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc40b2ba0 0x7f2dc4147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 >> 172.21.15.73:0/1419524403 conn(0x7f2dc40ce8e0 msgr2=0x7f2dc406a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 shutdown_connections 2024-04-05T22:31:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.216+0000 7f2dca800640 1 -- 172.21.15.73:0/1419524403 wait complete. 2024-04-05T22:31:59.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:31:40.727092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:31:41.315098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:31:59.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:31:59.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.449+0000 7efcef200640 1 Processor -- start 2024-04-05T22:31:59.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.450+0000 7efcef200640 1 -- start start 2024-04-05T22:31:59.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.451+0000 7efcef200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efce80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.451+0000 7efcef200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce815afc0 con 0x7efce806c9a0 2024-04-05T22:31:59.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.451+0000 7efcef200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce806f5b0 con 0x7efce806bc40 2024-04-05T22:31:59.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.451+0000 7efcef200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce806f890 con 0x7efce815abe0 2024-04-05T22:31:59.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.452+0000 7efce7400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efce806c9a0 0x7efce806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45820/0 (socket says 172.21.15.73:45820) 2024-04-05T22:31:59.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.452+0000 7efce7400640 1 -- 172.21.15.73:0/247778479 learned_addr learned my addr 172.21.15.73:0/247778479 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.452+0000 7efce7e00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efce80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.452+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891881495 0 0) 0x7efce815afc0 con 0x7efce806c9a0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.452+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efccc003520 con 0x7efce806c9a0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692930040 0 0) 0x7efccc003520 con 0x7efce806c9a0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 msgr2=0x7efce80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efce80712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 >> v1:172.21.15.157:6789/0 conn(0x7efce806bc40 legacy=0x7efce8164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce806fb70 con 0x7efce806c9a0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7efcd0001670 con 0x7efce806c9a0 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 >> v1:172.21.15.73:6789/0 conn(0x7efce806c9a0 legacy=0x7efce806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efccc0039a0 0x7efccc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efccc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efccc004d00 0x7efccc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce806f890 con 0x7efccc004d00 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00bb40 con 0x7efccc0039a0 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00bd00 con 0x7efce815abe0 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efced200640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efccc004d00 0x7efccc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce7400640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efccc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efce7e00640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efccc0039a0 0x7efccc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.453+0000 7efced200640 1 -- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 msgr2=0x7efccc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.454+0000 7efced200640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce815abe0 0x7efccc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.454+0000 7efced200640 1 -- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efccc0039a0 msgr2=0x7efccc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.454+0000 7efced200640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efccc0039a0 0x7efccc003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.454+0000 7efced200640 1 -- 172.21.15.73:0/247778479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc003520 con 0x7efccc004d00 2024-04-05T22:31:59.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efced200640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efccc004d00 0x7efccc005180 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7efce0001750 tx=0x7efccc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2642fccb5a487e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00052d0 con 0x7efccc004d00 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce002e050 con 0x7efccc004d00 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efce6a00640 1 -- 172.21.15.73:0/247778479 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00035f0 con 0x7efccc004d00 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efcef200640 1 -- 172.21.15.73:0/247778479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efccc004d00 msgr2=0x7efccc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efcef200640 1 --2- 172.21.15.73:0/247778479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efccc004d00 0x7efccc005180 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7efce0001750 tx=0x7efccc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.455+0000 7efced200640 1 -- 172.21.15.73:0/247778479 reap_dead start 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.456+0000 7efcef200640 1 -- 172.21.15.73:0/247778479 shutdown_connections 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.456+0000 7efcef200640 1 -- 172.21.15.73:0/247778479 >> 172.21.15.73:0/247778479 conn(0x7efce8069cb0 msgr2=0x7efce806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.456+0000 7efcef200640 1 -- 172.21.15.73:0/247778479 shutdown_connections 2024-04-05T22:31:59.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.456+0000 7efcef200640 1 -- 172.21.15.73:0/247778479 wait complete. 2024-04-05T22:31:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.457+0000 7efcef200640 1 Processor -- start 2024-04-05T22:31:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 -- start start 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8146860 0x7efce814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 0x7efce80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce814b350 con 0x7efce814dc50 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce814b630 con 0x7efce8146860 2024-04-05T22:31:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.458+0000 7efcef200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce814b910 con 0x7efce8146c40 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33474/0 (socket says 172.21.15.73:33474) 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 -- 172.21.15.73:0/1936465585 learned_addr learned my addr 172.21.15.73:0/1936465585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7e00640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8146860 0x7efce814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efced200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 0x7efce80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 -- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 msgr2=0x7efce80ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 0x7efce80ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 -- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8146860 msgr2=0x7efce814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8146860 0x7efce814d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efce7400640 1 -- 172.21.15.73:0/1936465585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce814bbf0 con 0x7efce814dc50 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.459+0000 7efced200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 0x7efce80ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efce7400640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7efcd000db10 tx=0x7efcd0007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004cb0 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0004e10 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce814bed0 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00072f0 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.460+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce814c5d0 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.461+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce8068160 con 0x7efce814dc50 2024-04-05T22:31:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.462+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 90063+0+0 (secure 0 0 0) 0x7efcd0007450 con 0x7efce814dc50 2024-04-05T22:31:59.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.463+0000 7efcdfe00640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 0x7efcc406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.463+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcd0087b30 con 0x7efce814dc50 2024-04-05T22:31:59.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.464+0000 7efce7e00640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 0x7efcc406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.465+0000 7efce7e00640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 0x7efcc406da40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efce815d7d0 tx=0x7efcd800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.468+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7efcd00545a0 con 0x7efce814dc50 2024-04-05T22:31:59.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.661+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efce8155b90 con 0x7efce814dc50 2024-04-05T22:31:59.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.662+0000 7efcdfe00640 1 -- 172.21.15.73:0/1936465585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcd0019020 con 0x7efce814dc50 2024-04-05T22:31:59.662 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:31:59.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 msgr2=0x7efcc406da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 0x7efcc406da40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efce815d7d0 tx=0x7efcd800d040 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 msgr2=0x7efce80cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7efcd000db10 tx=0x7efcd0007e30 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 shutdown_connections 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7efcc406b580 0x7efcc406da40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efce8146c40 0x7efce80ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efce814dc50 0x7efce80cde40 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 --2- 172.21.15.73:0/1936465585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce8146860 0x7efce814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 >> 172.21.15.73:0/1936465585 conn(0x7efce8069cb0 msgr2=0x7efce80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.665+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 shutdown_connections 2024-04-05T22:31:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.666+0000 7efcef200640 1 -- 172.21.15.73:0/1936465585 wait complete. 2024-04-05T22:31:59.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:31:59.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.898+0000 7f6a8f200640 1 Processor -- start 2024-04-05T22:31:59.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.898+0000 7f6a8f200640 1 -- start start 2024-04-05T22:31:59.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.900+0000 7f6a8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 0x7f6a8815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.900+0000 7f6a8f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8815ae50 con 0x7f6a8815dbc0 2024-04-05T22:31:59.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.900+0000 7f6a8f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8815e3b0 con 0x7f6a8815aa70 2024-04-05T22:31:59.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.900+0000 7f6a8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880ac9e0 con 0x7f6a88071670 2024-04-05T22:31:59.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.901+0000 7f6a8d200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6a8815dbc0 0x7f6a8806ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45822/0 (socket says 172.21.15.73:45822) 2024-04-05T22:31:59.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.901+0000 7f6a8d200640 1 -- 172.21.15.73:0/1781887055 learned_addr learned my addr 172.21.15.73:0/1781887055 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:31:59.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.901+0000 7f6a7fe00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 0x7f6a8815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.901+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311099091 0 0) 0x7f6a8815ae50 con 0x7f6a8815dbc0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.901+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a6c003520 con 0x7f6a8815dbc0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771915972 0 0) 0x7f6a8815e3b0 con 0x7f6a8815aa70 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a8815ae50 con 0x7f6a8815aa70 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3192407264 0 0) 0x7f6a6c003520 con 0x7f6a8815dbc0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 msgr2=0x7f6a8815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 0x7f6a8815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 >> v1:172.21.15.157:6789/0 conn(0x7f6a8815aa70 legacy=0x7f6a88070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880accc0 con 0x7f6a8815dbc0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f6a800019f0 con 0x7f6a8815dbc0 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 >> v1:172.21.15.73:6789/0 conn(0x7f6a8815dbc0 legacy=0x7f6a8806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c003b30 0x7f6a6c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c0047c0 0x7f6a6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 0x7f6a6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880ac9e0 con 0x7f6a6c0047c0 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c00ba30 con 0x7f6a6c003b30 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c00bbf0 con 0x7f6a88071670 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c0047c0 0x7f6a6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87e00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c003b30 0x7f6a6c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 -- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 msgr2=0x7f6a6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a88071670 0x7f6a6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 -- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c003b30 msgr2=0x7f6a6c003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c003b30 0x7f6a6c003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a7fe00640 1 -- 172.21.15.73:0/1781887055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c003520 con 0x7f6a6c0047c0 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.902+0000 7f6a87e00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c003b30 0x7f6a6c003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a7fe00640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c0047c0 0x7f6a6c004c40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009a90 tx=0x7f6a6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b25163b935310f server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7002c6b0 con 0x7f6a6c0047c0 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7002c810 con 0x7f6a6c0047c0 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a87400640 1 -- 172.21.15.73:0/1781887055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7002cba0 con 0x7f6a6c0047c0 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a8f200640 1 -- 172.21.15.73:0/1781887055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c0047c0 msgr2=0x7f6a6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a8f200640 1 --2- 172.21.15.73:0/1781887055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c0047c0 0x7f6a6c004c40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009a90 tx=0x7f6a6c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.904+0000 7f6a8d200640 1 -- 172.21.15.73:0/1781887055 reap_dead start 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.905+0000 7f6a8f200640 1 -- 172.21.15.73:0/1781887055 shutdown_connections 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.905+0000 7f6a8f200640 1 -- 172.21.15.73:0/1781887055 >> 172.21.15.73:0/1781887055 conn(0x7f6a88069cb0 msgr2=0x7f6a8806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.905+0000 7f6a8f200640 1 -- 172.21.15.73:0/1781887055 shutdown_connections 2024-04-05T22:31:59.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.905+0000 7f6a8f200640 1 -- 172.21.15.73:0/1781887055 wait complete. 2024-04-05T22:31:59.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.906+0000 7f6a8f200640 1 Processor -- start 2024-04-05T22:31:59.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.906+0000 7f6a8f200640 1 -- start start 2024-04-05T22:31:59.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c0047c0 0x7f6a8814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a881f0320 0x7f6a8814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880b0e90 con 0x7f6a880b04a0 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880b1140 con 0x7f6a6c0047c0 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.907+0000 7f6a8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881f2e60 con 0x7f6a881f0320 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33494/0 (socket says 172.21.15.73:33494) 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 -- 172.21.15.73:0/3892134640 learned_addr learned my addr 172.21.15.73:0/3892134640 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 -- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a881f0320 msgr2=0x7f6a8814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a881f0320 0x7f6a8814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 -- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c0047c0 msgr2=0x7f6a8814f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c0047c0 0x7f6a8814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 -- 172.21.15.73:0/3892134640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a881f3140 con 0x7f6a880b04a0 2024-04-05T22:31:59.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.908+0000 7f6a8d200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f6a800012e0 tx=0x7f6a8000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.909+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000a610 con 0x7f6a880b04a0 2024-04-05T22:31:59.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.909+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8000a770 con 0x7f6a880b04a0 2024-04-05T22:31:59.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.909+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000ab20 con 0x7f6a880b04a0 2024-04-05T22:31:59.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.909+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a881f3420 con 0x7f6a880b04a0 2024-04-05T22:31:59.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.909+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a881f3940 con 0x7f6a880b04a0 2024-04-05T22:31:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.910+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a880cf6f0 con 0x7f6a880b04a0 2024-04-05T22:31:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.911+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f6a8000ac80 con 0x7f6a880b04a0 2024-04-05T22:31:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.912+0000 7f6a84c00640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 0x7f6a6406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:31:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.912+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a80087760 con 0x7f6a880b04a0 2024-04-05T22:31:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.912+0000 7f6a87e00640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 0x7f6a6406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:31:59.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.914+0000 7f6a87e00640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 0x7f6a6406dc30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000f60 tx=0x7f6a78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:31:59.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:31:59.916+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6a800541a0 con 0x7f6a880b04a0 2024-04-05T22:32:00.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.112+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a8806cf50 con 0x7f6a880b04a0 2024-04-05T22:32:00.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.112+0000 7f6a84c00640 1 -- 172.21.15.73:0/3892134640 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a800577c0 con 0x7f6a880b04a0 2024-04-05T22:32:00.113 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:32:00.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 msgr2=0x7f6a6406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 0x7f6a6406dc30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000f60 tx=0x7f6a78009040 comp rx=0 tx=0).stop 2024-04-05T22:32:00.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 msgr2=0x7f6a880b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f6a800012e0 tx=0x7f6a8000ccd0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 shutdown_connections 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6a6406b770 0x7f6a6406dc30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a880b04a0 0x7f6a880b0950 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a881f0320 0x7f6a8814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 --2- 172.21.15.73:0/3892134640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a6c0047c0 0x7f6a8814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 >> 172.21.15.73:0/3892134640 conn(0x7f6a88069cb0 msgr2=0x7f6a8806f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 shutdown_connections 2024-04-05T22:32:00.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.115+0000 7f6a8f200640 1 -- 172.21.15.73:0/3892134640 wait complete. 2024-04-05T22:32:00.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:32:00.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.351+0000 7fe2c2a00640 1 Processor -- start 2024-04-05T22:32:00.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.352+0000 7fe2c2a00640 1 -- start start 2024-04-05T22:32:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2bc167ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c2a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0d1d50 con 0x7fe2bc15ce70 2024-04-05T22:32:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c2a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0d1660 con 0x7fe2bc0d18c0 2024-04-05T22:32:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c2a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0d03a0 con 0x7fe2bc0b36b0 2024-04-05T22:32:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c1600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe2bc15ce70 0x7fe2bc0d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45828/0 (socket says 172.21.15.73:45828) 2024-04-05T22:32:00.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.353+0000 7fe2c1600640 1 -- 172.21.15.73:0/3062075287 learned_addr learned my addr 172.21.15.73:0/3062075287 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2bc167ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590122091 0 0) 0x7fe2bc0d1d50 con 0x7fe2bc15ce70 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a4003520 con 0x7fe2bc15ce70 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180165074 0 0) 0x7fe2a4003520 con 0x7fe2bc15ce70 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 msgr2=0x7fe2bc167ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2bc167ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 >> v1:172.21.15.157:6789/0 conn(0x7fe2bc0d18c0 legacy=0x7fe2bc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0ce400 con 0x7fe2bc15ce70 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fe2b0001670 con 0x7fe2bc15ce70 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 >> v1:172.21.15.73:6789/0 conn(0x7fe2bc15ce70 legacy=0x7fe2bc0d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2a40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a4003fd0 0x7fe2a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2a4004d00 0x7fe2a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0d03a0 con 0x7fe2a4004d00 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a400bb40 con 0x7fe2a4003fd0 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.354+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a400bd00 con 0x7fe2bc0b36b0 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2a4004d00 0x7fe2a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c0c00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a4003fd0 0x7fe2a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c1600640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2a40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 -- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 msgr2=0x7fe2a40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2a40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 -- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a4003fd0 msgr2=0x7fe2a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a4003fd0 0x7fe2a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c2000640 1 -- 172.21.15.73:0/3062075287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4003520 con 0x7fe2a4004d00 2024-04-05T22:32:00.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.355+0000 7fe2c1600640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc0b36b0 0x7fe2a40039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.356+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2a4004d00 0x7fe2a4005180 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007330 tx=0x7fe2a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61a7d6771d7a286 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cca0 con 0x7fe2a4004d00 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b402ce00 con 0x7fe2a4004d00 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2bbe00640 1 -- 172.21.15.73:0/3062075287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4034680 con 0x7fe2a4004d00 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3062075287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2a4004d00 msgr2=0x7fe2a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3062075287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2a4004d00 0x7fe2a4005180 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007330 tx=0x7fe2a40057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2000640 1 -- 172.21.15.73:0/3062075287 reap_dead start 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3062075287 shutdown_connections 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3062075287 >> 172.21.15.73:0/3062075287 conn(0x7fe2bc0d2a60 msgr2=0x7fe2bc0d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3062075287 shutdown_connections 2024-04-05T22:32:00.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.357+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3062075287 wait complete. 2024-04-05T22:32:00.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.359+0000 7fe2c2a00640 1 Processor -- start 2024-04-05T22:32:00.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.359+0000 7fe2c2a00640 1 -- start start 2024-04-05T22:32:00.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a4004d00 0x7fe2bc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc0b55f0 0x7fe2bc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b6be0 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b7070 con 0x7fe2bc0b55f0 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b5fd0 con 0x7fe2a4004d00 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33520/0 (socket says 172.21.15.73:33520) 2024-04-05T22:32:00.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.360+0000 7fe2c2000640 1 -- 172.21.15.73:0/3762004756 learned_addr learned my addr 172.21.15.73:0/3762004756 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c0c00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc0b55f0 0x7fe2bc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c1600640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a4004d00 0x7fe2bc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 -- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a4004d00 msgr2=0x7fe2bc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a4004d00 0x7fe2bc15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 -- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc0b55f0 msgr2=0x7fe2bc0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc0b55f0 0x7fe2bc0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 -- 172.21.15.73:0/3762004756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0b62b0 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2c2000640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4037040 tx=0x7fe2b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b403e640 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.361+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b403e7a0 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.362+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc0b6590 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.362+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b403eb30 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.362+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc0c9230 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.363+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc0b6d90 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.364+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fe2b403ec90 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.365+0000 7fe2b9600640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 0x7fe29c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.365+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b40aa130 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.366+0000 7fe2c1600640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 0x7fe29c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.367+0000 7fe2c1600640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 0x7fe29c06dc30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0001eb0 tx=0x7fe2b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.370+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe2bc0b6d90 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.565+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2bc067250 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.566+0000 7fe2b9600640 1 -- 172.21.15.73:0/3762004756 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2b4039020 con 0x7fe2bc0b5a70 2024-04-05T22:32:00.566 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 msgr2=0x7fe29c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 0x7fe29c06dc30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0001eb0 tx=0x7fe2b0009040 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 msgr2=0x7fe2bc0cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4037040 tx=0x7fe2b4004ab0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 shutdown_connections 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe29c06b770 0x7fe29c06dc30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc0b5a70 0x7fe2bc0cba40 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc0b55f0 0x7fe2bc0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 --2- 172.21.15.73:0/3762004756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a4004d00 0x7fe2bc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 >> 172.21.15.73:0/3762004756 conn(0x7fe2bc0d2a60 msgr2=0x7fe2bc0b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 shutdown_connections 2024-04-05T22:32:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.569+0000 7fe2c2a00640 1 -- 172.21.15.73:0/3762004756 wait complete. 2024-04-05T22:32:00.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:32:00.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.798+0000 7f7edca00640 1 Processor -- start 2024-04-05T22:32:00.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.798+0000 7f7edca00640 1 -- start start 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.799+0000 7f7edca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ed81587d0 0x7f7ed80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.799+0000 7f7edca00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 0x7f7ed815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.799+0000 7f7edca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed806f5b0 con 0x7f7ed80ccd20 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7edca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed815f240 con 0x7f7ed81587d0 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7edca00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed8070410 con 0x7f7ed815e850 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ed81587d0 0x7f7ed80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ecfe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7ed80ccd20 0x7f7ed806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45838/0 (socket says 172.21.15.73:45838) 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ecfe00640 1 -- 172.21.15.73:0/2817778990 learned_addr learned my addr 172.21.15.73:0/2817778990 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:00.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ed81587d0 0x7f7ed80cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33536/0 (socket says 172.21.15.73:33536) 2024-04-05T22:32:00.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed6800640 1 --2- 172.21.15.73:0/2817778990 >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 0x7f7ed815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2817778990 >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 msgr2=0x7f7ed815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:00.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 --2- 172.21.15.73:0/2817778990 >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 0x7f7ed815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2817778990 >> v1:172.21.15.73:6789/0 conn(0x7f7ed80ccd20 legacy=0x7f7ed806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2817778990 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed8158bb0 con 0x7f7ed81587d0 2024-04-05T22:32:00.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.800+0000 7f7ed6800640 1 --2- 172.21.15.73:0/2817778990 >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 0x7f7ed815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:00.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.801+0000 7f7ed5e00640 1 --2- 172.21.15.73:0/2817778990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed80cc7e0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb838524616465a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7ed5400640 1 -- 172.21.15.73:0/2817778990 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8031830 con 0x7f7ed81587d0 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7ed5400640 1 -- 172.21.15.73:0/2817778990 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8031990 con 0x7f7ed81587d0 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7ed5400640 1 -- 172.21.15.73:0/2817778990 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8031d20 con 0x7f7ed81587d0 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 -- 172.21.15.73:0/2817778990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 msgr2=0x7f7ed80cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 --2- 172.21.15.73:0/2817778990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed80cc7e0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802b8d0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 -- 172.21.15.73:0/2817778990 shutdown_connections 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 --2- 172.21.15.73:0/2817778990 >> v2:172.21.15.157:3300/0 conn(0x7f7ed815e850 0x7f7ed815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 --2- 172.21.15.73:0/2817778990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed80cc7e0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 -- 172.21.15.73:0/2817778990 >> 172.21.15.73:0/2817778990 conn(0x7f7ed80ce8e0 msgr2=0x7f7ed80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.802+0000 7f7edca00640 1 -- 172.21.15.73:0/2817778990 shutdown_connections 2024-04-05T22:32:00.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.803+0000 7f7edca00640 1 -- 172.21.15.73:0/2817778990 wait complete. 2024-04-05T22:32:00.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.804+0000 7f7edca00640 1 Processor -- start 2024-04-05T22:32:00.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.804+0000 7f7edca00640 1 -- start start 2024-04-05T22:32:00.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 0x7f7ed8156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed81553f0 0x7f7ed81571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed81558b0 con 0x7f7ed81587d0 2024-04-05T22:32:00.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8155b90 con 0x7f7ed815e850 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7edca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8155e70 con 0x7f7ed81553f0 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33550/0 (socket says 172.21.15.73:33550) 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.805+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2748591798 learned_addr learned my addr 172.21.15.73:0/2748591798 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ecfe00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 0x7f7ed8156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed6800640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed81553f0 0x7f7ed81571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed81553f0 msgr2=0x7f7ed81571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed81553f0 0x7f7ed81571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 msgr2=0x7f7ed8156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 0x7f7ed8156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 -- 172.21.15.73:0/2748591798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8009400 con 0x7f7ed81587d0 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ecfe00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 0x7f7ed8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.806+0000 7f7ed5e00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f7ec802bee0 tx=0x7f7ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.807+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec803ba70 con 0x7f7ed81587d0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.807+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec803bbf0 con 0x7f7ed81587d0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.807+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80453f0 con 0x7f7ed81587d0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.807+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed8156150 con 0x7f7ed81587d0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.807+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed8149780 con 0x7f7ed81587d0 2024-04-05T22:32:00.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.808+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed8066af0 con 0x7f7ed81587d0 2024-04-05T22:32:00.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.809+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f7ec804e450 con 0x7f7ed81587d0 2024-04-05T22:32:00.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.810+0000 7f7ece000640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 0x7f7eb406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:00.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.810+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ec80aa030 con 0x7f7ed81587d0 2024-04-05T22:32:00.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.810+0000 7f7ecfe00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 0x7f7eb406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:00.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.812+0000 7f7ecfe00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 0x7f7eb406dc00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80cb9a0 tx=0x7f7ec0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:00.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:00.815+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7ec8076cf0 con 0x7f7ed81587d0 2024-04-05T22:32:01.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.025+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ed8149910 con 0x7f7ed81587d0 2024-04-05T22:32:01.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.026+0000 7f7ece000640 1 -- 172.21.15.73:0/2748591798 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v172) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7ec8035020 con 0x7f7ed81587d0 2024-04-05T22:32:01.026 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 msgr2=0x7f7eb406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 0x7f7eb406dc00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80cb9a0 tx=0x7f7ec0009040 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 msgr2=0x7f7ed8149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f7ec802bee0 tx=0x7f7ec80046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 shutdown_connections 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7eb406b740 0x7f7eb406dc00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed81553f0 0x7f7ed81571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed815e850 0x7f7ed8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.028+0000 7f7edca00640 1 --2- 172.21.15.73:0/2748591798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ed81587d0 0x7f7ed8149240 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.029+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 >> 172.21.15.73:0/2748591798 conn(0x7f7ed80ce8e0 msgr2=0x7f7ed80cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:01.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.029+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 shutdown_connections 2024-04-05T22:32:01.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.029+0000 7f7edca00640 1 -- 172.21.15.73:0/2748591798 wait complete. 2024-04-05T22:32:01.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-05T22:32:01.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:01.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-05T22:32:01.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:32:01.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:01.183 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-05T22:32:01.234 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:32:01.234 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:01.274 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:32:01.275 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:01.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:32:01.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.574+0000 7f935fc00640 1 Processor -- start 2024-04-05T22:32:01.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.575+0000 7f935fc00640 1 -- start start 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.576+0000 7f935fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.576+0000 7f935fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f935815e850 0x7f935815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.576+0000 7f935fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935806f5b0 con 0x7f93580ccd20 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.576+0000 7f935fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935815f240 con 0x7f935815e850 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.576+0000 7f935fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9358070410 con 0x7f93581587d0 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f9357e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f93580ccd20 0x7f935806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45840/0 (socket says 172.21.15.73:45840) 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f9357e00640 1 -- 172.21.15.73:0/487418759 learned_addr learned my addr 172.21.15.73:0/487418759 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935dc00640 1 --2- 172.21.15.73:0/487418759 >> v2:172.21.15.73:3300/0 conn(0x7f935815e850 0x7f935815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935d200640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f9357400640 1 -- 172.21.15.73:0/487418759 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784018364 0 0) 0x7f935806f5b0 con 0x7f93580ccd20 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f9357400640 1 -- 172.21.15.73:0/487418759 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9340003520 con 0x7f93580ccd20 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935dc00640 1 -- 172.21.15.73:0/487418759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 msgr2=0x7f93580cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935dc00640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935dc00640 1 -- 172.21.15.73:0/487418759 >> v1:172.21.15.73:6789/0 conn(0x7f93580ccd20 legacy=0x7f935806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.577+0000 7f935dc00640 1 -- 172.21.15.73:0/487418759 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358158bb0 con 0x7f935815e850 2024-04-05T22:32:01.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.578+0000 7f935dc00640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815e850 0x7f935815ed00 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9350009d90 tx=0x7f935002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3268d5fe3b9239 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:01.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f9357400640 1 -- 172.21.15.73:0/487418759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935002cc50 con 0x7f935815e850 2024-04-05T22:32:01.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f9357400640 1 -- 172.21.15.73:0/487418759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935002cdb0 con 0x7f935815e850 2024-04-05T22:32:01.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f9357400640 1 -- 172.21.15.73:0/487418759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350034660 con 0x7f935815e850 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 -- 172.21.15.73:0/487418759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815e850 msgr2=0x7f935815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815e850 0x7f935815ed00 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9350009d90 tx=0x7f935002bea0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 -- 172.21.15.73:0/487418759 shutdown_connections 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815e850 0x7f935815ed00 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 --2- 172.21.15.73:0/487418759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.579+0000 7f935fc00640 1 -- 172.21.15.73:0/487418759 >> 172.21.15.73:0/487418759 conn(0x7f93580ce8e0 msgr2=0x7f93580cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.580+0000 7f935fc00640 1 -- 172.21.15.73:0/487418759 shutdown_connections 2024-04-05T22:32:01.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.580+0000 7f935fc00640 1 -- 172.21.15.73:0/487418759 wait complete. 2024-04-05T22:32:01.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.581+0000 7f935fc00640 1 Processor -- start 2024-04-05T22:32:01.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.581+0000 7f935fc00640 1 -- start start 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935815e850 0x7f93581f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580b1970 con 0x7f935815d550 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581f31c0 con 0x7f935815e850 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.582+0000 7f935fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93581f34a0 con 0x7f93581587d0 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33558/0 (socket says 172.21.15.73:33558) 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 -- 172.21.15.73:0/3516827361 learned_addr learned my addr 172.21.15.73:0/3516827361 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935d200640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f9357e00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935815e850 0x7f93581f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 -- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 msgr2=0x7f93580cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 -- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935815e850 msgr2=0x7f93581f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935815e850 0x7f93581f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.583+0000 7f935dc00640 1 -- 172.21.15.73:0/3516827361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009a70 con 0x7f935815d550 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f935dc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f9350009d90 tx=0x7f9350004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350040650 con 0x7f935815d550 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500407b0 con 0x7f935815d550 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350040b60 con 0x7f935815d550 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93581f3780 con 0x7f935815d550 2024-04-05T22:32:01.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.584+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93581f4000 con 0x7f935815d550 2024-04-05T22:32:01.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.585+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358066af0 con 0x7f935815d550 2024-04-05T22:32:01.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.586+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f9350040cc0 con 0x7f935815d550 2024-04-05T22:32:01.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.587+0000 7f9354c00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 0x7f933c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:01.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.587+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93500aa770 con 0x7f935815d550 2024-04-05T22:32:01.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.588+0000 7f935d200640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 0x7f933c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:01.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.589+0000 7f935d200640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 0x7f933c06dc30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:01.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.591+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f93500773a0 con 0x7f935815d550 2024-04-05T22:32:01.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.787+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93580b1e50 con 0x7f935815d550 2024-04-05T22:32:01.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.789+0000 7f9354c00640 1 -- 172.21.15.73:0/3516827361 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7f9350009450 con 0x7f935815d550 2024-04-05T22:32:01.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:01.789 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:32:01.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 msgr2=0x7f933c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 0x7f933c06dc30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c008040 comp rx=0 tx=0).stop 2024-04-05T22:32:01.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 msgr2=0x7f93581f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:01.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f9350009d90 tx=0x7f9350004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:01.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 shutdown_connections 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f933c06b770 0x7f933c06dc30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935815d550 0x7f93581f28b0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935815e850 0x7f93581f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 --2- 172.21.15.73:0/3516827361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93581587d0 0x7f93580cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 >> 172.21.15.73:0/3516827361 conn(0x7f93580ce8e0 msgr2=0x7f935806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 shutdown_connections 2024-04-05T22:32:01.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:01.792+0000 7f935fc00640 1 -- 172.21.15.73:0/3516827361 wait complete. 2024-04-05T22:32:01.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:31:40.727092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:31:41.315098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:32:01.812 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:32:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.022+0000 7fdae4400640 1 Processor -- start 2024-04-05T22:32:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.022+0000 7fdae4400640 1 -- start start 2024-04-05T22:32:02.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdadc15aa70 0x7fdadc070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdadc0714a0 0x7fdadc15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc071930 con 0x7fdadc0714a0 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc15ae50 con 0x7fdadc15aa70 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc15e3b0 con 0x7fdadc15dbc0 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdadc0714a0 0x7fdadc15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.024+0000 7fdae1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdadc0714a0 0x7fdadc15d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33568/0 (socket says 172.21.15.73:33568) 2024-04-05T22:32:02.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 -- 172.21.15.73:0/3963520469 learned_addr learned my addr 172.21.15.73:0/3963520469 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 -- 172.21.15.73:0/3963520469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 msgr2=0x7fdadc1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 --2- 172.21.15.73:0/3963520469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 -- 172.21.15.73:0/3963520469 >> v2:172.21.15.157:3300/0 conn(0x7fdadc15aa70 msgr2=0x7fdadc070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1a00640 1 --2- 172.21.15.73:0/3963520469 >> v2:172.21.15.157:3300/0 conn(0x7fdadc15aa70 0x7fdadc070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 --2- 172.21.15.73:0/3963520469 >> v2:172.21.15.157:3300/0 conn(0x7fdadc15aa70 0x7fdadc070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.025+0000 7fdae1000640 1 -- 172.21.15.73:0/3963520469 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0ac9e0 con 0x7fdadc0714a0 2024-04-05T22:32:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.026+0000 7fdae1000640 1 --2- 172.21.15.73:0/3963520469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc15d680 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0b16a0 tx=0x7fdac802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c439dba05461e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.026+0000 7fdadbe00640 1 -- 172.21.15.73:0/3963520469 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8031870 con 0x7fdadc0714a0 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdadbe00640 1 -- 172.21.15.73:0/3963520469 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac80319d0 con 0x7fdadc0714a0 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdadbe00640 1 -- 172.21.15.73:0/3963520469 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8031d60 con 0x7fdadc0714a0 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 -- 172.21.15.73:0/3963520469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 msgr2=0x7fdadc15d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 --2- 172.21.15.73:0/3963520469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc15d680 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0b16a0 tx=0x7fdac802b910 comp rx=0 tx=0).stop 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 -- 172.21.15.73:0/3963520469 shutdown_connections 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 --2- 172.21.15.73:0/3963520469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc15d680 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 --2- 172.21.15.73:0/3963520469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 --2- 172.21.15.73:0/3963520469 >> v2:172.21.15.157:3300/0 conn(0x7fdadc15aa70 0x7fdadc070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.027+0000 7fdae4400640 1 -- 172.21.15.73:0/3963520469 >> 172.21.15.73:0/3963520469 conn(0x7fdadc069cb0 msgr2=0x7fdadc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.028+0000 7fdae4400640 1 -- 172.21.15.73:0/3963520469 shutdown_connections 2024-04-05T22:32:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.028+0000 7fdae4400640 1 -- 172.21.15.73:0/3963520469 wait complete. 2024-04-05T22:32:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.029+0000 7fdae4400640 1 Processor -- start 2024-04-05T22:32:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 -- start start 2024-04-05T22:32:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdadc15aa70 0x7fdadc158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc159e80 con 0x7fdadc0714a0 2024-04-05T22:32:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc1595e0 con 0x7fdadc15aa70 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.030+0000 7fdae4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc1598c0 con 0x7fdadc15dbc0 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33578/0 (socket says 172.21.15.73:33578) 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 -- 172.21.15.73:0/2937849818 learned_addr learned my addr 172.21.15.73:0/2937849818 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 -- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 msgr2=0x7fdadc158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 -- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdadc15aa70 msgr2=0x7fdadc158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdadc15aa70 0x7fdadc158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 -- 172.21.15.73:0/2937849818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80094d0 con 0x7fdadc0714a0 2024-04-05T22:32:02.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.031+0000 7fdae1a00640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c7f0 tx=0x7fdad000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.032+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0004220 con 0x7fdadc0714a0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.032+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad00043a0 con 0x7fdadc0714a0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.032+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0007860 con 0x7fdadc0714a0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.032+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc159ba0 con 0x7fdadc0714a0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.032+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc15c270 con 0x7fdadc0714a0 2024-04-05T22:32:02.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.033+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc1f2bb0 con 0x7fdadc0714a0 2024-04-05T22:32:02.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.034+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fdad0007bd0 con 0x7fdadc0714a0 2024-04-05T22:32:02.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.035+0000 7fdad9600640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 0x7fdabc06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.035+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdad0087f40 con 0x7fdadc0714a0 2024-04-05T22:32:02.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.035+0000 7fdae1000640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 0x7fdabc06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.037+0000 7fdae1000640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 0x7fdabc06dc00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdac8036000 tx=0x7fdac8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.039+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fdad0054920 con 0x7fdadc0714a0 2024-04-05T22:32:02.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.231+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdadc068160 con 0x7fdadc0714a0 2024-04-05T22:32:02.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.232+0000 7fdad9600640 1 -- 172.21.15.73:0/2937849818 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdad001e020 con 0x7fdadc0714a0 2024-04-05T22:32:02.232 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 msgr2=0x7fdabc06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 0x7fdabc06dc00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdac8036000 tx=0x7fdac8030000 comp rx=0 tx=0).stop 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 msgr2=0x7fdadc0aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c7f0 tx=0x7fdad000ccb0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 shutdown_connections 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fdabc06b740 0x7fdabc06dc00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdadc15dbc0 0x7fdadc158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdadc15aa70 0x7fdadc158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 --2- 172.21.15.73:0/2937849818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdadc0714a0 0x7fdadc0aefd0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 >> 172.21.15.73:0/2937849818 conn(0x7fdadc069cb0 msgr2=0x7fdadc06ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:02.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 shutdown_connections 2024-04-05T22:32:02.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.235+0000 7fdae4400640 1 -- 172.21.15.73:0/2937849818 wait complete. 2024-04-05T22:32:02.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:32:02.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.468+0000 7fbf5de00640 1 Processor -- start 2024-04-05T22:32:02.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.469+0000 7fbf5de00640 1 -- start start 2024-04-05T22:32:02.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.470+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf580cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.470+0000 7fbf5de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf581654e0 0x7fbf5806b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.470+0000 7fbf5de00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5815f260 con 0x7fbf580b1950 2024-04-05T22:32:02.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.470+0000 7fbf5de00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf58070d20 con 0x7fbf581654e0 2024-04-05T22:32:02.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.470+0000 7fbf5de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5806f5b0 con 0x7fbf5815ee80 2024-04-05T22:32:02.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf4fe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbf580b1950 0x7fbf58164dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45848/0 (socket says 172.21.15.73:45848) 2024-04-05T22:32:02.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf4fe00640 1 -- 172.21.15.73:0/1582636785 learned_addr learned my addr 172.21.15.73:0/1582636785 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571122960 0 0) 0x7fbf40001110 con 0x7fbf580b1950 2024-04-05T22:32:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf38003520 con 0x7fbf580b1950 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf57400640 1 --2- 172.21.15.73:0/1582636785 >> v2:172.21.15.157:3300/0 conn(0x7fbf581654e0 0x7fbf5806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56a00640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf580cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355057442 0 0) 0x7fbf38003520 con 0x7fbf580b1950 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 msgr2=0x7fbf580cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf580cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 >> v2:172.21.15.157:3300/0 conn(0x7fbf581654e0 msgr2=0x7fbf5806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 --2- 172.21.15.73:0/1582636785 >> v2:172.21.15.157:3300/0 conn(0x7fbf581654e0 0x7fbf5806b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5806fb70 con 0x7fbf580b1950 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbf40001800 con 0x7fbf580b1950 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 >> v1:172.21.15.73:6789/0 conn(0x7fbf580b1950 legacy=0x7fbf58164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.471+0000 7fbf56000640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf38003970 0x7fbf38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56000640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56000640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5806f5b0 con 0x7fbf38004d00 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3800ba30 con 0x7fbf38003970 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3800bb70 con 0x7fbf5815ee80 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf4fe00640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56a00640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf38003970 0x7fbf38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 -- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 msgr2=0x7fbf380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5815ee80 0x7fbf380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 -- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf38003970 msgr2=0x7fbf38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf38003970 0x7fbf38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf57400640 1 -- 172.21.15.73:0/1582636785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf38003520 con 0x7fbf38004d00 2024-04-05T22:32:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.472+0000 7fbf56a00640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf38003970 0x7fbf38003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.473+0000 7fbf57400640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf38005180 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fbf50007330 tx=0x7fbf380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcade86348338a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5002cc80 con 0x7fbf38004d00 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5002cde0 con 0x7fbf38004d00 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf56000640 1 -- 172.21.15.73:0/1582636785 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50034680 con 0x7fbf38004d00 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf5de00640 1 -- 172.21.15.73:0/1582636785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 msgr2=0x7fbf38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf5de00640 1 --2- 172.21.15.73:0/1582636785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf38005180 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fbf50007330 tx=0x7fbf380057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.474+0000 7fbf57400640 1 -- 172.21.15.73:0/1582636785 reap_dead start 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.475+0000 7fbf5de00640 1 -- 172.21.15.73:0/1582636785 shutdown_connections 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.475+0000 7fbf5de00640 1 -- 172.21.15.73:0/1582636785 >> 172.21.15.73:0/1582636785 conn(0x7fbf58069cb0 msgr2=0x7fbf5806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.475+0000 7fbf5de00640 1 -- 172.21.15.73:0/1582636785 shutdown_connections 2024-04-05T22:32:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.475+0000 7fbf5de00640 1 -- 172.21.15.73:0/1582636785 wait complete. 2024-04-05T22:32:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.477+0000 7fbf5de00640 1 Processor -- start 2024-04-05T22:32:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.477+0000 7fbf5de00640 1 -- start start 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.478+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.478+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5814cf30 0x7fbf5814a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.478+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5806b570 0x7fbf5806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.478+0000 7fbf5de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580c9030 con 0x7fbf38004d00 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf5de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580c92d0 con 0x7fbf5806b570 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf5de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5806bf30 con 0x7fbf5814cf30 2024-04-05T22:32:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33600/0 (socket says 172.21.15.73:33600) 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 -- 172.21.15.73:0/512303661 learned_addr learned my addr 172.21.15.73:0/512303661 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 -- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5814cf30 msgr2=0x7fbf5814a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5814cf30 0x7fbf5814a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 -- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5806b570 msgr2=0x7fbf5806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5806b570 0x7fbf5806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 -- 172.21.15.73:0/512303661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5814af80 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.479+0000 7fbf56a00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800e810 tx=0x7fbf4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.480+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800c5f0 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.480+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4800c750 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.480+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800cb00 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.480+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5814aca0 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.480+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5814d6a0 con 0x7fbf38004d00 2024-04-05T22:32:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.481+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58068160 con 0x7fbf38004d00 2024-04-05T22:32:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.482+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fbf4800cc60 con 0x7fbf38004d00 2024-04-05T22:32:02.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.483+0000 7fbf4f400640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 0x7fbf3406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.484+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf48087bf0 con 0x7fbf38004d00 2024-04-05T22:32:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.484+0000 7fbf4fe00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 0x7fbf3406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.485+0000 7fbf4fe00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 0x7fbf3406dc00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbf40001390 tx=0x7fbf40000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.488+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbf48054630 con 0x7fbf38004d00 2024-04-05T22:32:02.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.681+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf5806ce20 con 0x7fbf38004d00 2024-04-05T22:32:02.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.682+0000 7fbf4f400640 1 -- 172.21.15.73:0/512303661 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf48057c50 con 0x7fbf38004d00 2024-04-05T22:32:02.682 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.684+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 msgr2=0x7fbf3406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.684+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 0x7fbf3406dc00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbf40001390 tx=0x7fbf40000ed0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.684+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 msgr2=0x7fbf581fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.684+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800e810 tx=0x7fbf4800ecd0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 shutdown_connections 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbf3406b740 0x7fbf3406dc00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5806b570 0x7fbf5806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5814cf30 0x7fbf5814a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 --2- 172.21.15.73:0/512303661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf38004d00 0x7fbf581fabd0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 >> 172.21.15.73:0/512303661 conn(0x7fbf58069cb0 msgr2=0x7fbf58160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:02.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 shutdown_connections 2024-04-05T22:32:02.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.685+0000 7fbf5de00640 1 -- 172.21.15.73:0/512303661 wait complete. 2024-04-05T22:32:02.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:32:02.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.916+0000 7fd0b1400640 1 Processor -- start 2024-04-05T22:32:02.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.917+0000 7fd0b1400640 1 -- start start 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0ac160ff0 0x7fd0ac0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac1613d0 con 0x7fd0ac1654e0 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac06b4f0 con 0x7fd0ac160ff0 2024-04-05T22:32:02.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.918+0000 7fd0b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac06b760 con 0x7fd0ac0ae170 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33602/0 (socket says 172.21.15.73:33602) 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 -- 172.21.15.73:0/4106852215 learned_addr learned my addr 172.21.15.73:0/4106852215 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0a3400640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0abe00640 1 --2- 172.21.15.73:0/4106852215 >> v2:172.21.15.157:3300/0 conn(0x7fd0ac160ff0 0x7fd0ac0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 -- 172.21.15.73:0/4106852215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 msgr2=0x7fd0ac0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac0b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 -- 172.21.15.73:0/4106852215 >> v2:172.21.15.157:3300/0 conn(0x7fd0ac160ff0 msgr2=0x7fd0ac0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/4106852215 >> v2:172.21.15.157:3300/0 conn(0x7fd0ac160ff0 0x7fd0ac0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.919+0000 7fd0b0a00640 1 -- 172.21.15.73:0/4106852215 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac06ba40 con 0x7fd0ac1654e0 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.920+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06f740 tx=0x7fd0a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724880e822edb888 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0ab400640 1 -- 172.21.15.73:0/4106852215 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a402ca70 con 0x7fd0ac1654e0 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0ab400640 1 -- 172.21.15.73:0/4106852215 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a402cbd0 con 0x7fd0ac1654e0 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0ab400640 1 -- 172.21.15.73:0/4106852215 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40343f0 con 0x7fd0ac1654e0 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 -- 172.21.15.73:0/4106852215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 msgr2=0x7fd0ac0b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06f740 tx=0x7fd0a402bed0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 -- 172.21.15.73:0/4106852215 shutdown_connections 2024-04-05T22:32:02.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0b29c0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 --2- 172.21.15.73:0/4106852215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 --2- 172.21.15.73:0/4106852215 >> v2:172.21.15.157:3300/0 conn(0x7fd0ac160ff0 0x7fd0ac0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 -- 172.21.15.73:0/4106852215 >> 172.21.15.73:0/4106852215 conn(0x7fd0ac0b0830 msgr2=0x7fd0ac0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 -- 172.21.15.73:0/4106852215 shutdown_connections 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.921+0000 7fd0b1400640 1 -- 172.21.15.73:0/4106852215 wait complete. 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.923+0000 7fd0b1400640 1 Processor -- start 2024-04-05T22:32:02.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.923+0000 7fd0b1400640 1 -- start start 2024-04-05T22:32:02.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ac160ff0 0x7fd0ac162960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac163e50 con 0x7fd0ac1654e0 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0aa830 con 0x7fd0ac160ff0 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1510e0 con 0x7fd0ac0ae170 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33606/0 (socket says 172.21.15.73:33606) 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.924+0000 7fd0b0a00640 1 -- 172.21.15.73:0/1837483442 learned_addr learned my addr 172.21.15.73:0/1837483442 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0abe00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 -- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 msgr2=0x7fd0ac150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 -- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ac160ff0 msgr2=0x7fd0ac162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0a3400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ac160ff0 0x7fd0ac162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ac160ff0 0x7fd0ac162960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:02.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 -- 172.21.15.73:0/1837483442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4009b30 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0abe00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.925+0000 7fd0b0a00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4036040 tx=0x7fd0a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.926+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4009510 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.926+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a403e450 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.926+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a403e800 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.926+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac1632d0 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.926+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac163890 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.927+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac0aa9f0 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.928+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fd0a403e960 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.929+0000 7fd0a8c00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 0x7fd08806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.929+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0a40aaa60 con 0x7fd0ac1654e0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.930+0000 7fd0abe00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 0x7fd08806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:02.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.931+0000 7fd0abe00640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 0x7fd08806dc00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:02.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:02.933+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fd0a40776f0 con 0x7fd0ac1654e0 2024-04-05T22:32:03.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.127+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0ac04e160 con 0x7fd0ac1654e0 2024-04-05T22:32:03.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.128+0000 7fd0a8c00640 1 -- 172.21.15.73:0/1837483442 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0a407ad10 con 0x7fd0ac1654e0 2024-04-05T22:32:03.128 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:32:03.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.130+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 msgr2=0x7fd08806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.130+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 0x7fd08806dc00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c008040 comp rx=0 tx=0).stop 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.130+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 msgr2=0x7fd0ac0a8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.130+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4036040 tx=0x7fd0a4004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 shutdown_connections 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fd08806b740 0x7fd08806dc00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac1654e0 0x7fd0ac0a8820 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ac160ff0 0x7fd0ac162960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 --2- 172.21.15.73:0/1837483442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ac0ae170 0x7fd0ac150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 >> 172.21.15.73:0/1837483442 conn(0x7fd0ac0b0830 msgr2=0x7fd0ac15dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:03.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 shutdown_connections 2024-04-05T22:32:03.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.131+0000 7fd0b1400640 1 -- 172.21.15.73:0/1837483442 wait complete. 2024-04-05T22:32:03.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-05T22:32:03.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.363+0000 7f77b8800640 1 Processor -- start 2024-04-05T22:32:03.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.364+0000 7f77b8800640 1 -- start start 2024-04-05T22:32:03.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.365+0000 7f77b8800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.365+0000 7f77b8800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77b015e850 0x7f77b015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.365+0000 7f77b8800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b006f5b0 con 0x7f77b01587d0 2024-04-05T22:32:03.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.365+0000 7f77b8800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b006d500 con 0x7f77b015e850 2024-04-05T22:32:03.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.365+0000 7f77b8800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b015f220 con 0x7f77b006cb80 2024-04-05T22:32:03.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.366+0000 7f77b5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.366+0000 7f77b5e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f77b01587d0 0x7f77b00cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:49426/0 (socket says 172.21.15.73:49426) 2024-04-05T22:32:03.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.366+0000 7f77b5e00640 1 -- 172.21.15.73:0/601185239 learned_addr learned my addr 172.21.15.73:0/601185239 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:03.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.367+0000 7f77b6800640 1 --2- 172.21.15.73:0/601185239 >> v2:172.21.15.157:3300/0 conn(0x7f77b015e850 0x7f77b015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.367+0000 7f77b5400640 1 -- 172.21.15.73:0/601185239 >> v2:172.21.15.157:3300/0 conn(0x7f77b015e850 msgr2=0x7f77b015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:03.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.367+0000 7f77b5400640 1 --2- 172.21.15.73:0/601185239 >> v2:172.21.15.157:3300/0 conn(0x7f77b015e850 0x7f77b015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.367+0000 7f77b5400640 1 -- 172.21.15.73:0/601185239 >> v1:172.21.15.157:6789/0 conn(0x7f77b01587d0 legacy=0x7f77b00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.367+0000 7f77b5400640 1 -- 172.21.15.73:0/601185239 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b0070410 con 0x7f77b006cb80 2024-04-05T22:32:03.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.368+0000 7f77b5400640 1 --2- 172.21.15.73:0/601185239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b006cfc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77a00097a0 tx=0x7f77a002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75927ce1f27bb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:03.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.369+0000 7f77b4a00640 1 -- 172.21.15.73:0/601185239 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0031830 con 0x7f77b006cb80 2024-04-05T22:32:03.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.369+0000 7f77b4a00640 1 -- 172.21.15.73:0/601185239 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0031990 con 0x7f77b006cb80 2024-04-05T22:32:03.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.369+0000 7f77b4a00640 1 -- 172.21.15.73:0/601185239 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0031d20 con 0x7f77b006cb80 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 -- 172.21.15.73:0/601185239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 msgr2=0x7f77b006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 --2- 172.21.15.73:0/601185239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b006cfc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77a00097a0 tx=0x7f77a002b8d0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 -- 172.21.15.73:0/601185239 shutdown_connections 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 --2- 172.21.15.73:0/601185239 >> v2:172.21.15.157:3300/0 conn(0x7f77b015e850 0x7f77b015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 --2- 172.21.15.73:0/601185239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b006cfc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 -- 172.21.15.73:0/601185239 >> 172.21.15.73:0/601185239 conn(0x7f77b00ce8e0 msgr2=0x7f77b00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 -- 172.21.15.73:0/601185239 shutdown_connections 2024-04-05T22:32:03.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.370+0000 7f77b8800640 1 -- 172.21.15.73:0/601185239 wait complete. 2024-04-05T22:32:03.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.371+0000 7f77b8800640 1 Processor -- start 2024-04-05T22:32:03.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.372+0000 7f77b8800640 1 -- start start 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.372+0000 7f77b8800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b01f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.372+0000 7f77b8800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.372+0000 7f77b8800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b015e850 0x7f77b00acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.372+0000 7f77b8800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00ad100 con 0x7f77b01587d0 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b8800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b015a8e0 con 0x7f77b015e850 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b8800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b015abc0 con 0x7f77b006cb80 2024-04-05T22:32:03.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33616/0 (socket says 172.21.15.73:33616) 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 -- 172.21.15.73:0/3130790070 learned_addr learned my addr 172.21.15.73:0/3130790070 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5e00640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b01f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b6800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b015e850 0x7f77b00acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 -- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 msgr2=0x7f77b01f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b01f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 -- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b015e850 msgr2=0x7f77b00acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b015e850 0x7f77b00acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.373+0000 7f77b5400640 1 -- 172.21.15.73:0/3130790070 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009400 con 0x7f77b01587d0 2024-04-05T22:32:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77b5400640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f77a00098c0 tx=0x7f77a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a003a6f0 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a003a870 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a003ac20 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b00a8f70 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.374+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b00a9530 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.375+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b0066af0 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.376+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f77a0043410 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.377+0000 7f77aa000640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 0x7f779406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.377+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77a00a9890 con 0x7f77b01587d0 2024-04-05T22:32:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.378+0000 7f77b5e00640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 0x7f779406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:03.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.379+0000 7f77b5e00640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 0x7f779406dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77a4005880 tx=0x7f77a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:03.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.381+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f77a00764c0 con 0x7f77b01587d0 2024-04-05T22:32:03.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.585+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f77b0068160 con 0x7f77b01587d0 2024-04-05T22:32:03.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.586+0000 7f77aa000640 1 -- 172.21.15.73:0/3130790070 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v172) v1 ==== 80+0+0 (secure 0 0 0) 0x7f77a0079ae0 con 0x7f77b01587d0 2024-04-05T22:32:03.586 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-05T22:32:03.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.588+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 msgr2=0x7f779406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.588+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 0x7f779406dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77a4005880 tx=0x7f77a4009040 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.588+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 msgr2=0x7f77b00ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.588+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f77a00098c0 tx=0x7f77a00046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 shutdown_connections 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f779406b740 0x7f779406dc00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b015e850 0x7f77b00acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77b01587d0 0x7f77b00ac680 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 --2- 172.21.15.73:0/3130790070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77b006cb80 0x7f77b01f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 >> 172.21.15.73:0/3130790070 conn(0x7f77b00ce8e0 msgr2=0x7f77b0069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 shutdown_connections 2024-04-05T22:32:03.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:03.589+0000 7f77b8800640 1 -- 172.21.15.73:0/3130790070 wait complete. 2024-04-05T22:32:03.608 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-05T22:32:03.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:03.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-05T22:32:03.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:32:03.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:03.723 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-05T22:32:03.768 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:32:03.768 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:03.825 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:32:03.825 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:32:03.825 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:03.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:32:03.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:32:04.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.090+0000 7f5abd400640 1 Processor -- start 2024-04-05T22:32:04.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.090+0000 7f5abd400640 1 -- start start 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5abd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ab815ef60 0x7f5ab806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5abd400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab806f5b0 con 0x7f5ab806cb80 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5abd400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab8070410 con 0x7f5ab81587d0 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5abd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab806d500 con 0x7f5ab815ef60 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5ab5e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5ab806cb80 0x7f5ab815e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45852/0 (socket says 172.21.15.73:45852) 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5ab5e00640 1 -- 172.21.15.73:0/2976724758 learned_addr learned my addr 172.21.15.73:0/2976724758 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:04.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5ab7200640 1 --2- 172.21.15.73:0/2976724758 >> v2:172.21.15.73:3300/0 conn(0x7f5ab815ef60 0x7f5ab806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.092+0000 7f5ab5400640 1 -- 172.21.15.73:0/2976724758 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572322759 0 0) 0x7f5ab806f5b0 con 0x7f5ab806cb80 2024-04-05T22:32:04.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.093+0000 7f5ab5400640 1 -- 172.21.15.73:0/2976724758 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c003520 con 0x7f5ab806cb80 2024-04-05T22:32:04.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.093+0000 7f5ab7200640 1 -- 172.21.15.73:0/2976724758 >> v1:172.21.15.157:6789/0 conn(0x7f5ab81587d0 legacy=0x7f5ab80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:04.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.093+0000 7f5ab7200640 1 -- 172.21.15.73:0/2976724758 >> v1:172.21.15.73:6789/0 conn(0x7f5ab806cb80 legacy=0x7f5ab815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.093+0000 7f5ab7200640 1 -- 172.21.15.73:0/2976724758 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab80ccef0 con 0x7f5ab815ef60 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.094+0000 7f5ab7200640 1 --2- 172.21.15.73:0/2976724758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab806cfc0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd34b5ecca57fed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.094+0000 7f5ab5400640 1 -- 172.21.15.73:0/2976724758 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab002cd00 con 0x7f5ab815ef60 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.094+0000 7f5ab5400640 1 -- 172.21.15.73:0/2976724758 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab002ce60 con 0x7f5ab815ef60 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.094+0000 7f5ab5400640 1 -- 172.21.15.73:0/2976724758 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0034640 con 0x7f5ab815ef60 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 -- 172.21.15.73:0/2976724758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 msgr2=0x7f5ab806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 --2- 172.21.15.73:0/2976724758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab806cfc0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab002bf50 comp rx=0 tx=0).stop 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 -- 172.21.15.73:0/2976724758 shutdown_connections 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 --2- 172.21.15.73:0/2976724758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab806cfc0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 -- 172.21.15.73:0/2976724758 >> 172.21.15.73:0/2976724758 conn(0x7f5ab80ce8e0 msgr2=0x7f5ab80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 -- 172.21.15.73:0/2976724758 shutdown_connections 2024-04-05T22:32:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.095+0000 7f5abd400640 1 -- 172.21.15.73:0/2976724758 wait complete. 2024-04-05T22:32:04.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.096+0000 7f5abd400640 1 Processor -- start 2024-04-05T22:32:04.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.096+0000 7f5abd400640 1 -- start start 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 0x7f5ab80b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81587d0 0x7f5ab80acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81f2b70 con 0x7f5ab815ef60 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80b3160 con 0x7f5ab81587d0 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.097+0000 7f5abd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80b2550 con 0x7f5ab806cb80 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33624/0 (socket says 172.21.15.73:33624) 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 -- 172.21.15.73:0/1165791073 learned_addr learned my addr 172.21.15.73:0/1165791073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab6800640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 0x7f5ab80b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab5e00640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81587d0 0x7f5ab80acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 -- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 msgr2=0x7f5ab80b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 0x7f5ab80b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 -- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81587d0 msgr2=0x7f5ab80acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81587d0 0x7f5ab80acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 -- 172.21.15.73:0/1165791073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0009a70 con 0x7f5ab815ef60 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab6800640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 0x7f5ab80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.098+0000 7f5ab7200640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.099+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003b90 con 0x7f5ab815ef60 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.099+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab0003cf0 con 0x7f5ab815ef60 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.099+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0004250 con 0x7f5ab815ef60 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.099+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab80b2830 con 0x7f5ab815ef60 2024-04-05T22:32:04.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.099+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab80b2df0 con 0x7f5ab815ef60 2024-04-05T22:32:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.100+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8066af0 con 0x7f5ab815ef60 2024-04-05T22:32:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.101+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f5ab003b050 con 0x7f5ab815ef60 2024-04-05T22:32:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.102+0000 7f5aaea00640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 0x7f5a9806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.102+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ab00aa4e0 con 0x7f5ab815ef60 2024-04-05T22:32:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.103+0000 7f5ab6800640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 0x7f5a9806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.104+0000 7f5ab6800640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 0x7f5a9806dc00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000f60 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.107+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5ab0076ec0 con 0x7f5ab815ef60 2024-04-05T22:32:04.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.338+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ab806d1a0 con 0x7f5ab815ef60 2024-04-05T22:32:04.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.339+0000 7f5aaea00640 1 -- 172.21.15.73:0/1165791073 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4753 (secure 0 0 0) 0x7f5ab0039090 con 0x7f5ab815ef60 2024-04-05T22:32:04.339 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:04.339 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:32:04.339 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 110 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.341+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 msgr2=0x7f5a9806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.341+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 0x7f5a9806dc00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000f60 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.341+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 msgr2=0x7f5ab80b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.341+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab0004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 shutdown_connections 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5a9806b740 0x7f5a9806dc00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab815ef60 0x7f5ab80b1970 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81587d0 0x7f5ab80acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 --2- 172.21.15.73:0/1165791073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab806cb80 0x7f5ab80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 >> 172.21.15.73:0/1165791073 conn(0x7f5ab80ce8e0 msgr2=0x7f5ab80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:04.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 shutdown_connections 2024-04-05T22:32:04.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.342+0000 7f5abd400640 1 -- 172.21.15.73:0/1165791073 wait complete. 2024-04-05T22:32:04.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:32:04.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.570+0000 7f61e2e00640 1 Processor -- start 2024-04-05T22:32:04.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.570+0000 7f61e2e00640 1 -- start start 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e2e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61dc1587d0 0x7f61dc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61dc15e850 0x7f61dc15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e2e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc06f5b0 con 0x7f61dc0ccd20 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc15f240 con 0x7f61dc15e850 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e2e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc070410 con 0x7f61dc1587d0 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.572+0000 7f61e0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61dc15e850 0x7f61dc15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61dc15e850 0x7f61dc15ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33638/0 (socket says 172.21.15.73:33638) 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 -- 172.21.15.73:0/1341601614 learned_addr learned my addr 172.21.15.73:0/1341601614 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:04.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61dbe00640 1 --2- 172.21.15.73:0/1341601614 >> v2:172.21.15.157:3300/0 conn(0x7f61dc1587d0 0x7f61dc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 -- 172.21.15.73:0/1341601614 >> v2:172.21.15.157:3300/0 conn(0x7f61dc1587d0 msgr2=0x7f61dc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 --2- 172.21.15.73:0/1341601614 >> v2:172.21.15.157:3300/0 conn(0x7f61dc1587d0 0x7f61dc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 -- 172.21.15.73:0/1341601614 >> v1:172.21.15.73:6789/0 conn(0x7f61dc0ccd20 legacy=0x7f61dc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.573+0000 7f61e0e00640 1 -- 172.21.15.73:0/1341601614 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc158bb0 con 0x7f61dc15e850 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.574+0000 7f61e0e00640 1 --2- 172.21.15.73:0/1341601614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc15e850 0x7f61dc15ed00 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009e50 tx=0x7f61d402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93cf31b1c07d197 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.574+0000 7f61db400640 1 -- 172.21.15.73:0/1341601614 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d402cb20 con 0x7f61dc15e850 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.574+0000 7f61db400640 1 -- 172.21.15.73:0/1341601614 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d402cc80 con 0x7f61dc15e850 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.574+0000 7f61db400640 1 -- 172.21.15.73:0/1341601614 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40343f0 con 0x7f61dc15e850 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 -- 172.21.15.73:0/1341601614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc15e850 msgr2=0x7f61dc15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 --2- 172.21.15.73:0/1341601614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc15e850 0x7f61dc15ed00 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009e50 tx=0x7f61d402bf80 comp rx=0 tx=0).stop 2024-04-05T22:32:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 -- 172.21.15.73:0/1341601614 shutdown_connections 2024-04-05T22:32:04.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 --2- 172.21.15.73:0/1341601614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc15e850 0x7f61dc15ed00 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 --2- 172.21.15.73:0/1341601614 >> v2:172.21.15.157:3300/0 conn(0x7f61dc1587d0 0x7f61dc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 -- 172.21.15.73:0/1341601614 >> 172.21.15.73:0/1341601614 conn(0x7f61dc0ce8e0 msgr2=0x7f61dc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:04.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 -- 172.21.15.73:0/1341601614 shutdown_connections 2024-04-05T22:32:04.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.575+0000 7f61e2e00640 1 -- 172.21.15.73:0/1341601614 wait complete. 2024-04-05T22:32:04.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.577+0000 7f61e2e00640 1 Processor -- start 2024-04-05T22:32:04.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.577+0000 7f61e2e00640 1 -- start start 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc15e850 0x7f61dc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61dc0b21a0 0x7f61dc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0b33a0 con 0x7f61dc1587d0 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0b2720 con 0x7f61dc15e850 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61e2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0b2a00 con 0x7f61dc0b21a0 2024-04-05T22:32:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33650/0 (socket says 172.21.15.73:33650) 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 -- 172.21.15.73:0/2154782315 learned_addr learned my addr 172.21.15.73:0/2154782315 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 -- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61dc0b21a0 msgr2=0x7f61dc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61dc0b21a0 0x7f61dc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 -- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc15e850 msgr2=0x7f61dc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61d3400640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc15e850 0x7f61dc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc15e850 0x7f61dc0b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.578+0000 7f61dbe00640 1 -- 172.21.15.73:0/2154782315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4009b30 con 0x7f61dc1587d0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61dbe00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d800 tx=0x7f61cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00b6c0 con 0x7f61dc1587d0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00b820 con 0x7f61dc1587d0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00bbb0 con 0x7f61dc1587d0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc0b2ce0 con 0x7f61dc1587d0 2024-04-05T22:32:04.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.579+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc142ac0 con 0x7f61dc1587d0 2024-04-05T22:32:04.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.580+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc066af0 con 0x7f61dc1587d0 2024-04-05T22:32:04.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.581+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f61cc00bd10 con 0x7f61dc1587d0 2024-04-05T22:32:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.583+0000 7f61d8c00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 0x7f61b806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.583+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61cc088250 con 0x7f61dc1587d0 2024-04-05T22:32:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.583+0000 7f61d3400640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 0x7f61b806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:04.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.584+0000 7f61d3400640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 0x7f61b806dc00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cf570 tx=0x7f61c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.587+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f61cc054e80 con 0x7f61dc1587d0 2024-04-05T22:32:04.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.782+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61dc068160 con 0x7f61dc1587d0 2024-04-05T22:32:04.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.785+0000 7f61d8c00640 1 -- 172.21.15.73:0/2154782315 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7f61cc0584a0 con 0x7f61dc1587d0 2024-04-05T22:32:04.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:04.786 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:32:04.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 msgr2=0x7f61b806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 0x7f61b806dc00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cf570 tx=0x7f61c4009040 comp rx=0 tx=0).stop 2024-04-05T22:32:04.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 msgr2=0x7f61dc142010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d800 tx=0x7f61cc00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 shutdown_connections 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f61b806b740 0x7f61b806dc00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61dc0b21a0 0x7f61dc0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc15e850 0x7f61dc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 --2- 172.21.15.73:0/2154782315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc1587d0 0x7f61dc142010 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 >> 172.21.15.73:0/2154782315 conn(0x7f61dc0ce8e0 msgr2=0x7f61dc15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 shutdown_connections 2024-04-05T22:32:04.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:04.788+0000 7f61e2e00640 1 -- 172.21.15.73:0/2154782315 wait complete. 2024-04-05T22:32:04.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-05T22:21:08.558000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-05T22:31:40.727092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-05T22:31:41.315098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-05T22:32:04.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-05T22:32:05.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.017+0000 7f985a000640 1 Processor -- start 2024-04-05T22:32:05.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.017+0000 7f985a000640 1 -- start start 2024-04-05T22:32:05.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f985a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f985415aa70 0x7f9854070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f985a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f985406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f985a000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985415ae50 con 0x7f98540714a0 2024-04-05T22:32:05.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f985a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9854071930 con 0x7f985415aa70 2024-04-05T22:32:05.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f985a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985415e3b0 con 0x7f985415dd90 2024-04-05T22:32:05.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f984be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f98540714a0 0x7f985415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45862/0 (socket says 172.21.15.73:45862) 2024-04-05T22:32:05.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.019+0000 7f984be00640 1 -- 172.21.15.73:0/3703112442 learned_addr learned my addr 172.21.15.73:0/3703112442 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:05.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9853400640 1 --2- 172.21.15.73:0/3703112442 >> v2:172.21.15.157:3300/0 conn(0x7f985415aa70 0x7f9854070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400906105 0 0) 0x7f985415ae50 con 0x7f98540714a0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9838003520 con 0x7f98540714a0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9853e00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f985406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496055702 0 0) 0x7f9838003520 con 0x7f98540714a0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 msgr2=0x7f985406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f985406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 >> v2:172.21.15.157:3300/0 conn(0x7f985415aa70 msgr2=0x7f9854070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 --2- 172.21.15.73:0/3703112442 >> v2:172.21.15.157:3300/0 conn(0x7f985415aa70 0x7f9854070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540ac9e0 con 0x7f98540714a0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f983c001670 con 0x7f98540714a0 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 >> v1:172.21.15.73:6789/0 conn(0x7f98540714a0 legacy=0x7f985415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838003970 0x7f9838003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f98380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838004d00 0x7f9838005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854071930 con 0x7f9838004d00 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985415e3b0 con 0x7f9838003970 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.020+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983800b980 con 0x7f985415dd90 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838004d00 0x7f9838005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f984be00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f98380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 -- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 msgr2=0x7f98380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415dd90 0x7f98380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 -- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838003970 msgr2=0x7f9838003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838003970 0x7f9838003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.021+0000 7f9853e00640 1 -- 172.21.15.73:0/3703112442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9838003520 con 0x7f9838004d00 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.022+0000 7f9853e00640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838004d00 0x7f9838005180 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f984c007390 tx=0x7f98380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d952b8ff2697a7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.022+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c02ccb0 con 0x7f9838004d00 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c02ce10 con 0x7f9838004d00 2024-04-05T22:32:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f9852a00640 1 -- 172.21.15.73:0/3703112442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c034660 con 0x7f9838004d00 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 -- 172.21.15.73:0/3703112442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838004d00 msgr2=0x7f9838005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 --2- 172.21.15.73:0/3703112442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838004d00 0x7f9838005180 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f984c007390 tx=0x7f98380057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f9853e00640 1 -- 172.21.15.73:0/3703112442 reap_dead start 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 -- 172.21.15.73:0/3703112442 shutdown_connections 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 -- 172.21.15.73:0/3703112442 >> 172.21.15.73:0/3703112442 conn(0x7f9854069cb0 msgr2=0x7f985406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 -- 172.21.15.73:0/3703112442 shutdown_connections 2024-04-05T22:32:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.023+0000 7f985a000640 1 -- 172.21.15.73:0/3703112442 wait complete. 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.025+0000 7f985a000640 1 Processor -- start 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.025+0000 7f985a000640 1 -- start start 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838004d00 0x7f9854152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9854157c70 0x7f98541580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854070f50 con 0x7f9854070590 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541569e0 con 0x7f9838004d00 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f985a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9854156cc0 con 0x7f9854157c70 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f984be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f984be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33660/0 (socket says 172.21.15.73:33660) 2024-04-05T22:32:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.026+0000 7f984be00640 1 -- 172.21.15.73:0/3377152209 learned_addr learned my addr 172.21.15.73:0/3377152209 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 -- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9854157c70 msgr2=0x7f98541580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9854157c70 0x7f98541580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 -- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838004d00 msgr2=0x7f9854152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838004d00 0x7f9854152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 -- 172.21.15.73:0/3377152209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854156fa0 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984be00640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d8e0 tx=0x7f983c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0074e0 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c007640 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c004a20 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854157280 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.027+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854152970 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.029+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9854068160 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.029+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f983c004b80 con 0x7f9854070590 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.031+0000 7f984b400640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 0x7f983006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.031+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f983c087ad0 con 0x7f9854070590 2024-04-05T22:32:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.031+0000 7f9853400640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 0x7f983006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.033+0000 7f9853400640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 0x7f983006dc30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9844006a90 tx=0x7f9844006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.035+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f983c054410 con 0x7f9854070590 2024-04-05T22:32:05.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.224+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98540ae1c0 con 0x7f9854070590 2024-04-05T22:32:05.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.225+0000 7f984b400640 1 -- 172.21.15.73:0/3377152209 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f983c0190a0 con 0x7f9854070590 2024-04-05T22:32:05.225 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 msgr2=0x7f983006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 0x7f983006dc30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9844006a90 tx=0x7f9844006a20 comp rx=0 tx=0).stop 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 msgr2=0x7f9854070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d8e0 tx=0x7f983c00dda0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 shutdown_connections 2024-04-05T22:32:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f983006b770 0x7f983006dc30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9854157c70 0x7f98541580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9854070590 0x7f9854070a10 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 --2- 172.21.15.73:0/3377152209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838004d00 0x7f9854152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 >> 172.21.15.73:0/3377152209 conn(0x7f9854069cb0 msgr2=0x7f98541552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 shutdown_connections 2024-04-05T22:32:05.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.228+0000 7f985a000640 1 -- 172.21.15.73:0/3377152209 wait complete. 2024-04-05T22:32:05.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-05T22:32:05.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.456+0000 7f5be8400640 1 Processor -- start 2024-04-05T22:32:05.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.456+0000 7f5be8400640 1 -- start start 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be8400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5be006b530 0x7f5be00adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 0x7f5be0162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be8400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00b2ff0 con 0x7f5be00b2190 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be8400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be01597b0 con 0x7f5be006b530 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01599a0 con 0x7f5be00ae0b0 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be5a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5be00b2190 0x7f5be00b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45872/0 (socket says 172.21.15.73:45872) 2024-04-05T22:32:05.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be5a00640 1 -- 172.21.15.73:0/2776471447 learned_addr learned my addr 172.21.15.73:0/2776471447 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.458+0000 7f5be5000640 1 --2- 172.21.15.73:0/2776471447 >> v2:172.21.15.73:3300/0 conn(0x7f5be006b530 0x7f5be00adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2776471447 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236087971 0 0) 0x7f5be00b2ff0 con 0x7f5be00b2190 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be6400640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 0x7f5be0162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2776471447 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc8005730 con 0x7f5be00b2190 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be5000640 1 -- 172.21.15.73:0/2776471447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 msgr2=0x7f5be0162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be5000640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 0x7f5be0162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be5000640 1 -- 172.21.15.73:0/2776471447 >> v1:172.21.15.73:6789/0 conn(0x7f5be00b2190 legacy=0x7f5be00b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be5000640 1 -- 172.21.15.73:0/2776471447 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0159c80 con 0x7f5be006b530 2024-04-05T22:32:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.459+0000 7f5be6400640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 0x7f5be0162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.460+0000 7f5be5000640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be006b530 0x7f5be00adb70 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0097a0 tx=0x7f5bcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1919f3d189450a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.460+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2776471447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc02e070 con 0x7f5be006b530 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.460+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2776471447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0315a0 con 0x7f5be006b530 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.460+0000 7f5bdfe00640 1 -- 172.21.15.73:0/2776471447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc031910 con 0x7f5be006b530 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 -- 172.21.15.73:0/2776471447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be006b530 msgr2=0x7f5be00adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be006b530 0x7f5be00adb70 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0097a0 tx=0x7f5bcc02b840 comp rx=0 tx=0).stop 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 -- 172.21.15.73:0/2776471447 shutdown_connections 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00ae0b0 0x7f5be0162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 --2- 172.21.15.73:0/2776471447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be006b530 0x7f5be00adb70 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 -- 172.21.15.73:0/2776471447 >> 172.21.15.73:0/2776471447 conn(0x7f5be0069cb0 msgr2=0x7f5be006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 -- 172.21.15.73:0/2776471447 shutdown_connections 2024-04-05T22:32:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.461+0000 7f5be8400640 1 -- 172.21.15.73:0/2776471447 wait complete. 2024-04-05T22:32:05.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.463+0000 7f5be8400640 1 Processor -- start 2024-04-05T22:32:05.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.463+0000 7f5be8400640 1 -- start start 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b530 0x7f5be0140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00b2190 0x7f5be0141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be014e130 con 0x7f5be00ae0b0 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0150840 con 0x7f5be006b530 2024-04-05T22:32:05.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.464+0000 7f5be8400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0150b20 con 0x7f5be00b2190 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33672/0 (socket says 172.21.15.73:33672) 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 -- 172.21.15.73:0/3741159597 learned_addr learned my addr 172.21.15.73:0/3741159597 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 -- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00b2190 msgr2=0x7f5be0141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00b2190 0x7f5be0141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 -- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b530 msgr2=0x7f5be0140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b530 0x7f5be0140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 -- 172.21.15.73:0/3741159597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc009400 con 0x7f5be00ae0b0 2024-04-05T22:32:05.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.465+0000 7f5be5000640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc034040 tx=0x7f5bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.466+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc032630 con 0x7f5be00ae0b0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.466+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc032790 con 0x7f5be00ae0b0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.466+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc032b00 con 0x7f5be00ae0b0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.466+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be0151c00 con 0x7f5be00ae0b0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.466+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be0140fe0 con 0x7f5be00ae0b0 2024-04-05T22:32:05.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.467+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0152120 con 0x7f5be00ae0b0 2024-04-05T22:32:05.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.468+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f5bcc032c60 con 0x7f5be00ae0b0 2024-04-05T22:32:05.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.469+0000 7f5bdd600640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 0x7f5bc006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.469+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bcc0a9c40 con 0x7f5be00ae0b0 2024-04-05T22:32:05.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.469+0000 7f5be5a00640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 0x7f5bc006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.471+0000 7f5be5a00640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 0x7f5bc006dc30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd40078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.473+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5bcc076870 con 0x7f5be00ae0b0 2024-04-05T22:32:05.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.653+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5be00b2f70 con 0x7f5be00ae0b0 2024-04-05T22:32:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.653+0000 7f5bdd600640 1 -- 172.21.15.73:0/3741159597 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5bcc079e90 con 0x7f5be00ae0b0 2024-04-05T22:32:05.654 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-05T22:32:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 msgr2=0x7f5bc006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 0x7f5bc006dc30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd40078f0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 msgr2=0x7f5be0152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc034040 tx=0x7f5bcc0046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 shutdown_connections 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f5bc006b770 0x7f5bc006dc30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00b2190 0x7f5be0141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be00ae0b0 0x7f5be0152400 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 --2- 172.21.15.73:0/3741159597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be006b530 0x7f5be0140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 >> 172.21.15.73:0/3741159597 conn(0x7f5be0069cb0 msgr2=0x7f5be00ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 shutdown_connections 2024-04-05T22:32:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.656+0000 7f5be8400640 1 -- 172.21.15.73:0/3741159597 wait complete. 2024-04-05T22:32:05.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-05T22:32:05.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.893+0000 7fca8ba00640 1 Processor -- start 2024-04-05T22:32:05.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.894+0000 7fca8ba00640 1 -- start start 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.895+0000 7fca8ba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca840b32b0 0x7fca841657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.895+0000 7fca8ba00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca840b1b30 con 0x7fca840cbe00 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.895+0000 7fca8ba00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca840b1e10 con 0x7fca8406c630 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.895+0000 7fca8ba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca840ce7b0 con 0x7fca840b32b0 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca83e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fca840cbe00 0x7fca840ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57734/0 (socket says 172.21.15.73:57734) 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca83e00640 1 -- 172.21.15.73:0/1627419424 learned_addr learned my addr 172.21.15.73:0/1627419424 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca89a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca840b32b0 0x7fca841657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca83400640 1 -- 172.21.15.73:0/1627419424 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520273838 0 0) 0x7fca840b1b30 con 0x7fca840cbe00 2024-04-05T22:32:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca83400640 1 -- 172.21.15.73:0/1627419424 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca6c003520 con 0x7fca840cbe00 2024-04-05T22:32:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca89a00640 1 -- 172.21.15.73:0/1627419424 >> v1:172.21.15.157:6789/0 conn(0x7fca8406c630 legacy=0x7fca8406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca89a00640 1 -- 172.21.15.73:0/1627419424 >> v1:172.21.15.73:6789/0 conn(0x7fca840cbe00 legacy=0x7fca840ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.896+0000 7fca89a00640 1 -- 172.21.15.73:0/1627419424 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840cc300 con 0x7fca840b32b0 2024-04-05T22:32:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.898+0000 7fca89a00640 1 --2- 172.21.15.73:0/1627419424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b32b0 0x7fca841657d0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009d90 tx=0x7fca7c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b54c193458fd36b server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.898+0000 7fca83400640 1 -- 172.21.15.73:0/1627419424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c02cd00 con 0x7fca840b32b0 2024-04-05T22:32:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.898+0000 7fca83400640 1 -- 172.21.15.73:0/1627419424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c02ce60 con 0x7fca840b32b0 2024-04-05T22:32:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.898+0000 7fca83400640 1 -- 172.21.15.73:0/1627419424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c034640 con 0x7fca840b32b0 2024-04-05T22:32:05.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 -- 172.21.15.73:0/1627419424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b32b0 msgr2=0x7fca841657d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 --2- 172.21.15.73:0/1627419424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b32b0 0x7fca841657d0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009d90 tx=0x7fca7c02bf50 comp rx=0 tx=0).stop 2024-04-05T22:32:05.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 -- 172.21.15.73:0/1627419424 shutdown_connections 2024-04-05T22:32:05.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 --2- 172.21.15.73:0/1627419424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b32b0 0x7fca841657d0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 -- 172.21.15.73:0/1627419424 >> 172.21.15.73:0/1627419424 conn(0x7fca840b0830 msgr2=0x7fca840b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:05.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 -- 172.21.15.73:0/1627419424 shutdown_connections 2024-04-05T22:32:05.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.899+0000 7fca8ba00640 1 -- 172.21.15.73:0/1627419424 wait complete. 2024-04-05T22:32:05.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.901+0000 7fca8ba00640 1 Processor -- start 2024-04-05T22:32:05.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.901+0000 7fca8ba00640 1 -- start start 2024-04-05T22:32:05.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 0x7fca8415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 0x7fca84132b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca841304d0 con 0x7fca840cbe00 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8412f640 con 0x7fca840b32b0 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.902+0000 7fca8ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8415aa60 con 0x7fca8406c630 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60620/0 (socket says 172.21.15.73:60620) 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 -- 172.21.15.73:0/3811663380 learned_addr learned my addr 172.21.15.73:0/3811663380 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89000640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 0x7fca8415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca83e00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 0x7fca84132b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 -- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 msgr2=0x7fca8415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 0x7fca8415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 -- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 msgr2=0x7fca84132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 0x7fca84132b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89a00640 1 -- 172.21.15.73:0/3811663380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c009a70 con 0x7fca840cbe00 2024-04-05T22:32:05.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca83e00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 0x7fca84132b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:05.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.903+0000 7fca89000640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 0x7fca8415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:05.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca89a00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009d90 tx=0x7fca7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c003b90 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c003cf0 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004250 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca84133070 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.904+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8412fb30 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.905+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca84130610 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.906+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fca7c03b050 con 0x7fca840cbe00 2024-04-05T22:32:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.907+0000 7fca80c00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 0x7fca6406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:05.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.907+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca7c0aa4d0 con 0x7fca840cbe00 2024-04-05T22:32:05.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.907+0000 7fca89000640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 0x7fca6406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:05.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.909+0000 7fca89000640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 0x7fca6406dc00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fca78001680 tx=0x7fca78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:05.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:05.911+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fca84130610 con 0x7fca840cbe00 2024-04-05T22:32:06.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.106+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca84130610 con 0x7fca840cbe00 2024-04-05T22:32:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.107+0000 7fca80c00640 1 -- 172.21.15.73:0/3811663380 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca84130610 con 0x7fca840cbe00 2024-04-05T22:32:06.107 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-05T22:32:06.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 msgr2=0x7fca6406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 0x7fca6406dc00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fca78001680 tx=0x7fca78009040 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 msgr2=0x7fca840a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009d90 tx=0x7fca7c004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 shutdown_connections 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fca6406b740 0x7fca6406dc00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840cbe00 0x7fca840a9500 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca840b32b0 0x7fca84132b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 --2- 172.21.15.73:0/3811663380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8406c630 0x7fca8415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 >> 172.21.15.73:0/3811663380 conn(0x7fca840b0830 msgr2=0x7fca8415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 shutdown_connections 2024-04-05T22:32:06.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.110+0000 7fca8ba00640 1 -- 172.21.15.73:0/3811663380 wait complete. 2024-04-05T22:32:06.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:32:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.338+0000 7fbe22600640 1 Processor -- start 2024-04-05T22:32:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.338+0000 7fbe22600640 1 -- start start 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.339+0000 7fbe22600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe1c15e270 0x7fbe1c065b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe22600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 0x7fbe1c069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe22600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c161cd0 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe22600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c06b230 con 0x7fbe1c15e270 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe22600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c06a600 con 0x7fbe1c0cd440 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbe1c06a1f0 0x7fbe1c0cf180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57742/0 (socket says 172.21.15.73:57742) 2024-04-05T22:32:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1be00640 1 -- 172.21.15.73:0/1847067973 learned_addr learned my addr 172.21.15.73:0/1847067973 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1aa00640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 0x7fbe1c069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532498583 0 0) 0x7fbe1c161cd0 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe00003520 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.340+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1847067973 >> v2:172.21.15.157:3300/0 conn(0x7fbe1c15e270 0x7fbe1c065b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460975186 0 0) 0x7fbe00003520 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 msgr2=0x7fbe1c069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 0x7fbe1c069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 >> v2:172.21.15.157:3300/0 conn(0x7fbe1c15e270 msgr2=0x7fbe1c065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 --2- 172.21.15.73:0/1847067973 >> v2:172.21.15.157:3300/0 conn(0x7fbe1c15e270 0x7fbe1c065b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c1610e0 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbe140019f0 con 0x7fbe1c06a1f0 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 >> v1:172.21.15.73:6789/0 conn(0x7fbe1c06a1f0 legacy=0x7fbe1c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 0x7fbe000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe00004d00 0x7fbe00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c06a600 con 0x7fbe00003970 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000b9b0 con 0x7fbe00004d00 2024-04-05T22:32:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.341+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000bb70 con 0x7fbe1c0cd440 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1be00640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe00004d00 0x7fbe00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 -- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 msgr2=0x7fbe000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0cd440 0x7fbe000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 -- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe00004d00 msgr2=0x7fbe00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe00004d00 0x7fbe00005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1b400640 1 -- 172.21.15.73:0/1847067973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00003520 con 0x7fbe00003970 2024-04-05T22:32:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.342+0000 7fbe1be00640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe00004d00 0x7fbe00005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.343+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe00003d90 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00ba30 tx=0x7fbe000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf4ed79b4547fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c032680 con 0x7fbe00003970 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c0327e0 con 0x7fbe00003970 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe1a000640 1 -- 172.21.15.73:0/1847067973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c032b70 con 0x7fbe00003970 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 -- 172.21.15.73:0/1847067973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 msgr2=0x7fbe00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 --2- 172.21.15.73:0/1847067973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe00003d90 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00ba30 tx=0x7fbe000057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe1be00640 1 -- 172.21.15.73:0/1847067973 reap_dead start 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 -- 172.21.15.73:0/1847067973 shutdown_connections 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 -- 172.21.15.73:0/1847067973 >> 172.21.15.73:0/1847067973 conn(0x7fbe1c071250 msgr2=0x7fbe1c071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 -- 172.21.15.73:0/1847067973 shutdown_connections 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.344+0000 7fbe22600640 1 -- 172.21.15.73:0/1847067973 wait complete. 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.346+0000 7fbe22600640 1 Processor -- start 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.346+0000 7fbe22600640 1 -- start start 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0c6ee0 0x7fbe1c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe1c06d700 0x7fbe1c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c15b3c0 con 0x7fbe00003970 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c15b6a0 con 0x7fbe1c06d700 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.347+0000 7fbe22600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c15b980 con 0x7fbe1c0c6ee0 2024-04-05T22:32:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60638/0 (socket says 172.21.15.73:60638) 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 -- 172.21.15.73:0/1118220734 learned_addr learned my addr 172.21.15.73:0/1118220734 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1aa00640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0c6ee0 0x7fbe1c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 -- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0c6ee0 msgr2=0x7fbe1c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0c6ee0 0x7fbe1c0c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 -- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe1c06d700 msgr2=0x7fbe1c15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe1c06d700 0x7fbe1c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 -- 172.21.15.73:0/1118220734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c15bc60 con 0x7fbe00003970 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.348+0000 7fbe1b400640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02bf00 tx=0x7fbe0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.349+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c004220 con 0x7fbe00003970 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.349+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c0043a0 con 0x7fbe00003970 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.349+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c030aa0 con 0x7fbe00003970 2024-04-05T22:32:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.349+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c15bf40 con 0x7fbe00003970 2024-04-05T22:32:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.349+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c0c8f50 con 0x7fbe00003970 2024-04-05T22:32:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.350+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c0c9110 con 0x7fbe00003970 2024-04-05T22:32:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.351+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fbe0c030c00 con 0x7fbe00003970 2024-04-05T22:32:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.352+0000 7fbe13400640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 0x7fbdfc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.352+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe0c0aa630 con 0x7fbe00003970 2024-04-05T22:32:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.352+0000 7fbe1aa00640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 0x7fbdfc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.353+0000 7fbe1aa00640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 0x7fbdfc06dc30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c06c160 tx=0x7fbe08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.356+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbe0c0e0650 con 0x7fbe00003970 2024-04-05T22:32:06.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.587+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1c0c9110 con 0x7fbe00003970 2024-04-05T22:32:06.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.588+0000 7fbe13400640 1 -- 172.21.15.73:0/1118220734 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4753 (secure 0 0 0) 0x7fbe0c0772c0 con 0x7fbe00003970 2024-04-05T22:32:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:06.589 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:32:06.589 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 110 2024-04-05T22:32:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.590+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 msgr2=0x7fbdfc06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.590+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 0x7fbdfc06dc30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c06c160 tx=0x7fbe08008040 comp rx=0 tx=0).stop 2024-04-05T22:32:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.590+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 msgr2=0x7fbe1c152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.590+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02bf00 tx=0x7fbe0c004730 comp rx=0 tx=0).stop 2024-04-05T22:32:06.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 shutdown_connections 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbdfc06b770 0x7fbdfc06dc30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe1c06d700 0x7fbe1c15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe1c0c6ee0 0x7fbe1c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 --2- 172.21.15.73:0/1118220734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe00003970 0x7fbe1c152fd0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 >> 172.21.15.73:0/1118220734 conn(0x7fbe1c071250 msgr2=0x7fbe1c15ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 shutdown_connections 2024-04-05T22:32:06.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.591+0000 7fbe22600640 1 -- 172.21.15.73:0/1118220734 wait complete. 2024-04-05T22:32:06.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:32:06.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.811+0000 7f79b5000640 1 Processor -- start 2024-04-05T22:32:06.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.812+0000 7f79b5000640 1 -- start start 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.813+0000 7f79b5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f79b006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.813+0000 7f79b5000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006f5b0 con 0x7f79b015e850 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.813+0000 7f79b5000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b0070410 con 0x7f79b01587d0 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.813+0000 7f79b5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b006d500 con 0x7f79b006cb80 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ae800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f79b015e850 0x7f79b00708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57744/0 (socket says 172.21.15.73:57744) 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ae800640 1 -- 172.21.15.73:0/2696286894 learned_addr learned my addr 172.21.15.73:0/2696286894 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79a7e00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f79b006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120509637 0 0) 0x7f79b006f5b0 con 0x7f79b015e850 2024-04-05T22:32:06.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7994003520 con 0x7f79b015e850 2024-04-05T22:32:06.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104171121 0 0) 0x7f7994003520 con 0x7f79b015e850 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 msgr2=0x7f79b006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f79b006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 >> v1:172.21.15.157:6789/0 conn(0x7f79b01587d0 legacy=0x7f79b00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00ccef0 con 0x7f79b015e850 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f79a80019f0 con 0x7f79b015e850 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.814+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 >> v1:172.21.15.73:6789/0 conn(0x7f79b015e850 legacy=0x7f79b00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f7994003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79940047c0 0x7f7994004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f7994005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b006d500 con 0x7f79940039a0 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799400ba30 con 0x7f79940047c0 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799400bbf0 con 0x7f79b006cb80 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f7994003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ae800640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f7994005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 -- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 msgr2=0x7f7994005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79a7e00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79940047c0 0x7f7994004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f7994005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 -- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79940047c0 msgr2=0x7f7994004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79940047c0 0x7f7994004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ade00640 1 -- 172.21.15.73:0/2696286894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994003520 con 0x7f79940039a0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.815+0000 7f79ae800640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b006cb80 0x7f7994005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.816+0000 7f79ade00640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f7994003dc0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001720 tx=0x7f79940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508fd1b461e354e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00052d0 con 0x7f79940039a0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a002e050 con 0x7f79940039a0 2024-04-05T22:32:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79ad400640 1 -- 172.21.15.73:0/2696286894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00035f0 con 0x7f79940039a0 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 -- 172.21.15.73:0/2696286894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 msgr2=0x7f7994003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 --2- 172.21.15.73:0/2696286894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f7994003dc0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001720 tx=0x7f79940057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79ae800640 1 -- 172.21.15.73:0/2696286894 reap_dead start 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 -- 172.21.15.73:0/2696286894 shutdown_connections 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 -- 172.21.15.73:0/2696286894 >> 172.21.15.73:0/2696286894 conn(0x7f79b00ce8e0 msgr2=0x7f79b00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 -- 172.21.15.73:0/2696286894 shutdown_connections 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.817+0000 7f79b5000640 1 -- 172.21.15.73:0/2696286894 wait complete. 2024-04-05T22:32:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.819+0000 7f79b5000640 1 Processor -- start 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.819+0000 7f79b5000640 1 -- start start 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 0x7f79b00acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 0x7f79b00b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ad290 con 0x7f79940039a0 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00b2910 con 0x7f79b00ac8d0 2024-04-05T22:32:06.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79b5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00b2bf0 con 0x7f79b00b1ba0 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60656/0 (socket says 172.21.15.73:60656) 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.820+0000 7f79ade00640 1 -- 172.21.15.73:0/3634918643 learned_addr learned my addr 172.21.15.73:0/3634918643 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79a7e00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 0x7f79b00acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ae800640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 0x7f79b00b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 -- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 msgr2=0x7f79b00b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 0x7f79b00b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 -- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 msgr2=0x7f79b00acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 0x7f79b00acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 -- 172.21.15.73:0/3634918643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00b2ed0 con 0x7f79940039a0 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79a7e00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 0x7f79b00acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:06.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ae800640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 0x7f79b00b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79ade00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f79a0008fd0 tx=0x7f79a0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00076c0 con 0x7f79940039a0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0007a30 con 0x7f79940039a0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0006be0 con 0x7f79940039a0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b00b31b0 con 0x7f79940039a0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.821+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b014e9d0 con 0x7f79940039a0 2024-04-05T22:32:06.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.822+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b0066af0 con 0x7f79940039a0 2024-04-05T22:32:06.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.823+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f79a0006d40 con 0x7f79940039a0 2024-04-05T22:32:06.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.824+0000 7f79a6000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 0x7f798c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:06.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.824+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79a00aa060 con 0x7f79940039a0 2024-04-05T22:32:06.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.825+0000 7f79a7e00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 0x7f798c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:06.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.826+0000 7f79a7e00640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 0x7f798c06dc30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7998006ce0 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:06.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:06.828+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f79a0076b60 con 0x7f79940039a0 2024-04-05T22:32:07.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.008+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b00ad290 con 0x7f79940039a0 2024-04-05T22:32:07.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.010+0000 7f79a6000640 1 -- 172.21.15.73:0/3634918643 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7f79b00ad290 con 0x7f79940039a0 2024-04-05T22:32:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:07.011 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:32:07.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 msgr2=0x7f798c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 0x7f798c06dc30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7998006ce0 tx=0x7f7998008040 comp rx=0 tx=0).stop 2024-04-05T22:32:07.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 msgr2=0x7f79b014e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f79a0008fd0 tx=0x7f79a0001120 comp rx=0 tx=0).stop 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 shutdown_connections 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f798c06b770 0x7f798c06dc30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79b00b1ba0 0x7f79b00b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ac8d0 0x7f79b00acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 --2- 172.21.15.73:0/3634918643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79940039a0 0x7f79b014e490 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 >> 172.21.15.73:0/3634918643 conn(0x7f79b00ce8e0 msgr2=0x7f79b0159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 shutdown_connections 2024-04-05T22:32:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.013+0000 7f79b5000640 1 -- 172.21.15.73:0/3634918643 wait complete. 2024-04-05T22:32:07.031 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:32:07.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:32:07.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.223+0000 7ff4e4a00640 1 Processor -- start 2024-04-05T22:32:07.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.223+0000 7ff4e4a00640 1 -- start start 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4e4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4e01541a0 0x7ff4e00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4e4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4e4a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e015aa60 con 0x7ff4e006d710 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4e4a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e00b3b60 con 0x7ff4e01541a0 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4e4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00701c0 con 0x7ff4e00ce000 2024-04-05T22:32:07.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4dde00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4e01541a0 0x7ff4e00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4dde00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4e01541a0 0x7ff4e00b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60668/0 (socket says 172.21.15.73:60668) 2024-04-05T22:32:07.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.225+0000 7ff4dde00640 1 -- 172.21.15.73:0/2223704767 learned_addr learned my addr 172.21.15.73:0/2223704767 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:07.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.226+0000 7ff4de800640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.226+0000 7ff4dde00640 1 -- 172.21.15.73:0/2223704767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 msgr2=0x7ff4e015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:07.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.226+0000 7ff4dde00640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.226+0000 7ff4dde00640 1 -- 172.21.15.73:0/2223704767 >> v1:172.21.15.73:6789/0 conn(0x7ff4e006d710 legacy=0x7ff4e00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.226+0000 7ff4dde00640 1 -- 172.21.15.73:0/2223704767 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00b3880 con 0x7ff4e01541a0 2024-04-05T22:32:07.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.227+0000 7ff4dde00640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b3050 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000bb10 tx=0x7ff4d002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1afb5f9a6f88a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.227+0000 7ff4dd400640 1 -- 172.21.15.73:0/2223704767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0036440 con 0x7ff4e01541a0 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.227+0000 7ff4dd400640 1 -- 172.21.15.73:0/2223704767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d00365a0 con 0x7ff4e01541a0 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.227+0000 7ff4dd400640 1 -- 172.21.15.73:0/2223704767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0036930 con 0x7ff4e01541a0 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 -- 172.21.15.73:0/2223704767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 msgr2=0x7ff4e00b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b3050 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000bb10 tx=0x7ff4d002e150 comp rx=0 tx=0).stop 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 -- 172.21.15.73:0/2223704767 shutdown_connections 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/2223704767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b3050 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 -- 172.21.15.73:0/2223704767 >> 172.21.15.73:0/2223704767 conn(0x7ff4e00ce8e0 msgr2=0x7ff4e00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 -- 172.21.15.73:0/2223704767 shutdown_connections 2024-04-05T22:32:07.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.228+0000 7ff4e4a00640 1 -- 172.21.15.73:0/2223704767 wait complete. 2024-04-05T22:32:07.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.229+0000 7ff4e4a00640 1 Processor -- start 2024-04-05T22:32:07.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 -- start start 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e006d710 0x7ff4e00ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e00b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0140a80 con 0x7ff4e01541a0 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0143000 con 0x7ff4e006d710 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.230+0000 7ff4e4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e01403c0 con 0x7ff4e00ce000 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60674/0 (socket says 172.21.15.73:60674) 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 -- 172.21.15.73:0/1291897083 learned_addr learned my addr 172.21.15.73:0/1291897083 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 -- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 msgr2=0x7ff4e00b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e00b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 -- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e006d710 msgr2=0x7ff4e00ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e006d710 0x7ff4e00ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 -- 172.21.15.73:0/1291897083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d000b770 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.231+0000 7ff4de800640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800d800 tx=0x7ff4d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.232+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800b6c0 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.232+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d800b840 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.232+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e01406a0 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.232+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800bbf0 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.232+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e00acee0 con 0x7ff4e01541a0 2024-04-05T22:32:07.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.233+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e0066af0 con 0x7ff4e01541a0 2024-04-05T22:32:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.234+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7ff4d800bd50 con 0x7ff4e01541a0 2024-04-05T22:32:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.235+0000 7ff4d6a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 0x7ff4bc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.235+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4d8088240 con 0x7ff4e01541a0 2024-04-05T22:32:07.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.235+0000 7ff4dde00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 0x7ff4bc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.237+0000 7ff4dde00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 0x7ff4bc06dc30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0038000 tx=0x7ff4d0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.240+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff4d8054e70 con 0x7ff4e01541a0 2024-04-05T22:32:07.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.453+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4e0067db0 con 0x7ff4e01541a0 2024-04-05T22:32:07.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.454+0000 7ff4d6a00640 1 -- 172.21.15.73:0/1291897083 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4753 (secure 0 0 0) 0x7ff4d8058490 con 0x7ff4e01541a0 2024-04-05T22:32:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:07.455 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:32:07.455 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 110 2024-04-05T22:32:07.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 msgr2=0x7ff4bc06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 0x7ff4bc06dc30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0038000 tx=0x7ff4d0030000 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 msgr2=0x7ff4e00b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800d800 tx=0x7ff4d800dcc0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 shutdown_connections 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff4bc06b770 0x7ff4bc06dc30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01541a0 0x7ff4e00b22d0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e00ce000 0x7ff4e00b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 --2- 172.21.15.73:0/1291897083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e006d710 0x7ff4e00ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 >> 172.21.15.73:0/1291897083 conn(0x7ff4e00ce8e0 msgr2=0x7ff4e00cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.457+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 shutdown_connections 2024-04-05T22:32:07.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.458+0000 7ff4e4a00640 1 -- 172.21.15.73:0/1291897083 wait complete. 2024-04-05T22:32:07.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:32:07.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.680+0000 7f00b6200640 1 Processor -- start 2024-04-05T22:32:07.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.680+0000 7f00b6200640 1 -- start start 2024-04-05T22:32:07.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.682+0000 7f00b6200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 0x7f00b006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.682+0000 7f00b6200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.682+0000 7f00b6200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b006f5b0 con 0x7f00b01587d0 2024-04-05T22:32:07.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.682+0000 7f00b6200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b006d500 con 0x7f00b006cb80 2024-04-05T22:32:07.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.682+0000 7f00b6200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b015f220 con 0x7f00b015e850 2024-04-05T22:32:07.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.683+0000 7f00afe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.683+0000 7f00a6a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 0x7f00b006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.683+0000 7f00afe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:56096/0 (socket says 172.21.15.73:56096) 2024-04-05T22:32:07.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.683+0000 7f00afe00640 1 -- 172.21.15.73:0/2399603465 learned_addr learned my addr 172.21.15.73:0/2399603465 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:07.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.683+0000 7f00af400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f00b01587d0 0x7f00b00cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:34402/0 (socket says 172.21.15.73:34402) 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00afe00640 1 -- 172.21.15.73:0/2399603465 >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 msgr2=0x7f00b006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00afe00640 1 --2- 172.21.15.73:0/2399603465 >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 0x7f00b006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00afe00640 1 -- 172.21.15.73:0/2399603465 >> v1:172.21.15.157:6789/0 conn(0x7f00b01587d0 legacy=0x7f00b00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00afe00640 1 -- 172.21.15.73:0/2399603465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b0070410 con 0x7f00b015e850 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00a6a00640 1 --2- 172.21.15.73:0/2399603465 >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 0x7f00b006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-05T22:32:07.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.684+0000 7f00aea00640 1 -- 172.21.15.73:0/2399603465 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830287240 0 0) 0x7f00b006f5b0 con 0x7f00b01587d0 2024-04-05T22:32:07.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.685+0000 7f00afe00640 1 --2- 172.21.15.73:0/2399603465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009e50 tx=0x7f00a802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f8930f4f1bd681 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.686+0000 7f00aea00640 1 -- 172.21.15.73:0/2399603465 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a802cb00 con 0x7f00b015e850 2024-04-05T22:32:07.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.686+0000 7f00aea00640 1 -- 172.21.15.73:0/2399603465 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a802cc60 con 0x7f00b015e850 2024-04-05T22:32:07.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.686+0000 7f00aea00640 1 -- 172.21.15.73:0/2399603465 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a80343f0 con 0x7f00b015e850 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 -- 172.21.15.73:0/2399603465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 msgr2=0x7f00b015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 --2- 172.21.15.73:0/2399603465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009e50 tx=0x7f00a802bf60 comp rx=0 tx=0).stop 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 -- 172.21.15.73:0/2399603465 shutdown_connections 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 --2- 172.21.15.73:0/2399603465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b015e850 0x7f00b015ece0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 --2- 172.21.15.73:0/2399603465 >> v2:172.21.15.157:3300/0 conn(0x7f00b006cb80 0x7f00b006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 -- 172.21.15.73:0/2399603465 >> 172.21.15.73:0/2399603465 conn(0x7f00b00ce8e0 msgr2=0x7f00b00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 -- 172.21.15.73:0/2399603465 shutdown_connections 2024-04-05T22:32:07.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.687+0000 7f00b6200640 1 -- 172.21.15.73:0/2399603465 wait complete. 2024-04-05T22:32:07.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.688+0000 7f00b6200640 1 Processor -- start 2024-04-05T22:32:07.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.688+0000 7f00b6200640 1 -- start start 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b006cb80 0x7f00b00aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b01587d0 0x7f00b0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00addd0 con 0x7f00b015e850 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00ae050 con 0x7f00b01587d0 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.689+0000 7f00b6200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00ae330 con 0x7f00b006cb80 2024-04-05T22:32:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60682/0 (socket says 172.21.15.73:60682) 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 -- 172.21.15.73:0/2011638650 learned_addr learned my addr 172.21.15.73:0/2011638650 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 -- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b006cb80 msgr2=0x7f00b00aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b006cb80 0x7f00b00aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 -- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b01587d0 msgr2=0x7f00b0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00a6a00640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b01587d0 0x7f00b0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b01587d0 0x7f00b0153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.690+0000 7f00afe00640 1 -- 172.21.15.73:0/2011638650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0098009400 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00afe00640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f00a8036040 tx=0x7f00a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a803e430 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a803e590 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a8009b30 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a803e940 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.691+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b00ae8f0 con 0x7f00b015e850 2024-04-05T22:32:07.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.692+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b0066af0 con 0x7f00b015e850 2024-04-05T22:32:07.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.693+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f00a803eaa0 con 0x7f00b015e850 2024-04-05T22:32:07.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.694+0000 7f00a7e00640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 0x7f008c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:07.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.695+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00a80aadb0 con 0x7f00b015e850 2024-04-05T22:32:07.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.695+0000 7f00af400640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 0x7f008c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.696+0000 7f00af400640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 0x7f008c06dc00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007fd0 tx=0x7f00a0007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.699+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f00a8077790 con 0x7f00b015e850 2024-04-05T22:32:07.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.937+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00b00aeae0 con 0x7f00b015e850 2024-04-05T22:32:07.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.938+0000 7f00a7e00640 1 -- 172.21.15.73:0/2011638650 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4753 (secure 0 0 0) 0x7f00a807adb0 con 0x7f00b015e850 2024-04-05T22:32:07.939 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:07.939 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-05T22:31:42.297895+0000","modified":"2024-04-05T22:31:45.322696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10179":{"gid":10179,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6834/1107789803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1107789803},{"type":"v1","addr":"172.21.15.73:6834","nonce":1107789803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-05T22:32:07.939 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 110 2024-04-05T22:32:07.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 msgr2=0x7f008c06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 0x7f008c06dc00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007fd0 tx=0x7f00a0007920 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 msgr2=0x7f00b00a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f00a8036040 tx=0x7f00a8004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 shutdown_connections 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f008c06b740 0x7f008c06dc00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b015e850 0x7f00b00a86d0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b01587d0 0x7f00b0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 --2- 172.21.15.73:0/2011638650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00b006cb80 0x7f00b00aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 >> 172.21.15.73:0/2011638650 conn(0x7f00b00ce8e0 msgr2=0x7f00b00ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:07.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 shutdown_connections 2024-04-05T22:32:07.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:07.941+0000 7f00b6200640 1 -- 172.21.15.73:0/2011638650 wait complete. 2024-04-05T22:32:07.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:32:08.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.169+0000 7fbb2ce00640 1 Processor -- start 2024-04-05T22:32:08.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.169+0000 7fbb2ce00640 1 -- start start 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb2ce00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb2806d710 0x7fbb280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb2ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2815aa60 con 0x7fbb2815aef0 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb2ce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280b3b60 con 0x7fbb281541a0 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb2ce00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280701c0 con 0x7fbb2806d710 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb26800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbb2815aef0 0x7fbb2815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57754/0 (socket says 172.21.15.73:57754) 2024-04-05T22:32:08.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.171+0000 7fbb26800640 1 -- 172.21.15.73:0/586673963 learned_addr learned my addr 172.21.15.73:0/586673963 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881616825 0 0) 0x7fbb2815aa60 con 0x7fbb2815aef0 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c003520 con 0x7fbb2815aef0 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb1d400640 1 --2- 172.21.15.73:0/586673963 >> v2:172.21.15.157:3300/0 conn(0x7fbb2806d710 0x7fbb280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920689995 0 0) 0x7fbb0c003520 con 0x7fbb2815aef0 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 >> v2:172.21.15.157:3300/0 conn(0x7fbb2806d710 msgr2=0x7fbb280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 --2- 172.21.15.73:0/586673963 >> v2:172.21.15.157:3300/0 conn(0x7fbb2806d710 0x7fbb280cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 >> v1:172.21.15.157:6789/0 conn(0x7fbb281541a0 legacy=0x7fbb280b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280b3880 con 0x7fbb2815aef0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbb20001670 con 0x7fbb2815aef0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 >> v1:172.21.15.73:6789/0 conn(0x7fbb2815aef0 legacy=0x7fbb2815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c003970 0x7fbb0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb0c0047c0 0x7fbb0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 0x7fbb0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280701c0 con 0x7fbb0c005180 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c00c0d0 con 0x7fbb0c0047c0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.172+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c00c290 con 0x7fbb0c003970 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 0x7fbb0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb25e00640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c003970 0x7fbb0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb1d400640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb0c0047c0 0x7fbb0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 -- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c003970 msgr2=0x7fbb0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c003970 0x7fbb0c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 -- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb0c0047c0 msgr2=0x7fbb0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb0c0047c0 0x7fbb0c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.173+0000 7fbb26800640 1 -- 172.21.15.73:0/586673963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c003520 con 0x7fbb0c005180 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.174+0000 7fbb26800640 1 -- 172.21.15.73:0/586673963 reap_dead start 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.174+0000 7fbb26800640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 0x7fbb0c00b980 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c190 tx=0x7fbb0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6733ee3ae7ecc029 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.174+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20005ad0 con 0x7fbb0c005180 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.174+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20005c70 con 0x7fbb0c005180 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.174+0000 7fbb25400640 1 -- 172.21.15.73:0/586673963 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002c8a0 con 0x7fbb0c005180 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 -- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 msgr2=0x7fbb0c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 0x7fbb0c00b980 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c190 tx=0x7fbb0c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 -- 172.21.15.73:0/586673963 shutdown_connections 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/586673963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb0c005180 0x7fbb0c00b980 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 -- 172.21.15.73:0/586673963 >> 172.21.15.73:0/586673963 conn(0x7fbb280ce8e0 msgr2=0x7fbb280cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 -- 172.21.15.73:0/586673963 shutdown_connections 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.175+0000 7fbb2ce00640 1 -- 172.21.15.73:0/586673963 wait complete. 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.177+0000 7fbb2ce00640 1 Processor -- start 2024-04-05T22:32:08.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.177+0000 7fbb2ce00640 1 -- start start 2024-04-05T22:32:08.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 0x7fbb280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 0x7fbb281495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28154b60 con 0x7fbb281582e0 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28149b00 con 0x7fbb28149140 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb2ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28159010 con 0x7fbb0c005180 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb26800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb26800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60702/0 (socket says 172.21.15.73:60702) 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.178+0000 7fbb26800640 1 -- 172.21.15.73:0/2700627112 learned_addr learned my addr 172.21.15.73:0/2700627112 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb25e00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 0x7fbb280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb1d400640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 0x7fbb281495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 -- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 msgr2=0x7fbb280b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 0x7fbb280b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 -- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 msgr2=0x7fbb281495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 0x7fbb281495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 -- 172.21.15.73:0/2700627112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb281592f0 con 0x7fbb281582e0 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb25e00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 0x7fbb280b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb1d400640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 0x7fbb281495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:08.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb26800640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009000 tx=0x7fbb28070c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200049e0 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20004b40 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20000ed0 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb281595d0 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.179+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb280b2540 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.180+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb28066af0 con 0x7fbb281582e0 2024-04-05T22:32:08.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.181+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fbb20032070 con 0x7fbb281582e0 2024-04-05T22:32:08.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.183+0000 7fbb1ea00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 0x7fbb0406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.183+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb2003c070 con 0x7fbb281582e0 2024-04-05T22:32:08.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.183+0000 7fbb25e00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 0x7fbb0406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.184+0000 7fbb25e00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 0x7fbb0406dc00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007fd0 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.188+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbb200756c0 con 0x7fbb281582e0 2024-04-05T22:32:08.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.383+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb28154ca0 con 0x7fbb281582e0 2024-04-05T22:32:08.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.386+0000 7fbb1ea00640 1 -- 172.21.15.73:0/2700627112 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16115 (secure 0 0 0) 0x7fbb28154ca0 con 0x7fbb281582e0 2024-04-05T22:32:08.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:08.387 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":172,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:31:43.301911+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-05T22:31:40.727092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-05T22:31:41.315098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.389+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 msgr2=0x7fbb0406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.389+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 0x7fbb0406dc00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007fd0 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.389+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 msgr2=0x7fbb28158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.389+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009000 tx=0x7fbb28070c30 comp rx=0 tx=0).stop 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 shutdown_connections 2024-04-05T22:32:08.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbb0406b740 0x7fbb0406dc00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb281582e0 0x7fbb28158760 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb28149140 0x7fbb281495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 --2- 172.21.15.73:0/2700627112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb0c005180 0x7fbb280b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 >> 172.21.15.73:0/2700627112 conn(0x7fbb280ce8e0 msgr2=0x7fbb281527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 shutdown_connections 2024-04-05T22:32:08.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.390+0000 7fbb2ce00640 1 -- 172.21.15.73:0/2700627112 wait complete. 2024-04-05T22:32:08.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-05T22:32:08.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.610+0000 7f2975a00640 1 Processor -- start 2024-04-05T22:32:08.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.610+0000 7f2975a00640 1 -- start start 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f2975a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f297006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f2975a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297006f5b0 con 0x7f29701587d0 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f2975a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2970070410 con 0x7f297006cb80 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f2975a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297006d500 con 0x7f297015ef60 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f296ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f29701587d0 0x7f29700cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57768/0 (socket says 172.21.15.73:57768) 2024-04-05T22:32:08.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f296ea00640 1 -- 172.21.15.73:0/2491501231 learned_addr learned my addr 172.21.15.73:0/2491501231 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.612+0000 7f296f400640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f297006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830685043 0 0) 0x7f297006f5b0 con 0x7f29701587d0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2954003520 con 0x7f29701587d0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739086573 0 0) 0x7f2970070410 con 0x7f297006cb80 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f297006f5b0 con 0x7f297006cb80 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613389623 0 0) 0x7f2954003520 con 0x7f29701587d0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 msgr2=0x7f297006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f297006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 >> v1:172.21.15.157:6789/0 conn(0x7f297006cb80 legacy=0x7f297015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700ccef0 con 0x7f29701587d0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f29600015f0 con 0x7f29701587d0 2024-04-05T22:32:08.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 >> v1:172.21.15.73:6789/0 conn(0x7f29701587d0 legacy=0x7f29700cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f2954003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29540040e0 0x7f29540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2954004d00 0x7f2954005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297006d500 con 0x7f2954004d00 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295400bac0 con 0x7f29540040e0 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.613+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295400bc80 con 0x7f297015ef60 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2954004d00 0x7f2954005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296ea00640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f2954003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296e000640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29540040e0 0x7f29540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 -- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 msgr2=0x7f2954003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f297015ef60 0x7f2954003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 -- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29540040e0 msgr2=0x7f29540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29540040e0 0x7f29540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.614+0000 7f296f400640 1 -- 172.21.15.73:0/2491501231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954003520 con 0x7f2954004d00 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.615+0000 7f296f400640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2954004d00 0x7f2954005180 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fc0 tx=0x7f29540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fcff527bcc61f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.615+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296802cc20 con 0x7f2954004d00 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800f040 con 0x7f2954004d00 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f296d600640 1 -- 172.21.15.73:0/2491501231 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968034440 con 0x7f2954004d00 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 -- 172.21.15.73:0/2491501231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2954004d00 msgr2=0x7f2954005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 --2- 172.21.15.73:0/2491501231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2954004d00 0x7f2954005180 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fc0 tx=0x7f29540057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f296f400640 1 -- 172.21.15.73:0/2491501231 reap_dead start 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 -- 172.21.15.73:0/2491501231 shutdown_connections 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 -- 172.21.15.73:0/2491501231 >> 172.21.15.73:0/2491501231 conn(0x7f29700ce8e0 msgr2=0x7f29700cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 -- 172.21.15.73:0/2491501231 shutdown_connections 2024-04-05T22:32:08.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.616+0000 7f2975a00640 1 -- 172.21.15.73:0/2491501231 wait complete. 2024-04-05T22:32:08.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.618+0000 7f2975a00640 1 Processor -- start 2024-04-05T22:32:08.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.618+0000 7f2975a00640 1 -- start start 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2954004d00 0x7f29700afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29700c4f30 0x7f29700c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c37e0 con 0x7f29700c4ad0 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c4640 con 0x7f2954004d00 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f2975a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c3a30 con 0x7f29700c4f30 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f296e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.619+0000 7f296e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60716/0 (socket says 172.21.15.73:60716) 2024-04-05T22:32:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 -- 172.21.15.73:0/284028213 learned_addr learned my addr 172.21.15.73:0/284028213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296f400640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29700c4f30 0x7f29700c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296ea00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2954004d00 0x7f29700afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 -- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29700c4f30 msgr2=0x7f29700c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29700c4f30 0x7f29700c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 -- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2954004d00 msgr2=0x7f29700afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2954004d00 0x7f29700afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 -- 172.21.15.73:0/284028213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700c3d10 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f296e000640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f295c00e7b0 tx=0x7f295c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.620+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0050d0 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.621+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c005230 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.621+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29700c3ff0 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.621+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007030 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.621+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29700b0180 con 0x7f29700c4ad0 2024-04-05T22:32:08.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.622+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970066af0 con 0x7f29700c4ad0 2024-04-05T22:32:08.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.623+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f295c007190 con 0x7f29700c4ad0 2024-04-05T22:32:08.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.624+0000 7f2966a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 0x7f295006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:08.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.624+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f295c087b30 con 0x7f29700c4ad0 2024-04-05T22:32:08.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.624+0000 7f296ea00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 0x7f295006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:08.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.625+0000 7f296ea00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 0x7f295006dc30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2960001af0 tx=0x7f2960000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:08.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.628+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f295c054510 con 0x7f29700c4ad0 2024-04-05T22:32:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.862+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29700b0340 con 0x7f29700c4ad0 2024-04-05T22:32:08.864 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-05T22:32:08.864+0000 7ffaa8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-05T22:32:08.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.877+0000 7f2966a00640 1 -- 172.21.15.73:0/284028213 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7f295c08a020 con 0x7f29700c4ad0 2024-04-05T22:32:08.877 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 msgr2=0x7f295006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 0x7f295006dc30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2960001af0 tx=0x7f2960000ed0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 msgr2=0x7f29700c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f295c00e7b0 tx=0x7f295c00ec70 comp rx=0 tx=0).stop 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 shutdown_connections 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f295006b770 0x7f295006dc30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29700c4f30 0x7f29700c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29700c4ad0 0x7f29700c5710 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 --2- 172.21.15.73:0/284028213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2954004d00 0x7f29700afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:08.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 >> 172.21.15.73:0/284028213 conn(0x7f29700ce8e0 msgr2=0x7f29700708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:08.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 shutdown_connections 2024-04-05T22:32:08.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:08.880+0000 7f2975a00640 1 -- 172.21.15.73:0/284028213 wait complete. 2024-04-05T22:32:08.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-05T22:32:08.989 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-05T22:32:09.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.101+0000 7f738c800640 1 Processor -- start 2024-04-05T22:32:09.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.101+0000 7f738c800640 1 -- start start 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.102+0000 7f738c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 0x7f73840b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.102+0000 7f738c800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73840af110 0x7f738406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.102+0000 7f738c800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738415d3c0 con 0x7f73840ad220 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.102+0000 7f738c800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840ae3f0 con 0x7f73840af110 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.102+0000 7f738c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ad7e0 con 0x7f738415cfe0 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7389400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f73840ad220 0x7f73840aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57774/0 (socket says 172.21.15.73:57774) 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7389400640 1 -- 172.21.15.73:0/440616929 learned_addr learned my addr 172.21.15.73:0/440616929 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:09.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7389e00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 0x7f73840b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f738a800640 1 --2- 172.21.15.73:0/440616929 >> v2:172.21.15.157:3300/0 conn(0x7f73840af110 0x7f738406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245724698 0 0) 0x7f738415d3c0 con 0x7f73840ad220 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f736c003520 con 0x7f73840ad220 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913582810 0 0) 0x7f736c003520 con 0x7f73840ad220 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 msgr2=0x7f73840b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 0x7f73840b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 >> v2:172.21.15.157:3300/0 conn(0x7f73840af110 msgr2=0x7f738406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 --2- 172.21.15.73:0/440616929 >> v2:172.21.15.157:3300/0 conn(0x7f73840af110 0x7f738406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840adac0 con 0x7f73840ad220 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.103+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7374001670 con 0x7f73840ad220 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 >> v1:172.21.15.73:6789/0 conn(0x7f73840ad220 legacy=0x7f73840aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f736c003970 0x7f736c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 0x7f736c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c004d00 0x7f736c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ae3f0 con 0x7f736c003970 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ad7e0 con 0x7f736c004d00 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c00b980 con 0x7f738415cfe0 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f736c003970 0x7f736c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 -- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 msgr2=0x7f736c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f738415cfe0 0x7f736c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 -- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c004d00 msgr2=0x7f736c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c004d00 0x7f736c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.104+0000 7f7389e00640 1 -- 172.21.15.73:0/440616929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c003520 con 0x7f736c003970 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.105+0000 7f7389e00640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f736c003970 0x7f736c003d90 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f7378009ac0 tx=0x7f736c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0845bc141394cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.105+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737802c710 con 0x7f736c003970 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737802c870 con 0x7f736c003970 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f7388a00640 1 -- 172.21.15.73:0/440616929 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737802cc00 con 0x7f736c003970 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 -- 172.21.15.73:0/440616929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f736c003970 msgr2=0x7f736c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 --2- 172.21.15.73:0/440616929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f736c003970 0x7f736c003d90 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f7378009ac0 tx=0x7f736c0057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738a800640 1 -- 172.21.15.73:0/440616929 reap_dead start 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 -- 172.21.15.73:0/440616929 shutdown_connections 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 -- 172.21.15.73:0/440616929 >> 172.21.15.73:0/440616929 conn(0x7f73840b17b0 msgr2=0x7f73840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 -- 172.21.15.73:0/440616929 shutdown_connections 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.106+0000 7f738c800640 1 -- 172.21.15.73:0/440616929 wait complete. 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.108+0000 7f738c800640 1 Processor -- start 2024-04-05T22:32:09.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.108+0000 7f738c800640 1 -- start start 2024-04-05T22:32:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c003970 0x7f7384150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 0x7f7384156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415df00 con 0x7f73841514f0 2024-04-05T22:32:09.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415d610 con 0x7f738415e390 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738415d8b0 con 0x7f736c003970 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60760/0 (socket says 172.21.15.73:60760) 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.109+0000 7f738a800640 1 -- 172.21.15.73:0/2197212003 learned_addr learned my addr 172.21.15.73:0/2197212003 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f7389400640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 0x7f7384156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 -- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c003970 msgr2=0x7f7384150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c003970 0x7f7384150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 -- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 msgr2=0x7f7384156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 0x7f7384156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 -- 172.21.15.73:0/2197212003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738415db90 con 0x7f73841514f0 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f7389400640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 0x7f7384156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:09.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738a800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f73840af6d0 tx=0x7f738000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380010040 con 0x7f73841514f0 2024-04-05T22:32:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738000bae0 con 0x7f73841514f0 2024-04-05T22:32:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000be70 con 0x7f73841514f0 2024-04-05T22:32:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.110+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7384151090 con 0x7f73841514f0 2024-04-05T22:32:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.111+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7384151b50 con 0x7f73841514f0 2024-04-05T22:32:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.112+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7384068160 con 0x7f73841514f0 2024-04-05T22:32:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.113+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f738001d400 con 0x7f73841514f0 2024-04-05T22:32:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.114+0000 7f737e000640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 0x7f736806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:09.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.114+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7380088320 con 0x7f73841514f0 2024-04-05T22:32:09.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.114+0000 7f7389e00640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 0x7f736806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:09.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.116+0000 7f7389e00640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 0x7f736806dc00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7378036000 tx=0x7f737802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:09.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.118+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f73800589e0 con 0x7f73841514f0 2024-04-05T22:32:09.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.349+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f738415dd20 con 0x7f73841514f0 2024-04-05T22:32:09.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.883+0000 7f737e000640 1 -- 172.21.15.73:0/2197212003 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7380058570 con 0x7f73841514f0 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 msgr2=0x7f736806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 0x7f736806dc00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7378036000 tx=0x7f737802e000 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 msgr2=0x7f7384157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f73840af6d0 tx=0x7f738000b480 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 shutdown_connections 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f736806b740 0x7f736806dc00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73841514f0 0x7f7384157f00 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738415e390 0x7f7384156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 --2- 172.21.15.73:0/2197212003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c003970 0x7f7384150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.886+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 >> 172.21.15.73:0/2197212003 conn(0x7f73840b17b0 msgr2=0x7f738415c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.887+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 shutdown_connections 2024-04-05T22:32:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:09.887+0000 7f738c800640 1 -- 172.21.15.73:0/2197212003 wait complete. 2024-04-05T22:32:09.908 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-05T22:32:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.117+0000 7feb3d200640 1 Processor -- start 2024-04-05T22:32:10.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.118+0000 7feb3d200640 1 -- start start 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 0x7feb380ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb380d0fd0 0x7feb380d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb38160780 0x7feb38160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380d1950 con 0x7feb380d0fd0 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380ced00 con 0x7feb38160780 2024-04-05T22:32:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.119+0000 7feb3d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb380cefe0 con 0x7feb3815cd00 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb380d0fd0 0x7feb380d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb380d0fd0 0x7feb380d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60762/0 (socket says 172.21.15.73:60762) 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 -- 172.21.15.73:0/2991780862 learned_addr learned my addr 172.21.15.73:0/2991780862 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb37400640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 0x7feb380ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb37e00640 1 --2- 172.21.15.73:0/2991780862 >> v2:172.21.15.157:3300/0 conn(0x7feb38160780 0x7feb38160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 -- 172.21.15.73:0/2991780862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 msgr2=0x7feb380ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 0x7feb380ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 -- 172.21.15.73:0/2991780862 >> v2:172.21.15.157:3300/0 conn(0x7feb38160780 msgr2=0x7feb38160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 --2- 172.21.15.73:0/2991780862 >> v2:172.21.15.157:3300/0 conn(0x7feb38160780 0x7feb38160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb36a00640 1 -- 172.21.15.73:0/2991780862 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380cf2c0 con 0x7feb380d0fd0 2024-04-05T22:32:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.120+0000 7feb37400640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 0x7feb380ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.121+0000 7feb36a00640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb380d0fd0 0x7feb380d1410 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7feb380c9ac0 tx=0x7feb2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af547e886b2f2dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb36000640 1 -- 172.21.15.73:0/2991780862 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2002ca70 con 0x7feb380d0fd0 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb36000640 1 -- 172.21.15.73:0/2991780862 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2002cbd0 con 0x7feb380d0fd0 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb36000640 1 -- 172.21.15.73:0/2991780862 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200343f0 con 0x7feb380d0fd0 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 -- 172.21.15.73:0/2991780862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb380d0fd0 msgr2=0x7feb380d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb380d0fd0 0x7feb380d1410 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7feb380c9ac0 tx=0x7feb2002bed0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 -- 172.21.15.73:0/2991780862 shutdown_connections 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb380d0fd0 0x7feb380d1410 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 --2- 172.21.15.73:0/2991780862 >> v2:172.21.15.157:3300/0 conn(0x7feb38160780 0x7feb38160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 --2- 172.21.15.73:0/2991780862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3815cd00 0x7feb380ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.122+0000 7feb3d200640 1 -- 172.21.15.73:0/2991780862 >> 172.21.15.73:0/2991780862 conn(0x7feb3809bdd0 msgr2=0x7feb380d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.123+0000 7feb3d200640 1 -- 172.21.15.73:0/2991780862 shutdown_connections 2024-04-05T22:32:10.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.123+0000 7feb3d200640 1 -- 172.21.15.73:0/2991780862 wait complete. 2024-04-05T22:32:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.124+0000 7feb3d200640 1 Processor -- start 2024-04-05T22:32:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.124+0000 7feb3d200640 1 -- start start 2024-04-05T22:32:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb380d0fd0 0x7feb381f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb38160780 0x7feb381f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381f6600 con 0x7feb3815cd00 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381f68e0 con 0x7feb380d0fd0 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb3d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb381f27b0 con 0x7feb38160780 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.125+0000 7feb36a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60776/0 (socket says 172.21.15.73:60776) 2024-04-05T22:32:10.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 -- 172.21.15.73:0/1839933283 learned_addr learned my addr 172.21.15.73:0/1839933283 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 -- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb38160780 msgr2=0x7feb381f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:10.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb38160780 0x7feb381f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 -- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb380d0fd0 msgr2=0x7feb381f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb380d0fd0 0x7feb381f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 -- 172.21.15.73:0/1839933283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009b30 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb36a00640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7feb20000f80 tx=0x7feb20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.126+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20009510 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.127+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2003e450 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.127+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2003e800 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.127+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb381f2a90 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.127+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3815da90 con 0x7feb3815cd00 2024-04-05T22:32:10.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.128+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3815dc20 con 0x7feb3815cd00 2024-04-05T22:32:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.129+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7feb2003e960 con 0x7feb3815cd00 2024-04-05T22:32:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.129+0000 7feb2f400640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 0x7feb1406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:10.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.130+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feb200aab30 con 0x7feb3815cd00 2024-04-05T22:32:10.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.130+0000 7feb37400640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 0x7feb1406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:10.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.131+0000 7feb37400640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 0x7feb1406dc00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feb28006b10 tx=0x7feb28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:10.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.134+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feb20077760 con 0x7feb3815cd00 2024-04-05T22:32:10.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.325+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb380d1f30 con 0x7feb3815cd00 2024-04-05T22:32:10.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:10.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.895+0000 7feb2f400640 1 -- 172.21.15.73:0/1839933283 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v174) v1 ==== 181+0+0 (secure 0 0 0) 0x7feb200ad0d0 con 0x7feb3815cd00 2024-04-05T22:32:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-05T22:32:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.900+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 msgr2=0x7feb1406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.900+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 0x7feb1406dc00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feb28006b10 tx=0x7feb28008040 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.900+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 msgr2=0x7feb381f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.900+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7feb20000f80 tx=0x7feb20004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 shutdown_connections 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feb1406b740 0x7feb1406dc00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb38160780 0x7feb381f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3815cd00 0x7feb381f2fb0 secure :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7feb20000f80 tx=0x7feb20004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 --2- 172.21.15.73:0/1839933283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb380d0fd0 0x7feb381f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.901+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 >> 172.21.15.73:0/1839933283 conn(0x7feb3809bdd0 msgr2=0x7feb380d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:10.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.902+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 shutdown_connections 2024-04-05T22:32:10.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:10.902+0000 7feb3d200640 1 -- 172.21.15.73:0/1839933283 wait complete. 2024-04-05T22:32:10.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-05T22:32:11.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.128+0000 7fce97000640 1 Processor -- start 2024-04-05T22:32:11.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.128+0000 7fce97000640 1 -- start start 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce97000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 0x7fce9006b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce97000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce9006d060 0x7fce9006ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce97000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9015d420 con 0x7fce900ac8d0 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce97000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9015d700 con 0x7fce9006d060 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce97000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015d9e0 con 0x7fce9006cc00 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce95c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fce900ac8d0 0x7fce900accb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57790/0 (socket says 172.21.15.73:57790) 2024-04-05T22:32:11.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.130+0000 7fce95c00640 1 -- 172.21.15.73:0/2265513470 learned_addr learned my addr 172.21.15.73:0/2265513470 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:11.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578183791 0 0) 0x7fce9015d420 con 0x7fce900ac8d0 2024-04-05T22:32:11.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce96600640 1 --2- 172.21.15.73:0/2265513470 >> v2:172.21.15.157:3300/0 conn(0x7fce9006d060 0x7fce9006ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce95200640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 0x7fce9006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce78003520 con 0x7fce900ac8d0 2024-04-05T22:32:11.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258165090 0 0) 0x7fce78003520 con 0x7fce900ac8d0 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 msgr2=0x7fce9006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 0x7fce9006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 >> v2:172.21.15.157:3300/0 conn(0x7fce9006d060 msgr2=0x7fce9006ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 --2- 172.21.15.73:0/2265513470 >> v2:172.21.15.157:3300/0 conn(0x7fce9006d060 0x7fce9006ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9015dcc0 con 0x7fce900ac8d0 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fce84001670 con 0x7fce900ac8d0 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 >> v1:172.21.15.73:6789/0 conn(0x7fce900ac8d0 legacy=0x7fce900accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce78003970 0x7fce78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce780047c0 0x7fce78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 0x7fce78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015d700 con 0x7fce78003970 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015d9e0 con 0x7fce780047c0 2024-04-05T22:32:11.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7800b980 con 0x7fce9006cc00 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.131+0000 7fce95c00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce78003970 0x7fce78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.132+0000 7fce95c00640 1 -- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 msgr2=0x7fce78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.132+0000 7fce95c00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce9006cc00 0x7fce78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.132+0000 7fce95c00640 1 -- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce780047c0 msgr2=0x7fce78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.132+0000 7fce95c00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce780047c0 0x7fce78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.132+0000 7fce95c00640 1 -- 172.21.15.73:0/2265513470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78003520 con 0x7fce78003970 2024-04-05T22:32:11.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.133+0000 7fce95c00640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce78003970 0x7fce78003d90 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c240 tx=0x7fce780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8d24726e7477c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:11.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.133+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84005ad0 con 0x7fce78003970 2024-04-05T22:32:11.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.133+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84005c30 con 0x7fce78003970 2024-04-05T22:32:11.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.133+0000 7fce8fe00640 1 -- 172.21.15.73:0/2265513470 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8402c8b0 con 0x7fce78003970 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 -- 172.21.15.73:0/2265513470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce78003970 msgr2=0x7fce78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 --2- 172.21.15.73:0/2265513470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce78003970 0x7fce78003d90 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c240 tx=0x7fce780057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce96600640 1 -- 172.21.15.73:0/2265513470 reap_dead start 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 -- 172.21.15.73:0/2265513470 shutdown_connections 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 -- 172.21.15.73:0/2265513470 >> 172.21.15.73:0/2265513470 conn(0x7fce900b0360 msgr2=0x7fce900b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 -- 172.21.15.73:0/2265513470 shutdown_connections 2024-04-05T22:32:11.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.134+0000 7fce97000640 1 -- 172.21.15.73:0/2265513470 wait complete. 2024-04-05T22:32:11.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.136+0000 7fce97000640 1 Processor -- start 2024-04-05T22:32:11.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.136+0000 7fce97000640 1 -- start start 2024-04-05T22:32:11.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce78003970 0x7fce901f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 0x7fce90155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90158600 con 0x7fce90159eb0 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce901579f0 con 0x7fce78003970 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce97000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce90157cd0 con 0x7fce90159a30 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce96600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce96600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60790/0 (socket says 172.21.15.73:60790) 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.137+0000 7fce96600640 1 -- 172.21.15.73:0/2511958390 learned_addr learned my addr 172.21.15.73:0/2511958390 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce95c00640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce78003970 0x7fce901f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce95200640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 0x7fce90155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 -- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 msgr2=0x7fce90155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 0x7fce90155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 -- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce78003970 msgr2=0x7fce901f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce78003970 0x7fce901f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 -- 172.21.15.73:0/2511958390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90157fb0 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce95200640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 0x7fce90155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce96600640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fce9006d9a0 tx=0x7fce8800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88010040 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800bae0 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce90158290 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800be70 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.138+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce90149d70 con 0x7fce90159eb0 2024-04-05T22:32:11.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.140+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90158420 con 0x7fce90159eb0 2024-04-05T22:32:11.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.141+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fce8801d400 con 0x7fce90159eb0 2024-04-05T22:32:11.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.141+0000 7fce8d600640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 0x7fce7006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:11.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.142+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fce88088340 con 0x7fce90159eb0 2024-04-05T22:32:11.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.142+0000 7fce95c00640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 0x7fce7006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:11.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.143+0000 7fce95c00640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 0x7fce7006dc30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fce84034000 tx=0x7fce84009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:11.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.146+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fce88058a00 con 0x7fce90159eb0 2024-04-05T22:32:11.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.338+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce9004e160 con 0x7fce90159eb0 2024-04-05T22:32:11.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.914+0000 7fce8d600640 1 -- 172.21.15.73:0/2511958390 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7fce880180d0 con 0x7fce90159eb0 2024-04-05T22:32:11.919 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-05T22:32:11.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.921+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 msgr2=0x7fce7006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 0x7fce7006dc30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fce84034000 tx=0x7fce84009030 comp rx=0 tx=0).stop 2024-04-05T22:32:11.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 msgr2=0x7fce90155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:11.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fce9006d9a0 tx=0x7fce8800b480 comp rx=0 tx=0).stop 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 shutdown_connections 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fce7006b770 0x7fce7006dc30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce90159eb0 0x7fce90155e50 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce90159a30 0x7fce90155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 --2- 172.21.15.73:0/2511958390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce78003970 0x7fce901f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 >> 172.21.15.73:0/2511958390 conn(0x7fce900b0360 msgr2=0x7fce900af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.922+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 shutdown_connections 2024-04-05T22:32:11.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:11.923+0000 7fce97000640 1 -- 172.21.15.73:0/2511958390 wait complete. 2024-04-05T22:32:11.945 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-05T22:32:12.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.136+0000 7f9cd2e00640 1 Processor -- start 2024-04-05T22:32:12.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.136+0000 7f9cd2e00640 1 -- start start 2024-04-05T22:32:12.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9ccc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd2e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0cceb0 con 0x7f9ccc070660 2024-04-05T22:32:12.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd2e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0cd130 con 0x7f9ccc143660 2024-04-05T22:32:12.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0cb8c0 con 0x7f9ccc06cd40 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd0e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9ccc070660 0x7f9ccc0cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57796/0 (socket says 172.21.15.73:57796) 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.138+0000 7f9cd0e00640 1 -- 172.21.15.73:0/3969712945 learned_addr learned my addr 172.21.15.73:0/3969712945 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccb400640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9ccc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550309319 0 0) 0x7f9ccc0cceb0 con 0x7f9ccc070660 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb0003520 con 0x7f9ccc070660 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707297206 0 0) 0x7f9cb0003520 con 0x7f9ccc070660 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 msgr2=0x7f9ccc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9ccc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 >> v1:172.21.15.157:6789/0 conn(0x7f9ccc143660 legacy=0x7f9ccc06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc070b90 con 0x7f9ccc070660 2024-04-05T22:32:12.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f9cc40019f0 con 0x7f9ccc070660 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 >> v1:172.21.15.73:6789/0 conn(0x7f9ccc070660 legacy=0x7f9ccc0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.139+0000 7f9ccaa00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00039a0 0x7f9cb0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccaa00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9cb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccaa00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9cb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0cb8c0 con 0x7f9cb0004d00 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb000bab0 con 0x7f9cb00039a0 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb000de80 con 0x7f9ccc06cd40 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9cb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccbe00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00039a0 0x7f9cb0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9ccb400640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9cb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 -- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 msgr2=0x7f9cb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:12.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc06cd40 0x7f9cb00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 -- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00039a0 msgr2=0x7f9cb0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:12.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00039a0 0x7f9cb0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.140+0000 7f9cd0e00640 1 -- 172.21.15.73:0/3969712945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0003520 con 0x7f9cb0004d00 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.141+0000 7f9cd0e00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9cb0005180 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400a820 tx=0x7f9cb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d065a02ebe5b7f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40072d0 con 0x7f9cb0004d00 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc402f070 con 0x7f9cb0004d00 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9ccaa00640 1 -- 172.21.15.73:0/3969712945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40076b0 con 0x7f9cb0004d00 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3969712945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 msgr2=0x7f9cb0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3969712945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9cb0005180 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400a820 tx=0x7f9cb00057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd0e00640 1 -- 172.21.15.73:0/3969712945 reap_dead start 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3969712945 shutdown_connections 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3969712945 >> 172.21.15.73:0/3969712945 conn(0x7f9ccc0b0830 msgr2=0x7f9ccc0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3969712945 shutdown_connections 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.142+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3969712945 wait complete. 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.144+0000 7f9cd2e00640 1 Processor -- start 2024-04-05T22:32:12.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.144+0000 7f9cd2e00640 1 -- start start 2024-04-05T22:32:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc13a980 0x7f9ccc13ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc135f40 0x7f9ccc1363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc13c050 con 0x7f9cb0004d00 2024-04-05T22:32:12.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc13b440 con 0x7f9ccc13a980 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9cd2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc13b720 con 0x7f9ccc135f40 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.145+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60798/0 (socket says 172.21.15.73:60798) 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 -- 172.21.15.73:0/3209284149 learned_addr learned my addr 172.21.15.73:0/3209284149 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccb400640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc13a980 0x7f9ccc13ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 -- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc135f40 msgr2=0x7f9ccc1363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc135f40 0x7f9ccc1363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 -- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc13a980 msgr2=0x7f9ccc13ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc13a980 0x7f9ccc13ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 -- 172.21.15.73:0/3209284149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc13ba00 con 0x7f9cb0004d00 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9ccbe00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e550 tx=0x7f9cbc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0052e0 con 0x7f9cb0004d00 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc005440 con 0x7f9cb0004d00 2024-04-05T22:32:12.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.146+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc13bce0 con 0x7f9cb0004d00 2024-04-05T22:32:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.147+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003810 con 0x7f9cb0004d00 2024-04-05T22:32:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.147+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1593d0 con 0x7f9cb0004d00 2024-04-05T22:32:12.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.148+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc1595c0 con 0x7f9cb0004d00 2024-04-05T22:32:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.149+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f9cbc003970 con 0x7f9cb0004d00 2024-04-05T22:32:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.150+0000 7f9cc3e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 0x7f9cac06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.150+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9cbc0879d0 con 0x7f9cb0004d00 2024-04-05T22:32:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.150+0000 7f9ccb400640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 0x7f9cac06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.151+0000 7f9ccb400640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 0x7f9cac06dc00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc138020 tx=0x7f9cb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.154+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f9ccc1595c0 con 0x7f9cb0004d00 2024-04-05T22:32:12.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.358+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9ccc1595c0 con 0x7f9cb0004d00 2024-04-05T22:32:12.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.363+0000 7f9cc3e00640 1 -- 172.21.15.73:0/3209284149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9ccc1595c0 con 0x7f9cb0004d00 2024-04-05T22:32:12.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 msgr2=0x7f9cac06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 0x7f9cac06dc00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc138020 tx=0x7f9cb8008040 comp rx=0 tx=0).stop 2024-04-05T22:32:12.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 msgr2=0x7f9ccc14f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e550 tx=0x7f9cbc00ea10 comp rx=0 tx=0).stop 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 shutdown_connections 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f9cac06b740 0x7f9cac06dc00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ccc135f40 0x7f9ccc1363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ccc13a980 0x7f9ccc13ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 --2- 172.21.15.73:0/3209284149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb0004d00 0x7f9ccc14f3e0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 >> 172.21.15.73:0/3209284149 conn(0x7f9ccc0b0830 msgr2=0x7f9ccc070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 shutdown_connections 2024-04-05T22:32:12.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.366+0000 7f9cd2e00640 1 -- 172.21.15.73:0/3209284149 wait complete. 2024-04-05T22:32:12.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-05T22:32:12.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.587+0000 7f0ed4200640 1 Processor -- start 2024-04-05T22:32:12.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.588+0000 7f0ed4200640 1 -- start start 2024-04-05T22:32:12.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed4200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ecc161f90 0x7f0ecc162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed4200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed4200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0b3510 con 0x7f0ecc164e50 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed4200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc158790 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed4200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b2820 con 0x7f0ecc161f90 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.589+0000 7f0ed2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60806/0 (socket says 172.21.15.73:60806) 2024-04-05T22:32:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 -- 172.21.15.73:0/385173539 learned_addr learned my addr 172.21.15.73:0/385173539 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed1800640 1 --2- 172.21.15.73:0/385173539 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc161f90 0x7f0ecc162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 -- 172.21.15.73:0/385173539 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc161f90 msgr2=0x7f0ecc162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 --2- 172.21.15.73:0/385173539 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc161f90 0x7f0ecc162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 -- 172.21.15.73:0/385173539 >> v1:172.21.15.157:6789/0 conn(0x7f0ecc164e50 legacy=0x7f0ecc0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.590+0000 7f0ed2200640 1 -- 172.21.15.73:0/385173539 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc165cf0 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.591+0000 7f0ed2200640 1 --2- 172.21.15.73:0/385173539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc071a40 tx=0x7f0ec402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80317cdee13642da server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.591+0000 7f0ecbe00640 1 -- 172.21.15.73:0/385173539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec402ca50 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.591+0000 7f0ecbe00640 1 -- 172.21.15.73:0/385173539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec402cbb0 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.591+0000 7f0ecbe00640 1 -- 172.21.15.73:0/385173539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40343f0 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 -- 172.21.15.73:0/385173539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc0b2ba0 msgr2=0x7f0ecc0b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 --2- 172.21.15.73:0/385173539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc071a40 tx=0x7f0ec402beb0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 -- 172.21.15.73:0/385173539 shutdown_connections 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 --2- 172.21.15.73:0/385173539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc0b2ba0 0x7f0ecc0b2fd0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 --2- 172.21.15.73:0/385173539 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc161f90 0x7f0ecc162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 -- 172.21.15.73:0/385173539 >> 172.21.15.73:0/385173539 conn(0x7f0ecc0ce8e0 msgr2=0x7f0ecc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 -- 172.21.15.73:0/385173539 shutdown_connections 2024-04-05T22:32:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.592+0000 7f0ed4200640 1 -- 172.21.15.73:0/385173539 wait complete. 2024-04-05T22:32:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.594+0000 7f0ed4200640 1 Processor -- start 2024-04-05T22:32:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.594+0000 7f0ed4200640 1 -- start start 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 0x7f0ecc1449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 0x7f0ecc147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1480f0 con 0x7f0ecc164e50 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1456b0 con 0x7f0ecc161f90 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc145990 con 0x7f0ecc0b2ba0 2024-04-05T22:32:12.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60810/0 (socket says 172.21.15.73:60810) 2024-04-05T22:32:12.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed2200640 1 -- 172.21.15.73:0/460049002 learned_addr learned my addr 172.21.15.73:0/460049002 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed0e00640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 0x7f0ecc147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed2200640 1 -- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 msgr2=0x7f0ecc1449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.595+0000 7f0ed1800640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 0x7f0ecc1449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed2200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 0x7f0ecc1449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed2200640 1 -- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 msgr2=0x7f0ecc147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed2200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 0x7f0ecc147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed2200640 1 -- 172.21.15.73:0/460049002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4009b30 con 0x7f0ecc164e50 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed1800640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 0x7f0ecc1449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed0e00640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 0x7f0ecc147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed2200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036040 tx=0x7f0ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec403e450 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec403e5d0 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc145c70 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.596+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec403e980 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.597+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc146230 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.598+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc066af0 con 0x7f0ecc164e50 2024-04-05T22:32:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.599+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f0ec403eae0 con 0x7f0ecc164e50 2024-04-05T22:32:12.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.599+0000 7f0ec9600640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 0x7f0eac06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:12.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.600+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ec40aac00 con 0x7f0ecc164e50 2024-04-05T22:32:12.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.600+0000 7f0ed1800640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 0x7f0eac06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:12.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.601+0000 7f0ed1800640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 0x7f0eac06dc00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0008890 tx=0x7f0ec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.604+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f0ec40778e0 con 0x7f0ecc164e50 2024-04-05T22:32:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.872+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f0ecc068160 con 0x7f0ecc164e50 2024-04-05T22:32:12.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.911+0000 7f0ec9600640 1 -- 172.21.15.73:0/460049002 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v595) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0ec407af00 con 0x7f0ecc164e50 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.914+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 msgr2=0x7f0eac06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.914+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 0x7f0eac06dc00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0008890 tx=0x7f0ec0008040 comp rx=0 tx=0).stop 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 msgr2=0x7f0ecc146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036040 tx=0x7f0ec4004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 shutdown_connections 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f0eac06b740 0x7f0eac06dc00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ecc164e50 0x7f0ecc146750 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc161f90 0x7f0ecc147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 --2- 172.21.15.73:0/460049002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b2ba0 0x7f0ecc1449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:12.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 >> 172.21.15.73:0/460049002 conn(0x7f0ecc0ce8e0 msgr2=0x7f0ecc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:12.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 shutdown_connections 2024-04-05T22:32:12.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:12.915+0000 7f0ed4200640 1 -- 172.21.15.73:0/460049002 wait complete. 2024-04-05T22:32:12.933 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-04-05T22:32:12.934 INFO:tasks.cephfs_test_runner: 2024-04-05T22:32:12.934 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-05T22:32:12.934 INFO:tasks.cephfs_test_runner:Ran 11 tests in 620.510s 2024-04-05T22:32:12.934 INFO:tasks.cephfs_test_runner: 2024-04-05T22:32:12.934 INFO:tasks.cephfs_test_runner:OK 2024-04-05T22:32:12.934 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-05T22:32:12.945 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-05T22:32:12.964 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-05T22:32:12.965 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:32:12.965 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:13.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:32:13.020 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-05T22:32:13.020 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:13.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-05T22:32:13.071 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:32:13.071 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:13.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:32:13.128 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-05T22:32:13.128 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-05T22:32:13.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-05T22:32:13.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.181 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:32:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-05T22:32:13.238 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-05T22:32:13.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-05T22:32:13.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-05T22:32:13.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-05T22:32:13.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.542 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-05T22:32:13.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-05T22:32:13.719 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-04-05T22:32:13.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:32:13.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.724 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:32:13.802 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-05T22:32:13.802 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-05T22:32:13.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.803 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-05T22:32:13.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-05T22:32:13.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:13.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-05T22:32:14.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:14.121 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-05T22:32:14.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-05T22:32:14.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-05T22:32:14.307 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-05T22:32:14.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:32:14.312 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-05T22:32:14.322 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-05T22:32:14.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:32:14.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.536+0000 7fbfd4a00640 1 Processor -- start 2024-04-05T22:32:14.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.536+0000 7fbfd4a00640 1 -- start start 2024-04-05T22:32:14.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfd4a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfd00adf90 0x7fbfd015e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfd4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfd00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfd4a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd0159ef0 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfd4a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd00caa80 con 0x7fbfd00adf90 2024-04-05T22:32:14.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfd4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0071290 con 0x7fbfd00c3700 2024-04-05T22:32:14.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfce800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbfd00ca5f0 0x7fbfd00c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57800/0 (socket says 172.21.15.73:57800) 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.538+0000 7fbfce800640 1 -- 172.21.15.73:0/3000494241 learned_addr learned my addr 172.21.15.73:0/3000494241 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcde00640 1 --2- 172.21.15.73:0/3000494241 >> v2:172.21.15.157:3300/0 conn(0x7fbfd00adf90 0x7fbfd015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfd00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823685609 0 0) 0x7fbfd0159ef0 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb4003520 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147672264 0 0) 0x7fbfb4003520 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 msgr2=0x7fbfd00ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfd00ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 >> v2:172.21.15.157:3300/0 conn(0x7fbfd00adf90 msgr2=0x7fbfd015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 --2- 172.21.15.73:0/3000494241 >> v2:172.21.15.157:3300/0 conn(0x7fbfd00adf90 0x7fbfd015e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00c3b60 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7fbfc80019f0 con 0x7fbfd00ca5f0 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 >> v1:172.21.15.73:6789/0 conn(0x7fbfd00ca5f0 legacy=0x7fbfd00c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.539+0000 7fbfcd400640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003970 0x7fbfb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcd400640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb40047c0 0x7fbfb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcd400640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00caa80 con 0x7fbfb4003970 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0071290 con 0x7fbfb40047c0 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb400dc10 con 0x7fbfd00c3700 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003970 0x7fbfb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb40047c0 0x7fbfb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfce800640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 -- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 msgr2=0x7fbfb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd00c3700 0x7fbfb4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 -- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb40047c0 msgr2=0x7fbfb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb40047c0 0x7fbfb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfcde00640 1 -- 172.21.15.73:0/3000494241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4003520 con 0x7fbfb4003970 2024-04-05T22:32:14.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.540+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfb40047c0 0x7fbfb4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.541+0000 7fbfcde00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003970 0x7fbfb4003d90 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009ac0 tx=0x7fbfb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ee24b6dda35ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc002c710 con 0x7fbfb4003970 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc002c870 con 0x7fbfb4003970 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfcd400640 1 -- 172.21.15.73:0/3000494241 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc002cc00 con 0x7fbfb4003970 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 -- 172.21.15.73:0/3000494241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003970 msgr2=0x7fbfb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/3000494241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb4003970 0x7fbfb4003d90 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009ac0 tx=0x7fbfb40057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfce800640 1 -- 172.21.15.73:0/3000494241 reap_dead start 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 -- 172.21.15.73:0/3000494241 shutdown_connections 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 -- 172.21.15.73:0/3000494241 >> 172.21.15.73:0/3000494241 conn(0x7fbfd00ce840 msgr2=0x7fbfd00cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 -- 172.21.15.73:0/3000494241 shutdown_connections 2024-04-05T22:32:14.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.542+0000 7fbfd4a00640 1 -- 172.21.15.73:0/3000494241 wait complete. 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.544+0000 7fbfd4a00640 1 Processor -- start 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.544+0000 7fbfd4a00640 1 -- start start 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb4003970 0x7fbfd0146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd00b2840 0x7fbfd00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd01468b0 con 0x7fbfd00aec90 2024-04-05T22:32:14.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00b4630 con 0x7fbfd00b2840 2024-04-05T22:32:14.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.545+0000 7fbfd4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00af650 con 0x7fbfb4003970 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60822/0 (socket says 172.21.15.73:60822) 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 -- 172.21.15.73:0/866484176 learned_addr learned my addr 172.21.15.73:0/866484176 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 -- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb4003970 msgr2=0x7fbfd0146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb4003970 0x7fbfd0146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 -- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd00b2840 msgr2=0x7fbfd00b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd00b2840 0x7fbfd00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 -- 172.21.15.73:0/866484176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00b3590 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.546+0000 7fbfc7e00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80108b0 tx=0x7fbfb8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.547+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800eb60 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.547+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800ecc0 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.547+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd00b3870 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.547+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb801c400 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.547+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd00b3e30 con 0x7fbfd00aec90 2024-04-05T22:32:14.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.548+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd0068160 con 0x7fbfd00aec90 2024-04-05T22:32:14.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.549+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fbfb801c560 con 0x7fbfd00aec90 2024-04-05T22:32:14.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.550+0000 7fbfc6000640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 0x7fbfac06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.550+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbfb80884f0 con 0x7fbfd00aec90 2024-04-05T22:32:14.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.550+0000 7fbfcde00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 0x7fbfac06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.552+0000 7fbfcde00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 0x7fbfac06dc00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036000 tx=0x7fbfc002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:14.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.554+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fbfb8054fe0 con 0x7fbfd00aec90 2024-04-05T22:32:14.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.747+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfd0070f20 con 0x7fbfac06b740 2024-04-05T22:32:14.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.754+0000 7fbfc6000640 1 -- 172.21.15.73:0/866484176 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48850 (secure 0 0 0) 0x7fbfc002c9e0 con 0x7fbfac06b740 2024-04-05T22:32:14.755 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:14.755 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.756+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 msgr2=0x7fbfac06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.756+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 0x7fbfac06dc00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036000 tx=0x7fbfc002e000 comp rx=0 tx=0).stop 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 msgr2=0x7fbfd00af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80108b0 tx=0x7fbfb8010d70 comp rx=0 tx=0).stop 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 shutdown_connections 2024-04-05T22:32:14.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fbfac06b740 0x7fbfac06dc00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd00b2840 0x7fbfd00b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd00aec90 0x7fbfd00af110 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 --2- 172.21.15.73:0/866484176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfb4003970 0x7fbfd0146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 >> 172.21.15.73:0/866484176 conn(0x7fbfd00ce840 msgr2=0x7fbfd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 shutdown_connections 2024-04-05T22:32:14.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.757+0000 7fbfd4a00640 1 -- 172.21.15.73:0/866484176 wait complete. 2024-04-05T22:32:14.777 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-04-05T22:32:13.623360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291884,"kb_used_data":18376,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754682836,"statfs":{"total":773094113280,"available":772795224064,"internally_reserved":0,"allocated":18817024,"data_stored":8983228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":279957051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-66674,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-84,"num_write_kb":-100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002198"},"pg_stats":[{"pgid":"1.0","version":"75'128","reported_seq":334,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-04-05T22:32:09.019287+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:32:09.019287+0000","last_peered":"2024-04-05T22:32:09.019287+0000","last_clean":"2024-04-05T22:32:09.019287+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:32:09.019287+0000","last_undegraded":"2024-04-05T22:32:09.019287+0000","last_fullsized":"2024-04-05T22:32:09.019287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37992,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333848,"statfs":{"total":96636764160,"available":96597860352,"internally_reserved":0,"allocated":2846720,"data_stored":1618457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":1.216}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.4650000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4379999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37348,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2187264,"data_stored":963401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.339}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.466},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.8129999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.601},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.6379999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37344,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2183168,"data_stored":963098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.115},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.522}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":0.87},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.2709999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.216}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.4119999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.444}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.621},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.718},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.163}]}]},{"osd":4,"up_from":14,"seq":60129542279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":1.306}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3839999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.4930000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.625}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.341},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.6819999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33256,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2191360,"data_stored":962698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.159}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.88},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.0229999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542279,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33260,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.087}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.371}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.145},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.995}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.22}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.3919999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.008}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.236},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.413}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.504},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.379}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.3540000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.327}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.381},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:32:14.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:32:14.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.989+0000 7fa6c5600640 1 Processor -- start 2024-04-05T22:32:14.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.989+0000 7fa6c5600640 1 -- start start 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c01541a0 0x7fa6c00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6c006d710 0x7fa6c00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c015aef0 0x7fa6c00cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c015aa60 con 0x7fa6c015aef0 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c00b3b60 con 0x7fa6c006d710 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.990+0000 7fa6c5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00701c0 con 0x7fa6c01541a0 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c015aef0 0x7fa6c00cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c015aef0 0x7fa6c00cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60838/0 (socket says 172.21.15.73:60838) 2024-04-05T22:32:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 -- 172.21.15.73:0/505037804 learned_addr learned my addr 172.21.15.73:0/505037804 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 -- 172.21.15.73:0/505037804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c01541a0 msgr2=0x7fa6c00b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bea00640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c01541a0 0x7fa6c00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c01541a0 0x7fa6c00b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 -- 172.21.15.73:0/505037804 >> v2:172.21.15.157:3300/0 conn(0x7fa6c006d710 msgr2=0x7fa6c00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6b6000640 1 --2- 172.21.15.73:0/505037804 >> v2:172.21.15.157:3300/0 conn(0x7fa6c006d710 0x7fa6c00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 --2- 172.21.15.73:0/505037804 >> v2:172.21.15.157:3300/0 conn(0x7fa6c006d710 0x7fa6c00cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.991+0000 7fa6bf400640 1 -- 172.21.15.73:0/505037804 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00b3880 con 0x7fa6c015aef0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.992+0000 7fa6bf400640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c015aef0 0x7fa6c00cde30 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0071fd0 tx=0x7fa6b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f439065546bd61dd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6be000640 1 -- 172.21.15.73:0/505037804 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b802ca70 con 0x7fa6c015aef0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6be000640 1 -- 172.21.15.73:0/505037804 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b802cbd0 con 0x7fa6c015aef0 2024-04-05T22:32:14.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6be000640 1 -- 172.21.15.73:0/505037804 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80343f0 con 0x7fa6c015aef0 2024-04-05T22:32:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 -- 172.21.15.73:0/505037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c015aef0 msgr2=0x7fa6c00cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c015aef0 0x7fa6c00cde30 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0071fd0 tx=0x7fa6b802bed0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 -- 172.21.15.73:0/505037804 shutdown_connections 2024-04-05T22:32:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c015aef0 0x7fa6c00cde30 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 --2- 172.21.15.73:0/505037804 >> v2:172.21.15.157:3300/0 conn(0x7fa6c006d710 0x7fa6c00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 --2- 172.21.15.73:0/505037804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c01541a0 0x7fa6c00b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 -- 172.21.15.73:0/505037804 >> 172.21.15.73:0/505037804 conn(0x7fa6c00ce8e0 msgr2=0x7fa6c00cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.993+0000 7fa6c5600640 1 -- 172.21.15.73:0/505037804 shutdown_connections 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.994+0000 7fa6c5600640 1 -- 172.21.15.73:0/505037804 wait complete. 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.995+0000 7fa6c5600640 1 Processor -- start 2024-04-05T22:32:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.995+0000 7fa6c5600640 1 -- start start 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c006d710 0x7fa6c014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c015aef0 0x7fa6c00ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0164bf0 con 0x7fa6c01541a0 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c014ddf0 con 0x7fa6c015aef0 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6c5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0164e40 con 0x7fa6c006d710 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.996+0000 7fa6b6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60840/0 (socket says 172.21.15.73:60840) 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 -- 172.21.15.73:0/921668131 learned_addr learned my addr 172.21.15.73:0/921668131 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6bea00640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c006d710 0x7fa6c014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 -- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c006d710 msgr2=0x7fa6c014d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c006d710 0x7fa6c014d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 -- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c015aef0 msgr2=0x7fa6c00ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c015aef0 0x7fa6c00ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 -- 172.21.15.73:0/921668131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8009b30 con 0x7fa6c01541a0 2024-04-05T22:32:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b6000640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c7f0 tx=0x7fa6a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.997+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8004220 con 0x7fa6c01541a0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.998+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8004380 con 0x7fa6c01541a0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.998+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c0165120 con 0x7fa6c01541a0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.998+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8007860 con 0x7fa6c01541a0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.998+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c01656e0 con 0x7fa6c01541a0 2024-04-05T22:32:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:14.999+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c0066af0 con 0x7fa6c01541a0 2024-04-05T22:32:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.000+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fa6a8007bd0 con 0x7fa6c01541a0 2024-04-05T22:32:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.001+0000 7fa6b7400640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 0x7fa69c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.001+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa6a8087e80 con 0x7fa6c01541a0 2024-04-05T22:32:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.001+0000 7fa6bea00640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 0x7fa69c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.003+0000 7fa6bea00640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 0x7fa69c06dc30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006fd0 tx=0x7fa6b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.005+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa6a8054860 con 0x7fa6c01541a0 2024-04-05T22:32:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.196+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6c00b3590 con 0x7fa69c06b770 2024-04-05T22:32:15.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.203+0000 7fa6b7400640 1 -- 172.21.15.73:0/921668131 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48850 (secure 0 0 0) 0x7fa6b000c040 con 0x7fa69c06b770 2024-04-05T22:32:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:15.203 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:32:15.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 msgr2=0x7fa69c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 0x7fa69c06dc30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006fd0 tx=0x7fa6b0008040 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 msgr2=0x7fa6c00ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c7f0 tx=0x7fa6a800ccb0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 shutdown_connections 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa69c06b770 0x7fa69c06dc30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c015aef0 0x7fa6c00ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c01541a0 0x7fa6c00ac8d0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 --2- 172.21.15.73:0/921668131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c006d710 0x7fa6c014d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 >> 172.21.15.73:0/921668131 conn(0x7fa6c00ce8e0 msgr2=0x7fa6c00b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 shutdown_connections 2024-04-05T22:32:15.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.205+0000 7fa6c5600640 1 -- 172.21.15.73:0/921668131 wait complete. 2024-04-05T22:32:15.222 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-04-05T22:32:13.623360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291884,"kb_used_data":18376,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754682836,"statfs":{"total":773094113280,"available":772795224064,"internally_reserved":0,"allocated":18817024,"data_stored":8983228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":279957051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-66674,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-84,"num_write_kb":-100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002198"},"pg_stats":[{"pgid":"1.0","version":"75'128","reported_seq":334,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-04-05T22:32:09.019287+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:32:09.019287+0000","last_peered":"2024-04-05T22:32:09.019287+0000","last_clean":"2024-04-05T22:32:09.019287+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:32:09.019287+0000","last_undegraded":"2024-04-05T22:32:09.019287+0000","last_fullsized":"2024-04-05T22:32:09.019287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37992,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333848,"statfs":{"total":96636764160,"available":96597860352,"internally_reserved":0,"allocated":2846720,"data_stored":1618457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":1.216}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.4650000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4379999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37348,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2187264,"data_stored":963401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.339}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.466},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.8129999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.601},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.6379999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37344,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2183168,"data_stored":963098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.115},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.522}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":0.87},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.2709999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.216}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.4119999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.444}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.621},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.718},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.163}]}]},{"osd":4,"up_from":14,"seq":60129542279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":1.306}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3839999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.4930000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.625}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.341},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.6819999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33256,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2191360,"data_stored":962698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.159}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.88},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.0229999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542279,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33260,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.087}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.371}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.145},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.995}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.22}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.3919999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.008}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.236},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.413}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.504},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.379}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.3540000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.327}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.381},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:32:15.224 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-05T22:32:15.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:32:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.434+0000 7ff6f1400640 1 Processor -- start 2024-04-05T22:32:15.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.434+0000 7ff6f1400640 1 -- start start 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec0ccd20 0x7ff6ec06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6ec06d0c0 0x7ff6ec15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec06f5b0 con 0x7ff6ec1587d0 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec070410 con 0x7ff6ec06d0c0 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.435+0000 7ff6f1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec158bb0 con 0x7ff6ec0ccd20 2024-04-05T22:32:15.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60846/0 (socket says 172.21.15.73:60846) 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 -- 172.21.15.73:0/2095022466 learned_addr learned my addr 172.21.15.73:0/2095022466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 -- 172.21.15.73:0/2095022466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec0ccd20 msgr2=0x7ff6ec06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 --2- 172.21.15.73:0/2095022466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec0ccd20 0x7ff6ec06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 -- 172.21.15.73:0/2095022466 >> v2:172.21.15.157:3300/0 conn(0x7ff6ec06d0c0 msgr2=0x7ff6ec15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 --2- 172.21.15.73:0/2095022466 >> v2:172.21.15.157:3300/0 conn(0x7ff6ec06d0c0 0x7ff6ec15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.436+0000 7ff6ea800640 1 -- 172.21.15.73:0/2095022466 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec06f800 con 0x7ff6ec1587d0 2024-04-05T22:32:15.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.437+0000 7ff6ea800640 1 --2- 172.21.15.73:0/2095022466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0ae4b0 tx=0x7ff6dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9494e4992440f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6e9400640 1 -- 172.21.15.73:0/2095022466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc031870 con 0x7ff6ec1587d0 2024-04-05T22:32:15.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6e9400640 1 -- 172.21.15.73:0/2095022466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc0319d0 con 0x7ff6ec1587d0 2024-04-05T22:32:15.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6e9400640 1 -- 172.21.15.73:0/2095022466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc031d60 con 0x7ff6ec1587d0 2024-04-05T22:32:15.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 -- 172.21.15.73:0/2095022466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec1587d0 msgr2=0x7ff6ec0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 --2- 172.21.15.73:0/2095022466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0ae4b0 tx=0x7ff6dc02b910 comp rx=0 tx=0).stop 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 -- 172.21.15.73:0/2095022466 shutdown_connections 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 --2- 172.21.15.73:0/2095022466 >> v2:172.21.15.157:3300/0 conn(0x7ff6ec06d0c0 0x7ff6ec15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 --2- 172.21.15.73:0/2095022466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec0ccd20 0x7ff6ec06cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 --2- 172.21.15.73:0/2095022466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec1587d0 0x7ff6ec0cc7e0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 -- 172.21.15.73:0/2095022466 >> 172.21.15.73:0/2095022466 conn(0x7ff6ec0ce8e0 msgr2=0x7ff6ec0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 -- 172.21.15.73:0/2095022466 shutdown_connections 2024-04-05T22:32:15.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.438+0000 7ff6f1400640 1 -- 172.21.15.73:0/2095022466 wait complete. 2024-04-05T22:32:15.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.440+0000 7ff6f1400640 1 Processor -- start 2024-04-05T22:32:15.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.440+0000 7ff6f1400640 1 -- start start 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 0x7ff6ec148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec1587d0 0x7ff6ec0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14c250 con 0x7ff6ec149000 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14b6b0 con 0x7ff6ec06d0c0 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6f1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14b920 con 0x7ff6ec1587d0 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.441+0000 7ff6eb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60852/0 (socket says 172.21.15.73:60852) 2024-04-05T22:32:15.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 -- 172.21.15.73:0/201183171 learned_addr learned my addr 172.21.15.73:0/201183171 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6ea800640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 0x7ff6ec148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 -- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec1587d0 msgr2=0x7ff6ec0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6e9e00640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec1587d0 0x7ff6ec0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec1587d0 0x7ff6ec0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 -- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 msgr2=0x7ff6ec148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 0x7ff6ec148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 -- 172.21.15.73:0/201183171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0094d0 con 0x7ff6ec149000 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6ea800640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 0x7ff6ec148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:15.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6eb200640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fd0 tx=0x7ff6e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.442+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011830 con 0x7ff6ec149000 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.443+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4011990 con 0x7ff6ec149000 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.443+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011d20 con 0x7ff6ec149000 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.443+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec14bc00 con 0x7ff6ec149000 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.443+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec1f67c0 con 0x7ff6ec149000 2024-04-05T22:32:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.444+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec066af0 con 0x7ff6ec149000 2024-04-05T22:32:15.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.445+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7ff6e401b460 con 0x7ff6ec149000 2024-04-05T22:32:15.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.446+0000 7ff6e2a00640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 0x7ff6cc06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.446+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6e4087b00 con 0x7ff6ec149000 2024-04-05T22:32:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.446+0000 7ff6ea800640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 0x7ff6cc06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.447+0000 7ff6ea800640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 0x7ff6cc06dc00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc036000 tx=0x7ff6dc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.450+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff6e40581c0 con 0x7ff6ec149000 2024-04-05T22:32:15.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.641+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6ec14ba90 con 0x7ff6cc06b740 2024-04-05T22:32:15.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.648+0000 7ff6e2a00640 1 -- 172.21.15.73:0/201183171 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48850 (secure 0 0 0) 0x7ff6ec14ba90 con 0x7ff6cc06b740 2024-04-05T22:32:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:15.649 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 msgr2=0x7ff6cc06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 0x7ff6cc06dc00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc036000 tx=0x7ff6dc03c040 comp rx=0 tx=0).stop 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 msgr2=0x7ff6ec0ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fd0 tx=0x7ff6e400b8e0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 shutdown_connections 2024-04-05T22:32:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff6cc06b740 0x7ff6cc06dc00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6ec149000 0x7ff6ec0ce0f0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6ec1587d0 0x7ff6ec0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 --2- 172.21.15.73:0/201183171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6ec06d0c0 0x7ff6ec148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 >> 172.21.15.73:0/201183171 conn(0x7ff6ec0ce8e0 msgr2=0x7ff6ec15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 shutdown_connections 2024-04-05T22:32:15.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.651+0000 7ff6f1400640 1 -- 172.21.15.73:0/201183171 wait complete. 2024-04-05T22:32:15.669 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-05T22:32:15.624015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291884,"kb_used_data":18376,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754682836,"statfs":{"total":773094113280,"available":772795224064,"internally_reserved":0,"allocated":18817024,"data_stored":8983228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":279957051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-66674,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-84,"num_write_kb":-100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.754029"},"pg_stats":[{"pgid":"1.0","version":"75'128","reported_seq":334,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-04-05T22:32:09.019287+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:32:09.019287+0000","last_peered":"2024-04-05T22:32:09.019287+0000","last_clean":"2024-04-05T22:32:09.019287+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:32:09.019287+0000","last_undegraded":"2024-04-05T22:32:09.019287+0000","last_fullsized":"2024-04-05T22:32:09.019287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37992,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333848,"statfs":{"total":96636764160,"available":96597860352,"internally_reserved":0,"allocated":2846720,"data_stored":1618457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":1.216}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.4650000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.4379999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37348,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2187264,"data_stored":963401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.339}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3899999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.466},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.8129999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.601},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.6379999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37344,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2183168,"data_stored":963098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":1.115},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.522}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":0.87},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.2709999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.216}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.4119999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.444}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.621},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.718},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.163}]}]},{"osd":4,"up_from":14,"seq":60129542279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":1.306}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3839999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.4930000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.625}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.341},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.6819999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33256,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2191360,"data_stored":962698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.159}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.88},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.0229999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542279,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33260,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2195456,"data_stored":968519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.087}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.371}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.145},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.995}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.22}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.3919999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.008}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.236},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.413}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.504},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.379}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.3540000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.327}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.522}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.381},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:32:15.670 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-05T22:32:15.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.874+0000 7f7b02c00640 1 Processor -- start 2024-04-05T22:32:15.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.874+0000 7f7b02c00640 1 -- start start 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b02c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7afc1587d0 0x7f7afc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b02c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 0x7f7afc15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b02c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc06f5b0 con 0x7f7afc0ccd20 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b02c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc15f240 con 0x7f7afc1587d0 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b02c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc070410 con 0x7f7afc15e850 2024-04-05T22:32:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7afbe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7afc1587d0 0x7f7afc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7b00c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 0x7f7afc15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7af3e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7afc0ccd20 0x7f7afc06cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:51870/0 (socket says 172.21.15.73:51870) 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.876+0000 7f7af3e00640 1 -- 172.21.15.73:0/1908743246 learned_addr learned my addr 172.21.15.73:0/1908743246 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.877+0000 7f7b00c00640 1 -- 172.21.15.73:0/1908743246 >> v2:172.21.15.157:3300/0 conn(0x7f7afc1587d0 msgr2=0x7f7afc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.877+0000 7f7b00c00640 1 --2- 172.21.15.73:0/1908743246 >> v2:172.21.15.157:3300/0 conn(0x7f7afc1587d0 0x7f7afc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.877+0000 7f7b00c00640 1 -- 172.21.15.73:0/1908743246 >> v1:172.21.15.157:6789/0 conn(0x7f7afc0ccd20 legacy=0x7f7afc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.877+0000 7f7b00c00640 1 -- 172.21.15.73:0/1908743246 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc158bb0 con 0x7f7afc15e850 2024-04-05T22:32:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.879+0000 7f7b00c00640 1 --2- 172.21.15.73:0/1908743246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 0x7f7afc15ed00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009d90 tx=0x7f7af402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52adc0e2d017ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7afb400640 1 -- 172.21.15.73:0/1908743246 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402cc50 con 0x7f7afc15e850 2024-04-05T22:32:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7afb400640 1 -- 172.21.15.73:0/1908743246 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af402cdb0 con 0x7f7afc15e850 2024-04-05T22:32:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7afb400640 1 -- 172.21.15.73:0/1908743246 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4034660 con 0x7f7afc15e850 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 -- 172.21.15.73:0/1908743246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 msgr2=0x7f7afc15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1908743246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 0x7f7afc15ed00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009d90 tx=0x7f7af402bea0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 -- 172.21.15.73:0/1908743246 shutdown_connections 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1908743246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc15e850 0x7f7afc15ed00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1908743246 >> v2:172.21.15.157:3300/0 conn(0x7f7afc1587d0 0x7f7afc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 -- 172.21.15.73:0/1908743246 >> 172.21.15.73:0/1908743246 conn(0x7f7afc0ce8e0 msgr2=0x7f7afc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 -- 172.21.15.73:0/1908743246 shutdown_connections 2024-04-05T22:32:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.880+0000 7f7b02c00640 1 -- 172.21.15.73:0/1908743246 wait complete. 2024-04-05T22:32:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.882+0000 7f7b02c00640 1 Processor -- start 2024-04-05T22:32:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.882+0000 7f7b02c00640 1 -- start start 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc1587d0 0x7f7afc0af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0a84c0 0x7f7afc0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0a8ca0 con 0x7f7afc15e850 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1430c0 con 0x7f7afc1587d0 2024-04-05T22:32:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7b02c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc12e180 con 0x7f7afc0a84c0 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41520/0 (socket says 172.21.15.73:41520) 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.883+0000 7f7af3e00640 1 -- 172.21.15.73:0/1378287648 learned_addr learned my addr 172.21.15.73:0/1378287648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7afbe00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc1587d0 0x7f7afc0af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7b00c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0a84c0 0x7f7afc0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 -- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0a84c0 msgr2=0x7f7afc0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0a84c0 0x7f7afc0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 -- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc1587d0 msgr2=0x7f7afc0af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc1587d0 0x7f7afc0af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 -- 172.21.15.73:0/1378287648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009a70 con 0x7f7afc15e850 2024-04-05T22:32:15.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af3e00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c4a0 tx=0x7f7ae400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.884+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4005ce0 con 0x7f7afc15e850 2024-04-05T22:32:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.885+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4005e40 con 0x7f7afc15e850 2024-04-05T22:32:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.885+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0a89c0 con 0x7f7afc15e850 2024-04-05T22:32:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.885+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4007270 con 0x7f7afc15e850 2024-04-05T22:32:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.885+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc142d50 con 0x7f7afc15e850 2024-04-05T22:32:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.886+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc066af0 con 0x7f7afc15e850 2024-04-05T22:32:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.887+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f7ae40073d0 con 0x7f7afc15e850 2024-04-05T22:32:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.887+0000 7f7af8c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 0x7f7ad806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.888+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ae40873a0 con 0x7f7afc15e850 2024-04-05T22:32:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.888+0000 7f7afbe00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 0x7f7ad806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:15.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.889+0000 7f7afbe00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 0x7f7ad806dc00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009ad0 tx=0x7f7aec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:15.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:15.892+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f7ae4053e30 con 0x7f7afc15e850 2024-04-05T22:32:16.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.089+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7afc1428d0 con 0x7f7afc15e850 2024-04-05T22:32:16.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.091+0000 7f7af8c00640 1 -- 172.21.15.73:0/1378287648 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+12528 (secure 0 0 0) 0x7f7afc1428d0 con 0x7f7afc15e850 2024-04-05T22:32:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:16.091 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":175,"fsid":"38861c87-67cf-4180-a71f-97401becb77d","created":"2024-04-05T22:20:55.548628+0000","modified":"2024-04-05T22:32:11.893063+0000","last_up_change":"2024-04-05T22:21:07.094714+0000","last_in_change":"2024-04-05T22:20:59.166718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-05T22:21:08.558000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c57082f-7b11-49ad-9aa2-3d7c81942802","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":37310},{"type":"v1","addr":"172.21.15.73:6801","nonce":37310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":37310},{"type":"v1","addr":"172.21.15.73:6803","nonce":37310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":37310},{"type":"v1","addr":"172.21.15.73:6807","nonce":37310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":37310},{"type":"v1","addr":"172.21.15.73:6805","nonce":37310}]},"public_addr":"172.21.15.73:6801/37310","cluster_addr":"172.21.15.73:6803/37310","heartbeat_back_addr":"172.21.15.73:6807/37310","heartbeat_front_addr":"172.21.15.73:6805/37310","state":["exists","up"]},{"osd":1,"uuid":"620a6081-b2dd-4f3d-90de-3cb57d1f6788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":37311},{"type":"v1","addr":"172.21.15.73:6809","nonce":37311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":37311},{"type":"v1","addr":"172.21.15.73:6811","nonce":37311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":37311},{"type":"v1","addr":"172.21.15.73:6818","nonce":37311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":37311},{"type":"v1","addr":"172.21.15.73:6814","nonce":37311}]},"public_addr":"172.21.15.73:6809/37311","cluster_addr":"172.21.15.73:6811/37311","heartbeat_back_addr":"172.21.15.73:6818/37311","heartbeat_front_addr":"172.21.15.73:6814/37311","state":["exists","up"]},{"osd":2,"uuid":"236682ef-6293-4b5c-837f-c010a131e2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":37323},{"type":"v1","addr":"172.21.15.73:6815","nonce":37323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6817","nonce":37323},{"type":"v1","addr":"172.21.15.73:6819","nonce":37323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":37323},{"type":"v1","addr":"172.21.15.73:6823","nonce":37323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":37323},{"type":"v1","addr":"172.21.15.73:6821","nonce":37323}]},"public_addr":"172.21.15.73:6815/37323","cluster_addr":"172.21.15.73:6819/37323","heartbeat_back_addr":"172.21.15.73:6823/37323","heartbeat_front_addr":"172.21.15.73:6821/37323","state":["exists","up"]},{"osd":3,"uuid":"48454405-0107-42fb-a71e-ef13d6c8198c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":37326},{"type":"v1","addr":"172.21.15.73:6825","nonce":37326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":37326},{"type":"v1","addr":"172.21.15.73:6827","nonce":37326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":37326},{"type":"v1","addr":"172.21.15.73:6831","nonce":37326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":37326},{"type":"v1","addr":"172.21.15.73:6829","nonce":37326}]},"public_addr":"172.21.15.73:6825/37326","cluster_addr":"172.21.15.73:6827/37326","heartbeat_back_addr":"172.21.15.73:6831/37326","heartbeat_front_addr":"172.21.15.73:6829/37326","state":["exists","up"]},{"osd":4,"uuid":"3be8f606-e96f-484d-9db0-a5b60f39ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":37592},{"type":"v1","addr":"172.21.15.157:6801","nonce":37592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":37592},{"type":"v1","addr":"172.21.15.157:6803","nonce":37592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":37592},{"type":"v1","addr":"172.21.15.157:6807","nonce":37592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":37592},{"type":"v1","addr":"172.21.15.157:6805","nonce":37592}]},"public_addr":"172.21.15.157:6801/37592","cluster_addr":"172.21.15.157:6803/37592","heartbeat_back_addr":"172.21.15.157:6807/37592","heartbeat_front_addr":"172.21.15.157:6805/37592","state":["exists","up"]},{"osd":5,"uuid":"5dcdb399-2827-4da5-911d-dc7ced11744c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":37594},{"type":"v1","addr":"172.21.15.157:6822","nonce":37594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":37594},{"type":"v1","addr":"172.21.15.157:6826","nonce":37594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":37594},{"type":"v1","addr":"172.21.15.157:6831","nonce":37594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":37594},{"type":"v1","addr":"172.21.15.157:6829","nonce":37594}]},"public_addr":"172.21.15.157:6822/37594","cluster_addr":"172.21.15.157:6826/37594","heartbeat_back_addr":"172.21.15.157:6831/37594","heartbeat_front_addr":"172.21.15.157:6829/37594","state":["exists","up"]},{"osd":6,"uuid":"8047a6f8-c2c1-462e-876e-7700088d2ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":37593},{"type":"v1","addr":"172.21.15.157:6809","nonce":37593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":37593},{"type":"v1","addr":"172.21.15.157:6811","nonce":37593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":37593},{"type":"v1","addr":"172.21.15.157:6815","nonce":37593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":37593},{"type":"v1","addr":"172.21.15.157:6813","nonce":37593}]},"public_addr":"172.21.15.157:6809/37593","cluster_addr":"172.21.15.157:6811/37593","heartbeat_back_addr":"172.21.15.157:6815/37593","heartbeat_front_addr":"172.21.15.157:6813/37593","state":["exists","up"]},{"osd":7,"uuid":"09e1e050-1cef-45dc-9aa2-2fa40597a903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":37603},{"type":"v1","addr":"172.21.15.157:6817","nonce":37603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":37603},{"type":"v1","addr":"172.21.15.157:6819","nonce":37603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":37603},{"type":"v1","addr":"172.21.15.157:6827","nonce":37603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":37603},{"type":"v1","addr":"172.21.15.157:6823","nonce":37603}]},"public_addr":"172.21.15.157:6817/37603","cluster_addr":"172.21.15.157:6819/37603","heartbeat_back_addr":"172.21.15.157:6827/37603","heartbeat_front_addr":"172.21.15.157:6823/37603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.407971+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.456172+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.496639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.485173+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.402142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.514427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.744055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-05T22:21:04.550833+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6834/1107789803":"2024-04-06T22:32:08.864428+0000","172.21.15.73:6832/1107789803":"2024-04-06T22:32:08.864428+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.093+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 msgr2=0x7f7ad806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.093+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 0x7f7ad806dc00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009ad0 tx=0x7f7aec008040 comp rx=0 tx=0).stop 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.093+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 msgr2=0x7f7afc12dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.093+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c4a0 tx=0x7f7ae400c960 comp rx=0 tx=0).stop 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 shutdown_connections 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f7ad806b740 0x7f7ad806dc00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:16.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7afc0a84c0 0x7f7afc0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7afc15e850 0x7f7afc12dc40 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 --2- 172.21.15.73:0/1378287648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7afc1587d0 0x7f7afc0af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 >> 172.21.15.73:0/1378287648 conn(0x7f7afc0ce8e0 msgr2=0x7f7afc15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 shutdown_connections 2024-04-05T22:32:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:16.094+0000 7f7b02c00640 1 -- 172.21.15.73:0/1378287648 wait complete. 2024-04-05T22:32:17.113 INFO:tasks.ceph:Scrubbing osd.0 2024-04-05T22:32:17.114 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:17.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.337+0000 7ffaa6600640 1 Processor -- start 2024-04-05T22:32:17.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.337+0000 7ffaa6600640 1 -- start start 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.338+0000 7ffaa6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffaa01541a0 0x7ffaa00cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.338+0000 7ffaa6600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 0x7ffaa00cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.338+0000 7ffaa6600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa015d900 con 0x7ffaa00b2840 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.338+0000 7ffaa6600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa015e1a0 con 0x7ffaa01541a0 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.338+0000 7ffaa6600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa00b3490 con 0x7ffaa0154580 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffaa01541a0 0x7ffaa00cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffaa01541a0 0x7ffaa00cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41524/0 (socket says 172.21.15.73:41524) 2024-04-05T22:32:17.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9f400640 1 -- 172.21.15.73:0/1231633195 learned_addr learned my addr 172.21.15.73:0/1231633195 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9f400640 1 -- 172.21.15.73:0/1231633195 >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 msgr2=0x7ffaa00cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9ea00640 1 --2- 172.21.15.73:0/1231633195 >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 0x7ffaa00cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.339+0000 7ffa9e000640 1 -- 172.21.15.73:0/1231633195 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660993347 0 0) 0x7ffaa015d900 con 0x7ffaa00b2840 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.340+0000 7ffa9f400640 1 --2- 172.21.15.73:0/1231633195 >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 0x7ffaa00cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.340+0000 7ffa9f400640 1 -- 172.21.15.73:0/1231633195 >> v1:172.21.15.73:6789/0 conn(0x7ffaa00b2840 legacy=0x7ffaa00b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.340+0000 7ffa9f400640 1 -- 172.21.15.73:0/1231633195 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa015db50 con 0x7ffaa01541a0 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.340+0000 7ffa9ea00640 1 --2- 172.21.15.73:0/1231633195 >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 0x7ffaa00cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffa9f400640 1 --2- 172.21.15.73:0/1231633195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa00cecd0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00ce3d0 tx=0x7ffa9002e190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c849e4ab3562f85d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffa9e000640 1 -- 172.21.15.73:0/1231633195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000b0d0 con 0x7ffaa01541a0 2024-04-05T22:32:17.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffa9e000640 1 -- 172.21.15.73:0/1231633195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000b230 con 0x7ffaa01541a0 2024-04-05T22:32:17.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffa9e000640 1 -- 172.21.15.73:0/1231633195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90009600 con 0x7ffaa01541a0 2024-04-05T22:32:17.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffaa6600640 1 -- 172.21.15.73:0/1231633195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 msgr2=0x7ffaa00cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.341+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1231633195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa00cecd0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00ce3d0 tx=0x7ffa9002e190 comp rx=0 tx=0).stop 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 -- 172.21.15.73:0/1231633195 shutdown_connections 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1231633195 >> v2:172.21.15.157:3300/0 conn(0x7ffaa0154580 0x7ffaa00cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1231633195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa00cecd0 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 -- 172.21.15.73:0/1231633195 >> 172.21.15.73:0/1231633195 conn(0x7ffaa006a0d0 msgr2=0x7ffaa006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 -- 172.21.15.73:0/1231633195 shutdown_connections 2024-04-05T22:32:17.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.342+0000 7ffaa6600640 1 -- 172.21.15.73:0/1231633195 wait complete. 2024-04-05T22:32:17.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.343+0000 7ffaa6600640 1 Processor -- start 2024-04-05T22:32:17.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.344+0000 7ffaa6600640 1 -- start start 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.344+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffaa00b2840 0x7ffaa014b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.344+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.344+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa0154580 0x7ffaa015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.344+0000 7ffaa6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0152c60 con 0x7ffaa01541a0 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffaa6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00af6f0 con 0x7ffaa0154580 2024-04-05T22:32:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffaa6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa014b880 con 0x7ffaa00b2840 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41534/0 (socket says 172.21.15.73:41534) 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 -- 172.21.15.73:0/1193139081 learned_addr learned my addr 172.21.15.73:0/1193139081 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 -- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffaa00b2840 msgr2=0x7ffaa014b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffaa00b2840 0x7ffaa014b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 -- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa0154580 msgr2=0x7ffaa015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa0154580 0x7ffaa015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9000bd40 con 0x7ffaa01541a0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.345+0000 7ffa9ea00640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa015c500 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c7c0 tx=0x7ffa8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.346+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c007820 con 0x7ffaa01541a0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.346+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c007980 con 0x7ffaa01541a0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.346+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c007d10 con 0x7ffaa01541a0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.346+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa01555b0 con 0x7ffaa01541a0 2024-04-05T22:32:17.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.346+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa0155b70 con 0x7ffaa01541a0 2024-04-05T22:32:17.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.347+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaa006b420 con 0x7ffaa01541a0 2024-04-05T22:32:17.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.348+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7ffa8c0113f0 con 0x7ffaa01541a0 2024-04-05T22:32:17.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa97400640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa8006b740 0x7ffa8006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa8c087fc0 con 0x7ffaa01541a0 2024-04-05T22:32:17.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa97400640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7ffa800715a0 0x7ffa800739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa80074070 con 0x7ffa800715a0 2024-04-05T22:32:17.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa8c08ce10 con 0x7ffaa01541a0 2024-04-05T22:32:17.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa9fe00640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7ffa800715a0 0x7ffa800739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.349+0000 7ffa9f400640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa8006b740 0x7ffa8006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.350+0000 7ffa9fe00640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7ffa800715a0 0x7ffa800739c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.351+0000 7ffa9f400640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa8006b740 0x7ffa8006dc00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ffa90037000 tx=0x7ffa90030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.351+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== osd.0 v2:172.21.15.73:6800/37310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ffa80074070 con 0x7ffa800715a0 2024-04-05T22:32:17.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.365+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 --> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffaa0068160 con 0x7ffa800715a0 2024-04-05T22:32:17.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.365+0000 7ffa97400640 1 -- 172.21.15.73:0/1193139081 <== osd.0 v2:172.21.15.73:6800/37310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ffaa0068160 con 0x7ffa800715a0 2024-04-05T22:32:17.365 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:17.366 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:17.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.365+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7ffa800715a0 msgr2=0x7ffa800739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.365+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:6800/37310,v1:172.21.15.73:6801/37310] conn(0x7ffa800715a0 0x7ffa800739c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa8006b740 msgr2=0x7ffa8006dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ffa8006b740 0x7ffa8006dc00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ffa90037000 tx=0x7ffa90030000 comp rx=0 tx=0).stop 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 msgr2=0x7ffaa015c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 --2- 172.21.15.73:0/1193139081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa01541a0 0x7ffaa015c500 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c7c0 tx=0x7ffa8c00cc80 comp rx=0 tx=0).stop 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffa9fe00640 1 -- 172.21.15.73:0/1193139081 reap_dead start 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 shutdown_connections 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 >> 172.21.15.73:0/1193139081 conn(0x7ffaa006a0d0 msgr2=0x7ffaa006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 shutdown_connections 2024-04-05T22:32:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.366+0000 7ffaa6600640 1 -- 172.21.15.73:0/1193139081 wait complete. 2024-04-05T22:32:17.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-05T22:32:17.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.590+0000 7f6461a00640 1 Processor -- start 2024-04-05T22:32:17.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.590+0000 7f6461a00640 1 -- start start 2024-04-05T22:32:17.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6461a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f645c06cb80 0x7f645c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6461a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c15e850 0x7f645c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6461a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c06f5b0 con 0x7f645c1587d0 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6461a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c06d500 con 0x7f645c06cb80 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6461a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15f220 con 0x7f645c15e850 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6452000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f645c06cb80 0x7f645c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6452000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f645c06cb80 0x7f645c06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41538/0 (socket says 172.21.15.73:41538) 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.592+0000 7f6452000640 1 -- 172.21.15.73:0/3211483845 learned_addr learned my addr 172.21.15.73:0/3211483845 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:17.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.593+0000 7f645b400640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c15e850 0x7f645c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.593+0000 7f6452000640 1 -- 172.21.15.73:0/3211483845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c15e850 msgr2=0x7f645c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.593+0000 7f6452000640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c15e850 0x7f645c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.593+0000 7f6452000640 1 -- 172.21.15.73:0/3211483845 >> v1:172.21.15.157:6789/0 conn(0x7f645c1587d0 legacy=0x7f645c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.593+0000 7f6452000640 1 -- 172.21.15.73:0/3211483845 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c070410 con 0x7f645c06cb80 2024-04-05T22:32:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.594+0000 7f6452000640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c06cfc0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f64440098a0 tx=0x7f644402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210c92617c8ad9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f645a000640 1 -- 172.21.15.73:0/3211483845 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444030840 con 0x7f645c06cb80 2024-04-05T22:32:17.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f645a000640 1 -- 172.21.15.73:0/3211483845 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64440309a0 con 0x7f645c06cb80 2024-04-05T22:32:17.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f645a000640 1 -- 172.21.15.73:0/3211483845 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444030d30 con 0x7f645c06cb80 2024-04-05T22:32:17.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 -- 172.21.15.73:0/3211483845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 msgr2=0x7f645c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c06cfc0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f64440098a0 tx=0x7f644402b980 comp rx=0 tx=0).stop 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 -- 172.21.15.73:0/3211483845 shutdown_connections 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c15e850 0x7f645c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 --2- 172.21.15.73:0/3211483845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c06cfc0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 -- 172.21.15.73:0/3211483845 >> 172.21.15.73:0/3211483845 conn(0x7f645c0ce8e0 msgr2=0x7f645c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 -- 172.21.15.73:0/3211483845 shutdown_connections 2024-04-05T22:32:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.595+0000 7f6461a00640 1 -- 172.21.15.73:0/3211483845 wait complete. 2024-04-05T22:32:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.597+0000 7f6461a00640 1 Processor -- start 2024-04-05T22:32:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.597+0000 7f6461a00640 1 -- start start 2024-04-05T22:32:17.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 0x7f645c1f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 0x7f645c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c154cc0 con 0x7f645c06cb80 2024-04-05T22:32:17.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1f3490 con 0x7f645c15e850 2024-04-05T22:32:17.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f6461a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1593c0 con 0x7f645c1587d0 2024-04-05T22:32:17.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f645aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f645aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41548/0 (socket says 172.21.15.73:41548) 2024-04-05T22:32:17.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.598+0000 7f645aa00640 1 -- 172.21.15.73:0/2868359544 learned_addr learned my addr 172.21.15.73:0/2868359544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f6452000640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 0x7f645c1f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645b400640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 0x7f645c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 -- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 msgr2=0x7f645c1f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 0x7f645c1f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 -- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 msgr2=0x7f645c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 0x7f645c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 -- 172.21.15.73:0/2868359544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440094d0 con 0x7f645c06cb80 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645b400640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 0x7f645c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f6452000640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 0x7f645c1f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f645aa00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c450 tx=0x7f644c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0052d0 con 0x7f645c06cb80 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.599+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c005430 con 0x7f645c06cb80 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.600+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c1596a0 con 0x7f645c06cb80 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.600+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c012870 con 0x7f645c06cb80 2024-04-05T22:32:17.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.600+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c0b1c50 con 0x7f645c06cb80 2024-04-05T22:32:17.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.601+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c066af0 con 0x7f645c06cb80 2024-04-05T22:32:17.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.602+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f644c0129d0 con 0x7f645c06cb80 2024-04-05T22:32:17.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.603+0000 7f6453400640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 0x7f643806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:17.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.603+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f644c087b80 con 0x7f645c06cb80 2024-04-05T22:32:17.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.603+0000 7f6452000640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 0x7f643806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:17.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.604+0000 7f6452000640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 0x7f643806dc00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6444036000 tx=0x7f644403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:17.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.607+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f644c0544a0 con 0x7f645c06cb80 2024-04-05T22:32:17.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.800+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f645c06f800 con 0x7f643806b740 2024-04-05T22:32:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.801+0000 7f6453400640 1 -- 172.21.15.73:0/2868359544 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6444030be0 con 0x7f643806b740 2024-04-05T22:32:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 msgr2=0x7f643806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 0x7f643806dc00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6444036000 tx=0x7f644403b040 comp rx=0 tx=0).stop 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 msgr2=0x7f645c154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c450 tx=0x7f644c00c910 comp rx=0 tx=0).stop 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 shutdown_connections 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f643806b740 0x7f643806dc00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f645c15e850 0x7f645c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f645c1587d0 0x7f645c1f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 --2- 172.21.15.73:0/2868359544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f645c06cb80 0x7f645c154780 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:17.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.803+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 >> 172.21.15.73:0/2868359544 conn(0x7f645c0ce8e0 msgr2=0x7f645c15dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:17.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.804+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 shutdown_connections 2024-04-05T22:32:17.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:17.804+0000 7f6461a00640 1 -- 172.21.15.73:0/2868359544 wait complete. 2024-04-05T22:32:17.823 INFO:tasks.ceph:Scrubbing osd.1 2024-04-05T22:32:17.824 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:18.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.041+0000 7fb872800640 1 Processor -- start 2024-04-05T22:32:18.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.041+0000 7fb872800640 1 -- start start 2024-04-05T22:32:18.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb872800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb872800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c15b230 con 0x7fb86c0cce90 2024-04-05T22:32:18.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb872800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c15f4a0 con 0x7fb86c15ae50 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb872800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c0ce2f0 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb86aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb86aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41558/0 (socket says 172.21.15.73:41558) 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.043+0000 7fb86aa00640 1 -- 172.21.15.73:0/3655131681 learned_addr learned my addr 172.21.15.73:0/3655131681 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.044+0000 7fb86aa00640 1 -- 172.21.15.73:0/3655131681 >> v1:172.21.15.157:6789/0 conn(0x7fb86c15ae50 legacy=0x7fb86c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:18.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.044+0000 7fb86aa00640 1 -- 172.21.15.73:0/3655131681 >> v1:172.21.15.73:6789/0 conn(0x7fb86c0cce90 legacy=0x7fb86c0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.044+0000 7fb86aa00640 1 -- 172.21.15.73:0/3655131681 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c15f780 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.045+0000 7fb86aa00640 1 --2- 172.21.15.73:0/3655131681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb854009870 tx=0x7fb85402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232e94890179b1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.045+0000 7fb86a000640 1 -- 172.21.15.73:0/3655131681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854031870 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.045+0000 7fb86a000640 1 -- 172.21.15.73:0/3655131681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8540319d0 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.045+0000 7fb86a000640 1 -- 172.21.15.73:0/3655131681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854031d60 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 -- 172.21.15.73:0/3655131681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c0cf3e0 msgr2=0x7fb86c0b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 --2- 172.21.15.73:0/3655131681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb854009870 tx=0x7fb85402b910 comp rx=0 tx=0).stop 2024-04-05T22:32:18.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 -- 172.21.15.73:0/3655131681 shutdown_connections 2024-04-05T22:32:18.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 --2- 172.21.15.73:0/3655131681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c0cf3e0 0x7fb86c0b3aa0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 -- 172.21.15.73:0/3655131681 >> 172.21.15.73:0/3655131681 conn(0x7fb86c06a0d0 msgr2=0x7fb86c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 -- 172.21.15.73:0/3655131681 shutdown_connections 2024-04-05T22:32:18.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.046+0000 7fb872800640 1 -- 172.21.15.73:0/3655131681 wait complete. 2024-04-05T22:32:18.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.048+0000 7fb872800640 1 Processor -- start 2024-04-05T22:32:18.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.048+0000 7fb872800640 1 -- start start 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb86c0cce90 0x7fb86c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86c0cf3e0 0x7fb86c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 0x7fb86c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c153230 con 0x7fb86c15ae50 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c155630 con 0x7fb86c0cce90 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb872800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c155910 con 0x7fb86c0cf3e0 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb86be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 0x7fb86c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb86be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 0x7fb86c1430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41570/0 (socket says 172.21.15.73:41570) 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.049+0000 7fb86be00640 1 -- 172.21.15.73:0/1951641517 learned_addr learned my addr 172.21.15.73:0/1951641517 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86aa00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86c0cf3e0 0x7fb86c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86b400640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb86c0cce90 0x7fb86c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 -- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86c0cf3e0 msgr2=0x7fb86c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86c0cf3e0 0x7fb86c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 -- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb86c0cce90 msgr2=0x7fb86c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb86c0cce90 0x7fb86c15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8540094d0 con 0x7fb86c15ae50 2024-04-05T22:32:18.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb86be00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 0x7fb86c1430b0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb864001eb0 tx=0x7fb86400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8640052d0 con 0x7fb86c15ae50 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864005430 con 0x7fb86c15ae50 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864003810 con 0x7fb86c15ae50 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.050+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c155bf0 con 0x7fb86c15ae50 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.051+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c156110 con 0x7fb86c15ae50 2024-04-05T22:32:18.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.051+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb86c1557a0 con 0x7fb86c15ae50 2024-04-05T22:32:18.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.053+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fb864003970 con 0x7fb86c15ae50 2024-04-05T22:32:18.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.053+0000 7fb863400640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb84806b770 0x7fb84806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb864087bc0 con 0x7fb86c15ae50 2024-04-05T22:32:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb863400640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7fb8480715d0 0x7fb8480739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8480740a0 con 0x7fb8480715d0 2024-04-05T22:32:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb86408a6a0 con 0x7fb86c15ae50 2024-04-05T22:32:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb86b400640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb84806b770 0x7fb84806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.054+0000 7fb86aa00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7fb8480715d0 0x7fb8480739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.055+0000 7fb86aa00640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7fb8480715d0 0x7fb8480739f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.055+0000 7fb86b400640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb84806b770 0x7fb84806dc30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009f20 tx=0x7fb85c0094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.056+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== osd.1 v2:172.21.15.73:6808/37311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb8480740a0 con 0x7fb8480715d0 2024-04-05T22:32:18.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.069+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 --> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb86c0cb610 con 0x7fb8480715d0 2024-04-05T22:32:18.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb863400640 1 -- 172.21.15.73:0/1951641517 <== osd.1 v2:172.21.15.73:6808/37311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb854031810 con 0x7fb8480715d0 2024-04-05T22:32:18.070 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:18.070 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:18.070 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:18.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7fb8480715d0 msgr2=0x7fb8480739f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:6808/37311,v1:172.21.15.73:6809/37311] conn(0x7fb8480715d0 0x7fb8480739f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb84806b770 msgr2=0x7fb84806dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fb84806b770 0x7fb84806dc30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009f20 tx=0x7fb85c0094d0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 msgr2=0x7fb86c1430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb872800640 1 --2- 172.21.15.73:0/1951641517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb86c15ae50 0x7fb86c1430b0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb864001eb0 tx=0x7fb86400c890 comp rx=0 tx=0).stop 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.070+0000 7fb86be00640 1 -- 172.21.15.73:0/1951641517 reap_dead start 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.071+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 shutdown_connections 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.071+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 >> 172.21.15.73:0/1951641517 conn(0x7fb86c06a0d0 msgr2=0x7fb86c06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.071+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 shutdown_connections 2024-04-05T22:32:18.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.071+0000 7fb872800640 1 -- 172.21.15.73:0/1951641517 wait complete. 2024-04-05T22:32:18.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-05T22:32:18.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.303+0000 7fa2f7e00640 1 Processor -- start 2024-04-05T22:32:18.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.303+0000 7fa2f7e00640 1 -- start start 2024-04-05T22:32:18.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.304+0000 7fa2f7e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2f8158860 0x7fa2f80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.304+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.304+0000 7fa2f7e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f806b500 con 0x7fa2f815eb60 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.304+0000 7fa2f7e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f815eff0 con 0x7fa2f8158860 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.304+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80ccaf0 con 0x7fa2f80b1f00 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.305+0000 7fa2ee000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.305+0000 7fa2ee000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:40038/0 (socket says 172.21.15.73:40038) 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.305+0000 7fa2f7400640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa2f815eb60 0x7fa2f815afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.73:51906/0 (socket says 172.21.15.73:51906) 2024-04-05T22:32:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.305+0000 7fa2ee000640 1 -- 172.21.15.73:0/2626290405 learned_addr learned my addr 172.21.15.73:0/2626290405 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.305+0000 7fa2f6a00640 1 --2- 172.21.15.73:0/2626290405 >> v2:172.21.15.157:3300/0 conn(0x7fa2f8158860 0x7fa2f80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.306+0000 7fa2ee000640 1 -- 172.21.15.73:0/2626290405 >> v2:172.21.15.157:3300/0 conn(0x7fa2f8158860 msgr2=0x7fa2f80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.306+0000 7fa2ee000640 1 --2- 172.21.15.73:0/2626290405 >> v2:172.21.15.157:3300/0 conn(0x7fa2f8158860 0x7fa2f80b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.306+0000 7fa2ee000640 1 -- 172.21.15.73:0/2626290405 >> v1:172.21.15.157:6789/0 conn(0x7fa2f815eb60 legacy=0x7fa2f815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.306+0000 7fa2ee000640 1 -- 172.21.15.73:0/2626290405 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80b23a0 con 0x7fa2f80b1f00 2024-04-05T22:32:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.306+0000 7fa2f6000640 1 -- 172.21.15.73:0/2626290405 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546276486 0 0) 0x7fa2f806b500 con 0x7fa2f815eb60 2024-04-05T22:32:18.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.308+0000 7fa2ee000640 1 --2- 172.21.15.73:0/2626290405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009870 tx=0x7fa2e002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=376b60b977899e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.308+0000 7fa2f6000640 1 -- 172.21.15.73:0/2626290405 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0031640 con 0x7fa2f80b1f00 2024-04-05T22:32:18.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.308+0000 7fa2f6000640 1 -- 172.21.15.73:0/2626290405 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e00317a0 con 0x7fa2f80b1f00 2024-04-05T22:32:18.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.308+0000 7fa2f6000640 1 -- 172.21.15.73:0/2626290405 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0031b30 con 0x7fa2f80b1f00 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 -- 172.21.15.73:0/2626290405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 msgr2=0x7fa2f815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/2626290405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009870 tx=0x7fa2e002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 -- 172.21.15.73:0/2626290405 shutdown_connections 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/2626290405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f815e620 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/2626290405 >> v2:172.21.15.157:3300/0 conn(0x7fa2f8158860 0x7fa2f80b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 -- 172.21.15.73:0/2626290405 >> 172.21.15.73:0/2626290405 conn(0x7fa2f80b06c0 msgr2=0x7fa2f80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 -- 172.21.15.73:0/2626290405 shutdown_connections 2024-04-05T22:32:18.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.309+0000 7fa2f7e00640 1 -- 172.21.15.73:0/2626290405 wait complete. 2024-04-05T22:32:18.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.311+0000 7fa2f7e00640 1 Processor -- start 2024-04-05T22:32:18.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.311+0000 7fa2f7e00640 1 -- start start 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f814a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2f815eb60 0x7fa2f81570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8157ef0 con 0x7fa2f814aba0 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f815c330 con 0x7fa2f815eb60 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8157930 con 0x7fa2f80b1f00 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41586/0 (socket says 172.21.15.73:41586) 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f7400640 1 -- 172.21.15.73:0/721749419 learned_addr learned my addr 172.21.15.73:0/721749419 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.312+0000 7fa2f6a00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f814a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2ee000640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2f815eb60 0x7fa2f81570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 -- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 msgr2=0x7fa2f814a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f814a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 -- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2f815eb60 msgr2=0x7fa2f81570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2f815eb60 0x7fa2f81570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 -- 172.21.15.73:0/721749419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00094d0 con 0x7fa2f814aba0 2024-04-05T22:32:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7400640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c7a0 tx=0x7fa2f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000a640 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f000a7a0 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000abd0 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f8157c10 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.313+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f8157620 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.315+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f815c190 con 0x7fa2f814aba0 2024-04-05T22:32:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.316+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fa2f000ad30 con 0x7fa2f814aba0 2024-04-05T22:32:18.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.317+0000 7fa2efe00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 0x7fa2d406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.317+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa2f00874e0 con 0x7fa2f814aba0 2024-04-05T22:32:18.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.318+0000 7fa2f6a00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 0x7fa2d406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.319+0000 7fa2f6a00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 0x7fa2d406dc00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ad0 tx=0x7fa2e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.321+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fa2f815c190 con 0x7fa2f814aba0 2024-04-05T22:32:18.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.516+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2f806b750 con 0x7fa2d406b740 2024-04-05T22:32:18.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.518+0000 7fa2efe00640 1 -- 172.21.15.73:0/721749419 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa2e80119a0 con 0x7fa2d406b740 2024-04-05T22:32:18.517 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.519+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 msgr2=0x7fa2d406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.519+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 0x7fa2d406dc00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ad0 tx=0x7fa2e8008040 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.519+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 msgr2=0x7fa2f8159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.519+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c7a0 tx=0x7fa2f000cc60 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 shutdown_connections 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fa2d406b740 0x7fa2d406dc00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2f814aba0 0x7fa2f8159ca0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2f815eb60 0x7fa2f81570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 --2- 172.21.15.73:0/721749419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2f80b1f00 0x7fa2f814a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 >> 172.21.15.73:0/721749419 conn(0x7fa2f80b06c0 msgr2=0x7fa2f80cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 shutdown_connections 2024-04-05T22:32:18.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.520+0000 7fa2f7e00640 1 -- 172.21.15.73:0/721749419 wait complete. 2024-04-05T22:32:18.538 INFO:tasks.ceph:Scrubbing osd.2 2024-04-05T22:32:18.538 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:18.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.757+0000 7f8da1000640 1 Processor -- start 2024-04-05T22:32:18.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.757+0000 7f8da1000640 1 -- start start 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c15ac00 0x7f8d9c06cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d9c0cf210 0x7f8d9c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cf6a0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c15afe0 con 0x7f8d9c0cf210 2024-04-05T22:32:18.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8da1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c161760 con 0x7f8d9c15ac00 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8d99400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8d99400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41596/0 (socket says 172.21.15.73:41596) 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.759+0000 7f8d99400640 1 -- 172.21.15.73:0/698863658 learned_addr learned my addr 172.21.15.73:0/698863658 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99e00640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c15ac00 0x7f8d9c06cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d9a800640 1 --2- 172.21.15.73:0/698863658 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c0cf210 0x7f8d9c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99400640 1 -- 172.21.15.73:0/698863658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c15ac00 msgr2=0x7f8d9c06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99400640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c15ac00 0x7f8d9c06cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99400640 1 -- 172.21.15.73:0/698863658 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c0cf210 msgr2=0x7f8d9c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99400640 1 --2- 172.21.15.73:0/698863658 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c0cf210 0x7f8d9c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.760+0000 7f8d99400640 1 -- 172.21.15.73:0/698863658 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c06d4c0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.761+0000 7f8d99400640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cb300 tx=0x7f8d8802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8323fbcb52791e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.761+0000 7f8d98a00640 1 -- 172.21.15.73:0/698863658 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d880317e0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.761+0000 7f8d98a00640 1 -- 172.21.15.73:0/698863658 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88031940 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.761+0000 7f8d98a00640 1 -- 172.21.15.73:0/698863658 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88031cd0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 -- 172.21.15.73:0/698863658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 msgr2=0x7f8d9c0cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cb300 tx=0x7f8d8802b880 comp rx=0 tx=0).stop 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 -- 172.21.15.73:0/698863658 shutdown_connections 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cecd0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 --2- 172.21.15.73:0/698863658 >> v2:172.21.15.157:3300/0 conn(0x7f8d9c0cf210 0x7f8d9c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 --2- 172.21.15.73:0/698863658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c15ac00 0x7f8d9c06cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 -- 172.21.15.73:0/698863658 >> 172.21.15.73:0/698863658 conn(0x7f8d9c06a0d0 msgr2=0x7f8d9c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 -- 172.21.15.73:0/698863658 shutdown_connections 2024-04-05T22:32:18.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.762+0000 7f8da1000640 1 -- 172.21.15.73:0/698863658 wait complete. 2024-04-05T22:32:18.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.764+0000 7f8da1000640 1 Processor -- start 2024-04-05T22:32:18.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.764+0000 7f8da1000640 1 -- start start 2024-04-05T22:32:18.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c0cf210 0x7f8d9c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15ac00 0x7f8d9c1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1f38b0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c15f000 con 0x7f8d9c15ac00 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8da1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c15f2e0 con 0x7f8d9c0cf210 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.765+0000 7f8d99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41604/0 (socket says 172.21.15.73:41604) 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 -- 172.21.15.73:0/1970179442 learned_addr learned my addr 172.21.15.73:0/1970179442 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d9a800640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15ac00 0x7f8d9c1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 -- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c0cf210 msgr2=0x7f8d9c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99400640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c0cf210 0x7f8d9c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d9c0cf210 0x7f8d9c0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 -- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15ac00 msgr2=0x7f8d9c1540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15ac00 0x7f8d9c1540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880094d0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d9a800640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9c15ac00 0x7f8d9c1540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.766+0000 7f8d99e00640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cb300 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d5f0 tx=0x7f8d8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.767+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0125f0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.767+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c012750 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.767+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c012b00 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.767+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c15f5c0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.767+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c0cb840 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.768+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d9c15f170 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.769+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f8d8c012c60 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d92000640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7c06b740 0x7f8d7c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d8c0881f0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d92000640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f8d7c0715a0 0x7f8d7c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d7c074070 con 0x7f8d7c0715a0 2024-04-05T22:32:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d8c08f2a0 con 0x7f8d9c0b36e0 2024-04-05T22:32:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d9a800640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f8d7c0715a0 0x7f8d7c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.770+0000 7f8d99400640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7c06b740 0x7f8d7c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.771+0000 7f8d9a800640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f8d7c0715a0 0x7f8d7c0739c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.772+0000 7f8d99400640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7c06b740 0x7f8d7c06dc00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8d88036000 tx=0x7f8d8803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.772+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== osd.2 v2:172.21.15.73:6813/37323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8d7c074070 con 0x7f8d7c0715a0 2024-04-05T22:32:18.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.785+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 --> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8d9c161110 con 0x7f8d7c0715a0 2024-04-05T22:32:18.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8d92000640 1 -- 172.21.15.73:0/1970179442 <== osd.2 v2:172.21.15.73:6813/37323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8d94003990 con 0x7f8d7c0715a0 2024-04-05T22:32:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:18.786 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:18.786 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f8d7c0715a0 msgr2=0x7f8d7c0739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:6813/37323,v1:172.21.15.73:6815/37323] conn(0x7f8d7c0715a0 0x7f8d7c0739c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7c06b740 msgr2=0x7f8d7c06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f8d7c06b740 0x7f8d7c06dc00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8d88036000 tx=0x7f8d8803d040 comp rx=0 tx=0).stop 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 msgr2=0x7f8d9c0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 --2- 172.21.15.73:0/1970179442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c0b36e0 0x7f8d9c0cb300 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d5f0 tx=0x7f8d8c00dab0 comp rx=0 tx=0).stop 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8d9a800640 1 -- 172.21.15.73:0/1970179442 reap_dead start 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 shutdown_connections 2024-04-05T22:32:18.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 >> 172.21.15.73:0/1970179442 conn(0x7f8d9c06a0d0 msgr2=0x7f8d9c0b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:18.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 shutdown_connections 2024-04-05T22:32:18.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:18.786+0000 7f8da1000640 1 -- 172.21.15.73:0/1970179442 wait complete. 2024-04-05T22:32:18.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-05T22:32:19.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.009+0000 7feef0200640 1 Processor -- start 2024-04-05T22:32:19.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.010+0000 7feef0200640 1 -- start start 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feef0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815cf70 0x7feee80d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feef0200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feee80d0bd0 0x7feee80d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feef0200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80ceee0 con 0x7feee815d350 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feef0200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80ce5b0 con 0x7feee80d0bd0 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feef0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ce890 con 0x7feee815cf70 2024-04-05T22:32:19.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feeee200640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feee815d350 0x7feee80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44146/0 (socket says 172.21.15.73:44146) 2024-04-05T22:32:19.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.011+0000 7feeee200640 1 -- 172.21.15.73:0/3056664686 learned_addr learned my addr 172.21.15.73:0/3056664686 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feeece00640 1 --2- 172.21.15.73:0/3056664686 >> v2:172.21.15.157:3300/0 conn(0x7feee80d0bd0 0x7feee80d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feeed800640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815cf70 0x7feee80d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081241004 0 0) 0x7feee80ceee0 con 0x7feee815d350 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feecc003520 con 0x7feee815d350 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640888488 0 0) 0x7feecc003520 con 0x7feee815d350 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815cf70 msgr2=0x7feee80d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815cf70 0x7feee80d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 >> v2:172.21.15.157:3300/0 conn(0x7feee80d0bd0 msgr2=0x7feee80d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 --2- 172.21.15.73:0/3056664686 >> v2:172.21.15.157:3300/0 conn(0x7feee80d0bd0 0x7feee80d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80ceb70 con 0x7feee815d350 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7feee00019f0 con 0x7feee815d350 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 >> v1:172.21.15.73:6789/0 conn(0x7feee815d350 legacy=0x7feee80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feecc003970 0x7feecc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feecc004360 0x7feecc004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc005000 0x7feecc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ce5b0 con 0x7feecc004360 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.012+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ce890 con 0x7feecc005000 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc00bfa0 con 0x7feecc003970 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeed800640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815cf70 0x7feee80d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feecc004360 0x7feecc004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeee200640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc005000 0x7feecc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeed800640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feecc003970 0x7feecc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 -- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feecc003970 msgr2=0x7feecc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feecc003970 0x7feecc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 -- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc005000 msgr2=0x7feecc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc005000 0x7feecc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.013+0000 7feeece00640 1 -- 172.21.15.73:0/3056664686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc003520 con 0x7feecc004360 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.015+0000 7feeece00640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feecc004360 0x7feecc004840 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7feed4009a90 tx=0x7feecc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b133ac66ac8c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.015+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed402c6b0 con 0x7feecc004360 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.015+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed402c810 con 0x7feecc004360 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.015+0000 7feee7e00640 1 -- 172.21.15.73:0/3056664686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed402cba0 con 0x7feecc004360 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 -- 172.21.15.73:0/3056664686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feecc004360 msgr2=0x7feecc004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 --2- 172.21.15.73:0/3056664686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feecc004360 0x7feecc004840 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7feed4009a90 tx=0x7feecc0057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feeee200640 1 -- 172.21.15.73:0/3056664686 reap_dead start 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 -- 172.21.15.73:0/3056664686 shutdown_connections 2024-04-05T22:32:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 -- 172.21.15.73:0/3056664686 >> 172.21.15.73:0/3056664686 conn(0x7feee80d2df0 msgr2=0x7feee80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 -- 172.21.15.73:0/3056664686 shutdown_connections 2024-04-05T22:32:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.016+0000 7feef0200640 1 -- 172.21.15.73:0/3056664686 wait complete. 2024-04-05T22:32:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.018+0000 7feef0200640 1 Processor -- start 2024-04-05T22:32:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.018+0000 7feef0200640 1 -- start start 2024-04-05T22:32:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 0x7feee814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 0x7feee80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81e8490 con 0x7feee80b0f50 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b0ac0 con 0x7feecc004360 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feef0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee814f660 con 0x7feee81e7b80 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41624/0 (socket says 172.21.15.73:41624) 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 -- 172.21.15.73:0/2997885941 learned_addr learned my addr 172.21.15.73:0/2997885941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeee200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 0x7feee80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeed800640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 0x7feee814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 -- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 msgr2=0x7feee80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 0x7feee80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 -- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 msgr2=0x7feee814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 0x7feee814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeece00640 1 -- 172.21.15.73:0/2997885941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee815c3a0 con 0x7feee80b0f50 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.019+0000 7feeee200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 0x7feee80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feeed800640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 0x7feee814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:19.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feeece00640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7feed4037040 tx=0x7feed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4003810 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4003970 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4003d00 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee81e81b0 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.020+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee80b0750 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.021+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee8068160 con 0x7feee80b0f50 2024-04-05T22:32:19.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.022+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7feed4040450 con 0x7feee80b0f50 2024-04-05T22:32:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.023+0000 7feee5600640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 0x7feec806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.023+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feed40a9d00 con 0x7feee80b0f50 2024-04-05T22:32:19.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.024+0000 7feeed800640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 0x7feec806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.025+0000 7feeed800640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 0x7feec806dc00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feed8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.028+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7feed4076930 con 0x7feee80b0f50 2024-04-05T22:32:19.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.222+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7feee80b02f0 con 0x7feec806b740 2024-04-05T22:32:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.223+0000 7feee5600640 1 -- 172.21.15.73:0/2997885941 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7feed8015470 con 0x7feec806b740 2024-04-05T22:32:19.223 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-04-05T22:32:19.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 msgr2=0x7feec806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 0x7feec806dc00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feed8008040 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 msgr2=0x7feee81e7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7feed4037040 tx=0x7feed4004730 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 shutdown_connections 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7feec806b740 0x7feec806dc00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee81e7b80 0x7feee80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feee80b0f50 0x7feee81e7640 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 --2- 172.21.15.73:0/2997885941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feecc004360 0x7feee814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 >> 172.21.15.73:0/2997885941 conn(0x7feee80d2df0 msgr2=0x7feee80cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 shutdown_connections 2024-04-05T22:32:19.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.226+0000 7feef0200640 1 -- 172.21.15.73:0/2997885941 wait complete. 2024-04-05T22:32:19.245 INFO:tasks.ceph:Scrubbing osd.3 2024-04-05T22:32:19.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.452+0000 7ff18d400640 1 Processor -- start 2024-04-05T22:32:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.452+0000 7ff18d400640 1 -- start start 2024-04-05T22:32:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff18815d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1880704a0 0x7ff1880b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18d400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18815df30 con 0x7ff18815dad0 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18d400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880caa00 con 0x7ff1880704a0 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880caf80 con 0x7ff188159990 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.453+0000 7ff18ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1880704a0 0x7ff1880b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1880704a0 0x7ff1880b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41632/0 (socket says 172.21.15.73:41632) 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 -- 172.21.15.73:0/585822592 learned_addr learned my addr 172.21.15.73:0/585822592 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff187e00640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff18815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 -- 172.21.15.73:0/585822592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 msgr2=0x7ff18815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff18815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 -- 172.21.15.73:0/585822592 >> v1:172.21.15.157:6789/0 conn(0x7ff18815dad0 legacy=0x7ff1880b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.454+0000 7ff18ca00640 1 -- 172.21.15.73:0/585822592 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18806b610 con 0x7ff1880704a0 2024-04-05T22:32:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.455+0000 7ff18ca00640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1880704a0 0x7ff1880b2930 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7ff180009e50 tx=0x7ff18002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb4406a871b0de server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.455+0000 7ff186a00640 1 -- 172.21.15.73:0/585822592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18002ca70 con 0x7ff1880704a0 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.455+0000 7ff186a00640 1 -- 172.21.15.73:0/585822592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18002cbd0 con 0x7ff1880704a0 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.455+0000 7ff186a00640 1 -- 172.21.15.73:0/585822592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800343f0 con 0x7ff1880704a0 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 -- 172.21.15.73:0/585822592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1880704a0 msgr2=0x7ff1880b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1880704a0 0x7ff1880b2930 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7ff180009e50 tx=0x7ff18002bed0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 -- 172.21.15.73:0/585822592 shutdown_connections 2024-04-05T22:32:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1880704a0 0x7ff1880b2930 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 --2- 172.21.15.73:0/585822592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff18815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 -- 172.21.15.73:0/585822592 >> 172.21.15.73:0/585822592 conn(0x7ff1880b0360 msgr2=0x7ff1880b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 -- 172.21.15.73:0/585822592 shutdown_connections 2024-04-05T22:32:19.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.456+0000 7ff18d400640 1 -- 172.21.15.73:0/585822592 wait complete. 2024-04-05T22:32:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.458+0000 7ff18d400640 1 Processor -- start 2024-04-05T22:32:19.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.458+0000 7ff18d400640 1 -- start start 2024-04-05T22:32:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1880704a0 0x7ff188151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff1880ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 0x7ff1880ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881f6570 con 0x7ff18815dad0 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188151970 con 0x7ff1880704a0 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881f6990 con 0x7ff188159990 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 0x7ff1880ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 0x7ff1880ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41642/0 (socket says 172.21.15.73:41642) 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.459+0000 7ff18ca00640 1 -- 172.21.15.73:0/1859890171 learned_addr learned my addr 172.21.15.73:0/1859890171 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff187e00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1880704a0 0x7ff188151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff187400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff1880ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 -- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 msgr2=0x7ff1880ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff1880ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 -- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1880704a0 msgr2=0x7ff188151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1880704a0 0x7ff188151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009b30 con 0x7ff18815dad0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff187400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff188159990 0x7ff1880ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff187e00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1880704a0 0x7ff188151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff18ca00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 0x7ff1880ace10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7ff180009e50 tx=0x7ff180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.460+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180009510 con 0x7ff18815dad0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.461+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18003e450 con 0x7ff18815dad0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.461+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18003e800 con 0x7ff18815dad0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.461+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1881f5bd0 con 0x7ff18815dad0 2024-04-05T22:32:19.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.461+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1881f6190 con 0x7ff18815dad0 2024-04-05T22:32:19.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.461+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1880cac10 con 0x7ff18815dad0 2024-04-05T22:32:19.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.463+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7ff18003e960 con 0x7ff18815dad0 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff17fe00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff16406b740 0x7ff16406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff1800aab50 con 0x7ff18815dad0 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff17fe00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7ff1640715a0 0x7ff1640739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff164074070 con 0x7ff1640715a0 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1800af740 con 0x7ff18815dad0 2024-04-05T22:32:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff187400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7ff1640715a0 0x7ff1640739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.464+0000 7ff187e00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff16406b740 0x7ff16406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.465+0000 7ff187400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7ff1640715a0 0x7ff1640739c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.465+0000 7ff187e00640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff16406b740 0x7ff16406dc00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.465+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== osd.3 v2:172.21.15.73:6824/37326 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff164074070 con 0x7ff1640715a0 2024-04-05T22:32:19.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.479+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 --> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff1881f66b0 con 0x7ff1640715a0 2024-04-05T22:32:19.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.479+0000 7ff17fe00640 1 -- 172.21.15.73:0/1859890171 <== osd.3 v2:172.21.15.73:6824/37326 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff1881f66b0 con 0x7ff1640715a0 2024-04-05T22:32:19.479 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:19.479 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:19.479 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:19.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.479+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7ff1640715a0 msgr2=0x7ff1640739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.479+0000 7ff18d400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:6824/37326,v1:172.21.15.73:6825/37326] conn(0x7ff1640715a0 0x7ff1640739c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff16406b740 msgr2=0x7ff16406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff16406b740 0x7ff16406dc00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178008040 comp rx=0 tx=0).stop 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 msgr2=0x7ff1880ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 --2- 172.21.15.73:0/1859890171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18815dad0 0x7ff1880ace10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7ff180009e50 tx=0x7ff180004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18ca00640 1 -- 172.21.15.73:0/1859890171 reap_dead start 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 shutdown_connections 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 >> 172.21.15.73:0/1859890171 conn(0x7ff1880b0360 msgr2=0x7ff1880ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 shutdown_connections 2024-04-05T22:32:19.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.480+0000 7ff18d400640 1 -- 172.21.15.73:0/1859890171 wait complete. 2024-04-05T22:32:19.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-05T22:32:19.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.697+0000 7f4a89600640 1 Processor -- start 2024-04-05T22:32:19.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.698+0000 7f4a89600640 1 -- start start 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.699+0000 7f4a89600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a8415ccd0 0x7f4a8415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.699+0000 7f4a89600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a840d1600 0x7f4a840d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.699+0000 7f4a89600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a840b2420 con 0x7f4a840d35e0 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.699+0000 7f4a89600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840b2770 con 0x7f4a8415ccd0 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.699+0000 7f4a89600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840ce380 con 0x7f4a840d1600 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a8415ccd0 0x7f4a8415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a8415ccd0 0x7f4a8415d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41646/0 (socket says 172.21.15.73:41646) 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 -- 172.21.15.73:0/2334179467 learned_addr learned my addr 172.21.15.73:0/2334179467 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 -- 172.21.15.73:0/2334179467 >> v2:172.21.15.157:3300/0 conn(0x7f4a840d1600 msgr2=0x7f4a840d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 --2- 172.21.15.73:0/2334179467 >> v2:172.21.15.157:3300/0 conn(0x7f4a840d1600 0x7f4a840d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 -- 172.21.15.73:0/2334179467 >> v1:172.21.15.157:6789/0 conn(0x7f4a840d35e0 legacy=0x7f4a840d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.700+0000 7f4a83e00640 1 -- 172.21.15.73:0/2334179467 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840ce660 con 0x7f4a8415ccd0 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.701+0000 7f4a83e00640 1 --2- 172.21.15.73:0/2334179467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a8415d0b0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f4a8408cff0 tx=0x7f4a7402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842b06083888296d server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.701+0000 7f4a83400640 1 -- 172.21.15.73:0/2334179467 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74031750 con 0x7f4a8415ccd0 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.701+0000 7f4a83400640 1 -- 172.21.15.73:0/2334179467 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a740318b0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.701+0000 7f4a83400640 1 -- 172.21.15.73:0/2334179467 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74031c40 con 0x7f4a8415ccd0 2024-04-05T22:32:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 -- 172.21.15.73:0/2334179467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 msgr2=0x7f4a8415d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 --2- 172.21.15.73:0/2334179467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a8415d0b0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f4a8408cff0 tx=0x7f4a7402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 -- 172.21.15.73:0/2334179467 shutdown_connections 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 --2- 172.21.15.73:0/2334179467 >> v2:172.21.15.157:3300/0 conn(0x7f4a840d1600 0x7f4a840d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 --2- 172.21.15.73:0/2334179467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a8415d0b0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 -- 172.21.15.73:0/2334179467 >> 172.21.15.73:0/2334179467 conn(0x7f4a840d2ac0 msgr2=0x7f4a840d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 -- 172.21.15.73:0/2334179467 shutdown_connections 2024-04-05T22:32:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.702+0000 7f4a89600640 1 -- 172.21.15.73:0/2334179467 wait complete. 2024-04-05T22:32:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.704+0000 7f4a89600640 1 Processor -- start 2024-04-05T22:32:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.704+0000 7f4a89600640 1 -- start start 2024-04-05T22:32:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a840d1600 0x7f4a840c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a840d35e0 0x7f4a84150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c7520 con 0x7f4a8415ccd0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c7f50 con 0x7f4a840d1600 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a89600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c8230 con 0x7f4a840d35e0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41658/0 (socket says 172.21.15.73:41658) 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.705+0000 7f4a88c00640 1 -- 172.21.15.73:0/1568312951 learned_addr learned my addr 172.21.15.73:0/1568312951 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a7b400640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a840d35e0 0x7f4a84150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a83e00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a840d1600 0x7f4a840c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 -- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a840d35e0 msgr2=0x7f4a84150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a840d35e0 0x7f4a84150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 -- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a840d1600 msgr2=0x7f4a840c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a840d1600 0x7f4a840c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 -- 172.21.15.73:0/1568312951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a740094d0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a88c00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b9e0 tx=0x7f4a7c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00caa0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c00cc20 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840c8510 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.706+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c014440 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.707+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a840b59c0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.708+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a840b5be0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.708+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f4a7c01d3b0 con 0x7f4a8415ccd0 2024-04-05T22:32:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.709+0000 7f4a80c00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 0x7f4a6006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.709+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a7c087c10 con 0x7f4a8415ccd0 2024-04-05T22:32:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.710+0000 7f4a83e00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 0x7f4a6006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:19.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.711+0000 7f4a83e00640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 0x7f4a6006dc00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035000 tx=0x7f4a74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:19.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.714+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f4a7c054840 con 0x7f4a8415ccd0 2024-04-05T22:32:19.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.909+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a84067250 con 0x7f4a6006b740 2024-04-05T22:32:19.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.910+0000 7f4a80c00640 1 -- 172.21.15.73:0/1568312951 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4a84067250 con 0x7f4a6006b740 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 msgr2=0x7f4a6006dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 0x7f4a6006dc00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035000 tx=0x7f4a74008030 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 msgr2=0x7f4a84159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b9e0 tx=0x7f4a7c00bea0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 shutdown_connections 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f4a6006b740 0x7f4a6006dc00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8415ccd0 0x7f4a84159340 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a840d35e0 0x7f4a84150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 --2- 172.21.15.73:0/1568312951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a840d1600 0x7f4a840c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 >> 172.21.15.73:0/1568312951 conn(0x7f4a840d2ac0 msgr2=0x7f4a8406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 shutdown_connections 2024-04-05T22:32:19.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:19.912+0000 7f4a89600640 1 -- 172.21.15.73:0/1568312951 wait complete. 2024-04-05T22:32:19.949 INFO:tasks.ceph:Scrubbing osd.4 2024-04-05T22:32:19.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:20.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.154+0000 7fad07200640 1 Processor -- start 2024-04-05T22:32:20.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.154+0000 7fad07200640 1 -- start start 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad07200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad0815aa80 0x7fad0806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad07200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad080714b0 0x7fad0806b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad07200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad080ccc50 con 0x7fad0806b9d0 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad07200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad080cd5f0 con 0x7fad080714b0 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad07200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad080ccf30 con 0x7fad0815aa80 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad080714b0 0x7fad0806b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad080714b0 0x7fad0806b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41660/0 (socket says 172.21.15.73:41660) 2024-04-05T22:32:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 -- 172.21.15.73:0/1676945426 learned_addr learned my addr 172.21.15.73:0/1676945426 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 -- 172.21.15.73:0/1676945426 >> v2:172.21.15.157:3300/0 conn(0x7fad0815aa80 msgr2=0x7fad0806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 --2- 172.21.15.73:0/1676945426 >> v2:172.21.15.157:3300/0 conn(0x7fad0815aa80 0x7fad0806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 -- 172.21.15.73:0/1676945426 >> v1:172.21.15.157:6789/0 conn(0x7fad0806b9d0 legacy=0x7fad080cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.156+0000 7fad05400640 1 -- 172.21.15.73:0/1676945426 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080cd210 con 0x7fad080714b0 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad05400640 1 --2- 172.21.15.73:0/1676945426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad0806b490 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fad0806b2b0 tx=0x7facf402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ba14b194643d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad04a00640 1 -- 172.21.15.73:0/1676945426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4031750 con 0x7fad080714b0 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad04a00640 1 -- 172.21.15.73:0/1676945426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf40318b0 con 0x7fad080714b0 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad04a00640 1 -- 172.21.15.73:0/1676945426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4031c40 con 0x7fad080714b0 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad07200640 1 -- 172.21.15.73:0/1676945426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 msgr2=0x7fad0806b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.158+0000 7fad07200640 1 --2- 172.21.15.73:0/1676945426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad0806b490 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fad0806b2b0 tx=0x7facf402b7f0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 -- 172.21.15.73:0/1676945426 shutdown_connections 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 --2- 172.21.15.73:0/1676945426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad0806b490 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 --2- 172.21.15.73:0/1676945426 >> v2:172.21.15.157:3300/0 conn(0x7fad0815aa80 0x7fad0806fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 -- 172.21.15.73:0/1676945426 >> 172.21.15.73:0/1676945426 conn(0x7fad080b0360 msgr2=0x7fad080b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 -- 172.21.15.73:0/1676945426 shutdown_connections 2024-04-05T22:32:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.159+0000 7fad07200640 1 -- 172.21.15.73:0/1676945426 wait complete. 2024-04-05T22:32:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.161+0000 7fad07200640 1 Processor -- start 2024-04-05T22:32:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.161+0000 7fad07200640 1 -- start start 2024-04-05T22:32:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806b9d0 0x7fad08154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad081484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0815aa80 0x7fad080b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080b1760 con 0x7fad080714b0 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08158eb0 con 0x7fad0806b9d0 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad07200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad08159190 con 0x7fad0815aa80 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad081484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad081484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41668/0 (socket says 172.21.15.73:41668) 2024-04-05T22:32:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 -- 172.21.15.73:0/1298279938 learned_addr learned my addr 172.21.15.73:0/1298279938 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05e00640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806b9d0 0x7fad08154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad06800640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0815aa80 0x7fad080b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 -- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0815aa80 msgr2=0x7fad080b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0815aa80 0x7fad080b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 -- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806b9d0 msgr2=0x7fad08154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806b9d0 0x7fad08154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05400640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf40094d0 con 0x7fad080714b0 2024-04-05T22:32:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.162+0000 7fad05e00640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806b9d0 0x7fad08154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7fad05400640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad081484a0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7facf4000c00 tx=0x7facf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4033c60 con 0x7fad080714b0 2024-04-05T22:32:20.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4033de0 con 0x7fad080714b0 2024-04-05T22:32:20.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf403d6b0 con 0x7fad080714b0 2024-04-05T22:32:20.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08159470 con 0x7fad080714b0 2024-04-05T22:32:20.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.163+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad08155030 con 0x7fad080714b0 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.164+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad080cc870 con 0x7fad080714b0 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.165+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7facf403a040 con 0x7fad080714b0 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.166+0000 7facfe000640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7face806b740 0x7face806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.166+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7facf40aa5e0 con 0x7fad080714b0 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.166+0000 7facfe000640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7face80715a0 0x7face80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.166+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7face8074070 con 0x7face80715a0 2024-04-05T22:32:20.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.166+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf40acc10 con 0x7fad080714b0 2024-04-05T22:32:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.167+0000 7fad05e00640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7face806b740 0x7face806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.167+0000 7fad06800640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7face80715a0 0x7face80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.168+0000 7fad06800640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7face80715a0 0x7face80739c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.168+0000 7fad05e00640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7face806b740 0x7face806dc00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7facf8006fd0 tx=0x7facf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.170+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== osd.4 v2:172.21.15.157:6800/37592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7face8074070 con 0x7face80715a0 2024-04-05T22:32:20.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.184+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 --> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fad080b3970 con 0x7face80715a0 2024-04-05T22:32:20.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7facfe000640 1 -- 172.21.15.73:0/1298279938 <== osd.4 v2:172.21.15.157:6800/37592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fad080b3970 con 0x7face80715a0 2024-04-05T22:32:20.185 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:20.186 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:20.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7face80715a0 msgr2=0x7face80739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6800/37592,v1:172.21.15.157:6801/37592] conn(0x7face80715a0 0x7face80739c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7face806b740 msgr2=0x7face806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7face806b740 0x7face806dc00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7facf8006fd0 tx=0x7facf8008040 comp rx=0 tx=0).stop 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 msgr2=0x7fad081484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.185+0000 7fad07200640 1 --2- 172.21.15.73:0/1298279938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad080714b0 0x7fad081484a0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7facf4000c00 tx=0x7facf40046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.186+0000 7fad06800640 1 -- 172.21.15.73:0/1298279938 reap_dead start 2024-04-05T22:32:20.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.186+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 shutdown_connections 2024-04-05T22:32:20.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.186+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 >> 172.21.15.73:0/1298279938 conn(0x7fad080b0360 msgr2=0x7fad080cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.186+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 shutdown_connections 2024-04-05T22:32:20.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.186+0000 7fad07200640 1 -- 172.21.15.73:0/1298279938 wait complete. 2024-04-05T22:32:20.203 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-05T22:32:20.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.417+0000 7f625bc00640 1 Processor -- start 2024-04-05T22:32:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.417+0000 7f625bc00640 1 -- start start 2024-04-05T22:32:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f625bc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f625406cb80 0x7f625406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f625bc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 0x7f625415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f625bc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625406f5b0 con 0x7f62541587d0 2024-04-05T22:32:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f625bc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625406d500 con 0x7f625406cb80 2024-04-05T22:32:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f625bc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625415f220 con 0x7f625415e850 2024-04-05T22:32:20.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f6253e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f625406cb80 0x7f625406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f6253e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f625406cb80 0x7f625406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41678/0 (socket says 172.21.15.73:41678) 2024-04-05T22:32:20.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.419+0000 7f6253e00640 1 -- 172.21.15.73:0/1013854099 learned_addr learned my addr 172.21.15.73:0/1013854099 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6259c00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 0x7f625415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6253e00640 1 -- 172.21.15.73:0/1013854099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 msgr2=0x7f625415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6253e00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 0x7f625415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6253e00640 1 -- 172.21.15.73:0/1013854099 >> v1:172.21.15.157:6789/0 conn(0x7f62541587d0 legacy=0x7f62540cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6253e00640 1 -- 172.21.15.73:0/1013854099 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254070410 con 0x7f625406cb80 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.420+0000 7f6259c00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 0x7f625415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:20.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.421+0000 7f6253e00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f625406cb80 0x7f625406cfc0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f624002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926efc199f184d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.421+0000 7f6253400640 1 -- 172.21.15.73:0/1013854099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240031640 con 0x7f625406cb80 2024-04-05T22:32:20.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.421+0000 7f6253400640 1 -- 172.21.15.73:0/1013854099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62400317a0 con 0x7f625406cb80 2024-04-05T22:32:20.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.421+0000 7f6253400640 1 -- 172.21.15.73:0/1013854099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240031b30 con 0x7f625406cb80 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 -- 172.21.15.73:0/1013854099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f625406cb80 msgr2=0x7f625406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f625406cb80 0x7f625406cfc0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f624002b8f0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 -- 172.21.15.73:0/1013854099 shutdown_connections 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625415e850 0x7f625415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 --2- 172.21.15.73:0/1013854099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f625406cb80 0x7f625406cfc0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 -- 172.21.15.73:0/1013854099 >> 172.21.15.73:0/1013854099 conn(0x7f62540ce8e0 msgr2=0x7f62540cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 -- 172.21.15.73:0/1013854099 shutdown_connections 2024-04-05T22:32:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.422+0000 7f625bc00640 1 -- 172.21.15.73:0/1013854099 wait complete. 2024-04-05T22:32:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.424+0000 7f625bc00640 1 Processor -- start 2024-04-05T22:32:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.424+0000 7f625bc00640 1 -- start start 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625406cb80 0x7f6254159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 0x7f62540ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254156a60 con 0x7f62541587d0 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541f0250 con 0x7f625415e850 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f625bc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6254155de0 con 0x7f625406cb80 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f6253e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f6253e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41694/0 (socket says 172.21.15.73:41694) 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.425+0000 7f6253e00640 1 -- 172.21.15.73:0/3982739427 learned_addr learned my addr 172.21.15.73:0/3982739427 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6259c00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 0x7f62540ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6259200640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625406cb80 0x7f6254159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 -- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625406cb80 msgr2=0x7f6254159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625406cb80 0x7f6254159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 -- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 msgr2=0x7f62540ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 0x7f62540ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 -- 172.21.15.73:0/3982739427 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400094d0 con 0x7f62541587d0 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6259c00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 0x7f62540ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6253e00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f62400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624003ac60 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624003ade0 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62541560c0 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.426+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240044630 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.427+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6254156680 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.428+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254066af0 con 0x7f62541587d0 2024-04-05T22:32:20.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.429+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f624002f050 con 0x7f62541587d0 2024-04-05T22:32:20.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.430+0000 7f6250c00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 0x7f623406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.430+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62400a9e50 con 0x7f62541587d0 2024-04-05T22:32:20.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.431+0000 7f6259200640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 0x7f623406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.432+0000 7f6259200640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 0x7f623406dc00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6244003d90 tx=0x7f6244009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.435+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6240076a80 con 0x7f62541587d0 2024-04-05T22:32:20.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.629+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f625406f800 con 0x7f623406b740 2024-04-05T22:32:20.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.631+0000 7f6250c00640 1 -- 172.21.15.73:0/3982739427 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6244003a30 con 0x7f623406b740 2024-04-05T22:32:20.631 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2024-04-05T22:32:20.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 msgr2=0x7f623406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 0x7f623406dc00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6244003d90 tx=0x7f6244009040 comp rx=0 tx=0).stop 2024-04-05T22:32:20.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 msgr2=0x7f62540aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f62400046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 shutdown_connections 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f623406b740 0x7f623406dc00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415e850 0x7f62540ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62541587d0 0x7f62540aae20 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 --2- 172.21.15.73:0/3982739427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625406cb80 0x7f6254159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 >> 172.21.15.73:0/3982739427 conn(0x7f62540ce8e0 msgr2=0x7f625415a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 shutdown_connections 2024-04-05T22:32:20.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.633+0000 7f625bc00640 1 -- 172.21.15.73:0/3982739427 wait complete. 2024-04-05T22:32:20.651 INFO:tasks.ceph:Scrubbing osd.5 2024-04-05T22:32:20.651 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.857+0000 7f9306e00640 1 Processor -- start 2024-04-05T22:32:20.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.857+0000 7f9306e00640 1 -- start start 2024-04-05T22:32:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9306e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 0x7f930006b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9306e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f930006b9d0 0x7f93000cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9306e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93000ccc50 con 0x7f930015aa80 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9306e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93000cd5f0 con 0x7f930006b9d0 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9306e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93000705b0 con 0x7f93000714b0 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9305a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f930015aa80 0x7f930006fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44158/0 (socket says 172.21.15.73:44158) 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.859+0000 7f9305a00640 1 -- 172.21.15.73:0/1853042947 learned_addr learned my addr 172.21.15.73:0/1853042947 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f9305000640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 0x7f930006b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f9306400640 1 --2- 172.21.15.73:0/1853042947 >> v2:172.21.15.157:3300/0 conn(0x7f930006b9d0 0x7f93000cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189605905 0 0) 0x7f93000ccc50 con 0x7f930015aa80 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e8003520 con 0x7f930015aa80 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239071014 0 0) 0x7f92e8003520 con 0x7f930015aa80 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 msgr2=0x7f930006b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 0x7f930006b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 >> v2:172.21.15.157:3300/0 conn(0x7f930006b9d0 msgr2=0x7f93000cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 --2- 172.21.15.73:0/1853042947 >> v2:172.21.15.157:3300/0 conn(0x7f930006b9d0 0x7f93000cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000ccf30 con 0x7f930015aa80 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.860+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f92f4001670 con 0x7f930015aa80 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 >> v1:172.21.15.73:6789/0 conn(0x7f930015aa80 legacy=0x7f930006fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 0x7f92e8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e8003ef0 0x7f92e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92e8004d00 0x7f92e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93000cd5f0 con 0x7f92e8004d00 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e800b9e0 con 0x7f92e8003ef0 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e800bba0 con 0x7f93000714b0 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92e8004d00 0x7f92e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9305000640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e8003ef0 0x7f92e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 -- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 msgr2=0x7f92e8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93000714b0 0x7f92e8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 -- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e8003ef0 msgr2=0x7f92e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e8003ef0 0x7f92e80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.861+0000 7f9306400640 1 -- 172.21.15.73:0/1853042947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8003520 con 0x7f92e8004d00 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.862+0000 7f9306400640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92e8004d00 0x7f92e8005180 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009fc0 tx=0x7f92e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ab0706d60d8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f802cc60 con 0x7f92e8004d00 2024-04-05T22:32:20.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800f040 con 0x7f92e8004d00 2024-04-05T22:32:20.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f92ffe00640 1 -- 172.21.15.73:0/1853042947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8034440 con 0x7f92e8004d00 2024-04-05T22:32:20.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 -- 172.21.15.73:0/1853042947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92e8004d00 msgr2=0x7f92e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 --2- 172.21.15.73:0/1853042947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92e8004d00 0x7f92e8005180 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009fc0 tx=0x7f92e80057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306400640 1 -- 172.21.15.73:0/1853042947 reap_dead start 2024-04-05T22:32:20.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 -- 172.21.15.73:0/1853042947 shutdown_connections 2024-04-05T22:32:20.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 -- 172.21.15.73:0/1853042947 >> 172.21.15.73:0/1853042947 conn(0x7f93000b0360 msgr2=0x7f93000b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 -- 172.21.15.73:0/1853042947 shutdown_connections 2024-04-05T22:32:20.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.863+0000 7f9306e00640 1 -- 172.21.15.73:0/1853042947 wait complete. 2024-04-05T22:32:20.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.865+0000 7f9306e00640 1 Processor -- start 2024-04-05T22:32:20.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.865+0000 7f9306e00640 1 -- start start 2024-04-05T22:32:20.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e8004d00 0x7f930014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 0x7f9300150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93001509d0 0x7f9300152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300150e80 con 0x7f930014c220 2024-04-05T22:32:20.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300153bd0 con 0x7f93001509d0 2024-04-05T22:32:20.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9306e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930014d0f0 con 0x7f92e8004d00 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.866+0000 7f9305000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 0x7f9300150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 0x7f9300150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41710/0 (socket says 172.21.15.73:41710) 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 -- 172.21.15.73:0/1318040074 learned_addr learned my addr 172.21.15.73:0/1318040074 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305a00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e8004d00 0x7f930014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 -- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e8004d00 msgr2=0x7f930014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e8004d00 0x7f930014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 -- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93001509d0 msgr2=0x7f9300152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93001509d0 0x7f9300152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930014c7c0 con 0x7f930014c220 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305a00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e8004d00 0x7f930014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f9305000640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 0x7f9300150490 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f92f0010890 tx=0x7f92f0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.867+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0009570 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.868+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f000eae0 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.868+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930014caa0 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.868+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f000ee90 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.868+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930014d580 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.869+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f930015ae60 con 0x7f930014c220 2024-04-05T22:32:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.870+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f92f001c400 con 0x7f930014c220 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.870+0000 7f92fd600640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f92e406b740 0x7f92e406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92f0088370 con 0x7f930014c220 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f92fd600640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f92e40715a0 0x7f92e40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92e4074070 con 0x7f92e40715a0 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f008f2a0 con 0x7f930014c220 2024-04-05T22:32:20.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f9305a00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f92e406b740 0x7f92e406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.871+0000 7f9306400640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f92e40715a0 0x7f92e40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:20.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.872+0000 7f9306400640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f92e40715a0 0x7f92e40739c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.873+0000 7f9305a00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f92e406b740 0x7f92e406dc00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f92f40011e0 tx=0x7f92f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:20.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.874+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== osd.5 v2:172.21.15.157:6821/37594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f92e4074070 con 0x7f92e40715a0 2024-04-05T22:32:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.888+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 --> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f930014d260 con 0x7f92e40715a0 2024-04-05T22:32:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f92fd600640 1 -- 172.21.15.73:0/1318040074 <== osd.5 v2:172.21.15.157:6821/37594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f930014d260 con 0x7f92e40715a0 2024-04-05T22:32:20.889 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:20.889 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:20.889 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f92e40715a0 msgr2=0x7f92e40739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6821/37594,v1:172.21.15.157:6822/37594] conn(0x7f92e40715a0 0x7f92e40739c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f92e406b740 msgr2=0x7f92e406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f92e406b740 0x7f92e406dc00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f92f40011e0 tx=0x7f92f4009040 comp rx=0 tx=0).stop 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 msgr2=0x7f9300150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 --2- 172.21.15.73:0/1318040074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f930014c220 0x7f9300150490 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f92f0010890 tx=0x7f92f0010d50 comp rx=0 tx=0).stop 2024-04-05T22:32:20.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306400640 1 -- 172.21.15.73:0/1318040074 reap_dead start 2024-04-05T22:32:20.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 shutdown_connections 2024-04-05T22:32:20.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 >> 172.21.15.73:0/1318040074 conn(0x7f93000b0360 msgr2=0x7f93000ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:20.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 shutdown_connections 2024-04-05T22:32:20.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:20.889+0000 7f9306e00640 1 -- 172.21.15.73:0/1318040074 wait complete. 2024-04-05T22:32:20.908 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-05T22:32:21.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.116+0000 7f1f01400640 1 Processor -- start 2024-04-05T22:32:21.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.116+0000 7f1f01400640 1 -- start start 2024-04-05T22:32:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1f01400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1f01400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1efc15e850 0x7f1efc15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1f01400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc06f5b0 con 0x7f1efc0ccd20 2024-04-05T22:32:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1f01400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc15f240 con 0x7f1efc15e850 2024-04-05T22:32:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1f01400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc070410 con 0x7f1efc1587d0 2024-04-05T22:32:21.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1efa800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1efa800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.73:40180/0 (socket says 172.21.15.73:40180) 2024-04-05T22:32:21.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.118+0000 7f1efa800640 1 -- 172.21.15.73:0/3685108214 learned_addr learned my addr 172.21.15.73:0/3685108214 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:21.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.119+0000 7f1efb200640 1 --2- 172.21.15.73:0/3685108214 >> v2:172.21.15.157:3300/0 conn(0x7f1efc15e850 0x7f1efc15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.119+0000 7f1efa800640 1 -- 172.21.15.73:0/3685108214 >> v2:172.21.15.157:3300/0 conn(0x7f1efc15e850 msgr2=0x7f1efc15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.119+0000 7f1efa800640 1 --2- 172.21.15.73:0/3685108214 >> v2:172.21.15.157:3300/0 conn(0x7f1efc15e850 0x7f1efc15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.119+0000 7f1efa800640 1 -- 172.21.15.73:0/3685108214 >> v1:172.21.15.157:6789/0 conn(0x7f1efc0ccd20 legacy=0x7f1efc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.119+0000 7f1efa800640 1 -- 172.21.15.73:0/3685108214 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc158bb0 con 0x7f1efc1587d0 2024-04-05T22:32:21.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.120+0000 7f1efa800640 1 --2- 172.21.15.73:0/3685108214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009870 tx=0x7f1eec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0592ee51cc97f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.121+0000 7f1ef9400640 1 -- 172.21.15.73:0/3685108214 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec030840 con 0x7f1efc1587d0 2024-04-05T22:32:21.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.121+0000 7f1ef9400640 1 -- 172.21.15.73:0/3685108214 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0309a0 con 0x7f1efc1587d0 2024-04-05T22:32:21.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.121+0000 7f1ef9400640 1 -- 172.21.15.73:0/3685108214 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec030d30 con 0x7f1efc1587d0 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 -- 172.21.15.73:0/3685108214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 msgr2=0x7f1efc0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 --2- 172.21.15.73:0/3685108214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009870 tx=0x7f1eec02b980 comp rx=0 tx=0).stop 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 -- 172.21.15.73:0/3685108214 shutdown_connections 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 --2- 172.21.15.73:0/3685108214 >> v2:172.21.15.157:3300/0 conn(0x7f1efc15e850 0x7f1efc15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 --2- 172.21.15.73:0/3685108214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc0cc7e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 -- 172.21.15.73:0/3685108214 >> 172.21.15.73:0/3685108214 conn(0x7f1efc0ce8e0 msgr2=0x7f1efc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 -- 172.21.15.73:0/3685108214 shutdown_connections 2024-04-05T22:32:21.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.122+0000 7f1f01400640 1 -- 172.21.15.73:0/3685108214 wait complete. 2024-04-05T22:32:21.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.123+0000 7f1f01400640 1 Processor -- start 2024-04-05T22:32:21.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.123+0000 7f1f01400640 1 -- start start 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1efc15e850 0x7f1efc1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1f46d0 con 0x7f1efc1f3730 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc142ec0 con 0x7f1efc15e850 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.124+0000 7f1f01400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1428f0 con 0x7f1efc1587d0 2024-04-05T22:32:21.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41726/0 (socket says 172.21.15.73:41726) 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 -- 172.21.15.73:0/587011019 learned_addr learned my addr 172.21.15.73:0/587011019 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 -- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 msgr2=0x7f1efc1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 -- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1efc15e850 msgr2=0x7f1efc1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1efc15e850 0x7f1efc1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 -- 172.21.15.73:0/587011019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0094d0 con 0x7f1efc1f3730 2024-04-05T22:32:21.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.125+0000 7f1efb200640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d30 tx=0x7f1ef400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.126+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4009ba0 con 0x7f1efc1f3730 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.126+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef400f040 con 0x7f1efc1f3730 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.126+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40146c0 con 0x7f1efc1f3730 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.126+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc1f4110 con 0x7f1efc1f3730 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.126+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc142ae0 con 0x7f1efc1f3730 2024-04-05T22:32:21.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.127+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc066af0 con 0x7f1efc1f3730 2024-04-05T22:32:21.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.128+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f1ef4014820 con 0x7f1efc1f3730 2024-04-05T22:32:21.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.129+0000 7f1ef2a00640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 0x7f1edc06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.129+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1ef4088ae0 con 0x7f1efc1f3730 2024-04-05T22:32:21.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.130+0000 7f1efa800640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 0x7f1edc06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.131+0000 7f1efa800640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 0x7f1edc06dc00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1eec036000 tx=0x7f1eec03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.134+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f1ef4055710 con 0x7f1efc1f3730 2024-04-05T22:32:21.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.331+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1efc06fdc0 con 0x7f1edc06b740 2024-04-05T22:32:21.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.332+0000 7f1ef2a00640 1 -- 172.21.15.73:0/587011019 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1eec030b10 con 0x7f1edc06b740 2024-04-05T22:32:21.332 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 msgr2=0x7f1edc06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 0x7f1edc06dc00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1eec036000 tx=0x7f1eec03b040 comp rx=0 tx=0).stop 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 msgr2=0x7f1efc13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d30 tx=0x7f1ef400c490 comp rx=0 tx=0).stop 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 shutdown_connections 2024-04-05T22:32:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1edc06b740 0x7f1edc06dc00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc1f3730 0x7f1efc13fa30 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1efc15e850 0x7f1efc1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 --2- 172.21.15.73:0/587011019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1efc1587d0 0x7f1efc1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 >> 172.21.15.73:0/587011019 conn(0x7f1efc0ce8e0 msgr2=0x7f1efc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 shutdown_connections 2024-04-05T22:32:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.335+0000 7f1f01400640 1 -- 172.21.15.73:0/587011019 wait complete. 2024-04-05T22:32:21.354 INFO:tasks.ceph:Scrubbing osd.6 2024-04-05T22:32:21.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.568+0000 7f7178600640 1 Processor -- start 2024-04-05T22:32:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.568+0000 7f7178600640 1 -- start start 2024-04-05T22:32:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.569+0000 7f7178600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f71700b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.569+0000 7f7178600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71700cce90 0x7f71700b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.569+0000 7f7178600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700cd2e0 con 0x7f717015ae50 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.569+0000 7f7178600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71700ce2f0 con 0x7f71700cce90 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.569+0000 7f7178600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717015b230 con 0x7f71700cf3e0 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f7175c00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f717015ae50 0x7f71700cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44160/0 (socket says 172.21.15.73:44160) 2024-04-05T22:32:21.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f7175c00640 1 -- 172.21.15.73:0/1113123055 learned_addr learned my addr 172.21.15.73:0/1113123055 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f7175200640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f71700b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891088342 0 0) 0x7f71700cd2e0 con 0x7f717015ae50 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f7176600640 1 --2- 172.21.15.73:0/1113123055 >> v2:172.21.15.157:3300/0 conn(0x7f71700cce90 0x7f71700b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7158003520 con 0x7f717015ae50 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075390271 0 0) 0x7f7158003520 con 0x7f717015ae50 2024-04-05T22:32:21.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 msgr2=0x7f71700b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f71700b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 >> v2:172.21.15.157:3300/0 conn(0x7f71700cce90 msgr2=0x7f71700b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 --2- 172.21.15.73:0/1113123055 >> v2:172.21.15.157:3300/0 conn(0x7f71700cce90 0x7f71700b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717015f4a0 con 0x7f717015ae50 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.570+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1927619306 0 0) 0x7f7164001690 con 0x7f717015ae50 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 >> v1:172.21.15.73:6789/0 conn(0x7f717015ae50 legacy=0x7f71700cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f7158003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003fa0 0x7f71580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7158005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700ce2f0 con 0x7f7158004d00 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717015b230 con 0x7f7158003fa0 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715800bb40 con 0x7f71700cf3e0 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7158005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7175200640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003fa0 0x7f71580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7175c00640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f7158003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 -- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 msgr2=0x7f7158003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700cf3e0 0x7f7158003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 -- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003fa0 msgr2=0x7f71580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003fa0 0x7f71580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.571+0000 7f7176600640 1 -- 172.21.15.73:0/1113123055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7158003520 con 0x7f7158004d00 2024-04-05T22:32:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7176600640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7158005180 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f7168007360 tx=0x7f71580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b539719047ba8c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716802ccf0 con 0x7f7158004d00 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716802ce50 con 0x7f7158004d00 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f716fe00640 1 -- 172.21.15.73:0/1113123055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168034660 con 0x7f7158004d00 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7178600640 1 -- 172.21.15.73:0/1113123055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 msgr2=0x7f7158005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7178600640 1 --2- 172.21.15.73:0/1113123055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7158005180 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f7168007360 tx=0x7f71580057f0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7176600640 1 -- 172.21.15.73:0/1113123055 reap_dead start 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7178600640 1 -- 172.21.15.73:0/1113123055 shutdown_connections 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.573+0000 7f7178600640 1 -- 172.21.15.73:0/1113123055 >> 172.21.15.73:0/1113123055 conn(0x7f717006a0d0 msgr2=0x7f717006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.574+0000 7f7178600640 1 -- 172.21.15.73:0/1113123055 shutdown_connections 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.574+0000 7f7178600640 1 -- 172.21.15.73:0/1113123055 wait complete. 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.575+0000 7f7178600640 1 Processor -- start 2024-04-05T22:32:21.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.575+0000 7f7178600640 1 -- start start 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7170071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700b0cc0 0x7f71701587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717006f6a0 0x7f717015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7170070c00 con 0x7f7158004d00 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700715d0 con 0x7f71700b0cc0 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.576+0000 7f7178600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717006fff0 con 0x7f717006f6a0 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7170071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7170071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41748/0 (socket says 172.21.15.73:41748) 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 -- 172.21.15.73:0/80573988 learned_addr learned my addr 172.21.15.73:0/80573988 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 -- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717006f6a0 msgr2=0x7f717015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717006f6a0 0x7f717015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 -- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700b0cc0 msgr2=0x7f71701587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700b0cc0 0x7f71701587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700702d0 con 0x7f7158004d00 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.577+0000 7f7175c00640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7170071090 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f716400e840 tx=0x7f716400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.578+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164017070 con 0x7f7158004d00 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.578+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400c8d0 con 0x7f7158004d00 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.578+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400ca60 con 0x7f7158004d00 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.578+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71700705b0 con 0x7f7158004d00 2024-04-05T22:32:21.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.578+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170158d00 con 0x7f7158004d00 2024-04-05T22:32:21.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.579+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71700cb610 con 0x7f7158004d00 2024-04-05T22:32:21.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.580+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f716400cbc0 con 0x7f7158004d00 2024-04-05T22:32:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.581+0000 7f716d600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f715406b770 0x7f715406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.581+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7164087450 con 0x7f7158004d00 2024-04-05T22:32:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.582+0000 7f716d600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f71540715d0 0x7f71540739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.582+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71540740a0 con 0x7f71540715d0 2024-04-05T22:32:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.582+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7f716408e2a0 con 0x7f7158004d00 2024-04-05T22:32:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.582+0000 7f7175200640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f715406b770 0x7f715406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.582+0000 7f7176600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f71540715d0 0x7f71540739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.583+0000 7f7175200640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f715406b770 0x7f715406dc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f71701f5a30 tx=0x7f716000d280 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.584+0000 7f7176600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f71540715d0 0x7f71540739f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.586+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== osd.6 v2:172.21.15.157:6808/37593 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f71540740a0 con 0x7f71540715d0 2024-04-05T22:32:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.599+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 --> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f71700cb330 con 0x7f71540715d0 2024-04-05T22:32:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.600+0000 7f716d600640 1 -- 172.21.15.73:0/80573988 <== osd.6 v2:172.21.15.157:6808/37593 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f716802cc10 con 0x7f71540715d0 2024-04-05T22:32:21.600 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:21.601 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:21.601 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f71540715d0 msgr2=0x7f71540739f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6808/37593,v1:172.21.15.157:6809/37593] conn(0x7f71540715d0 0x7f71540739f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f715406b770 msgr2=0x7f715406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f715406b770 0x7f715406dc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f71701f5a30 tx=0x7f716000d280 comp rx=0 tx=0).stop 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 msgr2=0x7f7170071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 --2- 172.21.15.73:0/80573988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7158004d00 0x7f7170071090 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f716400e840 tx=0x7f716400ed00 comp rx=0 tx=0).stop 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7176600640 1 -- 172.21.15.73:0/80573988 reap_dead start 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 shutdown_connections 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 >> 172.21.15.73:0/80573988 conn(0x7f717006a0d0 msgr2=0x7f71700afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 shutdown_connections 2024-04-05T22:32:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.601+0000 7f7178600640 1 -- 172.21.15.73:0/80573988 wait complete. 2024-04-05T22:32:21.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-05T22:32:21.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.829+0000 7ff4a8c00640 1 Processor -- start 2024-04-05T22:32:21.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.830+0000 7ff4a8c00640 1 -- start start 2024-04-05T22:32:21.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4a406cb80 0x7ff4a406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a8c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4a415e850 0x7ff4a415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a8c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a406f5b0 con 0x7ff4a41587d0 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a8c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a406d500 con 0x7ff4a406cb80 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a8c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a415f220 con 0x7ff4a415e850 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.831+0000 7ff4a1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4a406cb80 0x7ff4a406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4a406cb80 0x7ff4a406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41758/0 (socket says 172.21.15.73:41758) 2024-04-05T22:32:21.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 -- 172.21.15.73:0/4120397944 learned_addr learned my addr 172.21.15.73:0/4120397944 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:21.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a2800640 1 --2- 172.21.15.73:0/4120397944 >> v2:172.21.15.157:3300/0 conn(0x7ff4a415e850 0x7ff4a415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 -- 172.21.15.73:0/4120397944 >> v2:172.21.15.157:3300/0 conn(0x7ff4a415e850 msgr2=0x7ff4a415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 --2- 172.21.15.73:0/4120397944 >> v2:172.21.15.157:3300/0 conn(0x7ff4a415e850 0x7ff4a415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 -- 172.21.15.73:0/4120397944 >> v1:172.21.15.157:6789/0 conn(0x7ff4a41587d0 legacy=0x7ff4a40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:21.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.832+0000 7ff4a1400640 1 -- 172.21.15.73:0/4120397944 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4070410 con 0x7ff4a406cb80 2024-04-05T22:32:21.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.833+0000 7ff4a1400640 1 --2- 172.21.15.73:0/4120397944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a406cfc0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7ff4a415c270 tx=0x7ff49002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a86f0d3b867bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.833+0000 7ff4a0a00640 1 -- 172.21.15.73:0/4120397944 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490031870 con 0x7ff4a406cb80 2024-04-05T22:32:21.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a0a00640 1 -- 172.21.15.73:0/4120397944 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4900319d0 con 0x7ff4a406cb80 2024-04-05T22:32:21.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a0a00640 1 -- 172.21.15.73:0/4120397944 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490031d60 con 0x7ff4a406cb80 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 -- 172.21.15.73:0/4120397944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 msgr2=0x7ff4a406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/4120397944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a406cfc0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7ff4a415c270 tx=0x7ff49002b910 comp rx=0 tx=0).stop 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 -- 172.21.15.73:0/4120397944 shutdown_connections 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/4120397944 >> v2:172.21.15.157:3300/0 conn(0x7ff4a415e850 0x7ff4a415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/4120397944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a406cfc0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 -- 172.21.15.73:0/4120397944 >> 172.21.15.73:0/4120397944 conn(0x7ff4a40ce8e0 msgr2=0x7ff4a40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 -- 172.21.15.73:0/4120397944 shutdown_connections 2024-04-05T22:32:21.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.834+0000 7ff4a8c00640 1 -- 172.21.15.73:0/4120397944 wait complete. 2024-04-05T22:32:21.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.836+0000 7ff4a8c00640 1 Processor -- start 2024-04-05T22:32:21.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.836+0000 7ff4a8c00640 1 -- start start 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a41587d0 0x7ff4a4148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4a415e850 0x7ff4a4148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40a9510 con 0x7ff4a406cb80 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c3d20 con 0x7ff4a41587d0 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.837+0000 7ff4a8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41f0e80 con 0x7ff4a415e850 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41770/0 (socket says 172.21.15.73:41770) 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 -- 172.21.15.73:0/2105840956 learned_addr learned my addr 172.21.15.73:0/2105840956 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1400640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a41587d0 0x7ff4a4148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 -- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4a415e850 msgr2=0x7ff4a4148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4a415e850 0x7ff4a4148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 -- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a41587d0 msgr2=0x7ff4a4148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a41587d0 0x7ff4a4148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 -- 172.21.15.73:0/2105840956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900094d0 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff4a1e00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c430 tx=0x7ff49400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.838+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494011790 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.839+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4940118f0 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.839+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494011ce0 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.839+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a41f1160 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.839+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40a9a40 con 0x7ff4a406cb80 2024-04-05T22:32:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.840+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4066af0 con 0x7ff4a406cb80 2024-04-05T22:32:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.841+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7ff49401c460 con 0x7ff4a406cb80 2024-04-05T22:32:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.842+0000 7ff49a000640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 0x7ff48406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.842+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff494087c70 con 0x7ff4a406cb80 2024-04-05T22:32:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.842+0000 7ff4a1400640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 0x7ff48406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.843+0000 7ff4a1400640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 0x7ff48406dc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff490036000 tx=0x7ff49003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:21.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:21.846+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7ff4940548a0 con 0x7ff4a406cb80 2024-04-05T22:32:22.042 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.042+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4a406f800 con 0x7ff48406b770 2024-04-05T22:32:22.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.043+0000 7ff49a000640 1 -- 172.21.15.73:0/2105840956 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff490031cc0 con 0x7ff48406b770 2024-04-05T22:32:22.043 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2024-04-05T22:32:22.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 msgr2=0x7ff48406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 0x7ff48406dc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff490036000 tx=0x7ff49003c040 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 msgr2=0x7ff4a40c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c430 tx=0x7ff49400c8f0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 shutdown_connections 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7ff48406b770 0x7ff48406dc30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4a415e850 0x7ff4a4148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a41587d0 0x7ff4a4148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 --2- 172.21.15.73:0/2105840956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a406cb80 0x7ff4a40c37e0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 >> 172.21.15.73:0/2105840956 conn(0x7ff4a40ce8e0 msgr2=0x7ff4a415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 shutdown_connections 2024-04-05T22:32:22.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.046+0000 7ff4a8c00640 1 -- 172.21.15.73:0/2105840956 wait complete. 2024-04-05T22:32:22.066 INFO:tasks.ceph:Scrubbing osd.7 2024-04-05T22:32:22.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-05T22:32:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.278+0000 7f1a97e00640 1 Processor -- start 2024-04-05T22:32:22.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.278+0000 7f1a97e00640 1 -- start start 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a97e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a9015ae50 0x7f1a900b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a97e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a900cf3e0 0x7f1a900b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a97e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9015b230 con 0x7f1a900cce90 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a97e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a900cd320 con 0x7f1a900cf3e0 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a97e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a900ce2f0 con 0x7f1a9015ae50 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a94a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1a900cce90 0x7f1a900cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44172/0 (socket says 172.21.15.73:44172) 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a94a00640 1 -- 172.21.15.73:0/3133149195 learned_addr learned my addr 172.21.15.73:0/3133149195 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.280+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3133149195 >> v2:172.21.15.73:3300/0 conn(0x7f1a900cf3e0 0x7f1a900b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a95400640 1 --2- 172.21.15.73:0/3133149195 >> v2:172.21.15.157:3300/0 conn(0x7f1a9015ae50 0x7f1a900b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a8fe00640 1 -- 172.21.15.73:0/3133149195 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058510286 0 0) 0x7f1a9015b230 con 0x7f1a900cce90 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a8fe00640 1 -- 172.21.15.73:0/3133149195 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a78003520 con 0x7f1a900cce90 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a95e00640 1 -- 172.21.15.73:0/3133149195 >> v2:172.21.15.157:3300/0 conn(0x7f1a9015ae50 msgr2=0x7f1a900b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3133149195 >> v2:172.21.15.157:3300/0 conn(0x7f1a9015ae50 0x7f1a900b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a95e00640 1 -- 172.21.15.73:0/3133149195 >> v1:172.21.15.73:6789/0 conn(0x7f1a900cce90 legacy=0x7f1a900cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.281+0000 7f1a95e00640 1 -- 172.21.15.73:0/3133149195 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9015f4a0 con 0x7f1a900cf3e0 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.282+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3133149195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a900cf3e0 0x7f1a900b2e00 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f1a9006dbb0 tx=0x7f1a8802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca9b467806071a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.282+0000 7f1a8fe00640 1 -- 172.21.15.73:0/3133149195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8802cbe0 con 0x7f1a900cf3e0 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.282+0000 7f1a8fe00640 1 -- 172.21.15.73:0/3133149195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8802cd40 con 0x7f1a900cf3e0 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.282+0000 7f1a8fe00640 1 -- 172.21.15.73:0/3133149195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88034660 con 0x7f1a900cf3e0 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 -- 172.21.15.73:0/3133149195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a900cf3e0 msgr2=0x7f1a900b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3133149195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a900cf3e0 0x7f1a900b2e00 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f1a9006dbb0 tx=0x7f1a8802be30 comp rx=0 tx=0).stop 2024-04-05T22:32:22.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 -- 172.21.15.73:0/3133149195 shutdown_connections 2024-04-05T22:32:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3133149195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a900cf3e0 0x7f1a900b2e00 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3133149195 >> v2:172.21.15.157:3300/0 conn(0x7f1a9015ae50 0x7f1a900b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 -- 172.21.15.73:0/3133149195 >> 172.21.15.73:0/3133149195 conn(0x7f1a9006a0d0 msgr2=0x7f1a9006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 -- 172.21.15.73:0/3133149195 shutdown_connections 2024-04-05T22:32:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.283+0000 7f1a97e00640 1 -- 172.21.15.73:0/3133149195 wait complete. 2024-04-05T22:32:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.285+0000 7f1a97e00640 1 Processor -- start 2024-04-05T22:32:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.285+0000 7f1a97e00640 1 -- start start 2024-04-05T22:32:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a900cce90 0x7f1a900ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a900cf3e0 0x7f1a900ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 0x7f1a900aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900ab030 con 0x7f1a9015ae50 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900ab310 con 0x7f1a900cf3e0 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a97e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900ab5f0 con 0x7f1a900cce90 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a95e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 0x7f1a900aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a95e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 0x7f1a900aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41788/0 (socket says 172.21.15.73:41788) 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.286+0000 7f1a95e00640 1 -- 172.21.15.73:0/3139349116 learned_addr learned my addr 172.21.15.73:0/3139349116 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95400640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a900cce90 0x7f1a900ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 -- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a900cce90 msgr2=0x7f1a900ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a900cce90 0x7f1a900ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a94a00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a900cf3e0 0x7f1a900ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 -- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a900cf3e0 msgr2=0x7f1a900ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a900cf3e0 0x7f1a900ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88009a70 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a94a00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a900cf3e0 0x7f1a900ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a95e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 0x7f1a900aea60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009e80 tx=0x7f1a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88040650 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a880407b0 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88040b60 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.287+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a900ab8d0 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.288+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a900ace40 con 0x7f1a9015ae50 2024-04-05T22:32:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.289+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a900cb610 con 0x7f1a9015ae50 2024-04-05T22:32:22.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.290+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f1a88040cc0 con 0x7f1a9015ae50 2024-04-05T22:32:22.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a8d600640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1a7006b740 0x7f1a7006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1a880ad390 con 0x7f1a9015ae50 2024-04-05T22:32:22.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a8d600640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f1a700715a0 0x7f1a700739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a70074070 con 0x7f1a700715a0 2024-04-05T22:32:22.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=175) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a880af7f0 con 0x7f1a9015ae50 2024-04-05T22:32:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.291+0000 7f1a95400640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1a7006b740 0x7f1a7006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.292+0000 7f1a94a00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f1a700715a0 0x7f1a700739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.292+0000 7f1a94a00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f1a700715a0 0x7f1a700739c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.293+0000 7f1a95400640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1a7006b740 0x7f1a7006dc00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006fd0 tx=0x7f1a84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.295+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== osd.7 v2:172.21.15.157:6816/37603 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1a70074070 con 0x7f1a700715a0 2024-04-05T22:32:22.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.308+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 --> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a900cb330 con 0x7f1a700715a0 2024-04-05T22:32:22.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.309+0000 7f1a8d600640 1 -- 172.21.15.73:0/3139349116 <== osd.7 v2:172.21.15.157:6816/37603 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1a7c0018a0 con 0x7f1a700715a0 2024-04-05T22:32:22.309 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-05T22:32:22.309 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-05T22:32:22.309 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-05T22:32:22.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.309+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f1a700715a0 msgr2=0x7f1a700739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.309+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6816/37603,v1:172.21.15.157:6817/37603] conn(0x7f1a700715a0 0x7f1a700739c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1a7006b740 msgr2=0x7f1a7006dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f1a7006b740 0x7f1a7006dc00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006fd0 tx=0x7f1a84008040 comp rx=0 tx=0).stop 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 msgr2=0x7f1a900aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 --2- 172.21.15.73:0/3139349116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a9015ae50 0x7f1a900aea60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009e80 tx=0x7f1a88004a40 comp rx=0 tx=0).stop 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a95e00640 1 -- 172.21.15.73:0/3139349116 reap_dead start 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 shutdown_connections 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 >> 172.21.15.73:0/3139349116 conn(0x7f1a9006a0d0 msgr2=0x7f1a9006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 shutdown_connections 2024-04-05T22:32:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.310+0000 7f1a97e00640 1 -- 172.21.15.73:0/3139349116 wait complete. 2024-04-05T22:32:22.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-05T22:32:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.533+0000 7f534a000640 1 Processor -- start 2024-04-05T22:32:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.534+0000 7f534a000640 1 -- start start 2024-04-05T22:32:22.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f53440ccd20 0x7f534406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f534406d0c0 0x7f534415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534406f5b0 con 0x7f534406d0c0 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5344070410 con 0x7f53440ccd20 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f534a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5344158bb0 con 0x7f53441587d0 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.535+0000 7f5343e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f534406d0c0 0x7f534415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f534406d0c0 0x7f534415e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41792/0 (socket says 172.21.15.73:41792) 2024-04-05T22:32:22.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 -- 172.21.15.73:0/116916220 learned_addr learned my addr 172.21.15.73:0/116916220 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343400640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5342a00640 1 --2- 172.21.15.73:0/116916220 >> v2:172.21.15.157:3300/0 conn(0x7f53440ccd20 0x7f534406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 -- 172.21.15.73:0/116916220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 msgr2=0x7f53440cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 -- 172.21.15.73:0/116916220 >> v2:172.21.15.157:3300/0 conn(0x7f53440ccd20 msgr2=0x7f534406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 --2- 172.21.15.73:0/116916220 >> v2:172.21.15.157:3300/0 conn(0x7f53440ccd20 0x7f534406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343e00640 1 -- 172.21.15.73:0/116916220 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534406f800 con 0x7f534406d0c0 2024-04-05T22:32:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.536+0000 7f5343400640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.537+0000 7f5343e00640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415e850 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f5344066850 tx=0x7f533c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9d7954bad3c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f5342000640 1 -- 172.21.15.73:0/116916220 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c031870 con 0x7f534406d0c0 2024-04-05T22:32:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f5342000640 1 -- 172.21.15.73:0/116916220 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0319d0 con 0x7f534406d0c0 2024-04-05T22:32:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f5342000640 1 -- 172.21.15.73:0/116916220 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c031d60 con 0x7f534406d0c0 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 -- 172.21.15.73:0/116916220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 msgr2=0x7f534415e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415e850 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f5344066850 tx=0x7f533c02b910 comp rx=0 tx=0).stop 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 -- 172.21.15.73:0/116916220 shutdown_connections 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415e850 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 --2- 172.21.15.73:0/116916220 >> v2:172.21.15.157:3300/0 conn(0x7f53440ccd20 0x7f534406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 --2- 172.21.15.73:0/116916220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 -- 172.21.15.73:0/116916220 >> 172.21.15.73:0/116916220 conn(0x7f53440ce8e0 msgr2=0x7f53440cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 -- 172.21.15.73:0/116916220 shutdown_connections 2024-04-05T22:32:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.538+0000 7f534a000640 1 -- 172.21.15.73:0/116916220 wait complete. 2024-04-05T22:32:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.540+0000 7f534a000640 1 Processor -- start 2024-04-05T22:32:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.540+0000 7f534a000640 1 -- start start 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53440c3d20 0x7f5344156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f6ae0 con 0x7f534406d0c0 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f5e60 con 0x7f53440c3d20 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.541+0000 7f534a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f6140 con 0x7f53441587d0 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41794/0 (socket says 172.21.15.73:41794) 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 -- 172.21.15.73:0/558466437 learned_addr learned my addr 172.21.15.73:0/558466437 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 -- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 msgr2=0x7f53440c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 -- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53440c3d20 msgr2=0x7f5344156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53440c3d20 0x7f5344156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 -- 172.21.15.73:0/558466437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0094d0 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.542+0000 7f5343400640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f533400c7f0 tx=0x7f533400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.543+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334007820 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.543+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334007980 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.543+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441f6420 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.543+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334007cf0 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.543+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534415a6b0 con 0x7f534406d0c0 2024-04-05T22:32:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.544+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5344066af0 con 0x7f534406d0c0 2024-04-05T22:32:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.545+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f53340113f0 con 0x7f534406d0c0 2024-04-05T22:32:22.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.546+0000 7f533b400640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 0x7f532006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.546+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5334088270 con 0x7f534406d0c0 2024-04-05T22:32:22.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.546+0000 7f5342a00640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 0x7f532006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.548+0000 7f5342a00640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 0x7f532006dc00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f532c007130 tx=0x7f532c007310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.550+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f5334054c50 con 0x7f534406d0c0 2024-04-05T22:32:22.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.741+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f53440700a0 con 0x7f532006b740 2024-04-05T22:32:22.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.742+0000 7f533b400640 1 -- 172.21.15.73:0/558466437 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f532c00cdf0 con 0x7f532006b740 2024-04-05T22:32:22.742 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2024-04-05T22:32:22.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 msgr2=0x7f532006dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 0x7f532006dc00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f532c007130 tx=0x7f532c007310 comp rx=0 tx=0).stop 2024-04-05T22:32:22.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 msgr2=0x7f534415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f533400c7f0 tx=0x7f533400ccb0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 shutdown_connections 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f532006b740 0x7f532006dc00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53440c3d20 0x7f5344156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53441587d0 0x7f53440c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 --2- 172.21.15.73:0/558466437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f534406d0c0 0x7f534415a170 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 >> 172.21.15.73:0/558466437 conn(0x7f53440ce8e0 msgr2=0x7f53440ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 shutdown_connections 2024-04-05T22:32:22.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.745+0000 7f534a000640 1 -- 172.21.15.73:0/558466437 wait complete. 2024-04-05T22:32:22.764 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:32:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.968+0000 7f39a2c00640 1 Processor -- start 2024-04-05T22:32:22.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.969+0000 7f39a2c00640 1 -- start start 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.970+0000 7f39a2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c1587d0 0x7f399c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.970+0000 7f39a2c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f399c0ccd20 0x7f399c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.970+0000 7f39a2c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c06f5b0 con 0x7f399c06d0c0 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.970+0000 7f39a2c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c070410 con 0x7f399c0ccd20 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.970+0000 7f39a2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c158bb0 con 0x7f399c1587d0 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f399c0ccd20 0x7f399c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f399c0ccd20 0x7f399c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41804/0 (socket says 172.21.15.73:41804) 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 -- 172.21.15.73:0/2317709119 learned_addr learned my addr 172.21.15.73:0/2317709119 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f399be00640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c1587d0 0x7f399c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 -- 172.21.15.73:0/2317709119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c1587d0 msgr2=0x7f399c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c1587d0 0x7f399c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 -- 172.21.15.73:0/2317709119 >> v1:172.21.15.157:6789/0 conn(0x7f399c06d0c0 legacy=0x7f399c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:22.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.971+0000 7f3993400640 1 -- 172.21.15.73:0/2317709119 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c06f800 con 0x7f399c0ccd20 2024-04-05T22:32:22.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.972+0000 7f3993400640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c0ccd20 0x7f399c06cb80 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3984009870 tx=0x7f398402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c161a575dde84b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f399b400640 1 -- 172.21.15.73:0/2317709119 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984031640 con 0x7f399c0ccd20 2024-04-05T22:32:22.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f399b400640 1 -- 172.21.15.73:0/2317709119 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39840317a0 con 0x7f399c0ccd20 2024-04-05T22:32:22.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f399b400640 1 -- 172.21.15.73:0/2317709119 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984031b30 con 0x7f399c0ccd20 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 -- 172.21.15.73:0/2317709119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c0ccd20 msgr2=0x7f399c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c0ccd20 0x7f399c06cb80 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3984009870 tx=0x7f398402b8f0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 -- 172.21.15.73:0/2317709119 shutdown_connections 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c0ccd20 0x7f399c06cb80 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 --2- 172.21.15.73:0/2317709119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c1587d0 0x7f399c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.973+0000 7f39a2c00640 1 -- 172.21.15.73:0/2317709119 >> 172.21.15.73:0/2317709119 conn(0x7f399c0ce8e0 msgr2=0x7f399c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.974+0000 7f39a2c00640 1 -- 172.21.15.73:0/2317709119 shutdown_connections 2024-04-05T22:32:22.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.974+0000 7f39a2c00640 1 -- 172.21.15.73:0/2317709119 wait complete. 2024-04-05T22:32:22.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.975+0000 7f39a2c00640 1 Processor -- start 2024-04-05T22:32:22.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.975+0000 7f39a2c00640 1 -- start start 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c06d0c0 0x7f399c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f399c1587d0 0x7f399c155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1566a0 con 0x7f399c155580 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c155a90 con 0x7f399c1587d0 2024-04-05T22:32:22.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c155d70 con 0x7f399c06d0c0 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.976+0000 7f39a0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41814/0 (socket says 172.21.15.73:41814) 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 -- 172.21.15.73:0/3689166577 learned_addr learned my addr 172.21.15.73:0/3689166577 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f399be00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c06d0c0 0x7f399c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f3993400640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f399c1587d0 0x7f399c155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 -- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c06d0c0 msgr2=0x7f399c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c06d0c0 0x7f399c143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 -- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f399c1587d0 msgr2=0x7f399c155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f399c1587d0 0x7f399c155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 -- 172.21.15.73:0/3689166577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840094d0 con 0x7f399c155580 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.977+0000 7f39a0c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f399400c780 tx=0x7f399400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.978+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994004b60 con 0x7f399c155580 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.978+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399400ce90 con 0x7f399c155580 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.978+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400a850 con 0x7f399c155580 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.978+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c156050 con 0x7f399c155580 2024-04-05T22:32:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.978+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c143f70 con 0x7f399c155580 2024-04-05T22:32:22.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.979+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c066af0 con 0x7f399c155580 2024-04-05T22:32:22.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.980+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f399400a9b0 con 0x7f399c155580 2024-04-05T22:32:22.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.981+0000 7f3998c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 0x7f397806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:22.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.981+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3994088050 con 0x7f399c155580 2024-04-05T22:32:22.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.981+0000 7f399be00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 0x7f397806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.983+0000 7f399be00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 0x7f397806dc00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f399c15c020 tx=0x7f398c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:22.985+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f3994054a30 con 0x7f399c155580 2024-04-05T22:32:23.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.174+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f399c0700a0 con 0x7f397806b740 2024-04-05T22:32:23.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.181+0000 7f3998c00640 1 -- 172.21.15.73:0/3689166577 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48654 (secure 0 0 0) 0x7f398c0108d0 con 0x7f397806b740 2024-04-05T22:32:23.181 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:23.182 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.183+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 msgr2=0x7f397806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.183+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 0x7f397806dc00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f399c15c020 tx=0x7f398c008040 comp rx=0 tx=0).stop 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.183+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 msgr2=0x7f399c0aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.183+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f399400c780 tx=0x7f399400cc40 comp rx=0 tx=0).stop 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 shutdown_connections 2024-04-05T22:32:23.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f397806b740 0x7f397806dc00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399c155580 0x7f399c0aacc0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f399c1587d0 0x7f399c155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 --2- 172.21.15.73:0/3689166577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f399c06d0c0 0x7f399c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 >> 172.21.15.73:0/3689166577 conn(0x7f399c0ce8e0 msgr2=0x7f399c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 shutdown_connections 2024-04-05T22:32:23.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:23.184+0000 7f39a2c00640 1 -- 172.21.15.73:0/3689166577 wait complete. 2024-04-05T22:32:23.203 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-04-05T22:32:21.625659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291956,"kb_used_data":18440,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754682764,"statfs":{"total":773094113280,"available":772795150336,"internally_reserved":0,"allocated":18882560,"data_stored":8973408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":279957051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002878"},"pg_stats":[{"pgid":"1.0","version":"75'128","reported_seq":334,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-04-05T22:32:09.019287+0000","last_change":"2024-04-05T22:21:10.419609+0000","last_active":"2024-04-05T22:32:09.019287+0000","last_peered":"2024-04-05T22:32:09.019287+0000","last_clean":"2024-04-05T22:32:09.019287+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:32:09.019287+0000","last_undegraded":"2024-04-05T22:32:09.019287+0000","last_fullsized":"2024-04-05T22:32:09.019287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-05T22:21:09.102542+0000","last_clean_scrub_stamp":"2024-04-05T22:21:09.102542+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T06:38:28.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":1.0389999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.159},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.097}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":1.371},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.339}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.5}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.623},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.6060000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37372,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.66200000000000003,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.774,"5min":1.774,"15min":2.2509999999999999},"last":1.125},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.0740000000000001,"15min":1.089},"min":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.129,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.42499999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":1.036},{"interface":"front","average":{"1min":1.143,"5min":1.125,"15min":1.135},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4890000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.7190000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.86099999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.6000000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.1859999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.43099999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.86699999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.046,"5min":0.73299999999999998,"15min":0.65500000000000003},"max":{"1min":1.6779999999999999,"5min":1.96,"15min":1.96},"last":1.569},{"interface":"front","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.97999999999999998,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5609999999999999,"5min":1.95,"15min":3.1720000000000002},"last":1.3600000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.387,"15min":1.3320000000000001},"min":{"1min":1.141,"5min":0.66000000000000003,"15min":0.60099999999999998},"max":{"1min":1.897,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.907},{"interface":"front","average":{"1min":1.381,"5min":1.4199999999999999,"15min":1.4930000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.8149999999999999,"5min":1.9399999999999999,"15min":3.27},"last":1.2509999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.3819999999999999},"min":{"1min":0.82299999999999995,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.508,"15min":1.518},"min":{"1min":0.99299999999999999,"5min":0.81100000000000005,"15min":0.76800000000000002},"max":{"1min":1.845,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.8180000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.57299999999999995},"max":{"1min":1.7090000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.49},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4510000000000001,"15min":1.496},"min":{"1min":0.91500000000000004,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.6519999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37372,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1439999999999999,"15min":1.0669999999999999},"min":{"1min":0.73699999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.4350000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.151,"5min":1.1890000000000001,"15min":1.1259999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.58199999999999996},"max":{"1min":1.7769999999999999,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.206,"15min":1.1839999999999999},"min":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.63500000000000001},"max":{"1min":1.591,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.159,"5min":1.379,"15min":1.242},"min":{"1min":0.72699999999999998,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.6040000000000001,"5min":17.381,"15min":17.381},"last":1.292}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.417,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.61799999999999999},"max":{"1min":1.903,"5min":24.52,"15min":24.52},"last":1.054},{"interface":"front","average":{"1min":1.155,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85099999999999998,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.484,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":0.94799999999999995}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2230000000000001,"15min":1.2030000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.7270000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.57099999999999995},"max":{"1min":1.573,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.163}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.456,"15min":1.327},"min":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.42899999999999999},"max":{"1min":1.7190000000000001,"5min":14.933,"15min":14.933},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3089999999999999},"min":{"1min":0.65000000000000002,"5min":0.52100000000000002,"15min":0.48699999999999999},"max":{"1min":1.599,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.448,"15min":1.361},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.8640000000000001,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.5569999999999999,"15min":1.5149999999999999},"min":{"1min":0.91600000000000004,"5min":0.77200000000000002,"15min":0.55200000000000005},"max":{"1min":1.829,"5min":3.782,"15min":3.782},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5369999999999999,"15min":1.4750000000000001},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.73,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.528},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5629999999999999,"15min":1.5800000000000001},"min":{"1min":0.81299999999999994,"5min":0.66700000000000004,"15min":0.47399999999999998},"max":{"1min":1.8080000000000001,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.577}]}]},{"osd":4,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0700000000000001,"15min":1.0309999999999999},"min":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59399999999999997},"max":{"1min":1.427,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1140000000000001,"15min":1.0900000000000001},"min":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.629,"5min":1.7769999999999999,"15min":1.859},"last":1.2030000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.119},"min":{"1min":0.68000000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.76,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.421},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1799999999999999,"15min":1.1659999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60299999999999998},"max":{"1min":1.476,"5min":2.577,"15min":2.577},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1379999999999999,"15min":1.1799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.974},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.194},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6160000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.272}]},{"osd":3,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.835,"15min":2.1840000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.363,"15min":1.383},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6240000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.5209999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3009999999999999,"15min":1.288},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7390000000000001,"5min":2.665,"15min":2.665},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8280000000000001,"5min":2.2360000000000002,"15min":2.3220000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4419999999999999,"15min":1.4359999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.8740000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4239999999999999,"15min":1.456},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.0209999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":1.788}]},{"osd":7,"last update":"Fri Apr 5 22:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5589999999999999,"15min":1.536},"min":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0579999999999998,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.609,"15min":1.585},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.9770000000000001,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":1.8520000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33276,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":1.363},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.018}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.54}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.645}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.206},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":1.474}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":1.405}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":1.591},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.2869999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.55600000000000005,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4350000000000001,"5min":2.879,"15min":2.879},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.98899999999999999,"15min":0.94299999999999995},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5760000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.2070000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2190000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.90200000000000002},"min":{"1min":0.57699999999999996,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.216,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.4750000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.04,"15min":1.0429999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5760000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.65200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.375}]},{"osd":4,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.325,"15min":1.268},"min":{"1min":0.97899999999999998,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.6399999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.647},{"interface":"front","average":{"1min":1.226,"5min":1.29,"15min":1.252},"min":{"1min":0.69499999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.8420000000000001,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.5529999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0409999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.5},{"interface":"front","average":{"1min":1.157,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5289999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2669999999999999,"15min":1.266},"min":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.5109999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.294,"15min":1.286},"min":{"1min":0.96699999999999997,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.7130000000000001,"5min":2.242,"15min":2.7890000000000001},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:24 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3500000000000001},"min":{"1min":0.85899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9339999999999999,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":0.92100000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.571,"5min":2.597,"15min":2.597},"last":1.6060000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58799999999999997,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.0740000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.36799999999999999},"max":{"1min":1.3520000000000001,"5min":1.5960000000000001,"15min":1.9119999999999999},"last":1.0309999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.63500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3049999999999999,"5min":1.488,"15min":1.5389999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.54,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.081},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.5429999999999999,"5min":1.6539999999999999,"15min":1.75},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.504,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.319}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.7769999999999999,"5min":1.8500000000000001,"15min":1.855},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.268,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.73299999999999998,"15min":0.68999999999999995},"max":{"1min":1.6479999999999999,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.72099999999999997,"15min":0.68100000000000005},"max":{"1min":1.845,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2330000000000001,"15min":1.236},"min":{"1min":0.85799999999999998,"5min":0.64700000000000002,"15min":0.629},"max":{"1min":1.544,"5min":2.0680000000000001,"15min":2.173},"last":1.077}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.335,"15min":1.3400000000000001},"min":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.758,"5min":1.857,"15min":1.917},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3340000000000001,"15min":1.345},"min":{"1min":0.98699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.952,"5min":2.157,"15min":2.157},"last":1.163}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.397,"15min":1.369},"min":{"1min":0.872,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8220000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.266},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.329},"min":{"1min":1.0049999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.905,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.4239999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.3620000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.456},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.6120000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6759999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:32:23.207 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-05T22:21:09.102542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=5, tm_hour=22, tm_min=21, tm_sec=9, tm_wday=4, tm_yday=96, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=5, tm_hour=22, tm_min=32, tm_sec=16, tm_wday=4, tm_yday=96, tm_isdst=0) 2024-04-05T22:32:23.207 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-05T22:32:40.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:40.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:43.209 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-05T22:32:43.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.421+0000 7f6f4e600640 1 Processor -- start 2024-04-05T22:32:43.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.421+0000 7f6f4e600640 1 -- start start 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.422+0000 7f6f4e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f481587d0 0x7f6f480cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.422+0000 7f6f4e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 0x7f6f4806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f4e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f4806d0c0 0x7f6f4815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f4e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4806f5b0 con 0x7f6f481587d0 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f4e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f48070410 con 0x7f6f480ccd20 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f4e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48158bb0 con 0x7f6f4806d0c0 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f481587d0 0x7f6f480cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f481587d0 0x7f6f480cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45112/0 (socket says 172.21.15.73:45112) 2024-04-05T22:32:43.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 -- 172.21.15.73:0/3103139071 learned_addr learned my addr 172.21.15.73:0/3103139071 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:43.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 -- 172.21.15.73:0/3103139071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f4806d0c0 msgr2=0x7f6f4815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:43.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47e00640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f4806d0c0 0x7f6f4815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f4806d0c0 0x7f6f4815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f3ea00640 1 --2- 172.21.15.73:0/3103139071 >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 0x7f6f4806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 -- 172.21.15.73:0/3103139071 >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 msgr2=0x7f6f4806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 --2- 172.21.15.73:0/3103139071 >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 0x7f6f4806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f47400640 1 -- 172.21.15.73:0/3103139071 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4806f800 con 0x7f6f481587d0 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.423+0000 7f6f3ea00640 1 --2- 172.21.15.73:0/3103139071 >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 0x7f6f4806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:43.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f47400640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f481587d0 0x7f6f480cc7e0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806b560 tx=0x7f6f3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e143e86cddaf198 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f46a00640 1 -- 172.21.15.73:0/3103139071 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38031870 con 0x7f6f481587d0 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f46a00640 1 -- 172.21.15.73:0/3103139071 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f380319d0 con 0x7f6f481587d0 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f46a00640 1 -- 172.21.15.73:0/3103139071 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38031d60 con 0x7f6f481587d0 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f4e600640 1 -- 172.21.15.73:0/3103139071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f481587d0 msgr2=0x7f6f480cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.425+0000 7f6f4e600640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f481587d0 0x7f6f480cc7e0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806b560 tx=0x7f6f3802b910 comp rx=0 tx=0).stop 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 -- 172.21.15.73:0/3103139071 shutdown_connections 2024-04-05T22:32:43.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f4806d0c0 0x7f6f4815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 --2- 172.21.15.73:0/3103139071 >> v2:172.21.15.157:3300/0 conn(0x7f6f480ccd20 0x7f6f4806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 --2- 172.21.15.73:0/3103139071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f481587d0 0x7f6f480cc7e0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 -- 172.21.15.73:0/3103139071 >> 172.21.15.73:0/3103139071 conn(0x7f6f480ce8e0 msgr2=0x7f6f480cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 -- 172.21.15.73:0/3103139071 shutdown_connections 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.426+0000 7f6f4e600640 1 -- 172.21.15.73:0/3103139071 wait complete. 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.427+0000 7f6f4e600640 1 Processor -- start 2024-04-05T22:32:43.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.427+0000 7f6f4e600640 1 -- start start 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f481587d0 0x7f6f480ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f480aee60 0x7f6f480c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480af310 con 0x7f6f4806d0c0 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48149800 con 0x7f6f480aee60 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.428+0000 7f6f4e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481576a0 con 0x7f6f481587d0 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45124/0 (socket says 172.21.15.73:45124) 2024-04-05T22:32:43.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 -- 172.21.15.73:0/2630390996 learned_addr learned my addr 172.21.15.73:0/2630390996 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:43.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 -- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f481587d0 msgr2=0x7f6f480ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f481587d0 0x7f6f480ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 -- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f480aee60 msgr2=0x7f6f480c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f480aee60 0x7f6f480c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 -- 172.21.15.73:0/2630390996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f380094d0 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.429+0000 7f6f47400640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802b950 tx=0x7f6f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.430+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38031860 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.430+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f380443f0 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.430+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380447a0 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.430+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480af530 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.430+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48149490 con 0x7f6f4806d0c0 2024-04-05T22:32:43.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.431+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f48066af0 con 0x7f6f4806d0c0 2024-04-05T22:32:43.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.432+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f6f38044900 con 0x7f6f4806d0c0 2024-04-05T22:32:43.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.433+0000 7f6f3fe00640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 0x7f6f2406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.433+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f3802d040 con 0x7f6f4806d0c0 2024-04-05T22:32:43.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.433+0000 7f6f3ea00640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 0x7f6f2406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.435+0000 7f6f3ea00640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 0x7f6f2406dc00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006fd0 tx=0x7f6f30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.438+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f6f38077180 con 0x7f6f4806d0c0 2024-04-05T22:32:43.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.629+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 --> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f480700a0 con 0x7f6f2406b740 2024-04-05T22:32:43.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.635+0000 7f6f3fe00640 1 -- 172.21.15.73:0/2630390996 <== mgr.4104 v2:172.21.15.157:6836/37023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48311 (secure 0 0 0) 0x7f6f3000ae40 con 0x7f6f2406b740 2024-04-05T22:32:43.636 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:43.636 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-05T22:32:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 msgr2=0x7f6f2406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 0x7f6f2406dc00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006fd0 tx=0x7f6f30008040 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 msgr2=0x7f6f480cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802b950 tx=0x7f6f380046c0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 shutdown_connections 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f6f2406b740 0x7f6f2406dc00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f480aee60 0x7f6f480c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f481587d0 0x7f6f480ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 --2- 172.21.15.73:0/2630390996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f4806d0c0 0x7f6f480cb880 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 >> 172.21.15.73:0/2630390996 conn(0x7f6f480ce8e0 msgr2=0x7f6f4806c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 shutdown_connections 2024-04-05T22:32:43.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.638+0000 7f6f4e600640 1 -- 172.21.15.73:0/2630390996 wait complete. 2024-04-05T22:32:43.657 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-05T22:32:41.631244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291948,"kb_used_data":18440,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754682772,"statfs":{"total":773094113280,"available":772795158528,"internally_reserved":0,"allocated":18882560,"data_stored":8973408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":279957051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003405"},"pg_stats":[{"pgid":"1.0","version":"175'130","reported_seq":348,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-04-05T22:32:31.228730+0000","last_change":"2024-04-05T22:32:31.228988+0000","last_active":"2024-04-05T22:32:31.228730+0000","last_peered":"2024-04-05T22:32:31.228730+0000","last_clean":"2024-04-05T22:32:31.228730+0000","last_became_active":"2024-04-05T22:21:10.417194+0000","last_became_peered":"2024-04-05T22:21:10.417194+0000","last_unstale":"2024-04-05T22:32:31.228730+0000","last_undegraded":"2024-04-05T22:32:31.228730+0000","last_fullsized":"2024-04-05T22:32:31.228730+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'130","last_scrub_stamp":"2024-04-05T22:32:31.228676+0000","last_deep_scrub":"175'130","last_deep_scrub_stamp":"2024-04-05T22:32:31.228676+0000","last_clean_scrub_stamp":"2024-04-05T22:32:31.228676+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T09:51:42.637269+0000","scrub_duration":0.012178408,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542285,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37948,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333892,"statfs":{"total":96636764160,"available":96597905408,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":36043212},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0629999999999999,"15min":1.046},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.698,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.196},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1040000000000001,"15min":1.0509999999999999},"min":{"1min":0.78300000000000003,"5min":0.64900000000000002,"15min":0.63800000000000001},"max":{"1min":1.575,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.0600000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1579999999999999,"15min":1.1020000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.58099999999999996},"max":{"1min":1.528,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.0800000000000001},"min":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.819,"15min":1.819},"last":1.109}]},{"osd":2,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.71399999999999997,"15min":0.66900000000000004},"max":{"1min":1.6140000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.409},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.75},"max":{"1min":1.601,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2290000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.163},"min":{"1min":0.80600000000000005,"5min":0.73099999999999998,"15min":0.72299999999999998},"max":{"1min":1.601,"5min":1.6359999999999999,"15min":1.712},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2290000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.65600000000000003},"max":{"1min":1.5249999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.4950000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.234,"15min":1.23},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.889,"5min":2.71,"15min":2.71},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3460000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.61299999999999999,"15min":0.56899999999999995},"max":{"1min":1.996,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.298}]},{"osd":5,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.335,"15min":1.2789999999999999},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8460000000000001,"5min":3.492,"15min":3.492},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.333,"5min":1.3700000000000001,"15min":1.363},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.782,"5min":2.992,"15min":2.992},"last":1.3680000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.53,"15min":1.4930000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8660000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.452},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.506,"15min":1.498},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.65000000000000002},"max":{"1min":1.8089999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":1.5509999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37372,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.73699999999999999,"5min":0.53500000000000003,"15min":0.45500000000000002},"max":{"1min":1.7470000000000001,"5min":1.774,"15min":2.2509999999999999},"last":1.458},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.073,"15min":1.0880000000000001},"min":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.439},"max":{"1min":1.5880000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.139,"15min":1.139},"min":{"1min":0.751,"5min":0.70499999999999996,"15min":0.42499999999999999},"max":{"1min":1.706,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1220000000000001,"15min":1.1319999999999999},"min":{"1min":0.80600000000000005,"5min":0.73199999999999998,"15min":0.51900000000000002},"max":{"1min":1.7190000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0589999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.159},"min":{"1min":0.86399999999999999,"5min":0.76500000000000001,"15min":0.57099999999999995},"max":{"1min":1.5660000000000001,"5min":1.7030000000000001,"15min":2.3450000000000002},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.1839999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.43099999999999999},"max":{"1min":1.6299999999999999,"5min":2.524,"15min":2.524},"last":1.274}]},{"osd":3,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2569999999999999,"15min":1.2729999999999999},"min":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.65500000000000003},"max":{"1min":1.6799999999999999,"5min":1.96,"15min":1.96},"last":1.206},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.26,"15min":1.335},"min":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.68100000000000005},"max":{"1min":1.5720000000000001,"5min":1.95,"15min":3.1720000000000002},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.393,"15min":1.331},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.60099999999999998},"max":{"1min":1.907,"5min":2.0390000000000001,"15min":2.1019999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.478},"min":{"1min":0.91900000000000004,"5min":0.73799999999999999,"15min":0.64000000000000001},"max":{"1min":1.7150000000000001,"5min":1.9399999999999999,"15min":3.27},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.351,"15min":1.371},"min":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.50600000000000001},"max":{"1min":1.7749999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.532,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.94399999999999995,"5min":0.83499999999999996,"15min":0.76800000000000002},"max":{"1min":1.823,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4179999999999999,"15min":1.468},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.57299999999999995},"max":{"1min":1.8100000000000001,"5min":1.9950000000000001,"15min":3.3540000000000001},"last":1.248},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.47,"15min":1.486},"min":{"1min":1.05,"5min":0.85399999999999998,"15min":0.66400000000000003},"max":{"1min":1.869,"5min":2.6459999999999999,"15min":3.0920000000000001},"last":1.3959999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37372,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.123,"15min":1.0660000000000001},"min":{"1min":0.67700000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.3580000000000001,"5min":7.2329999999999997,"15min":7.2329999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1839999999999999,"15min":1.131},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.58199999999999996},"max":{"1min":1.714,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.639}]},{"osd":1,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.21,"15min":1.1779999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.63500000000000001},"max":{"1min":1.4590000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":1.234},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.3660000000000001,"15min":1.242},"min":{"1min":0.77800000000000002,"5min":0.71599999999999997,"15min":0.47399999999999998},"max":{"1min":1.486,"5min":17.381,"15min":17.381},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.397,"15min":1.2450000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.61799999999999999},"max":{"1min":1.4650000000000001,"5min":24.52,"15min":24.52},"last":1.319},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.78300000000000003,"5min":0.73499999999999999,"15min":0.59299999999999997},"max":{"1min":1.532,"5min":7.5179999999999998,"15min":7.5179999999999998},"last":1.417}]},{"osd":3,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.24,"15min":1.206},"min":{"1min":0.89800000000000002,"5min":0.71499999999999997,"15min":0.68100000000000005},"max":{"1min":1.51,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.3280000000000001,"15min":1.331},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.57099999999999995},"max":{"1min":1.835,"5min":7.4000000000000004,"15min":7.4000000000000004},"last":1.3540000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.327},"min":{"1min":0.88400000000000001,"5min":0.67400000000000004,"15min":0.42899999999999999},"max":{"1min":1.617,"5min":14.933,"15min":14.933},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3360000000000001,"15min":1.306},"min":{"1min":0.97099999999999997,"5min":0.65000000000000002,"15min":0.48699999999999999},"max":{"1min":1.6279999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4970000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.456,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.57599999999999996,"15min":0.42499999999999999},"max":{"1min":1.827,"5min":7.1959999999999997,"15min":7.1959999999999997},"last":1.45},{"interface":"front","average":{"1min":1.468,"5min":1.5449999999999999,"15min":1.5129999999999999},"min":{"1min":1.2310000000000001,"5min":0.91600000000000004,"15min":0.55200000000000005},"max":{"1min":1.6759999999999999,"5min":3.782,"15min":3.782},"last":1.603}]},{"osd":7,"last update":"Fri Apr 5 22:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.55,"15min":1.476},"min":{"1min":1.2030000000000001,"5min":0.63800000000000001,"15min":0.55000000000000004},"max":{"1min":1.776,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":1.532},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.552,"15min":1.5629999999999999},"min":{"1min":1.163,"5min":0.81299999999999994,"15min":0.47399999999999998},"max":{"1min":1.6579999999999999,"5min":5.1429999999999998,"15min":5.1429999999999998},"last":1.48}]}]},{"osd":4,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0409999999999999,"15min":1.0349999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4730000000000001,"5min":1.51,"15min":2.4489999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.1040000000000001,"15min":1.0920000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.738,"5min":1.738,"15min":1.859},"last":1.0840000000000001}]},{"osd":1,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4730000000000001,"5min":1.76,"15min":2.0329999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1679999999999999,"15min":1.1619999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4730000000000001,"5min":1.659,"15min":2.577},"last":1.427}]},{"osd":2,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.153,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5860000000000001,"5min":1.8069999999999999,"15min":1.974},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.177,"15min":1.194},"min":{"1min":0.72599999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.526,"5min":1.7869999999999999,"15min":2.1880000000000002},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.306,"15min":1.341},"min":{"1min":0.75,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7210000000000001,"5min":1.7889999999999999,"15min":2.1840000000000002},"last":1.236},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.351,"15min":1.375},"min":{"1min":0.82799999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.548,"5min":1.8899999999999999,"15min":2.2759999999999998},"last":1.2110000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3,"15min":1.282},"min":{"1min":0.68999999999999995,"5min":0.56799999999999995,"15min":0.47099999999999997},"max":{"1min":1.8620000000000001,"5min":2.1459999999999999,"15min":2.665},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.399,"15min":1.3859999999999999},"min":{"1min":0.77400000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.96,"5min":1.96,"15min":2.3220000000000001},"last":1.4710000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.431,"15min":1.4279999999999999},"min":{"1min":0.73499999999999999,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.754,"5min":2.0800000000000001,"15min":2.3730000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.429,"15min":1.4510000000000001},"min":{"1min":0.75600000000000001,"5min":0.61799999999999999,"15min":0.53000000000000003},"max":{"1min":2.0299999999999998,"5min":2.3540000000000001,"15min":2.7909999999999999},"last":1.137}]},{"osd":7,"last update":"Fri Apr 5 22:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.5609999999999999,"15min":1.536},"min":{"1min":0.88300000000000001,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":2.093,"5min":2.2959999999999998,"15min":2.4630000000000001},"last":1.331},{"interface":"front","average":{"1min":1.605,"5min":1.5980000000000001,"15min":1.5860000000000001},"min":{"1min":0.95099999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0470000000000002,"5min":2.2530000000000001,"15min":3.4820000000000002},"last":1.52}]}]},{"osd":3,"up_from":14,"seq":60129542285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33276,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74399999999999999,"15min":0.81799999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.744},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.85699999999999998,"15min":0.88800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33900000000000002},"max":{"1min":1.393,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.054}]},{"osd":1,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":1.3939999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.90900000000000003,"15min":0.97399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.214,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4910000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.212},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40600000000000003},"max":{"1min":1.4550000000000001,"5min":1.956,"15min":2.2549999999999999},"last":1.538}]},{"osd":4,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1299999999999999,"15min":1.1930000000000001},"min":{"1min":0.65300000000000002,"5min":0.622,"15min":0.54800000000000004},"max":{"1min":1.359,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.228,"15min":1.266},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.52000000000000002},"max":{"1min":1.659,"5min":1.8089999999999999,"15min":2.1619999999999999},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2470000000000001,"15min":1.3129999999999999},"min":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.56599999999999995},"max":{"1min":1.4590000000000001,"5min":1.8979999999999999,"15min":2.5129999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.2549999999999999,"15min":1.2769999999999999},"min":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.57199999999999995},"max":{"1min":1.712,"5min":1.958,"15min":2.3239999999999998},"last":1.4450000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.505},"max":{"1min":1.7709999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.258},{"interface":"front","average":{"1min":1.212,"5min":1.3280000000000001,"15min":1.3620000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.298,"15min":2.298},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3,"15min":1.377},"min":{"1min":0.76500000000000001,"5min":0.63400000000000001,"15min":0.53400000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":2.468},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.29,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.639,"5min":2.508,"15min":2.798},"last":1.508}]}]},{"osd":2,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.82099999999999995,"15min":0.82599999999999996},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.117,"5min":2.879,"15min":2.879},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.98999999999999999,"15min":0.93899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":5.8979999999999997,"15min":5.8979999999999997},"last":1.0649999999999999}]},{"osd":1,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.82899999999999996,"15min":0.83099999999999996},"min":{"1min":0.36699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.431,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.93400000000000005,"15min":0.90100000000000002},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4750000000000001,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.036,"15min":1.0409999999999999},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":1.5269999999999999,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.032},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1040000000000001,"15min":1.119},"min":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4710000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.343,"15min":1.276},"min":{"1min":0.90600000000000003,"5min":0.61499999999999999,"15min":0.57799999999999996},"max":{"1min":1.7729999999999999,"5min":2.9630000000000001,"15min":2.9630000000000001},"last":1.405},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.3,"15min":1.2509999999999999},"min":{"1min":0.83799999999999997,"5min":0.69199999999999995,"15min":0.625},"max":{"1min":1.6919999999999999,"5min":5.8639999999999999,"15min":5.8639999999999999},"last":1.27}]},{"osd":5,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2689999999999999,"15min":1.244},"min":{"1min":0.76300000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6339999999999999,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.246,"15min":1.25},"min":{"1min":0.754,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7669999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.268,"15min":1.2649999999999999},"min":{"1min":0.82599999999999996,"5min":0.76500000000000001,"15min":0.76200000000000001},"max":{"1min":1.95,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.468},{"interface":"front","average":{"1min":1.252,"5min":1.292,"15min":1.2849999999999999},"min":{"1min":0.72899999999999998,"5min":0.72499999999999998,"15min":0.65300000000000002},"max":{"1min":1.968,"5min":2.242,"15min":2.7890000000000001},"last":1.4570000000000001}]},{"osd":7,"last update":"Fri Apr 5 22:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.331,"15min":1.3360000000000001},"min":{"1min":0.68400000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.575,"5min":5.7169999999999996,"15min":5.7169999999999996},"last":1.427},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.397,"15min":1.389},"min":{"1min":0.98799999999999999,"5min":0.748,"15min":0.748},"max":{"1min":1.873,"5min":2.597,"15min":2.597},"last":1.5669999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37376,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2211840,"data_stored":974084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.35299999999999998},"max":{"1min":1.3160000000000001,"5min":1.3680000000000001,"15min":1.6000000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.01,"5min":0.92200000000000004,"15min":0.88},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.83599999999999997},"min":{"1min":0.42599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.262,"5min":1.4139999999999999,"15min":1.5389999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.98799999999999999},"min":{"1min":0.55800000000000005,"5min":0.41699999999999998,"15min":0.32900000000000001},"max":{"1min":1.474,"5min":1.5649999999999999,"15min":1.7130000000000001},"last":1.1180000000000001}]},{"osd":3,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.042,"15min":1.079},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.47899999999999998},"max":{"1min":1.448,"5min":1.6539999999999999,"15min":1.75},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.012,"15min":0.98399999999999999},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42299999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.3100000000000001}]},{"osd":4,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1080000000000001,"15min":1.1339999999999999},"min":{"1min":0.83099999999999996,"5min":0.67700000000000005,"15min":0.625},"max":{"1min":1.4690000000000001,"5min":1.7769999999999999,"15min":1.855},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.27,"15min":1.278},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.68999999999999995},"max":{"1min":1.5840000000000001,"5min":1.7769999999999999,"15min":2.2789999999999999},"last":1.395}]},{"osd":5,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68100000000000005},"max":{"1min":1.7210000000000001,"5min":1.845,"15min":2.2050000000000001},"last":1.498},{"interface":"front","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2330000000000001},"min":{"1min":0.78700000000000003,"5min":0.70699999999999996,"15min":0.629},"max":{"1min":1.591,"5min":2.0680000000000001,"15min":2.173},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.329,"15min":1.3400000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6160000000000001,"5min":1.8540000000000001,"15min":1.917},"last":1.464},{"interface":"front","average":{"1min":1.373,"5min":1.3320000000000001,"15min":1.345},"min":{"1min":1.032,"5min":0.83399999999999996,"15min":0.70699999999999996},"max":{"1min":2.1339999999999999,"5min":2.157,"15min":2.157},"last":1.373}]},{"osd":7,"last update":"Fri Apr 5 22:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.393,"15min":1.3720000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.68400000000000005},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2490000000000001},"last":1.425},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2849999999999999,"15min":1.3280000000000001},"min":{"1min":0.89100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.679,"5min":1.9330000000000001,"15min":2.3879999999999999},"last":1.264}]}]},{"osd":0,"up_from":14,"seq":60129542286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37948,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333892,"statfs":{"total":96636764160,"available":96597905408,"internally_reserved":0,"allocated":2805760,"data_stored":1564452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.329,"5min":1.4339999999999999,"15min":2.1219999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84399999999999997,"15min":0.88},"min":{"1min":0.496,"5min":0.36299999999999999,"15min":0.31900000000000001},"max":{"1min":1.421,"5min":1.8680000000000001,"15min":3.4870000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.90800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2609999999999999,"5min":1.54,"15min":2.0779999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.91800000000000004,"15min":0.96099999999999997},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.318},"max":{"1min":1.3480000000000001,"5min":1.6639999999999999,"15min":3.4420000000000002},"last":1.123}]},{"osd":3,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.88600000000000001},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.5700000000000001,"15min":1.976},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":1.0680000000000001},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.44700000000000001},"max":{"1min":1.5640000000000001,"5min":1.6080000000000001,"15min":2.157},"last":1.016}]},{"osd":4,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1850000000000001,"15min":1.177},"min":{"1min":0.85299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.841,"5min":1.931,"15min":1.931},"last":1.391},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":1.8520000000000001,"15min":2.0790000000000002},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.64300000000000002},"max":{"1min":1.9359999999999999,"5min":2.0739999999999998,"15min":2.3879999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.9550000000000001},"last":1.5}]},{"osd":6,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2849999999999999,"15min":1.353},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.65500000000000003},"max":{"1min":1.5620000000000001,"5min":1.8340000000000001,"15min":2.1589999999999998},"last":1.466},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3069999999999999,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8720000000000001,"15min":2.4430000000000001},"last":1.4419999999999999}]},{"osd":7,"last update":"Fri Apr 5 22:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3100000000000001,"15min":1.337},"min":{"1min":0.873,"5min":0.78300000000000003,"15min":0.67100000000000004},"max":{"1min":2.0219999999999998,"5min":2.0449999999999999,"15min":2.2839999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.248,"15min":1.298},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.5669999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-05T22:32:43.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-05T22:32:43.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.852+0000 7fe05c400640 1 Processor -- start 2024-04-05T22:32:43.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.853+0000 7fe05c400640 1 -- start start 2024-04-05T22:32:43.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe05c400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe05406a850 0x7fe054069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe05c400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0540c9af0 0x7fe05406a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe05c400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0540ad3a0 con 0x7fe054155670 2024-04-05T22:32:43.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe05c400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05406af70 con 0x7fe05406a850 2024-04-05T22:32:43.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe05c400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05406b1e0 con 0x7fe0540c9af0 2024-04-05T22:32:43.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe059a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe054155670 0x7fe0540abfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49392/0 (socket says 172.21.15.73:49392) 2024-04-05T22:32:43.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe059a00640 1 -- 172.21.15.73:0/409797760 learned_addr learned my addr 172.21.15.73:0/409797760 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:43.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.854+0000 7fe059000640 1 --2- 172.21.15.73:0/409797760 >> v2:172.21.15.73:3300/0 conn(0x7fe05406a850 0x7fe054069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe05a400640 1 --2- 172.21.15.73:0/409797760 >> v2:172.21.15.157:3300/0 conn(0x7fe0540c9af0 0x7fe05406a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe059000640 1 -- 172.21.15.73:0/409797760 >> v2:172.21.15.157:3300/0 conn(0x7fe0540c9af0 msgr2=0x7fe05406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe059000640 1 --2- 172.21.15.73:0/409797760 >> v2:172.21.15.157:3300/0 conn(0x7fe0540c9af0 0x7fe05406a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe059000640 1 -- 172.21.15.73:0/409797760 >> v1:172.21.15.73:6789/0 conn(0x7fe054155670 legacy=0x7fe0540abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe059000640 1 -- 172.21.15.73:0/409797760 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05406b4c0 con 0x7fe05406a850 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.855+0000 7fe053e00640 1 -- 172.21.15.73:0/409797760 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949092203 0 0) 0x7fe048001110 con 0x7fe054155670 2024-04-05T22:32:43.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.856+0000 7fe059000640 1 --2- 172.21.15.73:0/409797760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe05406a850 0x7fe054069cb0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ca1b0 tx=0x7fe04402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901358228bb380cd server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.856+0000 7fe053e00640 1 -- 172.21.15.73:0/409797760 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04402cc80 con 0x7fe05406a850 2024-04-05T22:32:43.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.856+0000 7fe053e00640 1 -- 172.21.15.73:0/409797760 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04402cde0 con 0x7fe05406a850 2024-04-05T22:32:43.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.856+0000 7fe053e00640 1 -- 172.21.15.73:0/409797760 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044034660 con 0x7fe05406a850 2024-04-05T22:32:43.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 -- 172.21.15.73:0/409797760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe05406a850 msgr2=0x7fe054069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 --2- 172.21.15.73:0/409797760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe05406a850 0x7fe054069cb0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ca1b0 tx=0x7fe04402bed0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 -- 172.21.15.73:0/409797760 shutdown_connections 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 --2- 172.21.15.73:0/409797760 >> v2:172.21.15.157:3300/0 conn(0x7fe0540c9af0 0x7fe05406a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 --2- 172.21.15.73:0/409797760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe05406a850 0x7fe054069cb0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 -- 172.21.15.73:0/409797760 >> 172.21.15.73:0/409797760 conn(0x7fe0540b0ce0 msgr2=0x7fe0540b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 -- 172.21.15.73:0/409797760 shutdown_connections 2024-04-05T22:32:43.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.857+0000 7fe05c400640 1 -- 172.21.15.73:0/409797760 wait complete. 2024-04-05T22:32:43.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.859+0000 7fe05c400640 1 Processor -- start 2024-04-05T22:32:43.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.859+0000 7fe05c400640 1 -- start start 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05406a850 0x7fe054154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0540c9af0 0x7fe054159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415a410 con 0x7fe054155670 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054159560 con 0x7fe05406a850 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe054159800 con 0x7fe0540c9af0 2024-04-05T22:32:43.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45156/0 (socket says 172.21.15.73:45156) 2024-04-05T22:32:43.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.860+0000 7fe05a400640 1 -- 172.21.15.73:0/4130791492 learned_addr learned my addr 172.21.15.73:0/4130791492 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe059a00640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05406a850 0x7fe054154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe059000640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0540c9af0 0x7fe054159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 -- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0540c9af0 msgr2=0x7fe054159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0540c9af0 0x7fe054159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 -- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05406a850 msgr2=0x7fe054154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05406a850 0x7fe054154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 -- 172.21.15.73:0/4130791492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044009a70 con 0x7fe054155670 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05a400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00d800 tx=0x7fe04c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00b6c0 con 0x7fe054155670 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c00b840 con 0x7fe054155670 2024-04-05T22:32:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00bbf0 con 0x7fe054155670 2024-04-05T22:32:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.861+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe054159ae0 con 0x7fe054155670 2024-04-05T22:32:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.862+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05415a0a0 con 0x7fe054155670 2024-04-05T22:32:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.863+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05415a580 con 0x7fe054155670 2024-04-05T22:32:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.863+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7fe04c00bd50 con 0x7fe054155670 2024-04-05T22:32:43.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.864+0000 7fe051600640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 0x7fe03806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:43.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.864+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe04c0882c0 con 0x7fe054155670 2024-04-05T22:32:43.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.865+0000 7fe059a00640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 0x7fe03806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:43.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.866+0000 7fe059a00640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 0x7fe03806dc50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe048000f60 tx=0x7fe048007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:43.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:43.869+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7fe04c054ef0 con 0x7fe054155670 2024-04-05T22:32:44.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.064+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe054068160 con 0x7fe054155670 2024-04-05T22:32:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.073+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe04c054ef0 con 0x7fe054155670 2024-04-05T22:32:44.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.074+0000 7fe051600640 1 -- 172.21.15.73:0/4130791492 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe04c058510 con 0x7fe054155670 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.077+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 msgr2=0x7fe03806dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.077+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 0x7fe03806dc50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe048000f60 tx=0x7fe048007c00 comp rx=0 tx=0).stop 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.077+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 msgr2=0x7fe05414b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.077+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00d800 tx=0x7fe04c00dcc0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 shutdown_connections 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7fe03806b790 0x7fe03806dc50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054155670 0x7fe05414b9b0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0540c9af0 0x7fe054159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 --2- 172.21.15.73:0/4130791492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05406a850 0x7fe054154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 >> 172.21.15.73:0/4130791492 conn(0x7fe0540b0ce0 msgr2=0x7fe054153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 shutdown_connections 2024-04-05T22:32:44.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.078+0000 7fe05c400640 1 -- 172.21.15.73:0/4130791492 wait complete. 2024-04-05T22:32:44.097 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-05T22:32:44.098 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-05T22:32:44.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.309+0000 7f42afe00640 1 Processor -- start 2024-04-05T22:32:44.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.309+0000 7f42afe00640 1 -- start start 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.310+0000 7f42afe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42b013f3b0 0x7f42b00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.310+0000 7f42afe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b006ca50 con 0x7f42b006c4a0 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42afe00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b006cd30 con 0x7f42b00b2210 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42afe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b006f920 con 0x7f42b013f3b0 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42aea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42b013f3b0 0x7f42b00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42af400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f42b006c4a0 0x7f42b006fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:49398/0 (socket says 172.21.15.73:49398) 2024-04-05T22:32:44.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42af400640 1 -- 172.21.15.73:0/589865847 learned_addr learned my addr 172.21.15.73:0/589865847 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-05T22:32:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42aea00640 1 -- 172.21.15.73:0/589865847 >> v1:172.21.15.157:6789/0 conn(0x7f42b00b2210 legacy=0x7f42b006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-05T22:32:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42aea00640 1 -- 172.21.15.73:0/589865847 >> v1:172.21.15.73:6789/0 conn(0x7f42b006c4a0 legacy=0x7f42b006fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42aea00640 1 -- 172.21.15.73:0/589865847 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b006d010 con 0x7f42b013f3b0 2024-04-05T22:32:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.311+0000 7f42ae000640 1 -- 172.21.15.73:0/589865847 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095812134 0 0) 0x7f42b006ca50 con 0x7f42b006c4a0 2024-04-05T22:32:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.312+0000 7f42aea00640 1 --2- 172.21.15.73:0/589865847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b00b1cd0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f42a00097a0 tx=0x7f42a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f51992c2efa2945 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.313+0000 7f42ae000640 1 -- 172.21.15.73:0/589865847 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a002e070 con 0x7f42b013f3b0 2024-04-05T22:32:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.313+0000 7f42ae000640 1 -- 172.21.15.73:0/589865847 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f42a00315a0 con 0x7f42b013f3b0 2024-04-05T22:32:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.313+0000 7f42ae000640 1 -- 172.21.15.73:0/589865847 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0031950 con 0x7f42b013f3b0 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.313+0000 7f42afe00640 1 -- 172.21.15.73:0/589865847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 msgr2=0x7f42b00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.313+0000 7f42afe00640 1 --2- 172.21.15.73:0/589865847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b00b1cd0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f42a00097a0 tx=0x7f42a002b840 comp rx=0 tx=0).stop 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.314+0000 7f42afe00640 1 -- 172.21.15.73:0/589865847 shutdown_connections 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.314+0000 7f42afe00640 1 --2- 172.21.15.73:0/589865847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b00b1cd0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.314+0000 7f42afe00640 1 -- 172.21.15.73:0/589865847 >> 172.21.15.73:0/589865847 conn(0x7f42b00b06c0 msgr2=0x7f42b00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.314+0000 7f42afe00640 1 -- 172.21.15.73:0/589865847 shutdown_connections 2024-04-05T22:32:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.314+0000 7f42afe00640 1 -- 172.21.15.73:0/589865847 wait complete. 2024-04-05T22:32:44.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.315+0000 7f42afe00640 1 Processor -- start 2024-04-05T22:32:44.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.315+0000 7f42afe00640 1 -- start start 2024-04-05T22:32:44.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 0x7f42b013e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42b00b2210 0x7f42b0153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b0157330 con 0x7f42b013f3b0 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01544f0 con 0x7f42b006c4a0 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.316+0000 7f42afe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b0157050 con 0x7f42b00b2210 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45178/0 (socket says 172.21.15.73:45178) 2024-04-05T22:32:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 -- 172.21.15.73:0/1757131160 learned_addr learned my addr 172.21.15.73:0/1757131160 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42aea00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 0x7f42b013e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 -- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42b00b2210 msgr2=0x7f42b0153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42a6000640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42b00b2210 0x7f42b0153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42b00b2210 0x7f42b0153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 -- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 msgr2=0x7f42b013e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 0x7f42b013e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42af400640 1 -- 172.21.15.73:0/1757131160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009400 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.317+0000 7f42aea00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 0x7f42b013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42af400640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c7e0 tx=0x7f42a800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800a640 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f42a800a7c0 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b014ee20 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800ab90 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.318+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b014f3e0 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.319+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b00cf840 con 0x7f42b013f3b0 2024-04-05T22:32:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.320+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 90165+0+0 (secure 0 0 0) 0x7f42a800acf0 con 0x7f42b013f3b0 2024-04-05T22:32:44.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.321+0000 7f42a7400640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 0x7f428c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-05T22:32:44.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.321+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f42a80878c0 con 0x7f42b013f3b0 2024-04-05T22:32:44.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.322+0000 7f42aea00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 0x7f428c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-05T22:32:44.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.323+0000 7f42aea00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 0x7f428c06dc00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f42a0036000 tx=0x7f42a003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-05T22:32:44.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.325+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181812 (secure 0 0 0) 0x7f42a80bd650 con 0x7f42b013f3b0 2024-04-05T22:32:44.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.554+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42b014f600 con 0x7f42b013f3b0 2024-04-05T22:32:44.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.555+0000 7f42a7400640 1 -- 172.21.15.73:0/1757131160 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+3541 (secure 0 0 0) 0x7f42a8018020 con 0x7f42b013f3b0 2024-04-05T22:32:44.555 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-05T22:32:44.555 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":112,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1874725366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1874725366},{"type":"v1","addr":"172.21.15.157:6835","nonce":1874725366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/4085268024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6833","nonce":4085268024},{"type":"v1","addr":"172.21.15.73:6835","nonce":4085268024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2990256860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2990256860},{"type":"v1","addr":"172.21.15.157:6833","nonce":2990256860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":10563,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6834/3688991365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3688991365},{"type":"v1","addr":"172.21.15.73:6834","nonce":3688991365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[]} 2024-04-05T22:32:44.556 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 112 2024-04-05T22:32:44.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 msgr2=0x7f428c06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 0x7f428c06dc00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f42a0036000 tx=0x7f42a003d040 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 msgr2=0x7f42b0153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c7e0 tx=0x7f42a800cca0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 shutdown_connections 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:6836/37023,v1:172.21.15.157:6837/37023] conn(0x7f428c06b740 0x7f428c06dc00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42b013f3b0 0x7f42b0153fb0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42b00b2210 0x7f42b0153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 --2- 172.21.15.73:0/1757131160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42b006c4a0 0x7f42b013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 >> 172.21.15.73:0/1757131160 conn(0x7f42b00b06c0 msgr2=0x7f42b00711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 shutdown_connections 2024-04-05T22:32:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-05T22:32:44.558+0000 7f42afe00640 1 -- 172.21.15.73:0/1757131160 wait complete. 2024-04-05T22:32:44.577 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-05T22:32:44.578 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-05T22:32:44.578 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-05T22:32:44.578 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.594 INFO:tasks.ceph.mds.a:Stopped 2024-04-05T22:32:44.594 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-05T22:32:44.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.665 INFO:tasks.ceph.mds.c:Stopped 2024-04-05T22:32:44.665 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-05T22:32:44.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.697 INFO:tasks.ceph.mds.b:Stopped 2024-04-05T22:32:44.697 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-05T22:32:44.698 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.765 INFO:tasks.ceph.mds.d:Stopped 2024-04-05T22:32:44.765 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-05T22:32:44.765 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-05T22:32:44.765 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.851 INFO:tasks.ceph.osd.0:Stopped 2024-04-05T22:32:44.851 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-05T22:32:44.852 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.919 INFO:tasks.ceph.osd.1:Stopped 2024-04-05T22:32:44.919 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-05T22:32:44.919 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:44.976 INFO:tasks.ceph.osd.2:Stopped 2024-04-05T22:32:44.977 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-05T22:32:44.977 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.033 INFO:tasks.ceph.osd.3:Stopped 2024-04-05T22:32:45.034 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-05T22:32:45.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.081 INFO:tasks.ceph.osd.4:Stopped 2024-04-05T22:32:45.081 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-05T22:32:45.082 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.142 INFO:tasks.ceph.osd.5:Stopped 2024-04-05T22:32:45.143 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-05T22:32:45.143 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.227 INFO:tasks.ceph.osd.6:Stopped 2024-04-05T22:32:45.227 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-05T22:32:45.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.305 INFO:tasks.ceph.osd.7:Stopped 2024-04-05T22:32:45.305 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-05T22:32:45.305 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-05T22:32:45.305 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:45.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-05T22:32:45.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-05T22:32:45.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-05T22:32:51.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~6s 2024-04-05T22:32:51.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-05T22:32:51.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-05T22:32:51.407 INFO:tasks.ceph.mgr.y:Stopped 2024-04-05T22:32:51.407 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-05T22:32:51.408 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:51.484 INFO:tasks.ceph.mgr.x:Stopped 2024-04-05T22:32:51.484 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-05T22:32:51.485 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-05T22:32:51.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:51.538 INFO:tasks.ceph.mon.a:Stopped 2024-04-05T22:32:51.539 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-05T22:32:51.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:51.583 INFO:tasks.ceph.mon.b:Stopped 2024-04-05T22:32:51.583 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-05T22:32:51.584 INFO:teuthology.orchestra.run:waiting for 300 2024-04-05T22:32:51.619 INFO:tasks.ceph.mon.c:Stopped 2024-04-05T22:32:51.620 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-05T22:32:51.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-05T22:32:51.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:32:51.730 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-05T22:32:51.966 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:32:51.967 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-05T22:32:52.121 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:32:52.122 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-05T22:32:52.278 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:32:52.278 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-05T22:32:52.455 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:32:52.456 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-05T22:32:52.686 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:32:52.686 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-05T22:32:52.845 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:32:52.846 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-05T22:32:52.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:32:52.981 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-05T22:32:53.124 INFO:tasks.ceph:Archiving mon data... 2024-04-05T22:32:53.126 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/data/mon.a.tgz 2024-04-05T22:32:53.126 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-04-05T22:32:53.182 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.2V1QdIQbre 2024-04-05T22:32:53.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2V1QdIQbre 2024-04-05T22:32:53.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.2V1QdIQbre 2024-04-05T22:32:53.613 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.2V1QdIQbre is 988KB 2024-04-05T22:32:53.731 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.2V1QdIQbre 2024-04-05T22:32:53.786 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/data/mon.b.tgz 2024-04-05T22:32:53.787 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-05T22:32:53.843 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.bCjYetEB3O 2024-04-05T22:32:53.843 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.bCjYetEB3O 2024-04-05T22:32:54.147 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.bCjYetEB3O 2024-04-05T22:32:54.295 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.bCjYetEB3O is 991KB 2024-04-05T22:32:54.410 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.bCjYetEB3O 2024-04-05T22:32:54.466 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/data/mon.c.tgz 2024-04-05T22:32:54.467 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-05T22:32:54.519 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.6ZJTGq0UKx 2024-04-05T22:32:54.519 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.6ZJTGq0UKx 2024-04-05T22:32:54.790 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.6ZJTGq0UKx 2024-04-05T22:32:54.929 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.6ZJTGq0UKx is 992KB 2024-04-05T22:32:55.046 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.6ZJTGq0UKx 2024-04-05T22:32:55.103 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-05T22:32:55.103 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-05T22:32:55.111 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-05T22:32:55.206 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-05T22:32:55.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-05T22:32:55.213 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-05T22:32:55.305 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-05T22:32:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-05T22:32:55.325 INFO:tasks.ceph:Archiving crash dumps... 2024-04-05T22:32:55.327 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi073/crash 2024-04-05T22:32:55.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-05T22:32:55.412 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi157/crash 2024-04-05T22:32:55.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-05T22:32:55.505 INFO:tasks.ceph:Shutting down logrotate 2024-04-05T22:32:55.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:55.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:55.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:55.755 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-05T22:32:55.845 INFO:tasks.ceph:Compressing logs... 2024-04-05T22:32:55.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-05T22:32:55.854 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-05T22:32:56.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-05T22:33:01.370 INFO:tasks.ceph:Archiving logs... 2024-04-05T22:33:01.371 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi073/log 2024-04-05T22:33:01.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-05T22:33:02.787 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi157/log 2024-04-05T22:33:02.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-05T22:33:03.441 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-05T22:33:03.451 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-05T22:33:03.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-05T22:33:03.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-05T22:33:03.600 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-05T22:33:03.611 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-05T22:33:03.611 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-05T22:33:03.619 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-05T22:33:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-04-05T22:33:03.673 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-05T22:33:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-05T22:33:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-04-05T22:33:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -1422us[-1422us] +/- 76ms 2024-04-05T22:33:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -579us[ -594us] +/- 59ms 2024-04-05T22:33:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +2053us[+2037us] +/- 79ms 2024-04-05T22:33:03.676 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 -1357us[-1349us] +/- 76ms 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -527us[ -520us] +/- 59ms 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +2114us[+2122us] +/- 79ms 2024-04-05T22:33:03.683 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-05T22:33:03.684 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-05T22:33:03.694 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-05T22:33:03.694 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-05T22:33:03.702 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-05T22:33:03.952 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-05T22:33:04.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-05T22:33:04.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:33:04.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-05T22:33:04.373 INFO:teuthology.orchestra.run.smithi157.stdout:type=AVC msg=audit(1712355093.051:201): avc: denied { checkpoint_restore } for pid=1205 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-04-05T22:33:04.373 INFO:teuthology.orchestra.run.smithi157.stdout:type=AVC msg=audit(1712355093.051:200): avc: denied { checkpoint_restore } for pid=1203 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-04-05T22:33:04.375 DEBUG:teuthology.task.selinux:ubuntu@smithi157.front.sepia.ceph.com has 2 denials 2024-04-05T22:33:04.375 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi157.front.sepia.ceph.com: ['type=AVC msg=audit(1712355093.051:200): avc: denied { checkpoint_restore } for pid=1203 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1', 'type=AVC msg=audit(1712355093.051:201): avc: denied { checkpoint_restore } for pid=1205 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-04-05T22:33:04.376 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-05T22:33:04.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-05T22:33:04.395 INFO:teuthology.task.internal:Duration was 1292.851994 seconds 2024-04-05T22:33:04.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-05T22:33:04.404 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-05T22:33:04.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-05T22:33:04.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-05T22:33:04.563 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-05T22:33:04.574 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-05T22:33:05.036 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-05T22:33:05.036 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-04-05T22:33:05.036 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-05T22:33:05.096 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-05T22:33:05.097 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-05T22:33:05.169 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-05T22:33:05.169 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-05T22:33:05.176 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-05T22:33:05.264 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-05T22:33:05.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-05T22:33:05.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-05T22:33:05.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-05T22:33:05.485 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-05T22:33:05.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-05T22:33:05.572 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-05T22:33:05.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-05T22:33:05.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-05T22:33:05.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-05T22:33:05.789 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-04-05T22:33:05.802 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-05T22:33:05.854 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-05T22:33:05.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:33:05.904 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-05T22:33:05.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-05T22:33:05.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-05T22:33:05.971 INFO:teuthology.task.internal:Transferring archived files... 2024-04-05T22:33:05.972 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi073 2024-04-05T22:33:05.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-05T22:33:06.075 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672/remote/smithi157 2024-04-05T22:33:06.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-05T22:33:06.182 INFO:teuthology.task.internal:Removing archive directory... 2024-04-05T22:33:06.182 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-05T22:33:06.190 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-05T22:33:06.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-05T22:33:06.258 INFO:teuthology.task.internal:Not uploading archives. 2024-04-05T22:33:06.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-05T22:33:06.272 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-05T22:33:06.272 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-05T22:33:06.280 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-05T22:33:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 5 22:33 /home/ubuntu/cephtest 2024-04-05T22:33:06.334 INFO:teuthology.orchestra.run.smithi157.stdout: 395193 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 5 22:33 /home/ubuntu/cephtest 2024-04-05T22:33:06.337 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-05T22:33:06.348 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-05T22:33:06.360 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi157.front.sepia.ceph.com: ['type=AVC msg=audit(1712355093.051:200): avc: denied { checkpoint_restore } for pid=1203 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1', 'type=AVC msg=audit(1712355093.051:201): avc: denied { checkpoint_restore } for pid=1205 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-04-05T22:33:06.360 INFO:teuthology.nuke:Checking targets against current locks 2024-04-05T22:33:06.384 DEBUG:teuthology.nuke:shortname: smithi073 2024-04-05T22:33:06.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-05T22:33:06.417 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-05 22:01:05.931000', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbjYUP5vK2Qbb96znHaYroZFrZBg/XI0PMZan49bBI9xSKPeaNJDnXaZtsKsnRWmuhzAoICbPCRqEGiWnhE+VM='} 2024-04-05T22:33:06.429 DEBUG:teuthology.nuke:shortname: smithi157 2024-04-05T22:33:06.429 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-05T22:33:06.476 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-05 22:01:05.932694', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCnXSpAkWuSJg4Pt6OmsfWAKlOC9R3c3W4g9ge5OjmfvIsslsovZ1PLYWw8SmZK5Y0xBJs7GyUrzdfPi/UgmuQ='} 2024-04-05T22:33:06.506 INFO:teuthology.orchestra.console.smithi073:Power off 2024-04-05T22:33:06.507 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-05T22:33:06.543 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2024-04-05T22:33:06.543 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:06.560 INFO:teuthology.orchestra.console.smithi157:Power off 2024-04-05T22:33:06.560 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-05T22:33:06.580 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-04-05T22:33:06.595 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-04-05T22:33:06.595 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:06.632 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-05T22:33:10.582 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:10.633 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:10.694 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-04-05T22:33:10.743 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-05T22:33:14.698 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:14.745 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:14.811 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-04-05T22:33:14.857 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-04-05T22:33:14.957 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-04-05T22:33:18.814 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-05T22:33:18.924 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-04-05T22:33:19.025 INFO:teuthology.orchestra.console.smithi073:Power off completed 2024-04-05T22:33:19.131 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 client.1-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1292.8519940376282 failure_reason: 'SELinux denials found on ubuntu@smithi157.front.sepia.ceph.com: [''type=AVC msg=audit(1712355093.051:200): avc: denied { checkpoint_restore } for pid=1203 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'', ''type=AVC msg=audit(1712355093.051:201): avc: denied { checkpoint_restore } for pid=1205 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'']' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-04-05T22:33:19.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-05T22:33:19.231 INFO:teuthology.run:FAIL